RobotsTxt 2014.2.19
RobotsTxt : A robots.txt parser for .NET
A robots.txt parser for .NET
Supports ;
- Allow directives.
- Crawl-delay directives.
- Sitemap declarations.
- * and $ wildcards.
See https://bitbucket.org/cagdas/robotstxt for usage examples.
Install-Package RobotsTxt -Version 2014.2.19
dotnet add package RobotsTxt --version 2014.2.19
<PackageReference Include="RobotsTxt" Version="2014.2.19" />
For projects that support PackageReference, copy this XML node into the project file to reference the package.
paket add RobotsTxt --version 2014.2.19
The NuGet Team does not provide support for this client. Please contact its maintainers for support.
Dependencies
This package has no dependencies.
Version History
| Version | Downloads | Last updated | ||
|---|---|---|---|---|
| 2014.2.19 | 19,748 | 2/19/2014 | ||