RobotsTxt 2014.2.19

.NET Framework
NuGet\Install-Package RobotsTxt -Version 2014.2.19
This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package.
dotnet add package RobotsTxt --version 2014.2.19
<PackageReference Include="RobotsTxt" Version="2014.2.19" />
For projects that support PackageReference, copy this XML node into the project file to reference the package.
paket add RobotsTxt --version 2014.2.19
#r "nuget: RobotsTxt, 2014.2.19"
#r directive can be used in F# Interactive, C# scripting and .NET Interactive. Copy this into the interactive tool or source code of the script to reference the package.
// Install RobotsTxt as a Cake Addin
#addin nuget:?package=RobotsTxt&version=2014.2.19

// Install RobotsTxt as a Cake Tool
#tool nuget:?package=RobotsTxt&version=2014.2.19

A robots.txt parser for .NET

Supports ;
- Allow directives.
- Crawl-delay directives.
- Sitemap declarations.
- * and $ wildcards.

See https://bitbucket.org/cagdas/robotstxt for usage examples.

Product Versions
.NET Framework net
Compatible target framework(s)
Additional computed target framework(s)
Learn more about Target Frameworks and .NET Standard.

This package has no dependencies.

NuGet packages

This package is not used by any NuGet packages.

GitHub repositories (2)

Showing the top 2 popular GitHub repositories that depend on RobotsTxt:

Repository Stars
esbencarlsen/NCrawler
.NET based webcrawler
nazuke/SEOMacroscope
SEO Macroscope is a website scanning tool, to check your website for broken links; including some technical SEO functionality, site scraping, Excel reporting, and more.
Version Downloads Last updated
2014.2.19 46,377 2/19/2014