Rag.NET.DataProviders.Web 0.1.0

.NET 10.0

dotnet add package Rag.NET.DataProviders.Web --version 0.1.0

NuGet\Install-Package Rag.NET.DataProviders.Web -Version 0.1.0

This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package.

<PackageReference Include="Rag.NET.DataProviders.Web" Version="0.1.0" />

For projects that support PackageReference, copy this XML node into the project file to reference the package.

<PackageVersion Include="Rag.NET.DataProviders.Web" Version="0.1.0" />
                    

                            Directory.Packages.props

<PackageReference Include="Rag.NET.DataProviders.Web" />
                    

                            Project file

For projects that support Central Package Management (CPM), copy this XML node into the solution Directory.Packages.props file to version the package.

paket add Rag.NET.DataProviders.Web --version 0.1.0

The NuGet Team does not provide support for this client. Please contact its maintainers for support.

#r "nuget: Rag.NET.DataProviders.Web, 0.1.0"

#r directive can be used in F# Interactive and Polyglot Notebooks. Copy this into the interactive tool or source code of the script to reference the package.

#:package Rag.NET.DataProviders.Web@0.1.0

#:package directive can be used in C# file-based apps starting in .NET 10 preview 4. Copy this into a .cs file before any lines of code to reference the package.

#addin nuget:?package=Rag.NET.DataProviders.Web&version=0.1.0
                    

                            Install as a Cake Addin

#tool nuget:?package=Rag.NET.DataProviders.Web&version=0.1.0
                    

                            Install as a Cake Tool

The NuGet Team does not provide support for this client. Please contact its maintainers for support.

Rag.NET.DataProviders.Web

Web content providers for Rag.NET ingestion — no credentials, three shapes: a sitemap walker, an RSS/Atom feed reader, and a same-domain crawler with depth, page-count and robots.txt controls.

Install

dotnet add package Rag.NET.DataProviders.Web

Setup

These providers are constructed directly — no DI extension method:

using Microsoft.Extensions.DependencyInjection;
using Rag.NET.Abstractions;
using Rag.NET.DataProviders.Web;

var httpClient = new HttpClient();
var provider = new SitemapDataProvider("https://docs.example.com/sitemap.xml", httpClient);
services.AddSingleton<IFileContentProvider>(provider);

Pages arrive as HTML — register AddHtmlParser() from Rag.NET.Parsers.Html in your pipeline.

Example

The crawler variant, bounded so it cannot wander off:

using Rag.NET.DataProviders.Web;
using Rag.NET.DataProviders;
using Rag.NET.Models;

var crawler = new WebCrawlerDataProvider("https://docs.example.com", httpClient, new WebCrawlerOptions
{
    MaxDepth         = 3,
    MaxPages         = 500,
    SameDomain       = true,
    RespectRobotsTxt = true,
});

var result = await pipeline.IngestFromProviderAsync(crawler, new ProviderId("docs-site"), hashStore);
Console.WriteLine($"Ingested {result.Ingested} pages");

RssDataProvider follows the same pattern for feeds.

Full guide

Data providers

Product	Compatible and additional computed target framework versions.
.NET	net10.0 is compatible. net10.0-android was computed. net10.0-browser was computed. net10.0-ios was computed. net10.0-maccatalyst was computed. net10.0-macos was computed. net10.0-tvos was computed. net10.0-windows was computed.

Compatible target framework(s)

Included target framework(s) (in package)

Learn more about Target Frameworks and .NET Standard.

net10.0
- AngleSharp (>= 1.7.1)
- Rag.NET (>= 0.1.0)
- Rag.NET.DataProviders (>= 0.1.0)

NuGet packages

This package is not used by any NuGet packages.

GitHub repositories

This package is not used by any popular GitHub repositories.

Version	Downloads	Last Updated
0.1.0	54	8/11/2026