WebReaper.Mongo
11.3.0
dotnet add package WebReaper.Mongo --version 11.3.0
NuGet\Install-Package WebReaper.Mongo -Version 11.3.0
<PackageReference Include="WebReaper.Mongo" Version="11.3.0" />
<PackageVersion Include="WebReaper.Mongo" Version="11.3.0" />
<PackageReference Include="WebReaper.Mongo" />
paket add WebReaper.Mongo --version 11.3.0
#r "nuget: WebReaper.Mongo, 11.3.0"
#:package WebReaper.Mongo@11.3.0
#addin nuget:?package=WebReaper.Mongo&version=11.3.0
#tool nuget:?package=WebReaper.Mongo&version=11.3.0
WebReaper.Mongo
MongoDB adapters for WebReaper: a result sink, plus MongoDB-backed scraper-config and cookie storage.
Satellite package (ADR-0009): the Mongo adapters are kept out of the WebReaper core so the core stays dependency-light and Native-AOT-clean. MongoDB.Driver's BSON serialization is reflection-based and not trim/AOT-clean, so this package is deliberately not AOT-guaranteed.
Install
dotnet add package WebReaper.Mongo
Pulls WebReaper (the core) as a dependency.
Usage
Adds WriteToMongoDb, WithMongoDbConfigStorage and
WithMongoDbCookieStorage to ScraperEngineBuilder:
using WebReaper.Builders;
using WebReaper.Mongo;
var engine = await ScraperEngineBuilder
.Crawl("https://example.com/catalog")
.Extract(new() { new("title", "h1"), new("price", ".price") })
.WriteToMongoDb(
connectionString: "mongodb://localhost:27017",
databaseName: "scrape",
collectionName: "items",
dataCleanupOnStart: false)
.BuildAsync();
await engine.RunAsync();
License
MIT (ADR-0017). Part of the WebReaper project.
| Product | Versions Compatible and additional computed target framework versions. |
|---|---|
| .NET | net10.0 is compatible. net10.0-android was computed. net10.0-browser was computed. net10.0-ios was computed. net10.0-maccatalyst was computed. net10.0-macos was computed. net10.0-tvos was computed. net10.0-windows was computed. |
-
net10.0
- Microsoft.Extensions.Logging.Abstractions (>= 10.0.8)
- MongoDB.Driver (>= 3.8.1)
- WebReaper (>= 11.3.0)
NuGet packages
This package is not used by any NuGet packages.
GitHub repositories
This package is not used by any popular GitHub repositories.
10.0.0: lockstep republish against the core 10.0.0. Adopts the core's new shapes: durable adapters implement IAsyncInitializable (ADR-0033; the Crawl driver now warms storage uniformly before the loop) and MIT relicense (ADR-0017). Requires WebReaper 10.0.0. 9.0.0: lockstep republish against the core 9.0.0 (ADR-0023: the core public surface is now the documented contract; Tier-2 implementation types internalised). No functional change. 8.0.0: lockstep republish against the core 8.0.0 major (ADR-0022); no functional change. 7.0.0: initial release. MongoDB sink and MongoDB-backed scraper-config and cookie storage (ScraperEngineBuilder.WriteToMongoDb / WithMongoDbConfigStorage / WithMongoDbCookieStorage) extracted from the WebReaper core per ADR-0009 so the core stays dependency-light and Native-AOT-clean (core no longer references MongoDB.Driver).