Gravicode.ScrapyNet 0.1.0

.NET 10.0 This package targets .NET 10.0. The package is compatible with this framework or higher. 
dotnet add package Gravicode.ScrapyNet --version 0.1.0
                    


                    

                
NuGet\Install-Package Gravicode.ScrapyNet -Version 0.1.0
This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package. 
<PackageReference Include="Gravicode.ScrapyNet" Version="0.1.0" />
For projects that support PackageReference, copy this XML node into the project file to reference the package. 
<PackageVersion Include="Gravicode.ScrapyNet" Version="0.1.0" />
                    


                            Directory.Packages.props
                        

                    

                
<PackageReference Include="Gravicode.ScrapyNet" />
                    


                            Project file
For projects that support Central Package Management (CPM), copy this XML node into the solution Directory.Packages.props file to version the package. 
paket add Gravicode.ScrapyNet --version 0.1.0
                    


                    

                
#r "nuget: Gravicode.ScrapyNet, 0.1.0"
#r directive can be used in F# Interactive and Polyglot Notebooks. Copy this into the interactive tool or source code of the script to reference the package. 
#:package Gravicode.ScrapyNet@0.1.0
#:package directive can be used in C# file-based apps starting in .NET 10 preview 4. Copy this into a .cs file before any lines of code to reference the package. 
#addin nuget:?package=Gravicode.ScrapyNet&version=0.1.0
                    


                            Install as a Cake Addin
                        

                    

                
#tool nuget:?package=Gravicode.ScrapyNet&version=0.1.0
                    


                            Install as a Cake Tool

Scrapy.Net

Scrapy for .NET 10 — the web crawling and scraping framework, rewritten in C#.

Dibuat oleh Gravicode Studios dipimpin oleh Kang Fadhil. · Made by Gravicode Studios, led by Kang Fadhil.

Scrapy Gallery

Same engine architecture, component names and setting keys as Scrapy, with async callbacks, typed items and compile-time checked components.

using ScrapyNet;

public sealed class QuotesSpider : Spider
{
    public override IReadOnlyList<string> StartUrls => ["https://quotes.toscrape.com/"];

    public override async IAsyncEnumerable<object> Parse(Response response)
    {
        foreach (var quote in response.Css("div.quote"))
            yield return new { Text = quote.Css("span.text::text").Get(), Author = quote.Css("small.author::text").Get() };

        if (response.Css("li.next a::attr(href)").Get() is { } next)
            yield return response.Follow(next, Parse);
        await Task.CompletedTask;
    }
}

var settings = new Settings();
settings.Feeds.Add("quotes.jsonl");
Console.WriteLine(await Scrapy.CrawlAsync(new QuotesSpider(), settings));

Packages

Package Adds
Gravicode.ScrapyNet Engine, spiders (Crawl/Sitemap/XML/CSV), CSS/XPath selectors, items, loaders, middlewares (retry, redirect, cookies, proxies, robots.txt, cache, AutoThrottle), pipelines (files, images, databases), feed exports, pause/resume, signals, stats, CLI runner
Gravicode.ScrapyNet.Playwright JavaScript rendering with page actions
Gravicode.ScrapyNet.AI Extraction, chunking, dedup, embeddings, vector search, RAG with OpenAI/Azure/DeepSeek/Ollama
Gravicode.ScrapyNet.Platform Declarative spiders, jobs, cron, secrets, monitoring, alerts
Gravicode.ScrapyNet.Server REST API with roles (scrapyd-style)
Gravicode.ScrapyNet.Sandbox Offline practice website for learning and tests
Gravicode.ScrapyNet.Cli scrapynet tool: startproject, genspider, crawl, shell, fetch, parse, bench

Documentation

MIT License · © 2026 Gravicode Studios

Product Compatible and additional computed target framework versions.
.NET net10.0 is compatible.  net10.0-android was computed.  net10.0-browser was computed.  net10.0-ios was computed.  net10.0-maccatalyst was computed.  net10.0-macos was computed.  net10.0-tvos was computed.  net10.0-windows was computed. 
Compatible target framework(s)
Included target framework(s) (in package)
Learn more about Target Frameworks and .NET Standard.

NuGet packages (3)

Showing the top 3 NuGet packages that depend on Gravicode.ScrapyNet:

Package Downloads
Gravicode.ScrapyNet.Platform

Control plane for Scrapy.Net: spider catalog with versioning and cloning, declarative no-code spiders (targets and scraping rules as JSON), job manager (run, stop, pause, resume, cancel, retry, history), cron scheduler, encrypted secret store, live monitoring dashboard data, health alerts (failed jobs, zero or dropping items, inaccessible targets, expired logins), notifications (webhook, Slack, Teams, email) and raw content storage. Dibuat oleh Gravicode Studios, dipimpin oleh Kang Fadhil.
Gravicode.ScrapyNet.Playwright

A Playwright download handler for Scrapy.Net (like scrapy-playwright): requests marked with meta "playwright" are rendered in a headless Chromium, Firefox or WebKit, with page actions (click, fill, scroll, wait for selector, evaluate, screenshot), resource blocking and page reuse; other requests keep using plain HTTP. Dibuat oleh Gravicode Studios, dipimpin oleh Kang Fadhil.
Gravicode.ScrapyNet.AI

Turns Scrapy.Net crawls into AI-ready knowledge: article and metadata extraction (OpenGraph, JSON-LD), text normalization, chunking, SimHash near-duplicate detection, content change detection and incremental re-crawls, local hashing embeddings or OpenAI-compatible embeddings, an in-memory SIMD vector store (plus Qdrant), extractive and LLM summarization, keyword/Naive Bayes/LLM categorization, and a RAG ingestion pipeline with a queryable knowledge base. Dibuat oleh Gravicode Studios, dipimpin oleh Kang Fadhil.

GitHub repositories

This package is not used by any popular GitHub repositories.

Version Downloads Last Updated
0.1.0 40 9/27/2026

First release of the Scrapy.Net engine, selectors, items, loaders, pipelines, feed exports, middlewares, extensions and command line.