z3l.Scraper.Playwright 0.1.0-preview.6

This is a prerelease version of z3l.Scraper.Playwright.
dotnet add package z3l.Scraper.Playwright --version 0.1.0-preview.6
                    
NuGet\Install-Package z3l.Scraper.Playwright -Version 0.1.0-preview.6
                    
This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package.
<PackageReference Include="z3l.Scraper.Playwright" Version="0.1.0-preview.6" />
                    
For projects that support PackageReference, copy this XML node into the project file to reference the package.
<PackageVersion Include="z3l.Scraper.Playwright" Version="0.1.0-preview.6" />
                    
Directory.Packages.props
<PackageReference Include="z3l.Scraper.Playwright" />
                    
Project file
For projects that support Central Package Management (CPM), copy this XML node into the solution Directory.Packages.props file to version the package.
paket add z3l.Scraper.Playwright --version 0.1.0-preview.6
                    
#r "nuget: z3l.Scraper.Playwright, 0.1.0-preview.6"
                    
#r directive can be used in F# Interactive and Polyglot Notebooks. Copy this into the interactive tool or source code of the script to reference the package.
#:package z3l.Scraper.Playwright@0.1.0-preview.6
                    
#:package directive can be used in C# file-based apps starting in .NET 10 preview 4. Copy this into a .cs file before any lines of code to reference the package.
#addin nuget:?package=z3l.Scraper.Playwright&version=0.1.0-preview.6&prerelease
                    
Install as a Cake Addin
#tool nuget:?package=z3l.Scraper.Playwright&version=0.1.0-preview.6&prerelease
                    
Install as a Cake Tool

z3l.Scraper.Playwright

A headless browser page loader for z3l.Scraper on Microsoft Playwright: for sites that draw their pages with scripts or block plain HTTP clients. The scraper's queue, retries, response checks and throttling work on top, as with HttpPageLoader.

dotnet add package z3l.Scraper.Playwright --prerelease
using z3l.Scraper;
using z3l.Scraper.Loading;
using z3l.Scraper.Playwright;

await using var loader = new PlaywrightPageLoader(new PlaywrightPageLoaderOptions
{
    Channel = "chrome",                                      // the installed Chrome; or download Chromium, see below
    MaxPages = 4,                                            // tabs loading at once
    Proxies = ProxyList.FromFile("proxies.txt"),             // a browser context per proxy
    UserAgents = UserAgentList.FromFile("useragents.txt"),   // every context takes one
    WaitForSelector = "div.product",                         // the content is drawn by scripts
});

var scraper = new Scraper(loader)
    .Route(PageType.Item, ctx => ctx.Emit(ctx.Html.XPathValue("//h1")));

The browser

With Channel = "chrome" or "msedge" the browser installed on the machine is used. Otherwise download one once:

PlaywrightPageLoader.InstallBrowser(PlaywrightBrowser.Chromium);    // returns 0 when it went well

or pwsh bin/Debug/net10.0/playwright.ps1 install chromium after a build. In Docker start from mcr.microsoft.com/playwright/dotnet, which has the browsers and their libraries.

How pages are loaded

  • One browser, started with the first load and again if it crashes. MaxPages pages load at once, each in a new tab that is closed after the load; the scraper's RequestTimeout closes a tab that takes too long.
  • A browser context per proxy (or one without proxies): its own cookies, cache and user agent, like one browser per address. A blocked answer rests the proxy for that site (ProxyList.BlockPolicy); with ResetSessionOnBlock (on by default) the context that got the block is replaced by a new one, with new cookies and the next user agent.
  • GET requests navigate. The response is the page as the browser has drawn it (PageResponse.Text), after WaitUntil (Load), WaitForSelector for 2xx pages and your OnPageLoaded (scroll, click "show more"...). JSON, images and other non-HTML answers are returned as they came; downloads are fetched with the context's request API.
  • POST and other methods go through the context's request API, with its cookies.
  • Images, media and fonts are not loaded (BlockedResourceTypes).
  • ConfigureContext sets the locale, viewport, time zone or geolocation of new contexts.

Browsers cannot log in to socks5 proxies, so a socks5 proxy with a login goes through a small relay on 127.0.0.1: the browser talks to it without a login, the relay logs in to the proxy and passes the connection on (TLS stays end to end).

License

The same as z3l.Scraper (LICENSE.md). Microsoft Playwright is a separate package under the Apache License 2.0; the browsers it downloads have their own licenses.

Product Compatible and additional computed target framework versions.
.NET net9.0 is compatible.  net9.0-android was computed.  net9.0-browser was computed.  net9.0-ios was computed.  net9.0-maccatalyst was computed.  net9.0-macos was computed.  net9.0-tvos was computed.  net9.0-windows was computed.  net10.0 is compatible.  net10.0-android was computed.  net10.0-browser was computed.  net10.0-ios was computed.  net10.0-maccatalyst was computed.  net10.0-macos was computed.  net10.0-tvos was computed.  net10.0-windows was computed. 
Compatible target framework(s)
Included target framework(s) (in package)
Learn more about Target Frameworks and .NET Standard.

NuGet packages

This package is not used by any NuGet packages.

GitHub repositories

This package is not used by any popular GitHub repositories.

Version Downloads Last Updated
0.1.0-preview.6 0 11/16/2026