Zooper.Ant 1.0.0

Zooper.Ant

<img src="icon.png" alt="Zooper.Ant Logo" width="120" align="right"/>

Caller-selected paging for .NET: continuable, counted and capped pages over any store, with keyset continuation behind encrypted tokens.

Overview

Every list endpoint needs paging, and every team writes it again: offsets that skip rows when data changes, totals nobody asked for, cursors that leak database values, tokens that can be replayed against another query. Zooper.Ant writes it once.

A read is written once and serves three page kinds, chosen by the caller per request. Continuation is a keyset seek, never an offset, so it stays correct while rows are inserted and deleted, and costs the same on page 1000 as on page 2. Tokens are encrypted with ASP.NET Core Data Protection, bound to the read and its filters, and never expire. Any store can be paged: LINQ stores supply two methods, every other store implements one small interface.

Zooper.Ant depends on Microsoft.AspNetCore.DataProtection.Abstractions and Microsoft.Extensions.Options only. The companion package Zooper.Ant.AspNetCore adds the HTTP edge: query binding, 400 problem responses and the OpenAPI description of pages.

Key Concepts

Installation

dotnet add package Zooper.Ant
dotnet add package Zooper.Ant.AspNetCore   # ASP.NET Core hosts: binding, problem responses, OpenAPI

Getting Started

Register Ant once. The host owns Data Protection; Ant uses it for tokens.

builder.Services.AddDataProtection();
builder.Services.AddAnt();

Inject Pager and end a LINQ query with ToPageAsync:

public sealed class PlayerQueries(PlayerStore store, Pager pager)
{
    public async Task<Page<PlayerView>> ListAsync(string? team, Paging paging, CancellationToken ct)
    {
        var page = await store.Players
            .Where(p => team == null || p.Team == team)
            .ToPageAsync(
                paging,
                Order<Player>.ByDescending(p => p.Rating).ThenBy(p => p.Id),   // order, unique Id last
                PageScope.For(nameof(ListAsync), team),                         // the read and its filters
                pager,
                ct);

        return page.Map(PlayerView.From);                                       // keeps the page kind
    }
}

The caller picks the kind:

await queries.ListAsync("red", Paging.Continuable(pageToken: null, size: 20), ct); // first page + next token
await queries.ListAsync("red", Paging.Continuable(nextToken, size: 20), ct);     // the page after
await queries.ListAsync("red", Paging.Counted(page: 3, size: 20), ct);           // page 3 of N, with the total
await queries.ListAsync("red", Paging.Capped(cap: 5), ct);                       // top 5, and whether more matched

Example Project

Zooper.Ant.Example plays a narrated tour of every feature over twelve players, printing each call and what came back:

dotnet run --project Zooper.Ant.Example

1. Continuable pages, the default kind
   > queries.ListAsync(new PlayerFilter(), Paging.Continuable(null, size: 3))
   < page 1: continuable, size 3: Eve (blue, 95), Ada (red, 91), Ivy (green, 91)
     next token: ...ZVCLsuw0xZLyoILU (219 chars, encrypted, different every page)

With -- --serve it serves the same reads over HTTP through Zooper.Ant.AspNetCore (the ?paging= parameters, 400 problem responses, /openapi/v1.json); Zooper.Ant.Example.http walks through them. See Zooper.Ant.Example/README.md.

Page Kinds

Continuable

var page = (ContinuablePage<PlayerView>)await queries.ListAsync(null, Paging.Continuable(token, 20), ct);

page.ItemList;        // the players
page.NextPageToken;   // null on the last page; the only end signal

The token marks a position in the order ("after Ivy, rating 91, id 9"), so a row deleted before it does not shift the next page and a row added after it shows up later. The page size may change between pages.

Counted

var page = (CountedPage<PlayerView>)await queries.ListAsync(null, Paging.Counted(page: 2, size: 20), ct);

page.Page;         // 2
page.TotalCount;   // every match, across all pages

A page deeper than MaximumDepth rows is refused with a PagingException, never replaced by another page. Use continuable paging to go further.

Capped

var page = (CappedPage<PlayerView>)await queries.ListAsync("red", Paging.Capped(cap: 3), ct);

page.Truncated;   // true when more than 3 matched

Mapping and Empty Pages

page.Map(...) maps the items and keeps the kind with its page number, total or token. A filter known to match nothing can answer without reading:

if (minRating > 100)
    return paging.ToEmptyPage<PlayerView>();

Limits

builder.Services.AddAnt(options =>
    options.Limits = new PagingLimits(DefaultSize: 20, MaximumSize: 50, MaximumDepth: 1000));   // the defaults

Paging.Continuable(token, size, pager.Limits);   // build Paging from the configured limits

Ordering

Order<Player>.ByDescending(p => p.Rating)           // nullable: unrated players sort last
    .ThenBy(p => p.Name)
    .ThenBy(p => p.Id);                              // unique tie-breaker, always last

Order<Player>.By(p => p.Rating, nullsFirst: true);  // nulls first instead
order.Reverse();                                     // every direction and nulls placement flipped

• End with a unique key. Rows with equal values are then still in a fixed order, so a page boundary between them can never skip or repeat one.
• Keys are typed lambdas. No strings: renaming a property is a compile error.
• Mixed directions work. Each key seeks in its own direction.
• Timestamps are truncated to microseconds in the token, the precision of PostgreSQL and most stores.

Tokens

PageScope.For(nameof(ListAsync), team, minRating)

The scope names the read and lists every argument that decides its result set. A token is refused when:

• it is presented to another read,
• any scope value differs (team=red token used with team=blue),
• the order changed,
• a character was altered, or the keys that encrypted it are gone.

Never put the page size in the scope; it may change between pages. Pass scalars and records.

Tokens have no lifetime. A host with several instances must share one Data Protection key ring (PersistKeysToFileSystem, a database, Redis), otherwise a token from one instance is refused by another. The purpose root defaults to Zooper.Ant.PageToken; changing it invalidates every token in flight:

builder.Services.AddAnt(options => options.PurposeRoot = "MyService.PageToken");

To replace the encryption, register your own IPageTokenProtector before AddAnt.

PageToken implements IParsable<PageToken> and carries JSON and type converters, so it binds from a query string and serialises as a plain string.

Reading Everything

TraverseAsync follows the tokens for you. It reads the next page only when you reach it, keeps no snapshot, and ends with the error of a failed page:

await foreach (var player in pager.TraverseAsync<PlayerView>(
    (token, ct) => queries.ListAsync("red", Paging.Continuable(token, 100), ct), ct))
{
    await Sync(player);
}

Data is live: a row whose ordering value changes during the traversal may be skipped or seen twice. Order by an immutable key when that matters.

Error Handling

Map one type to one response. An ASP.NET Core host gets that mapping from Zooper.Ant.AspNetCore, below.

Hosting in ASP.NET Core

Zooper.Ant.AspNetCore is the HTTP edge, so no host writes it again. Every Ant host speaks the same five query parameters:

An argument of another kind (?paging=capped&cap=5&page=2) is refused, never ignored. An unknown paging value or a malformed pageToken is a 400 before the endpoint runs. The names are fixed: a host whose API already uses other names binds its own parameters and calls the Paging factories with pager.Limits.

Bind PagingRequest beside your own filter and pass the host's limits:

app.MapGet("/players", ([AsParameters] PlayerFilter filter, [AsParameters] PagingRequest paging, PlayerQueries queries, Pager pager, CancellationToken ct) =>
    queries.ListAsync(filter, paging.ToPaging(pager.Limits), ct));

// In an [ApiController]:
public Task<Page<PlayerView>> List([FromQuery] PlayerFilter filter, [FromQuery] PagingRequest paging, CancellationToken ct) =>
    queries.ListAsync(filter, paging.ToPaging(pager.Limits), ct);

Two independent one-liners do the rest; the binding needs none:

builder.Services.AddAntExceptionHandler();                               // PagingException → 400 problem response
builder.Services.AddOpenApi(options => options.AddAntPageResponses());   // Page<T> → oneOf the three page shapes

app.UseExceptionHandler();                                               // required, as for every IExceptionHandler
app.MapOpenApi();

The problem's type is urn:zooper:ant:invalid-page-token for a refused token (a client restarts from the first page) and urn:zooper:ant:invalid-paging for every other refusal; the detail is the refusal's message. A host with its own error convention skips AddAntExceptionHandler and maps PagingException itself. Controllers need [ApiController], which turns a failed parse into a 400; without it an unknown paging would bind as absent.

Adding a Store

A LINQ provider implements IQueryExecutor, two methods. QueryablePageSource<T> builds the ORDER BY and the seek from the Order<T>:

public sealed class MyProviderExecutor : IQueryExecutor
{
    public Task<IReadOnlyList<T>> ToListAsync<T>(IQueryable<T> query, CancellationToken ct) => ...;
    public Task<int> CountAsync<T>(IQueryable<T> query, CancellationToken ct) => ...;
}

var page = await pager.ReadAsync(new QueryablePageSource<Player>(query, order, new MyProviderExecutor()), paging, scope, ct);

Pass timestampPrecision when the store keeps timestamps more precisely than microseconds.

Any other store implements IPageSource<T>. The Pager builds every kind from it and owns the tokens; the source never sees one:

public interface IPageSource<TItem>
{
    string OrderSignature { get; }                                                    // changes when the order does
    Task<int> CountAsync(CancellationToken ct);                                       // counted kind
    Task<IReadOnlyList<TItem>> ReadAsync(int offset, int limit, CancellationToken ct);
    Task<IReadOnlyList<TItem>> ReadAfterAsync(PagePosition position, int limit, CancellationToken ct);
    PagePosition PositionOf(TItem item);                                              // goes into the token
    PagePosition ParsePosition(IReadOnlyList<JsonElement> payload);                   // comes out of it
}

A store with its own cursors (Elasticsearch search_after, Cosmos DB continuations, Dapr state queries) puts its cursor in the position and throws NotSupportedException from CountAsync and from ReadAsync with an offset. Continuable and capped pages work; the counted kind is refused with a PagingException. ActivityLogPageSource in the example project is a complete one.

API Quick Reference

More from Zooper

Check out the other libraries on GitHub.

Contributing

Contributions are welcome. Open an issue or submit a pull request.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Made with ❤️ by the Zooper team