Winche.Storage 8.0.0

Winche.Storage

Lightweight .NET libraries for storing file metadata in PostgreSQL and objects in S3-compatible archives via presigned URLs. The solution includes core abstractions, an S3 archive provider, an ASP.NET Core base + REST adapter, and a sample app.

Access control is enforced with Winche.Rules — the same Firestore-style rules engine used by Winche.Database.

Packages

Architecture

Package dependencies:

flowchart TD
    Rest["Winche.Storage.AspNetCore.Rest<br/>MapWincheStorageRestApi"]
    Base["Winche.Storage.AspNetCore<br/>FileClaimsAccessor · MapClaims"]
    S3["Winche.Storage.S3<br/>UseS3Archive · S3Archive"]
    Core["Winche.Storage<br/>IFileStorage · guard · schema · hooks"]
    Rules["Winche.Rules<br/>RuleEngine · RuleSetBuilder · Expr"]

    Rest --> Base
    Rest --> Core
    Base --> Core
    S3 --> Core
    Core --> Rules

Runtime components inside the core. The public IFileStorage is the rules guard; it authorizes through the RuleEngine, then delegates to the unprotected FileStorage, which talks to Postgres and the pluggable archive:

flowchart LR
    IFM["IFileStorage"] --> Guard["RuleGuardedFileStorage"]
    Guard -->|authorize| Engine["RuleEngine"]
    Guard -->|delegate| Core["FileStorage"]
    Engine --> Claims["IRuleClaimsAccessor"]
    Core --> DB[("PostgreSQL")]
    Core --> Archive["IArchive<br/>NullArchive / S3Archive"]

Install

dotnet add package Winche.Storage
dotnet add package Winche.Storage.S3
dotnet add package Winche.Storage.AspNetCore.Rest

Quick Start

1. Configure appsettings.json

{
  "ConnectionStrings": {
    "WincheStorage": "Host=localhost;Database=mydb;Username=user;Password=pass"
  },
  "WincheStorage:S3Archive": {
    "BucketName": "your-bucket",
    "RegionName": "us-east-1",
    "AccessKey": "YOUR_ACCESS_KEY",
    "SecretKey": "YOUR_SECRET_KEY",
    "PresignedUrlExpiry": "00:15:00"
  }
}

AccessKey and SecretKey are optional. Omit them when deploying to AWS with a task role or instance profile — the SDK uses ambient IAM credentials automatically.

For a non-public schema, add Search Path=myschema to the connection string (the metadata table is created in the connection's search_path).

2. Register services

using Winche.Rules;
using Winche.Rules.Expressions;
using Winche.Storage.AspNetCore.DependencyInjection;       // MapClaims
using Winche.Storage.DependencyInjection;                  // AddWincheStorage
using Winche.Storage.S3.DependencyInjection;               // UseS3Archive

builder.Services.AddWincheStorage(opts =>
{
    opts.ConnectionString = builder.Configuration.GetConnectionString("WincheStorage");

    // Default-deny. Grant access with one or more rule blocks.
    opts.UseRules(r => r.Match("userFiles/{userId}/{rest=**}", owned =>
        owned.Allow(RuleOperations.All, Expr.Auth("token", "userId").Eq(Expr.Param("userId")))));

    opts.UseS3Archive(s3 => builder.Configuration.GetSection("WincheStorage:S3Archive").Bind(s3));

    // Map HTTP requests to caller claims consumed by the rules engine.
    opts.MapClaims(ctx => new Dictionary<string, object?>
    {
        ["userId"] = ctx.Request.Headers["X-USER-ID"].ToString(),
    });
});

3. Initialize schema and map endpoints

await app.InitializeWincheStorageAsync();   // creates the winche_files table if it doesn't exist
app.MapWincheStorageRestApi();              // maps REST routes under the "files/" prefix

See samples/Winche.Storage.Sample for a complete working example.

Configuration

AddWincheStorage

A single overload takes an Action<WincheStorageOptions>. Set the connection string and register components inside the lambda:

services.AddWincheStorage(opts =>
{
    opts.ConnectionString = "Host=...;Database=...;Username=...;Password=...";
    opts.UseRules(/* ... */);
    opts.UseHooks(h => h.Add<AuditHook>("userFiles/{userId}/{file=**}"));
    opts.UseS3Archive(/* Action<S3ArchiveOptions> */);
    opts.MapClaims(/* Func<HttpContext, IReadOnlyDictionary<string, object?>?> */);
});

WincheStorageOptions

UseS3Archive (from Winche.Storage.S3) and MapClaims (from Winche.Storage.AspNetCore) are extension methods on WincheStorageOptions.

S3ArchiveOptions

Configure S3 with a delegate (bind from IConfiguration yourself if you prefer):

opts.UseS3Archive(s3 =>
{
    s3.BucketName = "my-bucket";
    s3.RegionName = "eu-west-1";
});

REST Endpoints

All {path} segments are base64url-encoded file paths (or directory paths for :list). CRUD uses HTTP methods; per-file operations use AIP-136 colon-verbs (all POST).

MapWincheStorageRestApi returns a single IEndpointConventionBuilder covering every endpoint, so cross-cutting policy applies to all of them:

app.MapWincheStorageRestApi(prefix: "storage")
   .RequireAuthorization();

Request pipeline

Every endpoint runs the built-in ClaimsAccessor filter (maps the request to caller claims) and the ExceptionHandler filter (translates exceptions to status codes), outermost, before the handler decodes the base64url path and calls IFileStorage:

flowchart LR
    Req["HTTP<br/>/files/{base64url}:verb"] --> CA["ClaimsAccessor<br/>SetClaims(HttpContext)"]
    CA --> EH["ExceptionHandler"]
    EH --> Handler["handler<br/>DecodePath → IFileStorage"]
    Handler --> Ok["Results.Json"]
    EH -. on exception .-> Err["403 AccessDenied<br/>404 NotFound<br/>400 InvalidStatus<br/>500 other"]

Upload lifecycle

A record is registered first (status pending), uploaded directly to the archive via presigned URLs (single or multipart), then confirmed (status complete). Downloads require complete:

sequenceDiagram
    actor Client
    participant API as Storage API
    participant Arch as Archive

    Client->>API: PUT /{path}  (register, status=pending)
    alt single upload
        Client->>API: POST /{path}:upload
        API-->>Client: presigned PUT URL
        Client->>Arch: upload object
    else multipart upload
        loop part 1..N
            Client->>API: POST /{path}:signPart {partNumber}
            API-->>Client: presigned part URL
            Client->>Arch: upload part
        end
    end
    Client->>API: POST /{path}:confirm
    API->>Arch: complete / verify object
    API-->>Client: FileRecord (status=complete)
    Client->>API: POST /{path}:download
    API-->>Client: presigned GET URL

stateDiagram-v2
    [*] --> pending: register
    pending --> complete: confirm
    complete --> [*]: download

Access Control

Authorization is expressed as Winche.Rules rule blocks via opts.UseRules(...). Access is default-deny: with no matching Allow, every protected call throws AccessDeniedException. Multiple UseRules calls accumulate and are OR-combined.

opts.UseRules(r => r
    .Match("userFiles/{userId}/{rest=**}", owned =>
    {
        // Owner has full access to their own subtree.
        owned.Allow(RuleOperations.All, Expr.Auth("token", "userId").Eq(Expr.Param("userId")));
    })
    .Match("public/{rest=**}", pub =>
    {
        // Anyone may read public files.
        pub.Allow(RuleOperations.Read, Expr.Const(true));
    }));

• Path patterns use {param} single-segment captures and a trailing {name=**} recursive capture, readable in conditions via Expr.Param("param").
• Operations: RuleOperations.Read (get + list), RuleOperations.Write (create + update + delete), RuleOperations.All, or individual RuleOperation values.
• The existing file is exposed to conditions as resource — e.g. Expr.Resource("mimeType"), Expr.Resource("sizeBytes"), Expr.Resource("metadata", "ownerId").
• Caller claims are exposed under request.auth: Expr.Auth("uid") (the uid claim, if any) and Expr.Auth("token", "<claim>") (any mapped claim).

Hot-swapping rules at runtime

The guard's engine reads its ruleset from a mutable repository on every evaluation, so rules can be replaced without restarting the app. The write side is registered as a keyed IMutableRuleSetRepository under WincheStorageKeys.RULE_ENGINE_KEY — resolve it and call Update:

using Winche.Rules;
using Winche.Rules.Expressions;
using Winche.Storage.Constants;

var repo = provider.GetRequiredKeyedService<IMutableRuleSetRepository>(WincheStorageKeys.RULE_ENGINE_KEY);
repo.Update(RuleSetBuilder.Build(r => r.Match("public/{rest=**}", pub =>
    pub.Allow(RuleOperations.Read, Expr.Const(true)))));

The swap is a lock-free atomic reference write: in-flight evaluations keep the ruleset they started with, and the next call observes the new rules. Update replaces the whole ruleset — build the full set you want live (it is not merged with the UseRules blocks registered at startup).

Storage operations map to rule operations as follows:

ListDirectoryIdsAsync is intentionally absent from this table: it exists only on the concrete FileStorage, never on IFileStorage, and is never rule-evaluated — a privileged, admin-only operation (see Privileged: ListDirectoryIdsAsync below).

Authorization flow

A protected call loads the current record (as resource), gathers caller claims, and asks the RuleEngine. If no rule allows the operation it throws AccessDeniedException (default-deny); otherwise it delegates to the unprotected core. Writes follow the same path, authorizing before the mutation (a create has no prior resource):

sequenceDiagram
    actor Caller
    participant Guard as RuleGuardedFileStorage
    participant Core as FileStorage
    participant Claims as IRuleClaimsAccessor
    participant Engine as RuleEngine

    Caller->>Guard: GetAsync(path)
    Guard->>Core: GetAsync(path)
    Core-->>Guard: FileRecord (resource)
    Guard->>Claims: GetClaims()
    Claims-->>Guard: caller claims
    Guard->>Engine: AllowsAsync(Get, path, request)
    alt allowed
        Engine-->>Guard: true
        Guard-->>Caller: FileRecord
    else denied
        Engine-->>Guard: false
        Guard--xCaller: AccessDeniedException
    end

Claims mapping

MapClaims (from Winche.Storage.AspNetCore) maps the HTTP request to the caller-claims dictionary the rules engine reads. The dictionary is exposed as request.auth.token.*, and a uid key (if present) also as request.auth.uid.

opts.MapClaims(ctx => new Dictionary<string, object?>
{
    ["uid"]    = ctx.User.FindFirst("sub")?.Value,
    ["userId"] = ctx.Request.Headers["X-USER-ID"].ToString(),
});

For non-HTTP callers (background services, tests), inject FileClaimsAccessor and call SetClaims(...) directly before invoking IFileStorage.

Hooks

Implement FileStoreHook (behavior only) and register it against a path with UseHooks(h => h.Add<T>(path)). The path is a Firestore-style pattern (literal segments, {id} single-segment captures, and a trailing {name=**} recursive wildcard matching one or more segments; bare */** are not valid). The same hook type can be bound to multiple paths. Hooks are dispatched asynchronously.

public class AuditHook : FileStoreHook
{
    public override Task OnFileRegisteredAsync(FileRecord record, CancellationToken ct) { ... }
    public override Task OnUploadConfirmedAsync(FileRecord record, CancellationToken ct) { ... }
    public override Task OnFileDeletedAsync(string path, CancellationToken ct) { ... }
    public override Task OnMetadataUpdatedAsync(FileRecord record, CancellationToken ct) { ... }
    public override Task OnUploadUrlGeneratedAsync(string path, UploadSession session, CancellationToken ct) { ... }
    public override Task OnDownloadUrlGeneratedAsync(string path, DownloadSession session, CancellationToken ct) { ... }
}

Register: opts.UseHooks(h => h.Add<AuditHook>("userFiles/{userId}/{file=**}")).

Orphan Sweep

DeleteAsync treats Postgres as the source of truth: it commits the row removal first, then deletes the archive object best-effort. If the archive call fails, the row is still gone and the object is left as a harmless orphan rather than rolling the delete back (which would leave an undeletable record). Archive deletes never throw out of DeleteAsync.

The background orphan sweep reclaims those leftovers. Enable it with UseOrphanSweep (requires a real archive such as UseS3Archive):

opts.UseOrphanSweep(o =>
{
    o.Interval    = TimeSpan.FromHours(6);    // how often the sweep runs
    o.GraceWindow = TimeSpan.FromHours(24);   // min age before an unreferenced object is purged
    o.Prefix      = null;                     // optional key prefix to scope the sweep
});

A hosted service periodically lists the archive and deletes every object that has no matching winche_files row and is older than GraceWindow. The grace window protects in-flight uploads (object PUT, row not yet committed) from being reaped.

To run a sweep on demand, call the privileged PurgeOrphansAsync on the concrete FileStorage (never on IFileStorage, never rule-evaluated — same privileged model as ListDirectoryIdsAsync). It returns the number of objects identified as orphans and submitted for deletion:

// concrete FileStorage only — privileged, not rule-guarded
Task<int> PurgeOrphansAsync(string? prefix, TimeSpan graceWindow, CancellationToken ct = default);

Custom IArchive implementations must implement ListObjectsAsync (added in 7.0) for the sweep to work; S3Archive already does.

IFileStorage

Inject IFileStorage to interact with the store — it always resolves to the rules guard, which authorizes every call via Winche.Rules. Trusted server-side callers that have no request claims (background services, hooks, schedulers) should inject the concrete FileStorage instead, which is the unguarded core. This mirrors Winche.Database's IDocumentDatabase (guarded) vs DocumentDatabase (unguarded) split.

// IFileStorage — every call is authorized via Winche.Rules
Task<FileRecord>               SetAsync(string path, string mimeType, long sizeBytes, JsonObject? metadata, CancellationToken ct);
Task<FileRecord?>              GetAsync(string path, CancellationToken ct);
Task<FileRecord?>              UpdateMetadataAsync(string path, JsonObject patch, CancellationToken ct);
Task<bool>                     DeleteAsync(string path, CancellationToken ct);
Task<UploadSession>            GenerateUploadUrlAsync(string path, CancellationToken ct);
Task<DownloadSession>          GenerateDownloadUrlAsync(string path, CancellationToken ct);
Task<FileRecord>               ConfirmUploadAsync(string path, CancellationToken ct);
Task<IEnumerable<FileRecord>>  ListAsync(string directory, string? mimeType, CancellationToken ct);
Task<UploadSession>            SignPartAsync(string path, int partNumber, CancellationToken ct);
Task<IEnumerable<FilePart>>    ListUploadedPartsAsync(string path, CancellationToken ct);

For trusted, claim-less callers, inject the concrete core — same surface, no authorization:

public sealed class ReportMailer(FileStorage files)   // concrete FileStorage = unguarded
{
    public Task<DownloadSession> LinkAsync(string path, CancellationToken ct) =>
        files.GenerateDownloadUrlAsync(path, ct);
}

Privileged: ListDirectoryIdsAsync

Listing the immediate sub-directory names under a directory is a privileged, admin-style operation. It lives only on the concrete FileStorage — never on IFileStorage — and is never evaluated by the rules engine, mirroring Firestore's Admin-SDK-only listCollectionIds (and Winche.Database's DocumentDatabase.ListCollectionIdsAsync). It is not exposed over the REST API.

// concrete FileStorage only — privileged, not rule-guarded
Task<ListDirectoryIdsResult> ListDirectoryIdsAsync(
    string? parentDirectory, int? pageSize = null, string? pageToken = null, CancellationToken ct = default);

Returns the distinct sub-directory names directly under parentDirectory (or the top-level directories when null/empty), ordered by UTF-8 byte order. Results are keyset-paginated: pass NextPageToken back as pageToken to walk pages (pageSize defaults to 100, capped at 300). Files sitting directly in parentDirectory are not sub-directories and are excluded.

public sealed class Browser(FileStorage files)   // concrete = privileged
{
    public async Task<IReadOnlyList<string>> SubdirsAsync(string dir, CancellationToken ct)
    {
        var all = new List<string>();
        string? token = null;
        do
        {
            var page = await files.ListDirectoryIdsAsync(dir, pageToken: token, ct: ct);
            all.AddRange(page.DirectoryIds);
            token = page.NextPageToken;
        } while (token is not null);
        return all;
    }
}

ListDirectoryIdsResult carries IReadOnlyList<string> DirectoryIds and a string? NextPageToken (null on the last page).

FileRecord

Requirements

• .NET 10 SDK (net10.0)
• PostgreSQL for metadata storage
• An S3-compatible bucket for object storage (AWS S3, MinIO, etc.)

License

Elastic License 2.0