NStalling.Avro 0.2.0

NStalling.Avro

A thin extension over Apache.Avro that adds runtime CLR type resolution to reflection-based Avro deserialization.

Apache.Avro understands the Avro schema and performs the decoding. NStalling.Avro determines the CLR type to materialize from the Avro record schema.

• Targets: netstandard2.1
• Depends on: Apache.Avro 1.12.1

Why

The Avro schema may identify exactly which record is being read while the corresponding CLR member does not identify the concrete application type to materialize.

For example:

public sealed class Envelope
{
    public string EventId { get; init; } = "";
    public object Payload { get; init; } = null!;
}

The schema might identify Payload as Acme.Events.CustomerCreated, but object does not tell Apache.Avro which CLR type should represent that record.

Runtime type resolution is also needed when:

• a member is declared as object, an interface, or an abstract base type;
• an Avro union contains multiple record branches;
• multiple CLR types share an Avro name and are distinguished by schema version; or
• an outer record carries an Avro payload as opaque bytes, with its writer schema supplied separately.

NStalling.Avro stays focused on CLR materialization around Apache's reflection reader. It does not generate schemas, implement a codec, add a schema registry client, or replace Apache's Avro behavior.

Install

dotnet add package NStalling.Avro

For Microsoft.Extensions.DependencyInjection integration (AddAvro), also add:

dotnet add package NStalling.Avro.DependencyInjection

Quick start

Register the concrete CLR types that correspond to known Avro record schemas:

using System.Runtime.Serialization;
using Avro;
using NStalling.Avro;
using NStalling.Avro.Serialization;

[DataContract(Name = "CustomerCreated", Namespace = "Acme.Events")]
public sealed class CustomerCreated
{
    public string CustomerId { get; init; } = "";
}

[DataContract(Name = "OrderPlaced", Namespace = "Acme.Events")]
public sealed class OrderPlaced
{
    public string OrderId { get; init; } = "";
}

public sealed class Envelope
{
    public string EventId { get; init; } = "";
    public object Payload { get; init; } = null!;
}

var resolver = new AvroTypeRegistry()
    .Add<CustomerCreated>()
    .Add<OrderPlaced>()
    .BuildResolver();

var serializer = new AvroSerializer(resolver);

var schema = Schema.Parse(envelopeSchemaJson);
var envelope = serializer.Deserialize<Envelope>(bytes, schema);

switch (envelope.Payload)
{
    case CustomerCreated customer:
        Handle(customer);
        break;

    case OrderPlaced order:
        Handle(order);
        break;
}

Apache.Avro reads the Avro data and schema; NStalling.Avro resolves the record schema to the registered CLR type.

Registering types

AvroTypeRegistry can derive an Avro full name from [DataContract] (Namespace + "." + Name) or from the exact CLR full name, or you can map a type explicitly.

Simple-name matching is never used, and schema versions are never inferred from CLR names.

var resolver = new AvroTypeRegistry()
    .Add<CustomerCreated>()
    .Map<OrderPlaced>(
        "OrderPlaced",
        "Acme.Events")
    .Map<Product>(
        "Product",
        "Acme.Events",
        schemaVersion: "3")
    .FromAssemblyContaining<CustomerCreated>()
    .BuildResolver();

Within a selected resolution bucket, precedence is:

Explicit > DataContract > CLR full-name convention

Conflicts between equal-precedence mappings fail fast at build time with AvroConfigurationException.

Attributes

NStalling.Avro can keep type-resolution metadata close to CLR models when that is the most natural place for it.

[DataContract(Name = "Customer", Namespace = "Acme.Events")]
[AvroSchemaVersion("2")]
public sealed class Customer
{
}

For members that need runtime discriminator metadata:

public sealed class ProfileEnvelope
{
    [AvroTypeDiscriminator]
    public string PayloadType { get; init; } = "";

    [AvroVersionDiscriminator]
    public string? PayloadVersion { get; init; }

    [AvroPolymorphic]
    public object Payload { get; set; } = null!;
}

The attributes have distinct roles:

• DataContract maps a CLR type to an Avro record name.
• [AvroSchemaVersion] declares which externally supplied schema version(s) a CLR type can represent.
• [AvroTypeDiscriminator] identifies metadata used to locate an opaque payload's writer schema.
• [AvroVersionDiscriminator] supplies runtime schema-version context.
• [AvroPolymorphic] marks or configures a member that needs runtime materialization behavior.

Fluent configuration can be used instead when this metadata belongs in application configuration rather than on the model.

Schema versions

One CLR type may represent multiple schema versions, and one Avro name may map to different CLR types under different supplied versions.

[DataContract(Name = "Customer", Namespace = "Acme.Events")]
[AvroSchemaVersion("1")]
public sealed class LegacyCustomer : ICustomer
{
}

[DataContract(Name = "Customer", Namespace = "Acme.Events")]
[AvroSchemaVersion("2")]
public sealed class CurrentCustomer : ICustomer
{
}

var customerSchema = (RecordSchema)Schema.Parse(customerSchemaJson);

var v1 = serializer.Deserialize(
    bytes,
    customerSchema,
    schemaVersion: "1"); // LegacyCustomer

var v2 = serializer.Deserialize(
    bytes,
    customerSchema,
    schemaVersion: "2"); // CurrentCustomer

A single CLR type can also represent several compatible versions:

[DataContract(Name = "Customer", Namespace = "Acme.Events")]
[AvroSchemaVersion("2")]
[AvroSchemaVersion("3")]
[AvroSchemaVersion("4")]
public sealed class Customer
{
}

Resolution is two-stage:

1. Select the version bucket.
2. Apply mapping precedence within that bucket.

An exact version wins. If version-specific mappings exist for an Avro name, an unknown version does not silently fall back to an unqualified mapping.

A type that declares [AvroSchemaVersion] is never placed in the unqualified bucket.

Configuration and dependency injection

AvroOptions compiles an immutable AvroConfiguration exposing a ready resolver and serializer.

using NStalling.Avro;

var config = new AvroOptions()
    .Types(t => t
        .Add<CustomerCreated>()
        .Add<OrderPlaced>())
    .Build();

var result = config.Serializer.Deserialize<Envelope>(bytes, schema);

The NStalling.Avro.DependencyInjection project adds AddAvro for Microsoft.Extensions.DependencyInjection. It compiles eagerly so configuration defects surface during registration.

using NStalling.Avro.DependencyInjection;

services.AddAvro(o =>
    o.Types(t => t
        .Add<CustomerCreated>()
        .Add<OrderPlaced>()));

AddAvro registers:

• AvroConfiguration
• IAvroTypeResolver
• AvroSerializer

as singletons.

Opaque payloads

When an outer Avro record carries an inner payload as opaque bytes, NStalling can perform a second decode after the outer record has been read.

The application supplies the inner writer schema through IAvroPayloadSchemaSource. NStalling then uses the resulting Avro schema to resolve the CLR type and lets Apache perform the inner decode.

outer record
    ↓
payload metadata / discriminator
    ↓
IAvroPayloadSchemaSource
    ↓
inner Avro schema
    ↓
CLR type resolution
    ↓
Apache second-pass decode

Example:

public sealed class ProfileEnvelope
{
    [AvroTypeDiscriminator]
    public string PayloadType { get; init; } = "";

    [AvroVersionDiscriminator]
    public string? PayloadVersion { get; init; }

    [AvroPolymorphic]
    public object Payload { get; set; } = null!;
}

var config = new AvroOptions()
    .Types(t => t
        .Add<LegacyProfile>()
        .Add<CurrentProfile>())
    .Polymorphic<ProfileEnvelope>(p => p
        .Member(e => e.Payload)
        .PayloadSchema(myPayloadSchemaSource))
    .Build();

IAvroPayloadSchemaSource.TryGetWriterSchema distinguishes an ordinary not-found from an infrastructure failure.

Discriminator values are never treated as arbitrary CLR type names. Resolved CLR types remain limited to the configured type registry and explicitly scanned assemblies.

Unknown or missing type identity is governed by AvroUnrecognizedTypeDiscriminatorHandling:

A missing version discriminator is different: it simply means no version qualifier was supplied, so normal unqualified version resolution applies.

Payload-schema, CLR-resolution, and inner-decode failures are typed separately and are never diverted by the type-discriminator handling policy.

Exceptions

Failures inside the Avro materialization pipeline derive from AvroSerializationException and carry relevant path/schema/version/discriminator context while preserving the originating exception as InnerException.

• AvroPayloadSchemaException — the payload schema source failed.
• AvroTypeResolutionException — no or ambiguous CLR mapping, declared-type incompatibility, or unrecognized type identity under Fail.
• AvroPayloadDecodeException — the second-pass decode of an isolated payload buffer failed.

Configuration defects use AvroConfigurationException.

Cancellation and ordinary API argument-validation exceptions retain normal .NET semantics and are not wrapped.

Samples

Runnable examples are available under NStalling.Avro.Samples:

• EventEnvelope — resolves concrete CLR types for an object payload backed by an Avro union.
• Annotations — demonstrates discriminators, opaque payloads, and versioned CLR types that share an Avro record name.
• DependencyInjection — configures the union scenario through IServiceCollection.AddAvro and resolves AvroSerializer from the service provider.
• TypeResolver — exercises IAvroTypeResolver directly, including version-qualified resolution, declared-type compatibility, and absence handling.

From the src directory:

dotnet run --project NStalling.Avro.Samples/EventEnvelope
dotnet run --project NStalling.Avro.Samples/Annotations
dotnet run --project NStalling.Avro.Samples/DependencyInjection
dotnet run --project NStalling.Avro.Samples/TypeResolver

Build and test

From the src directory:

dotnet build NStalling.Avro.sln
dotnet test NStalling.Avro.sln

License

Apache License 2.0. See LICENSE.