Optima.Net.Events 4.0.0

Optima.Net.Events

Schema-driven event serialization for .NET using Apache Avro and Protocol Buffers.

Optima.Net.Events provides a consistent event envelope, declared and dynamic payloads, deterministic schema generation, schema hashing, and serialization/deserialization for distributed applications. It is transport-independent and can be used with Kafka, RabbitMQ, Azure Service Bus, HTTP, and other messaging infrastructure.

The latest implementation adds payload List<T> support for standard C#/.NET scalar types in both Avro and Protobuf, while preserving the distinction between event metadata and business payload data.

If this project is useful, please consider supporting its development: Buy me a coffee.

Contents

• What the library provides
• Consuming events without a shared payload DLL
• Architecture and core types
• Event metadata
• EventContext: contextual event metadata
• Stream position: StreamId and Ordinal
• Getting started with UserAdded
• Avro: declared payload round trip
• Avro: dynamic payload round trip
• Protobuf: declared payload round trip
• Protobuf: dynamic payload round trip
• Supported payload List<T> types
• Optional<T> contract
• Schemas, hashes, and compatibility
• Critical: Schema identification before deserialization
• Central schema storage: Schema registry or Amazon S3
• Recommended schema-generation tooling
• Diagnostics and schema files
• Integration guidance
• Platform and license

What the library provides

• One generic envelope: GenericEvent<TPayload> contains event identity, origin, version, correlation, causation, deduplication, stream position, and payload.
• Two payload styles: declared domain payloads built on Payload<TPayload> and runtime-defined DynamicPayload instances.
• Avro support: schema generation from an event, binary serialization, and deserialization of declared or dynamic payloads; supports separate writer and reader schemas.
• Protobuf support: .proto generation from payload fields, binary serialization of event metadata and payload data, and deserialization into the caller-requested declared payload type or DynamicPayload.
• Deterministic schema field order: payload fields are sorted during schema generation and names are normalized, with duplicate normalized names rejected.
• Schema hash checking: the serialized envelope contains a hash of the schema text; deserialization checks the expected hash. For schema discovery, the publishing application should also include that hash in the transport message header.
• Payload scalar lists: round-trip support for an explicit set of List<T> element types, including List<DateOnly>, List<Guid>, and List<decimal>.
• Controlled optional metadata: Optional<Guid> and Optional<long> are supported for specified envelope fields. Payload-level Optional<T> is intentionally unsupported.
• Transport independence: serialization is not coupled to a broker, database, or messaging transport.

Critical: Schema identification before deserialization

The schema hash must be available in the transport message header (or the transport's equivalent message attributes). A consumer needs to identify and retrieve the correct writer schema before it can deserialize the event body.

Optima.Net.Events also writes SchemaHash inside the serialized event envelope. That embedded value is used for validation after the consumer supplies the schema. It does not replace the transport header: reading the envelope to discover the hash would itself require deserialization.

Recommended transport message headers

Header or attribute	Purpose
schema-hash	Required by the recommended integration contract. Identifies the exact published writer schema.
serialization-format	Identifies whether the body uses avro or protobuf.
event-type	Optional routing or observability information, such as UserAdded; not a substitute for the schema hash.

These are suggested header names, not headers automatically written by the NuGet package. The publishing application's transport adapter must set them, and the consuming adapter must read them. For example:

Message headers:
  schema-hash: 524A125E8FD3060BD98B96C294B9576AB408062C78177DA202B527B27E5B8674
  serialization-format: protobuf
  event-type: UserAdded

Message body:
  <Protobuf-encoded event bytes>

Consumer workflow:

1. Read schema-hash and serialization-format from the transport headers.
2. Look up the exact writer schema using the format and hash, from a schema registry, Amazon S3, or an approved local cache.
3. Verify that the retrieved schema text produces the supplied hash.
4. Pass the original event bytes and retrieved schema to the appropriate Avro or Protobuf deserializer.
5. Let Optima.Net.Events validate the SchemaHash embedded in the envelope against that supplied schema.

Do not select a schema using only EventType, SchemaVersion, or the receiving application's current payload definition. More than one historical schema may exist for an event type. Central schema storage and the associated schema-generation tooling are explained below.

Consuming events without a shared payload DLL

A consumer does not need the producer's UserAdded class or its assembly to read the event's business data. A producer can serialize a declared GenericEvent<UserAdded>, while an independent consumer can deserialize the same bytes as GenericEvent<DynamicPayload> by using the corresponding Avro or Protobuf schema.

This is schema-based consumption, not a conversion of the producer's original CLR type. The event still has EventType = "UserAdded", but the consumer deliberately chooses DynamicPayload because it does not reference the UserAdded contract. A consumer that does reference the payload assembly can choose strongly typed deserialization instead.

Producer publishes	Consumer has UserAdded DLL?	Consumer deserializes as
GenericEvent<UserAdded>	Yes	GenericEvent<UserAdded>
GenericEvent<UserAdded>	No	GenericEvent<DynamicPayload>
GenericEvent<DynamicPayload>	Not required	GenericEvent<DynamicPayload>

Producer: publish a strongly typed UserAdded event

The producing service references its UserAdded class, as defined in the getting-started example:

var userAdded = new UserAdded(1, "John Doe");

var evt = GenericEvent<UserAdded>.Create(
    Optional<Guid>.Some(Guid.CreateVersion7()),
    userAdded,
    Optional<string>.Some(userAdded.PayloadName),
    "user-service",
    "V1.0.0",
    EventContext.Create());

// Avro
var avroSchema = AvroSchemaGenerator.GenerateSchemaFromGenericEvent(evt);
byte[] avroBytes = AvroEventSerializer.Serialize(evt, avroSchema);

// Protobuf
var protoSchema = ProtobufSchemaGenerator.GenerateSchemaProto(
    userAdded.Fields, userAdded.PayloadName, "events");
byte[] protoBytes = ProtobufEventSerializer.Serialize(evt, protoSchema);

The producer's transport adapter should put the schema hash in the message header, alongside the serialization format. The consumer uses those headers to retrieve the exact corresponding schema before deserializing the body. The schema is not inferred from the event name alone. See Critical: Schema identification before deserialization.

Consumer: deserialize without UserAdded or its DLL

The following consumer code depends on Optima.Net.Events, not on UserAdded. Here receivedAvroBytes/receivedProtoBytes and the matching schemas represent data provided to the consuming application.

Avro:

GenericEvent<DynamicPayload> receivedAvro =
    AvroEventSerializer.Deserialize<DynamicPayload>(
        receivedAvroBytes, writerAvroSchemaJson);

int userId = (int)receivedAvro.Payload.Fields["UserID"];
string userName = (string)receivedAvro.Payload.Fields["UserName"];
var dates = (List<DateOnly>)receivedAvro.Payload.Fields["InterestingDates"];

Protobuf:

GenericEvent<DynamicPayload> receivedProto =
    ProtobufEventSerializer.Deserialize(receivedProtoBytes, protoSchema);

int userId = (int)receivedProto.Payload.Fields["UserID"];
string userName = (string)receivedProto.Payload.Fields["UserName"];
var dates = (List<DateOnly>)receivedProto.Payload.Fields["InterestingDates"];

Both consumers can inspect receivedAvro.EventType or receivedProto.EventType to identify "UserAdded", and use Fields.TryGetValue(...) when a field may be absent. Neither needs to declare a UserAdded CLR type, load the producer's domain assembly, or convert the data to JSON first.

List<DateOnly> reconstruction in these examples depends on retaining the generated Optima.Net.Events schema hints (optimaClrElementType for Avro and optima-clr-element-type for Protobuf). Removing those hints can leave the receiver with a list of underlying wire values such as strings rather than the original CLR element type. For general-purpose consumers, the Fields dictionary can be inspected without relying on a particular domain type.

The choice is made by the consumer, not by the producer: typed consumers get GenericEvent<UserAdded>; schema-only consumers get GenericEvent<DynamicPayload>. This decouples event-data access from sharing domain-specific payload DLLs while keeping schema management an explicit responsibility.

Architecture and core types

Domain / Application
    |
    +-- Declared payload: Payload<TPayload> / IPayload
    |             or
    +-- DynamicPayload / IPayload
                  |
           GenericEvent<TPayload>
                  |
           Schema generation
             |           |
       Avro schema   Protobuf schema
             |           |
       Avro serializer  Protobuf serializer
             |           |
             +----- Serialized bytes -----+
                          |
                    Your transport
                          |
                    Deserializer
                          |
               Reconstructed event

The main API types are:

Type	Responsibility
IPayload	Payload contract, including its name and field dictionary.
Payload<TPayload>	Base class for declared payloads.
DynamicPayload	Construct an event payload at runtime by adding named fields.
GenericEvent<TPayload>	Typed event envelope created through its Create factory.
EventContext	Immutable optional correlation, causation, deduplication, and originating-stream metadata; enforces the StreamId/Ordinal dependency.
AvroSchemaGenerator	Generates an Avro schema from a sample generic event.
AvroEventSerializer	Serializes and deserializes events using Avro schemas.
ProtobufSchemaGenerator	Generates a .proto schema from payload fields.
ProtobufEventSerializer	Serializes and deserializes declared and dynamic events through typed and dynamic overloads.

Event metadata

GenericEvent<TPayload> provides these fields:

Field	Meaning
EventId	Event identifier (Guid).
EventType	Event name/type.
Source	Originating application or component.
Timestamp	Event timestamp.
SchemaVersion	Producer-assigned schema version.
SchemaHash	Hash of the schema text used for serialization.
CausationEventId	Optional identifier of the event that caused this event.
CorrelationId	Optional identifier for linking a related sequence of events.
DeDuplicationId	Optional identifier for consumer-side deduplication workflows.
StreamId	Optional identifier of the event stream to which the event belongs (Optional<Guid>).
Ordinal	Optional position of the event within its stream (Optional<long>).
Payload	Business data implementing IPayload.

The library models metadata separately from payload fields, even though the Avro schema represents envelope metadata and payload fields together in one record.

Create events with GenericEvent<TPayload>.Create(...); do not rely on constructing a partially populated event and setting properties individually.

EventContext: contextual event metadata

EventContext is a sealed record containing context about an event, separate from the event's business payload. It tells consumers how an event relates to other events and where it belongs in an originating event stream. GenericEvent<TPayload>.Create(...) receives an EventContext and copies its contextual metadata onto the event envelope.

The five properties of EventContext are:

Property	CLR type	Purpose
CorrelationId	Optional<Guid>	Associates events taking part in the same logical operation, workflow, or business process. Several events may have the same correlation ID.
CausationEventId	Optional<Guid>	Identifies the event that directly caused this event. This is different from a correlation ID, which links the broader operation.
DeDuplicationId	Optional<Guid>	Supplies an identifier consumers or messaging infrastructure can use to recognize repeated delivery or processing of the same logical event. The library does not automatically deduplicate events.
StreamId	Optional<Guid>	Identifies the originating event stream to which the event belongs.
Ordinal	Optional<long>	Identifies the event's position within that originating stream, not a position across every stream.

Creating an EventContext

All five values are optional. Use EventContext.Create() when none is available:

var context = EventContext.Create();

When context is known, use the five-argument factory. A typical example is a follow-up event caused by another event, with a correlation ID for the overall workflow and a stream position supplied by the producing application:

var context = EventContext.Create(
    correlationId: Optional<Guid>.Some(workflowId),
    causationEventId: Optional<Guid>.Some(previousEventId),
    deDuplicationId: Optional<Guid>.Some(deduplicationId),
    streamId: Optional<Guid>.Some(originatingStreamId),
    ordinal: Optional<long>.Some(42));

Here workflowId, previousEventId, deduplicationId, and originatingStreamId are Guid values already known to the producer. The producer chooses those identifiers according to its own event-processing rules; the serialization library does not generate them.

Construction invariant: An ordinal is meaningful only within a stream. Consequently, EventContext.Create(...) throws ArgumentException if ordinal has a value but streamId is empty. A StreamId may be provided without an Ordinal. Neither field is required when no stream position is available.

// Valid: a stream is known, but the event position is not.
var contextWithStreamOnly = EventContext.Create(
    Optional<Guid>.Empty,
    Optional<Guid>.Empty,
    Optional<Guid>.Empty,
    Optional<Guid>.Some(Guid.NewGuid()),
    Optional<long>.Empty);

// Invalid: an ordinal without its originating stream.
// EventContext.Create(..., Optional<Guid>.Empty, Optional<long>.Some(42))
// throws ArgumentException.

Optional<T>.Empty means the contextual value was not provided. It does not mean the event is uncorrelated, unique, or outside a stream. These values are envelope metadata only: payload fields must not contain Optional<T>.

Correlation versus causation

For example, suppose one business operation results in UserAdded, then WelcomeEmailRequested:

• Both events can share a CorrelationId because they belong to the same business operation.
• WelcomeEmailRequested.CausationEventId can equal the EventId of UserAdded, because that particular event directly caused the next one.
• Each event still has its own EventId. A DeDuplicationId, when supplied, serves a separate duplicate-detection purpose.

The identifiers convey meaning to consumers; EventContext does not perform workflow orchestration, causality resolution, or duplicate removal.

Stream position: StreamId and Ordinal

StreamId identifies an event stream, while Ordinal records the position of an event within that stream. Together, they let a consumer distinguish events belonging to different streams and understand each event's position in its own stream.

Both values are optional envelope metadata:

• StreamId is Optional<Guid>.
• Ordinal is Optional<long>.

For example, when a producer knows the stream and event position:

var streamId = Guid.NewGuid();

var context = EventContext.Create(
    correlationId: Optional<Guid>.Empty,
    causationEventId: Optional<Guid>.Empty,
    deDuplicationId: Optional<Guid>.Empty,
    streamId: Optional<Guid>.Some(streamId),
    ordinal: Optional<long>.Some(42));

var evt = GenericEvent<UserAdded>.Create(
    Optional<Guid>.Some(Guid.CreateVersion7()),
    new UserAdded(1, "John Doe"),
    Optional<string>.Some("UserAdded"),
    "user-service",
    "V1.0.0",
    context);

Here, Ordinal = 42 indicates the event's position in the stream identified by StreamId. It is not a global event number or a Kafka partition offset. Different streams can have events with the same ordinal.

Within the complete originating stream, ordinals represent contiguous, monotonically increasing positions: there are no gaps in a complete stream. A filtered or projected view may legitimately show gaps because some originating events have been omitted. Do not renumber ordinals when filtering, projecting, or replaying events: their values must continue to identify positions in the original stream.

Consumers can use stream position metadata for ordered processing, replay, gap detection, and reconciliation when the producing system assigns and maintains meaningful positions. EventContext enforces that an ordinal has a stream ID, but does not check adjacent events for continuity or allocate positions. Optima.Net.Events carries both values through Avro and Protobuf serialization; it does not generate ordinals, guarantee delivery order, or enforce gap-free sequences. Those responsibilities belong to the event-producing and consuming systems.

When stream position is not available, Ordinal may be Optional<long>.Empty; consumers must not infer a position from a missing value.

Getting started with UserAdded

The examples below use the same UserAdded payload as the test harness. It carries two scalar fields and a List<DateOnly>.

using Optima.Net.Events.Models;
using Optima.Net.Events.Models.Interfaces;
using System;
using System.Collections.Generic;

public class UserAdded : Payload<UserAdded>, IPayload
{
    private UserAdded() : base(nameof(UserAdded)) { }

    public UserAdded(int userID, string userName)
        : base(nameof(UserAdded))
    {
        Add("UserID", userID);
        Add("UserName", userName);
        Add("InterestingDates", new List<DateOnly>
        {
            new(2024, 12, 2),
            new(2024, 12, 3),
            new(2024, 12, 4)
        });
    }
}

The non-public parameterless constructor is required by the Avro and Protobuf declared-payload deserializers, which reconstruct the payload without calling its business constructor.

Create the envelope:

using Optima.Net;
using Optima.Net.Events.Models;

var userAdded = new UserAdded(1, "John Doe");
var context = EventContext.Create();

var evt = GenericEvent<UserAdded>.Create(
    Optional<Guid>.Some(Guid.CreateVersion7()),
    userAdded,
    Optional<string>.Some(userAdded.PayloadName),
    "user-service",
    "V1.0.0",
    context);

For a real consumer, retain the schema that was used by the writer. Generating a new schema from unrelated sample values is not a substitute for retaining the event's original schema.

Avro: declared payload round trip

Avro generates a record schema containing metadata fields and payload fields.

using Optima.Net.Events.Services;

var avroSchema = AvroSchemaGenerator.GenerateSchemaFromGenericEvent(evt);

byte[] avroBytes = AvroEventSerializer.Serialize(
    evt,
    avroSchema,
    out string avroSchemaHash);

GenericEvent<UserAdded> avroResult =
    AvroEventSerializer.Deserialize<UserAdded>(avroBytes, avroSchema);

var dates = (List<DateOnly>)avroResult.Payload.Fields["InterestingDates"];

Console.WriteLine(avroSchemaHash);
Console.WriteLine(string.Join(", ", dates));
// 12/02/2024, 12/03/2024, 12/04/2024 (display depends on culture)

You can also provide separate writer and reader schemas:

var evolvedResult = AvroEventSerializer.Deserialize<UserAdded>(
    avroBytes,
    writerSchemaJson: avroSchema,
    readerSchemaJson: avroSchema);

The two-schema overload enables Avro reader/writer schema resolution; it does not guarantee that every possible schema change is compatible. The example passes the same schema for both arguments.

The relevant portion of the generated Avro schema for the date list looks like this:

{
  "name": "InterestingDates",
  "type": {
    "type": "array",
    "items": "string",
    "optimaClrElementType": "System.DateOnly"
  }
}

The extra optimaClrElementType attribute tells Optima.Net.Events that the serialized string values should be reconstructed as DateOnly elements.

Avro: dynamic payload round trip

A DynamicPayload does not automatically acquire fields from a declared payload just because the PayloadName is the same. Add every desired field explicitly.

using Optima.Net;
using Optima.Net.Events.Models;
using Optima.Net.Events.Payloads;
using Optima.Net.Events.Services;

var dynamicPayload = new DynamicPayload("UserAdded");
dynamicPayload.Add("UserID", 1);
dynamicPayload.Add("UserName", "John Doe");
dynamicPayload.Add("InterestingDates", new List<DateOnly>
{
    new(2024, 12, 2),
    new(2024, 12, 3),
    new(2024, 12, 4)
});

var dynamicEvent = GenericEvent<DynamicPayload>.Create(
    Optional<Guid>.Some(Guid.CreateVersion7()),
    dynamicPayload,
    Optional<string>.Some(dynamicPayload.PayloadName),
    "user-service",
    "V1.0.0",
    EventContext.Create());

var dynamicAvroSchema =
    AvroSchemaGenerator.GenerateSchemaFromGenericEvent(dynamicEvent);

var dynamicAvroBytes =
    AvroEventSerializer.Serialize(dynamicEvent, dynamicAvroSchema);

var dynamicAvroResult =
    AvroEventSerializer.Deserialize<DynamicPayload>(
        dynamicAvroBytes,
        dynamicAvroSchema);

var dynamicDates =
    (List<DateOnly>)dynamicAvroResult.Payload.Fields["InterestingDates"];

Protobuf: declared payload round trip

Protobuf generates a schema from the payload's field dictionary. It does not generate its metadata as ordinary payload fields; the serializer writes the envelope in its defined wire layout.

using Optima.Net.Events.Services;

var protoSchema = ProtobufSchemaGenerator.GenerateSchemaProto(
    userAdded.Fields,
    userAdded.PayloadName,
    "events");

byte[] protoBytes = ProtobufEventSerializer.Serialize(
    evt,
    protoSchema,
    out string protoSchemaHash);

GenericEvent<UserAdded> protoResult =
    ProtobufEventSerializer.Deserialize<UserAdded>(protoBytes, protoSchema);

var protoDates =
    (List<DateOnly>)protoResult.Payload.Fields["InterestingDates"];

Console.WriteLine(protoSchemaHash);
Console.WriteLine(protoDates.Count); // 3
Console.WriteLine(protoResult.EventId == evt.EventId); // True
Console.WriteLine(protoResult.Timestamp == evt.Timestamp); // True

The generated payload .proto includes:

syntax = "proto3";
package events;

message UserAdded {
  repeated string InterestingDates = 1; // optima-clr-element-type: System.DateOnly
  int32 UserID = 2;
  string UserName = 3;
}

repeated string is valid Protobuf. The comment is an Optima.Net.Events schema hint that enables the deserializer to recover List<DateOnly> rather than List<string>. Keep that hint when preserving or distributing the schema.

The type parameter is essential: Deserialize<UserAdded>(...) reconstructs GenericEvent<UserAdded>, including the original typed payload fields. The non-generic Deserialize(...) overload is retained for callers intentionally consuming a GenericEvent<DynamicPayload>. Neither overload silently changes the declared event type requested by the caller.

Protobuf: dynamic payload round trip

The same serializer works when the sending payload is already dynamic:

var dynamicProtoSchema = ProtobufSchemaGenerator.GenerateSchemaProto(
    dynamicPayload.Fields,
    dynamicPayload.PayloadName,
    "events");

var dynamicProtoBytes = ProtobufEventSerializer.Serialize(
    dynamicEvent,
    dynamicProtoSchema);

var dynamicProtoResult = ProtobufEventSerializer.Deserialize(
    dynamicProtoBytes,
    dynamicProtoSchema);

var dynamicProtoDates =
    (List<DateOnly>)dynamicProtoResult.Payload.Fields["InterestingDates"];

Console.WriteLine(dynamicProtoDates.Count); // 3

This reuses dynamicPayload and dynamicEvent from the preceding dynamic Avro example. The dynamic payload explicitly includes InterestingDates, so the list is present in the Protobuf schema, serialized data, and reconstructed payload.

Verify the original CLR payload type

The JSON representation alone is not proof of type preservation. Verify the actual event and list types in a test:

GenericEvent<UserAdded> restored =
    ProtobufEventSerializer.Deserialize<UserAdded>(protoBytes, protoSchema);

if (restored.Payload is null || restored.Payload.GetType() != typeof(UserAdded))
    throw new InvalidOperationException("Protobuf did not restore UserAdded.");

if (restored.Payload.Fields["InterestingDates"] is not List<DateOnly> dates ||
    dates.Count != 3 ||
    dates[0] != new DateOnly(2024, 12, 2) ||
    dates[1] != new DateOnly(2024, 12, 3) ||
    dates[2] != new DateOnly(2024, 12, 4))
    throw new InvalidOperationException("Protobuf did not restore List<DateOnly>.");

if (restored.EventId != evt.EventId || restored.Timestamp != evt.Timestamp)
    throw new InvalidOperationException("Protobuf event metadata changed.");

For an event originally created with DynamicPayload, use Deserialize<DynamicPayload>(...) or the existing non-generic Deserialize(...). Declared payloads and dynamic payloads are separate API choices; no automatic conversion from a declared payload to a dynamic payload is required.

Supported payload List<T> types

The supported element types are standard CLR scalar types. This contract applies to payload fields declared as List<T> only, not event metadata or arbitrary nested object graphs.

Category	Supported T in List<T>
Text and boolean	string, char, bool
Signed integers	sbyte, short, int, long, Int128, nint (IntPtr)
Unsigned integers	byte, ushort, uint, ulong, UInt128, nuint (UIntPtr)
Floating point	Half, float, double
Exact decimal	decimal
Identifier	Guid
Dates and times	DateTime, DateTimeOffset, DateOnly, TimeOnly, TimeSpan

Both schema generators and serializers use the same internal scalar-list mapping contract (PayloadScalarListCodec).

Value and type fidelity

• Native wire types are used where the Avro or Protobuf type identifies the CLR value sufficiently.
• Other CLR elements use a compatible underlying wire scalar, plus a schema-level type hint. For example, List<DateOnly> uses strings with a System.DateOnly hint.
• List<decimal> uses invariant string values rather than conversion to double, avoiding loss of decimal precision for list elements.
• Type hints are part of the supplied schema, not extra event metadata or payload fields.
• Reconstructing the original CLR type depends on using the matching, intact schema. A general-purpose Avro or Protobuf reader can read the wire primitives but may not understand Optima.Net.Events-specific hints.
• In Protobuf, repeated numeric fields can be read in packed or unpacked form.

Deliberate limits

This scalar-list contract does not include List<T?>, null list elements, nested lists such as List<List<int>>, enum lists, or arbitrary user-defined object/record/struct lists. Avro's existing handling of certain structured list elements is separate and is not a promise of equivalent Protobuf support.

It also does not extend the scalar non-list payload mappings. In particular, precision guarantees for List<decimal> must not be read as a new guarantee for an individual scalar decimal field.

nint and nuint depend on the receiver's native pointer size. A serialized value that exceeds that platform's supported range cannot be restored on that receiver.

Optional<T> contract

Optional<T> is part of the event envelope's controlled metadata model. It is not supported inside business payloads.

Supported envelope uses include:

var streamId = Guid.NewGuid();
var context = EventContext.Create(
    correlationId: Optional<Guid>.Some(Guid.NewGuid()),
    causationEventId: Optional<Guid>.Empty,
    deDuplicationId: Optional<Guid>.Empty,
    streamId: Optional<Guid>.Some(streamId),
    ordinal: Optional<long>.Some(42));

If application code uses Optional<T> for business decisions, resolve it before adding a payload value:

Optional<string> optionalNickname = Optional<string>.Some("Joey");

var payload = new DynamicPayload("NicknameChanged");
payload.Add("Nickname", optionalNickname.Value);

Payload-level Optional<T> (including List<Optional<T>> or an Optional<T> inside a nested payload type) is rejected rather than silently unwrapped. That keeps business interpretation of missing versus present values in the producing application.

Schemas, hashes, and compatibility

Deterministic schemas

The generators order payload fields using ordinal key/name ordering and normalize field names. They reject collisions resulting from normalization. Given the same event structure, schema-generation inputs, and version of the generator, the output is stable.

Schema hashes are computed from the schema text; they are not derived solely from a list of CLR types. Formatting changes, type hints, and field changes can change a hash.

Hash validation

During serialization, the serializer computes a hash from the supplied schema and embeds it in the event envelope. During deserialization, it checks that the embedded hash matches the writer/schema text it was given. A mismatch results in a deserialization error.

Schema-hash validation helps detect schema mismatches. It is not a cryptographic signature, authentication mechanism, or proof that the producer is trusted.

Schema evolution and release compatibility

Schema hashing does not automatically guarantee forward or backward compatibility. Compatibility must be established for each event/schema change and consumer combination.

• Preserve the writer schema used to produce each Avro event. Avro also supports passing an explicit reader schema where compatible.
• Protobuf consumers must use the schema expected by the current serializer. Treat field numbers, wire types, and repeated-field declarations as part of the contract.
• Preserve the optimaClrElementType Avro attribute and optima-clr-element-type Protobuf comments for CLR scalar-list reconstruction.
• Regenerate and version schemas when a payload contract changes, then publish or retain them alongside your events.
• Changes to the list implementation may change historical schema text and hashes; for example, Avro List<float> and List<decimal> mappings differ from earlier versions.
• Retain older writer schemas for historical messages and test consumers before rolling out new schemas.
• Existing serialized events are not automatically compatible simply because public C# method signatures remain unchanged.

Format-specific behavior to know

• Avro: Deserialize<TPayload>(...) supports typed and dynamic payload reconstruction; use the typed overload intentionally. A separate writer/reader schema overload is available.
• Protobuf: Deserialize<TPayload>(data, protoSchema) reconstructs GenericEvent<TPayload>; the non-generic Deserialize(data, protoSchema) explicitly reconstructs GenericEvent<DynamicPayload> for dynamic consumers. Both paths restore the transmitted event timestamp.
• Both: The source, metadata constraints, and embedded hash are validated according to each serializer's implementation. Type reconstruction is more specific than simply examining the JSON representation of a deserialized event.

Check actual list element types

JSON prints a DateOnly value as a date-like string, so visual JSON comparison is not enough to test CLR element types. In a test harness, assert the runtime type:

var restored = protoResult.Payload.Fields["InterestingDates"];

if (restored is not List<DateOnly> restoredDates || restoredDates.Count != 3)
    throw new InvalidOperationException("DateOnly list round trip failed.");

if (restoredDates[0] != new DateOnly(2024, 12, 2))
    throw new InvalidOperationException("DateOnly element value changed.");

Apply equivalent checks for Avro and other supported scalar-list types. Successful local tests validate your tested scenarios, not every potential schema evolution.

Central schema storage: Schema registry or Amazon S3

For production use, schemas should be centrally accessible to every service that publishes or consumes events and uniquely identified by their schema hashes. This is particularly important for consumers that do not reference the producer's payload DLL: they need the writer's schema to reconstruct the payload.

A central location can be:

• A schema registry: a dedicated service that stores and serves versioned Avro and Protobuf schemas. Use the SHA-256 schema hash computed by Optima.Net.Events as a lookup key, or maintain a reliable mapping between that hash and the registry's native schema identifier.
• An Amazon S3 bucket: store schema documents as immutable objects, using the hash and format in the object key, for example schemas/avro/<schema-hash>.avsc and schemas/protobuf/<schema-hash>.proto. The paths are a suggested convention, not a library requirement.

A recommended publication and consumption workflow is:

1. Generate the Avro .avsc or Protobuf .proto schema from the published payload contract.
2. Compute its schema hash using the same hashing routine as the serializer, and publish the exact schema text under that hash before events using it are sent.
3. Publish the serialized event with the schema hash in the transport message header (or equivalent message attributes) and the serialization format alongside it. The serializer also stores the schema hash inside the event envelope for subsequent validation.
4. Resolve the writer schema by hash on the consumer side, optionally caching it locally by (format, hash) to avoid retrieving it for every event.
5. Deserialize with the retrieved schema. The serializer checks the embedded schema hash against the supplied schema; the consumer can also check that the retrieved document hashes to the identifier supplied by the transport.
6. Retain schemas for as long as previously published events may need to be replayed or consumed. Do not overwrite a schema associated with an existing hash.

Important: The deserializers accept schema text; they do not retrieve schemas from a registry or S3 automatically. The transport adapter must place the schema hash in the message header and the consumer must read it before deserializing. The embedded envelope SchemaHash is for validation after schema resolution; it is not a pre-deserialization lookup mechanism. The serialization format must also be known, because a hash alone does not indicate whether the payload is Avro or Protobuf.

Keep the generated schema exactly as published. Its hash is derived from the schema text, so reformatting, editing or stripping the Optima.Net.Events CLR element-type hints can change the hash or affect List<T> reconstruction. A schema version or event name is useful metadata but should not replace the hash when selecting the precise writer schema.

Recommended schema-generation tooling

NuGet consumers should consider creating a small schema-generation and publication utility, such as a CLI, as part of their build or release process. This is recommended application tooling, not a CLI currently supplied by Optima.Net.Events.

A useful CLI could:

• Discover the project's declared payload types or load an explicitly supplied set of payload contracts.
• Generate the relevant Avro and/or Protobuf schema using AvroSchemaGenerator and ProtobufSchemaGenerator.
• Compute each schema's hash with the same helper used by serialization.
• Write .avsc and .proto artifacts and a manifest mapping event type, format, schema version and schema hash to each file.
• Compare new schemas with previously published schemas, check compatibility according to the team's rules, and fail the release if an incompatible change is not approved.
• Publish the artifacts to a central registry or S3 bucket and make the hash-to-schema mapping available to consuming services.

A conceptual command might look like this:

dotnet run --project MySchemaTool -- generate --assembly MyDomain.dll --output ./schemas

This command is illustrative only; you would implement it in your own CLI. Schema generators require appropriate inputs: Avro takes a sample GenericEvent<TPayload> and Protobuf takes a payload field dictionary and payload name. Merely finding a type in an assembly does not always provide enough information to build an accurate schema. The tool should construct representative payloads or use explicit schema definitions, and should keep field numbering and published contracts stable.

Schemas should be generated and published before deployment of producers that depend on them. For distributed systems, a CI/CD step that verifies the artifacts and their hashes is preferable to generating an untracked schema independently in every consuming service.

Diagnostics and schema files

AvroEventSerializer includes a debug-print helper:

AvroEventSerializer.PrintDebug(evt, avroSchema, includePayload: true);

The Protobuf generator can write a .proto file:

var path = ProtobufSchemaGenerator.SaveSchemaToFile(
    protoSchema,
    directoryPath: "schemas",
    fileName: "UserAdded.proto");

For production use, publish schemas centrally and make them retrievable by schema hash; see Central schema storage: Schema registry or Amazon S3. The library does not require a particular registry or transport.

Integration guidance

Layering

A typical arrangement is:

MySystem.Domain
    Payloads
    Domain events

MySystem.Application
    Event creation / handlers
    Optional payload discovery or registration

MySystem.Infrastructure
    Optima.Net.Events serializers
    Schema storage / retrieval
    Transport-specific publishing and consumption

The original project documentation references payload discovery through an application-registration extension such as AddEventPayloadDiscovery. That API was not part of the serializer and generator source used to verify this README; confirm its exact signature in your installed package before using it. Payload discovery does not change the requirement to retain and provide the correct schema.

A schema-generation CLI is recommended to generate, hash, publish and track the schemas used by producers and consumers. See Recommended schema-generation tooling; the CLI is not part of Optima.Net.Events.

Avro versus Protobuf

Concern	Avro	Protobuf
Schema	JSON record schema (.avsc)	Protobuf schema (.proto)
Generator input	Sample GenericEvent<TPayload>	Payload field dictionary and payload name
Declared payload serialization	Yes	Yes
Declared payload deserialization	Yes	Yes, through Deserialize<TPayload>(...)
Dynamic payload serialization/deserialization	Yes	Yes
Scalar List<T> payload support	Yes, for supported types	Yes, for supported types
Schema-evolution mechanism	Writer and reader schemas	Protobuf field numbering and schema management
Event metadata	Flattened into the Avro record	Serializer-defined envelope wire fields

Performance depends on message size, payload shape, schema parsing, and your infrastructure. Benchmark against representative traffic rather than treating either format as universally faster.

Platform and license

• .NET 8.0 or later (as specified by the existing project README).
• Cross-platform .NET environments: Windows, Linux, and macOS.
• MIT License. Copyright � 2025.

This README describes the scalar-list implementation and the new typed Protobuf deserialization overload. Validate the round-trip behavior in your solution before publishing the updated NuGet package.