Microsoft.Azure.Data.SchemaRegistry.ApacheAvro 1.0.0-beta.3

The ID prefix of this package has been reserved for one of the owners of this package by NuGet.org. Prefix Reserved
This is a prerelease version of Microsoft.Azure.Data.SchemaRegistry.ApacheAvro.
There is a newer version of this package available.
See the version list below for details.
dotnet add package Microsoft.Azure.Data.SchemaRegistry.ApacheAvro --version 1.0.0-beta.3
NuGet\Install-Package Microsoft.Azure.Data.SchemaRegistry.ApacheAvro -Version 1.0.0-beta.3
This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package.
<PackageReference Include="Microsoft.Azure.Data.SchemaRegistry.ApacheAvro" Version="1.0.0-beta.3" />
For projects that support PackageReference, copy this XML node into the project file to reference the package.
paket add Microsoft.Azure.Data.SchemaRegistry.ApacheAvro --version 1.0.0-beta.3
#r "nuget: Microsoft.Azure.Data.SchemaRegistry.ApacheAvro, 1.0.0-beta.3"
#r directive can be used in F# Interactive and Polyglot Notebooks. Copy this into the interactive tool or source code of the script to reference the package.
// Install Microsoft.Azure.Data.SchemaRegistry.ApacheAvro as a Cake Addin
#addin nuget:?package=Microsoft.Azure.Data.SchemaRegistry.ApacheAvro&version=1.0.0-beta.3&prerelease

// Install Microsoft.Azure.Data.SchemaRegistry.ApacheAvro as a Cake Tool
#tool nuget:?package=Microsoft.Azure.Data.SchemaRegistry.ApacheAvro&version=1.0.0-beta.3&prerelease

Azure Schema Registry Apache Avro client library for .NET

Azure Schema Registry is a schema repository service hosted by Azure Event Hubs, providing schema storage, versioning, and management. This package provides an Avro serializer capable of serializing and deserializing payloads containing Schema Registry schema identifiers and Avro-encoded data.

Getting started

Install the package

Install the Azure Schema Registry Apache Avro library for .NET with NuGet:

dotnet add package Microsoft.Azure.Data.SchemaRegistry.ApacheAvro --version 1.0.0-beta.1

Prerequisites

If you need to create an Event Hubs namespace, you can use the Azure Portal or Azure PowerShell.

You can use Azure PowerShell to create the Event Hubs namespace with the following command:

New-AzEventHubNamespace -ResourceGroupName myResourceGroup -NamespaceName namespace_name -Location eastus

Authenticate the client

In order to interact with the Azure Schema Registry service, you'll need to create an instance of the Schema Registry Client class. To create this client, you'll need Azure resource credentials and the Event Hubs namespace hostname.

Get credentials

To acquire authenicated credentials and start interacting with Azure resources, please see the quickstart guide here.

Get Event Hubs namespace hostname

The simpliest way is to use the Azure portal and navigate to your Event Hubs namespace. From the Overview tab, you'll see Host name. Copy the value from this field.

Create SchemaRegistryClient

Once you have the Azure resource credentials and the Event Hubs namespace hostname, you can create the SchemaRegistryClient. You'll also need the Azure.Identity package to create the credential.

// Create a new SchemaRegistry client using the default credential from Azure.Identity using environment variables previously set,
// including AZURE_CLIENT_ID, AZURE_CLIENT_SECRET, and AZURE_TENANT_ID.
// For more information on Azure.Identity usage, see: https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/identity/Azure.Identity/README.md
var schemaRegistryClient = new SchemaRegistryClient(fullyQualifiedNamespace: fullyQualifiedNamespace, credential: new DefaultAzureCredential());

Key concepts

ObjectSerializer

This library provides a serializer, SchemaRegistryAvroObjectSerializer, that implements the ObjectSerializer abstract class. This allows a developer to use this serializer in any .NET Azure SDKs that utilize ObjectSerializer. The SchemaRegistryAvroObjectSerializer utilitizes a SchemaRegistryClient to construct messages using a wire format containing schema information such as a schema ID.

This serializer requires the Apache Avro library. The payload types accepted by this serializer include GenericRecord and ISpecificRecord.

Wire Format

The serializer in this library creates messages in a wire format. The format is the following:

  • Bytes [0-3] – record format indicator – currently is \x00\x00\x00\x00
  • Bytes [4-35] – UTF-8 GUID, identifying the schema in a Schema Registry instance
  • Bytes [36-end] – serialized payload bytes

Examples

The following shows examples of what is available through the SchemaRegistryAvroObjectSerializer. There are both sync and async methods available for these operations. These examples use a generated Apache Avro class Employee.cs created using this schema:

{
   "type" : "record",
    "namespace" : "TestSchema",
    "name" : "Employee",
    "fields" : [
        { "name" : "Name" , "type" : "string" },
        { "name" : "Age", "type" : "int" }
    ]
}

Details on generating a class using the Apache Avro library can be found in the Avro C# Documentation.

Serialize

Register a schema to be stored in the Azure Schema Registry.

var employee = new Employee { Age = 42, Name = "John Doe" };

using var memoryStream = new MemoryStream();
var serializer = new SchemaRegistryAvroObjectSerializer(schemaRegistryClient, groupName, new SchemaRegistryAvroObjectSerializerOptions { AutoRegisterSchemas = true });
serializer.Serialize(memoryStream, employee, typeof(Employee), CancellationToken.None);

Deserialize

Retrieve a previously registered schema ID from the Azure Schema Registry.

var serializer = new SchemaRegistryAvroObjectSerializer(schemaRegistryClient, groupName, new SchemaRegistryAvroObjectSerializerOptions { AutoRegisterSchemas = true });
memoryStream.Position = 0;
Employee employee = (Employee)serializer.Deserialize(memoryStream, typeof(Employee), CancellationToken.None);

Troubleshooting

Information on troubleshooting steps will be provided as potential issues are discovered.

Next steps

See Azure Schema Registry for additional information.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit cla.microsoft.com.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

Product Compatible and additional computed target framework versions.
.NET net5.0 was computed.  net5.0-windows was computed.  net6.0 was computed.  net6.0-android was computed.  net6.0-ios was computed.  net6.0-maccatalyst was computed.  net6.0-macos was computed.  net6.0-tvos was computed.  net6.0-windows was computed.  net7.0 was computed.  net7.0-android was computed.  net7.0-ios was computed.  net7.0-maccatalyst was computed.  net7.0-macos was computed.  net7.0-tvos was computed.  net7.0-windows was computed.  net8.0 was computed.  net8.0-android was computed.  net8.0-browser was computed.  net8.0-ios was computed.  net8.0-maccatalyst was computed.  net8.0-macos was computed.  net8.0-tvos was computed.  net8.0-windows was computed. 
.NET Core netcoreapp2.0 was computed.  netcoreapp2.1 was computed.  netcoreapp2.2 was computed.  netcoreapp3.0 was computed.  netcoreapp3.1 was computed. 
.NET Standard netstandard2.0 is compatible.  netstandard2.1 was computed. 
.NET Framework net461 was computed.  net462 was computed.  net463 was computed.  net47 was computed.  net471 was computed.  net472 was computed.  net48 was computed.  net481 was computed. 
MonoAndroid monoandroid was computed. 
MonoMac monomac was computed. 
MonoTouch monotouch was computed. 
Tizen tizen40 was computed.  tizen60 was computed. 
Xamarin.iOS xamarinios was computed. 
Xamarin.Mac xamarinmac was computed. 
Xamarin.TVOS xamarintvos was computed. 
Xamarin.WatchOS xamarinwatchos was computed. 
Compatible target framework(s)
Included target framework(s) (in package)
Learn more about Target Frameworks and .NET Standard.

NuGet packages (2)

Showing the top 2 NuGet packages that depend on Microsoft.Azure.Data.SchemaRegistry.ApacheAvro:

Package Downloads
Furly.Azure.EventHubs

Furly Azure Event Hubs extension

Microsoft.Azure.Kafka.SchemaRegistry.Avro The ID prefix of this package has been reserved for one of the owners of this package by NuGet.org.

Azure Schema Registry is a hosted schema repository service provided by Azure Event Hubs, designed to simplify schema management and data governance.

GitHub repositories

This package is not used by any popular GitHub repositories.

Version Downloads Last updated
1.0.0 223,368 5/12/2022
1.0.0-beta.8 1,794 4/6/2022
1.0.0-beta.7 15,160 3/11/2022
1.0.0-beta.6 435 2/10/2022
1.0.0-beta.5 4,277 1/14/2022
1.0.0-beta.4 17,725 11/12/2021
1.0.0-beta.3 6,261 10/6/2021
1.0.0-beta.2 2,493 8/17/2021
1.0.0-beta.1 80,983 9/9/2020