Azure.Storage.Blobs.ChangeFeed 12.0.0-preview.29

The ID prefix of this package has been reserved for one of the owners of this package by NuGet.org. Prefix Reserved
.NET Standard 2.0
This is a prerelease version of Azure.Storage.Blobs.ChangeFeed.
dotnet add package Azure.Storage.Blobs.ChangeFeed --version 12.0.0-preview.29
NuGet\Install-Package Azure.Storage.Blobs.ChangeFeed -Version 12.0.0-preview.29
This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package.
<PackageReference Include="Azure.Storage.Blobs.ChangeFeed" Version="12.0.0-preview.29" />
For projects that support PackageReference, copy this XML node into the project file to reference the package.
paket add Azure.Storage.Blobs.ChangeFeed --version 12.0.0-preview.29
#r "nuget: Azure.Storage.Blobs.ChangeFeed, 12.0.0-preview.29"
#r directive can be used in F# Interactive, C# scripting and .NET Interactive. Copy this into the interactive tool or source code of the script to reference the package.
// Install Azure.Storage.Blobs.ChangeFeed as a Cake Addin
#addin nuget:?package=Azure.Storage.Blobs.ChangeFeed&version=12.0.0-preview.29&prerelease

// Install Azure.Storage.Blobs.ChangeFeed as a Cake Tool
#tool nuget:?package=Azure.Storage.Blobs.ChangeFeed&version=12.0.0-preview.29&prerelease

Azure Storage Blobs Change Feed client library for .NET

Server Version: 2021-02-12, 2020-12-06, 2020-10-02, 2020-08-04, 2020-06-12 2020-04-08, 2020-02-10, and 2019-12-12

The purpose of the change feed is to provide transaction logs of all the changes that occur to the blobs and the blob metadata in your storage account. The change feed provides ordered, guaranteed, durable, immutable, read-only log of these changes. Client applications can read these logs at any time. The change feed enables you to build efficient and scalable solutions that process change events that occur in your Blob Storage account at a low cost.

Source code | Package (NuGet) | Product documentation

Getting started

Install the package

Install the Azure Storage Blobs client library for .NET with NuGet:

dotnet add package Azure.Storage.Blobs.ChangeFeed --prerelease

Prerequisites

You need an Azure subscription and a Storage Account to use this package.

To create a new Storage Account, you can use the Azure Portal, Azure PowerShell, or the Azure CLI. Here's an example using the Azure CLI:

az storage account create --name MyStorageAccount --resource-group MyResourceGroup --location westus --sku Standard_LRS

Authenticate the Client

Authentication works the same as in Azure.Storage.Blobs.

Key concepts

The change feed is stored as blobs in a special container in your storage account at standard blob pricing cost. You can control the retention period of these files based on your requirements (See the conditions of the current release). Change events are appended to the change feed as records in the Apache Avro format specification: a compact, fast, binary format that provides rich data structures with inline schema. This format is widely used in the Hadoop ecosystem, Stream Analytics, and Azure Data Factory.

You can process these logs incrementally or in-full. Any number of client applications can independently read the change feed, in parallel, and at their own pace. Analytics applications such as Apache Drill or Apache Spark can consume logs directly as Avro files, which let you process them at a low-cost, with high-bandwidth, and without having to write a custom application.

Thread safety

We guarantee that all client instance methods are thread-safe and independent of each other (guideline). This ensures that the recommendation of reusing client instances is always safe, even across threads.

Additional concepts

Client options | Accessing the response | Long-running operations | Handling failures | Diagnostics | Mocking | Client lifetime

Examples

Get all events in the Change Feed

// Get all the events in the change feed.
List<BlobChangeFeedEvent> changeFeedEvents = new List<BlobChangeFeedEvent>();
await foreach (BlobChangeFeedEvent changeFeedEvent in changeFeedClient.GetChangesAsync())
{
    changeFeedEvents.Add(changeFeedEvent);
}

Get events between a start and end time

// Create the start and end time.  The change feed client will round start time down to
// the nearest hour, and round endTime up to the next hour if you provide DateTimeOffsets
// with minutes and seconds.
DateTimeOffset startTime = new DateTimeOffset(2017, 3, 2, 15, 0, 0, TimeSpan.Zero);
DateTimeOffset endTime = new DateTimeOffset(2020, 10, 7, 2, 0, 0, TimeSpan.Zero);

// You can also provide just a start or end time.
await foreach (BlobChangeFeedEvent changeFeedEvent in changeFeedClient.GetChangesAsync(
    start: startTime,
    end: endTime))
{
    changeFeedEvents.Add(changeFeedEvent);
}

Resume with continuationToken

string continuationToken = null;
await foreach (Page<BlobChangeFeedEvent> page in changeFeedClient.GetChangesAsync().AsPages(pageSizeHint: 10))
{
    foreach (BlobChangeFeedEvent changeFeedEvent in page.Values)
    {
        changeFeedEvents.Add(changeFeedEvent);
    }

    // Get the change feed continuation token.  The continuation token is not required to get each page of events,
    // it is intended to be saved and used to resume iterating at a later date.
    continuationToken = page.ContinuationToken;
    break;
}

// Resume iterating from the pervious position with the continuation token.
await foreach (BlobChangeFeedEvent changeFeedEvent in changeFeedClient.GetChangesAsync(
    continuationToken: continuationToken))
{
    changeFeedEvents.Add(changeFeedEvent);
}

Troubleshooting

All Blob service operations will throw a RequestFailedException on failure with helpful ErrorCodes. Many of these errors are recoverable.

Next steps

Get started with our Change Feed samples:

  1. Hello World: Get changes that have occured in your storage account (or asynchronously)
  2. Auth: Authenticate with connection strings, public access, shared keys, shared access signatures, and Azure Active Directory.

Contributing

See the Storage CONTRIBUTING.md for details on building, testing, and contributing to this library.

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit cla.microsoft.com.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

Product Versions
.NET net5.0 net5.0-windows net6.0 net6.0-android net6.0-ios net6.0-maccatalyst net6.0-macos net6.0-tvos net6.0-windows net7.0 net7.0-android net7.0-ios net7.0-maccatalyst net7.0-macos net7.0-tvos net7.0-windows
.NET Core netcoreapp2.0 netcoreapp2.1 netcoreapp2.2 netcoreapp3.0 netcoreapp3.1
.NET Standard netstandard2.0 netstandard2.1
.NET Framework net461 net462 net463 net47 net471 net472 net48 net481
MonoAndroid monoandroid
MonoMac monomac
MonoTouch monotouch
Tizen tizen40 tizen60
Xamarin.iOS xamarinios
Xamarin.Mac xamarinmac
Xamarin.TVOS xamarintvos
Xamarin.WatchOS xamarinwatchos
Compatible target framework(s)
Additional computed target framework(s)
Learn more about Target Frameworks and .NET Standard.

NuGet packages

This package is not used by any NuGet packages.

GitHub repositories

This package is not used by any popular GitHub repositories.

Version Downloads Last updated
12.0.0-preview.29 1,638 10/12/2022
12.0.0-preview.28 637 8/23/2022
12.0.0-preview.27 342 7/8/2022
12.0.0-preview.26 1,448 6/15/2022
12.0.0-preview.25 1,126 5/2/2022
12.0.0-preview.24 875 4/20/2022
12.0.0-preview.23 1,345 4/13/2022
12.0.0-preview.22 79 4/12/2022
12.0.0-preview.21 396 3/30/2022
12.0.0-preview.20 11,337 3/10/2022
12.0.0-preview.19 169 2/7/2022
12.0.0-preview.18 384 11/30/2021
12.0.0-preview.17 234 11/4/2021
12.0.0-preview.16 354 9/9/2021
12.0.0-preview.15 192 7/23/2021
12.0.0-preview.14 105 7/23/2021
12.0.0-preview.13 329 6/9/2021
12.0.0-preview.12 198 5/12/2021
12.0.0-preview.11 2,671 4/9/2021
12.0.0-preview.10 523 3/10/2021
12.0.0-preview.9 208 2/10/2021
12.0.0-preview.8 56,607 1/12/2021
12.0.0-preview.7 279 12/7/2020
12.0.0-preview.6 1,366 11/10/2020
12.0.0-preview.5 1,298 10/1/2020
12.0.0-preview.4 2,999 8/18/2020
12.0.0-preview.2 224 7/28/2020
12.0.0-preview.1 253 7/3/2020