Azure.Storage.Blobs.ChangeFeed 12.0.0-preview.16 The ID prefix of this package has been reserved for one of the owners of this package by NuGet.org.

This client library enables working with the Microsoft Azure Storage Change Feed feature to review and monitor changes to an Azure Storage account.
     For this release see notes - https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/storage/Azure.Storage.Blobs.ChangeFeed/README.md and https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/storage/Azure.Storage.Blobs.ChangeFeed/CHANGELOG.md
     in addition to the breaking changes https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/storage/Azure.Storage.Blobs.ChangeFeed/BreakingChanges.txt

This is a prerelease version of Azure.Storage.Blobs.ChangeFeed.
Install-Package Azure.Storage.Blobs.ChangeFeed -Version 12.0.0-preview.16
dotnet add package Azure.Storage.Blobs.ChangeFeed --version 12.0.0-preview.16
<PackageReference Include="Azure.Storage.Blobs.ChangeFeed" Version="12.0.0-preview.16" />
For projects that support PackageReference, copy this XML node into the project file to reference the package.
paket add Azure.Storage.Blobs.ChangeFeed --version 12.0.0-preview.16
The NuGet Team does not provide support for this client. Please contact its maintainers for support.
#r "nuget: Azure.Storage.Blobs.ChangeFeed, 12.0.0-preview.16"
#r directive can be used in F# Interactive, C# scripting and .NET Interactive. Copy this into the interactive tool or source code of the script to reference the package.
// Install Azure.Storage.Blobs.ChangeFeed as a Cake Addin
#addin nuget:?package=Azure.Storage.Blobs.ChangeFeed&version=12.0.0-preview.16&prerelease

// Install Azure.Storage.Blobs.ChangeFeed as a Cake Tool
#tool nuget:?package=Azure.Storage.Blobs.ChangeFeed&version=12.0.0-preview.16&prerelease
The NuGet Team does not provide support for this client. Please contact its maintainers for support.

Azure Storage Blobs Change Feed client library for .NET

Server Version: 2020-04-8, 2020-02-10, 2019-12-12

The purpose of the change feed is to provide transaction logs of all the changes that occur to the blobs and the blob metadata in your storage account. The change feed provides ordered, guaranteed, durable, immutable, read-only log of these changes. Client applications can read these logs at any time. The change feed enables you to build efficient and scalable solutions that process change events that occur in your Blob Storage account at a low cost.

Source code | Package (NuGet) | Product documentation

Getting started

Install the package

Install the Azure Storage Blobs client library for .NET with NuGet:

dotnet add package Azure.Storage.Blobs.ChangeFeed

Prerequisites

You need an Azure subscription and a Storage Account to use this package.

To create a new Storage Account, you can use the Azure Portal, Azure PowerShell, or the Azure CLI. Here's an example using the Azure CLI:

az storage account create --name MyStorageAccount --resource-group MyResourceGroup --location westus --sku Standard_LRS

Authenticate the Client

Authentication works the same as in Azure.Storage.Blobs.

Key concepts

The change feed is stored as blobs in a special container in your storage account at standard blob pricing cost. You can control the retention period of these files based on your requirements (See the conditions of the current release). Change events are appended to the change feed as records in the Apache Avro format specification: a compact, fast, binary format that provides rich data structures with inline schema. This format is widely used in the Hadoop ecosystem, Stream Analytics, and Azure Data Factory.

You can process these logs incrementally or in-full. Any number of client applications can independently read the change feed, in parallel, and at their own pace. Analytics applications such as Apache Drill or Apache Spark can consume logs directly as Avro files, which let you process them at a low-cost, with high-bandwidth, and without having to write a custom application.

Thread safety

We guarantee that all client instance methods are thread-safe and independent of each other (guideline). This ensures that the recommendation of reusing client instances is always safe, even across threads.

Additional concepts

Client options | Accessing the response | Long-running operations | Handling failures | Diagnostics | Mocking | Client lifetime

Examples

Get all events in the Change Feed

// Get all the events in the change feed.
List<BlobChangeFeedEvent> changeFeedEvents = new List<BlobChangeFeedEvent>();
await foreach (BlobChangeFeedEvent changeFeedEvent in changeFeedClient.GetChangesAsync())
{
    changeFeedEvents.Add(changeFeedEvent);
}

Get events between a start and end time

// Create the start and end time.  The change feed client will round start time down to
// the nearest hour, and round endTime up to the next hour if you provide DateTimeOffsets
// with minutes and seconds.
DateTimeOffset startTime = new DateTimeOffset(2017, 3, 2, 15, 0, 0, TimeSpan.Zero);
DateTimeOffset endTime = new DateTimeOffset(2020, 10, 7, 2, 0, 0, TimeSpan.Zero);

// You can also provide just a start or end time.
await foreach (BlobChangeFeedEvent changeFeedEvent in changeFeedClient.GetChangesAsync(
    start: startTime,
    end: endTime))
{
    changeFeedEvents.Add(changeFeedEvent);
}

Resume with continuationToken

string continuationToken = null;
await foreach (Page<BlobChangeFeedEvent> page in changeFeedClient.GetChangesAsync().AsPages(pageSizeHint: 10))
{
    foreach (BlobChangeFeedEvent changeFeedEvent in page.Values)
    {
        changeFeedEvents.Add(changeFeedEvent);
    }

    // Get the change feed continuation token.  The continuation token is not required to get each page of events,
    // it is intended to be saved and used to resume iterating at a later date.
    continuationToken = page.ContinuationToken;
    break;
}

// Resume iterating from the pervious position with the continuation token.
await foreach (BlobChangeFeedEvent changeFeedEvent in changeFeedClient.GetChangesAsync(
    continuationToken: continuationToken))
{
    changeFeedEvents.Add(changeFeedEvent);
}

Troubleshooting

All Blob service operations will throw a RequestFailedException on failure with helpful ErrorCodes. Many of these errors are recoverable.

Next steps

Get started with our Change Feed samples:

  1. Hello World: Get changes that have occured in your storage account (or asynchronously)
  2. Auth: Authenticate with connection strings, public access, shared keys, shared access signatures, and Azure Active Directory.

Contributing

See the Storage CONTRIBUTING.md for details on building, testing, and contributing to this library.

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit cla.microsoft.com.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

Azure Storage Blobs Change Feed client library for .NET

Server Version: 2020-04-8, 2020-02-10, 2019-12-12

The purpose of the change feed is to provide transaction logs of all the changes that occur to the blobs and the blob metadata in your storage account. The change feed provides ordered, guaranteed, durable, immutable, read-only log of these changes. Client applications can read these logs at any time. The change feed enables you to build efficient and scalable solutions that process change events that occur in your Blob Storage account at a low cost.

Source code | Package (NuGet) | Product documentation

Getting started

Install the package

Install the Azure Storage Blobs client library for .NET with NuGet:

dotnet add package Azure.Storage.Blobs.ChangeFeed

Prerequisites

You need an Azure subscription and a Storage Account to use this package.

To create a new Storage Account, you can use the Azure Portal, Azure PowerShell, or the Azure CLI. Here's an example using the Azure CLI:

az storage account create --name MyStorageAccount --resource-group MyResourceGroup --location westus --sku Standard_LRS

Authenticate the Client

Authentication works the same as in Azure.Storage.Blobs.

Key concepts

The change feed is stored as blobs in a special container in your storage account at standard blob pricing cost. You can control the retention period of these files based on your requirements (See the conditions of the current release). Change events are appended to the change feed as records in the Apache Avro format specification: a compact, fast, binary format that provides rich data structures with inline schema. This format is widely used in the Hadoop ecosystem, Stream Analytics, and Azure Data Factory.

You can process these logs incrementally or in-full. Any number of client applications can independently read the change feed, in parallel, and at their own pace. Analytics applications such as Apache Drill or Apache Spark can consume logs directly as Avro files, which let you process them at a low-cost, with high-bandwidth, and without having to write a custom application.

Thread safety

We guarantee that all client instance methods are thread-safe and independent of each other (guideline). This ensures that the recommendation of reusing client instances is always safe, even across threads.

Additional concepts

Client options | Accessing the response | Long-running operations | Handling failures | Diagnostics | Mocking | Client lifetime

Examples

Get all events in the Change Feed

// Get all the events in the change feed.
List<BlobChangeFeedEvent> changeFeedEvents = new List<BlobChangeFeedEvent>();
await foreach (BlobChangeFeedEvent changeFeedEvent in changeFeedClient.GetChangesAsync())
{
    changeFeedEvents.Add(changeFeedEvent);
}

Get events between a start and end time

// Create the start and end time.  The change feed client will round start time down to
// the nearest hour, and round endTime up to the next hour if you provide DateTimeOffsets
// with minutes and seconds.
DateTimeOffset startTime = new DateTimeOffset(2017, 3, 2, 15, 0, 0, TimeSpan.Zero);
DateTimeOffset endTime = new DateTimeOffset(2020, 10, 7, 2, 0, 0, TimeSpan.Zero);

// You can also provide just a start or end time.
await foreach (BlobChangeFeedEvent changeFeedEvent in changeFeedClient.GetChangesAsync(
    start: startTime,
    end: endTime))
{
    changeFeedEvents.Add(changeFeedEvent);
}

Resume with continuationToken

string continuationToken = null;
await foreach (Page<BlobChangeFeedEvent> page in changeFeedClient.GetChangesAsync().AsPages(pageSizeHint: 10))
{
    foreach (BlobChangeFeedEvent changeFeedEvent in page.Values)
    {
        changeFeedEvents.Add(changeFeedEvent);
    }

    // Get the change feed continuation token.  The continuation token is not required to get each page of events,
    // it is intended to be saved and used to resume iterating at a later date.
    continuationToken = page.ContinuationToken;
    break;
}

// Resume iterating from the pervious position with the continuation token.
await foreach (BlobChangeFeedEvent changeFeedEvent in changeFeedClient.GetChangesAsync(
    continuationToken: continuationToken))
{
    changeFeedEvents.Add(changeFeedEvent);
}

Troubleshooting

All Blob service operations will throw a RequestFailedException on failure with helpful ErrorCodes. Many of these errors are recoverable.

Next steps

Get started with our Change Feed samples:

  1. Hello World: Get changes that have occured in your storage account (or asynchronously)
  2. Auth: Authenticate with connection strings, public access, shared keys, shared access signatures, and Azure Active Directory.

Contributing

See the Storage CONTRIBUTING.md for details on building, testing, and contributing to this library.

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit cla.microsoft.com.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

Release Notes

https://github.com/Azure/azure-sdk-for-net/blob/Azure.Storage.Blobs.ChangeFeed_12.0.0-preview.16/sdk/storage/Azure.Storage.Blobs.ChangeFeed/CHANGELOG.md

NuGet packages

This package is not used by any NuGet packages.

GitHub repositories

This package is not used by any popular GitHub repositories.

Version History

Version Downloads Last updated
12.0.0-preview.16 97 9/9/2021
12.0.0-preview.15 124 7/23/2021
12.0.0-preview.14 35 7/23/2021
12.0.0-preview.13 253 6/9/2021
12.0.0-preview.12 137 5/12/2021
12.0.0-preview.11 673 4/9/2021
12.0.0-preview.10 271 3/10/2021
12.0.0-preview.9 137 2/10/2021
12.0.0-preview.8 17,370 1/12/2021
12.0.0-preview.7 210 12/7/2020
12.0.0-preview.6 1,205 11/10/2020
12.0.0-preview.5 999 10/1/2020
12.0.0-preview.4 1,797 8/18/2020
12.0.0-preview.2 150 7/28/2020
12.0.0-preview.1 184 7/3/2020