AzureFunctions.Extensions.CognitiveServices 1.0.0-preview2

Azure Function Extensions for Cognitive Services (Preview)

Azure Function Extensions for Cognitive Services is a set of custom Azure Function bindings for Azure's Cognitive Services.

Features:

• Supports a subset of the Computer Vision API including
  • Image Analysis
  • Optical Character Recognition (OCR)
  • Handwriting, Celebrities & Landmarks,
  • Thumbnail generation.
• Cognitive Service API calls with a couple lines of code
• Support for automatic retry policies for throttled requests
• Support for automatic image resize for files larger then 4mb (Cognitive Services Limit)
• Support for Key Vault storage of Cognitive Services Keys
• Exception, Warning, and request time metrics logging

Additional support for the remainder of the Vision offerings and additional support for Language and Knowledge will be coming very soon.

More details on the various Cognitive Services offerings can be found at https://azure.microsoft.com/en-us/services/cognitive-services/

Current Version

Latest Version: 1.0.0-preview2

Current Azure Function Dependency: Microsoft.Azure.Webjobs.Extensions (3.0.0-beta5)

Warning: There are a lot of previews at play with these bindings. The Bindings themselves are in preview. Azure Functions 2.0 is in preview. Some Cognitive Services are in preview. ImageSharp, the library used for native .net core image resizing is also in preview.

These bindings currently only support Azure Functions 2.0 . There are no current plans for 1.x support.

QuickStart

Prerequisites

You must have a valid Azure Cognitive Services Vision License Key and Url. You can obtain one with a Azure Subscription. If you do not have a subscription you can sign up for a free trial at https://azure.microsoft.com/en-us/try/cognitive-services/

Packages

Nuget Package Available at AzureFunctions.Extensions.CognitiveServices (https://www.nuget.org/packages/AzureFunctions.Extensions.CognitiveServices)

Language Support

This library has only been tested with C#. Other language support will be validated\added soon.

Configuration

Appsettings

The following default appsetting values are used if no argument is provided:

• VisionKey : The API Key for your Vision subscription
• VisionUrl : The API Url for your Vision subscription

Attribute Properties such as VisionUrl and VisionKey support the Azure Functions AutoBinding feature by using the %appsettingskey% value.

KeyVault

You can optionally specify the SecureVisionKey and provide the path to your key vault secret (directly or via appsettings).

For example: https://xxxxx.vault.azure.net/secrets/VisionApiKey/XXXXXXXXXXXXX

Note: The current implementation assumes you are using a Managed Service Identity and does not support providing direct client id and secrets for Azure AD Applications.

General Usage Guidelines

These extensions support numerouse combinations of bindings for each service available in cognitive services. The available binding and their associated behaviors depend on the Image Source provided within the binding attribute.

The following bindings are supported.

Model Class Binding

Model Binding is the fastest and simples way to automatically analyze a given image. It requires and image source to be available at the time of binding so a given cognitive services query can be immediatly executed. Because of this it only supports BlobStorage and Url sources. Blob Storage has the additional requirement of a connection and path.

...
[VisionAnalysis(BlobStorageConnection = "storageaccount",
                BlobStoragePath = "analysismodel/{name}",
                ImageSource = ImageSource.BlobStorage)]VisionAnalysisModel result,
string name
...

It is often advantageous to us these extensions in combination with a blob trigger where the name parameter is provided by the Blob Trigger and the path and connection is already known.

 [BlobTrigger("analysismodel/{name}")]Stream storageBlob,
 [VisionAnalysis(BlobStorageConnection = "%storageaccount%",
                           BlobStoragePath = "analysismodel/{name}",
                           ImageSource = ImageSource.BlobStorage)]VisionAnalysisModel result,
 string name,

Each Vision Binding has one or more corresponding models available:

Client Binding

If your image source meets one of the following criteria then working with a Vision Client may be more usefull:

• Your image is not available in Blob Storage or in a publically accessible URL
• Your image is not available at binding time
• Your image is in an external source that requires a seperate request
• You want to modify your image (beyond resize) before processing
• You do not want to use the default processing options for a given request.

Due to the unique requirements of each cognitive service each vision binding has a dedicated vision client and vision request object.

The following client and requests classes are available for each binding

Binding example with VisionAnalysis Attribute and VisionAnalysisClient binding:

...
[VisionAnalysis()]VisionAnalysisClient visionclient,
... {

    byte[] fileBytes = FetchMyFile();

    //Instantiate a new request
    var request = new VisionAnalysisRequest(fileBytes);
    request.AutoResize = true;

    //Set unique analysis options for the request
    request.Options = VisionAnalysisOptions.Categories | VisionAnalysisOptions.Description | VisionAnalysisOptions.Tags;

    //Make the request
    var result = await visionclient.AnalyzeAsync(request);

}