Azure.Search.Documents 11.5.0-beta.2

Azure Cognitive Search client library for .NET

Azure Cognitive Search is a search-as-a-service cloud solution that gives developers APIs and tools for adding a rich search experience over private, heterogeneous content in web, mobile, and enterprise applications.

The Azure Cognitive Search service is well suited for the following application scenarios:

• Consolidate varied content types into a single searchable index. To populate an index, you can push JSON documents that contain your content, or if your data is already in Azure, create an indexer to pull in data automatically.
• Attach skillsets to an indexer to create searchable content from images and large text documents. A skillset leverages AI from Cognitive Services for built-in OCR, entity recognition, key phrase extraction, language detection, text translation, and sentiment analysis. You can also add custom skills to integrate external processing of your content during data ingestion.
• In a search client application, implement query logic and user experiences similar to commercial web search engines.

Use the Azure.Search.Documents client library to:

• Submit queries for simple and advanced query forms that include fuzzy search, wildcard search, regular expressions.
• Implement filtered queries for faceted navigation, geospatial search, or to narrow results based on filter criteria.
• Create and manage search indexes.
• Upload and update documents in the search index.
• Create and manage indexers that pull data from Azure into an index.
• Create and manage skillsets that add AI enrichment to data ingestion.
• Create and manage analyzers for advanced text analysis or multi-lingual content.
• Optimize results through scoring profiles to factor in business logic or freshness.

Source code | Package (NuGet) | API reference documentation | REST API documentation | Product documentation | Samples

Getting started

Install the package

Install the Azure Cognitive Search client library for .NET with NuGet:

dotnet add package Azure.Search.Documents

Prerequisites

You need an Azure subscription and a search service to use this package.

To create a new search service, you can use the Azure portal, Azure PowerShell, or the Azure CLI. Here's an example using the Azure CLI to create a free instance for getting started:

az search service create --name <mysearch> --resource-group <mysearch-rg> --sku free --location westus

See choosing a pricing tier for more information about available options.

Authenticate the client

All requests to a search service need an api-key that was generated specifically for your service. The api-key is the sole mechanism for authenticating access to your search service endpoint. You can obtain your api-key from the Azure portal or via the Azure CLI:

az search admin-key show --service-name <mysearch> --resource-group <mysearch-rg>

There are two types of keys used to access your search service: admin (read-write) and query (read-only) keys. Restricting access and operations in client apps is essential to safeguarding the search assets on your service. Always use a query key rather than an admin key for any query originating from a client app.

Note: The example Azure CLI snippet above retrieves an admin key so it's easier to get started exploring APIs, but it should be managed carefully.

We can use the api-key to create a new SearchClient.

string indexName = "nycjobs";

// Get the service endpoint and API key from the environment
Uri endpoint = new Uri(Environment.GetEnvironmentVariable("SEARCH_ENDPOINT"));
string key = Environment.GetEnvironmentVariable("SEARCH_API_KEY");

// Create a client
AzureKeyCredential credential = new AzureKeyCredential(key);
SearchClient client = new SearchClient(endpoint, indexName, credential);

ASP.NET Core

To inject SearchClient as a dependency in an ASP.NET Core app, first install the package Microsoft.Extensions.Azure. Then register the client in the Startup.ConfigureServices method:

public void ConfigureServices(IServiceCollection services)
{
    services.AddAzureClients(builder =>
    {
        builder.AddSearchClient(Configuration.GetSection("SearchClient"));
    });
  
    services.AddControllers();
}

To use the preceding code, add this to your configuration:

{
    "SearchClient": {
      "endpoint": "https://<resource-name>.search.windows.net",
      "indexname": "nycjobs"
    }
}

You'll also need to provide your resource key to authenticate the client, but you shouldn't be putting that information in the configuration. Instead, when in development, use User-Secrets. Add the following to secrets.json:

{
    "SearchClient": {
      "credential": { "key": "<you resource key>" }
    }
}

When running in production, it's preferable to use environment variables:

SEARCH__CREDENTIAL__KEY="..."

Or use other secure ways of storing secrets like Azure Key Vault.

For more details about Dependency Injection in ASP.NET Core apps, see Dependency injection with the Azure SDK for .NET.

Key concepts

An Azure Cognitive Search service contains one or more indexes that provide persistent storage of searchable data in the form of JSON documents. (If you're brand new to search, you can make a very rough analogy between indexes and database tables.) The Azure.Search.Documents client library exposes operations on these resources through three main client types.

• SearchClient helps with:

  • Searching your indexed documents using rich queries and powerful data shaping
  • Autocompleting partially typed search terms based on documents in the index
  • Suggesting the most likely matching text in documents as a user types
  • Adding, Updating or Deleting Documents documents from an index

• SearchIndexClient allows you to:

  • Create, delete, update, or configure a search index
  • Declare custom synonym maps to expand or rewrite queries

• SearchIndexerClient allows you to:

  • Create indexers to automatically crawl data sources
  • Define AI powered Skillsets to transform and enrich your data

The Azure.Search.Documents client library (v11) is a brand new offering for .NET developers who want to use search technology in their applications. There is an older, fully featured Microsoft.Azure.Search client library (v10) with many similar looking APIs, so please be careful to avoid confusion when exploring online resources. A good rule of thumb is to check for the namespace using Azure.Search.Documents; when you're looking for us.

Thread safety

We guarantee that all client instance methods are thread-safe and independent of each other (guideline). This ensures that the recommendation of reusing client instances is always safe, even across threads.

Additional concepts

Client options | Accessing the response | Long-running operations | Handling failures | Diagnostics | Mocking | Client lifetime

Examples

The following examples all use a simple Hotel data set that you can import into your own index from the Azure portal. These are just a few of the basics - please check out our Samples for much more.

• Querying
  • Use C# types for search results
  • Use SearchDocument like a dictionary for search results
  • SearchOptions
• Creating an index
• Adding documents to your index
• Retrieving a specific document from your index
• Async APIs

Advanced authentication

• Create a client that can authenticate in a national cloud

Querying

Let's start by importing our namespaces.

using Azure;
using Azure.Search.Documents;
using Azure.Search.Documents.Indexes;
using Azure.Core.GeoJson;

We'll then create a SearchClient to access our hotels search index.

// Get the service endpoint and API key from the environment
Uri endpoint = new Uri(Environment.GetEnvironmentVariable("SEARCH_ENDPOINT"));
string key = Environment.GetEnvironmentVariable("SEARCH_API_KEY");
string indexName = "hotels";

// Create a client
AzureKeyCredential credential = new AzureKeyCredential(key);
SearchClient client = new SearchClient(endpoint, indexName, credential);

There are two ways to interact with the data returned from a search query. Let's explore them with a search for a "luxury" hotel.

Use C# types for search results

We can decorate our own C# types with attributes from System.Text.Json:

public class Hotel
{
    [JsonPropertyName("HotelId")]
    [SimpleField(IsKey = true, IsFilterable = true, IsSortable = true)]
    public string Id { get; set; }

    [JsonPropertyName("HotelName")]
    [SearchableField(IsFilterable = true, IsSortable = true)]
    public string Name { get; set; }

    [SimpleField(IsFilterable = true, IsSortable = true)]
    public GeoPoint GeoLocation { get; set; }

    // Complex fields are included automatically in an index if not ignored.
    public HotelAddress Address { get; set; }
}

public class HotelAddress
{
    public string StreetAddress { get; set; }

    [SimpleField(IsFilterable = true, IsSortable = true, IsFacetable = true)]
    public string City { get; set; }

    [SimpleField(IsFilterable = true, IsSortable = true, IsFacetable = true)]
    public string StateProvince { get; set; }

    [SimpleField(IsFilterable = true, IsSortable = true, IsFacetable = true)]
    public string Country { get; set; }

    [SimpleField(IsFilterable = true, IsSortable = true, IsFacetable = true)]
    public string PostalCode { get; set; }
}

Then we use them as the type parameter when querying to return strongly-typed search results:

SearchResults<Hotel> response = client.Search<Hotel>("luxury");
foreach (SearchResult<Hotel> result in response.GetResults())
{
    Hotel doc = result.Document;
    Console.WriteLine($"{doc.Id}: {doc.Name}");
}

If you're working with a search index and know the schema, creating C# types is recommended.

Use SearchDocument like a dictionary for search results

If you don't have your own type for search results, SearchDocument can be used instead. Here we perform the search, enumerate over the results, and extract data using SearchDocument's dictionary indexer.

SearchResults<SearchDocument> response = client.Search<SearchDocument>("luxury");
foreach (SearchResult<SearchDocument> result in response.GetResults())
{
    SearchDocument doc = result.Document;
    string id = (string)doc["HotelId"];
    string name = (string)doc["HotelName"];
    Console.WriteLine("{id}: {name}");
}

SearchOptions

The SearchOptions provide powerful control over the behavior of our queries. Let's search for the top 5 luxury hotels with a good rating.

int stars = 4;
SearchOptions options = new SearchOptions
{
    // Filter to only Rating greater than or equal our preference
    Filter = SearchFilter.Create($"Rating ge {stars}"),
    Size = 5, // Take only 5 results
    OrderBy = { "Rating desc" } // Sort by Rating from high to low
};
SearchResults<Hotel> response = client.Search<Hotel>("luxury", options);
// ...

Creating an index

You can use the SearchIndexClient to create a search index. Fields can be defined from a model class using FieldBuilder. Indexes can also define suggesters, lexical analyzers, and more.

Using the Hotel sample above, which defines both simple and complex fields:

Uri endpoint = new Uri(Environment.GetEnvironmentVariable("SEARCH_ENDPOINT"));
string key = Environment.GetEnvironmentVariable("SEARCH_API_KEY");

// Create a service client
AzureKeyCredential credential = new AzureKeyCredential(key);
SearchIndexClient client = new SearchIndexClient(endpoint, credential);

// Create the index using FieldBuilder.
SearchIndex index = new SearchIndex("hotels")
{
    Fields = new FieldBuilder().Build(typeof(Hotel)),
    Suggesters =
    {
        // Suggest query terms from the hotelName field.
        new SearchSuggester("sg", "hotelName")
    }
};

client.CreateIndex(index);

In scenarios when the model is not known or cannot be modified, you can also create fields explicitly using convenient SimpleField, SearchableField, or ComplexField classes:

// Create the index using field definitions.
SearchIndex index = new SearchIndex("hotels")
{
    Fields =
    {
        new SimpleField("hotelId", SearchFieldDataType.String) { IsKey = true, IsFilterable = true, IsSortable = true },
        new SearchableField("hotelName") { IsFilterable = true, IsSortable = true },
        new SearchableField("description") { AnalyzerName = LexicalAnalyzerName.EnLucene },
        new SearchableField("tags", collection: true) { IsFilterable = true, IsFacetable = true },
        new ComplexField("address")
        {
            Fields =
            {
                new SearchableField("streetAddress"),
                new SearchableField("city") { IsFilterable = true, IsSortable = true, IsFacetable = true },
                new SearchableField("stateProvince") { IsFilterable = true, IsSortable = true, IsFacetable = true },
                new SearchableField("country") { IsFilterable = true, IsSortable = true, IsFacetable = true },
                new SearchableField("postalCode") { IsFilterable = true, IsSortable = true, IsFacetable = true }
            }
        }
    },
    Suggesters =
    {
        // Suggest query terms from the hotelName field.
        new SearchSuggester("sg", "hotelName")
    }
};

client.CreateIndex(index);

Adding documents to your index

You can Upload, Merge, MergeOrUpload, and Delete multiple documents from an index in a single batched request. There are a few special rules for merging to be aware of.

IndexDocumentsBatch<Hotel> batch = IndexDocumentsBatch.Create(
    IndexDocumentsAction.Upload(new Hotel { Id = "783", Name = "Upload Inn" }),
    IndexDocumentsAction.Merge(new Hotel { Id = "12", Name = "Renovated Ranch" }));

IndexDocumentsOptions options = new IndexDocumentsOptions { ThrowOnAnyError = true };
client.IndexDocuments(batch, options);

The request will succeed even if any of the individual actions fail and return an IndexDocumentsResult for inspection. There's also a ThrowOnAnyError option if you only care about success or failure of the whole batch.

Retrieving a specific document from your index

In addition to querying for documents using keywords and optional filters, you can retrieve a specific document from your index if you already know the key. You could get the key from a query, for example, and want to show more information about it or navigate your customer to that document.

Hotel doc = client.GetDocument<Hotel>("1");
Console.WriteLine($"{doc.Id}: {doc.Name}");

Async APIs

All of the examples so far have been using synchronous APIs, but we provide full support for async APIs as well. You'll generally just add an Async suffix to the name of the method and await it.

SearchResults<Hotel> searchResponse = await client.SearchAsync<Hotel>("luxury");
await foreach (SearchResult<Hotel> result in searchResponse.GetResultsAsync())
{
    Hotel doc = result.Document;
    Console.WriteLine($"{doc.Id}: {doc.Name}");
}

Authenticate in a National Cloud

To authenticate in a National Cloud, you will need to make the following additions to your client configuration:

• Set the AuthorityHost in the credential options or via the AZURE_AUTHORITY_HOST environment variable
• Set the Audience in SearchClientOptions

// Create a SearchClient that will authenticate through AAD in the China national cloud
string indexName = "nycjobs";
Uri endpoint = new Uri(Environment.GetEnvironmentVariable("SEARCH_ENDPOINT"));
SearchClient client = new SearchClient(endpoint, indexName,
    new DefaultAzureCredential(
        new DefaultAzureCredentialOptions()
        {
            AuthorityHost = AzureAuthorityHosts.AzureChina
        }),
    new SearchClientOptions()
    {
        Audience = SearchAudience.AzureChina
    });

Troubleshooting

Any Azure.Search.Documents operation that fails will throw a RequestFailedException with helpful Status codes. Many of these errors are recoverable.

try
{
    return client.GetDocument<Hotel>("12345");
}
catch (RequestFailedException ex) when (ex.Status == 404)
{
    Console.WriteLine("We couldn't find the hotel you are looking for!");
    Console.WriteLine("Please try selecting another.");
    return null;
}

You can also easily enable console logging if you want to dig deeper into the requests you're making against the service.

See our troubleshooting guide for details on how to diagnose various failure scenarios.

Next steps

• Go further with Azure.Search.Documents and our samples
• Watch a demo or deep dive video
• Read more about the Azure Cognitive Search service

Contributing

See our Search CONTRIBUTING.md for details on building, testing, and contributing to this library.

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit cla.microsoft.com.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.