SoftSense.Databricks.Core 0.0.1-prerelease19-786528c

Databricks Client SDK

High-performance .NET SDK for Databricks SQL Warehouse.

Features

• Execute SQL queries against Databricks SQL Warehouses
• Stream large result sets efficiently
• Azure Entra ID and PAT authentication
• Async/await support
• Automatic retry with exponential backoff
• AOT-compatible

Installation

dotnet add package SoftSense.Databricks.Core
dotnet add package SoftSense.Databricks.SqlClient

Quick Start

C#

using Azure.Identity;
using SoftSense.Databricks.Core.Configuration;
using SoftSense.Databricks.SqlClient;

var config = new DatabricksConfig
{
    WorkspaceUrl = "https://adb-123456789.azuredatabricks.net",
    Credential = new DefaultAzureCredential(),
    WarehouseId = "your-warehouse-id"
};

using var client = new SqlWarehouseClient(config);

// Execute query
var result = await client.ExecuteQueryAsync("SELECT * FROM table");

// Stream large results
await foreach (var row in client.ExecuteQueryStreamAsync("SELECT * FROM large_table"))
{
    ProcessRow(row);
}

Authentication

Azure Entra ID (Recommended)

var config = new DatabricksConfig
{
    WorkspaceUrl = "https://adb-123456789.azuredatabricks.net",
    Credential = new DefaultAzureCredential()
};

Personal Access Token

var config = new DatabricksConfig
{
    WorkspaceUrl = "https://your-workspace.databricks.com",
    AccessToken = Environment.GetEnvironmentVariable("DatabricksConfig__AccessToken")
};

Environment Variables

Configuration can be loaded from environment variables using .NET configuration notation:

# Windows
set DatabricksConfig__WorkspaceUrl=https://your-workspace.databricks.com
set DatabricksConfig__WarehouseId=your-warehouse-id
set DatabricksConfig__AccessToken=your-token

# Linux/macOS
export DatabricksConfig__WorkspaceUrl=https://your-workspace.databricks.com
export DatabricksConfig__WarehouseId=your-warehouse-id
export DatabricksConfig__AccessToken=your-token

Then load with configuration builder:

using Microsoft.Extensions.Configuration;
using SoftSense.Databricks.Core.Configuration;

var configuration = new ConfigurationBuilder()
    .AddEnvironmentVariables()
    .Build();

var config = configuration.GetSection("DatabricksConfig").Get<DatabricksConfig>();

Configuration

Note: Either Credential or AccessToken is required.

Error Handling

using SoftSense.Databricks.Core.Exceptions;

try
{
    var result = await client.ExecuteQueryAsync(sql);
}
catch (DatabricksAuthenticationException ex)
{
    Console.WriteLine($"Auth failed: {ex.Message}");
}
catch (DatabricksRateLimitException ex)
{
    Console.WriteLine($"Rate limited. Retry after: {ex.RetryAfter}");
}
catch (DatabricksHttpException ex)
{
    Console.WriteLine($"HTTP {ex.StatusCode}: {ex.Message}");
}

Building from Source

cd source/csharp
dotnet restore
dotnet build
dotnet test

Benchmarks

Performance benchmarks comparing batched vs streamed query execution are available in source/csharp/SoftSense.Databricks.Benchmarks.

cd source/csharp/SoftSense.Databricks.Benchmarks
dotnet run -c Release

See benchmark documentation for details.

License

Apache License 2.0 - see LICENSE file.

Support

• Issues: GitHub Issues
• Discussions: GitHub Discussions