StorageContextLib 1.0.28

StorageContextLib - Unified Cloud Storage API

A powerful .NET library that provides a unified abstraction layer for Azure Blob Storage and Amazon S3, enabling seamless multi-cloud storage operations.

Features

✅ Multi-Cloud Support - Work with Azure Blob Storage and Amazon S3 through a single interface
✅ Batch Operations - High-performance batch read, write, and copy operations
✅ Streaming Support - Memory-efficient streaming with configurable chunk sizes
✅ Async Enumerable - Stream large files with IAsyncEnumerable<ReadOnlyMemory<byte>>
✅ Retry Logic - Built-in exponential backoff for transient failures
✅ Cross-Account Operations - Copy files between different storage accounts
✅ Background Services - Connection pool management with BackgroundService
✅ Move Operations - Atomic move with overwrite support

Installation

Or via NuGet Package Manager:

Configuration

1. Add to appsettings.json

{
  "AzureContext": [
    {
      "azureConnectionString": "connectionstring",
      "azureStorageName": "storageAccount1",
      "azureContainerName": "documents"
    },
    {
      "azureConnectionString": "connectionstring",
      "azureStorageName": "storageAccount1",
      "azureContainerName": "documents"
    }
  ],
  "AmazonContext": [
    {
      "awsAccessKeyId": "your-access-key-id",
      "awsSecretAccessKey": "your-secret-access-key",
      "awsBucketName": "your-bucket-name",
      "awsRegion": "your-region"
    }
  ]
}

Background Service Configuration (Performance Optimization)

add this in your appsettings.json to enable and configure the background service for connection management:

{
  "BackgroundBlobService": {
    "ConnectionsInPool": 5,
    "PoolInterval": 60,
    "ServiceInterval": 120
  }
}

For improved performance and connection management, configure the BackgroundBlobService:

Performance Impact:

• ✅ Reduces cold-start latency on blob operations
• ✅ Maintains warm connections to storage accounts
• ✅ Prevents connection timeouts during idle periods
• ⚡ Can improve operation speed by up to 40% under load

Set ConnectionsInPool: 0 to disable background service if not needed.

2. Register Services in Program.cs

    var AzureCtx = builder.Configuration.GetRequiredSection("AzureContext").Get<List<AzureStorageProps>>();
    var AmazonCtx = builder.Configuration.GetRequiredSection("AmazonContext").Get<List<AmazonStorageProps>>();

    builder.Services.AddSingleton<IEnumerable<IBlobStorageService>>(ctx =>
     {
         var storageContext = new MultiContext
         {

             Azureprops = builder.Configuration.GetRequiredSection("AzureContext").Get<List<AzureStorageProps>>(),
             Amazonprops = builder.Configuration.GetRequiredSection("AmazonContext").Get<List<AmazonStorageProps>>()

         };

         return storageContext.CreateService();
     });
//only incase you want to use the background service for connection management, add this line to register the hosted service:
   builder.Services.AddHostedService<BlobServiceBackgroundTask>();

Complete Usage Examples

Basic Controller Setup

public StorageController( IEnumerable<IBlobStorageService> storageServices, ILogger<StorageController> logger) { _storageServices = storageServices; _logger = logger; }

[HttpPost("upload")] public async Task<IActionResult> UploadFile(IFormFile file) { var storage = _storageServices.instance( storageAccountName: "mystorageaccount", containerName: "documents" ); using var stream = file.OpenReadStream(); await storage.UploadBlobAsync($"uploads/{file.FileName}", stream);

return Ok(new { message = "File uploaded successfully" });

[HttpGet("download/{*filePath}")] public async Task<IActionResult> DownloadFile(string filePath) { var storage = _storageServices.instance("mystorageaccount", "documents"); var stream = await storage.DownloadBlobAsync(filePath); if (stream == null) return NotFound();

return File(stream, "application/octet-stream", Path.GetFileName(filePath));

HttpGet("stream/{*blobPath}")] public async Task<IActionResult> StreamFile( string blobPath, [FromQuery] int chunkSizeInMB = 4) { var storage = _storageServices.instance("mystorageaccount", "documents"); var contentType = Path.GetExtension(blobPath).ToLower() switch { ".mp4" ⇒ "video/mp4", ".pdf" ⇒ "application/pdf", ".jpg" or ".jpeg" ⇒ "image/jpeg", ".png" ⇒ "image/png", _ ⇒ "application/octet-stream" };

Response.ContentType = contentType;

await foreach (var chunk in storage.StreamBlobInChunksAsync( blobPath, chunkSizeInMB, HttpContext.RequestAborted)) { await Response.Body.WriteAsync(chunk.ToArray()); await Response.Body.FlushAsync(); }

return new EmptyResult();

[HttpPost("batch-read")] public async Task<IActionResult> BatchRead([FromBody] List<string> filePaths) { var storage = _storageServices.instance("mystorageaccount", "documents"); var paths = new ConcurrentBag<string>(filePaths); var results = await storage.ReadBatch(paths, exGlobal: true);

var response = results .Where(r ⇒ r.stream != null) .Select(r ⇒ new { path = r.path, size = r.stream.Length, exists = true }) .ToList();

return Ok(response);

5. Batch Write Operations

[HttpPost("batch-write")] public async Task<IActionResult> BatchWrite([FromBody] Dictionary<string, string> files) { var storage = _storageServices.instance("mystorageaccount", "documents"); var writeBatch = new ConcurrentBag<Batch>();

foreach (var file in files) { var contentBytes = Encoding.UTF8.GetBytes(file.Value); writeBatch.Add(new Batch { path = file.Key, stream = new MemoryStream(contentBytes) }); }

await storage.WriteBatch(writeBatch);

return Ok(new { filesWritten = writeBatch.Count }); }

7. Copy Files Across Storage Accounts

[HttpPost("copy-across-accounts")] public async Task<IActionResult> CopyAcrossAccounts() { var storage = _storageServices.instance("sourceaccount", "documents"); var copyBatch = new CopyBatch { blobStorageServices = _storageServices, paths = new ConcurrentDictionary<string, string> { ["folder/source.pdf"] = "folder/destination.pdf", ["images/photo.jpg"] = "archive/photo.jpg" }, fromStorageAccount = "sourceaccount", toStorageAccount = "destaccount" };

await storage.CopyBatch(copyBatch);

return Ok(new { message = "Files copied successfully" }); }

7. Copy Files Across Storage Accounts

[HttpPost("copy-across-accounts")] public async Task<IActionResult> CopyAcrossAccounts() { var storage = _storageServices.instance("sourceaccount", "documents"); var copyBatch = new CopyBatch { blobStorageServices = _storageServices, paths = new ConcurrentDictionary<string, string> { ["folder/source.pdf"] = "folder/destination.pdf", ["images/photo.jpg"] = "archive/photo.jpg" }, fromStorageAccount = "sourceaccount", toStorageAccount = "destaccount" };

await storage.CopyBatch(copyBatch);

return Ok(new { message = "Files copied successfully" });

8. List Directory Contents

[HttpGet("list")] async Task<IActionResult> ListFiles( [FromQuery] string folder = "", [FromQuery] string pattern = "*") { var storage = _storageServices.instance("mystorageaccount", "documents"); var files = await storage.GetDirectoryFilesPath( basePath: folder, searchoption: SearchOption.AllDirectories, pattern: pattern );

return Ok(new { count = files.Count, files });

9. Delete Files

[HttpDelete("delete/{*filePath}")] public async Task<IActionResult> DeleteFile(string filePath) { var storage = _storageServices.instance("mystorageaccount", "documents"); var deleted = await storage.DeleteBlobAsync(filePath);

return deleted ? Ok(new { message = "File deleted" }) : NotFound();

10. Download Large Files in Chunks

[HttpGet("download-chunked/{*filePath}")] public async Task<IActionResult> DownloadChunked( string filePath, [FromQuery] int chunkSizeInMB = 16) { var storage = _storageServices.instance("mystorageaccount", "documents"); var sw = Stopwatch.StartNew(); var stream = await storage.DownloadBlobInChunksAsync(filePath, chunkSizeInMB); sw.Stop();

if (stream == null) return NotFound();

_logger.LogInformation( $"Downloaded {stream.Length} bytes in {sw.ElapsedMilliseconds}ms");

return File(stream, "application/octet-stream", Path.GetFileName(filePath)); }

Advanced Scenarios

Using Multiple Storage Accounts

// Production storage var prodStorage = _storageServices.instance("prodaccount", "documents"); await prodStorage.UploadBlobAsync("reports/report.pdf", stream); // Development storage var devStorage = _storageServices.instance("devaccount", "documents"); await devStorage.UploadBlobAsync("test/report.pdf", stream); // QA storage var qaStorage = _storageServices.instance("qaaccount", "documents"); await qaStorage.UploadBlobAsync("qa/report.pdf", stream);

Append Mode for Audio/Video Files

[HttpPost("append-audio")] public async Task<IActionResult> AppendAudio(IFormFile audioChunk) { var storage = _storageServices.instance("mystorageaccount", "documents"); using var stream = audioChunk.OpenReadStream();

// Append mode automatically handles WAV header merging await storage.UploadBlobAsync( "recordings/session1.wav", stream, appendMode: true );

return Ok(new { message = "Audio chunk appended" }); }

Read/Write Text Files

// Write text var content = JsonSerializer.Serialize(new { data = "example" }); await storage.WriteAllText("config/settings.json", content); // Read text var jsonContent = await storage.ReadAllText("config/settings.json"); var settings = JsonSerializer.Deserialize<MySettings>(jsonContent);

Performance Tips

1. Enable Background Service - Maintains warm connections
2. Use Batch Operations - 3-5x faster than individual operations
3. Stream Large Files - Use StreamBlobInChunksAsync for files > 100MB
4. Adjust Chunk Size - Larger chunks (16-32MB) for faster networks
5. Parallel Operations - Library is fully thread-safe

Amazon S3 Region Codes

Requirements

• .NET 8.0, 9.0, or 10.0
• Azure.Storage.Blobs 12.23.0+
• AWSSDK.S3 3.7.309.7+
• Microsoft.Extensions.Hosting 8.0.0+

Troubleshooting

Issue: Slow first request

Solution: Enable BlobServiceBackgroundTask with ConnectionsInPool: 5

Issue: 412 ConditionNotMet errors

Solution: Library has built-in retry logic with exponential backoff

Issue: Out of memory on large files

Solution: Use StreamBlobInChunksAsync instead of DownloadBlobAsync

Support & Contributing

• NuGet: https://www.nuget.org/packages/StorageContextLib
• Repository:
• Issues: Report bugs or request features via Azure DevOps

License

MIT License - Copyright © PracticeEHR 2024-2025

Roadmap

• ✅ Azure Blob Storage (Complete)
• 🚧 Amazon S3 (In Development)
• 📋 Google Cloud Storage (Planned)
• 📋 Cross-cloud migration tools (Planned)

Made with ❤️ by Habib PracticeEHR