E5Embedding.Net 2.0.2

E5Embedding.Net

A high-performance .NET library for generating text embeddings using E5 models with ONNX Runtime. This library provides GPU acceleration support (CUDA/DirectML) with automatic CPU fallback, making it ideal for production environments.

Features

• 🚀 High Performance: Optimized ONNX Runtime inference with GPU acceleration
• 🎯 E5 Model Support: Built specifically for E5 embedding models
• 🔧 Flexible Tokenization: Supports both SentencePiece and BERT-style tokenizers
• 💻 GPU Acceleration: Automatic GPU detection with CUDA and DirectML support
• 🔄 Automatic Fallback: Seamless fallback to CPU if GPU initialization fails
• 📦 Easy Integration: Simple API for embedding single texts or batches
• 🎨 Production Ready: Comprehensive error handling and logging support

Installation

Install the package via NuGet:

dotnet add package E5Embedding.Net

Or via Package Manager:

Install-Package E5Embedding.Net

Quick Start

Basic Usage

using E5Embedding.Net;
using Microsoft.Extensions.Logging;

string testText = "This is a test text for embedding generation using E5 model.";

var config = new E5EmbeddingConfiguration
{
    OnnxModelPath = "C:/WorkSpace/test/model.onnx",
    SentencePieceModelFile = "C:/WorkSpace/test/sentencepiece.bpe.model",
    TokenizerConfigFile = "C:/WorkSpace/test/tokenizer_config.json",
    TokenizerJsonFile = "C:/WorkSpace/test/tokenizer.json",
    MaxSequenceLength = 512,
    Dimension = 1024,
    BatchSize = 16
};

// Logger is optional and can be null
// If you don't need logging, you can pass null: new OnnxEmbeddingService(config, null)
var loggerFactory = LoggerFactory.Create(builder => builder.AddConsole());
var logger = loggerFactory.CreateLogger<OnnxEmbeddingService>();

using var embeddingService = new OnnxEmbeddingService(config, logger);

var embedding = await embeddingService.EmbedAsync(testText);

Console.WriteLine($"Embedding dimension: {embedding.Length}");
Console.WriteLine($"[{string.Join(", ", embedding.Select(v => v.ToString("F6")))}]");

// Generate embeddings for multiple texts
var texts = new[] { "Text 1", "Text 2", "Text 3" };
var embeddings = await embeddingService.EmbedBatchAsync(texts);

Dependency Injection

using E5Embedding.Net;
using Microsoft.Extensions.DependencyInjection;

// In your Startup.cs or Program.cs
services.AddSingleton<E5EmbeddingConfiguration>(sp =>
{
    var configuration = sp.GetRequiredService<IConfiguration>();
    return new E5EmbeddingConfiguration
    {
        OnnxModelPath = configuration["E5:OnnxModelPath"],
        SentencePieceModelFile = configuration["E5:SentencePieceModelFile"],
        TokenizerConfigFile = configuration["E5:TokenizerConfigFile"],
        TokenizerJsonFile = configuration["E5:TokenizerJsonFile"],
        MaxSequenceLength = configuration.GetValue<int>("E5:MaxSequenceLength"),
        Dimension = configuration.GetValue<int>("E5:Dimension"),
        BatchSize = configuration.GetValue<int>("E5:BatchSize", 16)
    };
});

services.AddSingleton<IEmbeddingService>(sp =>
{
    var config = sp.GetRequiredService<E5EmbeddingConfiguration>();
    var logger = sp.GetService<ILogger<OnnxEmbeddingService>>();
    return new OnnxEmbeddingService(config, logger);
});

Configuration

E5EmbeddingConfiguration Properties

GPU Acceleration

The library automatically detects and uses GPU acceleration when available:

• CUDA: Automatically used on systems with NVIDIA GPUs and CUDA support
• DirectML: Used on Windows systems with compatible GPUs
• CPU Fallback: Automatically falls back to CPU if GPU initialization fails

GPU usage is logged when the service is initialized. Check your logs to see which provider is being used.

Tokenizers

SentencePieceTokenizer

Used by default for E5 models. Supports BPE (Byte Pair Encoding) tokenization.

using E5Embedding.Net.Tokenization;

var tokenizer = new SentencePieceTokenizer(
    sentencePieceModelFile: "path/to/sentencepiece.bpe.model",
    tokenizerConfigFile: "path/to/tokenizer_config.json",
    tokenizerJsonFile: "path/to/tokenizer.json",
    maxSequenceLength: 512
);

var encoding = tokenizer.Encode("Your text here");

BertTokenizer

BERT-style WordPiece tokenizer for models that require it.

using E5Embedding.Net.Tokenization;

var tokenizer = new BertTokenizer(
    tokenizerConfigFile: "path/to/tokenizer_config.json",
    tokenizerJsonFile: "path/to/tokenizer.json",
    maxSequenceLength: 512
);

var encoding = tokenizer.Encode("Your text here");
var pairEncoding = tokenizer.EncodePair("Query text", "Passage text");

Requirements

• .NET 8.0 or later
• ONNX Runtime (included via NuGet package)
• E5 model files (ONNX format)
• Tokenizer files (SentencePiece model, config, and JSON)

Model Files

You need the following files from your E5 model:

1. model.onnx - The ONNX model file
2. sentencepiece.bpe.model - SentencePiece tokenizer model (for SentencePieceTokenizer)
3. tokenizer_config.json - Tokenizer configuration
4. tokenizer.json - Tokenizer vocabulary and metadata

Download Model Files

You can download the ONNX model files from Hugging Face:

• Download URL: https://huggingface.co/intfloat/multilingual-e5-large/tree/main/onnx

The repository contains various ONNX model formats:

• model.onnx - Standard ONNX model
• model.onnx_data - Model weights data
• model_O4.onnx - Optimized ONNX model (O4)
• model_qint8_avx512_vnni.onnx - Quantized model for AVX512 VNNI
• Tokenizer files (sentencepiece.bpe.model, tokenizer_config.json, tokenizer.json)

Performance Tips

1. Batch Processing: Use EmbedBatchAsync for multiple texts to improve throughput
2. Batch Size: Adjust BatchSize based on your memory and performance requirements
3. GPU: Ensure GPU drivers are installed for best performance
4. Dispose: Always dispose the service when done to free resources

Error Handling

The library provides comprehensive error handling:

• ArgumentNullException: When required parameters are null
• FileNotFoundException: When model or tokenizer files are missing
• InvalidOperationException: When configuration is invalid or dimensions don't match
• AggregateException: When both GPU and CPU initialization fail

License

MIT License - see LICENSE file for details

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

Support

For issues, questions, or contributions, please visit the GitHub repository.