Brevit.NET 0.1.0

Brevit.NET

A high-performance, type-safe .NET library for semantically compressing and optimizing data before sending it to a Large Language Model (LLM). Dramatically reduce token costs while maintaining data integrity and readability.

Table of Contents

• Why Brevit.NET?
• Key Features
• When Not to Use Brevit.NET
• Benchmarks
• Installation & Quick Start
• Playgrounds
• CLI
• Format Overview
• API
• Using Brevit.NET in LLM Prompts
• Syntax Cheatsheet
• Other Implementations
• Full Specification

Why Brevit.NET?

.NET-Specific Advantages

• First-Class POCO Support: Optimize C# objects directly without manual serialization
• Dependency Injection Ready: Seamless integration with ASP.NET Core DI container
• Type Safety: Full compile-time type checking with modern C# features
• Async/Await: Built-in async support for high-performance applications
• LINQ Compatible: Works seamlessly with LINQ queries and expressions

Performance Benefits

• 40-60% Token Reduction: Dramatically reduce LLM API costs
• Zero-Copy Operations: Efficient memory usage with Span<T> and Memory<T>
• High Throughput: Process thousands of objects per second
• Low Latency: Sub-millisecond optimization for typical objects

Example Cost Savings

// Before: 234 tokens = $0.000468 per request
var json = JsonSerializer.Serialize(complexOrder);

// After: 127 tokens = $0.000254 per request (46% reduction)
var optimized = await brevit.BrevityAsync(complexOrder); // Automatic optimization

// Or with explicit configuration
var explicit = await brevit.OptimizeAsync(complexOrder);

// Savings: $0.000214 per request
// At 1M requests/month: $214/month savings

Automatic Strategy Selection

Brevit.NET now includes the BrevityAsync() method that automatically analyzes your data and selects the optimal optimization strategy:

var data = new
{
    Friends = new[] { "ana", "luis", "sam" },
    Hikes = new[]
    {
        new { Id = 1, Name = "Blue Lake Trail", DistanceKm = 7.5 },
        new { Id = 2, Name = "Ridge Overlook", DistanceKm = 9.2 }
    }
};

// Automatically detects uniform arrays and applies tabular format
var optimized = await brevit.BrevityAsync(data);
// No configuration needed - Brevit analyzes and optimizes automatically!

Key Features

• JSON Optimization: Flatten nested JSON structures into token-efficient key-value pairs
• Text Optimization: Clean and summarize long text documents
• Image Optimization: Extract text from images via OCR
• Type-Safe: Built with modern C# and .NET 8
• Extensible: Plugin architecture for custom optimizers
• Lightweight: Minimal dependencies, high performance
• First-Class POCO Support: Optimize C# objects directly without manual serialization

POCO Example

public class Order
{
    public string OrderId { get; set; }
    public string Status { get; set; }
    public List<OrderItem> Items { get; set; }
}

var order = new Order 
{ 
    OrderId = "o-456", 
    Status = "SHIPPED",
    Items = new List<OrderItem> { ... }
};

// Direct optimization - no serialization needed!
var optimized = await brevit.OptimizeAsync(order);

When Not to Use Brevit.NET

Consider alternatives when:

1. API Responses: If returning JSON to HTTP clients, use standard JSON serialization
2. Data Contracts: When strict JSON schema validation is required
3. Small Objects: Objects under 100 tokens may not benefit significantly
4. Real-Time APIs: For REST APIs serving JSON, standard formatting is better
5. Legacy Systems: Systems expecting specific JSON formats

Best Use Cases:

• ✅ LLM prompt optimization
• ✅ Reducing OpenAI/Anthropic API costs
• ✅ Processing large datasets for AI
• ✅ Document summarization workflows
• ✅ OCR and image processing pipelines

Benchmarks

Token Reduction

Object Type	Original Tokens	Brevit Tokens	Reduction
Simple POCO	45	28	38%
Complex POCO	234	127	46%
Nested Arrays	156	89	43%
API Response	312	178	43%

Performance

Operation	Objects/sec	Avg Latency	Memory
Flatten (1KB)	2,000	0.5ms	2.1MB
Flatten (10KB)	450	2.2ms	8.5MB
Flatten (100KB)	55	18ms	45MB

Benchmarks: .NET 8, Intel i7-12700K, Release mode

Installation & Quick Start

Prerequisites

• .NET 8 SDK or later
• Visual Studio 2022, VS Code, or Rider

Install via NuGet (Recommended)

dotnet add package Brevit.NET

Install from Source

git clone https://github.com/JavianDev/Brevit.git
cd Brevit/Brevit.NET
dotnet build
dotnet add reference ../Brevit.NET/Brevit.NET.csproj

Quick Start

using Brevit.NET;

// 1. Create configuration
var config = new BrevitConfig(
    JsonMode: JsonOptimizationMode.Flatten,
    TextMode: TextOptimizationMode.Clean,
    ImageMode: ImageOptimizationMode.Ocr
)
{
    LongTextThreshold = 1000
};

// 2. Create optimizers
var jsonOptimizer = new DefaultJsonOptimizer();
var textOptimizer = new DefaultTextOptimizer();
var imageOptimizer = new DefaultImageOptimizer();

// 3. Create client
var brevit = new BrevitClient(config, jsonOptimizer, textOptimizer, imageOptimizer);

// 4. Optimize POCO directly
var order = new { OrderId = "o-456", Status = "SHIPPED" };
string optimized = await brevit.OptimizeAsync(order);

Dependency Injection (Recommended)

using Brevit.NET;

var builder = WebApplication.CreateBuilder(args);

// Configure Brevit
var brevitConfig = new BrevitConfig(
    JsonMode: JsonOptimizationMode.Flatten,
    TextMode: TextOptimizationMode.SummarizeFast,
    ImageMode: ImageOptimizationMode.Ocr
)
{
    LongTextThreshold = 1000
};

builder.Services.AddSingleton(brevitConfig);
builder.Services.AddSingleton<IJsonOptimizer, DefaultJsonOptimizer>();
builder.Services.AddSingleton<ITextOptimizer, DefaultTextOptimizer>();
builder.Services.AddSingleton<IImageOptimizer, DefaultImageOptimizer>();
builder.Services.AddScoped<BrevitClient>();

var app = builder.Build();

Playgrounds

Interactive Playground

# Clone and run
git clone https://github.com/JavianDev/Brevit.git
cd Brevit/Brevit.NET
dotnet run --project Playground

Online Playground

• Web Playground: https://brevit.dev/playground (Coming Soon)
• .NET Fiddle: https://dotnetfiddle.net/brevit (Coming Soon)

CLI

Installation

dotnet tool install -g Brevit.CLI

Usage

# Optimize a JSON file
brevit optimize input.json -o output.txt

# Optimize POCO from assembly
brevit optimize --assembly MyApp.dll --type MyApp.Order

# Optimize with custom config
brevit optimize input.json --mode flatten --threshold 1000

# Help
brevit --help

Examples

# Flatten JSON
brevit optimize order.json --mode flatten

# Convert to YAML
brevit optimize data.json --mode yaml

# Filter paths
brevit optimize data.json --mode filter --paths "user.name,order.id"

Format Overview

Flattened Format (Hybrid Optimization)

Brevit intelligently converts C# objects to flat key-value pairs with automatic tabular optimization:

Input (C# POCO):

var order = new Order
{
    OrderId = "o-456",
    Friends = new[] { "ana", "luis", "sam" },
    Items = new[]
    {
        new OrderItem { Sku = "A-88", Quantity = 1 },
        new OrderItem { Sku = "T-22", Quantity = 2 }
    }
};

Output (with tabular optimization):

OrderId: o-456
Friends[3]: ana,luis,sam
Items[2]{Quantity,Sku}:
  1,A-88
  2,T-22

For non-uniform arrays (fallback):

var mixed = new
{
    Items = new object[]
    {
        new { Sku = "A-88", Quantity = 1 },
        "special-item",
        new { Sku = "T-22", Quantity = 2 }
    }
};

Output (fallback to indexed format):

Items[0].Sku: A-88
Items[0].Quantity: 1
Items[1]: special-item
Items[2].Sku: T-22
Items[2].Quantity: 2

Key Features

• Property Names: Uses C# property names as-is
• Nested Objects: Dot notation for nested properties
• Tabular Arrays: Uniform object arrays automatically formatted in compact tabular format (Items[2]{Field1,Field2}:)
• Primitive Arrays: Comma-separated format (Friends[3]: ana,luis,sam)
• Hybrid Approach: Automatically detects optimal format, falls back to indexed format for mixed data
• Null Handling: Explicit null values
• Type Preservation: Numbers, booleans preserved as strings

API

BrevitClient

Main client class for optimization.

public class BrevitClient
{
    public BrevitClient(
        BrevitConfig config,
        IJsonOptimizer jsonOptimizer,
        ITextOptimizer textOptimizer,
        IImageOptimizer imageOptimizer
    );

    // Automatic optimization - analyzes data and selects best strategy
    public Task<string> BrevityAsync(object rawData, string? intent = null);
    
    // Explicit optimization with configured settings
    public Task<string> OptimizeAsync(object rawData, string? intent = null);
    
    // Register custom optimization strategy
    public void RegisterStrategy(string name, IOptimizationStrategy strategy);
}

Example - Automatic Optimization:

// Automatically analyzes data structure and selects best strategy
var optimized = await brevit.BrevityAsync(order);
// Automatically detects uniform arrays, long text, etc.

Example - Explicit Optimization:

// Use explicit configuration
var optimized = await brevit.OptimizeAsync(order, "extract_total");

Example - Custom Strategy:

// Register custom optimization strategy
brevit.RegisterStrategy("custom", new CustomOptimizationStrategy());

BrevitConfig

Configuration record for BrevitClient.

public record BrevitConfig(
    JsonOptimizationMode JsonMode = JsonOptimizationMode.Flatten,
    TextOptimizationMode TextMode = TextOptimizationMode.Clean,
    ImageOptimizationMode ImageMode = ImageOptimizationMode.Ocr
)
{
    public List<string> JsonPathsToKeep { get; init; } = new();
    public int LongTextThreshold { get; init; } = 500;
}

Interfaces

IJsonOptimizer

public interface IJsonOptimizer
{
    Task<string> OptimizeJsonAsync(string jsonString, BrevitConfig config);
}

ITextOptimizer

public interface ITextOptimizer
{
    Task<string> OptimizeTextAsync(string longText, BrevitConfig config);
}

IImageOptimizer

public interface IImageOptimizer
{
    Task<string> OptimizeImageAsync(byte[] imageData, BrevitConfig config);
}

Enums

JsonOptimizationMode

• None - No optimization
• Flatten - Flatten to key-value pairs (default)
• ToYaml - Convert to YAML
• Filter - Keep only specified paths

TextOptimizationMode

• None - No optimization
• Clean - Remove boilerplate
• SummarizeFast - Fast summarization
• SummarizeHighQuality - High-quality summarization

ImageOptimizationMode

• None - Skip processing
• Ocr - Extract text via OCR
• Metadata - Extract metadata only

Using Brevit.NET in LLM Prompts

Best Practices

1. Context First: Provide context before optimized data
2. Clear Instructions: Tell the LLM what format to expect
3. Examples: Include format examples in prompts

Example Prompt Template

var optimized = await brevit.OptimizeAsync(order);

var prompt = $@"You are analyzing order data. The data is in Brevit flattened format:

Context:
{optimized}

Task: Extract the order total and shipping address.

Format your response as JSON with keys: total, address";

Real-World Example

public class OrderAnalysisService
{
    private readonly BrevitClient _brevit;
    private readonly IOpenAIClient _openAI;

    public async Task<OrderAnalysis> AnalyzeOrderAsync(Order order)
    {
        // Optimize order data
        var optimized = await _brevit.OptimizeAsync(order);

        // Create prompt
        var prompt = $@"Analyze this order:

{optimized}

Questions:
1. What is the order total?
2. How many items?
3. Average item price?

Respond in JSON.";

        // Call LLM
        var response = await _openAI.GenerateAsync(prompt);
        return JsonSerializer.Deserialize<OrderAnalysis>(response);
    }
}

Syntax Cheatsheet

C# to Brevit Format

C# Structure	Brevit Format	Example
Property	PropertyName: value	OrderId: o-456
Nested property	Parent.Child: value	Customer.Name: John
Primitive array	Array[count]: val1,val2,val3	Friends[3]: ana,luis,sam
Uniform object array	Array[count]{Field1,Field2}:<br> val1,val2<br> val3,val4	Items[2]{Sku,Quantity}:<br> A-88,1<br> T-22,2
Array element (fallback)	Array[index].Property: value	Items[0].Sku: A-88
Nested array	Parent[index].Child[index]	Orders[0].Items[1].Sku
Null value	Property: null	Phone: null
Boolean	Property: True	IsActive: True
Number	Property: 123	Quantity: 5

Special Cases

• Empty Collections: Items: [] → Items: []
• Nested Empty Objects: Metadata: {} → Metadata: {}
• Nullable Types: Phone: null → Phone: null
• Enums: Converted to string representation
• Tabular Arrays: Automatically detected when all objects have same properties
• Primitive Arrays: Automatically detected when all elements are primitives

Other Implementations

Brevit is available in multiple languages:

Language	Package	Status
C# (.NET)	Brevit.NET	✅ Stable (This)
JavaScript	brevit-js	✅ Stable
Python	brevit-py	✅ Stable

Full Specification

Format Specification

1. Key-Value Pairs: One pair per line
2. Separator: : (colon + space)
3. Key Format: Property names with dot/bracket notation
4. Value Format: String representation of values
5. Line Endings: \n (newline)

Grammar

brevit := line*
line := key ": " value "\n"
key := identifier ("." identifier | "[" number "]")*
value := string | number | boolean | null
identifier := [A-Za-z_][A-Za-z0-9_]*

Examples

Simple Object:

OrderId: o-456
Status: SHIPPED

Nested Object:

Customer.Name: John Doe
Customer.Email: john@example.com

Array:

Items[0].Sku: A-88
Items[0].Quantity: 1
Items[1].Sku: T-22
Items[1].Quantity: 2

Complex Structure:

OrderId: o-456
Customer.Name: John Doe
Items[0].Sku: A-88
Items[0].Price: 29.99
Items[1].Sku: T-22
Items[1].Price: 39.99
Shipping.Address.Street: 123 Main St
Shipping.Address.City: Toronto

Advanced Usage

Custom Text Optimizer with Semantic Kernel

public class SemanticKernelTextOptimizer : ITextOptimizer
{
    private readonly Kernel _kernel;

    public SemanticKernelTextOptimizer(Kernel kernel)
    {
        _kernel = kernel;
    }

    public async Task<string> OptimizeTextAsync(string longText, BrevitConfig config)
    {
        var summarizeFunction = _kernel.CreateFunctionFromPrompt(
            "Summarize the following text: {{$input}}"
        );

        var result = await _kernel.InvokeAsync(
            summarizeFunction, 
            new() { ["input"] = longText }
        );
        
        return result.ToString();
    }
}

Custom Image Optimizer with Azure AI Vision

public class AzureVisionImageOptimizer : IImageOptimizer
{
    private readonly ImageAnalysisClient _client;

    public AzureVisionImageOptimizer(ImageAnalysisClient client)
    {
        _client = client;
    }

    public async Task<string> OptimizeImageAsync(byte[] imageData, BrevitConfig config)
    {
        var result = await _client.AnalyzeAsync(
            BinaryData.FromBytes(imageData),
            VisualFeatures.Read
        );

        return result.Value.Read.Text;
    }
}

Filter Mode

var config = new BrevitConfig(
    JsonMode: JsonOptimizationMode.Filter
)
{
    JsonPathsToKeep = new List<string>
    {
        "user.name",
        "order.orderId",
        "order.items[*].sku"
    }
};

Examples

Example 1: Optimize Complex POCO

public class User
{
    public string Id { get; set; }
    public string Name { get; set; }
    public bool IsActive { get; set; }
    public ContactInfo Contact { get; set; }
    public List<Order> Orders { get; set; }
}

var user = new User
{
    Id = "u-123",
    Name = "Javian",
    IsActive = true,
    Contact = new ContactInfo
    {
        Email = "support@javianpicardo.com",
        Phone = null
    },
    Orders = new List<Order>
    {
        new Order { OrderId = "o-456", Status = "SHIPPED" }
    }
};

// Automatic optimization - analyzes data structure and selects best strategy
string optimized = await brevit.BrevityAsync(user);

// Or use explicit optimization
string explicit = await brevit.OptimizeAsync(user);

Example 2: Optimize JSON String

string json = @"{
    ""order"": {
        ""orderId"": ""o-456"",
        ""status"": ""SHIPPED"",
        ""items"": [
            { ""sku"": ""A-88"", ""name"": ""Brevit Pro"", ""quantity"": 1 }
        ]
    }
}";

string optimized = await brevit.OptimizeAsync(json);

Example 3: Process Long Text

string longDocument = await File.ReadAllTextAsync("document.txt");
string optimized = await brevit.OptimizeAsync(longDocument);
// Triggers text optimization if length > LongTextThreshold

Best Practices

1. Use Dependency Injection: Register all components in DI container
2. Implement Custom Optimizers: Replace stubs with real LLM integrations
3. Configure Thresholds: Adjust LongTextThreshold based on use case
4. Monitor Token Usage: Track before/after token counts
5. Cache Results: Cache optimized results for repeated queries
6. Error Handling: Wrap optimize calls in try-catch blocks
7. Async All The Way: Use async/await throughout your pipeline

Troubleshooting

Issue: "Failed to serialize POCO"

Solution: Ensure object is serializable. Use [JsonIgnore] to exclude properties:

public class Order
{
    public string OrderId { get; set; }
    
    [JsonIgnore]
    public string InternalId { get; set; } // Excluded from optimization
}

Issue: YAML conversion not working

Solution: Install YamlDotNet and implement conversion:

dotnet add package YamlDotNet

Issue: Text summarization returns stub

Solution: Implement custom ITextOptimizer using Semantic Kernel or your LLM service.

Contributing

Contributions welcome! See CONTRIBUTING.md for guidelines.

License

MIT License - see LICENSE file for details.

Support

• Documentation: https://brevit.dev/docs
• Issues: https://github.com/JavianDev/Brevit/issues
• Email: support@javianpicardo.com

Version History

• 0.1.0 (Current): Initial release with core optimization features