AzureAICommunity.Agent.Middleware.ContextCompressionMiddleware 1.0.0

<div align="center">

🗜️ AzureAICommunity – Context Compression Middleware

A plug-and-play Microsoft.Extensions.AI middleware that automatically summarises older conversation history when the estimated token count approaches your configured limit — preventing context-window overflow while keeping the most recent messages intact.

Long-running conversations accumulate history. Every message is sent to the LLM on every turn, so token usage grows linearly. Eventually you hit the model's context-window limit and requests start failing.

</div>

The Solution

ContextCompressionMiddleware intercepts the request before it reaches the LLM. When the estimated token count exceeds a configurable trigger threshold, the middleware:

1. Splits the history into old and recent segments.
2. Calls the LLM to produce a compact summary of the old segment.
3. Replaces the old segment with a single system summary message.
4. Forwards the compressed history to the inner client as normal.

Features

Installation

dotnet add package AzureAICommunity.Agent.Middleware.ContextCompressionMiddleware

Quick Start

using AzureAICommunity.Agent.Middleware.ContextCompressionMiddleware;
using Microsoft.Extensions.AI;
using OllamaSharp;

IChatClient ollamaClient = new OllamaApiClient(new Uri("http://localhost:11434/"), "llama3.2");

var client = ollamaClient
    .AsBuilder()
    .Use(inner => new ContextCompressionMiddleware(
        inner,
        maxTokens: 4000,
        triggerRatio: 0.80,
        keepLastMessages: 6))
    .Build();

// Use like any other IChatClient — compression is transparent
var response = await client.GetResponseAsync(conversationHistory);

Configuration

Parameters

Threshold Callback

var client = ollamaClient
    .AsBuilder()
    .Use(inner => new ContextCompressionMiddleware(
        inner,
        maxTokens: 4000,
        onThresholdReached: info =>
        {
            Console.WriteLine($"Threshold hit: {info.TokensUsed} / {info.MaxTokens} tokens");
            return true; // true = compress, false = throw ContextCompressionThresholdException
        }))
    .Build();

Custom Token Counter

// Example using a precise tokeniser
var client = ollamaClient
    .AsBuilder()
    .Use(inner => new ContextCompressionMiddleware(
        inner,
        maxTokens: 4000,
        tokenCounter: messages =>
        {
            // supply your own counting logic here
            return messages.Sum(m => m.Text?.Length / 4 ?? 0);
        }))
    .Build();

Separate Summariser Client

// Use a cheaper model for summarisation
IChatClient summarizerClient = new OllamaApiClient(new Uri("http://localhost:11434/"), "phi3");

var client = expensiveClient
    .AsBuilder()
    .Use(inner => new ContextCompressionMiddleware(
        inner,
        maxTokens: 8000,
        summarizerClient: summarizerClient))
    .Build();

🤝 Contributing

Contributions are welcome! Please open an issue to discuss what you'd like to change before submitting a pull request.

📁 Repository: https://github.com/rvinothrajendran/AgentFramework

1. Fork the repository
2. Create a feature branch (git checkout -b feature/my-feature)
3. Commit your changes (git commit -m 'Add my feature')
4. Push to the branch (git push origin feature/my-feature)
5. Open a Pull Request

👤 Author

Built and maintained by Vinoth Rajendran.

• 🐙 GitHub: github.com/rvinothrajendran — follow for more projects!
• 📺 YouTube: youtube.com/@VinothRajendran — subscribe for tutorials and demos!
• 💼 LinkedIn: linkedin.com/in/rvinothrajendran — let's connect!

📄 License

MIT © 2026 Vinoth Rajendran – AzureAICommunity