EasyImageSharp.AI 1.2.0

<div align="center">

<img src="https://raw.githubusercontent.com/FarhanLodi/EasyImageSharp/main/src/EasyImageSharp/Assets/icon.png" alt="EasyImageSharp" width="128" height="128" />

EasyImageSharp.AI

</div>

ONNX-powered image operations for EasyImageSharp: document orientation, page dewarping, super-resolution, denoising, background removal and learned binarisation — with a model hub that downloads, checksum-verifies and caches models for you.

This package is optional. The core EasyImageSharp package has no dependencies beyond the framework; everything here is opt-in, and the ONNX Runtime native binaries live only in this package.

dotnet add package EasyImageSharp.AI

Quick start

using EasyImageSharp;
using EasyImageSharp.AI;
using EasyImageSharp.PixelFormats;
using EasyImageSharp.Processing;

using var ai = new ImageAiSession();

using Image<Rgb24> page = Image.Load<Rgb24>("phone-photo.jpg");

page.AutoOrient(ai);                                  // fix a sideways or upside-down page
page.DewarpDocument(ai);                              // flatten a curled page
page.DenoiseAI(ai);                                   // learned denoise
page.Mutate(ctx => ctx.Deskew().SauvolaThreshold());  // classical finish

page.SaveAsPng("clean.png");

ImageAiSession caches one inference session per model and is safe to keep for the lifetime of your application. Dispose it when you are done.

Document profiles with diagnostics

using var processed = await page.PreprocessDocumentAIAsync(ai, new()
{
    AutoOrient = true,
    Dewarp = true,
    Preprocessing = new() { Profile = DocumentPreprocessingProfile.NeuralOcr },
});
processed.Image.SaveAsPng("rectified.png");
foreach (var stage in processed.Pages[0].Stages)
    Console.WriteLine($"{stage.Name}: applied={stage.Applied}; {stage.Reason}; {stage.Elapsed}");

The result owns its output image and leaves the source unchanged. Each frame is processed independently. Orientation scores below MinimumOrientationConfidence cause abstention. Requested AI stages require their models; an unavailable model or incompatible export raises an error rather than silently skipping work.

The published UVDoc model consumes BGR at full resolution, with edge padding to a multiple of 32, and returns a rectified image. It provides no correspondence grid, so both coordinate mapping methods return false after this stage. Image output is opaque because the model does not return alpha. The default 16-million-pixel padded-input budget rejects larger inputs; configure DewarpContract.MaximumInputPixels according to available memory. It does not silently reduce resolution. Explicit FixedInputSize is available for exports requiring it; image-output detail is then limited by that size.

Custom model overrides can select RGB order, named inputs/outputs, fixed dimensions or a normalized [1,2,H,W] backward sampling grid through DocumentDewarpContract. Grid exports resample the original full-resolution image with bilinear premultiplied-alpha sampling and replicated borders. Set SizeMultiple=1 for unpadded grid coordinates. GridAlignCorners must match the export. Grid results expose processed-to-source mapping; forward mapping remains unavailable because an arbitrary grid may fold. Never assume unavailable mapping is identity. DewarpDocumentWithContractAsync provides the same explicit contract for an in-place operation.

The standard orientation path uses aspect-preserving bilinear resize to short side 256 followed by a 224-square center crop. These preprocessing corrections also apply to the existing AutoOrient and DewarpDocument methods. Their output changes from the previous stretched classifier input and fixed 488-by-712 RGB dewarp path. For generic image models, ImageModelContract.UseBgrChannelOrder applies normalization in the model's channel order.

Operations

Each has an ...Async counterpart taking a CancellationToken.

Why these, next to the classical operators

The core library already has projection-profile deskew, Sauvola binarisation and median denoising, and those remain the right default: they are fast, deterministic and need no model download. These operations handle what the classical ones cannot:

• Orientation — a projection profile is symmetric under 90° and 180°, so it cannot detect an upside-down page. The classifier can.
• Dewarping — a homography maps one plane to another; it cannot straighten a curved book spine.
• Super-resolution — recovers stroke topology on small glyphs that bicubic upscaling smears.
• Learned binarisation — predicts a per-pixel threshold instead of one global window and constant.

A good pipeline uses both: the model for the quadrant, the classical operator for the residual angle.

Models

Models are published at huggingface.co/EasyImageSharp/EasyImageSharp-models, fetched on first use and cached under %LOCALAPPDATA%/EasyImageSharp/models (~/.local/share on Linux and macOS). Every file has its SHA-256 pinned in this package.

Weights keep their upstream authors' licences. To run your own export instead — a re-trained model, an int8 variant or a file from an internal mirror — point the library at it by model name:

var options = new ImageAiOptions();
options.ModelPathOverrides["super-resolution-x4"] = @"C:\models\realesrgan_general_x4v3.onnx";

using var ai = new ImageAiSession(options);

Each model's exact input contract — tensor name, shape and normalisation — is documented on the corresponding ModelRegistry property, so an export that matches will work without code changes.

Configuration

var options = new ImageAiOptions
{
    ExecutionProvider = ExecutionProvider.Auto,   // Cpu, Cuda, DirectML, CoreML
    Quantize = true,                              // prefer int8 weights where available
    Offline = false,                              // true: never download, fail if not cached
    CachePath = null,                             // null: use the default cache directory
    AllowUnverifiedModels = false,                // keep checksum verification fail-closed
    IntraOpNumThreads = null,
    Log = Console.WriteLine,
};

using var ai = new ImageAiSession(options);

GPU execution. ExecutionProvider.Auto tries the GPU providers whose native packages are present and silently falls back to CPU. To enable one, add the matching ONNX Runtime package to your application — for example Microsoft.ML.OnnxRuntime.Gpu for CUDA or Microsoft.ML.OnnxRuntime.DirectML for DirectML. This package deliberately does not force those native assets on every consumer.

Environment overrides. EASYIMAGESHARP_CACHE sets the cache directory and EASYIMAGESHARP_MODEL_BASE_URL redirects downloads to a mirror.

Air-gapped and offline deployment

The model hub is fail-closed by design: downloads are HTTPS-only, every file is verified against a pinned SHA-256, and a mismatch deletes the file and throws rather than running an unverified model.

For an environment without internet access, pre-seed the cache and set Offline = true:

var options = new ImageAiOptions
{
    CachePath = "/opt/myapp/models",
    Offline = true,   // OfflineModelMissingException if a model is not already cached
};

Downloads are atomic (written to .part and renamed), resume with HTTP range requests, and retry with exponential backoff. Concurrent requests for the same model collapse into a single download.

Bring your own model

Any image-to-image ONNX model can be run through the same tiling and normalisation machinery:

using Image<Rgb24> output = ImageModelRunner.Run(
    ai,
    modelPath,
    input,
    new ImageModelContract
    {
        InputName = "input",
        Normalization = TensorNormalization.Unit,   // 0-1; also ImageNet mean/std
        ScaleFactor = 2,                            // output size relative to input
        TileSize = 256,
        TileOverlap = 16,
    });

Exceptions

Requirements

Targets net8.0 and net10.0, and depends on Microsoft.ML.OnnxRuntime.

License

MIT — Copyright © 2026 Farhan Lodi. Model weights are covered by their own upstream licences, listed in the table above and on each ModelRegistry entry.