FastCompute 0.5.0

FastCompute.NET

FastCompute.NET is a strongly named .NET 8 library for fast array processing. It provides one API for single-threaded CPU, multi-threaded CPU, SIMD, and ILGPU execution and supports float, double, and int arrays.

Version 0.5.0 is the first stable release. The assembly public key token is c76a60c96d65300c.

Quick start

1. Install the package

dotnet add package FastCompute --version 0.5.0

The consuming project must target .NET 8 or a compatible later framework.

2. Process an array

using FastCompute;

float[] source = [0.0f, 0.5f, 1.0f, 1.5f];

float[] result = Compute.Run(
    source,
    value => GpuMath.Sin(value) * 2.0f);

Compute.Run uses ComputeBackendKind.Auto by default. FastCompute evaluates the operation and array size and then selects Scalar CPU, Parallel CPU, SIMD, or GPU. Auto mode is transfer-conservative: merely having a GPU does not mean that every operation is sent to it.

3. Check which backend was selected

using FastCompute;
using FastCompute.Diagnostics;

ComputeResult<float[]> result = Compute.RunWithDiagnostics(
    source,
    value => GpuMath.Sin(value) * 2.0f);

Console.WriteLine($"Backend: {result.Diagnostics.Backend}");
Console.WriteLine($"Device:  {result.Diagnostics.DeviceName ?? "CPU"}");
Console.WriteLine($"Reason:  {result.Diagnostics.BackendSelectionReason}");

The computed array is available through result.Value.

4. Force a backend when required

float[] simdResult = Compute.Run(
    source,
    value => value * 2.0f + 1.0f,
    new ComputeOptions
    {
        Backend = ComputeBackendKind.Simd
    });

An explicitly selected backend is a strict request and never silently falls back. If the expression or machine does not support that backend, the operation throws an exception.

Detailed guide

Core operations

Map

Run applies a unary expression to every array element and returns a new array:

float[] mapped = Compute.Run(
    source,
    value => GpuMath.Clamp(value * 1.25f, 0.0f, 1.0f));

Zip

Zip combines arrays element by element:

float[] left = [1.0f, 2.0f, 3.0f];
float[] right = [10.0f, 20.0f, 30.0f];

float[] zipped = Compute.Zip(
    left,
    right,
    (x, y) => x + y);

Both arrays must have the same length.

In-place processing

Use the in-place variants when the result may overwrite the input array:

Compute.RunInPlace(
    source,
    value => value * 2.0f + 1.0f);

Compute.ZipInPlace(
    target: left,
    right: right,
    (x, y) => x + y);

The returned reference is the same array as the target. This reduces managed allocations, but cancellation or an execution failure can leave an in-place array partially modified.

Reductions

float sum = Compute.Sum(source);
float minimum = Compute.Min(source);
float maximum = Compute.Max(source);
float average = Compute.Average(source);

The same operations are available for double[] and int[]. Average returns the array element type, so integer average uses integer semantics.

Supported element types and expressions

Float expressions support arithmetic, comparisons, conditional expressions, captured primitive constants, and these GpuMath methods:

• Abs, Min, Max, and Clamp;
• Sqrt, Pow, Exp, Log, and Log10;
• Sin, Cos, and Tan;
• Floor, Ceiling, and Round.

float multiplier = 0.75f;
bool clamp = true;

float[] result = Compute.Run(
    source,
    value => clamp
        ? GpuMath.Clamp(GpuMath.Sin(value) * multiplier, 0.0f, 1.0f)
        : GpuMath.Sin(value) * multiplier);

Double and integer expressions use arithmetic and the applicable supported System.Math overloads:

double[] precise = Compute.Run(
    new[] { 0.0, 0.5, 1.0 },
    value => Math.Sin(value) * Math.Exp(-value));

int[] integers = Compute.Run(
    new[] { 1, 2, 3 },
    value => value * 2 + 1);

The expression API converts an expression tree to FastCompute's backend-independent instruction representation. Captured float, double, int, and bool values are supported. Calls through captured reference objects and arbitrary .NET methods are intentionally rejected because they cannot be translated to SIMD or GPU instructions.

Arbitrary user methods

When a transformation contains unrestricted application code, use RunDelegate. It executes a normal Func<float, float> on Scalar or Parallel CPU without translating it:

float CustomCalculation(float value) =>
    MathF.Sin(value) + GetApplicationCoefficient(value);

float[] result = Compute.RunDelegate(
    source,
    CustomCalculation,
    new ComputeOptions
    {
        Backend = ComputeBackendKind.ParallelCpu
    });

RunDelegate currently supports only float[] and the Scalar and Parallel CPU backends. It cannot execute arbitrary CLR methods on SIMD or GPU hardware.

Choosing a backend

var options = new ComputeOptions
{
    Backend = ComputeBackendKind.Auto,
    MaxDegreeOfParallelism = Environment.ProcessorCount
};

The available modes are:

SIMD is not another form of Parallel.For: it processes several values per CPU instruction on the calling thread. Parallel CPU and SIMD are separate backends.

Auto mode is appropriate when the library should make the decision. Use an explicit backend for reproducible performance tests, a known deployment environment, or full user control:

float[] result = source.RunExplicit(
    value => value * value,
    ComputeBackendKind.ParallelCpu);

RunExplicit is a LINQ-style extension for float[], double[], and int[]. It rejects Auto by design.

Selecting one of several GPUs

Accelerator indices are assigned by ILGPU and can include CPU accelerators. Always discover them on the target machine:

using FastCompute;

IReadOnlyList<ComputeDeviceInfo> accelerators =
    ComputeContext.GetAccelerators();

foreach (ComputeDeviceInfo device in accelerators)
{
    Console.WriteLine(
        $"{device.Index}: {device.Name} ({device.AcceleratorType})");
}

To prefer a particular hardware GPU without forcing its use, pass its index to Auto mode:

ComputeDeviceInfo preferredGpu = ComputeContext.GetAccelerators()
    .First(device => !device.AcceleratorType.Contains(
        "CPU",
        StringComparison.OrdinalIgnoreCase));

ComputeResult<float[]> result = Compute.RunWithDiagnostics(
    source,
    value => GpuMath.Sin(value) * GpuMath.Exp(-value * value),
    new ComputeOptions
    {
        Backend = ComputeBackendKind.Auto,
        PreferredGpuAcceleratorIndex = preferredGpu.Index
    });

PreferredGpuAcceleratorIndex means “use this GPU if the planner decides that GPU execution is beneficial.” Auto may still select SIMD, Parallel CPU, or Scalar. If the preferred index is unavailable or does not identify a hardware GPU, Auto continues with a CPU backend.

To require that accelerator, create a reusable context and explicitly request GPU execution:

using ComputeContext gpu = ComputeContext.Create(
    new ComputeContextOptions
    {
        AcceleratorIndex = preferredGpu.Index
    });

Console.WriteLine($"Selected accelerator: {gpu.DeviceName}");

float[] gpuResult = Compute.Run(
    source,
    value => GpuMath.Sin(value),
    new ComputeOptions
    {
        Backend = ComputeBackendKind.Gpu,
        GpuContext = gpu
    });

GpuContext and PreferredGpuAcceleratorIndex are mutually exclusive. A context created without an index prefers a non-CPU accelerator and falls back to the ILGPU CPU accelerator if no hardware GPU is available.

When GPU kernels are compiled

For the first operation in a ComputeContext, FastCompute:

1. validates and lowers the expression;
2. obtains or compiles the required ILGPU kernel template;
3. caches the lowered expression and compiled template in that context;
4. uploads data, starts the kernel, and downloads the result.

Repeated compatible operations on the same context reuse those caches. This is why the first GPU call is normally slower than subsequent calls. A new context has its own caches.

Use PrecompileAll during application warm-up to compile every implemented kernel template:

using ComputeContext gpu = ComputeContext.Create(
    new ComputeContextOptions
    {
        AcceleratorIndex = preferredGpu.Index
    });

IReadOnlyList<ComputeCompilationResult> templates = gpu.PrecompileAll();

Console.WriteLine(
    $"Prepared: {templates.Count}; " +
    $"cache hits: {templates.Count(item => item.CacheHit)}");

Use Precompile<T> to validate, lower, and cache a particular expression:

ComputeCompilationResult compilation =
    gpu.Precompile<float>(
        value => GpuMath.Sin(value) * GpuMath.Exp(-value));

Console.WriteLine($"Cache hit: {compilation.CacheHit}");
Console.WriteLine($"Compile time: {compilation.CompilationTime}");

For an operation that will be called repeatedly, create a prepared operation:

PreparedCompute<float> prepared =
    gpu.Prepare<float>(value => GpuMath.Sin(value) * 2.0f);

float[] first = prepared.Run(source);
float[] second = prepared.Run(otherSource);

Precompilation removes kernel compilation from the first business operation. It does not upload the future input array and cannot remove normal GPU transfer cost.

Arrays larger than available GPU memory

GPU Map, Zip, in-place operations, reductions, and Histogram can run in sequential chunks. Chunking is enabled by default and the effective working set is limited by the context safety limit and optional operation budget:

float[] result = Compute.Run(
    source,
    value => GpuMath.Sin(value),
    new ComputeOptions
    {
        Backend = ComputeBackendKind.Gpu,
        GpuContext = gpu,
        GpuMemoryBudgetBytes = 512L * 1024 * 1024,
        GpuChunkElementCount = 4_000_000
    });

GpuChunkElementCount is an optional upper bound. Without it, FastCompute calculates a chunk size from the memory budget. Setting EnableGpuChunking = false makes insufficient memory a hard error.

Explicit out-of-place float Map can optionally overlap transfers and execution using two accelerator streams:

float[] result = Compute.Run(
    source,
    value => GpuMath.Sin(value),
    new ComputeOptions
    {
        Backend = ComputeBackendKind.Gpu,
        GpuContext = gpu,
        GpuChunkElementCount = 4_000_000,
        EnableGpuStreaming = true
    });

Streaming is opt-in because its benefit depends on the accelerator, bus, and expression. Other operations use sequential chunks.

Keeping data on the accelerator

For several consecutive GPU operations, upload once and use ComputeBuffer<T>:

using ComputeBuffer<float> input = gpu.Upload(source);
using ComputeBuffer<float> scaled =
    input.Select(value => value * 0.75f);
using ComputeBuffer<float> transformed =
    scaled.Select(value => GpuMath.Sin(value));

float sum = transformed.Sum();

float[] output = new float[transformed.Length];
transformed.Download(output);

Console.WriteLine(transformed.Context.DeviceName);
Console.WriteLine(transformed.Location); // Host or Device

Available resident operations include Select, SelectInPlace, Zip, ZipInPlace, Sum, Min, Max, Average, and download to an array or Span<T>. Chained float selections use a lazy, copy-on-write execution graph where applicable. Dispose buffers and their context to release accelerator resources.

The context also owns a bounded, thread-safe transient float-buffer pool:

using ComputeContext gpu = ComputeContext.Create(
    new ComputeContextOptions
    {
        AcceleratorIndex = preferredGpu.Index,
        MemoryPoolLimitBytes = 256L * 1024 * 1024
    });

Console.WriteLine(gpu.MemoryPoolStatistics.RetainedBytes);
Console.WriteLine(gpu.MemoryPoolStatistics.EvictedBuffers);

The limit controls idle memory retained for reuse. Active operations may use more. Set MemoryPoolLimitBytes = 0 to disable idle-buffer retention.

Histogram

Histogram splits a numeric range into equal-width bins:

float[] samples = [0.0f, 0.1f, 0.5f, 0.9f, 1.0f];

int[] histogram = Compute.Histogram(
    samples,
    binCount: 256,
    minimum: 0.0f,
    maximum: 1.0f);

Finite values outside the range are clamped to the first or last bin by default. NaN is always ignored. To ignore all out-of-range values:

int[] histogram = Compute.Histogram(
    samples,
    binCount: 256,
    minimum: 0.0f,
    maximum: 1.0f,
    new HistogramOptions
    {
        OutOfRangeMode = HistogramOutOfRangeMode.Ignore
    });

Histogram supports Scalar, Parallel CPU, and GPU. Automatic GPU selection is opt-in through ComputeThresholdOptions.GpuHistogramThreshold.

Diagnostics

Diagnostic APIs return the value together with planning and execution details:

ComputeResult<float[]> result = Compute.RunWithDiagnostics(
    source,
    value => value * value,
    new ComputeOptions
    {
        Backend = ComputeBackendKind.Auto
    });

ComputeDiagnostics d = result.Diagnostics;

Console.WriteLine($"Backend:       {d.Backend}");
Console.WriteLine($"Device:        {d.DeviceName ?? "CPU"}");
Console.WriteLine($"Planning:      {d.PlanningTime}");
Console.WriteLine($"Compilation:   {d.CompilationTime}");
Console.WriteLine($"Execution:     {d.ExecutionTime}");
Console.WriteLine($"Upload bytes:  {d.UploadedBytes}");
Console.WriteLine($"Download bytes:{d.DownloadedBytes}");
Console.WriteLine($"Chunks:        {d.ChunkCount}");
Console.WriteLine($"Streaming:     {d.IsStreaming}");
Console.WriteLine($"Cache hit:     {d.KernelCacheHit}");

Variants include RunWithDiagnostics, RunInPlaceWithDiagnostics, ZipWithDiagnostics, ZipInPlaceWithDiagnostics, HistogramWithDiagnostics, and diagnostic reduction methods such as SumWithDiagnostics.

Async-compatible API and cancellation

using var cancellationSource = new CancellationTokenSource();

float[] mapped = await Compute.RunAsync(
    source,
    value => value * 2.0f,
    new ComputeOptions
    {
        CancellationToken = cancellationSource.Token
    });

using ComputeBuffer<float> buffer =
    await gpu.UploadAsync(source, cancellationSource.Token);

float[] downloaded =
    await buffer.DownloadAsync(cancellationSource.Token);

ILGPU currently exposes synchronous completion primitives. These methods therefore return completed tasks and do not hide blocking work in Task.Run. They are async-compatible API boundaries, not guaranteed non-blocking GPU execution.

Performance guidance

• Use Auto for general-purpose calls, but verify the selected backend with diagnostics for important workloads.
• A direct loop is normally best for very small arrays.
• GPU execution is most useful when enough computation compensates for upload and download cost.
• Reuse ComputeContext to reuse compiled kernels.
• Use resident buffers for multi-step GPU pipelines.
• Use in-place methods when overwriting the source is acceptable.
• Benchmark on the deployment hardware; GPU model, memory bandwidth, CPU SIMD, and array size all affect the result.

The opt-in performance gate compares Auto with an equivalent single-threaded loop on large simple, heavy, and in-place Map workloads:

dotnet run --project benchmarks/FastCompute.Benchmarks `
  --configuration Release -- `
  --assert-performance

It exits with code 1 if FastCompute is more than 5% slower than the loop. Run it on otherwise idle hardware. The complete backend and operation matrix can be run with:

dotnet run --project benchmarks/FastCompute.Benchmarks `
  --configuration Release -- `
  --filter "*SpecificationMatrixBenchmarks*"

Common problems

GPU is slower than Parallel CPU.

This is expected when transfer and compilation costs exceed the kernel work. Reuse a context, precompile, keep intermediate data in resident buffers, or let Auto choose CPU.

The wrong GPU was selected.

Print ComputeContext.GetAccelerators(), select the runtime index, and either set PreferredGpuAcceleratorIndex for Auto or create a context with AcceleratorIndex for strict GPU execution.

An explicit backend throws instead of using CPU.

This is the intended contract. Explicit modes never silently fall back. Use Auto when fallback is required.

An expression cannot be translated.

Use supported arithmetic and math methods, or use RunDelegate for unrestricted float CPU code.

The first GPU call is slow.

The first call includes expression lowering and kernel compilation. Reuse the context and call PrecompileAll, Precompile<T>, or Prepare<T> during warm-up.

Sample application

The console sample demonstrates Auto selection, diagnostics, captured constants, precompilation, and a resident GPU pipeline:

dotnet run --project samples/FastCompute.Sample.Console `
  --configuration Release

Build, test, and package

dotnet build FastCompute.sln --configuration Release
dotnet test FastCompute.sln --configuration Release --no-build
./pack.ps1 -Version 0.5.0

pack.ps1 builds and tests the solution, creates .nupkg and .snupkg artifacts, verifies the strong-name identity, and runs a package-only consumer smoke test. On a Windows or Linux CI machine without a hardware GPU:

./pack.ps1 -Version 0.5.0 -SkipGpuTests

Further documentation

• Stable release compliance
• Additional technical requirements
• Stage 1 architecture
• Stage 2 architecture
• Stage 3 GPU implementation and compilation
• SIMD architecture
• Stage 4 reductions and memory pooling
• Stage 6 execution graph
• Release history and known limitations

Authors

• staszx — project author and maintainer.
• OpenAI Codex — implementation and documentation assistance.