GP.Net 0.0.1

GP.Net - Gaussian Process Library for .NET

A high-performance Gaussian Process library for .NET, implemented based on scikit-learn's Gaussian Process module and "Gaussian Processes for Machine Learning" (GPML) by Rasmussen and Williams (2006).

Overview

GP.Net provides a production-ready implementation of Gaussian Process Regression and Classification with:

• Numerically equivalent to scikit-learn (validated via cross-testing)
• SIMD-accelerated kernel computations using System.Numerics.Vector<T>
• MathNet.Numerics for reliable linear algebra operations
• AOT-compatible and trimming-friendly for modern .NET deployment

Implementation Reference

This library is based on:

• scikit-learn 1.5.2 Gaussian Process module
• GPML textbook Algorithm 2.1 (Regression) and Algorithm 3.1 (Classification with Laplace Approximation)
• Cross-validated against sklearn to ensure correctness (see CrossTest)

Features

Models

• Regressor - Gaussian Process Regression

  • Implements GPML Algorithm 2.1
  • Uncertainty quantification (predictive variance/std)
  • Optional target normalization
  • Optional input scaling (MinMaxScaler, StandardScaler)
  • Efficient Cholesky decomposition

• Classifier - Binary Classification

  • Implements GPML Algorithm 3.1 (Laplace Approximation)
  • Probabilistic predictions
  • Newton-Raphson optimization for posterior mode
  • Optional input scaling

Kernels

All kernels implement IKernel interface:

• RBFKernel - Radial Basis Function (Squared Exponential)

  k(x, x') = exp(-||x - x'||² / (2 * length_scale²))
  

• ConstantKernel - Constant value kernel

  k(x, x') = constant_value
  

• WhiteKernel - White noise kernel for observation noise

  k(x, x') = noise_level if x == x' else 0
  

• SumKernel - Sum of two kernels

  k(x, x') = k1(x, x') + k2(x, x')
  

• MulKernel - Scalar multiplication or product of kernels

  k(x, x') = scalar * k1(x, x')  or  k1(x, x') * k2(x, x')
  

• ProductKernel - Element-wise product of kernels

  k(x, x') = k1(x, x') * k2(x, x')
  

Scalers

Implements IScaler interface for feature scaling:

• StandardScaler - Z-score normalization

  X_scaled = (X - mean) / std
  

• MinMaxScaler - Min-Max scaling to specified range

  X_scaled = (X - X_min) / (X_max - X_min) * (max - min) + min
  

Installation

dotnet add package GP.Net

Or add to your .csproj:

<PackageReference Include="GP.Net" Version="1.0.0" />

Quick Start

Regression Example

using GP.Net.Models;
using GP.Net.Kernels;

// Training data: simple sine wave
var X = new double[,] { { 0.0 }, { 1.0 }, { 2.0 }, { 3.0 }, { 4.0 } };
var y = new double[] { 0.0, 0.84, 0.91, 0.14, -0.76 };

// Create kernel: C * RBF (similar to sklearn's C(1.0) * RBF(1.0))
var kernel = new MulKernel(
    new ConstantKernel(1.0),
    new RBFKernel(lengthScale: 1.0)
);

// Create and fit model
var gpr = new Regressor(
    kernel: kernel,
    alpha: 1e-5,        // Regularization term
    normalizeY: true    // Normalize target values
);

gpr.Fit(X, y);

// Make predictions with uncertainty
var XTest = new double[,] { { 0.5 }, { 1.5 }, { 2.5 } };
var (predictions, std) = gpr.Predict(XTest, returnStd: true);

for (int i = 0; i < predictions.Length; i++)
{
    Console.WriteLine($"x={XTest[i, 0]}: y={predictions[i]:F4} ± {std![i]:F4}");
}

Classification Example

using GP.Net.Models;
using GP.Net.Kernels;

// Binary classification data: linearly separable
var X = new double[,]
{
    { 0.0 }, { 1.0 }, { 2.0 }, { 3.0 },
    { 4.0 }, { 5.0 }, { 6.0 }, { 7.0 }
};
var y = new int[] { 0, 0, 0, 0, 1, 1, 1, 1 };

// Create classifier with RBF kernel
var gpc = new Classifier(
    kernel: new RBFKernel(lengthScale: 1.0),
    maxIterPredict: 50
);

gpc.Fit(X, y);

// Predict classes
var XTest = new double[,] { { 0.5 }, { 3.5 }, { 5.5 } };
var predictions = gpc.Predict(XTest);

Console.WriteLine($"Predicted classes: {string.Join(", ", predictions)}");

// Predict probabilities
var proba = gpc.PredictProba(XTest);
for (int i = 0; i < XTest.GetLength(0); i++)
{
    Console.WriteLine($"x={XTest[i, 0]}: P(y=0)={proba[i, 0]:F4}, P(y=1)={proba[i, 1]:F4}");
}

Using Feature Scaling

using GP.Net.Models;
using GP.Net.Kernels;
using GP.Net.Scalers;

// Data with different scales
var X = new double[,] { { 100.0 }, { 200.0 }, { 300.0 }, { 400.0 } };
var y = new double[] { 1.0, 2.0, 3.0, 4.0 };

// Create regressor with MinMaxScaler
var gpr = new Regressor(
    kernel: new RBFKernel(1.0),
    alpha: 1e-5,
    normalizeY: true,
    normalizeX: true,
    scaler: new MinMaxScaler(featureRangeMin: 0.0, featureRangeMax: 1.0)
);

gpr.Fit(X, y);

// Test data will be automatically scaled
var XTest = new double[,] { { 150.0 }, { 350.0 } };
var (predictions, _) = gpr.Predict(XTest);

Console.WriteLine($"Predictions: {string.Join(", ", predictions.Select(p => p.ToString("F2")))}");

Custom Kernel Combinations

using GP.Net.Kernels;

// Example 1: Sum of RBF and White noise (common for noisy data)
var kernel1 = new SumKernel(
    new RBFKernel(lengthScale: 1.0),
    new WhiteKernel(noiseLevel: 0.1)
);

// Example 2: Constant * RBF (sklearn's C * RBF pattern)
var kernel2 = new MulKernel(
    new ConstantKernel(2.5),
    new RBFKernel(lengthScale: 0.5)
);

// Example 3: Product of two RBF kernels (for different feature groups)
var kernel3 = new ProductKernel(
    new RBFKernel(lengthScale: 1.0),
    new RBFKernel(lengthScale: 2.0)
);

// Example 4: Complex combination
var kernel4 = new SumKernel(
    new MulKernel(
        new ConstantKernel(1.0),
        new RBFKernel(1.0)
    ),
    new WhiteKernel(0.01)
);

API Reference

Regressor

public class Regressor : AbstractModel
{
    public Regressor(
        IKernel? kernel = null,
        double alpha = 1e-10,
        bool normalizeY = false,
        bool normalizeX = false,
        IScaler? scaler = null
    )

    public void Fit(double[,] X, double[] y)

    public (double[] Mean, double[]? Std) Predict(
        double[,] X,
        bool returnStd = false
    )
}

Parameters:

• kernel: Covariance function (default: RBFKernel(1.0))
• alpha: Regularization term added to kernel diagonal (default: 1e-10)
• normalizeY: Normalize target values to zero mean and unit variance (default: false)
• normalizeX: Apply input scaling using scaler (default: false)
• scaler: Scaler for input normalization (default: StandardScaler())

Methods:

• Fit(X, y): Fit the model to training data
  • X: Training features [n_samples, n_features]
  • y: Training targets [n_samples]
• Predict(X, returnStd): Make predictions
  • X: Test features [n_samples, n_features]
  • returnStd: If true, returns predictive standard deviation
  • Returns: (Mean, Std) tuple

Classifier

public class Classifier : AbstractModel
{
    public Classifier(
        IKernel? kernel = null,
        int maxIterPredict = 100,
        bool normalizeX = false,
        IScaler? scaler = null,
        double convergenceTol = 1e-10
    )

    public void Fit(double[,] X, int[] y)

    public int[] Predict(double[,] X)

    public double[,] PredictProba(double[,] X)
}

Parameters:

• kernel: Covariance function (default: RBFKernel(1.0))
• maxIterPredict: Maximum iterations for posterior mode finding (default: 100)
• normalizeX: Apply input scaling using scaler (default: false)
• scaler: Scaler for input normalization (default: StandardScaler())
• convergenceTol: Convergence tolerance for Newton-Raphson (default: 1e-10)

Methods:

• Fit(X, y): Fit the model to training data
  • X: Training features [n_samples, n_features]
  • y: Training labels [n_samples] (binary: 0 or 1)
• Predict(X): Predict class labels
  • X: Test features [n_samples, n_features]
  • Returns: Predicted labels [n_samples]
• PredictProba(X): Predict class probabilities
  • X: Test features [n_samples, n_features]
  • Returns: Probabilities [n_samples, 2] (columns: P(y=0), P(y=1))

Kernels

All kernels implement the IKernel interface:

public interface IKernel
{
    double[,] Compute(double[,] X, double[,]? Y = null);
    double[] ComputeDiag(double[,] X);
    bool IsStationary { get; }
}

Kernel Implementations:

Scalers

All scalers implement the IScaler interface:

public interface IScaler
{
    double[,] FitTransform(double[,] X);
    double[,] Transform(double[,] X);
    double[,] InverseTransform(double[,] X);
}

Scaler Implementations:

Requirements

• .NET 10.0 or later
• MathNet.Numerics 5.0.0 (automatically installed)
• Supports: Windows, Linux, macOS

Performance

GP.Net uses SIMD instructions via System.Numerics.Vector<T> for optimal performance:

• Kernel computations: Vectorized distance calculations
• Matrix operations: MathNet.Numerics with native BLAS/LAPACK when available
• AOT-compatible: Ready for Native AOT compilation
• Trimming-friendly: Minimal deployment size

Validation

GP.Net has been cross-validated against scikit-learn 1.5.2 to ensure correctness:

• ✅ Regression: MSE matches sklearn to 6+ decimal places
• ✅ Classification: Accuracy matches sklearn within 0.04%
• ✅ Uncertainty: Predictive std matches sklearn exactly

See CrossTest/README.md for detailed validation results.

Testing

GP.Net includes comprehensive unit tests and cross-validation tests:

# Run unit tests
cd GP.Net.Tests
dotnet test

# Run cross-validation tests
cd CrossTest/dotnet
dotnet run

See GP.Net.Tests/README.md for unit test details.

License

MIT License - see LICENSE for details.

References

1. Rasmussen, C. E., & Williams, C. K. I. (2006). Gaussian Processes for Machine Learning. MIT Press.

   • Available at: http://www.gaussianprocess.org/gpml/
   • Algorithm 2.1 (Regression), Algorithm 3.1 (Classification)

2. scikit-learn Gaussian Process module

   • Reference implementation: https://scikit-learn.org/stable/modules/gaussian_process.html
   • Source code: https://github.com/scikit-learn/scikit-learn/tree/main/sklearn/gaussian_process

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

Roadmap

Future enhancements planned:

1. Hyperparameter Optimization: Gradient-based MLE optimization (like sklearn's optimizer)
2. Multi-class Classification: One-vs-rest and one-vs-one strategies
3. Additional Kernels: Matern, RationalQuadratic, Periodic, Linear
4. Sparse GP: Inducing points for scalability (>10,000 samples)
5. GPU Acceleration: CUDA/ROCm support for large-scale problems