Realeyes.EmotionTracking 2.8.0

Realeyes.EmotionTracking

.NET wrapper for the Native Emotions Library, providing emotion tracking capabilities for video frames.

Features

• Async API for emotion tracking using TrackAsync
• Flattened emotion structure with individual properties (Happy, Confusion, etc.)
• Safe P/Invoke wrapper around the native C API
• Support for multiple image formats (RGB, BGR, Grayscale, RGBA, BGRA)
• Thread-safe async operations using TaskCompletionSource pattern

Usage

using Realeyes.EmotionTracking;

// Create tracker with model file
using var tracker = new EmotionTracker("path/to/model.realZ");

// Prepare image data
var imageHeader = new ImageHeader
{
    Data = imageBytes,
    Width = 640,
    Height = 480,
    Stride = 640 * 3,
    Format = ImageFormat.RGB
};

// Track emotions asynchronously
var result = await tracker.TrackAsync(imageHeader, timestamp);

// Access emotions
if (result.Emotions.Happy is { } happy)
{
    Console.WriteLine($"Happy: {happy.Probability:P2}, Active: {happy.IsActive}");
}

if (result.Emotions.Confusion is { } confusion)
{
    Console.WriteLine($"Confusion: {confusion.Probability:P2}, Active: {confusion.IsActive}");
}

// Access landmarks if available
if (result.LandmarkData != null)
{
    Console.WriteLine($"Landmarks 2D: {result.LandmarkData.Landmarks2d.Length} points");
    Console.WriteLine($"Landmarks 3D: {result.LandmarkData.Landmarks3d.Length} points");
}

Emotion Types

The library tracks the following emotions:

• Confusion (intensity-based)
• Contempt (intensity-based)
• Disgust (intensity-based)
• Fear (intensity-based)
• Happy (intensity-based)
• Empathy (intensity-based, also known as "sad")
• Surprise (intensity-based)
• Attention (intensity-based)
• Presence (binary)
• EyesOnScreen (binary, also known as "eyes_on")

Each emotion provides:

• Probability: Detection confidence (0.0 - 1.0)
• IsActive: Whether the emotion is currently active
• IsDetectionSuccessful: Whether detection was successful for this emotion

API Reference

EmotionTracker

Main class for tracking emotions.

Constructor

public EmotionTracker(string modelPath, int concurrentCalculations = 0, bool enableFaceTracking = true)

Methods

// Track emotions in a frame
public Task<TrackingResult> TrackAsync(ImageHeader imageHeader, TimeSpan timestamp);

// Get list of tracked emotion names
public static string[] GetEmotionNames();

// Get SDK version information
public static Version GetSdkVersion();
public static string GetSdkVersionString();

Properties

public string ModelName { get; }
public int ConcurrentCalculations { get; }
public bool IsFaceTrackingEnabled { get; }

Types

ImageHeader

public record struct ImageHeader
{
    public required byte[] Data { get; init; }
    public required int Width { get; init; }
    public required int Height { get; init; }
    public required int Stride { get; init; }
    public required ImageFormat Format { get; init; }
}

TrackingResult

public record TrackingResult(LandmarkData? LandmarkData, Emotions Emotions);

Emotions

public class Emotions
{
    public EmotionData? Confusion { get; init; }
    public EmotionData? Contempt { get; init; }
    public EmotionData? Disgust { get; init; }
    public EmotionData? Fear { get; init; }
    public EmotionData? Happy { get; init; }
    public EmotionData? Empathy { get; init; }
    public EmotionData? Surprise { get; init; }
    public EmotionData? Attention { get; init; }
    public EmotionData? Presence { get; init; }
    public EmotionData? EyesOnScreen { get; init; }
}

License

See license.md for third-party licenses.