NetComposer.Visualization 0.1.0

RTHDDNetComposer

A high-performance neural network compositor with dual-language architecture (C# for graph construction, C++ for runtime execution) using memory-mapped files for zero-copy communication.

Status: Phase 3 MVP + Phase 3D Partial + Phase 4A Complete ✅ (622 C# + 114 C++ unit + 17 C++ integration tests passing)

Overview

RTHDDNetComposer implements a unique dual-graph architecture where forward computation and gradient flow exist as separate, parallel computational graphs. This design enables:

• Multiple gradient sources: Rewards, supervised loss, intrinsic curiosity
• Complex gradient routing: Independent modification of gradient flow without changing forward computation
• Zero-copy execution: Memory-mapped files eliminate serialization overhead
• Real-time performance: < 16.67ms per timestep for 60 FPS operation

Three-Stage Model Lifecycle

1. Model Design (C#): Pure topology descriptions - graph structure, node specifications, hyperparameters
2. Model Package (C#): Complete model = topology + trained weights
3. Unfolded Memory Map (C++): Runtime-ready form with all buffers pre-allocated

Architecture

C# Layer (Graph Builder)

• Declarative graph construction using fluent API
• Memory analysis and layout optimization
• "Unfolding" process: transforms graphs into memory-mapped runtime representation
• LINQ-based port operations for flexible connection patterns

C++ Layer (Execution Runtime)

• Opens memory-mapped files created by C#
• Zero-copy direct buffer access
• Lock-free circular buffers for I/O streaming
• Thread pool with CPU affinity control
• Gradient signal routing and processing

Numeric Domains

Support for multiple numeric domains:

• Real: Standard floating-point
• Complex: Frequency domain processing
• Quaternion: 3D rotations and spatial relationships
• Octonion: Higher-dimensional algebraic structures

Current Phase Status

Completed Phases

• ✅ Phase 0 (Foundation): Core type system (C# + C++)
• ✅ Phase 1 (Graph Construction): Complete builder API with InstantNode/TemporalNode architecture
• ✅ Phase 1A (Constrained Types): Extended type system with unit-constrained domains
• ✅ Phase 1.5 (Visualization + Node Identity): Graph visualization, diagnostic output, GUID-based node identity
• ✅ Phase 2A-D (Memory Mapping): .mmap file generation with topology serialization
• ✅ Phase 3A-C (C++ Execution Engine): Memory-mapped graph loading, buffer access, execution engine
• ✅ Phase 3D (Partial): IntegratorNode temporal execution (LTCNode deferred)
• ✅ Phase 4A (Gradient Graph Construction): Complete C# gradient graph API

Phase 3D Achievements

IntegratorNode - First temporal/stateful node in C++ runtime:

• Exponential moving average: s(t+1) = (1-α)·s(t) + α·gain·input
• State buffer allocation and initialization
• Fixed critical MemoryAnalyzer bug (state buffer naming)
• 11 unit tests + 4 integration tests
• Mathematical correctness verified (convergence, tracking, state persistence)

Deferred to Future: LTCNode, conversion nodes (C++ execution), multi-threading, performance validation

Phase 4A Achievements

Gradient Graph Construction API - Dual-graph architecture foundation:

• 4 gradient source types (RewardSignal, SupervisedLoss, IntrinsicCuriosity, RegularizationLoss)
• 5 gradient transforms (Scale, Clip, Normalize, Gate, Accumulate)
• 4 gradient routing modes (Direct, Broadcast, Accumulate, Conditional)
• GradientGraphBuilder fluent API for declarative gradient graph construction
• Complete validation (DAG enforcement, cycle detection, orphan detection)
• Forward-gradient mapping via node GUIDs
• 106 new tests (all passing)

Next Phase

Phase 4B (Gradient Memory Mapping): Extend .mmap format to serialize gradient graphs for C++ runtime

Prerequisites

Required

• .NET SDK 8.0 or later
• CMake 3.18 or later
• C++20 compatible compiler (GCC 11+, Clang 13+, or MSVC 2022+)
• Git

Optional

• CUDA Toolkit 12.x (for GPU acceleration)
• Ninja (faster C++ builds)
• VS Code (recommended IDE)

Quick Start

1. Clone the Repository

git clone https://github.com/rjogilvie/RTHDDNetComposer.git
cd RTHDDNetComposer

2. Check Environment

# Run environment setup script
./scripts/setup_env.sh

# Check for CUDA (optional)
./scripts/detect_cuda.sh

3. Build C# Projects

# Restore dependencies
dotnet restore RTHDDNetComposer.sln

# Build all C# projects
dotnet build RTHDDNetComposer.sln --configuration Release

# Run tests
dotnet test RTHDDNetComposer.sln

4. Build C++ Projects

# Configure (without CUDA)
mkdir -p build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Release -DENABLE_CUDA=OFF

# Build
cmake --build . -j$(nproc)

# Run tests
ctest --output-on-failure

cd ..

5. Build with CUDA (Optional)

# Configure with CUDA
mkdir -p build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Release -DENABLE_CUDA=ON

# Build
cmake --build . -j$(nproc)

cd ..

Project Structure

RTHDDNetComposer/
├── src/
│   ├── NetComposer.Core/         # C# core data structures
│   ├── NetComposer.Builder/      # C# graph building API
│   ├── NetComposer.MemoryMap/    # C# memory mapping
│   ├── NetComposer.Simulation/   # C# simulation interfaces
│   └── cpp/                      # C++ runtime
│       ├── core/                 # Memory-mapped graph, buffers
│       ├── execution/            # Execution engine, threading
│       ├── gradient/             # Gradient processing
│       ├── simulation/           # Simulation adapters
│       └── io/                   # I/O controller
├── include/                      # C++ public headers
├── tests/
│   ├── NetComposer.Tests/        # C# NUnit tests
│   └── cpp/                      # C++ Google Test tests
├── docs/                         # Architecture documentation
├── scripts/                      # Build and setup scripts
├── .github/workflows/            # CI/CD pipelines
└── CLAUDE.md                     # AI assistant guidance

Development Workflow

Using VS Code

1. Open the project in VS Code
2. Install recommended extensions (prompted automatically)
3. Build tasks available via Ctrl+Shift+B / Cmd+Shift+B
4. Debug configurations in Run and Debug panel

Build Tasks (VS Code)

• Build C# (Debug/Release): Compile C# solution
• Build C++ (Debug/Release): Configure and compile C++ projects
• Test C#: Run NUnit tests
• Test C++: Run Google Test suite
• Build All: Build both C# and C++ projects

Command Line

# C# development
dotnet build                      # Build
dotnet test                       # Run tests
dotnet run --project <project>    # Run specific project

# C++ development
cmake -B build -S .               # Configure
cmake --build build               # Build
cd build && ctest                 # Test

# Clean builds
rm -rf build                      # Clean C++
dotnet clean                      # Clean C#

Testing

C# Tests (NUnit)

# Run all tests
dotnet test

# Run with detailed output
dotnet test --logger "console;verbosity=detailed"

# Run specific test
dotnet test --filter "TestName~<pattern>"

C++ Tests (Google Test)

cd build

# Run all tests
ctest --output-on-failure

# Run specific test
./tests/cpp/netcomposer_tests --gtest_filter="CoreTests.*"

Benchmarking

RTHDDNetComposer includes a comprehensive benchmark suite for performance validation and regression detection.

Running All Benchmarks

# Run complete benchmark suite (C++ + C#)
./scripts/run_benchmarks.sh

Results Location:

• C++ benchmarks: benchmarks/results/cpp_benchmarks.json
• C# benchmarks: benchmarks/results/csharp/
• Baseline report: benchmarks/baseline_report.md

Performance Baseline

C++ Runtime (24 benchmarks):

• Simple timestep: 84 ns (11.9M timesteps/sec)
• 10-layer network: 1.34 μs (749K timesteps/sec)
• Buffer registry lookup: 13.9 ns (72.1M lookups/sec)
• Gradient computation (BPTT): 329 ns (3.04M timesteps/sec)
• Scalability: O(n) complexity validated up to 10,000 nodes

Key Findings:

• ✅ All benchmarks meet 60 FPS target (< 16.67 ms per timestep)
• ✅ Up to 4,000-node graphs supported at 60 FPS
• ✅ 10,000-node graphs run at 25 FPS (38.9 ms per timestep)

For detailed results, see benchmarks/baseline_report.md.

Running Individual Benchmark Suites

C++ Only:

cmake -B build -DCMAKE_BUILD_TYPE=Release -DBUILD_BENCHMARKS=ON
cmake --build build --target netcomposer_benchmarks -j$(nproc)
cd build/benchmarks/cpp && ./netcomposer_benchmarks

C# Only:

cd benchmarks/NetComposer.Benchmarks
dotnet run -c Release

See benchmarks/README.md for complete benchmarking guide.

Documentation

• CLAUDE.md - Guidance for AI assistants working with this codebase
• docs/ml-compositor-api-spec.md - Complete C# API specification
• docs/memory-mapping-architecture.md - Memory layout and lifecycle
• docs/memory-mapped-implementation.md - C++ runtime implementation
• docs/compositor-data-structures.md - Core data structures
• docs/simulation-io-architecture.md - Simulation adapter interfaces
• docs/testing-plan-document.md - Testing strategy

Performance Targets

• Average timestep: < 16.67ms (60 FPS)
• Worst-case latency: < 20ms
• Jitter: < 2ms standard deviation
• Cache hit rate: > 90%
• Zero runtime allocations in execution loop

Contributing

Code Style

• C#: Follow .NET conventions (enforced by EditorConfig)
• C++: C++20 standard, follow project style (enforced by .editorconfig)
• Commits: Conventional commits format

Pull Requests

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Make your changes with clear commit messages
4. Run tests locally
5. Push to your fork
6. Open a Pull Request

Continuous Integration

GitHub Actions workflows automatically:

• Build C# and C++ projects
• Run test suites
• Check documentation
• Validate code style

See .github/workflows/ for CI configuration.

License

[Specify your license here]

Authors

• Robert Ogilvie - @rjogilvie

Acknowledgments

See docs/ for detailed architecture documentation and design decisions.

Troubleshooting

CUDA Not Found

Run the CUDA detection script:

./scripts/detect_cuda.sh

Follow the instructions to install CUDA or build without CUDA support using -DENABLE_CUDA=OFF.

CMake Configuration Issues

# Clean CMake cache
rm -rf build
mkdir build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Debug

.NET SDK Version Mismatch

Check global.json for pinned SDK version:

cat global.json
dotnet --list-sdks

Install the matching SDK version or update global.json.

Support

• Issues: GitHub Issues
• Documentation: docs/
• CLAUDE.md: Guidance for AI assistants