SmartPipe.Memory 0.2.0

SmartPipe.Memory

Embedded graph memory layer for the SmartPipe ecosystem.

Overview

SmartPipe.Memory is an embedded graph memory layer with predictive analytics, designed for ETL pipelines and AI agents. It provides a type-safe Fluent API for graph queries, in-memory traversal engine, and SQLite-backed persistence.

Packages

Installation

dotnet add package SmartPipe.Memory
dotnet add package SmartPipe.Memory.Extensions
dotnet add package SmartPipe.Memory.Health

Quick Start

var store = StoreFactory.CreateInMemory();

await store.UpsertNodeAsync(new Node { Id = "f1", Type = "File", Label = "doc.pdf" });
await store.UpsertNodeAsync(new Node { Id = "f2", Type = "File", Label = "copy.pdf" });
await store.UpsertEdgeAsync(new Edge { FromNodeId = "f1", ToNodeId = "f2", Type = EdgeType.DuplicateOf });

var query = MemoryQueryBuilder.Create(store, new NodeCache());

await foreach (var r in query.ShortestPath("f1", "f2", "DuplicateOf").ExecuteAsync())
    Console.WriteLine(string.Join(" -> ", r.Path));

Key Features

• Type-safe Fluent API — no text-based query language
• In-memory traversal engine (BFS, Dijkstra, Leiden clustering)
• Graph connectivity analysis — topological sort, cycle detection, SCC, WCC
• Predictive analytics — HealthVector, BottleneckPredictor, MemoryDecayPolicy
• Time-travel queries — AsOf and Between with full bitemporal support
• AND/OR composable filters with WhereNode, MinWeight, MinConfidence
• Centrality algorithms — PageRank, BetweennessCentrality, DegreeCentrality
• Graph reordering for cache locality
• Auto-classification of nodes and edges
• Immutable CSR/CSC snapshots — lock-free read-heavy graph algorithms
• COW (Copy-on-Write) edge arrays for thread-safe graph mutations
• Enhanced observability — CSR build/traversal metrics, PageRank convergence tracing
• Property-based tests (FsCheck) and concurrent stress tests
• SQLite consistency guarantees — atomic transactions, restart parity
• SQLite WAL persistence with automatic recovery and checkpointing
• False sharing prevention in metrics
• OpenTelemetry metrics and tracing — ActivitySource, Meter, EventCounters
• Dependency injection — AddSmartPipeMemory() / AddSmartPipeMemorySqlite()
• Zero external servers — embedded in-process library
• FastBitArray and array-based BFS queue — significant memory and performance improvements for graph traversals
• CSR-based traversal and pathfinding — ~5x faster BFS than dictionary-based approach
• 227 tests, 0 failures — 83.5% line coverage

Documentation

Dependencies

• SmartPipe.Core >= 1.0.6
• Microsoft.Data.Sqlite >= 9.0.0

License

MIT