UniversalQueryBuilder.Expressions 10.0.13-beta

QueryBuilder.Expressions

Expression compilation and caching layer for the Universal Query Builder

QueryBuilder.Expressions provides high-performance conversion of FilterDefinition objects into compiled LINQ expression trees with aggressive caching for 10-40x performance improvements. It serves as the critical bridge between declarative query definitions and executable in-memory predicates.

// Build expression from filter
var filter = FilterDefinitionExtensions.And(
    FilterDefinitionExtensions.Equal("IsActive", true),
    FilterDefinitionExtensions.GreaterThan("Age", 18)
);

var builder = serviceProvider.GetRequiredService<IExpressionBuilder<User>>();
var expression = await builder.BuildAsync(filter);
// Result: Expression<Func<User, bool>>
// Lambda: user => user.IsActive == true && user.Age > 18

// Apply to queryable
var users = dbContext.Users.Where(expression).ToList();

Performance Benefits:

• ✅ 10-40x faster compilation via FastExpressionCompiler (vs standard expression.Compile())
• ✅ 10-40x faster execution via two-level caching (vs recompilation)
• ✅ >95% cache hit rate in production workloads
• ✅ <0.1ms cache lookup for L1 hits

---

Table of Contents

• Overview
• Core Concepts
• Installation
• Expression Building
• Caching System
• Operator Support
• Type System
• Navigation Properties
• Performance
• Usage Examples
• Integration
• Configuration
• Optimization
• Testing
• Contributing

---

Overview

QueryBuilder.Expressions provides:

✅ Expression Compilation - FilterDefinition → Expression<Func<T, bool>> ✅ Two-Level Caching - L1 (ConcurrentDictionary) + L2 (MemoryCache) ✅ FastExpressionCompiler - 10-40x faster than standard compilation ✅ 18+ Operator Support - Comparison, string, collection, null, fuzzy ✅ Type Safety - Compile-time type checking via generics ✅ Navigation Properties - Dot notation with null-safe conditionals ✅ Collection Support - Automatic .Any() semantics for IEnumerable<T> ✅ Hierarchical Projections - Support for nested object and collection projections ✅ Nested Relation Options - Filtering, ordering, and pagination on nested collections ✅ Expression Optimization - Constant folding, boolean simplification

Role in the Universal Query Builder System

FilterDefinition
    ↓
┌─────────────────────────────────────────┐
│   QueryBuilder.Expressions              │
│   ├─ IExpressionBuilder<T>              │
│   ├─ INestedProjectionBuilder           │
│   ├─ JsonToExpressionBuilder            │
│   ├─ PropertyAccessBuilder              │
│   └─ CachedExpressionCompiler           │
└─────────────────────────────────────────┘
    ↓
Expression<Func<T, bool>> (LINQ expression tree)
    ↓
FastExpressionCompiler (IL generation)
    ↓
Func<T, bool> (Compiled predicate)
    ↓
LINQ .Where() / .Any() / .Count()
    ↓
Filtered Results

Integration Points:

• QueryBuilder.Core - Consumes FilterDefinition models
• QueryBuilder.InMemory - Uses compiled expressions for in-memory filtering
• SpecificationExpressionBuilder - Compiles Specification objects to expressions
• FastExpressionCompiler - High-performance IL generation library

---

Core Concepts

1. Expression Trees

What are Expression Trees?

Expression trees are data structures that represent code as a tree of nodes. They enable:

• Dynamic code generation at runtime
• Query translation (LINQ to SQL, LINQ to Entities)
• Compilation to executable delegates

Example:

// C# lambda
Func<User, bool> lambda = user => user.Age > 18;

// Equivalent expression tree
var param = Expression.Parameter(typeof(User), "user");
var property = Expression.Property(param, "Age");
var constant = Expression.Constant(18);
var comparison = Expression.GreaterThan(property, constant);
var expression = Expression.Lambda<Func<User, bool>>(comparison, param);

// Compile to delegate
var compiled = expression.Compile();

2. FilterDefinition → Expression Conversion

Conversion Pipeline:

FilterDefinition { Field: "Age", Operator: GreaterThan, Value: 18 }
    ↓
Cache Check (structural hash)
    ↓ (miss)
Expression Building
    ├─ BuildPropertyAccess("Age") → Expression.Property
    ├─ BuildConstant(18) → Expression.Constant
    └─ BuildGreaterThan() → Expression.GreaterThan
    ↓
Expression<Func<T, bool>>
    ↓
Optimization (constant folding, boolean simplification)
    ↓
Cache Store (L1 + L2)
    ↓
FastExpressionCompiler (IL generation)
    ↓
Func<T, bool> (Executable delegate)

3. Two-Level Caching Architecture

L1 Cache (Fast):

• Type: ConcurrentDictionary<StructuralHash, CompiledExpression>
• Max size: 1,000 entries
• Access time: <0.1ms
• Eviction: LRU when capacity exceeded (removes 25% oldest)
• Thread-safe: No locking required

L2 Cache (Durable):

• Type: IMemoryCache (Microsoft.Extensions.Caching.Memory)
• Expiration: 120 minutes (configurable)
• Size limit: 100MB default
• Eviction: Automatic on memory pressure
• Promotion: L2 hits promoted to L1

Cache Key Strategy:

// Structural hash based on expression tree, not object identity
var hash = HashVisitor.ComputeHash(expression);
var key = $"expr_{hash:X16}_{typeof(T)}_{typeof(bool)}";

// Same filter structure = same cache key
var filter1 = new FilterDefinition { Field = "Age", Operator = GreaterThan, Value = 18 };
var filter2 = new FilterDefinition { Field = "Age", Operator = GreaterThan, Value = 18 };
// hash(filter1) == hash(filter2) → cache hit!

4. FastExpressionCompiler

Why FastExpressionCompiler?

Standard expression.Compile():

• Interprets expression tree at runtime
• Slow for complex expressions (5-20ms)
• Higher memory overhead

FastExpressionCompiler:

• Generates optimized IL directly
• 10-40x faster compilation
• Smaller memory footprint
• Falls back to standard compiler on failure

Example:

var expression = /* ... */;

// Standard compilation (slow)
var standardCompiled = expression.Compile();  // ~10ms

// Fast compilation (optimized)
var fastCompiled = expression.CompileFast<Func<User, bool>>();  // ~0.5ms

---

Installation

NuGet Package (when published):

dotnet add package UniversalQueryBuilder.Expressions

Dependencies:

• .NET 10 or later
• QueryBuilder.Core - Core models and abstractions
• FastExpressionCompiler - High-performance IL generation
• Microsoft.Extensions.Caching.Memory - L2 caching
• Microsoft.Extensions.ObjectPool - Object pooling

Service Registration:

using QueryBuilder.Expressions.Extensions;

services.AddExpressionCompilation(options =>
{
    options.EnableCaching = true;
    options.L1CacheCapacity = 1000;
    options.L2CacheExpiration = TimeSpan.FromMinutes(120);
    options.EnableOptimization = true;
    options.MaxExpressionDepth = 100;
});

---

Expression Building

IExpressionBuilder<T>

Main Interface:

public interface IExpressionBuilder<T> where T : class
{
    /// <summary>Build expression from single filter</summary>
    Task<Expression<Func<T, bool>>> BuildAsync(
        FilterDefinition filter,
        CancellationToken cancellationToken = default);

    /// <summary>Build expression from multiple filters with logical operator</summary>
    Task<Expression<Func<T, bool>>> BuildAsync(
        IEnumerable<FilterDefinition> filters,
        LogicalOperator logicalOperator,
        CancellationToken cancellationToken = default);

    /// <summary>Validate filter before building</summary>
    ValidationResult Validate(FilterDefinition filter);
}

INestedProjectionBuilder

Main Interface:

public interface INestedProjectionBuilder
{
    /// <summary>
    /// Builds a hierarchical LINQ projection expression.
    /// Supports nested objects and collections with relation options.
    /// </summary>
    /// <typeparam name="TSource">The root entity type</typeparam>
    /// <param name="selection">The selection dictionary</param>
    /// <param name="dataSource">Data source metadata</param>
    /// <param name="target">Target execution environment (controls null-handling)</param>
    /// <param name="cancellationToken">Cancellation token</param>
    /// <returns>A projection expression: entity => Dictionary<string, object></returns>
    Task<Expression<Func<TSource, Dictionary<string, object>>>> BuildProjectionAsync<TSource>(
        SelectionDictionary selection,
        DataSourceDefinition dataSource,
        ProjectionTarget target = ProjectionTarget.InMemory,
        CancellationToken cancellationToken = default)
        where TSource : class;
}

Key Features:

• Recursive Navigation: Supports deeply nested object paths.
• Nested Collection Options: Applies Where, OrderBy, Limit, and Offset directly to nested collections.
• Schema-Aware Nested Access: Relation-level Where and OrderBy resolve against nested ColumnDefinition metadata, so aliases and non-CLR-backed fields (for example SQL Server JSON scalar fields) work inside collection projections.
• Performance: Uses IPropertyAccessBuilder for optimized member access and RuntimeTypeBuilder for efficient projection steps.
• Validated Shapes: Relation options (Where/OrderBy/Limit/Offset) apply only to collections, and a nested sub-selection requires a navigation with nested properties; applying relation options to a non-collection, or a sub-selection to a scalar, is rejected with a ValidationException. An excluded leaf (false) is omitted from the projection entirely.

IRuntimeTypeBuilder

Main Interface:

public interface IRuntimeTypeBuilder
{
    /// <summary>
    /// Gets or creates a runtime type matching the specified selection structure.
    /// </summary>
    Type GetProjectionType(Type parentType, SelectionDictionary selection);
}

Purpose: Generates CLR types at runtime that match the structure of a selection dictionary. These types are used as intermediate projection targets for EF Core queries, ensuring efficient SQL generation and avoiding IEnumerable materialization issues.

CollectionAggregateMethodResolver

Purpose: Static utility that caches Enumerable aggregate MethodInfo objects (Count, Sum, Average, Min, Max) and provides type-safe builder methods for constructing aggregate MethodCallExpression nodes. Used by JsonToExpressionBuilder (filter aggregates), AggregationExpressionBuilder (GROUP BY), and LinqQueryBuilder (aggregate ordering).

Methods:

Method	Description
BuildCountCall(collection, elementType)	collection.Count()
BuildSumCall(collection, selector, elementType)	collection.Sum(selector) — auto-selects decimal/double/int/long overload
BuildAverageCall(collection, selector, elementType)	collection.Average(selector) — auto-selects overload
BuildMinCall(collection, selector, elementType, resultType)	collection.Min(selector)
BuildMaxCall(collection, selector, elementType, resultType)	collection.Max(selector)
BuildWhereCall(collection, predicate, elementType)	collection.Where(predicate) — scopes a collection before aggregating
ConvertToDecimalForAggregate(expr)	Converts numeric expressions to decimal for Sum/Average, preserving nullability

AggregationExpressionBuilder

Purpose: Builds the GROUP BY projection for a grouped query. It dispatches on the GroupedSelectTerm results returned by GroupedSelectClassifier.Classify(select, source, groupByField) (in QueryBuilder.SchemaRegistry) — one term per top-level select entry — and emits the matching projection node for each:

Term	Projected expression
GroupKey(Path)	The grouping key (group.Key)
Count(Path)	COUNT(*) over the group
AggregateColumn(Path, Aggregates)	The requested aggregate(s) (sum/avg/average/min/max) over the column, built via CollectionAggregateMethodResolver
Invalid(Path, Reason)	Throws — the classifier already rejected the shape

Because the classifier is the single source of truth shared with the endpoint validator, what the builder will project and what the endpoint accepts cannot drift.

RuntimeFilterPredicateBuilder

Purpose: Static utility that builds a filter predicate (LambdaExpression) for an element type known only at runtime, by resolving the generic IExpressionBuilder<T> from a service provider. It caches the per-type dispatch delegate so the reflection cost is paid once per element type.

Task<LambdaExpression> BuildAsync(
    IServiceProvider serviceProvider,
    Type elementType,
    FilterDefinition filter,
    DataSourceDefinition? sourceMetadata,
    CancellationToken cancellationToken);

Shared by NestedProjectionBuilder (collection-relation where) and LinqQueryBuilder (aggregate-ordering filter), so both apply element-level filters identically. elementType must be a reference type; the caller is responsible for rejecting primitive collections before calling.

IFieldAccessExpressionBuilder

Main Interface:

public interface IFieldAccessExpressionBuilder
{
    /// <summary>Builds an expression that accesses a field value from the given root expression.</summary>
    Expression BuildFieldAccess(Expression rootExpression, ColumnDefinition column);
}

Purpose: Translates a ColumnDefinition's Accessor into a LINQ expression that reads the column's value off an entity instance. Downstream services (JsonToExpressionBuilder, NestedProjectionBuilder, AggregationExpressionBuilder, LinqQueryBuilder) call this instead of Expression.Property(...) directly so non-CLR-backed columns work transparently in filters, projections, ordering, and aggregation.

DefaultFieldAccessExpressionBuilder is the singleton registered by services.TryAddSingleton<IFieldAccessExpressionBuilder, DefaultFieldAccessExpressionBuilder>(). Provider-specific projects (e.g., QueryBuilder.EntityFramework) replace the registration to handle additional accessor variants.

ColumnAccessor is a closed discriminated union (an abstract record with a private constructor) of three sealed variants — the default builder handles Property and Coalesce; JsonScalar requires a provider-specific replacement (the EF builder).

Variant	Default builder emits
Property(PropertyInfo)	Expression.Property(root, propertyInfo)
JsonScalar(BackingJsonColumn, JsonPath)	Throws InvalidOperationException — requires the EF builder
Coalesce(Sources, Fallback?)	Expression.Coalesce folded left-associatively over child accessors; when LiteralFallback is non-null, it is appended as the terminal Expression.Constant(value, clrType)

Coalesce is associative, so the folding direction does not change the result.

The fold itself lives in CoalesceChainBuilder.Build(sources, fallback, recurse) — a public static helper in QueryBuilder.Expressions.Services that both the default and EF field-access builders call. The recurse callback is the provider-specific translation for a single ColumnAccessor.Property; the helper handles the Aggregate(Expression.Coalesce) fold and the optional literal terminal. Calling Build with an empty sources list and a null fallback throws ArgumentException — at least one expression is required to fold.

JsonToExpressionBuilder<T>

Main Implementation:

public class JsonToExpressionBuilder<T> : IExpressionBuilder<T> where T : class
{
    private readonly IExpressionCompiler _compiler;
    private readonly IExpressionOptimizer? _optimizer;
    private readonly IPropertyAccessBuilder _propertyAccessBuilder;
    private readonly ILogger<JsonToExpressionBuilder<T>> _logger;

    public async Task<Expression<Func<T, bool>>> BuildAsync(FilterDefinition filter, CancellationToken ct = default)
    {
        // 1. Validate
        var validation = Validate(filter);
        if (!validation.IsValid)
            throw new QueryValidationException(validation.Errors);

        // 2. Check cache
        var cacheKey = ComputeCacheKey(filter);
        if (_compiler.TryGetCached(cacheKey, out var cached))
            return cached;

        // 3. Build expression recursively
        var parameter = Expression.Parameter(typeof(T), "entity");
        var body = BuildExpressionRecursive(filter, parameter);
        var expression = Expression.Lambda<Func<T, bool>>(body, parameter);

        // 4. Optimize
        if (_optimizer != null)
            expression = _optimizer.Optimize(expression);

        // 5. Cache
        await _compiler.CompileAndCacheAsync(cacheKey, expression, ct);

        return expression;
    }
}

Building Process

Recursive Building:

private Expression BuildExpressionRecursive(FilterDefinition filter, ParameterExpression parameter)
{
    // Route based on operator type
    if (filter.LogicalOperator != null)
    {
        // Logical operators: AND, OR, NOT
        return BuildLogicalExpression(filter, parameter);
    }
    else if (filter.Operator != null)
    {
        // Filter operators: eq, gt, like, in, etc.
        return BuildFilterExpression(filter, parameter);
    }
    else
    {
        throw new InvalidOperationException("Filter must have either LogicalOperator or Operator");
    }
}

Logical Expression Building:

private Expression BuildLogicalExpression(FilterDefinition filter, ParameterExpression parameter)
{
    switch (filter.LogicalOperator)
    {
        case LogicalOperator.And:
            // Build all expressions and combine with AndAlso
            var andExpressions = filter.Expressions!
                .Select(e => BuildExpressionRecursive(e, parameter))
                .ToList();
            return andExpressions.Aggregate(Expression.AndAlso);

        case LogicalOperator.Or:
            // Build all expressions and combine with OrElse
            var orExpressions = filter.Expressions!
                .Select(e => BuildExpressionRecursive(e, parameter))
                .ToList();
            return orExpressions.Aggregate(Expression.OrElse);

        case LogicalOperator.Not:
            // Build single expression and negate
            var innerExpression = BuildExpressionRecursive(filter.Expressions![0], parameter);
            return Expression.Not(innerExpression);

        default:
            throw new NotSupportedException($"Logical operator {filter.LogicalOperator} not supported");
    }
}

Filter Expression Building:

private Expression BuildFilterExpression(FilterDefinition filter, ParameterExpression parameter)
{
    // 1. Build property access
    var property = _propertyAccessBuilder.BuildPropertyAccess(parameter, filter.Field!);

    // 2. Build constant with type coercion
    var constant = BuildConstant(filter.Value, property.Type);

    // 3. Build operator-specific expression
    return filter.Operator switch
    {
        FilterOperator.Equal => Expression.Equal(property, constant),
        FilterOperator.NotEqual => Expression.NotEqual(property, constant),
        FilterOperator.GreaterThan => Expression.GreaterThan(property, constant),
        FilterOperator.GreaterThanOrEqual => Expression.GreaterThanOrEqual(property, constant),
        FilterOperator.LessThan => Expression.LessThan(property, constant),
        FilterOperator.LessThanOrEqual => Expression.LessThanOrEqual(property, constant),
        FilterOperator.Contains => BuildContainsExpression(property, constant),
        FilterOperator.StartsWith => BuildStartsWithExpression(property, constant),
        FilterOperator.EndsWith => BuildEndsWithExpression(property, constant),
        FilterOperator.In => BuildInExpression(property, filter.Values!),
        FilterOperator.Between => BuildBetweenExpression(property, filter.Range!),
        FilterOperator.IsNull => Expression.Equal(property, Expression.Constant(null)),
        FilterOperator.IsNotNull => Expression.NotEqual(property, Expression.Constant(null)),
        // ... additional operators
        _ => throw new NotSupportedException($"Operator {filter.Operator} not supported")
    };
}

---

Caching System

Cache Architecture

L1 Cache (ConcurrentDictionary):

private readonly ConcurrentDictionary<string, CompiledExpression> _l1Cache = new();
private const int MaxL1Capacity = 1000;

public bool TryGetCached<TInput, TResult>(string key, out Expression<Func<TInput, TResult>>? expression)
{
    if (_l1Cache.TryGetValue(key, out var cached))
    {
        _statistics.RecordHit();
        expression = (Expression<Func<TInput, TResult>>)cached.Expression;
        return true;
    }

    // Try L2 cache
    if (_l2Cache.TryGetValue(key, out cached))
    {
        _statistics.RecordHit();
        // Promote to L1
        _l1Cache.TryAdd(key, cached);
        expression = (Expression<Func<TInput, TResult>>)cached.Expression;
        return true;
    }

    _statistics.RecordMiss();
    expression = null;
    return false;
}

L2 Cache (MemoryCache):

private readonly IMemoryCache _l2Cache;

public async Task CacheAsync<TInput, TResult>(
    string key,
    Expression<Func<TInput, TResult>> expression,
    CancellationToken ct = default)
{
    var compiled = new CompiledExpression
    {
        Expression = expression,
        CompiledFunc = expression.CompileFast<Func<TInput, TResult>>(),
        CachedAt = DateTimeOffset.UtcNow
    };

    // Store in L1
    _l1Cache.TryAdd(key, compiled);

    // Store in L2 with expiration
    var options = new MemoryCacheEntryOptions
    {
        AbsoluteExpirationRelativeToNow = TimeSpan.FromMinutes(120),
        Size = EstimateSize(expression)
    };
    _l2Cache.Set(key, compiled, options);

    // Evict oldest if L1 full
    if (_l1Cache.Count > MaxL1Capacity)
    {
        EvictOldestEntries();
    }
}

Cache Key Generation

Structural Hashing:

public static StructuralHash ComputeHash(Expression expression)
{
    var visitor = new HashVisitor();
    visitor.Visit(expression);
    return visitor.GetHash();
}

private class HashVisitor : ExpressionVisitor
{
    private readonly StringBuilder _hashBuilder = new();
    private int _hashCode = 0;

    protected override Expression VisitBinary(BinaryExpression node)
    {
        _hashBuilder.Append($"Binary:{node.NodeType}:");
        _hashCode = HashCode.Combine(_hashCode, node.NodeType);
        return base.VisitBinary(node);
    }

    protected override Expression VisitConstant(ConstantExpression node)
    {
        _hashBuilder.Append($"Const:{node.Value}:");
        _hashCode = HashCode.Combine(_hashCode, node.Value);
        return base.VisitConstant(node);
    }

    // ... other expression types
}

Cache Key Format:

expr_{hash:X16}_{InputType}_{ResultType}

Example:
expr_A3F2D8E1B4C6F9A7_User_Boolean

Cache Statistics

Metrics Tracked:

public class CacheStatistics
{
    public long HitCount { get; private set; }
    public long MissCount { get; private set; }
    public double HitRate => TotalRequests > 0 ? (double)HitCount / TotalRequests * 100 : 0;
    public long TotalRequests => HitCount + MissCount;
    public int L1EntryCount { get; set; }
    public int L2EntryCount { get; set; }
    public long EstimatedMemoryUsage { get; set; }
    public TimeSpan AverageRetrievalTime { get; set; }

    public void RecordHit() => Interlocked.Increment(ref HitCount);
    public void RecordMiss() => Interlocked.Increment(ref MissCount);
}

Monitoring:

// Log statistics periodically
_logger.LogInformation(
    "Expression cache: {HitRate:F2}% hit rate, {L1Count} L1 entries, {L2Count} L2 entries, {Memory} MB",
    stats.HitRate,
    stats.L1EntryCount,
    stats.L2EntryCount,
    stats.EstimatedMemoryUsage / 1024 / 1024);

---

Operator Support

Comparison Operators

Operator	Expression	Example
Equal	Expression.Equal(property, constant)	Age == 18
NotEqual	Expression.NotEqual(property, constant)	Status != "Inactive"
GreaterThan	Expression.GreaterThan(property, constant)	Age > 21
GreaterThanOrEqual	Expression.GreaterThanOrEqual(property, constant)	Price >= 100
LessThan	Expression.LessThan(property, constant)	Age < 65
LessThanOrEqual	Expression.LessThanOrEqual(property, constant)	Stock <= 10

String Operators

Contains:

private Expression BuildContainsExpression(Expression property, Expression value)
{
    var method = typeof(string).GetMethod(
        nameof(string.Contains),
        new[] { typeof(string), typeof(StringComparison) });

    var comparison = Expression.Constant(StringComparison.OrdinalIgnoreCase);

    return Expression.Call(property, method, value, comparison);
}

StartsWith / EndsWith:

private Expression BuildStartsWithExpression(Expression property, Expression value)
{
    var method = typeof(string).GetMethod(
        nameof(string.StartsWith),
        new[] { typeof(string), typeof(StringComparison) });

    var comparison = Expression.Constant(StringComparison.OrdinalIgnoreCase);

    return Expression.Call(property, method, value, comparison);
}

Like / ILike (Pattern Matching):

// Supports SQL-style wildcards: % (any chars), _ (single char)
private Expression BuildLikeExpression(Expression property, Expression pattern, bool caseSensitive)
{
    // Convert SQL wildcards to regex
    var regex = ConvertLikePatternToRegex(pattern.ToString());
    var options = caseSensitive ? RegexOptions.None : RegexOptions.IgnoreCase;

    var regexMethod = typeof(Regex).GetMethod(nameof(Regex.IsMatch), new[] { typeof(string), typeof(string), typeof(RegexOptions) });

    return Expression.Call(regexMethod, property, Expression.Constant(regex), Expression.Constant(options));
}

Collection Operators

IN Operator:

private Expression BuildInExpression(Expression property, IEnumerable<object> values)
{
    // Build: value == v1 || value == v2 || value == v3 || ...
    var comparisons = values
        .Select(v => Expression.Equal(property, Expression.Constant(v, property.Type)))
        .ToList();

    return comparisons.Aggregate(Expression.OrElse);
}

BETWEEN Operator:

private Expression BuildBetweenExpression(Expression property, RangeDefinition range)
{
    // Coerce range values to property type first (handles JsonElement, string, etc.)
    var coercedMin = CoerceValueToPropertyType(range.Min, property.Type);
    var coercedMax = CoerceValueToPropertyType(range.Max, property.Type);

    // Validate coerced values are IComparable and min <= max
    var min = Expression.Constant(coercedMin, property.Type);
    var max = Expression.Constant(coercedMax, property.Type);

    // Build: value >= min && value <= max (respects inclusivity settings)
    var lowerBound = Expression.GreaterThanOrEqual(property, min);
    var upperBound = Expression.LessThanOrEqual(property, max);

    return Expression.AndAlso(lowerBound, upperBound);
}

Flags Enum Operators

For [Flags] enum fields, the expression builder produces bitwise membership expressions instead of exact equality:

// Equal on [Flags] enum → (property & mask) == mask
// Example: access:Read generates:
//   (Convert(entity.Access, Int32) & 1) == 1
// This matches any record where the Read bit is set

// In on [Flags] enum → Has(a) || Has(b)
// Example: access:Read,Write generates:
//   ((Convert(entity.Access, Int32) & 1) == 1) || ((Convert(entity.Access, Int32) & 2) == 2)

// Nullable [Flags] enum adds HasValue guard:
//   entity.Access.HasValue && (Convert(entity.Access.Value, Int32) & 1) == 1

Zero-value flag members use direct equality (field == 0) instead of bitwise membership. Zero detection is value-based — any member whose numeric value is 0 triggers this behavior, regardless of its name (None, Unknown, Default, etc.). Zero is rejected in multi-value (In/NotIn) contexts with InvalidValueType. Quoted comma-composite strings (e.g., "Read,Write") are also rejected. Unsupported operators (comparison, string, range) throw QueryValidationException with OperatorNotSupported.

IsNull/IsNotNull on flags fields check CLR nullability only. For non-nullable types, they produce constant false/true respectively (vacuous but not an error). For nullable types, they check HasValue.

The FlagsEnumExpressionHelper in QueryBuilder.Core is shared by both JsonToExpressionBuilder and SpecificationExpressionBuilder to ensure consistent semantics across all query paths.

Null Operators

// IS NULL
Expression.Equal(property, Expression.Constant(null))

// IS NOT NULL
Expression.NotEqual(property, Expression.Constant(null))

Fuzzy Matching

Levenshtein Distance:

private Expression BuildFuzzyExpression(Expression property, Expression searchText, int maxDistance)
{
    // Convert edit distance to similarity threshold
    var threshold = maxDistance switch
    {
        0 => 100,  // Exact match
        1 => 80,   // Very close
        2 => 70,   // Close
        3 => 60,   // Somewhat close
        _ => Math.Max(0, 60 - (maxDistance - 3) * 10)
    };

    // Build: property != null && Fuzz.Ratio(property, searchText) >= threshold
    var notNull = Expression.NotEqual(property, Expression.Constant(null));

    var fuzzyMethod = typeof(Fuzz).GetMethod(nameof(Fuzz.Ratio), new[] { typeof(string), typeof(string) });
    var fuzzyCall = Expression.Call(fuzzyMethod, property, searchText);
    var thresholdConstant = Expression.Constant(threshold);
    var fuzzyCheck = Expression.GreaterThanOrEqual(fuzzyCall, thresholdConstant);

    return Expression.AndAlso(notNull, fuzzyCheck);
}

Logical Operators

// AND
var left = /* ... */;
var right = /* ... */;
Expression.AndAlso(left, right)

// OR
Expression.OrElse(left, right)

// NOT
Expression.Not(expression)

Null Safety (SQL 3VL)

String comparison operators and string field functions follow SQL three-valued logic when evaluated against a null field value: the row is excluded regardless of whether the filter is positive or negated.

Null-unsafe leaves that are guarded:

• Operators: Contains, StartsWith, EndsWith, Like, ILike, Regex, Fuzzy, NotIn, NotBetween
• Field functions: @lower, @upper, @trim, @length, @substring

Semantics (shorthand syntax with - for NOT):

Filter	Null field result
name:~foo (Contains)	false (excluded)
-name:~foo (Not Contains)	false (excluded)
@lower(name):foo	false (excluded)
-@lower(name):foo	false (excluded)
name:/pat/ (Regex)	false (excluded)
-name:/pat/ (Not Regex)	false (excluded)
-name:a,b (Not In)	false (excluded)
-modifiedAt:d1..d2 (Not Between)	false (excluded)

Other scalar operators (=, !=, <, >, <=, >=, In, Between, IsNull, IsNotNull) pass null values through LINQ's nullable-lifted comparisons and produce the same result SQL does.

NotIn and NotBetween also follow SQL 3VL on nullable fields: null-field rows are excluded under both positive and negated forms. The guard is type-aware — non-nullable value-type fields (e.g. int, DateTime) skip the guard since they can't be null.

The Entity Framework strategy produces the 3VL exclusion semantics for all operators via the database's handling of SQL NULL.

Implementation: JsonToExpressionBuilder threads a bool negated flag through its recursion for scalar leaves. BuildLogicalExpression applies De Morgan inline (And↔Or swap when negated); BuildNotExpression toggles the flag rather than wrapping with Expression.Not. Null-unsafe scalar leaves emit field != null && [!]predicate based on the flag. Collection-scoped predicates (e.g., items.name:"foo") evaluate the inner predicate positively and wrap the resulting .Any() with Expression.Not at the boundary when negated — existence semantics don't commute with predicate negation. Functions flagged with FieldTransformationMetadata.RequiresNullGuard drive the guard, including composed cases like @upper(@trim(name)) (resolved via nested arguments).

---

Type System

Type Coercion

Expression building does not maintain a separate conversion implementation. It delegates runtime coercion to the Core canonical coercer:

var result = TypeCoercer.TryCoerce(value, targetType);
if (!result.Success)
{
    throw new QueryValidationException(
        ValidationErrorCode.InvalidValueType,
        result.Error,
        documentationUrl: ValidationErrors.InvalidValueType.DocumentationUrl);
}

return result.Value;

This keeps expressions aligned with Core coercion policy (provided by the TypeCoercion NuGet package via TypeCoercer):

• strict failures (no silent string fallback)
• enum names only (numeric and numeric-string enum values rejected)
• JsonElement scalar coercion plus object/array typed deserialization for complex targets
• culture-invariant conversion behavior

DateOnly Range Transformation

Intelligent Date Filtering:

When filtering DateTime or DateTimeOffset properties with DateOnly values, the expression builder automatically transforms operators to match entire days:

Implementation:

private Expression BuildScalarFilterExpression(FilterDefinition filter, ParameterExpression parameter)
{
    var propertyAccess = /* build property access */;
    var filterValue = filter.Value;

    // Detect DateOnly value with DateTime/DateTimeOffset property
    var underlyingPropertyType = Nullable.GetUnderlyingType(propertyAccess.Type) ?? propertyAccess.Type;
    var isDateTimeProperty = underlyingPropertyType == typeof(DateTime)
                          || underlyingPropertyType == typeof(DateTimeOffset);

    if (filterValue is DateOnly dateOnly && isDateTimeProperty)
    {
        // Transform to UTC datetime range (entire day)
        var startOfDay = DateTime.SpecifyKind(
            dateOnly.ToDateTime(TimeOnly.MinValue),
            DateTimeKind.Utc
        );
        var endOfDay = DateTime.SpecifyKind(
            dateOnly.AddDays(1).ToDateTime(TimeOnly.MinValue),
            DateTimeKind.Utc
        );

        return filter.Operator switch
        {
            // Equality: match entire day
            FilterOperator.Equal =>
                BuildDateOnlyEqualityRange(propertyAccess, startOfDay, endOfDay),
                // Result: property >= start AND property < end

            // Inequality: exclude entire day
            FilterOperator.NotEqual =>
                BuildDateOnlyInequalityRange(propertyAccess, startOfDay, endOfDay),
                // Result: property < start OR property >= end

            // Comparison operators: boundary-based
            FilterOperator.GreaterThan =>
                BuildGreaterThanOrEqualExpression(propertyAccess, endOfDay),
                // Result: property >= end (after the day)

            FilterOperator.GreaterThanOrEqual =>
                BuildGreaterThanOrEqualExpression(propertyAccess, startOfDay),
                // Result: property >= start (from start of day)

            FilterOperator.LessThan =>
                BuildLessThanExpression(propertyAccess, startOfDay),
                // Result: property < start (before the day)

            FilterOperator.LessThanOrEqual =>
                BuildLessThanExpression(propertyAccess, endOfDay)
                // Result: property < end (through end of day)
        };
    }

    // Standard operator handling for non-DateOnly values...
}

Equality Range Builder:

private static BinaryExpression BuildDateOnlyEqualityRange(
    Expression property,
    DateTime startOfDay,
    DateTime endOfDay)
{
    // Coerce DateTime boundaries to property type (handles DateTime vs DateTimeOffset)
    var (startConstant, endConstant) = CoerceDateRangeBoundaries(
        startOfDay,
        endOfDay,
        property.Type
    );

    // Build: property >= startOfDay AND property < endOfDay
    var greaterThanOrEqualStart = Expression.GreaterThanOrEqual(property, startConstant);
    var lessThanEnd = Expression.LessThan(property, endConstant);

    return Expression.AndAlso(greaterThanOrEqualStart, lessThanEnd);
}

Type Coercion Helper:

private static (ConstantExpression Start, ConstantExpression End) CoerceDateRangeBoundaries(
    DateTime startOfDay,
    DateTime endOfDay,
    Type propertyType)
{
    // CoerceValueToPropertyType handles nullable types internally
    var coercedStart = CoerceValueToPropertyType(startOfDay, propertyType);
    var coercedEnd = CoerceValueToPropertyType(endOfDay, propertyType);

    return (
        Expression.Constant(coercedStart, propertyType),
        Expression.Constant(coercedEnd, propertyType)
    );
}

Bidirectional Type Conversions:

// DateOnly → DateTime (at midnight, unspecified timezone)
if (underlyingType == typeof(DateTime) && value is DateOnly dateOnly)
    return dateOnly.ToDateTime(TimeOnly.MinValue);

// DateOnly → DateTimeOffset (at midnight, local timezone)
if (underlyingType == typeof(DateTimeOffset) && value is DateOnly dateOnlyForOffset)
{
    var dt = dateOnlyForOffset.ToDateTime(TimeOnly.MinValue);
    return new DateTimeOffset(dt);
}

// DateTimeOffset → DateOnly (extract date part)
if (underlyingType == typeof(DateOnly) && value is DateTimeOffset dto)
    return DateOnly.FromDateTime(dto.DateTime);

Expression Output Examples:

// Query: createdAt:2025-03-15 (DateOnly value)
// Property: DateTime CreatedAt

// Generated Expression Tree:
// (createdAt >= DateTime.Parse("2025-03-15T00:00:00Z"))
//     AND
// (createdAt < DateTime.Parse("2025-03-16T00:00:00Z"))

// Compiled Lambda:
// user => user.CreatedAt >= new DateTime(2025, 3, 15, 0, 0, 0, DateTimeKind.Utc)
//      && user.CreatedAt < new DateTime(2025, 3, 16, 0, 0, 0, DateTimeKind.Utc)

Why UTC?

Using UTC for datetime boundaries prevents timezone-related bugs:

• Ensures consistent behavior across different server timezones
• Prevents ambiguous date boundaries (e.g., daylight saving time transitions)
• Aligns with DateTimeOffset best practices

DateOnly Property Behavior:

For DateOnly properties, no transformation occurs—direct comparison is used:

// Query: birthDate:1990-01-15 (DateOnly value)
// Property: DateOnly BirthDate

// Generated Expression Tree:
// birthDate == new DateOnly(1990, 1, 15)

// No range transformation because both are DateOnly types

Nullable Type Handling

Null-Safe Property Access:

private Expression BuildNullSafePropertyAccess(Expression instance, PropertyInfo property)
{
    // For nullable types: instance == null ? default(TProperty) : instance.Property
    if (instance.Type.IsNullableReferenceType() || Nullable.GetUnderlyingType(instance.Type) != null)
    {
        var propertyAccess = Expression.Property(instance, property);
        var nullCheck = Expression.Equal(instance, Expression.Constant(null));
        var defaultValue = Expression.Default(property.PropertyType);

        return Expression.Condition(nullCheck, defaultValue, propertyAccess);
    }

    return Expression.Property(instance, property);
}

Supported CLR Types

Type Category	Types
Integers	byte, sbyte, short, ushort, int, uint, long, ulong
Floating	float, double, decimal
Date/Time	DateTime, DateTimeOffset, DateOnly, TimeOnly
Text	string, char
Boolean	bool
Other	Guid, enum types

---

Navigation Properties

Dot Notation Support

Simple Navigation:

// Filter: { "field": "Customer.Address.City", "operator": "eq", "value": "Seattle" }

// Generated expression:
// user => user.Customer.Address.City == "Seattle"

var property = _propertyAccessBuilder.BuildPropertyAccess(parameter, "Customer.Address.City");
// Result: Expression.Property(
//   Expression.Property(
//     Expression.Property(parameter, "Customer"),
//     "Address"),
//   "City")

Null-Safe Navigation:

// Generated expression with null safety:
// user => user.Customer == null ? null :
//         user.Customer.Address == null ? null :
//         user.Customer.Address.City

var segments = PropertyPathAnalyzer.Analyze(typeof(User), "Customer.Address.City");
Expression current = parameter;

foreach (var segment in segments)
{
    var property = Expression.Property(current, segment.PropertyName);

    if (segment.IsNullable)
    {
        var nullCheck = Expression.Equal(current, Expression.Constant(null));
        var defaultValue = Expression.Default(segment.PropertyType);
        current = Expression.Condition(nullCheck, defaultValue, property);
    }
    else
    {
        current = property;
    }
}

Collection Navigation

Automatic .Any() Semantics:

// Filter: { "field": "Items.ProductId", "operator": "eq", "value": 123 }

// Generated expression:
// order => order.Items.Any(item => item.ProductId == 123)

var collectionProperty = Expression.Property(parameter, "Items");
var itemParameter = Expression.Parameter(itemElementType, "item");
var itemFilter = BuildFilterExpression(/* ProductId == 123 */, itemParameter);
var anyMethod = typeof(Enumerable).GetMethod("Any", /* ... */);
var anyCall = Expression.Call(anyMethod, collectionProperty, Expression.Lambda(itemFilter, itemParameter));

Nested Collection Navigation:

// Filter: { "field": "Orders.Items.Product.Price", "operator": "gt", "value": 100 }

// Generated expression:
// user => user.Orders.Any(order =>
//           order.Items.Any(item =>
//             item.Product.Price > 100))

// Automatically detects collection segments and applies .Any() at each level

PropertyPathAnalyzer

Path Analysis:

public class PropertyPathSegment
{
    public string PropertyName { get; set; }
    public Type PropertyType { get; set; }
    public bool IsCollection { get; set; }
    public Type? ElementType { get; set; }
    public bool IsNullable { get; set; }
}

// Example:
var segments = PropertyPathAnalyzer.Analyze(typeof(Order), "Items.Product.Price");

// Returns:
// [
//   { PropertyName: "Items", PropertyType: List<OrderItem>, IsCollection: true, ElementType: OrderItem },
//   { PropertyName: "Product", PropertyType: Product, IsCollection: false },
//   { PropertyName: "Price", PropertyType: decimal, IsCollection: false }
// ]

---

Performance

Benchmarks

Compilation Performance:

Scenario	Time	Notes
Simple filter (first compile)	~5-10ms	Standard compilation
Simple filter (FastExpression)	~0.3-0.5ms	10-20x faster
Complex nested filter (first)	~20-50ms	Standard compilation
Complex nested filter (FastExpression)	~1-2ms	20-40x faster
Cache hit (L1)	<0.1ms	Memory lookup
Cache hit (L2)	~0.5-2ms	Promotes to L1
Cache miss	+compilation time	Must compile

Execution Performance:

Scenario	Time per Iteration	Notes
Cached compiled expression	~0.001-0.01ms	Fastest
Fresh compilation + execute	~5-50ms	Slowest
Cache hit rate >95%	~0.001-0.01ms avg	Production typical

Memory Usage:

Component	Size	Notes
L1 cache entry	~2KB	Expression + metadata
L2 cache entry	~2KB	Same as L1
Full L1 cache (1000 entries)	~2MB	Max capacity
Expression tree (average)	~1-5KB	Depends on complexity
Compiled delegate	~200-500 bytes	Native code

Optimization Techniques

Constant Folding:

// Before: 2 + 3 > x
// After: 5 > x

protected override Expression VisitBinary(BinaryExpression node)
{
    var visited = base.VisitBinary(node);

    if (visited is BinaryExpression binary &&
        binary.Left is ConstantExpression leftConst &&
        binary.Right is ConstantExpression rightConst)
    {
        // Evaluate at compile time
        var result = EvaluateConstantOperation(binary.NodeType, leftConst.Value, rightConst.Value);
        return Expression.Constant(result);
    }

    return visited;
}

Boolean Simplification:

// true && x → x
// false || x → x
// x && true → x
// x || false → x
// !!x → x

if (binary.NodeType == ExpressionType.AndAlso)
{
    if (binary.Left is ConstantExpression { Value: true })
        return binary.Right;
    if (binary.Right is ConstantExpression { Value: true })
        return binary.Left;
    if (binary.Left is ConstantExpression { Value: false })
        return Expression.Constant(false);
}

Comparison Negation:

// !(x > y) → x <= y
// !(x >= y) → x < y
// !(x == y) → x != y

if (unary.NodeType == ExpressionType.Not &&
    unary.Operand is BinaryExpression comparison)
{
    return comparison.NodeType switch
    {
        ExpressionType.GreaterThan => Expression.LessThanOrEqual(comparison.Left, comparison.Right),
        ExpressionType.GreaterThanOrEqual => Expression.LessThan(comparison.Left, comparison.Right),
        ExpressionType.LessThan => Expression.GreaterThanOrEqual(comparison.Left, comparison.Right),
        ExpressionType.LessThanOrEqual => Expression.GreaterThan(comparison.Left, comparison.Right),
        ExpressionType.Equal => Expression.NotEqual(comparison.Left, comparison.Right),
        ExpressionType.NotEqual => Expression.Equal(comparison.Left, comparison.Right),
        _ => unary
    };
}

Performance Tips

1. Enable Caching:

services.AddExpressionCompilation(options =>
{
    options.EnableCaching = true;  // Critical for performance
});

2. Reuse Builder Instances:

// ✅ Good - Singleton registration
services.AddSingleton(typeof(IExpressionBuilder<>), typeof(JsonToExpressionBuilder<>));

// ❌ Bad - Creating new instances
var builder = new JsonToExpressionBuilder<User>(/* ... */);

3. Pre-compile Common Filters:

// On startup, compile common filters to warm cache
var commonFilters = new[]
{
    FilterDefinitionExtensions.Equal("IsActive", true),
    FilterDefinitionExtensions.GreaterThan("Age", 18),
    // ... more common filters
};

foreach (var filter in commonFilters)
{
    await builder.BuildAsync(filter);  // Warms cache
}

4. Monitor Cache Statistics:

var stats = compiler.GetStatistics();
if (stats.HitRate < 90)
{
    _logger.LogWarning("Low cache hit rate: {HitRate}%", stats.HitRate);
    // Consider increasing L1 capacity or L2 expiration
}

5. Use Async Methods:

// ✅ Async (non-blocking)
var expression = await builder.BuildAsync(filter, cancellationToken);

// ❌ Sync (blocks thread, only if necessary)
var expression = builder.BuildAsync(filter).Result;

---

Usage Examples

Example 1: Basic Expression Building

using QueryBuilder.Expressions.Builders;
using QueryBuilder.Core.Extensions;

// Create filter
var filter = FilterDefinitionExtensions.And(
    FilterDefinitionExtensions.Equal("IsActive", true),
    FilterDefinitionExtensions.GreaterThan("Age", 18)
);

// Build expression
var builder = serviceProvider.GetRequiredService<IExpressionBuilder<User>>();
var expression = await builder.BuildAsync(filter);

// Result: Expression<Func<User, bool>>
// Lambda: user => user.IsActive == true && user.Age > 18

// Use with LINQ
var users = dbContext.Users.Where(expression).ToList();

Example 2: Complex Nested Filters

var complexFilter = FilterDefinitionExtensions.And(
    FilterDefinitionExtensions.Equal("Status", "Active"),
    FilterDefinitionExtensions.Or(
        FilterDefinitionExtensions.Equal("Role", "Admin"),
        FilterDefinitionExtensions.GreaterThan("TotalPurchases", 1000)
    ),
    FilterDefinitionExtensions.Not(
        FilterDefinitionExtensions.Equal("IsDeleted", true)
    )
);

var expression = await builder.BuildAsync(complexFilter);

// Result: user => user.Status == "Active" &&
//                (user.Role == "Admin" || user.TotalPurchases > 1000) &&
//                !(user.IsDeleted == true)

Example 3: Navigation Properties

var filter = FilterDefinitionExtensions.Equal("Customer.Address.City", "Seattle");

var expression = await builder.BuildAsync(filter);

// Result: order => order.Customer.Address.City == "Seattle"
// (with null-safe conditionals)

Example 4: Collection Filtering

// Filter orders that have items with ProductId = 123
var filter = FilterDefinitionExtensions.Equal("Items.ProductId", 123);

var expression = await builder.BuildAsync(filter);

// Result: order => order.Items.Any(item => item.ProductId == 123)

Example 5: String Operations

var filter = FilterDefinitionExtensions.And(
    FilterDefinitionExtensions.Contains("Name", "John"),
    FilterDefinitionExtensions.StartsWith("Email", "admin"),
    FilterDefinitionExtensions.EndsWith("Email", "@company.com")
);

var expression = await builder.BuildAsync(filter);

// Result: user => user.Name.Contains("John", StringComparison.OrdinalIgnoreCase) &&
//                user.Email.StartsWith("admin", StringComparison.OrdinalIgnoreCase) &&
//                user.Email.EndsWith("@company.com", StringComparison.OrdinalIgnoreCase)

Example 6: IN and BETWEEN Operators

var filter = FilterDefinitionExtensions.And(
    FilterDefinitionExtensions.In("Status", "Active", "Pending", "InProgress"),
    FilterDefinitionExtensions.Between("Price", 10, 100)
);

var expression = await builder.BuildAsync(filter);

// Result: product => (product.Status == "Active" || product.Status == "Pending" || product.Status == "InProgress") &&
//                   (product.Price >= 10 && product.Price <= 100)

Example 7: Caching in Action

var filter = FilterDefinitionExtensions.Equal("IsActive", true);

// First call - cache miss, compiles expression
var sw = Stopwatch.StartNew();
var expression1 = await builder.BuildAsync(filter);
sw.Stop();
Console.WriteLine($"First call: {sw.ElapsedMilliseconds}ms");  // ~5-10ms

// Second call - cache hit
sw.Restart();
var expression2 = await builder.BuildAsync(filter);
sw.Stop();
Console.WriteLine($"Second call: {sw.ElapsedMilliseconds}ms");  // <0.1ms

// Expressions are cached, not object identity
Assert.NotSame(expression1, expression2);  // Different objects
Assert.Equal(expression1.ToString(), expression2.ToString());  // Same logic

Example 8: Type Coercion

// String to DateTime conversion
var filter = new FilterDefinition
{
    Field = "CreatedAt",
    Operator = FilterOperator.GreaterThan,
    Value = "2025-01-01"  // String value
};

var expression = await builder.BuildAsync(filter);

// Automatic coercion: "2025-01-01" → DateTime(2025, 1, 1)
// Result: entity => entity.CreatedAt > new DateTime(2025, 1, 1)

---

Integration

With Shorthand Batch Rule Compiler

Expression compilation powers the high-performance shorthand rule evaluation system for ETL scenarios.

Integration Architecture:

Shorthand Query
    ↓
[ShorthandQueryParser] → QueryDefinition (cached in shorthand layer)
    ↓
[IExpressionBuilder<T>] → Expression<Func<T, bool>> (uses L1+L2 cache)
    ↓
[IExpressionCompiler] → Func<T, bool> (uses FastExpressionCompiler)
    ↓
[BatchRuleCompiler<T>] → Dictionary<int, Func<T, bool>>
    ↓
Rapid instance evaluation (thousands of evaluations per second)

Example:

// Setup: Expression compiler registered in DI
services.AddQueryBuilderExpressions();  // Registers IExpressionBuilder, IExpressionCompiler
services.AddFilterShorthandParser();    // Registers shorthand parser
services.AddBatchRuleCompiler<Order>(); // Registers batch compiler (uses expression services)

// Usage: Expression compiler used internally by batch compiler
public class OrderTaggingService
{
    private readonly BatchRuleCompiler<Order> _compiler;

    public async Task InitializeAsync(IEnumerable<TagRule> rules)
    {
        // Internally:
        // 1. Parses shorthand → QueryDefinition
        // 2. Calls IExpressionBuilder<Order>.BuildAsync() → Expression<Func<Order, bool>>
        // 3. Calls IExpressionCompiler.CompileAsync() → Func<Order, bool>
        // 4. Stores compiled function for rapid evaluation
        await _compiler.PreCompileRulesAsync(rules, "orders");
    }

    public bool Matches(Order order, int ruleId)
    {
        // Direct function invocation - no parsing or compilation overhead
        return _compiler.Matches(order, ruleId);
    }
}

Caching Layers:

• L1: Shorthand query cache (eliminates ANTLR parsing)
• L2: Expression builder cache (eliminates expression tree building) - This layer
• L3: Expression compiler cache (eliminates delegate compilation) - This layer
• L4: Rule compiler cache (eliminates dictionary lookups)

Performance Impact:

• First rule compilation: Parses shorthand + builds expression + compiles delegate
• Subsequent identical queries: Cached at shorthand layer (skips all lower layers)
• Subsequent similar filters: Cached at expression layer (skips compilation)
• Hot path evaluation: Direct function invocation (microseconds per check)

---

With InMemoryExecutionStrategy

public class InMemoryExecutionStrategy : IExecutionStrategy
{
    private readonly IExpressionBuilder<T> _expressionBuilder;

    public async Task<IQueryResult<T>> ExecuteAsync<T>(
        QueryDefinition query,
        IExecutionContext? context = null) where T : class
    {
        // 1. Get data from provider
        var provider = _providerRegistry.GetProvider<T>(query.From.SourceName);
        var data = await provider.GetDataAsync();

        // 2. Build expression from filter
        var expression = await _expressionBuilder.BuildAsync(query.Where);

        // 3. Apply filter
        var filtered = data.AsQueryable().Where(expression);

        // 4. Apply ordering
        if (query.OrderBy != null)
            filtered = ApplyOrdering(filtered, query.OrderBy);

        // 5. Apply pagination
        if (query.Offset.HasValue)
            filtered = filtered.Skip(query.Offset.Value);
        if (query.Limit.HasValue)
            filtered = filtered.Take(query.Limit.Value);

        return new QueryResult<T> { Data = filtered.ToList() };
    }
}

With Specification Pattern

public static class SpecificationExpressionBuilder
{
    public static async Task<Expression<Func<T, bool>>> BuildExpressionAsync<T>(
        Specification specification,
        IExpressionBuilder<T> builder) where T : class
    {
        // Build expression from specification's filter
        return await builder.BuildAsync(specification.Filter);
    }

    public static async Task<Func<T, bool>> BuildCompiledFunctionAsync<T>(
        Specification specification,
        IExpressionBuilder<T> builder,
        IExpressionCompiler compiler) where T : class
    {
        // Build and compile expression
        var expression = await builder.BuildAsync(specification.Filter);
        return await compiler.CompileAsync(expression);
    }
}

Service Registration

// Startup.cs
services.AddExpressionCompilation(options =>
{
    options.EnableCaching = true;
    options.L1CacheCapacity = 1000;
    options.L2CacheExpiration = TimeSpan.FromMinutes(120);
    options.EnableOptimization = true;
    options.MaxExpressionDepth = 100;
});

// Register builder factory (generic type)
services.AddSingleton(typeof(IExpressionBuilder<>), typeof(JsonToExpressionBuilder<>));

// Register compiler
services.AddSingleton<IExpressionCompiler, CachedExpressionCompiler>();

// Register optimizer
services.AddSingleton<IExpressionOptimizer, OptimizationVisitor>();

// Register property access builder
services.AddSingleton<IPropertyAccessBuilder, PropertyAccessBuilder>();

---

Configuration

Expression compilation uses sensible defaults with no additional configuration required.

Service Registration

// Register expression compilation (caching, compiling)
services.AddExpressionCompilation();

// Register expression builders (includes compilation services)
services.AddExpressionBuilders();

What gets registered:

• IExpressionCompiler - Cached expression compiler (singleton)
• IExpressionCache - Memory-based expression cache (singleton)
• IExpressionBuilder<T> - Generic expression builder (singleton)
• INestedProjectionBuilder - Nested projection builder (singleton)
• IPropertyAccessBuilder - Property access utilities (singleton)
• Object pools for expression builder and compilation contexts

HashVisitor is a static helper (HashVisitor.ComputeHash(expression)) — no DI registration is required or supported.

Default Behavior:

• Two-level caching enabled (L1: ConcurrentDictionary, L2: IMemoryCache)
• FastExpressionCompiler used when available
• Maximum property navigation depth: 16 levels
• Thread-safe singleton instances for cache sharing

Disabling Caching: Use QueryBuilderOptions.DisableExpressionCaching() in the unified registration:

services.AddQueryBuilder(options =>
{
    options.DisableExpressionCaching(); // For testing or low-memory environments
});

---

Optimization

Expression Tree Optimization

OptimizationVisitor Passes:

1. Constant Folding Pass:

   • Evaluates constant operations at compile time
   • Example: 2 + 3 → 5

2. Boolean Simplification Pass:

   • Simplifies boolean algebra
   • Examples: true && x → x, false || x → x

3. Comparison Negation Pass:

   • Converts negated comparisons
   • Example: !(x > y) → x <= y

Multi-Pass Optimization:

public Expression<Func<T, bool>> Optimize<T>(Expression<Func<T, bool>> expression)
{
    var current = expression;
    var previousHash = "";

    // Up to 3 passes, stop if no changes
    for (int pass = 0; pass < 3; pass++)
    {
        var optimized = (Expression<Func<T, bool>>)Visit(current);
        var currentHash = HashVisitor.ComputeHash(optimized).ToString();

        if (currentHash == previousHash)
            break;  // No changes, optimization converged

        current = optimized;
        previousHash = currentHash;
    }

    return current;
}

Cache Tuning

Increase L1 Capacity for High Traffic:

options.L1CacheCapacity = 5000;  // Default: 1000

Extend L2 Expiration for Stable Queries:

options.L2CacheExpiration = TimeSpan.FromHours(6);  // Default: 2 hours

Monitor and Adjust:

var stats = compiler.GetStatistics();
if (stats.HitRate < 90)
{
    // Increase capacity or expiration
}
if (stats.EstimatedMemoryUsage > maxMemory)
{
    // Decrease capacity or expiration
}

---

Testing

Test Coverage

JsonToExpressionBuilderTests:

• Simple comparisons (Age > 18)
• String operations (Contains, StartsWith, EndsWith)
• Logical operators (AND, OR, NOT)
• Complex nested expressions
• Collection operators (In, Between)
• Null checks
• Navigation properties
• Type coercion

CachingTests:

• Store and retrieve expressions
• Cache hits and misses
• Clear operations
• Statistics tracking
• Eviction handling

PropertyAccessBuilderTests:

• Simple property access
• Nested navigation
• Null-safe conditionals
• Collection navigation
• Path validation

OptimizationVisitorTests:

• Constant folding
• Boolean simplification
• Comparison negation
• Multi-pass optimization

Running Tests

# All expression tests
dotnet test tests/QueryBuilder.Expressions.Tests/

# Specific category
dotnet test --filter "FullyQualifiedName~JsonToExpressionBuilderTests"

# With coverage
dotnet test --collect:"XPlat Code Coverage"

Example Tests

[Fact]
public async Task BuildAsync_SimpleComparison_GeneratesCorrectExpression()
{
    var filter = new FilterDefinition
    {
        Field = "Age",
        Operator = FilterOperator.GreaterThan,
        Value = 18
    };

    var expression = await _builder.BuildAsync(filter);

    var compiled = expression.Compile();
    var user = new User { Age = 25 };
    compiled(user).Should().BeTrue();
}

[Fact]
public async Task BuildAsync_CachedExpression_ReturnsFast()
{
    var filter = FilterDefinitionExtensions.Equal("IsActive", true);

    var sw = Stopwatch.StartNew();
    await _builder.BuildAsync(filter);
    var firstTime = sw.ElapsedMilliseconds;

    sw.Restart();
    await _builder.BuildAsync(filter);
    var secondTime = sw.ElapsedMilliseconds;

    secondTime.Should().BeLessThan(firstTime / 10);  // 10x+ faster
}

---

Contributing

Development Setup

Prerequisites:

• .NET 10 SDK
• Your favorite IDE

Build:

dotnet build src/QueryBuilder.Expressions/QueryBuilder.Expressions.csproj

Run Tests:

dotnet test tests/QueryBuilder.Expressions.Tests/

Adding New Operators

1. Add operator support:

// In JsonToExpressionBuilder.BuildFilterExpression()
return filter.Operator switch
{
    // ... existing operators
    FilterOperator.CustomOperator => BuildCustomOperatorExpression(property, constant),
    _ => throw new NotSupportedException()
};

2. Implement builder method:

private Expression BuildCustomOperatorExpression(Expression property, Expression value)
{
    // Your operator logic here
    return /* ... */;
}

3. Add tests:

[Fact]
public async Task BuildAsync_CustomOperator_GeneratesCorrectExpression()
{
    var filter = new FilterDefinition
    {
        Field = "Field",
        Operator = FilterOperator.CustomOperator,
        Value = "value"
    };

    var expression = await _builder.BuildAsync(filter);
    // Assert...
}

Code Style

• C# 12 features preferred
• Nullable reference types enabled
• XML documentation required for public APIs
• Unit tests required for new features
• Performance benchmarks for optimizations

---

Documentation

Project Documentation

• Main README: /README.md - Universal Query Builder overview
• Architecture: /docs/architecture-refactor-plan.md - System architecture
• CLAUDE.md: /CLAUDE.md - Development guide

API Documentation

All public APIs include XML documentation:

/// <summary>
/// Builds a LINQ expression tree from a filter definition.
/// </summary>
/// <typeparam name="T">The entity type to filter.</typeparam>
/// <param name="filter">The filter definition to convert.</param>
/// <param name="cancellationToken">Optional cancellation token.</param>
/// <returns>A compiled LINQ expression representing the filter.</returns>
/// <exception cref="QueryValidationException">Filter validation failed.</exception>
Task<Expression<Func<T, bool>>> BuildAsync(
    FilterDefinition filter,
    CancellationToken cancellationToken = default);

---

License

Part of the Universal Query Builder project.

See /LICENSE for license information.

---

Support

• Issues: GitHub Issues
• Documentation: /docs/
• Examples: /examples/

---

Built with ❤️ for high-performance expression compilation