Lyo.Query 1.0.0

Lyo.Query

Execution engine for Lyo.Query.Models filter trees. IWhereClauseService / BaseWhereClauseService translates polymorphic WhereClause ASTs (condition / group, optional SubClause) into LINQ expression trees on IQueryable<T>, applies multi-key SortBy, and can evaluate / explain the same AST against a loaded entity in memory.

What this package is: AST → expression / matcher. What it is not: HTTP endpoints, QueryConcreteReq / ProjectionQueryReq / QueryReq builders, or query-result caching — those live in Models + Lyo.Api.

EF-agnostic at the boundary (works on any IQueryable, including EF DbSet and in-memory). Requires ICacheService + CacheOptions for compiled predicate / matcher / metadata caches.

Targets net10.0.

Features

• AST → IQueryable — ApplyWhereClause builds EF-translatable (or provider) expression trees from WhereClause
• 16 comparators — Equals / NotEquals / Contains* / StartsWith* / EndsWith* / Greater* / Less* / In / NotIn / Regex / NotRegex
• Collection paths — Lines.Quantity → Any(...) on collection elements; bare collection / .Count style paths compare against count
• Two-phase SubClause — includeSubClauses: false for SQL root pass; load includes then match sub-tree in memory
• Multi-key sort — SortByProperty + ApplyOrdering with Priority and stable default tie-break
• In-memory match + explain — MatchesWhereClause / ExplainMatch with blocking path and OR-branch detail
• ICache hot path — compiled EF predicates (filter_ef_predicate:*), matchers (filter_matcher:*), sort keys, property-path metadata
• Path validation — TryValidatePropertyPath / InvalidQueryException for bad dotted paths
• Contains→Regex coalesce — adjacent Contains leaves on the same string field can merge into one Regex alternation
• Property diffing — IPropertyComparisonService for patch/update pipelines
• Value coercion — IValueConversionService for JSON literals → CLR property types (suppressible when Lyo.Api owns conversion)

Examples

Register services

using Lyo.Query;

services.AddLocalCache(/* … */); // or AddFusionCache(...)
services.AddLyoQueryServices();
// registers IValueConversionService, IPropertyComparisonService, IWhereClauseService (singletons)

// Hosting Lyo.Api (richer ITypeConversionService already registered):
// services.AddLyoQueryServices(registerValueConversion: false);

Apply WhereClause to IQueryable

using Lyo.Query;
using Lyo.Query.Models;
using Lyo.Query.Models.Builders;

var where = WhereClauseBuilder.And()
    .Equals("Status", "Active")
    .GreaterThan("Age", 18)
    .Build();

IQueryable<Person> people = db.People.AsQueryable();
var filtered = whereClauseService.ApplyWhereClause(people, where);
var page = await filtered.Skip(20).Take(20).ToListAsync(ct);

Two-phase SubClause (SQL then in-memory)

// Root runs as SQL; SubClause refined after Include load
var where = WhereClauseBuilder.And()
    .Equals("Status", "Open")
    .AddSubClause(sub => sub.Contains("Tags", "vip"))
    .Build();

var sqlPhase = whereClauseService.ApplyWhereClause(db.Orders, where, includeSubClauses: false);
var includePaths = whereClauseService.GetCollectionIncludePathsForWhereClause<Order>(where);
// apply Include(includePaths…) then materialize
var rows = await sqlPhase /* .Include(...) */ .ToListAsync(ct);
var matched = rows.Where(o => whereClauseService.MatchesWhereClause(o, where)).ToList();

Multi-key ordering

using Lyo.Query.Models;
using Lyo.Common.Enums;

var sorted = whereClauseService.ApplyOrdering(
    queryable: people,
    sortByProps:
    [
        new SortBy { PropertyName = "LastName", Direction = SortDirection.Ascending, Priority = 0 },
        new SortBy { PropertyName = "FirstName", Direction = SortDirection.Ascending, Priority = 1 },
    ],
    defaultOrder: p => p.Id,
    defaultSortDirection: SortDirection.Ascending);

Match and ExplainMatch

var entity = await db.People.AsNoTracking().FirstAsync(ct);

if (whereClauseService.MatchesWhereClause(entity, where))
{
    var explain = whereClauseService.ExplainMatch(entity, where);
    // explain.Passed, BlockingPath, FailureSummary, Nodes (per-condition ActualValueSummary)
    // In-memory only — not for SQL-only pipelines
}

Validate paths + collection includes

if (!whereClauseService.TryValidatePropertyPath<Person>("Addresses.City", out var err))
    throw new InvalidOperationException(err);

foreach (var path in whereClauseService.GetCollectionIncludePathsForWhereClause<Person>(where))
    query = query.Include(path); // e.g. "Addresses" for "Addresses.City"

WhereClauseHelpers fingerprints

using Lyo.Query.Services.WhereClause;

var hash = WhereClauseHelpers.GetWhereClauseTreeHash(where); // stable structural fingerprint
var hasSub = WhereClauseHelpers.HasAnySubClause(where);
if (WhereClauseHelpers.TryExtractConditions(where, out var conditions, out var op))
{
    // flat And/Or leaves — false when SubClause or unsupported nesting present
}

Property diff for patch

var diffs = propertyComparisonService.GetPropertyDifferences(entity, patchDto);
// Dictionary<propertyName, newValue> — only changed writable props

Benchmarks

• Portfolio suite: query

Package map

Builders and endpoint payloads are documented on Models / Api. This doc covers the runtime translator.

IWhereClauseService

BaseWhereClauseService is the default DI implementation.

Invalid paths / operators → InvalidQueryException (Lyo.Query.Models.Exceptions).

Comparators and path semantics

Operators come from ComparisonOperatorEnum on each ConditionClause:

Dotted paths

• Scalar: Status, Address.City
• Into collection: Lines.Quantity → lines.Any(e => e.Quantity …)
• Collection count: path metadata IsCountPath builds Enumerable.Count then compares

Group optimization: within an AND/OR group, multiple Contains leaves on the same string field may be coalesced into a single case-insensitive Regex alternation before expression build.

Two-phase SubClause execution

WhereClause.SubClause (on a condition or group) supports split execution — used heavily by Lyo.Api query pipelines:

1. Phase 1 (DB) — ApplyWhereClause(..., includeSubClauses: false) so only the primary predicate becomes SQL.
2. Load — GetCollectionIncludePathsForWhereClause → EF Include for collection segments referenced by the sub-tree.
3. Phase 2 (memory) — MatchesWhereClause / ExplainMatch on materialized entities with the full tree (includeSubClauses default true).

Helpers:

• WhereClauseHelpers.HasAnySubClause(node) — detect whether a two-phase path is needed
• WhereClauseHelpers.TryExtractConditions — flatten simple And/Or trees for projection-level filtering (returns false if any SubClause)
• WhereClauseHelpers.GetWhereClauseTreeHash — structural fingerprint for cache keys / logging (not cryptographic)

var where = WhereClauseBuilder.And()
    .Equals("Age", 10)
    .AddSubClause(sub => sub.AddAnd(s => s.Equals("Name", "Alice")))
    .Build();

ICache usage (predicate / matcher cache)

AddLyoQueryServices requires ICacheService + CacheOptions (e.g. AddLocalCache / AddFusionCache). This is not the same as Lyo.Api query-result caching (QueryOptions.CacheQueryResultsAsUtf8Payload).

Tags typically include the entity CLR type so hosts can invalidate by type when schemas change.

Api result cache (optional UTF-8 payload entries for /QueryConcrete + /QueryProject) is documented under Lyo.Api — Query result caching.

IPropertyComparisonService

GetPropertyDifferences<TEntity, TOther>(entity, newData) walks public writable properties on TEntity that have a same-named readable property on TOther, compares with an inferred strategy (direct equality, enum/string coercion, or Convert.ChangeType), and returns Dictionary<string, object?> of changed property → proposed new value.

Used by patch/update pipelines (e.g. Lyo.Api) to build minimal diffs. Strategies are cached via ICacheService.

IValueConversionService

• ConvertToTargetType(value, targetType) — JSON literals → CLR (primitives, nullables, Guid, enums, date/time, lists)
• GetUnderlyingType(type) — strip Nullable<T>
• IsObjectEnumerable(value) — non-string / non-byte[] IEnumerable

When hosting Lyo.Api, register Api’s richer ITypeConversionService (extends this interface) and call:

services.AddLyoQueryServices(registerValueConversion: false);

Otherwise two IValueConversionService registrations fight.

Registration

services.AddLocalCache(/* … */); // prerequisite
services.AddLyoQueryServices(registerValueConversion: true);

Registers as singletons:

• IValueConversionService → ValueConversionService (unless registerValueConversion: false)
• IPropertyComparisonService → PropertyComparisonService
• IWhereClauseService → BaseWhereClauseService

Optional: IMetrics / ILogger are taken from DI when present (Constants.Metrics.*).

Metrics

When IMetrics is registered, BaseWhereClauseService emits under Lyo.Query.Constants.Metrics:

Tags: entity_type, operation (Constants.Metrics.Tags).

Related docs

• DTOs / builders / request shapes → Lyo.Query.Models
• HTTP endpoints, projection SQL, result caching → Lyo.Api Query & Request Builders
• Blazor workbench → Lyo.Query.Web.Components
• Portfolio benchmarks suite → query

Links

• Lyo.Query.Models
• Lyo.Api query builders
• Lyo.Api query result caching

Dependencies

Generated from ProjectReference / PackageReference (same model as docs/Lyo.ProjectGraph.html).

• Lyo.Cache — (direct, lyo)
• Lyo.Common — (direct, lyo)
• Lyo.Exceptions — (direct, lyo)
• Lyo.Metrics — (direct, lyo)
• Lyo.Query.Models — (direct, lyo)
• Microsoft.Extensions.DependencyInjection.Abstractions 10.0.5 — (direct, microsoft)
• Microsoft.Extensions.Logging.Abstractions 10.0.5 — (direct, microsoft)
• Lyo.Compression — (transitive, lyo)
• Lyo.Encryption — (transitive, lyo)
• Lyo.Hashing — (transitive, lyo)
• Lyo.Health — (transitive, lyo)
• Lyo.KeyStore — (transitive, lyo)
• Lyo.Result — (transitive, lyo)
• Lyo.Streams — (transitive, lyo)
• BouncyCastle.Cryptography 2.6.2 — (transitive, third-party, netstandard2.0)
• EasyCompressor 2.1.0 — (transitive, third-party)
• Konscious.Security.Cryptography.Argon2 1.3.1 — (transitive, third-party)
• Microsoft.Bcl.AsyncInterfaces 10.0.5 — (transitive, microsoft, netstandard2.0)
• Microsoft.Extensions.Caching.Memory 10.0.5 — (transitive, microsoft)
• Microsoft.Extensions.Configuration.Binder 10.0.5 — (transitive, microsoft)
• Microsoft.Extensions.DependencyInjection 10.0.5 — (transitive, microsoft)
• Microsoft.Extensions.Options.ConfigurationExtensions 10.0.5 — (transitive, microsoft)
• System.Buffers 4.6.1 — (transitive, microsoft, netstandard2.0)
• System.IO.Hashing 10.0.5 — (transitive, microsoft, net10.0)
• System.Memory 4.6.3 — (transitive, microsoft, netstandard2.0)
• System.Text.Json 10.0.5 — (transitive, microsoft, netstandard2.0)
• System.Threading.Tasks.Extensions 4.6.3 — (transitive, microsoft, netstandard2.0)