FlexQuery.NET.Diagnostics 4.0.0

• net10.0

  • FlexQuery.NET (>= 4.0.0)

• net6.0

  • FlexQuery.NET (>= 4.0.0)

• net8.0

  • FlexQuery.NET (>= 4.0.0)

# FlexQuery.NET v4.0.0 Release Notes

**Release date:** 2026-09-16

## Overview

v4.0.0 is the largest release in the project's history: **369 commits since v3.1.1** (2026-07-04
through the `v4.0.0-rc.1` cutoff), spanning a rebuilt configuration model, typed DTO projection,
`include` relationship query blocks
(filtered, sorted, bounded collection slices), keyset pagination, an aggregate/HAVING redesign, the FQL parser,
provider modernization for both EF Core and Dapper, and a new OpenAPI package.

v4 is a **breaking release**. Package renames, restructured option classes, and removed legacy
syntaxes require code changes — most are mechanical. The authoritative step-by-step guide is
[Migrate from v3.1.1 to v4](https://github.com/peterjohncasasola/FlexQuery.NET/tree/main/docs-v4/content/docs/migration/v3-to-v4.mdx),
with a compact area-to-area mapping in the
[Change Matrix](https://github.com/peterjohncasasola/FlexQuery.NET/tree/main/docs-v4/content/docs/migration/change-matrix.mdx).

**Target frameworks** are now `.NET 6.0 / 8.0 / 10.0` (the .NET 7 target is dropped).
`FlexQuery.NET.OpenApi` targets `.NET 9.0 / 10.0` and requires `Microsoft.AspNetCore.OpenApi`.

---

## Highlights

### 1. Typed DTO Projection

Every provider now supports `FlexQueryAsync<TEntity, TResponse>` (4 EF Core + 4 Dapper overloads),
returning `QueryResult<TResponse>`. All pipeline stages — filter, sort, select, grouping, includes,
paging — run against the entity model; rows are materialized through entity→DTO maps.

```csharp
FlexQueryCore.Configure(options =>
{
   options.CreateMap<Customer, CustomerResponse>()
       .ForMember(d => d.CustomerName, e => e.FirstName);
});

var result = await db.Customers
   .FlexQueryAsync<Customer, CustomerResponse>(parameters, cancellationToken: ct);
```

New mapping vocabulary (`CreateMap`, `MapField`, `ForMember`, `ForNavigation`) lives in the global
`FlexQueryMapping` registry, with per-execution overrides via execution options. Navigation members
project only the requested navigation graph; computed `ForMember` expressions execute per row after
projection on EF Core.

**Affected packages:** `FlexQuery.NET`, `FlexQuery.NET.EntityFrameworkCore`, `FlexQuery.NET.Dapper`

### 2. Relationship Query Blocks on `include`

`FilteredIncludes` is replaced by options written directly in the `include` block - collection
relationships can carry their own `filter` / `sort` / `take` (and nested `include`):

```http
include=Orders(take=5;filter=Status:eq:'Active';sort=OrderDate:desc)
```

DSL and FQL parsers accept nested blocks; options inside a block are semicolon-separated. The
removed preview keyword `expand` (v4 previews only; it never appeared in a stable release) folds
into `include(...)` - and `expand=` is now rejected with a migration error in both DSL and FQL.
Include paths and every block level are governed by the new `AllowedIncludes` governance set (see
Security changes), and collection operations are rejected on single-valued relationships based on
the relationship metadata. `QueryOptions.Includes` (a single `IncludeNode` tree) and
`ApplyIncludes<T>()` are the programmatic entry points.

**Affected packages:** `FlexQuery.NET`, providers, parsers

### 3. Keyset (Cursor) Pagination

Offset paging gets slower the deeper you scroll. v4 adds keyset pagination end-to-end in both core
parameter parsing and the EF Core and Dapper executors:

```http
GET /api/customers?useKeysetPagination=true&sort=LastName:asc,Id:asc&pageSize=20
GET ...&cursor=eyJ2IjoxLCJ2YWx1ZXMiOlsiTWlsbGVyIiw4N119
```

`QueryResult.NextCursorToken` carries the opaque token; `SeekAfter` is the programmatic predicate
builder. Cursors must match the current sort; mixing `page` with keyset mode is an error.

**Affected packages:** `FlexQuery.NET`, `FlexQuery.NET.EntityFrameworkCore`, `FlexQuery.NET.Dapper`

### 4. Aggregates and HAVING Redesign

Aggregates move out of `select` into a dedicated `aggregate` parameter with a typed
`AggregateFunction` enum (was the string-function `AggregateModel`):

```http
# v3.1.1 — aggregates inside select
select=Status,sum(Total),count(Id)

# v4 — dedicated aggregate parameter
select=Status&aggregate=sum:Total,count:Id
```

Aliases default to PascalCase (`SumTotal`); explicit aliasing via `sum:Total:totalSales` (or FQL
`FUNCTION(field) [AS alias]`). HAVING became a recursive-descent-parsed condition
tree (`HavingNode` / `HavingLogicalNode` / `HavingConditionNode` / `HavingGroupNode`) with
AND/OR/parentheses support across DSL and FQL, and a dedicated grouped query executor
(`DynamicGroupedResult` model) runs grouped queries in both providers.

### 5. Nested `select` Trees with Aliases

`QueryOptions.Select` is now a `List<SelectNode>` tree (was `List<string>`), parsed through a
`SelectModel` AST. New syntax: `field:alias`, `field as alias`, nested select blocks (children
materialize through mapped TypeMaps), and mixed flat/nested forms. FQL and DSL parsers both gained
nested alias parsing, and `MergeTree` propagates aliases with conflict detection. Sort gained a
space form (`sort=Name ASC, Age DESC`) alongside the colon form.

### 6. FQL Parser (JQL Renamed) + DSL Grammar Extensions

- Package `FlexQuery.NET.Parsers.Jql` → **`FlexQuery.NET.Parsers.Fql`**; `QuerySyntax.Jql` →
 `QuerySyntax.Fql`; `JqlParseException` → `FqlParseException` (now derived from
 `FlexQueryException`).
- DSL filters accept `AND` / `OR` keywords in addition to `&` / `|` (additive). The logical forms
 build AND-groups inside OR-groups per precedence; `AND`/`OR` become reserved and must be quoted
 as values.
- All parsers report structured `Position` / `Expected` / `Found` on internal parse failures, and
 query parameters gained a `QueryParserRegistry` for per-syntax resolution.

### 7. Fluent Query Building

`Query.Create()` — a full fluent builder (filters via `FilterGroupBuilder`, sort, include blocks,
aggregates, paging) — complements the existing `FilterBuilder`. A strongly-typed
`FlexQueryRequest` object plus `FlexQueryRequestExtensions` cover the request layer, and
`MiniODataRequest.ToQueryOptions()` joins the adapter extension set.

### 8. Dapper Provider Modernized

- **Model definitions:** `MappingRegistry` is replaced by an EF-style `ModelBuilder` +
 `IEntityTypeConfiguration<T>`, configured through the provider facade:

 ```csharp
 FlexQueryDapper.Configure(options =>
 {
     options.Model.Entity<Customer>()
         .ToTable("Customers")
         .HasKey(c => c.Id)
         .HasMany(c => c.Orders)
         .HasForeignKey("CustomerId");
 });
 ```

- **Dialect auto-detection** from the open `DbConnection` — manual `Dialect` /
 `ISqlDialectResolver` configuration is removed.
- **SQL execution logging:** the final SQL and bound parameters are logged before every Dapper
 command, including copy-paste-ready `DECLARE` scripts for reproducing queries in SSMS.
- **DTO-aware SQL generation** with nested TypeMap field rewrites, include hydration reads the unified include tree,
  include-only joins excluded from the count query, split-query hydration internalized,
 and strongly-typed projected entities via the (now-internal) dynamic type builder honoring JSON
 naming policies.
- The five dynamic overloads consolidated to three (`FlexQueryParameters`,
 `IDictionary<string, StringValues>`, `QueryOptions`) plus four typed overloads.

### 9. New Package: `FlexQuery.NET.OpenApi`

Query-parameter surface for the modern `Microsoft.AspNetCore.OpenApi` (.NET 9/10) stack:
`AddFlexQueryOpenApi` / `AddFlexQuery` registrations plus an example provider aligned with the v4
`QueryOptions` model. Swashbuckle-era guidance is obsolete.

### 10. Result Surface and Cross-Cutting Additions

- `QueryResult<T>` gains `NextCursorToken` and `ResultShape` — response JSON is shaped to the
 selected DTO surface (with a JSON converter), keeping EF and Dapper output identical.
- `CancellationToken` added to **all** async provider overloads (trailing parameter).
- Per-request `QuerySyntax` and `DisablePaging` overrides in configuration.
- ASP.NET Core DI surface: `AddFlexQuerySecurity`, `AddFlexQueryJson`, `AddFlexQuery` (global
 configuration).

---

## Breaking Changes

### 1. Configuration Model Rebuilt — DI Registration Replaced by Static Facades

```csharp
// v3.1.1 — DI-era registration
services.AddFlexQueryDapper(...);
services.AddFlexQueryMiniOData(...);

// v4 — static facades, immutable after first use
FlexQueryCore.Configure(options => { ... });
FlexQueryDapper.Configure(options => { ... });
FlexQueryEFCore.Configure(options => options.UseNoTracking = true);
Fql.Register();
MiniOData.Register();
```

Calling any `Configure` **after a query has executed throws `InvalidOperationException`**. Global
options are immutable after the first call. No-tracking behavior moves from
`QueryExecutionOptions.UseNoTracking` / `UseSplitQuery` to provider facade or per-call
`EfCoreQueryOptions.UseNoTracking` (`bool?`); `UseSplitQuery` is gone — the provider decides the
SQL shape internally.

### 2. Renamed

| v3.1.1 | v4 | Migration |
|---|---|---|
| Package `FlexQuery.NET.Parsers.Jql` | `FlexQuery.NET.Parsers.Fql` | Update package reference |
| `QuerySyntax.Jql` | `QuerySyntax.Fql` | Find/replace |
| `JqlParseException : Exception` | `FqlParseException : FlexQueryException` | Update catch blocks |
| `QueryOptions.FilteredIncludes` (+ string `Includes`) | `QueryOptions.Includes` (single include tree) | Find/replace; block syntax on the Include guide |
| `ApplyFilteredIncludes<T>()` | `ApplyIncludes<T>()` | Find/replace |
| `AggregateModel` (string function) | `Aggregate` (typed `AggregateFunction`) | Update construction sites |
| `HavingCondition` | `HavingNode` tree | Update construction sites |
| `QueryOptions.Select` (`List<string>`) | `List<SelectNode>` | Update manual tree construction |
| `DebugResult` | `QueryDebugInfo` (via `ToFlexQueryDebug`) | Find/replace |
| `Models.IFlexQueryExecutionListener` | `Execution.IFlexQueryExecutionListener` | Update using (members unchanged) |
| `Models.QueryContext` | `Execution.QueryContext` (sealed) | Update using directives |
| `Models.BaseQueryOptions` | split: `Options.BaseQueryOptions` + `Options.QueryGovernanceOptions` | Adjust base-class references |

### 3. Removed

| Removed | Replacement |
|---|---|
| `expand` query keyword (v4 previews only; never stable) | move branch options into the `include(...)` block |
| JSON / Indexed / Generic query syntaxes (`JsonQueryParser`, `AutoDetect`) | Native DSL, FQL, or MiniOData |
| `CaseInsensitive` / `CaseInsensitiveFields` options | — (comparisons follow provider semantics) |
| Parser DI registration (`ServiceCollectionExtensions` in parser packages, `MiniODataFeature`) | Static `Fql.Register()` / `MiniOData.Register()` |
| Deprecated `QueryOptions` members: `Skip`, `Top`, `EnableCache`, `Items`, `Ast` | `PagingOptions`, per-call options |
| `InvalidFilterFieldException` / `InvalidSortFieldException` | `QueryValidationException` with structured errors |
| Manual Dapper `Dialect` config (`ISqlDialectResolver`, `DefaultSqlDialectResolver`) | Auto-detection from the `DbConnection` |
| Dapper `MappingRegistry` / `IMappingRegistry` / `IEntityMapping` / `JoinInfo` | `ModelBuilder` + `IEntityTypeConfiguration<T>` |
| Dapper conventions (`IEntityConvention`, `IForeignKeyConvention`, `IRelationshipConvention`, `Default*`) | Convention-first defaults (now internal) |
| `QueryableAspNetCoreExtensions.FlexQueryAsync` | Provider `FlexQueryAsync` + `[FieldAccess]` filter |
| `FromAgGridJson(string)` / `FromKendoJson(string)` | `JsonElement.ToQueryOptions()` |
| `AgGridQueryOptionsParser` / `AgGridResponseConverter` / `KendoQueryOptionsParser` | `ToQueryOptions()` / `ToAgGridServerSideResponse()` extensions |
| `UseSplitQuery` option | Split-query include hydration is now internal behavior |
| Public caches (`ExpressionCache`, `ParserCache`, `ProjectionExpressionCache`) | Internal caching (`FlexQueryCacheSettings` remains public) |
| Public helpers (`ExpressionBuilder`, `QueryBuilder`, `ProjectionOptimizer`, `GovernanceValidator`, `DynamicTypeBuilder`, `SelectTreeBuilder`, `ExpressionPrinter`, `ExpressionTreeVisualizer`, `ProjectionMetadata*`) | Not replaced — implementation detail |
| `FlexQueryParameters.RawParameters` (public) | Internal — use model binding |

### 4. DSL Grammar: Comma-Separated Conditions Became Single Values

v3 split combined filter conditions on `,`. In v4 a comma after the value is **part of the value**:
`Name:eq:Ann,Salary:gt:1000` now matches a `Name` literally equal to `"Ann,Salary:gt:1000"`, and
`;` in a filter is rejected outright. Rewrite multi-condition filters to join with `&` / `|` /
`AND` / `OR`. This is the one *silent* grammar change in the migration — grep stored/shared filter
strings. (`AND`/`OR` are also reserved and cannot appear as unquoted values.)

### 5. Aggregates and HAVING Tightened

Aggregates are parsed only from the dedicated `aggregate` parameter (function-first grammar);
wildcard aggregates are rejected; aliases must be non-empty. Every aggregate referenced in `having`
must be declared in `aggregate` (replacing v3.1.1's alias-integrity rule), and `having` without
`groupBy` is rejected. GROUP BY projection/sort rules now run through `GroupedSortValidator`
consistently across providers.

### 6. Paging/Parameter Validation Now Throws

```http
page=abc      → QueryParseException: not a valid page number
pageSize=-5   → QueryParseException: not a valid page size
distinct=x    → QueryParseException: not a valid distinct value
```

Out-of-range values (e.g. `page=0` or over-sized `pageSize`) are clamped within
1–`MaxPageSize` (default 1000) instead of erroring. `QuerySyntax` values are validated strictly
across all parameters.

### 7. Unified Exception Hierarchy

All query errors derive from `FlexQueryException`:

```csharp
catch (QueryValidationException ex) { ... }   // field access violations
catch (QueryParseException ex) { ... }        // malformed parameters
catch (FlexQueryException ex) { ... }         // safety net for all FlexQuery errors
```

### 8. Adapter Entry Points

AG Grid / Kendo `From*Json(string)` methods are replaced by `JsonElement.ToQueryOptions()`
extensions (null-safe), and the standalone parser/converter classes are removed in favor of
`ToQueryOptions()` / `ToAgGridServerSideResponse()` extension methods.

### 9. Target Frameworks

`net7.0` targets are dropped; packages build for `net6.0;net8.0;net10.0`, and
`FlexQuery.NET.OpenApi` for `net9.0;net10.0`.

---

## Security / Governance Changes

- Governance members keep their names but move to `QueryGovernanceOptions`;
 `DapperQueryOptions` now derives from it as well.
- `[FieldAccess]` gains `AllowedIncludes`; the attribute class and filter become sealed.
- Every include block level (including nested branches) is governed by `AllowedIncludes`; DTO public
 surfaces and navigation projections require their include paths via new validation rules
 (empty include lists rejected, unknown paths rejected recursively).
- Validation pipeline extended with include-cardinality/HAVING/grouping/keyset rules and a structured
 `ValidationResult` / `ValidationError` model; non-strict mode no longer calls
 `ValidateOrThrow`.

---

## Provider Behavior Changes

| Provider | Change |
|---|---|
| EF Core | Include hydration composed as EF Core filtered includes (provider decides SQL shape); include blocks carry per-branch filter/sort/take; grouped queries execute through a dedicated grouped executor; keyset paging wired end-to-end |
| Dapper | Dialect auto-detection from `DbConnection`; DTO-aware SQL generation with nested TypeMap field rewrites; include-only joins excluded from the COUNT query; SQL execution logging with DECLARE scripts; keyset paging with shared cursor validation |
| Both providers | Trailing `CancellationToken` on all async overloads; `QueryResult` structure stays field-identical across providers |

---

## Performance and Internals

- **Public-surface enforcement + namespace reorganization** (133 refactor commits): sort,
 projection, keyset, and group builders enforce public surfaces; the former monolithic model
 namespaces split (`Options`, `Execution`, `Paging`, serialization under `FlexQuery.NET.*`) while
 internals are hidden from consumers.
- **Dapper include materialization** takes a direct projection path and streams simple includes;
 projections are strongly typed via the internal dynamic type builder and respect configured JSON
 naming transformers.
- **Cache keys** updated to include `Sort`/`Take` on selection nodes, preventing stale shape reuse.
- **Reflection/caches** from v3 retained, now internalized.

---

## Notable Fixes

**Relationship queries / Dapper**
- Include-block filter/sort fields rewritten through nested TypeMaps instead of leaking entity-only names.
- Include-only joins excluded from the total-count query (correct `TotalCount` with filtered includes).
- Principal key resolved from entity mapping rather than hard-coded `Id`.
- Include validation walks the whole tree: relationship cardinality per level, unknown paths at
 any nesting depth, and duplicates are reported precisely.

**EF Core**
- Relationship query blocks are applied inside a single include-tree pass; keyset paging supported in the EF executor.
- Grouped-query notification path guards a null execution context.

**Parsers / Validation**
- OR logical operator corrected in `FqlHavingAstParser`.
- Improved error position reporting for HAVING and FQL safety validation; `*` token position
 reported in aggregate-field errors.
- Nested select aliases preserved; mixed flat/nested select syntax allowed.
- `FlexOptions.QuerySyntax` wired through to the parser via global syntax registration.
- Aggregate alias resolution uses the actual declared alias; Dapper/FQL/Kendo/AgGrid consumers
 moved to the `AggregateFunction` enum comparisons.

---

## Behavioral Changes

| Change | Impact | Mitigation |
|---|---|---|
| DSL comma becomes value text | Multi-condition filters stored as `a:x,b:y` behave differently | Join conditions with `&`/`|`/`AND`/`OR` |
| Paging param strictness | Previously tolerated malformed `page`/`pageSize`/`distinct` now throws | Send well-formed values; catch `QueryParseException` |
| Out-of-range paging clamped 1–1000 | Oversized `pageSize` silently clamped instead of erroring | Design around `MaxPageSize` |
| `Configure` freeze after first query | Late DI-style reconfiguration throws | Configure at startup |
| `CaseInsensitive` removed | String comparisons follow provider collation | Handle case at the provider/collation level |
| Semicolon rejected in DSL filter values | Legacy filters using `;` as separator fail | Restructure to `&`/`|` |
| `AND`/`OR` reserved tokens | Unquoted values equal to keywords rejected | Quote values: `name:eq:"AND"` |
| Aggregates only via `aggregate` param | `select=sum(Total)` no longer parses | Migrate to `aggregate=sum:Total` |
| HAVING requires declared aggregates | v3 alias-integrity escapes now rejected | Declare all referenced aggregates |

---

## Test Coverage

- **2,726 tests (2,726 passing, 0 skipped) on net8.0** — up from 888 total at v3.1.1, plus 25 OpenAPI schema tests.
- New coverage areas from the 65 test-side commits since v3.1.1:
 - Typed DTO: Dapper include boundary and `ForMember` nested select projection; DTO include-graph
   boundary semantics; navigation expansion.
 - Provider semantics: Dapper integration suite (split-query take preservation over include blocks, FQL include-block
   filter/sort/take), parent-preserving collection include semantics, EF/Dapper grouped executor
   behavior across the shared fixtures (navigation properties, nullable dates).
 - Validation: include cardinality-syntax and error-position tests, aggregate alias semantics,
   HAVING rule error codes, public DTO surface enforcement.
 - Serialization: EF ↔ Dapper projected-JSON naming-policy parity; DI registration unit tests for
   `IServiceCollection` extensions.

---

## Documentation

- New **docs-v4** site built with Next.js + Fumadocs (guides, concepts, migration, recipes,
 troubleshooting), plus the v3 static mirror for legacy pages.
- Root and per-package READMEs updated for the v4 API surface.

---

## Prerelease History

| Tag | Date |
|---|---|
| `v4.0.0-preview.1` | 2026-07-12 |
| `v4.0.0-preview.2` | 2026-07-14 |
| `v4.0.0-preview.3` | 2026-07-18 |
| `v4.0.0-beta1.0` | 2026-09-08 |
| `v4.0.0-rc.1` | 2026-09-10 (final code cutoff) |

---

## Upgrading

```bash
dotnet add package FlexQuery.NET --version 4.0.0
dotnet add package FlexQuery.NET.EntityFrameworkCore --version 4.0.0
dotnet add package FlexQuery.NET.Dapper --version 4.0.0
dotnet add package FlexQuery.NET.AspNetCore --version 4.0.0
dotnet add package FlexQuery.NET.Diagnostics --version 4.0.0
dotnet add package FlexQuery.NET.Parsers.Fql --version 4.0.0
dotnet add package FlexQuery.NET.Parsers.MiniOData --version 4.0.0
dotnet add package FlexQuery.NET.Adapters.AgGrid --version 4.0.0
dotnet add package FlexQuery.NET.Adapters.Kendo --version 4.0.0
dotnet add package FlexQuery.NET.OpenApi --version 4.0.0
```

Migration checklist (details in [Migrate from v3.1.1 to v4](https://github.com/peterjohncasasola/FlexQuery.NET/tree/main/docs-v4/content/docs/migration/v3-to-v4.mdx#migration-steps)):

1. Update package references (`Parsers.Jql` → `Parsers.Fql`; add `OpenApi` if used).
2. Replace DI registration with `Fql.Register()`, `MiniOData.Register()`,
  `FlexQueryEFCore.Configure()`, `FlexQueryDapper.Configure()`.
3. Replace `FilteredIncludes` (and preview `expand`) usage with `include(...)` blocks.
4. For Dapper: define the entity model via `options.Model` (tables, keys, relationships).
5. Move aggregates out of `select` into `aggregate`; verify `having` references declared aggregates.
6. Replace removed exception types with `QueryValidationException` handling.
7. Remove `CaseInsensitive` and JSON/Indexed/Generic syntax usage.
8. Replace `FromAgGridJson`/`FromKendoJson` with the `JsonElement` overloads.
9. Re-run test suites — paging validation and HAVING enforcement are stricter.