FlexQuery.NET.Parsers.Jql 3.0.4

• net10.0

  • FlexQuery.NET (>= 3.0.4)
  • Microsoft.Extensions.DependencyInjection.Abstractions (>= 10.0.0)

• net6.0

  • FlexQuery.NET (>= 3.0.4)
  • Microsoft.Extensions.DependencyInjection.Abstractions (>= 6.0.0)

• net8.0

  • FlexQuery.NET (>= 3.0.4)
  • Microsoft.Extensions.DependencyInjection.Abstractions (>= 8.0.0)

# FlexQuery.NET v3.0.4 Release Notes

**Release date:** 2026-06-24

## Overview

v3.0.4 closes the default projection governance gap, fixes a SelectTree governance bypass vulnerability, adds unified grouped sort validation across all providers, and provides a governance test coverage audit.

**Governance:** When no explicit `Select` is specified, the system now injects a governed default projection instead of returning all entity fields. Introduces `DefaultProjectionRule`, wildcard expansion at injection time, `GovernanceValidator` startup checks, and role-based auto-projection. **Critical fix:** SelectTree projection paths are now recursively validated against all governance rules, closing a bypass that allowed blocked fields through tree-based query entry points.

**Grouped Sort Validation:** All providers now share a centralized `GroupedSortValidator` that ensures grouped queries (GroupBy + Aggregates + Sort + Paging) produce valid SQL with deterministic ordering — removing invalid sorts, resolving aggregate field names to aliases, and injecting group-key fallbacks.

**Critical Security Fix — SelectTree Governance Bypass:** `FieldAccessValidator` previously validated only `options.Select` (the flat field list). The `options.SelectTree` projection path — used by `JsonParser.Parse`, all adapter parsers (AG Grid, Kendo, OData), and the JQL/DSL parsers — was never inspected. Blocked fields (`SSN`, `Orders.Total`) could survive validation when projected through SelectTree, effectively bypassing all governance rules: `BlockedFields`, `AllowedFields`, `SelectableFields`, `RoleAllowedFields`, `FieldAccessResolver`, and `MaxFieldDepth`.

---

## What's New

### 1. Default Projection Injection

**Problem:** When clients sent a query without `$select` (or equivalent), the projection builder returned all entity fields regardless of governance configuration. This meant `AllowedFields`, `BlockedFields`, and `RoleAllowedFields` were effectively ignored for unprojected queries, leaking sensitive fields by default.

**Fix:** A new `DefaultProjectionRule` runs first in the validation pipeline. When no explicit projection (`Select`, `SelectTree`, or `HasProjection()`) is present, it injects a default `Select` using the following priority:

| Priority | Source | Example |
|----------|--------|---------|
| 1 | `SelectableFields` | `{ "Id", "Name", "Email" }` |
| 2 | `RoleAllowedFields` (current role) | Admin → `{ "Id", "Name", "Salary" }` |
| 3 | `AllowedFields` | `{ "Id", "Name", "Email" }` |
| 4 | Entity metadata minus `BlockedFields` | `{ "Id", "Name", ... all scalar props }` |

If none of the above sources are configured, the rule skips injection (preserving the existing behavior of selecting all fields).

**Effect:** Unprojected queries now automatically respect field governance — blocked fields are excluded, allowed fields are whitelisted, and role-based restrictions take effect by default.

### 2. Wildcard Pattern Expansion

`DefaultProjectionHelper.ExpandWildcardFields` expands wildcard patterns like `Orders.*` at injection time by recursively walking navigation properties on the target entity type via reflection.

**Example:**

```csharp
SelectableFields = new HashSet<string> { "Id", "Name", "Orders.*" }
```

Expands to something like:

```
{ "Id", "Name", "Orders.OrderId", "Orders.Total", "Orders.Status", "Orders.CreatedAt" }
```

This ensures wildcard-based governance rules are resolved eagerly, before the projection builder receives the field list.

### 3. Non-Strict Re-Apply of Default Projection

In non-strict mode (`StrictFieldValidation = false`), if an explicit `Select` is provided but every field is removed by validation (e.g., the user selected fields not in `AllowedFields`), the default projection is re-injected automatically.

**Before:** An empty `Select` after non-strict validation resulted in no projection.
**After:** The system falls back to the governed default projection, ensuring the response is never empty or broken.

### 4. Role-Based Auto-Projection

`RoleAllowedFields` now serves as a valid source for the default projection. When no `SelectableFields` are configured but `RoleAllowedFields` contains entries for the `CurrentRole`, those fields are used as the default projection.

```csharp
opts.CurrentRole = "manager";
opts.RoleAllowedFields = new Dictionary<string, HashSet<string>>
{
   ["admin"] = new() { "Id", "Name", "Email", "Salary", "InternalNotes" },
   ["manager"] = new() { "Id", "Name", "Email", "Department" },
   ["user"] = new() { "Id", "Name", "Email" }
};

// When no Select is specified, manager inherits: { "Id", "Name", "Email", "Department" }
```

### 5. Paging Fallback Sort Respects Governance

`QueryBuilder.ApplyPaging` now uses the first field from `options.Select` as the fallback sort key (before falling back to `Id`, then `Key`, then the first scalar property). This ensures deterministic pagination respects the governed projection.

**Before:** Fallback was always `Id` → first property, regardless of projection.
**After:** Fallback prefers the first selected field, which is governed by the default projection injection.

### 6. Governance Configuration Validation

New `GovernanceValidator.ValidateConfiguration()` enables startup-time validation of governance config consistency:

| Check | Validates |
|-------|-----------|
| `BlockedFields` ∩ `AllowedFields` | No field can be both blocked and allowed |
| `SelectableFields` ⊆ `AllowedFields` | Selectable fields must be a subset of allowed fields |
| `FilterableFields` ⊆ `AllowedFields` | Filterable fields must be a subset of allowed fields |
| `SortableFields` ⊆ `AllowedFields` | Sortable fields must be a subset of allowed fields |
| `GroupableFields` ⊆ `AllowedFields` | Groupable fields must be a subset of allowed fields |
| `AggregatableFields` ⊆ `AllowedFields` | Aggregatable fields must be a subset of allowed fields |

Call at application startup:

```csharp
GovernanceValidator.ValidateConfiguration(execOptions);
```

Throws `InvalidOperationException` with a descriptive message when configuration is inconsistent.

### 7. New Test Coverage

- 24 new unit tests in `FieldSecurityTests.cs` covering:
 - `DefaultProjectionRule` priority (SelectableFields > RoleAllowedFields > AllowedFields > fallback)
 - `DefaultProjectionRule` skips when explicit `Select` is provided
 - `DefaultProjectionRule` skips when no governance is configured
 - `DefaultProjectionRule` excludes `BlockedFields`
 - `RoleAllowedFields` as default projection source
 - `RoleAllowedFields` with `AllowedFields` intersection
 - `DefaultProjectionRule` with grouped queries (skips injection)
 - Wildcard expansion via `ExpandWildcardFields`
 - Non-strict re-apply after AllowedFields removal
 - Strict mode throws even with no explicit Select
 - `GovernanceValidator` config validation (7 tests)
- 7 new SelectTree governance tests in `SecurityGovernanceEfCoreIntegrationTests.cs`
- 2 new tests in `PagingTests.cs` for fallback sort behavior

### 8. Documentation Updates

- Security & Governance guide updated with default projection, wildcard expansion, and config validation sections

### 9. Grouped Sort Validation (Cross-Provider)

**Problem:** Grouped queries (GroupBy + Aggregates + Sort + Paging) had inconsistent and broken sort behavior across the EF Core and Dapper providers:

| Scenario | EF Core (before) | Dapper (before) |
|---|---|---|
| Sort by non-projected field (`Id` with `GroupBy=[Category]`) | Silently dropped (nondeterministic order) | Generated `ORDER BY [Id]` — column not in GROUP BY, SQL error at runtime |
| Sort by aggregate source field (`Price` when `AVG(Price) AS priceAvg`) | Resolved to `priceAvg` (correct) | Generated `ORDER BY [Price]` — column not in GROUP BY, SQL error |
| All sorts invalid | No ORDER BY — nondeterministic | Generated invalid SQL |
| Paging without sort | Injected group-key sort (deterministic) | No ORDER BY — nondeterministic paging |

**Fix:** A new `GroupedSortValidator` in `FlexQuery.NET.Core` centralizes grouped sort logic and is shared by both providers:

1. **Valid sort fields:** Group-key fields and aggregate aliases are kept.
2. **Aggregate field → alias resolution:** Sorting by `Price` when `AVG(Price) AS priceAvg` exists resolves to `priceAvg`.
3. **Invalid sorts removed:** Fields not in the grouped projection (e.g., `Id` when grouping by `Category`) are silently removed.
4. **Fallback injection:** If all sorts are invalid or empty, a deterministic fallback by the first group-key ascending is injected.

**Provider changes:**

- **EF Core:** Replaced inline `BuildGroupedSorts<TShape>` / `ResolveGroupedSortField` (which depended on reflection over the dynamic TShape type) with a call to `GroupedSortValidator.Validate`, removing the generic parameter dependency.
- **Dapper:** `SqlTranslator.Translate` now validates sorts through `GroupedSortValidator.Validate` when the query has `GroupBy`, preventing invalid SQL and ensuring deterministic paging.

**Behavior after fix:**

| Scenario | EF Core | Dapper |
|---|---|---|
| Sort by group key (`Category` with `GroupBy=[Category]`) | `ORDER BY Category` | `ORDER BY "Category"` |
| Sort by aggregate alias (`priceAvg` with `AVG(Price) AS priceAvg`) | `ORDER BY priceAvg` | `ORDER BY "priceAvg"` |
| Sort by aggregate source field (`Price` with `AVG(Price) AS priceAvg`) | `ORDER BY priceAvg` | `ORDER BY "priceAvg"` |
| Sort by non-projected field (`Id` with `GroupBy=[Category]`) | Removed, fallback to `Category` | Removed, fallback to `"Category"` |
| All sorts invalid | Fallback to first group key | Fallback to first group key |
| Paging without sort | Injects group-key sort | Injects group-key sort |

### 10. Grouped Query Contract Documentation

New architecture document at `docs/architecture/grouped-query-contract.md` defining the expected behavior for grouped projections across both providers, including:

- Valid vs invalid sort fields
- Aggregate field → alias resolution
- Silent removal of invalid sorts
- Group-key fallback for deterministic paging
- Provider-specific code paths and risks
- Recommendations for future alignment

### 11. New Test Coverage

- 6 new grouped query behavior tests in `GroupedQueryBehaviorTests.cs` across EF Core and Dapper providers (28 total grouped query tests)
- Covers: invalid sort removal, aggregate alias sort, aggregate field → alias resolution, group key sort, and paging without sort
- Existing Dapper grouped SQL tests preserved for regression detection

### 12. SelectTree Governance Validation

**Vulnerability:** `FieldAccessValidator.Validate()` iterated `options.Select` (flat list) to check each field against governance rules, but completely ignored `options.SelectTree`. Since all JSON/DSL parser paths (`JsonParser.Parse`, `SelectTreeBuilder.ParseJsonSelect`, OData adapter, Kendo adapter, AG Grid adapter, JQL parser) populate `SelectTree` rather than the flat `Select` list, every governance rule was effectively bypassed for these entry points.

The flat `Select` path was secure — only SelectTree was vulnerable.

**Fix:** Added `ValidateSelectTree()` — a recursive method that walks every node in the SelectTree, builds dot-notation field paths (e.g., `Orders.Total`), and runs the existing `CheckAccess()` pipeline against each resolved field:

| Check | Applied? |
|-------|----------|
| `BlockedFields` | ✓ — each node path matched against blocked patterns |
| `AllowedFields` | ✓ — each node path matched against allowed patterns |
| `SelectableFields` | ✓ — checked with `QueryOperation.Select` |
| `RoleAllowedFields` | ✓ — role-based match per node |
| `FieldAccessResolver` | ✓ — custom resolver invoked per node |
| `MaxFieldDepth` | ✓ — path depth checked per node |
| `IncludeAllScalars` | ✓ — wildcard expands to scalar fields of the CLR type; each scalar validated individually |

**Strict mode behavior:** `ValidateSelectTree` calls `CheckAccess` which throws `QueryValidationException` on the first violation, rejecting the entire query.

**Non-strict mode behavior:** Violating child nodes are removed from the tree. When `IncludeAllScalars` is set and some scalars are blocked, the wildcard is expanded to explicit children (only the allowed ones). If the tree becomes empty after pruning, it is nullified and the governed default projection is injected (matching the flat `Select` behavior).

**Affected entry points (all now governed):**

| Entry Point | Before | After |
|---|---|---|
| `JsonParser.Parse(json)` | Bypass | Blocked |
| `JsonParser.Parse(JsonElement)` | Bypass | Blocked |
| `SelectTreeBuilder.ParseJsonSelect` | Bypass | Blocked |
| OData `$select` via `ODataQueryOptionsParser` | Bypass | Blocked |
| Kendo adapter `SelectTree` projection | Bypass | Blocked |
| AG Grid adapter `SelectTree` projection | Bypass | Blocked |
| JQL/DSL parser `SelectTree` projection | Bypass | Blocked |
| Direct `options.SelectTree = ...` in adapter code | Bypass | Blocked |
| Flat `options.Select = [...]` (always secure) | Already secure | Unchanged |

**Files changed:**

- `src/FlexQuery.NET/Validation/Rules/FieldAccessValidator.cs` — Added `ValidateSelectTree()` recursive validation method and `ResolveSelectTreeChildType()` type-resolution helper. Wired into `Validate()` after the flat `Select` block.
- `src/FlexQuery.NET/Models/SelectionNode.cs` — Added `ClearIncludeAllScalars()` and `RemoveChild(string)` for non-strict tree mutation.

**Vulnerability severity:** **High** — an authenticated client could explicitly request blocked fields through any SelectTree-based query adapter and have them projected into the response without governance enforcement. No data exfiltration was possible through the flat `Select` path.

### 13. Governance Test Coverage Audit

A complete governance coverage audit was performed post-fix, identifying 10 remaining gaps across the test suite:

| Priority | Gap | Description |
|---|---|---|
| High | SelectTree + `SelectableFields` | No test verifies `SelectableFields` restricts a SelectTree projection |
| High | SelectTree + `RoleAllowedFields` | No test for role-based field blocking in SelectTree |
| High | SelectTree + `IncludeAllScalars` + NonStrict | Non-strict expansion to explicit children untested |
| High | SelectTree + `FieldAccessResolver` | Custom resolver never tested via SelectTree |
| High | SelectTree + `MaxFieldDepth` | Depth validation never tested via SelectTree |
| Medium | SelectTree + `FieldMappings` | Field mapping translation has zero test coverage across all paths |
| Medium | `FilterableFields` explicit validation | No test filtering by a field in/not-in FilterableFields |
| Medium | `SortableFields` explicit sort validation | No sort-by-field-not-in-SortableFields test |
| Low | NonStrict SelectTree deep cleanup | Only root-level removal tested, not nested child pruning |
| Low | `RoleAllowedFields` + SelectTree + NonStrict | Non-strict role validation untested for SelectTree |

These gaps represent untested but **functional** code paths — the underlying `CheckAccess()` pipeline is shared across all validation paths, so the same governance rules apply even when not explicitly tested. The audit is a test-coverage improvement target, not a vulnerability disclosure.

### 14. New Test Coverage

- 7 new SelectTree governance enforcement tests in `SecurityGovernanceEfCoreIntegrationTests.cs`:
 - `SelectTree_Blocks_BlockedField_Strict` — simple field via SelectTree
 - `SelectTree_Blocks_NotInAllowedFields_Strict` — whitelist enforcement
 - `SelectTree_Blocks_NestedBlockedField_Strict` — navigation child field
 - `SelectTree_Blocks_IncludeAllScalarsWithBlocked_Strict` — wildcard expansion
 - `FlatSelect_CorrectlyBlocks_BlockedField` — control test
 - `SelectTree_Validation_Blocks_BlockedField_Strict` — validation-level test
 - `SelectTree_Removes_BlockedField_NonStrict` — non-strict tree mutation
- **Full test suite:** 773 tests passing, zero regressions
- **Security governance suite:** 62 field security tests + 40 security governance integration tests = 102 governance-related tests

---

## Migration Guide

### SelectTree Governance Enforcement

All `SelectTree` projection fields are now validated against governance rules. This is transparent for most users — blocked fields are rejected in strict mode and removed in non-strict mode, just like flat `Select` fields.

If you programmatically construct `options.SelectTree` with fields that conflict with governance rules, those queries will now fail in strict mode. To inspect what's allowed:

```csharp
// Before — SelectTree bypassed governance:
options.SelectTree = new SelectionNode();
options.SelectTree.GetOrAddChild("SSN");
var result = options.Validate(typeof(Customer), execOptions);  // passed even with BlockedFields=["SSN"]

// After — SelectTree is validated:
options.SelectTree = new SelectionNode();
options.SelectTree.GetOrAddChild("SSN");
var result = options.Validate(typeof(Customer), execOptions);  // throws if StrictFieldValidation=true
```

### Using Default Projection

No code changes required — the default projection is injected automatically. If you were relying on `Select` being `null` to return all fields, note that governed queries now return only the allowed fields by default (behavioral change).

To opt out of auto-projection, set an explicit `Select`:

```csharp
options.Select = new List<string> { "Id", "Name", "Email", "Description" };
```

If no governance is configured (`AllowedFields`, `BlockedFields`, `SelectableFields`, `RoleAllowedFields` are all empty), default projection injection is skipped entirely and the previous behavior is preserved.

### Using GovernanceValidator

Call during service registration:

```csharp
// At startup
var opts = new QueryExecutionOptions
{
   AllowedFields = new HashSet<string> { "Id", "Name" },
   BlockedFields = new HashSet<string> { "Id" }  // ❌ Conflict!
};
GovernanceValidator.ValidateConfiguration(opts);  // Throws InvalidOperationException
```

### Grouped Sort Behavior (Dapper)

**Dapper queries with GroupBy and invalid sorts** now automatically correct their ORDER BY instead of generating broken SQL. If you were intentionally sorting grouped queries by entity fields not in the GROUP BY, those sorts will now be silently removed and replaced with a group-key fallback.

**Dapper queries with GroupBy and paging but no sort** now receive a deterministic ORDER BY (first group key ascending). Previously this generated no ORDER BY, producing nondeterministic paging. No code changes required — the fix is automatic.

### EF Core Grouped Sort Resolution

EF Core's grouped sort resolution is unchanged in observable behavior (invalid sorts were already silently dropped, aggregate fields already resolved to aliases). The internal implementation now uses the shared `GroupedSortValidator` instead of reflection over the dynamic projection type.

---

## Upgrading

Update all relevant packages to v3.0.4:

```bash
dotnet add package FlexQuery.NET --version 3.0.4
dotnet add package FlexQuery.NET.Adapters.AgGrid --version 3.0.4
dotnet add package FlexQuery.NET.Dapper --version 3.0.4
```

No breaking changes in this release. Queries with an explicit `Select` are unaffected. Unprojected queries with governance configuration will now automatically respect field-level security settings. Dapper grouped queries with invalid or missing sorts now produce valid SQL with deterministic paging. **Full test suite: 773 tests passing.**