Apache.Calcite.EntityFrameworkCore.Adapter 1.0.0-pre.148

Apache.Calcite.EntityFrameworkCore.Adapter

An Apache Calcite adapter that executes relational plans as Entity Framework Core LINQ. Register a DbContext as a Calcite schema, and Calcite treats your EF Core model like any other adapter — plan SQL over it, federate it with CSV files, JDBC databases, and in-memory schemas, and join across all of them in one query.

This is the Calcite on EF Core direction. For the other direction — a DbContext whose store is Calcite — see Apache.Calcite.EntityFrameworkCore.

Calcite runs in-process through IKVM: no JDBC driver, no Avatica server, no second process.

dotnet add package Apache.Calcite.EntityFrameworkCore.Adapter

Quick start

using Apache.Calcite.Data;
using Apache.Calcite.EntityFrameworkCore.Adapter;

// Expose the DbSet<T> properties of ProductDbContext as tables in a Calcite schema named "efcore".
// The root schema belongs to the data source, so the context is named once here and every connection
// the data source opens sees it.
using var dataSource = new CalciteDataSourceBuilder("caseSensitive=false")
    .AddEfCoreSchema("efcore", () => new ProductDbContext(connectionString))
    .Build();

using var connection = dataSource.CreateConnection();
connection.Open();

using var cmd = connection.CreateCommand();
cmd.CommandText = @"SELECT ""Id"", ""Name"" FROM ""efcore"".""Product"" WHERE ""InStock"" = TRUE";

using var reader = cmd.ExecuteReader();
while (reader.Read())
    Console.WriteLine($"{reader.GetInt32(0)} {reader.GetString(1)}");

Two details worth knowing up front:

• Create does not register the schema for you — pass the parent schema so the adapter can resolve against it, then add it under the name you want, as above.
• Tables are named after the entity CLR type, not the DbSet property and not the mapped table. A DbSet<Product> Products on a table called Products is the Calcite table "Product".

The factory is called every time the adapter needs a context — once to read the model, and again for each execution — and the context is disposed afterwards, so it must return a fresh, independently usable instance each time rather than a shared one.

Registering from a Calcite model file

EfCoreSchemaFactory wires the same thing up from model JSON, so a schema can be declared in a connection string instead of in code. The operand map takes either dbContextType (an assembly-qualified DbContext subclass with a public parameterless constructor) or dbContextFactory (an assembly-qualified IDbContextFactory), plus an optional rexTranslatorFactory:

{
  "version": "1.0",
  "defaultSchema": "efcore",
  "schemas": [
    {
      "name": "efcore",
      "type": "custom",
      "factory": "Apache.Calcite.EntityFrameworkCore.Adapter.EfCoreSchemaFactory, Apache.Calcite.EntityFrameworkCore.Adapter",
      "operand": { "dbContextType": "MyApp.ProductDbContext, MyApp" }
    }
  ]
}

How it works

EfCoreConvention is a Calcite calling convention. Converter rules in EfCoreRules pull relational nodes into it — Filter, Project/Calc, Join and left join, Aggregate, Sort, Union, Intersect, Minus, Values — and each node's implement builds a LINQ Expression typed as IQueryable<T> over the DbSet<T> it came from. Rex trees become LINQ predicates and projections through RexToLinqTranslator, with SQL operator coverage supplied by a replaceable ISqlOperatorTranslationProvider.

The convention carries a cost multiplier below 1, so where the planner can express an operation either way it prefers pushing it into EF Core — which in turn means EF Core's own provider gets to push it down to the store. Work the convention cannot express stays in Calcite's enumerable convention above it and runs there, on rows the adapter streams out.

Pushing a predicate down means the store answers it, so the store's collation decides how strings compare, where Calcite in-process is always case- and accent-sensitive. =, LIKE and the ordered comparisons on a case-insensitive column therefore match more rows through the adapter than the same SQL would matching in Calcite. That is the trade pushdown is: the store is being asked the question, in the store's own terms.

There is exactly one way out: EfCoreToClrCursorConverter into ClrCursorConvention. A node of that convention has two bodies — Implement, whose open acquires synchronously, and ImplementAsync, whose open awaits — and both evaluate to an opened cursor with Read and ReadAsync(token). The adapter holds only an IQueryable, and reads it one way: through its IAsyncEnumerable, the path behind ToListAsync. Over a provider with real asynchronous I/O that holds no thread, and over one without it costs nothing, so EfCoreCursors has a pair of opens per row shape but one cursor, whose Read waits for its ReadAsync. EF Core's enumerator takes a token only when it is created, so the awaiting open's token is the one it sees; each ReadAsync checks its own token before advancing, and a synchronous Read polls the statement's cancel flag. Everything above the open — translating the subtree, resolving the columns, settling the row format — is shared. From there the bridge converters in Apache.Calcite.Extensions carry the rows onward to whatever convention the rest of the plan needs.

Extending the translation

EfCoreSchema.Create takes an optional IRexToLinqTranslatorFactory (rexTranslatorFactory in a model file), which is the single hook for everything below it. To add SQL functions the default translator does not cover, subclass SqlOperatorTranslationProvider, override Build — calling base.Build to keep the standard mappings — and hand your table to a RexToLinqTranslator from your own factory:

sealed class MyOperators : SqlOperatorTranslationProvider
{
    protected override void Build(Dictionary<SqlOperator, SqlOperatorTranslator> translators)
    {
        base.Build(translators);
        translators[SqlStdOperatorTable.INITCAP] = StaticCall(typeof(MyFunctions), nameof(MyFunctions.InitCap));
    }
}

sealed class MyTranslatorFactory : IRexToLinqTranslatorFactory
{
    public IRexToLinqTranslator Create() => new RexToLinqTranslator(new MyOperators());
}

Each translator delegate receives the already-translated CLR operand expressions and returns the expression that implements the function; StaticCall, InstanceCall, and PropRead cover the common shapes. SqlOperatorTranslationProvider.Default is the built-in table — UPPER, LOWER, CHAR_LENGTH, REPLACE, POSITION, the math operators, and the rest.

An untranslated SQL function is not an error. The planner simply leaves that part of the plan above the convention and evaluates it in Calcite, on rows the adapter feeds it.

Requirements

.NET 10 and EF Core 10. Any EF Core provider works as the underlying store — the adapter only builds IQueryable expressions and lets that provider execute them.

Links

• Repository and full documentation
• Apache.Calcite.EntityFrameworkCore — the EF Core provider
• calcite-dotnet — Apache Calcite for .NET
• Calcite adapter documentation

License

Apache 2.0