Devlooped.JQSharp 1.0.0

A .NET implementation of the jq filter language for querying and transforming JSON.

Usage

JQSharp exposes a minimal API surface via the Jq static class in the Devlooped namespace.

One-shot evaluation

using System.Text.Json;
using Devlooped;

using var doc = JsonDocument.Parse(
    """
    {"name":"Alice","age":30}
    """);

// Returns an IEnumerable<JsonElement> with the matching results
var results = Jq.Evaluate(".name", doc.RootElement);

foreach (var result in results)
    Console.WriteLine(result); // "Alice"

Parse once, evaluate many times

For hot paths or expressions that are applied to many inputs, parse the expression once into a reusable JqExpression. Parsed expressions are immutable and fully thread-safe, so the same instance can be evaluated concurrently from multiple threads.

using System.Text.Json;
using Devlooped;

// Parse once — this is the expensive step
JqExpression filter = Jq.Parse(".users[] | select(.active) | .name");

// Evaluate cheaply against many inputs
foreach (var json in GetJsonStream())
{
    using var doc = JsonDocument.Parse(json);
    foreach (var name in filter.Evaluate(doc.RootElement))
        Console.WriteLine(name);
}

Transforming JSON

jq's full filter language is available, including pipes, array/object construction, built-in functions, conditionals, and reductions:

using var doc = JsonDocument.Parse(
    """
    {
        "orders": [
            { "id": 1, "total": 42.5, "status": "paid" },
            { "id": 2, "total": 17.0, "status": "pending" },
            { "id": 3, "total": 99.9, "status": "paid" }
        ]
    }
    """);

// Sum paid order totals
var results = Jq.Evaluate(
    """
    [.orders[] | select(.status == "paid") | .total] | add
    """,
    doc.RootElement);

Console.WriteLine(results.Single()); // 142.4

Reshaping objects

Use object construction to project, rename, or combine fields from the input:

using var doc = JsonDocument.Parse(
    """
    {
        "user": { "id": 1, "name": "Alice", "email": "alice@example.com" },
        "role": "admin"
    }
    """);

// Pick and flatten specific fields into a new object
var results = Jq.Evaluate("{id: .user.id, name: .user.name, role: .role}", doc.RootElement);

Console.WriteLine(results.Single()); // {"id":1,"name":"Alice","role":"admin"}

Optional fields and defaults

Use the alternative operator // to supply a fallback when a field is missing or null, and the optional operator ? to silence errors on mismatched types:

using var doc = JsonDocument.Parse(
    """
    [
        { "name": "Alice", "email": "alice@example.com" },
        { "name": "Bob" },
        { "name": "Charlie", "email": "charlie@example.com" }
    ]
    """);

// .email // "n/a" returns the fallback when the field is absent
var results = Jq.Evaluate(
    """
    .[] | {name: .name, email: (.email // "n/a")}
    """,
    doc.RootElement);

foreach (var result in results)
    Console.WriteLine(result);
// {"name":"Alice","email":"alice@example.com"}
// {"name":"Bob","email":"n/a"}
// {"name":"Charlie","email":"charlie@example.com"}

Array operations

Built-in functions like map, select, sort_by, and group_by make it easy to slice and reshape collections:

using var doc = JsonDocument.Parse(
    """
    [
        { "name": "Alice", "dept": "Engineering", "salary": 95000 },
        { "name": "Bob",   "dept": "Marketing",   "salary": 72000 },
        { "name": "Carol", "dept": "Engineering", "salary": 105000 },
        { "name": "Dave",  "dept": "Marketing",   "salary": 68000 }
    ]
    """);

// Group by department and compute average salary per group
var results = Jq.Evaluate(
    "group_by(.dept) | map({dept: .[0].dept, avg_salary: (map(.salary) | add / length)})",
    doc.RootElement);

Console.WriteLine(results.Single());
// [{"dept":"Engineering","avg_salary":100000},{"dept":"Marketing","avg_salary":70000}]

Error handling

Both Jq.Parse and JqExpression.Evaluate throw JqException on invalid expressions or runtime errors:

try
{
    var results = Jq.Evaluate(".foo", doc.RootElement).ToList();
}
catch (JqException ex)
{
    Console.WriteLine($"jq error: {ex.Message}");
}

Streaming JSONL

Jq.EvaluateAsync accepts an IAsyncEnumerable<JsonElement> input and evaluates the filter against each element as it arrives, making it well-suited for processing JSONL (newline-delimited JSON) files or any other streaming JSON source without buffering the entire dataset in memory.

Reading a JSONL file

Use JsonSerializer.DeserializeAsyncEnumerable<JsonElement> to turn a stream of newline-delimited JSON objects into an IAsyncEnumerable<JsonElement>:

using System.Text.Json;
using Devlooped;

// users.jsonl — one JSON object per line:
// {"id":1,"name":"Alice","dept":"Engineering"}
// {"id":2,"name":"Bob","dept":"Marketing"}
// {"id":3,"name":"Charlie","dept":"Engineering"}

using var stream = File.OpenRead("users.jsonl");
var elements = JsonSerializer.DeserializeAsyncEnumerable<JsonElement>(
    stream, topLevelValues: true);

await foreach (var result in Jq.EvaluateAsync(
    """
    select(.dept == "Engineering") | .name
    """,
    elements))
    Console.WriteLine(result);
// "Alice"
// "Charlie"

Reusing a parsed expression across multiple streams

Parse the expression once and pass it to EvaluateAsync to avoid re-parsing on every call:

JqExpression filter = Jq.Parse(
    """
    select(.level == "error") | .message
    """);

foreach (var logFile in Directory.GetFiles("logs", "*.jsonl"))
{
    using var stream = File.OpenRead(logFile);
    var elements = JsonSerializer.DeserializeAsyncEnumerable<JsonElement>(
        stream, topLevelValues: true);

    await foreach (var result in Jq.EvaluateAsync(filter, elements))
        Console.WriteLine(result);
}

Cancellation

Both EvaluateAsync overloads accept an optional CancellationToken:

using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(30));

using var stream = File.OpenRead("large.jsonl");
var elements = JsonSerializer.DeserializeAsyncEnumerable<JsonElement>(
    stream, topLevelValues: true);

await foreach (var result in Jq.EvaluateAsync(".name", elements, cts.Token))
    Console.WriteLine(result);

jq Compatibility

JQSharp targets the jq 1.8 specification and passes the official jq test suite for the supported feature set.

Supported features include:

• Field access, array/object iteration and slicing
• Pipe (|), comma (,), and parentheses for grouping
• Array and object constructors ([], {})
• All built-in functions (map, select, group_by, to_entries, from_entries, env, limit, until, recurse, paths, walk, ascii, format, strftime, debug, …)
• String interpolation ("\(.foo)") and format strings (@base64, @uri, @csv, @tsv, @html, @json, @text, @sh)
• Variables (as $x), destructuring, and user-defined functions (def f(x): …)
• reduce, foreach, label-break, try-catch, ?// alternative operator
• Optional operator (?), path expressions, update (|=`) and assignment operators

Open Source Maintenance Fee

To ensure the long-term sustainability of this project, users of this package who generate revenue must pay an Open Source Maintenance Fee. While the source code is freely available under the terms of the License, this package and other aspects of the project require adherence to the Maintenance Fee.

To pay the Maintenance Fee, become a Sponsor at the proper OSMF tier. A single fee covers all of Devlooped packages.

Sponsors

Learn more about GitHub Sponsors