ToolUp.DataSources.Common 0.22.0

Prefix Reserved
dotnet add package ToolUp.DataSources.Common --version 0.22.0
                    
NuGet\Install-Package ToolUp.DataSources.Common -Version 0.22.0
                    
This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package.
<PackageReference Include="ToolUp.DataSources.Common" Version="0.22.0" />
                    
For projects that support PackageReference, copy this XML node into the project file to reference the package.
<PackageVersion Include="ToolUp.DataSources.Common" Version="0.22.0" />
                    
Directory.Packages.props
<PackageReference Include="ToolUp.DataSources.Common" />
                    
Project file
For projects that support Central Package Management (CPM), copy this XML node into the solution Directory.Packages.props file to version the package.
paket add ToolUp.DataSources.Common --version 0.22.0
                    
#r "nuget: ToolUp.DataSources.Common, 0.22.0"
                    
#r directive can be used in F# Interactive and Polyglot Notebooks. Copy this into the interactive tool or source code of the script to reference the package.
#:package ToolUp.DataSources.Common@0.22.0
                    
#:package directive can be used in C# file-based apps starting in .NET 10 preview 4. Copy this into a .cs file before any lines of code to reference the package.
#addin nuget:?package=ToolUp.DataSources.Common&version=0.22.0
                    
Install as a Cake Addin
#tool nuget:?package=ToolUp.DataSources.Common&version=0.22.0
                    
Install as a Cake Tool

ToolUp.DataSources.Common

The vendor-free support layer shared by the ToolUp.DataSources.* IDataSource companions. BCL only — no cloud SDK, no database driver (GP 1). A consumer never references this package directly; it arrives transitively with whichever connector they compose.

Six connectors needed the same five things, and duplicating them six times would have meant six places for the CSV escaping or the credential-fallback order to drift:

Module What it owns
Errors Classifies a connector exception onto the SDK's IngestionError taxonomy — transport-shaped exceptions to SourceUnreachable, everything else to UnexpectedFailure. Errors.guard wraps a body so nothing thrown escapes an IDataSource method, and an already-typed Error passes through unreclassified.
ConnectionScope Typed, failure-explicit reads out of the free-form DataSourceConfig.ConnectionScope map. require / optional / optionalOr / optionalInt / optionalBool / optionalEnum. A missing key is SchemaMismatch naming the key; a present but unparseable value is an error rather than a silent fallback, so a mistyped port cannot quietly become the default.
Credentials The credential thunk. resolve prefers DataSourceCallContext.Credential (which the shipped DataIngestor pre-resolves) and otherwise reads ISecretStore.GetSecret(ScopeId, CredentialKey)on every call, so a rotated secret takes effect without reconstructing the connector. resolveOptional is the same thing for connectors whose auth can legitimately fall back to an ambient provider chain.
SqlIdentifier Identifier safety for the connectors that build catalogue queries by hand: isSafe / require / quoteLiteral. See below.
CredentialJson Parses a JSON-object credential blob into a case-insensitive key → string map, with tryFind accepting several spellings of a key (vendors are not consistent about accessKeyId vs aws_access_key_id).
Csv RFC 4180 emission — the uniform IDataSource.Query wire format for the family. renderValue / escapeField / renderRow / toBytes, plus ofReader to drain a DbDataReader (System.Data.Common is BCL, so the ADO-shaped and API-shaped connectors share it).
TypeMap Native-type-name → coarse ColumnType classification: normalise, the ansi fallback table, and classify to compose a connector's own overrides in front of it.

Why one CSV format for the whole family

IDataSource.Query returns opaque bytes and the ingestor stores them through IDataObjectStore without looking inside, so the format is the connector's to choose. Choosing the same one everywhere means a module that reads one warehouse's output reads all of them.

CSV specifically: ToolUp.Tabular already reads it, and Athena stages result sets to S3 as CSV natively — so for at least one connector the wire format is a re-emission rather than a translation. Types are recovered from GetSchema, not from the payload.

Two details that are load-bearing rather than incidental:

  • Values render invariant-culture. A connector running on a comma-decimal host would otherwise corrupt every number in the payload, silently and unrecoverably. DateTime renders ISO-8601 round-trip ("O"), byte[] base64, bool as true/false, NULL and DBNull as the empty field.
  • No BOM. A UTF-8 BOM appears as a stray character in the first header cell for every naive downstream parser.

Why identifiers are validated rather than parameterised

SqlIdentifier exists because the eight backends across this family do not share one parameter marker — @p, :p, {p:String}, ? — and a connector whose safety depended on getting four markers right in eighteen places is a connector waiting to be wrong.

Catalogue queries therefore interpolate schema and table names, and every interpolated identifier is first validated against ^[A-Za-z_][A-Za-z0-9_$]{0,127}$ and refused as SchemaMismatch if it does not match. Quoted identifiers containing spaces, dots or Unicode are refused rather than escaped — deliberately narrower than what any backend permits. quoteLiteral doubles single quotes on top, as a second and independent line of defence.

Testing

Everything here is a total function or an in-process async, so all of it is covered by the always-on arm of src/ToolUp.DataSources.Tests — no credential, no server, no network.

Product Compatible and additional computed target framework versions.
.NET net10.0 is compatible.  net10.0-android was computed.  net10.0-browser was computed.  net10.0-ios was computed.  net10.0-maccatalyst was computed.  net10.0-macos was computed.  net10.0-tvos was computed.  net10.0-windows was computed. 
Compatible target framework(s)
Included target framework(s) (in package)
Learn more about Target Frameworks and .NET Standard.

NuGet packages (6)

Showing the top 5 NuGet packages that depend on ToolUp.DataSources.Common:

Package Downloads
ToolUp.DataSources.Synapse

Azure Synapse Analytics IDataSource companion for ToolUp.Platform, over the Synapse SQL endpoint (serverless or dedicated) via Microsoft.Data.SqlClient. Three auth modes — SQL login, a supplied AAD access token, or DefaultAzureCredential; INFORMATION_SCHEMA introspection; RFC 4180 CSV query output. Credentials resolved per call through ISecretStore.

ToolUp.DataSources.BigQuery

Google BigQuery IDataSource companion for ToolUp.Platform. Service-account JSON (or application-default) credentials resolved per call through ISecretStore; dataset/table introspection via the BigQuery API; query results returned as RFC 4180 CSV; a maximum_bytes_billed ceiling so a runaway scan fails instead of billing.

ToolUp.DataSources.Snowflake

Snowflake IDataSource companion for ToolUp.Platform, over the official Snowflake.Data ADO.NET driver. Password or key-pair (JWT) auth resolved per call through ISecretStore; per-warehouse / per-schema / per-role routing read from ConnectionScope; INFORMATION_SCHEMA introspection with Snowflake's identifier-case folding; RFC 4180 CSV query output.

ToolUp.DataSources.Athena

AWS Athena IDataSource companion for ToolUp.Platform. Glue-catalogue introspection, start/poll/page async query execution with a connector-enforced timeout, S3 result staging that shares a bucket with ToolUp.Storage.AwsS3, and RFC 4180 CSV output. Credentials resolved per call through ISecretStore, or the AWS default credential chain.

ToolUp.DataSources.Redshift

AWS Redshift IDataSource companion for ToolUp.Platform, over the Redshift Data API (no VPC route or database password needed). Provisioned clusters and Serverless workgroups; submit/poll/page async statement execution with a connector-enforced timeout; catalogue introspection via ListTables/DescribeTable; RFC 4180 CSV output. AWS credentials resolved per call through ISecretStore, or the AWS default credential chain.

GitHub repositories

This package is not used by any popular GitHub repositories.

Version Downloads Last Updated
0.22.0 44 8/27/2026