ToolUp.DataSources.Common
0.22.0
Prefix Reserved
dotnet add package ToolUp.DataSources.Common --version 0.22.0
NuGet\Install-Package ToolUp.DataSources.Common -Version 0.22.0
<PackageReference Include="ToolUp.DataSources.Common" Version="0.22.0" />
<PackageVersion Include="ToolUp.DataSources.Common" Version="0.22.0" />
<PackageReference Include="ToolUp.DataSources.Common" />
paket add ToolUp.DataSources.Common --version 0.22.0
#r "nuget: ToolUp.DataSources.Common, 0.22.0"
#:package ToolUp.DataSources.Common@0.22.0
#addin nuget:?package=ToolUp.DataSources.Common&version=0.22.0
#tool nuget:?package=ToolUp.DataSources.Common&version=0.22.0
ToolUp.DataSources.Common
The vendor-free support layer shared by the ToolUp.DataSources.* IDataSource companions. BCL
only — no cloud SDK, no database driver (GP 1). A consumer never references this package directly;
it arrives transitively with whichever connector they compose.
Six connectors needed the same five things, and duplicating them six times would have meant six places for the CSV escaping or the credential-fallback order to drift:
| Module | What it owns |
|---|---|
Errors |
Classifies a connector exception onto the SDK's IngestionError taxonomy — transport-shaped exceptions to SourceUnreachable, everything else to UnexpectedFailure. Errors.guard wraps a body so nothing thrown escapes an IDataSource method, and an already-typed Error passes through unreclassified. |
ConnectionScope |
Typed, failure-explicit reads out of the free-form DataSourceConfig.ConnectionScope map. require / optional / optionalOr / optionalInt / optionalBool / optionalEnum. A missing key is SchemaMismatch naming the key; a present but unparseable value is an error rather than a silent fallback, so a mistyped port cannot quietly become the default. |
Credentials |
The credential thunk. resolve prefers DataSourceCallContext.Credential (which the shipped DataIngestor pre-resolves) and otherwise reads ISecretStore.GetSecret(ScopeId, CredentialKey) — on every call, so a rotated secret takes effect without reconstructing the connector. resolveOptional is the same thing for connectors whose auth can legitimately fall back to an ambient provider chain. |
SqlIdentifier |
Identifier safety for the connectors that build catalogue queries by hand: isSafe / require / quoteLiteral. See below. |
CredentialJson |
Parses a JSON-object credential blob into a case-insensitive key → string map, with tryFind accepting several spellings of a key (vendors are not consistent about accessKeyId vs aws_access_key_id). |
Csv |
RFC 4180 emission — the uniform IDataSource.Query wire format for the family. renderValue / escapeField / renderRow / toBytes, plus ofReader to drain a DbDataReader (System.Data.Common is BCL, so the ADO-shaped and API-shaped connectors share it). |
TypeMap |
Native-type-name → coarse ColumnType classification: normalise, the ansi fallback table, and classify to compose a connector's own overrides in front of it. |
Why one CSV format for the whole family
IDataSource.Query returns opaque bytes and the ingestor stores them through IDataObjectStore
without looking inside, so the format is the connector's to choose. Choosing the same one
everywhere means a module that reads one warehouse's output reads all of them.
CSV specifically: ToolUp.Tabular already reads it, and Athena stages result sets to S3 as CSV
natively — so for at least one connector the wire format is a re-emission rather than a translation.
Types are recovered from GetSchema, not from the payload.
Two details that are load-bearing rather than incidental:
- Values render invariant-culture. A connector running on a comma-decimal host would otherwise
corrupt every number in the payload, silently and unrecoverably.
DateTimerenders ISO-8601 round-trip ("O"),byte[]base64,boolastrue/false,NULLandDBNullas the empty field. - No BOM. A UTF-8 BOM appears as a stray character in the first header cell for every naive downstream parser.
Why identifiers are validated rather than parameterised
SqlIdentifier exists because the eight backends across this family do not share one parameter marker
— @p, :p, {p:String}, ? — and a connector whose safety depended on getting four markers right
in eighteen places is a connector waiting to be wrong.
Catalogue queries therefore interpolate schema and table names, and every interpolated identifier is
first validated against ^[A-Za-z_][A-Za-z0-9_$]{0,127}$ and refused as SchemaMismatch if it
does not match. Quoted identifiers containing spaces, dots or Unicode are refused rather than escaped
— deliberately narrower than what any backend permits. quoteLiteral doubles single quotes on top, as
a second and independent line of defence.
Testing
Everything here is a total function or an in-process async, so all of it is covered by the always-on
arm of src/ToolUp.DataSources.Tests — no credential, no server, no network.
| Product | Versions Compatible and additional computed target framework versions. |
|---|---|
| .NET | net10.0 is compatible. net10.0-android was computed. net10.0-browser was computed. net10.0-ios was computed. net10.0-maccatalyst was computed. net10.0-macos was computed. net10.0-tvos was computed. net10.0-windows was computed. |
-
net10.0
- FSharp.Core (>= 10.1.300)
- ToolUp.Platform.Core (>= 0.22.0)
- ToolUp.Platform.Server (>= 0.22.0)
NuGet packages (6)
Showing the top 5 NuGet packages that depend on ToolUp.DataSources.Common:
| Package | Downloads |
|---|---|
|
ToolUp.DataSources.Synapse
Azure Synapse Analytics IDataSource companion for ToolUp.Platform, over the Synapse SQL endpoint (serverless or dedicated) via Microsoft.Data.SqlClient. Three auth modes — SQL login, a supplied AAD access token, or DefaultAzureCredential; INFORMATION_SCHEMA introspection; RFC 4180 CSV query output. Credentials resolved per call through ISecretStore. |
|
|
ToolUp.DataSources.BigQuery
Google BigQuery IDataSource companion for ToolUp.Platform. Service-account JSON (or application-default) credentials resolved per call through ISecretStore; dataset/table introspection via the BigQuery API; query results returned as RFC 4180 CSV; a maximum_bytes_billed ceiling so a runaway scan fails instead of billing. |
|
|
ToolUp.DataSources.Snowflake
Snowflake IDataSource companion for ToolUp.Platform, over the official Snowflake.Data ADO.NET driver. Password or key-pair (JWT) auth resolved per call through ISecretStore; per-warehouse / per-schema / per-role routing read from ConnectionScope; INFORMATION_SCHEMA introspection with Snowflake's identifier-case folding; RFC 4180 CSV query output. |
|
|
ToolUp.DataSources.Athena
AWS Athena IDataSource companion for ToolUp.Platform. Glue-catalogue introspection, start/poll/page async query execution with a connector-enforced timeout, S3 result staging that shares a bucket with ToolUp.Storage.AwsS3, and RFC 4180 CSV output. Credentials resolved per call through ISecretStore, or the AWS default credential chain. |
|
|
ToolUp.DataSources.Redshift
AWS Redshift IDataSource companion for ToolUp.Platform, over the Redshift Data API (no VPC route or database password needed). Provisioned clusters and Serverless workgroups; submit/poll/page async statement execution with a connector-enforced timeout; catalogue introspection via ListTables/DescribeTable; RFC 4180 CSV output. AWS credentials resolved per call through ISecretStore, or the AWS default credential chain. |
GitHub repositories
This package is not used by any popular GitHub repositories.
| Version | Downloads | Last Updated |
|---|---|---|
| 0.22.0 | 44 | 8/27/2026 |