NPipeline.Connectors.Snowflake 0.70.1

NPipeline Snowflake Connector

Source and sink nodes for Snowflake in NPipeline pipelines, on NPipeline's shared SQL layer.

About NPipeline

NPipeline is a high-performance, extensible data processing framework for .NET that enables developers to build scalable and efficient pipeline-based applications. It provides a rich set of components for data transformation, aggregation, branching, and parallel processing, with built-in support for resilience patterns and error handling.

Installation

dotnet add package NPipeline.Connectors.Snowflake

Targets .NET 8.0, 9.0 and 10.0.

Features

• Name-based binding: columns bind to members case-insensitively, once per query, through a compiled mapper that reads each value typed at its ordinal. Positional records, init accessors and required members work.
• Strict conversion: a value that does not fit its member is a row error, never a silent default; row errors can fail the read, be skipped, or go to the pipeline's dead-letter sink.
• Transactions per batch by default, so a batch lands whole and a transient failure is retried safely; or one transaction for the whole write.
• Failed batches fail the write, or go to the dead-letter sink with their items.
• Upserts on key columns, and identifiers that are always quoted and escaped.
• Three write strategies: multi-row INSERT statements, one statement per row, or a CSV file staged with PUT and loaded with COPY INTO, written so empty strings stay apart from NULL and a retry never loads a file twice.
• MERGE upserts and UPPER_SNAKE column names by default.

Usage

using NPipeline.Connectors.Snowflake;

var source = SnowflakeConnector.Source<Order>(connectionString, "SELECT * FROM PUBLIC.ORDERS ORDER BY ORDER_ID");
var sink = SnowflakeConnector.Sink<Order>(connectionString, "ORDERS", o => o with { WriteStrategy = SnowflakeWriteStrategy.StagedCopy });

See the Snowflake connector documentation and SQL Connectors: Shared Behaviour for every option.

Related Packages

• NPipeline - Core pipeline framework
• NPipeline.Connectors - Storage abstractions and base connectors
• NPipeline.Extensions.DependencyInjection - Dependency injection integration

License

This package is licensed under the Business Source License 1.1.

Free for non-production use. Production use is free for organizations with 4 or fewer developers and annual revenue of $5M AUD or less. Larger organizations require a commercial license. This license automatically converts to MIT two years after each release.