DataNet.Fuzzy 0.3.0

DataNet

A data-science toolkit for C#/.NET, built on an honest premise:

Don't rewrite Python. Use the .NET ecosystem where it's strong, and write native code only where .NET has a real gap: text (similarity, vectorization, semantic search). All of it with no Python at runtime.

Why

Python dominates data analysis through its ecosystem and its exploratory notebook workflow, not through the language itself. Its performance comes from C/Fortran kernels.

C# brings static typing, real parallelism without a global interpreter lock, safe refactoring, and simple deployment. The only objective reason to stay on Python for this domain was the lack of an equivalent .NET library. DataNet removes that reason.

Two deliverables

1. Native code where there's a gap → the packages below (string distances, vectorization, embeddings, fuzzy matching) — allocation-lean, Span-based, SIMD, zero external dependencies in the core.
2. Migration guides for people coming from Python → docs/migration/, which, for each need (NumPy, pandas, scikit-learn, statsmodels, PyTorch, matplotlib, seaborn), points to the right .NET building block and the pitfalls.

See the four-column migration inventory: it's the project map (use / build / decide).

Targets: .NET 10 (net10.0, all fast paths) and .NET Standard 2.0 (broad reach — also .NET Framework 4.6.1+, Mono, Xamarin, Unity). A single package carries both. See docs/decisions/0001.

Status — every lot of the project brief is delivered

Every building block is oracle-validated against rapidfuzz / jellyfish / textdistance / difflib / scikit-learn / nltk / HuggingFace tokenizers / sentencepiece / numpy / ONNX Runtime (see docs/equivalence.md).

Getting started

dotnet add package DataNet.Text

using DataNet.Text.Distances;

Levenshtein.Distance("kitten", "sitting");             // 3
Levenshtein.NormalizedSimilarity("kitten", "sitting"); // 0.5714…

A runnable version of the above, consuming the packages exactly as you would:

for p in src/DataNet.Text src/DataNet.Embeddings src/DataNet.Fuzzy src/DataNet.Metrics; do
  dotnet pack "$p" -c Release -o ./artifacts
done
dotnet run --project samples/DataNet.Sample -c Release

Full guide: docs/guides/quickstart.md. See also the vectorization, embeddings and fuzzy-matching guides.

Developing

dotnet build                                   # build the solution
dotnet test                                    # replay oracles + property tests
dotnet run -c Release --project bench/DataNet.Text.Benchmarks -- --filter '*Levenshtein*'

The project follows GitHub flow: main is always releasable, and every change arrives through a short-lived branch and a pull request. Branch conventions, the definition of done, the oracle-validation procedure and the analyzer-suppression policy are in CONTRIBUTING.md; release history is in CHANGELOG.md.

Oracle validation

Conformance to Python behavior is proven, not assumed (§4 of the brief). tools/generate_oracles.py freezes a few thousand reference cases from rapidfuzz/jellyfish/etc. into tests/oracles/*.json (versioned). The C# suite replays them with a 1e-9 tolerance. Python is a development-only dependency. See tools/README.md.

Structure

DataNet.slnx
├── src/DataNet.Text/            distances, similarity, tokenizers, vectorizers, stemmers (no dependencies)
├── src/DataNet.Embeddings/      sub-word tokenizers, pooling, SIMD kNN, ONNX inference (ONNX Runtime isolated here)
├── src/DataNet.Fuzzy/           fuzz.*, process.extract, deduplication
├── src/DataNet.Metrics/         confusion matrix, precision/recall/F1, report, ROC-AUC
├── tests/                       xUnit: oracles + properties (one project per module)
├── tests/oracles/               frozen JSON corpora (generated from Python) + a synthetic ONNX model
├── bench/DataNet.Text.Benchmarks/  BenchmarkDotNet
├── tools/generate_oracles.py    reference generation
├── Directory.Build.props        (root); src|tests/Directory.Packages.props (central package management)
├── src/*/Version.props          one version per publishable package (decision 0012)
└── docs/                        guides, equivalence table, decision log

Where a fact belongs

Each document below has one subject; content whose subject is another document's belongs there instead, with a link left behind. The source column is what tells you whether to correct the document itself or something upstream of it.

Publishing

Four NuGet packages are produced: DataNet.Text, DataNet.Embeddings, DataNet.Fuzzy, DataNet.Metrics. Each versions and releases on its own: shared metadata (license, README, repository) lives in Directory.Build.props, while the version is declared per project in src/<Package>/Version.props. DataNet.Fuzzy depends on DataNet.Text as a published package, not as a project reference — see docs/decisions/0012.

GitHub Packages (no nuget.org account needed — uses GitHub's automatic token). Bump the version, then tag it with the package name. The release workflow packs and publishes that package alone:

# 1. edit src/DataNet.Fuzzy/Version.props, commit, merge to main
# 2. tag the released version — <PackageId>/v<Version>
git tag DataNet.Fuzzy/v0.3.0
git push origin DataNet.Fuzzy/v0.3.0

The tag does not set the version; it names which declared version to release. The workflow refuses the job if the tag and Version.props disagree. Repository-wide v* tags are retired — there is no single version left for one to designate.

Step 1 is not optional. Because the tag only confirms the declared version, tagging without bumping first is a tag that agrees with Version.props and names a version the feed already has. The push is then rejected rather than absorbed. The workflows do not pass --skip-duplicate, which used to report that case as a successful release that shipped nothing. Keeping a declared version off the feed is also checked directly in CI by tools/check_version_floor.py.

To consume them, add a source pointing at the owner's feed (with a GitHub token that has read:packages):

dotnet nuget add source "https://nuget.pkg.github.com/CyrilB1531/index.json" \
  --name github --username CyrilB1531 --password <GITHUB_TOKEN>
dotnet add package DataNet.Text

nuget.org uses Trusted Publishing (OIDC, no stored key): run the Publish to nuget.org workflow from the Actions tab, choosing the package and confirming its version. By hand, with an API key, one package at a time:

dotnet pack src/DataNet.Text -c Release -o artifacts
dotnet nuget push "artifacts/DataNet.Text.*.nupkg" \
  --source https://api.nuget.org/v3/index.json --api-key <KEY>

License

Apache-2.0. See NOTICE and THIRD-PARTY-NOTICES.md for attributions. The license choice and the code-provenance rule are documented in docs/decisions/0003-provenance-and-licensing.md.

This repository is not legal advice.