PanoramicData.SoundPrint 1.0.1

PanoramicData.SoundPrint

Landmark-based audio fingerprinting for .NET: build, store and query fingerprint databases from broadcast audio.

Give it a few seconds of sound and it tells you which known track it is, where in the track it is, how sure it is, and where one song ends and the next begins. It is built for continuous streams (a radio station recorded around the clock) but works equally for single clips.

• Robust. On real music, 5 second clips are identified 98 to 100% of the time through MP3 64 kbit/s, AAC, Opus, radio-style EQ and compression, white noise down to 5 dB SNR, and a voice 5 dB below the music, with about 0.1% false matches.
• Fast and flat. 200 to 425 times real time on one core; a live stream uses under 1% of a core. A streaming fingerprinter allocates nothing, so memory is constant over weeks.
• Honest about partial matches. A window covering the end of one song and the start of the next returns both, each with the span of the query it explains.
• Yours. Written from the published landmark-hashing literature and first principles, MIT licensed, no native dependencies and no decoding: you pass mono PCM.
• Deterministic. The same audio produces bit-identical fingerprints however it is chunked; a golden-hash test passes on both Windows and the Linux CI runner.

Try it in your browser: a Blazor WebAssembly demo runs the whole library locally (microphone, files, and a visual explanation of how it works). See docs/DEMO.md.

Contents

• Installation · Quick start · Streaming · PostgreSQL · Results · How it works · Limits · Documentation · Development

Installation

dotnet add package PanoramicData.SoundPrint            # core library, no database dependency
dotnet add package PanoramicData.SoundPrint.Postgres   # optional PostgreSQL store (EF Core)

Targets .NET 10. Not yet published to NuGet: until the first release, reference the projects from this repository.

Quick start

using PanoramicData.SoundPrint;

var options = new FingerprintOptions();                       // tuned defaults
var store = new InMemoryFingerprintStore(options);

// Index a track you have decoded to mono float samples (any sample rate).
IReadOnlyList<Landmark> landmarks = Fingerprinter.Fingerprint(trackSamples, trackSampleRate, options);
await store.AddTrackAsync("my-song-42", landmarks, cancellationToken);

// Identify a short clip.
var matcher = new Matcher(store, options);
var query = Fingerprinter.Fingerprint(clipSamples, clipSampleRate, options);
var matches = await matcher.MatchAsync(query, null, cancellationToken);

var best = matches[0];
// best.TrackId       "my-song-42"
// best.TrackStart    where in the track the clip starts (best.TrackEnd: where it ends)
// best.QueryStart    the span of the clip this match explains (best.QueryEnd)
// best.Confidence    0 to 1, calibrated; best.Votes and best.Score give the evidence

Track ids are opaque strings you own; the library knows nothing about songs, artists or radio. Removing a mistaken label is await store.RemoveTrackAsync(id, ct), and adding an existing id replaces it.

Streaming a live feed

var fingerprinter = new Fingerprinter(22050, options);
var window = new RollingLandmarkWindow(TimeSpan.FromSeconds(10), options);
var chunks = 0;

foreach (var chunk in incoming.SelectMany(a => a.Chunk(22050)))
{
	fingerprinter.Push(chunk, window);

	// Every five seconds, match what has been heard in the last ten.
	if (++chunks % 5 == 0)
	{
		var matches = await matcher.MatchAsync(window.Snapshot(), null, cancellationToken);
		last = matches.Count > 0 ? matches[0] : last;
	}
}

Push any chunk size, from one reading thread per stream. Output lags input by about two to three seconds (peak lookahead, one density block, and the pairing zone). All times are on the stream's clock; add the UTC time at which the stream started to get wall-clock times. See docs/USAGE.md for more, and docs/INTEGRATION.md for a recording-archive workflow with labelling, boundaries and corrections.

PostgreSQL store

using PanoramicData.SoundPrint.Postgres;

var store = PostgresFingerprintStore.Create("Host=db;Database=soundprint;Username=...;Password=...", options);
await store.InitializeAsync(cancellationToken);   // applies migrations; refuses a database built with different parameters

Tracks are written in one transaction with binary COPY; lookups are index-only scans on fingerprints (hash) INCLUDE (track, frame). The matcher works identically over either store.

Results

All figures below are measured by commands in this repository; methodology, caveats and the full tables are in docs/BENCHMARKS.md.

Accuracy on real music

480 Creative Commons tracks in the database (Free Music Archive; sources and licences in benchmarks/SOURCES.md, audio not committed), 120 more held out as "absent" queries, 300 queries per cell. Recall means the right track is the best match and clears the recommended threshold.

• Minimum clip: about 5 seconds for reliable identification under broadcast-style degradation; 3 seconds works for clean or codec-only audio.
• Recommended threshold: a match score of 19.6, at which the pooled false-match rate was at most 0.1%. The default MatchOptions.MinConfidence of 0.5 corresponds to a score of 20.7, slightly stricter. Raise it (0.9 and up) when a wrong label is costly; lower it to gather candidates for review.
• Position: 98.4% of identified queries also had the matched position within 0.4 s of the true one.
• Song boundaries: a 10 s window straddling two songs found both in 98% of trials; the boundary was within a median of 0.06 s (hard cut) or 0.16 s (one-second crossfade).

Speed, memory, size and latency

For capacity planning: 100,000 tracks of 3.5 minutes is about 5,800 hours of audio, roughly 55 GB in memory or 320 GB in PostgreSQL at real-music density. Storage in PostgreSQL is the first thing to optimise if that matters.

How it works

Spectral peaks are paired into landmarks of (frequency 1, frequency difference, time difference) plus the time of the first peak, stored as hash -> (track, frame). A query's landmarks are looked up and each hit votes for the offset between query and track; a real match concentrates its votes on one offset, coincidences scatter. The vote is scored against the level of coincidences the same track received elsewhere, and the score is mapped to a calibrated confidence. Full description: docs/ARCHITECTURE.md. The algorithm follows A. Wang, "An Industrial-Strength Audio Search Algorithm" (2003), written from the paper and first principles with no code taken from other fingerprinting libraries.

Parameters were chosen by measurement; each decision and its effect is logged in docs/TUNING.md. The biggest wins were an 11.025 kHz internal sample rate (+20 points of recall on hard queries over 8 kHz) and fewer, stronger peaks per band.

Limits and honest caveats

• Not an audio decoder. Pass decoded mono PCM.
• Speed and pitch changes break matching. Playing a track noticeably faster or slower, or pitch-shifting it, changes the timing and frequencies the hash encodes. Drift of about a percent is tolerated.
• Covers, remixes and live versions are different recordings and will not match the original.
• Very short, very sparse or speech-only audio produces too few landmarks to vote reliably. A voice as loud as the music is the hardest case measured.
• The benchmarks are simulations on a small real sample. 600 Creative Commons tracks, simulated radio processing and a synthetic voice; results on a real station's feed will differ. Real-music results on a larger and more varied corpus have not been measured, and PostgreSQL latency at 100,000 tracks has not been measured.
• Confidence is calibrated to the benchmark's mix of present and absent queries; if almost all your queries are for tracks that are in the database, false matches will be rarer than the benchmark suggests, and if almost none are, rarer still matches deserve a higher threshold.
• Patents. The core landmark-hashing patents (US 6,990,453 and its family) appear to have expired (roughly 2021), but this is from public listings, not legal advice, and other parties' patents were not searched. Take advice before building a commercial service on it. Details in docs/DESIGN.md.

Documentation

Development

dotnet build                                   # warnings are errors
dotnet test --project PanoramicData.SoundPrint.Test                    # unit tests (xUnit v3, skips count as failures)
dotnet test --project PanoramicData.SoundPrint.IntegrationTest         # PostgreSQL in a container; needs Docker, fails (not skips) without it
dotnet run --project PanoramicData.SoundPrint.Demo                     # the browser demo
dotnet run -c Release --project PanoramicData.SoundPrint.Benchmarks -- # lists the benchmark commands

Quality: the core library has 100% line and branch coverage from the unit tests (Microsoft code coverage; settings in coverage.config), CI runs unit and PostgreSQL integration tests with coverage, and all test audio in the repository is synthetic: no recordings, broadcast audio or real fingerprint data are committed. Test projects, the synthetic audio generator, the demo's JavaScript and the benchmark harness are excluded from Codacy analysis (.codacy.yml) because they use seeded System.Random and throwaway SQL that pattern-based security rules flag.

Contributions: see CONTRIBUTING.md and SECURITY.md. Never commit recordings, copyrighted music, credentials or real fingerprint data.

Links

• NuGet: https://www.nuget.org/packages/PanoramicData.SoundPrint (once published)
• Source: https://github.com/panoramicdata/PanoramicData.SoundPrint
• Issues: https://github.com/panoramicdata/PanoramicData.SoundPrint/issues

License

MIT, see LICENSE.