Whisper.net 1.5.0

dotnet add package Whisper.net --version 1.5.0                
NuGet\Install-Package Whisper.net -Version 1.5.0                
This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package.
<PackageReference Include="Whisper.net" Version="1.5.0" />                
For projects that support PackageReference, copy this XML node into the project file to reference the package.
paket add Whisper.net --version 1.5.0                
#r "nuget: Whisper.net, 1.5.0"                
#r directive can be used in F# Interactive and Polyglot Notebooks. Copy this into the interactive tool or source code of the script to reference the package.
// Install Whisper.net as a Cake Addin
#addin nuget:?package=Whisper.net&version=1.5.0

// Install Whisper.net as a Cake Tool
#tool nuget:?package=Whisper.net&version=1.5.0                

Whisper.net

Open-Source Whisper.net

Dotnet bindings for OpenAI Whisper made possible by whisper.cpp

Native builds:

Linux Windows MacOs Android Wasm

Getting started

To install Whisper.net, run the following command in the Package Manager Console:

PM> Install-Package Whisper.net
PM> Install-Package Whisper.net.Runtime

or simply add a package reference in your csproj:

    <PackageReference Include="Whisper.net" Version="1.5.0" />
    <PackageReference Include="Whisper.net.Runtime" Version="1.5.0" />

TFL Community

We also have a Teams for Life community, where people can ask questions about the library or just discuss with others about their experience. You can join the community using this link

Runtime

The runtime package, Whisper.net.Runtime, contains the native whisper.cpp library and it is required in order to run Whisper.net.

CoreML Runtime

Whisper.net.Runtime.CoreML contains the native whisper.cpp library with Apple CoreML support enabled. Using this on Apple hardware (macOS, iOS, etc.) can net performance improvements over the core runtimes. To use it, reference the Whisper.net.Runtime.CoreML nuget,

    <PackageReference Include="Whisper.net" Version="1.5.0" />
    <PackageReference Include="Whisper.net.Runtime.CoreML" Version="1.5.0" />

Note that only the CoreML built libraries are available in this package and does not contain libraries for other platforms (Linux, Windows, etc). If you are creating a cross-platform application you can use conditional target frameworks to install the correct library package for each version.

Using the ggml whisper models with CoreML requires an additional mlmodelc file to be placed alongside your whisper model.

You can download and extract these using WhisperGgmlDownloader. Check the CoreML example.

You can also generate these via the whisper.cpp scripts. As whisper.cpp uses filepaths to detect this folder, you must load your whisper model with a file path.

If successful, the whisper output logs will announce:

whisper_init_state: loading Core ML model from...

If not, it will announce an error and use the original core library instead.

GPU Support

Dependeing on your GPU, you can use either Whisper.net.Runtime.Cublas or Whisper.net.Runtime.Clblast. For now, they are only available on Windows x64 and Linux x64 (only Cublas).

Check the Cublas and Clblast examples.

Blazor and WASM

Blazor is supported with both InteractivityServer and InteractivityWebAssemly. You can check the Blazor example here.

Versioning

Each version of Whisper.net is tied to a specific version of Whisper.cpp. The version of Whisper.net is the same as the version of Whisper it is based on. For example, Whisper.net 1.2.0 is based on Whisper.cpp 1.2.0.

However, the patch version is not tied to Whisper.cpp. For example, Whisper.net 1.2.1 is based on Whisper.cpp 1.2.0 and Whisper.net 1.5.0 is based on Whisper.cpp 1.5.1.

Ggml Models

Whisper.net uses Ggml models to perform speech recognition and translation.

You can find more about Ggml models here

Also, for easier integration Whisper.net provides a Downloader which is using https://huggingface.co.


    var modelName = "ggml-base.bin";
    if (!File.Exists(modelName))
    {
        using var modelStream = await WhisperGgmlDownloader.GetGgmlModelAsync(GgmlType.Base);
        using var fileWriter = File.OpenWrite(modelName);
        await modelStream.CopyToAsync(fileWriter);
    }

Usage


    using var whisperFactory = WhisperFactory.FromPath("ggml-base.bin");

    using var processor = whisperFactory.CreateBuilder()
        .WithLanguage("auto")
        .Build();

    using var fileStream = File.OpenRead(wavFileName);

    await foreach(var result in processor.ProcessAsync(fileStream))
    {
        Console.WriteLine($"{result.Start}->{result.End}: {result.Text}");
    }

Examples

Check more examples here

Documentation

You can find the documentation and code samples here: https://github.com/sandrohanea/whisper.net

Building The Runtime

The build scripts are a combination of PowerShell scripts and a Makefile. You can read each of them for the underlying cmake commands being used, or run them directly from the scripts.

Android:

make android

  • Before running, create an environment variable for NDK_PATH with the path to your Android NDK. For example,

NDK_PATH=/Users/UserName/Library/Developer/Xamarin/android-sdk-macosx/ndk-bundle

Apple:

make apple

  • Compiling the Apple libraries requires a Mac with Xcode installed.

Apple CoreML:

make apple_coreml

  • Compiling the Apple libraries requires a Mac with Xcode installed.

Linux:

make linux

Windows:

  • Import the powershel module Import-Module ./windows-scripts.ps1
  • Run BuildWindowsAll to build all Windows libraries

License

MIT Licence https://github.com/sandrohanea/whisper.net/blob/main/LICENSE

Supported platforms

Whisper.net is supported on the following platforms:

  • Windows x86
  • Windows x64
  • Windows ARM64
  • Windows ARM
  • Linux x64
  • Linux ARM64
  • Linux ARM
  • macOS x64
  • macOS ARM64 (Apple Silicon)
  • Android
  • iOS
  • MacCatalyst
  • tvOS
  • WebAssembly
Product Compatible and additional computed target framework versions.
.NET net5.0 was computed.  net5.0-windows was computed.  net6.0 is compatible.  net6.0-android was computed.  net6.0-ios was computed.  net6.0-maccatalyst was computed.  net6.0-macos was computed.  net6.0-tvos was computed.  net6.0-windows was computed.  net7.0 was computed.  net7.0-android was computed.  net7.0-ios was computed.  net7.0-maccatalyst was computed.  net7.0-macos was computed.  net7.0-tvos was computed.  net7.0-windows was computed.  net8.0 is compatible.  net8.0-android was computed.  net8.0-browser was computed.  net8.0-ios was computed.  net8.0-ios17.0 is compatible.  net8.0-maccatalyst was computed.  net8.0-maccatalyst17.0 is compatible.  net8.0-macos was computed.  net8.0-tvos was computed.  net8.0-tvos17.0 is compatible.  net8.0-windows was computed. 
.NET Core netcoreapp2.0 was computed.  netcoreapp2.1 was computed.  netcoreapp2.2 was computed.  netcoreapp3.0 was computed.  netcoreapp3.1 was computed. 
.NET Standard netstandard2.0 is compatible.  netstandard2.1 was computed. 
.NET Framework net461 was computed.  net462 was computed.  net463 was computed.  net47 was computed.  net471 was computed.  net472 was computed.  net48 was computed.  net481 was computed. 
MonoAndroid monoandroid was computed. 
MonoMac monomac was computed. 
MonoTouch monotouch was computed. 
Tizen tizen40 was computed.  tizen60 was computed. 
Xamarin.iOS xamarinios was computed. 
Xamarin.Mac xamarinmac was computed. 
Xamarin.TVOS xamarintvos was computed. 
Xamarin.WatchOS xamarinwatchos was computed. 
Compatible target framework(s)
Included target framework(s) (in package)
Learn more about Target Frameworks and .NET Standard.

NuGet packages (6)

Showing the top 5 NuGet packages that depend on Whisper.net:

Package Downloads
EachShow.AI

OpenAI, ChatGPT

LangChain.Providers.WhisperNet

OpenAI Whisper STT Provider using Whisper.Net library

OpenF1.Data

Package Description

BagheeraTranscriber

Package Description

SharpSpeech

SharpSpeech is free, local and open source way to speech and wake word recognition.

GitHub repositories

This package is not used by any popular GitHub repositories.

Version Downloads Last updated
1.5.0 20,616 11/25/2023
1.4.7 4,913 9/2/2023
1.4.6 2,732 7/2/2023
1.4.5 795 6/19/2023
1.4.4 208 6/18/2023
1.4.3 1,058 5/27/2023
1.4.2 367 5/20/2023
1.4.0 911 5/1/2023
1.3.0 348 4/29/2023
1.2.2 1,411 3/26/2023
1.2.1 823 2/21/2023
1.2.1-preview4 152 2/16/2023
1.2.1-preview3 104 2/16/2023
1.2.1-preview2 155 2/14/2023
1.2.1-preview1 138 2/14/2023
1.2.0 1,791 2/4/2023