VllmClient 0.2.0
There is a newer version of this package available.
See the version list below for details.
See the version list below for details.
dotnet add package VllmClient --version 0.2.0
NuGet\Install-Package VllmClient -Version 0.2.0
This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package.
<PackageReference Include="VllmClient" Version="0.2.0" />
For projects that support PackageReference, copy this XML node into the project file to reference the package.
<PackageVersion Include="VllmClient" Version="0.2.0" />
<PackageReference Include="VllmClient" />
For projects that support Central Package Management (CPM), copy this XML node into the solution Directory.Packages.props file to version the package.
paket add VllmClient --version 0.2.0
The NuGet Team does not provide support for this client. Please contact its maintainers for support.
#r "nuget: VllmClient, 0.2.0"
#r directive can be used in F# Interactive and Polyglot Notebooks. Copy this into the interactive tool or source code of the script to reference the package.
#addin nuget:?package=VllmClient&version=0.2.0
#tool nuget:?package=VllmClient&version=0.2.0
The NuGet Team does not provide support for this client. Please contact its maintainers for support.
vLLM Client
Overview
Client for the vLLM API with minimal dependencies.
Installation
Install vLLM Client from NuGet.
Examples
const string ApiUrl = "http://localhost:8000/generate";
var client = new AsyncVllmClient(ApiUrl);
// See the docstring of the SamplingParams [*] Python class
var @params = new SamplingParams { Temperature = 0.5f, MaxTokens = 2 };
const string prompt = "Hello! My name is";
// Full generation (list with N items, see SamplingParams)
IList<string> response = await client.Generate(prompt, @params);
// Streaming generation
await foreach (var response in client.Stream(prompt, @params)) {
// Response is an IList of N strings
}
See also the unit tests in the repository as examples.
Make sure to consider the prompt template of the model used (if any) to avoid confusion.
Notes
SamplingParamsneeds to be kept in sync with vLLM.
| Product | Versions Compatible and additional computed target framework versions. |
|---|---|
| .NET | net6.0 is compatible. net6.0-android was computed. net6.0-ios was computed. net6.0-maccatalyst was computed. net6.0-macos was computed. net6.0-tvos was computed. net6.0-windows was computed. net7.0 was computed. net7.0-android was computed. net7.0-ios was computed. net7.0-maccatalyst was computed. net7.0-macos was computed. net7.0-tvos was computed. net7.0-windows was computed. net8.0 was computed. net8.0-android was computed. net8.0-browser was computed. net8.0-ios was computed. net8.0-maccatalyst was computed. net8.0-macos was computed. net8.0-tvos was computed. net8.0-windows was computed. net9.0 was computed. net9.0-android was computed. net9.0-browser was computed. net9.0-ios was computed. net9.0-maccatalyst was computed. net9.0-macos was computed. net9.0-tvos was computed. net9.0-windows was computed. net10.0 was computed. net10.0-android was computed. net10.0-browser was computed. net10.0-ios was computed. net10.0-maccatalyst was computed. net10.0-macos was computed. net10.0-tvos was computed. net10.0-windows was computed. |
Compatible target framework(s)
Included target framework(s) (in package)
Learn more about Target Frameworks and .NET Standard.
-
net6.0
- No dependencies.
NuGet packages
This package is not used by any NuGet packages.
GitHub repositories
This package is not used by any popular GitHub repositories.