Microsoft.ML.Scoring
1.2.0
Microsoft Machine Learning Scoring library for deep learning model inference. Current version of the library supports inferencing on ONNX v1.3 and TensorFlow v1.10.0 models. The library supports CPU execution with MKL/MKLDNN acceleration. Also supports CUDA GPU devices. For CPU execution of ONNX models, no extra libraries are required. However for scoring TensorFlow models, the CUDA libraries are needed for both CPU and GPU execution. Download and install CUDA 9.2 toolkit, CUDNN and device drivers separately. This package provides a .Net standard 1.3 compatible module for maximum portability. Currently supported platforms include x64 CPU on Windows OS only.
Install-Package Microsoft.ML.Scoring -Version 1.2.0
dotnet add package Microsoft.ML.Scoring --version 1.2.0
<PackageReference Include="Microsoft.ML.Scoring" Version="1.2.0" />
paket add Microsoft.ML.Scoring --version 1.2.0
Microsoft.ML.Scoring library is a Model Inference Library that can used for scoring DNN models saved in either ONNX or TensorFlow format.. The library is .NET Standard 1.3 compatible library, with API for both managed as well as native application development. The following versions of ONNX and TensorFlow models are supported:
ONNX:
- ONNX Version: 1.3
- CPU with MKLDNN acceleration
- GPU via CUDA 9.2 Tookit + CUDNN library
TensorFlow:
- TensorFlow Version: 1.10.0
- CPU with MKL acceleration
- Note: CUDA libraries must be present on system path, even for CPU execution.
- GPU via CUDA 9.2 Tookit and CUDNN library
- Model formats: checkpoint and saved model only. Frozen model is not supported.
- For TensorFlow Checkpoint - all files including a checkpoint file, a meta file, and data files should be stored under the same folder. If your model contains TensorFlow lookup operations, please copy your vocabulary file to this folder as well.
- For TensorFlow SavedModel - all files including a pb file, data files and asset files should be stored under the same folder. Please do not import SavedModel files that were previously optimized by the library -this can result in unexpected errors.
Note: This library contains third party software. For more details, refer to ThirdPartyNotices.txt in the package.
Microsoft.ML.Scoring library is a Model Inference Library that can used for scoring DNN models saved in either ONNX or TensorFlow format.. The library is .NET Standard 1.3 compatible library, with API for both managed as well as native application development. The following versions of ONNX and TensorFlow models are supported:
ONNX:
- ONNX Version: 1.3
- CPU with MKLDNN acceleration
- GPU via CUDA 9.2 Tookit + CUDNN library
TensorFlow:
- TensorFlow Version: 1.10.0
- CPU with MKL acceleration
- Note: CUDA libraries must be present on system path, even for CPU execution.
- GPU via CUDA 9.2 Tookit and CUDNN library
- Model formats: checkpoint and saved model only. Frozen model is not supported.
- For TensorFlow Checkpoint - all files including a checkpoint file, a meta file, and data files should be stored under the same folder. If your model contains TensorFlow lookup operations, please copy your vocabulary file to this folder as well.
- For TensorFlow SavedModel - all files including a pb file, data files and asset files should be stored under the same folder. Please do not import SavedModel files that were previously optimized by the library -this can result in unexpected errors.
Note: This library contains third party software. For more details, refer to ThirdPartyNotices.txt in the package.
Release Notes
New additions in the version 1.2.0 release are -- .Net standard C# module; GPU support; updated ONNX runtime, supporting ONNX models of version 1.3
Dependencies
This package has no dependencies.
GitHub Usage
Showing the top 1 GitHub repositories that depend on Microsoft.ML.Scoring:
| Repository | Stars | |
|---|---|---|
|
microsoft/samples-for-ai
Samples for getting started with deep learning across TensorFlow, CNTK, Theano and more.
|