PythonEmbedded.Net 1.2.1.22

PythonEmbedded.Net

A .NET library for managing local, embeddable Python instances. Download, install, manage, and execute Python environments directly within .NET applications without requiring system-wide Python installations.

Installation

Nuget

The recommended installation approach is to use the available nuget package: PythonEmbedded.Net

Clone

Alternatively, you can clone this repo and reference the PythonEmbedded.Net project in your project.

Features

• ✅ Automatic Python Distribution Management: Download and install Python distributions from python-build-standalone
• ✅ Multiple Instance Support: Manage multiple Python versions and build dates simultaneously
• ✅ Smart Version Matching:
  • Exact versions (e.g., "3.12.5") match exactly
  • Partial versions (e.g., "3.12") automatically find the latest patch version (e.g., "3.12.19")
• ✅ Virtual Environment Management: Create and manage virtual environments for each Python instance
• ✅ Package Installation: Install packages via pip (single packages, requirements.txt, pyproject.toml)
• ✅ Python Execution: Execute Python code via subprocess or in-process using Python.NET
• ✅ Two Execution Modes:
  • PythonManager: Subprocess-based execution (standard Python execution)
  • PythonNetManager: Python.NET-based execution (in-process, high-performance)
• ✅ Cross-Platform: Supports Windows, Linux, and macOS
• ✅ Archive Format Support: Supports multiple archive formats (zip, tar.gz, tar.bz2, tar.bz, tar.zst)
• ✅ Modern C# Design: Abstract classes for extensibility, dependency injection support, IDisposable for resource management
• ✅ Structured Logging: Full support for Microsoft.Extensions.Logging
• ✅ Performance Optimizations: Optional caching for GitHub API responses, object pooling for frequently allocated objects

Installation

Install the NuGet package:

dotnet add package PythonEmbedded.Net

Requirements

• .NET 9.0 or later
• Octokit (included)
• Python.NET (included, optional - only needed for PythonNetManager)
• Tomlyn (for pyproject.toml support, included)

Quick Start

Basic Usage with PythonManager (Subprocess Mode)

using PythonEmbedded.Net;
using Octokit;
using Microsoft.Extensions.Logging;

// Create a PythonManager for subprocess execution
var manager = new PythonManager(
    directory: "/path/to/python/instances",
    githubClient: new GitHubClient(new ProductHeaderValue("MyApp")),
    logger: loggerFactory.CreateLogger<PythonManager>());

// Get or create a Python instance (downloads if needed)
var runtime = await manager.GetOrCreateInstanceAsync("3.12.0");

// Cast to root runtime to access virtual environment features
var rootRuntime = (BasePythonRootRuntime)runtime;

// Create a virtual environment
var venv = await rootRuntime.GetOrCreateVirtualEnvironmentAsync("myenv");

// Install packages
await venv.InstallPackageAsync("numpy");
await venv.InstallPackageAsync("requests==2.31.0");

// Execute Python code
var result = await venv.ExecuteCommandAsync("print('Hello, World!')");
Console.WriteLine(result.StandardOutput); // Output: Hello, World!

Using PythonNetManager (Python.NET Mode)

// Create a PythonNetManager for in-process Python.NET execution
var netManager = new PythonNetManager(
    directory: "/path/to/python/instances",
    githubClient: new GitHubClient(new ProductHeaderValue("MyApp")),
    logger: loggerFactory.CreateLogger<PythonNetManager>());

// Get or create a Python instance
var runtime = await netManager.GetOrCreateInstanceAsync("3.12.0");

// Create virtual environment
var rootRuntime = (IPythonRootRuntime)runtime;
var venv = await rootRuntime.GetOrCreateVirtualEnvironmentAsync("myenv");

// Install packages and execute code (same API as PythonManager)
await venv.InstallPackageAsync("numpy");
var result = await venv.ExecuteCommandAsync("import numpy; print(numpy.__version__)");

Multiple Instances and Version Matching

var manager = new PythonManager("/path/to/python/instances", githubClient);

// Exact version matching - matches exactly "3.12.5"
var runtime312 = await manager.GetOrCreateInstanceAsync("3.12.5");

// Partial version matching - finds latest patch version (e.g., "3.12.19")
var runtime312Latest = await manager.GetOrCreateInstanceAsync("3.12");

// With build date (DateTime? - was string? in older versions)
var runtime311 = await manager.GetOrCreateInstanceAsync(
    "3.11.5", 
    buildDate: new DateTime(2024, 2, 10));

// List all instances
var instances = manager.ListInstances();
foreach (var instance in instances)
{
    Console.WriteLine($"Python {instance.PythonVersion} (Build: {instance.BuildDate:yyyy-MM-dd})");
}

// Delete an instance
await manager.DeleteInstanceAsync("3.11.5", new DateTime(2024, 2, 10));

Package Installation Examples

var runtime = await manager.GetOrCreateInstanceAsync("3.12.0");
var rootRuntime = (IPythonRootRuntime)runtime;

// Install a single package to the root Python installation
await runtime.InstallPackageAsync("numpy");

// Install with version constraint
await runtime.InstallPackageAsync("requests==2.31.0");

// Create a virtual environment and install packages
var venv = await rootRuntime.GetOrCreateVirtualEnvironmentAsync("myenv");
await venv.InstallPackageAsync("pandas");

// Install from requirements.txt
await venv.InstallRequirementsAsync("requirements.txt");

// Install from pyproject.toml
await venv.InstallPyProjectAsync("pyproject.toml", editable: true);

Virtual Environment Management

var runtime = await manager.GetOrCreateInstanceAsync("3.12.0");
var rootRuntime = (IPythonRootRuntime)runtime;

// Create a virtual environment
var venv = await rootRuntime.GetOrCreateVirtualEnvironmentAsync("myenv");

// Use the virtual environment for operations
await venv.InstallPackageAsync("numpy");
var result = await venv.ExecuteCommandAsync("print('Hello from venv!')");

// List virtual environments
var venvNames = rootRuntime.ListVirtualEnvironments();
foreach (var name in venvNames)
{
    Console.WriteLine($"Virtual environment: {name}");
}

// Delete a virtual environment
await rootRuntime.DeleteVirtualEnvironmentAsync("myenv");

Execution Examples

Command Execution

var runtime = await manager.GetOrCreateInstanceAsync("3.12.0");

// Execute Python code
var result = await runtime.ExecuteCommandAsync("print('Hello from Python')");

// Access execution results
Console.WriteLine($"Exit Code: {result.ExitCode}");
Console.WriteLine($"Output: {result.StandardOutput}");
Console.WriteLine($"Error: {result.StandardError}");

// With stdin/stdout/stderr handlers
var result2 = await runtime.ExecuteCommandAsync(
    command: "for i in range(3): print(i)",
    stdoutHandler: line => Console.WriteLine($"Received: {line}"),
    stderrHandler: line => Console.Error.WriteLine($"Error: {line}"));

Script Execution

var runtime = await manager.GetOrCreateInstanceAsync("3.12.0");

// Execute a Python script
var scriptResult = await runtime.ExecuteScriptAsync(
    scriptPath: "script.py",
    arguments: new[] { "arg1", "arg2" });

// Execute script in virtual environment
var rootRuntime = (IPythonRootRuntime)runtime;
var venv = await rootRuntime.GetOrCreateVirtualEnvironmentAsync("myenv");
var venvResult = await venv.ExecuteScriptAsync("script.py");

Synchronous Versions

All async methods have synchronous counterparts:

// Synchronous execution
var result = runtime.ExecuteCommand("print('Hello')");
var venv = rootRuntime.GetOrCreateVirtualEnvironment("myenv");
venv.InstallPackage("numpy");

Advanced Usage

Using Abstract Classes for Dependency Injection

// Register in DI container
services.AddMemoryCache(); // For caching GitHub API responses

services.AddSingleton<PythonManager>(sp =>
{
    var loggerFactory = sp.GetRequiredService<ILoggerFactory>();
    var cache = sp.GetService<IMemoryCache>(); // Optional: improves performance
    var githubClient = new GitHubClient(new ProductHeaderValue("MyApp"));
    return new PythonManager("/path/to/instances", githubClient, 
        loggerFactory.CreateLogger<PythonManager>(), loggerFactory, cache);
});

// Inject and use
public class MyService
{
    private readonly PythonManager _pythonManager;
    
    public MyService(PythonManager pythonManager)
    {
        _pythonManager = pythonManager;
    }
    
    public async Task DoWorkAsync()
    {
        var runtime = await _pythonManager.GetOrCreateInstanceAsync("3.12.0");
        var result = await runtime.ExecuteCommandAsync("print('Hello')");
    }
}

Resource Management (Python.NET)

When using PythonNetManager, dispose of runtimes when done:

var netManager = new PythonNetManager("/path/to/instances", githubClient);
var runtime = await netManager.GetOrCreateInstanceAsync("3.12.0");

try
{
    // Use the runtime
    await runtime.ExecuteCommandAsync("print('Hello')");
}
finally
{
    // Dispose if it's a Python.NET runtime (PythonNetRootRuntime implements IDisposable)
    if (runtime is IDisposable disposable)
    {
        disposable.Dispose();
    }
}

Platform Support

The library automatically detects your platform and downloads the appropriate Python distribution from python-build-standalone:

• Windows: x64, x86 (Windows 7+)
• Linux: x64, ARM64, ARMv7 (GNU libc and musl)
• macOS: Intel (x64), Apple Silicon (ARM64)

Archive Format Support

The library supports multiple archive formats for Python distributions:

• .zip - Standard ZIP archives (Windows, cross-platform)
• .tar.gz - Gzip-compressed tar archives (Linux, macOS)
• .tar.bz2 - Bzip2-compressed tar archives (Linux, macOS)
• .tar.bz - Bzip-compressed tar archives (Linux, macOS)
• .tar.zst - Zstandard-compressed tar archives (Linux, macOS)

Archive extraction uses system tools (tar command) where available. The library automatically detects and handles the appropriate format based on the downloaded asset.

Architecture

Manager Classes

• PythonManager: Manages Python instances with subprocess-based execution
• PythonNetManager: Manages Python instances with Python.NET in-process execution

Both extend BasePythonManager and provide the same abstract interface for instance management.

Runtime Classes

• BasePythonRuntime: Abstract base class providing common execution and package installation functionality
• BasePythonRootRuntime: Extends BasePythonRuntime with virtual environment management
• BasePythonVirtualRuntime: Represents a virtual environment runtime

Concrete implementations:

• PythonRootRuntime: Subprocess-based root Python runtime
• PythonNetRootRuntime: Python.NET-based root Python runtime
• PythonRootVirtualEnvironment: Subprocess-based virtual environment
• PythonNetVirtualEnvironment: Python.NET-based virtual environment

Abstract Classes

The library uses abstract base classes for extensibility:

• BasePythonManager: Abstract manager base class
• BasePythonRuntime: Base runtime class
• BasePythonRootRuntime: Root runtime with virtual environment support
• BasePythonVirtualRuntime: Virtual environment runtime base class

Services

• IProcessExecutor: Interface for process execution (extracted for testability)
• ProcessExecutor: Default implementation of process execution

Directory Structure

specifieddir/
├── manager_metadata.json          # Central metadata file
└── python-{version}-{buildDate}/   # Instance directory
    ├── venvs/                      # Virtual environments
    │   └── {venv_name}/
    ├── python_instance/            # Python installation files
    └── instance_metadata.json      # Instance metadata

API Overview

BasePythonManager

• GetOrCreateInstanceAsync(pythonVersion?, buildDate?, cancellationToken?) - Get or create a Python runtime instance
  • pythonVersion: Optional. If null, uses default from configuration or "3.12"
  • Supports partial versions (e.g., "3.12" finds latest patch like "3.12.19")
  • Exact versions (e.g., "3.12.5") match exactly
• GetOrCreateInstance(pythonVersion?, buildDate?) - Synchronous version
• DeleteInstanceAsync(pythonVersion, buildDate?, cancellationToken?) - Delete an instance
  • buildDate: DateTime? - Optional build date (was string? in older versions)
• DeleteInstance(pythonVersion, buildDate?) - Synchronous version
• ListInstances() - List all cached instances
• ListAvailableVersionsAsync(releaseTag?, cancellationToken?) - List available versions from GitHub
• ListAvailableVersions(releaseTag?) - Synchronous version

BasePythonRuntime

• Execution:

  • ExecuteCommandAsync(command, stdinHandler?, stdoutHandler?, stderrHandler?) - Execute Python command
  • ExecuteCommand(...) - Synchronous version
  • ExecuteScriptAsync(scriptPath, arguments?, stdinHandler?, stdoutHandler?, stderrHandler?) - Execute Python script
  • ExecuteScript(...) - Synchronous version

• Package Installation:

  • InstallPackageAsync(packageSpec, upgrade?) - Install a single package
  • InstallPackage(...) - Synchronous version
  • InstallRequirementsAsync(requirementsFile, upgrade?) - Install from requirements.txt
  • InstallRequirements(...) - Synchronous version
  • InstallPyProjectAsync(pyProjectFile, editable?) - Install from pyproject.toml
  • InstallPyProject(...) - Synchronous version

BasePythonRootRuntime (extends BasePythonRuntime)

• Virtual Environment Management:
  • GetOrCreateVirtualEnvironmentAsync(name, recreateIfExists?) - Get or create a virtual environment
  • GetOrCreateVirtualEnvironment(...) - Synchronous version
  • DeleteVirtualEnvironmentAsync(name) - Delete a virtual environment
  • DeleteVirtualEnvironment(name) - Synchronous version
  • ListVirtualEnvironments() - List all virtual environments

PythonExecutionResult

A record type containing execution results:

public record PythonExecutionResult(
    int ExitCode,
    string StandardOutput = "",
    string StandardError = "");

GitHub API Authentication

For higher rate limits, provide an authenticated GitHub client:

var githubClient = new GitHubClient(new ProductHeaderValue("MyApp"))
{
    Credentials = new Credentials("your-github-token")
};

var manager = new PythonManager("/path/to/instances", githubClient);

Error Handling

The library uses custom exceptions for different error scenarios:

• PythonInstallationException - Base exception for installation errors
• PythonNotInstalledException - Python installation is missing or invalid
• PlatformNotSupportedException - Platform not supported
• PythonExecutionException - Python execution failed
• PackageInstallationException - Package installation failed
• RequirementsFileException - Requirements file installation failed
• VirtualEnvironmentNotFoundException - Virtual environment not found
• InstanceNotFoundException - Instance not found
• PythonNetInitializationException - Python.NET initialization failed
• PythonNetExecutionException - Python.NET execution failed
• And more...

Design Principles

This library follows modern C# best practices:

• Abstract Base Classes: Extensible architecture using abstract base classes
• Dependency Injection: Support for DI containers with logger factories and caching
• Resource Management: IDisposable support for Python.NET runtimes
• Modern C# Features: Records, file-scoped namespaces, collection expressions, pattern matching, ConfigureAwait(false)
• Structured Logging: Full Microsoft.Extensions.Logging integration
• Separation of Concerns: Process execution extracted to a service
• Performance Optimizations: Optional caching, object pooling for hot paths

Contributing

Contributions are welcome! Please read the contributing guidelines and submit pull requests.

License

MIT License - see LICENSE file for details.

Acknowledgments

This library utilizes python-build-standalone by astral-sh for providing high-quality, redistributable Python distributions. We are not associated with astral-sh, but we thank them for their fantastic work that makes this library possible.

Links

• Python Build Standalone - Source of Python distributions
• Python.NET - Python.NET integration
• Octokit - GitHub API client