Simkit 1.0.0
dotnet add package Simkit --version 1.0.0
NuGet\Install-Package Simkit -Version 1.0.0
<PackageReference Include="Simkit" Version="1.0.0" />
paket add Simkit --version 1.0.0
#r "nuget: Simkit, 1.0.0"
// Install Simkit as a Cake Addin #addin nuget:?package=Simkit&version=1.0.0 // Install Simkit as a Cake Tool #tool nuget:?package=Simkit&version=1.0.0
Simkit
Say that you are working on some complex piece of software engineering, for example a load balancer. You want to test it. What do you need?
- The test scenario needs to run at least 10 minutes to ensure it does not degrade over time, some cases perhaps even 24 hours.
- You may want to evaluate behavior under different conditions (steady load, spiky load, misbehaving clients, ...).
- Executing the same scenario multiple times should yield equivalent results - the behavior must be stable.
- You want to validate what happens when the environment changes (e.g. some servers become unhealthy and refuse traffic).
To do all of this manually would be so slow, expensive and complicated that few would bother and justifiably so.
Simkit is a framework for creating fast and accurate simulations that can validate such complex scenarios as part of an automated test suite.
Quick start
Prerequisites:
- .NET 7+
Steps to follow:
nuget install Simkit
- In your test method, create an instance of
SimulationParameters
andSimulator
. - Register any services via
ConfigureServices
. - Call
Simulator.ExecuteAsync()
. - After performing any scenario setup (to simulate inputs and observe outputs), call
ISimulation.ExecuteAsync()
. - Assert that the observed outputs indicate scenario success.
[TestMethod]
public async Task TickCounter_CountsTicksAtExpectedRate()
{
var parameters = new SimulationParameters();
var simulator = new Simulator(parameters);
simulator.ConfigureServices(services =>
{
services.AddSingleton<TickCounter>();
});
await simulator.ExecuteAsync(async (simulation, cancel) =>
{
var tickCounter = simulation.GetRequiredService<TickCounter>();
tickCounter.Start();
await simulation.ExecuteAsync();
// Validate results - did the simulated scenario actually succeed?
// The counter counts one tick per second, so that's how much we expect to see after the simulation is completed.
Assert.AreEqual((int)parameters.SimulationDuration.TotalSeconds, tickCounter.Ticks);
}, CancellationToken.None);
}
For reference, see TickCounter_CountsTicksAtExpectedRate().
Time
The simulator does not run in real time - it runs much faster. The code under test is expected to use the ITime interface for all its clock/timer/delay needs, replacing built-in .NET features such as Task.Delay
, PeriodicTimer
and DateTimeOffset.UtcNow
. The RealTime class provides an implementation suitable for real-time production code.
For optimal simulation performance, synchronous callbacks (at least ValueTask
) should be preferred over asynchronous Task
returning callbacks whereever possible.
Telemetry
Telemetry artifacts are stored a SimulationArtifacts
subdirectory, created in the current working directory. Each subdirectory is named after the (automatically generated) simulation ID, to uniquely identify each simulation.
Any ILogger
output is written to a log file. The simulator executes multiple runs of each simulation for comparison, each of which gets its own log file.
The simulator supports metrics export via the IMetricFactory
interface provided by the simulation engine. Metrics data is exported once per second in a format suitable for analysis via Azure Data Explorer, as a metrics.json.gz
file. This file contains metrics of all the runs.
Metrics analysis
- Create an Azure Data Explorer cluster and database.
- On dataexplorer.azure.com, go to Data → Ingest and pick either a new or existing table to import into (recommended table name
simkit
). - Upload one or more
metrics.json.gz
files from the simulations of interest. - Just hit "Next" when the ingest wizard asks you anything, the defaults are all fine.
- Wait for the happy news that data ingestion is completed (should take less than a minute).
Now what? Well, that is up to you - you can start querying and making use of the metrics. This is your data and how you use it to extract valuable insights is up to you.
The recommended visualization platform to connect Azure Data Explorer to is Grafana, although Azure Data Explorer also has rudimentary visualization capabilities in its own GUI, which may be sufficient depending on your use case.
Example Grafana dashboards
Load balancer simulation visualizes the status of the load balanced requests from LoadBalancerDemoScenarios.cs.
Simulation engine visualizes the state of the simulation engine, with a focus on the time logic. This allows you to observe the internals of the simulator, such as how many timers are registered and of what type they are.
Product | Versions Compatible and additional computed target framework versions. |
---|---|
.NET | net7.0 is compatible. net7.0-android was computed. net7.0-ios was computed. net7.0-maccatalyst was computed. net7.0-macos was computed. net7.0-tvos was computed. net7.0-windows was computed. net8.0 was computed. net8.0-android was computed. net8.0-browser was computed. net8.0-ios was computed. net8.0-maccatalyst was computed. net8.0-macos was computed. net8.0-tvos was computed. net8.0-windows was computed. |
-
net7.0
- Karambolo.Extensions.Logging.File (>= 3.3.1)
- Microsoft.Extensions.Hosting (>= 7.0.1)
- Microsoft.Extensions.Hosting.Abstractions (>= 7.0.0)
- Nito.AsyncEx (>= 5.1.2)
- prometheus-net (>= 8.0.0)
NuGet packages
This package is not used by any NuGet packages.
GitHub repositories
This package is not used by any popular GitHub repositories.
Version | Downloads | Last updated |
---|---|---|
1.0.0 | 287 | 2/17/2023 |
1.0.0-pre-20230220105811-a4... | 176 | 2/20/2023 |
1.0.0-pre-20230217081327-ff... | 155 | 2/17/2023 |
0.9.0-pre-20230217081049-63... | 155 | 2/17/2023 |
0.9.0-pre-20230217073550-4a... | 159 | 2/17/2023 |
0.9.0-pre-20230217073334-98... | 163 | 2/17/2023 |
0.9.0-pre-20230217073207-b1... | 156 | 2/17/2023 |