Simkit 0.9.0-pre-20230217073334-98e91d6

This is a prerelease version of Simkit.
There is a newer version of this package available.
See the version list below for details.
dotnet add package Simkit --version 0.9.0-pre-20230217073334-98e91d6                
NuGet\Install-Package Simkit -Version 0.9.0-pre-20230217073334-98e91d6                
This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package.
<PackageReference Include="Simkit" Version="0.9.0-pre-20230217073334-98e91d6" />                
For projects that support PackageReference, copy this XML node into the project file to reference the package.
paket add Simkit --version 0.9.0-pre-20230217073334-98e91d6                
#r "nuget: Simkit, 0.9.0-pre-20230217073334-98e91d6"                
#r directive can be used in F# Interactive and Polyglot Notebooks. Copy this into the interactive tool or source code of the script to reference the package.
// Install Simkit as a Cake Addin
#addin nuget:?package=Simkit&version=0.9.0-pre-20230217073334-98e91d6&prerelease

// Install Simkit as a Cake Tool
#tool nuget:?package=Simkit&version=0.9.0-pre-20230217073334-98e91d6&prerelease                

Simkit

Say that you are working on some complex piece of software engineering, for example a load balancer. You want to test it. What do you need?

  • The test scenario needs to run at least 10 minutes to ensure it does not degrade over time, some cases perhaps even 24 hours.
  • You may want to evaluate behavior under different conditions (steady load, spiky load, misbehaving clients, ...).
  • Executing the same scenario multiple times should yield equivalent results - the behavior must be stable.
  • You want to validate what happens when the environment changes (e.g. some servers become unhealthy and refuse traffic).

To do all of this manually would be so slow, expensive and complicated that few would bother and justifiably so.

Simkit is a framework for creating fast and accurate simulations that can validate such complex scenarios as part of an automated test suite.

Quick start

Prerequisites:

  • .NET 7+

Steps to follow:

  1. nuget install Simkit
  2. In your test method, create an instance of SimulationParameters and Simulator.
  3. Register any services via ConfigureServices.
  4. Call Simulator.ExecuteAsync().
  5. After performing any scenario setup (to simulate inputs and observe outputs), call ISimulation.ExecuteAsync().
  6. Assert that the observed outputs indicate scenario success.
[TestMethod]
public async Task TickCounter_CountsTicksAtExpectedRate()
{
    var parameters = new SimulationParameters();
    var simulator = new Simulator(parameters);

    simulator.ConfigureServices(services =>
    {
        services.AddSingleton<TickCounter>();
    });

    await simulator.ExecuteAsync(async (simulation, cancel) =>
    {
        var tickCounter = simulation.GetRequiredService<TickCounter>();
        tickCounter.Start();

        await simulation.ExecuteAsync();

        // Validate results - did the simulated scenario actually succeed?
        // The counter counts one tick per second, so that's how much we expect to see after the simulation is completed.
        Assert.AreEqual((int)parameters.SimulationDuration.TotalSeconds, tickCounter.Ticks);
    }, CancellationToken.None);
}

For reference, see TickCounter_CountsTicksAtExpectedRate().

Time

The simulator does not run in real time - it runs much faster. The code under test is expected to use the ITime interface for all its clock/timer/delay needs, replacing built-in .NET features such as Task.Delay, PeriodicTimer and DateTimeOffset.UtcNow. The RealTime class provides an implementation suitable for real-time production code.

For optimal simulation performance, synchronous callbacks (at least ValueTask) should be preferred over asynchronous Task returning callbacks whereever possible.

Telemetry

Telemetry artifacts are stored a SimulationArtifacts subdirectory, created in the current working directory. Each subdirectory is named after the (automatically generated) simulation ID, to uniquely identify each simulation.

alternate text is missing from this package README image

Any ILogger output is written to a log file. The simulator executes multiple runs of each simulation for comparison, each of which gets its own log file.

The simulator supports metrics export via the IMetricFactory interface provided by the simulation engine. Metrics data is exported once per second in a format suitable for analysis via Azure Data Explorer, as a metrics.json.gz file. This file contains metrics of all the runs.

Metrics analysis

  1. Create an Azure Data Explorer cluster and database.
  2. On dataexplorer.azure.com, go to Data → Ingest and pick either a new or existing table to import into (recommended table name simkit). alternate text is missing from this package README image
  3. Upload one or more metrics.json.gz files from the simulations of interest.
  4. Just hit "Next" when the ingest wizard asks you anything, the defaults are all fine.
  5. Wait for the happy news that data ingestion is completed (should take less than a minute). alternate text is missing from this package README image

Now what? Well, that is up to you - you can start querying and making use of the metrics. This is your data and how you use it to extract valuable insights is up to you.

The recommended visualization platform to connect Azure Data Explorer to is Grafana, although Azure Data Explorer also has rudimentary visualization capabilities in its own GUI, which may be sufficient depending on your use case.

Example Grafana dashboards

alternate text is missing from this package README image

Load balancer simulation visualizes the status of the load balanced requests from LoadBalancerDemoScenarios.cs.

alternate text is missing from this package README image

Simulation engine visualizes the state of the simulation engine, with a focus on the time logic. This allows you to observe the internals of the simulator, such as how many timers are registered and of what type they are.

Product Compatible and additional computed target framework versions.
.NET net7.0 is compatible.  net7.0-android was computed.  net7.0-ios was computed.  net7.0-maccatalyst was computed.  net7.0-macos was computed.  net7.0-tvos was computed.  net7.0-windows was computed.  net8.0 was computed.  net8.0-android was computed.  net8.0-browser was computed.  net8.0-ios was computed.  net8.0-maccatalyst was computed.  net8.0-macos was computed.  net8.0-tvos was computed.  net8.0-windows was computed. 
Compatible target framework(s)
Included target framework(s) (in package)
Learn more about Target Frameworks and .NET Standard.

NuGet packages

This package is not used by any NuGet packages.

GitHub repositories

This package is not used by any popular GitHub repositories.