Boos.TestProcessWrapper
6.2.373-alpha
See the version list below for details.
dotnet add package Boos.TestProcessWrapper --version 6.2.373-alpha
NuGet\Install-Package Boos.TestProcessWrapper -Version 6.2.373-alpha
<PackageReference Include="Boos.TestProcessWrapper" Version="6.2.373-alpha" />
paket add Boos.TestProcessWrapper --version 6.2.373-alpha
#r "nuget: Boos.TestProcessWrapper, 6.2.373-alpha"
// Install Boos.TestProcessWrapper as a Cake Addin #addin nuget:?package=Boos.TestProcessWrapper&version=6.2.373-alpha&prerelease // Install Boos.TestProcessWrapper as a Cake Tool #tool nuget:?package=Boos.TestProcessWrapper&version=6.2.373-alpha&prerelease
Test Helper: TestProcessWrapper
Table of Contents generated with DocToc
Overview
Launch and control dotnet
processes optionally wrapped into the
coverlet code coverage analyzer.
The class TestProcessWrapper
is intended to launch one ore more dotnet
processes for performing acceptance tests.
The class captures the messages written to the Console
and to Console.Error
. It allows to terminate the process
gracefully and forcefully. If multiple processes of the same executable (DLL) are running simultaneously, then one of
them can be wrapped by the coverlet command line tool in order to
calculate code coverage.
Example processes are given in this repository:
- TestProcessWrapper.LongLived.Application - daemon process; terminates after receiving a signal
- TestProcessWrapper.ShortLived.Application - command line process; terminates quickly by itself
Important
The tested process must report its process ID on the console. An example is TestProcessWrapper.ShortLived.Application/Program.cs.
Ensure that only one executable (DLL) is wrapped into coverlet at a time. Run instrumented integration tests sequentially. Otherwise the coverage results will be wrong. When using specflow with xUnit, enforce sequential execution by a specflow.json file like this.
Usage Examples
The most simple use is described by the acceptance test SmokeTests.cs.
You can find detailed usage examples in the Acceptance Test Suite (BDD). If you'd like to have easy to read HTML documentation, then generate LivingDoc as described in the corresponding section Create Feature Documentation (LivingDoc) below.
Next, read a Gherkin .feature
file from the
Features folder. It explains why each feature exists and which use
scenarios are addressed. Then read the corresponding *StepDefinitions.cs
file in the
Steps folder. It shows how the test steps from the feature file (given,
when, then) are actually implemented. Note, that some frequently used steps are implemented in the
Common folder.
Further information can be seen in the following GitHub repositories use TestProcessWrapper:
Attention
You can use the coverlet
wrapper only once per dotnet
application, because coverlet
instruments the dotnet
DLL.
If you use coverlet
with two or more instances of the same application, coverlet
will report an exception after
(or during) the application termination and the reported coverage will be 0.
Development and Support Standard
I am developing during my spare time and use this project for learning purposes. Please assume that I will need some days to answer your questions. Please keep this in mind when using this project in a production environment.
Thanks
Many thanks to JetBrains who provide an Open Source License for this project ❤️.
Development
Prerequisites
To compile, test and run this project the latest .NET SDK is required on your machine. For calculating code metrics I recommend metrix++. This requires Python. If you'd like to contribute, then please use the dotnet csharpier . command as described below.
To use the TestProcessWrapper
library and to run the unit tests you need the following tools installed:
dotnet tool install --global coverlet.console --configfile NuGet-OfficialOnly.config
dotnet tool install --global dotnet-reportgenerator-globaltool --configfile NuGet-OfficialOnly.config
Troubleshooting
If you are installing a dotnet tool for the first time, then you'll need to add the path to the dotnet tools to your system PATH. Please make sure that there is no "~" character in your PATH to coverlet.
E.g. add the following line to the end of your shell rc file (e.g. ~/.zshrc):
export PATH="$PATH:$HOME/.dotnet/tools"
Build, Test, Run
Run the following commands from the folder containing the TestProcessWrapper.sln
file in order to build, test and
run the application:
Build the Solution and Run the Acceptance Tests
Note: The script build.sh builds the NuGet package like the build pipeline does it. This can be helpful when debugging issues popping up in the build pipeline.
# Remove build output from previous runs
find . -iname "bin" -o -iname "obj" -exec rm -rf "{}" ";"
# The acceptance tests require both a debug and a release build of the long lived application
dotnet build --configuration Debug
dotnet build --configuration Release --no-restore TestProcessWrapper.LongLived.Application/TestProcessWrapper.LongLived.Application.csproj
# Simply run the tests
dotnet test
# As an alternative, run the tests with coverage and produce a coverage report
rm -r TestProcessWrapper.Acceptance.Tests/TestResults && \
dotnet test --no-restore --verbosity normal /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura /p:CoverletOutput='./TestResults/coverage.cobertura.xml' && \
reportgenerator "-reports:TestProcessWrapper.Acceptance.Tests/TestResults/*.xml" "-targetdir:TestProcessWrapper.Acceptance.Tests/TestResults/report" "-reporttypes:Html;lcov" "-title:TestProcessWrapper"
open TestProcessWrapper.Acceptance.Tests/TestResults/report/index.html
Run the Smoke Tests
In the ci pipeline, the smoke tests verify compatibility with all supported .net framework versions.
To run the smoke tests locally, issue the following console commands:
# build the nuget package with version 0.0.0
dotnet pack --configuration Debug TestProcessWrapper/TestProcessWrapper.csproj
# run the smoke tests
cd TestProcessWrapper.Nupkg.Tests
./smoketest.sh "net7.0"
You can replace the "net7.0" parameter with "net6.0", if you want to test that version of the .net framework.
Create Feature Documentation (LivingDoc)
As this project uses SpecFlow for acceptance tests, you can generate an HTML overview of all features including the test status as follows:
# Prerequisite: Install the LivingDoc CLI once
dotnet tool install --global SpecFlow.Plus.LivingDoc.CLI
# Run the acceptance tests to generate the TestExecution.json
# for LivingDoc
dotnet test TestProcessWrapper.Acceptance.Tests
# Generate LivingDoc
cd TestProcessWrapper.Acceptance.Tests/bin/Debug/net7.0
livingdoc test-assembly TestProcessWrapper.Acceptance.Tests.dll -t TestExecution.json
cd ../../../..
# Open the generated HTML in a browser
open TestProcessWrapper.Acceptance.Tests/bin/Debug/net7.0/LivingDoc.html
Before Creating a Pull Request ...
Fix Static Code Analysis Warnings
... fix static code analysis warnings reported by SonarLint and by CodeClimate.
Apply Code Formatting Rules
# Install https://csharpier.io globally, once
dotnet tool install -g csharpier
# Format code
dotnet csharpier .
Check Code Metrics
... check code metrics using metrix++
Configure the location of the cloned metrix++ scripts
export METRIXPP=/path/to/metrixplusplus
Collect metrics
python "$METRIXPP/metrix++.py" collect --std.code.complexity.cyclomatic --std.code.lines.code --std.code.todo.comments --std.code.maintindex.simple -- .
Get an overview
python "$METRIXPP/metrix++.py" view --db-file=./metrixpp.db
Apply thresholds
python "$METRIXPP/metrix++.py" limit --db-file=./metrixpp.db --max-limit=std.code.complexity:cyclomatic:5 --max-limit=std.code.lines:code:25:function --max-limit=std.code.todo:comments:0 --max-limit=std.code.mi:simple:1
At the time of writing, I want to stay below the following thresholds:
--max-limit=std.code.complexity:cyclomatic:5
--max-limit=std.code.lines:code:25:function
--max-limit=std.code.todo:comments:0
--max-limit=std.code.mi:simple:1
Finally, remove all code duplication. The next section describes how to detect code duplication.
Remove Code Duplication Where Appropriate
To detect duplicates I use the CPD Copy Paste Detector tool from the PMD Source Code Analyzer Project.
If you have installed PMD by download & unzip, replace pmd
by ./run.sh
.
The homebrew pmd formula makes the pmd
command globally available.
# Remove temporary and generated files
# 1. dry run
git clean -ndx
# 2. Remove the files shown by the dry run
git clean -fdx
# Identify duplicated code in files to push to GitHub
pmd cpd --minimum-tokens 50 --language cs --dir .
Make a Release
In order to create a release:
- Create a branch to prepare the release
- Update the CHANGELOG.md. The last number in the version is the build number. Assume that it will be the current build number + 2, because there will be one build to validate the pull request
- Create a pull request and wait for the validations to complete
- If there are validation errors, then fix them and update the release build number in the CHANGELOG.md
- The release will be published automatically from the main branch after the PR has been merged
References
.NET Core
Behavior Driven Development (BDD)
- Tricentis: SpecFlow - Getting Started
- The SpecFlow Team: SpecFlow.xUnit — documentation
- The SpecFlow Team: SpecFlow - Getting Started with a new project
- The SpecFlow Team: Add Living Documentation
- The SpecFlow Team: Generating LivingDoc using CLI
- Testcontainers
Code Style
- Bela VanderVoort: CSharpier - an opinionated code formatter
- Microsoft: dotnet format - dotnet code formatter
Code Analysis
- SonarSource S.A.: SonarLint IDE Extension
- Microsoft: Use code coverage for unit testing
- GitHub: coverlet-coverage / coverlet
- GitHub: danielpalme / ReportGenerator
- JetBrains s.r.o.: dupFinder Command-Line Tool
- Scott Hanselman: EditorConfig code formatting from the command line with .NET Core's dotnet format global tool
- EditorConfig.org
- GitHub: dotnet / roslyn - .editorconfig
- Check all the badges on top of this README
Product | Versions Compatible and additional computed target framework versions. |
---|---|
.NET | net6.0 is compatible. net6.0-android was computed. net6.0-ios was computed. net6.0-maccatalyst was computed. net6.0-macos was computed. net6.0-tvos was computed. net6.0-windows was computed. net7.0 is compatible. net7.0-android was computed. net7.0-ios was computed. net7.0-maccatalyst was computed. net7.0-macos was computed. net7.0-tvos was computed. net7.0-windows was computed. net8.0 was computed. net8.0-android was computed. net8.0-browser was computed. net8.0-ios was computed. net8.0-maccatalyst was computed. net8.0-macos was computed. net8.0-tvos was computed. net8.0-windows was computed. |
-
net6.0
- Microsoft.Extensions.Logging.Abstractions (>= 8.0.1)
- xunit.abstractions (>= 2.0.3)
-
net7.0
- Microsoft.Extensions.Logging.Abstractions (>= 8.0.1)
- xunit.abstractions (>= 2.0.3)
NuGet packages
This package is not used by any NuGet packages.
GitHub repositories
This package is not used by any popular GitHub repositories.
Version | Downloads | Last updated |
---|---|---|
6.3.376-alpha | 61 | 8/8/2024 |
6.2.373-alpha | 40 | 8/6/2024 |
6.2.367-alpha | 29 | 8/5/2024 |
6.2.350-alpha | 98 | 7/26/2023 |
6.2.343-alpha | 79 | 6/16/2023 |
6.2.330-alpha | 75 | 6/13/2023 |
6.1.328-alpha | 73 | 6/13/2023 |
6.1.325-alpha | 73 | 5/22/2023 |
6.1.323-alpha | 70 | 5/22/2023 |
6.1.320-alpha | 178 | 5/20/2023 |
6.0.317-alpha | 78 | 5/17/2023 |
5.0.306-alpha | 75 | 5/15/2023 |
5.0.300-alpha | 83 | 5/10/2023 |
5.0.298-alpha | 71 | 5/9/2023 |
5.0.289-alpha | 75 | 5/7/2023 |
5.0.287-alpha | 75 | 5/7/2023 |