JustChecking 1.1.0

JustChecking

JustChecking is an API namespace that assists with creating automated checks. Checks are observations made of computationally measurable facts where we want to report when a fact is demonstrated to be untrue. Checks are used part of testing a product behavior or attributes.

Usage

The recommended way to use the JustChecking namespace is to either derive the automated checks from CheckBaseClass, or wrap CheckBaseClass in case you intend to apply custom configuration of the logging objects. From CheckBaseClass, you can derive a configured Check class, and from that derive all the other classes as well as logging all objects, all consistently configured. From the Check object you can derive CheckFailsIf and NonProductFailure objects to record check failures and failures not against the product, respectively. Alternatively, you can use the ExpectedSteps object to allow product failure if necessary steps are skipped.

Hierarchical Logging

Logging in JustChecking is hierarchical, in this order:

CheckBaseClass
  Step
    [Step, ExpectedSteps, Check, NonProductFailure, LogEntry]
  Check
    CheckFailsIf
    [Step, ExpectedSteps, Check, NonProductFailure, LogEntry]
  ExpectedSteps
    Step
    [ExpectedSteps, Check, NonProductFailure, LogEntry]

Example code

The following C# code is a demonstration of using a Check and ExpectedSteps object to test an API on an object.

The following is the constructor for the base test class that wraps CheckBaseClass.

    public XmlFileBasedLoggingSUTBaseClass ([System.Runtime.CompilerServices.CallerMemberNameAttribute] string mainCheckName="")
    {
        string filePath = $"{mainCheckName}_{DateTime.Now.Year}_{DateTime.Now.Month}_{DateTime.Now.Day}_{DateTime.Now.Hour}_{DateTime.Now.Minute}_{DateTime.Now.Second}.log";
        // creating a duplicate logger we can use after the one we give to CheckBaseClass is disposed
        this._structuredLogger = InitializeLogger(filePath);
        this._structuredLogger.WriteLogEntry(new StructuredLogEntry() { EntryClass = "SUTBASECLASS", EntryGroup = EntryGroup.GroupBegin });
        _checkBase = new CheckBaseClass(InitializeLogger(filePath));
    }

Here is the wrapped CheckBaseClass in use. It starts with a usage statement that gets the Check object from the wrapped CheckBaseClass.

    [TestMethod]
    public void AddString()
    {
        using (JustChecking.Check check = this.CheckBase.Check())
        {
            using(JustChecking.ExpectedSteps expectedSteps = this.CheckBase.ExpectedSteps(new string[] { "create SUT", "add string 1", "add string 2", "check sut content" }))
            {
                SUT sut = null;
                expectedSteps.Step("create SUT", () => {  sut = new SUT(); });
                expectedSteps.Step("add string 1", () => { sut.AddString("string1"); });
                expectedSteps.Step("add string 2", () => { sut.AddString("string2"); });
                expectedSteps.Step("check sut content", () => { check.CheckFailsIf().AreNotEqual(2, sut.Strings.Length, "string count in sut compared to expected count.");});
            }
        }
    }

API

CheckBaseClass

Convenient class to either derive a checking class from, or to wrap with a class permitting non-default configuration. Provides access to configured Check, ExpectedSteps, NonProductFailure classes. All checking events executed after a CheckBaseClass is created are treated as child events inside the log until the CheckBaseClass object is disposed.

Check

Allows one to name their check, and provides convenient access to CheckfailsIf event. Implements IDisposable, making it most convenient to use within a using statement. All events in the scope of the using statement are treated as child events in the log.

CheckFailsIf

CheckFailsIf is the primary class in the API that exposes a number of comparison methods used to establish whether or not a given fact is untrue. CheckFailsIf is an instance rather than static class, allowing for proper isolation of logging behaviors in the case of parallel script execution. This is similar to other APIs used in automated product checking, except that rather than asserting correct behavior, we state the conditions under which a check is going to fail. This conforms more correctly and consistently with the realities of observation and checking, in that we cannot prove correctness of a product, but we can disprove correctness with evidence.

ExpectedSteps

A class that allows a script to specify that a set of steps are expected. ExpectedSteps implements IDisposable, and will report a check failure if any of the steps specified are not executed. Steps are executed by calling the Step method with the name of the corresponding step and a delegate action to perform the step. Steps can likewise call steps, which are represented in logged output hierarchically.

NonProductFailure

A class that will log failure when created. It is intended for use when the automated script knows for certain that a given failure is not due to any product under test behaviors. The caller may specify categories of reasons indicating if the failure occured prior to, after product interactions, or were simply during actions not related to product interaction.

IStructuredLogger

Takes structured log entries and converts them to formatted output, using ILogWriter objects to record logs to different storage locations.

StructuredLogEntry

A class used for writing log entries to an IStructuredLogger object. The Class field identifies the type of log entry, and defaults to the name of the method calling it. Source is usually the full type name of the object creating the log entry. Id varies per entry, for example on a check, it is the name of the check passed on Check object creation. The Type object indicates if the log entry indicates failure, just a warning, or is informative (there is no notion of "PASS", the equivalent being that a check execution is recorded as Information entry type). EntryGroup is used to indicate if the entry indicates the beginning or ending of a hierarchy group, or if it is a single line entry.

public class StructuredLogEntry
{
    public string Source = MethodBase.GetCurrentMethod().DeclaringType.FullName;
    public string Id;
    public string EntryClass = MethodBase.GetCurrentMethod().Name;
    public EntryType Type;
    public EntryGroup EntryGroup;
    public string Message;
}

XmlStructuredLogger

Implements IStructuredLogger and converts log entries to Xml format.

PlainStructuredLogger

Implements IStructuredLogger and converts log entries to CSV format.

ILogWriter

ILogger is a logging interface passed to checking objects that allow writing of diagnostic information to some location. It is a public interface, permitting integration of JustChecking with various data and reporting sources.

ConsoleLogger

Implements ILogger and writes information to standard output.

FileBasedLogger

Implements ILogger and writes information to a specified file path.