Test262Harness 1.0.1

dotnet add package Test262Harness --version 1.0.1                
NuGet\Install-Package Test262Harness -Version 1.0.1                
This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package.
<PackageReference Include="Test262Harness" Version="1.0.1" />                
For projects that support PackageReference, copy this XML node into the project file to reference the package.
paket add Test262Harness --version 1.0.1                
#r "nuget: Test262Harness, 1.0.1"                
#r directive can be used in F# Interactive and Polyglot Notebooks. Copy this into the interactive tool or source code of the script to reference the package.
// Install Test262Harness as a Cake Addin
#addin nuget:?package=Test262Harness&version=1.0.1

// Install Test262Harness as a Cake Tool
#tool nuget:?package=Test262Harness&version=1.0.1                

Build NuGet NuGet MyGet

Test262-Harness-dotnet

This is a .NET test runner for Test262: ECMAScript Test Suite. It includes parsing and downloading logic for the test suite in package Test262Harness and test suite generator functionality via CLI too, Test262Harness.Console

Usage

Following projects are utilizing the test suite generation and show how to create NUnit based test suite that is being generated by downloaded snapshot from test262 GitHub repository.

  • Jint
    • Generates NUnit test suite to ensure compliance, suite can be run in parallel for faster feedback loop
  • esprima-dotnet
    • Generates NUnit test suite for parsing tests, also has custom console logic to compare allow-list.txt for problematic files and progress getting the to parse properly
  • acornima
    • Generates NUnit test suite for parsing tests

First you need need to install the required package to your test project, it should look similar to this:

  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.9.0" />
    <PackageReference Include="NUnit" Version="4.0.1" />
    <PackageReference Include="NUnit3TestAdapter" Version="4.5.0" />
    <PackageReference Include="Test262Harness" Version="1.0.0" />
  </ItemGroup>

Next you will need a configuration, similar to this (also check the configuration format section):

Test262Harness.settings.json example

{
  "SuiteGitSha": "28b31c0bf1960878abb36ab8597a0cae224a684d",
  "TargetPath": "./Generated",
  "Namespace": "My.Tests.Test262",
  "ExcludedFeatures": [
    "Atomics",
    "Temporal"
  ],
  "ExcludedFlags": [
    "async" 
  ],
  "ExcludedDirectories": [
    "annexB",
    "intl402"
  ],
  "ExcludedFiles": [
    "language/expressions/object/dstr-async-gen-meth-*",
    "language/expressions/assignment/fn-name-lhs-cover.js"
  ]
}

You need to create minimal test file stub to initialize your testing target, example from Jint project.

using System;
using System.IO;
using Esprima;
using Jint.Native;
using Jint.Native.ArrayBuffer;
using Jint.Runtime;
using Jint.Runtime.Descriptors;
using Jint.Runtime.Interop;
using Test262Harness;

namespace Jint.Tests.Test262;

public static partial class State
{
    /// <summary>
    /// Pre-compiled scripts for faster execution.
    /// </summary>
    public static readonly Dictionary<string, Script> Sources = new(StringComparer.OrdinalIgnoreCase);
}

/// <summary>
/// Handles initializing testing state.
/// </summary>
public partial class TestHarness
{
    private static partial Task InitializeCustomState()
    {
        foreach (var file in State.HarnessFiles)
        {
            var source = file.Program;
            State.Sources[Path.GetFileName(file.FileName)] = new JavaScriptParser(source, new ParserOptions(file.FileName)).ParseScript();
        }

        return Task.CompletedTask;
    }
}

public abstract partial class Test262Test
{
    private Engine BuildTestExecutor(Test262File file)
    {
        var engine = new Engine(cfg =>
        {
            var relativePath = Path.GetDirectoryName(file.FileName);
            cfg.EnableModules(new Test262ModuleLoader(State.Test262Stream.Options.FileSystem, relativePath));
        });

        if (file.Flags.Contains("raw"))
        {
            // nothing should be loaded
            return engine;
        }

        engine.Execute(State.Sources["assert.js"]);
        engine.Execute(State.Sources["sta.js"]);

        // initialize engine with Test262 expected host defined functions here
        // https://github.com/tc39/test262/blob/main/INTERPRETING.md#host-defined-functions

        engine.SetValue("print", new ClrFunction(engine, "print", (_, args) => TypeConverter.ToString(args.At(0))));
        // and more...

        // the cinded files that that are expected
        foreach (var include in file.Includes)
        {
            engine.Execute(State.Sources[include]);
        }

        return engine;
    }

    private static void ExecuteTest(Engine engine, Test262File file)
    {
        if (file.Type == ProgramType.Module)
        {
            engine.AddModule(file.FileName, builder => builder.AddSource(file.Program));
            engine.ImportModule(file.FileName);
        }
        else
        {
            engine.Execute(new JavaScriptParser(file.Program, new ParserOptions(file.FileName)).ParseScript());
        }
    }

    private partial bool ShouldThrow(Test262File testCase, bool strict)
    {
        return testCase.Negative;
    }
}

And also the CLI tool for generating the test suite, run this in you test project directory.

dotnet tool add Test262Harness.Console

When everything is installed, you should be able to run:

dotnet tool restore
dotnet test262 generate

Test262Harness.settings.json configuration file

List of most important things you can tweak in configuration file:

Key Default Description
SuiteGitSha none The GitHub commit to use when downloading the test suite
SuiteDirectory none Alternatively, you can point to local repository root
TargetPath none Where to generate the file to
Namespace Test262Harness.TestSuite Namespace for the generated source files
ExcludedFeatures [] Any feature you want to ignore
ExcludedFlags [] Any flag you want to ignore
ExcludedDirectories [] Any sub-directory you would like to ignore, for example annexB
ExcludedFiles [] List of specific files you would like to ignore

Exclusion maps to setting [Ignore] attribute in test suite.

Branches and releases

Product Compatible and additional computed target framework versions.
.NET net5.0 was computed.  net5.0-windows was computed.  net6.0 is compatible.  net6.0-android was computed.  net6.0-ios was computed.  net6.0-maccatalyst was computed.  net6.0-macos was computed.  net6.0-tvos was computed.  net6.0-windows was computed.  net7.0 was computed.  net7.0-android was computed.  net7.0-ios was computed.  net7.0-maccatalyst was computed.  net7.0-macos was computed.  net7.0-tvos was computed.  net7.0-windows was computed.  net8.0 is compatible.  net8.0-android was computed.  net8.0-browser was computed.  net8.0-ios was computed.  net8.0-maccatalyst was computed.  net8.0-macos was computed.  net8.0-tvos was computed.  net8.0-windows was computed. 
.NET Core netcoreapp2.0 was computed.  netcoreapp2.1 was computed.  netcoreapp2.2 was computed.  netcoreapp3.0 was computed.  netcoreapp3.1 was computed. 
.NET Standard netstandard2.0 is compatible.  netstandard2.1 was computed. 
.NET Framework net461 was computed.  net462 was computed.  net463 was computed.  net47 was computed.  net471 was computed.  net472 was computed.  net48 was computed.  net481 was computed. 
MonoAndroid monoandroid was computed. 
MonoMac monomac was computed. 
MonoTouch monotouch was computed. 
Tizen tizen40 was computed.  tizen60 was computed. 
Xamarin.iOS xamarinios was computed. 
Xamarin.Mac xamarinmac was computed. 
Xamarin.TVOS xamarintvos was computed. 
Xamarin.WatchOS xamarinwatchos was computed. 
Compatible target framework(s)
Included target framework(s) (in package)
Learn more about Target Frameworks and .NET Standard.

NuGet packages (1)

Showing the top 1 NuGet packages that depend on Test262Harness:

Package Downloads
Test262Harness.Generator

Test262 Harness for .NET

GitHub repositories (2)

Showing the top 2 popular GitHub repositories that depend on Test262Harness:

Repository Stars
sebastienros/jint
Javascript Interpreter for .NET
sebastienros/esprima-dotnet
Esprima .NET (BSD license) is a .NET port of the esprima.org project. It is a standard-compliant ECMAScript parser (also popularly known as JavaScript).
Version Downloads Last updated
1.0.1 905 7/15/2024
1.0.0 1,649 2/16/2024