Nice3point.BenchmarkDotNet.Revit 2022.1.2

Benchmarking library for Revit

Write performance benchmarks for your Revit add-ins using the BenchmarkDotNet, and share reproducible measurement experiments.

Installation

You can install this library as a NuGet package.

The packages are compiled for specific versions of Revit. To support different versions of libraries in one project, use the RevitVersion property:

<PackageReference Include="Nice3point.BenchmarkDotNet.Revit" Version="$(RevitVersion).*"/>

Writing your first benchmark

Start by creating a new class inheriting from RevitApiBenchmark:

public class MyBenchmarks : RevitApiBenchmark
{

}

Add one or more methods marked with [Benchmark]:

using BenchmarkDotNet.Attributes;

public class MyBenchmarks : RevitApiBenchmark
{
    [Benchmark]
    public void MyBenchmark()
    {

    }
}

This is your runnable benchmark. The base class ensures the benchmark executes within Revit's single-threaded API context.

Running your benchmarks

You can run your benchmarks with a simple configuration. BenchmarkDotNet uses the Release configuration by default to run, which will cause a failure when running benchmarks for Revit, where API multi-versioning is required. In this case, use the WithCurrentConfiguration() extension for the Job, which will run the benchmark for your selected Solution configuration.

using BenchmarkDotNet.Configs;
using BenchmarkDotNet.Jobs;
using BenchmarkDotNet.Running;
using Nice3point.BenchmarkDotNet.Revit;

var configuration = ManualConfig.Create()
    .AddJob(Job.Default.WithCurrentConfiguration());

BenchmarkSwitcher.FromAssembly(typeof(Program).Assembly)
    .Run(args, configuration);

BenchmarkSwitcher runs the benchmarks the command-line arguments select:

dotnet run -c Release.R26 -- --filter '*MyBenchmarks*'

Application-level benchmarks

Benchmark Revit application-level operations:

using BenchmarkDotNet.Attributes;

public class RevitApplicationBenchmarks : RevitApiBenchmark
{
    [Benchmark]
    public XYZ NewXyz()
    {
        return new XYZ(3, 4, 5);
    }

    [Benchmark]
    public XYZ CreateNewXyz()
    {
        return Application.Create.NewXYZ(3, 4, 5);
    }
}

Document benchmarks

Use OnGlobalSetup and OnGlobalCleanup overrides to open and close a document around benchmark iterations. BenchmarkDotNet provides [GlobalSetup] and [GlobalCleanup] hooks, but due to library limitations, they cannot be assigned twice:

using BenchmarkDotNet.Attributes;

public class RevitCollectorBenchmarks : RevitApiBenchmark
{
    private Document _document = null!;

    protected sealed override void OnGlobalSetup()
    {
        _document = Application.NewProjectDocument(UnitSystem.Metric);
    }

    protected sealed override void OnGlobalCleanup()
    {
        _document.Close(false);
    }

    [Benchmark]
    public IList<Element> WhereElementIsNotElementTypeToElements()
    {
        return new FilteredElementCollector(_document)
            .WhereElementIsNotElementType()
            .ToElements();
    }

    [Benchmark]
    public List<Element> WhereElementIsNotElementTypeToList()
    {
        return new FilteredElementCollector(_document)
            .WhereElementIsNotElementType()
            .ToList();
    }
}

BenchmarkDotNet v0.15.8, Windows 11 (10.0.22631.6199/23H2/2023Update/SunValley3)
AMD Ryzen 5 5600 3.50GHz, 1 CPU, 12 logical and 6 physical cores
.NET SDK 10.0.100
  [Host]    : .NET 8.0.22 (8.0.22, 8.0.2225.52707), X64 RyuJIT x86-64-v3
  MediumRun : .NET 8.0.22 (8.0.22, 8.0.2225.52707), X64 RyuJIT x86-64-v3

Job=MediumRun  BuildConfiguration=Release.R26  IterationCount=15
LaunchCount=2  WarmupCount=10

User interface benchmarks

Benchmark the functionality that needs the Revit user interface by deriving from RevitApiUiBenchmark. The class exposes UiApplication, and runs every benchmark method and hook on the Revit thread inside a Revit API context. Use OnGlobalSetup and OnGlobalCleanup overrides to open a model around benchmark iterations:

using Autodesk.Revit.UI;
using BenchmarkDotNet.Attributes;

public class RevitSelectionBenchmarks : RevitApiUiBenchmark
{
    private UIDocument _uiDocument = null!;

    protected sealed override void OnGlobalSetup()
    {
        _uiDocument = UiApplication.OpenAndActivateDocument(modelPath);
    }

    [Benchmark]
    public List<Wall> GetElementOfTypeToList()
    {
        var document = _uiDocument.Document;
        return _uiDocument.Selection.GetElementIds()
            .Select(document.GetElement)
            .OfType<Wall>()
            .ToList();
    }

    [Benchmark]
    public List<Wall> FilteredElementCollectorOfClassToList()
    {
        return new FilteredElementCollector(_uiDocument.Document, _uiDocument.Selection.GetElementIds())
            .OfClass(typeof(Wall))
            .Cast<Wall>()
            .ToList();
    }
}

UI benchmarks share one Revit instance, which starts with the first UI benchmark and closes when the run ends. Asynchronous benchmark methods and the job settings of the benchmark process, such as the runtime and the GC mode, are not supported.

DotTrace profiler

Profile your benchmarks with dotTrace by adding the DotTraceDiagnoser from the BenchmarkDotNet.Diagnostics.dotTrace package to the configuration:

using BenchmarkDotNet.Configs;
using BenchmarkDotNet.Diagnostics.dotTrace;
using BenchmarkDotNet.Jobs;
using BenchmarkDotNet.Running;
using Nice3point.BenchmarkDotNet.Revit;

var configuration = ManualConfig.Create()
    .AddJob(Job.Default.WithCurrentConfiguration())
    .AddDiagnoser(new DotTraceDiagnoser());

BenchmarkSwitcher.FromAssembly(typeof(Program).Assembly)
    .Run(args, configuration);

The diagnoser profiles RevitApiBenchmark and RevitApiUiBenchmark classes alike. To profile a single class, mark it with the [DotTraceDiagnoser] attribute.

Benchmark configuration

BenchmarkDotNet initializes Revit with the English - United States language, the C:\Program Files\Autodesk\Revit {version} installation path, and an add-in with a random ID. To override these defaults:

• Add the assembly-level attribute to any .cs file in your project (e.g., Program.cs):

  using Nice3point.Revit.Injector.Attributes;
  
  [assembly: RevitLanguage("ENU")]
  [assembly: RevitInstallationPath(@"D:\Autodesk\Revit Preview")]
  [assembly: RevitClient(Id = "B3E7F5A2-9C4D-4E81-A6F0-2D8C51E7B934", Name = "Benchmarks", VendorId = "Contoso")]
  

• Or add the attributes directly to your .csproj file:

  
  <ItemGroup>
  
      <AssemblyAttribute Include="Nice3point.Revit.Injector.Attributes.RevitLanguageAttribute">
          <_Parameter1>ENU</_Parameter1>
      </AssemblyAttribute>
  
      <AssemblyAttribute Include="Nice3point.Revit.Injector.Attributes.RevitInstallationPathAttribute">
          <_Parameter1>D:\Autodesk\Revit $(RevitVersion)</_Parameter1>
      </AssemblyAttribute>
  
      <AssemblyAttribute Include="Nice3point.Revit.Injector.Attributes.RevitClientAttribute">
          <Id>B3E7F5A2-9C4D-4E81-A6F0-2D8C51E7B934</Id>
          <Name>Benchmarks</Name>
          <VendorId>Contoso</VendorId>
      </AssemblyAttribute>
  
  </ItemGroup>
  

The RevitLanguage attribute accepts a language name (e.g., "English - United States"), code (e.g., "ENU") or LanguageType enum value (e.g., "English_GB" or "15").

The RevitClient attribute sets the add-in that Revit attributes every API call of the benchmark to, and Application.ActiveAddInId returns it. Each property is optional, Id accepts any format Guid accepts, VendorId must match the vendor of an Extensible Storage schema the benchmark writes with vendor access.