OutWit.EngineNode 1.2.3

OutWit.EngineNode

Compute node implementation for WitEngine distributed computing system.

Overview

WitEngineNode is the worker component that runs on each compute device in the WitCloud network. It loads controller plugins, executes activities received from the host engine, and provides benchmark data for intelligent task distribution.

Note

This package is not published to nuget.org. It is intended for deployment on compute nodes in the WitCloud infrastructure. For plugin development and testing, use OutWit.Engine.Sdk with WitEngineNodeSdk instead.

Features

• Plugin loading with isolated assembly contexts
• Activity execution for distributed workloads
• Benchmark execution for performance measurement
• Capability-based compatibility checking
• Binary serialization for network transfer

Architecture

WitEngineNode (Compute Node)
       |
       +-- ControllerManager     (plugin adapters registry)
       |
       +-- ProcessingManager     (activity execution)
       |
       +-- ResourcesManager      (localized resources)
       |
       +-- Loaded Controllers
              +-- Variables Controller
              +-- Special Controller
              +-- Matrices Controller
              +-- ... (other plugins)

Usage

Basic Setup

// Initialize the node with default settings
WitEngineNode.Instance.Reload();

// Or with custom configuration
WitEngineNode.Instance.Reload(
    id: nodeId,
    useIsolatedContext: true,
    logger: myLogger,
    moduleFolder: @"C:\Controllers"
);

Running Benchmarks

Each node runs benchmarks for all supported activities to measure its performance:

// Run benchmark for a specific activity type
var result = await WitEngineNode.Instance.RunBenchmark<WitActivityMatrixGustavsonMultiply>(
    options,
    cancellationToken
);

Console.WriteLine($"Rate: {result.Rate} {result.Unit}");
Console.WriteLine($"Iterations: {result.Iterations}");
Console.WriteLine($"Elapsed: {result.Elapsed}");

Benchmark results are used by the host engine to distribute work proportionally to each node's capabilities.

Checking Capability Compatibility

Before assigning work, the host can check if a node supports an activity:

// Check if the node can run an activity given hardware capabilities
bool canRun = WitEngineNode.Instance.IsCompatibleWith<WitActivityGpuCompute>(nodeCapabilities);

// Or by type
bool canRun = WitEngineNode.Instance.IsCompatibleWith(activityType, nodeCapabilities);

Compatibility is determined by [Requirement*] attributes on activity classes:

[RequiresCpu(minCores: 4)]
[RequiresGpu(minMemoryMb: 2048)]
[RequiresOs(OsType.Linux)]
public class WitActivityGpuCompute : WitActivityFunction { }

Processing Activities

The node processes individual activities sent from the host:

// Process an activity with variable pool
var (status, returnVariables) = await WitEngineNode.Instance.Process(
    jobId,
    activity,
    variablePool,
    returnVariableNames
);

if (status.Result == WitProcessingResult.Completed)
{
    // Activity completed successfully
    // returnVariables contains the requested output variables
}

Distributed Execution Flow

1. Host engine compiles a job with Grid.ForEach
2. Host queries nodes for compatibility and benchmark data
3. Host distributes work based on benchmark rates
4. Each node processes its assigned tasks
5. Nodes return results to host
6. Host aggregates results into final output

Host (WitEngine)                    Nodes (WitEngineNode)
      |                                   |
      |-- GetCompatibleNodes() ---------> |
      |<-- [Node1, Node2, Node3] -------- |
      |                                   |
      |-- Distribute tasks by Rate -----> |
      |                                   |
      |-- Process(task1) ---------------> Node1
      |-- Process(task2) ---------------> Node2
      |-- Process(task3) ---------------> Node3
      |                                   |
      |<-- result1 ---------------------- Node1
      |<-- result2 ---------------------- Node2
      |<-- result3 ---------------------- Node3
      |                                   |
      |-- Aggregate results               |

Project Structure

OutWit.EngineNode/
  WitEngineNode.cs           - Main node singleton
  Managers/
    ResourcesManager.cs      - Localized resources manager

Plugin Loading

Controllers are loaded from the @Controllers folder relative to the node assembly:

NodeApplication/
  WitEngineNode.dll
  @Controllers/
    variables.module/
      OutWit.Controller.Variables.dll
    special.module/
      OutWit.Controller.Special.dll
    matrices.module/
      OutWit.Controller.Matrices.dll

Each .module folder contains a controller plugin with its dependencies.

Isolated Assembly Contexts

By default, plugins are loaded in isolated assembly load contexts. This provides:

• Plugin isolation (no conflicts between plugin dependencies)
• Independent versioning of shared libraries
• Safe unloading capability

Set useIsolatedContext: false for simpler debugging during development.

Differences from WitEngineNodeSdk

Dependencies

• OutWit.Engine.Shared
• OutWit.Common.Logging
• OutWit.Common.Plugins

Related Packages

License

This software is licensed under the Non-Commercial License (NCL).

• Free for personal, educational, and research purposes
• Commercial use requires a separate license agreement
• Contact licensing@ratner.io for commercial licensing inquiries

See the full LICENSE file for details.