Arch 1.0.8

Arch

A C# based Archetype Entity Component System (ECS).

Each Archetype stores their entities within 16KB sized chunks perfectly fitting into L1 Caches for maximum iteration performance.
This technique has two main advantages, first of all it provides an great entity allocation speed and second it lowers the cache misses to the best possible minimum. Its incredible fast, especially for well architectured component structures.

Supports .NetStandard 2.1, .Net Core 6 and 7.
Since .NetStandard is supported, you may also use it with Unity.

Download the package and get started today !

dotnet add PROJECT package Arch --version 1.0.8

Code Sample

public class Game {

    public struct Position { public float x, y; }
    public struct Velocity { public float dx, dy; }
    
    // The entity structure and or filter/query
    public static Type[] archetype = { typeof(Position), typeof(Velocity) };
    
    public static void Main(string[] args) {
        
        var world = World.Create();
        var query = new QueryDescription{ All = archetype };  // Query all entities with Position AND Velocity components

        // Create entities
        for (var index = 0; index < 1000; index++) {

            var entity = world.Create(archetype);
            entity.Set(new Position{ x = 0, y = 0});
            entity.Set(new Velocity{ dx = 1, dy = 1});
        }

        // Query and modify entities 
        world.Query(in query, (ref Position pos, ref Velocity vel) => {
            pos.x += vel.dx;
            pos.y += vel.dy;
        });
    }
}

Content

• Quickstart
  • ECS
  • World
  • Entity
  • Querying and Filtering
• Advanced usage
  • Highperformance Queries
  • Bulk adding
• Internal Structure and Memory layout
  • Archetype
  • Chunks
  • Archetype and Chunk usage
  • How to avoid pitfalls
• Performance
  • Benchmark
    • NET.7
    • Different Iteration & Acess Techniques
    • Legend

Quickstart

ECS

Entity Component System (ECS) is a software architectural pattern mostly used for the representation of game world objects or data oriented design in general. An ECS comprises entities composed from components of data, with systems or queries which operate on entities' components.

ECS follows the principle of composition over inheritance, meaning that every entity is defined not by a type hierarchy, but by the components that are associated with it.

World

The world acts as a management class for all its entities, it contains methods to create, destroy and query them and handles all the internal mechanics.
Therefore it is the most important class, you will use the world heavily.
Multiple worlds can be used in parallel, each instance and its entities are completly encapsulated from other worlds. Currently worlds and their content can not interact with each other, however this feature is already planned.

Worlds are created and destroyed like this...

var world = World.Create();
World.Destroy(world);

There can be up to 255 worlds in total.

Entity

A entity represents your game entity.
It is a simple struct with some metadata acting as a key to acess and manage its components.

public readonly struct Entity : IEquatable<Entity> {
        
    public readonly int EntityId;    // Its id/key in the world
    public readonly byte WorldId;    // The world the entity lives in
    public readonly ushort Version;  // Its version, how often the entity or its id was recycled
    ....
}

Entities are being created by a world and will "live" in the world in which they were created.
When an entity is being created, you need to specify the components it will have. Components are basically the additional data or structure the entity will have. This is called "Archetype".

var archetype = new []{ typeof(Position), typeof(Velocity), ... };
var entity = world.Create(archetype);
world.Destroy(in entity);

To ease writing code, you can acess the entity directly to modify its data or to check its metadata.
Lets take a look at the most important methods.

entity.IsAlive();                     // True if the entity is still existing in its world
entity.Has<Position>();               // True if the entity has a position component
entity.Set(new Position( x = 10 ));   // Replaces the position component and updates it data
entity.Get<Position>();               // Returns a reference to the entity position, can directly acess and update position attributes

entity.GetComponentTypes();           // Returns an array of its component types. Should be treated as readonly 
entity.GetComponents();               // Returns an array of all its components, allocates memory. 

With those utility methods you are able to implement your game logic.
A small example looks like this...

var archetype = new []{ typeof(Position), typeof(Velocity) };
var entity = world.Create(archetype);

ref var position = ref entity.Get<Position>();    // Get reference to the position
position.x++;                                     // Update x
position.y++;                                     // Update y

if(entity.Has<Position>())                        // Make sure that entity has a position ( Optional )
    entity.Set(new Position{ x = 10, y = 10 };    // Replaces the old position 

Querying and Filtering

To performs operations and to define your game logic, queries are used to iterate over entities.
This is performed by using the world ( remember, it manages your created entities ) and by defining a description of which entities we want to iterate over.

// Define a description of which entities you want to query
var query = new QueryDescription {
    All = new []{ typeof(Position), typeof(Velocity) },   // Should have all specified components
    Any = new []{ typeof(Player), typeof(Projectile) },   // Should have any of those
    None = new []{ typeof(AI) }                           // Should have none of those
};

// Execute the query
world.Query(in query, entity => { /* Do something */ });

// Execute the query and modify components in the same step, up to 10 generic components at the same time. 
world.Query(in query, (ref Position pos, ref Velocity vel) => {
    pos.x += vel.dx;
    pos.y += vel.dy;
});

In the example above we want to move our entities based on their Position and Velocity components. To perform this operation we need to iterate over all entities having both a Position and Velocity component (All). We also want that our entity either is a Player or a Projectile (Any). However, we do not want to iterate and perform that calculation on entities which are controlled by an AI (None).

The world.Query method than smartly searches for entities having both a Position and Velocity, either a Player or Projectile and no AI component and executes the defined logic for all of those fitting entities.

Its also important to know that there are multiple different overloads to perform such a query.

The less you query in terms of components and the size of components... the faster the query is !

world.Query(in query, entity => {});                                     // Passes the fitting entity
world.Query(in query, (ref T1 t1, T2 t2, ...) => {})                     // Passes the defined components of the fitting entity, up to 10 components
world.Query(in query, (in Entity en, ref T1 t1, ref T2 t2, ...) => {})   // Passed the fitting entity and its defined components, up to 10 components 

var filteredEntities = new List<Entity>();
var filteredArchetypes = new List<Archetype>();
var filteredChunks = new List<Chunk>();

world.GetEntities(query, filteredEntities);                             // Fills all fitting entities into the passed list
world.GetArchetypes(query, filteredArchetypes);                         // Fills all fitting archetypes into the list
world.GetChunks(query, filteredChunks);                                 // Fills all fitting chunks into the list 

Archetype's and Chunk's are internal structures of the world and store entities with the same component types. You will mostly never use them directly, therefore more on them later.

Advanced usage

Highperformance Queries

The default Query API is easy to use and still very fast, perfect for fast prototyping and the most features of your game. However sometimes you need even more power, thats where the highperformance queries kick in.

world.HPQuery<Struct,T0,T1...>(in queryDescription, ref myStruct);
world.HPEQuery<Struct,T0,T1...>(in queryDescription, ref myEStruct);

Those highperformance queries use an interfacee and its struct implementation. This allows the compiler to inline the method call which results in less adress jumping and even faster iteration speed. Therefore you need to know two important interfaces and how to implement them.

public interface IForEach<T0...T10>{
    void Update(ref T0 t0, ref T1 t1, ... ref T10 t10);
}

public interface IForEachEntity<T0...T10>{
    void Update(in Entity entity, ref T0 t0, ref T1 t1, ... ref T10 t10);
}

Those two interfaces provide Update methods with various generic overloads which can be used to implement the entity operations. All you need to do is implementing the interface in a struct and passing that struct to the highperformance query api.

public struct VelocityUpdate : IForEach<Position, Velocity> {

    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public void Update(ref Position pos, ref Velocity vel) { 
        pos.x += vel.x;
        pos.y += vel.y;
    }
}

world.HPQuery<VelocityUpdate, Position, Velocity>(in queryDescription);


// Also possible with a struct reference
public struct VelocityUpdate : IForEach<Position, Velocity> {

    public int counter;

    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public void Update(ref Position pos, ref Velocity vel) { 
        pos.x += vel.x;
        pos.y += vel.y;
        counter++;
    }
}

var velUpdate = new VelocityUpdate();
world.HPQuery<VelocityUpdate, Position, Velocity>(in queryDescription, ref velUpdate);
Console.WriteLine(velUpdate.counter);

Thats all, pretty cool right ? However in some cases you may also need a direct reference to the entity itself. In this case theres the IForEachEntity interface which is required.

public struct VelocityUpdate : IForEachEntity<Position, Velocity> {

    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public void Update(in Entity entity, ref Position pos, ref Velocity vel) { 
        pos.x += vel.x;
        pos.y += vel.y;
        Console.WriteLine(entity);
    }
}

world.HPEQuery<VelocityUpdate, Position, Velocity>(in queryDescription);  // <- Requires HPEQuery instead of HPQuery

Bulk adding

Arch supports bulk adding of entities, this is incredible fast since it allows us to allocate enough space for a certain set of entities in one go. This reservation happens on top of the already existing entities in an archetype. You only need to reserve space once and than it will be filled later or sooner.

var archetype = new []{ typeof(Position), typeof(Velocity) };

// Create 1k entities
for(var index = 0; index < 1000; index++)
    world.Create(archetype)

world.Reserve(archetype, 1000000);              // Reserves space for additional 1mil entities
for(var index = 0; index < 1000000; index++)    // Create additional 1 mil entities
    world.Create(archetype)

// In total there now 1mil and 1k entities in that certain archetype. 

Internal Structure and Memory layout

Arch is an archetype ecs. An archetype ecs groups entities with the same set of components in tightly packed arrays for the fastest possible iteration performance. This has no direct effect on its API useage or the way you develop your game. But understanding the internal structure can help you to improve your game performance even more. However it has an big impact on the internal structures being used and is the secret to its incredible performance.

Archetype

An archetype manages all entities with a common set of components. Like the world is used to manage ALL entities, the archetype is only used to manage a specific set of entities... all entities with the same component structure. The world stores those archetypes and acesses them to iterate, create, update and remove entities.

// Creates one entity, one archetype where all entities with Position and Velocity will be stored
var archetype = new []{ typeof(Position), typeof(Velocity) };
var entity = world.Create(archetype);                            

// Creates another entity, another archetype where all entities with Position, Velocity AND Rotation will be stored
var secondArchetype = new []{ typeof(Position), typeof(Velocity), typeof(Rotation) } ; 
var secondEntity = world.Create(secondArchetype);

You may probably now ask : "Why do this create two seperate archtypes ? Both entities share position and velocity, so they could be stored together."... Shared subsets of component do NOT matter... all what matters is the exact structure of an entity. Why is that ? Because this way we can utilize the cache during iterations, however explaining this would probably break the scope of this documentation.

Chunks

Chunks are where the entities and their components are stored. They utilize dense packed contiguous arrays ( SoA ) to store and acess entities with the same group of components. Its internal arrays are always 16KB in total, this is intended since 16kb fits perfectly into the L1 CPU Cache which gives us insane iteration speeds.

The internal structure simplified looks like this...

Chunk
[
    [Entity, Entity, Entity],
    [Position, Position, Position],
    [Velocity, Velocity, Velocity]
    ...
]

This way they are fast to (de)allocate which also reduces memory useage to a minimum. Each archetype contains multiple chunks and will create and destroy chunks based on worlds need.

Archetype and Chunk usage

Arch gives you acess to the internal structures aswell. You will mostly do not need this feature, however it can be usefull to leverage the performance, writing custom queries or add new features.

var archetypes = world.Archetypes;  // Returns direct acess to all archetypes of the world
var chunks = archetypes[0].Chunks   // Returns direct acess to all chunks of a archetype
 
world.GetArchetypes(in queryDescription, myArchetypeList);  // Fills all archetypes fitting the query description into the passed list
world.GetChunks(in queryDescription, myChunkList);          // Fills all chunks fitting the query description into the passed list

Acessing or using those methods, you will have the power to extend the features of this lib and acess its underlaying foundation. It can be used for e.g. :

• Writing custom queries
• Implementing different iteration techniques
• Serialisation
• Multithreading
• SIMD
• Code generation

Lets take a look at how the world.Query methods are implemented using that raw acess of archetypes and chunks.

// Inside the world
public void Query<T0>(in QueryDescription description, ForEach<T0> forEach)
{

    // Cache query
    if (!QueryCache.TryGetValue(description, out var query))
    {
        query = new Query(description);
        QueryCache[description] = query;
    }

    var size = Archetypes.Count;
    for (var index = 0; index < size; index++)
    {
        var archetype = Archetypes[index];      
        var archetypeSize = archetype.Size;
        var bitset = archetype.BitSet;
        
        if (!query.Valid(bitset)) continue;  // Process archetype & chunks if it fits the query description
        
        // Loop over all its chunks in an unsafe manner
        ref var chunkFirstElement = ref archetype.Chunks[0];
        for (var chunkIndex = 0; chunkIndex < archetypeSize; chunkIndex++)
        {
            ref readonly var chunk = ref Unsafe.Add(ref chunkFirstElement, chunkIndex);
            var chunkSize = chunk.Size;

            var t0Array = chunk.GetArray<T0>();        // Acess component array
            ref var t0FirstElement = ref t0Array[0];

            // Loop over all components in an unsafe manner
            for (var entityIndex = 0; entityIndex < chunkSize; entityIndex++)
            {
                ref var t0Component = ref Unsafe.Add(ref t0FirstElement, entityIndex);
                forEach(ref t0Component);
            }
        }
    }
}

With the power of acessing Archetype and Chunk directly from the world, you could easily write such high performance queries yourself. Great ! Isnt it ? 😃
Lets look at an small example of how you could utilize those features.

Lets say we want to iterate over every second entity with a Position component, since we dont wanna update ALL of them every tick. Well, you could do this easily like this e.g.

var archetype = new[]{ typeof(Position); }
var queryDesc = new QueryDescription{ All = archetype };

var allChunks = new List<Chunk>();
world.Query(in queryDesc, allChunks); // Get all chunks for entities with a position component

foreach(var chunk in allChunks){

    // Acess the position component array and loop over each position
    var positions = chunk.GetArray<Position>();
    for (var entityIndex = 0; entityIndex < chunk.Size; entityIndex+=2){ // <- Always skip one entity
    
        ref var pos = ref positions[entityIndex];
        pos.x++;
        pos.y++;

        Console.WriteLine($"{pos.x}/{pos.y}");
    }
}

How to avoid pitfalls

Since Archetype and Chunk do give you raw acess to the underlaying memory and foundation, there are some actions which should be avoided.

• Adding/Removing entities directly to Archetype and Chunk, use always the World for this
• Trimming or moving the internal arrays and hashmaps

Performance

Well... its fast, like REALLY fast.
However the iteration speed depends, the less you query, the faster it is.
This rule targets the amount of queried components aswell as their size.

Based on https://github.com/Doraku/Ecs.CSharp.Benchmark - Benchmark, it is among the fastest ecs frameworks in terms of allocation and iteration.

Benchmark

The current Benchmark only tests it Archetype/Chunk iteration performance.
Two different iteration techniques, 2 Components ( Transform & Rotation ) modification and Entity + 2 Components modification.

public struct Transform{ float x; float y; float z; }
public struct Rotation{ float x; float y; float z; float w; }

The used structs are actually quite big, the smaller the components, the faster the query. However i wanted to create a realistic approach and therefore used a combination of Transform and Rotation.

NET.7

Different Iteration and Acess Techniques

We have been testing different strategies and techniques to iterate over the archetype itself and all its chunks for providing the best overall performance. Suprisingly all of them are great but especially the Unsafe.Add Iterations were quite faster. Thats why we picked the techniques of IterationUnsafeAddTwoComponents and IterationUnsafeAddTwoComponentsUnsafeArray and decided that IterationUnsafeAddTwoComponents is the best overall since it comes along the least CacheMisses. We will run this benchmark regulary in the future to adjust the ECS performance based on new .NET improvements.

Legend

Legends

• amount : Value of the 'amount' parameter
• Mean : Arithmetic mean of all measurements
• Error : Half of 99.9% confidence interval
• StdDev : Standard deviation of all measurements
• Median : Value separating the higher half of all measurements (50th percentile)
• Allocated : Allocated memory per single operation (managed only, inclusive, 1KB = 1024B)
• 1 μs : 1 Microsecond (0.000001 sec)