BitwiseSerialiser 1.0.0

BitwiseSerialiser

Bitwise serialiser/deserialiser for dotnet. Handles endian and alignment issues.

Also contains various small helpers for byte stream formatting and display.

Use

Structures

To use, you need to create classes for your data structures, and include fields (not properties) to hold the binary data.

Decorate the class with [ByteLayout] attribute, and each field in the class with one of the BigEndian / LittleEndian / ByteString attributes.

Each of the field attributes has an order parameter. There must be an unbroken sequence from order: 0 up across all fields in the class.

[ByteLayout]
public class MyByteStructure
{
    [BigEndian(bytes: 2, order: 0)]
    public UInt16 StartMarker;
    
    [BigEndian(bytes: 3, order: 1)]
    public UInt32 ThreeBytesBig;
    
    [LittleEndian(bytes: 3, order: 2)]
    public UInt32 ThreeBytesSmall;
    
    [LittleEndian(bytes: 2, order: 3)]
    public UInt16 EndMarker;
}

The serialisers will ignore properties. This is intentional, so that properties can represent the code-side typed data (including any conversions)

Serialisation and Deserialisation

To get a byte array from a data structure, use

byte[] data = ByteSerialiser.ToBytes(myClass);

To restore a data structure from a byte array, use

byte[] data = ...
var ok = ByteSerialiser.FromBytes<MyByteLayoutClass>(data, out var resultingClass);

The returned ok value will be true if the serialiser read the supplied number of bytes or less. ok will be false if the structure required more bytes than were supplied.

Attributes

[ByteLayout]

class: no parameters

All classes in a structure used by the ByteSerialiser should be marked with this attribute.

[BigEndian]

field: bytes, order

Represents an unsigned integer value, taking the given number of bytes (1..8), Most Significant Byte first. Can handle non standard byte counts (e.g. 3 bytes into a UInt32). The field type must be large enough to hold the value, and be one of byte, UInt16, UInt32, UInt64, Int16, Int32, Int64.

[ByteLayout]
public class SimpleByteStructure
{
    [BigEndian(bytes: 2, order: 0)]
    public UInt16 TwoByteValue;
}

Note that BigEndian and LittleEndian values can be mixed inside a structure.

[LittleEndian]

field: bytes, order

Represents an unsigned integer value, taking the given number of bytes (1..8), Least Significant Byte first. Can handle non standard byte counts (e.g. 3 bytes into a UInt32). The field type must be large enough to hold the value, and be one of byte, UInt16, UInt32, UInt64, Int16, Int32, Int64.

[ByteLayout]
public class SimpleByteStructure
{
    [LittleEndian(bytes: 2, order: 0)]
    public UInt16 TwoByteValue;
}

Note that BigEndian and LittleEndian values can be mixed inside a structure.

[BigEndianPartial]

field: bits, order

Represents an unsigned integer value, taking the given number of BITS (1..64), Most Significant Byte first. Can handle non standard bit counts (e.g. 13 bits into a UInt16)

A sequence of BigEndianPartial attributes should line up to a byte boundary. It is not required, but subsequent byte access will be unaligned, and slower.

There is no little-endian variant of this attribute

[ByteLayout]
public class OneByteStructure
{
    [BigEndianPartial(bits:3, order: 0)]
    public byte FirstThreeBits;
    
    [BigEndianPartial(bits:2, order: 1)]
    public byte MiddleTwoBits;
    
    [BigEndianPartial(bits:3, order: 1)]
    public byte LastThreeBits;
}

[ByteString]

field: bytes, order

Represents a known-length list of bytes in input order

The field type must be byte[], and the array is read and written in stream order.

[ByteLayout]
public class FixedArrayStructure
{
    [ByteString(bytes: 5, order: 0)]
    public byte[] MyArray;
}

[RemainingBytes]

field: order

Represents an unknown length list of bytes in input order, from the current position to the end of input.

This should be the last field by order. During serialisation, this is treated as a normal byte string.

[ByteLayout]
public class RemainingBytesStructure
{
    [BigEndian(bytes: 2, order: 0)]
    public int SomethingElse;
    
    [RemainingBytes(order: 1)]
    public byte[]? VariableArray;
}

[VariableByteString]

field: source, order

Represents a list of bytes in input order, whose length is generated by a named function of the type.

The function should be a public instance method that takes no parameters and returns an int. The function is allowed to return zero or negative values, which will be interpreted as empty. The resulting byte array will be non-null and zero length.

When based on another field, that field MUST be in earlier order than the variable byte string.

[ByteLayout]
public class VariableArrayStructure
{
    [BigEndian(bytes: 2, order: 0)]
    public int Length;
    
    [VariableByteString(source: nameof(GetLength), order: 1)]
    public byte[] Variable = Array.Empty<byte>();
    
    public int GetLength()=>Length;
}

[ValueTerminatedByteString]

field: stopValue, order

Represents a variable length list of bytes in input order. The string ends when the given byte is reached. The value read includes this byte.

For example, with a StopValue of 0x00, this will read a C-style null-terminated character string.

[ByteLayout]
public class NullTerminatedStringStructure
{
    [ValueTerminatedByteString(stopValue: 0x00, order: 1)]
    public byte[]? MessageString;
    
    [BigEndian(bytes: 2, order: 2)]
    public int Checksum;
}

[FixedValueAttribute]

field: value

Marks the field as having a fixed value. The field must also have a BigEndian or LittleEndian attribute

[ByteLayout]
public class SimpleByteStructure
{
    [BigEndian(bytes: 2, order: 0), FixedValue(0x7F, 0x80)]
    public UInt16 StartMarker;
}

[ByteLayoutChild]

field: order

Include a sub-structure into this structure.

[ByteLayout]
public class ParentWithChild
{
    [ByteLayoutChild(order: 0)]
    public OtherStructure ChildStruct;
}

Marks a field as a substructure. The field type should be another class which is a [ByteLayout]. To repeat multiple times, see [ByteLayoutMultiChild] or [ByteLayoutVariableChild]

Child structures can be nested to arbitrary depth.

[ByteLayoutMultiChild]

field: count, order

Include a fixed number of sub-structures into this structure.

[ByteLayout]
public class ParentWithRepeatedChild
{
    [ByteLayoutMultiChild(count: 3, order: 0)]
    public OtherStructure[]? ChildStruct;
}

Marks a field as a substructure. The field type should be an array of another class which is a [ByteLayout]. See also [ByteLayoutChild] or [ByteLayoutVariableChild]

Child structures can be nested to arbitrary depth.

[ByteLayoutVariableChild]

field: source, order

Include a variable number of sub-structures into this structure. source should be the name of a method in this class that will give the number of repetitions required.

[ByteLayout]
public class ParentWithVariableRepeatChild
{
    [BigEndian(bytes: 2, order:0)]
    public int HowMany;
    
    [ByteLayoutVariableChild(nameof(CountHowMany), order: 1)]
    public OtherStructure[]? ChildStruct;

    public int CountHowMany() => HowMany;
}

Marks a field as a substructure. The field type should be an array of another class which is a [ByteLayout]. See also [ByteLayoutChild] or [ByteLayoutMultiChild]

Child structures can be nested to arbitrary depth.

Helpers

BCD conversions

Assert.That(34.DecToBcd(), Is.EqualTo(0x34));

var ok = ((byte)0x34).BcdToDec(out var dec);
Assert.That(ok, Is.True);
Assert.That(dec, Is.EqualTo(34));

Human Readable Binary Sizes

Assert.That(1000000ul.Human(), Is.EqualTo("976.56kb"));
Assert.That(1073741824ul.Human(), Is.EqualTo("1gb"));

Byte array to C# code

var sample = new byte[] { 1, 2, 3, 100, 200, 255, 0 };
Assert.That(sample.ToCsharpCode("myName"), Is.EqualTo(
    "var myName = new byte[] {0x01, 0x02, 0x03, 0x64, 0xC8, 0xFF, 0x00};"));

Byte array to Hex View

var sample = new byte[] {
    1,   2,   3,   4,  5,  6,  7,  8,  9,  10, 11, 12, 13, 14, 15, 16,
    17,  18,  19,  20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32,
    255, 255, 255, 0
};

Assert.That(sample.Describe("name of thing"), Is.EqualTo(
    "name of thing => 36bytes\r\n" +
    "0000: 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 10 \r\n" +
    "0016: 11 12 13 14 15 16 17 18 19 1A 1B 1C 1D 1E 1F 20 \r\n" +
    "0032: FF FF FF 00 \r\n"));

Byte to Binary String

Assert.That(((byte)0xAA).ToBinString(), Is.EqualTo("10101010"));
Assert.That(((byte)0x55).ToBinString(), Is.EqualTo("01010101"));

Converting between Byte Arrays and Hex Strings

Note: For dotnet 5+, you can use the built-in Convert.ToHexString() and Convert.FromHexString().

var original = new byte[] {
    1,2,4,8,16,32,64,128,255,127,63,31,15,7,3,1,0
};

var hexStr = original.ToHexString();
Assert.That(hexStr, Is.EqualTo("0102040810204080FF7F3F1F0F07030100"));

var result = hexStr.ParseBytes();
Assert.That(result, Is.EqualTo(original).AsCollection);

[END]