IndependentReserve.Grpc.Tools 4.3.274

IndependentReserve.Grpc.Tools

Purpose

This package contains MSBuild tasks and targets for automatic generation of gRPC/Protobuf code from plain C# interface(s).

How it works

When this package is referenced by a project it adds itself into compilation pipeline in the following way:
For every depended project which is marked by GenerateGrpc attribute it loads all eligible C# interfaces and for every such source interface it generates the following code:

• a set of *.proto files defining gRPC service and messages plus the hierarchy of all DTO classes referenced by the source interface;
• a set of C# partial classes which provide implicit conversion operators for Protobuf ↔ DTO conversion for every generated *.proto file;
• gRPC service implementation which depends on source interface and internally calls this interface for corresponding interface method implementation;
• gRPC client class which implements the source interface by calling gRPC server via gRPC.

How to use it

To generate all required gRPC/Protobuf code in a project you need to point the package to a separate project which contains interface (or interfaces) which you want to generate gRPC/Protobuf types for. To do that simply add GenerateGrpc="true" attribute to ProjectReference element of the relevant source project, e.g.:

<ItemGroup>
  <ProjectReference Include="..\Service.Interface.csproj" GenerateGrpc="true" />
</ItemGroup>

By default the tool searches for all public interfaces which names match Service$ regular expression (e.g. ISomeService) and generates all required gRPC-related code for every found interface.
To use a different pattern for interface search specify a custom regular expression (.NET flavor) via GrpcServicePattern attribute, e.g.:

<ItemGroup>
  <ProjectReference Include="..\Service.Interface.csproj" >
    <GenerateGrpc>true</GenerateGrpc>
    <GrpcServicePattern>I[^.]*ServiceInterface$</GrpcServicePattern>
  </ProjectReference>
</ItemGroup>  

Note: the source interfaces must be in a separate dependent project because the tool uses reflection to load and process source interfaces during the build.

All generated files are placed in obj/[Configuration]/[TargetFramework]/Grpc root folder and are automatically included into project's build.
All *.proto files are placed in Protos subfolder:

• [service-name].proto file: gRPC service definition file which defines all methods mirroring I[service-name] source interface methods;
• [service-name]/Messages/*.proto files: these files contain all *Request and *Response messages referenced from [service-name].proto file;
• [class-name].proto files: a hierarchy of messages which are referenced (directly or indirectly) by messages from [service-name]/Messages/*.proto.

All *.cs files are placed in Partials subfolder:

• [service-name]GrpcService.cs: gRPC service class:
  This class depends on the source interface and it is expected that the implementation of this interface passed to gRPC service class constructor (e.g. via dependency injection) contains the internal implementation of the service logic.
  Every gRPC method of this class is marked with [SourceMethod] attribute (SourceMethodAttribute from IndependentReserve.Grpc package) which identifies the source interface method it calls: the interface which declares the method, the method name and its parameter types. gRPC method names do not always match source method names: Async suffix is dropped (e.g. GetAttachmentAsync is served as GetAttachment) and methods which end up with the same gRPC name (overloads, or e.g. Foo and FooAsync) are numbered (Foo1, Foo2). So any code which maps a gRPC method back to its source method (e.g. to apply authorization attributes of the source interface implementation to the gRPC service) must use SourceMethodAttribute.Resolve() and must not look the source method up by gRPC method name, which does not find renamed methods;
• [service-name]GrpcClient.cs: gRPC client implementation:
  This class implements source interface by calling the service via gRPC (using internal gRPC client class in turn generated by Grpc.Tools). For each method from source interface both synchronous and asynchronous methods are generated.
• [class-name].cs files: a set of partial C# classes which add implicit conversion operators (Protobuf ↔ DTO) to generated by Grpc.Tools C# classes.
  There is one to one correspondence between this set and [class-name].proto files set.

From the above set of files normally only C# code is relevant to the user of this package. The Protobuf code can be (and should be) considered internal. In practice developer only needs to know about two classes:

• gRPC service class [service-name]GrpcService.cs: to host the service in ASP.NET Core;
• gRPC client class [service-name]GrpcClient.cs: to call the service.

Both classes either depend or implement the source interface while all boilerplate conversion to/from Protobuf messages is handled by AutoMapper framework.

.NET to Protobuf conversion rules

• Every public method of source interface is converted to gRPC service method;
• Every .NET type which is referenced by any convertible source interface method directly or transitively is converted to Protobuf message using the following approach:
  • Any well-known built-in type, i.e. a type which has direct corresponding Protobuf type (e.g. Boolean, Int32, String, etc.) or one of built-in .NET-specific types from IndependentReserve.Grpc package (e.g.: Decimal, Guid, DateTime etc.) is directly convertible;
  • A collection (i.e. a type which implements IEnumerable<> interface, including IDictionary<,>) or a single-dimensional array (including jagged arrays) with elements of convertible type are convertible;
  • Enums are convertible (but see note below);
  • For any other .NET type a custom Protobuf message is generated using all public fields and properties of convertible types;
  • Any .NET entity which is not convertible (i.e. which is ignored or is not currently supported) is omitted during Protobuf code generation (with detailed MSBuild warning issued in build log). Such ignored entity will be absent in Protobuf code and will most likely always have default value in source type.

Ignored .NET entities

The following .NET entities are ignored (i.e. they are not convertible):

• Open generic source interfaces;
• Interfaces and abstract classes (except for classes and interfaces which implement IEnumerable<> interface);
• Any type which is not built-in and does not have any convertible public fields or properties.

Currently not supported .NET entities

The following .NET entities are not currently supported (i.e. they are currently not convertible) but they might be supported in future versions:

• Nested .NET types;
• Multidimensional arrays;
• Enums with underlying type smaller than Int32;
• ref and out source interface method parameters;
• Source interface properties;
• Source interface open generic methods without default implementation;
• Source interface events;

Nullable annotations

The tool takes into account nullable annotations when they are available. Generally not-nullable arguments and type members produce more efficient Protobuf code. This is especially noticeable in collections, e.g. serialisation of a property of type string[] is several times faster than serialisation of a similar property but of type string?[].

gRPC Streaming

Generated code use gRPC streaming to handle return type of a method or a single argument of a method of type IEnumerable<> or IAsyncEnumerable<>.

Example of generated C# code

If we pass the following source interface to the tool:

public interface ITestService
{
    int Plus(int a, int b);
}

it will generate the following gRPC service class:

public partial class TestServiceGrpcService : TestServiceBase
{
    private readonly ILogger<TestServiceGrpcService> _logger;
    private readonly ITestService _testService;

    public TestServiceGrpcService(
        ILogger<TestServiceGrpcService> logger,
        ITestService testService)
    {
        _logger = logger;
        _testService = testService;
    }

    [SourceMethod(typeof(ITestService), "Plus", typeof(System.Int32), typeof(System.Int32))]
    public override async Task<PlusResponse> Plus(PlusRequest request, ServerCallContext context)
    {
        var args = MapperTo<ValueTuple<System.Int32, System.Int32>>.MapFrom(new { Item1 = request.A, Item2 = request.B });
        var result = _testService.Plus(args.Item1, args.Item2);
        return MapperTo<PlusResponse>.MapFrom(new { Result = result });
    }
}

along with the following gRPC client class:

public partial class TestServiceGrpcClient : GrpcClient, ITestService
{
    private readonly Lazy<TestServiceClient> _client;

    public TestServiceGrpcClient(IGrpcServiceConfiguration config, bool useGrpcWeb = true)
        : base(config, useGrpcWeb)
    {
        var invoker = Channel.CreateCallInvoker();
        SetupCallInvoker(ref invoker);
        _client = new(() => new(invoker));
    }

    partial void SetupCallInvoker(ref CallInvoker invoker);

    private TestServiceClient Client => _client.Value;

    public System.Int32 Plus(System.Int32 a, System.Int32 b)
    {
        var response = Client.Plus(MapperTo<PlusRequest>.MapFrom(new { A = a, B = b }));
        return MapperTo<Wrapper<System.Int32>>.MapFrom(response).Result;
    }

    public async Task<System.Int32> PlusAsync(System.Int32 a, System.Int32 b)
    {
        var response = await Client.PlusAsync(MapperTo<PlusRequest>.MapFrom(new { A = a, B = b }));
        return MapperTo<Wrapper<System.Int32>>.MapFrom(response).Result;
    }
}

Service hooks

A service hook is work that the generated gRPC service runs on the message before it calls the service, e.g. setting the log reference, which used to be a PostSharp aspect on the implementation. Like that aspect, it is an attribute on the method of the service implementation, whose type derives, directly or through other (e.g. abstract) classes, from Ir.Common.Grpc.Hooks.ServiceHookAttribute.

The hook contract is in the Ir.Common.Grpc.Hooks namespace of the IndependentReserve.Grpc package, which the generated code and the services already reference. What each hook does, and the order the hooks of a method run in, are up to the hooks:

• ServiceHooks.For(Type interfaceType, string methodName, params Type[] parameterTypes) - no hooks yet for an interface method;
• ServiceHooks.Bind(MethodInfo implementation) - reads, orders and binds the hooks of the implementation method, and fails when two of them share an order or one cannot run on the method;
• ServiceHooks.Run<T>(T message) - runs them on the message;
• RunsServiceHooksAttribute(string hooksField) - marks a generated method that runs the hooks in the named field.

The tool sees only the source interface, never the implementation, so every gRPC method with a message to run hooks on - the first argument of the source method, mapped whole - gets the same code, whether its implementation is hooked or not. The generated gRPC service:

• creates an empty ServiceHooks once, into a private static _[method-name]Hooks field (camel case, like its other fields);
• runs it on the message between mapping the request and calling the service, so a hook that throws stops the call;
• puts [RunsServiceHooks(nameof(_[method-name]Hooks))] on the gRPC method, below its [SourceMethod].

A method without such a message runs no hooks and gets no marker: one with no parameters, one whose first parameter is left out of the request (see Ignored .NET entities), or one with a streamed request (see gRPC Streaming). A streamed response is fine.

The host binds the hooks once at startup, before the service takes any request. For every gRPC method with the marker, it reads the static field that [RunsServiceHooks].HooksField names and calls Bind with the implementation method that serves the interface method [SourceMethod] names. It reads both on the generated service itself: a subclass of it, e.g. a proxy that overrides the gRPC methods to add attributes, inherits [SourceMethod] but neither the marker nor the private field, and its overrides run the hooks by calling the base method. It can fail its startup when an implementation method is hooked but the gRPC method that serves it has no marker: the service was generated by a tool version without hooks, or the method has no message. Until the host binds them, the hooks are empty, so a host that maps the service without binding runs none.

E.g. for the following source interface:

public interface IHookedService
{
    string Absorbed(string message);
}

the gRPC service class gets:

using Ir.Common.Grpc.Hooks;

public partial class HookedServiceGrpcService : HookedServiceBase
{
    private static readonly ServiceHooks _absorbedHooks = ServiceHooks.For(
        typeof(Ir.Grpc.Test.Service.IHookedService),
        nameof(Ir.Grpc.Test.Service.IHookedService.Absorbed),
        typeof(System.String));

    // ...

    [SourceMethod(typeof(Ir.Grpc.Test.Service.IHookedService), "Absorbed", typeof(System.String))]
    [RunsServiceHooks(nameof(_absorbedHooks))]
    public override async Task<AbsorbedResponse> Absorbed(AbsorbedRequest request, ServerCallContext context)
    {
        var args = MapperTo<ValueTuple<System.String>>.MapFrom(new { Item1 = request.Message });
        _absorbedHooks.Run(args.Item1);
        var result = _hookedService.Absorbed(@message: args.Item1);
        return MapperTo<AbsorbedResponse>.MapFrom(new { Result = result });
    }
}

and an implementation hooks it the way it put a PostSharp aspect on it, where FirstHook derives from ServiceHookAttribute:

public class HookedService : IHookedService
{
    [FirstHook]
    public string Absorbed(string message) => message;
}

DTO ↔ Protobuf conversion code test generation

This tool can also generate unit tests for DTO → Protobuf → byte[] → Protobuf → DTO (round-trip) conversion/serialization path which check that after round-trip transformations the resulting "after" DTO content is equal to the source "before" DTO content.
To add test generation step to a project add GenerateGrpcTests="true" attribute to ProjectReference referencing source interface project, e.g.:

<ItemGroup>
  <ProjectReference Include="..\Service.Interface.csproj" GenerateGrpcTests="true" />
</ItemGroup>

The tool will then generate unit tests for every source interface method which test conversion/serialisation of every DTO referenced (even transitively) by the source interface. Generated tests use xUnit test framework. Every test is executed against generated test data set which aims to cover the entire range of DTO values using minimum number of test data values.

Generated test code is split into the following classes:

• [service-name]ConversionTests.cs: a set of conversion tests for I[service-name] source interface:
  For every source interface method the tool generates up to two unit tests named:
  • [method-name]RequestTest: for all convertible method's arguments (if there is any);
  • [method-name]ResponseTest: for method's return value type (if it is convertible).
• [service-name]Arbitrary.cs: a set of test data generators referenced by tests via [ClassData] attribute;
• Asserter.cs: a per-project file which contains Assert.Equal implementation used by tests.

All generated test files are placed in obj/[Configuration]/[TargetFramework]/Grpc/Tests subfolder and are automatically included into project's build.

Note: GenerateGrpcTests="true" does not generate source Protobuf code (only tests) since normally unit tests are placed in a separate project which in turn references the project which contains generated (via GenerateGrpc="true") Protobuf code. If you require both code and the tests to be generated in the same project add both GenerateGrpc and GenerateGrpcTests attributes to the relevant ProjectReference.

Example of generated test code

For the above ITestService source interface the following code is generated in TestServiceConversionTests.cs file:

public partial class TestServiceConversionTests
{
    private static Asserter Assert { get; set; } = Asserter.Instance;

    [Theory]
    [ClassData(typeof(TestServiceArbitrary.PlusRequest))]
    public void PlusRequestTest(System.Int32 aBefore, System.Int32 bBefore)
    {
        var protoBefore = MapperTo<TestService.Messages.Grpc.PlusRequest>.MapFrom(new { A = aBefore, B = bBefore });
        var bytes = Serialize(protoBefore);
        var protoAfter = TestService.Messages.Grpc.PlusRequest.Parser.ParseFrom(bytes);
        var (aAfter, bAfter) = MapperTo<(System.Int32, System.Int32)>.MapFrom(new { Item1 = protoAfter.A, Item2 = protoAfter.B });
        Assert.Equal(aBefore, aAfter);
        Assert.Equal(bBefore, bAfter);
    }

    [Theory]
    [ClassData(typeof(TestServiceArbitrary.PlusResponse))]
    public void PlusResponseTest(System.Int32 resultBefore)
    {
        // Test code similar to above
    }

    private byte[] Serialize(IMessage message)
    {
        // Common Protobuf serialisation code
    }    
}

While the following test data generation code is placed in TestServiceArbitrary.cs:

public static partial class TestServiceArbitrary
{
    private static TestServiceArbitraryGens Gens { get; set; } = new();
    private static int? MaxTestCasesPerTest { get; set; } = 1000;
    // Other settings

    public class PlusRequest : IEnumerable<object[]>
    {
        public IEnumerator<object[]> GetEnumerator()
        {
            return Gens.GenPlusRequest()
                .Select(elem => new object[] { elem.Item1, elem.Item2 })
                .Take(MaxTestCasesPerTest ?? int.MaxValue)
                .GetEnumerator();
        }
        // Rest of the implementation
    }

    public class PlusResponse : IEnumerable<object[]>
    {
        // Implementation similar to above
    }

    public class TestServiceArbitraryGens
    {
        public virtual IEnumerable<(System.Int32, System.Int32)> GenPlusRequest() =>
            from @item1 in GenInt32()
            from @item2 in GenInt32()
            select MapperTo<(System.Int32, System.Int32)>.MapFrom(new
            {
                @Item1 = @item1,
                @Item2 = @item2
            });

        public virtual IEnumerable<ValueTuple<System.Int32>> GenPlusResponse() =>
            // Implementation similar to above

        public virtual IEnumerable<Int32> GenInt32() => new[]
        {
            default,
            Int32.MinValue,
            Int32.MaxValue,
            Int32.MinValue / 2,
            Int32.MaxValue / 2,
        };
    }
}

Per-project Asserter.cs:

public partial class Asserter
{
    public static Asserter Instance { get; private set; } = new();

    public virtual void Equal<T>(T expected, T actual)
    {
        Assert.Equal(JsonSerializer.Serialize(expected), JsonSerializer.Serialize(actual));
    }
}

Customisation of generated test code

Generated test code should work out of the box in most cases however there is also a way to customise it if, for example, generated test data cannot be handled by some DTO (e.g. DTO throws an exception when a property is set to int.MaxValue) or if a different content of test data is more suitable for testing of a particular DTO.

All generated top-level test classes are partial so they can be customised for example by implementing static constructor for a relevant file from where the following customisation options are available:

• Asserter class: derive from this class to customise Assert.Equal logic:
  • For all tests in the project: by assigning global Asserter.Instance property to derived implementation;
  • For a specific [service-name]ConversionTests class: by assigning [service-name]ConversionTests.Assert to derived implementation.
• [service-name]Arbitrary class: customise this class to change generated test data set content:
  • Some test data generation parameters and limits can be adjusted via:
    • MaxTestCasesPerTest: limits the number of test data instances generated for each test (default is 1000, set to null to remove the limit). In most cases generated test data set will contains much less than default maximum of 1000 number of items (usually less than 100);
    • MaxCollectionSize: limits the maximum number of items in test data collections (default is 5);
    • MaxNumberOfEnumValues: defines the maximum number of enum values which are used for test data (default is 3).
  • It is also possible to override any test data generator by providing a custom implementation which can completely change the content of the test data:
    • Derive from the default generated (nested) class [service-name]Arbitrary.[service-name]ArbitraryGens which defines a set of test data generators via a set of virtual IEnumerable<[DTO]> Gen* methods and replace any of them with a custom implementation by overriding base implementation in the derived class;
    • In static constructor of [service-name]Arbitrary class set Gens property to the instance of the derived class.