Atrea.PolicyEngine 4.0.0-preview-1

<div id="top"></div>

<br /> <div align="center"> <a href="https://github.com/wbaldoumas/atrea-policyengine"> <img src="images/engine_icon.png" alt="Logo" width="80" height="80"> </a> <h1 align="center">Atrea.PolicyEngine</h1> <p align="center"> A modular, composable policy engine for easy implementation of complex conditional processing pipelines. <br /> <a href="#documentation"><strong>Explore the docs »</strong></a> <br /> <br /> <a href="https://github.com/wbaldoumas/atrea-policyengine/tree/main/examples/Atrea.PolicyEngine.Examples/Examples">View Examples</a> · <a href="https://github.com/wbaldoumas/atrea-policyengine/issues">Report Bug</a> · <a href="https://github.com/wbaldoumas/atrea-policyengine/issues">Request Feature</a> </p> <a href="https://github.com/wbaldoumas/atrea-policyengine"> <img src="images/engine.png" alt="Engine" > </a> </div>

<details> <summary>Table of Contents</summary>

• Installation
• Documentation
  • Basic Usage
  • Input Policies
  • Processors
  • Output Policies
  • Asynchronous and Parallel Processing
  • Nesting Policy Engines
  • Code Examples
• Contributing
• License
• Contact
• Acknowledgements

</details>

<a name="installation"/>

Installation

Package manager:

Install-Package Atrea.PolicyEngine -Version 2.2.0

.NET CLI:

dotnet add package Atrea.PolicyEngine --version 2.2.0

<p align="right">(<a href="#top">back to top</a>)</p>

<a name="documentation"/>

Documentation

<a name="basic-usage"/>

Basic Usage

Once input policies, processors, and output policies have been implemented, a policy engine can be built using the PolicyEngineBuilder<T>. In the example below we configure a policy engine which performs translations between natural languages.

var engine = PolicyEngineBuilder<TranslatableItem>
    .Configure()
    .WithInputPolicies(
        // Only translate items which have not yet been translated and are 
        // translations from US English to UK English.
        new IsNotYetTranslated(),
        new IsFromUsEnglish(),
        new IsToUkEnglish()
    ).WithProcessors(
        // Use the Google machine translation API and a proprietary single-word
        // translator to perform translations.
        new GoogleTranslator(),
        new SingleWordTranslator()
    ).WithOutputPolicies(
        // Once an item is translated, publish the translation to an event stream
        // and mark the item as translated.
        new PublishTranslation(),
        new MarkItemTranslated()
    ).Build();

var translatableItem = _repository.GetTranslatableItem();

// Process the item.
engine.Process(translatableItem);

// Process multiple items at once.
var translatableItems =  _repository.GetTranslatableItems();

engine.Process(translatableItems);

<p align="right">(<a href="#top">back to top</a>)</p>

<a name="input-policies"/>

Input Policies

Input policies can be thought of as the gatekeepers that guard the rest of a policy engine's processing and post-processing steps. They should be used to check whether a given item that has entered the policy engine should be processed or not.

The IInputPolicy<T> interface is implemented by a given input policy, whose ShouldProcess(T item) method can return one of three InputPolicyResult values: Continue, Accept, or Reject. How the policy engine handles these input policy results is outlined in the table below.

Input policies are run in the order that they are passed to the PolicyEngineBuilder<T>.WithInputPolicies(...) or AsyncPolicyEngineBuilder<T>.WithAsyncInputPolicies(...) methods.

Note that when an async policy engine is configured with AsyncPolicyEngineBuilder<T>.WithParallelInputPolicies(...):

• All input policies are run and they are all run in parallel.
• There is no meaningful difference between InputPolicyResult.Continue and InputPolicyResult.Accept amongst the individual input policies.
• If one input policy evaluates to InputPolicyResult.Reject but another evaluates to InputPolicyResult.Accept, the item is accepted for processing by the policy engine and not rejected.

<p align="right">(<a href="#top">back to top</a>)</p>

<a name="implementing-input-policies"/>

Implementing Input Policies

Input policies should aim to follow the Single-Responsibility Principle such that each input policy inspects just one facet of information about the item to be processed by the engine. This allows for a much more flexibly configurable engine as well as better reusability for the input policies themselves.

Here is an example of a poorly implemented input policy that is doing too much:

public class ShouldCanadianFrenchToUsEnglishEngineProcess : IInputPolicy<TranslatableItem>
{
    public InputPolicyResult ShouldProcess(TranslatableItem item)
    {
        if (item.IsTranslated)
        {
            return InputPolicyResult.Reject;
        }

        if (item.IsQueuedByUser)
        {
            return InputPolicyResult.Accept;
        }

        if (item.FromLanguage != LanguageCode.CaFr)
        {
            return InputPolicyResult.Reject;
        }

        if (item.ToLanguage != LanguageCode.UsEn)
        {
            return InputPolicyResult.Reject;
        }

        return InputPolicyResult.Continue;
    }
}

Here are just some of the problems with the input policy implementation above:

• It isn't reusable by other policy engines.
• It doesn't follow the Single-Responsibility Principle.
• It is hard to unit test all possible branches for this input policy.

This can be refactored into a cleaner implementation by breaking down each of the checks above into separate input policies:

public class IsNotYetTranslated : IInputPolicy<TranslatableItem>
{
    public InputPolicyResult ShouldProcess(TranslatableItem item)
    {
        if (item.IsTranslated)
        {
            return InputPolicyResult.Reject;
        }

        return InputPolicyResult.Continue;
    }
}

public class IsQueuedByUser : IInputPolicy<TranslatableItem>
{
    public InputPolicyResult ShouldProcess(TranslatableItem item)
    {
        if (item.IsQueuedByUser)
        {
            return InputPolicyResult.Accept;
        }

        return InputPolicyResult.Continue;
    }
}

public class IsFromCanadianFrench : IInputPolicy<TranslatableItem>
{
    public InputPolicyResult ShouldProcess(TranslatableItem item)
    {
        if (item.FromLanguage != LanguageCode.CaFr)
        {
            return InputPolicyResult.Reject;
        }

        return InputPolicyResult.Continue;
    }
}

public class IsToUsEnglish : IInputPolicy<TranslatableItem>
{
    public InputPolicyResult ShouldProcess(TranslatableItem item)
    {
        if (item.ToLanguage != LanguageCode.UsEn)
        {
            return InputPolicyResult.Reject;
        }

        return InputPolicyResult.Continue;
    }
}

Note that although this produces more code, classes, and source files, each input policy follows the Single-Responsibility Principal, is reusable within the context of other policy engines, and is extremely easy to unit test! ✔️ ✔️ ✔️

These can then be passed to the policy engine builder's WithInputPolicies(...) method as such:

var engine = PolicyEngineBuilder<TranslatableItem>
    .Configure()
    .WithInputPolicies(
        new IsNotYetTranslated(),
        new IsQueuedByUser(),
        new IsFromCanadianFrench(),
        new IsToUsEnglish()
    )
    .WithProcessors(...)
    .WithOutputPolicies(...)
    .Build();

Async Input Policies

If some input policies are more complex and have dependencies that perform async operations, the IAsyncInputPolicy<T> interface can be implemented instead. See more about asynchronous and parallel processing below.

<p align="right">(<a href="#top">back to top</a>)</p>

<a name="compound-input-policies/>

Compound Input Policies

The Atrea.PolicyEngine library also provides a handful of useful compound input policies. These currently include And<T>, Or<T>, and Xor<T>.

These compound input policies can be created by passing other input policies constructor arguments:

var isFromCanadianFrenchAndToUsEnglish = new And<TranslatableItem>(
  new IsFromCanadianFrench(),
  new IsToUsEnglish();
)

or by using the built-in IInputPolicy<T> extension methods:

var isFromCanadianFrenchToUsEnglish = new IsFromCanadianFrench().And(new IsToUsEnglish());

Using these compound input policies allows for creation of complex input policies on the fly by composing together more granular input policies in an intuitive way.

var isFromCanadianFrench = new IsFromCanadianFrench();
var isToCanadianFrench = new IsToCanadianFrench();
var isFromUsEnglish = new IsFromUsEnglish();
var isToUsEnglish = new IsToUsEnglish();

var isFromCanadianFrenchToUsEnglish = isFromCanadianFrench.And(isToUsEnglish);
var isFromUsEnglishToCanadianFrench = isFromUsEnglish.And(isToCanadianFrench);

var isCanadianFrenchUsEnglishTranslation = isFromCanadianFrenchToUsEnglish.Xor(
    isFromUsEnglishToCanadianFrench
);

A Not<T> input policy is also available to easily reverse the output of any given input policy.

var isAlreadyTranslated = new Not<TranslatableItem>(new IsNotYetTranslated());

Not<T> is implemented in such a way that it produces the following InputPolicyResult values.

Versions of these compound input policies that support async operations are also available with AsyncAnd<T>, AsyncOr<T>, AsyncXor<T>, and AsyncNot<T>.

<p align="right">(<a href="#top">back to top</a>)</p>

<a name="processors"/>

Processors

A policy engine's processors should be where a brunt of the complex processing of items takes place. A given processor should implement the IProcessor<T> or IAsyncProcessor<T> interface, whose respective Process(T item) or ProcessAsync(T item) method will be called by the policy engine when an item has been accepted for processing by the engine's input policies. In our example, this is where we are actually reaching out to external APIs or data stores to perform machine translation between natural languages.

public class GoogleTranslator : IProcessor<TranslatableItem>
{
    private readonly ITranslationClient _googleTranslationClient;

    public GoogleTranslator(ITranslationClient googleTranslationClient)
    {
        _googleTranslationClient = googleTranslationClient;
    }

    public void Process(TranslatableItem item)
    {
        var response = _googleTranslationClient.TranslateText(
            item.SourceText,
            item.FromLanguage,
            item.ToLanguage
        );

        item.TranslatedText = response.TranslatedText;
    }
}

Processors can be configured to run synchronously, asynchronously, and in parallel. See more about asynchronous and parallel processing below.

Output Policies

A policy engine's output policies can be thought of as light post-processors that should be run after the engine's main processing step has been completed. They shouldn't be doing any heavy lifting. In our example we have an output policy that is pushing messages to an event stream.

public class PublishTranslation : IOutputPolicy<TranslatableItem>
{
    private readonly IKafkaProducer<TranslatableItemMessage> _messageProducer;

    public PublishTranslation(IKafkaProducer<TranslatableItemMessage> messageProducer)
    {
        _messageProducer = messageProducer;
    }

    public void Apply(TranslatableItem item)
    {
        var message = new TranslatableItemMessage(item);

        _messageProducer.Produce(message);
    }
}

Output policies can be configured to run synchronously, asynchronously, and in parallel. See more about asynchronous and parallel processing below.

Asynchronous and Parallel Processing

Async and parallel processing is supported in a myriad of configurations by implementing the IAsyncInputPolicy<T>, IAsyncProcessor<T>, and IAsyncOutputPolicy<T> interfaces and using the AsyncPolicyEngineBuilder<T>.

Here we configure async input policies to be awaited in order, processors to be run in parallel, and output policies to be run in parallel.

var engine = AsyncPolicyEngineBuilder<TranslatableItem>
    .Configure()
    .WithAsyncInputPolicies(
        // For this engine, we only want it to translate items which have not
        // yet been translated, and are translations from Canadian French to US English.
        new IsNotYetTranslated(),
        new IsFromCanadianFrench(),
        new IsToUsEnglish()
    ).WithParallelProcessors(
        // Use the Google and Microsoft machine translation APIs, and a proprietary cache-based
        // translator to perform translations.
        new GoogleTranslator(),
        new MicrosoftTranslator(),
        new CacheTranslator()
    ).WithParallelOutputPolicies(
        // Once an item is translated, publish the translation to an event stream
        // and mark the item as translated.
        new PublishTranslation(),
        new MarkItemTranslated()
    ).Build();

var translatableItem = _repository.GetTranslatableItem();

// Process the item.
await engine.ProcessAsync(translatableItem);

<p align="right">(<a href="#top">back to top</a>)</p>

<a name="nesting-policy-engines"/>

Nesting Policy Engines

Since the IPolicyEngine<T> interface implements IProcessor<T>, policy engines can be composed together and nested within another encompassing policy engine and act as individual processors within that engine.

In the example below, imagine that we have methods to build a policy engine that performs translation between US English and Canadian French, one to perform translations between US English and UK English, and one that specifically handles values containing numeric text. A full code example can be seen here.

var canadianFrenchTranslationEngine = BuildCanadianFrenchTranslationEngine();
var englishTranslationEngine = BuildEnglishTranslationEngine();
var numericTranslationEngine = BuildNumericTranslationEngine();

var translationEngine = PolicyEngineBuilder<TranslatableItem>
    .Configure()
    // Only process items which have not yet been translated.
    .WithInputPolicies(NotYetTranslated)
    .WithProcessors(
        // Use the Canadian French, English, and numeric translation engines to
        // perform translations.
        canadianFrenchTranslationEngine,
        englishTranslationEngine,
        numericTranslationEngine
    )
    // No output policies needed, since each individual engine handles its own
    // post-processing steps.
    .WithoutOutputPolicies()
    .Build();

var translatableItem = _repository.GetTranslatableItem();

translationEngine.Process(translatableItem);

This nesting of policy engines as processors is also possible with asynchronous by using the AsyncPolicyEngineBuilder<T> (see code example below).

<p align="right">(<a href="#top">back to top</a>)</p>

<a name="code-examples"/>

Code Examples

Full code examples can be found in this repository at the following links:

• Simple PolicyEngineBuilder<T> usage
• Simple AsyncPolicyEngineBuilder<T> usage
• PolicyEngineBuilder<T> usage with compound input policies
• AsyncPolicyEngineBuilder<T> usage with compound input policies
• Nested policy engine configuration
• Nested async policy engine configuration

<p align="right">(<a href="#top">back to top</a>)</p>

<a name="contributing"/>

Contributing

Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated. For detailed contributing guidelines, please see CONTRIBUTING.md

<p align="right">(<a href="#top">back to top</a>)</p>

<a name="license"/>

License

Distributed under the MIT License. See LICENSE for more information.

<p align="right">(<a href="#top">back to top</a>)</p>

<a name="contact"/>

Contact

William Baldoumas - wbaldoumas.github@gmail.com

Project Link: https://github.com/wbaldoumas/atrea-policyengine

<a name="acknowledgements"/>

<p align="right">(<a href="#top">back to top</a>)</p>

Acknowledgements

This template was adapted from https://github.com/othneildrew/Best-README-Template.

<p align="right">(<a href="#top">back to top</a>)</p>

Show your support by contributing or starring the repo! ⭐ ⭐ ⭐ ⭐ ⭐