AICentral 0.16.0-pullrequest0087-0003

This is a prerelease version of AICentral.
There is a newer version of this package available.
See the version list below for details.
dotnet add package AICentral --version 0.16.0-pullrequest0087-0003
                    
NuGet\Install-Package AICentral -Version 0.16.0-pullrequest0087-0003
                    
This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package.
<PackageReference Include="AICentral" Version="0.16.0-pullrequest0087-0003" />
                    
For projects that support PackageReference, copy this XML node into the project file to reference the package.
<PackageVersion Include="AICentral" Version="0.16.0-pullrequest0087-0003" />
                    
Directory.Packages.props
<PackageReference Include="AICentral" />
                    
Project file
For projects that support Central Package Management (CPM), copy this XML node into the solution Directory.Packages.props file to version the package.
paket add AICentral --version 0.16.0-pullrequest0087-0003
                    
#r "nuget: AICentral, 0.16.0-pullrequest0087-0003"
                    
#r directive can be used in F# Interactive and Polyglot Notebooks. Copy this into the interactive tool or source code of the script to reference the package.
#:package AICentral@0.16.0-pullrequest0087-0003
                    
#:package directive can be used in C# file-based apps starting in .NET 10 preview 4. Copy this into a .cs file before any lines of code to reference the package.
#addin nuget:?package=AICentral&version=0.16.0-pullrequest0087-0003&prerelease
                    
Install as a Cake Addin
#tool nuget:?package=AICentral&version=0.16.0-pullrequest0087-0003&prerelease
                    
Install as a Cake Tool

AI Central

AI Central gives you control over your AI services.

  • Intelligent Routing
  • Custom consumer OAuth2 authorisation
  • Fallback AI service
  • Round Robin AI services
  • Lowest Latency AI service
  • Circuit breakers, and backoff-retry over downstream AI services
  • Request based and Token based rate limiting
  • Prompt and usage logging
    • Works for streaming endpoints as-well as non streaming
  • Additional Open Telemetry Metrics and Traces

Configuration

See docs on Github for more details.

Minimal

This sample produces a AI-Central proxy that

  • Listens on a hostname of your choosing
  • Proxies directly through to a back-end Open AI server
  • Can be accessed using standard SDKs
  • Outputs open-telemetry metrics to capture usage information
{
  "AICentral": {
    "Endpoints": [
      {
        "Type": "AzureOpenAIEndpoint",
        "Name": "openai-1",
        "Properties": {
          "LanguageEndpoint": "https://<my-ai>.openai.azure.com",
          "AuthenticationType": "ApiKey",
          "ApiKey": "<key>"
        }
      }
    ],
    "EndpointSelectors": [
      {
        "Type": "SingleEndpoint",
        "Name": "default",
        "Properties": {
          "Endpoint": "openai-1"
        }
      }
    ],
    "AuthProviders": [
      {
        "Type": "ApiKey",
        "Name": "apikey",
        "Properties": {
          "Clients": [
            {
              "ClientName": "Consumer-1",
              "Key1": "<random-key>",
              "Key2": "<random-key>"
            }
          ]
        }
      }
    ],
    "Pipelines": [
      {
        "Name": "OpenAIPipeline",
        "Host": "mypipeline.mydomain.com",
        "EndpointSelector": "default",
        "AuthProvider": "apikey"
        
      }
    ]
  }
}

Full example

This pipeline will:

  • Present an Azure Open AI, and an Open AI downstream as a single upstream endpoint
    • maps incoming Azure Open AI deployments to Open AI models
  • Present it as an Azure Open AI style endpoint
  • Protect the front-end by requiring an AAD token issued for your own AAD application
  • Put a local Asp.Net core rate-limiting policy over the endpoint
  • Add logging to Azure monitor
    • Logs quota, client caller information (IP and identity name), and in this case the Prompt but not the response.
  • Publish the client-name as a tag in Open Telemetry
{
  "AICentral": {
    "Endpoints": [
      {
        "Type": "AzureOpenAIEndpoint",
        "Name": "openai-priority",
        "Properties": {
          "LanguageEndpoint": "https://<my-ai>.openai.azure.com",
          "AuthenticationType": "Entra|EntraPassThrough|ApiKey",
          "MaxConcurrency": 10
        }
      },
      {
        "Type": "OpenAIEndpoint",
        "Name": "openai-fallback",
        "Properties": {
          "LanguageEndpoint": "https://api.openai.com",
          "ModelMappings": {
            "Gpt35Turbo0613": "gpt-3.5-turbo",
            "Ada002Embedding": "text-embedding-ada-002"
          },
          "ApiKey": "<my-api-key>",
          "Organization": "<optional-organisation>"
        }
      }
    ],
    "AuthProviders": [
      {
        "Type": "Entra",
        "Name": "simple-aad",
        "Properties": {
          "Entra": {
            "ClientId": "<my-client-id>",
            "TenantId": "<my-tenant-id>",
            "Instance": "https://login.microsoftonline.com/",
            "Audience": "<custom-audience>"
          }
        }
      }
    ],
    "EndpointSelectors": [
      {
        "Type": "Prioritised",
        "Name": "my-endpoint-selector",
        "Properties": {
          "PriorityEndpoints": ["openai-1"],
          "FallbackEndpoints": ["openai-fallback"]
        }
      }
    ],
    "GenericSteps": [
      {
        "Type": "AspNetCoreFixedWindowRateLimiting",
        "Name": "token-rate-limiter",
        "Properties": {
          "LimitType": "PerConsumer|PerAICentralEndpoint",
          "MetricType": "Tokens",
          "Options": {
            "Window": "00:01:00",
            "PermitLimit": 1000
          }
        }
      },
      {
        "Type": "AspNetCoreFixedWindowRateLimiting",
        "Name": "window-rate-limiter",
        "Properties": {
          "LimitType": "PerConsumer|PerAICentralEndpoint",
          "MetricType": "Requests",
          "Options": {
            "Window": "00:00:10",
            "PermitLimit": 100
          }
        }
      },
      {
        "Type": "AzureMonitorLogger",
        "Name": "azure-monitor-logger",
        "Properties": {
          "WorkspaceId": "<workspace-id>",
          "Key": "<key>",
          "LogPrompt": true,
          "LogResponse": false,
          "LogClient": true
        }
      },
      {
        "Type": "BulkHead",
        "Name": "bulk-head",
        "Properties": {
          "MaxConcurrency": 20
        }
      }
    ],
    "Pipelines": [
      {
        "Name": "MyPipeline",
        "Host": "prioritypipeline.mydomain.com",
        "EndpointSelector": "my-endpoint-selector",
        "AuthProvider": "simple-aad",
        "Steps": [
          "window-rate-limiter",
          "bulk-head",
          "azure-monitor-logger"
        ],
        "OpenTelemetryConfig": {
          "AddClientNameTag": true
        }
      }
    ]
  }
}

Product Compatible and additional computed target framework versions.
.NET net8.0 is compatible.  net8.0-android was computed.  net8.0-browser was computed.  net8.0-ios was computed.  net8.0-maccatalyst was computed.  net8.0-macos was computed.  net8.0-tvos was computed.  net8.0-windows was computed.  net9.0 was computed.  net9.0-android was computed.  net9.0-browser was computed.  net9.0-ios was computed.  net9.0-maccatalyst was computed.  net9.0-macos was computed.  net9.0-tvos was computed.  net9.0-windows was computed.  net10.0 was computed.  net10.0-android was computed.  net10.0-browser was computed.  net10.0-ios was computed.  net10.0-maccatalyst was computed.  net10.0-macos was computed.  net10.0-tvos was computed.  net10.0-windows was computed. 
Compatible target framework(s)
Included target framework(s) (in package)
Learn more about Target Frameworks and .NET Standard.

NuGet packages

This package is not used by any NuGet packages.

GitHub repositories

This package is not used by any popular GitHub repositories.

Version Downloads Last Updated
0.21.5 99 7/18/2025
0.21.4 1,945 2/21/2025
0.21.4-pullrequest0155-0003 78 2/21/2025
0.21.3 98 2/21/2025
0.21.3-pullrequest0155-0002 77 1/30/2025
0.21.1 619 1/14/2025
0.21.1-pullrequest0153-0003 62 1/14/2025
0.21.0 388 12/9/2024
0.21.0-pullrequest0152-0005 81 12/9/2024
0.21.0-pullrequest0152-0004 76 12/9/2024
0.21.0-pullrequest0152-0003 72 12/9/2024
0.21.0-pullrequest0152-0002 74 12/9/2024
0.20.7 246 11/26/2024
0.20.7-pullrequest0151-0002 78 11/26/2024
0.20.6 386 11/1/2024
0.20.6-pullrequest0150-0003 70 11/1/2024
0.20.5 310 9/30/2024
0.20.5-pullrequest0148-0004 87 9/30/2024
0.20.5-pullrequest0148-0003 78 9/30/2024
0.20.5-pullrequest0148-0002 100 9/30/2024
0.20.3 135 9/5/2024
0.20.3-pullrequest0146-0005 73 9/5/2024
0.20.3-pullrequest0146-0004 72 9/5/2024
0.20.2 97 9/4/2024
0.20.2-pullrequest0145-0004 66 9/4/2024
0.20.0 124 8/17/2024
0.20.0-pullrequest0143-0013 90 8/17/2024
0.20.0-pullrequest0143-0012 92 8/17/2024
0.20.0-pullrequest0143-0011 89 8/17/2024
0.20.0-pullrequest0143-0010 88 8/17/2024
0.20.0-pullrequest0143-0009 88 8/17/2024
0.19.6-pullrequest0143-0008 100 8/17/2024
0.19.5 92 8/7/2024
0.19.5-pullrequest0141-0002 71 8/7/2024
0.19.4 94 8/1/2024
0.19.4-pullrequest0140-0003 70 8/1/2024
0.19.4-pullrequest0139-0001 72 7/29/2024
0.19.3 90 7/29/2024
0.19.2 103 7/29/2024
0.19.2-pullrequest0138-0004 69 7/29/2024
0.19.1 91 7/29/2024
0.19.1-pullrequest0137-0003 74 7/29/2024
0.19.0 88 7/29/2024
0.19.0-pullrequest0136-0014 66 7/29/2024
0.19.0-pullrequest0136-0013 66 7/29/2024
0.19.0-pullrequest0136-0012 156 7/29/2024
0.19.0-pullrequest0136-0011 70 7/29/2024
0.19.0-pullrequest0136-0010 72 7/29/2024
0.19.0-pullrequest0136-0009 81 7/27/2024
0.19.0-pullrequest0136-0008 76 7/27/2024
0.19.0-pullrequest0136-0007 77 7/27/2024
0.18.6 111 7/15/2024
0.18.6-pullrequest0135-0002 72 7/15/2024
0.18.5 91 7/15/2024
0.18.5-pullrequest0134-0003 71 7/15/2024
0.18.4 94 7/15/2024
0.18.4-pullrequest0133-0009 72 7/15/2024
0.18.4-pullrequest0133-0008 74 7/15/2024
0.18.4-pullrequest0133-0007 74 7/15/2024
0.18.3 102 7/12/2024
0.18.3-pullrequest0132-0005 72 7/12/2024
0.18.2 92 7/12/2024
0.18.2-pullrequest0131-0003 79 7/11/2024
0.18.2-pullrequest0130-0003 73 7/11/2024
0.18.1 95 7/10/2024
0.18.1-pullrequest0129-0007 71 7/10/2024
0.18.0 138 7/10/2024
0.18.0-pullrequest0125-0014 72 7/10/2024
0.18.0-pullrequest0125-0013 78 7/10/2024
0.18.0-pullrequest0125-0012 71 7/10/2024
0.18.0-pullrequest0125-0011 70 7/10/2024
0.18.0-pullrequest0125-0010 71 7/10/2024
0.18.0-pullrequest0125-0009 84 7/10/2024
0.17.2-pullrequest0125-0008 73 7/10/2024
0.17.2-pullrequest0125-0007 80 7/10/2024
0.17.2-pullrequest0125-0006 72 7/10/2024
0.17.2-pullrequest0125-0005 74 7/10/2024
0.17.2-pullrequest0125-0004 80 7/10/2024
0.17.2-pullrequest0125-0003 76 7/9/2024
0.17.1 95 7/9/2024
0.17.1-pullrequest0124-0005 80 7/9/2024
0.17.0 98 7/8/2024
0.17.0-pullrequest0123-0024 64 7/8/2024
0.17.0-pullrequest0123-0023 67 7/8/2024
0.17.0-pullrequest0123-0021 82 7/8/2024
0.17.0-pullrequest0123-0019 72 7/8/2024
0.17.0-pullrequest0123-0018 61 7/8/2024
0.17.0-pullrequest0123-0017 74 7/8/2024
0.17.0-pullrequest0123-0015 71 7/8/2024
0.17.0-pullrequest0123-0014 68 7/8/2024
0.17.0-pullrequest0123-0013 72 7/8/2024
0.17.0-pullrequest0123-0012 68 7/8/2024
0.17.0-pullrequest0123-0011 73 7/8/2024
0.17.0-pullrequest0123-0009 75 7/6/2024
0.17.0-pullrequest0123-0008 71 7/6/2024
0.17.0-pullrequest0123-0007 109 7/6/2024
0.16.23 103 6/12/2024
0.16.23-pullrequest0118-0005 79 6/12/2024
0.16.23-pullrequest0118-0004 71 6/12/2024
0.16.23-pullrequest0118-0003 72 6/12/2024
0.16.23-pullrequest0118-0002 73 6/12/2024
0.16.21 153 5/29/2024
0.16.21-pullrequest0114-0002 78 5/29/2024
0.16.20 100 5/28/2024
0.16.20-pullrequest0112-0002 73 5/28/2024
0.16.19 103 5/13/2024
0.16.19-pullrequest0110-0004 83 5/13/2024
0.16.19-pullrequest0110-0003 73 5/13/2024
0.16.17 123 4/22/2024
0.16.17-pullrequest0108-0005 87 4/19/2024
0.16.17-pullrequest0108-0004 78 4/19/2024
0.16.17-pullrequest0108-0003 80 4/19/2024
0.16.16 112 4/16/2024
0.16.16-pullrequest0107-0003 89 4/16/2024
0.16.15 111 4/16/2024
0.16.15-pullrequest0106-0004 78 4/16/2024
0.16.15-pullrequest0106-0003 75 4/16/2024
0.16.15-pullrequest0106-0002 88 4/16/2024
0.16.14 175 4/16/2024
0.16.13 122 3/23/2024
0.16.13-pullrequest0103-0003 94 3/23/2024
0.16.12 107 3/23/2024
0.16.12-pullrequest0102-0006 84 3/22/2024
0.16.11 109 3/21/2024
0.16.11-pullrequest0101-0004 77 3/21/2024
0.16.11-pullrequest0101-0003 83 3/21/2024
0.16.11-pullrequest0101-0002 78 3/21/2024
0.16.9 116 3/16/2024
0.16.9-pullrequest0098-0008 83 3/16/2024
0.16.9-pullrequest0098-0007 84 3/16/2024
0.16.8 131 3/15/2024
0.16.8-pullrequest0097-0002 80 3/15/2024
0.16.7 109 3/10/2024
0.16.7-pullrequest0095-0002 90 3/10/2024
0.16.6 192 3/9/2024
0.16.6-pullrequest0094-0003 81 3/9/2024
0.16.6-pullrequest0094-0002 77 3/9/2024
0.16.2 118 3/5/2024
0.16.2-pullrequest0089-0009 86 3/5/2024
0.16.1 117 3/3/2024
0.16.1-pullrequest0088-0003 82 3/2/2024
0.16.1-pullrequest0088-0002 70 3/2/2024
0.16.0 161 2/28/2024
0.16.0-pullrequest0087-0010 78 2/28/2024
0.16.0-pullrequest0087-0009 81 2/28/2024
0.16.0-pullrequest0087-0007 91 2/28/2024
0.16.0-pullrequest0087-0006 83 2/28/2024
0.16.0-pullrequest0087-0005 76 2/28/2024
0.16.0-pullrequest0087-0004 87 2/28/2024
0.16.0-pullrequest0087-0003 80 2/28/2024
0.15.2 104 2/26/2024
0.15.2-pullrequest0086-0007 80 2/26/2024
0.15.2-pullrequest0086-0006 69 2/26/2024
0.15.1 112 2/23/2024
0.15.1-pullrequest0085-0006 78 2/23/2024
0.15.0 112 2/22/2024
0.15.0-pullrequest0084-0004 81 2/22/2024
0.15.0-pullrequest0084-0003 111 2/22/2024
0.14.1-pullrequest0084-0002 95 2/22/2024
0.14.0 128 2/21/2024
0.14.0-pullrequest0083-0011 103 2/8/2024
0.14.0-pullrequest0083-0010 83 2/8/2024
0.14.0-pullrequest0083-0009 84 2/8/2024
0.13.7 130 2/8/2024
0.13.7-pullrequest0082-0002 94 2/8/2024
0.13.6-pullrequest0081-0006 82 2/8/2024
0.13.5 137 2/7/2024
0.13.5-pullrequest0080-0002 83 2/7/2024
0.13.4 104 2/7/2024
0.13.4-pullrequest0079-0003 76 2/7/2024
0.13.1 111 2/1/2024
0.13.1-pullrequest0076-0003 86 2/1/2024
0.13.1-pullrequest0076-0002 85 2/1/2024
0.13.0 131 2/1/2024
0.13.0-pullrequest0075-0005 84 2/1/2024
0.13.0-pullrequest0075-0004 80 2/1/2024
0.12.2-pullrequest0075-0003 81 2/1/2024
0.12.1 108 2/1/2024
0.12.1-pullrequest0074-0002 80 2/1/2024
0.12.0 104 2/1/2024
0.12.0-pullrequest0073-0002 83 2/1/2024
0.11.0 106 1/31/2024
0.11.0-pullrequest0072-0011 87 1/31/2024
0.11.0-pullrequest0072-0010 80 1/31/2024
0.11.0-pullrequest0072-0008 76 1/31/2024
0.10.0 107 1/31/2024
0.10.0-pullrequest0070-0005 84 1/29/2024
0.10.0-pullrequest0070-0004 82 1/29/2024
0.9.1-pullrequest0069-0001 79 1/29/2024
0.9.0 102 1/29/2024
0.9.0-pullrequest0069-0002 77 1/29/2024
0.8.7-pullrequest0068-0001 81 1/29/2024
0.8.6 106 1/29/2024
0.8.4 124 1/19/2024
0.8.4-pullrequest0066-0009 80 1/19/2024
0.8.4-pullrequest0066-0008 73 1/19/2024
0.8.4-pullrequest0066-0007 83 1/19/2024
0.8.4-pullrequest0066-0005 76 1/19/2024
0.8.4-pullrequest0066-0004 82 1/19/2024
0.8.4-pullrequest0066-0003 78 1/19/2024
0.8.3 106 1/19/2024
0.8.3-pullrequest0065-0003 80 1/19/2024
0.8.3-pullrequest0065-0002 81 1/19/2024
0.8.1 103 1/19/2024
0.8.1-pullrequest0063-0013 79 1/19/2024
0.8.1-pullrequest0063-0011 80 1/19/2024
0.7.16-pullrequest0063-0010 86 1/19/2024
0.7.15 110 1/18/2024
0.7.15-pullrequest0062-0002 84 1/18/2024
0.7.12 118 1/16/2024
0.7.12-pullrequest0059-0006 88 1/16/2024
0.7.10 110 1/15/2024
0.7.10-pullrequest0059-0007 79 1/16/2024
0.7.10-pullrequest0058-0004 81 1/15/2024
0.7.10-pullrequest0057-0002 81 1/15/2024
0.7.9 111 1/15/2024
0.7.9-pullrequest0057-0003 77 1/15/2024
0.7.7 108 1/15/2024
0.7.7-pullrequest0056-0003 92 1/15/2024
0.7.4 112 1/15/2024
0.7.4-pullrequest0055-0012 84 1/15/2024
0.7.4-pullrequest0055-0011 85 1/15/2024
0.7.4-pullrequest0055-0010 92 1/15/2024
0.7.4-pullrequest0055-0009 88 1/15/2024
0.7.4-pullrequest0055-0008 87 1/15/2024
0.7.4-pullrequest0055-0007 86 1/15/2024
0.7.4-pullrequest0054-0005 84 1/15/2024
0.7.4-pullrequest0053-0004 97 1/15/2024
0.7.4-pullrequest0052-0002 76 1/15/2024
0.7.2 107 1/15/2024
0.7.2-pullrequest0051-0013 88 1/15/2024
0.7.2-pullrequest0050-0010 78 1/15/2024
0.7.1 117 1/13/2024
0.7.0 116 1/13/2024
0.7.0-pullrequest0046-0079 95 1/13/2024
0.6.12 136 1/3/2024
0.6.12-pullrequest0045-0018 80 1/13/2024
0.6.12-pullrequest0042-0010 86 1/12/2024
0.6.12-pullrequest0041-0007 84 1/12/2024
0.6.12-pullrequest0039-0002 89 1/3/2024
0.6.10 130 12/22/2023
0.6.10-pullrequest0038-0007 108 1/2/2024
0.6.10-pullrequest0038-0006 92 1/2/2024
0.6.10-pullrequest0038-0005 94 1/2/2024
0.6.8 126 12/21/2023
0.6.6 118 12/18/2023
0.6.6-pullrequest0035-0007 96 12/21/2023
0.6.6-pullrequest0035-0006 99 12/21/2023
0.6.6-pullrequest0035-0005 89 12/21/2023
0.6.3 116 12/18/2023
0.6.3-pullrequest0034-0018 93 12/18/2023
0.6.1 112 12/15/2023
0.6.1-pullrequest0029-0008 98 12/15/2023
0.6.1-pullrequest0028-0006 103 12/15/2023
0.6.1-pullrequest0027-0004 79 12/15/2023
0.5.3 119 12/11/2023
0.5.3-pullrequest0025-0049 97 12/14/2023
0.5.3-pullrequest0025-0047 95 12/14/2023
0.5.3-pullrequest0025-0043 96 12/13/2023
0.5.3-pullrequest0022-0038 93 12/13/2023
0.5.1 105 12/7/2023
0.5.0 111 12/6/2023
0.4.3 119 12/4/2023
0.4.2 108 12/4/2023
0.4.1 119 12/4/2023
0.3.10 127 11/22/2023
0.3.10-pullrequest0022-0008 94 11/27/2023
0.3.10-pullrequest0021-0005 94 11/22/2023
0.3.9 105 11/20/2023
0.3.8 115 11/9/2023
0.3.8-pullrequest0020-0005 90 11/9/2023
0.3.6 109 11/9/2023
0.3.5 113 11/9/2023
0.3.4 109 11/8/2023
0.3.3 110 11/8/2023
0.3.1 111 11/8/2023
0.3.1-pullrequest0018-0004 101 11/8/2023
0.2.1 146 11/2/2023
0.2.1-pullrequest0017-0050 109 11/8/2023
0.2.1-pullrequest0016-0048 111 11/8/2023
0.2.1-pullrequest0015-0045 100 11/8/2023
0.2.1-pullrequest0014-0035 111 11/7/2023
0.2.1-pullrequest0013-0033 104 11/7/2023
0.2.1-pullrequest0012-0031 104 11/7/2023
0.2.1-pullrequest0011-0024 103 11/6/2023
0.2.1-pullrequest0010-0015 120 11/2/2023
0.2.1-pullrequest0009-0011 105 11/2/2023
0.2.1-pullrequest0008-0009 109 11/2/2023
0.2.1-pullrequest0007-0007 115 11/2/2023
0.2.1-pullrequest0006-0005 112 11/2/2023
0.2.1-pullrequest0005-0001 101 11/2/2023
0.1.0 140 10/27/2023
0.1.0-pullrequest0004-0024 110 11/1/2023
0.1.0-pullrequest0003-0022 110 11/1/2023
0.1.0-pullrequest0002-0019 108 11/1/2023
0.1.0-pullrequest0001-0017 112 11/1/2023
0.1.0-pullrequest0001-0010 112 10/30/2023

releasenotes.md