Universal.Operative.Sdk.Realtime.OpenAI
3.0.0
dotnet add package Universal.Operative.Sdk.Realtime.OpenAI --version 3.0.0
NuGet\Install-Package Universal.Operative.Sdk.Realtime.OpenAI -Version 3.0.0
<PackageReference Include="Universal.Operative.Sdk.Realtime.OpenAI" Version="3.0.0" />
<PackageVersion Include="Universal.Operative.Sdk.Realtime.OpenAI" Version="3.0.0" />
<PackageReference Include="Universal.Operative.Sdk.Realtime.OpenAI" />
paket add Universal.Operative.Sdk.Realtime.OpenAI --version 3.0.0
#r "nuget: Universal.Operative.Sdk.Realtime.OpenAI, 3.0.0"
#:package Universal.Operative.Sdk.Realtime.OpenAI@3.0.0
#addin nuget:?package=Universal.Operative.Sdk.Realtime.OpenAI&version=3.0.0
#tool nuget:?package=Universal.Operative.Sdk.Realtime.OpenAI&version=3.0.0
Universal.Operative.Sdk.Realtime.OpenAI
About
Universal.Operative.Sdk.Realtime.OpenAI adapts the OpenAI Realtime API to Universal.Operative.Sdk.Realtime's provider-neutral IModel/IRealtimeSession/SessionEvent model.
Two connection paths in this release: Model.ConnectAsync (server-mediated WebSocket — audio relays through your process) and Model.ConnectForCallAsync (SIP passthrough — audio flows directly between the caller and OpenAI, never touching your process). Browser-direct WebRTC (an ephemeral-key handshake so a browser tab talks to OpenAI without your server in the audio path) isn't implemented yet.
How to Use
Installation
dotnet add package Universal.Operative.Sdk.Realtime.OpenAI
Quick start
using Universal.Operative.Sdk;
using Universal.Operative.Sdk.Realtime;
using Universal.Operative.Sdk.Realtime.OpenAI;
Model model = new Model(apiKey, modelId: "gpt-realtime", defaultVoice: "marin");
OperativeBuilder builder = OperativeBuilder.Create(new RealtimeSessionOptions { SystemPrompt = [new TextBlock("You are a helpful voice assistant.")], }).AddTool(myTool);
// apiKey, myTool, micChunk (24kHz mono PCM16) and cancellationToken come from the host.
await using IOperative operative = await builder.ConnectAsync(model, cancellationToken);
await operative.SendAsync(new AppendAudioInputEvent { Audio = micChunk }, cancellationToken);
await foreach (SessionOutputEvent e in operative.RunAsync(cancellationToken))
{
switch (e)
{
case SessionReadyEvent:
Console.WriteLine("connected");
break;
case TranscriptDeltaEvent t:
Console.WriteLine($"{t.Role}: {t.Text}");
break;
case AudioOutputDeltaEvent a: /* play a.Audio (24kHz PCM16) */
break;
case InterruptedEvent: /* stop playback immediately */
break;
}
}
A continuous microphone producer runs concurrently with output enumeration. Cancel the session token to stop both; disposing the operative disposes its session. Server errors and response status details have serialized typed-event companions sharing the canonical event's sequence; no payloads are automatically logged. These preserve every field supplied by the pinned client, which exposes status-error code/type but does not expose a status-error message or the original wire JSON.
Fidelity notes
- Capabilities:
AudioInProcess | ToolCalling | Transcripts— audio flows through this same WebSocket, and there's no native session resumption (SessionCapabilities.NativeResumption/ClientAssistedResumptionare both unset). - Tool results: the full
ToolResultcontent (all blocks, not just text) is flattened into OpenAI's single-stringfunction_call_output— text blocks contribute their text verbatim, anything else is JSON-serialized rather than silently dropped. - Tool choice:
RealtimeSessionOptions.ToolChoiceis sent as OpenAI's owntool_choiceat connect time (same shape as Chat Completions/Responses — a bare string forauto/required/none, a structured object to force one specific tool). - Raw passthrough: OpenAI's Realtime API has substantially more event types than are mapped to canonical events here (item lifecycle events, rate-limit updates, built-in MCP tool events, text-modality deltas). Anything not explicitly mapped arrives as a
RawPassthroughEventrather than being dropped. - Mid-session reconfiguration and other provider-specific sends: OpenAI's
session.update(changing instructions/voice/tools/tool-choice after connecting) has no canonicalSessionInputEventyet — unconfirmed whether Gemini or Nova Sonic support anything equivalent, so it isn't promoted.RawSendEventcovers this and more generally: it resolves itsRawJsonagainst every client event this adapter can send (session.update,response.create,response.cancel, theconversation.item.*/input_audio_buffer.*family, etc.) by matching its"type"field. A constructed event (e.g.SessionUpdateEventfromUniversal.OpenAI.Realtime.Client) asPayloadalso works, and is used as-is with no deserialization. Explicitly requesting a response with no new input first (needed because OpenAI's server-side turn detection doesn't always resume on its own afterinterrupt_responseauto-cancels a response mid-generation) goes through this same path:RawSendEvent { RawJson = "{\"type\":\"response.create\"}" }. Any input event this doesn't recognize still throwsNotSupportedException.
| Product | Versions Compatible and additional computed target framework versions. |
|---|---|
| .NET | net10.0 is compatible. net10.0-android was computed. net10.0-browser was computed. net10.0-ios was computed. net10.0-maccatalyst was computed. net10.0-macos was computed. net10.0-tvos was computed. net10.0-windows was computed. net11.0 is compatible. |
-
net10.0
- Universal.OpenAI.Realtime.Client (>= 1.2.0)
- Universal.Operative.Sdk.Realtime (>= 3.0.0)
-
net11.0
- Universal.OpenAI.Realtime.Client (>= 1.2.0)
- Universal.Operative.Sdk.Realtime (>= 3.0.0)
NuGet packages
This package is not used by any NuGet packages.
GitHub repositories
This package is not used by any popular GitHub repositories.
Targets .NET 10 and .NET 11; removes .NET 8 support.