Universal.Operative.Sdk.Realtime
2.2.0
See the version list below for details.
dotnet add package Universal.Operative.Sdk.Realtime --version 2.2.0
NuGet\Install-Package Universal.Operative.Sdk.Realtime -Version 2.2.0
<PackageReference Include="Universal.Operative.Sdk.Realtime" Version="2.2.0" />
<PackageVersion Include="Universal.Operative.Sdk.Realtime" Version="2.2.0" />
<PackageReference Include="Universal.Operative.Sdk.Realtime" />
paket add Universal.Operative.Sdk.Realtime --version 2.2.0
#r "nuget: Universal.Operative.Sdk.Realtime, 2.2.0"
#:package Universal.Operative.Sdk.Realtime@2.2.0
#addin nuget:?package=Universal.Operative.Sdk.Realtime&version=2.2.0
#tool nuget:?package=Universal.Operative.Sdk.Realtime&version=2.2.0
About
Universal.Operative.Sdk.Realtime is the duplex-stream/voice half of the Universal.Operative.Sdk family — the counterpart to Universal.Operative.Sdk.Discrete's turn-based model, for building agents against live, bidirectional session APIs (OpenAI Realtime, Gemini Live, Amazon Nova Sonic, Grok Voice) instead of discrete request/response turns.
This package ships the provider-neutral core only: the canonical SessionEvent model, IRealtimeSession/IResumableSession, IModel, IOperative/OperativeBuilder/OperativeFactory, the ITransformer<TEvent> hook pipeline, and Conversation/ConversationWriter/ConversationReader for recording and replaying a session. It has no provider adapters on its own — pair it with a provider package (e.g. Universal.Operative.Sdk.Realtime.OpenAI) for a concrete IModel.
How to Use
Installation
dotnet add package Universal.Operative.Sdk
dotnet add package Universal.Operative.Sdk.Realtime
Core concepts
SessionEvent— the canonical event model, split intoSessionOutputEvent(whatIRealtimeSession.Outputemits) andSessionInputEvent(whatIRealtimeSession.SendAsyncaccepts) so the two directions are a compile-time distinction, not just a naming convention. Covers what's shared across providers — transcripts, audio deltas, tool calls, interruption, turn/session lifecycle — plus aRawPassthroughEvent(output) /RawSendEvent(input) escape-hatch pair for whatever a given provider does that hasn't been promoted into the canonical shape. Every event carriesRaw, a list ofRawEventRefs tracing it back to the provider-native event(s) it came from, and the whole hierarchy is JSON-serializable (seeConversationbelow) so nothing is ever silently lost between the canonical, raw, and recorded views.IRealtimeSession— one live duplex connection to a provider.Capabilities(aSessionCapabilitiesflags enum) tells you what this particular session can actually do — e.g. whether audio flows through the canonical stream at all (AudioInProcess), which is false for topologies where the media plane bypasses your process entirely (WebRTC-direct, SIP passthrough) even though tool-calling and transcripts might still work.IModel— connects and returns anIRealtimeSession. Every adapter's primary (WebSocket) connection mode shares this sameConnectAsyncshape; provider-specific connection modes that don't (SIP passthrough, browser-direct WebRTC) stay off this interface as extra methods on the concreteModeltype instead.IResumableSession— for providers with session resumption, native (Gemini's handle) or client-assisted (an adapter that synthesizes the same contract by splicing history into a fresh session, e.g. for Nova Sonic). CheckCapabilitiesfor which kind you have.IOperative/OperativeBuilder/OperativeFactory— wraps an already-connectedIRealtimeSessionand is the intended entry point for both directions of traffic:RunAsync()relays the (transformer-applied) event stream and dispatchesToolCallRequestedEvents to your registeredITools automatically (reusing the exact sameITool/ToolResultcontract as the turn-based side), andSendAsyncis where anything you send — including the tool resultsRunAsyncgenerates on your behalf — passes through before reaching the session.OperativeBuilderis the one-shot fluent path for a single session;OperativeFactorymints oneIOperativeper session from a reusable tool/transformer configuration, for a host spawning many concurrent sessions (e.g. one per incoming call).ITransformer<TEvent>— the hook seam for observing, rewriting, or suppressing one event at a time as it passes through anIOperative, registered separately per direction viaOperativeBuilder.AddOutputTransformer/AddInputTransformer. An output transform only shapes what a consumer (and anything downstream, like a recorder) sees — it can't undo what the provider already did; an input transform has real teeth, since it runs before the event is ever sent.Conversation/ConversationWriter/ConversationReader— the durable, serializable counterpart to a live session (deliberately sharingDiscrete.Conversation's name: same role, disambiguated by namespace).ConversationWriterappends one JSON line per event as a session runs;ConversationReaderreads it back, either fully materialized or as a lazy stream.RecordingSessionis a transparentIRealtimeSessiondecorator that wires a session straight into a writer;RecordedSessionreplays aConversationback out as a fakeIRealtimeSessionwith no provider connection, for debugging or as an adapter-agnostic test fixture.AudioAdapter— a duplex wrapper over an already-connectedIOperativethat resamples PCM16 audio in both directions, so your microphone/speaker hardware rate doesn't need to match a provider's wire rate.WriteMicrophoneAudioAsync/CommitMicrophoneAudioAsyncaccept audio at your chosen input rate and resample it before sending;PlaybackAudioyields provider output audio resampled to your chosen playback rate;Eventsyields every otherSessionOutputEventunchanged. Owns the wrappedIOperative'sRunAsync()pump internally -- disposing the adapter disposes the operative.
Quick start
using Universal.Operative.Sdk.Realtime;
IRealtimeSession session = /* from a provider's IModel — e.g. someProvider.ConnectAsync(options) */;
IOperative operative = OperativeBuilder.Create(session)
.AddTool(myTool)
.Build();
await foreach (SessionOutputEvent e in operative.RunAsync())
{
switch (e)
{
case TranscriptDeltaEvent t: Console.WriteLine($"{t.Role}: {t.Text}"); break;
case AudioOutputDeltaEvent a: /* play a.Audio */ break;
case InterruptedEvent: /* stop playback immediately */ break;
case ToolCallRequestedEvent call: Console.WriteLine($"Calling {call.Name}..."); break;
}
}
await operative.SendAsync(new AppendAudioInputEvent { Audio = micChunk });
Tool dispatch policy
Realtime tool requests pass through the same ToolCallDispatcher as discrete engine calls and callback-driven adapters. Configure the boundary on OperativeBuilder:
OperativeBuilder builder = OperativeBuilder.Create()
.AddTool(myTool)
.AddBeforeToolDispatchHook(auditHook)
.AddAfterToolDispatchHook(resultPolicyHook);
builder.ToolTimeout = TimeSpan.FromMinutes(2);
builder.MaxToolTimeout = TimeSpan.FromMinutes(10);
builder.BackgroundDispatch = DispatchBackgroundAsync;
builder.Diagnostics = diagnostics;
Hooks can rewrite or veto calls and transform results. The realtime operative retains the provider's original call ID when submitting a result even if a hook rewrites the local call, preserving wire correlation.
Tool dispatch observation
Set OperativeBuilder.ToolDispatchCompleted to observe the shared dispatcher's typed terminal facts without installing a logger or telemetry backend. Path distinguishes tool execution from hook results and background acceptance; Outcome distinguishes success, failure, timeout, cancellation, and controlled termination. Background acceptance does not mean background work has finished.
Observers are informational; observer failures remain secondary to the operation. Tool policy belongs in the before/after dispatch hooks. After-hooks now receive ordinary foreground error results even with a configured timeout. They do not run for unavailable, rejected, timed-out, or throwing paths; use the terminal observer to cover all paths.
Uncooperative foreground work may outlive dispatch. PendingExecution is a drain handle for the abandoned work and its SDK cleanup. The host chooses whether to await it before releasing owned tool resources. Payloads and exception context are not automatically logged or persisted.
License
Free for noncommercial use under the Universal.Operative Noncommercial License. Source is closed; commercial use requires a separate license from Andrew Ong.
| Product | Versions Compatible and additional computed target framework versions. |
|---|---|
| .NET | net8.0 is compatible. net8.0-android was computed. net8.0-browser was computed. net8.0-ios was computed. net8.0-maccatalyst was computed. net8.0-macos was computed. net8.0-tvos was computed. net8.0-windows was computed. net9.0 was computed. net9.0-android was computed. net9.0-browser was computed. net9.0-ios was computed. net9.0-maccatalyst was computed. net9.0-macos was computed. net9.0-tvos was computed. net9.0-windows was computed. net10.0 is compatible. net10.0-android was computed. net10.0-browser was computed. net10.0-ios was computed. net10.0-maccatalyst was computed. net10.0-macos was computed. net10.0-tvos was computed. net10.0-windows was computed. |
-
net10.0
- Universal.Operative.Sdk (>= 6.4.0)
-
net8.0
- Universal.Operative.Sdk (>= 6.4.0)
NuGet packages (4)
Showing the top 4 NuGet packages that depend on Universal.Operative.Sdk.Realtime:
| Package | Downloads |
|---|---|
|
Universal.Operative.Sdk.Realtime.OpenAI
OpenAI Realtime API adapter for Universal.Operative.Sdk.Realtime. |
|
|
Universal.Operative.Sdk.Realtime.xAI
xAI Grok Voice Agent adapter for Universal.Operative.Sdk.Realtime. |
|
|
Universal.Operative.Sdk.Realtime.Google.Gemini
Gemini Live API adapter for Universal.Operative.Sdk.Realtime. |
|
|
Universal.Operative.Sdk.Realtime.Aws.Bedrock
Amazon Bedrock Nova Sonic adapter for Universal.Operative.Sdk.Realtime. |
GitHub repositories
This package is not used by any popular GitHub repositories.
Exposes terminal tool dispatch observations with typed path, outcome, and abandonment facts. Shares improved timeout cleanup and error-result hook behavior from SDK 6.4.