Universal.Operative.Sdk.Realtime.xAI 1.1.0

About

Universal.Operative.Sdk.Realtime.xAI adapts xAI's Grok Voice Agent API to Universal.Operative.Sdk.Realtime's provider-neutral IModel/IRealtimeSession/SessionEvent model.

Grok's Voice Agent implements the OpenAI Realtime API's core events verbatim (session.update, input_audio_buffer.append, response.create, conversation.item.create, ...), so this adapter reuses Universal.OpenAI.Realtime.Client's event DTOs and polymorphic (de)serialization directly. It does not reuse RealtimeClient itself, since its ConnectAsync overloads are hardcoded to wss://api.openai.com with no endpoint override -- the WebSocket connection to wss://api.x.ai/v1/realtime is hand-rolled against the same wire types.

How to Use

Installation

dotnet add package Universal.Operative.Sdk
dotnet add package Universal.Operative.Sdk.Realtime
dotnet add package Universal.Operative.Sdk.Realtime.xAI

Quick start

using Universal.Operative.Sdk.Realtime;
using Universal.Operative.Sdk.Realtime.xAI;

var model = new Model(apiKey, modelId: "grok-voice-latest");

IRealtimeSession session = await model.ConnectAsync(new RealtimeSessionOptions
{
    SystemPrompt = [new TextBlock("You are a helpful voice assistant.")],
    Tools = [myTool],
});

await foreach (SessionOutputEvent e in session.Output)
{
    switch (e)
    {
        case SessionReadyEvent: Console.WriteLine("connected"); break;
        case TranscriptDeltaEvent t: Console.WriteLine($"{t.Role}: {t.Text}"); break;
        case AudioOutputDeltaEvent a: /* play a.Audio (24kHz PCM16 by default) */ break;
        case InterruptedEvent: /* stop playback immediately */ break;
    }
}

await session.SendAsync(new AppendAudioInputEvent { Audio = micChunk });

Fidelity notes

• Capabilities: AudioInProcess | ToolCalling | Transcripts -- xAI documents a resumption session parameter for context caching across reconnects, but it isn't wired up (no NativeResumption/ClientAssistedResumption).
• Voice default: Model's defaultVoice parameter is null unless you supply one, leaving the choice to xAI's own server-side default.
• Transcription default: GrokRealtimeSessionOptions.InputTranscription defaults to xAI's own "grok-transcribe" model.
• Known gap: xAI documents a conversation.item.input_audio_transcription.updated event (a cumulative transcript, specific to "grok-transcribe") with no OpenAI counterpart. The vendor event DTOs this adapter reuses don't recognize it, so it surfaces as a RawPassthroughEvent rather than a canonical TranscriptDeltaEvent.
• Tool results: the full ToolResult content is flattened into xAI's single-string function_call_output, same as the OpenAI adapter.
• Tool choice: RealtimeSessionOptions.ToolChoice is sent the same way as the OpenAI adapter.
• Mid-session reconfiguration and other provider-specific sends: RawSendEvent resolves its RawJson against every client event this adapter can send (session.update, response.create, response.cancel, the conversation.item.*/input_audio_buffer.* family, etc.) by matching its "type" field -- a real escape hatch, not a single allowlisted shape. A constructed event (e.g. SessionUpdateEvent) as Payload also works, and is used as-is with no deserialization.
• Known gap: cross-modal text sent before any audio gets no response at all. Priming with a spoken audio turn first works for a standalone turn, but a full round trip that also waits for the priming turn's own completion can hang with no response and no error.