WebSocket Protocol

After starting a session via REST, connect to the WebSocket endpoint for real-time bidirectional audio streaming.

TEXT

wss://ultravoice.us.inc/api/v1/voice/ws/{session_id}

Audio Format

Sending Audio (Client → Server)

Format: Raw PCM Int16 (16-bit signed integer)
Sample rate: 16,000 Hz (16kHz)
Channels: Mono (1 channel)
Frame size: Any size, 4096 samples recommended
Frame type: Binary WebSocket frames

Receiving Audio (Server → Client)

Format: WAV-wrapped PCM audio (binary frames)
Sample rate: Matches TTS provider output (typically 22050 or 24000 Hz)
Channels: Mono

Events Reference

The server sends JSON text frames alongside audio binary frames:

JAVASCRIPT

// Session start
{ "type": "session_start", "session_id": "...", "agent_id": "..." }

// Transcript (partial)
{ "type": "transcript_partial", "text": "Hello, how can I", "is_final": false }

// Transcript (final)
{ "type": "transcript_final", "text": "Hello, how can I help you today?", "is_final": true }

// AI response (streaming)
{ "type": "llm_token", "text": "I'd", "turn_id": "t1" }

// AI response (complete)
{ "type": "llm_complete", "text": "I'd be happy to help!", "turn_id": "t1" }

// Tool call
{ "type": "tool_call", "name": "lookup_order", "args": {"order_id": "ORD-123"} }

// Tool result
{ "type": "tool_result", "name": "lookup_order", "result": "Order shipped on Jan 15" }

// Session end
{ "type": "session_end", "reason": "user_ended" }

// Error
{ "type": "error", "code": "asr_timeout", "message": "ASR provider timed out" }

Browser Client Example

JAVASCRIPT

const ws = new WebSocket('wss://ultravoice.us.inc/api/v1/voice/ws/SESSION_ID');
ws.binaryType = 'arraybuffer';

// Capture microphone
const stream = await navigator.mediaDevices.getUserMedia({
  audio: { sampleRate: 16000, channelCount: 1, echoCancellation: true }
});
const ctx = new AudioContext({ sampleRate: 16000 });
const source = ctx.createMediaStreamSource(stream);
const processor = ctx.createScriptProcessor(4096, 1, 1);
source.connect(processor);
processor.connect(ctx.destination);

// Send audio
processor.onaudioprocess = (e) => {
  if (ws.readyState !== WebSocket.OPEN) return;
  const float32 = e.inputBuffer.getChannelData(0);
  const int16 = new Int16Array(float32.length);
  for (let i = 0; i < float32.length; i++) {
    int16[i] = Math.max(-32768, Math.min(32767, float32[i] * 32768));
  }
  ws.send(int16.buffer);
};

// Receive messages
ws.onmessage = (e) => {
  if (e.data instanceof ArrayBuffer) {
    playAudio(e.data);  // Play TTS audio
  } else {
    const event = JSON.parse(e.data);
    console.log(event.type, event.text || '');
  }
};