Generate Podcast Audio from Text

This skill generates real audio narratives from text content using Azure OpenAI's Realtime API and the gpt-realtime-mini model. The workflow connects via WebSocket to the Azure OpenAI Realtime endpoint, sends a text prompt, collects streaming PCM audio chunks and a transcript, converts the PCM stream to WAV format, and returns base64-encoded audio to the frontend for playback.

On the backend, the WebSocket connection is configured for audio-only output with narrator instructions, then a text message is submitted as a conversation item and a response is requested. Streaming events are collected as they arrive: response.output_audio.delta carries base64-encoded audio chunks, response.output_audio_transcript.delta carries transcript text, and response.done signals completion so the loop can break; error events are handled via event.error.message. Collected PCM chunks are joined and converted to WAV at a 24kHz sample rate using a helper script.

On the frontend, the base64 WAV payload is converted into a playable blob and played directly:

const base64ToBlob = (base64, mimeType) => {
  const bytes = atob(base64);
  const arr = new Uint8Array(bytes.length);
  for (let i = 0; i < bytes.length; i++) arr[i] = bytes.charCodeAt(i);
  return new Blob([arr], { type: mimeType });
};

const audioBlob = base64ToBlob(response.audio_data, 'audio/wav');
const audioUrl = URL.createObjectURL(audioBlob);
new Audio(audioUrl).play();

Six voice options are available - alloy (neutral), echo (warm), fable (expressive), onyx (deep), nova (friendly), and shimmer (clear) - covering a range of narrator tones. Audio output is PCM at 24kHz, 16-bit, mono, stored and transmitted as base64-encoded WAV.

Use it when you need to generate spoken-audio narration from text content - podcast-style narration, read-alouds, or voice output for an application - using Azure OpenAI's Realtime API rather than a separate text-to-speech service. Configuration requires three environment variables (API key, endpoint, and deployment name), and the endpoint must be the base Cognitive Services URL without an /openai/v1/ suffix appended.

Input is a text prompt to narrate. Output is base64-encoded WAV audio (24kHz/16-bit/mono) plus a transcript, generated by streaming the Realtime API's audio-delta and transcript-delta events until the response completes. The session is explicitly configured for audio-only output modality with narrator instructions before the text is submitted as a conversation item. Further reference material covers the complete stack design and production code patterns, plus a dedicated script for PCM-to-WAV conversion.

Developers building applications that need narrated audio output from text - podcast generation, voice narration features, or accessibility read-aloud - using Azure OpenAI's Realtime API. Full architecture guidance and production-ready code patterns are provided as separate reference material for teams taking this beyond a proof of concept.