Generate Podcast Audio from Text

Turns a text prompt into real spoken audio by connecting to Azure OpenAI's Realtime API using the gpt-realtime-mini deployment. The flow: configure the Realtime API environment variables, open a WebSocket connection to the Azure OpenAI Realtime endpoint, send the text prompt, collect the streamed PCM audio chunks and transcript, convert the PCM to WAV, and return the audio as base64 to the frontend for playback. Configuration goes through three environment variables:

AZURE_OPENAI_AUDIO_API_KEY=your_realtime_api_key
AZURE_OPENAI_AUDIO_ENDPOINT=https://your-resource.cognitiveservices.azure.com
AZURE_OPENAI_AUDIO_DEPLOYMENT=gpt-realtime-mini

The endpoint value must be the bare base URL, not one that already includes /openai/v1/. On the backend, the HTTPS endpoint is converted to a WebSocket URL by swapping the scheme to wss:// and appending /openai/v1, then used to open an AsyncOpenAI client connection via client.realtime.connect(model="gpt-realtime-mini"). The session is configured for audio-only output (output_modalities: ["audio"]) with narrator instructions, the text prompt is sent as a conversation item, and a response is requested. Streamed events are handled as they arrive: response.output_audio.delta carries a base64 audio chunk to decode and buffer, response.output_audio_transcript.delta carries transcript text, response.done signals the end of generation, and error events are handled via event.error.message. The buffered PCM (24kHz, 16-bit, mono) is then converted to WAV, using the bundled scripts/pcm_to_wav.py, and stored or returned as base64. On the frontend, that base64 WAV is decoded into a Blob, turned into an object URL, and played through a standard Audio element.

Six voices are available: alloy (neutral), echo (warm), fable (expressive), onyx (deep), nova (friendly), and shimmer (clear).

Use it to add narrated or spoken-audio output to an app backed by Azure OpenAI, such as turning generated text content into a playable narration. It is specifically built around Azure OpenAI's Realtime API and the gpt-realtime-mini deployment, so it does not apply to other TTS providers or to Azure deployments configured with a different endpoint shape, since the code appends /openai/v1 itself and expects a bare base URL.

Input is a text prompt plus the three Realtime API environment variables (API key, endpoint, deployment name). Output is base64-encoded WAV audio, converted from 24kHz, 16-bit mono PCM, alongside the streamed transcript text, ready for playback in a browser via a Blob-backed Audio element.

Integrates with Azure OpenAI's Realtime API over WebSocket, via the AsyncOpenAI client's realtime.connect() interface. Further architecture detail lives in references/architecture.md, production usage patterns in references/code-examples.md, and the PCM-to-WAV conversion utility in scripts/pcm_to_wav.py.

Developers building an app on Azure OpenAI that needs to turn text into narrated audio, for example converting generated article or report content into a spoken version for playback in a web frontend.