Build Real-Time Voice & Video Apps with Gemini Live API

A skill for building real-time, bidirectional streaming applications with the Gemini Live API - WebSocket-based audio, video, and text streaming, voice activity detection, native audio, function calling, session management, and ephemeral client-side tokens. Key capabilities: bidirectional mic-to-speaker audio streaming, video streaming (camera/screen frames alongside audio), text input/output within a live session, audio transcriptions of both input and output, voice activity detection with automatic interruption handling, native audio "thinking" via a configurable thinkingLevel, synchronous function calling, Google Search grounding, and session management (context compression, session resumption, GoAway signals). The Live API currently only supports WebSockets - WebRTC or simplified integration requires a partner (LiveKit, Pipecat, Fishjam, Vision Agents, Voximplant, or Firebase AI Logic). The recommended model is gemini-3.1-flash-live-preview (low-latency, native audio output, thinking support, 128k context); gemini-3.5-live-translate-preview handles real-time streaming translation; three older Live models (gemini-2.5-flash-native-audio-preview-12-2025, gemini-live-2.5-flash-preview, gemini-2.0-flash-live-001) are deprecated with a December 9, 2025 shutdown. SDKs are google-genai (Python) and @google/genai (JS/TS); the legacy google-generativeai/@google/generative-ai packages are deprecated. Audio format is fixed: input is raw PCM, little-endian, 16-bit, mono, 16kHz native (other rates get resampled, MIME audio/pcm;rate=16000); output is the same format at 24kHz. A critical usage rule: send_realtime_input/sendRealtimeInput handles all real-time user input (audio, video, and text) during a live conversation, while send_client_content/sendClientContent is only for seeding initial context history and must never be used to send new user messages mid-conversation; within sendRealtimeInput, the generic "media" key is disallowed in favor of specific "audio", "video", and "text" keys. The quick-start flow authenticates a client, opens a live.connect() session with a model and LiveConnectConfig (response modalities, system instruction), then sends text/audio/video via send_realtime_input and receives responses via an async iteration over session.receive() - a single server event can contain multiple content parts simultaneously (e.g. audio plus transcript), so every part in each event must be processed to avoid dropping content.

Use it when building real-time voice or video interactions with Gemini over WebSockets - live conversational audio, video streaming, or real-time translation. Not for simple request/response text generation, which the standard Interactions API handles instead.

Input is a live-connect config (model, response modalities, system instruction) plus a continuous stream of audio/video/text sent via send_realtime_input. Output is a stream of server events received via session.receive(), each potentially containing multiple simultaneous content parts (audio chunks, transcripts, function calls) that must all be processed.

pip install google-genai
npm install @google/genai

WebSocket-only natively; WebRTC or simplified integration is available through partner platforms (LiveKit, Pipecat, Fishjam, Vision Agents, Voximplant, Firebase AI Logic).

Developers building real-time voice/video AI applications - live conversational assistants, streaming translation, or interruptible spoken interfaces - who need the correct current model, exact PCM audio format, and the send_realtime_input vs send_client_content distinction that's easy to get wrong.