Test ElevenLabs Voice AI Agents

This Promptfoo config tests ElevenLabs Conversational AI agents via the elevenlabs:agents provider, using three agent variants that share the same voice-agent framework but differ in LLM, criteria, and tools. A GPT-4o-backed support agent (voice 21m00Tcm4TlvDq8ikWAM) runs a five-point system prompt - greet customers warmly, listen carefully, provide accurate information, show empathy and professionalism, resolve issues efficiently - and is scored against five weighted evaluation criteria: greeting (weight 0.8, passing threshold 0.8), understanding (1.0/0.9), accuracy (1.0/0.9), helpfulness (0.9/0.8), and professionalism (0.7/0.8). It's driven by a simulated user (a "polite but needs help" customer persona, temperature 0.7, casual response style) across up to 10 conversation turns. A Claude-backed variant (claude-sonnet-4-5, a different voice ID) uses a trimmed 3-criterion set - greeting, understanding, helpfulness. A tool-enabled agent can call get_order_status (taking an order ID) and create_ticket (taking a title, description, and a priority enum of low/medium/high/urgent), evaluated specifically on whether it understands when to use tools and uses them appropriately.

Test prompts range from a single greeting ("Hello, I need help with my order") to full multi-turn transcripts: a weather chat that resolves a location follow-up, an order-tracking scenario referencing a specific order number (ORDER-12345) with an urgency cue, and a password-reset technical-support scenario triggered by an "invalid password" error message.

Four named tests check different dimensions of the interaction: "Conversation completes successfully" (output includes a conversation_id or history), "Evaluation criteria are met" (a JavaScript assertion parsing the response's analysis.evaluation_criteria_results and requiring at least 2 passed criteria), "Cost is reasonable" (under a $0.50 threshold), and "Response time is acceptable" (under 30 seconds).

Use it as a template for testing a voice or chat conversational agent's ability to greet, understand, and resolve user requests - including tool-calling agents and multi-turn conversations - while tracking cost and latency. Do not use it if you're not building on ElevenLabs Agents, or if a simpler single contains/llm-rubric assertion already covers your evaluation needs.

Input: the YAML config - agent prompts, LLM model choice, voice ID, evaluation criteria with weights and thresholds, optional tool definitions, and a simulated-user persona - plus the multi-turn test prompts. Output: Promptfoo's evaluation report, combining ElevenLabs' own criteria-based analysis with Promptfoo's javascript, cost, and latency assertions.

Uses Promptfoo's elevenlabs:agents provider, which can back an agent with either GPT-4o or Claude (claude-sonnet-4-5) as the underlying LLM, plus Promptfoo's cost and latency assertion types for budget and performance checks.

Teams building voice or chat-based conversational agents on ElevenLabs who need multi-criterion, cost- and latency-aware evaluation, including for tool-calling agents.