Compare LLM Performance on Riddles

This promptfoo example compares three large language models - OpenAI's GPT-5.4, Anthropic's Claude Sonnet 4.6, and Google's Gemini 3.1 Pro Preview - on riddle-solving tasks. According to the source material, it includes cost, latency, and quality assertions.

Use this example when you want to compare these three specific models on riddle-solving tasks.

Do not use this if you don't have API access to all three providers (OpenAI, Anthropic, and Google), or if you need to evaluate models on different types of tasks without modification.

You provide API keys for OpenAI, Anthropic, and Google services as environment variables:

export OPENAI_API_KEY=your-openai-key
export ANTHROPIC_API_KEY=your-anthropic-key
export GOOGLE_API_KEY=your-google-key

You receive evaluation results from the promptfoo framework comparing the three models on riddle-solving tasks.

This example requires API keys for OpenAI, Anthropic, and Google services. It uses the promptfoo evaluation framework.

To initialize and run the comparison:

npx promptfoo@latest init --example compare-gpt-vs-claude-vs-gemini
cd compare-gpt-vs-claude-vs-gemini

Then execute the evaluation:

npx promptfoo@latest eval

View the results:

npx promptfoo@latest view

This example is for users who want to compare GPT-5.4, Claude Sonnet 4.6, and Gemini 3.1 Pro Preview on riddle-solving tasks using the promptfoo framework.