Evaluate New Model Performance on Existing Code Explanations

This Evals example compares gpt-4.1-mini against gpt-4o-mini by evaluating both on stored responses rather than a hand-built eval dataset, using the responses data source in the OpenAI Evals API. This matters because most developers already have real production responses sitting in their logs page - so the eval works directly against existing traffic instead of requiring a fresh dataset to be assembled.

Use this when you want to know how a new model would have performed on real requests your application already handled, without regenerating a synthetic eval set. Because the responses data source requires existing user traffic, the worked example first generates some example traffic with gpt-4o (asking it to explain code files from the OpenAI SDK) before comparing gpt-4.1-mini against it - this step is only needed if you don't already have logged responses to draw on.

This only works in an org where data logging isn't disabled (for example, via Zero Data Retention) - you can confirm logging is active by checking whether generated responses appear at platform.openai.com/logs?api=responses. A first eval run evaluates the quality of the original gpt-4o responses by filtering for which logged responses to include; a second run then evaluates gpt-4.1-mini the same way, so both models' scores land in the same eval for direct comparison.

Results for both runs appear together in the OpenAI Evals dashboard, letting you compare gpt-4.1-mini's performance against gpt-4o-mini's on the exact same underlying task - explaining code files - side by side.

Developers deciding whether to switch models who want to validate the new model's quality against real, already-logged production responses rather than building a separate benchmark dataset from scratch.