Select Best Output Using Max Score

This prompt chain demonstrates the max-score assertion type within promptfoo. It enables objective selection of the best LLM output by aggregating scores from multiple other assertions, effectively automating the process of identifying the highest-performing response.

Use this prompt chain when you need to objectively compare multiple LLM outputs and automatically select the one that performs best across a defined set of criteria. It is ideal for scenarios where you have established evaluation metrics and want to automate the selection of the optimal response for a given prompt. Do not use this if your evaluation is purely subjective or if you do not have a defined set of assertions to score the outputs against.

The primary input is a prompt that will be sent to one or more language models. The chain then applies a series of assertions to evaluate the outputs from these models. The output is the selected LLM response that achieved the highest aggregated score based on the max-score assertion configuration.

This example is designed to be used with promptfoo, an open-source tool for testing and evaluating LLM outputs. promptfoo supports integration with various language models and APIs, allowing you to test outputs from different providers.

This prompt chain is for developers, prompt engineers, and AI researchers who are focused on optimizing LLM performance and require a systematic, data-driven approach to selecting the best model outputs. It differs from manual selection by providing an automated, objective, and reproducible method for identifying top-performing responses based on predefined metrics.