Organize and label prompts across multiple providers

This prompt chain demonstrates how to use prompt labels in Promptfoo to organize prompts and providers. In a single evaluation, you can run one set of multiple prompts against a provider, another set of prompts against another provider, and get back a single result.

Use this workflow when you need to organize your prompts and providers with labels in Promptfoo evaluations. The example shows how to structure your configuration to run different prompt sets against different providers.

Do NOT use this if you only need to test a single prompt against one provider - the labeling overhead adds unnecessary complexity. Avoid this approach when your evaluation logic requires sequential dependencies between prompts.

You provide a promptfooconfig.yaml configuration file that defines your prompt sets with labels and associates them with specific providers. You also need to set your OPENAI_API_KEY environment variable before running the evaluation.

You receive a single consolidated evaluation result. Results can be viewed interactively using the promptfoo view command.

The example requires setting your OPENAI_API_KEY environment variable. The workflow is configured through YAML files and executed via the Promptfoo CLI.

This workflow is for users who want to organize their prompts and providers using labels in Promptfoo evaluations.

To get started:

npx promptfoo@latest init --example config-prompt-labels
cd config-prompt-labels

Then edit your promptfooconfig.yaml and run:

promptfoo eval

After evaluation completes, view results with promptfoo view.