Configure Prompts for Multiple Models

This promptfoo configuration example demonstrates how to assign custom prompt sets to specific AI models within a single evaluation workflow. It allows you to define which prompts should be used with which models in your configuration file.

Use this pattern when you need to configure different prompts for different AI models within a single promptfoo evaluation. It's useful when you want to set up model-specific prompt configurations rather than using the same prompts across all models.

Do not use this if you're testing a single prompt across multiple models for comparison purposes - standard promptfoo configuration handles that more simply. Avoid this approach if your prompts are truly model-agnostic and don't require per-model configuration.

You provide a promptfooconfig.yaml file that maps specific prompts to specific models, along with your OpenAI API key as an environment variable. The workflow requires you to define which prompts should be used with which models in the configuration file.

You receive evaluation results viewable through the promptfoo results viewer.

Initialize the example and navigate to its directory:

npx promptfoo@latest init --example config-prompts-per-model
cd config-prompts-per-model

Set your OpenAI API key as an environment variable, then edit the promptfooconfig.yaml file to configure your model-prompt mappings.

Run the evaluation:

promptfoo eval

After evaluation completes, view results by running promptfoo view.

OpenAI models - Requires OPENAI_API_KEY environment variable.

Promptfoo evaluation framework - Built as a native example within the promptfoo testing and evaluation system.

Prompt engineers and ML engineers who need to configure different prompts for different AI models within their promptfoo evaluation workflows.