What problem does separating test configs solve?

It simplifies large `promptfooconfig.yaml` files by splitting test definitions into separate files referenced by path or wildcard, making configurations easier to navigate and manage.

When should I use separate test config files?

Use this approach when a single `promptfooconfig.yaml` is getting large or hard to navigate; it's not needed for small configs where all tests fit cleanly inline.

How do I set up this example?

Run `npx promptfoo@latest init --example config-separate-test-configs` followed by `cd config-separate-test-configs` to initialize the example locally.

Does this require external service integration?

No, this is a pure promptfoo configuration mechanism with no external service integration required.

Prompt Chain

Organize Test Configs with Path Wildcards

Name: Config Separate Test Configs
Availability: OnlineOnly
Author: Promptfoo

Organize promptfoo test configurations using paths and wildcards to simplify large test suites and improve maintainability.

Copy chain

Works with promptfoo

Promptfoo

Own this? Claim it

Spark score

out of 100

Updated 19 days ago

Version 0.121.18

Add to Favorites

Why it matters

Simplify and organize your promptfoo configuration by separating test configurations into multiple files using paths and wildcards, making large test suites more maintainable and easier to manage.

Outcomes

What it gets done

Split test configurations across multiple files for better organization

Use wildcard patterns to reference multiple test config files

Maintain cleaner promptfooconfig.yaml with path-based imports

Scale test suites without cluttering the main configuration file

Install

Add it to your toolbox

Run in your project directory:

curl -fsSL https://spark.entire.vc/get/pfoo-config-separate-test-configs | bash

Overview

Config Separate Test Configs

A promptfoo configuration example showing how to use paths and wildcards to simplify promptfooconfig.yaml. Use this example when you want to learn how to use paths and wildcards in your promptfoo configuration files.

What it does

This promptfoo workflow example demonstrates how to use paths and wildcards to simplify promptfooconfig.yaml.

Inputs and outputs

You provide test configuration files organized in your project directory structure, along with a main promptfooconfig.yaml that references them using paths and wildcards.

Who it's for

This example is for developers working with promptfoo who want to learn how to use paths and wildcards in their configuration files.

To get started with this example:

npx promptfoo@latest init --example config-separate-test-configs
cd config-separate-test-configs

Once initialized, run the evaluation:

promptfoo eval

The example provides a working demonstration of how to use paths and wildcards in promptfooconfig.yaml.

Source README

config-separate-test-configs (Separate Test Configs)

You can run this example with:

npx promptfoo@latest init --example config-separate-test-configs
cd config-separate-test-configs

Usage

This example shows how you can use paths and wildcards to simplify for promptfooconfig.yaml.

Run:

promptfoo eval

FAQ

Common questions

Discussion