What metadata column types does this prompt chain support?

It supports two metadata column types: `__metadata:category` for a single categorization value per row, and `__metadata:tags[]` for multiple comma-separated tags with escaping support (e.g., "global\,warming" is treated as one tag).

How do I filter test runs by metadata in this example?

You can filter test runs using the `--filter-metadata` flag with `promptfoo eval`, such as `promptfoo eval --filter-metadata category=technology` or `promptfoo eval --filter-metadata tags=ai`.

What file formats does this example support for test cases?

It supports both CSV (`tests.csv`) and Excel (`tests.xlsx`) file formats; Excel support requires the optional `read-excel-file` npm package.

When should I use this metadata filtering pattern?

Use it when your test suite is large enough that you want to run only subsets of tests without maintaining separate config files, or when test cases are maintained in spreadsheets by non-engineers who can add categorization directly in the spreadsheet.

Prompt Chain

Organize Test Cases with CSV Metadata Filtering

Name: Config Csv Metadata
Availability: OnlineOnly
Author: Promptfoo

Structure and filter AI prompt test cases using metadata columns in CSV/Excel files for organized evaluation workflows.

Copy chain

Works with csvexcel

Promptfoo

Own this? Claim it

Spark score

out of 100

Updated 18 days ago

Version 0.121.18

Add to Favorites

Why it matters

Enable developers to structure, categorize, and selectively run AI prompt test cases using metadata columns in CSV or Excel files, making it easier to manage large test suites and filter evaluations by category or tags.

Outcomes

What it gets done

Add metadata columns to CSV/Excel test files for categorization

Filter test case execution by single-value category metadata

Tag test cases with multiple labels using array metadata syntax

Run selective prompt evaluations based on metadata filters

Install

Add it to your toolbox

Run in your project directory:

curl -fsSL https://spark.entire.vc/get/pfoo-config-csv-metadata | bash

Steps

Steps in the chain

Setup the config-csv-metadata example

Run basic evaluation

Install Excel support (optional)

Run evaluation with Excel file

Filter evaluation by category

Filter evaluation by tags

Overview

Config Csv Metadata

This example shows how to structure CSV and Excel files with metadata columns (`__metadata:category` and `__metadata:tags[]`) and use command-line filters to run evaluations on specific subsets of test cases. Use this example when you want to learn how to add metadata columns to CSV or Excel test files and filter evaluations by category or tags using the `--filter-metadata` flag.

What it does

This Promptfoo workflow example demonstrates how to use metadata columns in CSV and Excel test files to organize and filter test cases. It shows two types of metadata columns: __metadata:category for single-value categorization and __metadata:tags[] for array metadata with comma-separated tags.

When to use - and when NOT to

Use this when you need to organize test cases with metadata columns in CSV or Excel format and want to filter evaluations by category or tags. The example includes basic assertions using the __expected column to verify outputs contain relevant keywords.

Do not use this if you have a small test suite where metadata organization adds unnecessary complexity, or if your test cases don't require categorization or filtering capabilities.

Inputs and outputs

You provide CSV or Excel files with special metadata columns: __metadata:category for single-value metadata and __metadata:tags[] for multiple tags with comma separation. The files also include an __expected column for basic assertions. Array metadata values support escaped commas (e.g., "global,warming" is treated as a single tag).

You receive evaluation results with assertions verified against expected keywords.

Integrations

The example works with Promptfoo's native CSV support and optionally integrates with Excel files through the read-excel-file npm package. It uses the standard promptfooconfig.yaml configuration file for prompt and provider setup.

Who it's for

This is for users who need to organize test cases using metadata columns in CSV or Excel files and filter evaluations based on those metadata values.

Example usage

Initialize and run the basic evaluation:

npx promptfoo@latest init --example config-csv-metadata
cd config-csv-metadata
promptfoo eval

For Excel file support:

npm install read-excel-file
promptfoo eval -c promptfooconfig.yaml -t file://tests.xlsx

Filter evaluations by metadata:

promptfoo eval --filter-metadata category=technology
promptfoo eval --filter-metadata tags=ai

Source README

config-csv-metadata (CSV Metadata Example)

You can run this example with:

npx promptfoo@latest init --example config-csv-metadata
cd config-csv-metadata

Usage

This example demonstrates how to use metadata columns in CSV and Excel test files to organize and filter your test cases. You can see more details on how to build test cases from CSV in the Promptfoo docs

Files

promptfooconfig.yaml: Configuration file with a simple prompt and provider setup
tests.csv: Test cases with metadata columns for categorization and filtering
tests.xlsx: Same test cases in Excel format (optional - requires npm install read-excel-file)

Metadata Columns

The CSV/Excel file includes two types of metadata columns:

__metadata:category - Single value metadata for broad categorization
__metadata:tags[] - Array metadata for multiple tags, with comma separation and escape support

Running the Example

Basic evaluation:

promptfoo eval

To use Excel file instead of CSV:

### First install the read-excel-file package
npm install read-excel-file

### Then run with Excel file
promptfoo eval -c promptfooconfig.yaml -t file://tests.xlsx

Filter by category:

promptfoo eval --filter-metadata category=technology

Filter by tags:

promptfoo eval --filter-metadata tags=ai

Note that array metadata values (tags) support escaped commas. For example, "global,warming" is treated as a single tag.

Expected Output

The example includes basic assertions using the __expected column to verify outputs contain relevant keywords.

FAQ

Common questions

Discussion