Orchestrate Multi-Turn Conversations

This is a promptfoo example demonstrating multi-turn conversation evaluation, showing how to use the built-in _conversation variable inside a Nunjucks prompt template to reference previous turns. The variable is typed as an array of completions:

type Completion = {
  prompt: string | object;
  input: string;
  output: string;
};

type Conversation = Completion[];

When looping through _conversation, completion.prompt gives access to the full prior prompt (e.g. completion.prompt[completion.prompt.length - 1].content is the last user message in a chat-formatted prompt), completion.input gives the last part of the prompt (equal to that same last-message content in chat format, or the raw last prompt element otherwise), and completion.output gives the assistant's response to that turn.

npx promptfoo@latest init --example config-multi-turn
cd config-multi-turn

Use it as a reference for building promptfoo evaluations that need to reference conversation history - grading a response based on what was said in earlier turns, or constructing follow-up prompts that reference prior outputs.

It is a starter example, not a production eval - you're expected to edit prompt.json and promptfooconfig.yaml to fit your own multi-turn scenario before running it for real.

Input is the OPENAI_API_KEY environment variable plus the example's prompt.json and promptfooconfig.yaml configuration, edited to define your multi-turn prompt logic. Output is produced by running promptfoo eval, with results viewable via promptfoo view.

Developers building promptfoo evaluations for multi-turn conversational prompts who need a working reference for accessing prior turns' prompts, inputs, and outputs inside a Nunjucks template.