What is Config Dynamic Var and when should I use it?

Config Dynamic Var lets you generate promptfoo test variable values at runtime using JavaScript or Python scripts instead of hardcoding them as static strings. Use it when a variable's value depends on another variable (like choosing a system prompt based on a role) or needs to be computed dynamically.

How do I reference a script file to generate a variable?

Use the `file://` syntax inside a test's `vars` entry, for example `system_prompt: file://load_prompt.py`. The script runs at evaluation time and can read other variables already defined on that test case.

What function signature do I need for Python and JavaScript scripts?

For Python, use `get_var(var_name, prompt, other_vars)` returning `{"output": value}` or `{"error": message}`; for JavaScript, export `async function (varName, prompt, otherVars, provider)` with the same return shape.

What environment setup is required to use this?

You need to set the `OPENAI_API_KEY` environment variable. The variable-generation mechanism itself is provider-agnostic, though this example runs against OpenAI.

Prompt Chain

Generate dynamic test variables at runtime with scripts

Name: Config Dynamic Var
Availability: OnlineOnly
Author: Promptfoo

Dynamically generate prompt test variables at runtime using JavaScript or Python scripts that access test context.

Copy chain

Works with openaipythonjavascript

Promptfoo

Own this? Claim it

Spark score

out of 100

Updated 18 days ago

Version 0.121.18

Add to Favorites

Why it matters

Enable developers to dynamically generate variable values at evaluation time using JavaScript or Python scripts that can access test context and other variables, making prompt testing more flexible and context-aware.

Outcomes

What it gets done

Load role-specific prompts dynamically based on test variables

Execute Python or JavaScript scripts at evaluation time to compute values

Pass test context and other variables into generation scripts

Return structured output or error messages from variable generation functions

Install

Add it to your toolbox

Run in your project directory:

curl -fsSL https://spark.entire.vc/get/pfoo-config-dynamic-var | bash

Steps

Steps in the chain

Initialize the dynamic variable example

Set up environment variables

Reference dynamic variable scripts

Define Python variable function

Define JavaScript variable function

Run evaluation

View results

Overview

Config Dynamic Var

Runtime variable generation using external scripts When variable values must be computed from other test variables rather than hardcoded

What it does

Config Dynamic Var generates variable values at runtime using JavaScript or Python. Reference a script with file:// in your vars, and the script runs at evaluation time and can access other variables.

When to use - and when NOT to

Use this when you need to generate values that depend on other test inputs, such as loading different prompt templates based on test variables. The source example shows loading role-specific prompts where the role variable determines which prompt to return.

Do not use this for simple static variables that don't change between test runs - plain YAML variable definitions are simpler and faster. Avoid it when the script logic becomes complex enough to warrant a full test harness instead of embedded generation.

Inputs and outputs

Reference a script with file:// in your test configuration. The script receives the test context and returns the value.

tests:
  - vars:
      role: support
      query: 'My order is late'
      system_prompt: file://load_prompt.py

def get_var(var_name, prompt, other_vars):
    role = other_vars.get("role")
    return {"output": PROMPTS[role]}

Scripts return either {"output": "value"} for success or {"error": "message"} for failures.

Integrations

This example requires OPENAI_API_KEY environment variable.

Who it's for

Developers building test suites where prompt content varies based on test parameters.

Python developers define get_var(var_name, prompt, other_vars) returning a dictionary. JavaScript developers export an async function with signature async function (varName, prompt, otherVars, provider). Both return {output: 'value'} or {error: 'message'}.

promptfoo eval
promptfoo view

Source README

config-dynamic-var (Dynamic Variable Generation)

Generate variable values at runtime using JavaScript or Python.

You can run this example with:

npx promptfoo@latest init --example config-dynamic-var
cd config-dynamic-var

Environment Variables

This example requires:

OPENAI_API_KEY - Your OpenAI API key

How It Works

Reference a script with file:// in your vars. The script runs at evaluation time and can access other variables:

tests:
  - vars:
      role: support
      query: 'My order is late'
      system_prompt: file://load_prompt.py # Loads support-specific prompt

The script receives the test context and returns the value:

def get_var(var_name, prompt, other_vars):
    role = other_vars.get("role")
    return {"output": PROMPTS[role]}

Function Signatures

Python - Define get_var(var_name, prompt, other_vars):

def get_var(var_name, prompt, other_vars):
    return {"output": "value"}  # or {"error": "message"}

JavaScript - Export a function:

module.exports = async function (varName, prompt, otherVars, provider) {
  return { output: 'value' }; // or { error: "message" }
};

Running the Example

promptfoo eval
promptfoo view

FAQ

Common questions

Discussion