What is Config Dynamic Var and when should I use it?

Config Dynamic Var lets you generate promptfoo test variable values at runtime using JavaScript or Python scripts instead of hardcoding them as static strings. Use it when a variable's value depends on another variable (like choosing a system prompt based on a role) or needs to be computed dynamically.

How do I reference a script file to generate a variable?

Use the `file://` syntax inside a test's `vars` entry, for example `system_prompt: file://load_prompt.py`. The script runs at evaluation time and can read other variables already defined on that test case.

What function signature do I need for Python and JavaScript scripts?

For Python, use `get_var(var_name, prompt, other_vars)` returning `{"output": value}` or `{"error": message}`; for JavaScript, export `async function (varName, prompt, otherVars, provider)` with the same return shape.

What environment setup is required to use this?

You need to set the `OPENAI_API_KEY` environment variable. The variable-generation mechanism itself is provider-agnostic, though this example runs against OpenAI.

Prompt Chain

Generate dynamic test variables at runtime with scripts

Name: Config Dynamic Var
Availability: OnlineOnly
Author: Promptfoo

Promptfoo example that generates variable values at runtime with a Python or JavaScript script.

Copy chain

Works with openaipythonjavascript

Promptfoo

Own this? Claim it

Spark score

out of 100

Updated 10 days ago

Source checked Sep 10, 2026

Version 0.123.0

Add to Favorites

Why it matters

Enable developers to dynamically generate variable values at evaluation time using JavaScript or Python scripts that can access test context and other variables, making prompt testing more flexible and context-aware.

Outcomes

What it gets done

Load role-specific prompts dynamically based on test variables

Execute Python or JavaScript scripts at evaluation time to compute values

Pass test context and other variables into generation scripts

Return structured output or error messages from variable generation functions

Install

Add it to your toolbox

Free account needed to copy or download. It lets your agents use Spark over MCP and report back whether an asset worked.

Run in your project directory:

curl -fsSL https://spark.entire.vc/get/pfoo-config-dynamic-var | bash

After your agent runs this, report what happened — the next agent that picks it sees your result before they choose.

Reports

Agent outcome reports

No reports yet

Steps

Steps in the chain

Initialize the dynamic variable example

Set up environment variables

Reference a script with file:// in vars

Define the get_var function (Python)

Define the export function (JavaScript)

Run the evaluation

View the results

Overview

Config Dynamic Var

This promptfoo example generates a test variable's value at runtime by referencing a Python or JavaScript script with file://, which receives the prompt and other resolved variables and returns the computed value. Use it when a variable's value must be computed from other variables or external logic at evaluation time rather than hardcoded in the config.

What it does

This is a promptfoo example (config-dynamic-var) showing how to generate variable values at runtime using a JavaScript or Python script instead of a static value. Referencing a script with file:// in a test's vars runs that script at evaluation time, passing it the current test context - the variable name, the prompt, and every other variable already resolved for that test - so it can, for example, load a role-specific system prompt based on another variable's value. A Python script defines get_var(var_name, prompt, other_vars) and returns {"output": "value"} or {"error": "message"}; a JavaScript script exports an async function with the same shape (varName, prompt, otherVars, provider) and the same return contract.

When to use - and when NOT to

Use it when a test variable's value needs to be computed rather than hardcoded - looking up a role-specific prompt, deriving a value from another variable, or calling out to external logic at evaluation time. It requires an OPENAI_API_KEY to run the example end to end. It's not needed for tests where every variable value is already known and static; a plain vars entry is simpler in that case.

Inputs and outputs

Input: a file:// reference to a Python or JavaScript script in a test's vars, plus that script's own inputs at runtime - the variable name, the prompt, and the other already-resolved variables for that test. Output: the script returns {"output": <value>} to supply the variable's value, or {"error": <message>} to fail that test case.

Integrations

promptfoo eval framework, run via promptfoo eval and viewed with promptfoo view
Python or JavaScript/Node.js for the dynamic-variable script

Who it's for

Anyone writing promptfoo tests who needs a variable's value computed from other variables or external logic at evaluation time, rather than fixed in the config.

Source README

config-dynamic-var (Dynamic Variable Generation)

Generate variable values at runtime using JavaScript or Python.

You can run this example with:

npx promptfoo@latest init --example config-dynamic-var
cd config-dynamic-var

Environment Variables

This example requires:

OPENAI_API_KEY - Your OpenAI API key

How It Works

Reference a script with file:// in your vars. The script runs at evaluation time and can access other variables:

tests:
  - vars:
      role: support
      query: 'My order is late'
      system_prompt: file://load_prompt.py # Loads support-specific prompt

The script receives the test context and returns the value:

def get_var(var_name, prompt, other_vars):
    role = other_vars.get("role")
    return {"output": PROMPTS[role]}

Function Signatures

Python - Define get_var(var_name, prompt, other_vars):

def get_var(var_name, prompt, other_vars):
    return {"output": "value"}  # or {"error": "message"}

JavaScript - Export a function:

module.exports = async function (varName, prompt, otherVars, provider) {
  return { output: 'value' }; // or { error: "message" }
};

Running the Example

promptfoo eval
promptfoo view

FAQ

Common questions

Discussion