Configure Custom Prompt Functions
Promptfoo example covering every supported prompt format, with focus on dynamic-config prompt functions.
Why it matters
Integrate custom prompt functions into your AI development process. This asset helps you define and manage complex prompting logic for better AI output.
Outcomes
What it gets done
Define and register custom prompt functions.
Integrate custom logic into prompt chains.
Test and debug custom prompt implementations.
Install
Add it to your toolbox
Run in your project directory:
curl -fsSL https://spark.entire.vc/get/pfoo-config-custom-prompt-function | bash Steps
Steps in the chain
Overview
Config Custom Prompt Function
A promptfoo example demonstrating every supported prompt format and function type, with focus on prompt functions that return dynamic configuration alongside content. Use when choosing a prompt format or building dynamic-config prompt functions that adjust provider settings per request.
What it does
This promptfoo example demonstrates the full range of prompt formats promptfoo supports, with special focus on prompt functions that can return both content and configuration. Run it with:
npx promptfoo@latest init --example config-custom-prompt-function
cd config-custom-prompt-function
After setting OPENAI_API_KEY and ANTHROPIC_API_KEY, evaluation runs via npx promptfoo@latest eval, with results viewable via npx promptfoo@latest view.
When to use - and when NOT to
Use it to explore which prompt format fits your use case: raw text prompts, file-based prompts (txt, yaml, json, jsonl, md, j2), glob patterns for multiple files, JSON chat formats, Markdown-formatted prompts, and Jinja2 template files. On the function side it covers JavaScript prompt functions (CommonJS and ESM), TypeScript prompt functions, Python prompt functions and class methods, multiple functions defined in a single file, and dynamic configuration - functions returning a { prompt, config } object that overrides provider settings.
Inputs and outputs
The example structure maps each format to a file: prompt.txt/prompt.yaml for various file formats, prompt.j2 for the Jinja2 template, prompt_chat.js/prompt_chat.ts for chat format, prompt_multiple.js for multiple functions in one file, prompt_esm.mjs for ESM modules, prompt_python.py for Python examples, prompt_config.js/prompt_config.py for functions returning dynamic configuration, a subfolder/ demonstrating nested file structures, and a promptfooconfig.yaml tying all the prompt types together.
Integrations
Configuration features shown include dynamic adjustment of model parameters (temperature, tokens) based on content, content-aware JSON schema formatting, configuration merging where function-returned values override provider defaults, provider-specific prompt configurations, and side-by-side comparison across OpenAI and Anthropic providers.
To adapt the example: explore the different prompt formats to find what fits your use case, use dynamic configurations to optimize model behavior based on input context, compare how different providers respond to the same prompts, and create custom prompt functions for more advanced use cases.
Who it's for
Teams evaluating which promptfoo prompt format or function pattern fits their workflow, who want dynamic per-request configuration (adjusting temperature or token limits based on input content) and multi-provider comparison already wired up rather than assembled from documentation alone.
Source README
config-custom-prompt-function (Prompt Functions and Format Examples)
You can run this example with:
npx promptfoo@latest init --example config-custom-prompt-function
cd config-custom-prompt-function
This example demonstrates the full range of prompt formats supported by promptfoo, with special focus on prompt functions that can return both content and configuration.
Quick Start
### Copy this example to your project
npx promptfoo@latest init --example config-custom-prompt-function
### Set required API keys
export OPENAI_API_KEY=your_openai_key
export ANTHROPIC_API_KEY=your_anthropic_key
### Run the evaluation
npx promptfoo@latest eval
### View results
npx promptfoo@latest view
What This Example Shows
Supported Prompt Formats
- Raw text prompts
- File-based prompts (txt, yaml, json, jsonl, md, j2)
- Glob patterns for multiple files
- JSON chat formats
- Markdown formatted prompts
- Jinja2 template files
Prompt Functions
- JavaScript prompt functions (CommonJS and ESM)
- TypeScript prompt functions
- Python prompt functions and class methods
- Multiple functions in a single file
- Dynamic configuration - returning
{ prompt, config }objects that override provider settings
Example Structure
prompt.txt,prompt.yaml, etc. - Various prompt file formatsprompt.j2- Jinja2 template formatprompt_chat.js/ts- Chat format examplesprompt_multiple.js- Multiple functions in one fileprompt_esm.mjs- ESM module formatprompt_python.py- Python examplesprompt_config.js/py- Functions returning dynamic configurationsubfolder/- Demonstrates nested file structurespromptfooconfig.yaml- Complete configuration showing all prompt types
Configuration Features
- Dynamic adjustment of model parameters (temperature, tokens) based on content
- Content-aware JSON schema formatting
- Configuration merging (function values override provider defaults)
- Provider-specific prompt configurations
- Multiple provider comparisons (OpenAI and Anthropic)
Customization
To adapt this example:
- Explore different prompt formats to find what works best for your use case
- Use dynamic configurations to optimize model behavior based on input context
- Compare how different providers respond to the same prompts
- Create your own prompt functions for advanced use cases
Documentation
Learn more about:
FAQ
Common questions
Discussion
Questions & comments · 0
Sign In Sign in to leave a comment.