3,719 tools found
Promptfoo example covering every supported prompt format, with focus on dynamic-config prompt functions.
Dynamically generate prompt test variables at runtime using JavaScript or Python scripts that access test context.
Demonstrates using executable scripts or binaries as dynamic prompt generators in promptfoo.
Custom setup/teardown hooks for promptfoo tests via the extensions API - beforeAll, afterAll, beforeEach, afterEach.
Loads promptfoo's defaultTest config from an external file via file:// for reuse across multiple projects.
A promptfoo example generating test cases from JavaScript/TypeScript functions, both statically and dynamically.
Promptfoo example demonstrating multi-turn conversation evaluation using the built-in _conversation variable in Nunjucks prompt templates.
Label and organize prompts to run different sets against multiple providers in unified evaluations.
Set up a custom set of promptfoo prompts targeted at specific models within a single eval.
Dynamically generate Python test cases with configurable parameters, enabling reusable test functions customized through YAML configuration.
Hook into AI evaluation lifecycle to process results, send metrics, trigger alerts, and integrate with CI/CD pipelines programmatically.
Simulate high error rates to test and validate retry functionality in AI prompt evaluation pipelines.
