4,026 tools found
Promptfoo example enforcing schema-validated JSON output across OpenAI, Azure OpenAI and Anthropic with each provider's own syntax.
A Promptfoo example showing how the test-level prompts field limits specific tests to specific prompts, instead of the default cartesian product.
Promptfoo example restricting specific tests to specific providers via the providers filter - avoids N×M cross-product test explosion.
Promptfoo's HTTP streaming example shows why full-response eval scoring gains little from SSE/chunked streaming over a normal call.
Promptfoo example showing three ways to override default scoring - config-level defaultTest, a JavaScript export, or a Python function.
Promptfoo example scoring LLM output by semantic similarity to a reference answer using BERTScore, including multi-reference matching.
Promptfoo example testing the conversation-relevance assertion, which catches a chatbot drifting off-topic mid-conversation.
A promptfoo example demonstrating how to distinguish between test errors (provider failures) and test failures (failed assertions) using a custom provider.
Evaluates LLM sentiment classification on IMDB reviews using F-score, precision, recall, and accuracy metrics via promptfoo's assertion system.
A promptfoo workflow example that demonstrates how to use G-Eval, referencing the framework from the 2303.16634 paper.
Promptfoo example comparing GPT-4o and GPT-4o-mini on Fashion MNIST image classification via structured JSON schema output.
External JavaScript assertion example for promptfoo evaluations, demonstrating how to define test logic in separate assert.js files.
