4,052 tools found
Promptfoo example workflow for evaluating JSON output from language models using configuration files and command-line tools.
Promptfoo workflow example named eval-markdown-rendering that you can initialize and run using promptfoo commands.
Promptfoo example using the max-score assertion to pick the best LLM output via weighted, deterministic scoring, not LLM judgment.
Promptfoo example on custom metric naming - dynamic per-case names from template variables, static names, and derived formula metrics.
Promptfoo RAG example with intentionally mixed pass/fail context-recall assertions, showing both successful and failing RAG metrics.
End-to-end RAG evaluation example - ingests SEC filing PDFs into a vector DB via LangChain, then evals retrieval and answer quality.
Promptfoo example using the search-rubric assertion to verify time-sensitive claims (prices, news, weather) via real-time web search.
Promptfoo workflow example named 'eval-select-best' that demonstrates evaluation capabilities using the promptfoo framework.
Promptfoo workflow that enables LLMs to evaluate and grade their own outputs against predefined expectations, with YAML or CSV test configuration.
A promptfoo workflow that validates SQL query generation by testing language models against SQL syntax rules using node-sql-parser integration.
Promptfoo example evaluating function/tool calling across OpenAI, Anthropic, AWS Bedrock, and Groq, with strict-mode and finish-reason assertions.
Promptfoo example workflow for evaluating hate speech detection models using Hugging Face API with configurable test cases and result visualization.
