3,719 tools found
Compare LangChain LCEL chains against LLM providers using Promptfoo to validate prompt pipelines and output parsers before production.
Promptfoo example for Langfuse prompt management with labels: deploy, A/B test, and roll back prompts without code changes.
Promptfoo example showing built-in OpenTelemetry tracing for LLM provider calls, exportable to Jaeger or Honeycomb.
Promptfoo example instrumenting a custom LLM provider with OpenTelemetry and verifying traces via trace-based assertions.
Promptfoo example instrumenting a Python provider with OpenTelemetry to trace LLM calls via protobuf-format OTLP export.
Promptfoo example evaluating PydanticAI agents with structured-output validation and multiple assertion types.
Import AI evaluation test cases from SharePoint CSV files using Azure AD certificate authentication for enterprise test management.
Promptfoo example collecting human feedback via Slack to compare against AI model responses in an eval.
Promptfoo example evaluating a Strands Agents SDK weather agent with tool-usage and rubric-based assertions.
Promptfoo example testing a conversational airline-booking agent against 31 simulated customer personas via function calling.
Promptfoo example comparing OpenAI, Anthropic, and Google responses accessed through a single Vercel AI Gateway API.
Reports dynamically-constructed prompts from the Vercel AI SDK to promptfoo, so the real sent prompt is debuggable.
