Verify Markdown Rendering Accuracy
A promptfoo example for evaluating how a model's markdown-formatted output renders.
Why it matters
Ensure your markdown renders correctly across different environments by automatically evaluating its output against expected results.
Outcomes
What it gets done
Generate markdown content.
Render markdown in a controlled environment.
Compare rendered output to expected results.
Identify and report rendering discrepancies.
Install
Add it to your toolbox
Run in your project directory:
curl -fsSL https://spark.entire.vc/get/pfoo-eval-markdown-rendering | bash Overview
Eval Markdown Rendering
A promptfoo starting-point example for evals that check markdown-formatted model output. Use when setting up a promptfoo eval that needs to check markdown-formatted model output.
What it does
This example provides a promptfoo configuration for evaluating markdown-formatted model output, editable via promptfooconfig.yaml.
When to use - and when NOT to
Use this as a starting point when your eval needs to check markdown-formatted model output.
Not needed for evals scoring plain unformatted text with no markdown structure to check.
Inputs and outputs
Input is promptfooconfig.yaml. Output is a normal promptfoo eval / promptfoo view run.
Integrations
No specific external provider stated beyond promptfoo's standard eval flow.
Who it's for
promptfoo users setting up an eval that needs to check markdown-formatted model output.
npx promptfoo@latest init --example eval-markdown-rendering
cd eval-markdown-rendering
Source README
eval-markdown-rendering (Markdown Rendering)
You can run this example with:
npx promptfoo@latest init --example eval-markdown-rendering
cd eval-markdown-rendering
Usage
To get started, edit promptfooconfig.yaml.
Then run:
promptfoo eval
Afterwards, you can view the results by running promptfoo view
Discussion
Questions & comments · 0
Sign In Sign in to leave a comment.