What is the max-score assertion type and how does it select the best output?

The max-score assertion type selects the best output from multiple providers by aggregating scores from other assertions (like correctness tests and quality rubrics), applying configurable weights to each assertion type, and choosing the output with the highest weighted score. It's deterministic, objective, and transparent—the same inputs always produce the same selection based on a clear weighted-scoring methodology.

How does max-score differ from select-best?

Max-score is objective (uses quantifiable scores rather than LLM judgment), deterministic (same inputs always produce the same result), transparent (shows a clear weighted-scoring methodology), and cost-effective (no additional LLM calls). Select-best, by contrast, uses LLM judgment and is better suited for genuinely subjective or hard-to-quantify selection criteria.

What configuration options does max-score support?

Max-score takes a `method` (either `average` for weighted average or `sum` for weighted sum), a `weights` map from assertion type to weight (default 1.0 each), and an optional `threshold` for the minimum score required for selection.

What does the max-score selection process involve?

The process runs in five steps: multiple outputs are generated (one per provider), all assertions are evaluated on each output (Python tests return pass=1/fail=0, LLM rubrics score 0-1, etc.), max-score calculates each output's weighted score, the highest-scoring output is marked as passing, and results show which output won and why.

When should I use max-score instead of select-best?

Use max-score when you have objective criteria like passing tests or measurable metrics, want reproducible results, need to weight different assertion types differently, or want to avoid the extra API cost of LLM-judged selection. Use select-best when your criteria are genuinely subjective and need nuanced LLM judgment.

Prompt Chain

Evaluate Code Generation with Max Score Selection

Name: Eval Max Score Selection
Availability: OnlineOnly
Author: Promptfoo

Promptfoo example using the max-score assertion to pick the best LLM output via weighted, deterministic scoring, not LLM judgment.

Copy chain

Promptfoo

Own this? Claim it

Spark score

out of 100

Updated 10 days ago

Source checked Sep 10, 2026

Version 0.123.0

Models

gpt 4ogpt 4claude

Add to Favorites

Why it matters

Automate the evaluation of code generation prompts by selecting the highest-scoring output based on predefined criteria. This asset helps ensure the quality and correctness of generated code.

Outcomes

What it gets done

Define evaluation metrics for code generation.

Run multiple code generation prompts.

Select the best code output based on evaluation scores.

Integrate with code quality assessment tools.

Install

Add it to your toolbox

Free account needed to copy or download. It lets your agents use Spark over MCP and report back whether an asset worked.

Run in your project directory:

curl -fsSL https://spark.entire.vc/get/pfoo-eval-max-score-selection | bash

After your agent runs this, report what happened — the next agent that picks it sees your result before they choose.

Reports

Agent outcome reports

No reports yet

Steps

Steps in the chain

Generate Multiple Outputs

Evaluate All Assertions

Aggregate Scores

Select Best Output

Display Results

Overview

Eval Max Score Selection

This promptfoo example shows the max-score assertion, which selects the best of several LLM outputs using a weighted aggregate of other assertions' scores instead of subjective LLM judgment. Use max-score when selection criteria are objective and quantifiable and you want reproducible, cost-free selection; use select-best for subjective, hard-to-quantify judgment instead.

What it does

This promptfoo example demonstrates the max-score assertion type, which picks the best output among multiple providers by aggregating scores from other assertions - correctness tests, LLM-judged quality rubrics, and simple string matches - into a single weighted score, rather than relying on an LLM's subjective judgment.

When to use - and when NOT to

Use max-score when you have objective criteria to score against, such as tests or deterministic metrics, want reproducible selection with no extra LLM calls, and need to weight different aspects of correctness differently. Compared with select-best, it is objective (quantifiable scores rather than LLM judgment), deterministic (the same inputs always produce the same selection), transparent (a clear scoring methodology based on weighted assertions you can inspect), and cost-effective (no additional LLM calls are made just to pick a winner). Use select-best instead when the criteria are subjective or hard to quantify and you need nuanced LLM judgment rather than a deterministic weighted score.

Inputs and outputs

Configuration sets a method (average, the default, or sum), a weights map from assertion type to numeric weight (Python tests, JavaScript tests, LLM rubrics, simple contains checks, each defaulting to 1.0), and an optional threshold minimum score. Running npx promptfoo eval, after npx promptfoo@latest init --example eval-max-score-selection, with OpenAI/Anthropic API keys set, walks through five steps: each configured provider generates an output; every assertion runs against every output (Python tests scoring pass=1/fail=0, LLM rubrics scoring 0-1 on quality aspects, and any other configured assertions contributing their own scores); max-score aggregates those into one weighted score per output using the configured method; the output with the highest score is marked as passing; and the results view shows which output won and why. The example worked through in the README scores three outputs on Python correctness, documentation, and efficiency with weights python=3 and llm-rubric=1, producing weighted averages of 0.84, 0.94, selected, and 0.40 respectively - Output B wins despite Output C scoring higher on documentation and efficiency, because it also passes the heavily-weighted Python correctness test.

Integrations

promptfoo's assertion and provider-comparison framework; works alongside any other assertion types (python, javascript, llm-rubric, contains, and others) whose scores it aggregates, and OpenAI/Anthropic as the example's model providers.

Who it's for

Teams building automated LLM evaluation pipelines who want a deterministic, cost-free way, with no added judge-model calls, to pick the best of several candidate outputs based on quantifiable, weighted criteria.

Source README

eval-max-score-selection (Max-Score Selection)

You can run this example with:

npx promptfoo@latest init --example eval-max-score-selection
cd eval-max-score-selection

This example demonstrates the max-score assertion type for objective output selection based on aggregated scores from other assertions.

Overview

The max-score assertion provides a deterministic way to select the best output from multiple providers by:

Aggregating scores from other assertions (correctness, quality, documentation, etc.)
Applying configurable weights to different assertion types
Selecting the output with the highest weighted score
Providing objective, reproducible selection criteria

Key Differences from `select-best`

Objective: Uses quantifiable scores rather than LLM judgment
Deterministic: Same inputs always produce same selection
Transparent: Clear scoring methodology based on weighted assertions
Cost-effective: No additional LLM calls for selection

Configuration

- type: max-score
  value:
    method: average # 'average' (default) or 'sum'
    weights:
      python: 3 # Weight for Python code correctness tests
      llm-rubric: 1 # Weight for LLM-evaluated quality rubrics
      javascript: 2 # Weight for JavaScript tests
      contains: 0.5 # Weight for simple string matching
    threshold: 0.7 # Optional minimum score threshold

Options

method: How to aggregate scores
- average (default): Weighted average of assertion scores
- sum: Weighted sum of assertion scores
weights: Map of assertion types to their weights (default: 1.0)
threshold: Minimum score required for selection (optional)

Usage

Basic Example

### Run the main example (requires API keys for OpenAI/Anthropic)
npx promptfoo@latest eval

How It Works

Multiple Outputs Generated: Each provider generates a solution
Assertions Evaluated: All assertions run on each output:
- Python tests verify correctness (pass=1, fail=0)
- LLM rubrics evaluate quality aspects (0-1 score)
- Other assertions contribute their scores
Scores Aggregated: Max-score calculates weighted score for each output
Best Selected: Output with highest score is marked as passing
Results Shown: Clear indication of which output won and why

Example Scoring

Given three outputs with these assertion results:

Output A: python=1.0, documentation=0.5, efficiency=0.7
Output B: python=1.0, documentation=0.9, efficiency=0.8
Output C: python=0.0, documentation=1.0, efficiency=1.0

With weights: python=3, llm-rubric=1

Output A: (3×1.0 + 1×0.5 + 1×0.7) / 5 = 0.84
Output B: (3×1.0 + 1×0.9 + 1×0.8) / 5 = 0.94 ✓ (selected)
Output C: (3×0.0 + 1×1.0 + 1×1.0) / 5 = 0.40

When to Use max-score

Use max-score when:

You have objective criteria (tests, metrics)
You want reproducible results
You need to weight different aspects differently
You want to avoid additional API costs

Use select-best when:

You need subjective judgment
The criteria are hard to quantify
You want nuanced evaluation of quality

FAQ

Common questions

Discussion

Evaluate Code Generation with Max Score Selection

What it gets done

Add it to your toolbox

Agent outcome reports

Steps in the chain

Eval Max Score Selection

What it does

When to use - and when NOT to

Inputs and outputs

Integrations

Who it's for

eval-max-score-selection (Max-Score Selection)

Overview

Key Differences from select-best

Configuration

Options

Usage

Basic Example

How It Works

Example Scoring

When to Use max-score

Common questions

Questions & comments · 0

Key Differences from `select-best`