Monitor and optimize AI agent token costs in CI/CD

Find the tokens your AI agent wasted - in dollars, with the fix - and fail
the PR when a change makes your agent more expensive.

Wattage reads the traces and session logs your agents already produce - your
local Claude Code sessions, or any OpenTelemetry GenAI trace export -
prices every call against a verified, dated pricing snapshot (52 models
across Anthropic, OpenAI, Google, Mistral, and xAI), runs ten waste-pattern
detectors that each name a dollar figure and a concrete fix, and ships the
one thing no dashboard gives you: a CI cost-regression gate that fails
the build when an agent quietly gets more expensive. Fully offline, no API
key, nothing phones home.

Real output of uvx wattage demo - regenerate this GIF with vhs docs/assets/demo.tape.

30 seconds to your first report

uvx wattage demo                    # findings-rich sample report, zero setup
uvx wattage report --claude-code    # your latest Claude Code session — data you already have
uvx wattage report trace.json       # any OTLP GenAI trace export

The demo trace is a deliberately wasteful synthetic agent - here's what
Wattage does to it (abridged; every number below is the command's real
output):

╭─ ⚡ wattage — demo_trace.json ─────────────────────╮
│ Token Efficiency: D (67)   Total cost: $0.0557    │
╰───────────────────────────────────────────────────╯
┃ Detector             ┃ Severity ┃  Wasted $ ┃ Fix                                        ┃
│ nonconvergence       │ critical │   $0.0037 │ Add a convergence stop after repeated      │
│                      │          │           │ non-productive iterations…                 │
│ prefix_churn         │ high     │   $0.0123 │ Enable prompt caching on the stable prefix │
│                      │          │           │ (system prompt + tool schemas)…            │
│ cache_gap            │ high     │   $0.0001 │ Move volatile fields after the cache       │
│                      │          │           │ breakpoint…                                │
│ reasoning_overspend  │ medium   │  ~$0.0060 │ Lower reasoning_effort (or disable         │
│                      │          │           │ extended thinking) for this step.          │
measured waste: $0.0187 (counts toward the grade) · estimated (~) findings: $0.0065 (reported, never graded)

Prefer a visual? --html writes a self-contained, shareable burn map -
an interactive flame graph of every token, with a stat strip and findings
that light up the exact frames that burned the money:

uvx wattage report --claude-code --html burn.html

Fail the PR when your agent gets more expensive

This is the part no other open-source tool ships: a cost-regression gate
over real measured traces (not tokenized prompt-diff predictions), with
a committed baseline that only advances on passing runs.

# .github/workflows/wattage.yml
name: Wattage
on:
  pull_request:
    paths: ["agents/**", "prompts/**", "src/**"]
permissions:
  pull-requests: write   # for the sticky PR comment (report still lands in the step summary without it)
concurrency:
  group: wattage-${{ github.ref }}
  cancel-in-progress: true
jobs:
  token-efficiency:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Generate trace fixture
        run: python scripts/run_agent_fixture.py > trace.json   # replace with whatever produces a trace for YOUR agent
      - name: Wattage cost-regression gate
        uses: faizannraza/wattage@v0.2.0
        with:
          source: trace.json
          baseline: .wattage/baseline.json
          fail-on: "score_below:80,cost_delta_pct_above:5,any_critical:true"
          pr-comment: "true"

Fails the build (exit 1) on a regression, posts one sticky PR comment
with a per-detector delta table (updated in place on every push, never
spammed), writes the report to the job's step summary, and emits SARIF and
JUnit XML for any other CI system. any_critical is a hard stop for
runaway loops: a non-convergent loop that burned half its own spend after
its last productive step escalates to critical severity. One more workflow
(runs on merge, keeps the committed baseline fresh) completes the setup -
copy-paste pair in CI Integration.

What it is - and what it isn't

Wattage is diagnosis + prescription + gate, not another dashboard. It
consumes the traces your existing tools already produce; it replaces none of
them.

Who it's for: teams shipping LLM agents who've been surprised by a bill.
You have (or can get) a trace - a Claude Code session on your laptop already
counts - you review PRs, and you want "did this change make the agent more
expensive?" answered automatically, in CI, for free.

Works with

• Claude Code / Claude Agent SDK sessions - reads the session .jsonl
  files under ~/.claude/projects directly, validated against real
  sessions. Includes the 5-minute/1-hour cache-write TTL split, so 1-hour
  cache writes price at their real 2x rate (a distinction the OTel format
  can't even express). Costs are standard API rates - for subscription
  users that's the API-equivalent value of the session, and the report
  says so.
• OpenTelemetry GenAI semconv traces - every attribute generation ever
  shipped: current names (gen_ai.provider.name, gen_ai.usage.input_tokens),
  the pre-v1.37/v1.27 names most deployed instrumentation still emits
  (gen_ai.system, gen_ai.usage.prompt_tokens), OpenLLMetry/Traceloop
  variants, and OpenInference
  llm.* attributes (the default instrumentation for OpenAI Agents SDK,
  CrewAI, and LangGraph via Arize). Single-object OTLP JSON and
  spec-standard JSON Lines (what the OTel Collector file exporter actually
  writes), camelCase or snake_case.
• mozilla-ai/any-agent -
  validated against a real captured trace (provenance),
  including LiteLLM-style "provider/model" strings.

The format is auto-detected - wattage report <file> just works. Full
matrix and honesty notes: Adapters.

The ten detectors

Every finding is priced, comes with a concrete fix, and carries two honesty
labels. A basis: measured findings (real billed tokens at the real
rate card) drive the grade and the CI gate; estimated findings (chars÷4
projections, policy ceilings, hypothetical downgrades) are reported with a
~ and can never fail a build. And a quality risk: a fix that could
plausibly change output quality (a model downgrade, less reasoning) only
counts once a --quality map backs it with real evidence. Full detail:
Detectors.

Honest numbers, structurally

• An unpriced model leaves that call's cost at zero and fails
  wattage ci loudly (exit 4) - never a guessed rate.
• A trace with zero captured usage refuses to grade instead of printing
  a vacuous A (100).
• Dropped or duplicated spans are counted and reported, never silently
  swallowed.
• The pricing snapshot is dated and source-cited
  (2026-08-23-verified, every number from the provider's own pricing
  page), context-tier aware (Gemini/Grok reprice whole requests above 200k
  prompt tokens), and effective-date aware (promo rates that expire price
  by the call's own timestamp). A published-but-rateless range (OpenAI
  above 272K context) is left unpriced, not billed at the wrong tier.

Benchmarked, reproducibly

On a real captured agent trace, Wattage's prefix_churn fix simulation
shows a 44.7% cost reduction ($0.000199 → $0.000110) from enabling
prompt caching on the stable prefix - small absolute dollars because it's a
3-turn demo trace; the mechanism is identical at production scale.

The convergence engine's classifier scores 1.00 F1 vs 0.25 for a real
SHA-256 exact-match baseline on a 10-loop hand-labeled suite. Read that
number for what it is: the suite is small, written by us, and deliberately
constructed to demonstrate the blind spots exact-match loop guards
structurally cannot see (fresh timestamps every retry, oscillating
strategies, productive-looking stalls) - it's a blind-spot demonstration
and regression suite, not a field study. Both numbers reproduce from the
shipped code with no hidden setup:

uv run python -m benchmarks.harness
uv run python -c "from benchmarks.frontier import build_frontier; print(build_frontier())"

Full methodology, including what the benchmark does not show:
The Convergence Engine.

The badge

uvx wattage badge trace.json --out wattage-badge.svg

[](https://github.com/faizannraza/wattage)

Wire --badge-out into the post-merge CI job and your README carries a
live, provable claim that your agent is efficient.