Audit AI Coding Sessions for Cost, Latency & Failures

agenttrace Session Audit uses the agenttrace tool to inspect local AI coding-agent sessions. It focuses on the process behind a run: token and cost spikes, tool failures, retry loops, latency gaps, anomalies, health scores and session-to-session diffs. agenttrace is local-first and reads session logs from tools such as Claude Code, Codex CLI, Gemini CLI, Aider, Cursor exports, OpenCode, Qwen Code and Kimi, as well as generic JSON or JSONL traces. The skill is MIT licensed and marked safe.

Use it when a user asks why an AI coding run was slow, expensive, shallow or unreliable, when reviewing local agent logs before retrying a failed or suspicious task, when building a lightweight CI health gate for AI-assisted coding sessions, or when comparing two attempts for changed tool paths, retries or cost patterns.

Know its limits. agenttrace can only analyze logs that are present locally or provided as exports, and some agents do not expose enough fields to infer cost, model, cache use or latency. Healthy trace metrics do not prove the final code is correct, so you still need to run tests and review diffs. For production LLM application tracing the skill points to a separate langfuse skill, and for broader service monitoring, SLOs and incident workflows it points to an observability-engineer skill.

The workflow has five steps.

Step 1: discover available sessions. Prefer an installed agenttrace binary on the PATH, or run it with go run from the cmd folder if the current repository is the agenttrace repo itself. If no sessions are detected, report the directories the doctor command checked and ask for an exported session file or log directory.

agenttrace --doctor
agenttrace --overview

Step 2: produce a human-readable audit. Use Markdown output when the user wants a concise report to inspect or share. The report should lead with the highest-risk sessions and explain why they matter: critical anomalies, repeated tool failures, token or cost waste, long latency gaps, low health scores and suspiciously shallow sessions.

Step 3: inspect one session or directory. Use the latest session for a quick check, pass an explicit export path, or point the overview at a session directory, with JSON output available.

Step 4: compare attempts when semantics matter. Token and latency metrics can look healthy even when an agent confidently takes the wrong implementation path, so when the risk is semantic drift, pair the trace audit with a diff against a previous or known-good attempt. Look for changed files or commands that diverge from the task, missing tests or verification steps compared with the reference, repeated edits around the same files without a clear reason, and lower cost that came from skipping necessary exploration.

Step 5: add automation gates. For CI, use JSON output or health thresholds: a minimum health score, failing on critical anomalies and a maximum tool failure rate. The example gate uses a health floor of 80, fail-on-critical and a 15 percent tool failure ceiling. Tune thresholds to the project, and keep a gate advisory or reporting-only while the team learns its baseline.

Best practices: start with the doctor command when session discovery is uncertain, report missing fields plainly and never invent cost, model, latency or health data, treat prompts, code and session contents as private local data, prefer JSON for automation and Markdown for human review, and use trace metrics for process failures and diff review for semantic drift. Safety notes: do not upload private session logs externally without explicit approval, do not overwrite user reports unless they asked for that exact output path, and avoid printing secrets found in prompts, tool output, environment variables or logs.

Common pitfalls: if no sessions are found, run the doctor and point at the export; if a cheap, fast run produced the wrong refactor, compare against a prior attempt; and if CI fails too often after adding a health gate, start with reporting and tighten gradually.

It works with agenttrace and the local session logs of the listed coding agents, and can feed CI pipelines through JSON output and threshold flags.

Developers and teams who use AI coding agents and want visibility into cost, reliability and behavior of their sessions, including a simple CI guard against unhealthy runs.