Debug and repair failed Agent QA test runs from evidence

This skill repairs a failed Agent QA run using recorded evidence and the relevant local source code. It treats the failure classifier as a hypothesis rather than a verdict, makes the smallest justified change, and verifies only the narrowest affected behavior rather than rewriting a whole test or definition.

The workflow has eight steps. First, collect evidence with the run, run steps, run artifact, run logs, and run execution logs calls. Second, call the failure classifier and treat its category as a hypothesis, not a verdict. Third, identify the failing surface: a test definition, a hook, the application under test, runtime infrastructure, or agent behavior. Fourth, inspect the relevant local files directly rather than inferring a patch from artifacts alone. Fifth, explain the link from evidence to change, then apply the smallest code or YAML change that accounts for the evidence. Sixth, validate any changed Agent QA definition before executing it. Seventh, re-run the narrowest affected test, suite, hook, or unit test within the approved environment. Eighth, report the root cause, the changed files, the verification command or MCP action used, the result, and any remaining risk.

Before any of this, the skill confirms preconditions: the repository, workspace, target environment, and files the user authorizes for modification; it inspects the planned test's external side effects before rerunning it and gets explicit confirmation for anything production-facing, destructive, or irreversible; it preserves unrelated user changes and keeps the patch limited to the evidenced failure; and it does not expose credentials or sensitive application data found in artifacts and logs.

Use it when a failed Agent QA run has already been triaged and now needs a code or YAML repair, when artifacts and logs point to a test, hook, product, runtime, or agent-behavior defect, when a proposed fix must be verified with the narrowest possible Agent QA or unit-test rerun, or when the user asks to self-heal or update a stale Agent QA definition from evidence.

Fix rules set firm limits: never invent selectors, screen states, screenshots, logs, or source files; never rewrite a test merely to make it pass when the artifact shows a real product or runtime defect; preserve canonical Agent QA IDs when editing tests, suites, hooks, or memory files; and stop and report the blocker when the evidence cannot distinguish between materially different fixes.

The input is a failed run's recorded evidence and the local repository. The output is a minimal patch, a validated definition if one was changed, and a rerun result. When the Agent QA MCP tools are unavailable, the skill falls back to dashboard REST APIs or local .agent-qa artifacts and states plainly that MCP evidence was unavailable, rather than silently proceeding without it. Before rerunning edited YAML, it prefers the dedicated validate-test, validate-suite, and validate-definition calls.

The skill is built around the Agent QA MCP tool set: get_run, get_run_steps, get_run_artifact, get_run_logs, get_run_execution_logs, classify_failure, and the validate_test, validate_suite, and validate_definition calls, with dashboard REST APIs and local artifact files as a fallback path.

It is for engineers and QA owners who use Agent QA and want a disciplined way to turn a failed run into a minimal, evidenced fix, verified by the smallest possible rerun, without papering over a real product or infrastructure defect.