Triage failed Agent QA runs with evidence-backed diagnosis

Agent QA Result Triage classifies a failed Agent QA run from its recorded evidence instead of guessing. It inspects the run, steps, artifacts and logs, chooses exactly one fixed category, and returns confidence, likely ownership and the next evidence-backed action.

The eight categories are timeout (the run or step exceeded its timeout - checked via the failure summary, step duration and logs); appium_startup (Appium failed to start or acquire a mobile session - checked via the failure summary, execution logs and runtime errors); browser_disconnect (the browser, page or context closed unexpectedly - checked via error logs for "browser closed" or "target closed"); element_not_found (a locator, element, selector or UI description was unavailable - checked via the failed step's error, observation, screenshot and DOM/accessibility context); assertion_failure (the app was reachable but expected content or state did not match - checked via the failed assert/verify step, observation and screenshot); hook_failure (setup, teardown or hook execution blocked the run - checked via hook logs, hook artifact sections and registry errors); infrastructure (a network, Docker, device farm, filesystem or service dependency failed - checked via execution logs, stderr and runtime errors); and unknown_failure when the evidence is insufficient for a stronger category, in which case it names the missing sections and the next evidence to collect.

Use it when investigating a failed or interrupted Agent QA run, inspecting run artifacts, step results or execution logs, comparing recent related runs for recurring failure patterns, or deciding whether a failure belongs to a test, product, hook, browser/mobile runtime or infrastructure owner.

It does not modify tests or application code; an authorized repair is the separate agent-qa-debug-fix skill's job. Classification is only as reliable as the retained run artifacts and logs. A category identifies the most likely failure surface, not a proven root cause, and missing screenshots, DOM/accessibility context, device logs or prior runs must lower the stated confidence.

The input is a run in Agent QA. The workflow starts with agent_qa_get_run for run status, suite child context, steps and attempts, then fetches evidence before deciding anything: agent_qa_get_run_artifact, agent_qa_get_run_steps, agent_qa_get_run_logs and agent_qa_get_run_execution_logs. It then calls agent_qa_classify_failure and uses that category as the default classification unless stronger evidence contradicts it, and compares recent related runs when they appear in the classifier output.

The output is a concise triage result: category, confidence, evidence, likely fix area and next action.

{
  "category": "element_not_found",
  "confidence": "high",
  "evidence": ["Step 4 could not resolve the described checkout button"],
  "likely_fix_area": "test definition or changed product UI",
  "next_action": "Inspect the captured UI context, then compare the current checkout screen"
}

The evidence rules are strict: quote or summarize concrete artifact, log or step evidence; mention missing artifact sections when they limit confidence; never invent screenshots, videos, logs or memory context that the MCP tools did not actually return; and redact credentials, session tokens, personal data and unrelated application content from the report. When MCP is unavailable, it falls back to the dashboard REST APIs or Agent QA CLI output and states which evidence was unavailable as a result. The skill's license is FSL-1.1-ALv2.