Skill

Systematically Fix Failing Tests

Systematically identifies and fixes failing tests via smart error grouping - infrastructure first, then API changes, then logic bugs.

Works with github

87
Spark score
out of 100
Updated 8 days ago
Version 15.3.0

Add to Favorites

Why it matters

Automate the process of identifying and resolving all failing tests within your codebase, ensuring a stable and reliable test suite.

Outcomes

What it gets done

01

Run initial test suite to identify all failures.

02

Group similar test failures by error type, module, or root cause.

03

Implement systematic fixes for grouped errors, prioritizing infrastructure and API changes.

04

Verify fixes by running subsets of tests and the complete suite.

Install

Add it to your toolbox

Run in your project directory:

curl -fsSL https://spark.entire.vc/get/ag-test-fixing | bash

Overview

Test Fixing

Groups failing tests by error type, module, and root cause, then fixes and verifies each group in priority order - infrastructure, then API changes, then logic bugs. Use when tests are failing, after finishing an implementation, or when CI/CD is broken due to test failures.

What it does

This skill systematically identifies and fixes all failing tests using smart grouping strategies. It starts with an initial test run (make test) to identify all failures, analyzing the output for the total failure count, error types and patterns, and affected modules/files. It then groups similar failures by error type (ImportError, AttributeError, AssertionError, etc.), by module/file (same file causing multiple failures), and by root cause (missing dependencies, API changes, refactoring impacts), prioritizing groups by number of affected tests and by dependency order - fixing infrastructure before functionality.

For each group, starting with the highest-impact one, it follows a four-step fixing process: identify the root cause by reading the relevant code and checking recent changes with git diff; implement a fix using minimal, focused changes that follow the project's own conventions (per CLAUDE.md); verify the fix by running a focused subset of tests via pytest markers or file patterns; then move to the next group only once the current group passes. Its fix order strategy tackles infrastructure issues first (import errors, missing dependencies, configuration issues), then API changes (function signature changes, module reorganization, renamed variables/functions), and finally logic issues (assertion failures, business logic bugs, edge case handling). After all groups are fixed, it runs the complete test suite again to verify no regressions and confirm test coverage remains intact.

Best practices reinforced throughout: fix one group at a time, run focused tests after each fix, use git diff to understand recent changes, look for patterns in failures, don't advance to the next group until the current one passes, and keep changes minimal and focused. The skill's own example workflow illustrates this end to end: a user reports failing tests after a refactor, make test surfaces 15 failures, which get grouped into 8 ImportErrors (a renamed module), 5 AttributeErrors (a changed function signature), and 2 AssertionErrors (logic bugs); each group is fixed and verified with a subset run in turn, then the full suite is run to confirm all pass.

When to use - and when NOT to

Use when explicitly asked to fix tests ("fix these tests", "make tests pass"), when test failures are reported ("tests are failing", "test suite is broken"), after completing an implementation that needs its tests passing, or when CI/CD is failing due to tests. It is not a substitute for environment-specific validation or expert review, and should not be invoked outside an actual test-fixing scenario.

Inputs and outputs

Input is a codebase with a failing test suite (via make test) and access to git diff for recent-change context. Output is a passing test suite, reached by fixing each error group with minimal, convention-following code changes and confirming each group individually before a final full-suite verification run.

uv run pytest tests/path/to/test_file.py -v
uv run pytest -k "pattern" -v

Integrations

make test as the test runner entry point, git diff for change inspection, uv run pytest with markers or file-pattern filters for focused verification, and the project's own CLAUDE.md for convention guidance.

Who it's for

Developers who need to systematically clear a batch of failing tests - especially after a refactor, dependency update, or API change - without randomly patching individual failures out of order.

FAQ

Common questions

Discussion

Questions & comments · 0

Sign In Sign in to leave a comment.