Audit Skill Activation and Collisions
A CLI that grades every SKILL.md A-F on activation quality and simulates which skill fires for a given prompt.
Why it matters
Ensure your coding skills are discoverable and fire correctly. This tool audits skill activation, grades their effectiveness, and identifies collision clusters where one skill might be overshadowing another.
Outcomes
What it gets done
Grade skills A-F based on discoverability.
Simulate skill firing for specific prompts.
Identify and report skill collision clusters.
Optionally fix weak skill descriptions.
Install
Add it to your toolbox
Run in your project directory:
curl -fsSL https://spark.entire.vc/get/ag-skill-issue | bash Overview
skill-issue - skill activation audit
A CLI that grades coding-agent skills A-F on activation quality, simulates which skill fires for a given prompt, and flags collision clusters where skills shadow each other. Use it to diagnose a non-firing skill, verify a new SKILL.md will be picked, or gate CI against colliding skill metadata - its offline scoring is a triage signal, not a final verdict.
What it does
A coding agent decides which skill to run based on each skill's always-on name and description fields. skill-issue audits exactly that activation surface: it grades every skill in a directory A-F, simulates which skill would fire for a specific prompt (and why, showing confidence scores and margins), and reports collision clusters where one skill silently shadows a sibling. Installed via npm i -g @misha_misha/skill-issue, brew install mishanefedov/skill-issue/skill-issue, or npx @misha_misha/skill-issue, it runs offline heuristic scoring by default (with an optional --llm flag to judge activation quality using a local claude/codex CLI). Core commands: grade every skill in a directory with a collisions summary, check --why "<prompt>" to see which skill fires for a specific request and its confidence margin against runner-up skills, --collisions to list clusters of skills that shadow each other, --fix to append a "Use when..." trigger clause to weak descriptions, and --json for machine-readable output that exits non-zero on errors (suitable for CI).
When to use - and when NOT to
Use it when a skill never seems to trigger and you don't know why, when asking "why isn't my skill firing" or "which skill fires for X," right after writing or installing a new SKILL.md to confirm it will actually be picked, or in CI to fail a PR that adds a skill with empty, duplicate, or colliding metadata. Its offline heuristic scoring is a triage signal, not a final quality verdict; collision reports highlight likely shadowing but agent-specific routers may weight metadata differently; and --fix edits still need maintainer review before committing.
Inputs and outputs
Inputs: a directory of skills (e.g. ~/.claude/skills) and, for --why, a specific prompt to test against them.
Outputs: an A-F grade per skill with pass/fail reasons, a ranked list of which skill fires for a given prompt with confidence scores, a collision-cluster report, or auto-appended trigger-clause fixes.
skill-issue ~/.claude/skills
skill-issue ~/.codex/skills --why "deploy to prod"
skill-issue <dir> --collisions
Example output
Grading a skills directory looks like:
skill-issue ~/.claude/skills
# F deploy-helper no description -- can never fire
# C shipit no "use when..." trigger clause
# A rollback-prod will fire on its triggers
Diagnosing a collision for a specific prompt looks like:
skill-issue ~/.claude/skills --why "deploy the app to prod"
# 1. shipit 0.74 would fire
# 2. land-deploy 0.69 (margin 0.05 -- ambiguous, likely collision)
skill-issue itself is MIT-licensed, community-authored by mishanefedov.
Who it's for
Developers writing or maintaining coding-agent skills who need to diagnose why a skill isn't firing, resolve activation collisions between similar skills, or enforce skill-metadata quality in CI.
FAQ
Common questions
Discussion
Questions & comments · 0
Sign In Sign in to leave a comment.