Optimize Skill Performance and Quality
A read-only skill that scores installed skills across 8 research-grounded dimensions using session data plus static checks.
Why it matters
Automate the auditing and optimization of your AI agent's skills. This asset analyzes historical session data and static skill file quality to identify and prioritize areas for improvement, ensuring your skills trigger correctly and perform optimally.
Outcomes
What it gets done
Scan all skills for performance issues
Audit skill descriptions for clarity and effectiveness
Analyze skill trigger rates and user satisfaction
Generate prioritized reports with actionable suggestions
Install
Add it to your toolbox
Run in your project directory:
curl -fsSL https://spark.entire.vc/get/ag-skill-optimizer | bash Overview
Skill Optimization Report
A read-only skill that audits a skill library using session data and static checks across 8 dimensions - trigger rate, user reaction, workflow completion, undertrigger risk, conflicts, and token economics. Use it when skills aren't triggering as expected or to audit overall skill library quality; it only reports findings, never modifies skill files directly.
What it does
This skill analyzes an agent's skill library using historical session transcripts plus static quality checks, outputting a diagnostic report with P0/P1/P2 prioritized fixes and a 5-point composite score per skill. It calls this discipline CSO (Claude/Agent Search Optimization) - writing skill descriptions so agents select the right skill at the right time. Invoked via /optimize-skill (all skills), /optimize-skill my-skill (one skill), or with multiple names. It auto-detects the agent platform (Claude Code, Codex, or both) and scans the corresponding session transcript and skill-file locations. The workflow identifies target skills and extracts their name/description/trigger keywords/workflow steps/word count, then collects session data (Skill tool_use calls and surrounding messages for Claude Code; response_item/event_msg/turn_context events for Codex, where skill loading in context does NOT equal active invocation - actual use requires evidence the agent followed the skill's defined workflow). It runs all 8 analysis dimensions without skipping any: (4.1) Trigger Rate - actual invocations vs keyword-match opportunities; (4.2) Post-Invocation User Reaction - classifying the user's next messages as negative/correction/positive/silent-switch; (4.3) Workflow Completion Rate - how far execution got through the skill's defined steps; (4.4) Static Quality Analysis against 14 rules (frontmatter format, trigger-first description wording, no workflow-summary leak, MUST/NEVER density, word count, YAML quoting safety, critical info in the first 20% of the file, trigger keywords within the first 250 characters, at most 2 trigger conditions); (4.5a) False Positive/Overtrigger rate; (4.5b) Undertrigger Detection - the highest-value dimension, matching user tasks against the skill's capabilities to find missed opportunities, flagging chronic undertriggering (0 triggers across 5+ relevant sessions) as a compounding risk since undertriggered skills cannot self-improve via usage feedback; (4.6) Cross-Skill Conflicts - keyword/workflow overlap and contradictory guidance between skill pairs; (4.7) Environment Consistency - checking referenced file paths, CLI tools, and directories actually exist; and (4.8) Token Economics - cost-effectiveness as trigger count divided by word count, plus a 3-tier progressive-disclosure check (frontmatter ~100 tokens, SKILL.md body under 500 lines, reference files loaded on demand) flagging skills that cram 500+ words into SKILL.md without using reference files. It computes a weighted composite score (trigger rate 25%, user reaction 20%, workflow completion 15%, static quality 15%, undertrigger 15%, token economics 10%) plus qualitative-only reporting for overtrigger, conflicts, and environment checks. It is strictly read-only - it never modifies skill files, only outputs the report - and grounds its methodology in cited research (Memento-Skills on undertrigger/self-improvement loops, MCP Description Quality on 3.6x tool-selection improvement from good descriptions, Lost in the Middle on positional attention, format-variance and instruction-compliance studies).
When to use - and when NOT to
Use it when skills aren't triggering as expected or seem broken, when auditing and improving overall skill library quality, or when trying to understand which skills are underperforming or wasting context tokens. It only reports findings and specific wording suggestions - it never modifies skill files directly, so fixes must still be applied and reviewed by the user.
Inputs and outputs
Inputs: the target skill name(s) (or none, to scan all) and available session transcript history for the detected platform(s).
Outputs: a Skill Optimization Report with an overview table (triggers, reaction, completion, static grade, undertrigger misses, token count, composite score per skill), P0/P1/P2 prioritized fixes, and full per-skill diagnostics across all 8 dimensions.
Who it's for
Developers maintaining a library of Claude Code or Codex skills who need an evidence-based, quantified audit of which skills are firing correctly, satisfying users, completing their workflows, and worth their token cost - rather than guessing from anecdote.
FAQ
Common questions
Discussion
Questions & comments · 0
Sign In Sign in to leave a comment.