Skill

Optimize Skill Performance and Quality

A read-only skill that scores installed skills across 8 research-grounded dimensions using session data plus static checks.


90
Spark score
out of 100
Updated last month
Version 13.1.0

Add to Favorites

Why it matters

Automate the auditing and optimization of your AI agent's skills. This asset analyzes historical session data and static skill file quality to identify and prioritize areas for improvement, ensuring your skills trigger correctly and perform optimally.

Outcomes

What it gets done

01

Scan all skills for performance issues

02

Audit skill descriptions for clarity and effectiveness

03

Analyze skill trigger rates and user satisfaction

04

Generate prioritized reports with actionable suggestions

Install

Add it to your toolbox

Run in your project directory:

curl -fsSL https://spark.entire.vc/get/ag-skill-optimizer | bash

Overview

Skill Optimization Report

A read-only skill that audits a skill library using session data and static checks across 8 dimensions - trigger rate, user reaction, workflow completion, undertrigger risk, conflicts, and token economics. Use it when skills aren't triggering as expected or to audit overall skill library quality; it only reports findings, never modifies skill files directly.

What it does

This skill analyzes an agent's skill library using historical session transcripts plus static quality checks, outputting a diagnostic report with P0/P1/P2 prioritized fixes and a 5-point composite score per skill. It calls this discipline CSO (Claude/Agent Search Optimization) - writing skill descriptions so agents select the right skill at the right time. Invoked via /optimize-skill (all skills), /optimize-skill my-skill (one skill), or with multiple names. It auto-detects the agent platform (Claude Code, Codex, or both) and scans the corresponding session transcript and skill-file locations. The workflow identifies target skills and extracts their name/description/trigger keywords/workflow steps/word count, then collects session data (Skill tool_use calls and surrounding messages for Claude Code; response_item/event_msg/turn_context events for Codex, where skill loading in context does NOT equal active invocation - actual use requires evidence the agent followed the skill's defined workflow). It runs all 8 analysis dimensions without skipping any: (4.1) Trigger Rate - actual invocations vs keyword-match opportunities; (4.2) Post-Invocation User Reaction - classifying the user's next messages as negative/correction/positive/silent-switch; (4.3) Workflow Completion Rate - how far execution got through the skill's defined steps; (4.4) Static Quality Analysis against 14 rules (frontmatter format, trigger-first description wording, no workflow-summary leak, MUST/NEVER density, word count, YAML quoting safety, critical info in the first 20% of the file, trigger keywords within the first 250 characters, at most 2 trigger conditions); (4.5a) False Positive/Overtrigger rate; (4.5b) Undertrigger Detection - the highest-value dimension, matching user tasks against the skill's capabilities to find missed opportunities, flagging chronic undertriggering (0 triggers across 5+ relevant sessions) as a compounding risk since undertriggered skills cannot self-improve via usage feedback; (4.6) Cross-Skill Conflicts - keyword/workflow overlap and contradictory guidance between skill pairs; (4.7) Environment Consistency - checking referenced file paths, CLI tools, and directories actually exist; and (4.8) Token Economics - cost-effectiveness as trigger count divided by word count, plus a 3-tier progressive-disclosure check (frontmatter ~100 tokens, SKILL.md body under 500 lines, reference files loaded on demand) flagging skills that cram 500+ words into SKILL.md without using reference files. It computes a weighted composite score (trigger rate 25%, user reaction 20%, workflow completion 15%, static quality 15%, undertrigger 15%, token economics 10%) plus qualitative-only reporting for overtrigger, conflicts, and environment checks. It is strictly read-only - it never modifies skill files, only outputs the report - and grounds its methodology in cited research (Memento-Skills on undertrigger/self-improvement loops, MCP Description Quality on 3.6x tool-selection improvement from good descriptions, Lost in the Middle on positional attention, format-variance and instruction-compliance studies).

When to use - and when NOT to

Use it when skills aren't triggering as expected or seem broken, when auditing and improving overall skill library quality, or when trying to understand which skills are underperforming or wasting context tokens. It only reports findings and specific wording suggestions - it never modifies skill files directly, so fixes must still be applied and reviewed by the user.

Inputs and outputs

Inputs: the target skill name(s) (or none, to scan all) and available session transcript history for the detected platform(s).
Outputs: a Skill Optimization Report with an overview table (triggers, reaction, completion, static grade, undertrigger misses, token count, composite score per skill), P0/P1/P2 prioritized fixes, and full per-skill diagnostics across all 8 dimensions.

Who it's for

Developers maintaining a library of Claude Code or Codex skills who need an evidence-based, quantified audit of which skills are firing correctly, satisfying users, completing their workflows, and worth their token cost - rather than guessing from anecdote.

FAQ

Common questions

Discussion

Questions & comments · 0

Sign In Sign in to leave a comment.