Optimize LLM Prompts with Caching Strategies

Prompt Caching is a skill covering caching strategies for LLM applications: Anthropic prompt caching, response caching, and CAG (Cache Augmented Generation). Its capabilities list is prompt-cache, response-cache, kv-cache, cag-patterns and cache-invalidation. The scope is deliberately LLM-specific, so it does not cover CDN caching, database query caching or static asset caching.

The skill teaches three patterns, each with a TypeScript example. Anthropic prompt caching marks the stable parts of a request, such as a long system prompt and a large knowledge base, with cache_control of type ephemeral, keeps the dynamic user query in the messages, and reads cache_read_input_tokens and cache_creation_input_tokens from the response usage to check hits. It notes a 90% cost reduction on cached tokens and up to 2x lower latency.

Response caching stores full LLM responses for repeated queries. The example hashes the prompt with SHA-256, stores the answer in Redis with a one hour default TTL, adds semantic similarity matching with a 0.95 threshold, and only caches low-temperature responses (temperature at or below 0.5). CAG pre-computes document context and places it in a cached prompt instead of running RAG retrieval, and it suggests refreshing that context periodically, hourly in its example.

The skill is triggered when a user mentions prompt caching, cache prompt, response cache, CAG or cache augmented generation. Use each pattern where it fits: Anthropic caching for stable system prompts or context, response caching when the same queries are asked repeatedly, and CAG when the document corpus is stable and fits in context.

It gives a CAG versus RAG decision matrix. CAG is better for corpora under 100K tokens, low update frequency, critical latency needs and general queries. RAG is better above 100K tokens, with frequent updates, flexible latency and specific queries.

Do not use it for non-LLM caching such as CDN, database or static assets. Its limitations section also says not to treat the output as a substitute for environment-specific validation, testing or expert review, and to ask for clarification if required inputs or success criteria are missing.

The prerequisites are knowledge of caching fundamentals, LLM API usage and hash functions, with context-window-management as a recommended companion skill. The inputs are your prompts, stable context and query patterns.

Three sharp edges are documented with fixes:

• Cache miss latency (HIGH): a miss plus cache overhead can be slower than no caching. The fix is a non-blocking cache check with a 50ms timeout, asynchronous cache writes, and selective caching of high-frequency patterns only.
• Stale cached responses (HIGH): fix with version-based keys, content-hash invalidation and event-based invalidation when a source updates.
• Prefix changes defeat Anthropic caching (MEDIUM): caching needs an exact prefix match, so timestamps or reordered messages break it. Keep the prefix static, put dynamic content in messages, and keep component ordering consistent.

Four validation checks are defined: caching high-temperature responses (warning), cache without TTL (warning), dynamic content in a cached prefix (warning) and no cache metrics (info).

The primary tools are Anthropic Prompt Caching in the Claude API, Redis for response caching (ioredis on servers, or @upstash/redis over HTTP on serverless) and automatic caching in the OpenAI API. The skill delegates context window or token questions to context-window-management, RAG questions to rag-implementation and memory persistence to conversation-memory.

The suggested high-performance workflow is:

1. Analyze query patterns
2. Implement prompt caching for stable prefixes
3. Add response caching for frequent queries
4. Consider CAG for stable document sets
5. Monitor and optimize hit rates

It is for developers running LLM applications who want lower cost and latency from repeated prefixes, repeated queries or stable document sets, and who need to avoid the usual failure modes of stale answers and cache misses that slow requests down.