Optimize Context for Deeper Insights

Covers four strategies for extending an LLM agent's effective context capacity without a larger model or longer window: compaction (summarizing context near the limit and reinitializing with the summary, prioritizing tool outputs, then old conversation turns, then retrieved docs, and never the system prompt), observation masking (replacing verbose tool outputs, which can be 80%+ of token usage in agent trajectories, with compact references once their information has been used - masking observations 3+ turns old or already summarized, never masking observations critical to current reasoning or from the most recent turn), KV-cache optimization (reusing cached Key/Value tensors across requests sharing an identical prefix via hash-based block matching, ordering context with stable elements like the system prompt and tool definitions first, then reused templates, then unique content last, and avoiding dynamic content like timestamps that breaks cache stability), and context partitioning (splitting work across sub-agents with isolated, clean contexts per subtask, then aggregating and re-summarizing results if they're still too large). It also defines explicit context-budget allocation across categories - system prompt, tool definitions, retrieved docs, message history, reserved buffer - and trigger-based optimization when token utilization exceeds 80% or quality or latency degrade.

Activate when context limits constrain task complexity, when optimizing for token cost or latency in long conversations, when building long-running agent systems, or when handling larger documents or conversations at production scale. The decision framework: optimize when context utilization exceeds 70%, response quality degrades as conversations extend, or costs or latency rise with context length; pick observation masking when tool outputs dominate usage, summarization or partitioning when retrieved documents dominate, and compaction when message history dominates. This is a scaling technique, not a substitute for good context design - it builds on companion skills context-fundamentals and context-degradation, and shouldn't be reached for before those fundamentals are in place.

Input is an agent's accumulating context - tool outputs, conversation turns, retrieved documents - approaching a defined token budget. Output is a reduced context that preserves signal: compaction targets 50-70% token reduction with under 5% quality degradation, masking targets 60-80% reduction in masked observations, and cache optimization targets a 70%+ hit rate for stable workloads.

if context_tokens / context_limit > 0.8:
    context = compact_context(context)

Connects to multi-agent-patterns (partitioning as isolation), evaluation (measuring optimization effectiveness), and memory-systems (offloading context to external memory) as companion skills in the same context-engineering collection.

Engineers building production LLM agent systems - especially long-running or multi-turn ones - who are hitting context-window limits, rising token costs, or quality degradation from bloated context, and need concrete compression, masking, caching, and partitioning techniques with measurable targets.