@ahmetcantryk
An agent that nails the first three steps of a task and then quietly derails on step twelve almost never has a model problem. It has a context problem: the window filled with stale tool output, contradicting instructions, or irrelevant history, and the model started reasoning over noise instead of signal. Fixing that isn't about switching models — it's context engineering, and by mid-2026 it's the single most-cited limiting factor for production agents, ahead of raw model capability.
Agent failures are context failures, not model failures
Frontier models now ship with 1M+ token windows, but capacity isn't the bottleneck teams hit in practice. Research on long-running agents consistently finds that output quality degrades once the relevant information in context passes roughly 50K tokens — well before the window fills up — because irrelevant or stale content dilutes the signal the model needs to reason correctly. Anthropic's own engineering write-up on the topic frames context as a finite resource with diminishing marginal returns: every token you add has to earn its place against the tokens already competing for the model's attention.
That reframes the debugging question. When an agent goes wrong on a multi-step task, the first question isn't "is this the right model" — it's "what does this agent's context actually look like at step twelve, and how much of it is still relevant."
The context budget: quality beats volume
Treat context like a budget, not a bucket you keep filling. Most production teams never use anywhere near their model's full window — the problem is what they put in the tokens they do use.
Metric | Typical impact |
|---|---|
Context engineering applied to an existing agent | 30–50% lower per-task token cost |
Same intervention, output quality | 25–40% improvement |
Relevant-context threshold before quality drops | ~50K tokens, regardless of window size |
Teams using their model's full context window | A small minority |
The takeaway is not "use less context" — it's "make every token in context earn a place." A 200K-token window with 40K tokens of tightly curated, relevant state consistently outperforms the same window stuffed with 150K tokens of raw tool output and chat history.
The seven core patterns
Production context engineering resolves into seven recurring patterns. Most real agents combine three or four of them, not all seven at once.
Pattern | What it does | Use it when |
|---|---|---|
Selection | Choose the smallest model/context configuration that meets the task | Routing simple vs. complex sub-tasks |
Compression | Summarize past turns instead of keeping raw transcripts | Long-running sessions, multi-step plans |
RAG integration | Retrieve only the relevant slice of a larger knowledge base on demand | Codebases or docs too large for any window |
Shared context layers | Give multiple agents a common state store instead of duplicating context | Multi-agent and sub-agent workflows |
Context routing | Send different context to different agents based on their role | Specialized sub-agents (reviewer vs. implementer) |
Context eviction | Actively remove stale or superseded information | Long sessions where early state goes invalid |
Context caching | Reuse computed context across calls instead of resending it | Repeated calls sharing a stable prefix |
Selection and routing decide what enters context in the first place; compression and eviction decide what leaves; RAG and shared layers decide where the ground truth actually lives; caching is the only pattern that's purely about cost, not correctness.
The caching trap: only cache what's actually stable
Caching is the pattern most teams get wrong first, because it looks like a pure win with no design tradeoff — until a single unstable token silently breaks it. Prompt caches match on an exact prefix, so anything that changes between calls, including something as small as a live date stamp, has to sit after the cached block, not inside it. The safest default is to cache only the system prompt and static tool definitions, and to keep genuinely dynamic content — current date, session state, retrieved documents — outside the cached prefix entirely. We go deeper on the cost side of this exact failure mode in our guide to cutting LLM token costs.
A worked before/after
Take a coding agent doing a multi-file refactor across a 40-file module. In the "before" state, its context accumulates the full output of every file read, every test run, and every prior turn's reasoning verbatim — by file 15, half the window is dead weight from files already handled. The agent starts reintroducing bugs it already fixed because the fix is buried under newer, unrelated tool output.
The "after" state applies three of the seven patterns together: eviction drops the full content of files once their edits are verified, keeping only a one-line summary of what changed; compression collapses completed sub-tasks into a running plan-state block instead of full transcripts; and RAG-style retrieval pulls a file's current content back in only when the agent is about to touch it again. Net effect: token cost per file drops substantially, and — more importantly — the model stops losing track of decisions it already made, because the decisions are captured in a compact state block instead of buried in raw history.
A checklist for your existing agent
Apply this to one agent you already run in production, this week:
- Measure current context composition. Log what fraction of a typical context window is system prompt, tool output, history, and retrieved data.
- Cap tool output. Truncate or summarize verbose tool results (test runs, file listings, API responses) before they enter context.
- Add eviction for completed sub-tasks. Once a step is verified done, replace its full trace with a one-line summary.
- Split cacheable from dynamic content. Move anything that changes per-call (dates, session IDs, live state) out of the cached prefix.
- Introduce retrieval instead of pre-loading. If you're stuffing an entire doc or codebase into the system prompt "just in case," switch to on-demand retrieval — see our RAG system guide for the mechanics.
- Re-test at the 50K-token relevance mark. Specifically evaluate quality once relevant context crosses that threshold, not just at max window size.
Teams building coordinated agents on top of this foundation should also read our take on AI agents vs. workflows — the pattern you pick upstream determines how much context discipline you'll need downstream, and our piece on AI agent memory systems covers the persistence layer that context engineering eventually has to hand off to.
Frequently Asked Questions
Isn't a bigger context window the simpler fix?
It solves capacity, not relevance. Studies on long-running agents consistently show quality degrading once relevant context passes roughly 50K tokens, regardless of how large the window is — a bigger window mostly buys you room to be sloppier for longer before it shows.
Which pattern should I implement first?
Eviction and compression usually give the fastest measurable win because they attack the most common failure (stale tool output crowding the window) without touching your retrieval or caching architecture at all.
How is context engineering different from prompt engineering?
Prompt engineering optimizes a single, mostly static instruction. Context engineering treats the entire token budget — system prompt, tools, retrieved data, history — as a continuously managed resource across a multi-step, multi-turn task.
Does context engineering replace fine-tuning or a better model?
No — they solve different problems. A stronger model reasons better over the same context; context engineering ensures the model is reasoning over the right information in the first place. Most production quality gains in 2026 have come from the latter, because most teams' context was the actual bottleneck.
