Context failure modes are the performance degradation and logic errors that occur when a large language model (LLM) has its attention budget overwhelmed by increasing token counts.
Although frontier models now advertise context windows exceeding one million tokens, providing more context often makes an agent worse. This is because excessive context introduces distraction, increases the probability of retrieval errors, and forces the model to manage n² pairwise token relationships that stretch its attention budget thin.
In a production environment, context must be treated as a finite resource with diminishing marginal returns. Effectively engineering an agent requires considering the holistic state available to the LLM and recognizing that every additional token introduced depletes the attention budget. This leads to a performance gradient rather than a hard cliff, where a model remains technically capable of processing the window but loses the precision required for long-range reasoning.
What are context failure modes, and why does more context make an agent worse?
To understand context failure modes, it is helpful to view the LLM as the CPU and the context window as the RAM. Just as an operating system must curate what fits into physical memory to ensure the processor operates efficiently, context engineering is the practice of filling the window with only the highest-signal information. When the RAM—our context—is overstuffed with "junk drawer" data, the "CPU" begins to lag, causing the system to lose the precision necessary for agentic execution.
This performance drop is a direct consequence of the Transformer architecture. Because Transformers rely on an attention mechanism where every token attends to every other token, there are n² pairwise relationships for n tokens. As the token count grows, the model's ability to capture these relationships is stretched thin. This creates a performance gradient; while a model might technically support a 128k or 1M token window, its precision for information retrieval and complex reasoning degrades as the window fills because the attention budget is spread across too many dependencies.
Agents also operate with a highly skewed input-to-output ratio, often averaging 100:1 in prefill-to-decode tokens. This skew significantly impacts the KV-cache, which is critical for maintaining low latency and cost efficiency. In an agentic loop, where tool feedback and environment observations are constantly appended, the context grows with every turn. If this growth is not managed, the accumulation of feedback eventually saturates the attention budget, causing the model to lose focus on the primary instructions stored in the prefix.
Distinguishing between prompt engineering and context engineering matters for systems design. Prompt engineering is the discrete task of writing instructions. Context engineering, however, is the iterative architectural work of managing the holistic state—including system prompts, tool definitions, message history, and external data—to ensure the model remains steerable. Without active management, the context window ceases to be a working memory and becomes a source of noise that triggers logic failures and performance decay.
Context poisoning: when an error becomes a fact
Context poisoning occurs when a hallucination or logic error enters the context window and is subsequently treated as a ground-truth fact. The Gemini 2.5 technical report highlighted this phenomenon in agents playing Pokémon. If the agent hallucinated a specific game state or an impossible goal, that misinformation was saved into its summary or goal state. Because the model refers back to its own history, it becomes fixated on achieving these irrelevant or nonsensical objectives, leading to a loop of strategic failures that persist across turns.
This creates a compounding effect within long-running threads. As the model repeatedly references its prior errors, the poison spreads across the context. In the Pokémon case study, researchers noted that once the goal section was poisoned, it took a very long time to undo the damage. The model essentially develops a false belief system based on its own generated content, which eventually outweighs the reality of the actual environment observations provided by the game engine.
Economic factors often tempt engineers to keep these long, potentially poisoned histories in the window. Cached tokens—priced at approximately $0.30 per million tokens for Claude Sonnet—are 10x cheaper than uncached ones at $3.00 per million tokens. To maintain high KV-cache hit rates and low costs, developers are incentivized to keep the context append-only and stable. However, if an early turn contains an error, the cost-saving benefit of prefix caching actually preserves the poisoned data, ensuring the model remains stuck in an erroneous state to save a few cents.
To mitigate poisoning, engineers must treat failure as evidence rather than something to be erased. While the impulse might be to clean the trace, leaving wrong turns in the context can occasionally help a model adapt, provided the error is identified as a failure. True context engineering requires a mechanism to detect when the context has become fundamentally misaligned with the environment state, triggering a manual reset or a compaction step to purge the misinformation from the model's immediate attention.
Context distraction: when history outweighs training
Context distraction describes a phenomenon where an agent favors repeating actions from its own history over synthesizing novel plans. As an agent's context grows significantly—specifically beyond 100,000 tokens as observed in Gemini 2.5 Pro—the model tends to fixate on the patterns established in its recent history. Instead of applying its foundational training to solve a new problem, the model becomes a mimic of its own prior turns, repeating strategies even if they are no longer optimal for the current objective.
Research indicates that the distraction ceiling is often much lower for smaller models. A Databricks study demonstrated that model correctness begins to fall at roughly 32k tokens for Llama 3.1 405b, and even earlier for smaller variants. This suggests that while a model may have a large theoretical context window, its reasoning window is a fraction of that size. Once that threshold is crossed, the model transitions from a reasoning engine to a simple retrieval engine, losing the ability to perform the multi-step generative tasks required for autonomy.
This failure mode is often characterized by a U-shaped performance curve, a concept identified in the "Lost in the Middle" research. Models exhibit primacy bias (remembering the start of the context) and recency bias (remembering the end), but their ability to access and reason over information located in the middle of the context degrades significantly. As the middle of the context fills with history and observations, the agent's attention to its core instructions and long-range goals is sacrificed for the sake of the most recent, often repetitive, tokens.
Ultimately, context distraction turns the agent's memory against it. In long-horizon tasks, the sheer volume of tokens the model must attend to creates a noise floor that drowns out the signal from its original training data. When the context is too long, the model loses the context of the task in favor of the context of the conversation. This leads to logic loops, repetitive tool calls, and a fundamental inability to recognize when a goal has already been achieved or needs to be abandoned.
Context confusion: when irrelevant content still gets used
Context confusion occurs when superfluous or irrelevant information in the context influences the model's response, often leading to tool-use errors. A primary driver of this is tool bloat. Results from the Berkeley Function-Calling Leaderboard show that every model performs worse when provided with more than one tool. Even if the model is explicitly told that no tool is needed, it will occasionally trigger a tool call simply because the tool definitions are present in its attention budget, creating a statistical pressure to act.
A concrete example of this was seen in experiments using the GeoEngine benchmark. In this trial, a quantized Llama 3.1 8b model was provided with 46 different tools. Despite the context being well within the model's 16k window, the agent failed the task. However, when the toolset was pruned to just 19 relevant tools, the model succeeded. This confirms that it is not the window size that limits performance, but the confusion caused by an overstuffed junk drawer of possible actions that shifts the model's internal probability distributions.
When irrelevant documents or tool descriptions are included in the prompt, the model feels an inherent pressure to use them. For an agent, this results in unnecessary tool calls or the model paying attention to distractor snippets that seem topically related but do not actually answer the query. The model's self-attention mechanism is hijacked by the mere presence of these tokens, making it more likely to pick a near-neighbor tool or a related but incorrect document rather than the precise solution required.
Effective management requires finding the Goldilocks zone for system prompts and tool loadouts. On one hand, engineers may try to hardcode brittle logic to prevent confusion; on the other, they may provide vague guidance that assumes the model can sort through the noise autonomously. Neither is ideal. The goal of context engineering is to maintain a tight context where every token serves a specific purpose, minimizing the potential for the model to attend to irrelevant distractors that derail the logic flow of the agent.
Context clash: when the context contradicts itself
Context clash is a severe form of confusion where different parts of the context directly disagree with one another. This often happens in multi-turn exchanges where an LLM makes an early assumption or a guess in a previous turn. Once that incorrect guess is written into the history, it remains part of the context. Even if later turns provide more accurate information, the model may struggle to un-learn its first mistake, leading to a persistent contradiction that derails its reasoning.
The impact of this was documented in a study of sharded prompts, where information was gathered in stages across multiple chat rounds rather than all at once. The researchers found that sharded prompts yielded results that were 39% worse on average. Notably, OpenAI's o3 model saw its performance score drop from 98.1 to 64.1 when the same information was presented in pieces. The model attempted to generate solutions before it had all the facts, and those premature, incorrect attempts then clashed with the final, complete context provided at the end.
For agent builders, this is a major hurdle. Agents assemble context from diverse, often unverified sources like web searches and multiple tool calls. When a document retrieved in turn two contradicts a document retrieved in turn five, the model is forced to reconcile the clash within its attention budget. Because LLMs tend to rely heavily on their own early turns, the incorrect information often wins the attention battle, preventing the agent from recovering from a wrong turn even when the ground truth is later provided.
This difficulty in un-learning is why agents often get lost in a conversation. Once a model commits to a path or a fact in its history, the self-attention mechanism reinforces that token sequence. To fix context clash, the system must be capable of identifying and pruning contradictory information or using a quarantine strategy where specialized sub-agents process new data in isolation before it is synthesized into the lead agent's primary context thread, preventing premature assumptions from poisoning the state.
Context rot: accuracy falls before the window is full
Context rot refers to the non-uniform degradation of recall and reasoning precision as token counts grow, regardless of whether the context window is technically full. An 18-model evaluation involving state-of-the-art models like GPT-4.1 and Gemini 2.5 revealed that models do not process context uniformly. Instead, their performance on simple tasks, such as lexical retrieval, becomes increasingly unreliable as the input length scales toward the maximum. This is a gradient of decay, not a sudden failure at the limit.
Architectural findings show that models exhibit specific failure flavors. For instance, Claude Opus 4 has been observed to trigger a copyright-risk refusal when asked to replicate long sequences, incorrectly identifying repetitive strings as protected material. Similarly, GPT-4.1 nano has shown a lowercase "san" hallucination, where it begins outputting "san Francisco" instead of the capitalized "San Francisco" once the context reaches a certain depth. These specific errors demonstrate that the attention mechanism begins to fray at a structural level as token counts rise.
The structural properties of the text in the haystack also play a surprising role in context rot. Researchers found that models often perform better on shuffled haystacks—where sentences are randomly reordered—than on coherent essays. This suggests that the logical flow of a structured document might actually distract the attention mechanism, making it harder for the model to isolate specific needles of information. This counterintuitive finding highlights how sensitive models are to the arrangement of data within the window.
Task-specific rot is clearly visible in text replication experiments. In the repeated-words task, where a model must replicate a sequence of words with a single unique word inserted, models began to fail as context length increased. This failure manifested as under-generation, over-generation, or random outputs where the model generated nonsensical tokens. For an agent to remain reliable, it cannot treat the context window as a flat, reliable storage space; it must account for the reality that a token's index radically changes its accessibility.
Stale data: retrieval that is on-topic but no longer true
Stale-data failure occurs when a model retrieves information that is topically relevant but factually outdated or incorrect. Naive retrieval-augmented generation (RAG), the practice of throwing all retrieved documents into a prompt, reaches a point of diminishing returns very quickly. Reader performance typically saturates long before retriever recall does. Simply retrieving more documents is mathematically counterproductive if the attention tax of the reader exceeds the information gain.
A classic example of this is the Robert Griffin III query. In benchmarks using a 2018 Wikipedia dump, a model might state the player is on the Baltimore Ravens, even though the annotated answer is that he is currently a free agent. This is a temporal mismatch between the retrieval corpus and the actual state of the world. When an agent is forced to perform retrieval and reasoning simultaneously, it must use its limited attention budget to find the needle while also thinking. This dual tasking depletes the budget faster than reasoning alone, leading the model to settle for the most attention-grabbing but stale data.
The problem is exacerbated because retrieved documents are often treated as definitive knowledge by the model. If a retriever pulls five documents and only one is true, the model must disambiguate them—a process that acts as an additional attention tax. As the context grows, the model's ability to pay this tax decreases, and it begins to synthesize answers based on whatever documents are most topically similar, regardless of their current truth value or freshness.
To combat stale data, context engineering must move beyond simple retrieval and focus on ranked-list truncation or reranking. Pushing the most relevant and fresh information to the primacy or recency positions of the context window increases the likelihood of a correct answer. Naively filling the context with every retrieved doc assumes the model is a perfect filter; in reality, the model is a limited processor that performs best when the junk is pre-filtered by a separate system before the LLM ever sees it.
Which failure mode is your agent hitting?
Identifying which context failure mode is plaguing an agent requires looking for specific diagnostic symptoms. If an agent is stuck in a repetitive loop, it is likely suffering from context distraction, where it is over-indexing on its own history. If the agent is pursuing a nonsensical objective or repeating a previous mistake as if it were a ground truth, it is hitting context poisoning. If the model is picking the wrong tool from a massive set or calling tools when none are needed, the diagnosis is context confusion.
Observability tools like LangSmith are essential for this diagnostic process. By tracing agent trajectories and tracking token usage, engineers can identify exactly where performance drops occur relative to the window size. For instance, if an agent consistently fails once the context exceeds 32k tokens, you have found the distraction ceiling for that specific model. You can also watch for refusals driven by a temporal mismatch, a symptom of stale data where retrieved facts contradict the model's internal training or user-provided signals.
Of the six, I find poisoning the worst one to inherit, because it never stops the agent — it just makes the agent wrong while staying completely confident. The other five leave a mark in the trace you can go looking for; a poisoned run reads like a healthy one right up until you check the output.
Engineers should also monitor refusal rates as a metric of context health. In high-token tests, models occasionally refuse to attempt a task—69 refusals out of 194,480 LLM calls, or 0.035%—sometimes citing copyrighted material risks or simply stating they cannot help. These refusals often cluster around specific context lengths or stale data points, serving as a red flag that the model's attention is overloaded or conflicted by irrelevant or contradictory tokens.
Setting realistic benchmarks is the final step in diagnosis. Because model performance is non-uniform, you cannot assume a 90% success rate at 1k tokens will translate to 90% at 100k tokens. By measuring the gap between focused inputs (relevant data only) and full inputs (the whole history), you can quantify the exact attention tax your agent is paying. This delta defines the scope of the context engineering work required to restore reliable performance to the system.
Fixing each mode: RAG, tool loadout, quarantine, pruning, summarization, offloading
Fixing context failure modes requires a suite of aggressive information management tactics. To resolve confusion, implement a "less is more" tool loadout. Instead of providing 100 tools, use an LLM-powered recommender or a RAG-based tool search to select a relevant subset of fewer than 30 tools per turn. This drastically reduces tool-selection errors and speeds up inference by cutting the prompt length and reducing the pressure to act.
For clash and poisoning, use a quarantine strategy through multi-agent architectures. By splitting tasks among specialized sub-agents, each with its own isolated context window, you prevent irrelevant data from polluting the lead agent's attention. Anthropic's research system showed that this approach, where sub-agents distill information into condensed summaries for a lead agent, outperformed single-agent systems by 90.2% on internal evaluations because it enforces a strict separation of concerns.
To address rot and distraction, use pruning and summarization. Provence-based reranking can cull 95% of irrelevant markdown content while preserving the signal needed for the task. Then implement auto-compact logic that triggers once the context hits a threshold. This distills the conversation history into a high-fidelity summary, purging redundant tool outputs and observations while re-focusing the model on the primary goals stored in its prefix.
Finally, use offloading via scratchpads or a "think" tool to store notes outside the context window. This allows the model to log its progress and technical details without cluttering its active working memory. By maintaining lightweight identifiers, such as file paths or UUIDs, the agent can retrieve data just-in-time only when it is needed. This keeps the active window tight and high-signal, ensuring the attention budget is spent on reasoning rather than merely holding onto a bloated history.
Over-pruning is its own failure mode
While the goal of context engineering is to maintain a high-signal window, engineers must be wary of aggressive compaction. Over-pruning can discard subtle but critical details whose importance only becomes apparent much later in a long-horizon task. If the summarization step is too reductive, the agent may lose the fine detail required to resolve a bug or follow a complex policy, leading to secondary logic failures.
The smartest agents find a balance by employing a just-in-time retrieval model. By keeping the context window small and using tools to fetch specific data only when necessary, they avoid the pitfalls of up-front context bloat while ensuring information is available on demand. The most effective agents are not those with the largest context windows, but those that manage their attention budget with the most surgical precision.
Every fix above is a trade, though, and none of them comes with a threshold you can just look up: each token you prune is a token the agent no longer has to reason with. I still tune this by hand per system, and I don't think anyone gets out of that yet.
References
- How Long Contexts Fail — Drew Breunig
- How to Fix Your Context — Drew Breunig
- Context Rot: How Increasing Input Tokens Impacts LLM Performance — Chroma
- Lost in the Middle: How Language Models Use Long Contexts
- Effective context engineering for AI agents — Anthropic
- Context Engineering for Agents — LangChain
- Context Engineering for AI Agents: Lessons from Building Manus
- The RAG Freshness Problem: How Stale Embeddings Silently Wreck Retrieval Quality