# LLM Overthinking: Why Thinking More Can Make an LLM Dumb

> Allocating excess test-time compute to simple prompts triggers LLM overthinking, degrading model accuracy through premise drift and redundant reasoning.

Source: https://camnangai.com/en/knowledge/why-thinking-more-can-make-an-llm-dumb  
Published: 2026-11-20

Large language models generate text autoregressively, computing output probabilities token by token based on previously emitted sequence history. Allocating additional test-time compute through extended reasoning trajectories allows models to solve multi-step logic and complex mathematics. However, forcing additional inference compute on simple or noisy prompts triggers **LLM overthinking**, causing accuracy to degrade as extended reasoning re-evaluates and overturns verified conclusions.


  Instead of improving the output, generating surplus thinking tokens on trivial inputs leads the
  system to reconsider valid assumptions, drift away from core prompt instructions, or introduce
  false premises into its scratchpad.


Understanding why surplus test-time compute degrades model accuracy requires analyzing autoregressive attention dynamics, reinforcement learning reward structures, and production latency tradeoffs.

![LLM Overthinking: Why Extended Thinking Can Degrade Model Accuracy](https://camnangai.com/articles/vi-sao-suy-nghi-nhieu-khien-llm-kem-thong-minh/01-infographic-en.webp)

## The Promise of Extended Thinking: How Reasoning Models Evolved

![The Evolution of Reasoning Models: Hidden Scratchpads and Autonomous Verification](https://camnangai.com/articles/vi-sao-suy-nghi-nhieu-khien-llm-kem-thong-minh/02-en.webp)

Frontier architectures have evolved beyond single-pass next-token prediction toward test-time compute scaling. Rather than forcing a model to generate an immediate output token in a single forward pass, architectures like OpenAI o1, DeepSeek-R1, and Claude 3.7 Sonnet allocate test-time compute to generate intermediate "thinking tokens." These tokens exist in a hidden scratchpad where the model can decompose complex problems, test intermediate hypotheses, and execute self-correction prior to finalizing its response.


  Reinforcement learning algorithms like Group Relative Policy Optimization (GRPO) train models to
  use these intermediate scratchpads without requiring step-by-step human annotations.

Unlike traditional Proximal Policy Optimization (PPO), GRPO calculates baseline performance metrics
directly from group output statistics rather than maintaining a separate, memory-heavy critic
network, significantly reducing VRAM overhead during RL training scaling. This self-directed
exploration yields autonomous verification mechanisms, such as DeepSeek R1-Zero's emergent "aha
moments" in math problems, where the model squared complex algebraic expressions, recognized
unnecessary expansion, halted its execution, and backtracked to the original equation.


  On high-uncertainty tasks, extended test-time compute delivers measurable empirical gains.

On SWE-bench Verified, Claude 3.7 Sonnet achieves a 63.7% pass@1 score in standard execution, which
rises to 70.3% under high compute scaffolding using parallel attempts and rejection sampling against
visible regression tests. Similarly, expanding multi-turn trajectory allocations on TAU-bench from
30 to 100 maximum steps enables agents to execute deeper tool-use chains and complete complex
environment interactions.

## The Overthinking Trap on Simple Prompts

![The Overthinking Trap on Simple Prompts: Redundant Verification and Task Deviation](https://camnangai.com/articles/vi-sao-suy-nghi-nhieu-khien-llm-kem-thong-minh/03-en.webp)

While test-time compute scaling enhances multi-step problem solving, applying unconstrained reasoning workloads to trivial prompts causes systematic resource allocation failures. Research demonstrates that reasoning models often generate thousands of redundant tokens when executing basic operations. When presented with elementary arithmetic like `2 + 3`, models such as QwQ-32B-Preview can emit up to 13 distinct solution attempts in their internal scratchpad before outputting the final answer.

This overthinking stems from outcome-based reward modeling in reinforcement learning. Standard reward functions score attempts based solely on final output correctness, granting equal reward to a direct one-step answer of `5` and a 2,000-token trace that recalculates the addition through multiple redundant pathways. To eliminate this inefficiency, researchers performed supervised fine-tuning (SFT) on trajectory-pruned trajectories that cut reasoning after the first verified correct answer plus a single verification check. On the MATH500 benchmark, this SFT approach reduced average token usage from 2,408 to 1,331 tokens per problem while preserving accuracy (93.0% versus 92.8%).

In production environments, unpruned reasoning tokens introduce severe economic and SLA penalties. Output tokens—including hidden thinking tokens—are typically billed at premium rates, such as 15 USD per million output tokens versus 3 USD per million input tokens. Beyond raw financial expenditure, redundant reasoning tokens directly inflate Total Generation Latency and Time Between Tokens (TBT), penalizing real-time system performance and breaching operational SLA targets for zero marginal gain in output accuracy.

## Compounding Autoregressive Errors and the Drifting Premise

![Compounding Autoregressive Errors: Attention Drift Triggered by Distractor Clauses](https://camnangai.com/articles/vi-sao-suy-nghi-nhieu-khien-llm-kem-thong-minh/04-en.webp)


  Beyond generating financial and operational latency penalties, extended reasoning trajectories on
  simple prompts can actively corrupt final outputs through premise drift.


  Transformer architectures compute self-attention vectors over all previously generated tokens in
  the context window.


  Once a model writes an incorrect intermediate assumption, flawed edge case, or alternate
  interpretation into its scratchpad, subsequent tokens attend directly to that erroneous context,
  steering generation off-course.


Consider a prompt requesting a scheduled train departure from a timetable that lists 9:00, accompanied by a side note stating trains on the line average a 10-minute delay. A standard LLM extracts `9:00` instantly. An overthinking reasoning model, however, may spend multiple thinking cycles analyzing the operational impact of the delay. As the scratchpad accumulates intermediate calculations regarding actual predicted arrival times (9:10), self-attention weights shift toward these recent reasoning tokens. When generating the final user-facing response, the model outputs `9:10`, failing the explicit instruction of the prompt.

Empirical research using the GSM-Symbolic benchmark—specifically under the GSM-NoOp test setting—demonstrates this vulnerability. By inserting a single logically irrelevant clause into grade-school math problems, model accuracy dropped by up to 65% across state-of-the-art architectures. Because language models replicate logic templates from training data rather than performing pure symbolic deduction, intermediate reasoning noise triggers attention drift, compounding minor scratchpad hallucinations into final output failures.

## Diminishing Returns and the "Negative Flip" Phenomenon

![Diminishing Returns and the Negative Flip Phenomenon in Test-Time Compute Scaling](https://camnangai.com/articles/vi-sao-suy-nghi-nhieu-khien-llm-kem-thong-minh/05-en.webp)

Continuous test-time compute scaling exhibits clear non-linear boundaries. Investigations confirm that the marginal utility of additional reasoning tokens follows a steep curve of diminishing returns. Beyond a task-specific compute threshold, generating extra thinking tokens yields zero performance improvement and increases the risk of solution corruption.

```text
Model Accuracy
  ^
  |        /--- Maximum Utility Threshold
  |       /   |      /   \  <-- Negative Flip Zone (Abandoning Correct Answers)
  |     /       |    /       \--- Diminishing Returns
  +-------------------------------------> Test-Time Compute (Tokens)
```

Scaling compute past this optimal threshold triggers the "negative flip" phenomenon, where a model abandons an initially correct solution. In these instances, the model identifies the correct response within its initial reasoning steps but lacks a deterministic stopping condition. As it continues generating tokens to re-evaluate its work, it introduces unnecessary edge cases or hyper-analyzes noise, eventually overwriting its valid intermediate answer with an incorrect final conclusion.

In parallel, empirical evaluations establish that optimal thinking token distributions vary significantly by domain. Forcing uniform, extended token budgets across heterogeneous workloads degrades performance on creative generation, direct lookup, and structured output tasks, proving that unconstrained reasoning actively impairs accuracy outside complex mathematical and symbolic domains.

## How Modern Systems Mitigate Overthinking: From Dynamic Budgets to Hybrid Reasoning

![Mitigating LLM Overthinking: Early Truncation, Thinking Budgets, and Hybrid Reasoning](https://camnangai.com/articles/vi-sao-suy-nghi-nhieu-khien-llm-kem-thong-minh/06-en.webp)

To prevent overthinking while retaining extended reasoning capabilities, modern infrastructure incorporates architectural controls, API parameters, and specialized training pipelines. At the architecture tier, hybrid reasoning models like Claude 3.7 Sonnet decouple standard generation from extended thinking. System designers can route low-latency requests through standard execution or explicitly enable extended thinking mode when handling high-complexity workloads.


  At the inference API layer, providers offer granular budget management, allowing infrastructure
  engineers to set an explicit maximum cap on thinking tokens (N tokens up to 128K).

This token budget enforces strict cost and latency boundaries, terminating scratchpad generation
before compounding errors or redundant verification loops impair response quality.

At the model training layer, systems apply dynamic step reduction and input-prompt difficulty routing. Through Thinking-Optimal Scaling strategies, models like Qwen2.5-32B-Instruct are fine-tuned on seed datasets with varied reasoning length distributions. By scoring attempts on both correctness and token conciseness, the model learns to evaluate prompt difficulty at runtime and autonomously select the shortest valid reasoning path.

## When to Let an LLM Think — and When to Cut It Short

Optimizing test-time compute requires matching inference allocation directly to task complexity. Disable or strictly cap extended thinking tokens when processing deterministic lookups, basic arithmetic, string extractions, schema-constrained JSON outputs, or real-time interactive user routes where added latency violates SLAs and raises premise drift risk.

```text
                                  [ Incoming Request ]
                                           |
                                 Is the task complex?
                                 /                                               ( No )                ( Yes )
                               /                            [ Disable / Cap Extended Thinking ]     [ Enable Extended Thinking ]
      • Deterministic lookups                 • Repository-level coding
      • Basic arithmetic                      • Multi-step agent workflows
      • String & schema extraction            • Advanced math & physics
      • Low-latency API routes                • Ambiguity-heavy logic puzzles
```


  Enable extended thinking modes when deploying systems for repository-level software refactoring,
  multi-turn agentic tool integration, advanced mathematical derivations, and ambiguity-heavy logic
  problems.

Aligning your test-time compute budget strictly with prompt difficulty eliminates redundant token
expenditure, guarantees predictable SLA latency, and prevents overthinking from corrupting model
outputs.

## References

- [Why Thinking More Can Make an LLM Dumb — ByteByteGo Newsletter](https://blog.bytebytego.com/p/why-thinking-more-can-make-an-llm)
- [Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs — arXiv:2412.21187](https://arxiv.org/abs/2412.21187)
- [When More Thinking Hurts: Overthinking in LLM Test-Time Compute Scaling — arXiv:2604.10739](https://arxiv.org/abs/2604.10739)
- [Towards Thinking-Optimal Scaling of Test-Time Compute for LLM Reasoning — arXiv:2502.18080](https://arxiv.org/abs/2502.18080)
- [GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models — arXiv:2410.05229](https://arxiv.org/abs/2410.05229)
- [Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models — arXiv:2503.16419](https://arxiv.org/abs/2503.16419)
- [Claude 3.7 Sonnet and Claude Code — Anthropic](https://www.anthropic.com/news/claude-3-7-sonnet)
