# Loop Engineering: How to Build AI Systems That Improve Their Own Work

> Loop engineering replaces manual prompting by building autonomous recursive systems that execute tasks, evaluate outputs, and update state.

Source: https://camnangai.com/en/engineering/what-is-loop-engineering  
Published: 2026-10-24

**Loop engineering** is the practice of replacing yourself as the human prompter by designing an
  autonomous, recursive system that prompts agents, passes context, evaluates outputs, updates
  state, and iterates until a goal is met.


Instead of typing prompts turn-by-turn in a single chat session, you build a control loop that finds work, hands it out, verifies results, and feeds itself. The workflow shifts from holding the tool continuously during execution to designing a background process that pokes the agents for you. This system runs on a defined schedule or goal trigger, records progress outside transient chat context, and decides what action to take next based on environmental feedback.


  Loop engineering sits one floor above the agent execution runtime: while a runtime configures the
  local environment for a single execution run, a loop orchestrates automations, worktrees, skills,
  connectors, sub-agents, and disk-based memory into an ongoing process that manages itself.


![Shift from manual prompt fatigue to an autonomous recursive loop engineering architecture](https://camnangai.com/articles/loop-engineering-la-gi/01-infographic-en.webp)

## From manual prompting to loop engineering: why the mental model must shift

For years, working with AI meant sitting in a chat session: you wrote a prompt, shared context, read the generated reply, and caught bugs in your head. When a response missed the mark, you nudged the model with corrections and decided when the output finally met your quality bar. The engineer was the sole engine of the loop—handling memory tracking, eval checks, boundary enforcement, and stopping decisions entirely between their ears.

Prompting by hand breaks down the moment work scales. At ten tasks, you can review diffs with genuine care. At two hundred, your attention drifts while models churn out tokens at the exact same speed without any earned domain judgment. You can see this shift happening across engineering teams. Peter Steinberger argued that developers should stop prompting coding agents directly and start designing loops that prompt agents, while Boris Cherny pointed out that he no longer prompts Claude manually—he writes loops that run continuously to prompt the model and figure out next steps.

That shift takes shape as a formalized execution loop: **Act → observe → evaluate → update state → decide → repeat, stop, or escalate**. Instead of rerolling a prompt and hoping for a better turn, a feedback loop logs specific failure details into external memory. The loop uses those recorded failures to guide subsequent turns, making decisions based on measurable progress rather than random variance.

## The 9 architectural components of a reliable AI loop

![The 9 core architectural components of a reliable AI engineering loop in production](https://camnangai.com/articles/loop-engineering-la-gi/02-en.webp)

A reliable AI loop comes down to nine architectural primitives that constrain generation and enforce operational boundaries. First, the **Goal** must be an explicit, measurable objective rather than a vague assignment. **Context** supplies the boundaries and domain rules the model needs, while **Actions** stay restricted to narrow, traceable execution steps. For execution, **Tools** follow least privilege—granting only the permissions needed for the task—while **Evals** score outputs objectively using deterministic code checks, source comparisons, or calibrated model scoring.

To keep the system running across sessions, **Memory** must live outside the prompt as concise, disk-based state files (such as Markdown notes or Linear boards), because model context evaporates between runs. Architectural **Guardrails** enforce hard boundaries that no prompt can bypass, such as blocking direct merges to production branches. When an edge case falls outside those boundaries, **Escalation** routes the decision to a human with full context attached, and explicit **Stopping Conditions** cut the loop off on success, token budget caps, stall detection, or tool errors.

In modern tools like Claude Code or OpenAI Codex, these nine logical concepts map cleanly to five platform mechanisms. Automations discover and triage work on a schedule. Git worktrees isolate parallel agent tasks so they never trample each other's files. Project skills (`SKILL.md`) codify repo conventions so models do not guess missing context. External connectors tie into issue trackers and staging APIs, while dedicated sub-agents split creation from verification so the model writing code never grades its own homework.

| Primitive              | System Function                   | Implementation Example (Codex / Claude Code)                                           |
| :--------------------- | :-------------------------------- | :------------------------------------------------------------------------------------- |
| **Goal**               | Defines measurable objective      | `/goal` command with explicit verification criteria                                    |
| **Context**            | Supplies domain boundaries        | Retrieved repository architecture and `SKILL.md` rules                                 |
| **Actions**            | Defines traceable execution steps | File diff inspection, running test suites, drafting comments                           |
| **Tools**              | Scopes execution permissions      | Model Context Protocol (MCP) servers, read-only GitHub access, sandboxed shell         |
| **Evals**              | Scores outputs objectively        | Deterministic test runners, source comparison, Natural Language Inference (NLI) models |
| **Memory**             | Persists state across sessions    | Disk-based files (`AGENTS.md`, progress files, Linear)                                 |
| **Guardrails**         | Prevents unauthorized operations  | Sandboxed execution, blocking direct production merges                                 |
| **Escalation**         | Transfers control to human        | Triage inbox alerts with attached evidence and logs                                    |
| **Stopping Condition** | Terminates execution loop         | Explicit targets, budget caps, stall limits, failure thresholds                        |

```python
while target_not_reached:
    result = take_action(context, memory)
    evaluation = evaluate(result, rubric)
    log(result, evaluation)
    if evaluation.passes:
        stop_with_success(result)
    if human_decision_required(evaluation):
        escalate(result, evaluation)
    if budget_exhausted() or progress_stalled():
        stop_without_success()
    memory = update_memory(result, evaluation)
```

## The self-improvement engine: Champion-Challenger and eval-driven development

Self-improving loops work by adapting the Champion-Challenger pattern from production ML to iterate on prompts and workflows against empirical data. Production inputs are split into an **improvement set** for active tuning and a **holdout set** to catch overfitting. For long-running systems, keeping a third, untouched **audit set** is essential to prevent information leakage over repeated eval runs. In this setup, the active prompt is the champion, and every candidate prompt enters as a challenger that has to earn its promotion on holdout data.

![Champion-Challenger workflow and dataset splitting across improvement, holdout, and audit sets](https://camnangai.com/articles/loop-engineering-la-gi/03-en.webp)

To keep prompt optimization causal rather than superstitious, the iteration protocol needs strict boundaries. Take a system designed to handle inbound customer feature requests:
Start by assembling 40 real production inputs, splitting them into an improvement set of 25 cases and a holdout set of 15 cases. Running the champion prompt across the 15 holdout cases sets the baseline at 3.4 out of 5.0—with 10 out of 15 replies failing an overpromising check. Before launching the loop, you establish hard stopping boundaries: a target holdout score of 4.5, a budget cap of 12 rounds, and a stall threshold of 3 consecutive non-promoting rounds.

During each round, the loop proposes exactly one controlled prompt change. It benchmarks the challenger on the improvement set, tossing out anything that regresses. Promising candidates then run against the holdout set, winning promotion to champion only if holdout scores rise without breaking existing guardrails; ties go to the incumbent. In Round 4 of our run, the loop injects a targeted prompt rule: "If the requested feature is not on the committed roadmap, say that the team is tracking demand..." That single rule plugs the primary failure mode, driving the holdout score from 4.0 to 4.4. By Round 7, the prompt hits a 4.6 holdout score, stopping automatically after 40 minutes of unattended execution at an API cost of $2.00, yielding a prompt that is 30% different from the baseline.

None of this works if evals rely on vibes. Concrete parameters like word counts or schema validation belong in deterministic code checks, which run instantly for zero cost. To measure factual consistency, a fine-tuned Natural Language Inference (NLI) model treats the source document as the premise and the generated text as the hypothesis. Dropping the neutral probability dimension and taking a softmax over entailment and contradiction yields a contradiction probability that acts as a direct, calibrated metric for factual errors. For subjective quality scoring, reward models trained on paired human preference data output scalar scores you can track over time.

## Three core loop architectures: Builder-Critic, saturation, and evaluator-optimizer

The **Builder-Critic** architecture pairs a generator model with a dedicated adversary that opposes its outputs. The critic model does not rewrite anything; its sole job is to attack unexamined assumptions, unhandled edge cases, operational cost spikes, and vague metrics. The builder cannot simply ignore feedback. The system enforces an invariant: every flagged critique must be explicitly resolved, either _fixed_ through document changes or _accepted_ with a logged justification. Splitting generation from evaluation breaks the shared blind spot that ruins single-model review.

![Three production loop architectures: Builder-Critic, Saturation Loop, and Evaluator-Optimizer](https://camnangai.com/articles/loop-engineering-la-gi/04-en.webp)

A **Saturation Loop** tackles large, unstructured datasets by ingesting records in structured batches—say, batches of 20 unstructured records from a pool of 300 support tickets or user interviews. The loop extracts recurring themes, groups feedback into clusters, and attaches direct source quotes to each cluster. After processing each batch, the system evaluates qualitative saturation by checking whether the new batch created new clusters or materially altered existing ones. The loop terminates automatically when two consecutive batches yield no new information or when execution hits a hard cap of ten batches.

When tasks outgrow flat feedback, linear iteration like _Self-Refine_ or _Reflexion_ runs into a wall. Both frameworks rely on a single linear trajectory where an actor generates responses, a reflector critiques failures, and an episodic memory buffer stores verbal reinforcement. But on complex engineering tasks, single-path loops easily get trapped in local minima: the model tries to patch a flawed premise instead of backing out. To solve this, Language Agent Tree Search (LATS) merges generation, verbal reflection, and Monte Carlo Tree Search (MCTS).

LATS executes across four distinct phases during each turn. In the **Select** phase, the search algorithm traverses existing nodes using Upper Confidence Bounds applied to Trees (UCT), calculating `UCT = (value / visits) + c * sqrt(ln(parent.visits) / visits)` to balance expected reward with exploration. During **Expand and Simulate**, the agent samples $N=5$ parallel candidate actions. In **Reflect + Evaluate**, runtime feedback and reflection models score the simulated paths. Finally, in **Backpropagate**, those scalar scores update every ancestor node up to the root, allowing the agent to back out of architectural dead ends and discover optimal execution paths.

## Five failure modes of AI loops and how to prevent them

Without strict guardrails, unattended loops fail in predictable ways. The first is **drift**, where the system optimizes a proxy metric at the expense of the actual outcome. If you reward an agent for brevity, for example, it will happily shorten responses until the text becomes brief, polite, and completely useless. Guarding against drift requires restating the core objective during every turn, evaluating outputs with multi-dimensional rubrics, and spot-checking actual outputs instead of worshiping an aggregate score.

The second failure mode is **weak evals**. If your evaluator is easier to fool than the underlying task is to solve, the loop simply learns how to please the judge. You eliminate weak evals by implementing deterministic code checks for concrete rules, requiring exact source citations for claims, and analyzing class distribution separation using Jensen-Shannon Divergence (JSD). When you plot predicted probability distributions, a high JSD (such as ~11.0) shows clean separation between positive and negative classes; a low JSD (such as ~1.1) reveals distribution overlap where a minor 0.05 shift in evaluation threshold causes massive drops in precision or recall.

The remaining three operational hazards—**runaway cost**, **latency**, and **endless retries**—stem from unbounded execution. A broken loop will cheerfully burn through token budgets on repeated failed attempts at 3:00 AM or take hours to finish. You control costs by setting hard constraints (such as capping runs at 12 rounds, $2.00 token spend, or 3 non-improving iterations), while routing simple verification steps to smaller, cheaper model tiers. You mitigate latency by running loops asynchronously with resumable disk-based state checkpoints, and you eliminate endless retries by enforcing strict stall limits that terminate execution when consecutive turns fail to produce measurable progress.

## The autonomy ladder: earning bounded autonomy through observable evidence

Handing an AI loop write access to production on day one is asking for an incident. Autonomy must be earned gradually across four distinct rungs. In **Shadow Mode**, the loop processes live production inputs in parallel with existing human workflows, but external execution is locked down. You compare the loop's hypothetical outputs against actual human decisions to establish baseline reliability without introducing operational risk to your production environment.

![The 4 stages of the Autonomy Ladder: Shadow, Suggest, Approve, and Bounded Autonomy](https://camnangai.com/articles/loop-engineering-la-gi/05-en.webp)

Once the system proves reliable in shadow mode, you advance it to **Suggest Mode** and **Approve Mode**. In Suggest Mode, the loop generates recommendations accompanied by structured evidence for human operators, who record whether they accept, edit, or reject the suggestions. In Approve Mode, the loop constructs complete execution payloads, such as draft code changes or pull requests, but requires explicit human sign-off before executing the final action in production.

You promote the system to **Bounded Autonomy** only after it demonstrates consistent performance across lower stages. Under Bounded Autonomy, the loop executes actions independently within strict, documented parameters, such as handling specific low-risk request types within predefined daily cost and volume caps. You enforce the core principle that loops must earn autonomy through observable evidence (such as passing automated test suites, matching verified source records, and achieving high evaluation scores) rather than relying on model self-confidence scores.

## The loop decision framework: when to build (and when to skip) a loop

You should not build a loop for every task. If a single prompt or deterministic workflow satisfies your requirements, adding state tracking, evaluators, and retry logic creates unnecessary system complexity. You decide whether to build a loop by answering three operational questions:

1. Can you check the result deterministically or through grounded rubrics?
2. What is the blast radius if the output is wrong?
3. Can you define the exact stopping conditions, budgets, and escalation triggers before writing line one of execution code?

Build loops for tasks when you can check outputs deterministically, restrict the blast radius, and set explicit stopping parameters. If outputs carry high financial, legal, or customer-facing risks, you automate the verification steps but retain a human checkpoint before execution. Establish clear operational boundaries up front, run the system in shadow mode, and let observable evidence determine how much authority the loop receives.

## References

- [Loop Engineering: Build AI Systems That Improve Their Work — Addy Osmani](https://www.the-ai-corner.com/p/loop-engineering-masterclass-self-improving-ai-systems-2026)
- [Loop Engineering: The Next Frontier in AI Coding Agents — Addy Osmani](https://addyosmani.com/blog/loop-engineering/)
- [Building Effective Agents — Anthropic Research](https://www.anthropic.com/research/building-effective-agents)
- [Reflection Agents — LangChain](https://blog.langchain.dev/reflection-agents/)
- [Reflexion: Language Agents with Verbal Reinforcement Learning — Shinn et al.](https://arxiv.org/abs/2303.11366)
- [Self-Refine: Iterative Refinement with Self-Feedback — Madaan et al.](https://arxiv.org/abs/2303.17651)
- [Task-Specific LLM Evals that Do & Don't Work — Eugene Yan](https://eugeneyan.com/writing/evals/)