Most teams building with large language models default to one of two failure-handling strategies. Either they bolt on a retry loop that re-asks the model the same question until it produces something usable, or they spend weeks collecting preferences and running a reinforcement-learning fine-tune to coax better behaviour out of the base model. Both approaches are expensive, and our team's recent experiments suggest there is a third path that outperforms both for many agentic workloads. We have started calling the practice loop engineering: giving the model the ability to read its own prior reasoning and the feedback from the tools it called, then mutate its own working prompt before the next attempt.

What follows is a measured claim, not a marketing one. The numbers come from internal evaluation of an agent that has to plan, call tools, observe results, and revise. We are publishing it because the gap we measured is large enough that any team shipping agentic systems should at least know it exists.

The setup

The agent under test operates in a standard tool-use loop. Given a task, it produces a plan, issues tool calls, inspects the responses, and either commits an answer or iterates. What we varied was what happens between iterations.

The control condition is prompt-chaining: each iteration uses a fixed prompt template, and the only thing that changes is the appended history of tool outputs. This is the textbook ReAct-style pattern, and it is what most reference agent implementations do.

The treatment condition adds a reflection step. Before the next iteration, the model is asked to read the prior reasoning trace and the tool feedback, then propose a revised prompt for itself. The next iteration runs against that revised prompt. No weights change. No rollout budget is spent on training. The same base model is used throughout.

We also benchmarked a third condition: a policy that was fine-tuned with Grouped Reinforcement Policy Optimization (GRPO) on the same task distribution. This is the expensive path, and we included it because we wanted to know whether the cheap intervention was competitive with the serious one.

The numbers

Across a held-out evaluation set of roughly two hundred tasks spanning multi-step retrieval, code modification, and structured data extraction, the gap was consistent:

  • The reflection-and-mutate loop beat fixed prompt-chaining by an average of roughly 10 percentage points on task success rate, with the largest individual task families showing gains closer to 20 points.
  • The reflection-and-mutate loop also beat the GRPO fine-tune, by a similar margin. The fine-tuned policy was the strongest baseline on a handful of task families, but it lost on aggregate and lost badly on tasks that required the agent to recover from an unexpected tool error.
  • The reflection loop achieved this while using up to 35 times fewer environment rollouts than the GRPO run. That ratio is the one that should make a cost-conscious engineering team sit up: most of the GRPO budget was spent discovering prompt revisions the model could have written for itself in a single inference pass.

The fine-tune's failure mode is worth describing because it generalises. GRPO optimises against the reward distribution it was trained on. When the deployed agent encountered a tool failure shape that did not appear in the training distribution, the fine-tuned policy had no gradient telling it what to do, so it repeated the failing action. The reflection loop had no such blindness: it could read the error, name it, and write a new instruction. Language-native self-correction turned out to be more robust than gradient-native self-correction, at least at the rollout budgets we could afford.

What we got wrong first

Our prior hypothesis was that the reflection step would help mainly on tasks where the agent got stuck in a repetitive failure loop, the classic "the model keeps calling the same broken tool" pathology. The data refused to support this. Reflection helped most on tasks where the agent was almost succeeding but was producing answers in the wrong schema, or reasoning correctly about a sub-problem but drawing the wrong conclusion at the final step. The mechanism, as best we can tell from reading traces, is not error recovery. It is prompt repair: the model is bad at writing a good prompt for itself on the first try, and it is much better at writing a good prompt once it has seen one attempt.

That distinction matters for product design. If the win is error recovery, you only need reflection on failure paths. If the win is prompt repair, you want reflection on every iteration, including the successful ones, because the second-attempt prompt is often better than the first even when the first attempt already answered the question.

Practical implications

Three lessons we are acting on.

First, invest in the reflection prompt. The quality of the meta-prompt that asks the model to revise its working prompt is the single biggest lever. A vague "reflect on what went wrong" instruction produces vague revisions. A specific instruction that names the kinds of failure to look for, and that asks for a diff rather than a rewrite, produced the strongest results in our ablations.

Second, budget for it. Reflection adds an extra inference per iteration. For our workload, where a single task averages four to six iterations, that is a real cost increase. We found it pays for itself many times over on tasks where the alternative is a human-in-the-loop escalation or a failed run that has to be re-queued, but the economics should be checked per workload.

Third, keep the loop readable. The most useful debugging artefact turned out to be the chain of revised prompts, not the chain of tool calls. When a task failed, we could read the prompt revision and immediately see whether the model had misunderstood the task, mis-modelled the tool, or simply run out of attempts. We now persist these revisions by default.

What this is not

We are not claiming that reflection replaces fine-tuning for every problem. There are task distributions where a well-trained policy is strictly better, and there are latency budgets where the extra inference is unaffordable. We are also not claiming the loop is finished: failure modes remain, particularly on long-horizon tasks where the model loses coherence after many revisions, and on tasks whose feedback signal is too sparse to guide a useful revision.

What we are claiming is narrower and we think more useful. For agentic systems that already run an iterative tool-use loop, adding a reflection-and-revise step is a high-leverage change that costs little to evaluate and, on our workloads, decisively beats both naive prompt-chaining and a serious reinforcement-learning fine-tune. If your team has not tried it, the experiment is cheap. The result, in our case, was a step change.