Most write-ups about reflection loops in LLM agents celebrate the iteration: critic scores the draft, model revises, score goes up, ship it. The contrarian worry in our team was never whether loops work. It was whether we could tell, at runtime, when a loop had stopped helping. A loop that fools the operator faster than the operator can notice is, from an engineering standpoint, a metering failure dressed up as a quality win.
We had a long-running multi-agent system with a critic/verifier loop, a cost guard, and an eleven-stage middleware chain that already enforced budgets, validation, and verification on most paths. We assumed that wall was enough. It was not. The gap we eventually named was small in surface area and large in consequence: there was no reusable component that asked "is this iteration actually improving the output, or are we burning tokens to feel productive?" before allowing the next round.
The pattern that was missing
In critic-agent designs, three primitives tend to appear. The critic produces a score. The loop applies feedback and resamples. Some kind of improvement detector asks, between rounds, whether the new draft is materially better than the previous one. When that detector is missing, the loop defaults to either a fixed iteration count or, worse, a "score went up by epsilon, keep going" rule. Both of those are ways for the loop to look like it is converging while the underlying output drifts.
What we had built was a cost guard and a verifier. The cost guard answers "have we exceeded budget?" The verifier answers "does this satisfy structural constraints?" Neither of those is an improvement detector. The verifier, especially, is a poor substitute: it can be satisfied on the first iteration and stay satisfied forever, which is indistinguishable from a loop that is genuinely improving and one that converged on a mediocre output in round one and is now just polishing.
A hypothesis we disproved
The natural hypothesis in this situation is that the verifier chain, because it already runs on every candidate output, is doing the work of an improvement detector as a side effect. If structural checks pass, the loop must be converging. We pushed on this directly. It is false.
The falsification did not require exotic instrumentation. We instrumented loops to record, per iteration, the verifier pass status, the critic score, the delta in critic score against the previous round, and the token spend per round. We then ran a fixed batch of tasks where we knew, from human-rated ground truth, what a good final answer looked like.
Three things fell out. First, on tasks the verifier passed in round one, the critic score frequently oscillated without trending upward across rounds two through five. The verifier was a flat signal: passed or did not pass, with no sensitivity to "is this getting better." Second, token spend per round was roughly constant; a loop that is genuinely refining spends more on the first refinement and tapers as it converges, or at least shows a recognizable shape. Our loops were flat. Third, on a meaningful slice of tasks, the highest-scoring output was the round-one draft. Subsequent rounds added cost without adding quality, and on a small but real fraction they actually degraded quality, presumably because the critic's feedback steered revisions toward locally pleasing phrasing rather than correctness.
The disproof is the part worth keeping. The verifier was not secretly doing the job. It was a different job.
What the gate has to do
Once we named the gap, the requirements for the missing gate were straightforward to write and hard to build. The gate needs a per-task score history, not a single round's score. It needs a notion of improvement that is robust to the noise we actually saw in the critic, which means looking at the delta over a short window rather than the last round alone. It needs to be cheap; if the gate costs more than the round it is gating, the design is incoherent. And it needs a clear exit condition that the rest of the pipeline can rely on, because every caller we anticipated wanted a binary "continue or commit."
We also had to decide what the gate does when the signal is ambiguous. The default we kept reaching for was "keep going," because that is what the existing loop does. That default is the failure mode. When the gate cannot tell whether the loop is improving, the right answer is usually to commit the best draft seen so far and stop, not to spend another round to find out.
What to take back to your own stack
If you run any flavor of critic/refiner loop, three checks are worth running before you trust the loop's output. Look at score deltas across rounds, not the final score. Look at per-round token spend, and check whether it has the shape of convergence or the shape of a flat-rate subscription. And treat any single-pass verifier as orthogonal to improvement detection, because, by construction, it is.
The generalisable lesson is not "add an improvement detector." Most teams that hear that advice will bolt one on and call it done. The lesson is that improvement detection is a distinct component from verification and from cost control, and conflating any two of them produces a loop that looks rigorous while quietly wasting budget. Naming the gap is the engineering work. Wiring the gate is the easy part after that.
A confidence gate that scores whether refinement is actually improving output is the smallest piece of a critic loop, and the easiest to skip, and the one whose absence costs the most per day it is missing.