I spend most of my time looking for ways to keep LLM pipelines from quietly bleeding money. A while back I started digging into a token compression tool that claims 60–95% reductions on natural text — the kind of numbers that, if real, would matter for any agent system that shoves market data, search hits, or long documents into a context window. The investigation itself was unremarkable. What came out of it is worth talking about, because the failure shape is something most teams will recognise.
What the tool claimed
The compressor exposes three integration modes: a library you can call inline, an HTTP proxy you can put in front of a model endpoint, and an MCP server you can register as a tool. That last one is what caught my attention. In a multi-agent stack, the cheapest way to adopt almost anything is to register it as a callable instrument — the agent decides when to invoke it, the orchestration layer tracks it, and the cost shows up in the same accounting as everything else. No bespoke plumbing, no shadow path through the system.
The pitch was straightforward: run your text through the compressor before it hits the model, pay a small latency tax, save a large fraction of the token bill. On a workload that is mostly high-volume market data and memory recall results — the kind of stuff that tends to be repetitive, boilerplate-heavy, and structurally predictable — the claimed reductions sounded plausible. Plausible is not the same as measured.
What we actually saw
The interesting result was not in the wins. It was in the cases where the compressor failed, and what those failures looked like.
The pattern I want to flag is this: compression is not lossy in the way people assume, and the lossy part is not where you expect it. Most engineers, when they hear "token compression," mentally model it as aggressive summarisation — things get dropped, but the things that remain are at least faithful. What the data showed is closer to the opposite. On structured inputs — order book snapshots, JSON payloads, anything with strict schema — the compressor frequently produced output that looked like the input and parsed like the input, but contained values that were subtly wrong. A field that had been an integer became a string. A timestamp drifted by an order of magnitude. A price moved enough to be a real trade if an agent downstream acted on it without re-checking.
The failure mode is not "the model hallucinates." It is "the compression step produced a payload that passes every structural check an agent would normally run, and is wrong anyway." That is a much harder class of bug to catch, because the defensive code most teams write — schema validation, type checks, length bounds — is exactly the code that passes these payloads through.
For an agent system, the consequence is worse than for a single LLM call. If an agent invokes a tool, gets back a compressed result, and then either feeds it to another agent or uses it to populate a downstream action, the corruption travels. By the time anything notices, the original source is two or three hops away and nobody can say which compressed payload introduced the bad value.
What was disproved
The hypothesis I went in with was simple: a high compression ratio implies high information density, and information density is roughly the same thing as utility for downstream agents. That is wrong, or at least much weaker than it sounds. Compression ratio measured in tokens per byte is not compression ratio measured in safe-to-act-on facts per byte. The two diverge sharply on structured data, and the divergence is invisible until something downstream breaks.
The secondary hypothesis — that the lossiness would show up in long-tail, hard-to-summarise inputs — was also disproved. The failures clustered on the easy inputs. Orderly, repetitive, highly patterned text is exactly what the compressor is best at reshaping, and it is also where the silent corruption was most likely to land, because the compressor had the most freedom to "optimise" phrasing without tripping any obvious alarm.
What this changes in practice
The generalisable rule I took away: treat any preprocessing step that sits between a trusted source and an LLM as a potential untrusted source. Schema validation, type checks, and "looks plausible" heuristics are not a substitute for re-grounding critical values against the original. For high-stakes fields — prices, identifiers, counts, timestamps — the safe pattern is to pass them through the compressor for the model's context but keep an authoritative copy outside the compressed channel, and reconcile before any action is taken.
The second rule is about integration shape. The MCP-server-style registration that made the tool attractive is exactly what makes this kind of bug dangerous. A preprocessor that sits inside the agent's tool registry inherits the agent's trust. If you cannot prove the preprocessor is faithful on the specific shapes your agents handle, do not let its output be the only thing the agent sees.
I have not measured the same compressor on every workload, and I am not claiming it is unfit for purpose. For free-form prose summarisation, the savings may well be real and the risks bounded. For structured market data flowing into autonomous actions, the failure shape above is enough to keep it out of the hot path until the faithfulness story is solved. A 70% token saving that introduces a 0.1% silent corruption rate into an action loop is not cheaper than full-price tokens; it is just an audit problem with extra steps.