The first objection to context compression is always the same: what if it deletes something my agent needs?
It is a fair question. A compressor that throws content away bets that the model will never need it. When that bet loses, the agent gets confused, reads the file again, or gives a worse answer. You save tokens on one turn and lose them, plus some quality, on the next.
Anyray does not make that bet. When an edit removes content, it keeps the original and leaves a way back to it. Removed is not lost. It is one tool call away.
Three kinds of edit
Not every edit needs a way back. The engine’s transforms fall into three groups by what happens to the original:
- Lossless. The edit can be reversed byte for byte. A later copy of a file points at the first copy, which stays in the request. Nothing is gone.
- Exact fields. The edit cuts filler, such as long tool descriptions, but keeps types, enums and required fields byte for byte. Every tool stays callable.
- Retrievable. The edit removes real content, such as old tool output or the noise inside a large log. The original is stored, and the agent can fetch it back.
This post is about the third group. It is where the large savings are, and where the risk would be without a way back.
What the agent sees
Take an access log the agent pulled ten turns ago. It is large, and most of it is healthy requests. On a later turn, a Distill transform keeps only the lines that match the latest question and replaces the rest with markers.
Each marker says what was removed, how large it was, and gives a handle such as ctx_5a. The model reads the marker like any other text. It knows something was cut, and it knows how to get it.
The round trip
When the agent needs the removed lines, it calls one tool, anyray_retrieve, with the handle. The gateway returns the exact original bytes.
This turns most lossy edits into deferred reads. The agent pays for the full content again only on the turns that use it. On every other turn, it carries a short marker in place of thousands of lines.
For your own tool loop, the same read-back is a plain HTTP call, POST /v1/retrieve with the handle. Anyray Connect gives Claude Code, Codex and Cursor the retrieve tool without any setup on your side.
Only cut what can come back
A marker is only safe if the caller can follow it. So the engine checks first. A client that cannot restore a handle gets only self-contained edits, which need no read-back. Retrievable edits run only where the retrieve tool is in reach.
Some content is never cut at all, retrievable or not:
- The new input on the current turn.
- The output of a failed tool call, because the error text is what the agent needs next.
- A file the agent is editing, so edits always see real bytes.
- The user’s own messages, which are never shortened by relevance ranking.
- Tool call structure, so the provider always gets a valid request.
Who can read a handle
A stored original is your content, so a handle is treated like the content it points to. A handle is bound to the user whose key created it. It resolves for that user’s keys and their sub-agents. Anyone else gets a 404.
Originals live in bounded memory, and each strategy sets how long they live. When durable recall is on, they can also be stored encrypted with AES-256-GCM under your deployment’s content mode. In off mode, nothing is written.
When read-backs pile up
Read-backs are also a signal. One fetch means the model needed that content. Many fetches mean a strategy is cutting too much for this workload.
The task-outcome guard watches for that. It counts regret per strategy: a retrieve of an elided original, a tool re-run after its result became a marker, or a request retried after a crop. When the regret rate for one strategy is significantly above its threshold, the guard turns that strategy off for that client tool in that tenant.
After a cool-off, the guard probes the strategy on a share of sessions. If the probes show the rate is back down, the strategy returns. The guard counts sessions, not edits, so one confused session cannot shut a strategy off for everyone. Inside a session, a fetch of removed code also makes the engine remove less code for the rest of that session.
What this means for you
Compression without a way back forces a choice between savings and quality. A handle removes that choice. The engine can be bold with old, bulky content, because a wrong cut costs one tool call, not a wrong answer.
If you want to see it in your own traffic, look at two numbers: how often your agents fetch content back, and which strategies those fetches come from. Both are in the decision record of each request, with no prompt content.
Where to go next
- Guardrails docsReversibility, handles and the regression guard, in detail.
- Why context compression breaks prompt cachingThe other half of safe compression: the cached prefix.
- Fail-open by designWhat happens when any layer has a bad day.