Same input, same bytes
Compression is a pure function of each message. The provider sees the same bytes on every turn, so the cached prefix keeps matching.
We study how to remove context from agent requests without breaking the prompt cache, the agent or the answer. Then we measure it in dollars, in the open.
Removing tokens is not the hard part. The hard part is removing them so the session bill goes down and the agent still works.
Compression is a pure function of each message. The provider sees the same bytes on every turn, so the cached prefix keeps matching.
Each edit is priced against the provider’s cache rules. An edit ships only when it lowers the bill over the rest of the session.
Removed content stays one tool call away, byte for byte. Most lossy edits become deferred reads.
A decision engine picks edits per request, checks them against your own traffic and backs off when the model reacts badly.
Terminal-Bench 2 tasks on Claude Sonnet 5.5, graded by the Harbor task verifier.
We take the cost of every trial and divide it by the trials that passed. One solved task cost $0.228 with Anyray and $0.367 with Claude direct.
This is every trial added up, passed or failed, as a share of what the same trials cost with Claude direct. Anyray cost 68% of that, and it cost less on 95% of the tasks.
This is the time for 50 tasks, from the average time per task. Anyray took 6h 08m and Claude direct took 7h 30m. The gap follows the step count further down: fewer model calls, less waiting.
A step is one model call, and every call sends the session again. Fewer steps means less to pay for and less to wait for. The agent finished a task in 8.7 steps with Anyray and 11.4 with Claude direct.
A cache bust is a request that changes the start of the session. The provider then bills the whole context again at the full input price, instead of the cheap cache rate. Anyray had 8 per 100 trials. Claude direct had 16.
Evaluation-driven compression across workloads
By function
38%
30%
68%
79%
By workload
| workload | saved |
|---|
| Access logcompacting repetitive entries down to the relevant few | |
| Incident debuggingkeeping only the signals that point to the cause | |
| Synonym-gap logsfinding an OOM event the question never names | |
| GitHub issuesreading GitHub issues/comments to categorize/prioritize | |
| JSON arraykeeping only the fields and items the task uses | |
| Code searchgrep/semantic search returning many file+line hits | |
| Git diff outputreviewing what changed in a commit or PR | |
| Test-suite outputkeeping only the failed tests and the lines that explain them | |
| Flaky-suite re-runcollapsing a near-duplicate test re-run to what changed | |
| Codebase explorationreading source files to understand architecture | |
| Read a Python modulekeeping only the function bodies on the task path | |
| Read a large TS filekeeping only the code paths a TypeScript service task touches | |
| Multi-file tracefollowing one call path across several files | |
| Recurring reportremoving repeated instructions across a nightly batch | |
| RAG retrievalkeeping only the chunks that answer the question | |
| MCP tool bloatkeeping only the tool schemas the task calls |
The harness, the scenarios and every result are open source. Point it at your own key and compare the two arms on your own tasks.
anyrayHQ/benchmarks