Each solved task costs 42% less
We take the cost of every trial and divide it by the trials that passed. One solved task cost $0.228 with Anyray, $0.395 with Headroom and $0.367 with no tool.
Headroom uses probabilistic compression on one developer's machine. Anyray is a Headroom alternative with deterministic compression, for the whole organization. We ran Anyray vs Headroom on the same Terminal-Bench 2 tasks.
Terminal-Bench 2 · Claude Sonnet 5.5 · Harbor task verifier
| Metric | Anyray | Headroom |
|---|---|---|
| Cost per solved task | $0.228 | $0.395 |
| Total cost | 68% | 140% |
| Wall clock | 6h 08m | 9h 21m |
| Average task cost | 81% | 169% |
| Tasks that cost less | 95% | 11% |
| Steps per task | 8.7 | 11.4 |
| Cache busts | 8 | 24 |
Terminal-Bench 2 tasks on Claude Sonnet 5.5, graded by the Harbor task verifier.
We take the cost of every trial and divide it by the trials that passed. One solved task cost $0.228 with Anyray, $0.395 with Headroom and $0.367 with no tool.
This is every trial added up, passed or failed, as a share of what the same trials cost with no tool. Anyray cost 68% of that. Headroom cost 140%, which is more than running Claude with no tool at all.
This is the time for 50 tasks, from the average time per task. Anyray took 6h 08m, Headroom took 9h 21m and no tool took 7h 30m. The gap follows the step count further down: fewer model calls, less waiting.
The mean counts every task in full, so one long, costly task can pull it up. We show it as a share of the average task cost with no tool. An average task cost 81% of that with Anyray and 169% with Headroom.
For each task we check if it cost less than the same task with no tool. Anyray cost less on 95% of tasks, Headroom on 11%. A high share means the saving does not depend on a few long tasks. It shows up on almost all of them.
A step is one model call, and every call sends the session again. Fewer steps means less to pay for and less to wait for. The agent finished a task in 8.7 steps with Anyray, 11.4 with Headroom and 11.4 with no tool.
A cache bust is a request that changes the start of the session. The provider then bills the whole context again at the full input price, instead of the cheap cache rate. Anyray had 8 per 100 trials. Headroom had 24, more than with no tool. No tool had 16.
Headroom is a good fit for one developer who wants an open-source engine and accepts probabilistic results. Anyray is built for an organization that wants lower agent cost, spend by team and governance in one place.
Headroom is an open-source compression engine. A developer runs it on one machine. It covers tool outputs, logs, files and RAG chunks with probabilistic compression, so the savings vary and a quality loss can go unseen. It sits inline in the request path, and it adds round trips and a local model load.
Anyray is a self-hosted gateway on the path of every LLM request an organization makes. It works on the whole request, not on the output of one tool. Compression is deterministic, with bounded quality guardrails. One setup enrolls the whole organization and shows spend by user, team and project. It fails open: if the optimizer is slow or down, requests pass through untouched.
| Capability | Anyray | Headroom |
|---|---|---|
| Coverage | The whole request | Tool outputs, logs, files, RAG chunks |
| Compression | Deterministic | Probabilistic |
| Quality safety | Bounded guardrails | Weight-based, quiet fails |
| Reliability | Fail-open | Inline dependency in the request path |
| Overhead | Extra HTTP hop | Round trips, local model load |
| Analytics | By user, team and project | Metrics and savings |
| Governance | Auth, access control | None |
| Enrollment | Organization-wide | Per machine |
| Best fit | Organizations | Individual developers |
A cache bust is costly, because the provider bills the whole context again at the full input price. In this benchmark, Headroom had 24 cache busts per 100 trials, against 16 with no tool and 8 with Anyray. Its total bill was 140% of the bill with no tool, and only 11% of tasks cost less. Anyray compresses the same way each time, so the cache stays warm and the bill was 68%.
Headroom is an open-source compression engine for AI agents. It covers tool outputs, logs, files and RAG chunks with probabilistic compression. It runs on one developer's machine.
Headroom uses probabilistic compression on one machine, so the savings vary. Anyray is a self-hosted gateway with deterministic compression and bounded quality guardrails for every agent in the organization. Anyray also gives spend analytics, access control and fail-open reliability.
On Terminal-Bench 2 with Claude Sonnet 5.5, one solved task cost $0.228 with Anyray and $0.395 with Headroom. The total bill was 68% of the no-tool bill with Anyray and 140% with Headroom.
Each tool ran the same Terminal-Bench 2 tasks with Claude Sonnet 5.5. The Harbor task verifier graded each trial. The harness, the scenarios and every result are open source on GitHub.
Anyray needs no code changes. Agents send their requests to the Anyray gateway in place of the provider URL. One setup covers the whole organization, so there is no engine to run on each machine.
The harness, the scenarios and every result are open source. Point it at your own key and compare Anyray and Headroom on your own tasks.
View on GitHub