A Headroom alternative that does not guess

Headroom uses probabilistic compression on one developer's machine. Anyray is a Headroom alternative with deterministic compression, for the whole organization. We ran Anyray vs Headroom on the same Terminal-Bench 2 tasks.

Anyray vs Headroom: the short version

Terminal-Bench 2 · Claude Sonnet 5.5 · Harbor task verifier

MetricAnyrayHeadroom
Cost per solved task$0.228$0.395
Total cost68%140%
Wall clock6h 08m9h 21m
Average task cost81%169%
Tasks that cost less95%11%
Steps per task8.711.4
Cache busts824

Benchmarked on real-world tasks

Terminal-Bench 2 tasks on Claude Sonnet 5.5, graded by the Harbor task verifier.

Each solved task costs 42% less

We take the cost of every trial and divide it by the trials that passed. One solved task cost $0.228 with Anyray, $0.395 with Headroom and $0.367 with no tool.

The full bill is 51% lower

This is every trial added up, passed or failed, as a share of what the same trials cost with no tool. Anyray cost 68% of that. Headroom cost 140%, which is more than running Claude with no tool at all.

The same work finishes 34% sooner

This is the time for 50 tasks, from the average time per task. Anyray took 6h 08m, Headroom took 9h 21m and no tool took 7h 30m. The gap follows the step count further down: fewer model calls, less waiting.

An average task costs 52% less

The mean counts every task in full, so one long, costly task can pull it up. We show it as a share of the average task cost with no tool. An average task cost 81% of that with Anyray and 169% with Headroom.

Anyray saves on 95% of tasks

For each task we check if it cost less than the same task with no tool. Anyray cost less on 95% of tasks, Headroom on 11%. A high share means the saving does not depend on a few long tasks. It shows up on almost all of them.

The agent needs 24% fewer steps

A step is one model call, and every call sends the session again. Fewer steps means less to pay for and less to wait for. The agent finished a task in 8.7 steps with Anyray, 11.4 with Headroom and 11.4 with no tool.

3× fewer cache busts

A cache bust is a request that changes the start of the session. The provider then bills the whole context again at the full input price, instead of the cheap cache rate. Anyray had 8 per 100 trials. Headroom had 24, more than with no tool. No tool had 16.

How Headroom and Anyray differ

Headroom is a good fit for one developer who wants an open-source engine and accepts probabilistic results. Anyray is built for an organization that wants lower agent cost, spend by team and governance in one place.

  • Headroom

    Headroom is an open-source compression engine. A developer runs it on one machine. It covers tool outputs, logs, files and RAG chunks with probabilistic compression, so the savings vary and a quality loss can go unseen. It sits inline in the request path, and it adds round trips and a local model load.

  • Anyray

    Anyray is a self-hosted gateway on the path of every LLM request an organization makes. It works on the whole request, not on the output of one tool. Compression is deterministic, with bounded quality guardrails. One setup enrolls the whole organization and shows spend by user, team and project. It fails open: if the optimizer is slow or down, requests pass through untouched.

CapabilityAnyrayHeadroom
CoverageThe whole requestTool outputs, logs, files, RAG chunks
CompressionDeterministicProbabilistic
Quality safetyBounded guardrailsWeight-based, quiet fails
ReliabilityFail-openInline dependency in the request path
OverheadExtra HTTP hopRound trips, local model load
AnalyticsBy user, team and projectMetrics and savings
GovernanceAuth, access controlNone
EnrollmentOrganization-widePer machine
Best fitOrganizationsIndividual developers

Why the numbers differ

A cache bust is costly, because the provider bills the whole context again at the full input price. In this benchmark, Headroom had 24 cache busts per 100 trials, against 16 with no tool and 8 with Anyray. Its total bill was 140% of the bill with no tool, and only 11% of tasks cost less. Anyray compresses the same way each time, so the cache stays warm and the bill was 68%.

Headroom alternative: common questions

What is Headroom?

Headroom is an open-source compression engine for AI agents. It covers tool outputs, logs, files and RAG chunks with probabilistic compression. It runs on one developer's machine.

What is the difference between Anyray and Headroom?

Headroom uses probabilistic compression on one machine, so the savings vary. Anyray is a self-hosted gateway with deterministic compression and bounded quality guardrails for every agent in the organization. Anyray also gives spend analytics, access control and fail-open reliability.

Which costs less, Anyray or Headroom?

On Terminal-Bench 2 with Claude Sonnet 5.5, one solved task cost $0.228 with Anyray and $0.395 with Headroom. The total bill was 68% of the no-tool bill with Anyray and 140% with Headroom.

How was the Anyray vs Headroom benchmark run?

Each tool ran the same Terminal-Bench 2 tasks with Claude Sonnet 5.5. The Harbor task verifier graded each trial. The harness, the scenarios and every result are open source on GitHub.

How does a team move from Headroom to Anyray?

Anyray needs no code changes. Agents send their requests to the Anyray gateway in place of the provider URL. One setup covers the whole organization, so there is no engine to run on each machine.

Learn more

Run the benchmark yourself.

The harness, the scenarios and every result are open source. Point it at your own key and compare Anyray and Headroom on your own tasks.

View on GitHub

Ready to run AI without the waste?