01Estimate your savings
Engineers whose assistants, agents, or jobs hit an LLM.
Provider spend per dev (tokens across all models).
Estimate only. Actual savings depend on your workload mix, prompt sizes, and cache hit rate — run a demo to measure against real traffic.
02How the estimate works
The math is deliberately simple so you can sanity-check it:
- Current annual spend = developers × monthly spend per developer × 12.
- Estimated savings = current spend × savings rate.
- Optimized spend = current spend − savings.
The savings rate is the one assumption worth scrutinizing. Anyray's real-time optimizer typically removes around 60% of inference cost, but the honest number for your org depends on how much redundant context your agents send and how repetitive your traffic is. The slider lets you model a conservative floor.
03Where the savings come from
Inference cost is just tokens × per-model price. A gateway on the path of every request drives that down four ways, deterministically and with no change to your model:
| Lever | What it does | Why it saves |
|---|---|---|
| Prompt & context compression | Trims redundant context before it reaches the provider | Coding agents resend large context every turn — most of it is repeated |
| Tool pruning | Drops tool definitions the request won't use | Unused tool schemas are billed as input tokens on every call |
| Semantic cache | Serves repeated or near-identical calls from cache | Agent loops and CI jobs repeat the same calls constantly |
| Routing | Sends each call to the cheapest capable provider | The same request can cost very differently across providers |
Compression is deterministic and bounded by quality guardrails, and the gateway fails open — if the optimizer is slow or down, requests pass through untouched. Prompt and response content stays in your environment; it's encrypted at rest and never logged in plaintext.
04Assumptions & method
- The default 60% savings rate reflects Anyray's typical result; it is not a guarantee. Model your own floor with the slider.
- Spend is treated as fully addressable — in practice a small share of calls may be exempt from optimization by policy.
- Figures are provider inference spend only and exclude the cost of self-hosting the gateway, which is typically a small fraction of the savings.
- Savings are recurring: the annual figure repeats every year the traffic continues.
05See it on real traffic
- Get a demo — point one URL at the gateway and watch live traffic get cheaper.
- Anyray vs. alternatives — how it compares to other token-reduction tools.
- How the optimization library works in the docs.