You moved Claude Code onto your company gateway. Requests flow, spend shows up on the dashboard, and the security team is happy. Then the complaints start: the context meter sits at 100%, and Claude Code compacts the conversation long before a 1M-token model should be full.
Nothing is broken. Claude Code is doing what it does behind any custom base URL. This post explains why, and which settings fix it.
The 200k context trap
Claude Code looks up a model’s real context window only when it talks to api.anthropic.com directly. Behind any other base URL, it cannot be sure what model sits on the other side, so it uses a flat budget of 200,000 tokens. It auto-compacts at about 95% of that budget.
On a model with a 1M-token window, that is 800,000 tokens of context you paid for and never get. The session does not fail. It summarizes itself early, again and again, and the agent loses detail it would have kept.
The symptom is easy to spot once you know it. The context meter reads 100% while the model still has most of its window free, and /compact runs on its own far sooner than it should.
How Claude Code picks its endpoint
Two environment variables send Claude Code to a gateway:
ANTHROPIC_BASE_URLreplaceshttps://api.anthropic.comas the endpoint for every model call.ANTHROPIC_AUTH_TOKENis sent as the bearer token. The gateway checks it, then calls the provider with its own credentials.
Most companies end up here for the same three reasons. Governance: one place to set which models, providers and regions are allowed. Cost visibility: spend per developer, team and tool, not one invoice line. One key: developers never hold a provider key, and the gateway holds the keys for Anthropic, Bedrock or Vertex.
LiteLLM, an internal proxy and Anyray all work this way. They all hit the 200k trap, because Claude Code sees a base URL that is not Anthropic’s.
The fix, one setting at a time
1. _CLAUDE_CODE_ASSUME_FIRST_PARTY_BASE_URL
This variable tells Claude Code to treat the custom base URL as if it were Anthropic’s own API. Claude Code then uses its per-model context windows again, so a 1M model gets its 1M budget.
{
"env": {
"ANTHROPIC_BASE_URL": "https://gateway.example.com",
"_CLAUDE_CODE_ASSUME_FIRST_PARTY_BASE_URL": "1"
}
}
2. [1m] on the model id
Some models reach 1M tokens only through Claude Code’s long-context option, not by default. For those, add [1m] to the end of settings.model:
{ "model": "claude-sonnet-4-5[1m]" }
A model with a native 1M window does not need the suffix.
3. ENABLE_TOOL_SEARCH
The first flag has a side effect: it turns tool search on for every session. With tool search, Claude Code sends a short index of MCP tools and loads a full schema only when it needs one. That is a win when the tool list is large, and a loss when it is small.
In our paired runs, tool search cost 33% more with a few MCP tools and saved 33% with many. So set a threshold instead of leaving it always on. auto:N turns tool search on only when the tool schemas take more than N% of the model’s context window.
We recommend auto:4 on 1M-token models and auto:20 on all other models:
{
"env": {
"_CLAUDE_CODE_ASSUME_FIRST_PARTY_BASE_URL": "1",
"ENABLE_TOOL_SEARCH": "auto:4"
}
}
Side effects to know about
The flag makes Claude Code believe it talks to Anthropic’s own API. Two things follow from that, and your gateway must handle both.
First-party-only tools
Claude Code starts to offer tools that only Anthropic’s API can run, such as WebSearch. If your route reaches only Bedrock, those tools can never work, but their schemas still go with every request. Deny them:
{
"permissions": {
"deny": ["WebSearch"]
}
}
A cache mark your key cannot use
Claude Code also adds a prompt-cache scope value to its system prompt. Only a subscription sign-in may send it. On an org API key, the provider rejects every request with a 400:
system.2.cache_control.ephemeral.scope: Extra inputs are not permitted
The gateway must strip that field before it forwards the request. If your gateway does not, turning on the flag breaks every call.
_CLAUDE_CODE_ASSUME_FIRST_PARTY_BASE_URL, and the action for each.Where to set the values
Claude Code reads the env block of its settings files. Pick the file by who owns the value:
~/.claude/settings.jsonis your user settings. Use it on your own machine..claude/settings.jsonin a repository is the project settings. It is committed, so every teammate in that repository gets it.managed-settings.jsonis for a fleet. IT puts it in a system path with MDM:/Library/Application Support/ClaudeCode/on macOS,/etc/claude-code/on Linux, andC:\Program Files\ClaudeCode\on Windows.
When two places set the same variable, the managed file wins over everything: command-line arguments, project settings, user settings and shell exports. Then the command line wins over project settings, and project settings win over user settings.
This matters in a rollout. If the managed file sets its own ANTHROPIC_BASE_URL, no value in a user file changes where requests go. A developer can edit ~/.claude/settings.json all day and see no effect.
Troubleshooting
A 401 loop after a config change
A terminal that was open before the change still exports the old variables. Claude Code keeps sending requests to the old gateway with a credential that no longer works, while every file on disk looks correct. Clear that terminal:
unset OPENAI_BASE_URL ANTHROPIC_BASE_URL ANTHROPIC_AUTH_TOKEN OPENAI_API_KEY \
ENABLE_TOOL_SEARCH _CLAUDE_CODE_ASSUME_FIRST_PARTY_BASE_URL
Or open a new terminal.
No change after you edit the file
Claude Code reads its environment once, at startup. So does VS Code, and it gives that environment to every terminal it opens. Quit Claude Code fully and start it again. In VS Code, use Cmd+Q. Reload Window is not enough.
How Anyray does this for you
If you route Claude Code through Anyray, you do not set any of this by hand. anyray-connect writes the flag, the [1m] suffix and the tool search threshold for each model. It keeps any value that you, your admin or a teammate already set, and --revert removes only the values it wrote. The gateway strips the cache scope mark on every route that cannot use it, so org API keys do not get the 400.
Details are in the docs FAQ and Desktop apps and coding tools.
FAQ
What does _CLAUDE_CODE_ASSUME_FIRST_PARTY_BASE_URL do?
It tells Claude Code to treat a custom ANTHROPIC_BASE_URL as Anthropic’s own API. Claude Code then uses each model’s real context window instead of a flat 200k budget. It also turns on tool search, offers first-party-only tools and adds a cache scope mark, so set a tool search threshold and make sure your gateway strips the mark.
Why does Claude Code auto-compact at 200k behind a proxy?
Claude Code looks up a model’s context window only when it talks to api.anthropic.com. Behind any other base URL, it budgets 200,000 tokens and auto-compacts at about 95% of that, whatever the model can hold. Set _CLAUDE_CODE_ASSUME_FIRST_PARTY_BASE_URL to restore the real window.
Is it safe to set the flag on Bedrock?
Yes, if your gateway handles two side effects. Deny the first-party-only tools, such as WebSearch, because Bedrock cannot run them. And make sure the gateway strips the cache scope mark. Without that, requests on an API key fail with a 400.
Where to go next
- How we detect prompt-cache breaksFind the client change that breaks your cache.
- Fail-open by designWhat happens to your agents when the gateway is down.
- Anyray FAQClaude Code, desktop apps and routing questions.