Optimizations
The service can reshape a request to make it cheaper. Each optimization is a per-key toggle, off or on independently, and each reports what it saved: the dashboard shows tokens and EUR per toggle, computed against the unoptimized estimate. Most leave the meaning of the request as it was; the ones that take content out say so below.
Caching#
| Toggle | Default | What it does |
|---|---|---|
cache_key | on | Derives a session key and sends it as prompt_cache_key (rdc- and 24 hex characters) when the request has none, so the prompt cache is hit across turns |
cache_breakpoints | on | On Anthropic-format requests, adds cache_control to the last system block and the last tool when the request carries no cache_control. It never goes past a limit of four breakpoints |
response_cache | off | Caches an exact request for ten minutes and answers a repeat at 10 percent of the price |
Sending your own x-session-id is the reliable way to keep a multi-process client on one cached prefix.
Tool output#
These act on tool results only. They run in this order: clean, compact, trim, whitespace.
| Toggle | Default | What it does |
|---|---|---|
clean_tool_output | off, recommended for coding agents | Removes what a terminal would not show: ANSI escape sequences, carriage-return redraws, and runs of three or more identical lines, which become the line plus a repeat marker when that is shorter |
trim_tool_output | off | A result longer than the limit (12,000 characters by default, 256 to 1,000,000) keeps its first 70% and last 20%, with an elision marker. Lines of the cut middle that report errors are kept in their place |
compact_json | off | Minifies a result that is a JSON object or array |
trim_whitespace | off | Collapses runs of blank lines and trailing spaces outside code blocks |
What these never touch: reasoning and thinking blocks, tool call arguments, and text with citations.
Files keep their bytes. The result of a tool that reads files (by its name: Read, read_file, view, cat, open_file, get_file, and the Anthropic text editor) is left alone by clean_tool_output, compact_json and trim_whitespace: an agent later quotes that text in an exact-match edit, and an edit against reformatted text fails.
The current turn is sent whole. trim_tool_output cuts the output of shell tools (names with bash, shell, terminal, exec, command or run) as soon as it arrives. Any other result is sent whole while it belongs to the current turn, and cut only in later requests, once the model has seen it.
Context#
| Toggle | Default | What it does |
|---|---|---|
session_dedup | off | A paragraph of 240+ characters that already appears in an earlier message of the same request becomes one line: [… the same text appeared in an earlier message]. The last user message is never touched |
compact_tables | off | A tool result that is a JSON array of 5+ uniform, flat objects is rendered as a columnar table: the same rows in about a third of the tokens |
Guardrails#
These act on the request before the model sees it. Detection uses the service's own pattern set, so a guard adds no per-request latency. What a guard did is reported in x-token-notice.
| Setting | Options | What it does |
|---|---|---|
guard_injection | off (default), warn, block | Heuristic detection of instruction-override and jailbreak text. The first 16 KiB of each text is scanned. Warn annotates; block refuses the request with 400 injection_suspected, and only on the unambiguous heuristics |
guard_credentials | off (default), on | API-key-shaped strings (OpenAI, Anthropic, GitHub, Slack, AWS, Stripe, SendGrid, JWTs) are replaced with [REDACTED:<kind>] |
output_style | off (default), terse, action_first, less_code | A response-shaping instruction appended to the system text when the client sent none. Terse drops filler; Action first leads with a numbered next step; Less code asks for the smallest working change |
The guards run after the other optimizations, on the text as it will be sent. They judge the request and never rewrite an answer. The memory recall block gets the same pass; in block mode a block that trips an unambiguous heuristic is withheld, the request goes on without recall, and x-token-notice says so.
Defaults#
| Default | Effect |
|---|---|
Fit to the context window (fit_context) | Off by default. Shortens a request that nearly fills the window instead of refusing it. The model then sees less of the conversation |
| Reasoning effort | Sent when the request has none. Auto picks a level from the conversation: low for a short message with no tool results, high for a long conversation or a long message, medium otherwise |
| Max output cap | A ceiling on the output a model may generate |
| Fallback models | The models tried in order when the requested one cannot serve, at the price of the model that serves. Up to 3. A request's own list or X-Token-Fallback replaces it, and none turns it off |
Context window#
On a model whose context window is known, the output limit is lowered to what the estimated input leaves in the window, less a small margin, and never below 1 token. A request whose input alone fills the window is refused with context_length_exceeded before it is sent.
With fit_context on, an input over 95% of the window is shortened step by step until it is at 85% or under: tool results the model has already seen are cut to their start, end and error lines; older inline images are replaced by a note; the oldest turns are dropped, never the system prompt and never the last two user turns. A shortened request carries x-token-notice: request shortened to fit the context window.
Response cache#
response_cache caches only requests that are safe to repeat: an explicit temperature of 0 and no tools. The same request sent again on the same key within ten minutes is answered with x-token-cache: hit and billed at 10 percent of the price. A streaming client gets the event sequence it asked for, replayed from the stored answer. A request that involves the memory tools still goes to the model, because its answer depends on the vault.
The cache key is the whole request body, the model and the key's settings: an answer made for another request, or under settings the key no longer has, is never served. An answer cut short by its output limit is not stored, and neither is an empty one. Cache-Control: no-cache skips the lookup and stores the fresh answer; no-store keeps the exchange out of the cache.
A cache hit is guarded on the request that asks for it, so a key with guard_injection or guard_credentials on keeps both the cache and the guard.
Seeing the savings#
Every request records what each optimization saved, in tokens and EUR at the model's prices. saved_nano is the request's total, and the key's optimizations panel shows each toggle's figures for the current month, so a toggle that never pays off can be turned back off.