LLM Context Token-Budget Admission: Reservations, Degradation Ladder, and Rejection Contract

A model supporting a long context window does not mean every request should be admitted to fill it. Template expansion, tool schemas, history, retrieved evidence, and output limits all consume capacity. Waiting for the provider to return a context-overflow error and then truncating the end of a string may remove exactly the authorization restriction or disconfirming evidence that matters. Prompt Engineering Systems define instruction content, while Memory Compression and Consolidation manage persistent memory. This article rewrites neither; it defines only the admission and degradation transaction for production gateways. Context Engineering explains what information should be selected, while Context Budgets for Coding Agents discusses buckets for repository evidence. It also does not replace KV Cache Admission: that protocol commits GPU blocks, while this one commits the logical tokens sent to the model.

Request a budget for the final rendered payload

Client-reported length is useful only as an early warning. The gateway must lock the modelRevision, tokenizerDigest, promptBundleDigest, toolset, and output schema, render the actual messages, and then invoke the counter for the target model or use a tokenizer whose consistency has been validated. Google Gemini's token-counting and long-context documentation describe its counting API and window constraints, but billing, system content, and multimodal conversion differ by provider, so a single heuristic character ratio cannot be shared across them.

{
  "requestId": "req-74d1",
  "tenantId": "t-19",
  "taskClass": "contract-review",
  "modelRevision": "model@2026-07-01",
  "tokenizerDigest": "sha256:2f4a...",
  "promptBundleDigest": "sha256:80c1...",
  "policyRevision": 18,
  "buckets": {
    "immutableRules": 1460,
    "trustedState": 620,
    "evidence": 11840,
    "recentTurns": 3940,
    "tools": 2110,
    "maxOutput": 1600
  },
  "estimatorRevision": "admission-v6"
}

immutableRules, authorization scope, output safety constraints, and the current question are hard buckets and must never be degraded. Evidence records source IDs, ranking scores, and a minimum-retention set. History records turn boundaries and state provenance; it must not truncate away half of a tool call. The output budget is the maximum commitment capped server-side by task class, rather than a value taken directly from user input. The version isolation described in Tokenizer Migration applies equally to length caches.

Admission uses the states PROPOSED -> RESERVED -> DISPATCHED -> RUNNING -> PARTIALLY_SETTLED -> SETTLED. Rejections enter REJECTED, while unknown external results enter RECONCILING. The same transaction reserves input, output, and concurrency capacity by tenant, task class, and model pool. Requests with the same idempotency key but different rendered digests must conflict. A reservation may be released in full only when the provider call has not yet been dispatched and authoritative evidence confirms that fact. A successful dispatch confirms input consumption and reuses a stable provider request ID.

Streaming usage is captured idempotently by (requestId, usageEventId), atomically increasing captured usage and decreasing remaining output while preserving captured + remaining = reserved + topUps. Actual usage is settled and the remaining output reservation released only after final usage arrives or an authoritative terminal state is obtained through a provider query or billing reconciliation. Cancellations, timeouts, and failures first settle usage already incurred. Unknown results remain frozen in RECONCILING and must not be released as zero usage. After a lease expires, only a reclaimer with a higher fence may take over reconciliation; if non-execution cannot be proven, the reservation must not be released. Tenant Quota Isolation governs aggregate capacity, while this protocol provides atomic details for a single call.

Degradation is a fixed ladder, not ad hoc trimming

A policy version declares a candidate ladder for each task: FULL retains all eligible evidence; DEDUP_EVIDENCE removes only lower-ranked copies with identical content digests and matching permissions; COMPACT_HISTORY uses a request-specific compression artifact that has passed the fidelity gate; NARROW_WORKFLOW disables nonessential tools or switches to a clarification-oriented workflow; the final rung is QUEUE_OR_REJECT. Switching to a smaller model is a valid rung only when the compatibility matrix for capabilities, data residency, output schemas, and safety policies passes in every dimension. A capacity failure must not be disguised as randomized model routing.

At every rung, the gateway rerenders the structured components and recounts them, recording the IDs of removed objects, the reason for removal, and the token counts before and after. If the minimum evidence set still does not fit, return the machine-decidable CONTEXT_BUDGET_EXCEEDED and recommend narrowing the scope or processing it asynchronously; never truncate silently. Cost-aware retrieval can reduce the candidate set first, but relevance scores from Cost-Aware Retrieval for RAG cannot override admission-layer permissions or minimum-evidence constraints.

Validate estimation and ordering with fault injection

Fixed fixtures cover CJK, code, multimodal components, enormous tool schemas, duplicate evidence, and multi-turn conversations close to the context window. Inject tokenizer hot swaps, counting-service timeouts, tools added after rendering, two requests concurrently contending for the same tenant balance, cancellation before dispatch, streaming cancellation, missing final usage, and lost settlement responses. Assert that count caches from an old digest are never reused; when counting is unavailable, the system conservatively rejects the request or uses an approved upper bound without degrading hard buckets. Usage retries must not capture twice, cancellation before dispatch must release the full reservation, cancellation after dispatch must settle actual consumption, and unknown results must remain frozen until reconciliation.

Metrics must include request counts at each ladder rung, deviation between estimated and actual usage, reservation leaks, provider context-overflow errors, hard-bucket changes, and task success rates and evidence-citation completeness grouped by rung. Hard-bucket changes must remain at zero. Whether other gates should be relaxed must be determined from baseline distributions and error budgets, rather than a universal percentage detached from the model and task.

Begin rollout by shadow-computing decisions without changing requests, then enable reservations for internal tenants, and finally ramp traffic gradually across stable queues. During rollback, CAS the policy alias to the previous validated version and clear the associated length caches, but retain the newer, lower global cap. Do not disable admission and return to "send first and see what happens." An incident request can be replayed from its manifest to confirm why it was admitted, which rung it used, and which resource was released.

Further reading

• Context Engineering
• Context Budgets for Coding Agents
• KV Cache Admission
• Tokenizer Compatibility Migration