Coding Agent Context Budget: Keep Key Evidence Without Drowning the Model

As context windows grow longer, many teams' first instinct is to cram more code, logs, and conversation history at the agent. The result is often the opposite of what they want: key constraints get diluted by irrelevant detail, the model draws wrong associations between stale info and new code, and cost and latency rise in lockstep. The context budget isn't about saving a few tokens—it's about making every input segment justify "why it must appear in this decision."

You can split task context into four layers: non-negotiable system constraints (security policies, working directory), current-task facts (requirements, acceptance criteria, risk level), code evidence (target module, callers, tests), and optional background (past discussions, similar PRs). The first two must be precise; the third is selected along the dependency graph; the fourth is added only when it would change a decision.

Budget by Evidence Category

Don't set just one total token cap. Reserve quota for different categories—say, 15% for task and policy, 45% for current code and tests, 20% for interfaces and dependencies, 20% for history and retrieval results. The split isn't a fixed recipe, but it stops long logs or chat histories from swallowing all the code context. For each selection, record the file, snippet range, summary version, and reason for inclusion.

A context package can express priority explicitly:

{
  "task": {"priority": "required", "hash": "..."},
  "policy": {"priority": "required", "version": "repo-policy-v3"},
  "files": [
    {"path": "src/auth.ts", "reason": "authorization boundary", "priority": "high"},
    {"path": "tests/auth.test.ts", "reason": "acceptance evidence", "priority": "high"}
  ],
  "history": [{"id": "pr-812", "priority": "optional"}]
}

When the budget runs short, drop low-priority background first rather than compressing security rules or test assertions. Compressed content keeps references to the original file and line numbers, so the summary doesn't become an untraceable second-hand fact.

Summaries Aren't Permanent Memory

After code changes, an old summary can be more dangerous than no summary at all. Bind each summary to a source-file hash, generation time, and applicable scope; when any key source file changes, related summaries are automatically invalidated. For long-running tasks, persist decision summaries and evidence references rather than an ever-growing transcript. This way, resuming a task re-fetches the latest code on demand instead of inheriting conclusions that went stale hours ago.

Also distinguish "what the model once said" from "the repo's current facts." Historical assumptions can stay in the prompt but must be marked as pending verification, and the agent must confirm them with a file or command before executing. Build commands, dependency versions, and permission rules in particular can't be decided straight from an old conversation.

Check Context Quality with Counterfactual Tests

When evaluating context selection, don't just look at whether the final answer looks reasonable. Construct a few counterfactuals: after removing a key permission file, does the agent still wrongly suggest a write operation? After adding an unrelated large log, does it ignore test failures? After swapping in an expired summary, does it proactively re-read the source file? These tests reveal whether the selector is actually preserving signal or just trimming text.

In production, monitor tokens per context category, hit rate, rework caused by missing evidence, and cache-invalidation counts. If a historical summary never changed a decision, lower its priority; if some test file is repeatedly missing during bug fixes, add it to the task template. Context engineering should be continuously tuned like dependency management, not optimized as a one-off prompt tweak.

Design Safe Fallbacks for Budget Exhaustion

As context nears the limit, the agent shouldn't keep compressing key files until meaning is lost. A safer choice is to stop at a clear checkpoint and output confirmed facts, dependencies not yet read, and the minimum evidence needed for the next round. The scheduler can launch a new sub-task or ask the user to narrow scope, but it must not disguise incomplete context as a complete conclusion.

The fallback strategy should also cap summary cascading: a summary of a summary easily loses exception handling, negative conditions, and version constraints. Beyond one level of compression, require re-reading the original high-risk files and mark unverified regions in the output. By logging how often budgets are exhausted and the final rework rate, the team can tell whether the problem is an undersized model window, an oversized task, or a wrong repo-map selection.

Further Reading

• Context Engineering
• AI Codebase Onboarding
• Prompt Versioning