Context Engineering: Giving the Model the Right Information

Over the past two years everyone has been obsessed with "how to write a good prompt." But as models grow stronger and context windows get longer, the bottleneck has quietly shifted: what determines an Agent's performance is no longer "how you phrase things," but "what you put in its context." This is Context Engineering—it doesn't conflict with prompt engineering, but its focus is different.

1. Prompt Engineering vs. Context Engineering

A rough but useful distinction:

• Prompt engineering: cares about how to write the instructions—role, format, phrasing, few-shot.
• Context engineering: cares about how to select the information—which documents, history, state, and tool results the model should see at this moment.

A comparison: have an Agent answer a domain question.

• The prompt-engineering approach: write a great system prompt, asking it to "be rigorous, use bullet points, and cite sources."
• The context-engineering approach: first ensure that the right materials are retrieved, irrelevant content is trimmed, and key constraints are placed prominently.

Once the model is strong enough, it basically understands whatever you say; what really separates results is "the quality of the context it sees." The phrase "garbage in, garbage out" has, in the LLM era, become "context in, quality out."

2. The Four Types of Context

A production-grade Agent's context is typically assembled from four types of information:

Many Agent context problems stem from an imbalanced ratio across these four types: too many instructions drowning out the retrieval results, history so long it crowds out key instructions, or retrieval stuffed with irrelevant content. The core of context engineering is dynamically managing the priority and volume of these four types.

3. Context-Window Management Strategies

Long-context models (100K–1M) create the illusion that "just stuff everything in." But being able to stuff it in ≠ it should be stuffed in—the longer the context, the more the model's attention to middle content gets diluted (the lost-in-the-middle phenomenon), and it also slows down inference and burns tokens. Three strategies:

1. Priority ordering: put the important things at the two ends. The model pays the most attention to the beginning (system prompt, task definition) and the end (latest instructions, data to be processed). So: put stable instructions first, the data currently being processed last, and retrieval/history in the middle.

2. Compression: don't let history grow indefinitely. In multi-turn conversations/long tasks, history quickly overflows the window. Three compression approaches:

• Sliding window + summary: keep the most recent N turns verbatim, and compress earlier ones into a summary.
• Structured state: extract "confirmed facts" into JSON stored in state, not relying on natural-language history.
• Tool-result trimming: for large chunks returned by tools, extract the key fields rather than keeping the full text.

3. Active forgetting: more context isn't always better. Outdated information (early assumptions, judgments that have been overturned) should be actively removed from the context, otherwise it will mislead the model. An Agent that can "forget its own earlier wrong ideas" performs better than one that remembers everything.

4. In Practice: Assembling an Agent's Context

Suppose you're building a "codebase Q&A Agent." Each turn its context should be assembled like this:

[Instructions - stable, via caching]
  system: You are a codebase assistant; only answer based on the provided code snippets...
  Rules: Say "I don't know" when not found; cite file paths

[State - compressed]
  Confirmed: User is researching the payment module; using TypeScript

[Retrieval - recalled this turn, top-3 after rerank]
  <file src="payment/service.ts">…</file>
  <file src="payment/refund.ts">…</file>

[This turn's instruction - latest]
  User question: Where is the refund logic?

Key points:

• Stable instructions → hit prompt caching, save cost.
• State compressed to two lines, no history piling up.
• Retrieval feeds only a small amount after rerank—quality over quantity.
• This turn's question goes last, where attention is strongest.

This structure is far better than "stuffing the entire conversation + 20 recalled files + system prompt all in"—with the former the model answers precisely, while with the latter the model gets lost in a flood of information.

5. Concrete Actions for Context Engineering

Turning "context engineering" into daily practice is a set of actionable habits:

• Ask yourself each turn: what does the model truly need to see at this step? Not "stuff whatever is relevant," but "without this, can it answer at all?"
• Add structure to the context. Use tags/delimiters to clearly mark the boundaries between instructions, retrieval, and data, so the model finds information more accurately.
• Monitor context composition. In the trace, record how many tokens each request's "instructions/retrieval/history" consumes—a spike in a category is often a warning signal.
• Retrieval quality > retrieval quantity. Rather than recalling 20 items and feeding them in, spend the effort feeding in the 3 most relevant. That's where the value of reranking lies.
• Use state instead of history whenever possible. Facts like "the user has logged in" or "the order has been paid" should be stored in structured state, not left for the model to infer from dozens of turns of conversation.

Summary

Models grow stronger year by year, and the marginal return of "how to write a prompt" is diminishing; meanwhile the marginal return of "what context you give the model" is rising. Context Engineering is the new fundamental skill of the Agent era.

The four types of context (instructions/examples/retrieval/state), the three window strategies (priority/compression/forgetting), plus the discipline of "feeding only the necessary information each turn"—this combination determines whether your Agent is "powerful but flailing" or "precise and controllable." Remember one sentence: the strongest model paired with the worst context loses to a mid-tier model paired with carefully organized context.