AI Agent Architecture: Engineering Planning, Memory, and Tool Calls

The word "Agent" has been overused. It's often equated with "an AI that does multi-turn dialogue," but that's just a chatbot. The real difference for an Agent comes down to one sentence: an Agent is a loop system that can autonomously decide "what to do next"—it receives a goal, plans on its own, calls tools, observes results, and revises its plan, until it completes the task or actively asks for help.

Break that sentence apart and you get the four components of Agent architecture.

1. The Four Components

        ┌──────────── Goal / Task ────────────┐
        ▼
   ┌─────────┐   ┌──────────┐   ┌─────────┐
   │Planning │──▶│Execution │──▶│Tool Env │
   └─────────┘   └────┬─────┘   └─────────┘
        ▲            │
        │ Obs/Result ▼
   ┌─────────┐   ┌─────────┐
   │ Memory  │◀──│Reflect. │
   └─────────┘   └─────────┘

• Planning: breaks a vague goal into executable steps, deciding what means to use at each step.
• Memory: remembers conversation context (short-term) and cross-session facts/preferences (long-term).
• Tools: capabilities it can call—search, code execution, databases, APIs, file read/write.
• Execution: the runtime that drives the "think → act → observe" loop.

2. Two Mainstream Paradigms: ReAct and Plan-Execute

ReAct (Reason + Act) is the simplest and most universal paradigm: at every step, the model does "think—act—observe." It's flexible and suits exploratory, uncertain-step tasks. The cost is that it plans at every step—high token consumption and prone to drift.

Thought: I need to first look up the user's recent orders
Action: query_orders(user_id="u_123", days=30)
Observation: [{id:"o_1", total:299}, ...]
Thought: Next, sum the totals
Action: ...

Plan-Execute (plan first, then execute) first has the model produce a complete plan, then executes it step by step; if the plan turns out to be infeasible during execution, it returns to the planning layer to re-plan. Suits tasks with relatively clear steps and high controllability requirements (like data processing pipelines).

Engineering rule of thumb: use ReAct by default; when the task is structured, has many steps, and you want to cut tokens, layer in a Plan. Most production-grade Agent frameworks (LangGraph, AutoGen) are combinations of these two.

3. Memory: Short-Term and Long-Term

Short-term memory is the conversation history inside the context window. The hard part is the window is limited—long tasks blow it up fast. Three responses:

1. Sliding window + summary: keep the original text of the most recent N turns; compress earlier ones into a summary.
2. Structured state: extract key facts into JSON/entities stored in variables, not relying on natural-language history.
3. Retrieval-based memory (RAG): store history into a vector store, recall by relevance each turn.

Long-term memory is the key to an Agent "recognizing people": user preferences, confirmed facts, past decisions. Usually a memory store (vector store + structured tables), isolated by user_id / session_id, with relevant memory injected into the system prompt at the start of each turn. More memory isn't better—you need a write policy (what's worth remembering) and a forgetting mechanism (update or delete outdated/conflicting entries), or you'll be drowned in noise.

4. Tool Calls: Engineering Points for Function Calling

Tools are the only channel through which an Agent affects the real world, and also the place where things most easily go wrong.

// A well-structured tool definition
const tools = [{
  type: "function",
  function: {
    name: "create_issue",
    description: "Create a GitHub issue. Use when user asks to track a bug or task.",
    parameters: {
      type: "object",
      required: ["title", "repo"],
      properties: {
        repo:   { type: "string", description: "owner/name, e.g. facebook/react" },
        title:  { type: "string" },
        body:   { type: "string" },
        labels: { type: "array", items: { type: "string" } },
      },
    },
  },
}];

Key points:

• Descriptions are written for the model to read, and must clearly explain "when to use it and what not to use instead." When the model picks the wrong tool, nine times out of ten the description wasn't written well.
• Parameters must have a schema, with required clearly marked and enum values fully listed. This is the only basis for the model to fill in parameters.
• Dangerous operations need confirmation. Write operations (delete data, send messages, payments) default to a "human confirmation" gate—don't let the model touch the production environment unsupervised.
• Idempotency and timeouts. Tool calls fail and retry—design them to be idempotent where possible; set timeouts on external calls so the Agent doesn't get stuck waiting on an API that never responds.

5. Pushing an Agent to Production

Going from a working demo to production-ready requires filling in three things:

1. Loop control. There must be three gates: maximum steps, maximum tokens, and maximum elapsed time. Once an Agent falls into a "call tool → fail → call again" death loop, no upper bound means a money-burning incident.

2. Error recovery. When a tool fails, feed the readable error message back to the model ("insufficient permissions, need to log in first") instead of throwing an exception and aborting—the Agent can often find another path on its own. For genuine system errors, have a fallback (to a human or to rules).

3. Observability. Every step's thought / action / observation must be logged (ideally structured, so the entire trace can be replayed). An Agent without a trace is a black box—when something goes wrong, there's no way to troubleshoot. A production Agent should at least instrument: per-step latency, token consumption, tool-call success rate, and human-intervention rate.

6. Summary

The hard part of Agents is never "making the model say a few more things," but how to stay controllable, recoverable, and observable inside an open loop. Planning sets the direction, memory provides continuity, tools confer capability, and the execution layer's "upper bounds, retries, traces" are the watershed between a toy and a production system.

Once you understand this skeleton, whether you use LangGraph, AutoGen, or build your own, you'll have a clear target—they're all essentially different assemblies of these four components.