State Management for Multi-Turn Conversations

A single-turn conversation is a function call: one input, one output. A multi-turn conversation is a state machine: users follow up, change their minds, say "What about its price?" to refer to earlier context, switch topics midway, and then return. Most Agent demos fail on the second turn: as the context grows, it becomes muddled; coreference gets resolved incorrectly; and a topic switch loses the original task. The problem is not the model. The problem is that you have not taken "conversation state" seriously.

1. What Makes Multi-Turn Conversations Hard?

The complexity of multi-turn conversations consists of five overlapping subproblems:

• Context accumulation: Every turn adds more material to the history. The context window eventually overflows, token consumption rises, and lost-in-the-middle effects appear: the model simply ignores information in the middle.
• Anaphora: "Its price," "that file," or "the one we just discussed." The model has to guess what each reference means, and one wrong guess can corrupt the entire chain.
• Topic drift: A user moves from booking a flight to changing a hotel reservation and then asking about the weather. If you keep clinging to the "book a flight" state machine, every response becomes confused.
• Users changing their minds: "No, change the date to Friday." The previous confirmation must be invalidated and the slot rolled back, not supplemented with another value.
• Interruption and resumption: A user leaves to do something else and returns hours later with "Continue where we left off." You need to resume the task correctly.

Single-turn interactions hide all these problems. Multi-turn interactions expose every one of them. There is only one fundamental solution: manage "context" (natural language shown to the model) separately from "conversation state" (machine-readable structured facts). The former helps the model speak coherently; the latter keeps the conversation from falling apart.

2. Context Management: Windows, Summaries, and Structured Extraction

First, consider how to manage the context shown to the model. The core principle is: store critical facts in state, not in history. History exists to preserve conversational flow; it is not the source of truth.

A production-ready approach combines a sliding window, a rolling summary, and structured state:

interface TurnContext {
  // 1. 稳定区:system prompt + 当前任务定义(放最前,优先级最高)
  system: string
  // 2. 摘要区:更早的对话压成一段,丢细节保脉络
  summary: string
  // 3. 结构化状态区:已确认的事实——这才是真相源
  state: ConversationState
  // 4. 滑动窗口:最近 N 轮原文(N 通常 4~8)
  recentMessages: Message[]
  // 5. 当前输入
  userMessage: string
}

async function buildTurnContext(session: Session, userMessage: string): Promise<TurnContext> {
  // 历史超过窗口阈值,触发滚动摘要:把最早的 K 轮压进 summary
  if (session.messages.length > WINDOW_SIZE) {
    session.summary = await summarize(session.summary, session.messages.splice(0, COMPRESS_BATCH))
    // 顺手从被压缩的轮次里抽取关键事实,写进 state——别让事实跟着原文一起消失
    await extractFactsIntoState(session)
  }
  return { system: session.system, summary: session.summary, state: session.state,
           recentMessages: session.messages.slice(-WINDOW_SIZE), userMessage }
}

There are three compression techniques, ordered from least to most aggressive:

• Sliding window: Keep the most recent turns verbatim to preserve conversational flow. This is the cheapest approach, but also the shallowest.
• Summary: Compress older dialogue into natural language that preserves the thread of the conversation. Summaries lose details, so extract any facts that can be structured before summarization.
• Structured state extraction: Extract critical facts such as "The user wants to go to Shanghai, depart on Friday, and fly economy" into JSON stored in state. This is the compression-resistant source of truth: a summary can be wrong, but slots in state are explicit.

In practice, do not start with the most expensive compression method. A plain sliding window is enough for most conversations of six turns or fewer. Summaries usually become necessary only after ten turns. State extraction, however, should happen regardless of conversation length because it solves a different problem: reliably determining what has been remembered.

3. Conversation State: Slots, Entities, and Phases

Conversation state is the foundation of multi-turn interaction. At minimum, it must answer three questions: What is happening now (phase)? What has been collected (slots and entities)? Who or what is being discussed (entity stack)?

interface ConversationState {
  // 当前在哪个阶段:闲聊 / 收集信息 / 澄清 / 执行 / 确认
  phase: 'chitchat' | 'collecting' | 'clarifying' | 'executing' | 'confirming'

  // 任务型:槽位填充。每个槽位带值、置信度、是否已确认
  activeTask: {
    type: string                              // 'book_flight' | 'order_refund' ...
    slots: Record<string, Slot>
    requiredSlots: string[]
  } | null

  // 最近提及的实体栈——指代消解靠它
  entityStack: Entity[]

  // 跨话题的稳定事实(用户身份、偏好、长期约束)
  beliefs: Record<string, { value: unknown; source: 'user' | 'inferred'; updatedAt: number }>

  // 挂起的任务栈——中断时压栈,恢复时弹栈
  suspendedTasks: SuspendedTask[]
}

interface Slot {
  value: unknown
  rawText: string          // 原文,方便回显和 debug
  confidence: number       // 0~1,低置信度要主动澄清
  confirmed: boolean       // 模糊抽取的值要二次确认
  sourceTurn: number
}

Slot filling is the backbone of task-oriented dialogue. Two details matter. First, include confidence scores. If "Friday" was extracted from "This Friday... no, next Friday," confidence should be low. A low-confidence value should move the conversation into the clarifying phase and trigger a question instead of immediate execution. Second, support rollback. When a user changes their mind, do not "append another date"; overwrite the previous date. Slots are a Map, not a List, so changing a decision is an assignment.

The phase determines the goal of the current turn. In clarifying, do not rush to call a tool; resolve the ambiguity first. In confirming, accept only yes/no. In chitchat, do not force the user back into a task. Many bugs where an Agent seems to talk past the user are caused by incorrect phase classification: the user is making small talk while the Agent continues asking for required slots.

4. Coreference Resolution: What Does "It" Refer To?

Coreference is one of the easiest ways for a multi-turn conversation to fail. When a user asks, "What about its price?" "it" might refer to the first product mentioned earlier or to the most recently mentioned one. Models often resolve this incorrectly on their own, and the error is difficult to detect afterward.

Two techniques work best in combination:

Explicit state (primary): Maintain an entityStack and push an entity whenever the user mentions one or the system produces one. When resolving a reference, prefer the most recent entity in state whose type matches. This approach is deterministic and testable.

LLM rewriting (secondary): Perform a coreference-resolution rewrite on the user's input. Based on prior context, rewrite "What about its price?" as "What is the price of the iPhone 15?" before continuing with downstream logic. This works well for complex, multi-hop references.

async function resolveCoreference(input: string, state: ConversationState): Promise<string> {
  // 命中代词才走改写,省 token
  if (!/他|她|它|那个|这个|刚才/.test(input)) return input

  // 1. 先试显式:entityStack 顶部的同类型实体
  const topEntity = state.entityStack.at(-1)
  if (topEntity && looksLikeReferenceTo(input, topEntity)) {
    return input.replace(/(它|那个|这个)/g, topEntity.name)
  }

  // 2. 兜底:让小模型带上最近 3 轮 + entityStack 做改写
  return await rewriteWithModel(input, {
    recentTurns: state.recentMessages.slice(-3),
    candidates: state.entityStack.slice(-3),
  })
}

The key is to maintain entityStack explicitly. Push the target entity during slot filling, push entities produced by tool results, and clear the appropriate entries when the topic changes. Coreference resolution is not an NLP problem; it is a state-management problem. If an entity is missing from state, the model can only guess from history, and the error rate rises with the number of turns.

5. Topic Switching and Interruptions: Push and Pop

Suppose a user is booking a flight and suddenly asks, "What will the weather be like in Shanghai tomorrow?" That is an interruption. Poor handling leads to one of two failures. The first is stubborn continuation: the system remains inside the flight-booking slots and tries to put the weather into the date slot. The second is amnesia: after answering the weather question, it loses the flight-booking task.

The correct solution is a task stack:

async function onTopicSwitch(detected: boolean, state: ConversationState, newIntent: string) {
  if (!detected) return
  // 当前任务还没完 → 压栈挂起,带上当时的槽位快照
  if (state.activeTask && !isTaskComplete(state.activeTask)) {
    state.suspendedTasks.push({
      task: clone(state.activeTask),
      suspendedAt: Date.now(),
      summary: '用户去问天气了',
    })
  }
  // 开启新任务(天气可能是临时闲聊,不建槽位)
  state.activeTask = newIntent === 'weather' ? null : createTask(newIntent)
  state.phase = newIntent === 'weather' ? 'chitchat' : 'collecting'
}

// 检测到用户说"继续订机票""回到刚才那个"
function resumeSuspended(state: ConversationState, keyword: string) {
  const hit = state.suspendedTasks.pop()
  if (hit) { state.activeTask = hit.task; state.phase = 'collecting' }
}

Topic switching can be detected with a lightweight classification step or rules: phrases such as "By the way," "Let's change the subject," and "One more question" are switch signals. Always include a snapshot when suspending a task. Otherwise, its slots will be empty when resumed, which is equivalent to starting over. Limit stack depth to two or three levels. Users are unlikely to remember deeply nested tasks themselves, so proactive clarification is better than forced resumption.

6. Engineering Decisions: Per-Turn Context, Token Control, and Cross-Session Continuity

Several practical decisions matter during implementation:

Per-turn context assembly order should account for lost-in-the-middle effects: stable instructions first → structured state → summary → recent verbatim turns → user input last. Put important information at both ends, and place state near the beginning rather than burying it in history.

Partition the token budget: reserve a fixed allocation for the system prompt and state, such as 30%; dynamically allocate space to history; and reserve space for retrieval and tool results. When history exceeds its budget, trigger summarization instead of truncating blindly. Truncation may remove exactly the critical turn you need.

Cross-session memory: Session-level state should be cleared when the session ends, but stable facts in beliefs, such as user identity and long-term preferences, should be written to a long-term memory store and recalled into state at the beginning of the next session. This connects to the long-term memory described in "Designing an Agent Memory System": multi-turn conversation state is short-term memory, while long-term memory provides its background context. Do not put the entire conversation history into long-term memory. Store only extracted structured facts.

Idempotency and replay: Record every state change in an event log, following event-sourcing principles. This makes it possible to debug questions such as "Why did it change the date?", replay conversations, and recover from a checkpoint after a crash. In production, conversation state is more valuable than conversation text and should be persisted.

7. Failure Modes You Will Encounter

These common failures are worth recognizing in advance:

• Longer context causes more confusion: The model is not becoming less capable; its attention is being diluted. Monitor the answer-quality curve for long conversations. If it starts declining, your compression strategy is not keeping up.
• Critical information gets displaced: A constraint given in turn two is pushed out of state or the summary by turn eight. Structured state is the defense: once a constraint enters state, natural-language history cannot dilute it.
• Coreference is resolved incorrectly and silently: The user says, "Delete it," the model resolves the wrong object, and then deletes something else. Destructive operations require a second confirmation. The confirmation must display the resolved entity explicitly, such as "Confirm deletion of the iPhone 15 order?", so the user can catch a resolution error.
• A change of mind is treated as an addition: A slot is appended when it should be overwritten. The solution is described above: slots use assignment semantics, and every write records sourceTurn for traceability.
• Topic-switch detection is oversensitive: Every sentence is treated as a topic change, causing constant churn in the task stack. Set a threshold for switch signals. Prefer determining whether the user still cares about the current task's required slots instead of relying on a few keywords.

Summary

• Separate context from state: Context is natural language shown to the model; structured state is the source of truth. Put critical facts in state instead of relying on history.
• Apply the three compression techniques as needed: Sliding windows preserve tone, summaries preserve the thread, and state extraction preserves reliability. State extraction is not about saving tokens; it determines exactly what has been remembered.
• Coreference is a state problem, not an NLP problem: Maintain entityStack, resolve references explicitly, use a smaller model as a rewriting fallback, and display the resolved entity for confirmation before destructive operations.
• Use a stack for topic switching: Push snapshots when suspending tasks, pop them to resume, keep the stack shallow, and proactively clarify when it becomes too deep.
• Store only structured facts across sessions: Long-term memory provides background context for state; it is not a backup of the chat transcript.

The engineering essence of multi-turn conversation fits in one sentence: single-turn interactions rely on the model; multi-turn interactions rely on state. The model decides how to respond in the current turn, while state management keeps that turn from going wrong.