Agent Runtime State Machine Design: Splitting Planning, Execution, Waiting, and Recovery into Verifiable States

Many agent prototypes cram all their logic into a single loop: read the message, call the model, decide whether to call a tool, then push the result back into the context. In a demo it runs smoothly, but once it hits timeouts, user-confirmation steps, queue retries, or process restarts in production, it becomes hard to answer one basic question: what state is this task actually in, and is the next step allowed to run?

A more reliable approach is to treat the agent as a state-driven workflow rather than a perpetually running prompt. The state machine exists not to make the diagram look pretty, but to ensure that every side effect has an entry condition, an exit condition, and persisted evidence. The minimal state set usually includes RECEIVED, PLANNING, AWAITING_APPROVAL, EXECUTING_TOOL, OBSERVING_RESULT, RETRY_WAIT, COMPLETED, and FAILED. Different businesses can extend it, but do not allow "any state to jump to any other state".

Define the state contract first, not the nodes

Each state needs to answer four questions: where its input comes from, what side effects it is allowed to produce, who decides the next state, and which fields must be persisted. Take AWAITING_APPROVAL: the input should be an already-normalized execution plan plus a risk summary; this state is not allowed to initiate any write operation; it can only be advanced by user confirmation, user rejection, or an expiry timer; persisted fields must include at least the plan version, approver, approval scope, expiry time, and a hash of the request summary.

Here is a compact TypeScript representation. The point is not the enum itself, but confining transitions to a single function:

type RunState = "RECEIVED" | "PLANNING" | "AWAITING_APPROVAL" | "EXECUTING_TOOL" | "OBSERVING_RESULT" | "COMPLETED" | "FAILED";

const allowed: Record<RunState, RunState[]> = {
  RECEIVED: ["PLANNING", "FAILED"],
  PLANNING: ["AWAITING_APPROVAL", "EXECUTING_TOOL", "FAILED"],
  AWAITING_APPROVAL: ["EXECUTING_TOOL", "FAILED"],
  EXECUTING_TOOL: ["OBSERVING_RESULT", "FAILED"],
  OBSERVING_RESULT: ["PLANNING", "COMPLETED", "FAILED"],
  COMPLETED: [],
  FAILED: [],
};

function transition(run: { state: RunState }, next: RunState) {
  if (!allowed[run.state].includes(next)) throw new Error(`illegal transition: ${run.state} -> ${next}`);
  return { ...run, state: next };
}

Do not treat the text returned by the model as state directly. The model can only propose candidate actions; the deterministic runtime is responsible for validating parameters, permissions, budget, and the current state. That way, even if the model mislabels "retry later" as "execute now", the executor still rejects the illegal transition.

Give side effects a trackable boundary

A state transition and an external call do not always correspond one to one. For example, EXECUTING_TOOL can create a work item carrying an idempotency key, while the actual HTTP request is performed by a worker; OBSERVING_RESULT then consumes the result event. This way, if the process crashes after the request, the recovery logic does not blindly repeat the write; it first queries for any known result using the idempotency key.

For each run, it is recommended to persist runId, stateVersion, planHash, toolCallId, idempotencyKey, lastErrorCode, and updatedAt. stateVersion is used for optimistic concurrency control: updates require a matching version, preventing two workers from advancing the same run into different branches at the same time. Audit logs reference these IDs rather than copying the full prompt or sensitive tool parameters.

For scenarios that require the user to wait, the state machine must have time semantics. After an approval expires, the run should enter an explainable terminal state or fall back to replanning, rather than leaving a task stuck in the queue forever. A timeout is not an ordinary error message; it is an input to a state transition. Design timeout, cancellation, and human handoff as first-class events so that operations can quantify why runs get stuck.

Validate transitions with scenario tests, not just prompt tests

Tests should cover every dangerous edge: an unapproved run cannot enter a write tool; a tool timeout cannot be marked as completed directly; a duplicate result event arriving twice must not settle twice; a completed task cannot be advanced again by a stale worker. You can treat the transition table as test data, asserting allowed and rejected combinations item by item. For recovery tests, deliberately kill the process after each side effect, then replay from a database snapshot, confirming that the final state and external effects occur exactly once.

Before launch, run a small validation: create a write operation that needs approval and confirm it stays in AWAITING_APPROVAL when there is no confirmation event; after sending the confirmation, simulate a tool timeout; restart the worker and redeliver the same result event. If only one external change remains, the state has a complete timeline, and errors are explainable, then the state machine boundary is usable.

When not to over-model

Read-only, side-effect-free Q&A that completes within a single request does not need to be forced into ten states. State machines bring storage, monitoring, and migration costs. The criterion is: can the task span processes, span user interactions, or span external systems? If yes, model it explicitly; if not, simple request-level handling is often clearer.

Further reading

• AI Agent Architecture
• Agent Loop Design
• AI Agent Testing Strategy