Next.js AI Partial-Response Recovery: Preserve, Retry, and Resume After Stream Interruptions

When a traditional page request fails, reloading is usually enough. When an AI stream disconnects after producing half an answer, however, "try again" is not a single well-defined action. The network may have dropped only the final few frames, the model may have completed but the completion event never arrived, a tool call may have succeeded and produced an external side effect, while the browser knows only that the connection was lost. If the client clears the existing text and resends the entire request, the user loses usable content, and the system may charge them again, resend an email, or modify data twice.

The first principle of recovery design is to separate "how much text has been seen" from "how far execution has progressed." Partial text in React is only a projection; the server-side run is the source of truth. Every generation has a stable runId. The server assigns monotonically increasing sequence values to output events and retains events or checkpoints within a recovery window. The client records the sequence number of the last fully applied event and, after a disconnection, queries the status or resubscribes using runId + afterSequence instead of immediately creating a new run.

First Determine Which Layer Failed

At minimum, distinguish five outcomes: user-initiated cancellation, browser transport interruption, upstream model failure, tool-step failure, and server-side policy rejection. Each permits different recovery actions. User cancellation does not reconnect automatically by default; a transport-only interruption can resubscribe to the same run; a model failure can be retried only after confirming that execution is no longer continuing; recovery from a tool failure depends on whether the operation is idempotent; and a policy rejection must not be bypassed through retries.

An HTTP status can describe only errors that occur before the response starts. Once the stream has returned 200 and begun sending a body, subsequent failures should be represented by structured events such as error { code, retryable, phase, lastSequence }. If the connection simply disappears, the client marks the state as disconnected rather than failed, then asks the status endpoint whether the run is still executing, has completed, or has actually failed. This prevents a completed answer from being generated again merely because its completion event was lost.

Next.js route error handling is responsible for returning a consistent public error structure. A React Error Boundary is responsible for rendering a fallback when the rendering phase crashes, but neither automatically handles business errors from a stream-reading Promise. The read loop must catch exceptions and dispatch events carrying the runId to the reducer. React Error Boundaries can help clarify the boundary between rendering failures and asynchronous run failures.

Event Replay Must Be Inherently Deduplicated

The client reducer accepts only events where sequence > lastAppliedSequence. Text events should carry immutable deltas or segment IDs rather than merely reporting the "current complete string," because snapshots from before and after a disconnection can otherwise overwrite one another. Under consistent transactional semantics, the server recovery endpoint returns both the run state and the event upper bound. The client applies all replayed events before attaching to the live stream, preventing a gap between the query and subscription.

If the infrastructure cannot retain every token chunk, create checkpoints at semantic paragraph boundaries. During recovery, display confirmed paragraphs and mark the unconfirmed tail as provisional. Regenerate only the subsequent portion, explicitly telling the model the summary and ending position of the confirmed content. Do not claim that generation can resume precisely from any arbitrary token: most model APIs do not guarantee that the same input will produce exactly the same suffix, and forced stitching may repeat half a sentence or change the conclusion.

When the user selects "Regenerate," create a new runId and a new candidate version. Preserve the old partial answer and label its termination reason. Do not let the new stream continue writing into the old message, and do not include an unfinished answer in the next turn's context by default. See Building Streaming LLM Frontends for details about streaming state and cancellation.

Side Effects Define Retry Boundaries

Pure generation calls can usually be retried with backoff before the request has been accepted upstream, but automatic retries require greater caution after any token has been received. Tool calls must carry a business idempotency key, and the server must persist the prepared / executing / succeeded / failed / unknown state. Only steps that can be confirmed as unexecuted or that execute idempotently may be retried automatically. When the state is unknown, first query the external system; if the outcome cannot be confirmed, route it to manual handling.

For example, "query an order" can be repeated safely, "send an email" requires a stable deduplication key, and "transfer funds" must not be replayed directly by a front-end button. The recovery engine resumes from the most recent safe checkpoint instead of executing every tool again from the beginning of the conversation. For a more systematic sequence, see Designing Agent Failure Recovery. For upstream failover and data recovery across the service, see LLM Disaster Recovery.

Retries must also have a total budget: a maximum attempt count, cumulative duration, provider-switch count, and cost ceiling. Add jitter to exponential backoff and honor the server's Retry-After. When an error is not retryable or the budget is exhausted, the interface preserves confirmed content and presents action-specific options such as "Reconnect," "Create a new version from here," or "Contact support," rather than one identical retry button for every failure.

The Fallback Interface Must Protect Existing Work

A message component can render streaming, disconnected, partial, completed, and failed states separately. A failure in one message must not cause an Error Boundary to replace the entire conversation tree, and a failure in the sources panel must not clear the answer. Keep confirmed paragraphs readable and available to copy, and give the unconfirmed tail a clear status instead of using an endless spinner to conceal a stall.

Server logs correlate conversationId, runId, the last sequence number, execution phase, retry count, and upstream request ID, with body content redacted by default. Metrics track disconnection recovery success rate, replayed event count, discarded duplicate event count, partial-answer retention rate, unknown tool states, additional tokens consumed by retries, and recovery latency. Incorporate the failure budget into the release gates defined by SLOs and Error Budgets.

Validation must inject real failures: disconnect before the first byte, disconnect at a Chinese-character encoding boundary, disconnect before the completion event, duplicate and out-of-order events, browser refresh, service instance restart, model 429, a tool succeeding while its response is lost, and recovery-window expiration. For every scenario, verify that the final message contains no duplicate fragments, dangerous tools are not executed twice, an old run cannot contaminate a new run, and resources are eventually released. Only when these outcomes can be explained and demonstrated is the system actually recovering; simply calling the model again after an exception only amplifies the uncertainty.