MCP Tool Timeouts, Retries, and Idempotency: Turning Uncertain Network Calls into Recoverable Workflows

When a model calls a tool, failures are not exceptional. Networks can be unstable, downstream services can impose rate limits, users can repeat their questions, and clients may disconnect before results arrive. If the Server relies on a single try/catch, two problems can easily arise: read operations hang indefinitely, and write operations execute more than once. The key is not to add more retries, but to first define recovery semantics for each type of operation.

Allocate a Time Budget to Each Call

Do not give tools only a global 60-second timeout. A single call usually includes queuing, authentication, downstream requests, result transformation, and audit-log writes. Set an upper limit for the entire request, then shorter limits for critical downstream operations. For example, with a total budget of 20 seconds, allocate 8 seconds to a database query, 6 seconds to a third-party API, and 2 seconds to log submission. This leaves the system enough time to return a structured error when a dependency slows down.

A timeout does not mean the operation has stopped. For write operations, the most dangerous state after a timeout is: "The client did not receive a result, but the downstream operation may have succeeded." The Server therefore cannot treat a timeout as a failure and immediately retry. It must enter a status-check or compensation path.

Decide Whether to Retry by Operation Type

Tools can be divided into three categories:

1. Read-only operations: For example, querying a ticket. Limited retries are appropriate for connection resets or transient 503 responses.
2. Idempotent writes: For example, updating status using a deterministic resource ID. The operation can be replayed with the same idempotency key.
3. Non-idempotent writes: For example, creating a payment or sending an external notification. These must not be retried automatically by default. First check the result or wait for human confirmation.

Retry policies also need clear limits. Use exponential backoff with random jitter, and limit both the total number of attempts and the overall time budget. Do not enable automatic retries simultaneously at the model layer, in the MCP client and Server, and in the downstream SDK. Otherwise, one user request can expand into dozens of real writes.

Bind Idempotency Keys to Business Operations

Generate an idempotency key for every tool call that may write data, and store it with the downstream business record. A practical format can include the session, tool name, user-confirmation turn, and a hash of the request content:

mcp:session-8f2:change-priority:7:8a2d...

When the same key arrives again, the service returns the result of the first request instead of executing the operation again. The key considerations are the key's lifetime and storage location. Keeping it only in an in-memory cache cannot cover process restarts, while deduplicating only by user ID can incorrectly discard legitimate new requests. Persist the key together with the business resource, expiration time, and result snapshot to correctly handle cases where the request succeeds but the response is lost.

Support Cancellation, but Do Not Promise What Cannot Be Canceled

After the client disconnects or the user clicks Cancel, the Server should stop work that has not yet started and forward the cancellation signal to downstream requests that support cancellation. For an asynchronous job that has already been submitted, return its task ID and current status instead of pretending that it has been rolled back. The tool description must clearly state the cancellation semantics: whether cancellation removes queued work, stops running work, or only stops waiting for the result.

Long-running jobs are better split into pollable workflows such as start_export, get_export_status, and download_export. This allows the model to check at each step whether the user still wants to continue. It can also recover from a failure using the task ID instead of restarting the entire expensive workflow.

Errors Returned to the Model Must Also Be Recoverable

When a tool returns an error to the model, the worst response is simply "request failed." The model may retry with different wording or mistakenly assume it should call another write tool. An error response must state the stage that failed, whether the business outcome is known, whether retrying is safe, and what to do next. After a write timeout, for example, return outcome: unknown, the idempotency key, and the name of the tool that checks the result instead of returning retryable: true.

For read tools, the response can recommend narrowing the query scope, using pagination, or retrying later. For write tools, it should first recommend get_operation_status; replay should be allowed only after confirming that no side effects occurred. Encoding these policies in the tool contract and regression tests prevents the model from amplifying an intermittent error into a sequence of writes under stressful conditions.

Each tool also needs a concurrency limit. A single model session may launch several similar queries concurrently, while multiple sessions may hit the same downstream service at the same time. Limit concurrency by tenant, tool, and dependency service, and count queuing time against the total budget. Even under heavy retry traffic, the system can then protect critical dependencies through controlled rejection or queuing responses.

Observability Must Show Whether Recovery Is Correct

At minimum, record the tool name, downstream dependency, duration, retry count, timeout stage, idempotency-key status, and final business outcome. Do not monitor only the error rate. Also track the proportion of timeouts followed by a successful status check, the duplicate-key hit rate, and P95/P99 latency for each tool.

Validate the design with fault injection before release: deliberately discard the response after the downstream service completes a write; make the first request return 503; and disconnect the client while an asynchronous job is running. The MCP tool has production-grade recovery capabilities only if the system consistently returns the same result, never writes twice, and never occupies connections indefinitely.

Further Reading

• MCP Protocol Deep Dive
• LLM Observability
• Automated Agent Evaluation