Engineering LLM Tool Calls: Making Function Calling Stable, Idempotent, and Recoverable

Registering a function with a model and invoking it successfully is the most rewarding moment in an Agent demo. Yet the truly difficult part of production begins only here: the model may choose the wrong tool, omit arguments, or put strings into numeric fields; the network may time out, or the tool may already have succeeded but its response was lost; after the user refreshes the page, the same write operation may be submitted again.

Reliable tool calling is not a model capability, but an execution-system capability.

1. Treat Tool Definitions as Public APIs

A tool's name, description, and parameter schema collectively determine how the model uses it. Ambiguous descriptions force the model to guess, while overly broad tools make permissions and error handling unmanageable.

A good tool should meet these requirements:

• Its name should express a single action, such as create_invoice_draft, rather than manage_invoice.
• Its description should explain when to use it, when not to use it, and its key preconditions.
• Its parameters should use explicit types, enums, ranges, and formats.
• It should return stable, structured results and machine-readable error codes.
• Read and write operations should be separate, with high-risk actions modeled independently.

const CreateDraftSchema = z.object({
  customerId: z.string().min(1),
  currency: z.enum(["CNY", "USD", "EUR"]),
  lines: z.array(z.object({
    sku: z.string().min(1),
    quantity: z.number().int().positive(),
  })).min(1).max(100),
  idempotencyKey: z.string().uuid(),
});

Do not let the model populate authentication fields. userId, tenantId, and permission scopes should be injected into the execution context from a trusted session.

2. Keep the Executor Independent of the Model Loop

The model is responsible only for proposing what it wants to call; the executor determines whether the call is allowed and how to perform it. A reliable executor usually proceeds through these stages:

1. Parse and validate the tool name and arguments.
2. Apply the authorization policy for the current identity.
3. Determine whether human confirmation is required.
4. Generate an idempotency key and check previous executions.
5. Invoke the tool after setting timeout, concurrency, and resource limits.
6. Normalize the result or error.
7. Write an audit event, then return a safe summary to the model.

A tool implementation should not directly receive the entire conversation or system prompt. It should receive only validated business arguments and a trusted execution context, reducing the amount of unrelated data sent to sensitive services.

3. Idempotency Is the Lifeline of Write Operations

Consider a "create ticket" tool: the downstream service has successfully created the ticket, but the network connection fails before the response arrives. The Agent sees a timeout and retries. Without idempotency, this creates two tickets.

An idempotency key should be bound to stable business intent instead of being regenerated for every retry. It can consist of a higher-level task ID, a step ID, and a hash of canonicalized arguments. The server stores the idempotency key, request digest, execution state, and result reference.

At a minimum, distinguish these states:

• PENDING: Execution is in progress; duplicate requests wait or receive a processing status.
• SUCCEEDED: Return the result of the first successful execution without executing again.
• FAILED_RETRYABLE: Retries are allowed according to policy.
• FAILED_FINAL: An argument or permission error occurred and should not be retried.
• UNKNOWN: The downstream outcome cannot be confirmed and requires a query or human intervention.

A timeout does not necessarily mean failure. For actions such as payments, sending messages, or creating resources, first confirm the result through a query API, then decide whether to retry.

4. Errors Must Help the Model Recover

Returning a database stack trace directly to the model both exposes internal information and fails to guide the next step. The executor should normalize errors into stable types:

{
  "ok": false,
  "error": {
    "code": "CUSTOMER_NOT_FOUND",
    "message": "No customer matched the supplied customerId.",
    "retryable": false,
    "suggestedAction": "Search the customer before creating a draft."
  }
}

Argument validation failures can return specific field errors to the model and let it correct them once; insufficient permissions should stop execution and be explained to the user; rate limits and transient network errors can use exponential backoff; business conflicts require fetching the latest state again.

Do not retry automatically without limit. Cap retries for each step and for the entire task, and record error fingerprints. When the same error occurs repeatedly, terminate the loop or route the task to a human.

5. Parallel Calls Must Account for Dependencies

The model may propose multiple tool calls at once. Read-only, independent queries can run in parallel, such as fetching an order and a user profile simultaneously; "create a resource and then update it" has an explicit dependency and must run sequentially.

The executor should not blindly run calls concurrently based only on the order returned by the model. The model can output step dependencies, or workflow code can define a fixed DAG. Parallel writes must also account for transaction boundaries, locks, and compensation logic.

If a group of operations partially succeeds, the system must know how to recover. Design compensating operations for reversible actions; for irreversible actions, record the completion point and let subsequent steps continue from there instead of rerunning the entire group.

6. Bind Human Confirmation to a Specific Action

High-risk tools should not display only a vague "Allow the Agent to perform this action?" prompt. The confirmation interface should show the tool name, target resource, key arguments, scope of impact, cost, and irreversible consequences.

A confirmation token must be bound to a hash of the canonicalized arguments and have a short validity period. After the user confirms "send the draft to A," the old confirmation must become invalid if the model changes the recipient to B. The confirmation result should also be recorded in the audit log.

Low-risk, reversible, high-frequency actions can be approved automatically according to policy, but still require limits. Permission design is not a binary choice between "automate everything" and "ask at every step"; it should be tiered by risk.

7. Observability and Evaluation

For every call, record the tool version, argument summary, authorization result, confirmation result, idempotency key, duration, retries, downstream request ID, and final status. Record only redacted values or hashes for sensitive arguments.

Offline evaluation should assess more than whether the model selected the correct tool. It should also verify whether the arguments satisfy the schema, whether unnecessary tools were called, whether the call order was correct, whether failures led to a reasonable recovery path, and whether budgets or permissions were exceeded.

In production, focus on tool success rate, argument validation failure rate, average call count, duplicate call rate, human rejection rate, and the number of unknown states. A high call success rate combined with three extra calls per task still indicates a problem with tool descriptions or planning.

8. A Pragmatic Rollout Sequence

Start with read-only tools and establish schemas, auditing, and error conventions. Then add reversible write operations with idempotency and previews. Only after that should irreversible operations be enabled, with human confirmation, limits, and an emergency kill switch.

For every new tool, answer these questions: What data can it access? What is the worst possible impact? What happens if it executes twice? How is the outcome confirmed after a timeout? Who can approve it? How can it be undone? If these questions cannot be answered clearly, the tool should not be available for automatic Agent invocation.

Summary

Function Calling is merely a format for model output; reliable tool calling is a complete distributed execution protocol. Clear contracts reduce model guesswork, server-side authorization protects boundaries, idempotency prevents duplicate side effects, error classification supports recovery, and confirmation mechanisms control high-risk actions. Only after these capabilities are placed in an independent executor is an Agent ready to move from demonstration to production.