Tool Calling Failure Taxonomy: Stop Blaming Every Failure on the Model

When an Agent's tool call fails, "the model hallucinated" is the easiest and least useful conclusion. It conflates entirely different faults: the model selected the wrong tool, the schema allowed invalid arguments, the user lacked permission, a downstream API timed out, or a write operation required confirmation but the system failed to intercept it. Without a shared taxonomy, teams can only keep changing the prompt, with no way to determine whether the model, the tool, or the business workflow actually needs fixing.

Six Failure Categories You Must Distinguish

The first category is tool selection failure: the Agent calls a write tool when it should have called a query tool, or it calls a deprecated tool. The second is argument failure: the tool is correct, but the resource ID, enum value, time range, or pagination arguments are wrong. The third is authorization failure: the request should never have been executed, and whether the system explicitly rejects it matters more than whether it ultimately produces a result.

The fourth category is dependency failure, including rate limits, timeouts, connection errors, and abnormal third-party responses. The fifth is state failure, such as when a create request succeeds but its response is lost, causing the Agent to create the resource again. The sixth is interaction failure: user confirmation is missing, a clarification question is never asked, or the Agent performs an irreversible operation despite insufficient information.

These six categories can be further tagged by ownership boundary. An argument error is not necessarily a model problem; the schema may describe the date format ambiguously. An authorization denial also need not count as a failure if the system correctly blocks an unauthorized action. Metrics must distinguish "expected denials" from "unexpected denials."

Preserve the Minimum Evidence for Each Failure

For every tool call, record at least the tool name, candidate tool set, whether the input passed schema validation, the authorization result, downstream status code, retry count, idempotency key state, and final user-visible outcome. Do not record complete sensitive arguments, but do retain a structured summary sufficient to replay a sanitized scenario.

For example, an argument failure can be recorded as missing_required_field, invalid_enum, or resource_not_in_context; a dependency failure can record dependency_timeout and the stage where time was spent. Stable codes let offline evaluations and production monitoring use the same language instead of relying on natural-language error messages in logs.

Drive Fixes with the Taxonomy, Not Just Overall Success Rate

Suppose the overall tool-call success rate rises from 88% to 91%. If all new successes come from low-risk queries while payment writes still have a 4% duplicate execution rate, the system's actual risk has not decreased. Priorities should be set by tool risk, failure category, and user impact. State failures in high-risk write operations may account for only one in a thousand calls, yet they should be fixed before wording problems in low-risk summaries.

Every fix should also be added to the evaluation set. After improving a tool description, rerun the selection-failure samples. After adding an idempotency key, inject a lost response and verify that state failures decrease. After adjusting permissions, verify that unauthorized requests are still denied. This turns the taxonomy from an incident-report template into a feedback loop that continuously reduces the failure rate.

Make It Understandable to Product Teams

For non-engineering roles, present business-facing metrics such as "the percentage of tasks requiring human takeover," "the number of high-risk requests correctly denied," and "mean time to recovery." Do not report only model scores. Users care whether the task was completed, whether the system took an incorrect action, and whether they know what to do next after a failure, not about an abstract improvement on some benchmark.

Once failures are classified consistently, teams no longer need to optimize Agents by guesswork. Models, tools, policies, and dependency services each gain verifiable improvement targets, so investment can focus on what actually affects reliability.

From Taxonomy to an Actionable On-Call Workflow

Each failure code needs a clearly assigned team and an initial response action. invalid_enum may belong to the tool contract owner, dependency_timeout to the platform or external dependency owner, and confirmation_missing to the product policy owner. Categories without owners only make reports look better; on-call staff still will not know whom to escalate to.

Define one diagnostic question for each failure category. For a state failure, first ask, "Has the downstream system already produced a side effect?" For an authorization failure, ask, "Does the denial match policy?" For a selection failure, ask, "Do the candidate tool descriptions overlap?" These questions turn incident handling from log trawling into a fixed diagnostic path. They also make it easier to group recurring issues into one backlog instead of opening a new ticket each time.

Review changes in the category distribution every week. If argument failures suddenly rise after a model upgrade, the output format may have changed. If resource-not-found errors rise after a tool release, ID parsing or context injection may have regressed. Category trends reveal systemic problems earlier than any single error report.

Further Reading

• MCP Tool Schema Design
• Automated Agent Evaluation
• MCP Tool Timeouts, Retries, and Idempotency