Evaluating Tool-Calling Accuracy

For ML engineers, tool calling evaluation requires moving beyond simple match/no-match metrics to understand complex failure modes. When an LLM calls a function, it is not executing code; it is generating a structured text artifact that a host system parses. Measuring this capability seems straightforward at first glance: if the output matches the expected function and arguments, it’s a hit. However, this binary view obscures the complex failure modes inherent in constrained text generation. A model might correctly identify the intent to search for a user but fail because it formats the ID as a string instead of an integer, or it might produce valid JSON that references a non-existent function.

To evaluate tool-calling accurately, we must move beyond simple "match/no-match" metrics. We need a hierarchical error taxonomy that isolates where the generation process broke down. This allows engineers to distinguish between a model that understands the API contract but struggles with syntax, and one that fundamentally misunderstands the tool's purpose.

The Mechanism of Generation

Before diagnosing errors, we must understand the generation mechanism. Most modern LLMs perform tool calling through one of two pathways:

1. Constrained Decoding: The model is forced to emit tokens that strictly adhere to a JSON schema or a specific grammar (e.g., using libraries like outlines or guidance). Here, syntax errors are rare, but semantic errors (wrong arguments) remain common.
2. Free-form Generation with Post-hoc Parsing: The model generates raw text, which the host system attempts to parse into JSON. This is the most common pattern in open-source frameworks (like LangChain or LlamaIndex). In this scenario, syntax errors (malformed JSON) are frequent, and semantic errors are equally likely.

The evaluation strategy must account for this distinction. If you use constrained decoding, a syntax error is a system failure, not a model failure. If you use free-form generation, syntax is a primary metric of reliability. Effective tool calling evaluation must distinguish between these pathways to accurately attribute errors.

A Hierarchical Error Taxonomy

A comprehensive evaluation framework should classify every failed prediction into one of four distinct categories. This taxonomy helps pinpoint whether the bottleneck is in the model’s reasoning, its knowledge of the schema, or its ability to format text.

1. Intent Error (Wrong Tool)

The model selects an incorrect function entirely. This indicates a failure in grounding the user’s request to the available tool definitions.

• Scenario: User asks, "What's the weather in London?"
• Available Tools: get_weather(city), search_news(query).
• Model Output: search_news(query="London weather")
• Analysis: The model understood the domain (weather/news) but failed to map the intent to the specific tool designed for meteorological data. This is a semantic alignment failure.

2. Schema Error (Wrong Arguments)

The model selects the correct tool but uses incorrect argument names, types, or structures as defined in the function signature.

• Scenario: Tool definition requires {"city": "string", "units": "string (celsius/fahrenheit)"}.
• Model Output: get_weather(city="London", unit="c")
• Analysis: The model hallucinated an argument name (unit vs units) and an invalid value (c vs celsius). This often stems from poor adherence to the provided JSON schema during generation.

3. Value Error (Invalid Data)

The model uses the correct argument names and types, but the values themselves are invalid according to business logic or type constraints.

• Scenario: Tool definition requires {"user_id": "integer"}.
• Model Output: get_user(user_id="12345")
• Analysis: The model correctly identified the tool and the argument name, but failed to cast the extracted entity to an integer. In strict typing systems, this causes a runtime crash. In loose systems, it might work but is still a technical error.

4. Syntax Error (Malformed Structure)

The model’s output cannot be parsed into the expected data structure (e.g., invalid JSON).

• Model Output: {"tool": "get_weather", "args": {"city": "London"}}
• Analysis: If the expected format is a specific function call object, this structural mismatch is a syntax error. This is common in free-form generation when the model truncates output or misses closing braces.

Evaluation Strategy: Beyond Exact Matching

A common pitfall in evaluation is using exact string comparison for tool names and arguments. This is fragile. Consider these variations:

• Get_Weather vs get_weather
• "city": "London" vs "city":"London"
• {"city": "London"} vs {\n "city": "London"\n}

An accurate evaluation engine must implement relaxed matching.

Structural Normalization

Before comparing outputs, normalize the generated JSON and the expected JSON:

1. Case Insensitivity: Convert tool names and argument keys to lowercase.
2. Whitespace Agnostic: Strip all whitespace from the serialized JSON string.
3. Type Coercion: When comparing values, treat numeric strings ("123") and integers (123) as equivalent if the schema allows flexibility, but flag them for value error analysis if strict typing is required.

Metric Design

Do not report a single "Accuracy" score. Report a confusion matrix across the error taxonomy:

This granular reporting reveals that a model might have 95% Intent Accuracy but only 60% Schema Compliance, indicating it needs better fine-tuning on JSON formatting rather than prompt engineering for tool selection. When designing a benchmark for tool use, these metrics provide the necessary granularity to compare models fairly.

Benchmarking Frameworks and Implementation

Several benchmarks exist, but they vary in rigor. BFCL (Berkeley Function-Calling Leaderboard) provides a standardized suite with explicit error categorization. ToolBench focuses on instruction-following across diverse tools.

For a custom evaluation pipeline, you can implement a lightweight evaluator in Python using jsonschema for validation and a custom classifier for intent errors.

import json
import jsonschema
 
def evaluate_tool_call(model_output, expected_tool, schema):
    # 1. Parse Output
    try:
        parsed = json.loads(model_output)
    except json.JSONDecodeError:
        return {"error_type": "syntax_error", "details": "Invalid JSON"}
 
    # 2. Check Intent
    actual_tool = parsed.get("tool")
    if actual_tool != expected_tool:
        return {"error_type": "intent_error", "actual": actual_tool}
 
    # 3. Check Schema
    try:
        jsonschema.validate(instance=parsed["arguments"], schema=schema)
    except jsonschema.exceptions.ValidationError as e:
        return {"error_type": "schema_error", "details": str(e.message)}
 
    # 4. Check Values (Business Logic)
    # Custom logic goes here (e.g., checking if date is in the future)
    
    return {"status": "success"}

Conclusion

Evaluating tool calling evaluation is not about measuring how often an LLM gets it right; it’s about understanding why it gets it wrong. By adopting a hierarchical error taxonomy—separating intent, schema, value, and syntax failures—engineers can provide targeted feedback to models. Whether through fine-tuning on specific JSON patterns or improving system prompts, this diagnostic approach turns vague "accuracy" scores into actionable engineering insights. The goal is not just a higher benchmark score, but a more predictable and debuggable integration between the LLM and the external world. Ultimately, improving function call accuracy requires this level of detailed, taxonomy-driven analysis.