Tool calling is the mechanism that lets a language model invoke external tools — search, calculators, databases, APIs — as part of generating a response, the foundational capability that AI agents (see AI Agents) are built on.
How Tool Calling Actually Works
Rather than the model directly executing code, tool calling works through a structured protocol: the model is given a description of available tools (name, purpose, expected arguments), and when it determines a tool would help, it generates a structured request specifying which tool to call and with what arguments — the actual execution happens outside the model, in the surrounding application code, which then feeds the result back to the model as additional context.
Code — The Basic Tool Calling Flow
tools = [
{
"name": "get_weather",
"description": "Get the current weather for a given city",
"parameters": {"city": "string"}
},
{
"name": "calculate",
"description": "Evaluate a mathematical expression",
"parameters": {"expression": "string"}
}
]
response = llm.generate(
prompt="What's the weather in Mumbai, and what's 47 * 892?",
tools=tools
)
# The model may respond with structured tool call requests instead of plain text:
# [{"tool": "get_weather", "args": {"city": "Mumbai"}},
# {"tool": "calculate", "args": {"expression": "47 * 892"}}]
# The application executes each requested tool call
results = [execute_tool(call) for call in response.tool_calls]
# Results are fed back to the model to produce the final, complete answer
final_response = llm.generate(prompt=build_followup_prompt(response, results))Why Tool Calling Matters: Overcoming Genuine Model Limitations
Language models are notably unreliable at precise arithmetic, don't have access to real-time information beyond their training cutoff, and can't directly interact with external systems (databases, APIs) on their own. Tool calling routes these specific sub-tasks to systems actually built for them — a calculator for arithmetic, a search API for current information — while the language model focuses on what it does well: understanding the request, deciding which tools are needed, and synthesizing the final, coherent answer from the results.
Tool Calling vs Function Calling — Terminology Note
"Tool calling" and "function calling" (the next note) are largely used interchangeably in practice — both describe the same underlying mechanism. Some distinguish "function calling" as a specific API implementation detail (structured JSON schema-based requests) versus "tool calling" as the broader conceptual capability, but this distinction isn't consistently applied across the field.
Common Mistakes
- Giving a model a poorly-described tool (vague name, unclear parameter descriptions) — the model relies entirely on these descriptions to decide when and how to use a tool correctly, so a poor description directly leads to incorrect or missed tool usage.
- Not validating tool call arguments before execution — a model can occasionally generate malformed or nonsensical arguments, and executing these directly without validation can cause errors or unintended behavior in the underlying system.
Interview Relevance
Q: "Why does tool calling let a language model perform tasks it's otherwise unreliable at, like precise arithmetic or accessing current information?" Tool calling doesn't make the language model itself better at arithmetic or give it live internet access — instead, it lets the model recognize when a sub-task is better handled by a specialized external system, generate a structured request for that system, and incorporate the returned result into its response. The actual computation or lookup happens outside the model entirely, in genuinely reliable purpose-built tools (a calculator, a search API), while the model's role is limited to what it's actually good at: understanding intent, deciding which tool is needed, and synthesizing a coherent final answer from the results.
Practice Question
Why is a clear, precise tool description important for reliable tool calling, and what might go wrong with a vague one?
