Giving An Llm Tools Function Calling And Tool Use

Have you ever asked a chatbot for the weather and gotten a confident wrong answer? You know the feeling — you type “What’s the weather in Tokyo right now?” and the model cheerfully replies with something like “72°F and partly cloudy.” Only, it’s not 72°F in Tokyo. It’s actually raining. And the model has no way of knowing that.

This is the single greatest limitation of a pure LLM: it’s a frozen brain of training data, with no senses to perceive the live world. An LLM can reason, but it can’t act. It can’t look up real-time weather, query a database, or do exact arithmetic. Every answer is a blend of its training data pattern-recognition, not a query against a live data source.

But what if we could give the LLM a way to reach outside of itself when it knows it doesn’t know? That’s tool calling.

1. The chatbot that can’t open a window

Let’s see the problem in action. We’ll ask GPT-4o-mini a simple real-time question and watch it fail.

# Block 1: A dead-end query — the LLM guesses a real-time answer and gets it wrong
import openai
from openai import OpenAI

client = OpenAI()  # Make sure you have your API key set

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "What is the current weather in Tokyo?"}
]

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=messages
)

print("Model's answer:")
print(response.choices[0].message.content)

When you run this, you’ll get something like:

Model's answer:
I'm sorry, but I don't have access to real-time data, including current weather conditions. I can only provide information up to my last training cut-off in April 2024. For current weather, I recommend checking a weather website or app.

Or worse, it might hallucinate a number. The model is not being coy — it genuinely has no way to check. Every answer is a blend of its training data pattern-recognition, not a query against a live data source.

So what would it take to unstick it? We need to give the LLM a tool.

2. What’s actually a ‘tool’, anyway?

Before we show the mechanics, let’s build an intuition for what a ‘tool’ is in the LLM’s world.

Think of it like this: if you asked me to look up a colleague’s phone number, you’d tell me “use the directory tool; it takes a name and a department.” You don’t describe the internal database schema. You describe the interface.

A tool specification is a JSON document with three blocks:

• What the tool does (a description in plain English)
• What the tool needs (parameters, each with a name, type, and description)
• What the tool returns (not specified to the model — it just gets the result back)

Here’s what that looks like for a weather tool:

{
  "type": "function",
  "function": {
    "name": "get_weather",
    "description": "Get the current weather for a given city",
    "parameters": {
      "type": "object",
      "properties": {
        "location": {
          "type": "string",
          "description": "City name, e.g. Tokyo, Japan"
        },
        "units": {
          "type": "string",
          "enum": ["celsius", "fahrenheit"],
          "description": "Temperature unit (optional, defaults to celsius)"
        }
      },
      "required": ["location"]
    }
  }
}

Let’s walk through each field in plain English:

• name: The tool’s identifier. The model uses this to say “I want to call get_weather.”
• description: A natural-language explanation of what the tool does. The model reads this to decide if this tool is relevant to the user’s question.
• parameters: A JSON Schema describing the inputs. The model reads each parameter’s name and description to decide how to fill in the arguments.
• required: Which parameters the model must provide. If location is required, the model knows it can’t leave it blank.

The model is a playwright — it writes the stage directions. Your code is the stage crew that actually executes them.

3. The mechanics: how the model asks for a tool

Now we show the actual mechanism: you send the model a list of tool definitions alongside your prompt. The model’s response now has an extra field: tool_calls. Let’s walk through every field in that response.

# Block 2: Define the tool in OpenAI SDK format and send the API call
import openai
from openai import OpenAI
import json

client = OpenAI()

# Define the tool in the format the OpenAI API expects
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for a given city",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "City name, e.g. Tokyo, Japan"
                    },
                    "units": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                        "description": "Temperature unit (optional, defaults to celsius)"
                    }
                },
                "required": ["location"]
            }
        }
    }
]

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "What is the current weather in Tokyo?"}
]

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=messages,
    tools=tools
)

# Print the entire response message object
message = response.choices[0].message
print("Full message object:")
print(f"  content: {message.content}")
print(f"  tool_calls: {message.tool_calls}")
print()

# Walk through the tool call details
if message.tool_calls:
    tool_call = message.tool_calls[0]
    print("Tool call details:")
    print(f"  id: {tool_call.id}")
    print(f"  function.name: {tool_call.function.name}")
    print(f"  function.arguments: {tool_call.function.arguments}")
    print()
    # Parse the arguments JSON
    args = json.loads(tool_call.function.arguments)
    print(f"  Parsed arguments: {args}")

When you run this, you’ll see something like:

Full message object:
  content: None
  tool_calls: [ChatCompletionMessageToolCall(id='call_abc123', function=Function(arguments='{"location":"Tokyo, Japan"}', name='get_weather'), type='function')]

Tool call details:
  id: call_abc123
  function.name: get_weather
  function.arguments: {"location":"Tokyo, Japan"}

  Parsed arguments: {'location': 'Tokyo, Japan'}

Notice what happened:

• content is None. The model is not speaking — it’s calling a tool.
• tool_calls is a list with one object. Each object has:
  • id: A unique identifier for this call. We’ll use this later to match the result.
  • function.name: The name of the tool the model wants to call.
  • function.arguments: A JSON string containing the arguments the model chose.

The model has not run any code. It has sent back a slip of paper saying: “I want to call get_weather with location = ‘Tokyo, Japan’.” Now it’s our job to run that function and give the answer back.

This is the hardest part for first-time implementers: the model does not execute. It only requests.

4. Executing the tool and feeding the result back

We take the model’s tool call, run a real Python function that actually calls a real weather API (or simulates one for safety), and then send the result back to the model as a new message. This completes the tool-using loop.

# Block 3: Execute the tool and feed the result back to the model
import openai
from openai import OpenAI
import json

client = OpenAI()

# Re-define the tool and messages so this block is self-contained
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for a given city",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "City name, e.g. Tokyo, Japan"
                    },
                    "units": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                        "description": "Temperature unit (optional, defaults to celsius)"
                    }
                },
                "required": ["location"]
            }
        }
    }
]

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "What is the current weather in Tokyo?"}
]

# First API call — get the tool request
response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=messages,
    tools=tools
)

message = response.choices[0].message
print("Model requested tool call:")
print(f"  Tool: {message.tool_calls[0].function.name}")
print(f"  Arguments: {message.tool_calls[0].function.arguments}")
print()

# --- Execute the tool ---
# In a real app, this would call a weather API. Here we simulate it.
def get_weather(location: str, units: str = "celsius") -> str:
    """Simulate a weather API call."""
    # Mock data for demonstration
    weather_data = {
        "Tokyo, Japan": {"temperature": 22, "conditions": "sunny"},
        "London, UK": {"temperature": 15, "conditions": "cloudy"},
        "New York, USA": {"temperature": 18, "conditions": "rainy"}
    }
    data = weather_data.get(location, {"temperature": 20, "conditions": "unknown"})
    temp = data["temperature"]
    if units == "fahrenheit":
        temp = temp * 9/5 + 32
    return json.dumps({"location": location, "temperature": temp, "units": units, "conditions": data["conditions"]})

# Parse the arguments and call the function
args = json.loads(message.tool_calls[0].function.arguments)
tool_result = get_weather(**args)
print(f"Tool execution result: {tool_result}")
print()

# --- Feed the result back to the model ---
# Append the assistant's tool call message
messages.append(message)

# Append the tool result as a new message with role 'tool'
messages.append({
    "role": "tool",
    "tool_call_id": message.tool_calls[0].id,
    "content": tool_result
})

# Send the updated message history back to the model
final_response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=messages,
    tools=tools
)

print("Final model response:")
print(final_response.choices[0].message.content)

When you run this, you’ll see something like:

Model requested tool call:
  Tool: get_weather
  Arguments: {"location":"Tokyo, Japan"}

Tool execution result: {"location": "Tokyo, Japan", "temperature": 22, "units": "celsius", "conditions": "sunny"}

Final model response:
The current weather in Tokyo is 22°C and sunny.

Let’s interpret what just happened:

1. The model recognized it needed external data and requested the get_weather tool.
2. Our code executed the tool (simulated a weather API call) and got a JSON result.
3. We appended the tool result as a new message with role: 'tool' and the matching tool_call_id.
4. The model read the weather data and synthesized it into a natural-language answer.

The model is now a polite assistant that can read its own mail. It requested data through the tool, we delivered it, and it synthesized the result into a natural answer.

This pattern — request → execute → return → synthesize — is the fundamental loop of all tool-using LLMs.

5. Beyond one call: the reasoning-act loop (ReAct)

Simple single-tool calls are powerful, but the real magic is when the model reasons step by step, calls a tool, gets a result, reasons some more, and calls another tool. This is the ReAct pattern: Thought → Action (tool call) → Observation (result) → next Thought.

Think of it as the model thinking out loud. It says: “I need the population of Paris first.” Then it calls the population tool. It gets the number. Then it says: “Now I need to divide that by 1000.” Then it calls the calculator tool.

Let’s build this multi-turn loop.

# Block 4: A multi-turn ReAct loop that handles multiple tool calls
import openai
from openai import OpenAI
import json

client = OpenAI()

# Define two tools: one for population data, one for calculation
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_population",
            "description": "Get the population of a given city",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {
                        "type": "string",
                        "description": "The name of the city, e.g. Paris"
                    }
                },
                "required": ["city"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "calculate",
            "description": "Perform a mathematical calculation",
            "parameters": {
                "type": "object",
                "properties": {
                    "expression": {
                        "type": "string",
                        "description": "The mathematical expression to evaluate, e.g. 14000000 / 1000"
                    }
                },
                "required": ["expression"]
            }
        }
    }
]

# Tool dispatch function
def execute_tool(tool_name: str, arguments: dict) -> str:
    """Execute a tool by name with the given arguments."""
    if tool_name == "get_population":
        # Mock population data
        data = {
            "Paris": 2161000,
            "Tokyo": 13960000,
            "London": 8982000
        }
        population = data.get(arguments["city"], 0)
        return json.dumps({"city": arguments["city"], "population": population})
    elif tool_name == "calculate":
        # Safe eval for demonstration — in production, use a proper math parser
        result = eval(arguments["expression"])
        return json.dumps({"expression": arguments["expression"], "result": result})
    else:
        return json.dumps({"error": f"Unknown tool: {tool_name}"})

# Start the conversation
messages = [
    {"role": "system", "content": "You are a helpful assistant. When you need to use a tool, call it. When you have the final answer, respond directly."},
    {"role": "user", "content": "What is the population of the capital of France divided by 1000?"}
]

max_iterations = 5
iteration = 0

while iteration < max_iterations:
    iteration += 1
    print(f"\n--- Iteration {iteration} ---")
    
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=messages,
        tools=tools
    )
    
    message = response.choices[0].message
    
    # Check if the model wants to call a tool
    if message.tool_calls:
        print(f"Model called tool: {message.tool_calls[0].function.name}")
        print(f"With arguments: {message.tool_calls[0].function.arguments}")
        
        # Add the assistant's message to the history
        messages.append(message)
        
        # Execute each tool call
        for tool_call in message.tool_calls:
            args = json.loads(tool_call.function.arguments)
            result = execute_tool(tool_call.function.name, args)
            print(f"Tool result: {result}")
            
            # Add the tool result back to the conversation
            messages.append({
                "role": "tool",
                "tool_call_id": tool_call.id,
                "content": result
            })
    else:
        # No tool calls — the model is giving a final answer
        print("Model gave final answer:")
        print(message.content)
        break
    
    if iteration == max_iterations:
        print("Reached maximum iterations without final answer.")

When you run this, you’ll see the model’s reasoning unfold step by step:

--- Iteration 1 ---
Model called tool: get_population
With arguments: {"city": "Paris"}
Tool result: {"city": "Paris", "population": 2161000}

--- Iteration 2 ---
Model called tool: calculate
With arguments: {"expression": "2161000 / 1000"}
Tool result: {"expression": "2161000 / 1000", "result": 2161.0}

--- Iteration 3 ---
Model gave final answer:
The population of Paris (the capital of France) is 2,161,000. Dividing that by 1000 gives 2,161.

This pattern was formalized in the ReAct paper by Yao et al. (2022), which showed that interleaving reasoning traces with actions reduced hallucination and improved task completion. The model doesn’t just call tools blindly — it reasons about what to do next based on the results it receives.

Not every question needs a tool. The model should learn to decide when to call and when to just answer from knowledge. The Decision Token mechanism (Wang et al., 2024) addresses exactly this — it trains the model to output a special token that indicates whether to call a tool or respond directly.

6. Provider patterns: OpenAI vs. Anthropic (and the general case)

OpenAI and Anthropic have slightly different API shapes for the same concept. Let’s see both side by side so you can generalize the pattern rather than memorize one vendor’s format.

OpenAI pattern (what we’ve already used)

# Block 5: OpenAI tool call pattern (summary of what we've done)
import openai
from openai import OpenAI
import json

client = OpenAI()

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for a given city",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {"type": "string", "description": "City name"}
                },
                "required": ["location"]
            }
        }
    }
]

messages = [
    {"role": "user", "content": "What's the weather in London?"}
]

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=messages,
    tools=tools
)

message = response.choices[0].message
print("OpenAI response:")
print(f"  content: {message.content}")
print(f"  tool_calls: {message.tool_calls}")

Anthropic pattern

# Block 6: Anthropic equivalent — same concept, different field names
# Note: This block requires the anthropic package: pip install anthropic
# It's self-contained and demonstrates the Anthropic pattern

# We'll simulate the Anthropic pattern conceptually since it requires a different API key
# The key differences are:
# 1. Anthropic uses stop_reason: "tool_use" instead of tool_calls
# 2. Tool use is a block inside content (which is a list)
# 3. The result is returned as a tool_result block

print("Anthropic pattern (conceptual):")
print()
print("1. Tool definition format:")
print('''
{
  "name": "get_weather",
  "description": "Get current weather",
  "input_schema": {
    "type": "object",
    "properties": {
      "location": {"type": "string"}
    },
    "required": ["location"]
  }
}
''')
print()
print("2. Response when model wants to use a tool:")
print('''
{
  "content": [
    {"type": "text", "text": "I'll look up the weather."},
    {
      "type": "tool_use",
      "id": "toolu_abc123",
      "name": "get_weather",
      "input": {"location": "London"}
    }
  ],
  "stop_reason": "tool_use"
}
''')
print()
print("3. Returning the result:")
print('''
{
  "role": "user",
  "content": [
    {
      "type": "tool_result",
      "tool_use_id": "toolu_abc123",
      "content": "{\"temperature\": 15, \"conditions\": \"cloudy\"}"
    }
  ]
}
''')

Both follow the same conceptual loop:

1. Define the tool in a structured spec
2. The model returns a structured request to call the tool
3. Your code executes it
4. The result is returned as a structured observation

The field names differ, but the flow is identical. Tool calling is the mechanism that lets the LLM recognize when it needs outside help and issue a structured, machine-readable request for that help.

7. The hard stuff: when tools go wrong and how to fix it

Now that you understand the basic loop, we need to address the problems that appear in production. Here are the most common issues and how to handle them.

API hallucination: the model invents tools

The model can invent functions or use wrong arguments. The Gorilla paper (Patil et al., 2023) showed that pairing the model with a document retriever over known APIs reduces hallucination by providing a grounded tool list. Instead of hoping the model remembers your tool definitions, retrieve the most relevant ones from a database.

Too many tools: you can’t fit 500 definitions in the prompt

You can’t fit 500 tool definitions in the prompt. The DTDR method (Wang et al., 2025) selects only the relevant tools conditioned on the query, improving success by 23-104% over static retrieval. The key insight: don’t dump all your tools in the prompt. Select the top 5-10 most relevant ones based on the user’s query.

Parallel execution: multiple tool calls at once

A single user query might trigger multiple tool calls. The OpenAI API returns them in a single tool_calls list. You can execute them in parallel (using concurrent.futures) or sequentially, depending on latency and dependency constraints. If one tool’s output is needed as input to another, you must run them sequentially.

Error handling: what if the API is down?

What if the weather API is down? Return an error message as the tool result. The model can respond with an apology or try a fallback tool. Your code is the safety net.

# Block 7: Error handling pattern for tool calls (pseudocode)
print("Error handling pattern:")
print()
print("1. Wrap tool execution in try/except:")
print('''
def safe_execute_tool(tool_name, arguments):
    try:
        result = actual_tool_function(tool_name, arguments)
        return {"success": True, "result": result}
    except Exception as e:
        return {"success": False, "error": str(e)}
''')
print()
print("2. Return error as tool result:")
print('''
# If the tool fails, return an error message
messages.append({
    "role": "tool",
    "tool_call_id": tool_call.id,
    "content": json.dumps({"error": "Weather API is currently unavailable. Please try again later."})
})
''')
print()
print("3. The model can respond with an apology or try a fallback:")
print('''
# The model might say:
# "I'm sorry, the weather service is temporarily unavailable. 
#  Let me try a different source."
# And then call a different tool.
''')

Security: never trust tool results blindly

Never let the tool result text be directly interpolated into a system prompt or SQL query. The model’s output is not trustworthy — it can be manipulated to produce injection strings. Always validate and sanitize tool outputs before using them in sensitive operations.

Tool description quality matters

The PremAI blog found that better tool descriptions improved accuracy by up to 30%. Be specific and clear in your descriptions. Instead of “Get weather data,” write “Get the current temperature, humidity, and weather conditions for a specific city.” The model uses these descriptions to decide which tool to call.

8. What you’ve built and what’s next

You now understand the core mechanism that powers every tool-using LLM agent. Let’s recap what you’ve learned:

1. Tools are functions described in JSON that the model reads but does not execute.
2. The model writes a structured tool-call request when it decides it needs outside information.
3. Your code dispatches the call to a real function and returns the output.
4. The model reads the output and continues reasoning, possibly calling another tool.
5. The loop ends when the model returns a final text answer.

The single most important concept: the LLM writes function-call requests in structured JSON. Your code executes them. The result is fed back as a new message. Repeat until the model says “I’m done.”

We’ve given the LLM a hand — it can reach out and grab data from the world. But it still can’t remember context across conversations or guide its own long-term planning. That’s the agent memory loop, and it’s exactly what we’ll build next in Part 3 of this series.

You now understand the core mechanism that powers every tool-using LLM agent, from simple Q&A bots to complex automation systems. That’s a big deal.

Check Your Understanding

Let’s test your understanding of tool calling with questions at different levels.

Remember: What are the three parts of a tool specification that the model reads?

Understand: Explain in your own words why the model’s response has content: None when it decides to call a tool.

Apply: Given a tool that searches a database of products by name, write the JSON specification for it. Include parameters for product name (required) and category (optional).

Analyze: A user asks “What’s the population of Paris divided by 1000?” The model calls get_population with {"city": "Paris"} and gets back {"population": 2161000}. Then it calls calculate with {"expression": "2161000 / 1000"}. Why did the model need two tool calls? Could it have done it in one?

Evaluate: Compare the OpenAI and Anthropic patterns for tool calling. What are the key differences in how the model signals it wants to use a tool? Which pattern do you find clearer, and why?

Create: Design a tool-calling system for a recipe assistant. The assistant needs to search for recipes by ingredient, scale recipe quantities, and convert between units (cups to grams, Fahrenheit to Celsius). Define the tools, their parameters, and describe how the ReAct loop would handle a query like “Find a chocolate cake recipe and scale it to serve 12 people instead of 8.”

Related articles

• Part 1: What Makes an “Agent” an Agent? The Loop Behind Every LLM Agent — This article introduced the Think → Act → Observe loop that we built upon here. If you haven’t read it, start there to understand the foundation.
• Part 3: Building Agent Memory: Context, State, and Long-Term Planning — Coming next in this series. We’ll take the tool-using loop from this article and add memory so the agent can remember past conversations and plan over multiple steps.