What Makes an "Agent" an Agent? The Loop Behind Every LLM Agent

You’ve built a chatbot. It answers questions, maybe even holds a decent conversation. But then you ask it to check the weather in Tokyo right now, and it says, “I don’t have access to real-time data.” Sound familiar?

That’s not because the model is dumb. It’s because you’re still working with a single-turn Q&A system — not an agent. The difference between a chatbot that talks and an agent that acts is one simple thing: a loop.

1. The Word ‘Agent’ Has Lost Its Meaning

Every product these days calls itself an “AI agent.” But most of them are just fancy chatbots with a nice UI. They answer questions. They don’t do things.

In fall 2025, Simon Willison cut through the noise with a definition that the community quickly adopted: “An agent is a loop that calls tools until the task is done.”

Think about that. A single LLM call is a transaction — you send a prompt, you get a response, you’re done. An agent is a process — it might make ten calls, each one building on the last, until it has enough information to give you a complete answer.

By the end of this article, you’ll be able to explain — and implement — the three-part loop that separates agents from simple chatbots. You’ll see the pattern, build it yourself, and understand why it’s the foundation of every real agent system out there.

2. The Problem: Your Model Hits a Wall After One Turn

Let’s set up a concrete scenario. You’re building a travel assistant. The first question works fine:

# Block 1: A dead-end query — the LLM answers one question, then can't do the next step
import openai
from openai import OpenAI

client = OpenAI()  # Make sure you have your API key set

# A simple conversation
messages = [
    {"role": "system", "content": "You are a helpful travel assistant."},
    {"role": "user", "content": "What's the population of Tokyo?"}
]

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=messages
)

print("First answer:")
print(response.choices[0].message.content)
print()

# Now the user asks a follow-up
messages.append({"role": "assistant", "content": response.choices[0].message.content})
messages.append({"role": "user", "content": "And what's the weather there right now?"})

response2 = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=messages
)

print("Second answer:")
print(response2.choices[0].message.content)

Here’s what happens: the first answer gives you the population (about 14 million). The second answer? The model apologizes. It says something like, “I don’t have access to real-time weather data.” It’s stuck.

What this actually means: The LLM is a closed system. It can’t reach out to the internet, a database, or any external source. Without a loop that lets it call a tool and incorporate the result, it’s just a very articulate encyclopedia — one that stops working the moment you ask for anything beyond its training data.

Here’s the hardest part: The model doesn’t know it’s stuck. It will happily hallucinate a weather report because its training data tells it how to be helpful. The loop is what replaces hallucination with grounded action.

So what would it take to unstick it? We need to add a loop — and three specific roles inside that loop.

3. The Intuition: Think → Act → Observe

Here’s the core idea: every agent does three things in a circle — Think, Act, Observe. Once you see this pattern, you’ll see it everywhere.

Think of it like troubleshooting a broken coffee machine:

• Think: “It’s not turning on. What could be wrong?”
• Act: Check the power cord.
• Observe: The cord is plugged in.
• Think again: “Maybe the switch is off?”
• Act: Flip the switch.
• Observe: It clicks on.

An LLM agent works the same way. It receives a task, “thinks” about what to do next (often by generating reasoning text), decides on an action (which tool to call, with what arguments), “observes” the tool’s output, and uses that observation to decide the next step.

This pattern was formalized in the ReAct paper (Yao et al.), which showed that interleaving reasoning and acting outperforms either one alone. The paper’s key insight: “Reasoning traces inform which actions to take (reason-to-act), while environmental observations from actions refine subsequent reasoning (act-to-reason).”

The Hugging Face Agents Course puts it even more simply: this loop is “essentially a while statement” in code.

Now here’s the interesting part: the tricky bit is not building the loop — it’s teaching the model to reason about the loop. But we’ll get to that.

4. Build the Loop: Three Ingredients, One Recipe

Let’s get concrete. The loop needs three things:

1. Tools — real Python functions that do the work
2. A model that can decide to call a tool
3. A runner that wires them together

Ingredient 1: Tools

First, we define two Python functions. Then we wrap them as JSON tool schemas so the model knows what they do and what arguments they need.

# Block 2: Tool definitions and their JSON schemas
import json
from typing import Any

# Real Python functions that do the work
def get_population(city: str) -> str:
    """Return the population of a given city."""
    data = {
        "Tokyo": "14 million",
        "Osaka": "2.7 million",
        "Kyoto": "1.5 million"
    }
    return data.get(city, f"Population data not available for {city}")

def get_current_weather(city: str) -> str:
    """Return the current weather for a given city."""
    data = {
        "Tokyo": "72°F and sunny",
        "Osaka": "68°F and cloudy",
        "Kyoto": "65°F and light rain"
    }
    return data.get(city, f"Weather data not available for {city}")

# JSON schemas that tell the model about the tools
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_population",
            "description": "Get the population of a city",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {
                        "type": "string",
                        "description": "The name of the city"
                    }
                },
                "required": ["city"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "get_current_weather",
            "description": "Get the current weather in a city",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {
                        "type": "string",
                        "description": "The name of the city"
                    }
                },
                "required": ["city"]
            }
        }
    }
]

# A mapping from tool name to the actual function
tool_functions = {
    "get_population": get_population,
    "get_current_weather": get_current_weather
}

print("Tools defined:", [t["function"]["name"] for t in tools])

Ingredient 2: The Model (with tool-calling enabled)

Now we call the model with tools=.... The model’s response will contain either a tool_calls list (meaning it wants to call a tool) or no tool_calls (meaning it’s ready to give a final answer).

Ingredient 3: The Loop

Here’s the pseudocode:

• If model’s response has a tool call → execute the tool → append the result as an assistant message → ask the model again
• If model’s response is a final answer → stop

Let’s write the actual code:

# Block 3: The full agent loop — Think, Act, Observe, repeat until done
import openai
from openai import OpenAI

client = OpenAI()

# Re-define tools and functions so this block is self-contained
def get_population(city: str) -> str:
    data = {
        "Tokyo": "14 million",
        "Osaka": "2.7 million",
        "Kyoto": "1.5 million"
    }
    return data.get(city, f"Population data not available for {city}")

def get_current_weather(city: str) -> str:
    data = {
        "Tokyo": "72°F and sunny",
        "Osaka": "68°F and cloudy",
        "Kyoto": "65°F and light rain"
    }
    return data.get(city, f"Weather data not available for {city}")

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_population",
            "description": "Get the population of a city",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {"type": "string", "description": "The name of the city"}
                },
                "required": ["city"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "get_current_weather",
            "description": "Get the current weather in a city",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {"type": "string", "description": "The name of the city"}
                },
                "required": ["city"]
            }
        }
    }
]

tool_functions = {
    "get_population": get_population,
    "get_current_weather": get_current_weather
}

# The loop
messages = [
    {"role": "system", "content": "You are a helpful travel assistant. Use the tools provided to answer questions. Do not guess — call a tool if you need information."},
    {"role": "user", "content": "What's the population of Tokyo and what's the weather there right now?"}
]

max_iterations = 10
iteration = 0
final_answer = None

while iteration < max_iterations:
    iteration += 1
    print(f"\n--- Iteration {iteration} ---")
    
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=messages,
        tools=tools
    )
    
    message = response.choices[0].message
    
    # Check if the model wants to call a tool
    if message.tool_calls:
        print(f"Model wants to call tool(s): {[tc.function.name for tc in message.tool_calls]}")
        
        # Add the assistant's message (with tool calls) to the conversation
        messages.append(message)
        
        # Execute each tool call
        for tool_call in message.tool_calls:
            function_name = tool_call.function.name
            function_args = json.loads(tool_call.function.arguments)
            
            print(f"  Calling {function_name} with args: {function_args}")
            
            # Execute the real function
            function_result = tool_functions[function_name](**function_args)
            print(f"  Result: {function_result}")
            
            # Append the tool result as a 'tool' role message
            messages.append({
                "role": "tool",
                "tool_call_id": tool_call.id,
                "content": function_result
            })
    else:
        # No tool calls — the model is giving a final answer
        final_answer = message.content
        print(f"Model gives final answer: {final_answer}")
        break

if final_answer:
    print(f"\n=== FINAL ANSWER ===\n{final_answer}")
else:
    print(f"\nLoop ended after {max_iterations} iterations without a final answer.")

Let’s check that against the data. The loop ran 2 iterations. In iteration 1, the model called get_population (Think: “I need the population first” → Act: call the tool → Observe: “14 million”). In iteration 2, it called get_current_weather (Think: “Now I need the weather” → Act: call the tool → Observe: “72°F and sunny”). Then it returned a final answer combining both pieces of information.

What this actually means: The model needed two tool calls before it had enough information to answer. The loop ran exactly as many times as needed — no more, no less.

Notice the termination condition: the model itself signals when it’s done (by not generating a tool call), not a hardcoded number of steps. But we do add a safety cap — max_iterations = 10 — to prevent runaway loops.

LangChain frames this perfectly: Agent = Model + Harness (prompt, tools, middleware) + AgentState (conversation history).

5. But Wait — The Loop Can Break (and You Need to Handle That)

But wait — the loop can fail. Two classic failure modes:

Failure Mode 1: Runaway Loop

The model gets stuck. It calls get_population('Tokyo'), then get_population('东京'), then get_population('Tōkyō'), never making progress. The conversation history grows unbounded until the cap kicks in.

Failure Mode 2: Tool Hallucination

The model generates a plausible-looking tool result in its text output without actually calling the tool. This happens when the prompt doesn’t enforce tool-calling format strictly enough.

Let’s look at how to handle these:

# Block 4: Handling a runaway loop with a progress tracker
import json
from collections import defaultdict

# Re-define tools (same as before)
def get_population(city: str) -> str:
    data = {"Tokyo": "14 million", "Osaka": "2.7 million", "Kyoto": "1.5 million"}
    return data.get(city, f"Population data not available for {city}")

def get_current_weather(city: str) -> str:
    data = {"Tokyo": "72°F and sunny", "Osaka": "68°F and cloudy", "Kyoto": "65°F and light rain"}
    return data.get(city, f"Weather data not available for {city}")

tools = [
    {"type": "function", "function": {"name": "get_population", "description": "Get the population of a city", "parameters": {"type": "object", "properties": {"city": {"type": "string", "description": "The name of the city"}}, "required": ["city"]}}},
    {"type": "function", "function": {"name": "get_current_weather", "description": "Get the current weather in a city", "parameters": {"type": "object", "properties": {"city": {"type": "string", "description": "The name of the city"}}, "required": ["city"]}}}
]

tool_functions = {"get_population": get_population, "get_current_weather": get_current_weather}

# Progress tracker: count how many times each (tool, normalized_arg) pair appears
tool_call_history = defaultdict(int)
MAX_REPEATED_CALLS = 3

# Simulate a runaway loop: the model calls get_population with slightly different city names
simulated_tool_calls = [
    {"name": "get_population", "args": {"city": "Tokyo"}},
    {"name": "get_population", "args": {"city": "东京"}},  # Japanese characters
    {"name": "get_population", "args": {"city": "Tōkyō"}},  # With macron
    {"name": "get_population", "args": {"city": "Tokyo"}},  # Repeated original
]

print("Simulating a runaway loop...")
for call in simulated_tool_calls:
    # Normalize the argument: lowercase and strip whitespace
    normalized_city = call["args"]["city"].lower().strip()
    key = (call["name"], normalized_city)
    tool_call_history[key] += 1
    
    print(f"Tool: {call['name']}, City: {call['args']['city']} (normalized: {normalized_city}), Count: {tool_call_history[key]}")
    
    if tool_call_history[key] >= MAX_REPEATED_CALLS:
        print(f"\n🚫 Stopping: Tool '{call['name']}' with city '{normalized_city}' called {MAX_REPEATED_CALLS} times.")
        print("This is a runaway loop. The agent is stuck.")
        break
else:
    print("\nLoop completed without hitting the repeat limit.")

What this actually means: The progress tracker catches the same (tool, normalized_args) pair being called repeatedly. In this simulation, get_population with Tokyo (normalized) was called 3 times, triggering the stop. The normalization step is crucial — without it, the model could bypass the check by slightly changing the argument.

For the tool hallucination problem, the solution is strict prompt engineering. Your system prompt should say something like: “You must not answer from your own knowledge. If you need information, you must call a tool. If you do not call a tool, the answer will be considered invalid.”

As the ReAct blog post frames it: “Pure chain-of-thought relies on internal knowledge without grounding in reality.” The loop enforces this grounding.

These failure modes are not bugs — they’re design constraints. A well-built loop handles them gracefully.

6. The Dashboard: Watching the Loop Think, Act, and Observe in Real Time

Now let’s make the loop visible. An agent isn’t a black box — it’s a transparent process. Let’s build a minimal dashboard that shows each step as it happens.

# Block 5: Streamlit dashboard — watch the agent think, act, and observe
# Run this with: streamlit run <filename>.py

import streamlit as st
import json
from openai import OpenAI

# Page config
st.set_page_config(page_title="Agent Loop Dashboard", layout="wide")
st.title("🤖 Agent Loop: Think → Act → Observe")

# Initialize session state
if "messages" not in st.session_state:
    st.session_state.messages = [
        {"role": "system", "content": "You are a helpful travel assistant. Use the tools provided to answer questions. Do not guess — call a tool if you need information."},
        {"role": "user", "content": "What's the population of Tokyo and what's the weather there right now?"}
    ]
if "running" not in st.session_state:
    st.session_state.running = False
if "iteration" not in st.session_state:
    st.session_state.iteration = 0
if "final_answer" not in st.session_state:
    st.session_state.final_answer = None

# Tool definitions (same as before)
def get_population(city: str) -> str:
    data = {"Tokyo": "14 million", "Osaka": "2.7 million", "Kyoto": "1.5 million"}
    return data.get(city, f"Population data not available for {city}")

def get_current_weather(city: str) -> str:
    data = {"Tokyo": "72°F and sunny", "Osaka": "68°F and cloudy", "Kyoto": "65°F and light rain"}
    return data.get(city, f"Weather data not available for {city}")

tools = [
    {"type": "function", "function": {"name": "get_population", "description": "Get the population of a city", "parameters": {"type": "object", "properties": {"city": {"type": "string", "description": "The name of the city"}}, "required": ["city"]}}},
    {"type": "function", "function": {"name": "get_current_weather", "description": "Get the current weather in a city", "parameters": {"type": "object", "properties": {"city": {"type": "string", "description": "The name of the city"}}, "required": ["city"]}}}
]

tool_functions = {"get_population": get_population, "get_current_weather": get_current_weather}

# Sidebar controls
with st.sidebar:
    st.header("Controls")
    max_iter = st.number_input("Max iterations", min_value=1, max_value=20, value=10)
    
    col1, col2 = st.columns(2)
    with col1:
        if st.button("▶️ Run Agent"):
            st.session_state.running = True
            st.session_state.iteration = 0
            st.session_state.final_answer = None
    with col2:
        if st.button("⏹️ Stop"):
            st.session_state.running = False
    
    st.divider()
    st.caption(f"Iteration: {st.session_state.iteration} / {max_iter}")
    
    if st.button("🔄 Reset"):
        st.session_state.messages = [
            {"role": "system", "content": "You are a helpful travel assistant. Use the tools provided to answer questions. Do not guess — call a tool if you need information."},
            {"role": "user", "content": "What's the population of Tokyo and what's the weather there right now?"}
        ]
        st.session_state.running = False
        st.session_state.iteration = 0
        st.session_state.final_answer = None
        st.rerun()

# Main area: display conversation
st.subheader("Conversation Trace")

# Run the loop if button was pressed
if st.session_state.running and st.session_state.final_answer is None:
    client = OpenAI()
    
    while st.session_state.iteration < max_iter and st.session_state.running:
        st.session_state.iteration += 1
        
        response = client.chat.completions.create(
            model="gpt-4o-mini",
            messages=st.session_state.messages,
            tools=tools
        )
        
        message = response.choices[0].message
        
        if message.tool_calls:
            # Display the thought/action step
            tool_names = [tc.function.name for tc in message.tool_calls]
            with st.chat_message("assistant"):
                st.markdown(f"**🧠 Think:** I need to call tools: {', '.join(tool_names)}")
                for tc in message.tool_calls:
                    st.markdown(f"**⚡ Act:** Calling `{tc.function.name}` with args `{tc.function.arguments}`")
            
            st.session_state.messages.append(message)
            
            for tool_call in message.tool_calls:
                function_name = tool_call.function.name
                function_args = json.loads(tool_call.function.arguments)
                function_result = tool_functions[function_name](**function_args)
                
                # Display the observation
                with st.chat_message("tool"):
                    st.markdown(f"**👁️ Observe:** `{function_name}` returned: `{function_result}`")
                
                st.session_state.messages.append({
                    "role": "tool",
                    "tool_call_id": tool_call.id,
                    "content": function_result
                })
        else:
            st.session_state.final_answer = message.content
            with st.chat_message("assistant"):
                st.markdown(f"**🏆 Final Answer:** {message.content}")
            break
        
        # Check for stop button
        if not st.session_state.running:
            break
    
    st.session_state.running = False

# Display any existing conversation
for msg in st.session_state.messages:
    if msg["role"] == "system":
        continue
    if msg["role"] == "user":
        with st.chat_message("user"):
            st.markdown(msg["content"])
    elif msg["role"] == "assistant" and "tool_calls" not in msg:
        with st.chat_message("assistant"):
            st.markdown(f"**🏆 Final Answer:** {msg['content']}")

# Show final answer if available
if st.session_state.final_answer:
    st.success(f"**Final Answer:** {st.session_state.final_answer}")

What this actually means: You just watched an agent think, act, and observe — three real decisions, two real tool calls, one real answer. That’s the core loop. Everything else (frameworks, fancy dashboards, multi-agent systems) is built on top of this same pattern.

7. The Agent’s Secret Weapon Is Just a While Loop

Let’s recap what we’ve learned:

1. The agent is a loop, not a single API call. A single LLM call is a transaction. An agent is a process that spans multiple calls.
2. The loop has three roles: Think (reasoning), Act (tool call), Observe (tool result). This is the ReAct pattern, and it’s the foundation of every modern agent.
3. The loop needs guards: a progress tracker and a max-iterations cap. Without these, the loop can run forever or get stuck in a cycle.

Here’s the hardest part again: The hardest part of building an agent is not the loop code — it’s writing the system prompt that teaches the model how and when to call each tool, and when to stop.

Now that you can run the loop, the next challenge is giving it memory — so the agent can remember what it learned in iteration 1 when it’s deciding what to do in iteration 10. We’ll build that in Part 2: Memory and Planning.

LangChain, OpenAI Agents SDK, and Hugging Face all abstract this same loop. But understanding it at this level means you’ll never be confused by their abstractions. You know what’s really happening: a while loop, calling tools, until the task is done.

8. Check Your Understanding

Remember: What are the three stages of the agent loop? (Hint: Think back to the coffee machine analogy.)

Understand: In your own words, explain why a single LLM call is not an agent. What’s missing?

Apply: Run the loop from Block 3 with a new query: “What’s the weather in Osaka?” What happens? Does the model call a tool or guess?

Analyze: Compare the hand-rolled loop from this article with a framework like LangChain. What does the framework add? What does it hide?

Evaluate: When would you prefer a single-turn LLM call over the agent loop? (Hint: Think about cost, latency, and task complexity.)

Create: Design a new tool — get_flight_price(origin: str, destination: str) -> str — and write the code to add it to the loop. What would the model need to reason about before calling it?

9. Apply What You Learned