What Is Reinforcement Learning Really Agents Envir

You’re standing in the kitchen, watching a toddler reach for a cookie jar on the counter. They stretch their arm, wobble on their tiptoes, and miss. They try again — this time they grab the edge of the jar, but it tips over and their hand gets burned by the hot mug next to it. Ouch. They pull back, cry for a minute, then try again. Finally, they manage to grab a cookie and shove it in their mouth, grinning ear to ear.

That toddler just did something remarkable. They learned a complex skill — reaching for a cookie — without anyone giving them a single labeled example. No one said “reach 15 degrees to the left” or “grip with 3 Newtons of force.” They just tried stuff, got feedback from the world (burn = bad, cookie = good), and adjusted their behavior.

What if you had to teach a computer to learn the same way? No textbook, no labeled dataset, no teacher telling it the right answer. Just the ability to try actions, feel the consequences, and figure out what works.

That’s exactly what reinforcement learning (RL) is. It’s the field of machine learning where an agent learns by interacting with an environment, getting rewards and punishments, and figuring out a strategy to maximize the good stuff over time. As IBM puts it, RL is about “learning from rewards and punishments” without a supervisor telling the agent what to do at each step.

If you’ve only ever worked with supervised learning — where you feed the computer (input, label) pairs and ask it to predict the label — RL feels like a completely different world. And it is. But the intuition is already in your head. You’ve been doing RL since you were a toddler.

The Three Ingredients: Agent, Environment, Reward

Let’s name the pieces the toddler story introduced. Every RL problem has exactly three components:

1. Agent: The thing making decisions. That’s the toddler. Or a game character. Or a stock-trading algorithm.
2. Environment: Everything the agent interacts with. The kitchen, the cookie jar, the hot mug. In a video game, it’s the level geometry, the enemies, the coins.
3. Reward: A number the agent gets after each action. Positive means “good job,” negative means “don’t do that again.” The cookie is +10. The burn is -10.

The agent’s goal is simple: maximize the total reward it collects over time. Not just the immediate cookie, but all the cookies it can get across its whole “life.”

The Hugging Face Deep RL Course uses a great example: imagine a character in a platformer game. The agent is the character. The environment is the level with platforms, enemies, and coins. The reward is +1 for a coin, -1 for touching an enemy. The agent has to figure out how to move to collect coins while avoiding enemies — without being told the rules.

Here’s what this looks like in code. A minimal skeleton that mirrors the story:

import random

class Agent:
    """The decision-maker. Starts knowing nothing."""
    def __init__(self, name="Toddler"):
        self.name = name
        self.total_reward = 0

    def choose_action(self, possible_actions):
        """Pick an action. For now, just random."""
        return random.choice(possible_actions)

    def learn(self, reward):
        """Update based on feedback. (We'll fill this in later.)"""
        self.total_reward += reward


class Environment:
    """The world the agent interacts with."""
    def __init__(self, name="Kitchen"):
        self.name = name

    def get_possible_actions(self):
        """What can the agent do right now?"""
        return ["reach_left", "reach_right", "grab", "wait"]

    def take_action(self, action):
        """Apply the action and return a reward."""
        # In a real environment, this would check physics, position, etc.
        # For now, we just simulate a random outcome.
        if action == "grab":
            reward = 10 if random.random() > 0.5 else -10
        else:
            reward = -1  # didn't try to grab, small penalty
        return reward

# Let's see them interact once
toddler = Agent()
kitchen = Environment()

action = toddler.choose_action(kitchen.get_possible_actions())
reward = kitchen.take_action(action)
toddler.learn(reward)

print(f"{toddler.name} chose '{action}' and got reward {reward}")
print(f"Total reward so far: {toddler.total_reward}")

This is the simplest possible RL system. The agent picks an action randomly, the environment gives back a number, and the agent remembers it. That’s the core loop. Everything else in RL is about making that loop smarter.

The Loop: How an Agent Actually Learns

Now let’s zoom in on the interaction cycle. It’s a repeating loop that looks the same whether you’re training a robot, a chess AI, or a self-driving car:

1. Observe state: The agent looks at the environment and gets some information. For the toddler: “My hand is 10 cm from the jar.”
2. Choose action: Based on its current strategy (called a policy), the agent picks an action. “Reach forward.”
3. Receive reward: The environment says how good that action was. “+5 for getting closer.”
4. Observe new state: The environment has changed. “My hand is now 5 cm from the jar.”
5. Repeat from step 2.

OpenAI Spinning Up calls this the agent-environment loop. The agent doesn’t know the rules of the environment — it has to discover them through trial and error. The state is the information available at a given moment. The action is a move the agent can make. The policy is the agent’s strategy for choosing actions given the current state.

Here’s what the loop looks like in Python:

import random

# Define a simple environment: a 5-position line
# Positions: 0 (start), 1, 2, 3, 4 (goal)
class LineWorld:
    def __init__(self):
        self.state = 0  # start position
        self.goal = 4

    def get_possible_actions(self):
        return ["left", "right"]

    def step(self, action):
        """Take action, return (next_state, reward, done)."""
        if action == "right" and self.state < self.goal:
            self.state += 1
        elif action == "left" and self.state > 0:
            self.state -= 1
        # else: hit a wall, stay put

        if self.state == self.goal:
            reward = 10
            done = True
        else:
            reward = -1  # small penalty for each step
            done = False

        return self.state, reward, done


# The agent (random policy)
class RandomAgent:
    def choose_action(self, possible_actions):
        return random.choice(possible_actions)


# Run one episode (one complete attempt)
env = LineWorld()
agent = RandomAgent()

total_reward = 0
done = False
step = 0

print("Starting the agent-environment loop...")
print(f"Initial state: position {env.state}")
print()

while not done:
    step += 1
    possible_actions = env.get_possible_actions()
    action = agent.choose_action(possible_actions)
    next_state, reward, done = env.step(action)
    total_reward += reward

    print(f"Step {step}: State={env.state}, Action={action}, "
          f"Reward={reward}, Next state={next_state}, Done={done}")

print(f"\nEpisode finished. Total reward: {total_reward}")

When you run this, you’ll see the agent stumble around randomly. Sometimes it reaches the goal (+10). Sometimes it gets stuck bouncing between two positions, racking up -1 penalties. The key insight: the agent has no idea what “left” or “right” means. It’s just trying stuff. The loop is nothing more than a while loop with some logic inside.

What Makes RL Different from Other Machine Learning?

You might be thinking: “Wait, isn’t this just what a neural network does?” No — and the difference is crucial. Let’s compare RL to the two other main branches of machine learning.

Supervised learning: You have a dataset of (input, label) pairs. The model learns to predict the label from the input. Example: given an email, predict whether it’s spam. The labels are correct answers provided by a human.

Unsupervised learning: You have inputs only. The model finds structure — clusters, patterns, groupings. Example: grouping customers by purchasing behavior.

Reinforcement learning: You have an environment and a reward signal. The agent learns a sequence of actions that maximizes reward. No one tells it the “right” action at each step. The agent has to figure that out by trying things and seeing what happens.

As the IBM article explains, RL is fundamentally different because “the algorithm learns through trial and error, not from a dataset of correct answers.”

Here’s the side-by-side comparison:

# SUPERVISED LEARNING: we have correct answers
from sklearn.linear_model import LogisticRegression
import numpy as np

# Labeled dataset
X = np.array([[0], [1], [2], [3], [4]])  # positions
Y = np.array([0, 0, 0, 0, 1])  # 1 = goal reached

model = LogisticRegression()
model.fit(X, Y)  # learns from correct labels

# The model can now predict: "at position 3, probability of goal = ?"
print(f"Supervised: at position 2.5, goal probability = {model.predict_proba([[2.5]])[0][1]:.2f}")

# REINFORCEMENT LEARNING: no correct answers, just rewards
import random

class SimpleRL:
    def __init__(self):
        self.position = 0
        self.goal = 4

    def step(self, action):
        if action == "right" and self.position < self.goal:
            self.position += 1
        reward = 10 if self.position == self.goal else -1
        done = self.position == self.goal
        return self.position, reward, done

# The agent never sees a "correct" action — only rewards
env = SimpleRL()
total = 0
for _ in range(10):
    action = random.choice(["left", "right"])
    state, reward, done = env.step(action)
    total += reward
    print(f"State={state}, Action={action}, Reward={reward}")

print(f"Total reward: {total}")

Notice the difference? In the supervised version, the model sees (position, is_goal) pairs — it knows what the right answer is. In the RL version, the agent only sees rewards. It never gets told “at position 3, you should move right.” It has to discover that through trial and error.

But here’s the hardest part of RL: the agent’s own actions change the data it sees next. If the agent moves left, it ends up at a different position than if it moved right. The data distribution shifts based on the agent’s behavior. This creates the exploration-exploitation trade-off: the agent has to try new things (explore) to learn about the environment, but trying new things might lose rewards (exploit what it already knows).

As the classic Kaelbling, Littman & Moore survey puts it, RL is “learning how to act from trial-and-error interaction” — and the exploration-exploitation dilemma is what makes it fundamentally harder than supervised learning.

The Reward Signal: More Than Just a Score

Let’s dig deeper into rewards. A reward is just a number, but it carries enormous meaning. It’s the only feedback the agent gets — the carrot and the stick.

Crash Course AI has a great example: the “John Green Bot” that writes tweets. It gets +1 for every like, -1 for every block, and 0 for no reaction. The bot’s goal is to maximize likes over time. But here’s the problem: a tweet might get a like hours after it’s posted. How does the bot know which word in the tweet caused the like? That’s the credit assignment problem — figuring out which actions led to which rewards.

This brings us to two important concepts:

Return: The sum of all rewards over time. Not just the immediate reward, but everything the agent collects.

Discount factor (gamma, γ): A number between 0 and 1 that says how much the agent cares about future rewards vs. immediate ones. Gamma close to 1 means far-sighted (cares about future rewards). Gamma close to 0 means myopic (only cares about right now).

Why discount? Because a reward now is more certain than a reward later. If I offer you $10 today or$10 next year, you’d take it today. The discount factor captures that intuition.

Here’s how to compute discounted return:

def discounted_return(rewards, gamma=0.9):
    """
    Compute the discounted sum of rewards.
    Formula: G = r0 + gamma*r1 + gamma^2*r2 + ...
    """
    G = 0
    discount = 1.0  # gamma^0 = 1
    for reward in rewards:
        G += discount * reward
        discount *= gamma
    return G

# Example: agent gets +10 at step 10, but -1 each step before
rewards = [-1, -1, -1, -1, -1, -1, -1, -1, -1, 10]

print("Rewards:", rewards)
print(f"Undiscounted return: {sum(rewards)}")
print(f"Discounted return (gamma=0.9): {discounted_return(rewards, gamma=0.9):.2f}")
print(f"Discounted return (gamma=0.5): {discounted_return(rewards, gamma=0.5):.2f}")

Let’s interpret that output. With gamma=0.9, the +10 at step 10 is only worth about 3.9 in today’s terms (10 * 0.9^9 ≈ 3.87). The agent sees that distant reward as much less valuable than an immediate one. With gamma=0.5, that same +10 is worth only 0.02 — basically nothing. The agent is extremely myopic and only cares about the next few steps.

This is the hardest part of RL: the agent must learn to take actions that may not pay off immediately. The toddler reaches for the cookie even though most attempts fail. The chess player sacrifices a pawn to win the game 20 moves later. The discount factor controls how “patient” the agent is.

The Policy: The Agent’s Brain

The policy is the agent’s decision-making rule. It maps states to actions. Think of it as the agent’s brain — the thing that actually gets learned.

There are two types of policies:

Deterministic policy: Given a state, always pick the same action. “If the jar is within reach, grab it.” Simple and predictable.

Stochastic policy: Given a state, pick actions with certain probabilities. “If I’m not sure, reach with my left hand 70% of the time, right hand 30%.” This randomness is crucial for exploration.

Why would you ever want a stochastic policy? Because sometimes you need to try something random to discover a better path. If you always do what you think is best, you might never find out that a different action leads to a much bigger reward.

Here’s a stochastic policy in code:

import numpy as np

class StochasticPolicy:
    """A policy that picks actions randomly based on state."""
    def __init__(self):
        # For each state, define probabilities for ["left", "right"]
        # State 0: near left wall, more likely to go right
        # State 4: near right wall, more likely to go left
        self.probs = {
            0: [0.1, 0.9],  # 10% left, 90% right
            1: [0.3, 0.7],
            2: [0.5, 0.5],  # equally likely
            3: [0.7, 0.3],
            4: [0.9, 0.1],
        }
        self.actions = ["left", "right"]

    def choose_action(self, state):
        """Sample an action based on the probability distribution."""
        probs = self.probs.get(state, [0.5, 0.5])
        return np.random.choice(self.actions, p=probs)

    def get_action_probs(self, state):
        """Return the probability of each action (for learning)."""
        return self.probs.get(state, [0.5, 0.5])


# Test it
policy = StochasticPolicy()

print("Testing stochastic policy:")
for state in range(5):
    action = policy.choose_action(state)
    probs = policy.get_action_probs(state)
    print(f"State {state}: action={action}, "
          f"P(left)={probs[0]:.1f}, P(right)={probs[1]:.1f}")

# Run 10 steps to see the randomness
print("\n10 steps of random action selection:")
state = 2  # start in the middle
for step in range(10):
    action = policy.choose_action(state)
    print(f"Step {step}: state={state}, action={action}")
    # Update state (simplified)
    if action == "right" and state < 4:
        state += 1
    elif action == "left" and state > 0:
        state -= 1

Notice how the policy is just a weighted coin flip. The probabilities encode what the agent has learned so far. A good policy has high probability on actions that lead to high reward. A random policy (all actions equally likely) is the starting point — the agent knows nothing.

Putting It All Together: A Complete (Toy) RL Problem

Now you have all the pieces. Let’s build a complete, runnable toy RL problem: a 1D grid world where an agent must reach a goal (+10) while avoiding a pit (-10). The agent can move left or right. We’ll use a random policy so you can see the loop in action without any learning yet.

import random

class GridWorld:
    """A 1D world with a goal at one end and a pit at the other."""
    def __init__(self, size=5):
        self.size = size
        self.positions = list(range(size))
        self.goal = size - 1  # rightmost position
        self.pit = 0  # leftmost position
        self.current_pos = size // 2  # start in the middle

    def reset(self):
        """Start a new episode."""
        self.current_pos = self.size // 2
        return self.current_pos

    def get_possible_actions(self):
        return ["left", "right"]

    def step(self, action):
        """
        Take an action and return (next_state, reward, done).
        """
        if action == "right" and self.current_pos < self.size - 1:
            self.current_pos += 1
        elif action == "left" and self.current_pos > 0:
            self.current_pos -= 1
        # else: hit a wall, stay put

        # Check if we reached goal or pit
        if self.current_pos == self.goal:
            reward = 10
            done = True
        elif self.current_pos == self.pit:
            reward = -10
            done = True
        else:
            reward = -1  # small step penalty
            done = False

        return self.current_pos, reward, done


class RandomAgent:
    """Agent that picks actions uniformly at random."""
    def choose_action(self, possible_actions):
        return random.choice(possible_actions)


# Run one episode
env = GridWorld(size=5)
agent = RandomAgent()

state = env.reset()
total_reward = 0
done = False
step = 0

print("=" * 50)
print("GRID WORLD EPISODE")
print("=" * 50)
print(f"World: positions 0 to {env.size-1}")
print(f"Goal: position {env.goal} (+10 reward)")
print(f"Pit: position {env.pit} (-10 reward)")
print(f"Start: position {state}")
print()

while not done:
    step += 1
    possible_actions = env.get_possible_actions()
    action = agent.choose_action(possible_actions)
    next_state, reward, done = env.step(action)
    total_reward += reward

    print(f"Step {step}:")
    print(f"  State: {state}")
    print(f"  Action: {action}")
    print(f"  Reward: {reward:3d}")
    print(f"  Next state: {next_state}")
    print(f"  Done: {done}")
    print()

    state = next_state

print(f"Episode finished after {step} steps.")
print(f"Total reward: {total_reward}")

if total_reward > 0:
    print("The agent reached the goal! (Pure luck, since it's random.)")
else:
    print("The agent fell into the pit or wandered aimlessly.")
    print("It learned nothing yet — it's just guessing.")

When you run this, you’ll see the agent stumble around. Sometimes it reaches the goal by pure luck. Sometimes it falls into the pit. Most of the time it just bounces around collecting -1 penalties until it hits something.

This is the “aha” moment: the agent is doing RL, but it’s not learning yet. It’s just exploring randomly. The magic of RL is in the algorithms that take this random exploration and turn it into a smart policy. That’s what we’ll cover in Part 2.

The Bigger Picture: Where Does This Series Go?

Let’s recap what you’ve learned:

• Reinforcement learning is learning from interaction, not from labeled examples.
• The three ingredients are agent, environment, and reward.
• The agent-environment loop is a cycle of observe → act → receive reward → repeat.
• RL differs from supervised/unsupervised learning because there are no correct answers, the data is generated by the agent’s own actions, and the agent must balance exploration vs. exploitation.
• The reward signal is the only feedback. Return is the cumulative discounted reward. The discount factor controls how far-sighted the agent is.
• The policy is the agent’s strategy for choosing actions. It can be deterministic or stochastic.
• A complete RL problem has all these pieces working together in a loop.

Where do we go from here?

In Part 2, we’ll answer the big question: how do we actually improve the policy? We’ll introduce value functions — a way to estimate how good each state or action is — and Q-learning, the algorithm that lets the agent learn from its experience.

Later in the series, we’ll combine RL with deep neural networks (deep RL), which is how systems like AlphaGo and self-driving cars work. As the DeepMind blog explains, deep RL has achieved superhuman performance in games like Atari and Go by using neural networks as function approximators for the policy and value functions.

But before we get to the fancy stuff, you need to understand the fundamentals. And you just built them from scratch. The toddler and the cookie jar, the grid world with its goal and pit — these are the same principles that power the most advanced AI systems in the world.

You’re ready for Part 2.

Check Your Understanding

Remember: What are the three core components of any reinforcement learning problem?

Understand: Explain in your own words why RL is different from supervised learning. Why can’t you just use a neural network to solve an RL problem?

Apply: Given the reward sequence [-1, -1, -1, 10, -1, -1, 10], compute the discounted return with gamma=0.9. What does this number tell you about the agent’s performance?

Analyze: In the GridWorld example, the agent used a random policy. What would happen if the agent always chose “right”? What if it always chose “left”? Which policy would get the highest reward on average?

Evaluate: A friend says “RL is just trial and error — it’s not real learning.” Based on what you’ve learned, how would you respond? What makes RL more sophisticated than simple trial and error?

Create: Design a simple RL problem of your own (not a grid world). Describe the agent, environment, state, actions, and reward function. What makes it a good problem for RL?

Apply What You Learned

**Task**: Modify the GridWorld code from this article to create a new environment with: - 10 positions instead of 5 - Two goals (at positions 3 and 8) each giving +5 reward - Two pits (at positions 2 and 6) each giving -5 reward - A step penalty of -0.5

Run the random agent for 5 episodes and record the total reward for each. Then change the step penalty to -2 and run again. What do you observe about how the step penalty affects the agent’s behavior?

Deliverable: A Python script that defines your new environment, runs 5 episodes with a random agent, and prints the total reward for each episode.

Self-check:

• The environment has exactly 10 positions
• Goals and pits are at the specified positions
• Rewards match the specification (+5, -5, -0.5 or -2)
• The agent starts at position 5 (middle)
• Output shows 5 episodes with different total rewards
• You can explain why the step penalty changes the agent’s “success rate”