Skip to main content

Agents and Environments

The fundamental concepts of Reinforcement Learning revolve around the interaction between agents and environments. Here you'll learn the key definitions.

🤖 Agent​

An agent is the entity that learns and makes decisions in an environment. In MLVisual, our agents are the ants that learn to collect food.

Characteristics of an Agent​

  • Learns through experience
  • Makes decisions based on current state
  • Receives rewards for its actions
  • Improves performance over time

Example in Ants Saga​

In our project, the ant is the agent that must:

  • Decide which direction to move
  • Learn to avoid obstacles
  • Maximize food collection

🌍 Environment​

The environment is the world in which the agent interacts. It includes everything the agent can observe and interact with.

Characteristics of an Environment​

  • Provides states - Information about the current situation
  • Receives actions - From the agent
  • Returns rewards - Feedback on action quality
  • Transitions - Changes state based on actions

Example in Ants Saga​

Our environment includes:

  • Map layout - Walls, food, obstacles
  • Physics - Movement, collisions
  • Rules - How the world behaves

📊 States​

A state is a snapshot of the environment at a specific moment. It contains all the information the agent needs to make decisions.

State Representation​

  • Discrete - Finite set of possible states
  • Continuous - Infinite set of possible states
  • Partial - Agent can't see everything
  • Full - Agent can see everything

Example in Ants Saga​

state = {
    'ant_position': (x, y),
    'food_positions': [(x1, y1), (x2, y2), ...],
    'obstacle_positions': [(x1, y1), (x2, y2), ...],
    'ant_health': 100,
    'food_collected': 5
}

🎮 Actions​

An action is a decision the agent can make. The set of all possible actions is called the action space.

Action Types​

  • Discrete - Finite set of actions (left, right, up, down)
  • Continuous - Infinite set of actions (move 0.5 units left)
  • Multi-dimensional - Multiple actions at once

Example in Ants Saga​

actions = {
    'move_up': (0, 1),
    'move_down': (0, -1),
    'move_left': (-1, 0),
    'move_right': (1, 0),
    'stay': (0, 0)
}

🔄 Agent-Environment Loop​

The learning process follows this cycle:

  1. Agent observes the current state
  2. Agent chooses an action based on its policy
  3. Environment receives the action
  4. Environment transitions to a new state
  5. Environment provides a reward
  6. Agent updates its policy based on the reward
  7. Repeat until learning is complete

🎯 Learning Objectives​

Goal​

The agent learns to maximize cumulative reward over time.

Methods​

  • Value-based - Learn value functions (Q-learning)
  • Policy-based - Learn policies directly (REINFORCE)
  • Actor-Critic - Combine both approaches

📚 Further Reading​