Lesson 1: Python Environment Setup

Before implementing Q-Learning, you need to set up your Python environment with all the necessary libraries and tools.

🎯 Learning Objectives

By the end of this lesson, you'll have:

• A properly configured Python environment
• All required libraries installed
• Unity-Python communication working
• A basic project structure

📋 Prerequisites

Before starting, ensure you have:

• Python 3.8+ installed
• Unity project from GameDev Unity section
• Basic understanding of Python and virtual environments

🛠️ Step-by-Step Setup

1. Create Virtual Environment

# Create a new virtual environment
python -m venv ants-saga-env

# Activate the environment
# On Windows:
ants-saga-env\Scripts\activate
# On macOS/Linux:
source ants-saga-env/bin/activate

2. Install Required Packages

# Install core ML libraries
pip install numpy pandas matplotlib seaborn

# Install RL libraries
pip install gym gymnasium

# Install Unity communication
pip install mlagents

# Install additional utilities
pip install jupyter notebook tqdm

3. Verify Installation

Create a test script test_installation.py:

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import gym
import mlagents

print("✅ All packages installed successfully!")
print(f"NumPy version: {np.__version__}")
print(f"Pandas version: {pd.__version__}")
print(f"Gym version: {gym.__version__}")

Run the test:

python test_installation.py

🏗️ Project Structure

Create the following directory structure:

ants-saga-rl/
├── environments/
│   ├── __init__.py
│   ├── ants_saga_env.py
│   └── unity_connector.py
├── agents/
│   ├── __init__.py
│   ├── q_learning.py
│   └── base_agent.py
├── utils/
│   ├── __init__.py
│   ├── visualization.py
│   └── metrics.py
├── config/
│   ├── hyperparameters.py
│   └── environment_config.py
├── notebooks/
│   └── exploration.ipynb
├── models/
│   └── .gitkeep
├── logs/
│   └── .gitkeep
└── main.py

🔧 Unity-Python Connection

1. Configure Unity for ML-Agents

In your Unity project:

1. Go to Window > Package Manager
2. Install ML-Agents package
3. Add the ML-Agents component to your agent GameObject

2. Test Connection

Create a simple connection test:

# test_unity_connection.py
import mlagents
from mlagents_envs.environment import UnityEnvironment

def test_unity_connection():
    try:
        # Connect to Unity environment
        env = UnityEnvironment(file_name="path/to/your/unity/build")
        print("✅ Unity connection successful!")
        env.close()
    except Exception as e:
        print(f"❌ Unity connection failed: {e}")

if __name__ == "__main__":
    test_unity_connection()

✅ Verification Checklist

• Virtual environment created and activated
• All packages installed without errors
• Test script runs successfully
• Project structure created
• Unity connection working (if Unity is available)

🎯 Next Steps

Once your environment is set up:

1. Proceed to Lesson 2: Q-Learning Agent
2. Review Getting Started for troubleshooting
3. Check Glossary for any unfamiliar terms

💡 Tips

• Use virtual environments: Always isolate your project dependencies
• Pin versions: Create a requirements.txt file with specific versions
• Test early: Verify each component works before moving on
• Document issues: Keep notes of any problems and solutions

📝 Requirements File

Create a requirements.txt file:

numpy==1.24.3
pandas==2.0.3
matplotlib==3.7.2
seaborn==0.12.2
gym==0.29.1
gymnasium==0.29.0
mlagents==0.30.0
jupyter==1.0.0
tqdm==4.65.0

---

Perfect! Your Python environment is ready. Let's implement Q-Learning!