Lesson 2: Environment Design

Now that your Unity project is set up, let's design the simulation environment where your RL agents will learn and interact.

🎯 Learning Objectives

By the end of this lesson, you'll have:

• A basic 3D environment for RL simulation
• Interactive elements for agent interaction
• A foundation for reward system implementation

🏗️ Environment Components

Core Elements

• Ground Plane: The base where agents move
• Walls: Obstacles and boundaries
• Food Sources: Collectible items for rewards
• Spawn Points: Where agents start each episode

Interactive Elements

• Collectibles: Items that provide positive rewards
• Obstacles: Elements that provide negative rewards
• Goals: Target locations for agents to reach

🛠️ Step-by-Step Implementation

1. Create the Ground

1. Right-click in Hierarchy
2. Select 3D Object > Plane
3. Rename to "Ground"
4. Scale to (10, 1, 10) for a larger area
5. Apply a material (create a new material if needed)

2. Add Walls

1. Create a new GameObject: 3D Object > Cube
2. Rename to "Wall"
3. Scale to (1, 2, 10) for a long wall
4. Position at the edge of the ground
5. Duplicate and position walls around the perimeter

3. Create Food Sources

1. Create a new GameObject: 3D Object > Sphere
2. Rename to "Food"
3. Scale to (0.5, 0.5, 0.5)
4. Add a bright material (yellow or green)
5. Create a prefab from this object

4. Set Up Spawn Points

1. Create empty GameObjects
2. Name them "SpawnPoint1", "SpawnPoint2", etc.
3. Position them around the environment
4. Add a visual indicator (small cube or sphere)

🎨 Visual Design

Materials and Colors

• Ground: Neutral color (gray or brown)
• Walls: Dark color (dark gray or black)
• Food: Bright, attractive color (yellow or green)
• Spawn Points: Distinctive color (blue or red)

Lighting

1. Go to Window > Rendering > Lighting
2. Set Environment Lighting to "Skybox"
3. Adjust Ambient Intensity to 0.3
4. Add additional lights if needed

🔧 Scripting Basics

Environment Manager Script

Create a new C# script called EnvironmentManager:

using UnityEngine;

public class EnvironmentManager : MonoBehaviour
{
    [Header("Environment Settings")]
    public Transform[] spawnPoints;
    public GameObject foodPrefab;
    public int foodCount = 10;
    
    private GameObject[] foodItems;
    
    void Start()
    {
        SpawnFood();
    }
    
    void SpawnFood()
    {
        foodItems = new GameObject[foodCount];
        for (int i = 0; i < foodCount; i++)
        {
            Vector3 randomPos = new Vector3(
                Random.Range(-4f, 4f),
                0.5f,
                Random.Range(-4f, 4f)
            );
            foodItems[i] = Instantiate(foodPrefab, randomPos, Quaternion.identity);
        }
    }
    
    public void ResetEnvironment()
    {
        // Reset food positions
        foreach (GameObject food in foodItems)
        {
            if (food != null)
            {
                Vector3 randomPos = new Vector3(
                    Random.Range(-4f, 4f),
                    0.5f,
                    Random.Range(-4f, 4f)
                );
                food.transform.position = randomPos;
                food.SetActive(true);
            }
        }
    }
}

✅ Testing Your Environment

1. Play the scene to test the environment
2. Verify food spawning works correctly
3. Check lighting and visual appeal
4. Test reset functionality if implemented

🎯 Next Steps

Once your environment is ready:

1. Proceed to Q-Learning Python - Next section
2. Review Getting Started if needed
3. Check Glossary for more concepts

💡 Tips

• Keep it simple: Start with basic shapes and add complexity later
• Use prefabs: Create reusable components
• Test frequently: Play the scene often to catch issues early
• Document decisions: Note why you made specific design choices

---

Excellent! Your environment is taking shape. Next, let's add the agent that will learn to navigate it!