🧠 CartPole Reinforcement Learning Project

This project implements a from-scratch CartPole environment using physics, visualizes it with matplotlib, and trains a reinforcement learning agent (PPO) to balance the pole. It also records videos of both random and trained agents for comparison.

---

📦 Project Structure

cartpole_rl/
├── env.py                 # Custom Gymnasium-compatible CartPole environment
├── train.py               # Trains PPO agent using Stable Baselines3
├── visualize.py           # Realtime cart-pole animation using matplotlib
├── visualize_and_save.py  # Saves trained agent animation to MP4 (with reward plot)
├── record_random_policy.py# Saves random policy animation to MP4
├── ppo_cartpole_custom.zip# Trained PPO model (after training)

---

🧰 Setup

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Requirements:

gymnasium

stable-baselines3

matplotlib

opencv-python

numpy

🚀 Usage

1. Train the PPO Agent

python train.py

2. Visualize Trained Agent in Real-Time

python visualize.py

3. Save Trained Agent Video (with rewards)

python visualize_trained.py

4. Save Random Policy Video (for comparison)

python record_random_policy.py

🎥 Output Videos

cartpole_rl.mp4 — Trained agent balancing the pole

cartpole_random.mp4 — Untrained (random) agent failing quickly

Each video shows:

Left: live cart-pole animation

Right: reward plot over time

📚 Concepts Demonstrated

Physics-based custom CartPole environment

Gymnasium API compliance

PPO reinforcement learning with Stable Baselines3

Live and saved visualizations

OpenCV + matplotlib integration