RSERV - Remote Surveillance and Exploration Rover Vehicle

A reinforcement learning-powered autonomous rover built on ESP32-CAM hardware.

What is RSERV

RSERV is an ESP32-CAM based robotic rover that uses RL to explore uncharted terrain autonomously. The rover connects wirelessly to a base station that handles the computationally heavy reinforcement learning training while the lightweight ESP32-CAM handles sensing and actuation.

3D Model

The physical design files are in the 3D_Models/ directory in STL and Shapr3D formats.

Tech Stack

• Hardware: ESP32-CAM microcontroller with integrated camera and WiFi
• Firmware: Arduino sketch (see arduino_sketch/RSERV.ino)
• ML Framework: Stable-Baselines3 with PPO algorithm
• Backend: Python with Flask for web interface
• Communication: HTTP-based API between rover and base station

Concept

Hierarchical Computing Architecture with Base Stations

In this development setup, the local computer acts as the base station. But in a real-world deployment:

• Dedicated base station for a cluster of rovers.
• Rovers are inexpensive and offload compute to the base station
• Base stations coordinate multiple rovers in parallel, making it horizontally scalable as well.

Reinforcement Learning

I used PPO (Proximal Policy Optimization) to train the rover. PPO is an RL algo that's good at learning policies for robotic control.

Architecture

flowchart LR
    subgraph Rover["ESP32-CAM Rover"]
        direction TB
        CAM[Camera]
        MOT[Motors]
        SENS[Sensors]
    end
    
    subgraph Base["Base Station - Python"]
        direction TB
        RL[RL Server Flask]
        PPO[PPO Training SB3]
    end
    
    subgraph Storage["Persistent Storage"]
        direction TB
        LOGS[logs/]
        MODELS[saved_models/]
    end
    
    Rover <--> |WiFi/HTTP<br/>Telemetry| Base
    Base --> |Training Data| Storage
    Storage --> |Load Models| Base
    RL <--> PPO

Loading

Key Components:

• robot_api.py - API interface for rover communication
• robot_env.py - Gym environment wrapper for RL
• train.py - Training script using PPO
• rl_server.py - Flask server for real-time control and monitoring
• arduino_sketch/RSERV.ino - ESP32-CAM firmware

Project Structure

• 3D_Models/ - stl files for 3D printing the rover chassis
• arduino_sketch/ - ESP32-CAM firmware
• logs/ - tensorboard training logs
• saved_models/ - trained PPO models
• static/ & templates/ - web interface assets
• captured_samples/ - camera frame samples from the rover