AutoRC: Autonomous RC Car Navigation Using Computer Vision and Deep Learning

A Hybrid Edge-Cloud Architecture for Real-Time Autonomous Navigation

McMaster University -- SEP 742: Deep Learning and Applications

Overview

AutoRC is an autonomous RC car platform that combines classical computer vision with deep learning through a hybrid edge-cloud architecture. A Raspberry Pi 5 performs real-time lane detection and vehicle control at the edge, while a remote PC runs YOLOv8 inference for traffic sign recognition. The two nodes communicate over a dual-channel TCP socket link, enabling the system to maintain responsive steering control locally while leveraging GPU-accelerated perception remotely.

Architecture

flowchart LR
    subgraph Pi["Raspberry Pi 5 (Edge)"]
        CAM[OAK Camera<br/>640x480 @ 30fps]
        LD[Lane Detector<br/>Contour Centroid]
        PD[PD Controller<br/>Kp=0.5  Kd=0.1]
        PWM[Hardware PWM<br/>GPIO 12 + 13]
        CAM --> LD --> PD --> PWM
    end

    subgraph PC["Remote PC (Cloud)"]
        YOLO[YOLOv8-nano<br/>Traffic Sign Detection]
        CMD[Command Mapper<br/>STOP / SLOW / FAST]
        YOLO --> CMD
    end

    Pi -->|"TCP:8888<br/>Video Stream"| PC
    PC -->|"TCP:8899<br/>Control Commands"| Pi

Loading

Project Structure

AutoRC/
├── current/                    # Active codebase (start here)
│   ├── pi/
│   │   ├── main.py             # Pi main entry - lane keeping + motor control
│   │   ├── stream.py           # Video streaming (OAK / picamera2 / OpenCV)
│   │   └── cmd_receiver.py     # TCP command receiver with graceful degradation
│   ├── pc/
│   │   ├── yolo_server.py      # YOLO inference server (recommended)
│   │   ├── vlm_server.py       # VLM API server (alternative)
│   │   └── video_viewer.py     # Standalone video viewer for debugging
│   └── core/
│       ├── lane_detector.py    # Dual-line contour centroid lane detection
│       └── data_collector.py   # Async data collection for behavioral cloning
├── pc/
│   └── best.pt                 # YOLOv8 trained model (not tracked in Git)
├── test_signs/                 # Printed traffic sign images for testing
├── deploy/                     # systemd service files
├── docs/                       # Technical documentation
├── requirements-pi.txt         # Pi-side Python dependencies
├── requirements-pc.txt         # PC-side Python dependencies
├── experiments/                # Archived experimental code
├── legacy/                     # Archived legacy code
└── README.md

Note: Directories hybrid/, new/, src/, and examples/ contain archived code from earlier development phases. They are preserved for reference but are not part of the active system. All current development is in current/.

Hardware Requirements

Component	Specification	Role
Single-Board Computer	Raspberry Pi 5 (4 GB RAM)	Edge processing, lane detection, motor control
Camera	OAK (DepthAI) / Pi Camera Module v2	Forward-facing visual perception (640x480 @ 30fps)
Steering Servo	SG90 / 25kg Digital Servo (500-2500 us PWM)	Lateral steering actuation (GPIO 12)
Drive Motor + ESC	Brushed DC motor with ESC	Longitudinal velocity control (GPIO 13)
Remote Workstation	x86-64 PC with NVIDIA GPU (CUDA)	YOLOv8 inference for traffic sign detection
Power Supply	7.4V 2S LiPo battery	Vehicle power
Networking	802.11ac Wi-Fi	TCP socket communication

Software Dependencies

Raspberry Pi:

pip install -r requirements-pi.txt

PC (with NVIDIA GPU):

pip install -r requirements-pc.txt

Quick Start

1. PC Side -- Start YOLO Server

cd AutoRC
python current/pc/yolo_server.py

The server will load pc/best.pt and wait for the Pi to connect on ports 8888 (video) and 8899 (commands).

Note: The best.pt model file is not tracked in Git due to its size. Train your own model following the instructions in the report, or contact the team for the pre-trained weights.

2. Pi Side -- Start Autonomous Driving

cd AutoRC
python current/pi/main.py --pc-host <PC_IP_ADDRESS>

Replace <PC_IP_ADDRESS> with the IP of the machine running yolo_server.py.

3. Standalone Mode (No PC)

python current/pi/main.py --standalone

Runs lane-keeping only, without traffic sign recognition.

4. Record and Replay Mode

Manually drive the car once to teach a path, then replay it precisely:

# Record: use keyboard (W/A/S/D) to drive, X to save
python current/pi/main.py --standalone --mode record --video ~/record.avi

# Replay: reproduce the recorded path
python current/pi/main.py --standalone --mode replay --video ~/replay.avi

# Replay with YOLO: follow path + respond to traffic signs
python current/pi/main.py --mode replay --pc-host <PC_IP> --video ~/replay.avi

5. Video Viewer (PC, no YOLO)

View the Pi's debug video stream without running inference:

python current/pc/video_viewer.py

Key Features

• Real-time lane detection using contour centroid method on HSV-filtered black tape boundaries, with dual-line centering and single-line offset tracking
• PD controller for steering with configurable proportional (Kp) and derivative (Kd) gains, providing smooth corrections through curves
• YOLOv8-nano traffic sign recognition detecting Stop signs, Speed Limit signs, and Red Light indicators with consecutive-frame confirmation
• Graceful degradation on network loss: automatically reduces speed after 1.5s timeout, stops after 5s
• Record-and-replay teaching mode for precise path reproduction via manual keyboard control
• Video recording with debug overlay showing lane contours, offset, confidence, and control state
• Asynchronous data collection module for future behavioral cloning research

Communication Protocol

The system uses a dual-channel TCP architecture to avoid head-of-line blocking:

Channel	Port	Direction	Payload
Video	8888	Pi --> PC	JPEG-compressed frames with 8-byte length header
Command	8899	PC --> Pi	UTF-8 strings: STOP, SLOW, or FAST

Command priority: STOP > SLOW > FAST. Commands require 2 consecutive detection frames for confirmation to suppress false positives.

Expected Performance

Metric	Value
Lane detection throughput (Pi)	30 FPS
YOLOv8 inference (PC, CUDA GPU)	20-33 FPS
End-to-end latency (Pi to PC and back)	53-97 ms
Traffic sign mAP@0.5	0.85-0.92
Stop sign response compliance	90-95%
Straight lane tracking success	95%

Command-Line Arguments

python current/pi/main.py [OPTIONS]

--mode {auto,record,replay}   Operating mode (default: auto)
--pc-host PC_IP               PC IP address for YOLO server
--standalone                  Run without PC connection
--kp KP                       PD controller proportional gain (default: 0.5)
--kd KD                       PD controller derivative gain (default: 0.1)
--video PATH                  Record video to file (e.g., ~/output.avi)
--path-file PATH              Record/replay path file (default: ~/autorc_path.json)
--s1 N                        Phase 1 straight frames (default: 75)
--turn N                      Phase 2 turn frames (default: 75)
--turn-steer V                Turn steering value (default: 0.75)
--s2 N                        Phase 3 straight frames (default: 60)

Team

• Shengqin Chu
• Wenjie Lu
• Wenzhe Zuo

McMaster University, Hamilton, Ontario, Canada

License

This project is licensed under the MIT License. See LICENSE for details.