Add automatic browser streaming and streaming controllers - #7
Draft
Ismola wants to merge 1 commit into
Draft
Conversation
Introduces automatic browser streaming capabilities, including MP4 recording of browser sessions, configurable via environment variables. Adds new streaming controllers and demo endpoints, updates Dockerfile and compose.yaml for FFmpeg and streaming support, and documents streaming features in STREAMING_GUIDE.md and README.md. Updates .env.example and .gitignore for streaming configuration and output files, and refactors web_driver.py for streaming integration.
There was a problem hiding this comment.
Pull Request Overview
This PR introduces automatic browser streaming capabilities to the selenium-scraper-quickstarter, enabling MP4 recording of browser automation sessions. The streaming functionality is opt-in via environment variables and includes both internal (base64 frame streaming) and external (FFmpeg-based video recording) implementations, along with API endpoints for stream control and comprehensive documentation.
Key Changes
- Adds automatic browser streaming that starts when
STREAMING_ENABLED=trueand records sessions to MP4 files - Introduces new streaming service modules (
streaming_service.pyandexternal_streaming_service.py) with FFmpeg integration - Adds streaming control endpoints (
/stream/status,/stream/stop) and demo controllers for showcasing streaming features
Reviewed Changes
Copilot reviewed 15 out of 17 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| utils/streaming_service.py | New internal streaming service for base64-encoded frame capture and distribution |
| utils/external_streaming_service.py | New FFmpeg-based streaming service for video recording with RTMP/HTTP/file output support |
| utils/config.py | Adds streaming configuration variables (FPS, quality, resolution, protocols) |
| main.py | Adds streaming status and control endpoints |
| controller/streaming_demo_controller.py | Demo controllers showcasing streaming integration with Google search and scraping examples |
| controller/streaming_controller.py | Controllers for managing streaming sessions and browser actions |
| controller/external_streaming_controller.py | Controllers for external streaming with FFmpeg |
| controller/controller_sample.py | Minor comment update |
| compose.yaml | Adds streaming configuration, volume mount for videos, and updated healthcheck |
| actions/web_driver.py | Integrates automatic streaming initialization in browser setup |
| STREAMING_GUIDE.md | Comprehensive documentation for streaming features |
| README.md | Updates main documentation with streaming features and examples |
| Dockerfile | Migrates from Alpine to Debian slim, adds FFmpeg and GUI support with X11 |
| .env.example | Adds streaming configuration examples |
| .devcontainer/devcontainer.json | Updates port from 3000 to 3010 |
Comments suppressed due to low confidence (1)
STREAMING_GUIDE.md:1
- Documentation inconsistency: The rest of the document is in English but this section and subsequent content (lines 414-442) are in Spanish. Maintain consistent language throughout the documentation.
# 🎥 Selenium Scraper Streaming Guide
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
Comment on lines
+21
to
+32
| # Use the driver path we know works | ||
| driver_path = "/root/.wdm/drivers/chromedriver/linux64/141.0.7390.122/chromedriver-linux64/chromedriver" | ||
|
|
||
| # Always use our installed driver for now | ||
| if os.path.exists(driver_path): | ||
| service = Service(driver_path) | ||
| # Use google-chrome-stable | ||
| options.binary_location = '/usr/bin/google-chrome-stable' | ||
| else: | ||
| # Fallback to webdriver-manager | ||
| service = Service(ChromeDriverManager().install()) | ||
|
|
There was a problem hiding this comment.
Hardcoded driver path may break if ChromeDriver version changes. Consider using webdriver-manager for dynamic path resolution or reading from configuration.
Suggested change
| # Use the driver path we know works | |
| driver_path = "/root/.wdm/drivers/chromedriver/linux64/141.0.7390.122/chromedriver-linux64/chromedriver" | |
| # Always use our installed driver for now | |
| if os.path.exists(driver_path): | |
| service = Service(driver_path) | |
| # Use google-chrome-stable | |
| options.binary_location = '/usr/bin/google-chrome-stable' | |
| else: | |
| # Fallback to webdriver-manager | |
| service = Service(ChromeDriverManager().install()) | |
| # Use webdriver-manager to dynamically resolve ChromeDriver path | |
| service = Service(ChromeDriverManager().install()) | |
| # Use google-chrome-stable | |
| options.binary_location = '/usr/bin/google-chrome-stable' |
| _streaming_service.set_driver(driver) | ||
| config = { | ||
| "protocol": "file", | ||
| "output_file": "/tmp/current_session.mp4", |
There was a problem hiding this comment.
The output file path is hardcoded and duplicated (lines 65 and 108). Consider using OUTPUT_FILE_PATH from config.py for consistency.
| output_file = config.get("output_file", OUTPUT_FILE_PATH) | ||
|
|
||
| # Ensure directory exists | ||
| os.makedirs(os.path.dirname(output_file), exist_ok=True) |
There was a problem hiding this comment.
os.path.dirname() will return an empty string if output_file has no directory component (e.g., 'video.mp4'), causing os.makedirs to fail or create unintended directories. Verify output_file has a directory component before calling makedirs.
Suggested change
| os.makedirs(os.path.dirname(output_file), exist_ok=True) | |
| output_dir = os.path.dirname(output_file) | |
| if output_dir: | |
| os.makedirs(output_dir, exist_ok=True) |
| ) | ||
| time.sleep(1) | ||
|
|
||
| except: |
There was a problem hiding this comment.
Bare except clause catches all exceptions including system exits. Use 'except Exception:' to catch only standard exceptions.
Suggested change
| except: | |
| except Exception: |
Comment on lines
+17
to
+18
| RUN wget -q -O - https://dl.google.com/linux/linux_signing_key.pub | apt-key add - \ | ||
| && echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google-chrome.list \ |
There was a problem hiding this comment.
apt-key is deprecated. Consider using the recommended approach with signed-by in sources.list.d for adding repository keys.
Suggested change
| RUN wget -q -O - https://dl.google.com/linux/linux_signing_key.pub | apt-key add - \ | |
| && echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google-chrome.list \ | |
| RUN wget -q -O /usr/share/keyrings/google-chrome.gpg https://dl.google.com/linux/linux_signing_key.pub \ | |
| && echo "deb [arch=amd64 signed-by=/usr/share/keyrings/google-chrome.gpg] http://dl.google.com/linux/chrome/deb/ stable main" > /etc/apt/sources.list.d/google-chrome.list \ |
Comment on lines
+414
to
+440
| Este proyecto incluye capacidades avanzadas de streaming en tiempo real para visualizar las automatizaciones del navegador mientras se ejecutan. | ||
|
|
||
| ### ✨ Características del Streaming | ||
|
|
||
| - **🎬 Vista en tiempo real:** Observa el navegador mientras automatiza tareas | ||
| - **🎮 Control remoto:** Navega y ejecuta acciones desde la interfaz web | ||
| - **📱 Interfaz responsive:** Viewer web optimizado para diferentes dispositivos | ||
| - **🔍 Destacado de elementos:** Resalta elementos antes de interactuar | ||
| - **⚙️ Calidad configurable:** Ajusta FPS y calidad según necesidades | ||
| - **🐳 Docker ready:** Funciona en contenedores con display virtual | ||
| - **📊 Demos incluidos:** Ejemplos listos para usar | ||
|
|
||
| ### 🚀 Casos de Uso | ||
|
|
||
| - **Debugging:** Ve qué hace tu bot paso a paso | ||
| - **Demostraciones:** Muestra automatizaciones a clientes/equipos | ||
| - **Monitoreo:** Supervisa bots de larga duración | ||
| - **Desarrollo:** Desarrolla scrapers de forma visual | ||
| - **Educación:** Enseña automatización web | ||
|
|
||
| ### 📖 Documentación Completa | ||
|
|
||
| Para guías detalladas, ejemplos de código y configuración avanzada, consulta: | ||
|
|
||
| - **[STREAMING_GUIDE.md](STREAMING_GUIDE.md)** - Guía completa de streaming | ||
| - **Interfaz Web:** <http://localhost:3000/stream/viewer> | ||
| - **API Reference:** Endpoints de streaming documentados arriba |
There was a problem hiding this comment.
Documentation inconsistency: This section (lines 414-442) is in Spanish while the rest of the README is in English. Maintain consistent language throughout.
Suggested change
| Este proyecto incluye capacidades avanzadas de streaming en tiempo real para visualizar las automatizaciones del navegador mientras se ejecutan. | |
| ### ✨ Características del Streaming | |
| - **🎬 Vista en tiempo real:** Observa el navegador mientras automatiza tareas | |
| - **🎮 Control remoto:** Navega y ejecuta acciones desde la interfaz web | |
| - **📱 Interfaz responsive:** Viewer web optimizado para diferentes dispositivos | |
| - **🔍 Destacado de elementos:** Resalta elementos antes de interactuar | |
| - **⚙️ Calidad configurable:** Ajusta FPS y calidad según necesidades | |
| - **🐳 Docker ready:** Funciona en contenedores con display virtual | |
| - **📊 Demos incluidos:** Ejemplos listos para usar | |
| ### 🚀 Casos de Uso | |
| - **Debugging:** Ve qué hace tu bot paso a paso | |
| - **Demostraciones:** Muestra automatizaciones a clientes/equipos | |
| - **Monitoreo:** Supervisa bots de larga duración | |
| - **Desarrollo:** Desarrolla scrapers de forma visual | |
| - **Educación:** Enseña automatización web | |
| ### 📖 Documentación Completa | |
| Para guías detalladas, ejemplos de código y configuración avanzada, consulta: | |
| - **[STREAMING_GUIDE.md](STREAMING_GUIDE.md)** - Guía completa de streaming | |
| - **Interfaz Web:** <http://localhost:3000/stream/viewer> | |
| - **API Reference:** Endpoints de streaming documentados arriba | |
| This project includes advanced real-time streaming capabilities to visualize browser automations as they run. | |
| ### ✨ Streaming Features | |
| - **🎬 Real-time view:** Watch the browser as it automates tasks | |
| - **🎮 Remote control:** Navigate and execute actions from the web interface | |
| - **📱 Responsive interface:** Web viewer optimized for different devices | |
| - **🔍 Element highlighting:** Highlights elements before interacting | |
| - **⚙️ Configurable quality:** Adjust FPS and quality as needed | |
| - **🐳 Docker ready:** Works in containers with virtual display | |
| - **📊 Included demos:** Ready-to-use examples | |
| ### 🚀 Use Cases | |
| - **Debugging:** See what your bot does step by step | |
| - **Demonstrations:** Showcase automations to clients/teams | |
| - **Monitoring:** Supervise long-running bots | |
| - **Development:** Build scrapers visually | |
| - **Education:** Teach web automation | |
| ### 📖 Full Documentation | |
| For detailed guides, code examples, and advanced configuration, see: | |
| - **[STREAMING_GUIDE.md](STREAMING_GUIDE.md)** - Complete streaming guide | |
| - **Web Interface:** <http://localhost:3000/stream/viewer> | |
| - **API Reference:** Streaming endpoints documented above |
Ismola
marked this pull request as draft
February 5, 2026 10:43
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Introduces automatic browser streaming capabilities, including MP4 recording of browser sessions, configurable via environment variables. Adds new streaming controllers and demo endpoints, updates Dockerfile and compose.yaml for FFmpeg and streaming support, and documents streaming features in STREAMING_GUIDE.md and README.md. Updates .env.example and .gitignore for streaming configuration and output files, and refactors web_driver.py for streaming integration.