Scraping & Automation

A collection of data-collection and reporting-automation projects in Python and PHP. Each folder is a standalone project.

Project	What it does	Stack
Gulp_Scraping	Collects freelancer profiles from a freelance marketplace. A console command gathers the list pages, then queues one message per profile so detail pages are fetched and parsed asynchronously.	PHP, Symfony Console, Symfony Messenger, Goutte
Disney_Scraping	Collects the Disney+ home-page catalog (collections, sets and titles) through the platform's GraphQL search API, including token refresh.	Python, requests
Google_Data_Studio_Automation	Automates repetitive edits to Google Data Studio reports: adds calculated fields (unit conversions, price per kilo) and English/Spanish translations of workflow states.	Python, Selenium
Gmail_Scraping	Flask service for campaign-report automation, backed by MySQL.	Python, Flask, MySQL

Highlights

• Queue-based scraping: Gulp_Scraping separates list crawling from detail parsing with a message bus, so slow pages don't block the crawl, and each parser (FreelancerListParser, FreelancerDetailsParser) can be tested on its own.
• API-first collection: Disney_Scraping calls the same GraphQL endpoints the web client uses instead of parsing HTML, which is faster and less fragile.
• Browser automation for tools without an API: Google_Data_Studio_Automation drives the report editor with explicit waits to apply the same field definitions across reports.

Running a project

Each project has its own dependencies:

# Python projects
pip install -r requirements.txt   # where provided
python app.py

# Gulp_Scraping
composer install
php bin/console list