Problem: Job submission succeeds but progress remains stuck at 0% indefinitely. Terminal/logs show activity but UI never updates. Job appears 'running' but never completes.\n\nImpact: Users cannot track job progress or know when scraping completes. Creates impression that app is frozen/broken.\n\nSymptoms:\n- POST /api/submit returns job ID successfully\n- GET /jobs/{id} shows status 'running' with 0% progress\n- Backend logs show scraping activity\n- Progress never increments; job never reaches 'completed' state\n- /jobs/{id}/log endpoint may or may not stream logs\n\nSuspected Causes:\n- Supervisor thread may not be starting or processing queue\n- Job state updates not persisting to shared JOBS dict\n- Race condition in threading/locking\n- Progress calculation logic broken\n\nAcceptance Criteria:\n- ✅ Job progresses from 0% → 100% as steps complete\n- ✅ Status transitions: pending → running → completed/failed\n- ✅ Add health check endpoint that verifies supervisor is alive\n- ✅ Add debug logging for job state transitions\n- ✅ E2E test: submit job → poll until completion → verify final state\n\nPriority: P0 - Core functionality broken
Problem: Job submission succeeds but progress remains stuck at 0% indefinitely. Terminal/logs show activity but UI never updates. Job appears 'running' but never completes.\n\nImpact: Users cannot track job progress or know when scraping completes. Creates impression that app is frozen/broken.\n\nSymptoms:\n- POST /api/submit returns job ID successfully\n- GET /jobs/{id} shows status 'running' with 0% progress\n- Backend logs show scraping activity\n- Progress never increments; job never reaches 'completed' state\n- /jobs/{id}/log endpoint may or may not stream logs\n\nSuspected Causes:\n- Supervisor thread may not be starting or processing queue\n- Job state updates not persisting to shared JOBS dict\n- Race condition in threading/locking\n- Progress calculation logic broken\n\nAcceptance Criteria:\n- ✅ Job progresses from 0% → 100% as steps complete\n- ✅ Status transitions: pending → running → completed/failed\n- ✅ Add health check endpoint that verifies supervisor is alive\n- ✅ Add debug logging for job state transitions\n- ✅ E2E test: submit job → poll until completion → verify final state\n\nPriority: P0 - Core functionality broken