Skip to content

Feature: Batch saved-list scraping via CLI and library #66

Description

@MozzamShahid

File / lines: src/gmaps_scraper/scraper.py (scrape_saved_list); src/gmaps_scraper/cli.py (lines 495-498)

Problem: The saved-list workflow is strictly single-URL. scrape_saved_list() takes one URL, and the CLI errors if len(args.urls) != 1 and rejects --input for --kind list. Users with many lists must write their own wrapper, while the place workflow already supports batch input, session reuse, staggered starts, and retries.

Suggested fix:

  1. Add scrape_saved_lists(urls, *, max_concurrency=1, stagger_ms=0, ...) that reuses HttpSessionConfig / BrowserSessionConfig across calls (mirroring scrape_places).
  2. In the CLI, accept multiple URLs or --input when --kind list.
  3. Emit a {"results": [...]} envelope like place batch mode.
  4. Add tests for batch list scraping and input file handling.

Example:

printf %s\n LIST_URL_1 LIST_URL_2 | uv run gmaps-scraper --kind list --input -

Difficulty: medium
Impact: Medium — expands a core workflow and improves parity with place scraping.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions