Skip to content

Forward crawler status messages to the run status message #1188

Open

Description

@vdusek

crawlee-python crawlers emit Event.CRAWLER_STATUS with a status line such as "Crawled 120/500 pages, 3 failed requests, desired concurrency 8." on every status interval (source). Nothing in the Python SDK listens to it, so a crawler running inside an Actor never updates the run's status message in Console unless the user wires it up.

The JS SDK subscribes to crawlee's STATUS_MESSAGE event in Actor.init() and forwards each message to the run without logging it again (source). On exit it unsubscribes after the listeners finish, so a late crawler message cannot overwrite the terminal status message (source).

Acceptance criteria

  • On the platform, the Actor registers a CRAWLER_STATUS listener during init that updates the run status message, without logging it a second time.
  • The listener is removed before the terminal status message is set on exit.
  • A failed update logs a warning and does not break the crawler.

Implementation notes

  • In _ActorType.__aenter__, after the event manager is entered (_actor.py#L219), register a crawlee.events.Event.CRAWLER_STATUS listener only when is_at_home(). Crawlers emit on the same event manager through service_locator, so no crawler changes are needed.
  • The listener calls set_status_message(data.message) (non-terminal). Crawlee already logs the status line, and set_status_message logs only off the platform, so nothing is logged twice. Catch exceptions in the listener and log a warning.
  • In __aexit__, remove the listener before finalize() sets the terminal status message (#L295-L300), so a late crawler message can't overwrite it.
  • Add unit tests to tests/unit/actor/test_actor_lifecycle.py: an emitted CRAWLER_STATUS event updates the run status message on the platform, nothing happens locally, and no update follows the terminal message.

✍️ Drafted by Claude Code

Activity

  1. added
    enhancementNew feature or request.
    t-toolingIssues with this label are in the ownership of the tooling team.
    on Oct 16, 2026
  2. self-assigned this
    on Oct 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request.t-toolingIssues with this label are in the ownership of the tooling team.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions