You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Forward crawler status messages to the run status message #1188
crawlee-python crawlers emit Event.CRAWLER_STATUS with a status line such as "Crawled 120/500 pages, 3 failed requests, desired concurrency 8." on every status interval (source). Nothing in the Python SDK listens to it, so a crawler running inside an Actor never updates the run's status message in Console unless the user wires it up.
The JS SDK subscribes to crawlee's STATUS_MESSAGE event in Actor.init() and forwards each message to the run without logging it again (source). On exit it unsubscribes after the listeners finish, so a late crawler message cannot overwrite the terminal status message (source).
Acceptance criteria
On the platform, the Actor registers a CRAWLER_STATUS listener during init that updates the run status message, without logging it a second time.
The listener is removed before the terminal status message is set on exit.
A failed update logs a warning and does not break the crawler.
Implementation notes
In _ActorType.__aenter__, after the event manager is entered (_actor.py#L219), register a crawlee.events.Event.CRAWLER_STATUS listener only when is_at_home(). Crawlers emit on the same event manager through service_locator, so no crawler changes are needed.
The listener calls set_status_message(data.message) (non-terminal). Crawlee already logs the status line, and set_status_message logs only off the platform, so nothing is logged twice. Catch exceptions in the listener and log a warning.
In __aexit__, remove the listener before finalize() sets the terminal status message (#L295-L300), so a late crawler message can't overwrite it.
Add unit tests to tests/unit/actor/test_actor_lifecycle.py: an emitted CRAWLER_STATUS event updates the run status message on the platform, nothing happens locally, and no update follows the terminal message.
Activity