Skip to content

fix(test): stream replay suite progress and cancel active runs #794

@thymikee

Description

@thymikee

Summary

agent-device test --maestro can show only:

Running replay suite...

while the simulator is actively executing the replay. This was reproduced on at least two iOS simulators, including an iPhone 17 Pro simulator. The command appears idle in the terminal even though device-side work continues.

Observed behavior

  • CLI prints Running replay suite... and then no per-file/per-step progress until final summary.
  • Simulator keeps executing the scenario, so the process is not truly idle.
  • After killing the CLI, simulator-side work can continue, which suggests daemon/runner work is not reliably cancelled with the client.
  • During one run, the parent CLI and dedicated daemon were alive with low CPU, while no useful maestro-test session/request logs were visible in the expected state dir.

Code context

The code already has progress plumbing:

  • src/daemon-client-progress.ts
  • src/cli-test-progress.ts
  • src/daemon/request-progress.ts
  • src/daemon/handlers/session-test.ts
  • src/daemon/handlers/session-test-attempt.ts

But formatReplayTestProgressEvent currently suppresses passing events, and there is no start/running event. That means long successful tests are silent by design. Failures only stream when retrying; final non-retry failures are also suppressed until the final response.

Desired behavior

  • Print bounded progress for replay suites, for example:
Running replay suite: 8 files
[1/8] START checkout-flow.ad
[1/8] PASS checkout-flow.ad (12.4s)
[2/8] START tabs-flow.ad
  • Include attempt/retry info when relevant.
  • Keep output compact and agent-friendly; no raw runner logs by default.
  • Print the suite artifact/session path early enough that a user can inspect logs while the run is active.
  • Ctrl-C / client disconnect should cancel active daemon replay/test work and stop runner-driven simulator activity.
  • Add tests covering start/pass progress events and cancellation/disconnect behavior where practical.

Validation expectation

Verify with a real iOS simulator using a non-trivial Maestro replay suite and show before/after terminal output. Also verify interrupting the CLI stops active simulator-driving work instead of leaving it running in the daemon/runner.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions