Skip to content

iOS: selector commands displace the app under test and return false negatives when no app session is tracked #1881

Description

@thymikee

Summary

On iOS, a selector command run against a session with no tracked appBundleId does not merely
fail to observe the screen — it displaces the app under test and then returns a confident
negative. find reports success: true, matches: [] for an element that is unambiguously on
screen, and the app that was in the foreground is gone.

This is a silent false negative that also corrupts device state, which makes it worse than a
failure: an agent gets a definitive-looking answer and a changed device.

Reproduction (iOS Simulator, iPhone 17, iOS 26.2)

Three A/B runs on the same simulator, seconds apart, against main:

Setup Command Result Device afterwards
Settings foregrounded sessionless find "General" list success: true, matches: [] Settings displaced by AgentDeviceRunner
Home screen, "Settings" icon visible find "Settings" list on a session with no app success: true, matches: [] home screen displaced
Settings foregrounded sessionless snapshot refuses in 2.4s, SESSION_NOT_FOUND Settings untouched

"General" and the Settings icon were unambiguously visible in the first two cases.

Mechanism

appBundleId is the XCUITest attach target.

  • With a target: the runner attaches to (and relaunches) that app. Verified by killing Settings with
    simctl terminate mid-session — snapshot still returned 18 real Settings nodes.
  • With no target: the runner has nothing to attach to, ensureRunnerHostAppActive brings
    AgentDeviceRunner itself to the foreground, and the capture observes its own ~3-node blank
    screen. The runner app has real UI, so the nodes.length === 0 tripwire never fires and the
    result looks plausible.

snapshot and diff are protected because they refuse this state up front (introduced in #681 to
stop exactly this class of wrong answer). find, get, is, and wait never ask, so they run the
capture and report on the runner.

Same state on Android is fine: snapshot returns the real launcher (16 nodes), find "Chrome" exists → true, wait text "Gmail" → success in 385 ms. The platform facts already encode the
asymmetry — platform-apple marks withoutActiveApp app-required for iOS,
platform-android allows it — so this is an iOS-only defect.

Related but separate: unhelpful refusal wording

Sessionless get and is refuse with a generic SESSION_NOT_FOUND / "No active session", rather
than the actionable iOS app-session message with the open hint that snapshot emits
(ios_app_session_required + buildIosOpenCommandHint). Unifying the wording across the selector
family is cheap and removes a second class of confusing output.

Fix direction

The Wave 4 selector migrations (#1739) close this as a side effect: wait (#1875), get (#1877),
and is route their captures through fact-derived admission, which refuses on iOS-with-no-app and
proceeds on Android automatically. find's own cutover is deferred behind Wave 5 focus/type
(#1876), so find keeps this defect the longest and is the reason this is filed separately
rather than folded into the wave.

Worth reconsidering while fixing

The rationale currently in the code — that an iOS capture without an app session "cannot succeed" —
is too strong. #1296 live-validated that capture works fine when SpringBoard is named as the session
app; the real constraint is that XCUITest needs some bundle id to attach to, not the user's app.
So refusing is correct only relative to today's implementation. A no-app iOS session could attach to
SpringBoard and return the real screen, which is what #1296 wants and what would remove the agent
friction #1662 measured (27 of 30 benchmark tasks hitting this refusal, costing 2-3 turns and ~20 s
each). That is a larger change than this bug fix and is noted here so the two are not solved twice.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions