feat(ios): split raw and regular snapshot projections behind one capture hint (#1797)

.github/workflows/ios.yml

@@ -151,12 +151,18 @@ jobs:
             -only-testing:AgentDeviceRunnerUITests/RunnerTests/testSnapshotPresentationPreservesCurrentWireShape \
             -only-testing:AgentDeviceRunnerUITests/RunnerTests/testSnapshotPresentationOwnsBackendNeutralEligibility \
             -only-testing:AgentDeviceRunnerUITests/RunnerTests/testSnapshotPresentationOwnsScopeAndRelativeDepth \
+            -only-testing:AgentDeviceRunnerUITests/RunnerTests/testPresentationRefusesAnAcquisitionCapturedForTheOtherProjection \
+            -only-testing:AgentDeviceRunnerUITests/RunnerTests/testCaptureHintIsTheOnlyAcquisitionViewOfARequest \
             -only-testing:AgentDeviceRunnerUITests/RunnerTests/testSnapshotScopePolicyMatchesGoldenParityTable \
             -only-testing:AgentDeviceRunnerUITests/RunnerTests/testFlatSnapshotProjectionMatchesElementReverseScrollCapture \
             -only-testing:AgentDeviceRunnerUITests/RunnerTests/testPrivateAXDepthLimitedRequiresEveryFrontierResolved \
             -only-testing:AgentDeviceRunnerUITests/RunnerTests/testDeepExtensionCountsMissedFrontiers \
             -only-testing:AgentDeviceRunnerUITests/RunnerTests/testPreferredPrivateAXBackendPlansAsPenalized \
+            -only-testing:AgentDeviceRunnerUITests/RunnerTests/testRawDiagnosticPlanCarriesOnlyBackendsThatCanServeRaw \
+            -only-testing:AgentDeviceRunnerUITests/RunnerTests/testProjectionMismatchFailureIsStructuredAndNotAnAxFailure \
             -only-testing:AgentDeviceRunnerUITests/RunnerTests/testPrivateAXRegularPresentationProjectsToViewportAndKeepsScrollHint \
+            -only-testing:AgentDeviceRunnerUITests/RunnerTests/testPrivateAXRawProjectionKeepsEveryAcquiredNode \
+            -only-testing:AgentDeviceRunnerUITests/RunnerTests/testPrivateAXRawProjectionAppliesRequestedTraversalDepth \
             -only-testing:AgentDeviceRunnerUITests/RunnerTests/testPrivateAXPresentationKeepsOffscreenSubtreeExcludedWhenChildFramesAreClamped \
             -only-testing:AgentDeviceRunnerUITests/RunnerTests/testPrivateAXGeometrylessSemanticsAreNeverActionableOrScrollContexts \
             -only-testing:AgentDeviceRunnerUITests/RunnerTests/testPrivateAXAcquisitionDoesNotInterpretScope \

CHANGELOG.md

@@ -7,6 +7,7 @@
 - `--udid` with `--platform android` (and `--serial` with an Apple platform) now fails as the flag mistake it is — `INVALID_ARGS` naming the right flag — instead of reaching device resolution and answering `No Apple device with UDID emulator-5580` for an explicitly Android request. `--udid` addresses Apple devices, `--serial` addresses Android and HarmonyOS; matching pairs and requests that name no platform are unchanged.
 - Breaking (`--session-lock strip`): a device selector that names a different device than the bound session is no longer silently discarded. `strip` exists to drop redundant platform/scope selectors; when it also dropped `--udid`/`--serial`/`--device`, the command kept running against the *bound* device instead of the one the caller named — a wrong-device action that looks like a success. Such a request now fails with `INVALID_ARGS` under both `reject` and `strip`, and the error carries the two identities structurally (`requestedDevice`, `boundDevice`) plus a hint offering the two real recoveries: close the bound session if the requested device is intended, or remove the selector if the bound device is. The hint no longer suggests `--session-lock strip` for an identity conflict, since following that advice is what produced the wrong-device run. Scope-only stripping (`--platform`, `--target`, `--ios-simulator-device-set`, `--android-device-allowlist`) is unchanged.
 - iOS regular snapshots now apply one backend-neutral eligibility rule after every capture backend: a node survives when its accessibility type is interactive or it carries a non-empty label, identifier, or value. This removes the tree backend's extra "hittable non-Other" membership path and drops unlabeled decorative nodes consistently; labeled images, identifier-only nodes, and value-only nodes still survive. Raw snapshot membership is unchanged.
+- iOS `snapshot --raw` is now the acquired accessibility tree on every backend that can serve it (#1797). A raw request that recovered onto the private-AX backend — the route an app whose XCTest tree capture fails takes — returned the *regular* projection's viewport-pruned nodes labeled raw: everything scrolled out of the viewport, and every sub-pixel decoration, was missing from the one view whose purpose is showing what the pruned view hid. Raw now keeps every node the backend serialized, at traversal depth, and `--depth` still narrows it (for raw, presented depth *is* traversal depth). Two structural rules replace the hand-synchronized ones: the raw capture plan is derived from each backend's declared ability to serve raw, so the interactive query sweep — which has no hierarchy to return — cannot be planned for a raw request; and presentation refuses an acquisition captured for the other projection instead of relabeling it, dropping that tier with a structured failure. Breaking in the same direction: `snapshot --raw -i` now returns the acquired tree instead of an interactive-filtered one — `-i` narrows the regular projection, and the pair used to produce a third membership rule that differed per backend. Regular and `-i` output is unchanged. Backends now read one derived capture hint rather than the request itself, so what a capture is allowed to skip is stated once, next to the proof that skipping it keeps the projection complete.
 - Android `snapshot --raw` is now the acquired accessibility tree (#1832 C3): the three regular-projection pruners — nodes Android marks invisible, stale application windows, and covered same-window surfaces — no longer run at parse time, so `--raw` keeps everything the helper serialized (normalization only) and pruned content is recoverable for diagnosis. Regular and `-i` output is unchanged (property-checked identical to the previous implementation over 12,000 random tree × projection pairs, hidden-content hints included). Also: Android blocking-dialog recovery now reads the same daemon presentation an agent's `snapshot` sees instead of a hand-rolled subset, and acts on its occlusion result — a stale "App isn't responding" surface left under the foreground one no longer triggers recovery, and a covered "Close app" is never tapped ahead of the visible one; the Android freshness route signature no longer keys on `role`/`selected`, fields the Android backend never carries; and the Android helper's declared fidelity residues (no `checked`/`checkable`/`selected`/`long-clickable`, 5000-node cap before scoping, API-level cache-reset divergence) are recorded in `CONTEXT.md`.
 - `agent-device mcp` now carries its own usage guidance, so MCP-only clients (Codex CLI, Cursor, custom agents) no longer depend on a separately installed skill (#1833). The handshake `instructions` — returned by both `server/discover` and, newly, the legacy `initialize` — is a compact (< 2 KB, the Claude Code truncation limit) workflow card: start with `open {app, foreground: true}` instead of probing, act with `settle: true` and continue from the diff, verify with `wait`/`is`/`get`/`find`, copy `@refs` byte-for-byte, recover from sparse/AX-unavailable, follow error hints, `close`. A new MCP-only `help` tool serves the full guides on demand: no `topic` returns the CLI's decision card; `topic` returns `agent-device help <topic|command>` verbatim (workflow, gestures, scripting, tv, macos, web, remote, debugging, …, or any tool name for its complete flag reference), prefixed with the one-line CLI→tool-property mapping. `help` is router-owned rather than a command descriptor, so it appears in `tools/list` only — not in the CLI, Node client, or `batch` — and its description tells the model it is not a startup step. Legacy `initialize` gains the optional `instructions` field; no other legacy field changes.
 - Android `snapshot --scope` (and every selector command's `--scope`, e.g. `press "Save" --scope Panel`) now resolves scope exactly once, inside the Android projection, under the shared scope specification: the scope root is the first node **in document order** whose label, value, or identifier contains the scope text (case-insensitive) **and whose subtree still has content in the projection you asked for**, the result is that subtree re-rooted at depth 0, and no match returns an empty snapshot (#1832). That second clause is what makes `snapshot -i --scope panel` return the button inside a structural container `-i` drops, and stops a decorative heading that happens to match from emptying the snapshot. Before, Android ran two passes with contradictory rules — a breadth-first platform match that fell back to the full tree on a miss, then the daemon's document-order pass — so a shallower later container could win over an earlier match, and an interaction capture whose scope reached only the daemon layer was silently unscoped. `--depth` under `--scope` counts from the scope root, filtering the depths the response prints (a node shown at depth 0 is never hidden by `--depth 0`), and ancestor context above the scope root (a clickable row, a list) still shapes `-i` membership inside it. The rule is pinned by `contracts/fixtures/snapshot-scope-policy.json`, the same golden table the iOS runner will consume (#1797). Also new: `androidSnapshot.occlusionScanUnavailable: true` discloses an API 23 capture, where the helper cannot report `drawing-order` and covered same-window surfaces are therefore not pruned.

CONTEXT.md

@@ -190,6 +190,11 @@ One backend attempt's raw accessibility nodes and attempt-level capture facts.
 **Presentation options**:
 The policy input controlling how one snapshot acquisition becomes a public projection.
 
+**Capture hint**:
+The acquisition-facing view of a snapshot request, derived once from presentation options. It names
+the projection a backend must serve and may narrow acquisition only where that backend can prove the
+narrowing complete.
+
 **Snapshot eligibility**:
 Membership in a presented snapshot projection, independent of whether a node is currently hittable.
 

apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+AXSnapshotFallback.swift

@@ -105,14 +105,14 @@ extension RunnerTests {
 
   func privateAXSnapshotAcquisition(
     app: XCUIApplication,
-    options: PresentationOptions,
+    hint: CaptureHint,
     deadline: Date = .distantFuture
   ) -> SnapshotAcquisition? {
     #if os(iOS) && targetEnvironment(simulator)
-      let requestedDepth = options.depth ?? 64
+      let requestedDepth = hint.depth ?? 64
       // An explicit --depth request is honored as asked: no accepted-depth
       // memory, no frontier extension past it.
-      let exactDepthRequested = options.depth != nil
+      let exactDepthRequested = hint.depth != nil
       let rememberedDepth =
         exactDepthRequested
         ? nil
@@ -135,12 +135,16 @@ extension RunnerTests {
           NSLog("AGENT_DEVICE_RUNNER_PRIVATE_AX_SNAPSHOT_BUDGET_EXHAUSTED depth=%ld", depth)
           break
         }
+        // Declared residue (#1797): the bridge caps the tree at 5000 nodes while serializing,
+        // BEFORE either projection exists, so a raw capture of a huge screen is bounded rather
+        // than failing the way the tree backend's own raw cap does. The cap is disclosed as
+        // `truncated`, and it applied to the acquired tree before this projection split too.
         response = RunnerAXSnapshotBridge.snapshotTree(
           for: app,
           maxDepth: depth,
           maxNodes: Self.privateAXSnapshotMaxNodes,
           deepExtensionCallLimit: exactDepthRequested ? 0 : Self.privateAXDeepExtensionCallLimit,
-          customActionLimit: options.customActions ? Self.privateAXCustomActionLimit : 0,
+          customActionLimit: hint.customActions ? Self.privateAXCustomActionLimit : 0,
           deadline: deadline
         )
         if response["ok"] as? Bool == true {
@@ -175,9 +179,9 @@ extension RunnerTests {
 
       let rootFrame = privateAXRect(root["frame"])
       let viewport = privateAXSnapshotViewport(app: app, rootFrame: rootFrame)
-      let nodes = privateAXPresentation(
+      let nodes = privateAXAcquisition(
         rawRoot: root,
-        options: options,
+        hint: hint,
         viewport: viewport
       )
       if nodes.count <= 1 {
@@ -203,6 +207,7 @@ extension RunnerTests {
         deepExtension?[RunnerAXSnapshotDeepExtensionNodesAddedKey] as? Int ?? 0
       )
       return SnapshotAcquisition(
+        hint: hint,
         nodes: nodes,
         truncated: (response["truncated"] as? Bool) == true,
         effectiveDepth: depthLimited ? effectiveDepth : nil,
@@ -635,9 +640,10 @@ extension RunnerTests {
         ],
       ],
     ]
-    let nodes = privateAXPresentation(
+    let nodes = privateAXAcquisition(
       rawRoot: tree,
-      options: PresentationOptions(interactiveOnly: false, depth: nil, scope: nil, raw: false),
+      hint: CaptureHint(
+        projection: .regular, depth: nil, interactiveOnly: false, customActions: false),
       viewport: CGRect(x: 0, y: 0, width: 390, height: 844)
     )
 
@@ -660,13 +666,17 @@ extension RunnerTests {
         ["type": 9, "label": "unrelated sibling", "children": []],
       ],
     ]
-    let nodes = privateAXPresentation(
+    // Scope never reaches acquisition: the hint derived for a scoped request carries no scope,
+    // and the backend has no way to interpret one.
+    let nodes = privateAXAcquisition(
       rawRoot: tree,
-      options: PresentationOptions(
-        interactiveOnly: false,
-        depth: nil,
-        scope: "homeScreen",
-        raw: false
+      hint: SnapshotPresentation.captureHint(
+        for: PresentationOptions(
+          interactiveOnly: false,
+          depth: nil,
+          scope: "homeScreen",
+          raw: false
+        )
       ),
       viewport: .infinite
     )
@@ -742,9 +752,10 @@ extension RunnerTests {
         ]
       ],
     ]
-    let nodes = privateAXPresentation(
+    let nodes = privateAXAcquisition(
       rawRoot: tree,
-      options: PresentationOptions(interactiveOnly: true, depth: nil, scope: nil, raw: false),
+      hint: CaptureHint(
+        projection: .regular, depth: nil, interactiveOnly: true, customActions: false),
       viewport: CGRect(x: 0, y: 0, width: 390, height: 844)
     )
 

apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+FlatSnapshotFiltering.swift

@@ -108,16 +108,18 @@ extension RunnerTests {
     )
   }
 
+  /// Regular-projection acquisition gate. The raw projection never reaches it: raw acquisition is
+  /// the acquired tree, so it has no visibility filter to consult (#1797 D4).
   func flatSnapshotFilterDecision(
     _ node: FlatSnapshotFilterNode,
-    options: PresentationOptions,
+    hint: CaptureHint,
     visibilityPolicy: FlatSnapshotVisibilityPolicy
   ) -> FlatSnapshotFilterDecision {
     let include: Bool
     if node.isRoot {
       include = true
     } else if !node.visible
-      && (options.interactiveOnly || visibilityPolicy == .viewportProjected)
+      && (hint.interactiveOnly || visibilityPolicy == .viewportProjected)
     {
       include = false
     } else {
@@ -285,6 +287,10 @@ extension RunnerTests {
   }
 
   func testFlatSnapshotFilterDecisionMatrixCoversOptions() {
+    func hint(interactiveOnly: Bool) -> CaptureHint {
+      CaptureHint(
+        projection: .regular, depth: nil, interactiveOnly: interactiveOnly, customActions: false)
+    }
     let visibleContent = FlatSnapshotFilterNode(
       isRoot: false,
       visible: true
@@ -305,42 +311,42 @@ extension RunnerTests {
     XCTAssertTrue(
       flatSnapshotFilterDecision(
         visibleContent,
-        options: PresentationOptions(interactiveOnly: false, depth: nil, scope: nil, raw: false),
+        hint: hint(interactiveOnly: false),
         visibilityPolicy: .interactiveOnly
       ).include
     )
     XCTAssertFalse(
       flatSnapshotFilterDecision(
         hiddenInteractive,
-        options: PresentationOptions(interactiveOnly: true, depth: nil, scope: nil, raw: false),
+        hint: hint(interactiveOnly: true),
         visibilityPolicy: .interactiveOnly
       ).include
     )
     XCTAssertFalse(
       flatSnapshotFilterDecision(
         hiddenInteractive,
-        options: PresentationOptions(interactiveOnly: false, depth: nil, scope: nil, raw: false),
+        hint: hint(interactiveOnly: false),
         visibilityPolicy: .viewportProjected
       ).include
     )
     XCTAssertTrue(
       flatSnapshotFilterDecision(
         hiddenInteractive,
-        options: PresentationOptions(interactiveOnly: false, depth: nil, scope: nil, raw: false),
+        hint: hint(interactiveOnly: false),
         visibilityPolicy: .interactiveOnly
       ).include
     )
     XCTAssertTrue(
       flatSnapshotFilterDecision(
         hiddenRoot,
-        options: PresentationOptions(interactiveOnly: false, depth: nil, scope: nil, raw: false),
+        hint: hint(interactiveOnly: false),
         visibilityPolicy: .viewportProjected
       ).include
     )
     XCTAssertTrue(
       flatSnapshotFilterDecision(
         decorative,
-        options: PresentationOptions(interactiveOnly: false, depth: nil, scope: nil, raw: false),
+        hint: hint(interactiveOnly: false),
         visibilityPolicy: .interactiveOnly
       ).include
     )