diff --git a/docs/adr/0019-request-bound-platform-runtime.md b/docs/adr/0019-request-bound-platform-runtime.md index f95f343cc..5baab7a21 100644 --- a/docs/adr/0019-request-bound-platform-runtime.md +++ b/docs/adr/0019-request-bound-platform-runtime.md @@ -38,7 +38,8 @@ those descriptors or authorize a second local/provider chooser. tested for the exact platform leaf, device kind/backend, and provider mode; family ownership never implies uniform leaf support. - Command descriptors declare one typed execution shape: inventory use, or platform-neutral required - device operations with separately declared preferred fast paths. Runtime owners report + device operations with separately declared preferred optimizations and fact-conditional semantic + operations. Runtime owners report device-specific facts and expose behavior-bearing facets; platform and provider implementations never name commands. - `RequestExecutionScope.bindDevice(device, use)` resolves provider ownership, validates the facts @@ -67,8 +68,9 @@ those descriptors or authorize a second local/provider chooser. - Evidence is tiered by what a unit imports: request-scoped device units prove facts, operations, and parity cells; only durable-resource units carry the section 4–5 lifecycle evidence. - A handler binds once with its execution use. Admission, `capabilities`, and doctor questions use - side-effect-free facts inspection; required-only declarations are the default and a preferred - operation requires a recorded measurement. + side-effect-free facts inspection; required-only declarations are the default, a preferred + operation requires a recorded measurement, and a conditional operation requires parity evidence + explaining why correctness depends on the owner fact. - Cross-cutting facets land with their first consuming command unit. Daemon startup recovery is evidence-gated, daemon shutdown is two-phase (detach, then stop), and session-teardown steps belong to their owning domains — there is no generic lifecycle-hook API. @@ -156,11 +158,15 @@ procedure. ### 2. Runtime use joins facts and narrows the bound runtime `CommandDescriptor` remains the command declaration root. Its runtime-use declaration has a typed set -of required platform-neutral operations and may separately name preferred optimizations. Commands -whose use depends on normalized input first produce a discriminated execution plan that retains -literal required/preferred types. Required and preferred operation keys are disjoint, and the -required-only path is semantically complete; preferred operations may improve execution but are -never necessary for command correctness. +of required platform-neutral operations and may separately name preferred optimizations or +fact-conditional semantic operations. Commands whose use depends on normalized input first produce a +discriminated execution plan that retains literal required/preferred/conditional types. The three +operation sets are pairwise disjoint. Preferred operations may improve execution but are never +necessary for command correctness. A conditional operation is different: an owner that advertises +it must provide it because that owner's semantically complete path depends on the observation; an +owner that reports it unavailable must have a parity-proven complete path through the required +operations. This prevents a correctness-bearing owner variation from being mislabeled as an +optimization or made unconditionally required across owners without that semantic source. Inventory commands have a separate `inventoryUse` declaration. `devices` calls the composed `DeviceInventoryGateway`, which selects canonical family sources and provider-owned inventory sources @@ -180,14 +186,15 @@ operations from the same facet. `RequestExecutionScope.bindDevice(device, use)` is the trust choke point. It: 1. resolves the exact local or provider runtime owner; -2. checks every required operation and classifies each preferred operation against facts for the - platform leaf, device kind/backend, and provider mode; +2. checks every required operation and classifies each preferred and conditional operation against + facts for the platform leaf, device kind/backend, and provider mode; 3. creates or reuses one request binding for that ownership-qualified device; -4. verifies that every required operation and every preferred operation advertised as available has - a concrete facet implementation; an advertised operation with no implementation is a - runtime-contract error; and +4. verifies that every required operation and every preferred or conditional operation advertised as + available has a concrete facet implementation; an advertised operation with no implementation is + a runtime-contract error; and 5. returns a selected operation projection: required operations are non-optional, declared preferred - operations are optional and present only when available, and undeclared operations are inaccessible. + and conditional operations are optional and present only when available, and undeclared operations + are inaccessible. The cached broad runtime remains private to `RequestExecutionScope`; narrowing does not intersect a wide optional aggregate that would still expose undeclared facets. The descriptor and its specialized @@ -195,7 +202,7 @@ handler share one non-widened declaration, and a widened generic descriptor carr A compile-time contract test proves the selected projection. A structural **runtime-facet-narrowing gate** covers every runtime-migrated handler owner and rejects attempts to manufacture required-operation proof with assertions or optional admission. Optional access is -permitted only for descriptor-declared preferred operations. The tracking issue owns the gate +permitted only for descriptor-declared preferred or conditional operations. The tracking issue owns the gate implementation and its required planted violation. Absence or failure of a preferred path may change optimization/path disclosure, not whether the @@ -204,6 +211,11 @@ reason and an explicit descriptor/ADR 0011 path classification; it is never a ge fallback. Helper/session reuse hidden inside one required operation remains that facet's implementation detail and follows ADR 0002 rather than becoming a daemon-visible preferred operation. +Conditional operations do not inherit that fallback rule. Their absence is valid only when the owner +fact reports them unavailable and parity evidence establishes the required path for that owner cell. +When the fact reports one available, omission of its implementation is a runtime-contract failure; +the handler must not silently continue through a path known to lose the observation. + Family registration and support coverage are separate gates. The immutable registry owns each of the six canonical families exactly once. Before a command cuts over, an independent parity artifact freezes its legacy supported/unsupported cells and hints. Runtime-fact scenarios expand canonical @@ -677,6 +689,11 @@ a preferred operation requires a recorded measurement of the fast path's benefit review; the direct-selector fast path is the model. A preferred operation declared without a measurement is speculative surface and is rejected in review. +Declaring a conditional operation instead requires a parity artifact showing both sides: the owner +cells whose semantic path depends on the operation and the owner cells whose required path remains +complete without it. A conditional declaration justified only by speed belongs under `preferred`; +one justified only by universal correctness belongs under `required`. + ### 10. Process-lifetime and cross-cutting surfaces A cross-cutting facet — one consumed from more than one command's execution path, such as snapshot @@ -727,8 +744,8 @@ platform-freedom is structurally enforced rather than measured. [ADR 0007](0007-remote-device-leases.md): daemon request-policy traits, lease admission, and lock ordering remain daemon-owned. Binding happens only after their admission requirements are met. - [ADR 0008](0008-command-descriptor-registry.md): the descriptor registry remains the command root. - Device-command capability buckets evolve into typed required/preferred runtime use joined with - exact runtime facts; inventory commands declare inventory use. + Device-command capability buckets evolve into typed required/preferred/conditional runtime use + joined with exact runtime facts; inventory commands declare inventory use. - [ADR 0009](0009-apple-platform-consolidation.md): the Apple family and `AppleOS` leaf axis remain. The shallow `PlatformPlugin` shape is superseded as command units migrate; physical shared mechanics move only through the legal injected substrate transition or after their last legacy diff --git a/packages/contracts/src/command-platform-execution.test.ts b/packages/contracts/src/command-platform-execution.test.ts index 316f99bec..3ee7b08eb 100644 --- a/packages/contracts/src/command-platform-execution.test.ts +++ b/packages/contracts/src/command-platform-execution.test.ts @@ -8,7 +8,11 @@ describe('command platform execution declaration', () => { { kind: 'none' }, { kind: 'legacy' }, { kind: 'inventory', use: inventoryUse }, - { kind: 'device-runtime', use: { required: ['capture'], preferred: ['inspect'] } }, + { kind: 'device-runtime', use: { required: ['capture'], preferred: [] } }, + { + kind: 'device-runtime', + use: { required: ['capture'], preferred: ['inspect'], conditional: ['observe'] }, + }, ])('accepts one closed execution shape: %j', (value) => { expect(() => assertCommandPlatformExecution(value)).not.toThrow(); }); @@ -19,9 +23,27 @@ describe('command platform execution declaration', () => { { kind: 'legacy', use: inventoryUse }, { kind: 'inventory' }, { kind: 'inventory', use: inventoryUse, legacy: true }, - { kind: 'device-runtime', use: { required: [], preferred: [] }, inventory: true }, - { kind: 'device-runtime', use: { required: ['capture', 'capture'], preferred: [] } }, - { kind: 'device-runtime', use: { required: ['capture'], preferred: ['capture'] } }, + { + kind: 'device-runtime', + use: { required: [], preferred: [] }, + inventory: true, + }, + { + kind: 'device-runtime', + use: { required: ['capture', 'capture'], preferred: [] }, + }, + { + kind: 'device-runtime', + use: { required: ['capture'], preferred: ['capture'] }, + }, + { + kind: 'device-runtime', + use: { required: ['capture'], preferred: [], conditional: ['capture'] }, + }, + { + kind: 'device-runtime', + use: { required: [], preferred: ['capture'], conditional: ['capture'] }, + }, ])('rejects neither, mixed, widened, duplicate, or overlapping declarations: %j', (value) => { expect(() => assertCommandPlatformExecution(value)).toThrow(/exactly one/); }); @@ -36,7 +58,10 @@ describe('command platform execution declaration', () => { { kind: 'device-runtime', uses: [] }, { kind: 'device-runtime', use: appLogRuntimePlanUses[0], uses: appLogRuntimePlanUses }, { kind: 'device-runtime', uses: [appLogRuntimePlanUses[0], appLogRuntimePlanUses[0]] }, - { kind: 'device-runtime', uses: [{ required: ['appLogStart'], preferred: ['appLogStart'] }] }, + { + kind: 'device-runtime', + uses: [{ required: ['appLogStart'], preferred: ['appLogStart'] }], + }, ])('rejects empty, duplicate, overlapping, or both-form runtime uses: %j', (value) => { expect(() => assertCommandPlatformExecution(value)).toThrow(/exactly one/); }); diff --git a/packages/contracts/src/command-platform-execution.ts b/packages/contracts/src/command-platform-execution.ts index 27c7ab07d..c04e791f3 100644 --- a/packages/contracts/src/command-platform-execution.ts +++ b/packages/contracts/src/command-platform-execution.ts @@ -1,5 +1,6 @@ import type { InventoryUse } from './platform-module.ts'; import type { RuntimeUseDeclaration } from './platform-runtime.ts'; +import { runtimeUseIdentity } from './platform-runtime-use.ts'; export type CommandPlatformExecution = | Readonly<{ kind: 'none' }> @@ -11,7 +12,7 @@ export type CommandPlatformExecution = uses: readonly [RuntimeUseDeclaration, ...RuntimeUseDeclaration[]]; }>; -// The discriminated union cannot prove uniqueness or required/preferred disjointness inside +// The discriminated union cannot prove uniqueness or operation-category disjointness inside // readonly arrays. Validate those declaration invariants where descriptors enter the registry. export function assertCommandPlatformExecution( value: unknown, @@ -54,13 +55,6 @@ function hasRuntimeUseDeclarations( return new Set(identities).size === identities.length; } -function runtimeUseIdentity(use: RuntimeUseDeclaration): string { - return JSON.stringify({ - required: [...use.required].sort(), - preferred: [...use.preferred].sort(), - }); -} - function hasExactInventoryUse(value: unknown): boolean { if (value === null || typeof value !== 'object') return false; const use = value as Record; @@ -70,12 +64,28 @@ function hasExactInventoryUse(value: unknown): boolean { function hasRuntimeUseDeclaration(value: unknown): boolean { if (value === null || typeof value !== 'object') return false; const use = value as Record; - const required = stringArray(use['required']); - const preferred = stringArray(use['preferred']); - if (!required || !preferred) return false; - if (!hasUniqueValues(required) || !hasUniqueValues(preferred)) return false; - if (!areDisjoint(required, preferred)) return false; - return sameKeys(Object.keys(use).sort(), ['preferred', 'required']); + const categories = [ + stringArray(use['required']), + stringArray(use['preferred']), + stringArray(use['conditional'] ?? []), + ]; + if (!hasValidRuntimeUseCategories(categories)) return false; + const operations = categories.flat(); + if (new Set(operations).size !== operations.length) return false; + return sameKeys( + Object.keys(use).sort(), + use['conditional'] === undefined + ? ['preferred', 'required'] + : ['conditional', 'preferred', 'required'], + ); +} + +function hasValidRuntimeUseCategories( + categories: readonly (string[] | null)[], +): categories is readonly string[][] { + return categories.every( + (category): category is string[] => category !== null && hasUniqueValues(category), + ); } function stringArray(value: unknown): string[] | null { @@ -87,11 +97,6 @@ function hasUniqueValues(values: readonly string[]): boolean { return new Set(values).size === values.length; } -function areDisjoint(left: readonly string[], right: readonly string[]): boolean { - const leftValues = new Set(left); - return right.every((value) => !leftValues.has(value)); -} - function sameKeys(actual: readonly string[], expected: readonly string[]): boolean { return actual.length === expected.length && actual.every((key, index) => key === expected[index]); } diff --git a/packages/contracts/src/facades/platform.ts b/packages/contracts/src/facades/platform.ts index 45c1d8d50..44c6f5529 100644 --- a/packages/contracts/src/facades/platform.ts +++ b/packages/contracts/src/facades/platform.ts @@ -218,14 +218,19 @@ export { resolveSnapshotRuntimePlan, screenshotRuntimePlanUses, selectorCaptureRuntimePlanUses, + selectorTextCaptureRuntimePlanUses, snapshotRuntimePlanUses, + waitSelectorCaptureRuntimePlanUses, viewportRuntimeUse, } from '../platform-runtime-operations.ts'; export type { ScreenshotRuntimePlan, + SelectorCaptureRuntimeIntent, SelectorCaptureRuntimePlan, SnapshotRuntimePlan, } from '../platform-runtime-operations.ts'; +export { waitObservesDevice } from '../wait-runtime-plan.ts'; +export type { WaitRuntimeTarget } from '../wait-runtime-plan.ts'; export type { PlatformRuntimeHost, PlatformRuntimeModule, @@ -258,8 +263,21 @@ export type { export { bindLocalSnapshotInteractor, bindProviderSnapshotInteractor, + captureSnapshotSignal, snapshotRuntimeOperationFacts, } from '../snapshot-runtime.ts'; +export { selectorObservationRuntimeOperationFacts } from '../selector-observation-runtime.ts'; +export type { + FindSelectorInput, + FindSelectorResult, + FindSelectorRuntimeOperations, + FindTextInput, + FindTextResult, + FindTextRuntimeOperations, + SelectorObservationRuntimeOperationFacts, + SelectorObservationRuntimeOperations, + SelectorObservationResult, +} from '../selector-observation-runtime.ts'; export type { CaptureSnapshotInput, LocalSnapshotInteractorResolver, diff --git a/packages/contracts/src/interactor-types.ts b/packages/contracts/src/interactor-types.ts index 48041f3d4..c2941b9e4 100644 --- a/packages/contracts/src/interactor-types.ts +++ b/packages/contracts/src/interactor-types.ts @@ -232,6 +232,22 @@ export type Interactor = { point: Point, options?: { appBundleId?: string; surface?: SessionSurface; signal?: AbortSignal }, ): Promise; + /** + * Native text-presence reading, when the backend has one that does not require a tree capture. + * A `true` answer is authoritative; anything else means "not proven here" and the caller + * consults the canonical tree (see `FindTextResult`). + */ + findText?( + text: string, + options?: { appBundleId?: string; signal?: AbortSignal }, + ): Promise<{ + found: boolean; + }>; + /** Native one-sided observation for a simple selector; false defers to canonical capture. */ + findSelector?( + selector: Readonly<{ key: ElementSelectorKey; value: string }>, + options?: { appBundleId?: string; signal?: AbortSignal }, + ): Promise<{ found: boolean }>; gestureViewport?(): Promise; back(mode?: BackMode): Promise; home(): Promise; diff --git a/packages/contracts/src/logs-runtime-cutover.ts b/packages/contracts/src/logs-runtime-cutover.ts index 7977619c6..6bc3eca67 100644 --- a/packages/contracts/src/logs-runtime-cutover.ts +++ b/packages/contracts/src/logs-runtime-cutover.ts @@ -3,7 +3,7 @@ import { type CommandPlatformExecution, } from './command-platform-execution.ts'; import { appLogRuntimePlanUses } from './logs-runtime-plan.ts'; -import type { RuntimeUseDeclaration } from './platform-runtime.ts'; +import { runtimeUseIdentity } from './platform-runtime-use.ts'; /** Joins the input-dependent logs plans to one exhaustive descriptor declaration. */ export function assertLogsRuntimeExecution( @@ -18,13 +18,6 @@ export function assertLogsRuntimeExecution( } } -function runtimeUseIdentity(use: RuntimeUseDeclaration): string { - return JSON.stringify({ - required: [...use.required].sort(), - preferred: [...use.preferred].sort(), - }); -} - function invalidLogsExecution(): TypeError { return new TypeError( 'Logs runtime execution must declare exactly the distinct uses selected by its seven plans', diff --git a/packages/contracts/src/network-runtime-plan.test.ts b/packages/contracts/src/network-runtime-plan.test.ts index 7d89be9c0..1b4125412 100644 --- a/packages/contracts/src/network-runtime-plan.test.ts +++ b/packages/contracts/src/network-runtime-plan.test.ts @@ -40,8 +40,14 @@ test('normalizes defaults and retains the requested alias and projection', () => include: 'headers', use: networkDumpUse, }); - assert.deepEqual(networkDumpUse, { required: ['networkDump'], preferred: [] }); - assert.deepEqual(networkAdmissionUse, { required: [], preferred: ['networkDump'] }); + assert.deepEqual(networkDumpUse, { + required: ['networkDump'], + preferred: [], + }); + assert.deepEqual(networkAdmissionUse, { + required: [], + preferred: ['networkDump'], + }); }); test('rejects actions and projections outside the frozen cells', () => { diff --git a/packages/contracts/src/platform-runtime-operations.test.ts b/packages/contracts/src/platform-runtime-operations.test.ts index 7a3a8710f..fd9f3fb9a 100644 --- a/packages/contracts/src/platform-runtime-operations.test.ts +++ b/packages/contracts/src/platform-runtime-operations.test.ts @@ -60,14 +60,20 @@ test.each([ true, 'custom-actions-active-app', 'captureSnapshotWithCustomActions', - { required: ['captureSnapshot', 'captureSnapshotWithCustomActions'], preferred: [] }, + { + required: ['captureSnapshot', 'captureSnapshotWithCustomActions'], + preferred: [], + }, ], [ false, false, 'without-active-app', 'captureSnapshotWithoutActiveApp', - { required: ['captureSnapshot', 'captureSnapshotWithoutActiveApp'], preferred: [] }, + { + required: ['captureSnapshot', 'captureSnapshotWithoutActiveApp'], + preferred: [], + }, ], [ true, diff --git a/packages/contracts/src/platform-runtime-operations.ts b/packages/contracts/src/platform-runtime-operations.ts index cb9234f4d..f606d34f9 100644 --- a/packages/contracts/src/platform-runtime-operations.ts +++ b/packages/contracts/src/platform-runtime-operations.ts @@ -14,6 +14,7 @@ import type { ScreenRecordingRuntimeHost } from './screen-recording-runtime-host import type { ScreenRecordingRuntimeOperations } from './screen-recording-runtime.ts'; import type { ScreenshotRuntimeOperations } from './screenshot-runtime.ts'; import type { SnapshotRuntimeHost, SnapshotRuntimeOperations } from './snapshot-runtime.ts'; +import type { SelectorObservationRuntimeOperations } from './selector-observation-runtime.ts'; import type { ViewportRuntimeOperations } from './viewport-runtime.ts'; import type { ElementTextRuntimeOperations } from './element-text-runtime.ts'; import type { @@ -47,6 +48,7 @@ export type PlatformRuntimeOperations = AppLogRuntimeOperations & ScreenRecordingRuntimeOperations & ScreenshotRuntimeOperations & SnapshotRuntimeOperations & + SelectorObservationRuntimeOperations & ViewportRuntimeOperations & ElementTextRuntimeOperations & DeviceReadinessRuntimeOperations & @@ -82,22 +84,37 @@ const captureSnapshotWithCustomActionsWithoutActiveAppUse = defineUse({ ], }); -/** - * The selector family's capture uses. Declared ALONGSIDE the snapshot uses above, never in place - * of them: `snapshot`/`diff` keep binding exactly what they bind today. The only difference is the - * PREFERRED element read — every selector read's required path answers from the captured tree, so - * an owner without the read still executes the command completely (ADR 0019 §2), but an owner that - * has one lets `get text` return the live value a truncated snapshot node cannot. - */ const selectorCaptureUse = defineUse({ required: ['captureSnapshot'], - preferred: ['readTextAtPoint'], }); const selectorCaptureWithoutActiveAppUse = defineUse({ required: ['captureSnapshot', 'captureSnapshotWithoutActiveApp'], +}); + +/** `get` and read-only `find` may improve a captured result with a live element read. */ +const selectorTextCaptureUse = defineUse({ + required: ['captureSnapshot'], + preferred: ['readTextAtPoint'], +}); +const selectorTextCaptureWithoutActiveAppUse = defineUse({ + required: ['captureSnapshot', 'captureSnapshotWithoutActiveApp'], preferred: ['readTextAtPoint'], }); +/** + * Native wait observations are correctness-bearing only for owners that advertise them. They stay + * out of capture-only and element-text uses so an unrelated command cannot be rejected for a wait + * operation it never executes (ADR 0019 §2). + */ +const waitSelectorCaptureUse = defineUse({ + required: ['captureSnapshot'], + conditional: ['findText', 'findSelector'], +}); +const waitSelectorCaptureWithoutActiveAppUse = defineUse({ + required: ['captureSnapshot', 'captureSnapshotWithoutActiveApp'], + conditional: ['findText', 'findSelector'], +}); + /** * The selector family (`find`, `get`, `is`, `wait`) resolves targets from the plain accessibility * capture: it exposes no `--actions` surface, so only the active-app split applies. @@ -107,6 +124,16 @@ export const selectorCaptureRuntimePlanUses = Object.freeze([ selectorCaptureWithoutActiveAppUse, ] as const); +export const selectorTextCaptureRuntimePlanUses = Object.freeze([ + selectorTextCaptureUse, + selectorTextCaptureWithoutActiveAppUse, +] as const); + +export const waitSelectorCaptureRuntimePlanUses = Object.freeze([ + waitSelectorCaptureUse, + waitSelectorCaptureWithoutActiveAppUse, +] as const); + export const snapshotRuntimePlanUses = Object.freeze([ captureSnapshotUse, captureSnapshotWithCustomActionsUse, @@ -136,40 +163,85 @@ export type SnapshotRuntimePlan = use: typeof captureSnapshotWithoutActiveAppUse; }>; +const selectorUsesByIntent = Object.freeze({ + 'capture-only': selectorCaptureRuntimePlanUses, + 'element-text': selectorTextCaptureRuntimePlanUses, + 'wait-observation': waitSelectorCaptureRuntimePlanUses, +} as const); + +export type SelectorCaptureRuntimeIntent = keyof typeof selectorUsesByIntent; + /** - * Same two `kind`s the snapshot plan uses for this split — deliberately, so the shared - * admit-then-bind path keeps ONE set of arms rather than growing a parallel dispatch — but - * carrying the selector uses, which add the preferred element read. + * Same two `kind`s the snapshot plan uses for this split — deliberately, so capture-only, + * element-text, and wait-observation callers share one admit-then-bind path. */ -export type SelectorCaptureRuntimePlan = +type SelectorCapturePlanFor = | Readonly<{ kind: 'selector-active-app'; + intent: Intent; operation: 'captureSnapshot'; - use: typeof selectorCaptureUse; + use: (typeof selectorUsesByIntent)[Intent][0]; }> | Readonly<{ kind: 'selector-without-active-app'; + intent: Intent; operation: 'captureSnapshotWithoutActiveApp'; - use: typeof selectorCaptureWithoutActiveAppUse; + use: (typeof selectorUsesByIntent)[Intent][1]; }>; +export type SelectorCaptureRuntimePlan = { + [Intent in SelectorCaptureRuntimeIntent]: SelectorCapturePlanFor; +}[SelectorCaptureRuntimeIntent]; + /** * The active-app split every selector capture selects from. The selector family exposes no * `--actions` surface, so custom actions are outside its declaration. */ export function resolveSelectorCaptureRuntimePlan( - input: Readonly<{ hasActiveApp: boolean }>, + input: Readonly<{ + hasActiveApp: boolean; + intent: SelectorCaptureRuntimeIntent; + }>, ): SelectorCaptureRuntimePlan { - return input.hasActiveApp + switch (input.intent) { + case 'capture-only': + return selectorCapturePlan( + input.hasActiveApp, + input.intent, + selectorUsesByIntent[input.intent], + ); + case 'element-text': + return selectorCapturePlan( + input.hasActiveApp, + input.intent, + selectorUsesByIntent[input.intent], + ); + case 'wait-observation': + return selectorCapturePlan( + input.hasActiveApp, + input.intent, + selectorUsesByIntent[input.intent], + ); + } +} + +function selectorCapturePlan( + hasActiveApp: boolean, + intent: Intent, + uses: (typeof selectorUsesByIntent)[Intent], +): SelectorCapturePlanFor { + return hasActiveApp ? Object.freeze({ kind: 'selector-active-app', + intent, operation: 'captureSnapshot', - use: selectorCaptureUse, + use: uses[0], }) : Object.freeze({ kind: 'selector-without-active-app', + intent, operation: 'captureSnapshotWithoutActiveApp', - use: selectorCaptureWithoutActiveAppUse, + use: uses[1], }); } diff --git a/packages/contracts/src/platform-runtime-unavailable.ts b/packages/contracts/src/platform-runtime-unavailable.ts index 7a995a6a8..806bf4891 100644 --- a/packages/contracts/src/platform-runtime-unavailable.ts +++ b/packages/contracts/src/platform-runtime-unavailable.ts @@ -12,6 +12,7 @@ import type { } from './platform-runtime.ts'; import { screenshotRuntimeOperationFacts } from './screenshot-runtime.ts'; import { snapshotRuntimeOperationFacts } from './snapshot-runtime.ts'; +import { selectorObservationRuntimeOperationFacts } from './selector-observation-runtime.ts'; import { viewportRuntimeOperationFacts } from './viewport-runtime.ts'; import { elementTextRuntimeOperationFacts } from './element-text-runtime.ts'; @@ -105,6 +106,13 @@ export function createUnavailablePlatformRuntimeFacts( customActions: snapshot, withoutActiveApp: snapshot, }), + // The preferred text reading starts unavailable for every family, on the same sentinel as + // capture: an owner that has a native reading declares it explicitly, and one that does not + // sends every text wait to the canonical tree. + ...selectorObservationRuntimeOperationFacts({ + findText: snapshot, + findSelector: snapshot, + }), ...viewportRuntimeOperationFacts({ setViewport: viewport }), ...elementTextRuntimeOperationFacts({ readTextAtPoint: elementText }), ensureReady: readiness, diff --git a/packages/contracts/src/platform-runtime-use.ts b/packages/contracts/src/platform-runtime-use.ts index 088bd41c0..9c2e38784 100644 --- a/packages/contracts/src/platform-runtime-use.ts +++ b/packages/contracts/src/platform-runtime-use.ts @@ -1,16 +1,29 @@ -import type { RuntimeOperationKey, RuntimeUse } from './platform-runtime.ts'; +import type { RuntimeOperationKey, RuntimeUse, RuntimeUseDeclaration } from './platform-runtime.ts'; + +export function runtimeUseIdentity(use: RuntimeUseDeclaration): string { + return JSON.stringify({ + required: [...use.required].sort(), + preferred: [...use.preferred].sort(), + conditional: [...(use.conditional ?? [])].sort(), + }); +} type RuntimeUseInput< Operations extends object, Required extends readonly RuntimeOperationKey[], Preferred extends readonly Exclude, Required[number]>[], + Conditional extends readonly Exclude< + RuntimeOperationKey, + Required[number] | Preferred[number] + >[], > = Readonly<{ required: Required; preferred?: Preferred; + conditional?: Conditional; }>; /** - * Define one descriptor's non-widened required/preferred operation declaration. + * Define one descriptor's non-widened required/preferred/conditional operation declaration. * Runtime validation keeps declarations built from dynamic data fail-closed too. */ export function runtimeUse() { @@ -18,19 +31,42 @@ export function runtimeUse() { const Required extends readonly RuntimeOperationKey[], const Preferred extends readonly Exclude, Required[number]>[] = readonly [], + const Conditional extends readonly Exclude< + RuntimeOperationKey, + Required[number] | Preferred[number] + >[] = readonly [], >( - input: RuntimeUseInput, - ): RuntimeUse => { + input: RuntimeUseInput, + ): RuntimeUse => { const required = freezeUniqueKeys(input.required, 'required'); const preferred = freezeUniqueKeys( input.preferred ?? ([] as unknown as Preferred), 'preferred', ); - const overlap = preferred.find((key) => required.includes(key)); - if (overlap !== undefined) { - throw new TypeError(`Runtime operation cannot be both required and preferred: ${overlap}`); + const conditional = freezeUniqueKeys( + input.conditional ?? ([] as unknown as Conditional), + 'conditional', + ); + const categories: ReadonlyArray = [ + ['required', required], + ['preferred', preferred], + ['conditional', conditional], + ]; + for (let leftIndex = 0; leftIndex < categories.length; leftIndex += 1) { + const left = categories[leftIndex]; + if (!left) continue; + for (const right of categories.slice(leftIndex + 1)) { + const overlap = right[1].find((key) => left[1].includes(key)); + if (overlap !== undefined) { + throw new TypeError( + `Runtime operation cannot be both ${left[0]} and ${right[0]}: ${overlap}`, + ); + } + } } - return Object.freeze({ required, preferred }) as RuntimeUse; + return Object.freeze( + conditional.length === 0 ? { required, preferred } : { required, preferred, conditional }, + ) as RuntimeUse; }; } diff --git a/packages/contracts/src/platform-runtime.test.ts b/packages/contracts/src/platform-runtime.test.ts index 9e607ff98..b587fe4c3 100644 --- a/packages/contracts/src/platform-runtime.test.ts +++ b/packages/contracts/src/platform-runtime.test.ts @@ -15,19 +15,24 @@ import { runtimeUse } from './platform-runtime-use.ts'; type TestOperations = { inspect: (input: Readonly<{ depth: number }>) => Promise>; inspectFast: () => Promise>; + inspectConditional: () => Promise>; mutate: (input: Readonly<{ value: string }>) => Promise; }; const inspectUse = runtimeUse()({ required: ['inspect'], preferred: ['inspectFast'], + conditional: ['inspectConditional'], }); function compileTimeNarrowingProof(runtime: BoundDeviceRuntime): void { const required: TestOperations['inspect'] = runtime.operations.inspect; const preferred: TestOperations['inspectFast'] | undefined = runtime.operations.inspectFast; + const conditional: TestOperations['inspectConditional'] | undefined = + runtime.operations.inspectConditional; void required; void preferred; + void conditional; // @ts-expect-error An undeclared sibling operation cannot cross the selected projection. void runtime.operations.mutate; @@ -39,17 +44,32 @@ void compileTimeNarrowingProof; function compileTimeDisjointProof(): void { // @ts-expect-error Required and preferred keys are statically disjoint. runtimeUse()({ required: ['inspect'], preferred: ['inspect'] }); + runtimeUse()({ + required: ['inspect'], + preferred: ['inspectFast'], + // @ts-expect-error Preferred and conditional keys are statically disjoint. + conditional: ['inspectFast'], + }); } void compileTimeDisjointProof; +test('runtime use omits an empty conditional category from shipped declarations', () => { + const captureUse = runtimeUse()({ required: ['inspect'] }); + + assert.deepEqual(captureUse, { required: ['inspect'], preferred: [] }); + assert.equal('conditional' in captureUse, false); +}); + test('runtime use freezes declarations and rejects dynamic overlap or duplicates', () => { assert.deepEqual(inspectUse, { required: ['inspect'], preferred: ['inspectFast'], + conditional: ['inspectConditional'], }); assert.ok(Object.isFrozen(inspectUse)); assert.ok(Object.isFrozen(inspectUse.required)); assert.ok(Object.isFrozen(inspectUse.preferred)); + assert.ok(Object.isFrozen(inspectUse.conditional)); const dynamic = runtimeUse(); assert.throws( @@ -67,6 +87,15 @@ test('runtime use freezes declarations and rejects dynamic overlap or duplicates }), /duplicate required/, ); + assert.throws( + () => + dynamic({ + required: ['inspect'], + preferred: ['inspectFast'], + conditional: ['inspectFast'] as unknown as readonly ['inspectConditional'], + }), + /both preferred and conditional/, + ); }); test('runtime owner keys distinguish local families and configured provider instances', () => { @@ -93,12 +122,14 @@ test('binding narrowing proves required operations and omits unavailable preferr const binding = testBinding({ inspect: { available: true }, inspectFast: { available: false, reason: 'owner-capability-missing' }, + inspectConditional: { available: false, reason: 'owner-capability-missing' }, mutate: { available: true }, }); const runtime = narrowDeviceBinding(binding, inspectUse); assert.equal(runtime.operations.inspect, binding.operations.inspect); assert.equal(runtime.operations.inspectFast, undefined); + assert.equal(runtime.operations.inspectConditional, undefined); assert.deepEqual(runtime.facts.inspectFast, { available: false, reason: 'owner-capability-missing', @@ -110,6 +141,7 @@ test('binding narrowing fails closed on unsupported or falsely advertised requir const unsupported = testBinding({ inspect: { available: false, reason: 'unsupported-device-kind' }, inspectFast: { available: false, reason: 'owner-capability-missing' }, + inspectConditional: { available: false, reason: 'owner-capability-missing' }, mutate: { available: true }, }); assert.throws( @@ -120,6 +152,7 @@ test('binding narrowing fails closed on unsupported or falsely advertised requir const missing = testBinding({ inspect: { available: true }, inspectFast: { available: false, reason: 'owner-capability-missing' }, + inspectConditional: { available: false, reason: 'owner-capability-missing' }, mutate: { available: true }, }); delete (missing.operations as Partial).inspect; @@ -129,6 +162,25 @@ test('binding narrowing fails closed on unsupported or falsely advertised requir ); }); +test('binding narrowing requires every conditionally available operation implementation', () => { + const binding = testBinding({ + inspect: { available: true }, + inspectFast: { available: false, reason: 'owner-capability-missing' }, + inspectConditional: { available: true }, + mutate: { available: true }, + }); + assert.equal( + narrowDeviceBinding(binding, inspectUse).operations.inspectConditional, + binding.operations.inspectConditional, + ); + + delete (binding.operations as Partial).inspectConditional; + assert.throws( + () => narrowDeviceBinding(binding, inspectUse), + (error) => error instanceof AppError && error.details?.reason === 'runtime-contract-invalid', + ); +}); + function testBinding( facts: DeviceBinding['facts']['operations'], ): DeviceBinding { @@ -149,6 +201,7 @@ function testBinding( operations: { inspect: async () => ({ nodes: 1 }), inspectFast: async () => ({ nodes: 1 }), + inspectConditional: async () => ({ nodes: 1 }), mutate: async () => undefined, }, [Symbol.asyncDispose]: async () => undefined, diff --git a/packages/contracts/src/platform-runtime.ts b/packages/contracts/src/platform-runtime.ts index 5d5f8355f..3891f846a 100644 --- a/packages/contracts/src/platform-runtime.ts +++ b/packages/contracts/src/platform-runtime.ts @@ -27,16 +27,22 @@ declare const runtimeOperations: unique symbol; export type RuntimeUseDeclaration = Readonly<{ required: readonly string[]; preferred: readonly string[]; + conditional?: readonly string[]; }>; export type RuntimeUse< Operations extends object, Required extends readonly RuntimeOperationKey[], Preferred extends readonly Exclude, Required[number]>[], + Conditional extends readonly Exclude< + RuntimeOperationKey, + Required[number] | Preferred[number] + >[], > = RuntimeUseDeclaration & Readonly<{ required: Required; preferred: Preferred; + conditional?: Conditional; /** Type-only link to the operation catalog; never emitted in descriptor metadata. */ readonly [runtimeOperations]?: Operations; }>; @@ -165,27 +171,38 @@ export type RuntimePlatformModule = PlatformMod }>; type OperationsOf = - Use extends RuntimeUse ? Operations : never; + Use extends RuntimeUse + ? Operations + : never; type RequiredOf = - Use extends RuntimeUse + Use extends RuntimeUse ? Required[number] : never; type PreferredOf = - Use extends RuntimeUse + Use extends RuntimeUse ? Preferred[number] : never; +type ConditionalOf = + Use extends RuntimeUse + ? Conditional[number] + : never; + /** The non-disposable operation projection returned to a specialized handler. */ export type BoundDeviceRuntime = Readonly<{ device: DeviceInfo; owner: RuntimeOwnerRef; facts: Readonly< - Pick>['operations'], RequiredOf | PreferredOf> + Pick< + RuntimeFacts>['operations'], + RequiredOf | PreferredOf | ConditionalOf + > >; operations: Readonly< - Pick, RequiredOf> & Partial, PreferredOf>> + Pick, RequiredOf> & + Partial, PreferredOf | ConditionalOf>> >; }>; @@ -194,10 +211,14 @@ export function narrowDeviceBinding< Operations extends object, const Required extends readonly RuntimeOperationKey[], const Preferred extends readonly Exclude, Required[number]>[], + const Conditional extends readonly Exclude< + RuntimeOperationKey, + Required[number] | Preferred[number] + >[], >( binding: DeviceBinding, - use: RuntimeUse, -): BoundDeviceRuntime> { + use: RuntimeUse, +): BoundDeviceRuntime> { const selectedFacts: Record = {}; const selectedOperations: Record = {}; @@ -216,12 +237,20 @@ export function narrowDeviceBinding< } } + for (const key of use.conditional ?? []) { + const fact = requireRuntimeFact(binding.facts.operations, key); + selectedFacts[key] = fact; + if (fact.available) { + selectedOperations[key] = requireRuntimeOperation(binding.operations, key); + } + } + return Object.freeze({ device: binding.device, owner: binding.owner, facts: Object.freeze(selectedFacts), operations: Object.freeze(selectedOperations), - }) as BoundDeviceRuntime>; + }) as BoundDeviceRuntime>; } function requireRuntimeFact( diff --git a/packages/contracts/src/record-runtime-cutover.ts b/packages/contracts/src/record-runtime-cutover.ts index 9a70b0850..9d23a0d51 100644 --- a/packages/contracts/src/record-runtime-cutover.ts +++ b/packages/contracts/src/record-runtime-cutover.ts @@ -3,7 +3,7 @@ import { type CommandPlatformExecution, } from './command-platform-execution.ts'; import { screenRecordingRuntimePlanUses } from './screen-recording-runtime-plan.ts'; -import type { RuntimeUseDeclaration } from './platform-runtime.ts'; +import { runtimeUseIdentity } from './platform-runtime-use.ts'; /** Joins every normalized record plan to the descriptor's exhaustive runtime declaration. */ export function assertRecordRuntimeExecution( @@ -24,13 +24,6 @@ export function assertRecordRuntimeExecution( } } -function runtimeUseIdentity(use: RuntimeUseDeclaration): string { - return JSON.stringify({ - required: [...use.required].sort(), - preferred: [...use.preferred].sort(), - }); -} - function invalidRecordExecution(): TypeError { return new TypeError( 'Record runtime execution must declare exactly the uses selected by its two runtime-bearing plans', diff --git a/packages/contracts/src/selector-observation-runtime.ts b/packages/contracts/src/selector-observation-runtime.ts new file mode 100644 index 000000000..4b978b194 --- /dev/null +++ b/packages/contracts/src/selector-observation-runtime.ts @@ -0,0 +1,45 @@ +import type { ElementSelectorKey } from './interactor-types.ts'; +import type { RuntimeOperationFact } from './platform-runtime.ts'; +import type { SessionSurface } from './session-surface.ts'; +import type { SnapshotRuntimeExecution } from './snapshot-runtime.ts'; + +type ObservationInput = Readonly<{ + options?: Readonly<{ appBundleId?: string; surface?: SessionSurface }>; + execution?: SnapshotRuntimeExecution; + signal?: AbortSignal; +}>; + +export type FindTextInput = ObservationInput & Readonly<{ text: string }>; +export type FindSelectorInput = ObservationInput & + Readonly<{ selector: Readonly<{ key: ElementSelectorKey; value: string }> }>; + +/** + * Positive native observations are authoritative and preserve matches that an advertising owner's + * bulk capture may omit. A negative observation means only "not proven by this owner"; the caller + * must still consult its required canonical capture. Owners without either semantic source report + * that conditional operation unavailable and rely on their parity-proven capture path. + */ +export type SelectorObservationResult = Readonly<{ found: boolean }>; +export type FindTextResult = SelectorObservationResult; +export type FindSelectorResult = SelectorObservationResult; + +export type SelectorObservationRuntimeOperations = Readonly<{ + findText(input: FindTextInput): Promise; + findSelector(input: FindSelectorInput): Promise; +}>; +export type FindTextRuntimeOperations = Pick; +export type FindSelectorRuntimeOperations = Pick< + SelectorObservationRuntimeOperations, + 'findSelector' +>; + +export type SelectorObservationRuntimeOperationFacts = Readonly<{ + findText: RuntimeOperationFact; + findSelector: RuntimeOperationFact; +}>; + +export function selectorObservationRuntimeOperationFacts( + input: SelectorObservationRuntimeOperationFacts, +): SelectorObservationRuntimeOperationFacts { + return Object.freeze({ findText: input.findText, findSelector: input.findSelector }); +} diff --git a/packages/contracts/src/snapshot-runtime.test.ts b/packages/contracts/src/snapshot-runtime.test.ts index 6b3858995..da363411d 100644 --- a/packages/contracts/src/snapshot-runtime.test.ts +++ b/packages/contracts/src/snapshot-runtime.test.ts @@ -2,7 +2,11 @@ import assert from 'node:assert/strict'; import { test } from 'vitest'; import type { DeviceInfo } from '@agent-device/kernel/device'; import type { Interactor, RunnerContext } from './interactor-types.ts'; -import { bindLocalSnapshotInteractor, bindProviderSnapshotInteractor } from './snapshot-runtime.ts'; +import { + bindLocalSnapshotInteractor, + bindProviderSnapshotInteractor, + captureSnapshotSignal, +} from './snapshot-runtime.ts'; const device: DeviceInfo = { id: 'snapshot-device', @@ -66,3 +70,69 @@ test('provider snapshot binding fails closed when its selected owner loses the i error.message === 'Provider-owned snapshot operation has no bound provider interactor.', ); }); + +// --------------------------------------------------------------------------- +// Per-capture cancellation (`CaptureSnapshotInput.signal`). A `DeviceBinding`'s +// signal is fixed at bind time, but `wait` binds once and polls many times, so +// each poll must be able to cancel its own capture without cancelling the +// binding. These are the contract-level halves of that claim; the poll-deadline +// behaviour itself is proven end to end in `src/daemon/__tests__/wait-runtime.test.ts`. +// --------------------------------------------------------------------------- + +test('a capture with no per-capture signal receives the binding signal itself, not a wrapper', () => { + const binding = new AbortController().signal; + + // Identity, deliberately: a wrapper would satisfy a deep-equal while quietly changing + // cancellation semantics for every single-capture consumer (`snapshot`, `diff`), which pass + // no signal at all. + assert.equal(captureSnapshotSignal(binding, {}), binding); + assert.equal(captureSnapshotSignal(binding, { options: { appBundleId: 'x' } }), binding); +}); + +test('binding-level cancellation still aborts a capture that carries its own signal', () => { + const binding = new AbortController(); + const perCapture = new AbortController(); + + const composed = captureSnapshotSignal(binding.signal, { signal: perCapture.signal }); + + assert.equal(composed.aborted, false); + binding.abort(); + // The composition adds a second way to cancel; it must not have replaced the first. + assert.equal(composed.aborted, true); +}); + +test('a per-capture signal aborts its own capture without aborting the binding', () => { + const binding = new AbortController(); + const perCapture = new AbortController(); + + const composed = captureSnapshotSignal(binding.signal, { signal: perCapture.signal }); + perCapture.abort(new DOMException('Wait deadline exceeded', 'TimeoutError')); + + assert.equal(composed.aborted, true); + // The binding outlives the poll: the next poll of the same wait still has a live binding. + assert.equal(binding.signal.aborted, false); +}); + +test('the shared interactor binding composes the per-capture signal it is handed', async () => { + const binding = new AbortController(); + const perCapture = new AbortController(); + let capturedSignal: AbortSignal | undefined; + const operations = bindLocalSnapshotInteractor({ + device, + signal: binding.signal, + resolveInteractor: async () => + ({ + snapshot: async (options: Parameters[0]) => { + capturedSignal = options?.signal; + return { backend: 'android', nodes: [] }; + }, + }) as unknown as Interactor, + }); + + await operations.captureSnapshot({ signal: perCapture.signal }); + + assert.ok(capturedSignal, 'the interactor must receive a signal'); + assert.equal(capturedSignal.aborted, false); + perCapture.abort(new DOMException('Wait deadline exceeded', 'TimeoutError')); + assert.equal(capturedSignal.aborted, true, 'the poll deadline must reach the platform'); +}); diff --git a/packages/contracts/src/snapshot-runtime.ts b/packages/contracts/src/snapshot-runtime.ts index 1ddbdb2e0..81f093142 100644 --- a/packages/contracts/src/snapshot-runtime.ts +++ b/packages/contracts/src/snapshot-runtime.ts @@ -13,12 +13,34 @@ export type { SnapshotResult } from './interactor-types.ts'; /** Runner metadata needed by the selected snapshot implementation, without request-owned state. */ export type SnapshotRuntimeExecution = Readonly>; -/** Neutral snapshot intent. The request binding supplies cancellation and exact-owner authority. */ +/** Neutral snapshot intent. The request binding supplies exact-owner authority. */ export type CaptureSnapshotInput = Readonly<{ options?: Readonly>; execution?: SnapshotRuntimeExecution; + /** + * Per-capture cancellation, composed with the binding's own signal. A command that captures + * once needs nothing here. A POLLING command does: `wait` enforces each poll's remaining + * budget by aborting that capture and then waiting for it to quiesce (it deliberately does + * not race-and-abandon, so a late capture cannot mutate session state or keep a helper). + * Without this a stalled capture consumes the whole request instead of producing the poll's + * stalled-capture verdict. + */ + signal?: AbortSignal; }>; +/** + * The one place a per-capture signal joins its binding's: the binding always cancels, the + * caller may cancel sooner — identically for app and desktop surface captures. + */ +export function captureSnapshotSignal( + bindingSignal: AbortSignal, + input: CaptureSnapshotInput, +): AbortSignal { + return input.signal === undefined + ? bindingSignal + : AbortSignal.any([bindingSignal, input.signal]); +} + export type SnapshotRuntimeOperations = Readonly<{ captureSnapshot(input: CaptureSnapshotInput): Promise; captureSnapshotWithCustomActions(input: CaptureSnapshotInput): Promise; @@ -81,10 +103,11 @@ function bindSnapshotInteractor( params: SnapshotInteractorBindingParams, ): SnapshotRuntimeOperations { const captureSnapshot = async (input: CaptureSnapshotInput) => { + const signal = captureSnapshotSignal(params.signal, input); const runner: RunnerContext = { ...input.execution, appBundleId: input.options?.appBundleId, - signal: params.signal, + signal, }; const interactor = params.ownership === 'local' @@ -97,7 +120,7 @@ function bindSnapshotInteractor( { reason: 'provider-runtime-interactor-missing', deviceId: params.device.id }, ); } - return await interactor.snapshot({ ...input.options, signal: params.signal }); + return await interactor.snapshot({ ...input.options, signal }); }; return Object.freeze({ captureSnapshot, diff --git a/packages/contracts/src/wait-runtime-plan.ts b/packages/contracts/src/wait-runtime-plan.ts new file mode 100644 index 000000000..d33046d58 --- /dev/null +++ b/packages/contracts/src/wait-runtime-plan.ts @@ -0,0 +1,15 @@ +/** The normalized wait target, independent of the positional grammar that produced it. */ +export type WaitRuntimeTarget = 'sleep' | 'text' | 'ref' | 'selector' | 'stable'; + +/** + * Which wait shapes reach a device at all. A duration wait observes nothing, so it never asks + * for a plan and therefore never admits or binds — the absence of platform execution for that + * shape is stated here rather than left as an incidental branch in the handler. + * + * Every other shape polls the selector family's wait-observation capture plan. The binder remains + * shared, but wait's correctness-bearing conditionals are absent from capture-only and element-text + * uses, so unrelated commands cannot be rejected for operations they never execute. + */ +export function waitObservesDevice(target: WaitRuntimeTarget): boolean { + return target !== 'sleep'; +} diff --git a/packages/platform-android/src/runtime.ts b/packages/platform-android/src/runtime.ts index 7d11af67b..c9b3cbc21 100644 --- a/packages/platform-android/src/runtime.ts +++ b/packages/platform-android/src/runtime.ts @@ -15,6 +15,7 @@ import { elementTextRuntimeOperationFacts, localRuntimeOwner, screenshotRuntimeOperationFacts, + selectorObservationRuntimeOperationFacts, snapshotRuntimeOperationFacts, viewportRuntimeOperationFacts, } from '@agent-device/contracts/platform'; @@ -153,6 +154,11 @@ export function createAndroidPlatformRuntime(host: PlatformRuntimeHost): Platfor ...screenshotRuntimeOperationFacts({ capture: device.kind === 'simulator' ? screenshotKindUnavailable : available, }), + // No native text reading: every text wait on this owner polls the canonical tree. + ...selectorObservationRuntimeOperationFacts({ + findText: snapshotKindUnavailable, + findSelector: snapshotKindUnavailable, + }), ...viewportRuntimeOperationFacts({ setViewport: viewportUnavailable }), // uiautomator reads text at a point through the same adb path the snapshot uses, so the // synthetic `simulator` row is the only Android kind without a live read. diff --git a/packages/platform-apple/src/runtime-snapshot.test.ts b/packages/platform-apple/src/runtime-snapshot.test.ts new file mode 100644 index 000000000..6d39302da --- /dev/null +++ b/packages/platform-apple/src/runtime-snapshot.test.ts @@ -0,0 +1,79 @@ +import { expect, test, vi } from 'vitest'; +import type { ElementSelectorKey } from '@agent-device/contracts/interaction'; +import type { DeviceInfo } from '@agent-device/kernel/device'; +import { platformRuntimeHostFixture } from './runtime.fixtures.ts'; +import { bindAppleFindSelectorRuntime } from './runtime-snapshot.ts'; + +const ios = { + platform: 'apple', + appleOs: 'ios', + id: 'ios', + name: 'iOS', + kind: 'simulator', + target: 'mobile', + booted: true, +} as const satisfies DeviceInfo; + +test('findSelector resolves the owner interactor once with request execution and cancellation', async () => { + const findSelector = vi.fn( + async ( + _selector: Readonly<{ key: ElementSelectorKey; value: string }>, + _options?: { appBundleId?: string; signal?: AbortSignal }, + ) => ({ found: true }), + ); + const resolve = vi.fn(async () => ({ findSelector }) as never); + const host = { ...platformRuntimeHostFixture(), localInteractors: { resolve } }; + const request = new AbortController(); + const poll = new AbortController(); + const operation = bindAppleFindSelectorRuntime(host, { device: ios, signal: request.signal }); + + await expect( + operation.findSelector({ + selector: { key: 'id', value: 'submit' }, + options: { appBundleId: 'com.example.app', surface: 'app' }, + execution: { requestId: 'request-1' }, + signal: poll.signal, + }), + ).resolves.toEqual({ found: true }); + + expect(resolve).toHaveBeenCalledOnce(); + expect(resolve).toHaveBeenCalledWith( + ios, + expect.objectContaining({ appBundleId: 'com.example.app', requestId: 'request-1' }), + ); + expect(findSelector).toHaveBeenCalledWith( + { key: 'id', value: 'submit' }, + expect.objectContaining({ appBundleId: 'com.example.app', signal: expect.any(AbortSignal) }), + ); + const signal = findSelector.mock.calls[0]?.[1]?.signal; + poll.abort(new DOMException('poll ended', 'AbortError')); + expect(signal?.aborted).toBe(true); +}); + +test.each([ + ['no active app', ios, undefined, undefined], + [ + 'non-app macOS surface', + { ...ios, appleOs: 'macos', kind: 'device', target: 'desktop' } as const, + 'com.example.app', + 'desktop' as const, + ], +])( + 'findSelector declines %s without resolving an interactor', + async (_name, device, appBundleId, surface) => { + const resolve = vi.fn(async () => ({}) as never); + const host = { ...platformRuntimeHostFixture(), localInteractors: { resolve } }; + const operation = bindAppleFindSelectorRuntime(host, { + device, + signal: new AbortController().signal, + }); + + await expect( + operation.findSelector({ + selector: { key: 'label', value: 'Settings' }, + options: { ...(appBundleId ? { appBundleId } : {}), ...(surface ? { surface } : {}) }, + }), + ).resolves.toEqual({ found: false }); + expect(resolve).not.toHaveBeenCalled(); + }, +); diff --git a/packages/platform-apple/src/runtime-snapshot.ts b/packages/platform-apple/src/runtime-snapshot.ts index fea587a39..d61d5095a 100644 --- a/packages/platform-apple/src/runtime-snapshot.ts +++ b/packages/platform-apple/src/runtime-snapshot.ts @@ -1,9 +1,16 @@ import type { CaptureSnapshotInput, + FindSelectorInput, + FindSelectorResult, + FindTextInput, + FindTextResult, PlatformRuntimeHost, PlatformRuntimeOperations, } from '@agent-device/contracts/platform'; -import { bindLocalSnapshotInteractor } from '@agent-device/contracts/platform'; +import { + bindLocalSnapshotInteractor, + captureSnapshotSignal, +} from '@agent-device/contracts/platform'; import { isMacOs, type DeviceInfo } from '@agent-device/kernel/device'; /** Apple-owned selection between app snapshots and explicit macOS surface snapshots. */ @@ -22,7 +29,11 @@ export function bindAppleSnapshotRuntime( input.options?.surface !== undefined && input.options.surface !== 'app' ) { - return await host.snapshot.captureSurface(request.device, input.options, request.signal); + return await host.snapshot.captureSurface( + request.device, + input.options, + captureSnapshotSignal(request.signal, input), + ); } return await appSnapshot.captureSnapshot(input); }; @@ -37,3 +48,75 @@ type SnapshotRuntimeOperation = Pick< PlatformRuntimeOperations, 'captureSnapshot' | 'captureSnapshotWithCustomActions' | 'captureSnapshotWithoutActiveApp' >; + +/** + * The runner's native text reading. Every condition under which Apple cannot answer lives here + * rather than in the daemon, which is the point of the migration: no caller inspects the family, + * the surface, or the session to decide whether to consult it. + * + * - No tracked app bundle id: the runner query is scoped to an application, so there is nothing + * to ask about. + * - macOS on an explicit non-app surface: the runner reads the *application*, so a positive + * answer would describe the wrong surface. Reporting `false` sends the poll to the desktop + * surface capture, which is the reading that matches the request. + * + * Both report `found: false` — "not proven here" — never an error, so the caller's canonical tree + * remains the complete path (ADR 0019 section 2). + */ +export function bindAppleFindTextRuntime( + host: PlatformRuntimeHost, + request: Readonly<{ device: DeviceInfo; signal: AbortSignal }>, +): Pick { + return Object.freeze({ + findText: async (input: FindTextInput): Promise => { + const appBundleId = input.options?.appBundleId; + if (appBundleId === undefined) return { found: false }; + if (isMacOs(request.device) && input.options?.surface !== undefined) { + if (input.options.surface !== 'app') return { found: false }; + } + const signal = + input.signal === undefined + ? request.signal + : AbortSignal.any([request.signal, input.signal]); + signal.throwIfAborted(); + const interactor = await host.localInteractors.resolve(request.device, { + ...input.execution, + appBundleId, + signal, + }); + if (!interactor.findText) return { found: false }; + return await interactor.findText(input.text, { appBundleId, signal }); + }, + }); +} + +/** Apple owns the native simple-selector observation; callers never inspect Apple/provider state. */ +export function bindAppleFindSelectorRuntime( + host: PlatformRuntimeHost, + request: Readonly<{ device: DeviceInfo; signal: AbortSignal }>, +): Pick { + return Object.freeze({ + findSelector: async (input: FindSelectorInput): Promise => { + const appBundleId = input.options?.appBundleId; + if (appBundleId === undefined) return { found: false }; + if ( + isMacOs(request.device) && + input.options?.surface !== undefined && + input.options.surface !== 'app' + ) { + return { found: false }; + } + const signal = input.signal + ? AbortSignal.any([request.signal, input.signal]) + : request.signal; + signal.throwIfAborted(); + const interactor = await host.localInteractors.resolve(request.device, { + ...input.execution, + appBundleId, + signal, + }); + if (!interactor.findSelector) return { found: false }; + return await interactor.findSelector(input.selector, { appBundleId, signal }); + }, + }); +} diff --git a/packages/platform-apple/src/runtime.test.ts b/packages/platform-apple/src/runtime.test.ts index b51e45fad..a837cf7cb 100644 --- a/packages/platform-apple/src/runtime.test.ts +++ b/packages/platform-apple/src/runtime.test.ts @@ -3,6 +3,7 @@ import type { DeviceBinding, PlatformRuntimeOperations, RuntimeFacts, + SnapshotRuntimeHost, } from '@agent-device/contracts/platform'; import type { AppleOS, DeviceInfo } from '@agent-device/kernel/device'; import { createApplePlatformRuntime } from './runtime.ts'; @@ -138,6 +139,11 @@ function expectAppleSnapshotAvailability( // capture does; every other supported leaf advertises and binds it. expect(binding.facts.operations.readTextAtPoint.available).toBe(available); expect(binding.operations.readTextAtPoint).toBeTypeOf(available ? 'function' : 'undefined'); + const nativeSelectorAvailable = available && device.appleOs !== 'macos'; + expect(binding.facts.operations.findSelector.available).toBe(nativeSelectorAvailable); + expect(binding.operations.findSelector).toBeTypeOf( + nativeSelectorAvailable ? 'function' : 'undefined', + ); } test.each(['frontmost-app', 'desktop', 'menubar'] as const)( @@ -431,4 +437,44 @@ function expectLegacyLifecycleFactCell( (facts.device.kind === 'simulator' || facts.device.kind === 'device'); expect(facts.operations.captureSnapshot.available).toBe(snapshotAvailable); expect(facts.operations.readTextAtPoint.available).toBe(snapshotAvailable); + expect(facts.operations.findSelector.available).toBe( + snapshotAvailable && facts.device.appleOs !== 'macos', + ); } + +// The macOS non-app surface branch calls `captureSurface` directly instead of going through +// `bindSnapshotInteractor`, so the shared composition does NOT cover it. If this branch drops the +// per-capture signal, desktop-surface captures silently ignore a wait's poll deadline while every +// other family honours it. +test('the macOS surface branch composes the per-capture signal with the binding signal', async () => { + const host = platformRuntimeHostFixture(); + const captureSurface = vi.fn(async () => ({ + backend: 'macos-helper' as const, + nodes: [], + truncated: false, + })); + const binding = await createApplePlatformRuntime({ + ...host, + localInteractors: { resolve: vi.fn(async () => ({}) as never) }, + snapshot: { captureSurface }, + }).bind({ + device: leaves.macos, + intent: { kind: 'ordinary' }, + scope: { + signal: new AbortController().signal, + diagnostics: { emit: () => {} }, + progress: { report: () => {} }, + }, + }); + + const poll = new AbortController(); + await binding.operations.captureSnapshot?.({ + options: { surface: 'desktop', appBundleId: 'com.example.app' }, + signal: poll.signal, + }); + + const passed = captureSurface.mock.calls[0]?.[2] as AbortSignal; + expect(passed.aborted).toBe(false); + poll.abort(new DOMException('Wait deadline exceeded', 'TimeoutError')); + expect(passed.aborted).toBe(true); +}); diff --git a/packages/platform-apple/src/runtime.ts b/packages/platform-apple/src/runtime.ts index 1b94570c3..d16b777ed 100644 --- a/packages/platform-apple/src/runtime.ts +++ b/packages/platform-apple/src/runtime.ts @@ -1,6 +1,7 @@ import type { DeviceBinding, NetworkDumpInput, + RuntimeOperationFact, PlatformRuntimeHost, PlatformRuntimeOperations, PlatformRuntimeOwner, @@ -13,6 +14,7 @@ import { elementTextRuntimeOperationFacts, localRuntimeOwner, screenshotRuntimeOperationFacts, + selectorObservationRuntimeOperationFacts, snapshotRuntimeOperationFacts, viewportRuntimeOperationFacts, } from '@agent-device/contracts/platform'; @@ -34,7 +36,11 @@ import { appleAppDeploymentFacts, createAppleAppDeploymentOperations, } from './deployment/runtime.ts'; -import { bindAppleSnapshotRuntime } from './runtime-snapshot.ts'; +import { + bindAppleFindSelectorRuntime, + bindAppleFindTextRuntime, + bindAppleSnapshotRuntime, +} from './runtime-snapshot.ts'; const owner = localRuntimeOwner('apple'); const available = Object.freeze({ available: true } as const); @@ -127,6 +133,11 @@ const snapshotActiveAppRequired = Object.freeze({ reason: 'owner-capability-missing', hint: 'Open the app under test before capturing its snapshot.', } as const); +const nativeSelectorUnavailable = Object.freeze({ + available: false, + reason: 'unsupported-platform-leaf', + hint: 'Native selector observation is available only on the Apple touch family.', +} as const); function unsupportedAppleDeviceKind(hint: string) { return Object.freeze({ available: false, reason: 'unsupported-device-kind', hint } as const); @@ -234,6 +245,10 @@ export function createApplePlatformRuntime(host: PlatformRuntimeHost): PlatformR screenRecordingCleanup: recordingFacts, ...appleSnapshotFacts(device), ...screenshotRuntimeOperationFacts({ capture: appleScreenshotFact(device) }), + ...selectorObservationRuntimeOperationFacts({ + findText: appleSnapshotFact(device), + findSelector: appleFindSelectorFact(device), + }), ...viewportRuntimeOperationFacts({ setViewport: viewportUnavailable }), ...elementTextRuntimeOperationFacts({ readTextAtPoint: appleElementTextFact(device) }), ensureReady: readiness, @@ -266,56 +281,62 @@ export function createApplePlatformRuntime(host: PlatformRuntimeHost): PlatformR }), networkDump: async (input: NetworkDumpInput) => await dumpAppleNetworkTraffic(host, request.device, input, request.scope.signal), - ...(recordingFacts.available - ? createAppleScreenRecordingOperations({ - host, - device: request.device, - owner, - signal: request.scope.signal, - }) - : {}), - ...(facts.operations.captureSnapshot.available - ? bindAppleSnapshotRuntime(host, { - device: request.device, - signal: request.scope.signal, - }) - : {}), - ...(facts.operations.captureScreenshot.available - ? bindLocalScreenshotInteractor({ - device: request.device, - signal: request.scope.signal, - resolveInteractor: host.localInteractors.resolve, - }) - : {}), - ...(facts.operations.readTextAtPoint.available - ? bindElementTextRuntime({ - device: request.device, - signal: request.scope.signal, - resolveInteractor: host.localInteractors.resolve, - }) - : {}), - ...(facts.operations.ensureReady.available - ? { - ensureReady: async () => - await ensureAppleReady(host, request.device, request.scope.signal), - } - : {}), - ...(facts.operations.bootTarget.available - ? { - bootTarget: async () => - await ensureAppleReady(host, request.device, request.scope.signal), - } - : {}), - ...(facts.operations.listApps.available - ? { - listApps: async (input: { device: DeviceInfo; filter: 'all' | 'user-installed' }) => - await host.appInventory.apple.listApps( - input.device, - input.filter, - request.scope.signal, - ), - } - : {}), + ...whenAdmitted(recordingFacts, () => + createAppleScreenRecordingOperations({ + host, + device: request.device, + owner, + signal: request.scope.signal, + }), + ), + ...whenAdmitted(facts.operations.captureSnapshot, () => + bindAppleSnapshotRuntime(host, { + device: request.device, + signal: request.scope.signal, + }), + ), + ...whenAdmitted(facts.operations.captureScreenshot, () => + bindLocalScreenshotInteractor({ + device: request.device, + signal: request.scope.signal, + resolveInteractor: host.localInteractors.resolve, + }), + ), + ...whenAdmitted(facts.operations.readTextAtPoint, () => + bindElementTextRuntime({ + device: request.device, + signal: request.scope.signal, + resolveInteractor: host.localInteractors.resolve, + }), + ), + ...whenAdmitted(facts.operations.findText, () => + bindAppleFindTextRuntime(host, { + device: request.device, + signal: request.scope.signal, + }), + ), + ...whenAdmitted(facts.operations.findSelector, () => + bindAppleFindSelectorRuntime(host, { + device: request.device, + signal: request.scope.signal, + }), + ), + ...whenAdmitted(facts.operations.ensureReady, () => ({ + ensureReady: async () => + await ensureAppleReady(host, request.device, request.scope.signal), + })), + ...whenAdmitted(facts.operations.bootTarget, () => ({ + bootTarget: async () => + await ensureAppleReady(host, request.device, request.scope.signal), + })), + ...whenAdmitted(facts.operations.listApps, () => ({ + listApps: async (input: { device: DeviceInfo; filter: 'all' | 'user-installed' }) => + await host.appInventory.apple.listApps( + input.device, + input.filter, + request.scope.signal, + ), + })), ...availableApplicationLifecycleOperations( bindAppleApplicationLifecycle({ host, @@ -324,15 +345,10 @@ export function createApplePlatformRuntime(host: PlatformRuntimeHost): PlatformR }), facts.operations, ), - ...(facts.operations.shutdownTarget.available - ? { - shutdownTarget: async () => - await host.deviceShutdown.apple.shutdownTarget( - request.device, - request.scope.signal, - ), - } - : {}), + ...whenAdmitted(facts.operations.shutdownTarget, () => ({ + shutdownTarget: async () => + await host.deviceShutdown.apple.shutdownTarget(request.device, request.scope.signal), + })), }), [Symbol.asyncDispose]: async () => await logs[Symbol.asyncDispose](), }) satisfies DeviceBinding; @@ -360,6 +376,10 @@ function appleSnapshotFact(device: DeviceInfo) { : snapshotKindUnavailable; } +function appleFindSelectorFact(device: DeviceInfo) { + return isIosFamily(device) ? appleSnapshotFact(device) : nativeSelectorUnavailable; +} + /** * macOS surface selection (app window vs desktop/menubar) lives inside the Apple interactor's own * capture, so the ordinary local interactor binding covers every admitted Apple cell — unlike @@ -383,3 +403,15 @@ function appleSnapshotFacts(device: DeviceInfo) { withoutActiveApp: isIosFamily(device) ? snapshotActiveAppRequired : capture, }); } + +/** + * An operation is present on a binding only when the owner's own facts admitted it. One helper so + * the binding below reads as a list of admitted operations rather than a chain of branches — and + * so the next operation added here costs no additional complexity. + */ +function whenAdmitted( + fact: RuntimeOperationFact, + build: () => T, +): T | Record { + return fact.available ? build() : {}; +} diff --git a/packages/platform-harmonyos/src/runtime.ts b/packages/platform-harmonyos/src/runtime.ts index f81aebb05..a99f6b875 100644 --- a/packages/platform-harmonyos/src/runtime.ts +++ b/packages/platform-harmonyos/src/runtime.ts @@ -12,6 +12,7 @@ import { elementTextRuntimeOperationFacts, localRuntimeOwner, screenshotRuntimeOperationFacts, + selectorObservationRuntimeOperationFacts, snapshotRuntimeOperationFacts, viewportRuntimeOperationFacts, } from '@agent-device/contracts/platform'; @@ -148,6 +149,11 @@ export function createHarmonyPlatformRuntime(host: PlatformRuntimeHost): Platfor ? available : screenshotKindUnavailable, }), + // No native text reading: every text wait on this owner polls the canonical tree. + ...selectorObservationRuntimeOperationFacts({ + findText: snapshotKindUnavailable, + findSelector: snapshotKindUnavailable, + }), ...viewportRuntimeOperationFacts({ setViewport: viewportUnavailable }), // HarmonyOS has no point-read tool: `get` answers from the captured tree, which is what // the legacy dispatch already did after its Apple-runner attempt failed. diff --git a/packages/platform-linux/src/runtime.test.ts b/packages/platform-linux/src/runtime.test.ts index 56ce74812..c2378092e 100644 --- a/packages/platform-linux/src/runtime.test.ts +++ b/packages/platform-linux/src/runtime.test.ts @@ -3,6 +3,7 @@ import type { DeviceBinding, PlatformRuntimeHost, PlatformRuntimeOperations, + SnapshotRuntimeHost, } from '@agent-device/contracts/platform'; import type { DeviceInfo } from '@agent-device/kernel/device'; import { createLinuxPlatformRuntime } from './runtime.ts'; @@ -131,7 +132,7 @@ test.each([ ); function lifecycleHost( - captureSurface = async () => ({ + captureSurface: SnapshotRuntimeHost['captureSurface'] = async () => ({ backend: 'linux-atspi' as const, nodes: [], truncated: false, @@ -162,3 +163,47 @@ function expectLifecycleFacts( } } } + +// `linuxSnapshotOperations` is the other direct `captureSurface` caller, so the shared +// `bindSnapshotInteractor` composition does not cover it either. Note it composes against +// `request.scope.signal` rather than `request.signal` — the Linux owner takes its signal off the +// request scope, which is exactly the detail a patch application can get wrong silently. +test('the Linux surface capture composes the per-capture signal with the request scope signal', async () => { + const captureSurface = vi.fn(async () => ({ + backend: 'linux-atspi' as const, + nodes: [], + truncated: false, + })); + const scope = new AbortController(); + const binding = await createLinuxPlatformRuntime(lifecycleHost(captureSurface)).bind({ + device: { + platform: 'linux' as const, + id: 'linux', + name: 'Linux', + kind: 'device' as const, + target: 'desktop' as const, + booted: true, + }, + intent: { kind: 'ordinary' }, + scope: { + signal: scope.signal, + diagnostics: { emit: () => {} }, + progress: { report: () => {} }, + }, + }); + + const poll = new AbortController(); + await binding.operations.captureSnapshot?.({ options: {}, signal: poll.signal }); + + const passed = captureSurface.mock.calls[0]?.[2] as AbortSignal; + expect(passed.aborted).toBe(false); + poll.abort(new DOMException('Wait deadline exceeded', 'TimeoutError')); + expect(passed.aborted).toBe(true); + + // The scope signal must still cancel too: composition adds a way to cancel, never replaces one. + const second = new AbortController(); + await binding.operations.captureSnapshot?.({ options: {}, signal: second.signal }); + const passedSecond = captureSurface.mock.calls[1]?.[2] as AbortSignal; + scope.abort(); + expect(passedSecond.aborted).toBe(true); +}); diff --git a/packages/platform-linux/src/runtime.ts b/packages/platform-linux/src/runtime.ts index f8842b4f2..99d1e578d 100644 --- a/packages/platform-linux/src/runtime.ts +++ b/packages/platform-linux/src/runtime.ts @@ -12,6 +12,7 @@ import { availableApplicationLifecycleOperations, bindLocalScreenshotInteractor, bindElementTextRuntime, + captureSnapshotSignal, createUnavailablePlatformRuntimeFacts, elementTextRuntimeOperationFacts, localRuntimeOwner, @@ -160,7 +161,11 @@ function linuxSnapshotOperations( request: Parameters[0], ) { const captureSnapshot = async (input: CaptureSnapshotInput) => - await host.snapshot.captureSurface(request.device, input.options, request.scope.signal); + await host.snapshot.captureSurface( + request.device, + input.options, + captureSnapshotSignal(request.scope.signal, input), + ); return Object.freeze({ captureSnapshot, captureSnapshotWithCustomActions: captureSnapshot, diff --git a/packages/platform-web/src/runtime.ts b/packages/platform-web/src/runtime.ts index 82390e1c7..099c0efe4 100644 --- a/packages/platform-web/src/runtime.ts +++ b/packages/platform-web/src/runtime.ts @@ -13,6 +13,7 @@ import { elementTextRuntimeOperationFacts, localRuntimeOwner, screenshotRuntimeOperationFacts, + selectorObservationRuntimeOperationFacts, snapshotRuntimeOperationFacts, sameRuntimeOwner, viewportRuntimeOperationFacts, @@ -248,6 +249,14 @@ function webRuntimeFacts( customActions: snapshotCustomActionsUnavailable, withoutActiveApp: browserDevice, }), + // No native text reading: every text wait on this owner polls the canonical tree. + ...selectorObservationRuntimeOperationFacts({ + findText: openTargetKindUnavailable, + findSelector: openTargetKindUnavailable, + }), + ...viewportRuntimeOperationFacts({ + setViewport: device.kind === 'device' ? available : openTargetKindUnavailable, + }), ...screenshotRuntimeOperationFacts({ capture: browserDevice }), ...viewportRuntimeOperationFacts({ setViewport: browserDevice }), // The web backend has no point-addressed read: `get` answers from the captured DOM tree, diff --git a/packages/provider-limrun/src/app-log-runtime.ts b/packages/provider-limrun/src/app-log-runtime.ts index 68e4b7841..c52ac3c99 100644 --- a/packages/provider-limrun/src/app-log-runtime.ts +++ b/packages/provider-limrun/src/app-log-runtime.ts @@ -28,6 +28,7 @@ import { sameRuntimeOwner, screenshotRuntimeOperationFacts, elementTextRuntimeOperationFacts, + selectorObservationRuntimeOperationFacts, snapshotRuntimeOperationFacts, viewportRuntimeOperationFacts, } from '@agent-device/contracts/platform'; @@ -453,6 +454,12 @@ function facts( withoutActiveApp: available, }), ...screenshotRuntimeOperationFacts({ capture: available }), + // Provider ownership is authoritative: no native text reading is exposed, so text waits + // on a Limrun-owned device poll the canonical tree rather than borrowing Apple's. + ...selectorObservationRuntimeOperationFacts({ + findText: customSnapshotUnavailable, + findSelector: customSnapshotUnavailable, + }), ...viewportRuntimeOperationFacts({ setViewport: viewportUnavailable }), ...elementTextRuntimeOperationFacts({ readTextAtPoint: elementTextUnavailable }), ensureReady: available, @@ -494,6 +501,10 @@ function recoveryFacts( withoutActiveApp: liveSessionUnavailable, }), ...screenshotRuntimeOperationFacts({ capture: liveSessionUnavailable }), + ...selectorObservationRuntimeOperationFacts({ + findText: liveSessionUnavailable, + findSelector: liveSessionUnavailable, + }), ...viewportRuntimeOperationFacts({ setViewport: liveSessionUnavailable }), ensureReady: liveSessionUnavailable, bootTarget: liveSessionUnavailable, diff --git a/scripts/layering/runtime-command-cutover-table.ts b/scripts/layering/runtime-command-cutover-table.ts index 1e5783e3f..f11059e9d 100644 --- a/scripts/layering/runtime-command-cutover-table.ts +++ b/scripts/layering/runtime-command-cutover-table.ts @@ -545,7 +545,7 @@ export const MIGRATED_COMMAND_CUTOVERS: readonly MigratedCommandCutover[] = [ operationOwners: { captureSnapshot: ['selectActiveAppSnapshot'], captureSnapshotWithoutActiveApp: ['selectSnapshotWithoutActiveApp'], - readTextAtPoint: ['bindElementRead'], + readTextAtPoint: ['selectElementTextOperation'], }, }, }, @@ -627,6 +627,53 @@ export const MIGRATED_COMMAND_CUTOVERS: readonly MigratedCommandCutover[] = [ operationOwners: { captureScreenshot: ['selectScreenshotCapture'] }, }, }, + { + rule: 'R38 wait-runtime-cutover', + command: 'wait', + subject: 'wait polling capture', + tier: 'request-scoped', + execution: 'device-runtime', + legacyRetirement: { + // The direct iOS selector probe wait took before polling, and the daemon-side Apple + // plumbing that used to answer `findText` by selecting on family and provider. The + // reading itself did not go away — it moved behind the owner's declared `findText` + // operation — but nothing in the daemon may reach the runner for it again. + routeNames: [ + 'dispatchDirectIosSelectorWait', + 'findTextWithAppleRunner', + 'findTextInMacosNonAppSurface', + 'readAppleRunnerFindTextTarget', + 'buildAppleRunnerFindTextOptions', + 'captureWaitSnapshot', + 'AppleRunnerFindTextTarget', + ], + }, + runtimeTypeNames: [ + 'SnapshotRuntimeOperations', + 'FindTextRuntimeOperations', + 'FindSelectorRuntimeOperations', + ], + operations: { + names: ['captureSnapshot', 'captureSnapshotWithoutActiveApp', 'findText', 'findSelector'], + }, + singularExecution: { + routes: ['handleSnapshotCommands'], + operations: [ + 'captureSnapshot', + 'captureSnapshotWithoutActiveApp', + 'findText', + 'findSelector', + ], + // The selector family's shared owners: wait binds through the same admit-then-bind entry + // as find/get/is, so it introduces no parallel binder for either operation. + operationOwners: { + captureSnapshot: ['selectActiveAppSnapshot'], + captureSnapshotWithoutActiveApp: ['selectSnapshotWithoutActiveApp'], + findText: ['selectWaitObservationOperations'], + findSelector: ['selectWaitObservationOperations'], + }, + }, + }, ]; function snapshotRetiredDispatchProjectionProof( diff --git a/src/__tests__/test-file-size-ratchet.test.ts b/src/__tests__/test-file-size-ratchet.test.ts index d3120b72f..504d4f23d 100644 --- a/src/__tests__/test-file-size-ratchet.test.ts +++ b/src/__tests__/test-file-size-ratchet.test.ts @@ -34,7 +34,7 @@ const TRIPWIRE_LINES = 1_000; // Exact current lengths. Lower a pin when its file shrinks; never raise one — extract instead. const PINNED_TEST_FILE_LINES: Readonly> = Object.freeze({ 'src/__tests__/remote-connection.test.ts': 2973, - 'src/daemon/handlers/__tests__/snapshot-handler.test.ts': 2638, + 'src/daemon/handlers/__tests__/snapshot-handler.test.ts': 2310, 'src/commands/interaction/runtime/settle.test.ts': 2361, 'src/daemon/handlers/__tests__/session-replay-runtime-maestro.test.ts': 2031, 'src/platforms/apple/core/__tests__/runner-session.test.ts': 2001, diff --git a/src/__tests__/test-utils/runtime-operation-facts.ts b/src/__tests__/test-utils/runtime-operation-facts.ts index d7400257d..caa8fd859 100644 --- a/src/__tests__/test-utils/runtime-operation-facts.ts +++ b/src/__tests__/test-utils/runtime-operation-facts.ts @@ -33,6 +33,8 @@ export const unavailableDeploymentSnapshotAndShutdownOperationFacts = Object.fre }), ...unavailableShutdownOperationFacts, ...screenshotRuntimeOperationFacts({ capture: unavailable }), + findText: unavailable, + findSelector: unavailable, setViewport: unavailable, ...elementTextRuntimeOperationFacts({ readTextAtPoint: unavailable }), }); diff --git a/src/backend.ts b/src/backend.ts index 501509825..e75b774e2 100644 --- a/src/backend.ts +++ b/src/backend.ts @@ -66,6 +66,10 @@ export type BackendReadTextResult = { text: string; }; +/** + * A backend's native text-presence reading. `found: true` is authoritative; `false` means + * "not proven by this backend" and the caller still consults the canonical tree. + */ export type BackendFindTextResult = { found: boolean; }; @@ -434,6 +438,7 @@ export type AgentDeviceBackend = { options?: BackendScreenshotOptions, ): Promise; readText?(context: BackendCommandContext, node: SnapshotNode): Promise; + /** Present only when the bound runtime advertised the conditional `findText` operation. */ findText?(context: BackendCommandContext, text: string): Promise; /** * #1542 off-screen refusal double-check: called ONLY at the moment the diff --git a/src/commands/interaction/runtime/__tests__/test-utils/index.ts b/src/commands/interaction/runtime/__tests__/test-utils/index.ts index fe549d49a..e905e33ca 100644 --- a/src/commands/interaction/runtime/__tests__/test-utils/index.ts +++ b/src/commands/interaction/runtime/__tests__/test-utils/index.ts @@ -559,7 +559,6 @@ export function createSelectorDevice( snapshot: SnapshotState, options: { readText?: string; - findText?: boolean; now?: number; captureSnapshot?: () => BackendSnapshotResult | Promise; /** @@ -584,7 +583,6 @@ export function createSelectorDevice( captureSnapshot: async () => options.captureSnapshot ? await options.captureSnapshot() : { snapshot }, readText: async () => ({ text: options.readText ?? '' }), - findText: async () => ({ found: options.findText ?? false }), } satisfies AgentDeviceBackend, artifacts: createLocalArtifactAdapter(), sessions, diff --git a/src/commands/interaction/runtime/selector-read.test.ts b/src/commands/interaction/runtime/selector-read.test.ts index a36ba1310..1112aba12 100644 --- a/src/commands/interaction/runtime/selector-read.test.ts +++ b/src/commands/interaction/runtime/selector-read.test.ts @@ -476,7 +476,6 @@ test('runtime find wait cancels and joins a capture that consumes its full deadl test('runtime selector convenience methods use explicit target helpers', async () => { const device = createSelectorDevice(selectorReadSnapshot(), { readText: 'Continue', - findText: true, }); const text = await device.selectors.getText(selector('label=Continue'), { session: 'default' }); @@ -484,7 +483,7 @@ test('runtime selector convenience methods use explicit target helpers', async ( const visible = await device.selectors.isVisible(selector('label=Continue'), { session: 'default', }); - const waited = await device.selectors.waitForText('Ready', { + const waited = await device.selectors.waitForText('Continue', { session: 'default', timeoutMs: 100, }); @@ -492,7 +491,7 @@ test('runtime selector convenience methods use explicit target helpers', async ( assert.equal(text.kind, 'text'); assert.equal(attrs.kind, 'attrs'); assert.equal(visible.pass, true); - assert.deepEqual(waited, { kind: 'text', text: 'Ready', waitedMs: 0 }); + assert.deepEqual(waited, { kind: 'text', text: 'Continue', waitedMs: 0 }); }); // --------------------------------------------------------------------------- diff --git a/src/commands/interaction/runtime/selector-wait.test.ts b/src/commands/interaction/runtime/selector-wait.test.ts index 8b1b9175d..5747a82a2 100644 --- a/src/commands/interaction/runtime/selector-wait.test.ts +++ b/src/commands/interaction/runtime/selector-wait.test.ts @@ -52,18 +52,31 @@ test('runtime focused selector waits against a full snapshot', async () => { assert.equal(captureOptions?.interactiveOnly, false); }); -test('runtime wait can use backend text search', async () => { - const device = createSelectorDevice(selectorReadSnapshot(), { - findText: true, - now: 10, - }); +// A text wait has exactly one source of truth: the polled capture. The backend `findText` seam +// that short-circuited it on Apple was wait's second platform-execution path and retired with +// wait's ADR 0019 cutover, so the tree answer is the only answer — in both directions. +test('runtime wait resolves text from the polled snapshot', async () => { + const device = createSelectorDevice(selectorReadSnapshot(), { now: 10 }); const result = await device.selectors.wait({ session: 'default', - target: { kind: 'text', text: 'Ready', timeoutMs: 100 }, + target: { kind: 'text', text: 'Continue', timeoutMs: 100 }, }); - assert.deepEqual(result, { kind: 'text', text: 'Ready', waitedMs: 0 }); + assert.deepEqual(result, { kind: 'text', text: 'Continue', waitedMs: 0 }); +}); + +test('runtime wait times out on text the polled snapshot does not carry', async () => { + const device = createSelectorDevice(selectorReadSnapshot(), { clock: createFakeClock() }); + + await assert.rejects( + async () => + await device.selectors.wait({ + session: 'default', + target: { kind: 'text', text: 'Ready', timeoutMs: 100 }, + }), + (error: Error) => error.message.includes('wait timed out for text: Ready'), + ); }); // --------------------------------------------------------------------------- diff --git a/src/commands/interaction/runtime/selector-wait.ts b/src/commands/interaction/runtime/selector-wait.ts index 60a3dc1ae..2b72e560e 100644 --- a/src/commands/interaction/runtime/selector-wait.ts +++ b/src/commands/interaction/runtime/selector-wait.ts @@ -118,6 +118,11 @@ export type WaitForTextCommandOptions = WaitCommandContext & type SelectorWaitRuntime = { backend: { platform: PublicPlatform; + /** + * The backend's native text reading, present only when the bound runtime advertised the + * conditional `findText` operation. It is consulted first and is authoritative ONLY when it + * answers `true`; see `waitForText`. + */ findText?: (context: WaitCommandContext, text: string) => Promise<{ found: boolean }>; }; clock?: { @@ -364,11 +369,8 @@ async function waitForText( const polling = createWaitPolling(runtime, options, timeoutMs, SELECTOR_PIPELINE_POLICIES.wait); let deadline: WaitPollDeadline | undefined; while (polling.hasTimeRemaining()) { - const poll = await polling.capture(async (signal) => - runtime.backend.findText - ? (await runtime.backend.findText(backendContext(runtime, { ...options, signal }), text)) - .found - : await snapshotContainsText(operations, runtime, { ...options, signal }, text), + const poll = await polling.capture( + async (signal) => await observeText(operations, runtime, { ...options, signal }, text), ); if (poll.timedOut) { deadline = poll.deadline; @@ -381,6 +383,32 @@ async function waitForText( throw waitTimeoutError(`wait timed out for text: ${text}`, polling, deadline); } +/** + * One poll's answer to "is this text on screen", from two sources with deliberately asymmetric + * authority: + * + * 1. the owner's native reading, when the bound runtime advertised it — a `true` here short- + * circuits the poll and skips the capture entirely, which is the whole benefit of the + * preferred operation; and + * 2. the canonical tree, always consulted when (1) did not answer `true`. + * + * So the fast path can only ever make a satisfied wait return sooner. It can never make a wait + * that the tree would satisfy fail, and it is never the reason a wait times out — the required + * tree path below it is complete on its own. + */ +async function observeText( + operations: SelectorWaitOperations, + runtime: Runtime, + options: WaitCommandOptions, + text: string, +): Promise { + if (runtime.backend.findText) { + const native = await runtime.backend.findText(backendContext(runtime, options), text); + if (native.found) return true; + } + return await snapshotContainsText(operations, runtime, options, text); +} + async function snapshotContainsText( operations: SelectorWaitOperations, runtime: Runtime, diff --git a/src/core/__tests__/capability-plugin-routing-parity.test.ts b/src/core/__tests__/capability-plugin-routing-parity.test.ts index 3abb51df8..db06ff044 100644 --- a/src/core/__tests__/capability-plugin-routing-parity.test.ts +++ b/src/core/__tests__/capability-plugin-routing-parity.test.ts @@ -277,7 +277,6 @@ test('HarmonyOS static capabilities omit runtime-backed command admissions', () 'settings', 'swipe', 'type', - 'wait', ]); }); diff --git a/src/core/capabilities.ts b/src/core/capabilities.ts index 2a5e4c2aa..bff7640b8 100644 --- a/src/core/capabilities.ts +++ b/src/core/capabilities.ts @@ -53,9 +53,8 @@ const HARMONYOS_SUPPORTED_COMMANDS = new Set([ 'settings', 'swipe', 'type', - 'wait', ]); -const WEB_QUERY_COMMANDS = ['audio', 'find', 'wait'] as const; +const WEB_QUERY_COMMANDS = ['audio', 'find'] as const; const WEB_INTERACTION_COMMANDS = [ 'click', 'fill', diff --git a/src/core/command-descriptor/__tests__/parity.test.ts b/src/core/command-descriptor/__tests__/parity.test.ts index 1f5c32863..b23e8738c 100644 --- a/src/core/command-descriptor/__tests__/parity.test.ts +++ b/src/core/command-descriptor/__tests__/parity.test.ts @@ -75,6 +75,7 @@ const NO_CAPABILITY_PUBLIC_COMMANDS = new Set([ PUBLIC_COMMANDS.test, PUBLIC_COMMANDS.trace, PUBLIC_COMMANDS.viewport, + PUBLIC_COMMANDS.wait, ]); type TestCommandDescriptor = (typeof commandDescriptors)[number]; diff --git a/src/core/command-descriptor/__tests__/viewport-runtime-execution.test.ts b/src/core/command-descriptor/__tests__/viewport-runtime-execution.test.ts index 425441561..f70e0eeb9 100644 --- a/src/core/command-descriptor/__tests__/viewport-runtime-execution.test.ts +++ b/src/core/command-descriptor/__tests__/viewport-runtime-execution.test.ts @@ -11,5 +11,8 @@ test('viewport descriptor declares its complete runtime use with no legacy proje kind: 'device-runtime', uses: [viewportRuntimeUse], }); - expect(viewportRuntimeUse).toEqual({ required: ['setViewport'], preferred: [] }); + expect(viewportRuntimeUse).toEqual({ + required: ['setViewport'], + preferred: [], + }); }); diff --git a/src/core/command-descriptor/__tests__/wait-runtime-execution.test.ts b/src/core/command-descriptor/__tests__/wait-runtime-execution.test.ts new file mode 100644 index 000000000..8dadc591c --- /dev/null +++ b/src/core/command-descriptor/__tests__/wait-runtime-execution.test.ts @@ -0,0 +1,52 @@ +import { expect, test } from 'vitest'; +import { + waitSelectorCaptureRuntimePlanUses, + waitObservesDevice, +} from '@agent-device/contracts/platform'; +import { commandDescriptors } from '../registry.ts'; + +test('wait descriptor declares its complete runtime use with no capability bucket', () => { + const wait = commandDescriptors.find(({ name }) => name === 'wait'); + + expect(wait).not.toHaveProperty('capability'); + expect(wait).not.toHaveProperty('dispatch'); + expect(wait?.platformExecution).toEqual({ + kind: 'device-runtime', + uses: waitSelectorCaptureRuntimePlanUses, + }); + // wait binds the family plan; its measured reading rides that plan's preferred set. + expect(waitSelectorCaptureRuntimePlanUses).toEqual([ + { + required: ['captureSnapshot'], + preferred: [], + conditional: ['findText', 'findSelector'], + }, + { + required: ['captureSnapshot', 'captureSnapshotWithoutActiveApp'], + preferred: [], + conditional: ['findText', 'findSelector'], + }, + ]); +}); + +test('only the duration shape reaches no device at all', () => { + expect(waitObservesDevice('sleep')).toBe(false); + for (const target of ['text', 'ref', 'selector', 'stable'] as const) { + expect(waitObservesDevice(target)).toBe(true); + } +}); + +// Native observations are correctness-bearing where owner facts advertise them. They cannot be +// preferred optimizations, while making them unconditionally required would reject owners whose +// complete semantic path is capture-backed. +test('wait declares native observations as fact-conditional operations, never optimizations', () => { + for (const use of waitSelectorCaptureRuntimePlanUses) { + expect(use.conditional).toContain('findText'); + expect(use.required).not.toContain('findText'); + expect(use.preferred).not.toContain('findText'); + expect(use.conditional).toContain('findSelector'); + expect(use.required).not.toContain('findSelector'); + expect(use.preferred).not.toContain('findSelector'); + expect(use.required).toContain('captureSnapshot'); + } +}); diff --git a/src/core/command-descriptor/registry.ts b/src/core/command-descriptor/registry.ts index cf79556c7..55b049d80 100644 --- a/src/core/command-descriptor/registry.ts +++ b/src/core/command-descriptor/registry.ts @@ -27,6 +27,8 @@ import { openApplicationRuntimePlanUses, closeApplicationRuntimePlanUses, selectorCaptureRuntimePlanUses, + selectorTextCaptureRuntimePlanUses, + waitSelectorCaptureRuntimePlanUses, snapshotRuntimePlanUses, prepareAppleRunnerRuntimeUse, runtimeCommandRuntimePlanUses, @@ -960,7 +962,6 @@ export const RAW_COMMAND_DESCRIPTORS = [ // starts, so identity verification runs inside its polling resolution. targetIdentityVerification: 'post-resolution', daemon: { route: 'snapshot', refFrameEffect: 'preserve' }, - capability: ALL_DEVICE_COMMAND_CAPABILITY, // The wait budget travels as a positional, not a flag; parse it the same // way the daemon will so the request envelope extends past it (#1075). timeoutPolicy: { @@ -968,7 +969,20 @@ export const RAW_COMMAND_DESCRIPTORS = [ budget: { source: 'positional-parser', parser: resolveWaitBudgetMs }, }, batchable: true, - platformExecution: LEGACY_PLATFORM_EXECUTION, + // A duration wait declares no operation and never binds (`waitObservesDevice`); every + // observing shape uses the wait-specific capture plan so its conditional native observations + // cannot affect capture-only or element-text selector commands. + // + // ACCEPTED BEHAVIOUR CHANGE (#1875, ruled rather than assumed): binding the family plan means + // wait asks the owner whether it can observe a device with no app attached, and the families + // answer differently. On iOS `appBundleId` is the XCUITest attach target, so with none set + // local Apple refuses `captureSnapshotWithoutActiveApp` and wait now fails immediately naming + // `open`, where it used to poll to its deadline. It could never have succeeded: the runner's + // own process foregrounds and displaces the app under test, then answers about its own blank + // screen — so `wait stable` and `wait @ref` stopped returning a success that was describing + // the runner, not the app. Android captures the real launcher in that state, its facts say + // so, and wait proceeds unchanged. Same plan, opposite outcomes, chosen by the owner. + platformExecution: { kind: 'device-runtime', uses: waitSelectorCaptureRuntimePlanUses }, }, { name: 'alert', @@ -1191,7 +1205,7 @@ export const RAW_COMMAND_DESCRIPTORS = [ daemon: { route: 'interaction', refFrameEffect: 'preserve' }, timeoutPolicy: postActionObservationTimeoutPolicy('get', PRESERVE_DAEMON_TIMEOUT_POLICY), batchable: true, - platformExecution: { kind: 'device-runtime', uses: selectorCaptureRuntimePlanUses }, + platformExecution: { kind: 'device-runtime', uses: selectorTextCaptureRuntimePlanUses }, }, { name: 'is', diff --git a/src/daemon/__tests__/device-claim-admission.test.ts b/src/daemon/__tests__/device-claim-admission.test.ts index d39d0b9fe..568cb719c 100644 --- a/src/daemon/__tests__/device-claim-admission.test.ts +++ b/src/daemon/__tests__/device-claim-admission.test.ts @@ -133,7 +133,10 @@ async function claimsWhileBound(command: string, stateDir: string) { }, }); try { - await scope.bindDevice(ANDROID_EMULATOR, { required: [], preferred: [] }); + await scope.bindDevice(ANDROID_EMULATOR, { + required: [], + preferred: [], + }); return claimedSessions(); } finally { await scope[Symbol.asyncDispose](); diff --git a/src/daemon/__tests__/selector-capture-binding.test.ts b/src/daemon/__tests__/selector-capture-binding.test.ts index 4aa78fb79..1d9412675 100644 --- a/src/daemon/__tests__/selector-capture-binding.test.ts +++ b/src/daemon/__tests__/selector-capture-binding.test.ts @@ -64,6 +64,38 @@ test('an unavailable required operation refuses before any bind', async () => { expect(fixture.binds).toEqual([]); }); +test.each(['find', 'get', 'is'] as const)( + '%s ignores an advertised wait observation it does not execute', + async (command) => { + const fixture = selectorCaptureFixture({ findText: { available: true } }); + + const bound = await resolveBoundSelectorCapture({ + command, + device: ANDROID_EMULATOR, + session: makeAndroidSession('selector'), + inspectFacts: fixture.inspectFacts, + bindDevice: fixture.bindDevice, + }); + + expect(bound.ok).toBe(true); + expect(fixture.binds).toEqual([ANDROID_EMULATOR]); + }, +); + +test('wait rejects an advertised observation with no implementation', async () => { + const fixture = selectorCaptureFixture({ findText: { available: true } }); + + await expect( + resolveBoundSelectorCapture({ + command: 'wait', + device: ANDROID_EMULATOR, + session: makeAndroidSession('selector'), + inspectFacts: fixture.inspectFacts, + bindDevice: fixture.bindDevice, + }), + ).rejects.toMatchObject({ details: { reason: 'runtime-contract-invalid' } }); +}); + // The active-app split is the only plan axis selector commands have: no `--actions` surface, // so `captureSnapshotWithCustomActions` is never required and never admitted for them. test('a session without a tracked app selects the without-active-app plan', async () => { diff --git a/src/daemon/__tests__/selector-capture-fixture.ts b/src/daemon/__tests__/selector-capture-fixture.ts index 2f974d5c3..71fe9037f 100644 --- a/src/daemon/__tests__/selector-capture-fixture.ts +++ b/src/daemon/__tests__/selector-capture-fixture.ts @@ -27,6 +27,7 @@ export function selectorCaptureFixture( params: Readonly<{ capture?: RuntimeOperationFact; withoutActiveApp?: RuntimeOperationFact; + findText?: RuntimeOperationFact; snapshot?: (input: CaptureSnapshotInput, index: number) => SnapshotResult; }> = {}, ): Readonly<{ @@ -54,6 +55,7 @@ export function selectorCaptureFixture( customActions: { available: false, reason: 'unsupported-platform-leaf' }, withoutActiveApp: params.withoutActiveApp ?? params.capture ?? available, }), + ...(params.findText ? { findText: params.findText } : {}), }, }; }; diff --git a/src/daemon/__tests__/selector-operation-binding.test.ts b/src/daemon/__tests__/selector-operation-binding.test.ts new file mode 100644 index 000000000..b5bfc03f2 --- /dev/null +++ b/src/daemon/__tests__/selector-operation-binding.test.ts @@ -0,0 +1,28 @@ +import { expect, test, vi } from 'vitest'; +import { + selectElementTextOperation, + selectWaitObservationOperations, +} from '../selector-operation-binding.ts'; + +test('selector operations are projected by presence and retain their exact bindings', async () => { + const findText = vi.fn(async () => ({ found: true })); + const findSelector = vi.fn(async () => ({ found: true })); + const selected = selectWaitObservationOperations({ operations: { findText, findSelector } }); + + await expect(selected.findText?.({ text: 'Ready' })).resolves.toEqual({ found: true }); + await expect( + selected.findSelector?.({ selector: { key: 'id', value: 'ready' } }), + ).resolves.toEqual({ found: true }); + expect(findText).toHaveBeenCalledOnce(); + expect(findSelector).toHaveBeenCalledOnce(); +}); + +test('selects the admitted element-text operation', async () => { + const readTextAtPoint = vi.fn(async () => ({ status: 'read' as const, text: 'Ready' })); + const selected = selectElementTextOperation({ operations: { readTextAtPoint } }); + + await expect(selected.readTextAtPoint?.({ point: { x: 1, y: 2 } })).resolves.toMatchObject({ + text: 'Ready', + }); + expect(readTextAtPoint).toHaveBeenCalledOnce(); +}); diff --git a/src/daemon/__tests__/wait-conditional-selector.test.ts b/src/daemon/__tests__/wait-conditional-selector.test.ts new file mode 100644 index 000000000..a007e1117 --- /dev/null +++ b/src/daemon/__tests__/wait-conditional-selector.test.ts @@ -0,0 +1,165 @@ +import { expect, test, vi } from 'vitest'; +import { + localRuntimeOwner, + narrowDeviceBinding, + snapshotRuntimeOperationFacts, + type DeviceBinding, + type FindSelectorInput, + type PlatformRuntimeOperations, + type RuntimeFacts, + type RuntimeOperationFact, + waitSelectorCaptureRuntimePlanUses, +} from '@agent-device/contracts/platform'; +import { deviceShape } from '@agent-device/kernel/device'; +import { IOS_SIMULATOR } from '../../__tests__/test-utils/device-fixtures.ts'; +import { + makeAuthoringSession, + makeIosAppSession, + makeIosSession, +} from '../../__tests__/test-utils/session-factories.ts'; +import { makeSessionStore } from '../../__tests__/test-utils/store-factory.ts'; +import { unavailableDeploymentSnapshotAndShutdownOperationFacts } from '../../__tests__/test-utils/runtime-operation-facts.ts'; +import { handleSnapshotCommands } from '../handlers/snapshot.ts'; +import type { BindDeviceRuntime, InspectDeviceRuntimeFacts } from '../request-runtime-binding.ts'; +import type { DaemonRequest, SessionState } from '../types.ts'; + +const available = Object.freeze({ available: true } as const); +const unavailable = Object.freeze({ + available: false, + reason: 'unsupported-platform-leaf' as const, + hint: 'No native selector observation.', +}); + +function harness(options: { + fact?: RuntimeOperationFact; + found?: boolean; + nodes?: Array<{ index: number; depth: number; type: string; identifier?: string }>; +}) { + const findSelector = vi.fn(async (_input: FindSelectorInput) => ({ + found: options.found === true, + })); + const captureSnapshot = vi.fn(async () => ({ + nodes: options.nodes ?? [], + backend: 'xctest' as const, + })); + const fact = options.fact ?? available; + const facts: RuntimeFacts = { + device: { ...deviceShape(IOS_SIMULATOR), providerMode: 'local' }, + operations: { + ...unavailableDeploymentSnapshotAndShutdownOperationFacts, + ...snapshotRuntimeOperationFacts({ + capture: available, + customActions: unavailable, + withoutActiveApp: available, + }), + findSelector: fact, + } as RuntimeFacts['operations'], + }; + const binding = { + device: IOS_SIMULATOR, + owner: localRuntimeOwner('apple'), + facts, + operations: { + captureSnapshot, + captureSnapshotWithoutActiveApp: captureSnapshot, + ...(fact.available ? { findSelector } : {}), + }, + [Symbol.asyncDispose]: async () => undefined, + } as unknown as DeviceBinding; + const inspectFacts: InspectDeviceRuntimeFacts = vi.fn(async () => facts); + const bindDevice = vi.fn(async (_device, use) => + narrowDeviceBinding(binding, use), + ) as unknown as BindDeviceRuntime; + return { findSelector, captureSnapshot, inspectFacts, bindDevice }; +} + +async function run(session: SessionState, runtime: ReturnType) { + const sessionStore = makeSessionStore('wait-conditional-selector-'); + sessionStore.set(session.name, session); + const req = { + command: 'wait', + positionals: ['id=runner-only', '200'], + token: 'token', + session: session.name, + flags: {}, + meta: { requestId: 'wait-conditional-selector' }, + } as unknown as DaemonRequest; + const response = await handleSnapshotCommands({ + req, + sessionName: session.name, + logPath: '/tmp/wait-conditional-selector.log', + sessionStore, + inspectFacts: runtime.inspectFacts, + bindDevice: runtime.bindDevice, + }); + if (!response) throw new Error('wait route did not handle the request'); + return response; +} + +test('an admitted positive native selector observation satisfies wait without a sparse tree', async () => { + const runtime = harness({ found: true, nodes: [] }); + const response = await run(makeIosAppSession('wait-conditional-selector'), runtime); + + expect(response).toMatchObject({ + ok: true, + data: { kind: 'selector', selector: 'id=runner-only' }, + }); + expect(runtime.inspectFacts).toHaveBeenCalledTimes(1); + expect(runtime.bindDevice).toHaveBeenCalledTimes(1); + expect(runtime.bindDevice).toHaveBeenCalledWith( + IOS_SIMULATOR, + waitSelectorCaptureRuntimePlanUses[0], + ); + expect(runtime.findSelector).toHaveBeenCalledOnce(); + expect(runtime.findSelector.mock.calls[0]?.[0]).toMatchObject({ + selector: { key: 'id', value: 'runner-only' }, + options: { appBundleId: 'com.example.app' }, + execution: { requestId: 'wait-conditional-selector' }, + }); + expect(runtime.captureSnapshot).not.toHaveBeenCalled(); +}); + +test('a negative native observation falls through to the same bound canonical capture', async () => { + const runtime = harness({ + found: false, + nodes: [{ index: 0, depth: 0, type: 'Button', identifier: 'runner-only' }], + }); + const response = await run(makeIosAppSession('wait-conditional-selector'), runtime); + + expect(response.ok).toBe(true); + expect(runtime.findSelector).toHaveBeenCalledOnce(); + expect(runtime.captureSnapshot).toHaveBeenCalled(); + expect(runtime.bindDevice).toHaveBeenCalledTimes(1); +}); + +test('an unavailable conditional observation preserves the capture-backed owner path', async () => { + const runtime = harness({ + fact: unavailable, + nodes: [{ index: 0, depth: 0, type: 'Button', identifier: 'runner-only' }], + }); + const response = await run(makeIosAppSession('wait-conditional-selector'), runtime); + + expect(response.ok).toBe(true); + expect(runtime.findSelector).not.toHaveBeenCalled(); + expect(runtime.captureSnapshot).toHaveBeenCalled(); + expect(runtime.inspectFacts).toHaveBeenCalledTimes(1); + expect(runtime.bindDevice).toHaveBeenCalledTimes(1); +}); + +test('recording and no-app waits retain capture-owned evidence semantics', async () => { + for (const session of [ + makeAuthoringSession('wait-conditional-selector', { + appBundleId: 'com.example.app', + }), + makeIosSession('wait-conditional-selector'), + ]) { + const runtime = harness({ + found: true, + nodes: [{ index: 0, depth: 0, type: 'Button', identifier: 'runner-only' }], + }); + const response = await run(session, runtime); + expect(response.ok).toBe(true); + expect(runtime.findSelector).not.toHaveBeenCalled(); + expect(runtime.captureSnapshot).toHaveBeenCalled(); + } +}); diff --git a/src/daemon/__tests__/wait-runtime.test.ts b/src/daemon/__tests__/wait-runtime.test.ts new file mode 100644 index 000000000..84e179e94 --- /dev/null +++ b/src/daemon/__tests__/wait-runtime.test.ts @@ -0,0 +1,535 @@ +import { expect, test, vi } from 'vitest'; +import { WAIT_REASONS } from '@agent-device/contracts/interaction'; +import { + localRuntimeOwner, + narrowDeviceBinding, + providerRuntimeOwner, + snapshotRuntimeOperationFacts, + type CaptureSnapshotInput, + type DeviceBinding, + type PlatformRuntimeOperations, + type RuntimeFacts, + type RuntimeOperationFact, + waitSelectorCaptureRuntimePlanUses, + type FindTextInput, + type SnapshotResult, +} from '@agent-device/contracts/platform'; +import { deviceShape, type DeviceInfo } from '@agent-device/kernel/device'; +import { makeSession } from '../../__tests__/test-utils/session-factories.ts'; +import { makeSessionStore } from '../../__tests__/test-utils/store-factory.ts'; +import { unavailableDeploymentSnapshotAndShutdownOperationFacts } from '../../__tests__/test-utils/runtime-operation-facts.ts'; +import type { BindDeviceRuntime, InspectDeviceRuntimeFacts } from '../request-runtime-binding.ts'; +import { handleSnapshotCommands } from '../handlers/snapshot.ts'; +import { resolveBoundSelectorCapture } from '../selector-capture-binding.ts'; +import type { DaemonRequest } from '../types.ts'; + +const webDevice = { + id: 'web', + name: 'Web', + platform: 'web', + kind: 'device', + booted: true, +} as const satisfies DeviceInfo; + +const available = Object.freeze({ available: true } as const); +/** The harness session tracks no app bundle id, so wait's plan is the without-active-app one. */ +const waitWithoutActiveAppUse = waitSelectorCaptureRuntimePlanUses[1]; + +const findTextUnavailable = Object.freeze({ + available: false, + reason: 'unsupported-platform-leaf' as const, + hint: 'This target exposes no native text reading.', +}); +const captureUnavailable = Object.freeze({ + available: false, + reason: 'unsupported-platform-leaf' as const, + hint: 'wait observes a snapshot; this target exposes none.', +}); + +type CaptureNode = { + index: number; + depth: number; + type: string; + label?: string; + hittable?: boolean; +}; + +/** + * Binds the fake at the seam the handler consumes — `inspectFacts` / `bindDevice` — never at + * `core/dispatch.ts`. `captureSnapshot` is the ONE operation `wait` declares, so this harness is + * also the proof that no sibling snapshot operation is reachable from wait's narrowed binding. + */ +function waitRuntimeHarness( + options: { + capture?: RuntimeOperationFact; + device?: DeviceInfo; + providerOwned?: boolean; + nodesPerPoll?: CaptureNode[][]; + /** The owner's native text reading; absent means the owner advertises none. */ + findText?: RuntimeOperationFact; + findTextAnswers?: (text: string) => boolean; + /** A scenario-supplied capture operation, for poll-deadline behaviour. */ + captureSnapshot?: (input: CaptureSnapshotInput) => Promise; + } = {}, +) { + const device = options.device ?? webDevice; + const capture = options.capture ?? available; + const polls = options.nodesPerPoll ?? [ + [{ index: 0, depth: 0, type: 'Button', label: 'Ready', hittable: true }], + ]; + let pollIndex = 0; + + const captureSnapshot = vi.fn( + async (input: CaptureSnapshotInput): Promise => + options.captureSnapshot + ? await options.captureSnapshot(input) + : { + nodes: polls[Math.min(pollIndex++, polls.length - 1)] ?? [], + backend: 'web', + }, + ); + const findTextFact = options.findText ?? findTextUnavailable; + const findText = vi.fn(async (input: FindTextInput) => ({ + found: options.findTextAnswers?.(input.text) ?? false, + })); + const facts: RuntimeFacts = { + device: { + ...deviceShape(device), + providerMode: options.providerOwned ? 'provider-runtime' : 'local', + }, + operations: { + ...unavailableDeploymentSnapshotAndShutdownOperationFacts, + ...snapshotRuntimeOperationFacts({ + capture, + customActions: captureUnavailable, + withoutActiveApp: capture, + }), + findText: findTextFact, + } as RuntimeFacts['operations'], + }; + const binding = { + device, + owner: options.providerOwned + ? providerRuntimeOwner('test', 'wait-runtime-test') + : localRuntimeOwner(device.platform), + facts, + operations: { + captureSnapshot, + captureSnapshotWithCustomActions: captureSnapshot, + captureSnapshotWithoutActiveApp: captureSnapshot, + // Only advertised operations get an implementation, so an owner whose facts refuse + // `findText` cannot expose one through the narrowed projection either. + ...(findTextFact.available ? { findText } : {}), + }, + [Symbol.asyncDispose]: async () => {}, + } as unknown as DeviceBinding; + + const inspectFacts: InspectDeviceRuntimeFacts = vi.fn(async () => facts); + const bindDevice = vi.fn(async (_device, use) => + narrowDeviceBinding(binding, use), + ) as unknown as BindDeviceRuntime; + return { device, captureSnapshot, findText, inspectFacts, bindDevice, facts }; +} + +function waitRequest(positionals: string[], flags: Record = {}): DaemonRequest { + return { + command: 'wait', + positionals, + token: 't', + session: 'wait-runtime', + flags, + meta: { requestId: 'wait-runtime-test' }, + } as unknown as DaemonRequest; +} + +async function runWait( + positionals: string[], + harness: ReturnType, + sessionNodes: CaptureNode[] = [], + flags: Record = {}, +) { + const sessionStore = makeSessionStore('agent-device-wait-runtime-'); + const session = makeSession('wait-runtime', { device: harness.device }); + if (sessionNodes.length > 0) { + session.snapshot = { + nodes: sessionNodes.map((node) => ({ ...node, ref: `e${node.index + 1}` })), + createdAt: Date.now(), + backend: 'web', + } as unknown as NonNullable; + } + sessionStore.set(session.name, session); + const response = await handleSnapshotCommands({ + req: waitRequest(positionals, flags), + sessionName: session.name, + logPath: '/tmp/daemon.log', + sessionStore, + inspectFacts: harness.inspectFacts, + bindDevice: harness.bindDevice, + }); + if (!response) throw new Error('the snapshot route did not handle wait'); + return { response, session, sessionStore }; +} + +// --------------------------------------------------------------------------- +// The duration shape. It is the one wait shape that reaches no device at all, +// and legacy admission skipped `requireCommandSupported` for exactly that +// reason. `resolveWaitRuntimePlan` returns a plan with no `use`, so there is +// nothing to inspect and nothing to bind — including on a target whose capture +// fact is unavailable. +// --------------------------------------------------------------------------- + +test('a duration wait inspects no facts and binds no device', async () => { + const harness = waitRuntimeHarness({ capture: captureUnavailable }); + + const { response } = await runWait(['5'], harness); + + expect(response).toMatchObject({ ok: true, data: { waitedMs: 5 } }); + expect(harness.inspectFacts).not.toHaveBeenCalled(); + expect(harness.bindDevice).not.toHaveBeenCalled(); + expect(harness.captureSnapshot).not.toHaveBeenCalled(); +}); + +test('an observing wait binds through the shared selector entry', async () => { + const harness = waitRuntimeHarness({ findText: available }); + + const bound = await resolveBoundSelectorCapture({ + command: 'wait', + session: undefined, + device: harness.device, + inspectFacts: harness.inspectFacts, + bindDevice: harness.bindDevice, + }); + + expect(bound.ok).toBe(true); + if (!bound.ok) return; + // Wait reaches the platform through the same owning interface as every other selector + // command; its preferred reading rides that record rather than a parallel binder. + expect(bound.operations.capture).toBeTypeOf('function'); + expect(bound.operations.findText).toBeTypeOf('function'); + expect(harness.inspectFacts).toHaveBeenCalledTimes(1); + expect(harness.bindDevice).toHaveBeenCalledTimes(1); +}); + +// --------------------------------------------------------------------------- +// Every observing shape: one inspection, one bind, one declared operation. +// --------------------------------------------------------------------------- + +test('a text wait binds the capture use once and polls through the bound operation', async () => { + const harness = waitRuntimeHarness(); + + const { response } = await runWait(['text', 'Ready'], harness); + + expect(response).toMatchObject({ ok: true, data: { text: 'Ready' } }); + expect(harness.inspectFacts).toHaveBeenCalledTimes(1); + expect(harness.inspectFacts).toHaveBeenCalledWith(harness.device); + expect(harness.bindDevice).toHaveBeenCalledTimes(1); + expect(harness.bindDevice).toHaveBeenCalledWith(harness.device, waitWithoutActiveAppUse); + expect(harness.captureSnapshot).toHaveBeenCalled(); +}); + +test('a selector wait binds the capture use once and polls through the bound operation', async () => { + const harness = waitRuntimeHarness(); + + const { response } = await runWait(['label=Ready'], harness); + + expect(response).toMatchObject({ ok: true, data: { selector: 'label=Ready' } }); + expect(harness.inspectFacts).toHaveBeenCalledTimes(1); + expect(harness.bindDevice).toHaveBeenCalledTimes(1); + expect(harness.bindDevice).toHaveBeenCalledWith(harness.device, waitWithoutActiveAppUse); +}); + +test('a @ref wait binds the capture use once and polls through the bound operation', async () => { + const harness = waitRuntimeHarness(); + + const { response } = await runWait(['@e1'], harness, [ + { index: 0, depth: 0, type: 'Button', label: 'Ready', hittable: true }, + ]); + + expect(response).toMatchObject({ ok: true, data: { text: 'Ready' } }); + expect(harness.inspectFacts).toHaveBeenCalledTimes(1); + expect(harness.bindDevice).toHaveBeenCalledTimes(1); + expect(harness.bindDevice).toHaveBeenCalledWith(harness.device, waitWithoutActiveAppUse); +}); + +test('a stable wait binds the capture use once and polls through the bound operation', async () => { + const harness = waitRuntimeHarness(); + + const { response } = await runWait(['stable', '1', '5000'], harness); + + expect(response).toMatchObject({ ok: true, data: { captures: expect.any(Number) } }); + expect(harness.inspectFacts).toHaveBeenCalledTimes(1); + expect(harness.bindDevice).toHaveBeenCalledTimes(1); + expect(harness.bindDevice).toHaveBeenCalledWith(harness.device, waitWithoutActiveAppUse); +}); + +// --------------------------------------------------------------------------- +// Facts are the only support authority: an unavailable exact-owner fact +// refuses BEFORE any binding, and provider ownership never borrows the local +// family runtime. +// --------------------------------------------------------------------------- + +test('an unavailable capture fact refuses an observing wait before binding', async () => { + const harness = waitRuntimeHarness({ capture: captureUnavailable }); + + const { response } = await runWait(['text', 'Ready'], harness); + + expect(response).toEqual({ + ok: false, + error: { + code: 'UNSUPPORTED_OPERATION', + message: 'wait is not supported on this device', + details: { reason: captureUnavailable.reason }, + hint: captureUnavailable.hint, + }, + }); + expect(harness.inspectFacts).toHaveBeenCalledTimes(1); + expect(harness.bindDevice).not.toHaveBeenCalled(); + expect(harness.captureSnapshot).not.toHaveBeenCalled(); +}); + +test('a provider owner that cannot capture fails closed instead of borrowing the local runtime', async () => { + const harness = waitRuntimeHarness({ capture: captureUnavailable, providerOwned: true }); + + const { response } = await runWait(['label=Ready'], harness); + + expect(response).toMatchObject({ + ok: false, + error: { code: 'UNSUPPORTED_OPERATION' }, + }); + expect(harness.bindDevice).not.toHaveBeenCalled(); +}); + +// --------------------------------------------------------------------------- +// The timeout-surface decoration is wait's platform execution too. It reuses +// the SAME binding rather than reaching a second capture owner. +// --------------------------------------------------------------------------- + +test('a timed-out wait decorates its failure through the same single binding', async () => { + const harness = waitRuntimeHarness({ + nodesPerPoll: [[{ index: 0, depth: 0, type: 'Button', label: 'Checkout', hittable: true }]], + }); + + const { response } = await runWait(['text', 'Ready', '1'], harness); + + expect(response.ok).toBe(false); + if (response.ok) return; + expect(response.error.message).toContain('wait timed out for text: Ready'); + expect(response.error.message).toContain('Current surface: Checkout'); + expect(harness.inspectFacts).toHaveBeenCalledTimes(1); + expect(harness.bindDevice).toHaveBeenCalledTimes(1); +}); + +// --------------------------------------------------------------------------- +// The fact-conditional native observation (ADR 0019 §2). These are the regressions +// for the iOS Smoke failure this unit's first revision caused: +// `wait text "Last input: press"` +// performed 17 readable canonical-tree captures, never observed the target, +// and timed out with `wait_target_absent`. The target is reported by the +// owner's native reading and is absent from the canonical tree, so a +// tree-only poll cannot satisfy it — which is exactly what these assert. +// --------------------------------------------------------------------------- + +test('a text wait is satisfied by the owner native reading when the tree never carries it', async () => { + // The target appears on the owner's native reading from the second poll onward and NEVER + // appears in the canonical tree, which keeps reporting a different screen — the divergence + // iOS Smoke hit. The first poll therefore proves the tree cannot satisfy this wait, and the + // second proves the native reading can. + let nativeReads = 0; + const harness = waitRuntimeHarness({ + findText: available, + findTextAnswers: (text) => { + nativeReads += 1; + return text === 'Last input: press' && nativeReads > 1; + }, + nodesPerPoll: [[{ index: 0, depth: 0, type: 'Button', label: 'Checkout', hittable: true }]], + }); + + const { response } = await runWait(['text', 'Last input: press', '2000'], harness); + + expect(response).toMatchObject({ ok: true, data: { text: 'Last input: press' } }); + expect(harness.findText.mock.calls[0]?.[0]).toMatchObject({ text: 'Last input: press' }); + // Not vacuous in either direction: the tree WAS consulted and could not satisfy the target, + // so the wait is satisfied only because the native reading answered. Delete the `findText` + // arm from `observeText` and this times out with `wait_target_absent` after burning the whole + // budget on readable captures — the exact shape iOS Smoke reported. + expect(harness.captureSnapshot).toHaveBeenCalled(); +}); + +test('a satisfied native reading short-circuits the poll without capturing', async () => { + const harness = waitRuntimeHarness({ + findText: available, + findTextAnswers: () => true, + }); + + const { response } = await runWait(['text', 'Ready', '2000'], harness); + + expect(response).toMatchObject({ ok: true, data: { text: 'Ready' } }); + expect(harness.findText).toHaveBeenCalledTimes(1); + // The benefit the §9 measurement records: a satisfied wait costs one native query, not a + // full accessibility capture. + expect(harness.captureSnapshot).not.toHaveBeenCalled(); +}); + +test('a negative native reading is not an answer — the same poll consults the canonical tree', async () => { + const harness = waitRuntimeHarness({ + findText: available, + findTextAnswers: () => false, + nodesPerPoll: [[{ index: 0, depth: 0, type: 'Button', label: 'Ready', hittable: true }]], + }); + + const { response } = await runWait(['text', 'Ready', '2000'], harness); + + expect(response).toMatchObject({ ok: true, data: { text: 'Ready' } }); + expect(harness.findText).toHaveBeenCalled(); + // A negative native observation never replaces the capture-backed half of the same poll. + expect(harness.captureSnapshot).toHaveBeenCalled(); +}); + +test('an owner that advertises no native reading polls the tree only', async () => { + const harness = waitRuntimeHarness({ + nodesPerPoll: [[{ index: 0, depth: 0, type: 'Button', label: 'Ready', hittable: true }]], + }); + + const { response } = await runWait(['text', 'Ready', '2000'], harness); + + expect(response).toMatchObject({ ok: true, data: { text: 'Ready' } }); + expect(harness.findText).not.toHaveBeenCalled(); + expect(harness.captureSnapshot).toHaveBeenCalled(); + expect(harness.bindDevice).toHaveBeenCalledTimes(1); +}); + +test('an unavailable conditional observation preserves the capture-backed owner path', async () => { + const harness = waitRuntimeHarness({ + findText: findTextUnavailable, + nodesPerPoll: [[{ index: 0, depth: 0, type: 'Button', label: 'Ready', hittable: true }]], + }); + + const { response } = await runWait(['text', 'Ready', '2000'], harness); + + expect(response.ok).toBe(true); + expect(harness.inspectFacts).toHaveBeenCalledTimes(1); + expect(harness.bindDevice).toHaveBeenCalledTimes(1); +}); + +// --------------------------------------------------------------------------- +// The conditional reading and the capture it short-circuits must run in the SAME +// runner context. Asserted as an equality against capture's own context rather +// than field by field, so the check cannot rot into a partial one when a new +// execution field is added: whatever capture carries, findText must carry too. +// --------------------------------------------------------------------------- + +test('the native text reading runs in the same runner context as the capture', async () => { + let nativeReads = 0; + const harness = waitRuntimeHarness({ + findText: available, + // Miss on the first poll so BOTH legs run and their contexts can be compared. + findTextAnswers: () => { + nativeReads += 1; + return nativeReads > 1; + }, + nodesPerPoll: [[{ index: 0, depth: 0, type: 'Button', label: 'Checkout', hittable: true }]], + }); + + const { response } = await runWait(['text', 'Ready', '4000'], harness, [], { + debug: true, + iosXctestrunFile: '/tmp/override.xctestrun', + iosXctestDerivedDataPath: '/tmp/derived', + }); + + expect(response.ok).toBe(true); + const captureExecution = harness.captureSnapshot.mock.calls[0]?.[0]?.execution; + const findTextExecution = harness.findText.mock.calls[0]?.[0]?.execution; + // Non-trivially populated: a comparison of two undefineds would prove nothing. + expect(captureExecution).toMatchObject({ + requestId: 'wait-runtime-test', + iosXctestrunFile: '/tmp/override.xctestrun', + iosXctestDerivedDataPath: '/tmp/derived', + }); + expect(findTextExecution).toEqual(captureExecution); +}); + +// --------------------------------------------------------------------------- +// Per-poll cancellation, end to end (`CaptureSnapshotInput.signal`). +// +// A binding's signal is fixed at bind time and `wait` binds once and polls many +// times, so each poll enforces its own remaining budget by ABORTING that +// capture and then waiting for it to quiesce. Without the per-capture signal +// the deadline never reaches the platform: a stalled capture consumes the whole +// request and `WAIT_REASONS.captureStalled` becomes unreachable. +// --------------------------------------------------------------------------- + +/** A capture that never settles on its own — it resolves only once its poll deadline aborts it. */ +function stallingCapture() { + const seen: AbortSignal[] = []; + const settledAt: number[] = []; + const captureSnapshot = vi.fn(async (input: CaptureSnapshotInput) => { + const signal = input.signal; + if (!signal) throw new Error('the poll deadline never reached the platform'); + seen.push(signal); + await new Promise((resolve) => { + if (signal.aborted) return resolve(); + signal.addEventListener('abort', () => resolve(), { once: true }); + }); + // A late capture: it settles only after its deadline fired. + settledAt.push(Date.now()); + throw new DOMException('Capture aborted', 'AbortError'); + }); + return { captureSnapshot, seen, settledAt }; +} + +test('a poll deadline aborts the in-flight capture, and aborts it as a deadline', async () => { + const stalling = stallingCapture(); + const harness = waitRuntimeHarness({ captureSnapshot: stalling.captureSnapshot }); + + await runWait(['text', 'Ready', '150'], harness); + + expect(stalling.seen.length).toBeGreaterThan(0); + const signal = stalling.seen[0]!; + expect(signal.aborted).toBe(true); + // Not merely "aborted": a request-end abort would also satisfy that. The deadline controller + // aborts with its own TimeoutError, which is what proves the POLL budget did the cancelling. + expect((signal.reason as DOMException | undefined)?.name).toBe('TimeoutError'); +}); + +test('the poll awaits the aborted capture instead of abandoning it', async () => { + const stalling = stallingCapture(); + const harness = waitRuntimeHarness({ captureSnapshot: stalling.captureSnapshot }); + + await runWait(['text', 'Ready', '150'], harness); + const returnedAt = Date.now(); + + // Ordering is the property, not the abort. `runWithinWaitDeadline` deliberately does not + // race-and-abandon: a late capture must not be able to write session state after the wait + // has already returned. + expect(stalling.settledAt.length).toBeGreaterThan(0); + for (const settled of stalling.settledAt) expect(settled).toBeLessThanOrEqual(returnedAt); +}); + +test('a stalled capture reports capture-stalled with no readable captures', async () => { + const stalling = stallingCapture(); + const harness = waitRuntimeHarness({ captureSnapshot: stalling.captureSnapshot }); + + const { response } = await runWait(['text', 'Ready', '150'], harness); + + expect(response.ok).toBe(false); + if (response.ok) return; + // Distinct from `wait_target_absent` (readable captures that lacked the target) and from + // `wait_deadline_exceeded` (a poll truncated after readable captures). Three separate paths in + // `waitTimeoutError`; collapsing any pair loses real diagnostic information. + expect(response.error.details?.reason).toBe(WAIT_REASONS.captureStalled); + expect(response.error.details?.captureStalled).toBe(true); + expect(response.error.details?.readableCaptures).toBe(0); +}); + +test('a readable capture that lacks the target stays target-absent, not capture-stalled', async () => { + const harness = waitRuntimeHarness({ + nodesPerPoll: [[{ index: 0, depth: 0, type: 'Button', label: 'Checkout', hittable: true }]], + }); + + const { response } = await runWait(['text', 'Ready', '400'], harness); + + expect(response.ok).toBe(false); + if (response.ok) return; + expect(response.error.details?.reason).toBe(WAIT_REASONS.targetAbsent); + expect(response.error.details?.readableCaptures).toBeGreaterThan(0); +}); diff --git a/src/daemon/handlers/__tests__/install-source.test.ts b/src/daemon/handlers/__tests__/install-source.test.ts index 9fbe562ef..67211f943 100644 --- a/src/daemon/handlers/__tests__/install-source.test.ts +++ b/src/daemon/handlers/__tests__/install-source.test.ts @@ -358,6 +358,8 @@ function sourceRuntimeFacts( withoutActiveApp: unavailable, }), ...screenshotRuntimeOperationFacts({ capture: unavailable }), + findText: unavailable, + findSelector: unavailable, setViewport: unavailable, readTextAtPoint: unavailable, deployApp: unavailable, diff --git a/src/daemon/handlers/__tests__/network-runtime-harness.ts b/src/daemon/handlers/__tests__/network-runtime-harness.ts index 587a06a37..1bf5b1487 100644 --- a/src/daemon/handlers/__tests__/network-runtime-harness.ts +++ b/src/daemon/handlers/__tests__/network-runtime-harness.ts @@ -19,14 +19,22 @@ export function createNetworkRuntime( networkFact: RuntimeOperationFact = { available: true }, owner: RuntimeOwnerRef = localRuntimeOwner(device.platform), ) { - const uses: Array<{ required: readonly string[]; preferred: readonly string[] }> = []; + const uses: Array<{ + required: readonly string[]; + preferred: readonly string[]; + conditional?: readonly string[]; + }> = []; const networkDump = vi.fn(implementation); const unavailable = { available: false, reason: 'owner-capability-missing', } as const; const bindDevice: BindDeviceRuntime = async (selected, use) => { - uses.push({ required: [...use.required], preferred: [...use.preferred] }); + uses.push({ + required: [...use.required], + preferred: [...use.preferred], + ...(use.conditional === undefined ? {} : { conditional: [...use.conditional] }), + }); return narrowDeviceBinding( { device: selected, diff --git a/src/daemon/handlers/__tests__/session-capabilities.fixtures.ts b/src/daemon/handlers/__tests__/session-capabilities.fixtures.ts index 759a3154b..da2b8d707 100644 --- a/src/daemon/handlers/__tests__/session-capabilities.fixtures.ts +++ b/src/daemon/handlers/__tests__/session-capabilities.fixtures.ts @@ -39,10 +39,18 @@ export const legacyCapabilityUses = [ ]; export function createCapabilitiesAdmissionRuntime(options: CapabilitiesAdmissionRuntimeOptions) { - const uses: Array<{ required: readonly string[]; preferred: readonly string[] }> = []; + const uses: Array<{ + required: readonly string[]; + preferred: readonly string[]; + conditional?: readonly string[]; + }> = []; const inspections: DeviceInfo[] = []; const bindDevice: BindDeviceRuntime = async (device, use) => { - uses.push({ required: [...use.required], preferred: [...use.preferred] }); + uses.push({ + required: [...use.required], + preferred: [...use.preferred], + ...(use.conditional === undefined ? {} : { conditional: [...use.conditional] }), + }); return narrowDeviceBinding(createAdmissionBinding(device, options), use); }; const inspectFacts: InspectDeviceRuntimeFacts = async (device) => { @@ -78,6 +86,8 @@ function createAdmissionFacts( withoutActiveApp: unavailable, }), ...screenshotRuntimeOperationFacts({ capture: screenshotFact }), + findText: unavailable, + findSelector: unavailable, setViewport: unavailable, deployApp: cell(options.deployAvailable), materializeAppSource: cell(options.sourceAvailable), diff --git a/src/daemon/handlers/__tests__/session-capabilities.test.ts b/src/daemon/handlers/__tests__/session-capabilities.test.ts index a6189ce0d..d64d06bd6 100644 --- a/src/daemon/handlers/__tests__/session-capabilities.test.ts +++ b/src/daemon/handlers/__tests__/session-capabilities.test.ts @@ -504,14 +504,22 @@ function createAdmissionRuntime(options: { screenshotAvailable?: boolean; providerMode: RuntimeProviderMode; }) { - const uses: Array<{ required: readonly string[]; preferred: readonly string[] }> = []; + const uses: Array<{ + required: readonly string[]; + preferred: readonly string[]; + conditional?: readonly string[]; + }> = []; const inspections: DeviceInfo[] = []; const inspectFacts: InspectDeviceRuntimeFacts = vi.fn(async (device: DeviceInfo) => { inspections.push(device); return createAdmissionBinding(device, options).facts; }); const bindDevice: BindDeviceRuntime = async (device, use) => { - uses.push({ required: [...use.required], preferred: [...use.preferred] }); + uses.push({ + required: [...use.required], + preferred: [...use.preferred], + ...(use.conditional === undefined ? {} : { conditional: [...use.conditional] }), + }); return narrowDeviceBinding(createAdmissionBinding(device, options), use); }; return { bindDevice, inspectFacts, inspections, uses }; diff --git a/src/daemon/handlers/__tests__/session-command-harness.ts b/src/daemon/handlers/__tests__/session-command-harness.ts index 3b8f90b6a..7c4018fbf 100644 --- a/src/daemon/handlers/__tests__/session-command-harness.ts +++ b/src/daemon/handlers/__tests__/session-command-harness.ts @@ -144,6 +144,8 @@ function readinessFacts(device: DeviceInfo): RuntimeFacts = []; + const uses: Array<{ + required: readonly string[]; + preferred: readonly string[]; + conditional?: readonly string[]; + }> = []; const bind = vi.fn( async (device: DeviceInfo): Promise> => ({ device, @@ -436,14 +440,22 @@ function createRuntimeHarness(options: { inspectAvailable?: boolean } = {}) { }), ); const bindDevice: BindDeviceRuntime = async (device, use) => { - uses.push(use); + uses.push({ + required: use.required, + preferred: use.preferred, + ...(use.conditional === undefined ? {} : { conditional: use.conditional }), + }); return narrowDeviceBinding(await bind(device), use); }; return { bind, bindDevice, boundUses: () => - uses.map((use) => ({ required: [...use.required], preferred: [...use.preferred] })), + uses.map((use) => ({ + required: [...use.required], + preferred: [...use.preferred], + ...(use.conditional === undefined ? {} : { conditional: [...use.conditional] }), + })), resetUses: () => { uses.length = 0; }, diff --git a/src/daemon/handlers/__tests__/snapshot-handler.test.ts b/src/daemon/handlers/__tests__/snapshot-handler.test.ts index 045607c43..44f262f30 100644 --- a/src/daemon/handlers/__tests__/snapshot-handler.test.ts +++ b/src/daemon/handlers/__tests__/snapshot-handler.test.ts @@ -107,14 +107,14 @@ function makeSessionStore(): SessionStore { return new SessionStore(path.join(root, 'sessions')); } -function makeSession(name: string, device: SessionState['device']): SessionState { - return { - name, - device, - createdAt: Date.now(), - actions: [], - }; +type SessionExtra = Partial; +function makeSession(name: string, d: SessionState['device'], extra?: SessionExtra): SessionState { + return { name, device: d, createdAt: Date.now(), actions: [], ...extra }; } +// An Apple wait runs inside an opened app: that bundle id is XCUITest's attach target, and +// without one the plan asks for the without-active-app row local Apple refuses. +const appAttach = (d: SessionState['device']): Partial => + d.platform === 'apple' ? { appBundleId: 'com.example.app' } : {}; const iosSimulatorDevice: SessionState['device'] = { platform: 'apple', @@ -250,7 +250,7 @@ async function runWaitCommand( positionals: string[], ) { const sessionStore = makeSessionStore(); - sessionStore.set(sessionName, makeSession(sessionName, device)); + sessionStore.set(sessionName, makeSession(sessionName, device, appAttach(device))); return await handleSnapshotCommands({ req: { token: 't', @@ -262,6 +262,7 @@ async function runWaitCommand( sessionName, logPath: '/tmp/daemon.log', sessionStore, + ...snapshotRuntimeFixture(), }); } @@ -1715,6 +1716,7 @@ test('wait text on Android uses freshness-aware capture instead of one-shot snap sessionName, logPath: '/tmp/daemon.log', sessionStore, + ...snapshotRuntimeFixture(), }); expect(response?.ok).toBe(true); @@ -2090,337 +2092,6 @@ test('diff screenshot is not handled daemon-side (client-backed command)', async } }); -test('wait text uses Apple runner path on macOS desktop sessions', async () => { - const sessionStore = makeSessionStore(); - const sessionName = 'macos-wait'; - sessionStore.set(sessionName, { - ...makeSession(sessionName, macOsDevice), - appBundleId: 'com.apple.systempreferences', - }); - - mockRunnerCommand.mockResolvedValue({ found: true }); - - const response = await handleSnapshotCommands({ - req: { - token: 't', - session: sessionName, - command: 'wait', - positionals: ['Accessibility', '10'], - flags: {}, - }, - sessionName, - logPath: '/tmp/daemon.log', - sessionStore, - }); - - expect(response?.ok).toBe(true); - expect(mockRunnerCommand).toHaveBeenCalledTimes(1); - const callArgs = mockRunnerCommand.mock.calls[0]; - expect((callArgs?.[1] as any)?.command).toBe('findText'); - expect((callArgs?.[1] as any)?.text).toBe('Accessibility'); -}); - -test('wait text on iOS without app bundle id uses snapshot path', async () => { - const sessionStore = makeSessionStore(); - const sessionName = 'ios-wait-no-app-bundle'; - sessionStore.set(sessionName, makeSession(sessionName, iosSimulatorDevice)); - - mockDispatch.mockResolvedValue({ - nodes: [ - { - index: 0, - depth: 0, - type: 'Window', - rect: { x: 0, y: 0, width: 390, height: 844 }, - }, - { - index: 1, - depth: 1, - parentIndex: 0, - type: 'StaticText', - label: 'Agent Device Tester', - rect: { x: 20, y: 80, width: 240, height: 40 }, - }, - ], - }); - - const response = await handleSnapshotCommands({ - req: { - token: 't', - session: sessionName, - command: 'wait', - positionals: ['Agent Device Tester', '5000'], - flags: {}, - }, - sessionName, - logPath: '/tmp/daemon.log', - sessionStore, - }); - - expect(response?.ok).toBe(true); - expect(mockRunnerCommand).not.toHaveBeenCalled(); - expect(mockDispatch).toHaveBeenCalledWith( - expect.anything(), - 'snapshot', - [], - undefined, - expect.anything(), - ); -}); - -test('wait text falls back to the canonical snapshot after an Apple runner miss', async () => { - const sessionStore = makeSessionStore(); - const sessionName = 'ios-wait-runner-miss'; - sessionStore.set(sessionName, { - ...makeSession(sessionName, iosSimulatorDevice), - appBundleId: 'com.example.app', - }); - - mockRunnerCommand.mockResolvedValue({ found: false }); - mockDispatch.mockResolvedValue({ - nodes: [ - { - index: 0, - depth: 0, - type: 'Window', - rect: { x: 0, y: 0, width: 390, height: 844 }, - }, - { - index: 1, - depth: 1, - parentIndex: 0, - type: 'StaticText', - label: 'Agent Device Tester', - rect: { x: 20, y: 80, width: 240, height: 40 }, - }, - ], - }); - - const response = await handleSnapshotCommands({ - req: { - token: 't', - session: sessionName, - command: 'wait', - positionals: ['Agent Device Tester', '5000'], - flags: {}, - }, - sessionName, - logPath: '/tmp/daemon.log', - sessionStore, - }); - - expect(response?.ok).toBe(true); - expect(mockRunnerCommand).toHaveBeenCalledTimes(1); - expect(mockDispatch).toHaveBeenCalledWith( - expect.anything(), - 'snapshot', - [], - undefined, - expect.anything(), - ); -}); - -// fallow-ignore-next-line complexity -test('wait selector uses direct iOS selector query when possible', async () => { - const sessionStore = makeSessionStore(); - const sessionName = 'ios-wait-selector'; - sessionStore.set(sessionName, { - ...makeSession(sessionName, iosSimulatorDevice), - appBundleId: 'com.example.app', - }); - - mockRunnerCommand.mockResolvedValue({ - found: true, - nodes: [ - { - ref: 'e1', - type: 'Button', - label: 'Continue', - rect: { x: 10, y: 20, width: 120, height: 44 }, - }, - ], - }); - - const response = await handleSnapshotCommands({ - req: { - token: 't', - session: sessionName, - command: 'wait', - positionals: ['id="continue-button"', '5000'], - flags: {}, - }, - sessionName, - logPath: '/tmp/daemon.log', - sessionStore, - }); - - expect(response?.ok).toBe(true); - if (response?.ok) { - expect(response.data?.selector).toBe('id="continue-button"'); - } - expect(mockRunnerCommand).toHaveBeenCalledTimes(1); - const callArgs = mockRunnerCommand.mock.calls[0]; - expect((callArgs?.[1] as any)?.command).toBe('querySelector'); - expect((callArgs?.[1] as any)?.selectorKey).toBe('id'); - expect((callArgs?.[1] as any)?.selectorValue).toBe('continue-button'); -}); - -test('wait selector falls back to snapshot runtime when direct iOS selector misses', async () => { - const sessionStore = makeSessionStore(); - const sessionName = 'ios-wait-selector-fallback'; - sessionStore.set(sessionName, { - ...makeSession(sessionName, iosSimulatorDevice), - appBundleId: 'com.example.app', - }); - - mockRunnerCommand.mockResolvedValue({ found: false }); - mockDispatch.mockResolvedValue({ - nodes: [ - { - index: 0, - depth: 0, - type: 'Window', - rect: { x: 0, y: 0, width: 390, height: 844 }, - }, - { - index: 1, - depth: 1, - parentIndex: 0, - type: 'Button', - identifier: 'continue-button', - label: 'Continue', - rect: { x: 10, y: 20, width: 120, height: 44 }, - }, - ], - }); - - const response = await handleSnapshotCommands({ - req: { - token: 't', - session: sessionName, - command: 'wait', - positionals: ['id="continue-button"', '5000'], - flags: {}, - }, - sessionName, - logPath: '/tmp/daemon.log', - sessionStore, - }); - - expect(response?.ok).toBe(true); - expect(mockRunnerCommand).toHaveBeenCalledTimes(1); - expect(mockDispatch).toHaveBeenCalledWith( - expect.anything(), - 'snapshot', - [], - undefined, - expect.anything(), - ); -}); - -test('wait selector bypasses fresh snapshot cache after direct iOS selector misses', async () => { - const sessionStore = makeSessionStore(); - const sessionName = 'ios-wait-selector-fresh-snapshot'; - const session = { - ...makeSession(sessionName, iosSimulatorDevice), - appBundleId: 'com.example.app', - }; - session.snapshot = { - createdAt: Date.now(), - presentationKey: buildSnapshotPresentationKey({}), - nodes: [ - { - ref: 'e0', - index: 0, - type: 'Window', - rect: { x: 0, y: 0, width: 390, height: 844 }, - }, - ], - }; - sessionStore.set(sessionName, session); - - mockRunnerCommand.mockResolvedValue({ found: false }); - mockDispatch.mockResolvedValue({ - nodes: [ - { - index: 0, - depth: 0, - type: 'Window', - rect: { x: 0, y: 0, width: 390, height: 844 }, - }, - { - index: 1, - depth: 1, - parentIndex: 0, - type: 'Button', - identifier: 'continue-button', - label: 'Continue', - rect: { x: 10, y: 20, width: 120, height: 44 }, - }, - ], - }); - - const response = await handleSnapshotCommands({ - req: { - token: 't', - session: sessionName, - command: 'wait', - positionals: ['id="continue-button"', '5000'], - flags: {}, - }, - sessionName, - logPath: '/tmp/daemon.log', - sessionStore, - }); - - expect(response?.ok).toBe(true); - expect(mockDispatch).toHaveBeenCalledWith( - expect.anything(), - 'snapshot', - [], - undefined, - expect.anything(), - ); -}); - -test('wait selector does not snapshot-fallback on ambiguous direct iOS selector match', async () => { - const sessionStore = makeSessionStore(); - const sessionName = 'ios-wait-selector-ambiguous'; - sessionStore.set(sessionName, { - ...makeSession(sessionName, iosSimulatorDevice), - appBundleId: 'com.example.app', - }); - - mockRunnerCommand.mockRejectedValue( - new AppError('AMBIGUOUS_MATCH', 'Selector matched multiple elements'), - ); - - const response = await handleSnapshotCommands({ - req: { - token: 't', - session: sessionName, - command: 'wait', - positionals: ['id="continue-button"', '5000'], - flags: {}, - }, - sessionName, - logPath: '/tmp/daemon.log', - sessionStore, - }); - - expect(response?.ok).toBe(false); - if (response?.ok === false) { - expect(response.error.code).toBe('AMBIGUOUS_MATCH'); - } - expect(mockDispatch).not.toHaveBeenCalledWith( - expect.anything(), - 'snapshot', - expect.anything(), - expect.anything(), - expect.anything(), - ); -}); - test('wait selector bypasses a fresh matching session snapshot', async () => { const sessionStore = makeSessionStore(); const sessionName = 'android-wait-fresh-capture'; @@ -2459,6 +2130,7 @@ test('wait selector bypasses a fresh matching session snapshot', async () => { sessionName, logPath: '/tmp/daemon.log', sessionStore, + ...snapshotRuntimeFixture(), }); expect(response?.ok).toBe(true); diff --git a/src/daemon/handlers/__tests__/system-surface-disclosure.test.ts b/src/daemon/handlers/__tests__/system-surface-disclosure.test.ts index aed6b991c..1bef498cd 100644 --- a/src/daemon/handlers/__tests__/system-surface-disclosure.test.ts +++ b/src/daemon/handlers/__tests__/system-surface-disclosure.test.ts @@ -4,6 +4,7 @@ import { getRuntimeBindings } from './interaction-get-runtime-fixture.ts'; import { dispatchFindReadOnlyViaRuntime, dispatchWaitViaRuntime } from '../../selector-runtime.ts'; import type { DaemonRequest, DaemonResponse } from '../../types.ts'; import { ANDROID_SYSTEM_SURFACE_DISCLOSURE } from '../../../core/android-system-surface-disclosure.ts'; +import { snapshotRuntimeFixture } from '../../__tests__/snapshot-runtime-fixture.ts'; import { makeSessionStore } from '../../../__tests__/test-utils/store-factory.ts'; import { makeAndroidSession } from '../../../__tests__/test-utils/session-factories.ts'; @@ -132,7 +133,7 @@ test('wait timeout for app text hidden behind a system surface discloses the occ sessionName: 'default', logPath: '/tmp/test.log', sessionStore, - ...getRuntimeBindings(), + ...snapshotRuntimeFixture(), }); expect(response.ok).toBe(false); @@ -217,7 +218,7 @@ test('sessionless wait success on shade content still discloses the occluding sy sessionName: 'default', logPath: '/tmp/test.log', sessionStore, - ...getRuntimeBindings(), + ...snapshotRuntimeFixture(), }); expect(response.ok).toBe(true); @@ -243,7 +244,7 @@ test('sessionless wait timeout still discloses the occluding system surface', as sessionName: 'default', logPath: '/tmp/test.log', sessionStore, - ...getRuntimeBindings(), + ...snapshotRuntimeFixture(), }); expect(response.ok).toBe(false); diff --git a/src/daemon/handlers/__tests__/wait-landmark-recording.test.ts b/src/daemon/handlers/__tests__/wait-landmark-recording.test.ts index 9e8c7405b..de028b4e6 100644 --- a/src/daemon/handlers/__tests__/wait-landmark-recording.test.ts +++ b/src/daemon/handlers/__tests__/wait-landmark-recording.test.ts @@ -16,6 +16,7 @@ import { dispatchWaitViaRuntime } from '../../selector-runtime.ts'; import type { DaemonRequest } from '../../types.ts'; import { WAIT_LANDMARK_MISMATCH_REASON } from '@agent-device/contracts/replay'; import type { TargetAnnotationV1 } from '@agent-device/contracts/replay'; +import { snapshotRuntimeFixture } from '../../__tests__/snapshot-runtime-fixture.ts'; import { makeSessionStore } from '../../../__tests__/test-utils/store-factory.ts'; import { makeAndroidSession, @@ -96,6 +97,7 @@ async function runWait(options: { recording?: boolean; req?: DaemonRequest } = { sessionName: 'default', logPath: '/tmp/test.log', sessionStore, + ...snapshotRuntimeFixture(), }); return { response, sessionStore }; } diff --git a/src/daemon/handlers/session-install-capability-projection.test.ts b/src/daemon/handlers/session-install-capability-projection.test.ts index 697367f8b..6f9e59f83 100644 --- a/src/daemon/handlers/session-install-capability-projection.test.ts +++ b/src/daemon/handlers/session-install-capability-projection.test.ts @@ -28,8 +28,14 @@ test('guards the public source-install alias against descriptor-use drift', () = expect(runtimeUse(PUBLIC_COMMANDS.installFromSource)).toEqual( runtimeUse(INTERNAL_COMMANDS.installSource), ); - expect(runtimeUse(PUBLIC_COMMANDS.install)).toEqual({ required: ['deployApp'], preferred: [] }); - expect(runtimeUse(PUBLIC_COMMANDS.reinstall)).toEqual({ required: ['deployApp'], preferred: [] }); + expect(runtimeUse(PUBLIC_COMMANDS.install)).toEqual({ + required: ['deployApp'], + preferred: [], + }); + expect(runtimeUse(PUBLIC_COMMANDS.reinstall)).toEqual({ + required: ['deployApp'], + preferred: [], + }); expect(runtimeUse(PUBLIC_COMMANDS.push)).toEqual({ required: ['ensureReady', 'sendPushNotification'], preferred: [], diff --git a/src/daemon/handlers/snapshot.ts b/src/daemon/handlers/snapshot.ts index f804385f5..72b2d114d 100644 --- a/src/daemon/handlers/snapshot.ts +++ b/src/daemon/handlers/snapshot.ts @@ -43,8 +43,15 @@ const SNAPSHOT_COMMAND_HANDLER_IMPLS = { bindDevice, }); }, - wait: async ({ req, sessionName, logPath, sessionStore }) => - await dispatchWaitViaRuntime({ req, sessionName, logPath, sessionStore }), + wait: async ({ req, sessionName, logPath, sessionStore, inspectFacts, bindDevice }) => + await dispatchWaitViaRuntime({ + req, + sessionName, + logPath, + sessionStore, + inspectFacts, + bindDevice, + }), alert: async ({ req, sessionName, logPath, sessionStore }) => { const { session, device } = await resolveSessionDevice(sessionStore, sessionName, req.flags); return await withSessionlessRunnerCleanup(session, device, async () => { diff --git a/src/daemon/request-runtime-binding.ts b/src/daemon/request-runtime-binding.ts index a42dbee98..ec3108e0d 100644 --- a/src/daemon/request-runtime-binding.ts +++ b/src/daemon/request-runtime-binding.ts @@ -20,10 +20,16 @@ export type BindDeviceRuntime = < RuntimeOperationKey, Required[number] >[], + const Conditional extends readonly Exclude< + RuntimeOperationKey, + Required[number] | Preferred[number] + >[], >( device: DeviceInfo, - use: RuntimeUse, -) => Promise>>; + use: RuntimeUse, +) => Promise< + BoundDeviceRuntime> +>; export type BindExactDeviceRuntime = < const Required extends readonly RuntimeOperationKey[], @@ -31,13 +37,19 @@ export type BindExactDeviceRuntime = < RuntimeOperationKey, Required[number] >[], + const Conditional extends readonly Exclude< + RuntimeOperationKey, + Required[number] | Preferred[number] + >[], >( device: DeviceInfo, owner: RuntimeOwnerRef, fence: ResourceOwnershipFence, - use: RuntimeUse, + use: RuntimeUse, scope: PlatformRequestScope, -) => Promise>>; +) => Promise< + BoundDeviceRuntime> +>; export type InspectDeviceRuntimeFacts = ( device: DeviceInfo, diff --git a/src/daemon/runtime-admission.ts b/src/daemon/runtime-admission.ts index 9874ec3ac..378821d10 100644 --- a/src/daemon/runtime-admission.ts +++ b/src/daemon/runtime-admission.ts @@ -72,10 +72,18 @@ export async function admitRuntimeUse< RuntimeOperationKey, Required[number] >[], + const Conditional extends readonly Exclude< + RuntimeOperationKey, + Required[number] | Preferred[number] + >[], >( request: Omit & - Readonly<{ use: RuntimeUse }>, -): Promise>> { + Readonly<{ + use: RuntimeUse; + }>, +): Promise< + RuntimeAdmission> +> { const admitted = await admitRuntimeOperations({ ...request, required: request.use.required }); if (admitted.type === 'response') return admitted; return { type: 'runtime', runtime: await admitted.bind(request.device, request.use) }; diff --git a/src/daemon/selector-capture-binding.ts b/src/daemon/selector-capture-binding.ts index 48c259e14..915d7b6e7 100644 --- a/src/daemon/selector-capture-binding.ts +++ b/src/daemon/selector-capture-binding.ts @@ -2,13 +2,15 @@ import { resolveSelectorCaptureRuntimePlan, type CaptureSnapshotInput, type ElementTextRuntimeOperations, + type FindTextRuntimeOperations, + type FindSelectorRuntimeOperations, type SnapshotResult, } from '@agent-device/contracts/platform'; import type { BindDeviceRuntime, InspectDeviceRuntimeFacts } from './request-runtime-binding.ts'; import { admitAndBindSnapshotCapture } from './snapshot-runtime-binding.ts'; import type { DaemonResponse, SessionState } from './types.ts'; -/** The selector commands that resolve their targets from a request-bound capture. */ +/** The selector commands that resolve their targets from the shared request-bound capture seam. */ export type SelectorCaptureCommand = 'find' | 'get' | 'is' | 'wait'; /** @@ -25,14 +27,15 @@ export type BoundSelectorCapture = (input: CaptureSnapshotInput) => Promise; export type ResolvedSelectorCapture = @@ -57,6 +60,7 @@ export async function resolveBoundSelectorCapture( ...params, plan: resolveSelectorCaptureRuntimePlan({ hasActiveApp: params.session?.appBundleId !== undefined, + intent: selectorCaptureIntent(params.command), }), }); if (!bound.ok) return bound; @@ -67,6 +71,22 @@ export async function resolveBoundSelectorCapture( operations: { capture: bound.capture, ...(bound.readTextAtPoint ? { readText: bound.readTextAtPoint } : {}), + ...(bound.findText ? { findText: bound.findText } : {}), + ...(bound.findSelector ? { findSelector: bound.findSelector } : {}), }, }; } + +function selectorCaptureIntent( + command: SelectorCaptureCommand, +): 'capture-only' | 'element-text' | 'wait-observation' { + switch (command) { + case 'is': + return 'capture-only'; + case 'find': + case 'get': + return 'element-text'; + case 'wait': + return 'wait-observation'; + } +} diff --git a/src/daemon/selector-capture-runtime.ts b/src/daemon/selector-capture-runtime.ts index 6eda22b0f..29b55fd29 100644 --- a/src/daemon/selector-capture-runtime.ts +++ b/src/daemon/selector-capture-runtime.ts @@ -204,6 +204,11 @@ async function runCapture( session: params.session, snapshotScope, includeRects: request.includeRects, + // The POLL's remaining budget, not the request's. A binding's signal is fixed at + // bind time and `wait` binds once and polls many times, so without this the + // deadline never reaches the platform: a stalled capture would consume the whole + // request instead of producing the `capture-stalled` verdict. + signal: request.signal, }), ), }), diff --git a/src/daemon/selector-operation-binding.ts b/src/daemon/selector-operation-binding.ts new file mode 100644 index 000000000..a2ed91635 --- /dev/null +++ b/src/daemon/selector-operation-binding.ts @@ -0,0 +1,63 @@ +import type { + ElementTextRuntimeOperations, + FindSelectorInput, + FindSelectorRuntimeOperations, + FindTextInput, + FindTextRuntimeOperations, + ReadTextAtPointInput, +} from '@agent-device/contracts/platform'; + +export type BoundElementRead = ElementTextRuntimeOperations['readTextAtPoint']; +export type BoundNativeTextRead = FindTextRuntimeOperations['findText']; +export type BoundNativeSelectorRead = FindSelectorRuntimeOperations['findSelector']; + +type SelectorOperations = Readonly<{ + readTextAtPoint?: BoundElementRead; + findText?: BoundNativeTextRead; + findSelector?: BoundNativeSelectorRead; +}>; + +/** Projects the one preferred operation admitted for `get` and read-only `find`. */ +export function selectElementTextOperation( + runtime: Readonly<{ + operations: Readonly<{ readTextAtPoint?: BoundElementRead }>; + }>, +): Pick { + const { readTextAtPoint } = runtime.operations; + const selected = readTextAtPoint ? { operations: { readTextAtPoint } } : undefined; + return Object.freeze({ + ...(selected + ? { + readTextAtPoint: async (input: ReadTextAtPointInput) => + await selected.operations.readTextAtPoint(input), + } + : {}), + }); +} + +/** Projects only the fact-conditional observations admitted for `wait`. */ +export function selectWaitObservationOperations( + runtime: Readonly<{ + operations: Readonly<{ + findText?: BoundNativeTextRead; + findSelector?: BoundNativeSelectorRead; + }>; + }>, +): Pick { + const { findText, findSelector } = runtime.operations; + const selectedText = findText ? { operations: { findText } } : undefined; + const selectedSelector = findSelector ? { operations: { findSelector } } : undefined; + return Object.freeze({ + ...(selectedText + ? { + findText: async (input: FindTextInput) => await selectedText.operations.findText(input), + } + : {}), + ...(selectedSelector + ? { + findSelector: async (input: FindSelectorInput) => + await selectedSelector.operations.findSelector(input), + } + : {}), + }); +} diff --git a/src/daemon/selector-runtime-backend.ts b/src/daemon/selector-runtime-backend.ts index 7c0158477..4e3002ad2 100644 --- a/src/daemon/selector-runtime-backend.ts +++ b/src/daemon/selector-runtime-backend.ts @@ -1,12 +1,13 @@ -import type { AgentDeviceBackend, BackendSnapshotResult } from '../backend.ts'; +import type { + AgentDeviceBackend, + BackendCommandContext, + BackendSnapshotResult, +} from '../backend.ts'; import { resolveTargetDevice } from '../core/dispatch.ts'; import { createAgentDevice } from '../runtime.ts'; -import { isMacOs, isApplePlatform, publicPlatformString } from '@agent-device/kernel/device'; +import { publicPlatformString } from '@agent-device/kernel/device'; import { noActiveSessionError } from './handlers/response.ts'; import type { SnapshotState, SnapshotNode } from '@agent-device/kernel/snapshot'; -import { findNodeByLabel } from '../core/snapshot-node-lookup.ts'; -import { runAppleRunnerCommand } from '../platforms/apple/core/runner/runner-client.ts'; -import { buildAppleRunnerRequestOptions } from './apple-runner-options.ts'; import { createDaemonRuntimePolicy } from './runtime-policy.ts'; import { createDaemonRuntimeSessionStore } from './runtime-session.ts'; import { contextFromFlags } from './context.ts'; @@ -17,13 +18,13 @@ import type { ContextFromFlags } from './handlers/interaction-common.ts'; import { SessionStore } from './session-store.ts'; import type { DaemonRequest, DaemonResponse, SessionState } from './types.ts'; import { createSelectorCaptureRuntime } from './selector-capture-runtime.ts'; +import { buildRuntimeCaptureInput } from './snapshot-runtime-capture-input.ts'; import { resolveBoundSelectorCapture, type BoundSelectorOperations, type SelectorCaptureCommand, } from './selector-capture-binding.ts'; import type { BindDeviceRuntime, InspectDeviceRuntimeFacts } from './request-runtime-binding.ts'; -import { isActiveProviderDevice } from '../provider-device-runtime.ts'; import { getRequestSignal } from '../request/cancel.ts'; import { snapshotOptionsToFlags } from '../backend-snapshot-options.ts'; @@ -57,12 +58,6 @@ type ResolvedSelectorDevice = | { ok: true; session: SessionState | undefined; device: SessionState['device'] } | { ok: false; response: DaemonResponse }; -type AppleRunnerFindTextTarget = { - device: SessionState['device']; - appBundleId: string; - traceLogPath?: string; -}; - export function createSelectorRuntimeForDevice(params: SelectorRuntimeDeviceParams) { return createAgentDevice({ backend: createSelectorBackend(params), @@ -142,6 +137,18 @@ function createSelectorBackend(params: SelectorRuntimeDeviceParams): AgentDevice ((flags, appBundleId, traceLogPath) => contextFromFlags(logPath ?? '', flags, appBundleId, traceLogPath)); const readTextAtPoint = params.bound?.readText; + const boundFindText = params.bound?.findText; + // The native reading must run in the SAME runner context as the capture it short-circuits: + // one requestId so diagnostics land in one request file, the session's log/trace paths, and + // the XCUITest override + runner-lease context the caller configured. Built through the one + // capture-intent builder the capture leg uses, never a second hand-rolled context. + const runnerExecution = buildRuntimeCaptureInput({ + flags: req.flags, + logPath: logPath ?? '', + meta: req.meta, + session, + snapshotScope: undefined, + }).execution; const captureRuntime = createSelectorCaptureRuntime({ device, session, @@ -189,107 +196,22 @@ function createSelectorBackend(params: SelectorRuntimeDeviceParams): AgentDevice contextFromFlags: resolveContextFromFlags, }), }), - findText: async (context, text) => ({ - found: await findText(params, text, context.signal), - }), - }; -} - -async function findText( - params: SelectorRuntimeDeviceParams, - text: string, - signal?: AbortSignal, -): Promise { - const macosSurfaceResult = await findTextInMacosNonAppSurface(params, text, signal); - if (macosSurfaceResult !== null) return macosSurfaceResult; - const appleRunnerResult = await findTextWithAppleRunner(params, text, signal); - // The runner query is a fast path, not the semantic source of truth. XCTest can report a - // transient miss for visible SwiftUI text that the canonical snapshot already contains. - if (appleRunnerResult === true) return true; - return await findTextInWaitSnapshot(params, text, signal); -} - -async function findTextInMacosNonAppSurface( - params: SelectorRuntimeDeviceParams, - text: string, - signal?: AbortSignal, -): Promise { - if (!isMacOs(params.device)) return null; - if (!params.session?.surface || params.session.surface === 'app') return null; - return await findTextInWaitSnapshot(params, text, signal); -} - -async function findTextWithAppleRunner( - params: SelectorRuntimeDeviceParams, - text: string, - signal?: AbortSignal, -): Promise { - const target = readAppleRunnerFindTextTarget(params); - if (!target) return null; - const result = (await runAppleRunnerCommand( - target.device, - { command: 'findText', text, appBundleId: target.appBundleId }, - { ...buildAppleRunnerFindTextOptions(params, target), signal }, - )) as { found?: boolean }; - return result?.found === true; -} - -function readAppleRunnerFindTextTarget( - params: SelectorRuntimeDeviceParams, -): AppleRunnerFindTextTarget | null { - if (!isApplePlatform(params.device.platform)) return null; - if (isActiveProviderDevice(params.device)) return null; - if (!params.session?.appBundleId) return null; - return { - device: params.device, - appBundleId: params.session.appBundleId, - traceLogPath: params.session.trace?.outPath, + // The owner's native text reading, forwarded only when its facts advertised it. The daemon + // makes no family, provider, surface, or session decision here: an owner that cannot answer + // reports `found: false` and the poll consults the canonical tree. + ...(boundFindText + ? { + findText: async (context: BackendCommandContext, text: string) => ({ + found: ( + await boundFindText({ + text, + options: { appBundleId: session?.appBundleId, surface: session?.surface }, + execution: runnerExecution, + ...(context.signal ? { signal: context.signal } : {}), + }) + ).found, + }), + } + : {}), }; } - -function buildAppleRunnerFindTextOptions( - params: SelectorRuntimeDeviceParams, - target: AppleRunnerFindTextTarget, -) { - return buildAppleRunnerRequestOptions({ - req: params.req, - logPath: params.logPath, - traceLogPath: target.traceLogPath, - }); -} - -async function findTextInWaitSnapshot( - params: SelectorRuntimeDeviceParams, - text: string, - signal?: AbortSignal, -): Promise { - const snapshot = await captureWaitSnapshot(params, signal); - return Boolean(findNodeByLabel(snapshot.nodes, text)); -} - -async function captureWaitSnapshot(params: SelectorRuntimeDeviceParams, signal?: AbortSignal) { - const captureRuntime = createSelectorCaptureRuntime({ - device: params.device, - session: params.session, - sessionStore: params.sessionStore, - sessionName: params.sessionName, - req: params.req, - logPath: params.logPath, - consumedSnapshot: params.consumedSnapshot, - }); - const { snapshot } = await captureRuntime.capture({ - flags: { - ...params.req.flags, - snapshotInteractiveOnly: false, - // Presence-only wait poll (findText is only called from waitForText): - // skip scroll-hint derivation (#1270). - snapshotIncludeHiddenContentHints: false, - }, - signal, - cache: { - forceFresh: true, - bypassForPostGestureStabilization: true, - }, - }); - return snapshot; -} diff --git a/src/daemon/selector-runtime.ts b/src/daemon/selector-runtime.ts index 1f255ecb2..a7ddd1a98 100644 --- a/src/daemon/selector-runtime.ts +++ b/src/daemon/selector-runtime.ts @@ -1,14 +1,12 @@ +import { waitObservesDevice } from '@agent-device/contracts/platform'; import { parseWaitPositionals } from '../core/wait-positionals.ts'; import type { WaitParsed } from '../core/wait-positionals.ts'; -import { AppError, asAppError, normalizeError } from '@agent-device/kernel/errors'; +import { AppError, asAppError } from '@agent-device/kernel/errors'; import type { SnapshotNode } from '@agent-device/kernel/snapshot'; -import { runAppleRunnerCommand } from '../platforms/apple/core/runner/runner-client.ts'; -import { - buildAppleRunnerRequestOptions, - type AppleRunnerRequestOptions, -} from './apple-runner-options.ts'; +import { queryAppleRunnerSelector } from '../platforms/apple/core/runner/runner-selector-query.ts'; +import type { AppleRunnerRequestOptions } from './apple-runner-options.ts'; import type { DaemonRequest, DaemonResponse, SessionState } from './types.ts'; -import { errorResponse, requireCommandSupported } from './handlers/response.ts'; +import { errorResponse } from './handlers/response.ts'; import { markSessionPartialRefsIssued, resolveRefStalenessWarning } from './session-snapshot.ts'; import { resolveSessionDevice, withSessionlessRunnerCleanup } from './handlers/snapshot-session.ts'; import { @@ -39,17 +37,18 @@ import type { RecordedTargetCapture } from './session-target-evidence.ts'; import type { TargetAnnotationV1 } from '@agent-device/contracts/replay'; import { maybeWaitTimeoutSurfaceResponse } from './wait-current-surface.ts'; import { withSystemSurfaceDisclosure } from './handlers/system-surface-disclosure.ts'; -import { - isDirectIosSelectorFallbackError, - readSimpleIosSelectorTarget, - type DirectIosSelectorTarget, -} from './direct-ios-selector.ts'; -import { isSessionRecording } from './session-script-publication-capability.ts'; +import type { DirectIosSelectorTarget } from './direct-ios-selector.ts'; import { createBoundSelectorRuntime, createSelectorRuntimeForDevice, type SelectorRuntimeParams, } from './selector-runtime-backend.ts'; +import type { BindDeviceRuntime, InspectDeviceRuntimeFacts } from './request-runtime-binding.ts'; +import { + resolveBoundSelectorCapture, + type BoundSelectorOperations, +} from './selector-capture-binding.ts'; +import { dispatchConditionalWaitSelector } from './wait-conditional-selector.ts'; export type DirectIosSelectorQueryResult = { found: boolean; @@ -57,13 +56,6 @@ export type DirectIosSelectorQueryResult = { node?: SnapshotNode; }; -type DirectIosSelectorErrorResult = { kind: 'error'; response: DaemonResponse }; - -type DirectIosSelectorFallbackResult = - | DirectIosSelectorQueryResult - | DirectIosSelectorErrorResult - | null; - export async function dispatchFindReadOnlyViaRuntime( params: SelectorRuntimeParams, ): Promise { @@ -259,31 +251,31 @@ export async function dispatchIsViaRuntime( } export async function dispatchWaitViaRuntime( - params: SelectorRuntimeParams, + params: SelectorRuntimeParams & + Readonly<{ inspectFacts?: InspectDeviceRuntimeFacts; bindDevice?: BindDeviceRuntime }>, ): Promise { const { req, sessionName, sessionStore } = params; const parsed = parseWaitPositionals(req.positionals ?? []); if (!parsed) return errorResponse('INVALID_ARGS', 'wait requires a duration or text'); if (parsed.kind === 'invalid') return errorResponse('INVALID_ARGS', parsed.message); const { session, device } = await resolveSessionDevice(sessionStore, sessionName, req.flags); - if (parsed.kind !== 'sleep') { - const unsupported = requireCommandSupported('wait', device); - if (unsupported) return unsupported; - } - // ADR 0012 / #1349: recording and a replayed landmark check both need the - // snapshot polling path — evidence and the identity comparison are computed - // from the resolution tree the direct runner query never captures. - const recordedLandmark = req.internal?.replayLandmarkGuard; - if (parsed.kind === 'selector' && !recordedLandmark && !isSessionRecording(session)) { - const directResponse = await dispatchDirectIosSelectorWait({ - ...params, - session, + // ADR 0019: facts are the only support authority, through the selector family's one + // admit-then-bind entry. A duration wait observes nothing, so it never asks for a binding — + // exactly the cell legacy admission skipped by testing `parsed.kind !== 'sleep'`. + let waitOperations: BoundSelectorOperations | undefined; + if (waitObservesDevice(parsed.kind)) { + const bound = await resolveBoundSelectorCapture({ + command: 'wait', device, - selectorExpression: parsed.selectorExpression, - timeoutMs: parsed.timeoutMs, + session, + inspectFacts: params.inspectFacts, + bindDevice: params.bindDevice, }); - if (directResponse) return directResponse; + if (!bound.ok) return bound.response; + waitOperations = bound.operations; } + // ADR 0012 / #1349, replay-only: the recorded landmark identity this wait must observe. + const recordedLandmark = req.internal?.replayLandmarkGuard; // #1076 + ADR 0014: a wait @ref names an element from the retained ref-frame // evidence, and its staleness is frame-derived rather than a property of the // live polling capture the condition is checked against. Once the ref frame @@ -303,6 +295,20 @@ export async function dispatchWaitViaRuntime( mintedGeneration: versionedRef.generation, }); } + if (waitParsed.kind === 'selector') { + const conditionalResponse = await dispatchConditionalWaitSelector({ + selectorExpression: waitParsed.selectorExpression, + operation: waitOperations?.findSelector, + recordedLandmark, + req, + session, + sessionName, + sessionStore, + logPath: params.logPath, + signal: params.signal, + }); + if (conditionalResponse) return conditionalResponse; + } // Wait builds its runtime directly (no createBoundSelectorRuntime), so the consumed-snapshot slot // must be initialized here too or sessionless waits have nowhere to report the capture from. params.consumedSnapshot ??= {}; @@ -311,6 +317,7 @@ export async function dispatchWaitViaRuntime( ...params, session, device, + bound: waitOperations, }); const response = await toDaemonResponse(async () => { const result = await runtime.selectors.wait({ @@ -330,10 +337,14 @@ export async function dispatchWaitViaRuntime( const data = toDaemonWaitData(result); return staleRefsWarning ? { ...data, warning: staleRefsWarning } : data; }); - const enrichedResponse = await maybeWaitTimeoutSurfaceResponse( - { req, logPath: params.logPath, session, device }, - response, - ); + // Only a polling wait can fail with `targetAbsent`/`stableTimeout`, and only it holds a + // capture binding to describe the surface with. A duration wait has neither. + const enrichedResponse = waitOperations + ? await maybeWaitTimeoutSurfaceResponse( + { req, logPath: params.logPath, session, device, capture: waitOperations.capture }, + response, + ) + : response; // Keep generic wait-surface details first so Android blocker detection can own the top-level message. return await maybeAndroidForegroundBlockerResponse(params, enrichedResponse, 'wait'); }; @@ -357,37 +368,6 @@ function readRecordedResolutionTarget( return { node: node as SnapshotNode, preActionNodes: preActionNodes as SnapshotNode[] }; } -async function dispatchDirectIosSelectorWait( - params: SelectorRuntimeParams & { - session: SessionState | undefined; - device: SessionState['device']; - selectorExpression: string; - timeoutMs: number | null; - }, -): Promise { - const selector = readSimpleIosSelectorTarget({ - session: params.session, - selectorExpression: params.selectorExpression, - }); - if (!params.session || !selector) return null; - const startedAt = Date.now(); - const result = await queryDirectIosSelectorOrFallback(params, params.session, selector); - if (isDirectIosSelectorErrorResult(result)) return result.response; - if (!result?.found) return null; - const payload = { - kind: 'selector', - selector: selector.raw, - waitedMs: Date.now() - startedAt, - selectorChain: [selector.raw], - }; - recordIfSession(params.sessionStore, params.sessionName, params.req, payload); - const response: DaemonResponse = { ok: true, data: stripSelectorChain(payload) }; - return await maybeWaitTimeoutSurfaceResponse( - { req: params.req, logPath: params.logPath, session: params.session, device: params.device }, - response, - ); -} - /** * The single querySelector client for the local XCTest runner: a live, * tree-independent read (and its found/text/node shape) for exactly one @@ -401,14 +381,10 @@ export async function queryDirectIosSelector( selector: Pick, requestOptions: AppleRunnerRequestOptions, ): Promise { - const data = await runAppleRunnerCommand( + const data = await queryAppleRunnerSelector( session.device, - { - command: 'querySelector', - selectorKey: selector.key, - selectorValue: selector.value, - appBundleId: session.appBundleId, - }, + selector, + session.appBundleId, requestOptions, ); const found = data.found === true; @@ -420,33 +396,6 @@ export async function queryDirectIosSelector( }; } -async function queryDirectIosSelectorOrFallback( - params: SelectorRuntimeParams, - session: SessionState, - selector: DirectIosSelectorTarget, -): Promise { - try { - return await queryDirectIosSelector( - session, - selector, - buildAppleRunnerRequestOptions({ - req: params.req, - logPath: params.logPath, - traceLogPath: session.trace?.outPath, - }), - ); - } catch (error) { - if (isDirectIosSelectorFallbackError(error, { allowElementNotFound: true })) return null; - return { kind: 'error', response: { ok: false, error: normalizeError(error) } }; - } -} - -function isDirectIosSelectorErrorResult( - result: DirectIosSelectorFallbackResult, -): result is DirectIosSelectorErrorResult { - return result !== null && 'kind' in result && result.kind === 'error'; -} - function readDirectIosSelectorNode(data: Record): SnapshotNode | undefined { const nodes = data.nodes; if (!Array.isArray(nodes)) return undefined; diff --git a/src/daemon/snapshot-runtime-binding.ts b/src/daemon/snapshot-runtime-binding.ts index c8e1e8410..50cda0ac2 100644 --- a/src/daemon/snapshot-runtime-binding.ts +++ b/src/daemon/snapshot-runtime-binding.ts @@ -1,8 +1,6 @@ import { resolveSnapshotRuntimePlan, type CaptureSnapshotInput, - type ElementTextRuntimeOperations, - type ReadTextAtPointInput, type RuntimeOperationFact, type SelectorCaptureRuntimePlan, type SnapshotResult, @@ -24,6 +22,13 @@ import { import { errorResponse } from './handlers/response.ts'; import { resolveSnapshotScope } from './handlers/snapshot-capture.ts'; import { resolveSessionDevice } from './handlers/snapshot-session.ts'; +import { + selectElementTextOperation, + selectWaitObservationOperations, + type BoundElementRead, + type BoundNativeSelectorRead, + type BoundNativeTextRead, +} from './selector-operation-binding.ts'; export type SnapshotRuntimeRouteParams = { req: DaemonRequest; @@ -49,18 +54,20 @@ type ResolvedSnapshotCaptureRuntime = export type BoundSnapshotCapture = (input: CaptureSnapshotInput) => Promise; /** The owner's live element read, when its facts advertise one. */ -export type BoundElementRead = ElementTextRuntimeOperations['readTextAtPoint']; - export type AdmittedSnapshotCapture = | Readonly<{ ok: true; capture: BoundSnapshotCapture; /** - * The owner's live element read, present only when the caller's plan declared it PREFERRED + * The owner's live element read, present only when the caller's plan declared it preferred * and the admitted owner advertised it. `snapshot`/`diff` plans declare no read, so this is * simply absent for them — the member is additive and they are unchanged. */ readTextAtPoint?: BoundElementRead; + /** A fact-conditional native text observation, present when the owner advertises it. */ + findText?: BoundNativeTextRead; + /** A fact-conditional one-sided simple-selector observation. */ + findSelector?: BoundNativeSelectorRead; }> | Readonly<{ ok: false; response: DaemonResponse }>; @@ -99,6 +106,8 @@ export async function admitAndBindSnapshotCapture( ok: true, capture: async (input: CaptureSnapshotInput) => await bound.captureSnapshot(input), ...(bound.readTextAtPoint ? { readTextAtPoint: bound.readTextAtPoint } : {}), + ...(bound.findText ? { findText: bound.findText } : {}), + ...(bound.findSelector ? { findSelector: bound.findSelector } : {}), }); } @@ -153,13 +162,15 @@ async function bindSnapshotCaptureRuntime( Readonly<{ captureSnapshot(input: CaptureSnapshotInput): Promise; readTextAtPoint?: BoundElementRead; + findText?: BoundNativeTextRead; + findSelector?: BoundNativeSelectorRead; }> > { const bind = requireRuntimeBinding(bindDevice); const { device, plan } = unwrapAdmittedRuntimePlan(admission); // One switch, one set of operation selectors. The selector arms reuse the SAME // `selectActiveAppSnapshot` / `selectSnapshotWithoutActiveApp` the snapshot arms use and only - // add the preferred element read; the discriminants differ solely so the compiler can narrow + // add selector operation projection; the discriminants differ solely so the compiler can narrow // `plan.use` per family. No parallel plan-to-operation dispatch is introduced. switch (plan.kind) { case 'active-app': { @@ -167,8 +178,7 @@ async function bindSnapshotCaptureRuntime( return selectActiveAppSnapshot(runtime); } case 'selector-active-app': { - const runtime = await bind(device, plan.use); - return { ...selectActiveAppSnapshot(runtime), ...selectElementRead(runtime) }; + return await bindActiveAppSelectorRuntime(bind, device, plan); } case 'custom-actions-active-app': { const runtime = await bind(device, plan.use); @@ -179,8 +189,7 @@ async function bindSnapshotCaptureRuntime( return selectSnapshotWithoutActiveApp(runtime); } case 'selector-without-active-app': { - const runtime = await bind(device, plan.use); - return { ...selectSnapshotWithoutActiveApp(runtime), ...selectElementRead(runtime) }; + return await bindSelectorRuntimeWithoutActiveApp(bind, device, plan); } case 'custom-actions-without-active-app': { const runtime = await bind(device, plan.use); @@ -189,26 +198,68 @@ async function bindSnapshotCaptureRuntime( } } -/** - * Projects the preferred element read when the admitted owner advertised it. A projection whose - * use never declared it simply has no such member, so this yields `{}` for `snapshot`/`diff`. - */ -function selectElementRead( - runtime: Readonly<{ operations: Readonly<{ readTextAtPoint?: BoundElementRead }> }>, -): Readonly<{ readTextAtPoint?: BoundElementRead }> { - const readTextAtPoint = runtime.operations.readTextAtPoint; - // Narrowed by CONSTRUCTION rather than by assertion: the projection below is only buildable - // from a non-undefined local, so presence is carried by the value that captured it. - return readTextAtPoint - ? { readTextAtPoint: bindElementRead({ operations: { readTextAtPoint } }) } - : {}; +type ActiveAppSelectorRuntimePlan = Extract< + SelectorCaptureRuntimePlan, + { kind: 'selector-active-app' } +>; + +async function bindActiveAppSelectorRuntime( + bind: BindDeviceRuntime, + device: SessionState['device'], + plan: ActiveAppSelectorRuntimePlan, +) { + switch (plan.intent) { + case 'capture-only': { + const runtime = await bind(device, plan.use); + return selectActiveAppSnapshot(runtime); + } + case 'element-text': { + const runtime = await bind(device, plan.use); + return { + ...selectActiveAppSnapshot(runtime), + ...selectElementTextOperation(runtime), + }; + } + case 'wait-observation': { + const runtime = await bind(device, plan.use); + return { + ...selectActiveAppSnapshot(runtime), + ...selectWaitObservationOperations(runtime), + }; + } + } } -/** The one lexical owner of the narrowed `readTextAtPoint` call. */ -function bindElementRead( - runtime: Readonly<{ operations: Readonly<{ readTextAtPoint: BoundElementRead }> }>, -): BoundElementRead { - return async (input: ReadTextAtPointInput) => await runtime.operations.readTextAtPoint(input); +type SelectorRuntimePlanWithoutActiveApp = Extract< + SelectorCaptureRuntimePlan, + { kind: 'selector-without-active-app' } +>; + +async function bindSelectorRuntimeWithoutActiveApp( + bind: BindDeviceRuntime, + device: SessionState['device'], + plan: SelectorRuntimePlanWithoutActiveApp, +) { + switch (plan.intent) { + case 'capture-only': { + const runtime = await bind(device, plan.use); + return selectSnapshotWithoutActiveApp(runtime); + } + case 'element-text': { + const runtime = await bind(device, plan.use); + return { + ...selectSnapshotWithoutActiveApp(runtime), + ...selectElementTextOperation(runtime), + }; + } + case 'wait-observation': { + const runtime = await bind(device, plan.use); + return { + ...selectSnapshotWithoutActiveApp(runtime), + ...selectWaitObservationOperations(runtime), + }; + } + } } type BoundSnapshotOperation = Readonly<{ diff --git a/src/daemon/snapshot-runtime-capture-input.ts b/src/daemon/snapshot-runtime-capture-input.ts index 3e8dd51f0..06f02ed33 100644 --- a/src/daemon/snapshot-runtime-capture-input.ts +++ b/src/daemon/snapshot-runtime-capture-input.ts @@ -23,6 +23,8 @@ export function buildRuntimeCaptureInput( * its first consumer; `snapshot`/`diff` pass nothing and are unaffected. */ includeRects?: boolean; + /** Per-capture cancellation for polling consumers; see `CaptureSnapshotInput.signal`. */ + signal?: AbortSignal; }>, ): CaptureSnapshotInput { const { flags, logPath, meta, session, snapshotScope } = params; @@ -49,6 +51,7 @@ export function buildRuntimeCaptureInput( surface, }, execution: runtimeExecutionFromContext(context), + ...(params.signal === undefined ? {} : { signal: params.signal }), }; } diff --git a/src/daemon/wait-conditional-selector.ts b/src/daemon/wait-conditional-selector.ts new file mode 100644 index 000000000..8c8841839 --- /dev/null +++ b/src/daemon/wait-conditional-selector.ts @@ -0,0 +1,63 @@ +import type { FindSelectorRuntimeOperations } from '@agent-device/contracts/platform'; +import type { TargetAnnotationV1 } from '@agent-device/contracts/replay'; +import { readSimpleSelectorTarget } from '@agent-device/selectors'; +import { getRequestSignal } from '../request/cancel.ts'; +import { buildRuntimeCaptureInput } from './snapshot-runtime-capture-input.ts'; +import { recordIfSession, stripSelectorChain } from './selector-recording.ts'; +import { isSessionRecording } from './session-script-publication-capability.ts'; +import type { SessionStore } from './session-store.ts'; +import type { DaemonRequest, DaemonResponse, SessionState } from './types.ts'; + +/** + * Tries the admitted owner's one-sided native observation for an immediate simple-selector wait. + * Any non-positive answer returns control to the required canonical capture in the same request. + */ +export async function dispatchConditionalWaitSelector( + input: Readonly<{ + selectorExpression: string; + operation: FindSelectorRuntimeOperations['findSelector'] | undefined; + recordedLandmark: TargetAnnotationV1 | undefined; + req: DaemonRequest; + session: SessionState | undefined; + sessionName: string; + sessionStore: SessionStore; + logPath: string | undefined; + signal?: AbortSignal; + }>, +): Promise { + if ( + !input.operation || + !input.session?.appBundleId || + input.recordedLandmark || + isSessionRecording(input.session) + ) { + return null; + } + const selector = readSimpleSelectorTarget(input.selectorExpression); + if (!selector) return null; + const startedAt = Date.now(); + const result = await input.operation({ + selector: { key: selector.key, value: selector.value }, + options: { + appBundleId: input.session.appBundleId, + surface: input.session.surface, + }, + execution: buildRuntimeCaptureInput({ + flags: input.req.flags, + logPath: input.logPath ?? '', + meta: input.req.meta, + session: input.session, + snapshotScope: undefined, + }).execution, + signal: input.signal ?? getRequestSignal(input.req.meta?.requestId), + }); + if (!result.found) return null; + const payload = { + kind: 'selector', + selector: selector.raw, + waitedMs: Date.now() - startedAt, + selectorChain: [selector.raw], + }; + recordIfSession(input.sessionStore, input.sessionName, input.req, payload); + return { ok: true, data: stripSelectorChain(payload) }; +} diff --git a/src/daemon/wait-current-surface.test.ts b/src/daemon/wait-current-surface.test.ts index be6a6b20f..299bda493 100644 --- a/src/daemon/wait-current-surface.test.ts +++ b/src/daemon/wait-current-surface.test.ts @@ -1,17 +1,28 @@ -import { beforeEach, expect, test, vi } from 'vitest'; - -const captureSnapshot = vi.hoisted(() => vi.fn()); - -vi.mock('./handlers/snapshot-capture.ts', () => ({ captureSnapshot })); +import { expect, test, vi } from 'vitest'; import { ANDROID_EMULATOR } from '../__tests__/test-utils/index.ts'; import { maybeWaitTimeoutSurfaceResponse } from './wait-current-surface.ts'; +import type { BoundSelectorCapture } from './selector-capture-binding.ts'; -beforeEach(() => { - captureSnapshot.mockReset(); -}); +const req = { + command: 'wait', + positionals: ['Agent Device Tester', '10000'], + session: 'android-e2e', + token: 'test-token', +}; + +/** + * The decoration capture rides wait's own request binding (ADR 0019), so the fake binds there — + * not on the `snapshot-capture` module — and "no capture started" is provable as "the binding + * was never invoked". + */ +function boundCapture(nodes: { index: number; depth: number; type: string; label?: string }[]) { + return vi.fn(async () => ({ nodes, backend: 'android' })) as unknown as BoundSelectorCapture & + ReturnType; +} test('deadline-truncated wait does not start a post-deadline diagnostic capture', async () => { + const capture = boundCapture([]); const response = { ok: false as const, error: { @@ -26,24 +37,16 @@ test('deadline-truncated wait does not start a post-deadline diagnostic capture' }; const result = await maybeWaitTimeoutSurfaceResponse( - { - req: { - command: 'wait', - positionals: ['Agent Device Tester', '10000'], - session: 'android-e2e', - token: 'test-token', - }, - session: undefined, - device: ANDROID_EMULATOR, - }, + { req, session: undefined, device: ANDROID_EMULATOR, capture }, response, ); expect(result).toBe(response); - expect(captureSnapshot).not.toHaveBeenCalled(); + expect(capture).not.toHaveBeenCalled(); }); test('wait surface decoration requires a structured wait timeout reason', async () => { + const capture = boundCapture([]); const response = { ok: false as const, error: { @@ -53,19 +56,42 @@ test('wait surface decoration requires a structured wait timeout reason', async }; const result = await maybeWaitTimeoutSurfaceResponse( - { - req: { - command: 'wait', - positionals: ['Agent Device Tester', '10000'], - session: 'android-e2e', - token: 'test-token', - }, - session: undefined, - device: ANDROID_EMULATOR, - }, + { req, session: undefined, device: ANDROID_EMULATOR, capture }, response, ); expect(result).toBe(response); - expect(captureSnapshot).not.toHaveBeenCalled(); + expect(capture).not.toHaveBeenCalled(); +}); + +test('an absent-target wait describes the current surface through its own request binding', async () => { + const capture = boundCapture([ + { index: 0, depth: 0, type: 'TextView', label: 'Checkout' }, + { index: 1, depth: 1, type: 'Button', label: 'Pay now' }, + ]); + const response = { + ok: false as const, + error: { + code: 'COMMAND_FAILED' as const, + message: 'wait timed out for text: Agent Device Tester', + details: { reason: 'wait_target_absent', timeoutMs: 10_000 }, + }, + }; + + const result = await maybeWaitTimeoutSurfaceResponse( + { req, session: undefined, device: ANDROID_EMULATOR, capture }, + response, + ); + + expect(capture).toHaveBeenCalledTimes(1); + expect(capture.mock.calls[0]?.[0]).toMatchObject({ + options: { interactiveOnly: true }, + }); + expect(result.ok).toBe(false); + if (result.ok) return; + expect(result.error.message).toContain('Current surface: Checkout, Pay now'); + expect(result.error.details?.currentSurface).toMatchObject({ + labels: ['Checkout', 'Pay now'], + buttons: ['Pay now'], + }); }); diff --git a/src/daemon/wait-current-surface.ts b/src/daemon/wait-current-surface.ts index 3bfc0e119..1ea7098d6 100644 --- a/src/daemon/wait-current-surface.ts +++ b/src/daemon/wait-current-surface.ts @@ -4,12 +4,19 @@ import type { DaemonRequest, DaemonResponse, SessionState } from './types.ts'; import { captureSnapshot } from './handlers/snapshot-capture.ts'; import { errorResponse } from './handlers/response.ts'; import { normalizeType } from '@agent-device/contracts/snapshot'; +import { buildRuntimeCaptureInput } from './snapshot-runtime-capture-input.ts'; +import type { BoundSelectorCapture } from './selector-capture-binding.ts'; type WaitCurrentSurfaceParams = { req: DaemonRequest; logPath?: string; session: SessionState | undefined; device: SessionState['device']; + /** + * The wait's own request binding. The decoration capture is wait's platform execution too, so + * it reuses the single admitted binding rather than reaching a second capture owner. + */ + capture: BoundSelectorCapture; }; type CurrentSurfaceDetails = { @@ -53,14 +60,25 @@ function canInspectWaitSurface(reason: unknown): boolean { async function inspectCurrentSurface( params: WaitCurrentSurfaceParams, ): Promise<{ summary: string; details: CurrentSurfaceDetails } | null> { + const flags = { + ...params.req.flags, + snapshotInteractiveOnly: true, + }; const capture = await captureSnapshot({ device: params.device, session: params.session, - flags: { - ...params.req.flags, - snapshotInteractiveOnly: true, - }, + flags, logPath: params.logPath ?? '', + captureData: async () => + await params.capture( + buildRuntimeCaptureInput({ + flags, + logPath: params.logPath ?? '', + meta: params.req.meta, + session: params.session, + snapshotScope: undefined, + }), + ), }); const orderedNodes = [...capture.snapshot.nodes].sort(compareSurfacePriority); const labels = topSurfaceTexts(orderedNodes, 6, { includeIdentifiers: true }); diff --git a/src/platforms/apple/__tests__/interactor-runner-provider.test.ts b/src/platforms/apple/__tests__/interactor-runner-provider.test.ts index dc222fdd7..5c2f8a73e 100644 --- a/src/platforms/apple/__tests__/interactor-runner-provider.test.ts +++ b/src/platforms/apple/__tests__/interactor-runner-provider.test.ts @@ -47,6 +47,11 @@ const RUNNER_TRANSPORT_METHODS: Record< invoke: (i) => i.readTextAtPoint!({ x: 10, y: 20 }), runnerCommand: 'readText', }, + findText: { invoke: (i) => i.findText!('Ready'), runnerCommand: 'findText' }, + findSelector: { + invoke: (i) => i.findSelector!({ key: 'id', value: 'ready' }), + runnerCommand: 'querySelector', + }, back: { invoke: (i) => i.back(), runnerCommand: 'backInApp' }, home: { invoke: (i) => i.home(), runnerCommand: 'home' }, setOrientation: { invoke: (i) => i.setOrientation('portrait'), runnerCommand: 'rotate' }, diff --git a/src/platforms/apple/core/__tests__/runner-selector-query.test.ts b/src/platforms/apple/core/__tests__/runner-selector-query.test.ts new file mode 100644 index 000000000..4f7a32f0b --- /dev/null +++ b/src/platforms/apple/core/__tests__/runner-selector-query.test.ts @@ -0,0 +1,33 @@ +import { expect, test, vi } from 'vitest'; +import { IOS_SIMULATOR } from '../../../../__tests__/test-utils/device-fixtures.ts'; + +const { runAppleRunnerCommand } = vi.hoisted(() => ({ + runAppleRunnerCommand: vi.fn(async () => ({ found: true })), +})); + +vi.mock('../runner/runner-client.ts', () => ({ runAppleRunnerCommand })); + +import { queryAppleRunnerSelector } from '../runner/runner-selector-query.ts'; + +test('builds the one querySelector runner command for every caller', async () => { + const options = { requestId: 'request-1' }; + await expect( + queryAppleRunnerSelector( + IOS_SIMULATOR, + { key: 'id', value: 'submit' }, + 'com.example.app', + options, + ), + ).resolves.toEqual({ found: true }); + + expect(runAppleRunnerCommand).toHaveBeenCalledWith( + IOS_SIMULATOR, + { + command: 'querySelector', + selectorKey: 'id', + selectorValue: 'submit', + appBundleId: 'com.example.app', + }, + options, + ); +}); diff --git a/src/platforms/apple/core/runner/runner-selector-query.ts b/src/platforms/apple/core/runner/runner-selector-query.ts new file mode 100644 index 000000000..e1115131e --- /dev/null +++ b/src/platforms/apple/core/runner/runner-selector-query.ts @@ -0,0 +1,23 @@ +import type { ElementSelectorKey } from '@agent-device/contracts/interaction'; +import type { DeviceInfo } from '@agent-device/kernel/device'; +import { runAppleRunnerCommand } from './runner-client.ts'; +import type { AppleRunnerCommandOptions } from './runner-provider.ts'; + +/** The single Apple runner `querySelector` command builder shared by bound and safety-probe reads. */ +export async function queryAppleRunnerSelector( + device: DeviceInfo, + selector: Readonly<{ key: ElementSelectorKey; value: string }>, + appBundleId: string | undefined, + options: AppleRunnerCommandOptions, +): Promise> { + return await runAppleRunnerCommand( + device, + { + command: 'querySelector', + selectorKey: selector.key, + selectorValue: selector.value, + appBundleId, + }, + options, + ); +} diff --git a/src/platforms/apple/interactor.ts b/src/platforms/apple/interactor.ts index 8fc8a9bcb..62b44ef15 100644 --- a/src/platforms/apple/interactor.ts +++ b/src/platforms/apple/interactor.ts @@ -12,6 +12,7 @@ import { iosRunnerOverrides, resolveAppleBackRunnerCommand } from './interaction import { appleRemotePressCommand } from './os/tvos/remote.ts'; import { runMacOsScreenshotAction } from './os/macos/helper.ts'; import { runAppleRunnerCommand } from './core/runner/runner-client.ts'; +import { queryAppleRunnerSelector } from './core/runner/runner-selector-query.ts'; import { withAppleRunnerProvider, type AppleRunnerCommandExecutor, @@ -72,6 +73,26 @@ export function createAppleInteractor( usesMacOsHelperSurface(device, options?.surface) ? await readMacOsSurfaceTextAtPoint(point, options) : await readRunnerTextAtPoint(device, point, options, runnerOpts), + // The XCTest runner's own text reading: it observes the live accessibility hierarchy + // directly, so it answers without the cost — and without the pruning — of a tree capture. + // Only a positive answer is authoritative; see `FindTextResult`. + findText: async (text, options) => { + const result = (await runAppleRunnerCommand( + device, + { command: 'findText', text, appBundleId: options?.appBundleId }, + options?.signal ? { ...runnerOpts, signal: options.signal } : runnerOpts, + )) as { found?: boolean }; + return { found: result?.found === true }; + }, + findSelector: async (selector, options) => { + const result = await queryAppleRunnerSelector( + device, + selector, + options?.appBundleId, + options?.signal ? { ...runnerOpts, signal: options.signal } : runnerOpts, + ); + return { found: result.found === true }; + }, back: async (mode) => { if (isTvOsDevice(device)) { // tvOS focus-only navigation: the Menu button pops focus, not a coordinate tap. diff --git a/test/integration/provider-scenarios/interaction-direct-selector-fallback.test.ts b/test/integration/provider-scenarios/interaction-direct-selector-fallback.test.ts index 3e1e45663..b38f08f62 100644 --- a/test/integration/provider-scenarios/interaction-direct-selector-fallback.test.ts +++ b/test/integration/provider-scenarios/interaction-direct-selector-fallback.test.ts @@ -135,7 +135,10 @@ async function withDirectSelectorScenario( ); } -test('Provider-backed direct iOS selector wait strips selectorChain from the public response', async () => { +// `wait` asks the admitted Apple owner first for simple selector existence. A positive +// owner observation avoids a sparse canonical tree; a miss still falls through to the +// request-bound capture. The public response strips the internal `selectorChain` either way. +test('Provider-backed iOS selector wait accepts the owner observation and strips selectorChain', async () => { const transcript = createProviderTranscript([ { command: 'ios.runner.querySelector', @@ -147,14 +150,48 @@ test('Provider-backed direct iOS selector wait strips selectorChain from the pub selectorValue: 'Continue', appBundleId: APP, }, - result: { found: true, node: { label: 'Continue' } }, + result: { found: true, nodes: [] }, }, ]); await withDirectSelectorScenario(transcript, async (daemon) => { const wait = await daemon.callCommand('wait', ['label="Continue"']); const data = assertRpcOk(wait); - assert.equal(data.kind, 'selector'); + assert.equal(data.selector, 'label="Continue"'); + assert.equal('selectorChain' in data, false); + }); +}); + +test('Provider-backed iOS selector wait falls through to capture after an owner miss', async () => { + const transcript = createProviderTranscript([ + { + command: 'ios.runner.querySelector', + deviceId: DEVICE_ID, + platform: 'apple', + request: { + command: 'querySelector', + selectorKey: 'label', + selectorValue: 'Continue', + appBundleId: APP, + }, + result: { found: false, nodes: [] }, + }, + snapshotEntry([ + APPLICATION_NODE, + { + index: 1, + parentIndex: 0, + type: 'Button', + label: 'Continue', + hittable: true, + rect: { x: 100, y: 300, width: 200, height: 44 }, + }, + ]), + ]); + + await withDirectSelectorScenario(transcript, async (daemon) => { + const wait = await daemon.callCommand('wait', ['label="Continue"']); + const data = assertRpcOk(wait); assert.equal(data.selector, 'label="Continue"'); assert.equal('selectorChain' in data, false); });