feat: cancel timed-out tasks; late ref writes cannot overwrite newer refs

README.md

@@ -366,9 +366,13 @@ type ToolContext = {
   get refMap(): Map<string, string>; // lazy — returns empty map if no session
   workflowContext: WorkflowContext;
   toolRegistry: Map<string, ToolFunction<unknown, unknown>>;
+  driver?: IPlatformDriver;
+  signal?: AbortSignal; // aborted when the request times out
 };
 ```
 
+Tools check `signal?.aborted` between steps and stop. In the daemon, `context.sessionManager.setRefMap` goes through a guard (`RefStore`) that drops a write from a request that timed out or was replaced by a newer one, so a cancelled call cannot overwrite the refs of later calls.
+
 The daemon routes `POST /tool/:name` requests through the registry, applies Zod validation on inputs, executes the tool through the request queue, and captures observations (test IDs, a11y snapshot) according to the call's `observe` level (see [Observations](#observations)).
 
 **Registered tools:**
@@ -443,6 +447,10 @@ mm type e3 "hello"    → types into the element mapped to e3
 
 This accessibility-first approach provides reliable element targeting that survives minor UI changes.
 
+Refs are numbered in tree order each time the daemon reads the screen (`describe_screen`, `accessibility_snapshot`, or an observation). The same screen gives the same refs, so an unchanged screen diffs as unchanged. When elements appear or disappear above an element, its ref can change: use the refs from the latest response. When a call returns both a tree and an observation (for example `describe_screen` with `observe`), the observation's refs are the ones that resolve. `list_testids` does not change refs.
+
+On mobile, after a mutating or batch call, the daemon may take the snapshot again after 300 ms: while it shows a busy element (a label ending in `, busy`, an Android `ProgressBar`, or an iOS `ActivityIndicator` / `ProgressIndicator` / `UIActivityIndicatorView`), and at most once when it still equals the previous screen. In total at most 3 extra snapshots and about 2 s. The test IDs come from the same snapshot as the tree. `describe_screen` on mobile also takes one snapshot for both.
+
 ### Session Metadata
 
 Consumers can persist launch metadata with `writeSessionMetadata(artifactsDir, metadata)`. It writes `<artifactsDir>/<sessionId>/session.json` (for example `test-artifacts/<sessionId>/session.json` with `artifactsDir: 'test-artifacts'`) and returns the file path.
@@ -541,6 +549,8 @@ For `run_steps`, the top-level `observe` sets the level of the one observation c
 
 Observations contain `testIds` and `a11y` only. They do not include app state; call `get_state` for that.
 
+A call that times out is cancelled: its signal is aborted, it stops at its next step, it collects no observation, and its refs are dropped.
+
 ## CLI Reference
 
 The `mm` CLI provides a unified interface for agents and developers. All commands communicate with the daemon over HTTP — the daemon is auto-started on `mm launch` if not already running.
@@ -795,6 +805,7 @@ Tool errors are classified into specific error codes for structured handling:
 | `MM_PORT_IN_USE`                    | Port already in use                                                                                     |
 | **System**                          |                                                                                                         |
 | `MM_UNKNOWN_TOOL`                   | Unknown tool name                                                                                       |
+| `MM_REQUEST_ABORTED`                | Internal: a cancelled (timed-out) call stopped; the caller gets the timeout error                       |
 | `MM_INTERNAL_ERROR`                 | Internal server error                                                                                   |
 | `MM_BATCH_TIMEOUT`                  | `batchTimeoutMs` deadline exceeded in run_steps                                                         |
 | `MM_CDP_BLOCKED`                    | CDP method is blocked (destructive to session); on mobile the Hermes blocklist maps here too            |

SKILL.md

@@ -18,7 +18,7 @@ mm cleanup --shutdown      # 5. Clean up when done
 
 **Critical rules:**
 
-- **Always `describe-screen` before interacting.** Refs like `e1`, `e2` are ephemeral — they change after every action.
+- **Always `describe-screen` before interacting.** Refs like `e1`, `e2` are ephemeral — use the refs from the latest response (`describe-screen` or the `observations` of your last action).
 - **Always `describe-screen` after interacting** — OR use inline `observations` from mutating tool responses. Mutating tools (click, type, navigate, etc.) return an `observations` object with fresh `testIds` and `a11y` refs. You can use these refs directly for the next interaction without calling `describe-screen`. Call `describe-screen` when you need the full tree or screenshots, and `get-state` when you need app state (observations carry none).
 - **One target per command.** Specify exactly ONE of: a11y ref (`e5`), testId, or CSS selector.
 - **Errors are structured.** Check the `error.code` field to decide recovery strategy (see Error Codes below).
@@ -85,7 +85,9 @@ After a mutating action, the response includes fresh screen data:
 }
 ```
 
-You can use the `ref` values from `observations.a11y.nodes` for the next interaction — no `describe-screen` needed. Note that refs in compacted observations may be summary nodes (e.g., `"55 options (refs e2–e56)"`) when there are 3+ options under a combobox or listbox.
+You can use the `ref` values from `observations.a11y.nodes` for the next interaction — no `describe-screen` needed. Refs are numbered in screen order: on an unchanged screen they stay the same, but when elements appear or disappear above an element its ref can change. With `--observe diff`, those elements are listed in the diff with their new refs, so always take refs from the latest response. Note that refs in compacted observations may be summary nodes (e.g., `"55 options (refs e2–e56)"`) when there are 3+ options under a combobox or listbox.
+
+**Timeouts cancel the call:** when a call times out (`Tool execution timed out`), it is cancelled. It stops at its next step and cannot overwrite the refs of later calls. The action itself may still have happened, so run `describe-screen` before retrying.
 
 **Quick reference:**
 

src/index.ts

@@ -10,6 +10,7 @@ export type {
   TypeActionResult,
   GetTextActionResult,
   PlatformScreenshotOptions,
+  ScreenCapture,
 } from './platform';
 export { PlaywrightPlatformDriver, MobilePlatformDriver } from './platform';
 

src/platform/index.ts

@@ -7,6 +7,7 @@ export type {
   PlatformScreenshotOptions,
   WithinScope,
   IPlatformDriver,
+  ScreenCapture,
 } from './types.js';
 
 export { PlaywrightPlatformDriver } from './playwright-driver.js';

src/platform/mobile-platform-driver.test.ts

@@ -71,6 +71,52 @@ describe('MobilePlatformDriver', () => {
     vi.restoreAllMocks();
   });
 
+  describe('captureScreen', () => {
+    it('returns the tree and the test IDs of one snapshot', async () => {
+      const backend = createMockBackend({
+        getWindowSize: vi.fn().mockResolvedValue({ width: 402, height: 874 }),
+        snapshot: vi
+          .fn()
+          .mockResolvedValueOnce({
+            platform: 'ios',
+            hierarchy: [
+              makeElement({ identifier: 'login', label: 'Unlock' }),
+              makeElement({
+                identifier: 'below',
+                type: 'StaticText',
+                frame: { x: 0, y: 2000, width: 10, height: 10 },
+              }),
+            ],
+            raw: '[]',
+            timestamp: Date.now(),
+          })
+          .mockResolvedValueOnce({
+            platform: 'ios',
+            hierarchy: [makeElement({ identifier: 'wallet' })],
+            raw: '[]',
+            timestamp: Date.now(),
+          }),
+      });
+      const driver = new MobilePlatformDriver(backend);
+
+      const screen = await driver.captureScreen();
+      const testIds = await screen.getTestIds();
+
+      expect(backend.snapshot).toHaveBeenCalledTimes(1);
+      expect(screen.nodes.map((node) => node.testId)).toStrictEqual([
+        'login',
+        'below',
+      ]);
+      expect(screen.refMap.get('e1')).toBe('identifier:login');
+      // Same items, order and visibility as getTestIds on that snapshot.
+      expect(testIds).toStrictEqual([
+        { testId: 'login', tag: 'Button', text: 'Unlock', visible: true },
+        { testId: 'below', tag: 'StaticText', text: undefined, visible: false },
+      ]);
+      expect(await screen.getTestIds(1)).toHaveLength(1);
+    });
+  });
+
   describe('getPlatform', () => {
     it('returns ios for ios backend', () => {
       const driver = new MobilePlatformDriver(createMockBackend());

src/platform/mobile-platform-driver.ts

@@ -22,6 +22,7 @@ import type {
   GetTextActionResult,
   PlatformScreenshotOptions,
   PlatformType,
+  ScreenCapture,
   WithinScope,
 } from './types.js';
 import type {
@@ -208,6 +209,33 @@ export class MobilePlatformDriver implements IPlatformDriver {
    */
   async getTestIds(limit?: number): Promise<TestIdItem[]> {
     const snapshot = await this.#backend.snapshot();
+    return this.#testIdsOf(snapshot.hierarchy, limit);
+  }
+
+  /**
+   * Takes one device snapshot and returns its accessibility tree and test
+   * IDs, so both describe the same screen.
+   *
+   * @returns The nodes, refMap and a test ID reader for the same snapshot.
+   */
+  async captureScreen(): Promise<ScreenCapture> {
+    const snapshot = await this.#backend.snapshot();
+    return {
+      ...normalizeSnapshot(snapshot.hierarchy),
+      getTestIds: async (limit?: number) =>
+        this.#testIdsOf(snapshot.hierarchy, limit),
+    };
+  }
+
+  /**
+   * @param hierarchy - The UIElement tree of a snapshot.
+   * @param limit - Maximum number of test IDs to return.
+   * @returns The test IDs, with visibility against the device viewport.
+   */
+  async #testIdsOf(
+    hierarchy: UIElement[],
+    limit?: number,
+  ): Promise<TestIdItem[]> {
     const items: TestIdItem[] = [];
     const max = limit ?? OBSERVATION_TESTID_LIMIT;
     let viewport: { width: number; height: number } | undefined;
@@ -217,7 +245,7 @@ export class MobilePlatformDriver implements IPlatformDriver {
       // Without a viewport we cannot judge visibility; report everything.
       viewport = undefined;
     }
-    collectTestIds(snapshot.hierarchy, items, max, viewport);
+    collectTestIds(hierarchy, items, max, viewport);
     return items;
   }
 

src/platform/types.ts

@@ -48,6 +48,21 @@ export type WithinScope = {
   value: string;
 };
 
+/**
+ * One snapshot of the screen: the accessibility tree, and the test IDs of the
+ * same snapshot.
+ */
+export type ScreenCapture = {
+  nodes: A11yNodeTrimmed[];
+  refMap: Map<string, string>;
+  /**
+   * Lists the test IDs of this snapshot (no new snapshot).
+   *
+   * @param limit - Maximum number of test IDs to return.
+   */
+  getTestIds(limit?: number): Promise<TestIdItem[]>;
+};
+
 export type IPlatformDriver = {
   click(
     targetType: TargetType,
@@ -88,6 +103,13 @@ export type IPlatformDriver = {
 
   getTestIds(limit?: number): Promise<TestIdItem[]>;
 
+  /**
+   * Takes one snapshot and returns both the accessibility tree and its test
+   * IDs, so they describe the same screen. Optional: without it, callers use
+   * `getAccessibilityTree` and `getTestIds` (two snapshots).
+   */
+  captureScreen?(): Promise<ScreenCapture>;
+
   screenshot(options: PlatformScreenshotOptions): Promise<ScreenshotResult>;
 
   getAppState(): Promise<ExtensionState>;