Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -451,7 +451,7 @@ Refs are numbered in tree order each time the daemon reads the screen (`describe

On mobile, after a mutating or batch call, the daemon may take the snapshot again after 300 ms: while it shows a busy element (a label ending in `, busy`, an Android `ProgressBar`, or an iOS `ActivityIndicator` / `ProgressIndicator` / `UIActivityIndicatorView`), and at most once when it still equals the previous screen (not when the first snapshot took 1 s or more: the screen already had that time to change). In total at most 3 extra snapshots, and no wait that would end more than about 2 s after the first snapshot started. The test IDs come from the same snapshot as the tree. `describe_screen` on mobile also takes one snapshot for both.

On mobile, each observation, `describe_screen`, `list_testids` and `accessibility_snapshot` takes one device snapshot (plus the settle retries above). The viewport for `visible` comes from the snapshot's root frames; the device window size is read at most once per session (and not refreshed after a rotation), only when a snapshot has no root frame. Elements that are not part of the app are removed from the tree, the refs, the test IDs and `generate_locators`: on Android, the system status bar and navigation bar windows (a full-width or full-height bar window whose ids are all `com.android.systemui:id/*`; other System UI windows such as the biometric prompt stay); on both platforms, the Expo DevLauncher "Tools" button of development builds (iOS `gearshape.fill` and its "Tools" text). Pass `{ excludeDevOverlay: false }` as the third `MobilePlatformDriver` argument to keep the DevLauncher button. Taps and waits inside `@metamask/device-mcp` still search the whole snapshot.
On mobile, each observation, `describe_screen`, `list_testids` and `accessibility_snapshot` takes one device snapshot (plus the settle retries above). The consumer's `DeviceBackend` takes the snapshot. With the `@metamask/device-mcp` ADB backend (0.6.0 or later), Android snapshots use a signed test-only helper APK by default (`helper-first`). The backend installs it on the device the first time, so the first snapshot takes longer. To use only `uiautomator dump`, set `DEVICE_MCP_ADB_SNAPSHOT=dump` or the backend's `adb.snapshotMode` option. See the device-mcp README. The viewport for `visible` comes from the snapshot's root frames; the device window size is read at most once per session (and not refreshed after a rotation), only when a snapshot has no root frame. Elements that are not part of the app are removed from the tree, the refs, the test IDs and `generate_locators`: on Android, the system status bar and navigation bar windows (a full-width or full-height bar window whose ids are all `com.android.systemui:id/*`; other System UI windows such as the biometric prompt stay); on both platforms, the Expo DevLauncher "Tools" button of development builds (iOS `gearshape.fill` and its "Tools" text). Pass `{ excludeDevOverlay: false }` as the third `MobilePlatformDriver` argument to keep the DevLauncher button. Taps and waits inside `@metamask/device-mcp` still search the whole snapshot.

The mobile accessibility tree (`describe_screen`, `accessibility_snapshot`, observations) is a flat list in pre-order with no `path` (on the browser, `path` holds the `role:name` of the ancestor dialogs and headings; on mobile the ancestors are native view classes). Layout-only elements are left out (`pruneLayoutOnly` in `src/platform/mobile-hierarchy.ts`): an element with no label, no value, no identifier, and a type that is not a control or a progress indicator. Its kept descendants take its place, in order. Controls are iOS `Button`, `Link`, `TextField`, `SecureTextField`, `SearchField`, `TextView`, `TextArea`, `Switch`, `Toggle`, `Slider`, `Stepper`, `Picker`, `Cell`, `Tab`, `MenuItem` and similar AX types (exact match, without the `XCUIElementType` prefix), and Android classes ending in `Button`, `EditText`, `CheckBox`, `CheckedTextView`, `Switch`, `SeekBar`, `Spinner`, `RatingBar` and similar. The device snapshot has no `clickable` flag, so a layout-only plain React Native view (`RCTViewComponentView` on iOS, `android.view.ViewGroup` on Android) that has no kept children and is at least 24×24 (iOS points, Android pixels) stays as an unlabelled tap target, for example the iOS wallet home action buttons. Node `bounds` are `{ x, y, width, height }`, in that key order. Node refs number the kept nodes only, and ambiguity is computed over them. Only nodes with a testId, label or value have a refMap entry, so only their refs can be targeted; unlabelled tap targets, progress indicators and unnamed controls are listed with `bounds` (use `tap_coordinates`) but their refs do not resolve. On real MetaMask screens: iOS wallet home 177 → 67 nodes (183 kB with `path` → 10 kB), Android wallet home 625 → 242 nodes (649 kB → 35 kB), Android login 47 → 15 nodes. `list_testids` and `generate_locators` are not affected, and `describe_screen` still returns the full `testIds` list next to the tree.

Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@
"hono": "^4.12.25"
},
"dependencies": {
"@metamask/device-mcp": "^0.5.0",
"@metamask/device-mcp": "^0.6.0",
"cosmiconfig": "^9.0.0",
"express": "^5.2.1",
"zod": "^4.4.3"
Expand Down
10 changes: 5 additions & 5 deletions yarn.lock
Original file line number Diff line number Diff line change
Expand Up @@ -926,7 +926,7 @@ __metadata:
"@lavamoat/allow-scripts": "npm:^3.0.4"
"@lavamoat/preinstall-always-fail": "npm:^2.0.0"
"@metamask/auto-changelog": "npm:^5.3.0"
"@metamask/device-mcp": "npm:^0.5.0"
"@metamask/device-mcp": "npm:^0.6.0"
"@metamask/eslint-config": "npm:^15.0.0"
"@metamask/eslint-config-nodejs": "npm:^15.0.0"
"@metamask/eslint-config-typescript": "npm:^15.0.0"
Expand Down Expand Up @@ -968,15 +968,15 @@ __metadata:
languageName: unknown
linkType: soft

"@metamask/device-mcp@npm:^0.5.0":
version: 0.5.0
resolution: "@metamask/device-mcp@npm:0.5.0"
"@metamask/device-mcp@npm:^0.6.0":
version: 0.6.0
resolution: "@metamask/device-mcp@npm:0.6.0"
dependencies:
"@modelcontextprotocol/sdk": "npm:^1.12.1"
zod: "npm:^4.4.3"
bin:
device-mcp: ./dist/cli/device-mcp.mjs
checksum: 10/130003f592927469d5297baade1cdd9ae98c5d2b9ae37305fdb662735c3cc3764315cbeb60802e1378d578154238016375532ada61c2381d5af093cd6455a9ce
checksum: 10/5b9443598050734b8ba6569fe460e5ecad0f8f9301464d22cdcc72561df48d524809ecaa7ec1dab674eabd44ea24e69e36f13b453bb312dc114c7b4b332e92b9
languageName: node
linkType: hard

Expand Down
Loading