Add XCUITest flow interpreter for iOS by j0ntz · Pull Request #5 · EdgeApp/edge-dev-agents

j0ntz · 2026-10-01T01:03:48Z

Description

Adds a native XCUITest flow interpreter and makes it the default iOS driver for build-and-test. Android stays on Maestro.

Runner (build-and-test/xcuitest/EdgeFlowRunner, scripts/xcuitest-build.sh, scripts/xcuitest-run.sh):

One generic XCUITest bundle interprets Maestro flow YAML at run time. maestro-yaml-to-json.rb converts the flow (inlining runFlow and retry files) and the JSON reaches the runner through a TEST_RUNNER_ env var. Each run is one xcodebuild test-without-building.
The build is cached per Xcode build, iOS runtime and source hash under ~/Library/Caches/edge-flow-runner. A mkdir lock lets concurrent slots share one build.
Each run keeps its JSON, log and result bundle under TMPDIR keyed by UDID. One xcodebuild per UDID, no host port. The slot's maestro MCP daemon is stopped by PID, never by a broad pkill.
Selectors keep Maestro semantics: id matches testID, text matches label, value or placeholder, both as whole-string case-insensitive regexes. index orders on-screen matches.
${...} and evalScript run in JavaScriptCore, so expressions are real JavaScript.
waitForAnimationToEnd waits for two consecutive identical screenshots.
Preflight walks the whole flow tree and exits 2 on any unsupported command, argument or selector key, naming the flow and step. There is no Maestro fallback.
A swizzle on XCUIApplicationProcess caps XCUITest's quiescence waits: default 1s, --quiescence-cap N per run, env EDGE_QUIESCENCE_CAP per flow, 0 skips the wait. Each hit prints a quiescence cap hit line naming the step, and the last line counts them.
--animations off|fast|on (default off) passes -EdgeTestAnimations on launchApp for the edge-react-gui test-mode switch.

Routing: the build-and-test rule ios-flows-run-on-xcuitest makes xcuitest-run.sh the default iOS runner. Maestro runs iOS only when a task asks for it or preflight rejects a command. capture-buy-quote.sh gains --driver xcuitest|maestro (default xcuitest). references/xcuitest-interpreter.md documents the runner.

Library flow fixes, found while driving all 12 flows natively. Each one is also a latent Maestro race:

Retry the Buy tab tap after login, and retry the ramp region tap when the keyboard takes the first tap.
select-swap-pair picks wallets by the picker row testID walletListRow.<name>.<code>.
Gate PIN digit taps on the PIN screen being visible, in login and buy-quote.
Clear the wallet search and the buy amount before typing. Both remember the last value.

Orch flow coverage (second commit). The target is every flow an orch run drives. A preflight survey of the agent task flows under /tmp rejected about half of them on tapOn point: and hideKeyboard. This commit adds:

tapOn point: in both Maestro forms, percentages ("50%,80%") and absolute points ("120,640"), through a normalized XCUICoordinate on the app.
hideKeyboard, following Maestro 2.6.1 on iOS: nothing when no keyboard is up, otherwise a short swipe up from the screen center, then a short swipe left if the keyboard is still there.
longPressOn (3s, selector or point:), assertVisible with enabled:, copyTextFrom and pasteText (the copied text is also maestro.copiedText in JavaScriptCore), inputRandomText, and scrollUntilVisible with centerElement:.
back and pressKey: back are accepted and do nothing, as on Maestro iOS.
One deviation from Maestro in centerElement: an element that is on screen but outside the center band is dragged by its own distance from the screen center. Maestro repeats its full swipe, which carried a wallet row past the band and off screen.

launchApp clearState: true stays rejected on purpose because it wipes the roster accounts. Porting the edge-react-gui maestro/ suite is not a goal.

Dependencies

EdgeApp/edge-react-gui#6232 adds the EdgeTestAnimations switch. The runner works without it; --animations then has no effect.

Testing

iOS simulator, edge-react-gui debug build at 2efa138b0 plus the gui PR. Default cap 1s, animations off.

All 12 library flows pass with 0 quiescence cap hits:

Flow	Flow time (s)	Wall incl. startup (s)
login-if-needed	7.79	16
dismiss-startup-modals	8.13	12
find-wallet	14.77	19
open-settings	1.76	6
ramp-set-region-fiat	16.60	21
buy-quote-input	19.51	24
buy-quote	23.52	27
select-swap-pair	48.18
confirm-slider	6.79
send-to-address	37.62
create-throwaway	59.92
delete-throwaway	23.17

The last five ran inside a swap drive, a send drive and a throwaway create/delete drive. The swap and the send were real mainnet transactions.

Wall time vs Maestro (find-wallet for Zcash, startup included): Maestro CLI 47.8s, interpreter 24.5s.
Concurrency: two flows on two sims at once, sharing one cached build: 25s and 38s wall, both pass, 0 cap hits.
Quiescence cap: no library flow hit it. A scratch flow hit it once on a swipe step, and open-settings with --quiescence-cap 0.05 hit it once with the step logged.
Animations: on the Dev Tab's permanent spinner, waitForAnimationToEnd settles in 0.16s with off and runs to its 5s timeout with on. login-if-needed took 7.86s on / 7.82s off and open-settings 2.01s on / 2.09s off, all passing with 0 cap hits. Spinners never hold the quiescence wait: taps beside one take 1.5s in either mode.
verify-repo passes.

Orch flow coverage commit, iOS simulator, edge-react-gui debug build at b50957606:

Preflight survey: FlowPreflight compiled into a CLI and run over every agent task flow under /tmp: 302 flows, 302 pass, 0 rejected. A flow with an unknown command is still rejected.
One flow per new command, each driven on the simulator to its terminal state:

Flow	Commands	Terminal state	Flow time (s)
t1-tap-point	`tapOn point:` percent and absolute	tab bar switches scene on each tap	22.20
t2-keyboard	`inputRandomText`, `copyTextFrom`, `pasteText`, `hideKeyboard`	copied text pasted into the search field, keyboard gone	20.19
t3-longpress	`scrollUntilVisible centerElement:`, `longPressOn`	wallet row menu open	26.31
t4-enabled-center	`assertVisible enabled:` true and false, `centerElement:`	disabled Save button and enabled row both asserted	26.34

Regression: all 12 library flows pass on the same build: login-if-needed 9.64s, dismiss-startup-modals 9.83s, open-settings 3.35s, find-wallet 15.02s, ramp-set-region-fiat 16.62s, buy-quote-input 18.91s, buy-quote 19.71s, select-swap-pair 31.42s, confirm-slider 4.97s, send-to-address 26.23s, create-throwaway-account 56.88s, delete-throwaway-account 12.48s. The swap (ETH to SOL) and the send (SOL) were real mainnet transactions.
The proof flows found four runner bugs, all fixed in this commit: enabled: false parsed as true, the scroll drag ended in a fling, a snapshot race in element lookup, and the centering overshoot described above.

Deep-link commits, iOS simulator, same debug build. The branch is rebased onto the build-and-test split, so each rule edit now sits in the slice that owns it (references/drive.md, references/build.md), and hooks/tests/build-and-test-slices.test.py passes on the branch tree.

openLink hands the URL to the app under test through XCUIApplication.open. Both edge:// links and https://edge.app/redirect/payment/ links arrive with no "Open in Edge?" dialog. Scalar and link: forms are accepted, and autoVerify and browser are accepted and ignored.
Flows open by deep link by default. select-swap-pair (SRC_ASSET, DST_ASSET), buy-quote-input (BUY_ASSET) and send-to-address (CURRENCY_CODE) each take MANUAL_PATH: "true" to walk the old path. Existing params keep their meaning. send-to-address with no CURRENCY_CODE takes the manual path, since the payment link needs a currency code.
When the account holds several wallets for a linked asset the app raises its own picker. send-to-address answers it with WALLET_SEARCH.

Drive	Result	Flow time (s)
swap link, bitcoin to ethereum	Exchange scene, My Bitcoin and My Ether selected	21.3 (with the next row)
payment link, BTC	Send scene with address and 0.0001 BTC filled	same run
payment link, ETH	app wallet picker, then filled Send scene	pass
select-swap-pair by link, through to the quote	quote scene	29.9
select-swap-pair `MANUAL_PATH`, ETH to LTC	quote scene	47.5
buy-quote-input by link	Buy scene with the amount entered	19
buy-quote-input `MANUAL_PATH`	Buy scene with the amount entered	20
`hideKeyboard` on the Buy amount field	keyboard gone, app running	pass

No send or swap was confirmed in these drives. The send flow was run without its final slider step.

The screenshots below come from the interpreter's takeScreenshot during these runs.

Test evidence

`7ea7001` _{Document deep-link drives per driver} _2026-10-01	_{1. find wallet}	_{4. ramp region}	_{9. throwaway created}
	_{12. tap point}	_{13. copy paste random text}	_{14. hide keyboard}
	_{15. long press menu}	_{16. scroll center element}	_{17. assert enabled}
	_{21. reg throwaway deleted}	_{22. openlink swap pair}	_{23. openlink send prefilled}
	_{24. openlink swap quote}	_{25. openlink buy amount}

The skill becomes a core (rules that bind in every phase, a step map, steps 1-3) plus one reference per phase: build.md (step 0a-0c), drive.md (step 0d-0f and the drive rules), evidence.md and funding.md. Rule ids are unchanged. The skill-read gate delivers the build slice with the build scripts and the drive and evidence slices with capture-buy-quote.sh and with every maestro drive. The funding slice arrives with a value-moving log-attempt.sh call. build-and-test-slices.test.py holds the core and each slice under 20,000 characters and checks that every pre-split rule id is placed once.

One generic XCUITest bundle (xcuitest/EdgeFlowRunner) interprets Maestro flow YAML at run time. The host converts the YAML to JSON (maestro-yaml-to-json.rb) and passes it through xcodebuild test-without-building via TEST_RUNNER_ env, so the bundle is built once per Xcode build, runtime and source hash (xcuitest-build.sh, cached under ~/Library/Caches/edge-flow-runner, mkdir lock). xcuitest-run.sh drives one flow on one UDID with per-run JSON, log and result bundle under TMPDIR keyed by UDID, and no host port. It stops the slot's maestro MCP daemon by explicit PID first. The runner preflights every command before step 1 and fails naming the flow and step for anything unsupported. ${} and evalScript run in JavaScriptCore. A swizzle caps XCUITest quiescence waits (default 1s, --quiescence-cap or env EDGE_QUIESCENCE_CAP per flow, 0 skips) and logs each cap hit with its step.

build-and-test's new ios-flows-run-on-xcuitest rule makes xcuitest-run.sh the default iOS flow runner; the maestro CLI runs iOS flows only when a task asks or the interpreter's preflight rejects a command. Android stays on Maestro. capture-buy-quote.sh takes --driver xcuitest|maestro (default xcuitest). references/xcuitest-interpreter.md lists the supported commands, the Maestro semantics kept, the quiescence cap, the test-mode animation switch and the unsupported commands.

The search field holds the typed text, so a plain name match hit the field instead of the first row; pick match 1 of a contains-regex. Header env uses ${KEY || default} so runFlow env wins, and the Exchange tap retries through the post-login hidden tab bar.

YOLO auto-login can finish mid-entry and leave the PIN screen, so a later tap found no digit and failed the flow. Each tap now runs only while Exit PIN is visible.

The Search Wallets text survives navigation, so inputText appended to a stale query and the row tap hit the search field.

A preflight survey of the agent task flows under /tmp found flows the interpreter rejected, almost all on tapOn point: and hideKeyboard. Add those and the remaining parity commands so every surveyed flow runs natively: - tapOn point: as screen percentages or absolute points, and relative to the element when a selector is present - longPressOn (3s hold, as Maestro on iOS) - hideKeyboard with Maestro's iOS behavior: nothing when no keyboard is up, else a short swipe up from the screen center, then a short swipe left - copyTextFrom and pasteText, with maestro.copiedText in the script context - inputRandomText - enabled: on selectors - scrollUntilVisible centerElement: - back and pressKey: back as no-ops, as on Maestro iOS launchApp clearState: true stays rejected.

j0ntz force-pushed the jon/xcuitest-interpreter branch from a085b79 to 7ea7001 Compare October 1, 2026 21:38

j0ntz added 12 commits October 1, 2026 16:00

Retry Buy tab tap after login

b7e41bf

Gate PIN taps on the PIN screen

49cf984

YOLO auto-login can finish mid-entry and leave the PIN screen, so a later tap found no digit and failed the flow. Each tap now runs only while Exit PIN is visible.

Clear wallet search before typing

ce30bfc

The Search Wallets text survives navigation, so inputText appended to a stale query and the row tap hit the search field.

Gate buy-quote PIN taps, clear amount

ea24bd7

Retry ramp region tap behind keyboard

ae0cf03

Add openLink to the XCUITest interpreter

8a391d0

Open committed flows by deep link

59d8da9

Document deep-link drives per driver

1d19ad8

j0ntz force-pushed the jon/xcuitest-interpreter branch from 7ea7001 to 1d19ad8 Compare October 1, 2026 23:00

j0ntz merged commit fd83db9 into main Oct 1, 2026

Navigation Menu

Add XCUITest flow interpreter for iOS - #5

Add XCUITest flow interpreter for iOS#5
j0ntz merged 13 commits into
mainfrom
jon/xcuitest-interpreter

j0ntz commented Oct 1, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

j0ntz commented Oct 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Dependencies

Testing

Test evidence

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

j0ntz commented Oct 1, 2026 •

edited

Loading