Repository navigation
Conversation
…ev-inc#3516) * fix(android): wait out a late locale flip with a grace poll setDeviceLocale fires the locale broadcast detached and confirms via getprop within a ~24s budget (3x32x250ms). Under post-resume GMS churn the receiver runs late, so the prop flips just after the budget and the driver reports UPDATE_CONFIGURATION_FAILED for a locale that did apply. Re-firing the broadcast can't cure a late-running receiver; only waiting does. After the detached retries, poll getprop once more over a longer grace window (default 120x250ms = 30s, spent only on the slow path) before classifying as failure. Also skip the trailing sleep after the final poll. * fix(android): poll the locale grace window coarsely (30x1s) The grace window runs on every unapplied path, not just the slow-success one, so a genuine failure waits it out in full. Poll at 1s intervals instead of 250ms to keep the same 30s window while cutting getprop calls 120 -> 30.
* Fix CDP target for web tests against new Chrome versions
Chrome 151 publishes its omnibox WebUI as `browser_ui` targets, so `listTargets().first()` no longer lands on the page under test. Resolve the target by Selenium's window handle instead.
Also:
- scrollToPoint's scroll expression was never valid JS: `() => {}()` does not parse. It was unreachable while innerHeight read correctly, so any tap below the fold failed with a SyntaxError that surfaced as a null Int cast
- Bound the CDP reply wait at 30s, so a target that never answers fails the command instead of stalling the run
* ci: test web against Chrome 151
The web job pinned Chrome 142, nine majors behind what users run. Bumping it keeps the suite closer to the browsers under test.
It does not add coverage of CDP target selection, which was the hope: a run with that fix reverted passes on Chrome 151 here too. It's specific to host OS too.
…ular circular DOMs (mobile-dev-inc#3521) androidWebViewHierarchy: devtools serializes the WebView hierarchy in-page with JSON.stringify. React/Vue/Angular attach circular back-references to DOM nodes (e.g. __reactFiber$…); when one reaches the snapshot, JSON.stringify throws "Converting circular structure to JSON" and the whole capture fails. - Serialize with a cycle-safe replacer that drops live DOM nodes and breaks cycles, so the snapshot always completes. Normal/deep pages are unchanged. - Classify a residual in-page serialization error as a WebView-content (test) error instead of a retried infra error. The deepDom e2e screen now also carries a circular DOM node, so one flow covers both deep and circular DOMs.
* refactor(e2e): describe workspaces in a manifest rather than in run_tests
Four facts about each workspace were spread through the script, three of them
duplicated between single-app mode and the full-suite loop: where its flows live,
which platforms it runs on, whether it carries failing-tagged flows, and — implied
by a glob of workspaces/ — that it is an app at all. Adding one meant editing five
places and remembering the sixth in the workflow.
workspaces.txt holds all four, and is also where the reasons live: the note that
demo_app is skipped on ios because it hits OOM on GHA sat next to a bare
`&& continue`.
The workflow validates the app input against the same file instead of globbing
workspaces/, so the two lists cannot drift.
Behaviour-preserving: with maestro stubbed to record its arguments, the
invocations for android, ios and web, and for MAESTRO_APP of demo_app, wikipedia
and simple_web_view, are byte-identical before and after. An unknown app now
names the manifest rather than a path it guessed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(e2e): validate workspaces.txt before running anything
A positional data file fails quietly when it is edited wrongly. A space where a
comma belongs shifts every column right, so `android ios yes` left the workspace
android-only and its failing flows unrun; a misspelt platform matched nothing and
dropped the workspace from every platform. Both are silent — the run just gets
smaller.
Check the shape first and name the line: four fields, platforms drawn from
android/ios/web, failing yes or no, no duplicate workspace, and a flow directory
that exists.
workspaces.txt:14: unknown platform "androd" in "androd" (want android, ios or web, comma-separated)
A valid manifest behaves exactly as before — the stubbed-maestro comparison across
android, ios, web and the three MAESTRO_APP values is unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(e2e): read manifest rows into named variables
Reading fields by number meant `_field "$1" 3` at the call sites and a comment
elsewhere explaining that 3 means platforms — and each helper re-queried the file
for a row the caller already had, three awk invocations per workspace to read one
line.
`read` names the fields instead, so no index appears anywhere, and the predicate
takes what it tests: `_supports_platform "$platforms" "$platform"` rather than a
workspace name it would look the list up from.
Single-app mode's directory-exists check goes with it: _validate_manifest already
checks every row's path up front, so a bad path now fails before any work starts
instead of when that one workspace is reached.
Invocations across android, ios, web and the three MAESTRO_APP values are
unchanged, as are the exit codes for an unknown app and a skipped platform.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(e2e): run one loop over the manifest, filtered when an app is named
Single-app mode was a second copy of the full-suite loop: same platform check,
same _run_passing, same failing decision, differing only in that it handled one
row. That duplication is what the manifest was introduced to remove, and it was
still there.
One loop now, skipping rows that do not match MAESTRO_APP when it is set, which
also retires _row_for and the here-doc that fed it. The two behaviours that were
particular to the single-app path are kept deliberately: "skipping X on Y" is
printed only when that workspace was the one asked for, and an app matching no row
is still an error rather than a quiet no-op.
Invocations across android, ios, web and the three MAESTRO_APP values are
unchanged, as are the exit codes: 1 for an unknown app, 0 for a workspace skipped
on a platform it does not support.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(e2e): say when a workspace is skipped, and name what the code reads
_supports_platform gave its arguments names, since "$1" and "$2" told the reader
nothing about which side was the list and which the platform. The commas around
both are now explained too: they make it a whole-word match, so "web" cannot match
a hypothetical "webview".
The comment/blank skip in the loop says why it is written against app_name, which
is not obvious — read puts a comment line's first word there, and leaves every
field empty for a blank line.
"skipping X on Y" now always prints. It was suppressed unless that workspace was
the one asked for, on the grounds that it was noise; in CI it is the opposite —
the log should say why a workspace produced no runs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(e2e): read the workspace list through a script, not awk in the workflow
Validating the app input meant two awk programs embedded in YAML, which is a lot
to ask of a reader who only wants to know what the step checks. It also put
knowledge of the manifest's layout in a second place.
e2e/list_workspaces prints the names, one per line, so the step reads as what it
does:
if ! e2e/list_workspaces | grep -qx "$app"; then
It resolves paths against itself rather than the caller's directory, so it works
from the repo root as the workflow needs, or from e2e/ by hand.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs(e2e): explain which side of _supports_platform's case is the pattern
`case X in PATTERN)` matches the word on the left against the pattern on the
right, so the list is the subject and the platform is what we search for inside
it — backwards from how the keyword `in` reads, and enough to make a reader stop
and check whether the arguments had been swapped.
Say so, and name the variables for the roles they play rather than for the
directionless "supported" and "wanted".
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(e2e): give the web flows their own workspace
They lived in demo_app/.maestro/web_flows because that was the workspace that
existed when the first one was written, alongside a maestro-web.js fix. Nothing
about them belongs to that app: none names its appId, and the CI job that runs
them builds no Flutter app, installs no APK and boots no emulator. The cost was
that a web run loaded and validated the whole demo_app workspace — every Flutter
flow, both device-configuration directories, commands/ — to select thirteen files.
web is now an ordinary row in the manifest, which retires the bespoke branch that
hardcoded demo_app's path and selected by the `web` tag where every other run
selects `passing`. Headless moves to the platform block next to the tag
exclusions, since it is a property of running a browser rather than of that
branch.
The web job no longer passes MAESTRO_APP. The old branch ignored it, so this
preserves behaviour — and it matters now: the app input defaults to demo_app,
which does not support web, so honouring it would have run nothing at all.
Verified: ./run_tests web passes 11/11 against the new location, and the stubbed
invocations for android and ios are unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(web): tag the textarea flow, and give it a page that still has a textarea
It was the first web flow ever written, added untagged alongside the
maestro-web.js change that put a textarea's contents in the hierarchy — so
--include-tags passing has never selected it and it has never run in CI.
Tagging it alone would have turned the suite red. It drove pastebin.com, which has
since replaced its plain textarea with a rich editor: the id the flow targeted now
belongs to a wrapper reporting "New Paste", and the typed text reaches the
hierarchy nowhere. Confirmed that Maestro's side is fine by typing into a local
textarea, which reports its value correctly.
The behaviour under test is Maestro's, so the page is ours now. It also covers the
other half of getNodeText that pastebin never exercised: the placeholder shows
while the textarea is empty, and gives way once there is a value.
12/12 web flows pass, textarea among them for the first time.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(web): serve fixture pages over http instead of inlining them as data: URLs
A page carried in the flow header had to be percent-encoded on one line, which
put date_input's markup in 1022 unreadable characters and made editing it a
decode-edit-reencode round trip. It also gave every fixture an opaque origin,
which rules out anything depending on cookies or localStorage.
The pages are files now, fetched from a static server on 7357. run_tests starts
one for the web platform and stops it again, so CI needs nothing extra; it reuses
a server that is already up, since anyone driving flows by hand or through the MCP
will have started one with serve_fixtures. If the port answers as something else
it says so rather than running against whatever is there.
Bound to 127.0.0.1, and addressed by that literal rather than localhost, so the
flows cannot resolve to ::1 and miss an IPv4-only listener.
The four pages were laid out by hand rather than mechanically reflowed: whitespace
inside an element changes the text a flow asserts on.
12/12 web flows pass — starting the server, reusing one, and with FIXTURES_PORT
moved — and the server is gone afterwards in each case.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(web): cover both iframe branches, and stop testing a third party's SPA
iframe.yaml drove automationui.pages.dev, a third-party Vite app whose markup is
client-rendered behind a hashed bundle — and whose frame turned out to be
same-origin, so the flow exercised the contentDocument recursion rather than the
Selenium frame switch that issue mobile-dev-inc#3009 was about. The path with the frame switch
had no e2e coverage at all.
Two flows now, one per branch of maestro-web.js:
iframe_same_origin both pages from the fixtures server
iframe_cross_origin a local page framing https://www.saucedemo.com
Confirmed they cover different code: with fetchCrossOriginIframeContent stubbed to
return null, the same-origin flow still passes and the cross-origin one fails.
Both frames sit at an offset on either axis, so the driver has to add the frame's
own position to the bounds it reports from inside rather than getting away with
zero.
The cross-origin flow deliberately does not log in. saucedemo keeps its session in
storage Chrome partitions for a third-party frame, so a login inside one fails
with "you can only access '/inventory.html' when you are logged in" — the site's
constraint, unrelated to iframe support.
13/13 web flows pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(web): identify the fixtures server by its index, not a marker file
fixtures.marker was a file whose name told a reader nothing and whose contents
were a bare token with no hint of why it existed — and it showed up in the
directory listing that http.server generates, explaining itself to nobody.
An index.html at the fixtures root does the same job without needing a name: the
probe fetches `/` and matches the heading, so there is no filename to misread. It
also replaces that generated listing with a page that says what the port is, for
anyone who opens it while driving flows by hand.
Verified all three outcomes: reusing a server already up, starting one when the
port is free, and refusing when the port answers as something else. 13/13 web
flows pass and no server is left behind.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs: bring AGENTS.md in line with the e2e workspace changes
The e2e section described a layout this branch changed and omitted the pieces that
now drive the suite: workspaces.txt, serve_fixtures, list_workspaces and
FIXTURES_PORT appeared nowhere.
The instruction for adding a workspace was the costly one. It said to put flow
YAMLs under a `.maestro/` subdirectory, which was already wrong — simple_web_view
and wikipedia have always held theirs directly — and it said nothing about the
manifest, so following it produced a workspace run_tests never runs and nothing
reports.
Also records that web flows need the fixtures server, and what it looks like when
they do not have one: launchApp succeeds against the dead port because Chrome
serves its own error page, so the flow reports Element not found for a selector
that is correct. Verified both directions.
demo_app/CLAUDE.md loses its web_flows entry, a directory that no longer exists.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(e2e): make the manifest validator a script of its own
It was 41 of run_tests' 269 lines, called once, which on its own is a poor reason
to move anything. What earns the move is a second caller: validate-inputs runs
before build, so a typo in workspaces.txt can now be reported in seconds instead
of after a three-platform CLI build. It is also placed before that job's app
check, which reads the manifest and would otherwise reject a valid name on the
strength of a malformed file.
Not added to the build job: that matrix includes windows-latest, and a POSIX shell
script has no business there. So a pull request still discovers a bad manifest in
the test jobs, as it did before — no worse, just not better.
Behaviour-preserving, checked by stashing the change and comparing the stubbed
invocations for android, ios and web either side. All six failure modes still
report with their line numbers.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(e2e): make readying the fixtures server callable from anywhere
serve_fixtures was already a script, but everything that made it usable lived in
run_tests: whether one is already up, whether the port belongs to something else,
waiting until it answers, and stopping only a server we started. An agent or a
Studio session got none of that — just "serve_fixtures &" and a race, whose
failure points nowhere near the cause, since launchApp succeeds against the dead
port and the flow reports Element not found for a correct selector.
ensure_fixtures does that work and is idempotent, so it is safe to call before
every flow. The pid of a server it started goes to stdout and nothing does when it
reused one, so a caller stops only its own; everything a human reads goes to
stderr, leaving stdout parseable. run_tests keeps its two-line trap and loses 50
lines.
Verified: starting, reusing, refusing a port held by something else, and that
run_tests leaves a server it did not start running while still cleaning up one it
did. 13/13 web flows pass and the documented recipe works as written.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(e2e): catch a workspace with no manifest row, and check the manifest before building
Two gaps in how a broken manifest is caught.
A workspace whose row was forgotten was not caught at all: validate_workspaces
passed, run_tests never mentioned it, CI went green, and the contributed flows
simply never ran. Adding the row is the one step in creating a workspace that the
filesystem does not show, which makes it the easy one to miss and the expensive one
to miss quietly. The validator already checked rows against directories; it now
checks directories against rows, ignoring anything under workspaces/ that holds no
flows.
A malformed row was caught, but only inside the test jobs — validate-inputs is
dispatch-only, so a pull request paid for a three-OS build first. A validate-manifest
job now runs on every event and build gates on it, which needs the result asserted
explicitly since always() would otherwise ignore it.
The check stays in validate-inputs too, so a dispatch reports a malformed manifest
before that job's app check reads it and blames the app instead.
Verified: a forgotten row fails both the validator and run_tests, adding the row
fixes it, a directory with no flows is ignored, every previous failure mode still
reports, and the stubbed invocations for android, ios and web are unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* ci: check the workspace manifest in the build job, not a job of its own
A job plus an always()-aware gate was a lot of YAML for one check, and the gate
was the fragile part: build's `if` had to assert the new job's result explicitly
or always() would have ignored it, making the dependency decorative.
A step at the top of build does the same work — it runs before anything expensive
and the test jobs wait on build either way — and it takes the job count back down.
Scoped to the Linux leg: the file is committed and identical on all three, so once
is enough, and nothing else in this workflow asks the Windows runner for awk.
Also puts detect-changes back under the comment describing it, which the removed
job had split apart.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* ci: gate the build on a manifest check that runs on every event
A step inside build was the wrong shape: the matrix has no fail-fast, so the macOS
and Windows legs would build anyway while only one leg reported the problem — and
running it on every leg would have made this the first thing in the workflow to
want awk from Git Bash.
A job of its own, between validate-inputs and detect-changes, with build waiting
on it. Ubuntu-only, no `if`, so it covers pull requests too — a manifest can be
broken by any push, not only by a manual dispatch.
The gate needs care and now says so: always() is what keeps the build running when
validate-inputs is skipped, which is every pull request, and the price is that a
failed dependency is ignored unless it is named in the condition. Evaluated the
expression across the cases that matter — on a pull request with a broken
manifest, build and every test job skip, so no platform starts Gradle.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
…dev-inc#3510) * fix(android): don't report an empty input's hint as its text On API 26+, AccessibilityNodeInfo.getText() returns the hint text while an empty input is showing it, flagged via isShowingHintText(). The view hierarchy dump serialized node.text unconditionally, so an empty field showing a placeholder was indistinguishable from a field that actually contains that value: its text attribute duplicated hintText. Serialize text as empty when the node is showing its hint (guarded to API 26+, where isShowingHintText exists). The hint remains available in the hintText attribute, and text matching already falls back to hintText, so selecting an input by its placeholder keeps working. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Sjrzn3RX7yq6NUQD5mk4zZ * test: cover textMatches hintText fallback for empty inputs Now that the Android hierarchy dump reports an empty input's text as empty instead of duplicating its hint, matching such an input by its placeholder relies on textMatches falling back to hintText. Pin that behavior with unit tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Sjrzn3RX7yq6NUQD5mk4zZ --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Dan Caseley <dan@mobile.dev>
Add SystemImageTag so device creation can select an Android system-image tag, with round-trip serialization and tests.
One skill (.claude/skills/release) is now the release doc and automation: draft changelog/version PR, one human gate after approval, then merge, tag, publish CLI, verify. RELEASING.md points at it; prepare-release skill removed.
…obile-dev-inc#3537) Improves the `release` skill: - Checkpoint 1 confirms the version + changelog before anything is written - Checkpoint 2 is the point of no return; on go, verify required checks (gh pr checks --required) and stop if they aren't green - Reframe the up-front plan as "here's what I'll do" with the two checkpoints interleaved, dropping the per-line role tags - Dry-run is opt-in only, never offered up front - Match house changelog style (lowercase after the area, no PR numbers) - Add a read-only brew-formula check to the verify step
…c#3536) Adds an e2e-gate job that depends on the three platform jobs and fails unless all succeeded, so the ruleset can require one stable check instead of enumerating each.
…itive) (mobile-dev-inc#3542) Additive-only expand: introduces a private systemImageOverride ctor param (serialized as "systemImage" via @JsonProperty) plus a public systemImage getter that resolves to the override or falls back to the google_apis image string, independent of `tag`. `emulatorImage` now aliases `systemImage` instead of deriving from `tag`. `tag` and `emulatorImage` stay in place (no @deprecated) so unmigrated callers in copilot's backend and maestro-cli keep compiling once the submodule bumps to this branch. Also teaches DeviceSpecSparseSerializer to honor @JsonProperty on a primary-ctor param when choosing the emitted JSON key, so the override round-trips under "systemImage" rather than "systemImageOverride". No CLI flags, no wire field, no init validation -- those land in PR 2.
…ile-dev-inc#3543) Overload --device-os to also take a full `system-images;<os>;<tag>;<abi>` path (it still accepts a version like android-34). A full path builds a typed DeviceSpec.Android and sends it as the typed `deviceSpec` upload field — the first real caller of that path — with the loose device fields nulled; any other --device-os value keeps today's loose-field send. - DeviceSpec.Android: expose systemImageOverride, add a systemImage getter, drop SystemImageTag/tag/emulatorImage. init validates the full-string image, including that its abi segment matches cpuArchitecture. - ApiClient.upload gains a deviceSpec param and registers DeviceSpecModule so systemImageOverride serializes under the `systemImage` key. - Fail fast when a full-path --device-os targets a non-Android app.
Fixed a typo in the case of the --help command.
* Fix file separators on Windows for Cloud uploads * Defend against single quotes in paths
… types (mobile-dev-inc#3560) * feat(orchestra): derive the flow-command schema from the parser's own types Adds maestro.orchestra.yaml.schema.FlowCommandSchema — the command surface (names, arguments, kinds, enum vocabularies, alternative shapes, which commands may be written bare) derived by reflection from YamlFluentCommand's constructor, each command's Yaml* data class and the parser's stringCommands map. Nothing is hand-maintained, so it cannot drift from what the parser accepts. It ships as an API, not an artifact: consumers already depending on dev.mobile:maestro-orchestra call FlowCommandSchema.commands() or asJson(). .github/workflows/publish-schemas.yaml, which uploaded a hand-written JSON schema to GCS, is deleted. Supporting changes: - DarkModeValue / AirplaneValue carry their YAML wire names as @JsonProperty instead of hiding them in string literals inside the deserializers. This is the only behaviour change: setDarkMode's map form now honours `optional`, which it previously dropped, and both commands reject near-miss values that the old substring matching let through. - YamlSwipe becomes a sealed interface so its four shapes are discoverable. - stringCommands becomes internal so the schema can read its keys. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012naxaR7AjmbDCmckC2Y4ru * fix(orchestra): only call an argument required when the parser cannot fill it in `required` was derived from `isOptional`, which answers a Kotlin question -- does the parameter have a default -- not the YAML one. A nullable parameter without a default still deserializes when the key is absent, because Jackson supplies null. That made the schema claim 9 arguments across 4 commands were mandatory when they are not, `- launchApp` on its own being the obvious counter-example. RequiredClaimTest pins the boundary in both directions: what the schema calls required must be rejected when omitted, what it calls optional must be accepted. Commands with alternative shapes are excluded and asserted, since dropping a variant's distinguishing argument leaves a shape a sibling variant legitimately accepts. `inputText.text` and `evalScript.script` are pinned as known parser bugs -- their delegating creators read the field with getOrDefault(name, ""), so omitting it yields an empty string instead of an error. Tightening that is a parsing behaviour change and is left out of this change; the exception fails the day it is fixed rather than lingering. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012naxaR7AjmbDCmckC2Y4ru * Revert the publish-schemas workflow deletion The workflow publishes the artifact-manifest schema, which is a separate concern from the flow-command schema this PR introduces. Deleting it here left the worst of three states: the bucket object and SCHEMA_URL stay, every manifest keeps stamping $schema, and BundleUploader deliberately preserves that field into customer-downloadable bundles -- but nothing would keep the published copy current. The first edit to v1.schema.json would hand customers a URL describing a format that no longer exists, with no test to catch it. Retiring or replacing artifact-manifest publishing is its own decision and belongs in its own change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012naxaR7AjmbDCmckC2Y4ru * feat(orchestra): let a String YAML field declare the vocabulary it is validated against pressKey.key and setOrientation.orientation are typed String -- they have to be, because both accept ${VAR} -- but the parser only accepts a fixed set of words, via KeyCode.getByName and DeviceOrientation.getByName. The schema could not see that, so it reported both as free strings and every consumer hand-copied the lists. Studio's copy had already lost `escape`. @YamlValues(of = ...) names the enum the parser looks the value up in, and FlowCommandSchema reports the field -- in the map form and in the shorthand -- as ENUM with those words. pressKey now advertises 30 keys and setOrientation 4, read off the enums rather than written down anywhere. KeyCode's YAML spelling is its `description` ("Volume Up"), not its constant name, and it cannot say so with @JsonProperty the way DarkModeValue and AirplaneValue do: Jackson also serializes KeyCode as PressKeyCommand.code on the MaestroCommand wire, where the constant name is what is written and read back. So @YamlValues carries an optional `spelledBy` naming the property that holds the spelling, and KeyCode is left alone. The existing `every advertised enum value parses` test picks both commands up automatically now that their arguments are ENUM -- 60 pressKey and 8 setOrientation values go through the real parser. A second test pins that a String field carrying the annotation surfaces its vocabulary in both forms. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012naxaR7AjmbDCmckC2Y4ru * test(orchestra): pin the schema against commands being added and changed Every existing schema test reads its expectations off the Yaml* types as they are today, so none could say what happens the next time one changes. FlowCommandSchemaEvolutionTest drives the derivation with synthetic Yaml* shapes instead. It found three bugs, fixed here: - `@JsonProperty` on a constructor parameter was ignored, so renaming a parameter behind its YAML spelling made the schema advertise a key the parser rejects. `wireNameOf` now reads it. - `@JsonAlias` spellings were never advertised: `launchApp: {url: ...}` parses today and was absent from the schema. `ArgumentSchema.aliases` carries them. - A stale `spelledBy` killed every `commands()` call with a bare NoSuchElementException naming neither the enum nor the property. `@YamlRequiresOneOf` closes the last gap the tests found: `required` is derived from the Kotlin constructor, so runFlow, retry, extendedWaitUntil and addMedia were published as having no required arguments while the parser rejects their empty form. The annotation declares the rule the way `@YamlValues` declares a vocabulary, and `CommandSchema.requiredOneOf` reports it. The two existing test classes were reworked: two assertions that could never fail are gone, both now parse through `parseCommand` rather than `checkSyntax` (which never runs `toCommands`, making the enum guarantee vacuous for exactly the `@YamlValues` fields), and the optional-argument test is one question per command since it rendered identical YAML per argument. That surfaced two gaps left pinned, not fixed: `openBrowser` has no `_toCommands` branch at all, and `action` publishes its five-word vocabulary as free-form String. Verify: - ./gradlew :maestro-orchestra:test - 427 tests, 0 failures - ./gradlew :maestro-orchestra-models:test - 51 tests, 0 failures - ./gradlew :maestro-cli:compileKotlin - passes Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015vHfmsiT7skZdV13zcbeof * test(orchestra): make the schema-vs-Jackson check actually able to fail Two holes in the guard added by the previous commit, both found in review. `binds()` asked whether a key was bound by handing it to `readValue` and looking at which exception came back. On a type with a required non-null constructor parameter Jackson fails at the creator step and never reports the unknown key, so the probe answered "bound" for anything: 26 of 54 shapes, including inputText, openLink, pressKey, setDarkMode and swipe. The direction that exists to catch a renamed argument could not fail for half the surface. It now asks the resolved BeanDeserializerBase, which is what does the binding. That also resolves `@JsonAlias` through a delegating creator, so the introspection floor and its caveat are gone, and the four commands with a hand-written deserializer are asserted rather than silently skipped. `wireNameOf` and the alias read used `parameter.findAnnotation`, which only sees Kotlin's `param` target. `@field:` and `@get:` are legal and Jackson honours them, so a rename written that way left the schema advertising the old name. `annotationOf` now looks at the parameter, the property, the getter and the backing field. Kotlin already warns that the default target is changing, which would make this the common case rather than the unusual one. Both are pinned by mutation: renaming inputText's key behind @JsonProperty with wireNameOf reverted now fails in both directions, and reverting annotationOf to parameter-only fails the new use-site-target test. Each passed before. Verify: - ./gradlew :maestro-orchestra:test - 428 tests, 0 failures - ./gradlew :maestro-orchestra-models:test - 51 tests, 0 failures - ./gradlew :maestro-cli:compileKotlin - passes Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015vHfmsiT7skZdV13zcbeof * test(orchestra): check the one-of rule against the parser, and say when it is exclusive @YamlRequiresOneOf is the one claim in the schema that is written by hand rather than derived, so unlike everything else it can simply be wrong — and nothing checked it. A typo published a rule naming an argument that does not exist, and removing the annotation from runFlow or addMedia left the suite green. - The rule now carries `exclusive`. runFlow and retry reject `file` and `commands` together; extendedWaitUntil accepts `visible` and `notVisible` together. One annotation was describing both, so a consumer generating from the schema could emit a pair the parser refuses. - `requiredOneOf` becomes an `OneOfSchema` rather than a bare list, so the command carries one rule with its mode instead of two parallel fields. - A new test asserts every part of the claim against the parser: the names exist, none of them is also `required`, the empty form really is rejected, each member alone really is enough, and `exclusive` really matches. - `render`'s shorthand fallback was firing for any command with no *kept* arguments, so `launchApp: {}` and `addMedia: {}` were never written — the single-value form was tested in their place. It is now gated on the command having no named arguments at all, which is what the comment always claimed. Pinned by mutation: dropping the annotation from addMedia, dropping runFlow's exclusivity, marking extendedWaitUntil exclusive, and misspelling an argument name each fail with a message naming the command. All four passed before. Verify: - ./gradlew :maestro-orchestra:test - 429 tests, 0 failures - ./gradlew :maestro-orchestra-models:test - 51 tests, 0 failures - ./gradlew :maestro-cli:compileKotlin - passes Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015vHfmsiT7skZdV13zcbeof * fix(orchestra): keep the YAML spelling off the MaestroCommand wire `@JsonProperty("enabled")` on DarkModeValue / AirplaneValue moved the YAML word onto the enum so the schema could derive it. But that enum is on two wires, and only one of them wanted changing: YAML input enabled -> enabled (already right) MaestroCommand JSON Enable -> enabled (not intended) The second is what `run_step.maestro_command_meta` holds and what the worker and backend exchange in `CommandResult.command`. The old spelling stopped being read at all, so historic rows and in-flight payloads would fail to deserialize, and a rolling deploy would have the two sides disagree. KeyCode already solves this in this same branch: hold the YAML word in a plain property and point `@YamlValues` at it. A constructor property does not change how Jackson writes an enum, so the wire keeps the constant names. The two enums now do the same, and the deserializers look the word up on the enum rather than letting Jackson convert it -- still derived, so the parser and schema still cannot disagree. Both real fixes from the original change survive: `optional` is still honoured in the map form, and near-miss values are still rejected. Reading the value node as text also restores the guidance message for `setDarkMode:\n value:`, which had degraded to "Failed to parse content". YamlSetModeTest now pins the wire format directly, because nothing else looks at it and every YAML test passes either way -- reintroducing `@JsonProperty` fails it. Verify: - ./gradlew :maestro-orchestra:test - 430 tests, 0 failures - ./gradlew :maestro-orchestra-models:test - 51 tests, 0 failures - ./gradlew :maestro-test:test - 213 tests, 0 failures - ./gradlew :maestro-cli:compileKotlin - passes Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015vHfmsiT7skZdV13zcbeof * fix(orchestra): publish what the parser actually accepts for selectors, drop a dead command `openBrowser` was declared on YamlFluentCommand but `_toCommands` has no branch for it, so nothing it accepted ever parsed — while the schema published it as a real command and the KDoc used it as the worked example of a scalar one. Nothing else in the repo refers to it; AndroidDriver's `openBrowser` is a private helper behind `openLink`. Selector commands published `{"selector": true, "arguments": []}`. tapOn, assertVisible, copyTextFrom and their siblings take a YamlElementSelector rather than named arguments of their own, so Maestro's most-used commands were the ones the schema said least about. `selectorArguments` derives the 27 of them once, beside `commonArguments`, rather than repeating them under every command. Writing that test turned up `selector.traits`: a String holding space-separated ElementTrait names, published as free-form text, so anything generated from the schema wrote a value the parser rejects. It is the third `@YamlValues` field. Verify: - ./gradlew :maestro-orchestra:test - 0 failures - ./gradlew :maestro-cli:compileKotlin - passes Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015vHfmsiT7skZdV13zcbeof * feat(orchestra): version the schema document and fix its ordering `asJson()` had no version, so a consumer caching or diffing it could not tell a change in the document's shape from Maestro gaining a command — and this branch has already added three fields to it. Two orderings were unspecified. `variants` comes from `sealedSubclasses`, which the language does not order, and the enum fallback used `Class.getFields()`, whose order the JVM does not fix — while the `spelledBy` path next to it used `enumConstants`, which is declaration order. A compiler or JDK upgrade would have silently reshuffled every published document. Variants are now sorted by name and both enum paths use the same primitive. Verify: - ./gradlew :maestro-orchestra:test - 0 failures Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015vHfmsiT7skZdV13zcbeof * test(orchestra): check the schema's hand-written claims, and correct two of them `commonArguments` is the one hand-maintained claim in an object whose KDoc says nothing is: `argumentsOf` strips any parameter named `label` or `optional` and the schema then asserts every command takes both. The parser has FAIL_ON_UNKNOWN_PROPERTIES on, so a command declaring only one of them would have the schema advertising a key the parser rejects. Now asserted. Restores the guard that `commands()` cannot silently drop a command. It was deleted last week as a tautology, which was half right: it duplicated the name list, but it was also the only thing covering the `as? KClass` filter, and every other test rebuilds that same filter. Two claims were wrong rather than unchecked. The `@YamlValues` KDoc says `pressKey` needs to stay a String because it accepts `${VAR}`; it does not — `KeyCode.getByName` runs before substitution, so only `setOrientation` does. And sealing YamlSwipe made the `else` branch of its `when` unreachable, which the compiler does not warn about for a statement `when`. Also: the alias test now asserts `url` lands in `appId` rather than merely parsing, and the broad `assertThrows<Exception>` calls are narrowed — to MismatchedInputException, not UnrecognizedPropertyException, because the throw is MissingKotlinParameterException. That masking is the same one the Jackson-binding check was rewritten to avoid, and the test says so. Verify: - ./gradlew :maestro-orchestra:test - 0 failures Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015vHfmsiT7skZdV13zcbeof * feat(orchestra): give `action` the vocabulary it has always had `action` accepts exactly five words — back, hideKeyboard, scroll, clearKeychain, pasteText — held as literals in a `when`, and the schema published it as free-form String. Anything generating a flow from the schema wrote a value the parser rejects with "Unknown navigation target". The words move onto YamlNavigationAction and the parser looks them up there, so they cannot drift. Reaching them needed one change: `action` is a plain String on YamlFluentCommand itself, so `schemaOf` now consults the declaring parameter and not only its type, which is where `@YamlValues` had to go. With this and openBrowser gone, every command the schema declares can be written from the schema — RequiredClaimTest's exception list is empty and deleted. Verify: - ./gradlew :maestro-orchestra:test - 434 tests, 0 failures - ./gradlew :maestro-cli:compileKotlin - passes Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015vHfmsiT7skZdV13zcbeof * docs(orchestra): correct the setDarkMode/setAirplaneMode vocabulary comments e44a632 moved the YAML words off `@JsonProperty` and onto `yamlValue`, to keep the constant names as the MaestroCommand wire format, but left three comments saying the vocabulary lives on `@JsonProperty`. That points a reader at an annotation which is not there, and invites putting one back -- which is the change that commit exists to undo. The corrected text says why it is not a `@JsonProperty`, not only where the words are. Comments only. Verify: - ./gradlew :maestro-orchestra:test - 0 failures - ./gradlew :maestro-orchestra-models:test - 0 failures Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015vHfmsiT7skZdV13zcbeof * feat(orchestra): publish swipe's alternative shapes under YAML names `VariantSchema.name` came from `subclass.simpleName`, so the schema published `YamlCoordinateSwipe` and `YamlSwipeElement` -- Kotlin class names, in a document a consumer reads and generates from. `@YamlVariant` declares the YAML-facing name instead, the way `@YamlValues` and `@YamlRequiresOneOf` declare the other things reflection cannot see: byDirection direction byCoordinates start, end byRelativeCoordinates start, end byElement direction, from Nothing consumes the schema yet, so these strings are free to choose now and are not once something does. A test fails if any variant ships under a class name; removing one annotation gives `[swipe.YamlSwipeElement]`. `variantNameOf` is internal so the binding test correlates a subclass with its variant through the same mapping rather than a second copy of the rule. `byCoordinates` and `byRelativeCoordinates` still carry identical arguments -- they differ only by the `%` in the value, which the schema has no way to say. Describing a string's *shape* is a concept it lacks in four places (these two, `extendedWaitUntil.timeout`, `retry.maxRetries`, and the space separator in `selector.traits`), so it is worth designing once rather than for swipe alone. Merging the two variants falls out for free once it exists. Verify: - ./gradlew :maestro-orchestra:test - 435 tests, 0 failures - ./gradlew :maestro-orchestra-models:test - 51 tests, 0 failures - ./gradlew :maestro-cli:compileKotlin - passes Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015vHfmsiT7skZdV13zcbeof * fix(orchestra): stop calling a primitive required when Jackson fills it in `required` was derived as "no Kotlin default and not nullable", which is right for reference types and wrong for primitives. Jackson hands the creator the JVM default for a missing `Int` or `Boolean` rather than raising, so an argument the schema called mandatory is one the parser is perfectly happy to fill in: {"text":"a","flag":true,"ratio":1.5} -> count=0, no error {"text":"a","count":1,"ratio":1.5} -> flag=false, no error {"count":1,"flag":true,"ratio":1.5} -> MissingKotlinParameterException (text) Nothing published changes: no command declares a non-nullable primitive without a default today, which is why this never showed up against the real parser. It was live in the suite, though -- the evolution test pinned `count`, `big`, `ratio` and `flag` as required, asserting a Jackson behaviour that does not exist. Left alone, the first command to declare `val timeout: Int` turns RequiredClaimTest red in a file its author never touched. The new case asserts the rule against Jackson rather than restating it, so it stays honest if the module's behaviour changes. Verify: - ./gradlew :maestro-orchestra:test - 436 tests, 0 failures Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018urBBzo8pVbwhcdWVvZFKr * test(orchestra): let JUnit own the temp flow both schema tests write Both tests built their referenced flow through `TempFileHandler()`, constructed inline and never closed. Nothing leaked -- `createTempFile` registers a `deleteOnExit`, so the JVM removed the file at shutdown -- but a `Closeable` used as if it were not one is the kind of thing that gets copied. `TempFileHandler` is a lifecycle helper for driver code that juggles many files. These tests want exactly one, and JUnit already has the tool for that: `@TempDir` is created and removed per test, so there is nothing to close and nothing left for shutdown. It also drops the `maestro-utils` import from both. Verify: - ./gradlew :maestro-orchestra:test - 436 tests, 0 failures Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018urBBzo8pVbwhcdWVvZFKr * fix(orchestra): drop two selector fields nothing has ever read `YamlElementSelector.start` and `.end` were added to the selector in "Add swipe command (mobile-dev-inc#27)" on 18 Jul 2022, when swipe still reused that type. Swipe grew its own `YamlCoordinateSwipe` and `YamlRelativeCoordinateSwipe`, each with their own start/end, and these two were left behind. Nothing reads them. `toElementSelector` never looks at them, the only `.start` / `.end` reads in the repo are on a YamlSwipe subtype, and the downstream `ElementSelector` has no field to carry them even if something wanted to. Every `start:` in the repo's flows -- tests and e2e -- is under `swipe:`. Jackson bound them and the parser threw them away, for three years. That was invisible until the schema published them: `selectorArguments` advertised two fields to every consumer, and Studio's autocomplete offered them under `tapOn` with hand-written prose describing behaviour that does not exist. `selectorArguments` goes 27 -> 25. `swipe` is untouched. The one behaviour change: `tapOn: {start: "0,0"}` parsed and was ignored, and now fails with "The property `start` is not recognized". A flow written that way was never doing what its author meant; it now says so. Worth a changelog line. Verify: - ./gradlew :maestro-orchestra:test :maestro-orchestra-models:test :maestro-test:test :maestro-client:test - 980 tests, 0 failures. No test needed changing, which is the point. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018urBBzo8pVbwhcdWVvZFKr * fix(orchestra): order retry's one-of by what a consumer should write first `@YamlRequiresOneOf` publishes its names in the order they are written, and a consumer with no other signal takes the first -- Studio seeds it into the snippet Monaco expands. `retry` listed `file` first, which was never a decision: the annotation was modelled on `runFlow`, where `file` genuinely is primary because `runFlow: "login.yaml"` is a valid shorthand. `retry` has no such shorthand, so picking `file` sent people to a flow file when the common form is an inline block. Reordered, with a comment saying the order is deliberate, so the next person tidying the argument list does not silently change what consumers write. `runFlow` is unchanged and unaffected either way: it returns on its shorthand before the one-of is consulted. Nothing pins the order -- RequiredClaimTest checks each member parses alone, whichever it is. Verify: - ./gradlew :maestro-orchestra:test :maestro-orchestra-models:test :maestro-test:test - 700 tests, 0 failures Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018urBBzo8pVbwhcdWVvZFKr * fix(orchestra): give the config's app id its YAML name on the wire `_appId` is named that only because YamlConfig also exposes a computed `appId`, and Kotlin will not let a constructor parameter and a property share a name. The rename was covered with `@JsonAlias("appId")`, which makes `appId` an *alternate* spelling and leaves `_appId` as the primary one. Two things followed: appId: com.example.app parses _appId: com.example.app parses too, which nobody intended and anything reading the type back gets the names inverted -- deriving a schema from YamlConfig today advertises `_appId`, with `appId` demoted to an alias. `@JsonProperty` makes `appId` the wire name, so the collision stays an implementation detail of this class. `appId:` is unchanged, `_appId:` is now a config error, and the private spelling stops leaking to anything that reads the type. The one behaviour change is that `_appId:` no longer parses. It was never documented or intended. Verify: - ./gradlew :maestro-orchestra:test :maestro-orchestra-models:test :maestro-test:test :maestro-client:test - 980 tests, 0 failures Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018urBBzo8pVbwhcdWVvZFKr --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
new assets for the read-me
* Update README.md * Update README.md Co-authored-by: Dan Caseley <dan@mobile.dev> * Update README.md Co-authored-by: Dan Caseley <dan@mobile.dev> --------- Co-authored-by: Dan Caseley <dan@mobile.dev>
… images) (mobile-dev-inc#3558) * fix(device): parse the major API level from a minor-versioned os Google's minor-SDK scheme (Android 16 QPR2+) means platforms like "android-37.1"; osVersion must yield 37, not 0, so API-level checks keep working. Take the major component. * feat(device): resolve the system image against what sdkmanager offers start-device with a plain --device-os (no full image) resolved system-images;<os>;google_apis;<abi>, which does not exist for newer levels (API 37 ships only google_apis_ps16k). Instead of hardcoding a per-API image, ask sdkmanager what actually exists and pick by preference: installed over available, google_apis over other rootable variants, never a playstore image. An explicit --device-os full image still wins; unresolved falls back to the old default. * refactor(device): resolve the system image on the creating host, not in the spec The DeviceSpec is the contract that travels CLI -> backend -> worker, so it carries intent (model, os, abi, optional pinned image) and stays a catalog key. Resolution against sdkmanager only makes sense on the host that creates the AVD, so it moves out of StartDeviceCommand.buildDeviceSpec and into DeviceCreateUtil.getOrCreateAndroidDevice: an existing AVD is reused as-is, a pinned full path wins, otherwise this host's SDK picks the package. spec.os is left untouched (no adopting "android-37.1"). Selection mirrors what sdkmanager really publishes for API 37: 37.0 still ships plain google_apis (4 KB pages), 37.1 ships only ps16k, 37.2 is beta. Pick the newest stable minor, only google_apis-family tags (no wear/TV/AOSP), google_apis over other rootable variants, never playstore. On a real host android-37 resolves to system-images;android-37.1;google_apis_ps16k;<abi>; 29..36 are unchanged. * fix(device): respect the spec's image and fail with the exact --device-os to use Instead of silently substituting a different package, start-device now tries the spec's own image. If it is not installed and this host's SDK offers a different image for that os (API 37 publishes only minor-versioned ps16k images), the command fails and names the flag: Re-run with: --device-os system-images;android-37.1;google_apis_ps16k;arm64-v8a An installed image never triggers an SDK lookup, a missing image the SDK does offer keeps the install prompt, and a pinned full path is used as-is. * fix(device): default to google_apis_ps16k system image for API 37+ DeviceSpec.systemImage hardcoded the google_apis tag, which does not exist for API 37 (only the 16 KB-page ps16k variant is published). Make the default tag os-aware so a minor-versioned os (android-37.1) deterministically resolves to a valid package inside the spec. A user-provided --device-os still wins, and a bare-major android-37 (no platform of its own) is still corrected by the host resolver. * fix(e2e): stabilize setOrientation flow on API 37 emulator Reset to portrait before each Back-and-re-enter so navigation always happens from the stable portrait home state. This avoids the fragile reverse (180-degree) landscape_left -> landscape_right flip and the ambiguous "Orientation Test" selector (which matches both the home-grid button and the AppBar title) while the device is still rotated. On the API 37 emulator that sequence left the app on the Home page, timing out the "Landscape Right" assertion. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016RhsjQHyvbpecDrkDs1smC --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* docs: fix dead link to the demo_app flows The README pointed at e2e/workspaces/demo_app, which 404s. That directory holds simple_web_view, web and wikipedia; demo_app's flows live in e2e/demo_app/.maestro, which is why run_tests refers to them separately as 'demo_app/.maestro and the workspaces directory'. Also drops the word 'workspace' from the sentence, since that is the part that was misleading - demo_app is not one of the workspaces. * chore: re-run CI after runner allocation failure No change. Unit Test on Java 17 never started: GitHub reported the job "was not started because it repeatedly failed to be acquired (5 attempts)", so no steps ran. This commit only re-triggers the workflows. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* chore(android): bump compile/target SDK from 36 to 37 * chore(android): rebuild driver APKs against API 37 * fix(android): retry transient system_server unavailability in the driver On newer Android system images (notably API 37's 16 KB page-size image) system_server can restart within seconds of boot and again mid-run, so the driver's own pm/am shell calls briefly fail with "Can't find service: package" or "Failure calling service ... Broken pipe" and were thrown as operation failures with no retry. Gate open() on device readiness before installing the driver APKs, and in isPackageInstalled() (the first call in launchApp/clearAppState) wait for the device to settle and retry once when the error is a transient service unavailability. Genuine failures (e.g. a real "pm list" error, a truly missing app) still surface promptly. * ci: validate e2e against Android API 37 Add android-37 to the test-e2e dispatch choices and resolve its 37.1 16 KB page-size (ps16k) image. Harden emulator readiness on that image: wait for the package manager before device config and again after unroot, and retry adb install while the package manager is still coming up. * fix(android): drop the driver retry band-aid; fix the real API 37 e2e cause (cmdline-tools) Root cause of the API 37 e2e failure was CI's cmdline-tools 12.x: avdmanager wrote target=android-0 into the AVD .ini, which misconfigured gfxstream so the guest SIGABRTed surfaceflinger (mapper.ranchu ReadColorBufferDma) at boot and took system_server down, surfacing as 'Broken pipe' on the driver's first pm/am call. - Revert the AndroidDriver awaitDeviceReady/retry (proven insufficient on 37.1 and unnecessary once the tools are correct) and the install_apps retry. - Bump the e2e cmdline-tools to build 15859902 so avdmanager resolves target=android-37.1 (verified in CI: 0 surfaceflinger/ReadColorBufferDma aborts). - The newer avdmanager creates the AVD under ~/.android/avd, so point ANDROID_AVD_HOME and the config.ini append path there.
…-inc#3610) Route all file paths through one place. FileAccessScope.resolve(anchor, candidate) has two forms: everything (relative to the anchor, or absolute as given — today's behavior) and under(root) (the same, but the result has to land under root, canonicalized with toRealPath so symlinks resolve before the check, walking up to the nearest existing ancestor for a not-yet-created path).
Connected iPhones were listed as selectable devices and greeted with "Detected connected iPhone!", then a multi-minute xcodebuild, before the flow's first launchApp hit an unimplemented method in the devicectl driver. Fail at device selection instead, and stop advertising the capability. - MaestroSessionManager: throw CliError for physical iOS devices - DeviceService: stop listing connected iPhones as available devices - RecordCommand: hide --apple-team-id, as test and print-hierarchy do - README: state that physical iOS devices aren't supported yet
…e-dev-inc#3615) The Create AVD step appends hardware settings to a hardcoded ~/.android/avd/MyAVD.avd/config.ini, but the Android tooling resolves its prefs root to $XDG_CONFIG_HOME/.android whenever that directory already exists — ahead of ANDROID_AVD_HOME. On a demo_app APK cache miss the Flutter Gradle build runs first and creates /home/runner/.config/.android, so avdmanager puts the AVD there and the append fails the step under `bash -e`. Setting ANDROID_PREFS_ROOT keeps the prefs root at ~/.android regardless. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
…e-dev-inc#3630) The new Wikipedia Android app (versionCode 50606) rebuilt its onboarding and search entry points in Jetpack Compose, so the flows' Android resource-id selectors no longer resolve: - onboarding: id/fragment_onboarding_forward_button and _done_button are gone; forward is now content-desc "Forward" (x3) and the final screen is dismissed with "Skip". - search: id/search_container is gone; search is reached via the bottom nav "Search" tab (which shows a one-time "A Faster way to Search" widget promo, dismissed conditionally) then the "Search Wikipedia" bar. - launch-clearstate: "Continue"/"Skip" no longer exist on the first onboarding screen; assert the new "All the world's knowledge" / "Learn more about Wikipedia" content instead. Verified end-to-end (android-flow + android-advanced-flow) against the new APK on a local API 34 emulator. Local API 37 emulators could not be used to verify due to an unrelated arm64 emulator graphics crash (surfaceflinger RegionSamplingThread); API 37 is validated via CI on x86_64.
…ile-dev-inc#3632) * feat: record when the full-run screen recording actually started The artifact manifest said nothing about when the full-run recording began, so anyone lining the video up against command timestamps had to assume 0:00 was the first command. That is off by the recorder's startup latency: a fixed 2s sleep on the iOS simulator, and an unbounded screenrecord spawn plus encoder warm-up on Android. Each driver now observes the recorder going live instead of guessing: - Android polls for the output file screenrecord creates right before its first frame and stamps the instant it appears. A stale file from a previous recording is removed first so it cannot trip the probe. - The iOS simulator script waits for simctl to create the file (bounded at 2s) instead of always sleeping 2s, and the host stamps the moment the script reports the recording live. The value is forwarded through IOSScreenRecording and IOSDriver. - Maestro uses the driver's value and falls back to the instant the driver returned for drivers that cannot tell (web). The SCREEN_RECORDING manifest entry carries the value as metadata.startedAtEpochMs, on the same clock as command timestamps. ArtifactCollector.annotate attaches it as soon as the recording starts. The manifest layout is unchanged; older consumers ignore the key. Also fixes Maestro.startScreenRecording leaving screenRecordingInProgress set when the driver throws, which silently turned every later recording on that instance into a no-op. * fix(e2e): make wikipedia iOS onboarding swipe until Get started is visible The subflow swiped a fixed 3 times right after launch. On a slow simulator the first swipe lands while the pager is still animating in and is dropped, leaving the flow on page 3 of 4 with no "Get started" to tap. Wait for the pager to be interactive first, then swipe until the button appears. * feat(web): record screencast frames on a wall-clock timeline and report the start Chrome only sends a screencast frame when the page changes, and the encoder wrote every frame at a fixed 29.97fps. A 30s session with ten page changes became a 0.3s video, and its 0:00 was the first change, not the start of the recording. Nothing in the video could be aligned with command timestamps. Frames are now stamped on arrival and written with their real duration: each shows until the next one, the first is stretched back to 0:00 and the last runs until the screencast stops. The recorder returns the instant the screencast was requested, which is the video's 0:00, so the web driver can report startedAt like the other drivers. Also: a frame that fails to encode is acknowledged so Chrome keeps sending frames instead of freezing the recording; the failure is logged at stop. Odd-sized frames are trimmed to even dimensions, which H264 4:2:0 requires. A recording with no frames writes nothing rather than a header-only file. The scratch file goes through TempFileHandler. * fix(recording): fail the start when the recorder cannot be seen going live A start time that was never observed was reported as unknown on Android and guessed on iOS. Both are now a failed start: - Android: if screenrecord exits before creating its file, its own failure is thrown at start instead of at stop. If the file has not appeared within 10s (was 5s) the recorder is stopped and the start fails. Any other exit from the wait, such as a cancellation, also stops the recorder so it cannot outlive the call. - iOS: screenrecord.sh waited up to 2s for simctl to create the file and then reported the recording started regardless. It now waits up to 10s and reports RECORDING_FAILED if the file never appears. * feat(recording): report the observed start time as is, never a host-clock guess Maestro.startScreenRecording filled in the host clock when a driver reported no start time. Now that every driver either observes its recorder going live or fails the start, ScreenRecording.startedAt is non-null and passed through untouched. The manifest's startedAtEpochMs is therefore always the driver's observation. With no value that could stand for "no recording", the in-progress no-op is gone: startScreenRecording returns null when a recording is already running on the instance and writes nothing. ArtifactsGenerator skips the manifest entry, and Orchestra's startRecording closes and removes the file it had allocated while keeping the running recording stoppable. Also resets the in-progress flag in a finally on the stop path. A driver that failed to stop left the flag set, so every later recording on that instance was silently skipped. * stopping the screencast failed, the encoder must release the output sink. * fix(orchestra): clean up startRecording output when the recorder fails to start A startRecording whose driver threw left its sink open and a 0-byte mp4 on disk, which the manifest then listed as a recording. Route both the command and the full-run recording through one helper that closes the sink and deletes the file whenever no recording starts. * fix(web): drop a frame that fails to encode and always close the scratch video A frame that failed to write stayed held and was retried by every later frame, so one failure lost the rest of the recording. A failure writing the last frame also skipped closing the scratch file's channel. Clear the held frame before writing it, lay frames end to end so the next one covers a dropped frame's time, and close the channel in a finally. Split decoding and color conversion into helpers. * fix(android): wait for the recorder to exit when a recording start fails A start that timed out or was cancelled signalled screenrecord and returned at once, so the next start could remove and reuse the device file while the old recorder was still exiting. Wait (bounded) for it to exit before rethrowing. Shut the recorder's executor down once the task is submitted, so its thread no longer outlives every recording. Split launching, running and stopping the recorder into helpers. * fix(web): register the screencast listener once and stop a half-started screencast Every window change added another frame listener, and Selenium keeps listeners for the whole connection, so each frame was encoded and acknowledged once per window change. A closed recorder's listener also kept receiving later recordings' frames and threw on its shut-down executor. Register one listener per recorder and make it inert once the recorder closes. Record how to stop the screencast before requesting it, so a start that fails midway still stops it, and release a failed start through the same path as close(). * fix(recording): stamp the recording start at the middle of the detection window The start was stamped when the check that found the recording file returned, so it was late by about half a poll interval plus the check itself (about 80ms measured on an API 33 emulator and an iOS 18.2 simulator). Poll every 20ms instead of 100ms on both platforms, and on Android return the midpoint between the last check that missed the file and the one that found it. * docs(manifest): say when a screen recording has no start time startedAtEpochMs is absent when Maestro did not make the recording, not when a driver failed to observe the start: a start that cannot be observed now fails instead. * chore(e2e): drop the comment above the Wikipedia iOS onboarding wait * test: remove the recording-twice integration case
* Prepare for release v2.11.0 * Drop FlowCommandSchema line from 2.11.0 changelog * Apply review wording suggestions to 2.11.0 changelog * Clarify Windows runFlow changelog entry
DouweBos
marked this pull request as ready for review
October 9, 2026 16:24
DouweBos
force-pushed
the
update-from-upstream-74af4e
branch
6 times, most recently
from
October 9, 2026 21:14
6fc6cd9 to
3119b7a
Compare
Flows referencing shared commands previously needed long relative chains like `runFlow: ../../../../command/login.yaml`, since every `file:` path resolves relative to the containing flow's own directory. Add named directory aliases declared once under `paths:` in the workspace config.yaml and referenced as `@alias/rest` (quoted, since `@` is a YAML reserved indicator). Resolution finds the nearest config.yaml by walking up from the flow file, so top-level and nested sub-flows behave identically. Routes runFlow/retry/runScript/addMedia and dependency-watching through a shared FlowPathResolver; non-@ paths keep the existing sibling-relative behavior.
Add a `readFile` command that reads a JSON file from disk and exposes its
parsed contents as a namespaced object variable, so flows reference data
(test ids, queries, etc.) via `${data.field}` instead of hardcoding it.
- New object-typed JS binding (`JsEngine.putObjectEnv`) since env vars are
string-only; nested objects/arrays wrapped as GraalJS proxies for native
`${data.a.b[0]}` access.
- File read at parse time (like runScript); JSON parsed at execution time.
- `outputVariable` reserved-name guard fails fast at parse time.
Also add FORK.md, a reference for every capability this fork adds on top of
upstream Maestro, and link it from AGENTS.md so agents discover fork-only
features instead of reading commits.
The web driver could only launch its own Chrome via Selenium and always drove the first CDP target. Add an attach mode (--cdp-url / --cdp-target) so flows can run against a chosen already-running webview, e.g. a specific Lightning emulator tile. In attach mode input is routed over CDP (Input.dispatchKeyEvent / dispatchMouseEvent) since there is no Selenium session, and the target is resolved by explicit id or APP_ID URL match and cached. The launched-Chrome path is unchanged.
The fork stack had drifted from its own documentation: update-from-upstream listed 8 of 14 commits, FORK.md had no entry for the CDP attach mode, and every commit SHA it cites had been invalidated by the rebase. Add a verify-fork-stack skill that reconciles `git log upstream/main..main` against both tables, checks the cited SHAs still resolve into the stack, and compile-checks — upstream adding a Driver interface member silently breaks fork-only drivers, which is the failure this rebase actually hit.
* fix(ios): bound simulator screen recording teardown simctl can ignore the SIGINT that stops recordVideo (seen on headless tvOS simulators), and the unbounded waitFor in stopScreenRecording then hangs the whole suite. Wait with a timeout and force-kill the recorder plus its descendants on expiry, escalate to SIGKILL inside screenrecord.sh so the moov atom still gets a chance to flush, and never let normalization throw out of teardown. * fix(ios): record the external display on tvOS simulators simctl io recordVideo defaults to --display internal, which tvOS does not have, so it never processed a frame, wrote no file, and ignored the SIGINT that stops it - which is what wedged the recorder teardown. Pass --display external for tvOS simulators, plumbing the platform down from the session manager, which already knows it. * feat(web): record the screen over CDP in attach mode WebScreenRecorder is Selenium-backed, so attaching to a running Chrome/Electron left startScreenRecording a silent no-op that produced an empty file. Drive Page.startScreencast over the raw CDP socket instead and feed the frames to the same encoder, acking each frame so Chrome's delivery gates on the consumer. * fix: handle unsupported screen recording uniformly Drivers signalled 'cannot record' three different ways - TODO(), error() and a silent no-op - so the same situation crashed, failed a flow or produced an empty file depending on the platform. Report it as ScreenRecordingUnsupported and let each caller decide: flows and debug artifacts skip the recording with a warning and no 0-byte file, while 'maestro record' fails with a clear message. Also stop marking a session as recording before the driver actually started, which left every later recording a silent no-op.
getVisiblePercentage only short-circuited when both dimensions were zero, so a box with one zero dimension divided zero by zero and returned NaN. filterOutOfBounds keeps anything failing `visiblePercentage < 0.1`, and NaN fails every comparison, so a collapsed element stayed in the hierarchy and read as visible. A Lightning nav drawer collapsed to [-452,0][-452,72] is what surfaced it.
Post the release tag, release/run links, and commit subjects since the previous cli-* release to #project-automated-testing after publish-cli succeeds. Skipped when SLACK_BOT_TOKEN is unset; never fails the release. Guard upstream's notify-release-comms workflow to mobile-dev-inc/maestro, since its dispatch target is unreachable from the fork.
DouweBos
force-pushed
the
update-from-upstream-74af4e
branch
from
October 9, 2026 22:02
3119b7a to
92c200a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
11 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rebases the fork stack onto upstream Maestro
c73b1202c(v2.11.0 plus later fixes). It was 23 ahead / 38 behind (including #10); it is now 23 ahead / 0 behind. #10 (Slack notification) is carried over on top; its workflow files matchmainexactly.Conflict resolutions
CLI_VERSIONis now upstream's 2.11.0, andPLEX_BUILDis kept, so the version is2.11.0.0.launchApp.CdpClient.openUrlis re-added for it, since upstream removed it.web_flows/iframe.yamland replaced it with its own fixture, so our tweak to it is dropped.launchAppmoved into the attach-mode branch.Maestro.startScreenRecordingtakes upstream's version, which already fixes the stuck "in progress" flag.startScreenRecordingInto, plus our skip for platforms that can't record.screenrecord.shcombines upstream's wait-for-file with our tvOS--display external.FileAccessScopechange (refactor: resolve flow file paths through FileAccessScope mobile-dev-inc/Maestro#3610): added a scopedFlowPathResolver.resolveoverload.Fixes folded into fork commits
resolvePath, and classifyReadFileCommandinRunOutputStaysInBundleTest.FileAccessScopetoparseCommand.main, Case 075 looped forever and 082 failed; upstream passes all three..xctestrun: removed theCodeCoverageBuildableInfosblock that the driver-update bot commits added.run_testsrun only workspaces listed ine2e/workspaces.txt): added rows fortvos_settings,tvos_demo_app,vega_demo_appandroku_demo_app. Each runs only on its own platform with no failing-flow pass, matching the fork's old routing.validate_workspacesnow accepts thetvos,vegaandrokuplatforms. Each change is folded into the commit that adds that platform. Without this, every e2e run failed at the manifest check.tvosadded as a dispatch option;tvOS26.2runtime, sincetvOS26.1is gone from the runner image;tvos_settingscan take about 20 min on CI.TVOS_DEPLOYMENT_TARGET = 16.0so the driver isn't tied to the newest tvOS SDK.Testing
Every fork commit compiles (
:maestro-cli:compileKotlin, plus Studio server where present).Local unit tests: utils 45, orchestra-models 82, web 20, ios 2, ios-driver 64, client 359, orchestra 460, cli 166, test 235, all passing.
CI
Testworkflow passes.CI e2e, verified by flow counts:
A manual dispatch can't select "all apps": GitHub substitutes the
demo_appdefault for an empty input, so each workspace was dispatched by name.tvOS driver built locally: minimum OS is 16.0 for the app, the test bundle and
MaestroDriverLib.Known follow-ups (not in this PR)
tvos_settings), and tvOS screenshots are slow everywhere on CI.IPHONEOS_DEPLOYMENT_TARGET = 14.0is rejected by Xcode 27; bump it to 15.0 before CI moves to Xcode 27.Before releasing
update-driversbot commit to rebuild the iOS simulator driver. Until then the bundled iPhone driver is upstream's build, without the fork's runner changes.