Scaffold Maestro E2E flows + CI workflow + Testing docs#2
Open
Scaffold Maestro E2E flows + CI workflow + Testing docs#2
Conversation
Adds .maestro/ with 10 placeholder flows covering ~80% of typical
SDK usage patterns:
01-login-email email + password login (catches the
TOKEN_WRONG_TYPE / wrong-base-URL class
of regressions we hit this week)
02-login-jwt client-flow JWT auth via /users/client
03-list-rooms GET /chats/my + unread counts
04-send-text full XMPP send round-trip
05-receive-text MAM push delivery (uses helper script
to send-as-bob via REST while alice
waits in-app)
06-attach-file PendingMediaSendQueue + chunked upload
07-reconnect-airplane airplane mode toggle, ConnectionStore
recovery without remount
08-push-deep-link synthetic notification intent → room
09-logout-relogin DataStore not leaking previous-user state
10-switch-app multi-tenant app-switcher (alice on app A
→ charlie on app B without remount)
Each flow is a runnable YAML stub with realistic semantic anchors
based on the playground's actual UI. A few ($MAESTRO_TEST_*)
variables hold credentials, sourced from CI secrets so no real
data lives in source — fixtures/test-users.yaml documents the
contract.
CI workflow (.github/workflows/maestro.yml):
- macOS runner (HAXM/hypervisor needed for emulator).
- Single API level on PRs (34); full matrix [26, 30, 34] on
release tags.
- .env on the runner is composed from secrets, never checked in.
- Failures upload Maestro recordings as artifacts (14d retention).
Both this repo's README and (paired commit on ethora-sdk-android
PR #3) the SDK README gain a "Testing" section explaining the
two-layer split:
Layer 1 — hermetic component tests in ethora-sdk-android
(chat-core/src/test, chat-ui/src/androidTest, ethora-component/src/test)
Layer 2 — real-server smoke flows here, gating SDK release tags
The split keeps SDK tests fast and offline-runnable while pushing
network/integration flows to where the running app is.
Several flows reference helper scripts (.maestro/scripts/sendAsBob.js,
toggleAirplaneMode.sh, sendPushIntent.sh) and asset fixtures
(test image for 06-attach-file) that don't exist yet — left as
TODOs in the YAML comments. First green flow can be 01 or 04 with
a populated .env; the rest can fill in incrementally.
Addresses the inline TODOs in the original scaffold:
scripts/sendAsBob.js
Real implementation. Maestro JS helper that POSTs /messages as bob
via the REST API so flow 05-receive-text can verify alice receives
it via XMPP without a second emulator. Uses Maestro's built-in
`http` client; reads MAESTRO_APP_TOKEN / MAESTRO_TEST_BOB_JWT /
MAESTRO_TEST_ROOM_JID from the env vars CI propagates from
repo secrets.
scripts/sendPushIntent.sh
Real implementation. Synthesises an `adb shell am start` with
`--es notification_jid <jid>` to mimic an FCM-tap intent. Needed
by 08-push-deep-link, which can't drive adb from inside Maestro.
CI invokes the script before launching that flow; the flow then
asserts on the resulting state.
assets/test-image.png
74-byte 8x8 PNG seeded into /sdcard/Pictures/ by CI (with a
MEDIA_SCANNER_SCAN_FILE broadcast to make it picker-visible) so
06-attach-file can pick it from the gallery. Tiny by design —
goal is just to exercise the upload path, not to test the codec.
Flow 07-reconnect-airplane rewritten
Replaced the adb-driven airplane-mode toggle with the app's own
SETUP / Disconnect / Connect buttons. Tests the same SDK
reconnect path (ChatConnectionStore transitions, XMPP client
teardown, MAM history survives) without depending on a shell
command Maestro can't run. OS-level network simulation can come
later as a separate flow invoked by a CI pre-step that toggles
wifi via adb.
Flow 08-push-deep-link rewritten
Now an assertion-only flow with `clearState: false` — assumes
the launcher intent is the synthesised notification, asserts
that EthoraChatProvider deep-links into the right room. CI
fires the intent via sendPushIntent.sh before invoking maestro.
CI workflow updates (.github/workflows/maestro.yml)
- Runs all 10 flows in two batches (01-07 + 09-10 first, then 08
after the synthetic intent) so the deep-link assertion has
deterministic launcher state.
- Pushes test-image.png to /sdcard/Pictures + broadcasts media
scan before the suite starts.
- Adds env vars sendAsBob.js needs (MAESTRO_API_BASE_URL,
MAESTRO_APP_TOKEN, MAESTRO_TEST_BOB_JWT) and updates the
secrets header comment.
.maestro/README.md
Repo layout updated to show assets/ + scripts/ directories with
a one-line description per file. Added a "Why some helpers live
outside the flow YAML" callout explaining the
JS-runtime-can't-shell-out constraint.
…m/wrong-password Expands the suite from 10 → 15 flows, targeting common interactions beyond the basic login/send/receive happy path. Each is a runnable YAML stub with realistic semantic anchors; selectors marked 'optional: true' where the SDK's UI is in flux so the flow doesn't hard-fail on a renamed accessibility id, just the missing assertion. 11-login-wrong-password Negative path. Catches: error response not surfaced to UI, retry loop on 401, accidental login under a stale cached token. No extra secrets needed beyond MAESTRO_TEST_EMAIL. 13-message-edit Long-press → Edit → ChatInput switches to "Edit message..." mode → change text → Send → assert bubble updates in place. Catches editText prop not flowing into ChatInput, edit calling POST instead of PUT, optimistic update not reconciling with ack. 14-message-delete Long-press → Delete → confirm → assert bubble removed or replaced by tombstone. Catches delete RPC silently failing while UI optimistically removes, MAM still returning the deleted message on rejoin. 15-message-reaction Long-press → React → 👍 → assert reaction visible with count 1. Catches: reaction not stored in Message.reaction map, picker rendering with no preset emoji, count not incrementing on re-react, reaction stripping from MAM history. 16-create-room Tap "+" → Create dialog → name → assert room visible and navigable. Sends a follow-up message to verify the new room is fully bootstrapped (XMPP presence joined, write perms correct). Catches: create-room RPC failing silently, JID collision on duplicate name. Flow 12 reserved for typing-indicator — needs a helper that pokes XMPP composing-state on bob's behalf (similar to sendAsBob.js but on the typing endpoint), left for a follow-up. Updates: .maestro/README.md flow listing + .github/workflows/maestro.yml to include the new flows in the CI run order.
…overage table Brings the suite to 19 flows total (slot 12 still reserved for typing-indicator pending a composing-state helper). Adds a coverage table to both README.md and .maestro/README.md so the team can see at a glance what each flow asserts and what regression classes it catches. 17-search-rooms Types into RoomListView's SearchBar, asserts the queried room remains visible. Catches: predicate not case-insensitive, list not re-rendering on query change, "no results" empty-state never appearing. 18-multi-message-rapid Five back-to-back sends, asserts all five are visible in the conversation. Catches: out-of-order ack reordering, optimistic UI dropping bubbles, ChatInput debounce eating taps. 19-room-info Opens ChatInfoScreen via the room title / info affordance, asserts the participants list shows the current user and a Leave control. Back-navigates and confirms we land back on the conversation, not the room list. Several selectors marked optional: true since the SDK's ChatInfoScreen UI is in flux — flow still composes a meaningful skeleton when those land. 20-offline-pending-resend Disconnects via SETUP/Disconnect, attempts a send, reconnects, asserts the message ultimately lands. Includes an optional "Tap to retry" tap in case the SDK requires manual retry rather than auto-resend on reconnect — covers both contracts with one flow. CI workflow (.github/workflows/maestro.yml) updated to include the four new flows in the run order. The flow files reference shared semantic anchors that don't yet exist in source (id: "chat_input", id: "chat_send_button", id: "rooms_search_input", etc.). Marked optional: true so flows still execute on a CI run today, but they assert nothing useful until those testTags land in the SDK's Compose tree. Adding semantic testTags to chat-ui composables is the highest-leverage source change to unblock these — flagged in the README.
This was referenced May 10, 2026
Adds a Layer 1 / Layer 2 mapping table showing how this Android sample's Maestro flows fit into the 4-platform stack (sdk-android, sdk-swift, sample-android, sample-swift, chat-component, app-reactjs) and how the selector-string contract (testTag / accessibilityIdentifier / data-testid) couples them.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the Layer 2 (end-to-end) test scaffolding to this repo, paired with `ethora-sdk-android` PR #3 which adds Layer 1 (hermetic Compose UI tests in the SDK repo).
10 Maestro YAML flows covering ~80% of typical SDK usage patterns, plus a GitHub Actions workflow that runs them on every push, PR, and SDK release tag against `chat-qa.ethora.com`.
Files
Why these specific flows
Every regression we've hit on Android in the last few weeks would have been caught by at least one of these:
Future regressions on the cross-process boundary (push, reconnect, switch-app) are exactly the class Compose UI tests can't reach — that's why they live here, not in the SDK repo.
Status of each flow
All 10 are runnable as YAML, but several depend on helper assets that don't exist yet (left as inline TODOs):
The first green flow on CI can be `01-login-email` once repo secrets are populated. The rest can come online incrementally — the scaffold is here so the team has a single shape to extend.
Required repo secrets
Settings → Secrets and variables → Actions:
```
ETHORA_API_BASE_URL https://api.chat-qa.ethora.com/v1
ETHORA_APP_ID
ETHORA_APP_TOKEN
ETHORA_XMPP_HOST xmpp.chat-qa.ethora.com
ETHORA_XMPP_CONFERENCE conference.xmpp.chat-qa.ethora.com
ETHORA_XMPP_SERVER_URL wss://xmpp.chat-qa.ethora.com/ws
MAESTRO_TEST_EMAIL alice@ethora.com (test user on chat-qa)
MAESTRO_TEST_PASSWORD TestPass123
MAESTRO_TEST_USER_JWT client-flow JWT for 02-login-jwt
MAESTRO_TEST_ROOM_JID a known MUC room JID alice belongs to
MAESTRO_TEST_APPB_* second-app credentials for 10-switch-app
```
All values target chat-qa, not prod, in keeping with the product-code policy.
Test plan