Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 45 additions & 21 deletions .github/workflows/run-maestro-tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -168,31 +168,27 @@ jobs:
adb wait-for-device
adb shell 'while [ "$(getprop sys.boot_completed)" != "1" ]; do sleep 2; done'

# Put SELinux in permissive mode. The offline map extract is pushed
# into the app's scoped external-storage dir by the shell/root user
# below, so it lands with the SELinux label "storage_file" and none of
# the app's per-process MLS categories. Enforcing SELinux then denies
# the app ("untrusted_app") read access to its own extract
# (avc: denied { read } ... tcontext=...:storage_file), so every tile
# falls through to the network. Permissive mode lets the app read the
# pushed file; it is acceptable on this userdebug test emulator and is
# set after the framework restart above so it is in effect for the run.
adb shell setenforce 0

adb logcat -c # clear logs
touch app/emulator.log # create log file
chmod 777 app/emulator.log # allow writing to log file
adb logcat >> app/emulator.log & # pipe all logcat messages into log file as a background process
adb install ${{ steps.find-apk.outputs.apk_path }}

# Push the Glasgow offline map extract into the app's offline-extracts
# directory and pin the emulator GPS to the STA office, so the
# location-dependent flows run against known map data and a fixed
# location. The extracts directory is the app-specific external files
# dir + "/Download" (see GeoEngine.offlineExtractPath); scoped storage
# blocks writes there even for the shell user, which is why we took
# root above.
#
# The android-emulator-runner runs each script line in its own
# "sh -c", so a shell variable assigned on one line is empty by the
# next; the extract directory path is therefore inlined rather than
# held in a variable. It is the app-specific external files dir +
# "/Download" (see GeoEngine.offlineExtractPath).
adb shell mkdir -p /storage/emulated/0/Android/data/org.scottishtecharmy.soundscape/files/Download
adb push glasgow-gb.pmtiles /storage/emulated/0/Android/data/org.scottishtecharmy.soundscape/files/Download/
# The .geojson metadata sidecar (findExtracts reads "<pmtiles>.geojson")
# is committed under .github/fixtures as it is too small to be worth a
# download, unlike the pmtiles itself.
adb push .github/fixtures/glasgow-gb.pmtiles.geojson /storage/emulated/0/Android/data/org.scottishtecharmy.soundscape/files/Download/
# Pin the emulator GPS to a fixed location (Edinburgh) so the
# location-dependent flows run against known map data. This is emulator
# GPS state, not app state, so it survives the app data being cleared
# and only needs setting once here.
# adb emu geo fix takes longitude then latitude.
adb emu geo fix -3.223538 55.955360

Expand All @@ -201,15 +197,43 @@ jobs:
# the end if any of them failed.
#
# This MUST stay a single line. android-emulator-runner runs each
# script line in its own "sh -c" (see the comment above), so a
# script line in its own "sh -c", so a shell variable assigned on one
# line is empty by the next; a
# "status" variable accumulated across separate lines, with a trailing
# "exit $status", does not work: each line gets a fresh shell, the
# "|| status=1" makes the failing line still exit 0, and the final
# "exit $status" sees an empty variable and exits 0 - which left the
# step (and the whole run) green even when every flow failed. Keeping
# the loop and the exit in one shell makes the non-zero status stick.
#
# The Onboarding flow runs "launchApp: clearState: true" (pm clear),
# which deletes the app's external-storage directory - including any
# offline map extract pushed there. So Onboarding runs FIRST, and only
# then is the Glasgow extract pushed into the (now app-created)
# offline-extracts directory, before the location-dependent flows run.
# Pushing earlier is pointless: clearState would wipe it and every tile
# request would fall through to the network (which has no provider URL
# on fork PRs, producing hundreds of errors). The extracts directory is
# the app-specific external files dir + "/Download" (see
# GeoEngine.offlineExtractPath); scoped storage blocks writes there even
# for the shell user, which is why we took root above. The .geojson
# metadata sidecar (findExtracts reads "<pmtiles>.geojson") is committed
# under .github/fixtures as it is too small to be worth a download,
# unlike the pmtiles itself. The push stays inside this one shell so the
# "status" aggregation above keeps working.
#
# "Unable to launch app" launch timeouts are intermittent on the
# heavier google_apis images - the app itself does not crash and the
# next flow launches fine. The rf() helper runs a flow and retries it
# ONCE, but ONLY when its JUnit report shows that exact launch failure.
# launchApp is the first command, before a flow does any work, so a
# retry then is safe. A blind retry-on-any-failure would NOT be: the
# stateful flows (e.g. LocationDetails assumes an empty database and
# creates markers A and B, asserting their order) would double-apply
# their changes if re-run after a mid-flow failure. The retry reuses the
# same --output path so the report reflects the final attempt.
# To add a flow, add its name (without .yaml) to the list below.
status=0; for flow in Onboarding HomePage LocationDetails PlacesNearby MarkersAndRoutes RouteCreation; do $HOME/.maestro/bin/maestro test --format=junit --output="maestro_outputs/report-$flow.xml" --test-output-dir=maestro_outputs --no-ansi "maestro/$flow.yaml" || status=1; done; exit $status
status=0; rf(){ $HOME/.maestro/bin/maestro test --format=junit --output="maestro_outputs/report-$1.xml" --test-output-dir=maestro_outputs --no-ansi "maestro/$1.yaml"; r=$?; if [ "$r" -ne 0 ] && grep -q "Unable to launch app" "maestro_outputs/report-$1.xml" 2>/dev/null; then echo "::warning::launch of $1 failed; retrying once"; $HOME/.maestro/bin/maestro test --format=junit --output="maestro_outputs/report-$1.xml" --test-output-dir=maestro_outputs --no-ansi "maestro/$1.yaml"; r=$?; fi; return "$r"; }; rf Onboarding || status=1; adb shell mkdir -p /storage/emulated/0/Android/data/org.scottishtecharmy.soundscape/files/Download; adb push glasgow-gb.pmtiles /storage/emulated/0/Android/data/org.scottishtecharmy.soundscape/files/Download/; adb push .github/fixtures/glasgow-gb.pmtiles.geojson /storage/emulated/0/Android/data/org.scottishtecharmy.soundscape/files/Download/; for flow in HomePage LocationDetails PlacesNearby MarkersAndRoutes RouteCreation; do rf "$flow" || status=1; done; exit $status

# Artifact names must be unique across the matrix, so suffix them with the
# combination (e.g. maestro-reports-35-large-ja-JP).
Expand Down