Skip to content

GHA: split test.yml into per-platform files to drop cross-platform skip noise #245

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
compnerd merged 1 commit into from
Jul 28, 2026
Merged

GHA: split test.yml into per-platform files to drop cross-platform skip noise #245

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 42 additions & 0 deletions .github/actions/resolve-test-subdirs/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
name: Resolve lldb test subdirectories
description: >-
Maps a test-category matrix value (misc/commands/Python API/functionalities)
to the concrete lldb test/API subdirectories to run. The misc category's
directory list is platform-specific (each platform test.yml file has its
own set of OS-specific test dirs mixed in), so it's passed in by the caller
rather than hardcoded here.

inputs:
test-category:
description: The matrix.test-category value (misc, commands, Python API, functionalities)
required: true
misc-dirs:
description: Space-separated list of directories to use for the "misc" category on this platform
required: true

outputs:
dirs:
description: Space-separated list of resolved test subdirectories
value: ${{ steps.resolve.outputs.dirs }}

runs:
using: composite
steps:
- id: resolve
shell: bash
run: |
case "${{ inputs.test-category }}" in
misc)
dirs="${{ inputs.misc-dirs }}"
;;
commands)
dirs="SourceCache/llvm/lldb/test/API/commands"
;;
"Python API")
dirs="SourceCache/llvm/lldb/test/API/python_api"
;;
functionalities)
dirs="SourceCache/llvm/lldb/test/API/functionalities"
;;
esac
echo "dirs=$dirs" >> "$GITHUB_OUTPUT"
6 changes: 2 additions & 4 deletions .github/workflows/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -472,14 +472,12 @@ jobs:
# waiting for every build job to complete before any test starts.
test-linux:
needs: [vars, linux, build-lldb-linux]
uses: ./.github/workflows/test.yml
uses: ./.github/workflows/test-linux.yml
with:
llvm-ref: ${{ needs.vars.outputs.llvm-ref }}
platform: linux

test-android:
needs: [vars, android-windows-ndk, build-lldb-linux]
uses: ./.github/workflows/test.yml
uses: ./.github/workflows/test-android.yml
with:
llvm-ref: ${{ needs.vars.outputs.llvm-ref }}
platform: android
232 changes: 232 additions & 0 deletions .github/workflows/test-android.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,232 @@
name: Test Android

on:
workflow_call:
inputs:
llvm-ref:
description: llvm-project commit/ref the build artifacts were built against
required: true
type: string

jobs:

# -----------------------------------------------------------------------
# Android
# -----------------------------------------------------------------------
#
# The emulator-runner action takes the whole test invocation as a single
# shell script string executed inside its own wrapper, so it can't invoke
# the run-lldb-tests composite action as a normal step the way the Linux
# jobs do -- the dotest invocation stays inlined here, same as before this
# file existed.

test-android-x86_64:
runs-on: ubuntu-latest
name: Test Android x86_64

strategy:
fail-fast: false
matrix:
test-category: [misc, commands, Python API, functionalities]

steps:
- uses: actions/download-artifact@v8
with:
name: llvm-Linux-X64
path: ${{ github.workspace }}/BinaryCache
- run: tar -C ${{ github.workspace }}/BinaryCache -xvf ${{ github.workspace }}/BinaryCache/llvm.tar

- uses: actions/download-artifact@v8
with:
name: android-x86_64-ds2
path: ${{ github.workspace }}/BinaryCache
- run: tar -C ${{ github.workspace }}/BinaryCache -xvf ${{ github.workspace }}/BinaryCache/ds2.tar

- name: Checkout llvm project
uses: actions/checkout@v7
with:
repository: llvm/llvm-project
ref: ${{ inputs.llvm-ref }}
path: ${{ github.workspace }}/SourceCache/llvm

- name: Checkout ds2
uses: actions/checkout@v7
with:
path: ${{ github.workspace }}/SourceCache/ds2

- name: Resolve test subdirectories
id: test-subdirs
uses: ./SourceCache/ds2/.github/actions/resolve-test-subdirs
with:
test-category: ${{ matrix.test-category }}
misc-dirs: "SourceCache/llvm/lldb/test/API/android SourceCache/llvm/lldb/test/API/api SourceCache/llvm/lldb/test/API/lang SourceCache/llvm/lldb/test/API/linux SourceCache/llvm/lldb/test/API/tools SourceCache/llvm/lldb/test/API/sanity SourceCache/llvm/lldb/test/API/types"

# Cache the contents of $HOME/.android which contains emulators,
# snapshots, and adb state.
- name: Setup AVD cache
uses: actions/cache@v6
id: avd-cache
with:
path: ~/.android/**
key: avd-${{ runner.os }}-x86_64-28

- name: Enable KVM
run: |
echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' | sudo tee /etc/udev/rules.d/99-kvm4all.rules
sudo udevadm control --reload-rules
sudo udevadm trigger --name-match=kvm

- name: Create AVD and generate snapshot for caching
if: steps.avd-cache.outputs.cache-hit != 'true'
uses: reactivecircus/android-emulator-runner@v2
with:
api-level: 28
arch: x86_64
force-avd-creation: false
emulator-options: -no-window -noaudio -no-boot-anim -camera-back none -gpu swiftshader_indirect
disable-animations: false
script: echo "Generated AVD snapshot for caching."

- name: Run lldb tests against ds2 on Android emulator
uses: reactivecircus/android-emulator-runner@v2
with:
api-level: 28
arch: x86_64
force-avd-creation: false
# NOTE: -no-snapshot will not load or save state and the emulator will
# cold boot. Ideally, we would specify -no-snapshot-save instead so it
# would load the snapshot created in the caching step (but not save a
# new one). However, doing that causes the emulator to hang during
# shutdown. We're still saving time by reusing the device created in
# the caching step and not wiping userdata on boot.
emulator-options: -no-snapshot -no-window -noaudio -no-boot-anim -camera-back none -gpu swiftshader_indirect
disable-animations: true
script: >
$ANDROID_SDK_ROOT/platform-tools/adb forward tcp:5432 tcp:5432

$ANDROID_SDK_ROOT/platform-tools/adb push "${{ github.workspace }}/BinaryCache/ds2/ds2" /data/local/tmp

$ANDROID_SDK_ROOT/platform-tools/adb shell "chmod +x /data/local/tmp/ds2"

$ANDROID_SDK_ROOT/platform-tools/adb shell "/data/local/tmp/ds2 platform --server --daemonize --listen localhost:5432"

${{ github.workspace }}/BinaryCache/llvm/bin/lldb-dotest
--out-of-tree-debugserver
--arch x86_64
--platform-name remote-android
--platform-url connect://localhost:5432
--platform-working-dir /data/local/tmp
--compiler=$ANDROID_NDK_LATEST_HOME/toolchains/llvm/prebuilt/linux-x86_64/bin/x86_64-linux-android28-clang
--excluded ${{ github.workspace }}/SourceCache/ds2/Support/Testing/Excluded/ds2/android-x86_64.excluded
--excluded ${{ github.workspace }}/SourceCache/ds2/Support/Testing/Excluded/upstream/android-x86_64.excluded
--excluded ${{ github.workspace }}/SourceCache/ds2/Support/Testing/Excluded/upstream/non-debugserver-tests.excluded
-v
${{ steps.test-subdirs.outputs.dirs }}
1>/dev/null 2>/tmp/dotest-android-x86_64.log;
rc=$?; grep -v '^UNSUPPORTED: LLDB ' /tmp/dotest-android-x86_64.log; exit $rc

test-android-x86_64-status:
needs: [test-android-x86_64]
if: always()
runs-on: ubuntu-latest
name: Test Android x86_64 (status)
steps:
- if: needs.test-android-x86_64.result != 'success'
run: exit 1

# Run a small sub-set of tests on arm64. The emulator is very slow so it is
# not practical to run more tests until we can run on an arm64 runner with
# virtualization. Only one category runs here, so this already reports as
# a single check with no matrix/grouping concerns.
test-android-arm64-v8a:
runs-on: ubuntu-latest
name: Test Android arm64-v8a sanity

steps:
- uses: actions/download-artifact@v8
with:
name: llvm-Linux-X64
path: ${{ github.workspace }}/BinaryCache
- run: tar -C ${{ github.workspace }}/BinaryCache -xvf ${{ github.workspace }}/BinaryCache/llvm.tar

- uses: actions/download-artifact@v8
with:
name: android-arm64-v8a-ds2
path: ${{ github.workspace }}/BinaryCache
- run: tar -C ${{ github.workspace }}/BinaryCache -xvf ${{ github.workspace }}/BinaryCache/ds2.tar

- name: Checkout llvm project
uses: actions/checkout@v7
with:
repository: llvm/llvm-project
ref: ${{ inputs.llvm-ref }}
path: ${{ github.workspace }}/SourceCache/llvm

- name: Checkout ds2
uses: actions/checkout@v7
with:
path: ${{ github.workspace }}/SourceCache/ds2

- name: Setup AVD cache
uses: actions/cache@v6
id: avd-cache
with:
path: ~/.android/**
key: avd-${{ runner.os }}-arm64-v8a-27

- name: Enable KVM
run: |
echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' | sudo tee /etc/udev/rules.d/99-kvm4all.rules
sudo udevadm control --reload-rules
sudo udevadm trigger --name-match=kvm

- name: Create AVD and generate snapshot for caching
if: steps.avd-cache.outputs.cache-hit != 'true'
uses: reactivecircus/android-emulator-runner@v2
with:
# Android emulator supports ARM on x86 only on API < 28.
api-level: 27
arch: arm64-v8a
force-avd-creation: false
# This is the most recent build of the emulator that supports
# booting arm64 images.
emulator-build: 12234392
emulator-options: -no-window -noaudio -no-boot-anim -camera-back none -gpu swiftshader_indirect -qemu -machine virt -cpu max
disable-animations: false
script: echo "Generated AVD snapshot for caching."

- name: Run lldb tests against ds2 on Android emulator
uses: reactivecircus/android-emulator-runner@v2
with:
api-level: 27
arch: arm64-v8a
force-avd-creation: false
emulator-build: 12234392
emulator-options: -no-snapshot -no-window -noaudio -no-boot-anim -camera-back none -gpu swiftshader_indirect -qemu -machine virt -cpu max
disable-animations: true
script: >
$ANDROID_SDK_ROOT/platform-tools/adb forward tcp:5432 tcp:5432

$ANDROID_SDK_ROOT/platform-tools/adb push "${{ github.workspace }}/BinaryCache/ds2/ds2" /data/local/tmp

$ANDROID_SDK_ROOT/platform-tools/adb shell "chmod +x /data/local/tmp/ds2"

$ANDROID_SDK_ROOT/platform-tools/adb shell "/data/local/tmp/ds2 platform --server --daemonize --listen localhost:5432"

${{ github.workspace }}/BinaryCache/llvm/bin/lldb-dotest
--out-of-tree-debugserver
--arch aarch64
--platform-name remote-android
--platform-url connect://localhost:5432
--platform-working-dir /data/local/tmp
--compiler=$ANDROID_NDK_LATEST_HOME/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android27-clang
--excluded ${{ github.workspace }}/SourceCache/ds2/Support/Testing/Excluded/ds2/android-aarch64.excluded
--excluded ${{ github.workspace }}/SourceCache/ds2/Support/Testing/Excluded/upstream/android-aarch64.excluded
--excluded ${{ github.workspace }}/SourceCache/ds2/Support/Testing/Excluded/upstream/non-debugserver-tests.excluded
-v
SourceCache/llvm/lldb/test/API/android
SourceCache/llvm/lldb/test/API/api
SourceCache/llvm/lldb/test/API/linux
SourceCache/llvm/lldb/test/API/sanity
1>/dev/null 2>/tmp/dotest-android-arm64-v8a.log;
rc=$?; grep -v '^UNSUPPORTED: LLDB ' /tmp/dotest-android-arm64-v8a.log; exit $rc
Loading
Loading