Add rocm CI - #343
Open
victorapm wants to merge 30 commits into
Open
Conversation
…hirdPartyLibs into feature/paludettomag1/rocm-ci
…es before docker build
…t explicitly in the Dockerfile
- build.sh: single driver for z5tux (cpu/cuda/hip) and LC (matrix, tioga, tuo-cpu, tuo-gpu), cmake+ninja by default, --spack for uberenv builds - CMakeLists: HIP plumbing for raja, chai, caliper and hypre - Make python explicit in the py-sphinx externals (spack v1.2 deprecation)
Master reworked the docker CI: the per-platform TPL Dockerfiles and docker/spack.yaml were replaced by parameterized docker/tpl-ubuntu.Dockerfile and docker/tpl-rockylinux.Dockerfile layered on the geosx/<os> base images from docker_base_images, driven by SPEC/DOCKER_BASE_IMAGE from the workflow matrix, plus a new scripts/reproduce_ci.sh for running a CI row locally. The ROCm work is ported onto that structure rather than merged alongside it: * docker/tpl-ubuntu-hip.Dockerfile takes DOCKER_BASE_IMAGE (was DOCKER_ROOT_IMAGE) and SPEC as build args instead of hardcoding the spec, and reads the build context from the readwrite mount like the other TPL Dockerfiles, dropping the uberenv named-context workaround. It keeps its own Dockerfile because AMD's ROCm image is not a docker_base_images image and ships its own toolchain. Also redeclares AMDGPU_TARGET/ROCM_VERSION in the final stage so the ROCm ENV defaults interpolate. * The ROCm row moves into the workflow matrix in master's style. Rows that do not build on a docker_base_images image now set DOCKER_BASE_IMAGE_OVERRIDE. * CA injection handles apt-based images as well as dnf-based ones, which the ROCm and Ubuntu CUDA rows on streak2 need. * scripts/reproduce_ci.sh gains the ROCm row via two optional trailing matrix fields (base_image, extra_env); existing rows are unchanged. * scripts/run-rocm-ci-local.sh is removed, superseded by reproduce_ci.sh. Conflict resolutions: took master's newer hypre pin and new hypredrive pin; took master's hypredrive/addr2line/multi-arch-CUDA changes in the geosx package and kept this branch's cmake_cache_path/cmake_cache_string conversion on top.
Brings spack v1.2.2 (and the Jul 20th 2026 package repo), the unified build.sh with HIP support in the TPL superbuild, removal of blueos/Lassen/ESSL support, and Trilinos defaulting off. Conflicts: kept this branch's spack_setup_clingo=0 alongside the v1.2.2 spack pins (all three TPL Dockerfiles pip-install clingo themselves); took v1.2.2's removal of the ppc64le/blueos and ESSL blocks in the geosx package while keeping this branch's cmake_cache_path/cmake_cache_string conversion. docker/spack-rocm.yaml was the only spack env still carrying externals copied from an Ubuntu 22.04 config, which spack v1.2.2 would take at face value since they are declared buildable:false. Versions verified against rocm/dev-ubuntu-24.04:6.4.3 and corrected: python 3.10.12 -> 3.12.3, py-sphinx 4.3.2 -> 7.2.6, openmpi 4.1.2 -> 4.1.6, zlib 1.2.11 -> 1.3, readline 8.0 -> 8.2, perl 5.34.0 -> 5.38.2, tar 1.34 -> 1.35, xz 5.2.5 -> 5.6.1, findutils 4.7.0 -> 4.9.0, netlib-blas/lapack 3.10.0 -> 3.12.0, gmake 4.4.1 -> 4.3, cmake 3.28.3 -> 3.28.6, and mpfr 6.0.2 -> 4.2.1 (6.0.2 was never an mpfr release). It also picks up v1.2.2's external-spec syntax change, moving the py-sphinx python constraint from an inline '^' into a dependencies list.
docker/rocky-spack.yaml declares xz as a non-buildable external at /usr, but
the image only installed the xz command, not its headers, so /usr/include/lzma.h
was absent. Nothing needed it until the spack v1.2.2 package repo pulled in
file@5.46, whose configure then failed the Rocky builds with:
configure: error: xzlib support requested but not found
Ubuntu is unaffected: file never enters its dependency graph.
Note the same file still declares xz@5.2.5 while Rocky 8 actually ships 5.2.4;
that entry is shared with the Rocky 9 rows, which do ship 5.2.5, so it is left
alone here rather than split per-distro.
Bash only treats literal VAR=value words as assignments, so the expanded
${extra_env} was parsed as the command name and the ROCm row died with
'ROCM_VERSION=6.4.3: command not found' before docker build ran.
The earlier external-version corrections updated the external specs but not the
matching 'version:' preference for cmake and findutils. Both are buildable:false,
so the stale preference matched no known version and spack refused to concretize:
==> Error: Preference for version 3.28.3 does not match any known version of cmake
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add Ubuntu 24.04, gcc 13.3.0 + ROCm 6.4.3 Docker image