[AMDGPU] Surface QuadrantsAssertionError after in-kernel assert (fix barrier hang)

[paveltc]

Summary

On AMDGPU, a failed in-kernel assert previously emitted asm("S_ENDPGM"), which only terminates the faulting wavefront. Peer wavefronts still waiting on s_barrier then deadlock, and the host hangs forever in hipStreamSynchronize. This PR replaces that with a dispatch-wide __builtin_trap() and translates the resulting fault back into a proper QuadrantsAssertionError on the host, preserving the debug-mode assertion contract without hanging.

CUDA / CPU / Metal paths are unchanged. All new behavior is gated on debug + Arch::amdgpu.

Approach

__builtin_trap() faults the whole dispatch, so the host gets hipErrorLaunchFailure (719) rather than a hang — but the context is then dead, so the usual device-side error-retrieval kernels can no longer run. To preserve the error message:

1. Pinned host-coherent mirror. In materialize_runtime (debug + amdgpu only) we hipHostMalloc(...Coherent) an AmdgpuAssertErrorState and publish its device-mapped address into the runtime. This mirrors the existing adstack_overflow_flag_dev_ptr precedent and survives a device fault.
2. Device publish before trap. In quadrants_assert_format, the faulting wavefront (serialized under the existing error_message_lock) copies the message template + arguments into the pinned buffer, issues a system-scope fence (amdgpu_system_mem_fence, patched to an LLVM seq_cst fence in llvm_context.cpp), stores error_code last, then __builtin_trap()s.
3. Host translation. AMDGPUFunction::operator() intercepts hipErrorLaunchFailure, and a debug-only hook reads the pinned state and raises QuadrantsAssertionError (a subclass of AssertionError). Subsequent 719s on the now-dead context are ignored so Program teardown does not terminate() from a destructor.

Testing

Validated on an AMD Instinct MI308X (gfx942), ROCm 7.2.4, base upstream/main:

• tests/python/test_assert.py::test_amdgpu_assert_raises — a failed assert raises QuadrantsAssertionError with the formatted message; isinstance(e, AssertionError) holds.
• tests/python/test_assert.py::test_amdgpu_assert_barrier_no_hang — one thread asserts while siblings hit block.sync(); raises instead of hanging (the original bug).
• CPU sanity (test_assert_*) unchanged.

Both new tests run each case in an isolated child subprocess (the HIP context is dead after a trap; HIP is unsafe after fork) with a wall-clock timeout that fails on the barrier-hang regression.

CI notes

Upstream AMDGPU CI (test_gpu.yml → test_linux_amdgpu, runs-on: amdgpu) runs bare-metal on the self-hosted runner — no container — which matches the environment where the trap returns a catchable hipErrorLaunchFailure. Some ROCm/HSA configs (notably inside Docker) instead escalate the trap to an uncatchable SIGABRT; the tests treat a SIGABRT-killed child as pytest.skip (environment limitation) while still failing on timeout or on a wrong/absent exception, so no runner goes spuriously red.

Known limitations / possible follow-ups

• After an assert, the HIP context is dead — one assert per process (tests isolate per subprocess). Accepted debug-mode limitation.
• The pinned struct is duplicated as a layout-compatible host view (AmdgpuAssertErrorStateHostView) in llvm_runtime_executor.cpp; a static_assert on size/offsets would harden this.
• The launch-failure hook and surfaced-flag are process-global singletons (assume a single active Program).

Made with Cursor

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: eb26004957

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

Publish assert state before peer wavefronts can trap

When an assertion fails in multiple wavefronts, the winning wave sets runtime->error_code = 1 before copying and publishing the pinned state. Another wavefront can observe that value, skip the locked section, and reach this unconditional trap before st->error_code is stored; the host hook then sees code 0 and reports a generic HIP launch failure instead of QuadrantsAssertionError. Ensure only the publisher can trap, or make other failing wavefronts wait until the pinned error state has been published.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Do not swallow post-assert failures during normal execution

After the first caught AMDGPU assertion marks this process-global flag, every later HIP call returning 719 silently succeeds through this return, not just calls made during teardown. For example, the existing test_assert_minimal flow catches func() and then expects func2() to raise, but the dead context's launch/synchronization failures are ignored, allowing subsequent kernels or copies to appear successful and expose stale results. Limit suppression to the destructor/finalization path and continue reporting the dead context during ordinary API calls.

Useful? React with 👍 / 👎.

[paveltc]

Superseded by #871, which raises the same change from the AMD-Ecosystem organization fork per policy (topic branch on an AMD org fork → upstream). Closing this one to avoid duplicates.