Skip to content

[BUG]: Free-threaded CI: coredump bool regression test races on application-wide settings #3225

Closed

Description

@jpascucci-nv

Is this a duplicate?

Type of Bug

Runtime Error

Component

cuda.bindings

Describe the bug

(From GPT 6.1 Sol High)

Both ARM64 free-threaded jobs in CI run 40126604939 failed in test_coredump_bool_uses_single_byte_storage:

driver.coredump_set_attribute_global(driver.CoredumpSettings.LIGHTWEIGHT, False)
assert driver.coredump_get_attribute_global(driver.CoredumpSettings.LIGHTWEIGHT) is False
# AssertionError: assert True is False

Likely cause
PR #3095 changed this test from inspecting a private helper’s storage to round-tripping an application-wide driver setting. The test remains eligible for parallel execution, and free-threaded CI uses --parallel-threads=4, running multiple copies concurrently.
The setter/getter sequence can therefore interleave:

  1. Worker A sets LIGHTWEIGHT=False.
  2. Worker B sets LIGHTWEIGHT=True.
  3. Worker A reads True and fails.
    The same configurations passed immediately after Ship _v2 lowpp API for the driver module #3095, consistent with an intermittent race. Executing the test against simulated, synchronized global storage also reproduces the assertion failure under this interleaving. A direct CUDA reproduction has not yet been performed.
    Suggested fix
    Mark the test pytest.mark.thread_unsafe, with a reason explaining that it modifies application-wide coredump settings. Consider restoring the original setting in finally to avoid leaving state behind.
    The test should pass consistently under free-threaded CI while retaining coverage of boolean attribute handling.

How to Reproduce

Might not be reproducible. You can re-run CI.

Expected behavior

Expected asserts to be equal.

Operating System

No response

nvidia-smi output

No response

Activity

  1. leofang commented on Oct 27, 2026

    @leofang
    Member

    Closed by #3226.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions