cuda.core: add LaunchConfig.programmatic_stream_serialization for programmatic dependent launch - #2456
cuda.core: add LaunchConfig.programmatic_stream_serialization for programmatic dependent launch#2456lijinf2 wants to merge 4 commits into
Conversation
…alization Allow users to set CU_LAUNCH_ATTRIBUTE_PROGRAMMATIC_STREAM_SERIALIZATION through LaunchConfig, matching the is_cooperative attribute pattern (NVIDIA#1334).
Add an end-to-end Hopper+ test that launches primary and secondary kernels on the same stream with programmatic_stream_serialization, and asserts overlap only when the PDL attribute is enabled (NVIDIA#1334).
Drop unused secondary sync/sleep from the overlap test, clarify the primary clock window comment, and print a short success line for CI.
github-actions
Bot
added
the
cuda.core
|
/ok to test d3b3b25 |
|
lijinf2
changed the title
| is_cooperative : bool, optional | ||
| Whether this config can be used to launch a cooperative kernel. | ||
| programmatic_stream_serialization : bool, optional | ||
| Whether to allow programmatic stream serialization (PDL). When True, |
There was a problem hiding this comment.
PDL is programmatic dependent launch, "programmatic stream serialization" is a sub-concept inside PDL?
| attr.value.cooperative = 1 | ||
| attrs.append(attr) | ||
|
|
||
| if config.programmatic_stream_serialization: |
There was a problem hiding this comment.
Non blocking Q: I see from the comment to this function "once all modules are cythonized, this function can be dropped in favor of the cdef method above". Are all modules cythonized?
| # PDL overlap is opportunistic; retry a few times on a quiet GPU. | ||
| saw_overlap = False | ||
| for _ in range(5): | ||
| if _run(secondary_cfg) == 1: | ||
| saw_overlap = True | ||
| break |
There was a problem hiding this comment.
Frankly, I have huge doubt over testing a non-deterministic behavior and asserting it in the test. We shouldn't be introducing flakiness in the test ourselves.
However kernel overlapping is opportunistic in nature. I vibe coded with my agent for a few rounds and I couldn't find a way to write a kernel and turn this into a deterministic test. My agent suggested that we should actually inspect the edge data from cuda graph to validate its dependency. But we don't have PDL in graph for now. (right?)
IMO, it's probably also acceptable to not expect saw_overlap here to always pass. Instead, we can use pytest.xfail when we didn't observe an overlap run. In that case, we validate the API via inspecting over a period of CI runs, not just through a single flaky failure.
2 participants
Description
Part of #1334
Expose
LaunchConfig.programmatic_stream_serializationto setCU_LAUNCH_ATTRIBUTE_PROGRAMMATIC_STREAM_SERIALIZATION, and add mapping plusHopper+ primary/secondary overlap tests.
Reference: https://docs.nvidia.com/cuda/cuda-programming-guide/04-special-topics/programmatic-dependent-launch.html
Release notes / Graph showcase will be in next PR.
Checklist