[dev] Share CUDA graph slots across dynamic CP variants - #6479
Draft
xiaoyao0115 wants to merge 11 commits into
Draft
[dev] Share CUDA graph slots across dynamic CP variants#6479xiaoyao0115 wants to merge 11 commits into
xiaoyao0115 wants to merge 11 commits into
Conversation
xiaoyao0115
force-pushed
the
dcp-cg-slot-pool
branch
from
August 12, 2026 09:12
51b9fad to
33082d6
Compare
xiaoyao0115
force-pushed
the
dcp-cg-slot-pool
branch
from
August 21, 2026 16:03
21f2169 to
3de21ef
Compare
Contributor
Author
|
/ok to test 2a1d969 |
This was referenced Aug 24, 2026
Contributor
Author
|
/ok to test a24ea4d |
Contributor
Author
|
/ok to test 7e6ba4c |
xiaoyao0115
force-pushed
the
dcp-cg-slot-pool
branch
from
August 25, 2026 08:00
7e6ba4c to
8bb6345
Compare
Contributor
Author
|
/ok to test 8bb6345 |
xiaoyao0115
force-pushed
the
dcp-cg-slot-pool
branch
from
August 25, 2026 09:54
8bb6345 to
e64ea93
Compare
Contributor
Author
|
/ok to test e64ea93 |
…IDIA#6646) Signed-off-by: Ajay Balasa <abalasa@nvidia.com> Signed-off-by: Charlie Truong <chtruong@nvidia.com>
Signed-off-by: Charlie Truong <chtruong@nvidia.com>
Signed-off-by: Charlie Truong <chtruong@nvidia.com>
Signed-off-by: Charlie Truong <chtruong@nvidia.com>
Signed-off-by: Ajay Balasa <abalasa@nvidia.com> Signed-off-by: Charlie Truong <chtruong@nvidia.com> Co-authored-by: Ajay <abalasa@nvidia.com>
Signed-off-by: Jingqi Yang <jingqiny@nvidia.com> Signed-off-by: jingqiny-99 <jingqiny@nvidia.com>
…VIDIA#3635) Signed-off-by: sophiayyya <yangjingyi0730@163.com> Signed-off-by: root <root@eos0294.eos.clusters.nvidia.com> Signed-off-by: Sophia Yang <sopyang@cw-dfw-cs-001-dc-02.cm.cluster> Signed-off-by: Deyu Fu <deyuf@nvidia.com> Signed-off-by: Sophia Yang <sopyang@cw-dfw-cs-001-vscode-01.cm.cluster> Co-authored-by: root <root@eos0294.eos.clusters.nvidia.com> Co-authored-by: Sophia Yang <sopyang@cw-dfw-cs-001-dc-02.cm.cluster> Co-authored-by: Deyu Fu <deyuf@nvidia.com> Co-authored-by: Sophia Yang <sopyang@cw-dfw-cs-001-vscode-01.cm.cluster>
Signed-off-by: Robin Zhang <robinz@nvidia.com>
Signed-off-by: Hongxiao Bai <hongxiaob@nvidia.com>
xiaoyao0115
force-pushed
the
dcp-cg-slot-pool
branch
3 times, most recently
from
August 27, 2026 14:05
374f7c5 to
76e7c93
Compare
added 2 commits
August 27, 2026 23:31
Signed-off-by: Tailai Ma <tailaim@ferries-work.nvidia.com>
Signed-off-by: Tailai Ma <tailaim@ferries-work.nvidia.com>
xiaoyao0115
force-pushed
the
dcp-cg-slot-pool
branch
from
August 28, 2026 01:44
76e7c93 to
853e321
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
torch.cuda.MemPooland explicit saved/output/user-gradient arenas across mutually exclusive CP variantsGraph executables remain separate because different CP sizes have different compute and communication graphs. Memory addresses are shared only when the schedule proves that their lifetimes cannot overlap, including cases where the total microbatch count is larger than the physical slot count.
Implementation
MCore builds the complete capture order for every CP variant and passes a compact seven-field slot plan to Transformer Engine. TE discovers tensor layouts during warmup, fixes boundary addresses in the slot arenas, and captures every variant against the shared pool.
Dependencies and merge order
This PR is intentionally stacked on
pull-request/6801; its diff contains only the CUDA Graph and slot-planning layer.Validation