Add CUDA graph capture (CudaStream.BeginCapture/EndCapture, CudaGraph, CudaGraphExec)

[mfagerlund]

Summary

Adds a small, additive wrapper over the CUDA driver's stream-capture and graph API: record a fixed-shape sequence of launches once (CudaStream.BeginCapture/EndCapture → CudaGraph), instantiate it (CudaGraph.Instantiate → CudaGraphExec), and replay the whole sequence with a single cuGraphLaunch (CudaGraphExec.Launch).

The new types live in ILGPU.Runtime.Cuda and are CUDA-only by design — the same shape of contribution as CuBlas/CuFFT/CuRand. Nothing in the cross-backend abstraction changes; OpenCL/CPU/Velocity are untouched, because the methods are on the sealed CudaStream, not on AcceleratorStream.

Performance characterization (please read — this is not a universal speedup)

CUDA graphs amortize host-side launch dispatch across a repeated, fixed-shape sequence of launches.

• It helps when per-step host dispatch dominates — small models, small batches, many tiny kernels per step (the regime where the GPU sits mostly idle waiting on the host launch path).
• It is inert when the step is GPU-bound — there is no launch overhead left to remove, so expect roughly 1×. This is expected and correct, not a regression.

Measured on an RTX 4090 on this branch (kernels only; eager vs. one cuGraphLaunch per step):

regime	per-step shape	eager µs/step	graph µs/step	speedup
host-bound	n=1024, 48 kernels	325	50	6.4×
host-bound	n=4096, 48 kernels	312	48	6.5×
mixed	n=65536, 48 kernels	296	51	5.8×
gpu-bound	n=4.2M, 8 kernels	88	73	~1.2× (no win — expected)

The intended consumers are small-network training/inference loops (RL policies, control nets, small MLPs) where the C# launch path, not the GPU, is the bottleneck.

API

using var stream = (CudaStream)accelerator.CreateStream();   // the NULL/default stream is not capturable
stream.BeginCapture();
kernel(stream, n, view);            // any number of launches; a cuBLAS call on the stream is recordable too
using var graph = stream.EndCapture();
using var exec  = graph.Instantiate();
exec.Launch(stream);                // one driver call replays the whole sequence

Because an instantiated graph is a reusable handle and Launch is one call, graphs compose: capture prep, batch, and finalize separately and run prep; for (i < N) batch; finalize; with N a plain host loop bound — no re-capture when N changes.

Scope

• First cut is capture → instantiate → launch → destroy. cuGraphExecUpdate (re-binding parameters without re-instantiating) is intentionally deferred to a follow-up; manual node-graph construction is out of scope.
• cuGraphInstantiateWithFlags is CUDA 11.4+. If older drivers need supporting, that entry point can be selected by driver version — happy to adjust.

Implementation notes

• Driver entry points are added to Src/ILGPU/Runtime/Cuda/CudaAPI.xml and the P/Invoke layer is regenerated by the existing DllImports.tt; CudaError-wrapped helpers sit in CudaAPI.cs next to the stream helpers.
• CudaGraph/CudaGraphExec are AcceleratorObject subclasses using the same lifetime/BindScoped/CudaException.VerifyDisposed pattern as CudaProfilingMarker.

Tests

Src/ILGPU.Tests/CudaGraphCapture.cs (registered in Configurations.txt): SkippableFacts guarded on AcceleratorType != Cuda, so they exercise CUDA hardware and Skip elsewhere (CUDA runners are disabled in CI). They assert that capture records without executing, each Launch replays exactly once, and an N-launch capture replays the whole sequence per launch. 6/6 green on a 4090.

A note on timing

I can see you're mid-flight on 2.0 (new_architecture_v2); this targets master as a purely additive change to the current CUDA runtime. No rush at all — happy to align it with your design preferences, or to hold and fold it into the 2.0 runtime if you'd prefer that.

[mfagerlund]

Heads-up on the second commit: it bumps the repo-wide copyright year 2016-2025 → 2016-2026 in LICENSE.txt / README.md / the two nuspec.targets. That's exactly the output of Tools/CopyrightUpdateTool, which the check-style-copyright-headers job runs and fails on any diff. master already has 2026 commits, so this bump is currently needed by any fork PR; happy to split it out or let you handle it via the scheduled copyright job if you'd prefer.

[m4rs-mt]

Thank you for this contribution 👍! The implementation is minimal and idiomatic, and it follows the established CUDA runtime conventions closely (CudaAPI.xml -> generated externs pipeline, AcceleratorObject-derived handles, and the BindScoped/VerifyDisposed disposal pattern).

Target branch: I recommend retargeting this PR to branch/v1.6.x where changes can be applied there without modification. v1.6.x retains the traditional CudaStream layout that BeginCapture/EndCapture build on, so it avoids merge conflicts this would encounter against the in-flight v2.0 work. branch/v1.6.x also maintains per-file copyright headers.

Finally, a note on our roadmap. We have a v2.0 line in progress. Graph capture ports forward cleanly, as it resides in the Cuda runtime directory Src/ILGPU/Runtime/Cuda/ that v2.0 retains and operates below the compiler layer, so no second implementation is required — only a minor rebase and will fit in nicely. We would love to see your contribution being added to v2.0 also!