Ship the CUDA delegate in the wheel

[shoumikhin]

The CUDA delegate runs a model on an NVIDIA GPU. It is built into the Python extension, so only
Python can use it. A C++ application has no way to link it, and nothing else can reuse it either.

There is a sharing problem too. A program may use more than one GPU backend at once, and they need
to agree on which CUDA stream (the queue the GPU runs work on) the caller chose. If each backend
carries its own copy of that state, work queued through one is invisible to the other.

Ship the CUDA delegate and a small stream helper as their own shared libraries, and name both as
CMake components. The stream helper is shared so a process has exactly one copy of the caller's
stream choice, which is what lets two backends agree on it.

find_package(executorch REQUIRED COMPONENTS backend_cuda)
target_link_libraries(my_app PRIVATE executorch::runtime
                                     executorch::backend_cuda)

A CUDA wheel does not bundle the CUDA runtime. It declares it as a dependency, the way the PyTorch
CUDA wheels do, so one copy is shared with torch rather than shipping a second one:

executorch/lib/libexecutorch_backend_cuda.so     the delegate
executorch/lib/libexecutorch_extension_cuda.so   the stream helper
executorch/backends/cuda/libaoti_cuda_shims.so   the GPU device code

Each library records a relative path to where pip installs the CUDA runtime, so it resolves without
the caller setting a library search path and without depending on a toolkit being installed.

The declared set is only what a shipped library actually loads, which is the CUDA runtime and
nothing else. cuRAND is used through its device-side header API, which compiles into the object
rather than linking a library, and the generated model library embeds its kernels rather than
compiling them at run time, so there is no runtime compiler to declare. Each declared package also
needs its own directory recorded, since that is where the loader looks. CUDA 13 collects every
component in one directory while CUDA 12 gives each its own, so the recorded set differs by train.

The stream helper's header no longer includes cuda_runtime.h, which the wheel does not publish. It
only ever uses a CUDA stream as an opaque handle, so it declares that handle itself, and a consumer
can compile against the wheel with no CUDA toolkit installed. A ROCm build keeps the runtime alias
header it already used, where the same handle is a hip type that only that header can name.

Built a CUDA wheel, installed it into a clean environment, and:

• ran a GPU model from Python on an NVIDIA GPU, with output identical to eager PyTorch on the
  same weights and inputs (largest absolute difference 0).
• built a C++ application against the installed wheel alone and ran the same model, matching the
  same reference.
• confirmed one library defines the stream state and the GPU shims, not several. Extracting them
  into every consumer put three copies in one wheel, and a stream selected through one was invisible
  to the others.
• confirmed no shipped library records a CUDA toolkit path from the build machine, and every library
  that links the CUDA runtime has a relative path to it.
• confirmed a CPU wheel ships none of the CUDA libraries and no CUDA-only header.
• a row that names a CUDA train is built with the CUDA option on rather than left to autodetection,
  so a builder without a matching toolkit fails while configuring. Before this, such a row produced
  a wheel tagged for CUDA, carrying no CUDA library, that still declared the CUDA runtime packages:
  it installed cleanly and then reported the backend as unregistered when a model ran.
• a row whose major CUDA version does not match the installed toolkit now fails the build. The
  declared packages and the loader paths come from the row while the binaries come from the
  toolkit, and nothing compared the two, so a cu126 row built against a 13.0 toolkit attached
  CUDA 12 metadata to binaries needing libcudart.so.13. An unrecognised train fails too, instead
  of silently reporting whatever the builder happened to have. Detection reads the toolkit major
  directly, so the guard fires on any mismatch rather than only on the three exact (major, minor)
  pairs the supported list carries; on those three pairs it behaved correctly before, and on every
  other minor it saw an empty detection and skipped the check.
• the row classifier and the packaging read the row the same way now, so both agree on what a row
  spelled with an unsupported minor means. The shell classifier reduces the row to digits and
  matches against SUPPORTED_CUDA_VERSIONS. Packaging did the same shape on the outer decision
  and then took only the first two digits when picking runtime packages, so cu125 classified as
  CPU on one side and declared CUDA 12 on the other. Packaging now matches on the same digits and
  raises loudly on an unsupported train instead.
• whether a row is a CUDA row is decided by asking if it names a supported train, rather than by
  listing the spellings that mean "no CUDA". Checked 16 row values including cpu-aarch64,
  rocm6.2 and cu118; the previous list-based form was wrong on several, and each wrong answer
  made a non-CUDA wheel declare the CUDA runtime.
• the CUDA components are required when the wheel's own version says it is a CUDA wheel. They were
  optional unconditionally, so a wheel tagged +cu126 with no CUDA library at all passed every
  check.
• the stream helper ships under either name it can be built with. The shim layer records it as a
  dependency whenever CUDA is on, while packaging named only the shared-build spelling, so a
  non-shared build shipped a shim whose dependency resolved to nothing.
• the relative hops between shipped libraries are sized by how deep the library sits in the package.
  A fixed pair was correct at one depth only: measured over every shipped location, 6 of 12 hops
  landed on a directory that does not exist, and the hop from lib/ climbed out of the package
  entirely, where an unrelated library with a matching soname could satisfy the dependency first.
• the stream helper no longer links or includes the CUDA toolkit. It uses a stream only as an
  opaque handle and calls no CUDA function, so it needs no toolkit include and no libcudart link.
  A ROCm build still links hip, because the alias header it uses there does include hip's own.
• the GPU shims no longer link cuRAND. It is reached through its device-side header API, which
  compiles into the object rather than resolving against a library, so the link only recorded a
  dependency nothing calls into: the shipped library names libcurand.so.10 as needed and leaves
  not one cuRAND symbol undefined. Dropping it removes a runtime package the wheel would otherwise
  have to place for a library that never opens it.
• the hops are added to a library that recorded no runtime search path of its own. The linker
  records one only when something resolved from outside a default directory, which is a property of
  the builder image rather than of the wheel, so keying the hops off it would drop the route to the
  CUDA wheels this package declares as dependencies on an image that keeps the runtime in /usr/lib64.
  Measured on a library built with nothing outside a default directory: it now carries the CUDA and
  sibling hops, and a non-CUDA build of the same library is rewritten only where a hop applies.

Also fixed in this commit:

• The pre-build classifier resolves the Python interpreter (python3 or python, whichever
  exists) instead of assuming one name, and no longer discards stderr. Builders disagree on the
  name: Linux and macOS provide python3, while the Windows builder runs inside a conda
  environment that provides only python. Assuming either name breaks the other platform, and
  treating the failure as "not a CUDA row" silently rebuilt a CUDA row as a CPU row.
• CU_VERSION=cpu pip install . is handled explicitly instead of running the CUDA-train parser
  over it, which previously turned cpu into pu through a character-set strip and reached the
  unsupported-train error.

Ran end to end on H100, A100 and Jetson Thor, covering compute capabilities 9.0, 8.0 and 11.0.

Known gap, not introduced here: the Python Runtime.load_program path allocates activation memory
on the host, so a program exported to keep activations on the GPU fails there. The supported Python
loader and the C++ path both work. This is upstream in the Python bindings, which this change does
not touch.

cc @digantdesai @freddan80 @per @zingo @oscarandersson8218 @mansnils @Sebastian-Larsson @robell @rascani

[shoumikhin]

Stack from ghstack (oldest at bottom):

• Name the Python extension after what it contains #21816
• Ship linkable libraries in the macOS wheel #21771
• Ship the OpenVINO delegate as its own library #21770
• Build and publish CUDA wheels #21668
• -> Ship the CUDA delegate in the wheel #21645
• Ship the quantized kernels as their own library #21642
• Ship a C++ SDK in the wheel #21639
• Split the runtime out of the Python bindings extension #21610

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21645

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 Cancelled Jobs, 3 Pending, 2 Unrelated Failures, 1 Unclassified Failure

As of commit 3a2908b with merge base 46f9696 ():

UNCLASSIFIED FAILURE - DrCI could not classify the following job because the workflow did not run on the merge base. The failure may be pre-existing on trunk or introduced by this PR:

• trunk / test-models-macos-coreml (efficient_sam) / macos-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  ##[error]The operation was canceled.

CANCELLED JOBS - The following jobs were cancelled. Please retry:

• Test ARM Backend / test-arm / package-golden-artifacts (gh)
• Test ARM Backend / test-arm / test-backend-linux (arm_vgf_fp, models) / linux-job (gh)

FLAKY - The following job failed but was likely due to flakiness present on trunk:

• pull / unittest-editable / macos / macos-job (gh) (detected as infra flaky with no log or failing log classifier)

BROKEN TRUNK - The following job failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Apple / build-demo-ios / macos-job (gh) (trunk failure)
  RuntimeError: Command bash /Users/runner/work/_temp/exec_script failed with exit code 65

This comment was automatically generated by Dr. CI and updates every 15 minutes.