Add NVIDIA / CUDA driver support for images built without it

[NewComer00]

Description

Currently users can start up a sandbox like nvidia/cuda:12.9.1-devel-ubuntu24.04 to access the host's NVIDIA GPU drivers and run CUDA applications with Enroot, as stated in the Usage documentation.

But sometimes there is a need to run an image built without NVIDIA / CUDA support, for example debian:bookworm-slim. It would be great if Enroot could provide access to the host's NVIDIA GPU drivers in this case as well, similar to how Docker and Podman do it.

Environment

• Host OS: Rocky Linux 8.10
• Enroot version: 3.5.0
• NVIDIA driver version: 570.153.02

Expected Behavior

The nvidia-smi command works inside an Enroot sandbox started from an image without NVIDIA / CUDA support, as long as the host has the NVIDIA drivers installed and running, like in Docker or Podman:

podman run --rm --device nvidia.com/gpu=all debian:bookworm-slim nvidia-smi

Output example:

Wed Dec 17 07:18:11 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 570.153.02             Driver Version: 570.153.02     CUDA Version: 12.8     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 4090 D      Off |   00000000:41:00.0 Off |                  Off |
| 31%   26C    P8             15W /  425W |       1MiB /  24564MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

Current Behavior

Currently, if I start an Enroot sandbox from an image without NVIDIA / CUDA support, no NVIDIA libraries or binaries are available inside the container, so running nvidia-smi fails:

enroot start --root --rw debian+bookworm-slim nvidia-smi

Output example:

/etc/rc: line 9: exec: nvidia-smi: not found

Workaround

My workaround is to manually mount the host's nvidia-smi binary and the relevant libnvidia-ml.so.1 libcuda.so.1 libraries into the container.

The NVIDIA driver from my host was installed via a .run file. The relevant files are located in /usr/lib64/ and /usr/bin/. Here is the command:

enroot start \
  --root \
  --rw \
  --env LD_LIBRARY_PATH=/usr/lib64 \
  --mount /usr/lib64/libcuda.so.1:/usr/lib64/libcuda.so.1 \
  --mount /usr/lib64/libnvidia-ml.so.1:/usr/lib64/libnvidia-ml.so.1 \
  --mount /usr/bin/nvidia-smi:/usr/bin/nvidia-smi \
  debian+bookworm-slim \
  nvidia-smi

Output example:

Wed Dec 17 15:40:17 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 570.153.02             Driver Version: 570.153.02     CUDA Version: 12.8     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 4090 D      Off |   00000000:41:00.0 Off |                  Off |
| 30%   26C    P8             15W /  425W |       1MiB /  24564MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

Applications like PyTorch(with CUDA runtime provided through PyPI packages) can now access the GPU as expected.

Additional Information

Each time I start an Enroot sandbox, there is an error message popping up:

/etc/enroot/hooks.d/10-shadow.sh: line 70: 312531 Broken pipe             yes 2> /dev/null
     312532 Segmentation fault      (core dumped) | pwck -R "${ENROOT_ROOTFS}" "${pwddb#${ENROOT_ROOTFS}}" /etc/shadow > /dev/null 2>&1

This seems unrelated to the NVIDIA driver, but I thought I should mention it in case it is relevant.

[elezar]

As far as I understand, Enroot uses the NVIDIA_VISIBLE_DEVICES envvar to indicate which devices should be included in a container. The nvidia/cuda base image have this environment variable set to NVIDIA_VISIBLE_DEVICES=all. Does starting the container with --end NVIDIA_VISIBLE_DEVICES=all acheive what you want:

enroot start --root --rw \
    --env NVIDIA_VISIBLE_DEVICES=all \
    debian+bookworm-slim nvidia-smi -L