Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 6 additions & 4 deletions cuda_bindings_13/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,13 +44,15 @@ subpackage in the `cuda-python` monorepo.
- **Cython tests**:
- build: `tests/cython/build_tests.sh` (or platform equivalent)
- run: `pytest tests/cython/`
- **Samples**: sample coverage is pytest-based under `../samples/cuda_bindings/`.
- **Benchmarks**: run with `pytest --benchmark-only benchmarks/` when needed.
- **Samples**: run pytest-based sample coverage in `tests/example_tests/`;
sample sources and instructions live in `../samples/cuda_bindings/`.
- **Benchmarks**: see the [benchmark README](../benchmarks/cuda_bindings/README.md)
for the Pixi tasks that run the Python and C++ benchmarks.

The `legacy_tests` subdirectory tests the old pre-v2 APIs of `driver`, `runtime`
The `tests/legacy_api` subdirectory tests the old pre-v2 APIs of `driver`, `runtime`
and `nvrtc`. These test files should not be added to, only updated when
necessary to fix test failures. The canonical set of tests are those outside of
the `legacy_tests` subdirectory.
the `tests/legacy_api` subdirectory.

## Build and environment notes

Expand Down
20 changes: 9 additions & 11 deletions cuda_bindings_13/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,31 +42,29 @@ To run these tests:
Cython tests are located in `tests/cython` and need to be built. These builds have the same CUDA Toolkit header requirements as [Installing from Source](https://nvidia.github.io/cuda-python/cuda-bindings/latest/install.html#requirements) where the major.minor version must match `cuda.bindings`. To build them:

1. Setup environment variable `CUDA_PATH` (or `CUDA_HOME`) with the path to the CUDA Toolkit installation. Note: If both are set, `CUDA_PATH` takes precedence.
2. Run `build_tests` script located in `test/cython` appropriate to your platform. This will both cythonize the tests and build them.
2. Run the `build_tests` script located in `tests/cython` appropriate to your platform. This will both cythonize the tests and build them.

To run these tests:
* `python -m pytest tests/cython/` against editable installations
* `pytest tests/cython/` against installed packages

### Samples

Various [CUDA Samples](https://github.com/NVIDIA/cuda-samples/tree/master) that were rewritten using CUDA Python are located in `examples`.
Various [CUDA Samples](https://github.com/NVIDIA/cuda-samples/tree/master) that were rewritten using CUDA Python are located in [samples/cuda_bindings](../samples/cuda_bindings/). Each sample directory has its own README with dependencies and instructions for running it.

In addition, extra examples are included:

* `examples/extra/jit_program_test.py`: Demonstrates the use of the API to compile and
* [jitProgram.py](../samples/cuda_bindings/extra/jitProgram/jitProgram.py): Demonstrates the use of the API to compile and
launch a kernel on the device. Includes device memory allocation /
deallocation, transfers between host and device, creation and usage of
streams, and context management.

To run these samples:
* `python -m pytest tests/cython/` against editable installations
* `pytest tests/cython/` against installed packages
To run the automated sample coverage in `tests/example_tests/`, run
`pixi run test-samples` from this directory.

### Benchmark

Allows for analyzing binding performance using plugin [pytest-benchmark](https://github.com/ionelmc/pytest-benchmark).

To run these benchmarks:
* `python -m pytest --benchmark-only benchmarks/` against editable installations
* `pytest --benchmark-only benchmarks/` against installed packages
Python and C++ benchmarks for analyzing binding overhead live in
[benchmarks/cuda_bindings](../benchmarks/cuda_bindings/). See the
[benchmark README](../benchmarks/cuda_bindings/README.md) for the Pixi environments,
benchmark commands, and result comparison instructions.
32 changes: 19 additions & 13 deletions cuda_core/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -156,14 +156,17 @@ and agents should flag violations.
objects that are not meant to be shared (e.g., the thread-local `Device`) do not
need such guards (see #2321). Reference-count integrity is guaranteed; cache
value-identity/idempotency is not.
- **Entry points work with or without the GIL**: Cython calls the helpers in
`_cpp/rt/` both inside and outside `with nogil` blocks. They never require
the GIL, release it around driver calls, and never acquire it while they
hold a C++ lock. The only paths that acquire it are the reporting wrappers
(`pw_*`, `report_*`) and the one-time driver function-table fill
(`ensure_fn_table()`, see `_cpp/rt/DESIGN.md`). Driver and destructor
callbacks run at arbitrary times, so they take the GIL (`with gil`) and probe
for interpreter shutdown before touching Python objects.
- **GIL handling depends on the path**: Cython calls the helpers in `_cpp/rt/`
both inside and outside `with nogil` blocks. Follow each helper's GIL
contract, release the GIL around driver calls, and never acquire it while
holding a C++ lock. Reporting wrappers (`pw_*`, `report_*`), the one-time
function-table fill (`ensure_fn_table()`), and Python-owner cleanup may
acquire it internally. Owner-release deleters use `GILAcquireGuard` to probe
for interpreter shutdown before touching Python objects. Python host
callbacks use `_py_host_trampoline` (`with gil`); CUDA user-object destructor
callbacks only enqueue deferred cleanup, without acquiring the GIL,
touching Python objects, or calling the driver (see `_cpp/rt/DESIGN.md` and
`_cpp/rt/py_deferred_cleanup.cpp`).
- **Driver calls go through the table**: C++ calls the driver with
`DRIVER_CALL(name, args...)`. Its pointers come from the table that
cuda-bindings resolves, never from the Cython wrappers. If the installed
Expand Down Expand Up @@ -212,10 +215,10 @@ below are for contributors. Reviewers and agents should flag violations.
`HANDLE_RETURN`; an `except` block whose rollback failed uses
`attach_rollback_failure()`; a deleter or cleanup path uses a `pw_*`
wrapper or `report_cuda_error()`; the same situation in Cython or Python
uses `warnings.warn(..., CUDAWarning)`; a CUDA callback thread does nothing
that needs the GIL and hands its work to the deferred-cleanup queue
(`Py_AddPendingCall` is GIL-free and allowed there). The table in
`_cpp/rt/DESIGN.md` ("Which channel to use") spells this out.
uses `warnings.warn(..., CUDAWarning)`; a CUDA user-object destructor
callback does nothing that needs the GIL and hands its work to the
deferred-cleanup queue (`Py_AddPendingCall` is GIL-free and allowed there).
The table in `_cpp/rt/DESIGN.md` ("Which channel to use") spells this out.
- **`pw_*` runs user Python**: a `p_` pointer only calls the driver; its `pw_`
twin also acquires the GIL on failure and runs the warning filters,
`showwarning`, or `sys.unraisablehook`, any of which may call back into
Expand Down Expand Up @@ -320,7 +323,10 @@ Python or Cython type annotations should be included for all public APIs. Avoid
the use of `Any` unless absolutely necessary. The argument and return types as
defined in the docstrings should match the type annotations.

Python imports should generally be outside of an if typing.TYPE_CHECK: block, even if the imported object is only used in type annotations. Use if typing.TYPE_CHECK: only to avoid creating import cycles. (This guidance maximizes compatibility with the cross-reference mechanisms in Sphinx.)
Python imports should generally be outside of an `if typing.TYPE_CHECKING:`
block, even if the imported object is only used in type annotations. Use
`if typing.TYPE_CHECKING:` only to avoid creating import cycles. (This guidance
maximizes compatibility with the cross-reference mechanisms in Sphinx.)

### Semantics

Expand Down
8 changes: 4 additions & 4 deletions cuda_core/cuda/core/_cpp/rt/DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -430,7 +430,7 @@ exactly one of these; none is ever dropped.
| Cython, inside an `except` block whose rollback failed | `attach_rollback_failure(op, status, detail)` | Adds a note to the exception being handled. Reports instead if nothing is being handled. |
| C++, a deleter or deferred cleanup | A `pw_*` wrapper, or `report_cuda_error()` / `report_message()` | Emits `CUDAWarning`. Never raises. |
| Cython or Python, a `__dealloc__` or destructor-path callback | `warnings.warn(msg, CUDAWarning, stacklevel=2)` | Same. |
| A CUDA callback thread | Nothing that needs the GIL. Hand the work to the deferred-cleanup queue with `Py_AddPendingCall` | CUDA forbids driver calls there, and acquiring the GIL there can deadlock with a GIL holder blocked in a driver call. GIL-free C API that only schedules work is fine. |
| A CUDA user-object destructor callback | Nothing that needs the GIL. Hand the work to the deferred-cleanup queue with `Py_AddPendingCall` | CUDA forbids driver calls there, and acquiring the GIL there can deadlock with a GIL holder blocked in a driver call. GIL-free C API that only schedules work is fine. |

### `p_` versus `pw_`

Expand All @@ -455,9 +455,9 @@ allocate or throw, since a deleter is `noexcept`.
So: use `pw_` only in deleters and cleanup paths that hold no C++ lock and have
finished updating the layer's own state. Where a lock must stay held, call
`p_`, keep the status, and report after the lock is released, as
`deviceptr_import_ipc` does. CUDA callback threads need no extra rule for
`pw_`: the driver call is forbidden there, so the wrapper is too. The general
rule for those threads is no GIL and no Python objects; GIL-free scheduling
`deviceptr_import_ipc` does. CUDA user-object destructor callbacks need no extra
rule for `pw_`: the driver call is forbidden there, so the wrapper is too. Those
callbacks must not acquire the GIL or touch Python objects; GIL-free scheduling
calls such as `Py_AddPendingCall` are how work leaves them.

## Usage from Cython
Expand Down
Loading