NanoVDB: add cuda::Buffer, cuda::BufferView, and the synchronous resource concept (CUDA) by harrism · Pull Request #2251 · AcademySoftwareFoundation/openvdb

harrism · 2026-07-16T03:13:41Z

Step 2 (part A) of #2232 (NanoVDB injectable CUDA memory resources). New types and tests only — no existing call site changes; the scratch retrofit that consumes these lands separately.

What

cuda::Buffer<T, R> — typed, resource-aware, stream-ordered container modeled on CCCL's cuda::buffer<T>. Constructor order and shape follow cuda::buffer exactly — (stream, resource, count, noInit), with defaulted-resource conveniences and stream-less synchronous forms. size() counts elements, size_bytes() bytes. Move-only with an explicit copy(). The retained-stream rule throughout: frees are ordered on the stream of the last allocation (or setStream), and resize orders everything — including the free of the old block, whose last use is the prefix copy — on the passed stream.
cuda::BufferView<T> — typed non-owning view with span semantics (element constness in T, shallow const, trivially copyable, no resource, no stream) satisfying the same static interface, so GridHandle<BufferView<std::byte>> wraps externally owned memory with zero copies (tested end-to-end against a createLevelSetSphere grid — pointer-equal, no double free).
is_resource<R> (synchronous tier) beside is_async_resource<R>, which now enforces CCCL's refinement: an async resource provides the stream-ordered and the synchronous interface, so is_async_resource<R> implies is_resource<R>. DeviceResource gains the delegating sync pair; PinnedResource remains synchronous-only.
DeviceResource's legacy static allocateAsync/deallocateAsync are [[deprecated]] (the implementation moved into the instance methods, so nothing internal trips the warning). Migration note for v13-era custom resources is on NanoVDB: injectable CUDA memory resources — design & roadmap #2232.

Design decisions (rationale on #2232)

No implicitly initializing count constructor — the noInit tag is mandatory, matching cuda::buffer (whose count constructor requires cuda::no_init) and the RMM device_uvector lesson: implicit fill of freshly allocated memory is a hidden cost the dominant allocate-then-overwrite pattern wastes. Enforced by is_constructible static_asserts.
A synchronous-R Buffer has no stream API at all (member, constructor parameter, stream(), setStream — all absent, verified by detection-idiom static_asserts): a synchronous allocation is immediately valid on every stream, so no parameter exists to ignore.
CamelCase type names, standard member names: type names are aliasable (using Buffer = ::cuda::buffer<T> remains available at CUDA ≥ 13.2), member names are structural and keep CCCL's spelling.
CUDA graph capture: the async path is capture-safe — allocation/use/free inside a stream capture records, instantiates, and relaunches (tested). The sync tier synchronizes and is excluded from capture by construction.
A failed resize (overflow or allocation error) leaves the buffer fully untouched, including its retained stream (tested).

Testing

New unittest/TestBuffer.cu (TestBuffer suite, 20 tests; ctest nanovdb_cuda_buffer_unit_test): trait satisfaction for both tiers and the refinement; exact allocation sizes through a counting resource; stream retention, setStream redirection, and resize stream ordering via a stream-recording resource; move/copy semantics without double-free; byte-size overflow throws; pinned page-lock; graph capture; BufferView semantics incl. the zero-copy GridHandle round-trip. Full suites green: CPU 153/153, nanovdb_test_cuda 53/53, memory-resource 8/8; compute-sanitizer memcheck clean (sm_89, CUDA 12.6).

Non-CUDA builds

No impact on NANOVDB_USE_CUDA=OFF: new headers live under nanovdb/cuda/, and the test target is registered inside if(NANOVDB_USE_CUDA).

🤖 Generated with Claude Code

…urce trait (CUDA) Adds nanovdb/cuda/Buffer.h with two typed containers: - cuda::Buffer<T, R>: owning, move-only container of T elements allocated from a memory resource R held by value. With a stream-ordered resource (is_async_resource) it retains the stream of the most recent allocation and orders deallocation, resize, and copy on it; with a synchronous resource (is_resource) it carries no stream state and exposes no stream API, enforced via an empty-base StreamHolder and enable_if-gated overloads. Constructors value-initialize elements to zero unless the NoInit tag is passed. Element counts whose byte size would overflow size_t are rejected with an exception. - cuda::BufferView<T>: non-owning, trivially copyable view with span semantics plus a detaching clear(), which is what the GridHandle buffer static interface requires, enabling zero-copy GridHandle<cuda::BufferView<std::byte>> over externally owned grids. Also adds the is_resource detection trait (synchronous allocate/deallocate) to nanovdb/cuda/DeviceResource.h alongside is_async_resource, and a new unit-test suite (TestBuffer.cu, ctest name nanovdb_cuda_buffer_unit_test) covering traits, counting and stream-recording resources, stream retention, resize, move semantics, deep copy, pinned-host buffers, and a zero-copy GridHandle round trip. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Mark Harris <mharris@nvidia.com>

kmuseth

looks good - only non-blocking changes

how much VRAM does the unit-test require? I've recently notice that tests would fail my my Nvidia laptop with only 8GB.

kmuseth · 2026-07-28T00:25:33Z

+    /// @param count number of elements
+    template<typename S = R, std::enable_if_t<is_async_resource<S>::value, int> = 0>
+    explicit Buffer(cudaStream_t stream, R resource, size_t count, NoInit)
+        : detail::StreamHolder<true>{stream}, mResource(std::move(resource))


very minor but I prefer:

explicit Buffer(cudaStream_t stream, R resource, size_t count, NoInit) : detail::StreamHolder<true>{stream} , mResource(std::move(resource)) {

I see you use the same layout below

Done in 0583b27 — split onto one member per line, matching the move c-tor below.

FWIW, this should be dictated by automatic formatting (clang-format) so people never need to think about it.

kmuseth · 2026-07-28T00:27:27Z

+    /// @brief Convenience c-tor using a default-constructed resource.
+    template<typename S = R, std::enable_if_t<!is_async_resource<S>::value && is_resource<S>::value, int> = 0>
+    Buffer(size_t count, NoInit) : Buffer(R(), count, noInit) {}
+


add:

/// Explicitly disallow copy construction and assignment operation

Added in 0583b27, with your wording.

kmuseth · 2026-07-28T00:29:50Z

+    /// @brief Returns the retained stream, i.e. the stream the buffer's memory
+    ///        will be freed on.
+    template<typename S = R, std::enable_if_t<is_async_resource<S>::value, int> = 0>
+    cudaStream_t stream() const { return this->mStream; }


is there a reason this is not called getStream()? I see setStream(..) below so it seems a little strange that the get method uses a different semantic.

Deliberate, though I agree the asymmetry looks odd in isolation.

stream() follows the precedent already in the tree: cuda::TempPool::stream() (nanovdb/cuda/TempPool.h:57), which landed in #2231. It also matches cuda::buffer's accessor, which this class is shaped after — same reasoning we used in #2231 for keeping allocate/allocate_async in snake_case rather than renaming them to NanoVDB style: where a name is part of a contract we're deliberately mirroring, matching it exactly is worth more than local consistency.

setStream() isn't a plain setter — it re-targets the retained stream without synchronizing, and the caller takes on the ordering obligation — so I wanted the verb there to make the mutation obvious at the call site.

That said, this is your call and it's cheap either way. Say the word and I'll rename to getStream()/setStream(); if I do, I'd want to rename TempPool::stream() in the same commit so we don't end up with both spellings.

I accept you argument for keeping it as is

kmuseth

The async resize wraps its cudaMemcpyAsync in a try/catch to deallocate newData on failure, leaving the buffer untouched. The synchronous overload (Buffer.h:228-237) has no such guard:

kmuseth · 2026-07-28T01:21:12Z

+    ///       initialization of freshly allocated memory costs a hidden fill
+    ///       pass that the dominant allocate-then-overwrite pattern wastes,
+    ///       so initialization is always explicit (matching cuda::buffer,
+    ///       whose count c-tor likewise requires cuda::no_init).


please use "@param count" etc

Done in 0583b27.

kmuseth · 2026-07-28T01:22:42Z

+    }
+
+    /// @brief Resizes the buffer to @c count elements through the synchronous
+    ///        resource, preserving the leading min(old, new) elements.


please use @param count

Done in 0583b27.

kmuseth · 2026-07-28T01:23:11Z

+
+    /// @brief Allocates @c count elements through the resource and records the
+    ///        new extent; @c stream is used by the stream-ordered form and
+    ///        ignored by the synchronous one. A zero count allocates nothing.


instead of documenting all the arguments in @brief please use "@param count" etc

Done in 0583b27. I applied the same treatment everywhere the pattern appeared rather than just at the three you flagged, so it's consistent across the header: copy(stream), setStream, the async resize, checkedBytes, deallocate, and the BufferView c-tor all document their arguments with @param now instead of describing them inside @brief.

@brief

The stream-ordered resize guarded its prefix copy so a failure frees the new allocation and leaves the buffer untouched; the synchronous overload had no such guard. Mirror it so both overloads share one invariant. Also document parameters with @param rather than describing them in @brief, split the stream-ordered c-tor's initializer list onto one member per line, and note that copy construction and assignment are explicitly disallowed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Mark Harris <mharris@nvidia.com>

harrism · 2026-07-31T06:52:27Z

how much VRAM does the unit-test require? I've recently noticed that tests would fail on my Nvidia laptop with only 8GB.

Measured it: ~490 MiB peak for the whole process, and essentially none of that is the test's own data.

TestBuffer allocates almost nothing. The element counts across the suite are 64, 96, 100, 128, 256, 300 and 1000, of int/float — the largest single buffer in the file is 1000 ints, i.e. 4 KB. The two oversized cases throw in checkedBytes before any allocation is attempted, so they never reserve anything either. The ~490 MiB is CUDA context and runtime overhead that any CUDA process on this machine pays before it allocates a byte.

Method: built TestBuffer.cu against CUDA 12.6 on an RTX 6000 Ada, ran it under --gtest_repeat=40, and sampled nvidia-smi --query-compute-apps=used_memory for the test PID throughout. Context overhead does vary with driver, architecture and how many modules get loaded, so the absolute number will differ on a laptop part — but the test's own footprint is ~4 KB no matter where it runs.

So this suite shouldn't be what's failing on your 8 GB laptop. If you can tell me which test binary and which case, I'm happy to dig — TestNanoVDB.cu builds real grids and is a far more plausible candidate. Worth noting that nothing would catch a VRAM regression for us anyway, since NanoVDB CI never runs the CUDA tests on a GPU.

Also: all 20 tests pass on the updated branch (0583b27), and I've pushed fixes for the other seven comments — replies are on the individual threads.

kmuseth

thanks for the changes

harrism requested a review from kmuseth as a code owner July 16, 2026 03:13

swahtz added the nanovdb label Jul 16, 2026

harrism mentioned this pull request Jul 22, 2026

NanoVDB: injectable CUDA memory resources — design & roadmap #2232

Open

5 tasks

kmuseth reviewed Jul 28, 2026

View reviewed changes

kmuseth requested changes Jul 28, 2026

View reviewed changes

This was referenced Jul 31, 2026

NanoVDB: please run the CUDA unit tests on a GPU in CI #2264

Open

NanoVDB: cudaCheck should throw instead of calling exit(), and teardown paths need a non-throwing check #2265

Open

kmuseth approved these changes Jul 31, 2026

View reviewed changes

kmuseth merged commit 7be8164 into AcademySoftwareFoundation:master Jul 31, 2026
15 checks passed

Uh oh!

Conversation

harrism commented Jul 16, 2026

What

Design decisions (rationale on #2232)

Testing

Non-CUDA builds

Uh oh!

kmuseth left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kmuseth left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

harrism commented Jul 31, 2026

Uh oh!

kmuseth left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants