diff --git a/docs/index.md b/docs/index.md index 2965aa7..deea0c7 100644 --- a/docs/index.md +++ b/docs/index.md @@ -12,3 +12,4 @@ | `cthreads documentation` | [link](./COMPILER.md) | | **Release**: GitHub Actions, TestPyPI, PyPI trusted publishing | [link](./release.md) | | End-to-end example (`@Thread` / `@Threadable` through codegen) | [link](./Example.md) | +| **Vulkan / GPU backend guide** (cthreads compute path, Issues 1+) | [link](./vk_guide/README.md) | diff --git a/docs/vk_guide/00-read-me-first.md b/docs/vk_guide/00-read-me-first.md new file mode 100644 index 0000000..1320c22 --- /dev/null +++ b/docs/vk_guide/00-read-me-first.md @@ -0,0 +1,93 @@ +# 00 — Read me first + +## What cthreads is building + +cthreads already turns typed Python (`@Thread`) into **CPU** native kernels: + +```text +Python args -> pack (C++ struct) -> run on OS thread -> writeback -> same Python objects +``` + +The GPU path follows the **same product idea** with a different middle: + +```text +Python args -> GpuPack (Vulkan buffers) -> run SPIR-V compute -> writeback -> same Python objects +``` + +Users still pass `list` / `int` / `float`. They do **not** learn a second buffer type. +Vulkan stays inside `_ext`. + +## What Vulkan is (intuition) + +Vulkan is a **low-level remote control for the GPU**. + +- The API does not hide a high-level "draw a triangle" call. +- Callers create memory, bind it, record commands, submit them to a queue, and wait until done. +- The driver does less magic. The application owns lifetimes, synchronization, and layouts. + +That is why Vulkan feels verbose. Every step that OpenGL/CUDA hid becomes a function call. +The upside: portable compute on NVIDIA / AMD / Intel with one API, and predictable behavior. + +## What this project deliberately ignores + +These topics are **out of scope** for the cthreads compute backend: + +| Topic | Why skip | +|-------|----------| +| Swapchain / windows / present | Not drawing to the screen | +| Render passes / framebuffers | Graphics only | +| Images / textures / samplers | Buffer-first compute path | +| Graphics pipelines / vertex input | Compute only | +| Multi-GPU / sparse memory | Out of scope for now | +| Buffer device address / bindless | Rejected for the current design | +| Ray tracing | Out of scope | + +If a random tutorial spends chapters on "hello triangle," skim or skip those sections. +The cthreads "hello world" is: **upload floats, run compute (when landed), download floats**. + +## How Vulkan compares to things contributors might know + +### vs writing normal C++ + +| C++ | Vulkan | +|-----|--------| +| `new[]` / `malloc` | `vkAllocateMemory` + choose memory type | +| `memcpy` to a buffer the CPU owns | `memcpy` only works if memory is **host-visible** | +| Call a function | Record commands, then **submit** to a queue (async GPU) | +| Function returns when done | GPU may still be running; **fence** or similar to wait | + +### vs CUDA (if familiar) + +| CUDA | cthreads Vulkan path | +|------|----------------------| +| `cudaMalloc` | device-local `VkBuffer` + memory | +| `cudaMemcpy` | staging buffer + `vkCmdCopyBuffer` | +| `<<>>` kernel launch | `vkCmdDispatch` in a command buffer | +| CUDA toolkit for end users | **Drivers only** for end users; SDK headers for *building* GPU-enabled `_ext` | + +### vs OpenGL compute + +OpenGL hides a lot of binding and sync. Vulkan makes bind points and barriers explicit. +Same idea (SSBO = storage buffer), more paperwork. + +## The one sentence to remember + +**Vulkan objects are handles to driver resources; almost nothing happens until a command buffer is submitted to a queue; CPU and GPU run out of sync unless the CPU waits.** + +## Locked decisions in this project (so generic tutorials do not confuse contributors) + +1. **Option 5 pack:** one scalar SSBO + one SSBO per `list`. +2. **Device-local** data for shaders; **staging** for CPU copies. +3. **Descriptors** bind buffers by binding index (not pointers in the scalar struct). +4. **Launch then join** — no mid-run Python `__sync_state` on GPU. +5. **Dynamic load** `vulkan-1.dll` / `libvulkan.so.1` — do not hard-link for default CPU wheels. + +If a blog says "just map the SSBO and memcpy," that is a shortcut cthreads does **not** use as the production list path. + +## Suggested study rhythm + +1. Read 01-03 for intuition (no code required). +2. Read 04-05 while looking at `gpu/headers/context.hpp` and `gpu/impl/context.cpp`. +3. Read 06-08 while looking at `gpu/headers/memory.hpp` and `gpu/impl/memory.cpp`. +4. Read 09-12 before working on pipelines, descriptors, or `@Gpu` emit. +5. Keep 14 glossary open while coding. diff --git a/docs/vk_guide/01-cthreads-gpu-big-picture.md b/docs/vk_guide/01-cthreads-gpu-big-picture.md new file mode 100644 index 0000000..ef58c0e --- /dev/null +++ b/docs/vk_guide/01-cthreads-gpu-big-picture.md @@ -0,0 +1,115 @@ +# 01 — cthreads GPU big picture + +## The CPU path + +A typical typed kernel looks like: + +```python +@Thread +def add(n: int, x: list[float], y: list[float]) -> None: + i: int = 0 + while i < n: + y[i] = x[i] + y[i] + i = i + 1 + +job = thread(add, 4, [1,2,3,4], [10,10,10,10]).start() +job.join() +``` + +Roughly: + +1. **Marshal** copies Python values into a C++ **pack** (args struct). +2. A worker thread runs the compiled C++ body on that pack. +3. **Writeback** copies mutated pack fields into the **same** Python lists. + +Scalars live as fields in the pack. Lists live as native containers owned by / pointed from the pack. + +## The GPU path (same story, different storage) + +Same Python. Different backend: + +1. **GPU marshal** creates a **GpuPack**: + - one **device-local** buffer holding all scalars as a `std430` struct + - one **device-local** buffer per list (tight array of floats/ints) +2. Host bytes go into those buffers via **staging + GPU copy** (upload). +3. A **compute shader** reads/writes those buffers through **descriptor bindings**. +4. After the GPU finishes (`join`): **download** + writeback into the same Python objects. + +```text + CPU world GPU world + ----------------- -------------------- +Python list[float] --upload--> staging --copy--> device-local SSBO +Python int/float --upload--> staging --copy--> scalar SSBO (struct) + | + compute shader + | +Python list[float] <--writeback-- staging <--copy-- device-local SSBO +``` + +## Why not one giant buffer for everything? + +cthreads uses **option 5**: + +| Piece | Where it lives | +|-------|----------------| +| Scalars (`n`, `a`, …) | One small SSBO interpreted as a C-like struct | +| Each `list[...]` | Its own SSBO | + +Reasons: + +- Matches how compute shaders usually declare `buffer` blocks (one binding per array). +- Easy to reuse the same list buffer across multiple dispatches in a batch. +- Layout for arrays stays simple (tight packed floats). +- Scalar writeback is one download of a small blob. + +The design **rejects** putting GPU pointers inside the scalar struct (that needs buffer device address). Bindings connect the shader to buffers instead. + +## Job contract (important product rule) + +On CPU, `__sync_state` can mirror pack -> Python mid-run. + +On the GPU path: **only launch then wait**. + +- No observing Python lists while the shader runs. +- Results appear after `GpuJob.join()` (fence wait + download + writeback). + +That matches how GPUs want to work: record work, submit, wait once. + +## Layers of the GPU stack + +| Layer | Responsibility | +|-------|----------------| +| Context | Talk to Vulkan: instance, device, queue, entry points | +| Memory helpers | Allocate buffers, staging upload/download | +| GpuPack | Option 5: scalar SSBO + per-list SSBOs; marshal/writeback | +| Launch path | Descriptors, pipeline, dispatch, `GpuJob.join` | +| `@Gpu` / `gpu()` | Compile and run user kernels (same typing model as CPU) | +| Workloads / packaging | Real numeric steps, docs, CI, capability gates | + +## Mental link: pack vs GpuPack + +| CPU pack | GpuPack | +|----------|---------| +| One C++ struct in process memory | Several `VkBuffer`s on the device | +| Field `int n` | Bytes at offset 0 in scalar SSBO | +| Field `vector x` | Separate SSBO whose bytes are the floats | +| Kernel gets C++ references | Shader gets bindings 0, 1, 2, … | + +Same **roles**. Different **implementation**. + +## What "done" looks like for a library user + +```python +from cthreads import Gpu, gpu + +@Gpu +def saxpy(n: int, a: float, x: list[float], y: list[float]) -> None: + ... + +x = [1.0, 2.0, 3.0, 4.0] +y = [10.0, 10.0, 10.0, 10.0] +gpu(saxpy, 4, 2.0, x, y).join() +# y is updated in place, like a CPU Thread job +``` + +No `DeviceBuffer` in that story. Vulkan is an implementation detail of `_ext`. diff --git a/docs/vk_guide/02-mental-model.md b/docs/vk_guide/02-mental-model.md new file mode 100644 index 0000000..cb70a1d --- /dev/null +++ b/docs/vk_guide/02-mental-model.md @@ -0,0 +1,143 @@ +# 02 — Mental model: objects, handles, and "nothing runs until submit" + +## Vulkan is an object graph + +Almost everything you create is an **object**: + +- Instance +- Physical device (the GPU hardware the loader knows about) +- Logical device (your open connection to that GPU) +- Queue (a port you submit work to) +- Buffer (a typed "array of bytes" object) +- Device memory (the actual allocated slab) +- Command pool / command buffer (recorded GPU instructions) +- Fence (CPU-side "GPU finished" signal) +- Shader module, pipeline, descriptor set (launch path) + +Objects form a tree of ownership. Typical rule: + +**Destroy children before parents.** Example: destroy buffers and command pools before destroying the logical device; destroy the device before the instance. + +## Handles are not pointers you dereference + +When Vulkan returns a `VkBuffer`, you get an opaque **handle** (often an integer-sized id). + +You cannot do: + +```cpp +buffer->data[i] = 1.0f; // NOT how Vulkan works +``` + +You talk to objects only through API functions: + +```cpp +vkDestroyBuffer(device, buffer, nullptr); +``` + +`VK_NULL_HANDLE` means "no object" (like a null pointer, but for handles). + +In our code, `Context` stores many handles and many **function pointers** (`PFN_vkCreateBuffer`, …) because we load Vulkan dynamically. + +## Explicit means: you fill structs + +Vulkan APIs almost always take a `*CreateInfo` struct: + +```cpp +VkBufferCreateInfo info{}; +info.sType = VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO; +info.size = 1024; +info.usage = VK_BUFFER_USAGE_STORAGE_BUFFER_BIT; +... +vkCreateBuffer(device, &info, nullptr, &buffer); +``` + +Rules of thumb: + +1. Zero the struct (`{}` in C++). +2. Set `sType` to the matching enum (tells the driver which struct this is). +3. Set only the fields you need; leave the rest zero/null. +4. Check the `VkResult` return (`VK_SUCCESS` or throw). + +This pattern repeats everywhere. Once you recognize it, Vulkan stops feeling like "a million APIs" and starts feeling like "the same paperwork for every object." + +## CPU time vs GPU time + +This is the biggest intuition jump. + +```text +CPU thread GPU +--------- --- +record commands into CB +submit CB to queue ------------> (maybe starts later) +do other CPU work execute copies / dispatches +vkWaitForFences <--------------- signals fence when done +``` + +If you forget to wait, you might download a buffer the GPU has not finished writing. +That is a classic bug class (CPU/GPU race). + +The upload/download helpers wait on a fence after the copy so the CPU memcpy of staging memory is safe. + +## Queues are submission ports + +A GPU exposes **queue families** (groups of queues with capabilities): + +- graphics +- compute +- transfer +- present (for windows — we ignore) + +cthreads picks a family that supports **compute** (and also uses it for buffer copies). +We create one logical device requesting that family, then call `vkGetDeviceQueue` to get `VkQueue queue`. + +All work we care about goes: **record -> vkQueueSubmit(queue, ...)**. + +## Two kinds of "memory" people confuse + +1. **Host memory** — normal RAM your C++ `memcpy` can touch. +2. **Device memory** — memory the GPU allocates through Vulkan (`VkDeviceMemory`). + +Some device memory is **host-visible**: the driver can give you a CPU pointer via `vkMapMemory` (staging). +Some is **device-local**: fast for the GPU, **not** for persistent CPU poking (our SSBOs). + +A `VkBuffer` alone is not enough. You must: + +1. Create the buffer object (describes size/usage). +2. Ask for memory requirements. +3. Allocate matching `VkDeviceMemory`. +4. **Bind** memory to the buffer. + +Until bind succeeds, the buffer cannot store data. + +## Layers of our stack (keep this map) + +```text +Python cthreads.gpu / future gpu() + | + v +pybind _ext.gpu + | + v +GpuPack (option 5) + | + v +memory:: create/upload/download + | + v +Context (instance/device/queue/PFNs) + | + v +vulkan-1.dll / libvulkan.so.1 (loader) + | + v +GPU driver (ICD) +``` + +## Validation layers (optional while learning) + +The Vulkan SDK can enable **validation layers**: extra checks that print readable errors when you misuse the API. + +For shipping cthreads to users we do **not** require layers. +While developing, turning them on is highly recommended (see SDK docs for `VK_LAYER_KHRONOS_validation`). + +We may gate this behind something like `CTHREADS_VK_VALIDATE=1` later. Not required to understand Context and memory. diff --git a/docs/vk_guide/03-sdk-runtime-drivers.md b/docs/vk_guide/03-sdk-runtime-drivers.md new file mode 100644 index 0000000..bf6be77 --- /dev/null +++ b/docs/vk_guide/03-sdk-runtime-drivers.md @@ -0,0 +1,120 @@ +# 03 — SDK vs runtime vs drivers + +This chapter clears the most common confusion: **"Do end users need the Vulkan SDK?"** + +**Short answer for cthreads:** + +- **End users:** GPU drivers only (they typically already have `vulkan-1.dll` / `libvulkan.so.1`). +- **Contributors building with `CTHREADS_GPU=ON`:** Vulkan SDK (headers) + drivers. +- **Default CPU wheels do not hard-link** `vulkan-1.lib`. + +## Three different things + +### 1. GPU driver (required to *run* Vulkan apps) + +NVIDIA / AMD / Intel install a **Vulkan Installable Client Driver (ICD)** with the graphics driver. + +That is what makes games and compute apps find a GPU. + +Sanity checks on Windows: + +- Device Manager shows the GPU. +- `vulkaninfo` (from SDK tools) lists the GPU if the runtime works. + +### 2. Vulkan loader (`vulkan-1.dll` / `libvulkan.so.1`) + +The **loader** is a small library that: + +- finds ICDs +- exposes `vkGetInstanceProcAddr` +- dispatches calls to the right driver + +On Windows it is typically `vulkan-1.dll` (comes with the driver / runtime). +On Linux it is `libvulkan.so.1` (package like `vulkan-icd-loader` + vendor ICD). + +The Context bootstrap does: + +```text +LoadLibrary("vulkan-1.dll") / dlopen("libvulkan.so.1") +-> get vkGetInstanceProcAddr +-> resolve every other function by name +``` + +So the **process must find that DLL/SO at runtime**. Linking the import library at build time is optional; we chose **not** to require it for default CPU builds. + +### 3. Vulkan SDK (LunarG) — mainly for *developers* + +The SDK gives you: + +| Piece | Use | +|-------|-----| +| Headers (`vulkan/vulkan.h`) | Compile C++ that mentions `VkBuffer`, `PFN_vkCreateInstance`, … | +| `vulkaninfo` | Diagnose devices/extensions | +| Validation layers | Catch API misuse while developing | +| Shader tools (`glslangValidator`, etc.) | Compile GLSL -> SPIR-V (we may use shaderc in-library later) | + +**End users of cthreads do not install the SDK** just to `pip install` a CPU wheel. +**Contributors** install the SDK so `find_package(Vulkan)` can see headers when `CTHREADS_GPU=ON`. + +## How this shows up in our CMake + +When `CTHREADS_GPU=ON`: + +- CMake runs `find_package(Vulkan REQUIRED)` — needs SDK/headers on the machine building `_ext`. +- We compile `context.cpp`, `memory.cpp`, … +- We define `CTHREADS_WITH_GPU=1`. +- We still **do not** need to link `Vulkan::Vulkan` if we resolve symbols dynamically (our design). + +When `CTHREADS_GPU=OFF`: + +- GPU sources are not compiled. +- Python `cthreads.gpu` soft-fails (`available()` false / `VulkanNotBuiltError`). + +## Practical Windows setup (contributors) + +1. Install recent GPU drivers. +2. Install [LunarG Vulkan SDK](https://vulkan.lunarg.com/). +3. Open **x64 Native Tools** / VS Dev Cmd. +4. Build: + +```bat +set CMAKE_ARGS=-DCTHREADS_GPU=ON +pip install -e . -v +``` + +Look for CMake status: `cthreads GPU: ON (Vulkan)`. + +5. Smoke: + +```python +from cthreads import gpu +print(gpu.available(), gpu.device_name() if gpu.available() else None) +``` + +## What can go wrong + +| Symptom | Likely cause | +|---------|----------------| +| `VulkanLoaderNotFound` | No `vulkan-1.dll` on PATH / not installed with driver | +| `VulkanNoDevice` | Loader ok but no compute-capable ICD / bad driver | +| `missing vkDestroyInstance` style bugs | Resolved instance functions with null instance (we fixed this) | +| CMake cannot find Vulkan | SDK not installed or env not visible to the build | +| Tiny wheel / no GPU log | `CTHREADS_GPU` not actually ON (wrong shell env, or cached CMake) | + +## Mental picture + +```text +Your C++ (_ext) + | dynamic LoadLibrary + v +vulkan-1.dll (LOADER) + | finds + v +nvoglv64.dll / amdvlk / igvx64 ... (ICD / DRIVER) + | + v +GPU hardware +``` + +Headers from the SDK are only a **compile-time** description of that API. +They are not the DLL. diff --git a/docs/vk_guide/04-instance-device-queue.md b/docs/vk_guide/04-instance-device-queue.md new file mode 100644 index 0000000..94647f4 --- /dev/null +++ b/docs/vk_guide/04-instance-device-queue.md @@ -0,0 +1,145 @@ +# 04 — Instance, physical device, logical device, queue + +This is the "plug the GPU in" chapter. In cthreads this lives in `Context`. + +## The four names people mix up + +| Name | What it is | Analogy | +|------|------------|---------| +| **Loader** | DLL/SO that finds drivers | "Phone book + switchboard" | +| **Instance** (`VkInstance`) | Your app's connection to Vulkan | "Logged into the Vulkan system" | +| **Physical device** (`VkPhysicalDevice`) | One GPU the loader can see | "The hardware card" | +| **Logical device** (`VkDevice`) | Your open session on that GPU | "File handle / open connection" | +| **Queue** (`VkQueue`) | Submission port for work | "Inbox where GPU jobs go" | + +You always need: loader -> instance -> pick physical device -> create logical device -> get queue. + +## Step by step (what `init()` does) + +### Step A — Open the loader + +Windows: `LoadLibraryA("vulkan-1.dll")` +Linux: `dlopen("libvulkan.so.1", …)` + +Then get **one** symbol by name: `vkGetInstanceProcAddr`. + +Everything else is resolved through that function. + +### Step B — Resolve `vkCreateInstance` (global) + +Some functions can be queried with a **null instance**: + +- `vkCreateInstance` +- (and a few enumerate-instance helpers we do not need yet) + +```text +get_fn(NULL, "vkCreateInstance") +``` + +### Step C — Create the instance + +You fill `VkApplicationInfo` (app name, engine name, API version) and `VkInstanceCreateInfo`. + +We request **Vulkan 1.1** (`VK_API_VERSION_1_1`). That is enough for our compute path. + +Out parameter: `c.instance`. + +### Step D — Resolve instance-level functions + +**Important rule (easy to get wrong):** + +After you have an instance, resolve functions like: + +- `vkDestroyInstance` +- `vkEnumeratePhysicalDevices` +- `vkGetPhysicalDeviceProperties` +- `vkCreateDevice` +- … + +with **`c.instance`**, not `NULL`. + +If you pass `NULL`, many loaders return nullptr for those names. That is a common failure mode (`missing vkDestroyInstance` / similar). + +### Step E — Enumerate physical devices + +Vulkan uses a two-call pattern constantly: + +1. Call with `nullptr` data pointer to get **count**. +2. Allocate a vector of that size. +3. Call again to fill the vector. + +```text +enumerate(instance, &count, nullptr) +vector devices(count) +enumerate(instance, &count, devices.data()) +``` + +### Step F — Pick a device + queue family + +For each physical device: + +1. Read properties (name, discrete vs integrated). +2. Read queue family properties. +3. Find a family whose flags include `VK_QUEUE_COMPUTE_BIT`. +4. Prefer `VK_PHYSICAL_DEVICE_TYPE_DISCRETE_GPU` (score higher). + +Store: + +- `physical_device` +- `queue_family` (index) +- `device_name` (string for Python) + +### Step G — Create the logical device + +You request queues: + +```text +VkDeviceQueueCreateInfo: family index = queue_family, count = 1, priority = 1.0 +VkDeviceCreateInfo: those queue infos +vkCreateDevice(physical_device, ...) +``` + +Out: `c.device`. + +### Step H — Get the queue handle + +```text +vkGetDeviceQueue(device, queue_family, 0, &queue) +``` + +Queue index `0` means "first queue of that family." + +### Step I — Mark ready + +`c.ready = true`. + +On any failure: destroy what you created and unload the loader (`shutdown_unlocked`). + +## Thread safety in our Context + +`init` / `shutdown` take a mutex. Double-checked locking: + +- Fast path: if `ready`, return. +- Lock, check again, then initialize once. + +`available()` tries `init()` and returns false on failure (no throw). +`device_name()` tries `init()` and may throw mapped errors. + +## Shutdown order + +1. Destroy logical device (also invalidates queues). +2. Destroy instance. +3. Clear physical device handle. +4. `FreeLibrary` / `dlclose` the loader. +5. Null all function pointers so a late call cannot jump into unloaded code. + +Command pools, fences, and buffers must be destroyed **before** destroying the logical device. + +## What the Context layer does *not* do + +- No buffers +- No command pools +- No shaders +- No descriptors + +It only proves: **we can talk to a compute-capable GPU and print its name.** diff --git a/docs/vk_guide/05-dynamic-loading.md b/docs/vk_guide/05-dynamic-loading.md new file mode 100644 index 0000000..1cd449a --- /dev/null +++ b/docs/vk_guide/05-dynamic-loading.md @@ -0,0 +1,84 @@ +# 05 — Dynamic loading and `PFN_*` function pointers + +## Why we do not link `vulkan-1.lib` by default + +If you link the Vulkan import library: + +- Every machine that imports `_ext` needs the loader present **or** load fails at process start. +- CPU-only users would pay a Vulkan dependency they do not need. + +cthreads design: + +- Default build: no Vulkan. +- `CTHREADS_GPU=ON`: compile GPU code against **headers only**, resolve symbols at runtime. + +## What a `PFN_` is + +Vulkan headers define typedefs like: + +```cpp +typedef VkResult (VKAPI_PTR *PFN_vkCreateBuffer)(VkDevice, const VkBufferCreateInfo*, ...); +``` + +So `PFN_vkCreateBuffer` means: "pointer to a function with that signature." + +In `Context` we store: + +```cpp +PFN_vkCreateBuffer vkCreateBuffer = nullptr; +``` + +After init: + +```cpp +context.vkCreateBuffer(device, &info, nullptr, &buffer); +``` + +That is a normal indirect call through a function pointer. + +## The bootstrap chain + +```text +1. LoadLibrary / dlopen +2. GetProcAddress / dlsym("vkGetInstanceProcAddr") +3. vkGetInstanceProcAddr(instance_or_null, "vkCreateInstance") +4. create instance +5. vkGetInstanceProcAddr(real_instance, "vkCreateBuffer") // etc. +``` + +The Context helper `get_fn(context, instance, "name")`: + +1. Calls `vkGetInstanceProcAddr`. +2. Throws if null (`cthreads.gpu.VulkanInitFailed: missing …`). +3. Casts to the typed `PFN_*`. + +## Global vs instance vs device level (practical rules) + +You do not need the full spec table. Use these project rules: + +1. **Before instance exists:** only resolve truly global entry points (`vkCreateInstance`, …) with `VK_NULL_HANDLE`. +2. **After instance exists:** resolve everything else we need with `c.instance`. +3. **After device exists:** we still resolve device commands via `vkGetInstanceProcAddr(instance, name)` (loader trampoline). That matches our code. (Advanced apps often use `vkGetDeviceProcAddr`; not required for our path.) + +## Why so many pointers on `Context`? + +Buffer, command, and fence functions are loaded once during Context init and cleared on shutdown. + +Groups in `context.hpp`: + +1. Instance / device bootstrap +2. Buffer + memory +3. Command pool / command buffer / copy / fence / submit + +When adding a new Vulkan call in `memory.cpp`, first check: is its `PFN_` on `Context` and loaded in `create_instance_and_device`? If not, add it there. + +## Errors contributors will see + +| Message prefix | Meaning | +|----------------|---------| +| `VulkanLoaderNotFound` | DLL/SO missing | +| `VulkanInitFailed: missing X` | `get_fn` returned null for name `X` | +| `VulkanNoDevice` | No compute-capable GPU | +| `VulkanInitFailed: vkCreate… failed` | Driver rejected create | + +Python maps these string prefixes to exception types in `cthreads.gpu.errors`. diff --git a/docs/vk_guide/06-buffers-and-memory.md b/docs/vk_guide/06-buffers-and-memory.md new file mode 100644 index 0000000..d1c0321 --- /dev/null +++ b/docs/vk_guide/06-buffers-and-memory.md @@ -0,0 +1,142 @@ +# 06 — Buffers and device memory + +This is the core of the memory layer (before copies). + +## Two objects, one job + +To store N bytes the GPU can use, you need **both**: + +1. **`VkBuffer`** — describes a buffer resource: size, usage flags, sharing mode. +2. **`VkDeviceMemory`** — the actual allocated memory slab. + +Then you **bind** them: + +```text +vkBindBufferMemory(device, buffer, memory, offset=0) +``` + +Until bind succeeds, the buffer is an empty shell. + +In our API this pair is `GpuBuffer`: + +```text +GpuBuffer { + buffer, memory, size, mapped, kind +} +``` + +## Usage flags (what the buffer is allowed to do) + +When creating a buffer you set `VkBufferUsageFlags`. Think of them as permissions. + +| Flag | Meaning for us | +|------|----------------| +| `TRANSFER_SRC` | May be source of a GPU copy | +| `TRANSFER_DST` | May be destination of a GPU copy | +| `STORAGE_BUFFER` | May be bound as an SSBO for compute shaders | + +cthreads buffer kinds: + +**Staging** + +```text +TRANSFER_SRC | TRANSFER_DST +``` + +**DeviceLocal** (shader data) + +```text +STORAGE_BUFFER | TRANSFER_SRC | TRANSFER_DST +``` + +Why transfer bits on device-local? Because upload/download copy through them. + +## Memory properties (where the bytes live) + +After `vkCreateBuffer`, call `vkGetBufferMemoryRequirements`: + +- `size` — how many bytes to allocate (may be larger than you asked; alignment) +- `alignment` +- `memoryTypeBits` — bitmask of legal memory type indices + +Then look at the physical device's memory types (`vkGetPhysicalDeviceMemoryProperties`). + +Each type has flags like: + +| Flag | Meaning | +|------|---------| +| `HOST_VISIBLE` | CPU can map it and read/write | +| `HOST_COHERENT` | No manual flush/invalidate needed for visibility | +| `DEVICE_LOCAL` | Lives in GPU-friendly memory (often VRAM) | + +### `find_memory_type` in cthreads + +```text +for i in 0 .. memoryTypeCount-1: + if bit i not set in type_bits: skip + if (type.flags & requested) == requested: return i +throw if none +``` + +Requested properties: + +- Staging: `HOST_VISIBLE | HOST_COHERENT` +- DeviceLocal: `DEVICE_LOCAL` + +## Mapping (CPU pointer into device memory) + +`vkMapMemory` returns a `void*` the CPU can `memcpy` into. + +**Only do this for host-visible memory.** + +cthreads rules: + +- Staging: map once at create, keep `GpuBuffer.mapped` until destroy. +- DeviceLocal: **never** persistently map. Upload goes through staging. + +Unmap with `vkUnmapMemory` before freeing memory. + +## Create algorithm (what `create_buffer` does) + +```text +1. Validate context.ready, size > 0 +2. Choose usage + memory property flags from BufferKind +3. vkCreateBuffer +4. vkGetBufferMemoryRequirements +5. find_memory_type(...) +6. vkAllocateMemory (allocationSize = mem_reqs.size) +7. vkBindBufferMemory(..., offset 0) +8. If Staging: vkMapMemory -> mapped +9. Store caller size in GpuBuffer.size +``` + +On failure: destroy whatever was created (buffer/memory) before throwing. + +## Destroy algorithm + +```text +if mapped: unmap +destroy buffer +free memory +zero the GpuBuffer fields +``` + +Empty buffer (already null handles): no-op. + +## Why size 0 is rejected + +Vulkan implementations often dislike zero-sized buffers. +Empty Python lists are handled by **GpuPack** (no buffer / skip), not by creating a 0-byte `VkBuffer`. + +## Sharing mode + +cthreads uses `VK_SHARING_MODE_EXCLUSIVE`: one queue family owns the buffer. +We only use one compute/transfer family, so exclusive is correct and simpler. + +## Common mistakes + +1. Creating a buffer and forgetting to allocate/bind memory. +2. Mapping device-local memory that is not host-visible. +3. Using `GpuBuffer.size` wrong vs `mem_reqs.size` (we store the **caller** size; allocation may be larger — copies use caller size). +4. Destroying the device while buffers still exist. +5. Assuming `list` memory in Python is the GPU buffer — it is not; marshal must copy. diff --git a/docs/vk_guide/07-staging-upload-download.md b/docs/vk_guide/07-staging-upload-download.md new file mode 100644 index 0000000..cc4235a --- /dev/null +++ b/docs/vk_guide/07-staging-upload-download.md @@ -0,0 +1,96 @@ +# 07 — Staging, upload, and download (final memory model) + +## The problem + +Shaders want **device-local** buffers (fast). +The CPU wants to `memcpy` from Python lists (host RAM). + +Those are often **different** memory heaps. You cannot always map the SSBO and pretend it is a C array. + +## The solution in cthreads + +```text +UPLOAD: + host pointer + --memcpy--> staging (host-visible, mapped) + --GPU copy--> device-local SSBO + +DOWNLOAD: + device-local SSBO + --GPU copy--> staging (mapped) + --memcpy--> host pointer +``` + +This is the permanent design for list/scalar marshal traffic. + +## What `upload_buffer` does (in `memory.cpp` today) + +Given a **device-local** `GpuBuffer`: + +1. Check kind, null data, size bounds. +2. `create_buffer(..., Staging)` of `size` bytes. +3. `memcpy(staging.mapped, data, size)`. +4. Record/submit `vkCmdCopyBuffer(staging -> device)` and wait on a fence. +5. `destroy_buffer(staging)`. + +`download_buffer` reverses the copy direction, then `memcpy` out of staging. + +## One-shot command pool vs TransferEngine + +In the current memory implementation, each upload/download creates a temporary: + +- command pool +- command buffer +- fence + +then destroys them after wait. + +That is **correct** and matches the final *API*. + +A later **TransferEngine** on `Context` can reuse one pool/fence/scratch staging for speed. +That is an optimization of the same path, not a new memory model. + +## Barriers (intuition for later) + +GPUs reorder work. Sometimes you need a **pipeline barrier** so "copy finished" happens before "shader reads." + +For CPU-side round-trips (copy then CPU wait then CPU read), a fence wait is enough because the CPU does not read device memory until the GPU signaled. + +When a shader runs in the same command buffer after a copy, insert a barrier between copy and dispatch. See the pipelines chapter. + +## Why not host-visible SSBOs only? + +It works on some integrated GPUs and for tiny demos. +It is the wrong long-term model for: + +- discrete GPUs (VRAM vs sysmem) +- large lists +- matching how production Vulkan compute is written + +cthreads does not ship a "map the SSBO forever" production path. + +## End-to-end round-trip (what tests will prove) + +```text +floats = [1, 2, 3, 4] +create device-local buffer (16 bytes) +upload_buffer(buf, floats) +download_buffer(buf, out) +assert out == floats +destroy +``` + +Same idea for a small scalar struct blob. + +## How this becomes GpuPack + +```text +GpuPack: + scalars: GpuBuffer DeviceLocal # bytes of std430 struct + lists[]: GpuBuffer DeviceLocal # one per list arg + +upload pack: upload each piece +download pack: download each piece into Python objects +``` + +Staging is an implementation detail inside `upload_buffer` / `download_buffer` (or the transfer engine). Users never see it. diff --git a/docs/vk_guide/08-commands-fences.md b/docs/vk_guide/08-commands-fences.md new file mode 100644 index 0000000..fe4833e --- /dev/null +++ b/docs/vk_guide/08-commands-fences.md @@ -0,0 +1,114 @@ +# 08 — Command buffers, queues, and fences + +## Why command buffers exist + +The GPU does not execute your C++ line by line. +You **record** a list of GPU commands, then **submit** that list to a queue. + +Think of a command buffer as a recipe card: + +```text +Begin + CopyBuffer A -> B + (later) Bind pipeline + (later) Dispatch +End +Submit to queue +``` + +Recording is cheap CPU work. Executing happens on the GPU, possibly later. + +## Objects involved + +| Object | Role | +|--------|------| +| `VkCommandPool` | Allocator/owner of command buffers for one queue family | +| `VkCommandBuffer` | The recorded recipe | +| `VkQueue` | Where recipes are submitted | +| `VkFence` | CPU-waitable "this submit finished" flag | + +## Record / submit / wait pattern (copy) + +This is exactly what our `copy_buffer_and_wait` helper does: + +```text +1. vkCreateCommandPool (family = context.queue_family) +2. vkAllocateCommandBuffers (one primary buffer) +3. vkBeginCommandBuffer (ONE_TIME_SUBMIT) +4. vkCmdCopyBuffer(src, dst, region) +5. vkEndCommandBuffer +6. vkCreateFence +7. vkQueueSubmit(queue, cmd, fence) +8. vkWaitForFences(..., UINT64_MAX) +9. destroy fence, free command buffer, destroy pool +``` + +### Primary vs secondary + +cthreads uses **primary** command buffers (can be submitted directly). +Secondary buffers are for advanced recording reuse; ignore for now. + +### One-time submit + +`VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT` means: we will record, submit once, then throw away / reset. +Perfect for copies. + +## Fences vs semaphores (only fences matter for us now) + +| Primitive | Who waits | Typical use | +|-----------|-----------|-------------| +| **Fence** | CPU | `join`, wait for copy before memcpy | +| **Semaphore** | GPU | Sync between queue submits / present (graphics) | + +cthreads launch/wait uses **fences**. +Semaphores can be ignored until multi-queue or swapchains (not part of this compute path). + +## Reset and reuse (launch hot-path) + +Instead of destroy/recreate every time: + +- Keep a pool with `RESET` flag or reset individual buffers. +- `vkResetFences` before reuse. +- Re-record and submit again. + +Reusing pools and cached pipelines is how launches stay cheap. + +## Queue submit is asynchronous + +After `vkQueueSubmit` returns `VK_SUCCESS`, the GPU might still be working. +Only after the fence signals is it safe to: + +- destroy the buffers you copied from/to (if you are done with them) +- `memcpy` from staging after a download copy +- tell Python the job finished + +`GpuJob.join()` waits on the fence for the dispatch submit. + +## Ordering: destroy vs in-flight work + +Never destroy a buffer or pool while the GPU still has commands referencing it. +Wait for fences first, then destroy. + +The upload helper waits, then destroys staging — correct order. + +## Transient pools + +`VK_COMMAND_POOL_CREATE_TRANSIENT_BIT` hints that command buffers are short-lived. +cthreads uses that for one-shot copies. + +## How this connects to compute dispatch (preview) + +A full kernel launch command buffer looks like: + +```text +Begin + (optional) barriers after uploads + Bind pipeline + Bind descriptor set (buffers) + Push constants (optional) + vkCmdDispatch(groups_x, groups_y, groups_z) +End +Submit + fence +``` + +Same machinery as copy; different commands inside. diff --git a/docs/vk_guide/09-descriptors-ssbo.md b/docs/vk_guide/09-descriptors-ssbo.md new file mode 100644 index 0000000..ba0d26a --- /dev/null +++ b/docs/vk_guide/09-descriptors-ssbo.md @@ -0,0 +1,100 @@ +# 09 — Descriptors and SSBOs (how shaders see buffers) + +The memory layer creates buffers. The launch path lets shaders read them. This chapter is the bridge. + +## The problem descriptors solve + +You have device-local buffers on the CPU/C++ side (`VkBuffer` handles). +A shader is SPIR-V running on the GPU. It cannot see your C++ variables. + +You need a **binding table**: "shader binding 0 is this buffer, binding 1 is that buffer." + +That table is built with: + +- Descriptor set layout (the schema: binding i is a storage buffer) +- Descriptor pool + descriptor set (an instance of that schema) +- Descriptor writes (`vkUpdateDescriptorSets`) pointing at actual `VkBuffer`s +- `vkCmdBindDescriptorSets` before dispatch + +## SSBO = storage buffer + +In GLSL compute: + +```glsl +layout(set = 0, binding = 1, std430) buffer XBlock { + float data[]; +} x; +``` + +That means: + +- descriptor set 0 +- binding 1 +- storage buffer +- std430 packing +- unsized array of floats at the end of the block + +On the C++ side, binding 1's descriptor must reference the `VkBuffer` that holds those floats. + +## cthreads binding convention (option 5) + +| Binding | Contents | +|---------|----------| +| 0 | Scalar SSBO (`std430` struct: `n`, `a`, …) | +| 1 | First list SSBO | +| 2 | Second list SSBO | +| … | More lists | + +Saxpy example: + +```glsl +layout(set=0, binding=0, std430) buffer Scalars { int n; float a; } scalars; +layout(set=0, binding=1, std430) buffer X { float data[]; } x; +layout(set=0, binding=2, std430) buffer Y { float data[]; } y; +``` + +No pointers inside `Scalars`. Bindings do the wiring. + +## Descriptor types we care about + +| Vulkan type | GLSL idea | +|-------------|-----------| +| `STORAGE_BUFFER` | `buffer { ... }` (read/write) | +| `UNIFORM_BUFFER` | `uniform` block (we avoid for mutable pack) | + +cthreads uses **storage buffers for scalars and lists** (option 5 = all SSBO). + +## Lifecycle sketch (launch path) + +```text +1. Create descriptor set layout (bindings 0..N as STORAGE_BUFFER) +2. Create pipeline layout (includes that set layout) +3. Create compute pipeline (shader module + layout) +4. Create descriptor pool; allocate one set +5. For each launch: + write descriptors to point at this GpuPack's buffers + record: bind pipeline, bind set, dispatch +6. On shutdown: destroy sets/pool/pipeline/layout/module (order matters) +``` + +## Why update descriptors per launch? + +Because each job has its own GpuPack buffers (or at least different list buffers). +The layout stays cached; the **buffer handles** inside the set change. + +CB reuse and pipeline caching optimize recording/submit; the descriptor model stays. + +## Ranges + +When writing a descriptor for a buffer you specify offset + range (often `VK_WHOLE_SIZE` or `buffer.size`). +For our tightly packed arrays, whole buffer / exact size both work if the buffer was created for that array only. + +## Mental model + +```text +Shader source says: binding 1 is float array x +Descriptor set says: binding 1 -> VkBuffer handle of list x +Dispatch runs: loads/stores hit that memory +``` + +If descriptors are wrong, you get garbage or crashes — validation layers help. diff --git a/docs/vk_guide/10-std430-layouts.md b/docs/vk_guide/10-std430-layouts.md new file mode 100644 index 0000000..ff81405 --- /dev/null +++ b/docs/vk_guide/10-std430-layouts.md @@ -0,0 +1,93 @@ +# 10 — `std430` layouts (scalar structs that match C++) + +## Why layout rules exist + +The CPU uploads a blob of bytes for scalars. +The shader interprets those bytes as a struct. + +If padding differs, `n` and `a` get scrambled. Silent corruption — nasty to debug. + +GLSL storage buffers default to **`std430`** packing when you write: + +```glsl +layout(..., std430) buffer Scalars { ... }; +``` + +Your host C++ struct must match **std430**, not "whatever the C++ compiler naturally did" unless you carefully match. + +## Rules of thumb (std430) + +For members we use in v1: + +| Type | Size | Alignment | +|------|------|-----------| +| `int` / `uint` / `float` | 4 | 4 | +| `bool` in GLSL | treat carefully; prefer `int` as 0/1 on host | 4 | +| `vec2` | 8 | 8 | +| `vec3` | 12 | **16** (align like vec4) | +| `vec4` | 16 | 16 | + +Arrays of `float`/`int` in an SSBO are tightly packed (stride 4). + +Structs get padding so each member starts at a multiple of its alignment. +The struct's overall alignment is the max of its members (roughly). + +## Easy case (saxpy scalars) + +GLSL: + +```glsl +layout(set=0, binding=0, std430) buffer Scalars { + int n; + float a; +} scalars; +``` + +Host: + +```cpp +struct ScalarPack { + int32_t n; // offset 0 + float a; // offset 4 +}; +// sizeof == 8, no surprise padding +``` + +Use fixed-width types (`int32_t`) so Windows LLP64 and Linux agree. + +## Hard case (why vec3 bites) + +```glsl +float a; +vec3 v; +``` + +`v` must start at offset 16, not 4. Bytes 4..15 are padding. + +Host must insert the same padding or use `alignas`. + +**For the current cthreads GPU path:** prefer scalars that are int/float/bool-as-int. Avoid vec3 in the scalar blob until a layout emitter matches std430 automatically. + +## Lists are not inside the scalar struct + +Do not put `float x[];` inside the scalar block when you also need `y[]`. +Unsized arrays in std430 must be last, and you only get one. + +That is another reason cthreads option 5 uses **separate list SSBOs**. + +## How codegen will help later + +The `@Gpu` emitter should emit: + +1. GLSL `std430` scalar block from kernel meta field order. +2. Matching host mirror struct / byte offsets for marshal. + +Until then, hand-written smoke tests use a tiny fixed struct both sides agree on. + +## Checklist when adding a scalar field + +1. Append field to GLSL block in the same order. +2. Append field to host struct with matching type width. +3. Recompute offsets; watch for padding. +4. Bump any cached shader hash (old SPIR-V would mismatch). +5. Upload `sizeof(host_struct)` bytes (include trailing padding if required by std430). diff --git a/docs/vk_guide/11-spirv-pipelines-dispatch.md b/docs/vk_guide/11-spirv-pipelines-dispatch.md new file mode 100644 index 0000000..320514d --- /dev/null +++ b/docs/vk_guide/11-spirv-pipelines-dispatch.md @@ -0,0 +1,121 @@ +# 11 — SPIR-V, pipelines, and compute dispatch + +## Shader languages in our stack + +| Form | Role | +|------|------| +| GLSL compute (human-written or emitted) | Source you can read | +| SPIR-V | Binary IR Vulkan drivers consume | +| `VkShaderModule` | Vulkan object wrapping SPIR-V bytes | +| `VkPipeline` (compute) | Compiled shader + layout ready to bind | + +Flow: + +```text +GLSL -> (shaderc / glslang at library build or runtime) -> SPIR-V bytes + -> vkCreateShaderModule + -> vkCreateComputePipelines +``` + +Users never run `glslangValidator` themselves. Either: + +- we commit `.spv` files, or +- we compile with shaderc inside `_ext` when `CTHREADS_GPU=ON`. + +## Compute shader shape (intuition) + +```glsl +#version 450 +layout(local_size_x = 64) in; // workgroup size + +layout(set=0, binding=0, std430) buffer Scalars { int n; float a; } scalars; +layout(set=0, binding=1, std430) buffer X { float data[]; } x; +layout(set=0, binding=2, std430) buffer Y { float data[]; } y; + +void main() { + uint i = gl_GlobalInvocationID.x; + if (i >= uint(scalars.n)) return; + y.data[i] = scalars.a * x.data[i] + y.data[i]; +} +``` + +### Workgroups and local size + +- `local_size_x = 64` means each workgroup runs 64 invocations. +- `vkCmdDispatch(groupCountX, 1, 1)` launches that many workgroups. +- Global id roughly: `groupIndex * local_size + localIndex`. + +For `n` elements: + +```text +groups = ceil(n / 64) +vkCmdDispatch(groups, 1, 1) +``` + +cthreads may also support serial-looking `@Gpu` bodies later (`local_size=1`). The product API stays launch/wait; the emitter chooses a parallelism model and documents it. + +## Pipeline layout + +Connects: + +- descriptor set layouts (which bindings exist) +- push constant ranges (optional) + +Even if you use a scalar SSBO, you might later mirror hot scalars into push constants for speed. Writeback truth remains the scalar SSBO. + +## Creating a compute pipeline (conceptual) + +```text +VkShaderModuleCreateInfo <- SPIR-V code +vkCreateShaderModule + +VkPipelineShaderStageCreateInfo stage: COMPUTE, module, entry "main" +VkComputePipelineCreateInfo: stage + pipelineLayout +vkCreateComputePipelines +``` + +Pipelines should be cached by shader hash so launches do not rebuild every time. + +## Dispatch command buffer (full story) + +```text +Begin command buffer + // ensure uploads visible to compute (barrier) when needed + vkCmdBindPipeline(COMPUTE, pipeline) + vkCmdBindDescriptorSets(... pack's set ...) + vkCmdDispatch(groupsX, 1, 1) +End +vkQueueSubmit(..., fence) +``` + +`GpuJob.join()`: + +```text +vkWaitForFences +download GpuPack -> Python writeback +``` + +## Barriers in one sentence + +A **pipeline barrier** tells the GPU: finish these earlier commands' memory effects before later commands use the data. + +Example: copy into `y` finished before compute reads/writes `y`. + +CPU-side round-trips can rely on fence waits. +In-GPU sequences need barriers between copy and dispatch (and between dispatches in a batch). + +## What to skip in graphics tutorials + +When reading vkguide / vulkan-tutorial, skip: + +- swapchain +- render pass +- framebuffer +- vertex buffers for drawing + +Keep: + +- buffer creation +- command buffers +- descriptors +- compute pipeline / dispatch (some tutorials bury this) diff --git a/docs/vk_guide/12-gpupack-marshal.md b/docs/vk_guide/12-gpupack-marshal.md new file mode 100644 index 0000000..3ff6a05 --- /dev/null +++ b/docs/vk_guide/12-gpupack-marshal.md @@ -0,0 +1,90 @@ +# 12 — GpuPack and marshal (option 5 end-to-end) + +## Definition + +**GpuPack** is the GPU analogue of the CPU args pack for one launch: + +```text +GpuPack { + GpuBuffer scalars; // DeviceLocal, std430 blob + vector lists; // DeviceLocal, one per list arg +} +``` + +Optional host-side metadata: + +- dtype per list (float vs int) +- element counts +- pointer/identity of Python objects for writeback + +## Build from Python args (marshal / smoke) + +Example call: + +```text +n: int = 4 +a: float = 2.0 +x: list[float] len 4 +y: list[float] len 4 +``` + +Steps: + +1. Build host `ScalarPack { n, a }` bytes. +2. `scalars = create_buffer(sizeof(ScalarPack), DeviceLocal)`. +3. `upload_buffer(scalars, &host_scalars, sizeof)`. +4. For `x`: create device-local buffer `4*sizeof(float)`; upload from a temporary C array filled from the Python list (or from a contiguous buffer you built while iterating). +5. Same for `y`. +6. Store Python object references for writeback. + +Empty list rule: **do not** create a 0-size VkBuffer; store "no buffer / count 0". + +## After compute (join) + +1. Wait fence. +2. `download_buffer` scalars -> host struct -> write Python ints/floats if they were mutable outputs (usually scalars are inputs; returns go in scalar blob too). +3. `download_buffer` each list -> update the **same** Python list objects in place (like CPU writeback). +4. Destroy pack buffers when the job is done (or retain if you design persistent buffers later — v1 can destroy per job). + +## Comparison table + +| Concern | CPU | GPU | +|---------|-----|-----| +| Where scalars live | Fields in C++ pack | Scalar SSBO bytes | +| Where lists live | `vector` / container in pack | Per-list SSBO | +| Fill | `pack_params` trampolines | `upload_buffer` | +| Run | call C++ function | dispatch SPIR-V | +| Sync mid-run | optional `__sync_state` | **not in v1** | +| Finish | writeback | download + writeback | + +## What marshal must guarantee + +1. Host scalar blob matches shader `std430` block. +2. List buffers contain tightly packed elements of the declared dtype. +3. Descriptor bindings match the emitter's binding assignment. +4. Download size matches what was uploaded (or the mutated full buffer). + +## Internal test API vs public API + +Tests may use an internal helper such as `_ext.gpu.roundtrip_lists(...)`. +That is **not** a user-facing `DeviceBuffer`. + +Public surface stays: + +```python +from cthreads import gpu +gpu.available() +# later: gpu(fn, *args).join() +``` + +## Lifetime vs Context + +```text +Context process lifetime (init/shutdown) +TransferEngine process lifetime (optional reuse) +GpuPack per job (or per batch) +GpuBuffer owned by pack / staging temps +``` + +Destroy packs before shutting down Context. +Destroy transfer engine resources before destroying the device. diff --git a/docs/vk_guide/13-map-to-our-code.md b/docs/vk_guide/13-map-to-our-code.md new file mode 100644 index 0000000..ecedd62 --- /dev/null +++ b/docs/vk_guide/13-map-to-our-code.md @@ -0,0 +1,67 @@ +# 13 — Map this guide to our repository + +## Files (GPU) + +| Path | Role | +|------|------| +| `src/cthreads/cpp/gpu/headers/context.hpp` | `Context` handles + all `PFN_*` | +| `src/cthreads/cpp/gpu/impl/context.cpp` | Load loader, init/shutdown, resolve entry points | +| `src/cthreads/cpp/gpu/headers/memory.hpp` | `BufferKind`, `GpuBuffer`, memory API | +| `src/cthreads/cpp/gpu/impl/memory.cpp` | find/create/destroy/upload/download | +| `src/cthreads/cpp/bindings/gpu_module.cpp` | pybind `cthreads._ext.gpu` | +| `src/cthreads/cpp/bindings/module.cpp` | `#ifdef CTHREADS_WITH_GPU` calls `bind_gpu` | +| `src/cthreads/cpp/CMakeLists.txt` | `CTHREADS_GPU` option + sources | +| `src/cthreads/python/cthreads/gpu/` | Python façade + errors | +| `tests/unit/test_gpu_context.py` | Context / availability tests (skip if no GPU) | + +Expected as the GPU stack grows: + +| Path (expected) | Role | +|-----------------|------| +| `gpu/headers/pack.hpp` (name may vary) | GpuPack | +| `gpu/.../transfer.*` | Reused command pool / staging | +| shader `.spv` / GLSL | Reference saxpy | +| more bindings | roundtrip / launch | + +## How to study with the code open + +### Pass 1 — Context + +1. Read guide 03-05. +2. Open `context.hpp` — read every PFN comment. +3. Open `context.cpp` — follow `open_loader` -> `create_instance_and_device` -> `shutdown_unlocked`. +4. Run `gpu.available()` / `device_name()` with `CTHREADS_GPU=ON`. + +### Pass 2 — Memory + +1. Read guide 06-08. +2. Open `memory.hpp` — `BufferKind` + `GpuBuffer`. +3. Walk `create_buffer` and `upload_buffer` in `memory.cpp` line by line. +4. Mentally simulate uploading 4 floats. + +### Pass 3 — Descriptors and shaders + +1. Read guide 09-12. +2. Sketch on paper the saxpy descriptor layout. +3. Sketch the command buffer for copy + dispatch + fence. + +## Build flag reminder + +```bat +set CMAKE_ARGS=-DCTHREADS_GPU=ON +pip install -e . -v +``` + +Wipe `build/` if CMake cached GPU off. + +## Python error mapping + +C++ throws `std::runtime_error` with prefixes like: + +```text +cthreads.gpu.VulkanLoaderNotFound: ... +cthreads.gpu.VulkanInitFailed: ... +cthreads.gpu.VulkanNoDevice: ... +``` + +`cthreads.gpu` catches and raises typed exceptions. Keep prefixes stable when adding new errors. diff --git a/docs/vk_guide/14-glossary.md b/docs/vk_guide/14-glossary.md new file mode 100644 index 0000000..baa145d --- /dev/null +++ b/docs/vk_guide/14-glossary.md @@ -0,0 +1,51 @@ +# 14 — Glossary + +| Term | Meaning in this project | +|------|-------------------------| +| **Vulkan** | Low-level cross-vendor GPU API | +| **Loader** | `vulkan-1.dll` / `libvulkan.so.1` — finds drivers, exports `vkGetInstanceProcAddr` | +| **ICD** | Installable Client Driver — vendor Vulkan implementation inside the GPU driver | +| **SDK** | LunarG Vulkan SDK — headers and tools for *developers* | +| **Instance** | App-wide Vulkan connection (`VkInstance`) | +| **Physical device** | A GPU enumerated by the loader | +| **Logical device** | Opened GPU connection (`VkDevice`) | +| **Queue family** | Group of queues with shared capabilities (compute/graphics/…) | +| **Queue** | Submission port (`VkQueue`) | +| **Handle** | Opaque id for a Vulkan object (`VkBuffer`, …) | +| **PFN_*** | Typed function pointer to a Vulkan entry point | +| **Buffer (`VkBuffer`)** | Buffer resource object (needs memory bound) | +| **Device memory** | Allocated GPU memory slab (`VkDeviceMemory`) | +| **Host-visible** | Memory the CPU can map | +| **Host-coherent** | Mapped writes/reads without manual flush | +| **Device-local** | GPU-fast memory; not our persistently mapped SSBO path | +| **Staging buffer** | Host-visible buffer used only for upload/download traffic | +| **Map** | Get a CPU pointer into host-visible device memory | +| **SSBO** | Storage buffer — shader-readable/writable buffer block | +| **Descriptor** | Binding that tells a shader which buffer is at binding i | +| **std430** | Packing rules for storage buffer structs/arrays | +| **Command pool** | Owns command buffers for a queue family | +| **Command buffer** | Recorded list of GPU commands | +| **Submit** | Give a command buffer to a queue for execution | +| **Fence** | CPU-waitable completion signal for a submit | +| **Semaphore** | GPU-side sync (we mostly ignore for now) | +| **Barrier** | GPU ordering/visibility constraint between commands | +| **SPIR-V** | Portable shader IR Vulkan consumes | +| **Shader module** | Vulkan object holding SPIR-V | +| **Pipeline (compute)** | Prepared compute shader + layout | +| **Dispatch** | Launch compute workgroups (`vkCmdDispatch`) | +| **Workgroup / local size** | Group of invocations that run together | +| **GpuPack** | Per-launch scalar SSBO + list SSBOs (option 5) | +| **Marshal** | Copy Python values into native/GPU pack storage | +| **Writeback** | Copy native/GPU results into the same Python objects | +| **Join** | Wait for GPU job completion then writeback | + +## Abbreviations + +| Short | Full | +|-------|------| +| H2D | Host to device (upload) | +| D2H | Device to host (download) | +| SSBO | Shader Storage Buffer Object | +| GIPA | `vkGetInstanceProcAddr` | +| BDA | Buffer device address (rejected for v1) | +| AoS | Array of structures (Threadable lists later) | diff --git a/docs/vk_guide/15-checklist.md b/docs/vk_guide/15-checklist.md new file mode 100644 index 0000000..f424b7d --- /dev/null +++ b/docs/vk_guide/15-checklist.md @@ -0,0 +1,44 @@ +# 15 — Checklist: concepts contributors should be able to explain + +Use this after reading. If a concept cannot be explained in plain language, revisit that chapter. + +## Foundations + +- [ ] Why cthreads GPU uses normal Python types, not a public DeviceBuffer +- [ ] Difference between loader, SDK, and GPU driver +- [ ] Why we dynamic-load Vulkan instead of linking for default wheels +- [ ] What `VK_NULL_HANDLE` means + +## Context layer + +- [ ] Instance vs physical device vs logical device vs queue +- [ ] Why instance-level functions must be resolved with a real instance +- [ ] How we pick a compute queue family and prefer discrete GPUs +- [ ] Shutdown order (device before instance before FreeLibrary) + +## Memory / transfer layer + +- [ ] Why a buffer needs both `VkBuffer` and `VkDeviceMemory` +- [ ] What `find_memory_type` does with `type_bits` and property flags +- [ ] Staging vs device-local (`BufferKind`) +- [ ] Why device-local SSBOs are not persistently mapped +- [ ] Upload path: memcpy staging -> GPU copy -> device-local +- [ ] Download path reverse +- [ ] Why size 0 buffers are rejected +- [ ] What a fence wait guarantees after `vkQueueSubmit` + +## Launch path / GpuPack / shaders + +- [ ] What an SSBO is in GLSL +- [ ] cthreads binding 0 = scalars, 1..N = lists convention +- [ ] What std430 padding is and why host/GPU must match +- [ ] SPIR-V vs GLSL vs pipeline vs dispatch +- [ ] What GpuPack contains (option 5) +- [ ] Why GPU jobs are launch/wait only (no mid-run sync) + +## When stuck + +1. Find the term in [14-glossary.md](./14-glossary.md). +2. Open the matching chapter from [README.md](./README.md). +3. Open the matching file in [13-map-to-our-code.md](./13-map-to-our-code.md). +4. If still unclear, ask with the chapter number + expected vs observed behavior in code. diff --git a/docs/vk_guide/16-api-reference.md b/docs/vk_guide/16-api-reference.md new file mode 100644 index 0000000..4ea3da5 --- /dev/null +++ b/docs/vk_guide/16-api-reference.md @@ -0,0 +1,1358 @@ +# 16 — Vulkan types, structs, enums, and functions (cthreads compute) + +This is a **reference chapter** for Vulkan symbols that matter to the cthreads +GPU compute path (Context, memory/transfers, and the planned launch path: +descriptors, pipelines, dispatch). It is not the full Vulkan API. + +Conventions used below: + +- **Handle** means an opaque Vulkan object id (do not dereference it as a C pointer). +- **`sType`** on create/info structs must be set to the matching `VK_STRUCTURE_TYPE_*`. +- **`pNext`** is usually `nullptr` in cthreads (extension chaining). +- **`pAllocator`** is always `nullptr` in cthreads (use the default allocator). +- **In / out** describes whether the caller fills the argument or the driver writes it. + +Related conceptual chapters: [02](./02-mental-model.md), [04](./04-instance-device-queue.md), +[06](./06-buffers-and-memory.md), [08](./08-commands-fences.md), [09](./09-descriptors-ssbo.md), +[11](./11-spirv-pipelines-dispatch.md). + +--- + +## Table of contents + +1. [Scalar and handle types](#1-scalar-and-handle-types) +2. [Results and booleans](#2-results-and-booleans) +3. [Structure type tags (`sType`)](#3-structure-type-tags-stype) +4. [Version helpers](#4-version-helpers) +5. [Instance and device bootstrap](#5-instance-and-device-bootstrap) +6. [Physical device queries](#6-physical-device-queries) +7. [Queues](#7-queues) +8. [Buffers and device memory](#8-buffers-and-device-memory) +9. [Command pools, command buffers, copies](#9-command-pools-command-buffers-copies) +10. [Fences and submit](#10-fences-and-submit) +11. [Launch path: descriptors](#11-launch-path-descriptors) +12. [Launch path: shaders, pipelines, dispatch](#12-launch-path-shaders-pipelines-dispatch) +13. [Launch path: barriers (overview)](#13-launch-path-barriers-overview) +14. [Function pointer typedefs (`PFN_*`)](#14-function-pointer-typedefs-pfn_) +15. [Quick index by name](#15-quick-index-by-name) + +--- + +## 1. Scalar and handle types + +### `VkBool32` + +- **What:** Vulkan boolean. Not C++ `bool`. +- **Values:** `VK_TRUE` (1), `VK_FALSE` (0). +- **Where used:** e.g. `vkWaitForFences(..., waitAll, ...)`. + +### `VkDeviceSize` + +- **What:** Unsigned 64-bit size/offset type for buffer sizes, copy sizes, memory offsets. +- **Why not `size_t`:** Vulkan wants a fixed width across platforms. +- **cthreads:** `GpuBuffer.size`, upload/download byte counts. + +### `VK_NULL_HANDLE` + +- **What:** Sentinel meaning "no object" for handle types. +- **Analogy:** Like `nullptr` for Vulkan handles. +- **cthreads:** Initial value of all handles on `Context` / `GpuBuffer`. + +### `VK_WHOLE_SIZE` + +- **What:** Special size meaning "the rest of the resource" (often used with map/descriptor ranges). +- **cthreads:** May appear when mapping whole allocations or descriptor buffer ranges. + +### Opaque handles (object ids) + +Handles are opaque ids created by `vkCreate*` / allocate functions and destroyed by +matching destroy/free functions. Destroy children before parents. + +Below is what each object is **for** (purpose), not only what the typedef is called. + +#### `VkInstance` + +**Purpose:** The process-wide "logged into Vulkan" object. It owns the connection to +the loader and is required before enumerating GPUs or creating a device. + +**How it works:** After `vkCreateInstance`, the loader knows this application exists +and can route instance-level calls. Almost every later bootstrap call takes the +instance (or a device created from it). Destroying the instance last tears down that +connection. cthreads keeps one instance on `Context`. + +#### `VkPhysicalDevice` + +**Purpose:** A handle representing one GPU the loader can see (hardware or software ICD). + +**How it works:** It is not "opened" yet. It is used only to **query** capabilities +(name, queue families, memory types) and as input to `vkCreateDevice`. Destroying the +instance invalidates physical device handles. cthreads picks one compute-capable +physical device (preferring discrete). + +#### `VkDevice` (logical device) + +**Purpose:** The opened session on a chosen GPU. Almost all GPU work objects (buffers, +pipelines, command pools) are created from a device. + +**How it works:** `vkCreateDevice` tells the driver which queue families this app will +use. After that, device-level entry points operate on this handle. Destroying the +device invalidates all objects created from it. cthreads uses one logical device. + +#### `VkQueue` + +**Purpose:** A submission port. Recorded command buffers are sent here to run on the GPU. + +**How it works:** Queues belong to a queue family (graphics/compute/transfer). +`vkGetDeviceQueue` returns a handle owned by the device (no separate destroy). +`vkQueueSubmit` is asynchronous: it returns when work is queued, not when finished. +cthreads uses one compute-capable queue for copies and (later) dispatches. + +#### `VkBuffer` + +**Purpose:** A GPU resource that represents a contiguous byte region with allowed uses +(copy, storage buffer, and so on). + +**How it works:** Creating a buffer only creates the *object*. Bytes live in +`VkDeviceMemory` after `vkBindBufferMemory`. Shaders never see the C++ handle directly; +descriptors bind the buffer to a binding number. cthreads uses staging buffers and +device-local SSBOs (`GpuBuffer`). + +#### `VkDeviceMemory` + +**Purpose:** An allocated slab of memory from a chosen memory type (host-visible, +device-local, …). + +**How it works:** `vkAllocateMemory` reserves memory; `vkBindBufferMemory` attaches it +to a buffer. Host-visible memory can be `vkMapMemory`'d for CPU `memcpy`. Device-local +memory is typically filled via GPU copies from staging. Free with `vkFreeMemory` +after unmapping and after the bound buffer is destroyed or no longer needs it +(cthreads destroys buffer then frees memory). + +#### `VkCommandPool` + +**Purpose:** An allocator/owner for command buffers tied to one queue family. + +**How it works:** Command buffers must come from a pool that matches the queue that +will submit them. Pools can be optimized for short-lived ("transient") buffers. +Destroying a pool frees its command buffers. cthreads currently creates a transient +pool per copy; a reused transfer engine would keep one pool alive on Context. + +#### `VkCommandBuffer` + +**Purpose:** A recorded recipe of GPU commands (copies, binds, dispatches). + +**How it works:** The CPU calls `vkBeginCommandBuffer`, then `vkCmd*` functions, then +`vkEndCommandBuffer`. Nothing runs on the GPU until `vkQueueSubmit`. After submit, the +CPU can continue; completion is tracked with a fence (or semaphores). Think of it as +building a batch job, then mailing it to the GPU. + +#### `VkFence` + +**Purpose:** A CPU-visible "this GPU submit is finished" signal. + +**How it works:** A fence starts unsignaled (unless created signaled). Passing it to +`vkQueueSubmit` asks the driver to signal it when that submit completes. The CPU calls +`vkWaitForFences` to block until signaled, then may safely read staging memory, +destroy temporary buffers, or mark a job joined. `vkResetFences` returns it to +unsignaled for reuse. Unlike semaphores (GPU-to-GPU), fences are the usual tool for +**CPU waiting on GPU** in cthreads. + +#### `VkSemaphore` + +**Purpose:** GPU-side synchronization between queue submits (and present in graphics). + +**How it works:** One submit can signal a semaphore; a later submit can wait on it at a +pipeline stage. The CPU does not typically `wait` on semaphores the way it waits on +fences. cthreads compute/copy path can ignore semaphores while everything uses one +queue and fence waits. + +#### `VkDescriptorSetLayout` + +**Purpose:** The schema for a descriptor set: which binding numbers exist and what +types they are (e.g. binding 0 = storage buffer). + +**How it works:** Created once for a shader interface. Pipeline layouts reference it. +It does not point at real buffers yet; it only describes the shape. + +#### `VkDescriptorPool` + +**Purpose:** A pool from which concrete descriptor sets are allocated. + +**How it works:** Sized by how many sets and how many descriptors of each type may be +allocated. Destroying the pool frees its sets (unless using free-individual flags). + +#### `VkDescriptorSet` + +**Purpose:** One instance of a layout filled with real resources (which `VkBuffer` is +binding 1, and so on). + +**How it works:** Allocate from a pool, then `vkUpdateDescriptorSets` to point bindings +at buffers. Before dispatch, `vkCmdBindDescriptorSets` attaches the set to the command +buffer so the shader's `layout(binding=N)` reads the right memory. + +#### `VkShaderModule` + +**Purpose:** Holds SPIR-V code for a shader stage. + +**How it works:** Created from SPIR-V bytes. Referenced when creating a pipeline. The +module itself is not "run"; the pipeline binds the compiled stage. Safe to destroy the +module after the pipeline is created (the pipeline keeps what it needs), depending on +driver rules; many apps keep modules until shutdown. + +#### `VkPipelineLayout` + +**Purpose:** Declares the interface a pipeline expects: descriptor set layouts and +optional push-constant ranges. + +**How it works:** Must match what the shader bindings and push constants use. +`vkCmdBindDescriptorSets` and `vkCmdPushConstants` take this layout. + +#### `VkPipeline` (compute) + +**Purpose:** A ready-to-bind compute program: shader stage + layout (+ state). + +**How it works:** `vkCreateComputePipelines` compiles/links the compute stage for the +device. Recording `vkCmdBindPipeline` then `vkCmdDispatch` runs it. Creating pipelines +is relatively expensive, so cthreads should cache them by shader hash. + +#### `VkPipelineCache` + +**Purpose:** Optional cache so pipeline creation can reuse previous driver compilations. + +**How it works:** Pass a cache into `vkCreateComputePipelines`. Can be saved to disk +across runs. Not required for correctness. + +--- + +## 2. Results and booleans + +### `VkResult` (enum) + +Most Vulkan functions return `VkResult`. + +| Value | Meaning for cthreads | +|-------|----------------------| +| `VK_SUCCESS` | Call succeeded | +| `VK_NOT_READY` | Not done yet (fences/queries) | +| `VK_TIMEOUT` | Wait timed out | +| `VK_EVENT_SET` / `VK_EVENT_RESET` | Event status (unused here) | +| `VK_INCOMPLETE` | Enumeration truncated (rare if count pattern is used correctly) | +| `VK_ERROR_OUT_OF_HOST_MEMORY` | CPU OOM | +| `VK_ERROR_OUT_OF_DEVICE_MEMORY` | GPU OOM | +| `VK_ERROR_INITIALIZATION_FAILED` | Init failed | +| `VK_ERROR_DEVICE_LOST` | Device lost (serious) | +| `VK_ERROR_MEMORY_MAP_FAILED` | `vkMapMemory` failed | +| `VK_ERROR_LAYER_NOT_PRESENT` | Requested layer missing | +| `VK_ERROR_EXTENSION_NOT_PRESENT` | Requested extension missing | +| `VK_ERROR_FEATURE_NOT_PRESENT` | Feature not available | +| `VK_ERROR_INCOMPATIBLE_DRIVER` | Driver too old / incompatible | + +**cthreads pattern:** treat anything other than `VK_SUCCESS` as failure and throw `cthreads.gpu.VulkanInitFailed: …` (or a more specific mapped error). + +--- + +## 3. Structure type tags (`sType`) + +### `VkStructureType` (enum, partial) + +Every create/info struct starts with: + +```text +sType: VkStructureType +pNext: const void* // usually nullptr +``` + +`sType` tells the driver which struct layout follows. Wrong `sType` is undefined behavior / validation errors. + +Values cthreads uses (or will use): + +| Enum | Struct | +|------|--------| +| `VK_STRUCTURE_TYPE_APPLICATION_INFO` | `VkApplicationInfo` | +| `VK_STRUCTURE_TYPE_INSTANCE_CREATE_INFO` | `VkInstanceCreateInfo` | +| `VK_STRUCTURE_TYPE_DEVICE_QUEUE_CREATE_INFO` | `VkDeviceQueueCreateInfo` | +| `VK_STRUCTURE_TYPE_DEVICE_CREATE_INFO` | `VkDeviceCreateInfo` | +| `VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO` | `VkBufferCreateInfo` | +| `VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO` | `VkMemoryAllocateInfo` | +| `VK_STRUCTURE_TYPE_COMMAND_POOL_CREATE_INFO` | `VkCommandPoolCreateInfo` | +| `VK_STRUCTURE_TYPE_COMMAND_BUFFER_ALLOCATE_INFO` | `VkCommandBufferAllocateInfo` | +| `VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO` | `VkCommandBufferBeginInfo` | +| `VK_STRUCTURE_TYPE_FENCE_CREATE_INFO` | `VkFenceCreateInfo` | +| `VK_STRUCTURE_TYPE_SUBMIT_INFO` | `VkSubmitInfo` | +| `VK_STRUCTURE_TYPE_SHADER_MODULE_CREATE_INFO` | `VkShaderModuleCreateInfo` | +| `VK_STRUCTURE_TYPE_PIPELINE_LAYOUT_CREATE_INFO` | `VkPipelineLayoutCreateInfo` | +| `VK_STRUCTURE_TYPE_COMPUTE_PIPELINE_CREATE_INFO` | `VkComputePipelineCreateInfo` | +| `VK_STRUCTURE_TYPE_PIPELINE_SHADER_STAGE_CREATE_INFO` | `VkPipelineShaderStageCreateInfo` | +| `VK_STRUCTURE_TYPE_DESCRIPTOR_SET_LAYOUT_CREATE_INFO` | `VkDescriptorSetLayoutCreateInfo` | +| `VK_STRUCTURE_TYPE_DESCRIPTOR_POOL_CREATE_INFO` | `VkDescriptorPoolCreateInfo` | +| `VK_STRUCTURE_TYPE_DESCRIPTOR_SET_ALLOCATE_INFO` | `VkDescriptorSetAllocateInfo` | +| `VK_STRUCTURE_TYPE_WRITE_DESCRIPTOR_SET` | `VkWriteDescriptorSet` | +| `VK_STRUCTURE_TYPE_MEMORY_BARRIER` / `BUFFER_MEMORY_BARRIER` / … | Barrier structs | + +**Rule:** always zero the struct (`{}`), then set `sType`, then set needed fields. + +--- + +## 4. Version helpers + +### `VK_MAKE_VERSION(major, minor, patch)` + +Packs a version into a `uint32_t` for `VkApplicationInfo`. + +### `VK_API_VERSION_1_0` / `VK_API_VERSION_1_1` / … + +API version requested in `VkApplicationInfo.apiVersion`. + +**cthreads:** requests `VK_API_VERSION_1_1` in Context init. + +--- + +## 5. Instance and device bootstrap + +**Purpose of this layer:** turn "Vulkan exists on this machine" into a usable +`VkDevice` + `VkQueue`. Without it, no buffers or submits are possible. + +Flow: create `VkInstance` -> enumerate `VkPhysicalDevice`s -> create `VkDevice` with +a compute queue family -> `vkGetDeviceQueue`. See handle purposes in section 1. + +### `VkApplicationInfo` + +Metadata about the application (mostly for tools/drivers). + +| Field | Type | Meaning | +|-------|------|---------| +| `sType` | `VkStructureType` | Must be `APPLICATION_INFO` | +| `pNext` | `const void*` | Usually `nullptr` | +| `pApplicationName` | `const char*` | App name string | +| `applicationVersion` | `uint32_t` | App version (`VK_MAKE_VERSION`) | +| `pEngineName` | `const char*` | Engine name (`"cthreads"`) | +| `engineVersion` | `uint32_t` | Engine version | +| `apiVersion` | `uint32_t` | Highest Vulkan API version the app uses | + +### `VkInstanceCreateInfo` + +Arguments to `vkCreateInstance`. + +| Field | Type | Meaning | +|-------|------|---------| +| `sType` | `VkStructureType` | `INSTANCE_CREATE_INFO` | +| `pNext` | `const void*` | Extensions via pNext (unused in basic cthreads) | +| `flags` | `VkInstanceCreateFlags` | Usually 0 | +| `pApplicationInfo` | `const VkApplicationInfo*` | Pointer to app info | +| `enabledLayerCount` | `uint32_t` | Validation layers count (0 unless debugging) | +| `ppEnabledLayerNames` | `const char* const*` | Layer name list | +| `enabledExtensionCount` | `uint32_t` | Instance extensions (0 for basic compute) | +| `ppEnabledExtensionNames` | `const char* const*` | Extension name list | + +### `vkCreateInstance` + +```text +VkResult vkCreateInstance( + const VkInstanceCreateInfo* pCreateInfo, // in: create parameters + const VkAllocationCallbacks* pAllocator, // in: nullptr in cthreads + VkInstance* pInstance // out: created instance handle +) +``` + +Creates the Vulkan instance. Resolve this entry point with a **null** instance via `vkGetInstanceProcAddr`. + +### `vkDestroyInstance` + +```text +void vkDestroyInstance( + VkInstance instance, // in: instance to destroy + const VkAllocationCallbacks* pAllocator // in: nullptr +) +``` + +Destroys the instance. Resolve with the **real** instance handle (not null). + +### `vkEnumeratePhysicalDevices` + +```text +VkResult vkEnumeratePhysicalDevices( + VkInstance instance, // in + uint32_t* pPhysicalDeviceCount, // in/out: count + VkPhysicalDevice* pPhysicalDevices // out: array, or nullptr to query count only +) +``` + +**Two-call idiom:** first call with `pPhysicalDevices == nullptr` to get count; allocate; second call to fill. + +### `VkDeviceQueueCreateInfo` + +Requests queues when creating a logical device. + +| Field | Type | Meaning | +|-------|------|---------| +| `sType` | `VkStructureType` | `DEVICE_QUEUE_CREATE_INFO` | +| `pNext` | `const void*` | Usually `nullptr` | +| `flags` | `VkDeviceQueueCreateFlags` | Usually 0 | +| `queueFamilyIndex` | `uint32_t` | Which family (cthreads compute family) | +| `queueCount` | `uint32_t` | How many queues (cthreads: 1) | +| `pQueuePriorities` | `const float*` | Array of priorities in `[0,1]` (cthreads: `{1.0f}`) | + +### `VkDeviceCreateInfo` + +| Field | Type | Meaning | +|-------|------|---------| +| `sType` | `VkStructureType` | `DEVICE_CREATE_INFO` | +| `pNext` | `const void*` | Features/extensions via pNext if needed | +| `flags` | `VkDeviceCreateFlags` | Usually 0 | +| `queueCreateInfoCount` | `uint32_t` | Number of queue infos | +| `pQueueCreateInfos` | `const VkDeviceQueueCreateInfo*` | Queue requests | +| `enabledLayerCount` | `uint32_t` | Deprecated for device; usually 0 | +| `ppEnabledLayerNames` | `const char* const*` | Usually unused | +| `enabledExtensionCount` | `uint32_t` | Device extensions (0 for basic path) | +| `ppEnabledExtensionNames` | `const char* const*` | Extension names | +| `pEnabledFeatures` | `const VkPhysicalDeviceFeatures*` | Optional features; can be `nullptr` | + +### `vkCreateDevice` + +```text +VkResult vkCreateDevice( + VkPhysicalDevice physicalDevice, // in: chosen GPU + const VkDeviceCreateInfo* pCreateInfo, // in + const VkAllocationCallbacks* pAllocator, // in: nullptr + VkDevice* pDevice // out +) +``` + +### `vkDestroyDevice` + +```text +void vkDestroyDevice( + VkDevice device, + const VkAllocationCallbacks* pAllocator +) +``` + +Destroys the logical device. All device-child objects must already be destroyed. + +### `vkGetDeviceQueue` + +```text +void vkGetDeviceQueue( + VkDevice device, // in + uint32_t queueFamilyIndex, // in + uint32_t queueIndex, // in: 0 for first queue of that family + VkQueue* pQueue // out +) +``` + +Does not create a queue; retrieves a handle owned by the device. + +--- + +## 6. Physical device queries + +**Purpose of this layer:** ask a `VkPhysicalDevice` what it can do before opening it. +cthreads uses these queries to (1) prefer a discrete GPU with a compute queue, and +(2) pick legal memory types for staging vs device-local buffers. + +### `VkPhysicalDeviceType` (enum) + +| Value | Meaning | +|-------|---------| +| `VK_PHYSICAL_DEVICE_TYPE_OTHER` | Other / unknown | +| `VK_PHYSICAL_DEVICE_TYPE_INTEGRATED_GPU` | iGPU (CPU die) | +| `VK_PHYSICAL_DEVICE_TYPE_DISCRETE_GPU` | Discrete card (preferred by cthreads scoring) | +| `VK_PHYSICAL_DEVICE_TYPE_VIRTUAL_GPU` | Virtualized | +| `VK_PHYSICAL_DEVICE_TYPE_CPU` | CPU fallback implementation | + +### `VkPhysicalDeviceProperties` + +Large struct. Fields cthreads cares about: + +| Field | Type | Meaning | +|-------|------|---------| +| `apiVersion` | `uint32_t` | Max API version supported | +| `driverVersion` | `uint32_t` | Vendor driver version | +| `vendorID` | `uint32_t` | PCI vendor id | +| `deviceID` | `uint32_t` | PCI device id | +| `deviceType` | `VkPhysicalDeviceType` | Discrete vs integrated, etc. | +| `deviceName` | `char[VK_MAX_PHYSICAL_DEVICE_NAME_SIZE]` | Human-readable GPU name | +| `pipelineCacheUUID` | `uint8_t[VK_UUID_SIZE]` | Cache identity | +| `limits` | `VkPhysicalDeviceLimits` | Many limits (max bindings, etc.) | + +### `vkGetPhysicalDeviceProperties` + +```text +void vkGetPhysicalDeviceProperties( + VkPhysicalDevice physicalDevice, // in + VkPhysicalDeviceProperties* pProperties // out +) +``` + +### `VkQueueFlagBits` / `VkQueueFlags` + +Bit flags describing what a queue family can do: + +| Flag | Meaning | +|------|---------| +| `VK_QUEUE_GRAPHICS_BIT` | Graphics commands | +| `VK_QUEUE_COMPUTE_BIT` | Compute dispatches (required by cthreads) | +| `VK_QUEUE_TRANSFER_BIT` | Dedicated transfer (often also implied by graphics/compute) | +| `VK_QUEUE_SPARSE_BINDING_BIT` | Sparse memory (unused) | + +### `VkQueueFamilyProperties` + +| Field | Type | Meaning | +|-------|------|---------| +| `queueFlags` | `VkQueueFlags` | Capability bits | +| `queueCount` | `uint32_t` | How many queues in this family | +| `timestampValidBits` | `uint32_t` | Timestamp support | +| `minImageTransferGranularity` | `VkExtent3D` | Image copy granularity | + +### `vkGetPhysicalDeviceQueueFamilyProperties` + +Two-call idiom (count then array), same as device enumeration. + +### `VkMemoryPropertyFlagBits` / `VkMemoryPropertyFlags` + +| Flag | Meaning | +|------|---------| +| `VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT` | GPU-fast memory (VRAM-like). cthreads device-local SSBOs | +| `VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT` | CPU can map. cthreads staging | +| `VK_MEMORY_PROPERTY_HOST_COHERENT_BIT` | No manual flush for CPU/GPU visibility. cthreads staging | +| `VK_MEMORY_PROPERTY_HOST_CACHED_BIT` | CPU cached (optional) | +| `VK_MEMORY_PROPERTY_LAZILY_ALLOCATED_BIT` | Transient attachments (graphics; unused) | + +### `VkMemoryType` + +| Field | Type | Meaning | +|-------|------|---------| +| `propertyFlags` | `VkMemoryPropertyFlags` | What this type can do | +| `heapIndex` | `uint32_t` | Which heap it comes from | + +### `VkMemoryHeap` + +| Field | Type | Meaning | +|-------|------|---------| +| `size` | `VkDeviceSize` | Heap size in bytes | +| `flags` | `VkMemoryHeapFlags` | e.g. device-local heap | + +### `VkPhysicalDeviceMemoryProperties` + +| Field | Type | Meaning | +|-------|------|---------| +| `memoryTypeCount` | `uint32_t` | Number of types | +| `memoryTypes` | `VkMemoryType[]` | Types array | +| `memoryHeapCount` | `uint32_t` | Number of heaps | +| `memoryHeaps` | `VkMemoryHeap[]` | Heaps array | + +### `vkGetPhysicalDeviceMemoryProperties` + +```text +void vkGetPhysicalDeviceMemoryProperties( + VkPhysicalDevice physicalDevice, + VkPhysicalDeviceMemoryProperties* pMemoryProperties +) +``` + +Used by `find_memory_type` together with `memoryTypeBits` from buffer requirements. + +--- + +## 7. Queues + +**Purpose:** a `VkQueue` is where work enters the GPU. The CPU records command +buffers, then `vkQueueSubmit` hands them to a queue. Execution is asynchronous +relative to the CPU. + +Queues are obtained with `vkGetDeviceQueue` (section 5). There is no +`vkDestroyQueue`; destroying the device invalidates queues. + +cthreads uses a single compute-capable queue for buffer copies and later for +compute dispatches. + +--- + +## 8. Buffers and device memory + +**Purpose of this layer:** give the GPU (and optionally the CPU) a place to store +bytes. A `VkBuffer` is the typed resource; `VkDeviceMemory` is the backing store. +Binding connects them. Staging memory is host-visible for `memcpy`; device-local +memory is what shaders should use for hot data. + +See also the handle write-ups for `VkBuffer` and `VkDeviceMemory` in section 1. + +### `VkBufferUsageFlagBits` / `VkBufferUsageFlags` + +| Flag | Meaning | +|------|---------| +| `VK_BUFFER_USAGE_TRANSFER_SRC_BIT` | May be source of `vkCmdCopyBuffer` | +| `VK_BUFFER_USAGE_TRANSFER_DST_BIT` | May be destination of a copy | +| `VK_BUFFER_USAGE_UNIFORM_BUFFER_BIT` | UBO (cthreads prefers SSBO for mutable pack) | +| `VK_BUFFER_USAGE_STORAGE_BUFFER_BIT` | SSBO for compute read/write | +| `VK_BUFFER_USAGE_INDEX_BUFFER_BIT` | Graphics index buffer (unused) | +| `VK_BUFFER_USAGE_VERTEX_BUFFER_BIT` | Graphics vertex buffer (unused) | +| `VK_BUFFER_USAGE_INDIRECT_BUFFER_BIT` | Indirect dispatch/draw (optional later) | + +**cthreads Staging:** `TRANSFER_SRC | TRANSFER_DST` +**cthreads DeviceLocal:** `STORAGE_BUFFER | TRANSFER_SRC | TRANSFER_DST` + +### `VkSharingMode` (enum) + +| Value | Meaning | +|-------|---------| +| `VK_SHARING_MODE_EXCLUSIVE` | One queue family owns it (cthreads default) | +| `VK_SHARING_MODE_CONCURRENT` | Multiple families; must list indices | + +### `VkBufferCreateInfo` + +| Field | Type | Meaning | +|-------|------|---------| +| `sType` | `VkStructureType` | `BUFFER_CREATE_INFO` | +| `pNext` | `const void*` | Usually `nullptr` | +| `flags` | `VkBufferCreateFlags` | Sparse flags etc.; usually 0 | +| `size` | `VkDeviceSize` | Size in bytes (> 0 in cthreads) | +| `usage` | `VkBufferUsageFlags` | Allowed uses | +| `sharingMode` | `VkSharingMode` | Exclusive vs concurrent | +| `queueFamilyIndexCount` | `uint32_t` | Needed if concurrent | +| `pQueueFamilyIndices` | `const uint32_t*` | Families if concurrent | + +### `vkCreateBuffer` + +```text +VkResult vkCreateBuffer( + VkDevice device, + const VkBufferCreateInfo* pCreateInfo, + const VkAllocationCallbacks* pAllocator, + VkBuffer* pBuffer +) +``` + +Creates the buffer object only. Memory is separate. + +### `vkDestroyBuffer` + +```text +void vkDestroyBuffer( + VkDevice device, + VkBuffer buffer, + const VkAllocationCallbacks* pAllocator +) +``` + +### `VkMemoryRequirements` + +| Field | Type | Meaning | +|-------|------|---------| +| `size` | `VkDeviceSize` | Bytes to allocate (may exceed create size due to alignment) | +| `alignment` | `VkDeviceSize` | Required alignment of the bind offset | +| `memoryTypeBits` | `uint32_t` | Bit i set => memory type i is legal | + +### `vkGetBufferMemoryRequirements` + +```text +void vkGetBufferMemoryRequirements( + VkDevice device, + VkBuffer buffer, + VkMemoryRequirements* pMemoryRequirements +) +``` + +### `VkMemoryAllocateInfo` + +| Field | Type | Meaning | +|-------|------|---------| +| `sType` | `VkStructureType` | `MEMORY_ALLOCATE_INFO` | +| `pNext` | `const void*` | Usually `nullptr` | +| `allocationSize` | `VkDeviceSize` | Usually `mem_reqs.size` | +| `memoryTypeIndex` | `uint32_t` | From `find_memory_type` | + +### `vkAllocateMemory` + +```text +VkResult vkAllocateMemory( + VkDevice device, + const VkMemoryAllocateInfo* pAllocateInfo, + const VkAllocationCallbacks* pAllocator, + VkDeviceMemory* pMemory +) +``` + +### `vkFreeMemory` + +```text +void vkFreeMemory( + VkDevice device, + VkDeviceMemory memory, + const VkAllocationCallbacks* pAllocator +) +``` + +### `vkBindBufferMemory` + +```text +VkResult vkBindBufferMemory( + VkDevice device, + VkBuffer buffer, + VkDeviceMemory memory, + VkDeviceSize memoryOffset // must respect alignment; cthreads uses 0 with dedicated alloc +) +``` + +Associates memory with the buffer. A buffer is bound at most once (without sparse extensions). + +### `VkMemoryMapFlags` + +Usually `0` for `vkMapMemory`. + +### `vkMapMemory` + +```text +VkResult vkMapMemory( + VkDevice device, + VkDeviceMemory memory, + VkDeviceSize offset, // start offset in the allocation + VkDeviceSize size, // bytes to map, or VK_WHOLE_SIZE + VkMemoryMapFlags flags, // usually 0 + void** ppData // out: CPU pointer +) +``` + +Only valid for **host-visible** memory. cthreads maps staging buffers. + +### `vkUnmapMemory` + +```text +void vkUnmapMemory( + VkDevice device, + VkDeviceMemory memory +) +``` + +--- + +## 9. Command pools, command buffers, copies + +**Purpose of this layer:** build and own GPU "to-do lists." A command buffer is the +list; a command pool allocates those lists for a specific queue family; +`vkCmdCopyBuffer` is one kind of item on the list. Recording is CPU-side; running +happens only after submit. + +See handle write-ups for `VkCommandPool` and `VkCommandBuffer` in section 1. + +### `VkCommandPoolCreateFlagBits` + +| Flag | Meaning | +|------|---------| +| `VK_COMMAND_POOL_CREATE_TRANSIENT_BIT` | Short-lived CBs (cthreads one-shot copies) | +| `VK_COMMAND_POOL_CREATE_RESET_COMMAND_BUFFER_BIT` | Allow resetting individual CBs | + +### `VkCommandPoolCreateInfo` + +| Field | Type | Meaning | +|-------|------|---------| +| `sType` | `VkStructureType` | `COMMAND_POOL_CREATE_INFO` | +| `pNext` | `const void*` | Usually `nullptr` | +| `flags` | `VkCommandPoolCreateFlags` | Transient / reset bits | +| `queueFamilyIndex` | `uint32_t` | Must match the queue that will submit | + +### `vkCreateCommandPool` / `vkDestroyCommandPool` + +Standard create/destroy pair on a `VkDevice`. + +### `VkCommandBufferLevel` (enum) + +| Value | Meaning | +|-------|---------| +| `VK_COMMAND_BUFFER_LEVEL_PRIMARY` | Can be submitted to a queue (cthreads) | +| `VK_COMMAND_BUFFER_LEVEL_SECONDARY` | Can be called from primary (advanced) | + +### `VkCommandBufferAllocateInfo` + +| Field | Type | Meaning | +|-------|------|---------| +| `sType` | `VkStructureType` | `COMMAND_BUFFER_ALLOCATE_INFO` | +| `pNext` | `const void*` | Usually `nullptr` | +| `commandPool` | `VkCommandPool` | Pool to allocate from | +| `level` | `VkCommandBufferLevel` | Primary in cthreads | +| `commandBufferCount` | `uint32_t` | How many to allocate | + +### `vkAllocateCommandBuffers` / `vkFreeCommandBuffers` + +```text +VkResult vkAllocateCommandBuffers( + VkDevice device, + const VkCommandBufferAllocateInfo* pAllocateInfo, + VkCommandBuffer* pCommandBuffers // out array +) + +void vkFreeCommandBuffers( + VkDevice device, + VkCommandPool commandPool, + uint32_t commandBufferCount, + const VkCommandBuffer* pCommandBuffers +) +``` + +### `VkCommandBufferUsageFlagBits` + +| Flag | Meaning | +|------|---------| +| `VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT` | Record, submit once (copies) | +| `VK_COMMAND_BUFFER_USAGE_RENDER_PASS_CONTINUE_BIT` | Graphics secondary (unused) | +| `VK_COMMAND_BUFFER_USAGE_SIMULTANEOUS_USE_BIT` | Allow concurrent resubmit (careful) | + +### `VkCommandBufferBeginInfo` + +| Field | Type | Meaning | +|-------|------|---------| +| `sType` | `VkStructureType` | `COMMAND_BUFFER_BEGIN_INFO` | +| `pNext` | `const void*` | Usually `nullptr` | +| `flags` | `VkCommandBufferUsageFlags` | e.g. one-time | +| `pInheritanceInfo` | `const VkCommandBufferInheritanceInfo*` | For secondary; nullptr for primary | + +### `vkBeginCommandBuffer` / `vkEndCommandBuffer` + +```text +VkResult vkBeginCommandBuffer( + VkCommandBuffer commandBuffer, + const VkCommandBufferBeginInfo* pBeginInfo +) + +VkResult vkEndCommandBuffer( + VkCommandBuffer commandBuffer +) +``` + +Recording happens between begin and end. `vkCmd*` calls are only valid while recording. + +### `vkResetCommandBuffer` + +```text +VkResult vkResetCommandBuffer( + VkCommandBuffer commandBuffer, + VkCommandBufferResetFlags flags // usually 0 +) +``` + +Clears recorded commands so the buffer can be recorded again (pool must allow reset, or reset the whole pool). + +### `VkBufferCopy` + +| Field | Type | Meaning | +|-------|------|---------| +| `srcOffset` | `VkDeviceSize` | Byte offset in source | +| `dstOffset` | `VkDeviceSize` | Byte offset in destination | +| `size` | `VkDeviceSize` | Bytes to copy | + +### `vkCmdCopyBuffer` + +```text +void vkCmdCopyBuffer( + VkCommandBuffer commandBuffer, // recording CB + VkBuffer srcBuffer, + VkBuffer dstBuffer, + uint32_t regionCount, + const VkBufferCopy* pRegions +) +``` + +Records a GPU-side copy. Does not run until the CB is submitted and the GPU executes it. + +Both buffers need appropriate `TRANSFER_SRC` / `TRANSFER_DST` usage bits. + +--- + +## 10. Fences and submit + +**Purpose of this layer:** (1) send recorded work to the GPU (`vkQueueSubmit`), and +(2) let the CPU know when that work is done (`VkFence`). + +### How fences work (detail) + +The GPU and CPU run on different timelines. After `vkQueueSubmit`, the CPU might +immediately try to `memcpy` from a staging buffer that a download copy has not +finished writing. That is a race. + +A **fence** closes that gap: + +1. Create a fence (usually **unsignaled**). +2. Pass it as the last argument to `vkQueueSubmit`. +3. When the GPU finishes that submit, the driver **signals** the fence. +4. `vkWaitForFences` blocks the CPU until the fence is signaled (or times out). +5. After the wait returns successfully, it is safe to read staging memory, free + temporary buffers used by that submit, or complete a Python `join()`. +6. `vkResetFences` sets it back to unsignaled before the next submit if reusing it. + +Mental model: the fence is a doorbell the GPU rings when a batch of work is done; +the CPU sleeps on `vkWaitForFences` until it hears the ring. + +**Fence vs semaphore:** fences are for **CPU waits**. Semaphores are for **GPU waits** +between submits. cthreads upload/download and `GpuJob.join` are CPU-wait problems, +so fences are the right tool. + +### `VkFenceCreateFlagBits` + +| Flag | Meaning | +|------|---------| +| `0` | Created unsignaled (typical) | +| `VK_FENCE_CREATE_SIGNALED_BIT` | Created already signaled | + +### `VkFenceCreateInfo` + +| Field | Type | Meaning | +|-------|------|---------| +| `sType` | `VkStructureType` | `FENCE_CREATE_INFO` | +| `pNext` | `const void*` | Usually `nullptr` | +| `flags` | `VkFenceCreateFlags` | Usually 0 | + +### `vkCreateFence` / `vkDestroyFence` + +Standard create/destroy on device. + +### `vkResetFences` + +```text +VkResult vkResetFences( + VkDevice device, + uint32_t fenceCount, + const VkFence* pFences +) +``` + +Returns fences to unsignaled so they can be reused on the next submit. + +### `vkWaitForFences` + +```text +VkResult vkWaitForFences( + VkDevice device, + uint32_t fenceCount, + const VkFence* pFences, + VkBool32 waitAll, // VK_TRUE: wait until all signal + uint64_t timeout // nanoseconds; UINT64_MAX = forever +) +``` + +CPU blocks until the fence(s) signal (or timeout). + +### `VkSubmitInfo` + +| Field | Type | Meaning | +|-------|------|---------| +| `sType` | `VkStructureType` | `SUBMIT_INFO` | +| `pNext` | `const void*` | Usually `nullptr` | +| `waitSemaphoreCount` | `uint32_t` | GPU waits (0 in basic cthreads copies) | +| `pWaitSemaphores` | `const VkSemaphore*` | Semaphores to wait on | +| `pWaitDstStageMask` | `const VkPipelineStageFlags*` | Stages to wait at | +| `commandBufferCount` | `uint32_t` | How many CBs | +| `pCommandBuffers` | `const VkCommandBuffer*` | CBs to execute | +| `signalSemaphoreCount` | `uint32_t` | Semaphores to signal (often 0) | +| `pSignalSemaphores` | `const VkSemaphore*` | Signal list | + +### `vkQueueSubmit` + +```text +VkResult vkQueueSubmit( + VkQueue queue, + uint32_t submitCount, + const VkSubmitInfo* pSubmits, + VkFence fence // optional; signals when this submit finishes +) +``` + +Returns when the submit is **queued**, not when the GPU finished. Use the fence to wait on the CPU. + +--- + +## 11. Launch path: descriptors + +**Purpose of this layer:** connect shader binding numbers to real `VkBuffer`s. +Without descriptors, a compute shader has no way to know which device memory is +`binding = 1`. Layouts describe the shape; sets hold the actual pointers/handles; +updates and binds install them for a dispatch. + +See handle write-ups for descriptor layout/pool/set in section 1. + +These are required once compute shaders bind GpuPack buffers. + +### `VkDescriptorType` (enum, partial) + +| Value | Meaning | +|-------|---------| +| `VK_DESCRIPTOR_TYPE_UNIFORM_BUFFER` | UBO | +| `VK_DESCRIPTOR_TYPE_STORAGE_BUFFER` | SSBO (cthreads scalars + lists) | +| `VK_DESCRIPTOR_TYPE_UNIFORM_BUFFER_DYNAMIC` | Dynamic-offset UBO | +| `VK_DESCRIPTOR_TYPE_STORAGE_BUFFER_DYNAMIC` | Dynamic-offset SSBO | + +### `VkDescriptorSetLayoutBinding` + +| Field | Type | Meaning | +|-------|------|---------| +| `binding` | `uint32_t` | Binding number (0 = scalars, 1..N = lists) | +| `descriptorType` | `VkDescriptorType` | `STORAGE_BUFFER` for cthreads | +| `descriptorCount` | `uint32_t` | Usually 1 | +| `stageFlags` | `VkShaderStageFlags` | `VK_SHADER_STAGE_COMPUTE_BIT` | +| `pImmutableSamplers` | `const VkSampler*` | nullptr for buffers | + +### `VkShaderStageFlagBits` + +| Flag | Meaning | +|------|---------| +| `VK_SHADER_STAGE_COMPUTE_BIT` | Compute shader | +| `VK_SHADER_STAGE_VERTEX_BIT` | Graphics (unused) | +| `VK_SHADER_STAGE_ALL` | All stages | + +### `VkDescriptorSetLayoutCreateInfo` + +Holds an array of `VkDescriptorSetLayoutBinding`. + +### `vkCreateDescriptorSetLayout` / `vkDestroyDescriptorSetLayout` + +Create/destroy the layout object on a device. + +### `VkDescriptorPoolSize` + +| Field | Type | Meaning | +|-------|------|---------| +| `type` | `VkDescriptorType` | e.g. storage buffer | +| `descriptorCount` | `uint32_t` | How many of that type the pool can allocate | + +### `VkDescriptorPoolCreateInfo` + +| Field | Type | Meaning | +|-------|------|---------| +| `flags` | `VkDescriptorPoolCreateFlags` | e.g. free-set bit if freeing individual sets | +| `maxSets` | `uint32_t` | Max sets allocatable | +| `poolSizeCount` | `uint32_t` | Length of pool sizes | +| `pPoolSizes` | `const VkDescriptorPoolSize*` | Capacities per type | + +### `vkCreateDescriptorPool` / `vkDestroyDescriptorPool` + +### `VkDescriptorSetAllocateInfo` + +| Field | Type | Meaning | +|-------|------|---------| +| `descriptorPool` | `VkDescriptorPool` | Pool | +| `descriptorSetCount` | `uint32_t` | How many sets | +| `pSetLayouts` | `const VkDescriptorSetLayout*` | Layout per set | + +### `vkAllocateDescriptorSets` / `vkFreeDescriptorSets` + +### `VkDescriptorBufferInfo` + +| Field | Type | Meaning | +|-------|------|---------| +| `buffer` | `VkBuffer` | Buffer to bind | +| `offset` | `VkDeviceSize` | Start offset | +| `range` | `VkDeviceSize` | Bytes, or `VK_WHOLE_SIZE` | + +### `VkWriteDescriptorSet` + +| Field | Type | Meaning | +|-------|------|---------| +| `dstSet` | `VkDescriptorSet` | Set to update | +| `dstBinding` | `uint32_t` | Binding index | +| `dstArrayElement` | `uint32_t` | Usually 0 | +| `descriptorCount` | `uint32_t` | Usually 1 | +| `descriptorType` | `VkDescriptorType` | Must match layout | +| `pImageInfo` | `const VkDescriptorImageInfo*` | For images; nullptr for buffers | +| `pBufferInfo` | `const VkDescriptorBufferInfo*` | Buffer info | +| `pTexelBufferView` | `const VkBufferView*` | Texel buffers; nullptr | + +### `vkUpdateDescriptorSets` + +```text +void vkUpdateDescriptorSets( + VkDevice device, + uint32_t descriptorWriteCount, + const VkWriteDescriptorSet* pDescriptorWrites, + uint32_t descriptorCopyCount, + const VkCopyDescriptorSet* pDescriptorCopies +) +``` + +CPU-side update; no command buffer needed. + +### `vkCmdBindDescriptorSets` + +```text +void vkCmdBindDescriptorSets( + VkCommandBuffer commandBuffer, + VkPipelineBindPoint pipelineBindPoint, // COMPUTE + VkPipelineLayout layout, + uint32_t firstSet, + uint32_t descriptorSetCount, + const VkDescriptorSet* pDescriptorSets, + uint32_t dynamicOffsetCount, + const uint32_t* pDynamicOffsets +) +``` + +--- + +## 12. Launch path: shaders, pipelines, dispatch + +**Purpose of this layer:** turn SPIR-V into something the GPU can run, then launch it. + +- **Shader module:** raw SPIR-V wrapped as a Vulkan object. +- **Pipeline layout:** declares descriptor/push-constant interface. +- **Compute pipeline:** prepared program ready to bind. +- **Dispatch:** "run N workgroups of this pipeline." + +Recording order in a command buffer is typically: bind pipeline -> bind descriptor +sets -> (optional push constants) -> `vkCmdDispatch`. Then submit + fence wait as in +section 10. + +### `VkShaderModuleCreateInfo` + +| Field | Type | Meaning | +|-------|------|---------| +| `flags` | `VkShaderModuleCreateFlags` | Usually 0 | +| `codeSize` | `size_t` | SPIR-V size in **bytes** | +| `pCode` | `const uint32_t*` | SPIR-V words | + +### `vkCreateShaderModule` / `vkDestroyShaderModule` + +### `VkPipelineBindPoint` (enum) + +| Value | Meaning | +|-------|---------| +| `VK_PIPELINE_BIND_POINT_COMPUTE` | Compute pipeline | +| `VK_PIPELINE_BIND_POINT_GRAPHICS` | Graphics (unused) | + +### `VkPipelineShaderStageCreateInfo` + +| Field | Type | Meaning | +|-------|------|---------| +| `stage` | `VkShaderStageFlagBits` | `COMPUTE_BIT` | +| `module` | `VkShaderModule` | Shader module | +| `pName` | `const char*` | Entry point, usually `"main"` | +| `pSpecializationInfo` | `const VkSpecializationInfo*` | Optional constants | + +### `VkPipelineLayoutCreateInfo` + +| Field | Type | Meaning | +|-------|------|---------| +| `setLayoutCount` | `uint32_t` | Number of descriptor set layouts | +| `pSetLayouts` | `const VkDescriptorSetLayout*` | Layouts | +| `pushConstantRangeCount` | `uint32_t` | 0 if unused | +| `pPushConstantRanges` | `const VkPushConstantRange*` | Optional | + +### `vkCreatePipelineLayout` / `vkDestroyPipelineLayout` + +### `VkComputePipelineCreateInfo` + +| Field | Type | Meaning | +|-------|------|---------| +| `flags` | `VkPipelineCreateFlags` | Usually 0 | +| `stage` | `VkPipelineShaderStageCreateInfo` | Compute stage | +| `layout` | `VkPipelineLayout` | Pipeline layout | +| `basePipelineHandle` | `VkPipeline` | For derivatives; often null | +| `basePipelineIndex` | `int32_t` | Or -1 | + +### `vkCreateComputePipelines` + +```text +VkResult vkCreateComputePipelines( + VkDevice device, + VkPipelineCache pipelineCache, // optional VK_NULL_HANDLE + uint32_t createInfoCount, + const VkComputePipelineCreateInfo* pCreateInfos, + const VkAllocationCallbacks* pAllocator, + VkPipeline* pPipelines +) +``` + +### `vkDestroyPipeline` + +### `vkCmdBindPipeline` + +```text +void vkCmdBindPipeline( + VkCommandBuffer commandBuffer, + VkPipelineBindPoint pipelineBindPoint, + VkPipeline pipeline +) +``` + +### `vkCmdDispatch` + +```text +void vkCmdDispatch( + VkCommandBuffer commandBuffer, + uint32_t groupCountX, + uint32_t groupCountY, + uint32_t groupCountZ +) +``` + +Launches `groupCountX * groupCountY * groupCountZ` workgroups. +Each workgroup size comes from the shader `layout(local_size_x = …)`. + +Rough element coverage for 1D: + +```text +groupsX = ceil(elementCount / local_size_x) +``` + +### `vkCmdPushConstants` (optional) + +```text +void vkCmdPushConstants( + VkCommandBuffer commandBuffer, + VkPipelineLayout layout, + VkShaderStageFlags stageFlags, + uint32_t offset, + uint32_t size, + const void* pValues +) +``` + +Small fast uniforms. In cthreads, scalar SSBO remains writeback source of truth if both are used. + +--- + +## 13. Launch path: barriers (overview) + +**Purpose:** force ordering and memory visibility between GPU commands. The GPU may +overlap or reorder work. A barrier says: "finish these earlier writes, then allow +these later reads/writes." + +**When fences are enough:** if the CPU submits a copy-only command buffer and +`vkWaitForFences` before touching staging memory, the CPU wait provides the needed +ordering for that CPU read. No barrier required for that pattern. + +**When barriers are required:** if the **same** command buffer (or back-to-back GPU +work without a CPU wait) does `copy into buffer Y` then `dispatch that reads Y`, +insert a pipeline barrier between them so the shader cannot read stale data. + +### `VkPipelineStageFlagBits` (partial) + +| Flag | Meaning | +|------|---------| +| `VK_PIPELINE_STAGE_TRANSFER_BIT` | Copy engine | +| `VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT` | Compute shader | +| `VK_PIPELINE_STAGE_HOST_BIT` | Host access | +| `VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT` | Earliest | +| `VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT` | Latest | + +### `VkAccessFlagBits` (partial) + +| Flag | Meaning | +|------|---------| +| `VK_ACCESS_TRANSFER_WRITE_BIT` | Copy write | +| `VK_ACCESS_TRANSFER_READ_BIT` | Copy read | +| `VK_ACCESS_SHADER_READ_BIT` | Shader read | +| `VK_ACCESS_SHADER_WRITE_BIT` | Shader write | +| `VK_ACCESS_HOST_READ_BIT` | CPU read | +| `VK_ACCESS_HOST_WRITE_BIT` | CPU write | + +### `vkCmdPipelineBarrier` + +Records a barrier between earlier and later commands in the same CB. Exact struct +fields (`VkMemoryBarrier`, `VkBufferMemoryBarrier`) are filled when implementing +the launch path; until then, CPU fence waits after submit are enough for +upload/download helpers that do not dispatch in the same CB. + +--- + +## 14. Function pointer typedefs (`PFN_*`) + +Vulkan headers define: + +```text +typedef (VKAPI_PTR *PFN_vkXxx)(); +``` + +Examples: + +| Typedef | Points to | +|---------|-----------| +| `PFN_vkGetInstanceProcAddr` | `vkGetInstanceProcAddr` | +| `PFN_vkCreateInstance` | `vkCreateInstance` | +| `PFN_vkCreateBuffer` | `vkCreateBuffer` | +| `PFN_vkCmdCopyBuffer` | `vkCmdCopyBuffer` | +| … | every entry point | + +cthreads stores these on `Context` and loads them through `vkGetInstanceProcAddr` +(see [05-dynamic-loading.md](./05-dynamic-loading.md)). + +### `vkGetInstanceProcAddr` + +```text +PFN_vkVoidFunction vkGetInstanceProcAddr( + VkInstance instance, // NULL for a few globals; real instance otherwise + const char* pName // e.g. "vkCreateBuffer" +) +``` + +Returns a generic function pointer or `nullptr` if unavailable. + +--- + +## 15. Quick index by name + +### Handles / scalars + +`VkBool32`, `VkDeviceSize`, `VK_NULL_HANDLE`, `VK_WHOLE_SIZE`, +`VkInstance`, `VkPhysicalDevice`, `VkDevice`, `VkQueue`, +`VkBuffer`, `VkDeviceMemory`, `VkCommandPool`, `VkCommandBuffer`, `VkFence`, +`VkDescriptorSetLayout`, `VkDescriptorPool`, `VkDescriptorSet`, +`VkShaderModule`, `VkPipelineLayout`, `VkPipeline`, `VkPipelineCache` + +### Enums / flags (high traffic) + +`VkResult`, `VkStructureType`, `VkPhysicalDeviceType`, +`VkQueueFlagBits`, `VkMemoryPropertyFlagBits`, `VkBufferUsageFlagBits`, +`VkSharingMode`, `VkCommandBufferLevel`, `VkCommandPoolCreateFlagBits`, +`VkCommandBufferUsageFlagBits`, `VkDescriptorType`, `VkShaderStageFlagBits`, +`VkPipelineBindPoint`, `VkPipelineStageFlagBits`, `VkAccessFlagBits` + +### Structs (high traffic) + +`VkApplicationInfo`, `VkInstanceCreateInfo`, +`VkDeviceQueueCreateInfo`, `VkDeviceCreateInfo`, +`VkPhysicalDeviceProperties`, `VkQueueFamilyProperties`, +`VkPhysicalDeviceMemoryProperties`, `VkMemoryType`, `VkMemoryHeap`, +`VkBufferCreateInfo`, `VkMemoryRequirements`, `VkMemoryAllocateInfo`, +`VkCommandPoolCreateInfo`, `VkCommandBufferAllocateInfo`, `VkCommandBufferBeginInfo`, +`VkBufferCopy`, `VkFenceCreateInfo`, `VkSubmitInfo`, +`VkDescriptorSetLayoutBinding`, `VkDescriptorBufferInfo`, `VkWriteDescriptorSet`, +`VkShaderModuleCreateInfo`, `VkPipelineShaderStageCreateInfo`, +`VkPipelineLayoutCreateInfo`, `VkComputePipelineCreateInfo` + +### Functions already used in cthreads GPU code + +`vkGetInstanceProcAddr`, +`vkCreateInstance`, `vkDestroyInstance`, +`vkEnumeratePhysicalDevices`, +`vkGetPhysicalDeviceProperties`, `vkGetPhysicalDeviceQueueFamilyProperties`, +`vkGetPhysicalDeviceMemoryProperties`, +`vkCreateDevice`, `vkDestroyDevice`, `vkGetDeviceQueue`, +`vkCreateBuffer`, `vkDestroyBuffer`, `vkGetBufferMemoryRequirements`, +`vkAllocateMemory`, `vkFreeMemory`, `vkBindBufferMemory`, +`vkMapMemory`, `vkUnmapMemory`, +`vkCreateCommandPool`, `vkDestroyCommandPool`, +`vkAllocateCommandBuffers`, `vkFreeCommandBuffers`, `vkResetCommandBuffer`, +`vkBeginCommandBuffer`, `vkEndCommandBuffer`, `vkCmdCopyBuffer`, +`vkCreateFence`, `vkDestroyFence`, `vkQueueSubmit`, `vkWaitForFences`, `vkResetFences` + +### Functions needed for the launch path (not all may be loaded yet) + +`vkCreateDescriptorSetLayout`, `vkDestroyDescriptorSetLayout`, +`vkCreateDescriptorPool`, `vkDestroyDescriptorPool`, +`vkAllocateDescriptorSets`, `vkFreeDescriptorSets`, `vkUpdateDescriptorSets`, +`vkCreateShaderModule`, `vkDestroyShaderModule`, +`vkCreatePipelineLayout`, `vkDestroyPipelineLayout`, +`vkCreateComputePipelines`, `vkDestroyPipeline`, +`vkCmdBindPipeline`, `vkCmdBindDescriptorSets`, `vkCmdDispatch`, +`vkCmdPipelineBarrier`, `vkCmdPushConstants` (optional) + +When adding any of these in C++, also add the matching `PFN_*` field on `Context`, +resolve it in init, and clear it in shutdown. + +--- + +## How to use this file + +1. When reading `context.cpp` / `memory.cpp`, look up each `Vk*` / `vk*` here. +2. When implementing the launch path, start from sections 11-13 and extend Context PFNs. +3. For intuition (why fences exist, why staging exists), return to chapters 02-08. + +This reference intentionally over-explains. Prefer it over skimming random internet snippets that mix graphics-only APIs into compute work. diff --git a/docs/vk_guide/README.md b/docs/vk_guide/README.md new file mode 100644 index 0000000..75b72c8 --- /dev/null +++ b/docs/vk_guide/README.md @@ -0,0 +1,51 @@ +# Vulkan guide for cthreads (compute path) + +This folder is a **project-specific** Vulkan tutorial for **cthreads contributors**. +It covers the compute path used by the GPU backend: buffers, copies, descriptors, +shaders, and launch/wait — **not** the full graphics stack (swapchains, render +passes, images, and so on). + +Architecture choices here match the cthreads GPU design (same Python types as the +CPU backend, GpuPack option 5, device-local + staging). Implementation status in +the tree may move faster or slower than any particular roadmap document; treat +this guide as the conceptual reference, and the source under `src/cthreads/cpp/gpu/` +as ground truth for what is already landed. + +## Audience + +Contributors who know C++ and the cthreads CPU model (`@Thread`, pack, marshal, +writeback), and who may know little or no Vulkan. That is a normal starting point. + +## How to read + +Read in order the first time. Later use the glossary and the "map to the codebase" +chapter as a reference. + +| # | File | Topics | +|---|------|--------| +| 00 | [00-read-me-first.md](./00-read-me-first.md) | Goals, out of scope, Vulkan vs CUDA/OpenGL | +| 01 | [01-cthreads-gpu-big-picture.md](./01-cthreads-gpu-big-picture.md) | How GPU fits next to CPU pack/marshal | +| 02 | [02-mental-model.md](./02-mental-model.md) | Objects, handles, explicit control | +| 03 | [03-sdk-runtime-drivers.md](./03-sdk-runtime-drivers.md) | SDK vs drivers vs `vulkan-1.dll` | +| 04 | [04-instance-device-queue.md](./04-instance-device-queue.md) | Connecting to a GPU (`Context`) | +| 05 | [05-dynamic-loading.md](./05-dynamic-loading.md) | `LoadLibrary` / `PFN_*` resolution | +| 06 | [06-buffers-and-memory.md](./06-buffers-and-memory.md) | `VkBuffer`, memory types, bind | +| 07 | [07-staging-upload-download.md](./07-staging-upload-download.md) | Device-local + staging copies | +| 08 | [08-commands-fences.md](./08-commands-fences.md) | Command buffers, submit, fences | +| 09 | [09-descriptors-ssbo.md](./09-descriptors-ssbo.md) | How shaders see buffers | +| 10 | [10-std430-layouts.md](./10-std430-layouts.md) | Scalar struct packing | +| 11 | [11-spirv-pipelines-dispatch.md](./11-spirv-pipelines-dispatch.md) | Shaders, pipelines, `dispatch` | +| 12 | [12-gpupack-marshal.md](./12-gpupack-marshal.md) | Option 5 pack end-to-end | +| 13 | [13-map-to-our-code.md](./13-map-to-our-code.md) | Files in `src/cthreads/cpp/gpu/` | +| 14 | [14-glossary.md](./14-glossary.md) | Terms in one place | +| 15 | [15-checklist.md](./15-checklist.md) | Concepts a contributor should be able to explain | +| 16 | [16-api-reference.md](./16-api-reference.md) | Vulkan types, structs, enums, and functions used by cthreads | + +## Official docs (optional later) + +- [Vulkan Guide](https://vkguide.dev/) — general tutorial (more graphics-heavy) +- [Vulkan Tutorial](https://vulkan-tutorial.com/) — classic; skip swapchain chapters for this project +- [Khronos Vulkan Spec](https://registry.khronos.org/vulkan/) — reference, not a textbook + +This guide is intentionally longer and more hand-holding than those, and tied to +**cthreads** architecture decisions rather than a generic hello-triangle path. diff --git a/src/cthreads/cpp/CMakeLists.txt b/src/cthreads/cpp/CMakeLists.txt index ea20f6c..186014b 100644 --- a/src/cthreads/cpp/CMakeLists.txt +++ b/src/cthreads/cpp/CMakeLists.txt @@ -147,11 +147,16 @@ if(CTHREADS_GPU) message(STATUS "cthreads GPU: ON (Vulkan)") target_sources(_ext PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}/gpu/impl/context.cpp" + "${CMAKE_CURRENT_SOURCE_DIR}/gpu/impl/memory.cpp" + "${CMAKE_CURRENT_SOURCE_DIR}/gpu/impl/pack.cpp" + "${CMAKE_CURRENT_SOURCE_DIR}/gpu/testing/pack_roundtrip.cpp" "${CMAKE_CURRENT_SOURCE_DIR}/bindings/gpu_module.cpp" - # extend more source when implemented (current stage: GPU-01) + "${CMAKE_CURRENT_SOURCE_DIR}/bindings/gpu_testing_module.cpp" + # GPU-02: memory + GpuPack marshal substrate (+ test-only roundtrip bindings) ) target_include_directories(_ext PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/gpu/headers + ${CMAKE_CURRENT_SOURCE_DIR}/gpu ${Vulkan_INCLUDE_DIRS} ) target_compile_definitions(_ext PRIVATE CTHREADS_WITH_GPU=1) diff --git a/src/cthreads/cpp/bindings/gpu_module.cpp b/src/cthreads/cpp/bindings/gpu_module.cpp index 49f897e..c63e42e 100644 --- a/src/cthreads/cpp/bindings/gpu_module.cpp +++ b/src/cthreads/cpp/bindings/gpu_module.cpp @@ -3,6 +3,7 @@ // LICENSE file in the root directory of this source tree. #include "gpu_module.hpp" +#include "gpu_testing_module.hpp" #include "../gpu/headers/context.hpp" @@ -39,4 +40,8 @@ void bind_gpu(py::module_& parent) { &cthreads::gpu::shutdown, "Destroy device/instance and unload the Vulkan loader." ); + + // Test-only pack round-trips live in a separate submodule / translation unit + // so product bindings stay small. Not re-exported by cthreads.gpu. + bind_gpu_testing(g); } diff --git a/src/cthreads/cpp/bindings/gpu_module.hpp b/src/cthreads/cpp/bindings/gpu_module.hpp index 1897633..60e4d1c 100644 --- a/src/cthreads/cpp/bindings/gpu_module.hpp +++ b/src/cthreads/cpp/bindings/gpu_module.hpp @@ -4,5 +4,5 @@ namespace py = pybind11; -/** Register ``cthreads._ext.gpu`` (Vulkan context probe API). */ -void bind_gpu(py::module_& parent); +/** Register ``cthreads._ext.gpu`` (probe API + test-only ``testing`` submodule). */ +void bind_gpu(py::module_& parent); \ No newline at end of file diff --git a/src/cthreads/cpp/bindings/gpu_testing_module.cpp b/src/cthreads/cpp/bindings/gpu_testing_module.cpp new file mode 100644 index 0000000..a4760ac --- /dev/null +++ b/src/cthreads/cpp/bindings/gpu_testing_module.cpp @@ -0,0 +1,76 @@ +// Copyright (c) 2026 Tobias Karusseit +// This source code is licensed under the MIT license found in the +// LICENSE file in the root directory of this source tree. + +#include "gpu_testing_module.hpp" + +#include "../gpu/testing/pack_roundtrip.hpp" + +#include +#include + +#include +#include +#include +#include + +namespace py = pybind11; + +namespace { + +py::bytes vector_to_bytes(const std::vector& data) { + return py::bytes(reinterpret_cast(data.data()), static_cast(data.size())); +} + +} // namespace + +void bind_gpu_testing(py::module_& gpu_parent) { + py::module_ t = gpu_parent.def_submodule( + "testing", + "TEST-ONLY GpuPack helpers. Not a product API; do not use from library code." + ); + + t.def( + "roundtrip_float_pack", + [](const py::bytes& scalar, const std::vector>& lists) { + const std::string raw = scalar; + auto result = cthreads::gpu::testing::roundtrip_float_pack( + raw.empty() ? nullptr : raw.data(), + raw.size(), + lists + ); + return py::make_tuple(vector_to_bytes(result.scalars), py::cast(result.float_lists)); + }, + py::arg("scalar"), + py::arg("float_lists"), + "Upload/download scalar bytes + list[list[float]] through GpuPack (test-only)." + ); + + t.def( + "roundtrip_int_pack", + [](const py::bytes& scalar, const std::vector>& lists) { + const std::string raw = scalar; + auto result = cthreads::gpu::testing::roundtrip_int_pack( + raw.empty() ? nullptr : raw.data(), + raw.size(), + lists + ); + return py::make_tuple(vector_to_bytes(result.scalars), py::cast(result.int_lists)); + }, + py::arg("scalar"), + py::arg("int_lists"), + "Upload/download scalar bytes + list[list[int]] through GpuPack (test-only)." + ); + + t.def( + "probe_invalid_elem_bytes", + &cthreads::gpu::testing::probe_invalid_elem_bytes, + "Raises GpuInvalidArgument (zero elem_bytes). Test-only." + ); + + t.def( + "probe_use_after_destroy_scalars", + &cthreads::gpu::testing::probe_use_after_destroy_scalars, + "Raises GpuUseAfterDestroy (upload into pack with no scalar buffer). Test-only." + ); +} diff --git a/src/cthreads/cpp/bindings/gpu_testing_module.hpp b/src/cthreads/cpp/bindings/gpu_testing_module.hpp new file mode 100644 index 0000000..df0276b --- /dev/null +++ b/src/cthreads/cpp/bindings/gpu_testing_module.hpp @@ -0,0 +1,11 @@ +#pragma once + +#include + +namespace py = pybind11; + +/** + * Register ``cthreads._ext.gpu.testing`` (GpuPack round-trip smoke API). + * Test-only — not part of the public ``cthreads.gpu`` package. + */ +void bind_gpu_testing(py::module_& gpu_parent); diff --git a/src/cthreads/cpp/gpu/headers/context.hpp b/src/cthreads/cpp/gpu/headers/context.hpp index 0fd3f0f..21e2401 100644 --- a/src/cthreads/cpp/gpu/headers/context.hpp +++ b/src/cthreads/cpp/gpu/headers/context.hpp @@ -3,8 +3,25 @@ #include #include +#include "memory.hpp" + namespace cthreads::gpu { +/** + * Reused copy machinery owned by Context for the process lifetime. + * + * Pool + fence are created in init. Staging scratch is optional and may stay + * empty until the first upload/download grows it (see memory::). + * + * Destroy via shutdown_transfer_engine before destroying the logical device. + */ +struct TransferEngine { + VkCommandPool command_pool = VK_NULL_HANDLE; + VkFence fence = VK_NULL_HANDLE; + // Host-visible scratch for H2D/D2H; empty until allocated. Grow-on-demand. + memory::GpuBuffer staging; +}; + struct Context { // OS handle to the loader shared library (HMODULE on Windows, void* on Linux). // Purpose: keep the DLL mapped and FreeLibrary/dlclose on shutdown. @@ -17,14 +34,42 @@ struct Context { PFN_vkGetInstanceProcAddr vkGetInstanceProcAddr = nullptr; // Global (pre-instance) / instance-level entry points. // Each is a typed function pointer; assigned in init() via GetInstanceProcAddr. - PFN_vkCreateInstance vkCreateInstance = nullptr; - PFN_vkDestroyInstance vkDestroyInstance = nullptr; - PFN_vkEnumeratePhysicalDevices vkEnumeratePhysicalDevices = nullptr; - PFN_vkGetPhysicalDeviceProperties vkGetPhysicalDeviceProperties = nullptr; - PFN_vkGetPhysicalDeviceQueueFamilyProperties vkGetPhysicalDeviceQueueFamilyProperties = nullptr; - PFN_vkCreateDevice vkCreateDevice = nullptr; - PFN_vkDestroyDevice vkDestroyDevice = nullptr; - PFN_vkGetDeviceQueue vkGetDeviceQueue = nullptr; + PFN_vkCreateInstance vkCreateInstance = nullptr; // creates the Vulkan instance (app <-> loader connection) + PFN_vkDestroyInstance vkDestroyInstance = nullptr; // destroys the instance and its child resources owned at instance level + PFN_vkEnumeratePhysicalDevices vkEnumeratePhysicalDevices = nullptr; // lists GPUs the loader can see + PFN_vkGetPhysicalDeviceProperties vkGetPhysicalDeviceProperties = nullptr; // reads name, type, limits for one GPU + PFN_vkGetPhysicalDeviceQueueFamilyProperties vkGetPhysicalDeviceQueueFamilyProperties = nullptr; // lists queue families (graphics/compute/transfer) on one GPU + PFN_vkCreateDevice vkCreateDevice = nullptr; // opens a logical device on a chosen physical GPU + PFN_vkDestroyDevice vkDestroyDevice = nullptr; // destroys the logical device + PFN_vkGetDeviceQueue vkGetDeviceQueue = nullptr; // gets a queue handle used to submit work + + // Buffer and memory functions + PFN_vkCreateBuffer vkCreateBuffer = nullptr; // creates a buffer object (byte region descriptor; needs memory bound) + PFN_vkDestroyBuffer vkDestroyBuffer = nullptr; // destroys a buffer object + PFN_vkGetBufferMemoryRequirements vkGetBufferMemoryRequirements = nullptr; // reports size, alignment, and allowed memory types for a buffer + PFN_vkAllocateMemory vkAllocateMemory = nullptr; // allocates a block of device (or host-visible) memory + PFN_vkFreeMemory vkFreeMemory = nullptr; // frees a memory block from AllocateMemory + PFN_vkBindBufferMemory vkBindBufferMemory = nullptr; // attaches a memory block to a buffer at an offset + PFN_vkMapMemory vkMapMemory = nullptr; // exposes host-visible memory as a CPU pointer for read/write + PFN_vkUnmapMemory vkUnmapMemory = nullptr; // releases a CPU mapping from MapMemory + PFN_vkGetPhysicalDeviceMemoryProperties vkGetPhysicalDeviceMemoryProperties = nullptr; // lists memory heaps/types and their flags (e.g. host-visible, device-local) + + // Command pool, command buffer, copy, and sync functions + PFN_vkCreateCommandPool vkCreateCommandPool = nullptr; // creates a pool that owns command buffers for one queue family + PFN_vkDestroyCommandPool vkDestroyCommandPool = nullptr; // destroys a command pool and its buffers + PFN_vkAllocateCommandBuffers vkAllocateCommandBuffers = nullptr; // allocates one or more command buffers from a pool + PFN_vkFreeCommandBuffers vkFreeCommandBuffers = nullptr; // returns command buffers to the pool / frees them + PFN_vkResetCommandBuffer vkResetCommandBuffer = nullptr; // clears a command buffer so it can be recorded again + PFN_vkBeginCommandBuffer vkBeginCommandBuffer = nullptr; // starts recording commands into a command buffer + PFN_vkEndCommandBuffer vkEndCommandBuffer = nullptr; // finishes recording; buffer is ready to submit + PFN_vkCmdCopyBuffer vkCmdCopyBuffer = nullptr; // records a GPU copy from one buffer to another + PFN_vkCreateFence vkCreateFence = nullptr; // creates a fence (CPU waits until GPU work finishes) + PFN_vkDestroyFence vkDestroyFence = nullptr; // destroys a fence + PFN_vkQueueSubmit vkQueueSubmit = nullptr; // submits recorded command buffers to a queue + PFN_vkWaitForFences vkWaitForFences = nullptr; // blocks the CPU until the given fences signal + PFN_vkResetFences vkResetFences = nullptr; // resets fences back to unsignaled for reuse + + // Opaque Vulkan handles. VkInstance instance = VK_NULL_HANDLE; // connection to the loader/app VkPhysicalDevice physical_device = VK_NULL_HANDLE; // chosen GPU @@ -36,6 +81,8 @@ struct Context { std::string device_name; // True only after init() fully succeeded. bool ready = false; + + TransferEngine transfer_engine; }; // Process-wide singleton accessor. Context& context(); diff --git a/src/cthreads/cpp/gpu/headers/memory.hpp b/src/cthreads/cpp/gpu/headers/memory.hpp new file mode 100644 index 0000000..cfd0d65 --- /dev/null +++ b/src/cthreads/cpp/gpu/headers/memory.hpp @@ -0,0 +1,194 @@ +#pragma once + +#include +#include + +namespace cthreads::gpu { +struct Context; +} + +/** + * Stateless helpers for one GPU byte region at a time: allocate, free, and + * copy between host memory and device-local storage buffers. + * + * Shader-facing buffers are device-local. Host traffic goes through the + * Context TransferEngine: reused command pool + fence, and a grow-on-demand + * host-visible staging scratch. Pass a ready Context each time. GpuPack builds + * on top. + * + * #### Technical terms: + * - Context: process-wide Vulkan connection (device, queue, loaded entry points). + * - host: CPU / process memory used by C++ and Python. + * - device-local: GPU memory optimized for shader access; not persistently mapped. + * - staging: host-visible buffer used only as a temporary for uploads/downloads. + * - SSBO: storage buffer - a buffer shaders can read and write. + */ +namespace cthreads::gpu::memory { + +/** + * Which memory path create_buffer should use. + * + * One create API, two property sets - not two parallel designs. + */ +enum class BufferKind : uint8_t { + // Host-visible + coherent; used only as the memcpy side of transfers. + Staging = 0, + // Device-local storage buffer for shaders (scalar SSBO or one list SSBO). + DeviceLocal = 1, +}; + +/** + * One contiguous byte region on the GPU (staging scratch, scalar SSBO, or one list). + * + * Owns the Vulkan buffer object and the device memory bound to it. Option 5 packs + * use several device-local GpuBuffers: one for all scalars, then one per list. + * + * #### Fields: + * - buffer: VkBuffer = Vulkan handle for the buffer object. VK_NULL_HANDLE if empty. + * - memory: VkDeviceMemory = allocated memory slab bound to buffer. VK_NULL_HANDLE if empty. + * - size: VkDeviceSize = caller-facing byte count requested at create (0 if empty). + * - mapped: void* = CPU pointer when this is a mapped staging buffer; nullptr for + * device-local buffers (they are never persistently mapped). + * - kind: BufferKind = Staging or DeviceLocal; drives destroy and upload/download. + * + * #### Technical terms: + * - VkBuffer: opaque id for a buffer resource (not a raw C pointer to bytes). + * - VkDeviceMemory: opaque id for an allocated memory block from the driver. + * - VK_NULL_HANDLE: sentinel meaning no object. + */ +struct GpuBuffer { + VkBuffer buffer = VK_NULL_HANDLE; + VkDeviceMemory memory = VK_NULL_HANDLE; + VkDeviceSize size = 0; + void* mapped = nullptr; + BufferKind kind = BufferKind::DeviceLocal; +}; + +/** + * Picks which memory type index on this GPU can back a new buffer. + * + * Vulkan gives a bitmask of allowed types for a buffer (type_bits). You also + * request properties (for example host-visible, or device-local). This walks the + * device memory types and returns the first index that is allowed and has every + * requested property flag. + * + * #### Parameters: + * - context: Context& = live GPU context (needs physical device + memory query entry point). + * - type_bits: uint32_t = bit i set means memory type i is allowed (from memory requirements). + * - properties: VkMemoryPropertyFlags = required flags for that type. + * + * #### Returns: + * - uint32_t = memory type index for vkAllocateMemory. + * + * #### Throws: + * - runtime_error if context is not ready or no type matches. + * + * #### Technical terms: + * - memory type: one of the heaps/types the driver exposes (device-local, host-visible, etc.). + * - type_bits: bitmask; bit N means this buffer may use memory type N. + */ +uint32_t find_memory_type( + cthreads::gpu::Context& context, + uint32_t type_bits, + VkMemoryPropertyFlags properties +); + +/** + * Creates a GpuBuffer of the given kind and size. + * + * Staging: host-visible + coherent, transfer usage, left mapped for memcpy. + * DeviceLocal: device-local memory, storage + transfer usage, not mapped. + * Size must be greater than 0 (empty lists are handled by GpuPack, not a zero-size buffer). + * + * #### Parameters: + * - context: Context& = initialized device and buffer/memory entry points. + * - size: VkDeviceSize = number of bytes to store (must be > 0). + * - kind: BufferKind = Staging or DeviceLocal. + * + * #### Returns: + * - GpuBuffer = owned handles; caller must destroy_buffer when done. + * + * #### Throws: + * - runtime_error on zero size, missing entry points, or Vulkan create/allocate/bind/map failure. + */ +GpuBuffer create_buffer( + cthreads::gpu::Context& context, + VkDeviceSize size, + BufferKind kind +); + +/** + * Releases a GpuBuffer and clears its fields. + * + * Unmaps if mapped, destroys the VkBuffer, frees VkDeviceMemory, then zeroes the + * handle. Safe no-op if the buffer is already empty. + * + * #### Parameters: + * - context: Context& = same device that created the buffer. + * - buffer: GpuBuffer& = allocation to free; left empty after return. + * + * #### Returns: + * - void + */ +void destroy_buffer(cthreads::gpu::Context& context, GpuBuffer& buffer); + +/** + * Copies host bytes into a device-local GpuBuffer (host to device). + * + * Final path: memcpy into Context TransferEngine staging, then GPU-copy into + * the device-local buffer (engine pool + fence). + * + * #### Parameters: + * - context: Context& = device + transfer engine entry points. + * - buffer: GpuBuffer& = device-local destination; must be large enough. + * - data: const void* = host source bytes. + * - size: VkDeviceSize = bytes to copy; must be <= buffer.size. + * + * #### Returns: + * - void + * + * #### Throws: + * - runtime_error if buffer is not device-local, data is null, size is invalid, + * transfer engine is missing, or the transfer fails. + * + * #### Technical terms: + * - upload: copy from host (CPU) toward device (GPU) memory. + */ +void upload_buffer( + cthreads::gpu::Context& context, + GpuBuffer& buffer, + const void* data, + VkDeviceSize size +); + +/** + * Copies bytes from a device-local GpuBuffer into host memory (device to host). + * + * Final path: GPU-copy device-local into TransferEngine staging, then memcpy + * staging to data (engine pool + fence). + * + * #### Parameters: + * - context: Context& = device + transfer engine entry points. + * - buffer: GpuBuffer& = device-local source; must be large enough. + * - data: void* = host destination bytes. + * - size: VkDeviceSize = bytes to copy; must be <= buffer.size. + * + * #### Returns: + * - void + * + * #### Throws: + * - runtime_error if buffer is not device-local, data is null, size is invalid, + * transfer engine is missing, or the transfer fails. + * + * #### Technical terms: + * - download: copy from device (GPU) memory to host (CPU). + * - writeback: copying native results into the same host/Python objects the caller passed in. + */ +void download_buffer( + cthreads::gpu::Context& context, + GpuBuffer& buffer, + void* data, + VkDeviceSize size +); + +} // namespace cthreads::gpu::memory diff --git a/src/cthreads/cpp/gpu/headers/pack.hpp b/src/cthreads/cpp/gpu/headers/pack.hpp new file mode 100644 index 0000000..67ea1ba --- /dev/null +++ b/src/cthreads/cpp/gpu/headers/pack.hpp @@ -0,0 +1,226 @@ +#pragma once + +#include +#include +#include + +#include "memory.hpp" + +namespace cthreads::gpu { +struct Context; +} + +/** + * Option 5 GpuPack: one device-local scalar SSBO plus one device-local SSBO per + * list/container. Host traffic goes through memory:: upload/download helpers. + * + * This is a generic runtime bag of buffers. Per-kernel std430 layout and which + * Python arg maps to which slot are marshal/codegen concerns, not this type. + * + * #### Technical terms: + * - scalar SSBO: single buffer holding all packed scalar bytes for one launch. + * - container slot: one list (or similar) argument; empty numel means no VkBuffer. + * - upload / download: host <-> device-local copy via staging (see memory::). + */ +namespace cthreads::gpu::pack { + +/** + * How large one container slot should be at create time. + * + * #### Fields: + * - elem_bytes: size_t = bytes per element (e.g. 4 for float or int32). + * - numel: size_t = element count. 0 means no buffer is allocated for that slot. + */ +struct ContainerSpec { + size_t elem_bytes = 0; + size_t numel = 0; +}; + +/** + * One list/container argument inside a GpuPack. + * + * #### Fields: + * - buffer: GpuBuffer = device-local SSBO when numel > 0; empty handles when numel == 0. + * - spec: ContainerSpec = elem size and count used at create (and for bounds checks). + */ +struct ContainerSlot { + cthreads::gpu::memory::GpuBuffer buffer; + ContainerSpec spec; +}; + +/** + * Per-launch GPU argument pack (option 5). + * + * Owns Vulkan allocations until destroy_gpu_pack. Returning GpuPack by value + * moves handles only; device bytes are not copied. + * + * #### Fields: + * - scalar_buffer: GpuBuffer = device-local blob for all scalars (empty if scalar_bytes was 0). + * - container_slots: vector of ContainerSlot = binding order 1..N matching create specs. + */ +struct GpuPack { + cthreads::gpu::memory::GpuBuffer scalar_buffer; + std::vector container_slots; +}; + +/** + * Allocates a GpuPack: device-local scalar buffer (if scalar_bytes > 0) and one + * device-local buffer per container with numel > 0. + * + * Empty containers (numel == 0) keep a slot with no VkBuffer. Zero-size Vulkan + * buffers are never created. + * + * #### Parameters: + * - context: Context& = initialized GPU context with buffer/memory entry points. + * - scalar_bytes: size_t = byte size of the scalar blob (0 = no scalar buffer). + * - container_specs: vector = per-list elem_bytes and numel, in binding order. + * + * #### Returns: + * - GpuPack = owned buffers; caller must destroy_gpu_pack when done. + * + * #### Throws: + * - runtime_error on bad specs (e.g. numel > 0 but elem_bytes == 0) or Vulkan alloc failure. + */ +GpuPack create_gpu_pack( + cthreads::gpu::Context& context, + size_t scalar_bytes, + std::vector container_specs +); + +/** + * Copies host scalar bytes into pack.scalar_buffer (host to device). + * + * #### Parameters: + * - context: Context& = same device that created the pack. + * - pack: GpuPack& = destination pack (must have a scalar buffer large enough). + * - data: const void* = host source bytes. + * - size: size_t = bytes to copy; must be <= scalar_buffer.size. + * + * #### Throws: + * - runtime_error if there is no scalar buffer, data is null, size is invalid, or transfer fails. + */ +void upload_scalars( + cthreads::gpu::Context& context, + GpuPack& pack, + const void* data, + size_t size +); + +/** + * Copies host bytes into one container slot (host to device). + * + * #### Parameters: + * - context: Context& = same device that created the pack. + * - pack: GpuPack& = destination pack. + * - index: size_t = container_slots index. + * - data: const void* = host source bytes. + * - size: size_t = bytes to copy; must equal elem_bytes * numel for that slot. + * + * #### Throws: + * - runtime_error if index is out of range, the slot is empty (numel == 0), data is null, + * size mismatches, or transfer fails. + */ +void upload_container( + cthreads::gpu::Context& context, + GpuPack& pack, + size_t index, + const void* data, + size_t size +); + +/** + * Uploads every non-empty container slot from parallel host pointers. + * + * data[i] / sizes[i] correspond to container_slots[i]. Slots with numel == 0 + * are skipped (pointer/size for those entries are ignored). + * + * #### Parameters: + * - context: Context& = same device that created the pack. + * - pack: GpuPack& = destination pack. + * - data: vector of host pointers, one per slot. + * - sizes: vector of byte counts, one per slot (same length as data and slots). + * + * #### Throws: + * - runtime_error if lengths mismatch pack.container_slots or any per-slot upload fails. + */ +void upload_containers( + cthreads::gpu::Context& context, + GpuPack& pack, + const std::vector& data, + const std::vector& sizes +); + +/** + * Copies pack.scalar_buffer into host memory (device to host). + * + * #### Parameters: + * - context: Context& = same device that created the pack. + * - pack: GpuPack& = source pack. + * - data: void* = host destination bytes. + * - size: size_t = bytes to copy; must be non-zero and <= scalar_buffer.size. + * + * #### Throws: + * - runtime_error if there is no scalar buffer, data is null, size is invalid, or transfer fails. + */ +void download_scalars( + cthreads::gpu::Context& context, + GpuPack& pack, + void* data, + size_t size +); + +/** + * Copies one container slot into host memory (device to host). + * + * #### Parameters: + * - context: Context& = same device that created the pack. + * - pack: GpuPack& = source pack. + * - index: size_t = container_slots index. + * - data: void* = host destination bytes. + * - size: size_t = bytes to copy; must be <= container_slots[index].buffer.size. + * + * #### Throws: + * - runtime_error if index is out of range, the slot is empty (numel == 0), data is null, + * size mismatches, or transfer fails. + */ +void download_container( + cthreads::gpu::Context& context, + GpuPack& pack, + size_t index, + void* data, + size_t size +); + +/** + * Downloads every non-empty container slot into parallel host pointers. + * + * Slots with numel == 0 are skipped. data/sizes length must match container_slots. + * + * #### Parameters: + * - context: Context& = same device that created the pack. + * - pack: GpuPack& = source pack. + * - data: vector of host destinations, one per slot. + * - sizes: vector of byte counts, one per slot (same length as data and slots). + * + * #### Throws: + * - runtime_error if lengths mismatch pack.container_slots or any per-slot download fails. + */ +void download_containers( + cthreads::gpu::Context& context, + GpuPack& pack, + std::vector& data, + const std::vector& sizes +); + +/** + * Destroys all buffers owned by the pack and clears its fields. + * + * Safe to call on an already-empty pack. Does not destroy the Context. + * + * #### Parameters: + * - context: Context& = same device that created the pack. + * - pack: GpuPack& = pack to free; left empty after return. + */ +void destroy_gpu_pack(cthreads::gpu::Context& context, GpuPack& pack); + +} // namespace cthreads::gpu::pack diff --git a/src/cthreads/cpp/gpu/impl/context.cpp b/src/cthreads/cpp/gpu/impl/context.cpp index 0826dbb..4586ae0 100644 --- a/src/cthreads/cpp/gpu/impl/context.cpp +++ b/src/cthreads/cpp/gpu/impl/context.cpp @@ -5,6 +5,8 @@ #include #include +#include "../headers/memory.hpp" + #if defined(_WIN32) // Windows (32-bit or 64-bit) #include @@ -20,6 +22,93 @@ namespace cthreads::gpu { namespace { + + // ------ Hidden TransferEngineHelpers ------ + + void shutdown_transfer_engine(Context& c) { + // Safe no-op if the engine was never created or already cleared. + TransferEngine& te = c.transfer_engine; + if (te.command_pool == VK_NULL_HANDLE && + te.fence == VK_NULL_HANDLE && + te.staging.buffer == VK_NULL_HANDLE) { + return; + } + + // Finish any in-flight copy before freeing GPU objects. + if (te.fence != VK_NULL_HANDLE && c.device != VK_NULL_HANDLE && + c.vkWaitForFences) { + c.vkWaitForFences(c.device, 1, &te.fence, VK_TRUE, UINT64_MAX); + } + + // Staging first (uses device + buffer PFNs), then fence, then pool. + if (te.staging.buffer != VK_NULL_HANDLE && c.device != VK_NULL_HANDLE) { + memory::destroy_buffer(c, te.staging); + } + if (te.fence != VK_NULL_HANDLE && c.device != VK_NULL_HANDLE && + c.vkDestroyFence) { + c.vkDestroyFence(c.device, te.fence, nullptr); + te.fence = VK_NULL_HANDLE; + } + if (te.command_pool != VK_NULL_HANDLE && c.device != VK_NULL_HANDLE && + c.vkDestroyCommandPool) { + c.vkDestroyCommandPool(c.device, te.command_pool, nullptr); + te.command_pool = VK_NULL_HANDLE; + } + te = TransferEngine{}; + } + + void init_transfer_engine(Context& c) { + // Pool + fence only. Staging is allocated later on demand so idle + // contexts do not hold a fixed 1 MiB host-visible buffer. + if (c.transfer_engine.command_pool != VK_NULL_HANDLE && + c.transfer_engine.fence != VK_NULL_HANDLE) { + return; + } + + if (c.device == VK_NULL_HANDLE || !c.vkCreateCommandPool || + !c.vkDestroyCommandPool || !c.vkCreateFence || !c.vkDestroyFence) { + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: init_transfer_engine missing " + "device or command/fence entry points"); + } + + // If a previous attempt left a half-built engine, clear it first. + if (c.transfer_engine.command_pool != VK_NULL_HANDLE || + c.transfer_engine.fence != VK_NULL_HANDLE || + c.transfer_engine.staging.buffer != VK_NULL_HANDLE) { + shutdown_transfer_engine(c); + } + + VkCommandPoolCreateInfo pool_info{}; + pool_info.sType = VK_STRUCTURE_TYPE_COMMAND_POOL_CREATE_INFO; + pool_info.queueFamilyIndex = c.queue_family; + // TRANSIENT: short-lived recordings. RESET: allow vkResetCommandBuffer reuse. + pool_info.flags = VK_COMMAND_POOL_CREATE_TRANSIENT_BIT | + VK_COMMAND_POOL_CREATE_RESET_COMMAND_BUFFER_BIT; + if (c.vkCreateCommandPool( + c.device, &pool_info, nullptr, &c.transfer_engine.command_pool) != + VK_SUCCESS) { + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: vkCreateCommandPool failed"); + } + + VkFenceCreateInfo fence_info{}; + fence_info.sType = VK_STRUCTURE_TYPE_FENCE_CREATE_INFO; + // Signaled so the first wait/reset path can treat it as "idle". + fence_info.flags = VK_FENCE_CREATE_SIGNALED_BIT; + if (c.vkCreateFence( + c.device, &fence_info, nullptr, &c.transfer_engine.fence) != + VK_SUCCESS) { + c.vkDestroyCommandPool( + c.device, c.transfer_engine.command_pool, nullptr); + c.transfer_engine.command_pool = VK_NULL_HANDLE; + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: vkCreateFence failed"); + } + } + + // ------ Hidden Context Helpers ------ + // Look up one export inside the already-loaded loader module. // module: void* (HMODULE on Windows). name: C string export name. // returns: raw code address, or nullptr if missing. @@ -115,6 +204,9 @@ namespace { c.vkGetPhysicalDeviceQueueFamilyProperties = get_fn( c, c.instance, "vkGetPhysicalDeviceQueueFamilyProperties"); + c.vkGetPhysicalDeviceMemoryProperties = + get_fn( + c, c.instance, "vkGetPhysicalDeviceMemoryProperties"); c.vkCreateDevice = get_fn( c, c.instance, "vkCreateDevice"); c.vkDestroyDevice = get_fn( @@ -175,11 +267,61 @@ namespace { } // Queue handle is owned by the device; index 0 of that family. c.vkGetDeviceQueue(c.device, c.queue_family, 0, &c.queue); + + // Device-level buffer / memory / transfer entry points (Issue 2). + // Resolved after the logical device exists; GIPA still returns loader trampolines. + c.vkCreateBuffer = get_fn( + c, c.instance, "vkCreateBuffer"); + c.vkDestroyBuffer = get_fn( + c, c.instance, "vkDestroyBuffer"); + c.vkGetBufferMemoryRequirements = get_fn( + c, c.instance, "vkGetBufferMemoryRequirements"); + c.vkAllocateMemory = get_fn( + c, c.instance, "vkAllocateMemory"); + c.vkFreeMemory = get_fn( + c, c.instance, "vkFreeMemory"); + c.vkBindBufferMemory = get_fn( + c, c.instance, "vkBindBufferMemory"); + c.vkMapMemory = get_fn( + c, c.instance, "vkMapMemory"); + c.vkUnmapMemory = get_fn( + c, c.instance, "vkUnmapMemory"); + c.vkCreateCommandPool = get_fn( + c, c.instance, "vkCreateCommandPool"); + c.vkDestroyCommandPool = get_fn( + c, c.instance, "vkDestroyCommandPool"); + c.vkAllocateCommandBuffers = get_fn( + c, c.instance, "vkAllocateCommandBuffers"); + c.vkFreeCommandBuffers = get_fn( + c, c.instance, "vkFreeCommandBuffers"); + c.vkResetCommandBuffer = get_fn( + c, c.instance, "vkResetCommandBuffer"); + c.vkBeginCommandBuffer = get_fn( + c, c.instance, "vkBeginCommandBuffer"); + c.vkEndCommandBuffer = get_fn( + c, c.instance, "vkEndCommandBuffer"); + c.vkCmdCopyBuffer = get_fn( + c, c.instance, "vkCmdCopyBuffer"); + c.vkCreateFence = get_fn( + c, c.instance, "vkCreateFence"); + c.vkDestroyFence = get_fn( + c, c.instance, "vkDestroyFence"); + c.vkQueueSubmit = get_fn( + c, c.instance, "vkQueueSubmit"); + c.vkWaitForFences = get_fn( + c, c.instance, "vkWaitForFences"); + c.vkResetFences = get_fn( + c, c.instance, "vkResetFences"); + c.ready = true; + // After device + PFNs + ready: reusable copy pool/fence (staging grows later). + init_transfer_engine(c); } void shutdown_unlocked(Context& c) { - + // Children before parents: transfer engine (pool/fence/staging) then device. + shutdown_transfer_engine(c); + // 1) release logical device if (c.device != VK_NULL_HANDLE && c.vkDestroyDevice) { // check if device is set and if theres a destroy fn for it c.vkDestroyDevice(c.device, nullptr); // set nullptr @@ -206,6 +348,7 @@ namespace { } // 4) Clear function pointers so a buggy late call can't jump into freed DLL! + // instance functions c.vkGetInstanceProcAddr = nullptr; c.vkCreateInstance = nullptr; c.vkDestroyInstance = nullptr; @@ -215,6 +358,32 @@ namespace { c.vkCreateDevice = nullptr; c.vkDestroyDevice = nullptr; c.vkGetDeviceQueue = nullptr; + + // buffer and memory functions + c.vkCreateBuffer = nullptr; + c.vkDestroyBuffer = nullptr; + c.vkGetBufferMemoryRequirements = nullptr; + c.vkAllocateMemory = nullptr; + c.vkFreeMemory = nullptr; + c.vkBindBufferMemory = nullptr; + c.vkMapMemory = nullptr; + c.vkUnmapMemory = nullptr; + c.vkGetPhysicalDeviceMemoryProperties = nullptr; + + // Command pool, command buffer, copy, and sync functions + c.vkCreateCommandPool = nullptr; + c.vkDestroyCommandPool = nullptr; + c.vkAllocateCommandBuffers = nullptr; + c.vkFreeCommandBuffers = nullptr; + c.vkResetCommandBuffer = nullptr; + c.vkBeginCommandBuffer = nullptr; + c.vkEndCommandBuffer = nullptr; + c.vkCmdCopyBuffer = nullptr; + c.vkCreateFence = nullptr; + c.vkDestroyFence = nullptr; + c.vkQueueSubmit = nullptr; + c.vkWaitForFences = nullptr; + c.vkResetFences = nullptr; c.queue_family = 0; c.device_name.clear(); diff --git a/src/cthreads/cpp/gpu/impl/memory.cpp b/src/cthreads/cpp/gpu/impl/memory.cpp new file mode 100644 index 0000000..6789fb9 --- /dev/null +++ b/src/cthreads/cpp/gpu/impl/memory.cpp @@ -0,0 +1,367 @@ +#include "../headers/memory.hpp" +#include "../headers/context.hpp" + +#include +#include +#include + +namespace cthreads::gpu::memory { + +namespace { + +/** +* Ensure that the global context singleton has been initialized and is ready to use. +* +* #### Parameters: +* - context: The global context singleton (holds all dynamically linked vulkan function ptrs aswell as the device information) +* - where: The name of the function that is calling this helper. This is used to construct the error message. +* +* #### Throws: +* - std::runtime_error: If the context is not ready. +*/ +void require_ready(const Context& context, const char* where) { + if (!context.ready || context.device == VK_NULL_HANDLE) { + throw std::runtime_error( + std::string("cthreads.gpu.VulkanInitFailed: ") + where + + " needs an initialized device"); + } +} + +void require_transfer_engine(const Context& context, const char* where) { + if (context.transfer_engine.command_pool == VK_NULL_HANDLE || + context.transfer_engine.fence == VK_NULL_HANDLE) { + throw std::runtime_error( + std::string("cthreads.gpu.VulkanInitFailed: ") + where + + " needs an initialized TransferEngine (pool + fence)"); + } +} + +// GPU copy via Context TransferEngine pool + fence, then CPU wait. +void copy_buffer_and_wait( + Context& context, + VkBuffer src, + VkBuffer dst, + VkDeviceSize size +) { + require_transfer_engine(context, "copy_buffer_and_wait"); + if (!context.vkAllocateCommandBuffers || !context.vkFreeCommandBuffers || + !context.vkBeginCommandBuffer || !context.vkEndCommandBuffer || + !context.vkCmdCopyBuffer || !context.vkQueueSubmit || + !context.vkWaitForFences || !context.vkResetFences || !context.queue) { + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: copy_buffer_and_wait missing " + "command/fence entry points or queue"); + } + + TransferEngine& te = context.transfer_engine; + VkCommandPool pool = te.command_pool; + VkFence fence = te.fence; + + VkCommandBufferAllocateInfo alloc_info{}; + alloc_info.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_ALLOCATE_INFO; + alloc_info.commandPool = pool; + alloc_info.level = VK_COMMAND_BUFFER_LEVEL_PRIMARY; + alloc_info.commandBufferCount = 1; + VkCommandBuffer cmd = VK_NULL_HANDLE; + if (context.vkAllocateCommandBuffers(context.device, &alloc_info, &cmd) != + VK_SUCCESS) { + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: vkAllocateCommandBuffers failed"); + } + + VkCommandBufferBeginInfo begin_info{}; + begin_info.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO; + begin_info.flags = VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT; + if (context.vkBeginCommandBuffer(cmd, &begin_info) != VK_SUCCESS) { + context.vkFreeCommandBuffers(context.device, pool, 1, &cmd); + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: vkBeginCommandBuffer failed"); + } + + VkBufferCopy region{}; + region.srcOffset = 0; + region.dstOffset = 0; + region.size = size; + context.vkCmdCopyBuffer(cmd, src, dst, 1, ®ion); + + if (context.vkEndCommandBuffer(cmd) != VK_SUCCESS) { + context.vkFreeCommandBuffers(context.device, pool, 1, &cmd); + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: vkEndCommandBuffer failed"); + } + + // Fence is created signaled and left signaled after each wait; reset for reuse. + if (context.vkResetFences(context.device, 1, &fence) != VK_SUCCESS) { + context.vkFreeCommandBuffers(context.device, pool, 1, &cmd); + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: vkResetFences failed"); + } + + VkSubmitInfo submit{}; + submit.sType = VK_STRUCTURE_TYPE_SUBMIT_INFO; + submit.commandBufferCount = 1; + submit.pCommandBuffers = &cmd; + if (context.vkQueueSubmit(context.queue, 1, &submit, fence) != VK_SUCCESS) { + context.vkFreeCommandBuffers(context.device, pool, 1, &cmd); + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: vkQueueSubmit failed"); + } + + if (context.vkWaitForFences( + context.device, 1, &fence, VK_TRUE, UINT64_MAX) != VK_SUCCESS) { + context.vkFreeCommandBuffers(context.device, pool, 1, &cmd); + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: vkWaitForFences failed"); + } + + context.vkFreeCommandBuffers(context.device, pool, 1, &cmd); +} + +} // namespace + +uint32_t find_memory_type( + Context& context, + uint32_t type_bits, + VkMemoryPropertyFlags properties +) { + // ensure the context is setup correctly + if (context.physical_device == VK_NULL_HANDLE || + !context.vkGetPhysicalDeviceMemoryProperties) { + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: find_memory_type needs a ready " + "physical device and vkGetPhysicalDeviceMemoryProperties" + ); + } + + VkPhysicalDeviceMemoryProperties memory_props{}; // get mem properties from the ctx device + context.vkGetPhysicalDeviceMemoryProperties( + context.physical_device, &memory_props); + + for (uint32_t i = 0; i < memory_props.memoryTypeCount; ++i) { + const bool allowed_by_buffer = (type_bits & (1u << i)) != 0; // check if the ith bit is 1 + if (!allowed_by_buffer) { + continue; + } + const VkMemoryPropertyFlags flags = + memory_props.memoryTypes[i].propertyFlags; + if ((flags & properties) == properties) { // checck if all properties bits are set in flags + return i; + } + } + + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: no memory type matches type_bits and " + "requested properties"); +} + +GpuBuffer create_buffer( + Context& context, + VkDeviceSize size, + BufferKind kind +) { + require_ready(context, "create_buffer"); // ensure the ctx is ready (the device must be initialized) + if (size == 0) { + throw std::runtime_error( + "cthreads.gpu.GpuInvalidArgument: create_buffer size must be greater than 0"); + } + if (!context.vkCreateBuffer || !context.vkGetBufferMemoryRequirements || + !context.vkAllocateMemory || !context.vkBindBufferMemory || + !context.vkDestroyBuffer || !context.vkFreeMemory) { + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: create_buffer missing Vulkan entry points"); + } + + // setup flags based of the buffers kind + VkBufferUsageFlags usage = 0; + VkMemoryPropertyFlags mem_props = 0; + if (kind == BufferKind::Staging) { + // Host can memcpy here; GPU copies to/from device-local buffers. + usage = VK_BUFFER_USAGE_TRANSFER_SRC_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT; + mem_props = VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | + VK_MEMORY_PROPERTY_HOST_COHERENT_BIT; + } else { + // Shader SSBO + copies for marshal upload/download. + usage = VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | + VK_BUFFER_USAGE_TRANSFER_SRC_BIT | + VK_BUFFER_USAGE_TRANSFER_DST_BIT; + mem_props = VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT; + } + + GpuBuffer out{}; // create gpu buffer struct (internally owns vulkan buffer and memory handles) + out.kind = kind; + + // create the vk buffer + VkBufferCreateInfo bci{}; + bci.sType = VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO; + bci.size = size; + bci.usage = usage; + bci.sharingMode = VK_SHARING_MODE_EXCLUSIVE; + if (context.vkCreateBuffer(context.device, &bci, nullptr, &out.buffer) != + VK_SUCCESS) { + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: vkCreateBuffer failed"); + } + + VkMemoryRequirements mem_reqs{}; // get the memory requirements for the buffer + context.vkGetBufferMemoryRequirements(context.device, out.buffer, &mem_reqs); + + const uint32_t memory_type = // find the memory type that matches the requirements + find_memory_type(context, mem_reqs.memoryTypeBits, mem_props); + + // allocate the memory (allocated to the gpuBuffers memory handle) + VkMemoryAllocateInfo mai{}; + mai.sType = VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO; + mai.allocationSize = mem_reqs.size; + mai.memoryTypeIndex = memory_type; + if (context.vkAllocateMemory(context.device, &mai, nullptr, &out.memory) != + VK_SUCCESS) { + context.vkDestroyBuffer(context.device, out.buffer, nullptr); + out.buffer = VK_NULL_HANDLE; + throw std::runtime_error( + "cthreads.gpu.VulkanOutOfMemory: vkAllocateMemory failed"); + } + + if (context.vkBindBufferMemory(context.device, out.buffer, out.memory, 0) != + VK_SUCCESS) { + context.vkFreeMemory(context.device, out.memory, nullptr); + context.vkDestroyBuffer(context.device, out.buffer, nullptr); + out.memory = VK_NULL_HANDLE; + out.buffer = VK_NULL_HANDLE; + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: vkBindBufferMemory failed"); + } + + // if the buffer is for staging then also set the cpu pointer to the memory in buffer.mapped + if (kind == BufferKind::Staging) { + if (!context.vkMapMemory) { // if the mem isnt mapped sth went wrong -> free and throw + context.vkFreeMemory(context.device, out.memory, nullptr); + context.vkDestroyBuffer(context.device, out.buffer, nullptr); + out.memory = VK_NULL_HANDLE; + out.buffer = VK_NULL_HANDLE; + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: vkMapMemory missing"); + } + if (context.vkMapMemory( // map the memory to the cpu pointer + context.device, out.memory, 0, mem_reqs.size, 0, &out.mapped) != + VK_SUCCESS) { + context.vkFreeMemory(context.device, out.memory, nullptr); + context.vkDestroyBuffer(context.device, out.buffer, nullptr); + out.memory = VK_NULL_HANDLE; + out.buffer = VK_NULL_HANDLE; + out.mapped = nullptr; + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: vkMapMemory failed"); + } + } + + out.size = size; + return out; +} + +void destroy_buffer(Context& context, GpuBuffer& buffer) { + // check if the buffer is already destroyed + if (buffer.buffer == VK_NULL_HANDLE && buffer.memory == VK_NULL_HANDLE) { + buffer = GpuBuffer{}; + return; + } + require_ready(context, "destroy_buffer"); // ensure the ctx is ready (the device must be initialized) + + // if the buffer is mapped then unmap it + if (buffer.mapped != nullptr && context.vkUnmapMemory && + buffer.memory != VK_NULL_HANDLE) { + context.vkUnmapMemory(context.device, buffer.memory); + buffer.mapped = nullptr; + } + // destroy the vk buffer + if (buffer.buffer != VK_NULL_HANDLE && context.vkDestroyBuffer) { + context.vkDestroyBuffer(context.device, buffer.buffer, nullptr); + buffer.buffer = VK_NULL_HANDLE; + } + // free the vk memory + if (buffer.memory != VK_NULL_HANDLE && context.vkFreeMemory) { + context.vkFreeMemory(context.device, buffer.memory, nullptr); + buffer.memory = VK_NULL_HANDLE; + } + buffer.size = 0; + buffer.kind = BufferKind::DeviceLocal; +} + +namespace { + +// Grow-only host-visible scratch on the TransferEngine. Never shrinks until +// Context shutdown. Safe because upload/download wait on the engine fence. +void ensure_staging(Context& context, VkDeviceSize size) { + TransferEngine& te = context.transfer_engine; + if (te.staging.buffer != VK_NULL_HANDLE && te.staging.size >= size && + te.staging.mapped != nullptr) { + return; + } + if (te.staging.buffer != VK_NULL_HANDLE) { + destroy_buffer(context, te.staging); + } + te.staging = create_buffer(context, size, BufferKind::Staging); +} + +} // namespace + +void upload_buffer( + Context& context, + GpuBuffer& buffer, + const void* data, + VkDeviceSize size +) { + require_ready(context, "upload_buffer"); + require_transfer_engine(context, "upload_buffer"); + if (buffer.kind != BufferKind::DeviceLocal || + buffer.buffer == VK_NULL_HANDLE) { + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: upload_buffer requires a device-local " + "GpuBuffer"); + } + if (data == nullptr) { + throw std::runtime_error( + "cthreads.gpu.GpuInvalidArgument: upload_buffer data is null"); + } + if (size == 0 || size > buffer.size) { + throw std::runtime_error( + "cthreads.gpu.GpuInvalidArgument: upload_buffer size invalid"); + } + + // Host -> engine staging (memcpy) -> device-local (GPU copy). + ensure_staging(context, size); + GpuBuffer& staging = context.transfer_engine.staging; + std::memcpy(staging.mapped, data, static_cast(size)); + copy_buffer_and_wait(context, staging.buffer, buffer.buffer, size); +} + +void download_buffer( + Context& context, + GpuBuffer& buffer, + void* data, + VkDeviceSize size +) { + require_ready(context, "download_buffer"); + require_transfer_engine(context, "download_buffer"); + if (buffer.kind != BufferKind::DeviceLocal || + buffer.buffer == VK_NULL_HANDLE) { + throw std::runtime_error( + "cthreads.gpu.VulkanInitFailed: download_buffer requires a " + "device-local GpuBuffer"); + } + if (data == nullptr) { + throw std::runtime_error( + "cthreads.gpu.GpuInvalidArgument: download_buffer data is null"); + } + if (size == 0 || size > buffer.size) { + throw std::runtime_error( + "cthreads.gpu.GpuInvalidArgument: download_buffer size invalid"); + } + + // Device-local -> engine staging (GPU copy) -> host (memcpy). + ensure_staging(context, size); + GpuBuffer& staging = context.transfer_engine.staging; + copy_buffer_and_wait(context, buffer.buffer, staging.buffer, size); + std::memcpy(data, staging.mapped, static_cast(size)); +} + +} // namespace cthreads::gpu::memory diff --git a/src/cthreads/cpp/gpu/impl/pack.cpp b/src/cthreads/cpp/gpu/impl/pack.cpp new file mode 100644 index 0000000..1da3cff --- /dev/null +++ b/src/cthreads/cpp/gpu/impl/pack.cpp @@ -0,0 +1,227 @@ +#include "../headers/pack.hpp" +#include "../headers/memory.hpp" +#include "../headers/context.hpp" + +#include +#include + +namespace cthreads::gpu::pack { +namespace { + +[[noreturn]] void invalid_arg(const char* detail) { + throw std::invalid_argument( + std::string("cthreads.gpu.GpuInvalidArgument: ") + detail + ); +} + +[[noreturn]] void use_after_destroy(const char* detail) { + throw std::runtime_error( + std::string("cthreads.gpu.GpuUseAfterDestroy: ") + detail + ); +} + +} // namespace + +GpuPack create_gpu_pack( + cthreads::gpu::Context& context, + size_t scalar_bytes, + std::vector container_specs +) { + GpuPack pack; + pack.container_slots.reserve(container_specs.size()); + + if (scalar_bytes > 0) { + pack.scalar_buffer = memory::create_buffer( + context, + scalar_bytes, + memory::BufferKind::DeviceLocal + ); + } + + for (const auto& spec : container_specs) { + if (spec.elem_bytes == 0) { + invalid_arg("ContainerSpec must have positive elem_bytes"); + } + + ContainerSlot slot; + slot.spec = spec; + if (spec.numel > 0) { + slot.buffer = memory::create_buffer( + context, + static_cast(spec.numel * spec.elem_bytes), + memory::BufferKind::DeviceLocal + ); + } + pack.container_slots.push_back(std::move(slot)); + } + return pack; +} + +void upload_scalars( + cthreads::gpu::Context& context, + GpuPack& pack, + const void* data, + size_t size +) { + if (!data) { + invalid_arg("data is null"); + } + if (size == 0) { + invalid_arg("size must be non-zero"); + } + if (pack.scalar_buffer.buffer == VK_NULL_HANDLE) { + use_after_destroy("scalar buffer is not initialized"); + } + if (size > pack.scalar_buffer.size) { + invalid_arg("size is greater than the scalar buffer size"); + } + + memory::upload_buffer(context, pack.scalar_buffer, data, size); +} + +void upload_container( + cthreads::gpu::Context& context, + GpuPack& pack, + size_t index, + const void* data, + size_t size +) { + if (index >= pack.container_slots.size()) { + invalid_arg("container index is out of range"); + } + if (!data) { + invalid_arg("data is null"); + } + if (size == 0) { + invalid_arg("size must be non-zero"); + } + + ContainerSlot& slot = pack.container_slots[index]; + if (slot.spec.numel == 0 || slot.buffer.buffer == VK_NULL_HANDLE) { + use_after_destroy("container buffer is not initialized"); + } + const size_t expected = slot.spec.elem_bytes * slot.spec.numel; + if (size != expected) { + invalid_arg("size does not match the container size"); + } + + memory::upload_buffer(context, slot.buffer, data, size); +} + +void upload_containers( + cthreads::gpu::Context& context, + GpuPack& pack, + const std::vector& data, + const std::vector& sizes +) { + if (data.size() != sizes.size()) { + invalid_arg("data and sizes must have the same length"); + } + if (data.size() != pack.container_slots.size()) { + invalid_arg("data length must match container_slots size"); + } + + for (size_t i = 0; i < data.size(); ++i) { + if (pack.container_slots[i].spec.numel == 0) { + continue; + } + try { + upload_container(context, pack, i, data[i], sizes[i]); + } catch (const std::exception& e) { + throw std::runtime_error( + std::string(e.what()) + " [container " + std::to_string(i) + "]" + ); + } + } +} + +void download_scalars( + cthreads::gpu::Context& context, + GpuPack& pack, + void* data, + size_t size +) { + if (!data) { + invalid_arg("data is null"); + } + if (size == 0) { + invalid_arg("size must be non-zero"); + } + if (pack.scalar_buffer.buffer == VK_NULL_HANDLE) { + use_after_destroy("scalar buffer is not initialized"); + } + if (size > pack.scalar_buffer.size) { + invalid_arg("size is greater than the scalar buffer size"); + } + + memory::download_buffer(context, pack.scalar_buffer, data, size); +} + +void download_container( + cthreads::gpu::Context& context, + GpuPack& pack, + size_t index, + void* data, + size_t size +) { + if (index >= pack.container_slots.size()) { + invalid_arg("container index is out of range"); + } + if (!data) { + invalid_arg("data is null"); + } + if (size == 0) { + invalid_arg("size must be non-zero"); + } + + ContainerSlot& slot = pack.container_slots[index]; + if (slot.spec.numel == 0 || slot.buffer.buffer == VK_NULL_HANDLE) { + use_after_destroy("container buffer is not initialized"); + } + if (size > slot.buffer.size) { + invalid_arg("size is greater than the container buffer size"); + } + + memory::download_buffer(context, slot.buffer, data, size); +} + +void download_containers( + cthreads::gpu::Context& context, + GpuPack& pack, + std::vector& data, + const std::vector& sizes +) { + if (data.size() != sizes.size()) { + invalid_arg("data and sizes must have the same length"); + } + if (data.size() != pack.container_slots.size()) { + invalid_arg("data length must match container_slots size"); + } + + for (size_t i = 0; i < data.size(); ++i) { + if (pack.container_slots[i].spec.numel == 0) { + continue; + } + try { + download_container(context, pack, i, data[i], sizes[i]); + } catch (const std::exception& e) { + throw std::runtime_error( + std::string(e.what()) + " [container " + std::to_string(i) + "]" + ); + } + } +} + +void destroy_gpu_pack(cthreads::gpu::Context& context, GpuPack& pack) { + if (pack.scalar_buffer.buffer != VK_NULL_HANDLE) { + memory::destroy_buffer(context, pack.scalar_buffer); + } + for (auto& slot : pack.container_slots) { + if (slot.buffer.buffer != VK_NULL_HANDLE) { + memory::destroy_buffer(context, slot.buffer); + } + } + pack = GpuPack{}; +} + +} // namespace cthreads::gpu::pack diff --git a/src/cthreads/python/cthreads/gpu/__init__.py b/src/cthreads/python/cthreads/gpu/__init__.py index 35348b9..108b975 100644 --- a/src/cthreads/python/cthreads/gpu/__init__.py +++ b/src/cthreads/python/cthreads/gpu/__init__.py @@ -8,10 +8,13 @@ from .errors import ( CThreadsGPUError, GPUNotAvailable, + GpuInvalidArgument, + GpuUseAfterDestroy, VulkanInitFailed, VulkanLoaderNotFound, VulkanNoDevice, VulkanNotBuiltError, + VulkanOutOfMemory, ) try: @@ -26,10 +29,16 @@ def _map_error(exc: BaseException) -> CThreadsGPUError: return VulkanLoaderNotFound(msg) if "VulkanNoDevice" in msg: return VulkanNoDevice(msg) - if "VulkanInitFailed" in msg: - return VulkanInitFailed(msg) + if "VulkanOutOfMemory" in msg: + return VulkanOutOfMemory(msg) + if "GpuUseAfterDestroy" in msg: + return GpuUseAfterDestroy(msg) + if "GpuInvalidArgument" in msg: + return GpuInvalidArgument(msg) if "VulkanNotBuilt" in msg: return VulkanNotBuiltError(msg) + if "VulkanInitFailed" in msg: + return VulkanInitFailed(msg) return VulkanInitFailed(msg) @@ -74,10 +83,13 @@ def shutdown() -> None: __all__ = [ "CThreadsGPUError", "GPUNotAvailable", + "GpuInvalidArgument", + "GpuUseAfterDestroy", "VulkanInitFailed", "VulkanLoaderNotFound", "VulkanNoDevice", "VulkanNotBuiltError", + "VulkanOutOfMemory", "available", "device_name", "init", diff --git a/src/cthreads/python/cthreads/gpu/errors.py b/src/cthreads/python/cthreads/gpu/errors.py index d201a85..5808db9 100644 --- a/src/cthreads/python/cthreads/gpu/errors.py +++ b/src/cthreads/python/cthreads/gpu/errors.py @@ -39,6 +39,27 @@ def __init__(self, detail: str = "Vulkan init failed") -> None: super().__init__(detail) +class VulkanOutOfMemory(CThreadsGPUError): + """vkAllocateMemory (or related) failed — device/host GPU memory exhausted.""" + + def __init__(self, detail: str = "Vulkan out of memory") -> None: + super().__init__(detail) + + +class GpuInvalidArgument(CThreadsGPUError): + """Bad size, dtype, null pointer, index, or other pack/memory argument error.""" + + def __init__(self, detail: str = "Invalid GPU argument") -> None: + super().__init__(detail) + + +class GpuUseAfterDestroy(CThreadsGPUError): + """Buffer/pack used after destroy or never initialized for the requested op.""" + + def __init__(self, detail: str = "GPU resource used after destroy") -> None: + super().__init__(detail) + + class GPUNotAvailable(CThreadsGPUError): """Generic: GPU path not usable (not built, no loader, or no device).""" diff --git a/src/cthreads/python/cthreads/marshal.py b/src/cthreads/python/cthreads/marshal.py index 9a0c219..56afda4 100644 --- a/src/cthreads/python/cthreads/marshal.py +++ b/src/cthreads/python/cthreads/marshal.py @@ -228,7 +228,7 @@ def _pack_c(pack: int | ctypes.c_void_p) -> ctypes.c_void_p: return pack if not pack: raise RuntimeError("cthreads.marshal: null pack pointer") - return ctypes.c_void_p(int(pack)) + return ctypes.c_void_p(int(pack)) # cast to void (trampolines expect a void pointer and static cast internally) def _extra(path: _Path) -> list: diff --git a/tests/unit/test_gpu_context.py b/tests/unit/test_gpu_context.py index e8acd10..f29371c 100644 --- a/tests/unit/test_gpu_context.py +++ b/tests/unit/test_gpu_context.py @@ -14,10 +14,13 @@ from cthreads.gpu.errors import ( CThreadsGPUError, GPUNotAvailable, + GpuInvalidArgument, + GpuUseAfterDestroy, VulkanInitFailed, VulkanLoaderNotFound, VulkanNoDevice, VulkanNotBuiltError, + VulkanOutOfMemory, ) @@ -34,6 +37,9 @@ VulkanLoaderNotFound, VulkanNoDevice, VulkanInitFailed, + VulkanOutOfMemory, + GpuInvalidArgument, + GpuUseAfterDestroy, GPUNotAvailable, ], ) @@ -62,6 +68,9 @@ def test_gpu_module_exports(): "VulkanLoaderNotFound", "VulkanNoDevice", "VulkanInitFailed", + "VulkanOutOfMemory", + "GpuInvalidArgument", + "GpuUseAfterDestroy", "GPUNotAvailable", ): assert hasattr(gpu, name) @@ -131,6 +140,9 @@ def shutdown(self) -> None: ("cthreads.gpu.VulkanLoaderNotFound: vulkan-1.dll not found", VulkanLoaderNotFound), ("cthreads.gpu.VulkanNoDevice: no physical devices", VulkanNoDevice), ("cthreads.gpu.VulkanInitFailed: vkCreateInstance failed", VulkanInitFailed), + ("cthreads.gpu.VulkanOutOfMemory: vkAllocateMemory failed", VulkanOutOfMemory), + ("cthreads.gpu.GpuInvalidArgument: size invalid", GpuInvalidArgument), + ("cthreads.gpu.GpuUseAfterDestroy: scalar buffer is not initialized", GpuUseAfterDestroy), ("cthreads.gpu.VulkanNotBuilt: should not happen from C++", VulkanNotBuiltError), ("something else entirely", VulkanInitFailed), ], diff --git a/tests/unit/test_gpu_pack.py b/tests/unit/test_gpu_pack.py new file mode 100644 index 0000000..810272e --- /dev/null +++ b/tests/unit/test_gpu_pack.py @@ -0,0 +1,170 @@ +""" +Issue GPU-02: GpuPack upload/download round-trips (test-only _ext.gpu.testing). + +Does not use public cthreads.gpu pack APIs (there are none). Live checks skip +when the extension was built without CTHREADS_GPU or when no device is available. +""" + +from __future__ import annotations + +import struct + +import pytest + +from cthreads import gpu +from cthreads.gpu.errors import ( + GpuInvalidArgument, + GpuUseAfterDestroy, +) + + +def _ext_gpu(): + return gpu._gpu + + +def _require_gpu_testing(): + ext = _ext_gpu() + if ext is None: + pytest.skip("cthreads built without CTHREADS_GPU (_ext.gpu missing)") + if not hasattr(ext, "testing"): + pytest.skip("_ext.gpu.testing missing (rebuild with CTHREADS_GPU=ON)") + if not gpu.available(): + pytest.skip("Vulkan loader/device not available in this environment") + return ext.testing + + +def _map_probe(exc: BaseException): + return gpu._map_error(exc) + + +def test_public_gpu_has_no_pack_roundtrip_exports(): + """Product package must not re-export test-only pack helpers.""" + assert not hasattr(gpu, "roundtrip_float_pack") + assert not hasattr(gpu, "roundtrip_int_pack") + assert not hasattr(gpu, "testing") + assert "roundtrip_float_pack" not in gpu.__all__ + for name in ("GpuInvalidArgument", "GpuUseAfterDestroy", "VulkanOutOfMemory"): + assert name in gpu.__all__ + + +def test_testing_submodule_absent_when_not_built(monkeypatch): + monkeypatch.setattr(gpu, "_gpu", None) + assert _ext_gpu() is None + + +def test_live_roundtrip_float_scalars_and_lists(): + testing = _require_gpu_testing() + try: + scalar = struct.pack("