20 gou 02 memory gpupack marshal substrate

docs/index.md

@@ -12,3 +12,4 @@
 | `cthreads documentation` | [link](./COMPILER.md) |
 | **Release**: GitHub Actions, TestPyPI, PyPI trusted publishing | [link](./release.md) |
 | End-to-end example (`@Thread` / `@Threadable` through codegen) | [link](./Example.md) |
+| **Vulkan / GPU backend guide** (cthreads compute path, Issues 1+) | [link](./vk_guide/README.md) |

docs/vk_guide/00-read-me-first.md

@@ -0,0 +1,93 @@
+# 00 — Read me first
+
+## What cthreads is building
+
+cthreads already turns typed Python (`@Thread`) into **CPU** native kernels:
+
+```text
+Python args -> pack (C++ struct) -> run on OS thread -> writeback -> same Python objects
+```
+
+The GPU path follows the **same product idea** with a different middle:
+
+```text
+Python args -> GpuPack (Vulkan buffers) -> run SPIR-V compute -> writeback -> same Python objects
+```
+
+Users still pass `list` / `int` / `float`. They do **not** learn a second buffer type.
+Vulkan stays inside `_ext`.
+
+## What Vulkan is (intuition)
+
+Vulkan is a **low-level remote control for the GPU**.
+
+- The API does not hide a high-level "draw a triangle" call.
+- Callers create memory, bind it, record commands, submit them to a queue, and wait until done.
+- The driver does less magic. The application owns lifetimes, synchronization, and layouts.
+
+That is why Vulkan feels verbose. Every step that OpenGL/CUDA hid becomes a function call.
+The upside: portable compute on NVIDIA / AMD / Intel with one API, and predictable behavior.
+
+## What this project deliberately ignores
+
+These topics are **out of scope** for the cthreads compute backend:
+
+| Topic | Why skip |
+|-------|----------|
+| Swapchain / windows / present | Not drawing to the screen |
+| Render passes / framebuffers | Graphics only |
+| Images / textures / samplers | Buffer-first compute path |
+| Graphics pipelines / vertex input | Compute only |
+| Multi-GPU / sparse memory | Out of scope for now |
+| Buffer device address / bindless | Rejected for the current design |
+| Ray tracing | Out of scope |
+
+If a random tutorial spends chapters on "hello triangle," skim or skip those sections.
+The cthreads "hello world" is: **upload floats, run compute (when landed), download floats**.
+
+## How Vulkan compares to things contributors might know
+
+### vs writing normal C++
+
+| C++ | Vulkan |
+|-----|--------|
+| `new[]` / `malloc` | `vkAllocateMemory` + choose memory type |
+| `memcpy` to a buffer the CPU owns | `memcpy` only works if memory is **host-visible** |
+| Call a function | Record commands, then **submit** to a queue (async GPU) |
+| Function returns when done | GPU may still be running; **fence** or similar to wait |
+
+### vs CUDA (if familiar)
+
+| CUDA | cthreads Vulkan path |
+|------|----------------------|
+| `cudaMalloc` | device-local `VkBuffer` + memory |
+| `cudaMemcpy` | staging buffer + `vkCmdCopyBuffer` |
+| `<<<grid,block>>>` kernel launch | `vkCmdDispatch` in a command buffer |
+| CUDA toolkit for end users | **Drivers only** for end users; SDK headers for *building* GPU-enabled `_ext` |
+
+### vs OpenGL compute
+
+OpenGL hides a lot of binding and sync. Vulkan makes bind points and barriers explicit.
+Same idea (SSBO = storage buffer), more paperwork.
+
+## The one sentence to remember
+
+**Vulkan objects are handles to driver resources; almost nothing happens until a command buffer is submitted to a queue; CPU and GPU run out of sync unless the CPU waits.**
+
+## Locked decisions in this project (so generic tutorials do not confuse contributors)
+
+1. **Option 5 pack:** one scalar SSBO + one SSBO per `list`.
+2. **Device-local** data for shaders; **staging** for CPU copies.
+3. **Descriptors** bind buffers by binding index (not pointers in the scalar struct).
+4. **Launch then join** — no mid-run Python `__sync_state` on GPU.
+5. **Dynamic load** `vulkan-1.dll` / `libvulkan.so.1` — do not hard-link for default CPU wheels.
+
+If a blog says "just map the SSBO and memcpy," that is a shortcut cthreads does **not** use as the production list path.
+
+## Suggested study rhythm
+
+1. Read 01-03 for intuition (no code required).
+2. Read 04-05 while looking at `gpu/headers/context.hpp` and `gpu/impl/context.cpp`.
+3. Read 06-08 while looking at `gpu/headers/memory.hpp` and `gpu/impl/memory.cpp`.
+4. Read 09-12 before working on pipelines, descriptors, or `@Gpu` emit.
+5. Keep 14 glossary open while coding.

docs/vk_guide/01-cthreads-gpu-big-picture.md

@@ -0,0 +1,115 @@
+# 01 — cthreads GPU big picture
+
+## The CPU path
+
+A typical typed kernel looks like:
+
+```python
+@Thread
+def add(n: int, x: list[float], y: list[float]) -> None:
+    i: int = 0
+    while i < n:
+        y[i] = x[i] + y[i]
+        i = i + 1
+
+job = thread(add, 4, [1,2,3,4], [10,10,10,10]).start()
+job.join()
+```
+
+Roughly:
+
+1. **Marshal** copies Python values into a C++ **pack** (args struct).
+2. A worker thread runs the compiled C++ body on that pack.
+3. **Writeback** copies mutated pack fields into the **same** Python lists.
+
+Scalars live as fields in the pack. Lists live as native containers owned by / pointed from the pack.
+
+## The GPU path (same story, different storage)
+
+Same Python. Different backend:
+
+1. **GPU marshal** creates a **GpuPack**:
+   - one **device-local** buffer holding all scalars as a `std430` struct
+   - one **device-local** buffer per list (tight array of floats/ints)
+2. Host bytes go into those buffers via **staging + GPU copy** (upload).
+3. A **compute shader** reads/writes those buffers through **descriptor bindings**.
+4. After the GPU finishes (`join`): **download** + writeback into the same Python objects.
+
+```text
+                 CPU world                         GPU world
+            -----------------                 --------------------
+Python list[float]  --upload-->  staging  --copy-->  device-local SSBO
+Python int/float    --upload-->  staging  --copy-->  scalar SSBO (struct)
+                                                      |
+                                                 compute shader
+                                                      |
+Python list[float]  <--writeback-- staging <--copy-- device-local SSBO
+```
+
+## Why not one giant buffer for everything?
+
+cthreads uses **option 5**:
+
+| Piece | Where it lives |
+|-------|----------------|
+| Scalars (`n`, `a`, …) | One small SSBO interpreted as a C-like struct |
+| Each `list[...]` | Its own SSBO |
+
+Reasons:
+
+- Matches how compute shaders usually declare `buffer` blocks (one binding per array).
+- Easy to reuse the same list buffer across multiple dispatches in a batch.
+- Layout for arrays stays simple (tight packed floats).
+- Scalar writeback is one download of a small blob.
+
+The design **rejects** putting GPU pointers inside the scalar struct (that needs buffer device address). Bindings connect the shader to buffers instead.
+
+## Job contract (important product rule)
+
+On CPU, `__sync_state` can mirror pack -> Python mid-run.
+
+On the GPU path: **only launch then wait**.
+
+- No observing Python lists while the shader runs.
+- Results appear after `GpuJob.join()` (fence wait + download + writeback).
+
+That matches how GPUs want to work: record work, submit, wait once.
+
+## Layers of the GPU stack
+
+| Layer | Responsibility |
+|-------|----------------|
+| Context | Talk to Vulkan: instance, device, queue, entry points |
+| Memory helpers | Allocate buffers, staging upload/download |
+| GpuPack | Option 5: scalar SSBO + per-list SSBOs; marshal/writeback |
+| Launch path | Descriptors, pipeline, dispatch, `GpuJob.join` |
+| `@Gpu` / `gpu()` | Compile and run user kernels (same typing model as CPU) |
+| Workloads / packaging | Real numeric steps, docs, CI, capability gates |
+
+## Mental link: pack vs GpuPack
+
+| CPU pack | GpuPack |
+|----------|---------|
+| One C++ struct in process memory | Several `VkBuffer`s on the device |
+| Field `int n` | Bytes at offset 0 in scalar SSBO |
+| Field `vector<float> x` | Separate SSBO whose bytes are the floats |
+| Kernel gets C++ references | Shader gets bindings 0, 1, 2, … |
+
+Same **roles**. Different **implementation**.
+
+## What "done" looks like for a library user
+
+```python
+from cthreads import Gpu, gpu
+
+@Gpu
+def saxpy(n: int, a: float, x: list[float], y: list[float]) -> None:
+    ...
+
+x = [1.0, 2.0, 3.0, 4.0]
+y = [10.0, 10.0, 10.0, 10.0]
+gpu(saxpy, 4, 2.0, x, y).join()
+# y is updated in place, like a CPU Thread job
+```
+
+No `DeviceBuffer` in that story. Vulkan is an implementation detail of `_ext`.

docs/vk_guide/02-mental-model.md

@@ -0,0 +1,143 @@
+# 02 — Mental model: objects, handles, and "nothing runs until submit"
+
+## Vulkan is an object graph
+
+Almost everything you create is an **object**:
+
+- Instance
+- Physical device (the GPU hardware the loader knows about)
+- Logical device (your open connection to that GPU)
+- Queue (a port you submit work to)
+- Buffer (a typed "array of bytes" object)
+- Device memory (the actual allocated slab)
+- Command pool / command buffer (recorded GPU instructions)
+- Fence (CPU-side "GPU finished" signal)
+- Shader module, pipeline, descriptor set (launch path)
+
+Objects form a tree of ownership. Typical rule:
+
+**Destroy children before parents.** Example: destroy buffers and command pools before destroying the logical device; destroy the device before the instance.
+
+## Handles are not pointers you dereference
+
+When Vulkan returns a `VkBuffer`, you get an opaque **handle** (often an integer-sized id).
+
+You cannot do:
+
+```cpp
+buffer->data[i] = 1.0f;  // NOT how Vulkan works
+```
+
+You talk to objects only through API functions:
+
+```cpp
+vkDestroyBuffer(device, buffer, nullptr);
+```
+
+`VK_NULL_HANDLE` means "no object" (like a null pointer, but for handles).
+
+In our code, `Context` stores many handles and many **function pointers** (`PFN_vkCreateBuffer`, …) because we load Vulkan dynamically.
+
+## Explicit means: you fill structs
+
+Vulkan APIs almost always take a `*CreateInfo` struct:
+
+```cpp
+VkBufferCreateInfo info{};
+info.sType = VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO;
+info.size = 1024;
+info.usage = VK_BUFFER_USAGE_STORAGE_BUFFER_BIT;
+...
+vkCreateBuffer(device, &info, nullptr, &buffer);
+```
+
+Rules of thumb:
+
+1. Zero the struct (`{}` in C++).
+2. Set `sType` to the matching enum (tells the driver which struct this is).
+3. Set only the fields you need; leave the rest zero/null.
+4. Check the `VkResult` return (`VK_SUCCESS` or throw).
+
+This pattern repeats everywhere. Once you recognize it, Vulkan stops feeling like "a million APIs" and starts feeling like "the same paperwork for every object."
+
+## CPU time vs GPU time
+
+This is the biggest intuition jump.
+
+```text
+CPU thread                         GPU
+---------                         ---
+record commands into CB
+submit CB to queue  ------------>  (maybe starts later)
+do other CPU work                  execute copies / dispatches
+vkWaitForFences  <---------------  signals fence when done
+```
+
+If you forget to wait, you might download a buffer the GPU has not finished writing.
+That is a classic bug class (CPU/GPU race).
+
+The upload/download helpers wait on a fence after the copy so the CPU memcpy of staging memory is safe.
+
+## Queues are submission ports
+
+A GPU exposes **queue families** (groups of queues with capabilities):
+
+- graphics
+- compute
+- transfer
+- present (for windows — we ignore)
+
+cthreads picks a family that supports **compute** (and also uses it for buffer copies).
+We create one logical device requesting that family, then call `vkGetDeviceQueue` to get `VkQueue queue`.
+
+All work we care about goes: **record -> vkQueueSubmit(queue, ...)**.
+
+## Two kinds of "memory" people confuse
+
+1. **Host memory** — normal RAM your C++ `memcpy` can touch.
+2. **Device memory** — memory the GPU allocates through Vulkan (`VkDeviceMemory`).
+
+Some device memory is **host-visible**: the driver can give you a CPU pointer via `vkMapMemory` (staging).
+Some is **device-local**: fast for the GPU, **not** for persistent CPU poking (our SSBOs).
+
+A `VkBuffer` alone is not enough. You must:
+
+1. Create the buffer object (describes size/usage).
+2. Ask for memory requirements.
+3. Allocate matching `VkDeviceMemory`.
+4. **Bind** memory to the buffer.
+
+Until bind succeeds, the buffer cannot store data.
+
+## Layers of our stack (keep this map)
+
+```text
+Python cthreads.gpu / future gpu()
+        |
+        v
+pybind _ext.gpu
+        |
+        v
+GpuPack (option 5)
+        |
+        v
+memory:: create/upload/download
+        |
+        v
+Context (instance/device/queue/PFNs)
+        |
+        v
+vulkan-1.dll / libvulkan.so.1  (loader)
+        |
+        v
+GPU driver (ICD)
+```
+
+## Validation layers (optional while learning)
+
+The Vulkan SDK can enable **validation layers**: extra checks that print readable errors when you misuse the API.
+
+For shipping cthreads to users we do **not** require layers.
+While developing, turning them on is highly recommended (see SDK docs for `VK_LAYER_KHRONOS_validation`).
+
+We may gate this behind something like `CTHREADS_VK_VALIDATE=1` later. Not required to understand Context and memory.