Skip to content

Add graph-safe 1-bit PQ KV cache support - #18

Open
zzz0906 wants to merge 1 commit into
mainfrom
zhongzhu/1bit-pq-kv
Open

Add graph-safe 1-bit PQ KV cache support#18
zzz0906 wants to merge 1 commit into
mainfrom
zhongzhu/1bit-pq-kv

Conversation

@zzz0906

@zzz0906 zzz0906 commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add compact INT1 and product-quantized K/V storage with per-layer codebooks
  • integrate mixed BF16/quantized allocation, flush, prefill, and decode paths
  • add CUDA Graph-safe Triton attention kernels and focused allocator/kernel coverage
  • exclude experiment logs, generated codebooks, result artifacts, and the obsolete ternary path

Test plan

  • Ruff syntax/undefined-name checks (E9, F821)
  • Compile all changed Python files
  • Source snapshot H100 validation: 43 tests plus 2 parameterized subtests; Qwen3-8B CUDA Graph batch 1/2/4
  • Re-run the GPU suite in CI or a CUDA environment (the PR host has no PyTorch/CUDA)

Implement compact INT1/PQ storage and mixed-window prefill, flush, and decode paths with CUDA Graph-safe Triton coverage.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant