fable-5's improvements to the CIFAR-10 speedrun over the hiverge baseline, measured
-
Updated
Jul 6, 2026 - Python
#
cuda-graphs
Here are 17 public repositories matching this topic...
Optimized CSM-1B TTS pipeline for RTX 5090 (Blackwell sm_120). CUDA graph replay via patched HF Transformers. ~0.46x RTF. Topics (tags): csm text-to-speech rtx-5090 blackwell cuda-graphs torch-compile sesame streaming pytorch
text-to-speech streaming pytorch tts sesame csm huggingface blackwell torch-compile rtx-5090 sm-120 cuda-graphs
-
Updated
Apr 5, 2026 - Python
Three-tier GPU model state management — hold many models on one GPU: sub-second swaps via CUDA VMM, deep-freeze to NVMe via CRIU, reboot-proof. Nothing re-loaded, re-warmed, or re-compiled.
gpu cuda inference pytorch gpu-memory criu model-serving vram mlops checkpoint-restore stable-diffusion diffusers torch-compile cuda-graphs model-swapping
-
Updated
Jul 12, 2026 - Python
Measuring what makes a VLA fast enough to run on the robot: a 5.9x CUDA-graph win, four experiments on why low-bit doesn't, a budget-driven deploy-compiler, and a runtime safety supervisor. Live demo: hf.co/spaces/LaelaZ/embodied-efficiency
-
Updated
Jun 7, 2026 - Python
Technical note + runnable demo: cudaMemcpyPeerAsync isn't capturable into a CUDA graph; capture a peer-to-peer copy as a DeviceToDevice UVA memcpy or a peer-access kernel. Confirmed identical on WSL2 and native Linux.
-
Updated
Jun 13, 2026 - Cuda
Silicon phonons with a MACE foundation model, benchmarked across four machines — phono3py displacement force sets as the textbook CUDA-graph workload (capture once, replay 111x; x12 single, x28 batched). Sister project to phosbench.
benchmarking pytorch phonons phonopy gpu-acceleration mace silicon machine-learning-interatomic-potentials cuda-graphs phono3py
-
Updated
Jul 6, 2026 - Python
GB10 inference port; see fork.md
-
Updated
May 14, 2026 - Python
vLLM v0.21 + Qwen 3.6 DFlash + real thinking_token_budget enforcement on Blackwell (sm_120 / sm_121a)
reasoning blackwell fp8 vllm qwen speculative-decoding qwen3 dgx-spark cuda-graphs dflash thinking-budget
-
Updated
May 26, 2026 - Python
Microsecond-scale limit order book inference with custom CUDA/cuTile kernels, TensorRT Plugin V3, CUDA Graphs, and p99 latency benchmarking.
deep-learning cuda pytorch low-latency limit-order-book tensorrt high-frequency-trading onnx fi-2010 gpu-inference nsight-systems nsight-compute cutile cuda-graphs
-
Updated
Jul 14, 2026 - Python
Adaptive CUDA Graph runtime for faster, validated AI inference.
-
Updated
Jun 21, 2026 - Python
Prefill performance study on Qwen2.5-7B using vLLM. Compares static vs mixed (bucketed) prefill under eager execution and CUDA Graphs, with controlled concurrency and real-world latency/throughput metrics.
-
Updated
Feb 10, 2026 - Python
From-scratch, heavily-annotated CUDA inference runtime for Qwen2.5-Coder-7B on H100 (sm_90). Custom INT4 packer, fused GEMV, paged KV, split-KV attention, CUDA graph decode — every hot path commented for the why. Educational, not a llama.cpp replacement.
-
Updated
Jul 14, 2026 - Cuda
From-scratch C++/CUDA LLM inference engine: paged KV cache, continuous batching, CUDA-graph decode. 4,748 tok/s on an RTX 4090 - benchmarked against vLLM and llama.cpp with byte-identical-output gating and fully committed raw data.
benchmark cpp gpu cuda transformer inference-engine vllm llm-inference qwen paged-attention continuous-batching cuda-graphs
-
Updated
Jul 13, 2026 - C++
Three measured notebooks on GPU inference optimization: roofline analysis, KV-cache decode optimization (4.21x), and CUDA-graph launch-overhead elimination (5.38x). Pure PyTorch.
gpu pytorch performance-optimization inference-optimization kv-cache llm-inference torch-compile roofline-analysis cuda-graphs
-
Updated
Jul 7, 2026 - Jupyter Notebook
vLLM v0.21 + DFlash + thinking_token_budget for Gemma 4 & Qwen 3.6 on Blackwell GB10 (sm_121a / sm_120)
gemma reasoning blackwell vllm speculative-decoding nvfp4 dgx-spark cuda-graphs dflash gemma-4 thinking-budget
-
Updated
May 26, 2026 - Python
DiSpec — a from-scratch LLM inference engine: paged attention, continuous batching, CUDA-graph decode, speculative decoding, and prefill/decode disaggregation
cuda inference kv-cache llm-inference speculative-decoding paged-attention cuda-graphs disaggregated-inference prefill-decode continuous-batch
-
Updated
Jun 24, 2026 - Python
PyTorch GPU inference performance exercises: roofline analysis, decode-loop profiling, KV-cache optimization, torch.compile, and CUDA graphs.
performance gpu cuda inference pytorch profiling roofline kv-cache llm-inference h100 torch-compile cuda-graphs
-
Updated
Jun 26, 2026 - Jupyter Notebook
Improve this page
Add a description, image, and links to the cuda-graphs topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the cuda-graphs topic, visit your repo's landing page and select "manage topics."