gpu-memory

A fast GPU memory copy library based on NVIDIA GPUDirect RDMA technology

Thin, unified, C++-flavored wrappers for the CUDA APIs

Training neural networks in TensorFlow 2.0 with 5x less memory

A Toolkit for Training, Tracking, Saving Models and Syncing Results

A memory profiler for NVIDIA GPUs to explore memory inefficiencies in GPU-accelerated applications.

OpenCV & Spout C++ library. Shared GPU memory and processing at reach.

Rust embedded things running on the seL4 microkernel for the Raspberry Pi 3

A simple tool to find out GPU VRAM requirements for running LLMs

Python library for memory-constrained activation checkpoint optimization, recomputation scheduling, and GPU training performance analysis.

A tiny, useful command-line tool to show each user gpu usage, pid under each gpu, provide more details than nvidia-smi/gpustat

Demonstration of generating mini-batches in Tensorlfow from GPU memory.

Accurate VRAM calculator for Local LLMs (Llama 4, DeepSeek V3, Qwen 2.5). Calculates GGUF quantization, GQA context overhead, and offloading limits

A prefix-cache advisor for LLM serving infrastructure that recommends KV-cache capacity and eviction policies from your request traces/logs.

A fork of Kubernetes with support of schedulable resource of NVIDIA GPU memory

Dynamic GPU Layer Swapping: Train large models on consumer GPUs with intelligent memory management

Detailed VRAM profiler for transformer inference with per-layer breakdown, activation analysis, and a predictive memory model that predicts VRAM with <1.2% error. Shows that FFN layers dominate static memory and that measured runtime VRAM exceeds KV-cache estimates by 2-4x.

A CLI tool for estimating GPU VRAM requirements for Hugging Face models, supporting various data types, parallelization strategies, and fine-tuning scenarios like LoRA.

Event-driven benchmark of adaptive batch composition policies for LLM serving, measuring how prefill and decode interference affects TTFT, TPOT, and throughput under different memory pressure regimes.