aibrix

AI/LLM on Kubernetes 基础设施知识库：涵盖 GPU、大数据、调度、分布式训练、LLM 推理、RAG、Agent 与生产运维。| AI/LLM on Kubernetes Infrastructure Knowledge Base: Covering GPUs, big data, scheduling, distributed training, LLM inference, RAG, agents, and production operations.

Experimental hybrid KV offloading with variable-length records, bounded MEMORY storage and a verifiable CPU byte path. Model equivalence remains unresolved.

A comprehensive guide and set of Kubernetes manifests for serving LLMs using vLLM and managing them with the Envoy AI Gateway.