A practical handbook for software engineers to learn AI, Large Language Models (LLMs), and Inference Engineering—from fundamentals to production systems.
-
Updated
Jul 4, 2026
#
inference-engineering
Here are 4 public repositories matching this topic...
Real-time LLM inference benchmarking dashboard: multi-config comparison (TTFT/TBT/throughput), FastAPI backend, async orchestration engine. 21/21 tests passing (100%).
python docker benchmarking docker-compose gpu websocket grafana prometheus pytest throughput observability mlops github-actions fastapi ttft llm vllm llm-inference mi300x inference-engineering
-
Updated
Jul 14, 2026 - HTML
Deep Agents and SvelteKit harness for authoring verifier-gated Bonsai workflow packs.
-
Updated
Jun 27, 2026 - TypeScript
Rust + cuTile research prototype for paged latent-cache LLM decode attention, validated on an RTX 4060.
rust gpu cuda attention kv-cache ml-systems llm-inference paged-attention inference-engineering cutile
-
Updated
Jul 14, 2026 - Rust
Improve this page
Add a description, image, and links to the inference-engineering topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the inference-engineering topic, visit your repo's landing page and select "manage topics."