Discover, manage, download, and chat with GGUF models with unparalleled elegance.
A beautiful, robust desktop-class web application for Hugging Face.
Discover Workspace: Quickly search Hugging Face for GGUF models directly within the application.
Downloads Workspace: Manage your active downloads and view locally downloaded models in a beautiful grid layout.
Chat Workspace: Seamlessly converse with your locally downloaded GGUF models using a beautiful, built-in chat interface with Markdown support.
Local Inference: Run AI models completely locally with complete privacy, leveraging llama-cpp-python.
Zero-Config GPU Acceleration: Auto-detects and compiles for NVIDIA (CUDA), Apple Silicon (Metal), Intel (SYCL/Vulkan), and AMD (ROCm).
Robust Downloading: Reliable downloading mechanism that uses chunks and supports pausing, resuming, and safe cancellation.
Native OS Integration: Instantly open the containing folder of any downloaded model in your native file explorer (Windows).
Beautiful Glassmorphism UI: A dark-mode, minimalist interface designed for speed, aesthetics, and premium user experience.
- Python 3.8+ installed on your system.
- Git (optional, for cloning).
-
Clone the repository:
git clone https://github.com/gowtham2thrive/Hive.git cd hive -
Run the start script (Windows):
start.bat
This script will automatically:
- Set up a Python virtual environment (
venv). - Install the required dependencies (
FastAPI,llama-cpp-python,aiosqlite,huggingface_hub, etc.). - Start the backend server on port
8080. - Open the app in your default web browser.
- Set up a Python virtual environment (
The application is built using a modern yet lightweight tech stack:
- Frontend: Vanilla HTML, CSS, JavaScript (No frameworks!) with Feather Icons.
- Backend: Python, FastAPI, Uvicorn, llama-cpp-python, SQLite.
Hive Chat lets you have conversations with your locally downloaded GGUF models — entirely on your machine, with zero data leaving your device.
- Navigate to the Chat tab from the sidebar
- Open Settings (
) → Select your .ggufmodel from the dropdown → Click Load Model - Start chatting — responses stream in real-time with full markdown rendering
| Component | Technology | Purpose |
|---|---|---|
| Inference Engine | llama-cpp-python |
Loads and runs GGUF models locally with CPU auto-tuning |
| Streaming | WebSocket | Real-time token-by-token response streaming with micro-batching |
| Persistence | SQLite (async, WAL) | Stores conversations and messages across sessions |
| Markdown | Custom renderer | Syntax highlighting, code blocks, tables, copy-to-clipboard |
| API | FastAPI REST + WS | Conversation CRUD, model management, streaming chat |
- Real-time streaming — Tokens stream as they're generated with adaptive render intervals
- Conversation management — Create, rename, search, and delete conversations
- Model management — Load/unload models, browse for
.gguffiles via native OS file picker - Markdown rendering — Code blocks with syntax highlighting (Python, JS, CSS, etc.), tables, lists, blockquotes
- Context truncation — Automatically trims old messages to fit the model's context window
- Cancel generation — Stop responses mid-stream with visual feedback
- Creativity slider — Adjustable temperature from Precise (0.0) to Creative (2.0) with dynamic labels
- Auto-scroll — Smart scrolling that pauses when you scroll up to read
- Inactivity watchdog — Auto-resets if no tokens arrive for 20 seconds
Hive features an advanced, zero-configuration GPU detection and optimization system. Upon the first launch, start.bat will probe your hardware and automatically install the optimal backend for llama-cpp-python.
Depending on your hardware, you may need to install standard drivers or toolkits to unlock full GPU offloading. Hive will attempt to build and run regardless, falling back to CPU mode if necessary.
| Platform / GPU | What to Install | Backend Used |
|---|---|---|
| NVIDIA (RTX, GTX) | CUDA Toolkit 12.x or 11.x | cuda (Pre-built wheel) |
| Apple Silicon (M1/M2/M3) | Xcode Command Line Tools (xcode-select --install) |
metal (Pre-built wheel) |
| Intel Arc / iGPU | Intel oneAPI Base Toolkit | sycl (Source build) |
| AMD Radeon | ROCm SDK | rocm (Source build) |
| Generic / Cross-platform | Vulkan SDK + CMake | vulkan (Source build) |
If you update your drivers or install a new toolkit, you can trigger a re-detection directly from the Settings panel in the Chat interface by clicking Rebuild GPU Backend, or by deleting the .gpu_backend file in the project root.
This project is licensed under the MIT License. See the LICENSE file for more details.