DevVault AI

DevVault AI is a local source-cited RAG backend for technical documents. It supports document ingestion, batch embeddings, hybrid retrieval, rank fusion, CrossEncoder reranking, relevance filtering, cited chat responses, and Server-Sent Events streaming.

The project is built as a learning and portfolio backend that keeps the retrieval pipeline explicit instead of hiding the core RAG behavior behind a high-level framework. It is designed to demonstrate practical backend architecture, database-backed retrieval, local AI serving, and inspectable RAG quality signals on a developer machine.

Highlights

• FastAPI backend with SQLAlchemy 2 and PostgreSQL persistence
• pgvector VECTOR(768) embedding storage with HNSW cosine-distance indexing
• Batch embedding generation through local Ollama embeddinggemma
• Hybrid retrieval with pgvector semantic search and PostgreSQL full-text search
• Reciprocal Rank Fusion for combining vector and lexical candidate rankings
• Local sentence-transformers CrossEncoder reranking
• Configurable reranker relevance threshold for coarse context-quality filtering
• Optional document_ids scoping for search and chat requests
• Source-cited /chat endpoint returning JSON answers and citations
• SSE-based /chat/stream endpoint with token, citations, done, and error events
• Lightweight structured logs for upload, retrieval, chat, and streaming latency
• Alembic migrations and Docker Compose local PostgreSQL infrastructure

Features

Document Ingestion

• Upload Markdown and plain text files through POST /documents/upload
• Validate filename, extension, UTF-8 decoding, empty content, and chunk creation
• Split uploaded text into overlapping character-based chunks
• Generate all chunk embeddings for an uploaded document in one Ollama batch request
• Validate chunk-to-embedding count alignment before persistence
• Store document metadata and ordered chunks in PostgreSQL

Retrieval Pipeline

• Apply optional document_ids filtering to first-stage retrieval when provided
• Embed the user query with the same Ollama embedding model used for document chunks
• Retrieve semantic candidates from pgvector using cosine distance
• Use an HNSW index on document_chunks.embedding with vector_cosine_ops
• Retrieve lexical candidates with PostgreSQL to_tsvector, plainto_tsquery, and ts_rank_cd
• Merge vector and lexical candidates by chunk_id
• Apply Reciprocal Rank Fusion to produce an intermediate hybrid ranking
• Rerank the strongest fused candidates with a local CrossEncoder
• Apply a configurable reranker relevance threshold before returning final context
• Return retrieval metadata including vector rank, lexical rank, RRF score, rerank score, and rerank rank

Chat and Citations

• Answer questions through POST /chat using retrieved document context
• Return a fallback answer when no relevant context survives retrieval and thresholding
• Include citation metadata for each source chunk used by the answer
• Preserve chunk_id, document_id, filename, and content snippets in citations
• Support optional document-scoped chat with document_ids

Streaming

• Stream answers through POST /chat/stream using Server-Sent Events
• Complete retrieval before streaming starts so the database session is not held during generation
• Emit generated text through token events
• Emit citations after text generation completes
• Emit a done event on successful completion
• Emit an error event if generation fails after the stream has started

Observability

• Configure an application logger under the app namespace
• Emit structured JSON payloads inside log messages
• Measure upload timing for chunking, embedding, persistence, and total request time
• Measure retrieval timing for query embedding, vector retrieval, lexical retrieval, fusion, reranking, and total retrieval time
• Measure chat generation time and final context count
• Measure streaming time to first token, generation time, total stream time, and final context count

Document Management

• List uploaded documents with chunk counts
• Retrieve one document summary by ID
• Delete a document and cascade-delete its chunks
• Keep chunk ordering stable with a (document_id, chunk_index) uniqueness constraint

Tech Stack

Area	Technology
Language	Python
API framework	FastAPI
Database	PostgreSQL
Vector storage/search	pgvector with HNSW indexing
ORM	SQLAlchemy 2
Migrations	Alembic
Validation	Pydantic
Configuration	Pydantic Settings
Embeddings	Ollama embeddinggemma
Chat generation	Ollama llama3.2
Reranking	sentence-transformers CrossEncoder
Local infrastructure	Docker Compose

Architecture

Upload flow:

Document upload
  -> filename and extension validation
  -> UTF-8 decoding
  -> character-based chunking
  -> batch embedding generation through Ollama
  -> document row
  -> ordered chunk rows with VECTOR(768) embeddings
  -> PostgreSQL + pgvector

Query flow:

Question
  -> optional document_ids scope
  -> query embedding
  -> vector retrieval with pgvector/HNSW
  -> lexical retrieval with PostgreSQL full-text search
  -> candidate merge by chunk_id
  -> Reciprocal Rank Fusion
  -> CrossEncoder reranking
  -> relevance threshold
  -> final context
  -> Ollama generation
  -> answer + citations

Streaming chat uses the same retrieval path. After final context and citations are prepared, /chat/stream starts an SSE response, streams generated text as token events, sends citations, and closes with a done event.

Project Structure

.
|-- README.md
|-- docker-compose.yml
|-- backend/
|   |-- .env.example
|   |-- alembic.ini
|   |-- requirements.txt
|   |-- app/
|   |   |-- main.py
|   |   |-- api/routes/       # Health, documents, search, and chat routes
|   |   |-- core/             # Settings and logging configuration
|   |   |-- db/               # SQLAlchemy base, session, and models
|   |   |-- integrations/     # Ollama sync and async clients
|   |   |-- schemas/          # Request and response models
|   |   `-- services/         # Ingestion, embedding, retrieval, reranking, and chat logic
|   |-- migrations/          # Alembic migrations for pgvector, tables, embeddings, and HNSW
|   `-- scripts/             # Small local smoke-test scripts

API Overview

Method	Endpoint	Purpose
GET	/	Root welcome response
GET	/health	Application health and runtime metadata
GET	/health/db	Database connectivity check
POST	/documents/upload	Upload a Markdown or TXT document
GET	/documents	List uploaded documents with chunk counts
GET	/documents/{document_id}	Get one document summary
DELETE	/documents/{document_id}	Delete a document and its chunks
POST	/search	Return filtered and reranked document chunks
POST	/chat	Return a source-cited answer as JSON
POST	/chat/stream	Stream a source-cited answer with SSE

Retrieval Response Signals

/search returns final chunks along with the scores and ranks that explain how each result moved through the retrieval pipeline.

Field	Meaning
vector_distance	Cosine distance from pgvector retrieval; lower is closer
vector_rank	Rank assigned by vector retrieval
lexical_score	PostgreSQL full-text relevance score
lexical_rank	Rank assigned by lexical retrieval
rrf_score	Reciprocal Rank Fusion score after combining vector and lexical rankings
rrf_rank	Rank after RRF, before CrossEncoder reranking
rerank_score	CrossEncoder query-chunk relevance score
rerank_rank	Final rank after CrossEncoder reranking

document_ids can be supplied to /search, /chat, and /chat/stream to restrict retrieval to selected uploaded documents.

Database Design

Table	Responsibility
documents	Uploaded document metadata, content type, source type, and creation time
document_chunks	Ordered chunk text, parent document reference, creation time, and VECTOR(768) embedding

Important schema choices:

• document_chunks.document_id uses ondelete="CASCADE" so deleting a document removes its chunks.
• uq_document_chunks_document_id_chunk_index prevents duplicate chunk indexes within the same document.
• ix_document_chunks_document_id supports document-scoped chunk lookups.
• ix_document_chunks_embedding_hnsw supports pgvector HNSW cosine-distance retrieval.
• The embedding column is nullable at the database level, but upload ingestion stores embeddings for generated chunks.

Local Development

1. Start PostgreSQL

Run from the repository root:

docker compose up -d postgres

The local PostgreSQL container listens on host port 5433.

2. Prepare the Backend Environment

Run from the backend/ folder:

python -m venv .venv
.\.venv\Scripts\python.exe -m pip install -r requirements.txt
Copy-Item .env.example .env

Update backend/.env if your local database or Ollama settings differ. For the current project corpus, set RERANKER_RELEVANCE_THRESHOLD=-8.0.

3. Prepare Ollama Models

Make sure Ollama is running, then pull the required local models:

ollama pull embeddinggemma
ollama pull llama3.2

The CrossEncoder reranker is loaded through sentence-transformers and may download on first use.

4. Apply Migrations

Run from the backend/ folder:

.\.venv\Scripts\python.exe -m alembic upgrade head

The migrations enable pgvector, create the document tables, add the embedding column, and create the HNSW vector index.

5. Run the API

Run from the backend/ folder:

.\.venv\Scripts\python.exe -m uvicorn app.main:app --reload

The API runs at:

http://127.0.0.1:8000

OpenAPI UI:

http://127.0.0.1:8000/docs

Environment Variables

Variable	Purpose
APP_NAME	FastAPI application title
APP_ENV	Runtime environment label
DATABASE_URL	PostgreSQL connection string
OLLAMA_BASE_URL	Ollama server URL
OLLAMA_EMBED_MODEL	Ollama model used for document and query embeddings
OLLAMA_CHAT_MODEL	Ollama model used for answer generation
RERANKER_MODEL	CrossEncoder model used for reranking
RERANKER_RELEVANCE_THRESHOLD	Minimum reranker score required for final context inclusion

Example local values:

APP_NAME=DevVault AI
APP_ENV=local
DATABASE_URL=postgresql+psycopg://devvault:devvault@localhost:5433/devvault?connect_timeout=5
OLLAMA_BASE_URL=http://localhost:11434
OLLAMA_EMBED_MODEL=embeddinggemma
OLLAMA_CHAT_MODEL=llama3.2
RERANKER_MODEL=cross-encoder/ms-marco-MiniLM-L6-v2
RERANKER_RELEVANCE_THRESHOLD=-8.0

RERANKER_RELEVANCE_THRESHOLD=-8.0 is an empirically chosen starting value for the current local corpus and cross-encoder/ms-marco-MiniLM-L6-v2. It is configurable and should not be treated as a universal threshold.

Useful Commands

Command	Purpose
docker compose up -d postgres	Start local PostgreSQL
.\.venv\Scripts\python.exe -m pip install -r requirements.txt	Install backend dependencies
.\.venv\Scripts\python.exe -m alembic upgrade head	Apply database migrations
.\.venv\Scripts\python.exe -m uvicorn app.main:app --reload	Run the API locally
ollama pull embeddinggemma	Pull the embedding model
ollama pull llama3.2	Pull the chat model

Verification and Debugging

Check pgvector extension:

SELECT extname
FROM pg_extension
WHERE extname = 'vector';

Check stored chunks and embeddings:

SELECT
  document_id,
  chunk_index,
  embedding IS NOT NULL AS has_embedding
FROM document_chunks
ORDER BY document_id, chunk_index;

Check embedding dimensions:

SELECT
  id,
  vector_dims(embedding) AS dimensions
FROM document_chunks
WHERE embedding IS NOT NULL;

Check the HNSW vector index:

SELECT
  indexname,
  indexdef
FROM pg_indexes
WHERE tablename = 'document_chunks'
  AND indexname = 'ix_document_chunks_embedding_hnsw';

Expected embedding dimension:

768

Observability

The backend emits lightweight structured log events to standard output. These logs are useful for local inspection and debugging; they are not a full tracing, metrics, or monitoring stack.

Event	Measures
rag_upload_metrics	Chunking latency, embedding latency, persistence latency, total upload time, chunk count
rag_retrieval_metrics	Query embedding latency, vector retrieval latency, lexical retrieval latency, fusion latency, reranking latency, total retrieval time, candidate count, final context count
rag_chat_metrics	Non-streaming generation time, total chat time, final context count
rag_stream_chat_metrics	Time to first token, streaming generation time, total stream time, final context count

License

This project is developed for educational and portfolio purposes.