RAGChunkEval

RAGChunkEval is a document chunk quality evaluation and benchmark toolkit for RAG systems.

It keeps chunking strategies separate from evaluation. The current codebase preserves the retained researched strategies and exposes them through one platform interface:

• sliding-window-merge: structure-aware windowing, semantic merge, optimization, and validation.
• semantic-structure: embedding semantic-boundary detection with document-structure preservation and chunk optimization.

Structure

backend/ragchunk_eval/          Backend package
backend/ragchunk_eval/chunkers/  Document chunking adapters and registry
backend/ragchunk_eval/evaluation/ Chunk quality and retrieval evaluation
backend/ragchunk_eval/api/       FastAPI service for the frontend
frontend/                       Browser workbench
Sliding Window + Merge/         Retained sliding-window + merge research strategy
AntSK-FileChunk_alfix/          Retained semantic-structure research implementation
scripts/                        Offline benchmark runners
data/                           Retrieval evaluation data
test/                           Sample files
tests/                          Platform tests

Generated reports, vectors, model caches, old dashboards, and historical run outputs are intentionally not kept in the repository.

CLI

$env:PYTHONPATH="backend"
python -m ragchunk_eval.cli split --input test\pdf\example.pdf --method auto --output chunks.json
python -m ragchunk_eval.cli evaluate --chunks chunks.json
python -m ragchunk_eval.cli benchmark --input test\pdf\example.pdf
python scripts\run_three_scheme_comparison.py --input test

Supported methods:

• auto
• sliding-window-merge
• semantic-structure

Auto routing:

• PDF: sliding-window-merge
• Word: sliding-window-merge
• TXT/Markdown: sliding-window-merge
• Spreadsheet/CSV: semantic-structure

scripts/run_three_scheme_comparison.py is the mixed-strategy runner. It compares the retained researched strategies through the unified pipeline.

Backend API

$env:PYTHONPATH="backend"
uvicorn ragchunk_eval.api.app:app --reload --host 127.0.0.1 --port 8000

Or from scripts/:

python .\run_api.py --reload

Windows one-click launch:

启动 RAGChunkEval.cmd

Endpoints:

• GET /health
• GET /strategies
• POST /split
• POST /evaluate

Frontend

Open http://127.0.0.1:8000/ after the backend API is running.

Test

pytest tests "Sliding Window + Merge\tests" -q