LayoutVLM-Lab

English | 简体中文

LayoutVLM-Lab is a modular document parsing framework designed to evaluate and compare your own VLMs on real document layouts. It lets you swap Layout and VLM components, run them at scale, and export consistent outputs for objective assessment.

Why LayoutVLM-Lab

• Built for benchmarking self-trained VLMs on document parsing tasks
• Interchangeable Layout/VLM implementations via registries
• Unified outputs (Markdown + JSON) for reliable comparisons
• A high-concurrency pipeline for batch processing

Quickstart

pip install -e .
cp .env.example .env
python run.py -c config/paddle_layoutv2.yaml

Output Structure

Default output is ./results/<experiment.name>/:

• *.md Markdown (saved directly in this folder)
• json/*.json structured results
• imgs/* cropped images (optional)

Installation

Core dependencies only (no Paddle):

pip install -e .

Install PP-DocLayoutV2 example dependencies (optional):

pip install paddlepaddle-gpu==3.2.0 -i https://www.paddlepaddle.org.cn/packages/stable/cu126/
pip install -U "paddleocr[doc-parser]"

Note

1. paddlepaddle-gpu requires the official CUDA wheel index (example for CUDA 12.6)

2. Paddle and vLLM can conflict when installed in the same environment. Install vLLM in a separate environment, then deploy your VLM model from that environment.

vLLM Serve

Before running, deploy the model first and ensure the model path is accessible.

CUDA_VISIBLE_DEVICES=0 \
  vllm serve Qwen/Qwen3-VL-4B-Instruct \
  --served-model-name Qwen3-VL-4B \
  --port 8000 \
  --dtype bfloat16 \
  --tensor-parallel-size 1 \
  --pipeline_parallel_size 1 \
  --data-parallel-size 1 \
  --max_model_len 22000 \
  --max_num_batched_tokens 8192 \
  --enable_chunked_prefill \
  --gpu-memory-utilization 0.9

Configuration

Copy and edit environment variables:

cp .env.example .env

Example config: config/paddle_layoutv2.yaml. Key sections:

• experiment / output: run naming and output base directory
• input: image folder and filtering behavior
• pipeline: concurrency settings
• layout / vlm: backend selection and model settings
• Full reference: docs/config_reference.md

Documentation

• Custom layout tutorial
• Configuration reference

Architecture Overview

OmniDocBench Results

Our implementation (PP-DocLayoutV2 + Qwen3-VL)

Model	Overall ↑	Text Edit ↓	Formula CDM ↑	Table TEDS ↑	Table TEDS-S ↑	Read Order Edit ↓
Qwen3-VL-4B	90.850	0.039	91.130	85.320	89.812	0.041

Leaderboard (OmniDocBench v1.5)

Model Type	Methods	Size	Overall ↑	Text Edit ↓	Formula CDM ↑	Table TEDS ↑	Table TEDS-S ↑	Read Order Edit ↓
Specialized VLMs	PaddleOCR-VL	0.9B	92.86	0.035	91.22	90.89	94.76	0.043
	MinerU2.5	1.2B	90.67	0.047	88.46	88.22	92.38	0.044
	MonkeyOCR-pro-3B	3B	88.85	0.075	87.25	86.78	90.63	0.128
	OCRVerse	4B	88.56	0.058	86.91	84.55	88.45	0.071
General VLMs	Qwen3-VL-235B-A22B-Instruct	235B	89.15	0.069	88.14	86.21	90.55	0.068
	Gemini-2.5 Pro	-	88.03	0.075	85.82	85.71	90.29	0.097
	Qwen2.5-VL	72B	87.02	0.094	88.27	82.15	86.22	0.102
Pipeline Tools	PP-StructureV3	-	86.73	0.073	85.79	81.68	89.48	0.073
	Mineru2-pipeline	-	75.51	0.209	76.55	70.90	79.11	0.225

Source: OmniDocBench official leaderboard — GitHub repository.

Extensibility

• Register new implementations in LAYOUT_REGISTRY / VLM_REGISTRY
• PP-DocLayoutV2 is an example implementation, not a required dependency

TODO

• Add non-OpenAI VLM adapters (local models, other APIs)
• Add PDF input support (PDF -> images pipeline)