addressed issue 61

CONTRIBUTING.md

@@ -58,6 +58,16 @@ the README badge always reflects the real state of `main`. The Go toolchain
 version comes from `go.mod` via `go-version-file`, so bumping the `go` directive
 is all it takes to move CI with it.
 
+The RAG store tests skip unless there is a database to run against. If you are
+touching `internal/rag`, give them one — they are what keeps the schema in this
+repo and the one `streamcore-cli` writes from drifting apart:
+
+```bash
+docker run -d --name pgvector-test -e POSTGRES_PASSWORD=test -e POSTGRES_DB=ragtest \
+    -p 55433:5432 pgvector/pgvector:pg16
+STREAMCORE_TEST_PG=postgres://postgres:test@localhost:55433/ragtest go test -race ./internal/rag/
+```
+
 Formatting and vet failures are the single most common reason a PR sits.
 
 ## Where things live

config.toml.example

@@ -209,6 +209,9 @@ tts_url = ""            # Optional; defaults to wss://openspeech.bytedance.com/a
 provider = "supabase"   # or "pgvector", or omit entirely to disable
 top_k = 3
 embedding_model = "text-embedding-3-small"  # optional, this is the default
+# The model fixes the column width: text-embedding-3-small and -ada-002 need
+# vector(1536), text-embedding-3-large needs vector(3072). The server checks
+# this against the store at startup and refuses to boot on a mismatch.
 
 [supabase]
 url = "https://xxx.supabase.co"

docs/agent-runtime.md

@@ -78,11 +78,17 @@ CREATE EXTENSION IF NOT EXISTS vector;
 CREATE TABLE documents (
     id SERIAL PRIMARY KEY,
     content TEXT NOT NULL,
-    embedding vector(1536),
-    source TEXT
+    embedding vector(1536) NOT NULL,
+    embedding_model TEXT NOT NULL,
+    source TEXT,
+    created_at TIMESTAMP DEFAULT NOW()
 );
+
+CREATE INDEX IF NOT EXISTS documents_embedding_model_idx ON documents (embedding_model);
 ```
 
+`streamcore-cli setup` creates all of this for you; the SQL is here for anyone who would rather run it themselves.
+
 ```toml
 [rag]
 provider = "pgvector"
@@ -99,11 +105,14 @@ CREATE EXTENSION IF NOT EXISTS vector;
 CREATE TABLE documents (
     id SERIAL PRIMARY KEY,
     content TEXT NOT NULL,
-    embedding vector(1536),
+    embedding vector(1536) NOT NULL,
+    embedding_model TEXT NOT NULL,
     source TEXT,
     created_at TIMESTAMP DEFAULT NOW()
 );
 
+CREATE INDEX IF NOT EXISTS documents_embedding_model_idx ON documents (embedding_model);
+
 CREATE OR REPLACE FUNCTION match_documents(
     query_embedding vector(1536),
     match_count int DEFAULT 3
@@ -142,10 +151,37 @@ function = "match_documents"
 table = "documents"
 ```
 
+### Embedding model and vector width
+
+Every row records the model that embedded it, and the vector column is sized for that model. Both halves are checked when the server boots, and a store it cannot use is a startup failure rather than retrieval that quietly returns the wrong chunks.
+
+| `embedding_model` | Column type |
+|---|---|
+| `text-embedding-3-small` (default) | `vector(1536)` |
+| `text-embedding-ada-002` | `vector(1536)` |
+| `text-embedding-3-large` | `vector(3072)` |
+
+The width alone is not enough. `ada-002` and `3-small` are both 1536 wide, so vectors written by one and searched with the other produce no error at all — just answers drawn from the wrong chunks. That is what `embedding_model` is for: the server looks for a single row disagreeing with `rag.embedding_model` and refuses to start if it finds one.
+
+For `pgvector` the width comes from the column definition and the model check is one indexed query. For Supabase there is no catalog to read over PostgREST, so both come from rows: an empty table stays unverified until something has been ingested, and a project that is unreachable at boot logs a warning instead of blocking startup.
+
+If you ingested before `embedding_model` existed, the server will tell you so and print this migration:
+
+```sql
+ALTER TABLE documents ADD COLUMN embedding_model TEXT;
+UPDATE documents SET embedding_model = '<the model you ingested with>';
+ALTER TABLE documents ALTER COLUMN embedding_model SET NOT NULL;
+CREATE INDEX IF NOT EXISTS documents_embedding_model_idx ON documents (embedding_model);
+```
+
+Only you know which model those rows came from, which is why the backfill is a placeholder. If you no longer know, re-ingest.
+
 ### Ingesting documents
 
 The server handles query-time retrieval only. Populate your vector store with [`streamcore-cli`](https://github.com/streamcoreai/streamcore-cli), a separate Go binary. It stays separate so the PDF, docx and xlsx parsers never end up in the server image.
 
+Prebuilt binaries for macOS and Linux, both architectures, are on the [releases page](https://github.com/streamcoreai/streamcore-cli/releases). With a Go toolchain:
+
 ```bash
 go install github.com/streamcoreai/streamcore-cli@latest
 
@@ -154,7 +190,9 @@ git clone https://github.com/streamcoreai/streamcore-cli
 cd streamcore-cli && go build -o streamcore-cli .
 ```
 
-`streamcore-cli setup` asks for your provider, OpenAI key and credentials, then writes `~/.streamcore/config.toml`. If you already have a server `config.toml`, skip it — the CLI reads the same format and falls back to the server's file, so nothing is configured twice.
+`streamcore-cli setup` asks for your provider, OpenAI key and credentials, writes `~/.streamcore/config.toml`, and creates the table with the vector width your chosen model needs. For Supabase it asks for the project's direct Postgres connection string, since PostgREST can insert rows but cannot run DDL; leave that blank and it prints the SQL for the dashboard's editor instead.
+
+If you already have a server `config.toml`, the CLI reads the same format and falls back to the server's file, so credentials are never configured twice.
 
 ```bash
 streamcore-cli setup
@@ -174,6 +212,43 @@ Config is looked up in order: `--config`, `~/.streamcore/config.toml`, `./config
 | `--chunk-size` | 512 | Target chunk size in words |
 | `--chunk-overlap` | 64 | Overlap between chunks in words |
 
-Ingest and query must use the same `embedding_model`. Vectors written by one model and searched by another are not comparable, and the result is bad retrieval rather than an error — so if you change the model, re-ingest.
+Ingest and query must use the same `embedding_model` — see [Embedding model and vector width](#embedding-model-and-vector-width) for which column each model needs. `ingest` checks the store against the configured model before it writes anything, so a mismatch costs one query rather than a table full of unusable vectors.
 
 Full command reference, supported formats and database DDL: [streamcore-cli README](https://github.com/streamcoreai/streamcore-cli#readme).
+
+### End to end
+
+Pick something the model cannot already know. A PDF of your own release notes works; so does a text file you write on the spot.
+
+```bash
+cat > closing-hours.md <<'EOF'
+The Wellington workshop closes at 3pm on the last Friday of every month
+for maintenance. All other Fridays it closes at 6pm.
+EOF
+
+streamcore-cli setup                    # provider, key, model, and the table
+streamcore-cli ingest closing-hours.md
+```
+
+```
+Using config: /Users/you/.streamcore/config.toml
+Processing closing-hours.md ...
+  Extracted 1 chunks
+  Uploaded 1/1 chunks
+Done. 1 chunks uploaded to supabase (text-embedding-3-small).
+```
+
+Point the server at the same config and start it. It reads the table at boot and says nothing if the contract holds:
+
+```
+RAG enabled — provider: supabase
+Voice agent server listening on :8080
+```
+
+Then connect a client and ask *"when does the Wellington workshop close on the last Friday of the month?"* The answer should be 3pm. Ask before ingesting, or against a store built with a different model, and you get a generic non-answer instead — which is the failure this contract exists to make loud.
+
+### Why ingestion is a separate binary
+
+The parsers are the reason. PDF, docx and xlsx pull in dependencies that the server has no use for at query time, and the server image is something people deploy; the ingestion tool is something they run once from a laptop. Keeping them apart means the server image never carries a document parser it will never call.
+
+The cost is a second artifact with its own config, which is why the CLI reads the server's `config.toml` and falls back to it — one set of credentials, two binaries. If that stops holding, the argument for the split is worth revisiting.

docs/agent-runtime.zh-CN.md

@@ -78,11 +78,17 @@ CREATE EXTENSION IF NOT EXISTS vector;
 CREATE TABLE documents (
     id SERIAL PRIMARY KEY,
     content TEXT NOT NULL,
-    embedding vector(1536),
-    source TEXT
+    embedding vector(1536) NOT NULL,
+    embedding_model TEXT NOT NULL,
+    source TEXT,
+    created_at TIMESTAMP DEFAULT NOW()
 );
+
+CREATE INDEX IF NOT EXISTS documents_embedding_model_idx ON documents (embedding_model);
 ```
 
+`streamcore-cli setup` 会替你建好这一切；这段 SQL 是留给想自己动手的人的。
+
 ```toml
 [rag]
 provider = "pgvector"
@@ -99,11 +105,14 @@ CREATE EXTENSION IF NOT EXISTS vector;
 CREATE TABLE documents (
     id SERIAL PRIMARY KEY,
     content TEXT NOT NULL,
-    embedding vector(1536),
+    embedding vector(1536) NOT NULL,
+    embedding_model TEXT NOT NULL,
     source TEXT,
     created_at TIMESTAMP DEFAULT NOW()
 );
 
+CREATE INDEX IF NOT EXISTS documents_embedding_model_idx ON documents (embedding_model);
+
 CREATE OR REPLACE FUNCTION match_documents(
     query_embedding vector(1536),
     match_count int DEFAULT 3
@@ -142,10 +151,37 @@ function = "match_documents"
 table = "documents"
 ```
 
+### 嵌入模型与向量宽度
+
+每一行都记录了给它做 embedding 的模型，而向量列的宽度也是按那个模型定的。服务端启动时两边都会检查，用不了的向量库会直接导致启动失败，而不是让检索悄悄返回错误的片段。
+
+| `embedding_model` | 列类型 |
+|---|---|
+| `text-embedding-3-small`（默认） | `vector(1536)` |
+| `text-embedding-ada-002` | `vector(1536)` |
+| `text-embedding-3-large` | `vector(3072)` |
+
+光看宽度是不够的。`ada-002` 和 `3-small` 都是 1536 维，用一个写入、用另一个检索，不会报任何错，只会从错误的片段里给出答案。`embedding_model` 就是为此存在的：服务端会去找一行与 `rag.embedding_model` 不一致的记录，找到就拒绝启动。
+
+`pgvector` 的宽度取自列定义，模型检查是一次走索引的查询。Supabase 那边没有可以通过 PostgREST 读到的系统目录，所以两项都靠取行来判断：空表在入库之前无法校验，启动时连不上的项目只记一条警告，不会阻塞启动。
+
+如果你是在 `embedding_model` 这一列出现之前入的库，服务端会告诉你，并打印出这段迁移 SQL：
+
+```sql
+ALTER TABLE documents ADD COLUMN embedding_model TEXT;
+UPDATE documents SET embedding_model = '<你当初入库用的模型>';
+ALTER TABLE documents ALTER COLUMN embedding_model SET NOT NULL;
+CREATE INDEX IF NOT EXISTS documents_embedding_model_idx ON documents (embedding_model);
+```
+
+那些行到底来自哪个模型，只有你自己知道，所以回填的值留成了占位符。要是已经想不起来了，就重新入库一遍。
+
 ### 文档入库
 
 服务端只负责查询时的检索。向量库的内容由 [`streamcore-cli`](https://github.com/streamcoreai/streamcore-cli) 填充 —— 那是一个独立的 Go 二进制。之所以独立，是为了让 PDF、docx、xlsx 的解析依赖不会进到服务端镜像里。
 
+macOS 和 Linux 两种架构的预编译二进制都在[发布页](https://github.com/streamcoreai/streamcore-cli/releases)。如果本机有 Go 工具链：
+
 ```bash
 go install github.com/streamcoreai/streamcore-cli@latest
 
@@ -154,7 +190,9 @@ git clone https://github.com/streamcoreai/streamcore-cli
 cd streamcore-cli && go build -o streamcore-cli .
 ```
 
-`streamcore-cli setup` 会依次询问服务商、OpenAI key 和凭据，然后写入 `~/.streamcore/config.toml`。如果你已经有服务端的 `config.toml`，可以跳过这一步 —— CLI 读的是同一种格式，会回退到服务端那份文件，因此没有任何东西需要配置两遍。
+`streamcore-cli setup` 会依次询问服务商、OpenAI key 和凭据，写入 `~/.streamcore/config.toml`，并按你选的模型所需的向量宽度把表建好。Supabase 会额外问一个项目的 Postgres 直连串 —— PostgREST 能插入数据，但跑不了 DDL；这一项留空，它就把 SQL 打印出来给你贴到控制台的 SQL 编辑器里。
+
+如果你已经有服务端的 `config.toml`，CLI 读的是同一种格式，会回退到服务端那份文件，因此凭据不需要配置两遍。
 
 ```bash
 streamcore-cli setup
@@ -174,6 +212,43 @@ streamcore-cli ingest --chunk-size 256 --chunk-overlap 32 manual.docx
 | `--chunk-size` | 512 | 目标分块大小（词数） |
 | `--chunk-overlap` | 64 | 分块之间的重叠（词数） |
 
-入库和查询必须使用同一个 `embedding_model`。用一个模型写入、用另一个模型检索出来的向量之间没有可比性，而且不会报错，只会让召回变差 —— 所以换了模型就重新入库一遍。
+入库和查询必须使用同一个 `embedding_model` —— 每个模型对应哪种列，见[嵌入模型与向量宽度](#嵌入模型与向量宽度)。`ingest` 在写入任何数据之前，会先拿配置里的模型去校验向量库，所以对不上的代价是一次查询，而不是一整张写满了用不了的向量的表。
 
 完整的命令说明、支持的格式和建表 SQL：[streamcore-cli README](https://github.com/streamcoreai/streamcore-cli/blob/main/README.zh-CN.md)。
+
+### 端到端跑一遍
+
+挑一份模型不可能已经知道的内容。你自己的发布说明 PDF 可以，现写一个文本文件也可以。
+
+```bash
+cat > closing-hours.md <<'EOF'
+惠灵顿工坊每月最后一个周五下午 3 点闭店做维护。
+其余的周五都是 6 点闭店。
+EOF
+
+streamcore-cli setup                    # 服务商、key、模型，以及建表
+streamcore-cli ingest closing-hours.md
+```
+
+```
+Using config: /Users/you/.streamcore/config.toml
+Processing closing-hours.md ...
+  Extracted 1 chunks
+  Uploaded 1/1 chunks
+Done. 1 chunks uploaded to supabase (text-embedding-3-small).
+```
+
+让服务端指向同一份配置并启动。它会在启动时读一遍这张表，契约没问题就什么都不说：
+
+```
+RAG enabled — provider: supabase
+Voice agent server listening on :8080
+```
+
+然后接一个客户端上去，问「惠灵顿工坊每月最后一个周五几点闭店？」，答案应该是下午 3 点。入库之前问，或者对着用另一个模型建起来的库问，得到的就是一句没有信息量的场面话 —— 而这正是这套契约要让它变响的那种失败。
+
+### 为什么入库是一个单独的二进制
+
+原因在解析器。PDF、docx、xlsx 会拖进一堆依赖，而服务端在查询时根本用不上它们；服务端镜像是要部署出去的东西，入库工具则是在自己笔记本上跑一次的东西。分开之后，服务端镜像里就不会带着一个永远不会被调用的文档解析器。
+
+代价是多了一个带自己配置的产物 —— 所以 CLI 会去读服务端的 `config.toml` 并回退到它：一份凭据，两个二进制。哪天这一点不再成立了，这个拆分就值得重新讨论。

docs/roadmap.md

@@ -33,7 +33,7 @@ Want one of these? Say so in [Discord](https://discord.gg/xKGFaGWawT) or open an
 
 - [ ] **Broader examples** proving the positioning: realtime translator, AI-hosted voice room, browser copilot, embedded device, SIP application, and a raw audio-processing app with no LLM at all.
 - [ ] **Embedded client hardening.** The ESP32-S3 firmware in [`esp32`](https://github.com/streamcoreai/esp32) connects over WHIP but is not production-ready.
-- [x] **`streamcore-cli` release.** Shipped as [`streamcoreai/streamcore-cli`](https://github.com/streamcoreai/streamcore-cli) — RAG document ingestion, see [Agent runtime → Ingesting documents](./agent-runtime.md#ingesting-documents). Tagged releases with prebuilt binaries remain open.
+- [x] **`streamcore-cli` release.** Shipped as [`streamcoreai/streamcore-cli`](https://github.com/streamcoreai/streamcore-cli) — RAG document ingestion, see [Agent runtime → Ingesting documents](./agent-runtime.md#ingesting-documents). Tagging a version publishes prebuilt binaries for macOS and Linux on both architectures.
 - [ ] **React Native SDK on npm.** `@streamcore/react-native-sdk` is built and usable from source, but unpublished.
 
 ---

docs/roadmap.zh-CN.md

@@ -33,7 +33,7 @@
 
 - [ ] **更多示例**，以印证产品定位：实时翻译、AI 主持的语音房间、浏览器副驾、嵌入式设备、SIP 应用，以及一个完全不含 LLM 的纯音频处理应用。
 - [ ] **嵌入式客户端加固。** [`esp32`](https://github.com/streamcoreai/esp32) 中的 ESP32-S3 固件已能通过 WHIP 连接，但尚未达到生产可用。
-- [x] **`streamcore-cli` 发布。** 已发布为 [`streamcoreai/streamcore-cli`](https://github.com/streamcoreai/streamcore-cli) —— 提供 RAG 文档入库，见[智能体运行时 → 文档入库](./agent-runtime.zh-CN.md#文档入库)。打 tag 的预编译二进制仍在计划中。
+- [x] **`streamcore-cli` 发布。** 已发布为 [`streamcoreai/streamcore-cli`](https://github.com/streamcoreai/streamcore-cli) —— 提供 RAG 文档入库，见[智能体运行时 → 文档入库](./agent-runtime.zh-CN.md#文档入库)。打一个版本 tag 即会发布 macOS 与 Linux 两种架构的预编译二进制。
 - [ ] **React Native SDK 发布到 npm。** `@streamcore/react-native-sdk` 已完成、可从源码使用，但尚未发布。
 
 ---