addressed issue 61

[jason-shen]

This pull request introduces significant improvements to the RAG (Retrieval-Augmented Generation) document ingestion and retrieval system, focusing on enforcing and documenting the contract between embedding models and vector storage, as well as improving developer experience and reliability. The main changes include adding explicit model and vector width checks, updating documentation for both English and Chinese audiences, and providing better guidance for contributors and users.

RAG Model and Vector Dimension Contract

• Added a new internal/rag/dimensions.go module that centralizes knowledge of supported embedding models, their output dimensions, and provides functions for validating vector width and embedding model consistency. This ensures that documents are always ingested and retrieved with compatible models, preventing subtle retrieval errors.
• Updated the SQL schema for the documents table to require the embedding_model column (with NOT NULL constraint), and created an index on it. This makes every row record the model used for embedding, allowing the server to enforce model consistency at startup. [1] [2] [3] [4]
• Documented the embedding model and vector width contract in detail, including migration instructions for existing tables and clear error messaging if mismatches are detected. [1] [2]

Developer and User Documentation Enhancements

• Improved the English and Chinese documentation to explain the importance of matching embedding models and vector widths, including end-to-end ingestion and retrieval examples, migration steps, and rationale for separating the ingestion tool from the server. [1] [2] [3] [4]
• Updated the roadmap to clarify that tagging a streamcore-cli release now publishes prebuilt binaries for both macOS and Linux architectures, improving accessibility for end users. [1] [2]
• Added a note to CONTRIBUTING.md to guide contributors on running RAG store tests with a local Postgres database, ensuring schema consistency between repositories.
• Clarified configuration options and the relationship between embedding models and vector column types in config.toml.example.

These changes collectively improve the robustness, usability, and maintainability of the RAG ingestion and retrieval pipeline, and provide clearer guidance for both developers and end users.