Skip to content

feat: type-level BM25 search corpus and search_types() - #8

Open
jakerobinson wants to merge 2 commits into
mainfrom
feature/types-bm25-corpus
Open

feat: type-level BM25 search corpus and search_types()#8
jakerobinson wants to merge 2 commits into
mainfrom
feature/types-bm25-corpus

Conversation

@jakerobinson

Copy link
Copy Markdown
Contributor

Summary

Adds a fourth BM25 search surface to the rsc-client: a type-level corpus that aggregates all field vocabulary for each GraphQL type into a single document. Exposes it via search_types(search), mirroring the existing search_operations / search_fields pattern.

Why this matters: Operations like allClusterConnection and slaDomains return complex types (Cluster, GlobalSlaReply) with 50+ fields each. When an agent searches for "CDM clusters" or "retention-locked SLAs", BM25 can't reliably surface these operations via the ops corpus — the type's rich vocabulary gets diluted across 228-token operation documents and length-normalized into oblivion. The type corpus solves this by indexing each type's complete vocabulary in one place and resolving back to operations.

Changes

  • mcp_indexer.pybuild_types_bm25_corpus() as a fourth indexer step. Tokens: camelCase type/field name splits with org→organization and vm→virtual machine synonym expansion, first-8-word field descriptions, and one level of Connection field traversal to include child type vocabulary. Op resolution covers direct return-type, Connection→node, and interface→implementor paths.

  • index.pysearch_types(search) with lazy BM25Okapi loading. Returns up to 10 results with name, ops, and score keys.

  • __init__.py — exports search_types alongside existing exports.

  • mcp_types_bm25_corpus.json — 1,562 reachable types, 1.3MB.

  • schema-update.yml — adds mcp_types_bm25_corpus.json to the git add in CI so it's committed on each schema update.

Test plan

  • 8 new tests in tests/test_search_types.py
  • search_types("cluster") returns Cluster type with cluster operations in ops
  • Scores > 0 for relevant queries
  • Results capped at 10
  • CI schema-update workflow commits the regenerated corpus on schema changes

jakerobinson and others added 2 commits August 7, 2026 15:51
Introduces mcp_types_bm25_corpus.json (1562 reachable types, 1.3MB) built by
a new build_types_bm25_corpus() indexer step. Tokens aggregate type-name and
field-name camelCase splits (with org/vm synonym expansion), first-8-word field
descriptions, and one level of Connection field traversal. Ops resolution covers
direct, node-via-Connection, and interface-inheritance paths.

Exposes search_types(search) from rsc.index and rsc.__init__, mirroring the
search_operations/search_fields pattern with lazy BM25Okapi loading. Returns up
to 10 results ranked by relevance, each with name, ops, and score keys.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant