Stop serving 2,400 tokens of schema nobody asked for - #146
Merged
Merged
Conversation
Nothing this server runs reaches a tool through Tool.get_schema(), which is where upstream cleans parameter schemas. RouteTable._tool_to_route reads tool.parameters directly, so both the MCP and REST surfaces receive the uncleaned dict, and two pieces of dead weight rode along on every request. toolcall_reason: Tool.model_post_init writes the key into parameters on every construction, unconditionally. The switch meant to govern it lives in get_schema() and defaults to off, which is the registry built here. The feature has been disabled since this registry was written and the schema shipped regardless, on all 33 core tools, for an argument execution discards. Pydantic title: every generated property carried a title restating its own name in title case, which no client reads. Both are trimmed at registry build, in _trim_wire_schema. The core surface falls from 11,361 to 8,974 cl100k_base tokens, 21% off the fixed cost of every request. The toolcall_reason pop becomes a no-op if upstream closes the gap; neither pass forks anything. The title trim recurses into property values only. Upstream's blanket key filter does not, which is why plot_dataset's real title argument is missing from registry.get_schemas() -- a test pins the distinction so the fix cannot regress into the bug it works around. The budget that should have caught all of this was measuring the same unserved surface: TestToolSpecBudget and scripts/measure_payload.py both read get_schemas(), understating the catalog by 4,263 bytes and passing. Both now measure RouteTable, with budgets ratcheted onto the trimmed figures. Separately, list_datasets reported an archive root as empty. A non-recursive scan of a directory whose datasets sit one level down returned total_files: 0 and no groups, which reads as a missing dataset rather than a wrong flag. That layout is GDEX and most model archives. The zero-file case now counts the subdirectories and names the first ten. Applied to the worker copy too, which is the one that meets GDEX.
rajeeja
added a commit
that referenced
this pull request
Sep 12, 2026
Nothing this server runs reaches a tool through Tool.get_schema(), which is where upstream cleans parameter schemas. RouteTable._tool_to_route reads tool.parameters directly, so both the MCP and REST surfaces receive the uncleaned dict, and two pieces of dead weight rode along on every request. toolcall_reason: Tool.model_post_init writes the key into parameters on every construction, unconditionally. The switch meant to govern it lives in get_schema() and defaults to off, which is the registry built here. The feature has been disabled since this registry was written and the schema shipped regardless, on all 33 core tools, for an argument execution discards. Pydantic title: every generated property carried a title restating its own name in title case, which no client reads. Both are trimmed at registry build, in _trim_wire_schema. The core surface falls from 11,361 to 8,974 cl100k_base tokens, 21% off the fixed cost of every request. The toolcall_reason pop becomes a no-op if upstream closes the gap; neither pass forks anything. The title trim recurses into property values only. Upstream's blanket key filter does not, which is why plot_dataset's real title argument is missing from registry.get_schemas() -- a test pins the distinction so the fix cannot regress into the bug it works around. The budget that should have caught all of this was measuring the same unserved surface: TestToolSpecBudget and scripts/measure_payload.py both read get_schemas(), understating the catalog by 4,263 bytes and passing. Both now measure RouteTable, with budgets ratcheted onto the trimmed figures. Separately, list_datasets reported an archive root as empty. A non-recursive scan of a directory whose datasets sit one level down returned total_files: 0 and no groups, which reads as a missing dataset rather than a wrong flag. That layout is GDEX and most model archives. The zero-file case now counts the subdirectories and names the first ten. Applied to the worker copy too, which is the one that meets GDEX.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Nothing this server runs reaches a tool through
Tool.get_schema(), so both surfaces served the uncleaned parameter dict:toolcall_reasonon all 33 core tools for a feature that has been disabled since the registry was written, plus a Pydantictitleon every property. Trimming both at registry build takes the core surface from 11,361 to 8,974cl100k_basetokens, and the budget test that should have caught it was measuringget_schemas(), a surface nobody receives. Also fixeslist_datasetsreporting an archive root as empty when its datasets sit one level down, which is how GDEX is laid out.