Repository navigation
Proposal: shared tool spec + conformance tests so 7-tool parity can't drift #1731
Description
Problem
The 7-tool surface is defined by hand in each SDK: packages/tools/src/tools-shared.ts (TOOL_DESCRIPTIONS), supermemory_openai/tools.py (TOOL_DESCRIPTIONS), and now supermemory_agent_framework/tools.py (#1730). Nothing checks that they agree, and they already don't.
Comparing TOOL_DESCRIPTIONS in TS and supermemory_openai on current main (normalizing camelCase ↔ snake_case tool names):
| Tool | Identical? |
|---|---|
| add_memory | yes |
| document_add | yes |
| search_memories | no |
| get_profile | no |
| document_list | no |
| document_delete | no |
| memory_forget | no |
Some differences are probably intentional (TS has the containerTag override, Python doesn't). But some change model behavior:
-
search_memoriesgives the model opposite guidance.- TS: "Use when explicitly asked to search or recall, or when past context could materially improve the response; do not invoke reflexively on every turn."
- Python: "Use proactively whenever prior context could help."
The same agent searches on every turn or only when needed, depending on the SDK.
-
A description promises a guard the code doesn't have.
supermemory_openai'sdocument_deletedescription says deletion is refused for documents "still processing", butSupermemoryTools.document_deletenever checksstatus. The TS version does (assertDocumentCanBeDeleted,TERMINAL_DOCUMENT_STATUSES).
Each parity pass (#1430, #1432, #1433) has had to fix this by hand across packages, and #1730 exists because one SDK was missed.
Proposal
-
One spec file, e.g.
packages/tools/tool-spec.json, holding each tool's name, parameters (name, type, required) and canonical description. Descriptions refer to other tools with a placeholder like{memory_forget}, which each SDK renders in its own naming (memoryForget/memory_forget). -
A conformance test in each SDK that loads the spec and fails if the SDK's tools don't match it:
packages/tools: vitest, againstTOOL_DESCRIPTIONSand the tool schemassupermemory_openai: pytest, againstMEMORY_TOOL_SCHEMASsupermemory_agent_framework: pytest, againstget_tools()schemas
These run in the CI jobs that already exist (
ci.yml,ci-python.yml), with no new workflow. -
Intentional differences are declared in the spec (e.g.
"sdks": ["ts"]on thecontainerTagparameter), so they're documented instead of accidental.
The Python packages would still ship their own copy of the strings, since the wheels can't read a repo file at runtime. The test just keeps those copies honest.
Questions before I build it
- Is TS the source of truth for descriptions, or should the Python wording win anywhere, e.g. the
search_memoriesguidance? - Is
packages/tools/tool-spec.jsona good location, or would you rather have it in its own package? - Should the Python SDKs vendor the spec at build time instead of keeping a hand copy? That's more work but removes the duplication entirely.
Happy to open the PR once the direction looks right. I'd also fix the missing status check in supermemory_openai.document_delete separately, since that's a bug either way.
Activity
linear-code commentedon Sep 30, 2026
ENG-1282
MaheshtheDev commentedon Sep 30, 2026
thanks for the issue but, we are working on making it even simpler and should be out in next few days. thanks for the issue though