feat(index): per-vault SQLite FTS5 index + scan + reindex (M2a)#215
Merged
Conversation
- vault-index.ts: node:sqlite FTS5 index (files + files_fts + meta), ported from the archived v0.24 module; search/stat/list/reconcile, bm25-ranked, lazy-loaded - vault-scan.ts: walk markdown + mtime/size/sha diff -> reconcile; reindexVault() - vault reindex [name]: rebuild one or all vault indexes from markdown - cli.ts: re-exec bootstrap with --experimental-sqlite for index commands on Node < 23.4, and silence the node:sqlite ExperimentalWarning - vitest execArgv passes the flag to fork workers - Direct in-process access = the DirectClient path per the daemon-index-ownership design (V10); the daemon will reuse this same engine - 127 tests green, coverage 88/86/77/89; verified the built binary reindexes a real vault + is idempotent
Contributor
|
🎉 PR Validation ✅ PASSED Commit: Checks:
Ready to merge! ✨ 🔗 View workflow run |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
First slice of M2 - Offline CRUD + index. The index engine +
vault reindex, ported tonode:sqlitefrom the tested v0.24 module. Nomemory-coredependency (V3 = harvest), so this is not gated on the PR-0 relicense.What
vault-index.ts- per-vault SQLite FTS5 index (~/.agentage/index/<name>.db):files+files_fts(bm25-ranked, porter/unicode61) +meta.search/stat/list/reconcile/fileCount/indexedAt. Lazy-loadsnode:sqliteso non-index commands never touch it.vault-scan.ts- walk the markdown tree, diff by mtime/size then sha256,reconcileadded/modified/removed.reindexVault(vaultPath, dbPath).vault reindex [name]- rebuild one vault's index (or all) from its markdown.cli.tsbootstrap -node:sqliteis behind--experimental-sqliteon Node < 23.4, so index commands re-exec once with the flag when it can't load; also silences the SQLiteExperimentalWarning. Non-index commands are untouched.poolOptions.forks.execArgvpasses the flag to test workers.Design fit
Per [[daemon-index-ownership]] (V10): the CLI will reach the index through a
MemoryClientwhose default is the daemon (auto-started, harvested from v0.24) and whose fallback is direct in-process. This PR is the shared engine + the direct path - the daemon (M2.5) reusesvault-index.tsverbatim, so nothing here gets reworked.reindexruns direct today; it routes via the daemon once that lands.Verify
npm run verifygreen: type-check (src + e2e), eslint, prettier, 127 unit tests, build. Coverage 88 / 86 / 77 / 89.vault adda local vault of.md->vault reindexreports+2, a second run is idempotent (=2), db lands atindex/<name>.db,vault reindex(no name) does all. Warning is silenced.indexedAt; scan build + drop-db-and-reindex = identical results (index is a pure cache) + modified/removed/unchanged detection + dot-dir skip;reindexcommand routing (all / named / unknown / empty).Next (M2b): the 6
memoryverbs behindMemoryClient+DirectClient. Then M2.5 harvests the daemon.