feat(engine): Phase 11.2 logical clock + active-tx registry (SQLR-22)#123
Merged
feat(engine): Phase 11.2 logical clock + active-tx registry (SQLR-22)#123
Conversation
Second slice of Phase 11 (concurrent writes via MVCC + BEGIN CONCURRENT). Adds the MVCC primitives that 11.3 will plug an in-memory version index into and 11.4 will hand begin/commit timestamps to. New `sqlrite::mvcc` module: - MvccClock — process-wide monotonic u64 over AtomicU64. tick(), now(), observe(value). The observe-CAS-loop guarantees the clock never moves backwards under contention; used at WAL replay to bring the in-memory clock up to the persisted high-water mark. - ActiveTxRegistry — Mutex<BTreeMap> over in-flight TxId → begin_ts mappings. register(&clock) returns a RAII TxHandle that unregisters on drop. min_active_begin_ts() answers Phase 11.6 GC's "what's still possibly visible" question in O(log N). - TxId newtype + TxTimestampOrId tagged union, defined now so 11.4 can plug in without re-litigating the type shape. WAL format bumps v1 → v2: - Bytes 24..32 of the WAL header (previously reserved-zero) now carry the persisted clock_high_water u64. - v1 WALs open cleanly — those zero bytes read as "clock never advanced" — and the next checkpoint rewrites the header at v2. No offline upgrade step. - New WAL_FORMAT_VERSION_MIN_SUPPORTED = 1 keeps the version-range check explicit. Forward versions (e.g. v3) reject with a clean error rather than misinterpreting bytes. - Wal::clock_high_water() / Wal::set_clock_high_water(value) accessors. The setter rejects regressions with a typed error (same value is a no-op). Not yet wired into the executor — that's 11.3. The clock is standalone today; only the mvcc module's own tests tick it. The plumbing is in place so 11.3 can read Wal::clock_high_water() at open time and feed the in-memory clock into a Database field. 21 new tests: - 8 in mvcc::clock::tests — seed, default, monotonicity, observe, contention (8 threads × 250 ticks unique). - 7 in mvcc::registry::tests — empty-min, register-advances-clock, unique-ids, unregister-arbitrary-order, Send+Sync compile check, concurrent registrations. - 6 in sql::pager::wal::tests — fresh-zero, round-trip-through- truncate, monotonic-persistence-across-truncates, regression-rejected, v1-opens-with-zero-clock, unknown-future-version-rejected. 599/599 workspace tests pass. fmt + clippy clean. cargo doc clean on changed files. Docs: docs/file-format.md WAL header diagram updated for v2 + the v1→v2 compatibility note. docs/roadmap.md marks 11.1 ✅ and 11.2 🚧. docs/design-decisions.md gets a 12b entry on the clock + WAL persistence design. docs/_index.md project-state line picks up the 11.2 summary. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Second slice of Phase 11 — concurrent writes via MVCC +
BEGIN CONCURRENT(SQLR-22). Adds the MVCC primitives that 11.3 will plug an in-memory version index into and 11.4 will hand begin / commit timestamps to.New
sqlrite::mvccmoduleMvccClock— process-wide monotonicu64overAtomicU64.tick(),now(),observe(value). The observe-CAS-loop guarantees the clock never moves backwards under contention; used at WAL replay to bring the in-memory clock up to the persisted high-water mark.ActiveTxRegistry—Mutex<BTreeMap>over in-flightTxId → begin_tsmappings.register(&clock)returns a RAIITxHandlethat unregisters on drop.min_active_begin_ts()answers Phase 11.6 GC's "what's still possibly visible" question inO(log N).TxIdnewtype +TxTimestampOrIdtagged union, defined now so 11.4 can plug in without re-litigating the type shape.WAL format bumps v1 → v2
clock_high_water: u64.WAL_FORMAT_VERSION_MIN_SUPPORTED = 1keeps the version-range check explicit. Forward versions (e.g. v3) reject with a clean error rather than misinterpreting bytes.Wal::clock_high_water()/Wal::set_clock_high_water(value)accessors. The setter rejects regressions with a typed error (same value is a no-op).Scope honesty
Not yet wired into the executor — that's 11.3. The clock is standalone today; only the
mvccmodule's own tests tick it. The plumbing is in place so 11.3 can readWal::clock_high_water()at open time and feed the in-memory clock into aDatabasefield.Test plan
cargo build --workspace --exclude sqlrite-desktop --exclude sqlrite-python --exclude sqlrite-nodejs --exclude sqlrite-benchmarks --all-targets— cleancargo test --workspace --exclude sqlrite-desktop --exclude sqlrite-python --exclude sqlrite-nodejs --exclude sqlrite-benchmarks— 599/599 pass (21 new)cargo clippy --workspace --exclude sqlrite-desktop --exclude sqlrite-python --exclude sqlrite-nodejs --exclude sqlrite-benchmarks --all-targets— no new warnings on changed filescargo fmt --all -- --check— cleancargo doc --workspace --exclude sqlrite-desktop --exclude sqlrite-python --exclude sqlrite-nodejs --exclude sqlrite-benchmarks --no-deps— no new warnings on changed filesv1_wal_opens_with_zero_clockNew tests (21 total)
mvcc::clock::tests— seed, default, monotonicity, observe, contention (8 threads × 250 ticks unique), observe-under-contention.mvcc::registry::tests— empty-min, register-advances-clock, unique-ids, unregister-arbitrary-order, Send+Sync compile check, concurrent registrations, TxId display, TxTimestampOrId round-trip.sql::pager::wal::tests— fresh-zero, round-trip-through-truncate, monotonic-persistence-across-truncates, regression-rejected, v1-opens-with-zero-clock, unknown-future-version-rejected.Followups (next sub-phases, separate PRs)
MvStoreskeleton +PRAGMA journal_mode = mvccsnapshot-isolated reads. WiresMvccClockintoDatabase; readsWal::clock_high_water()at open.BEGIN CONCURRENTwrites + commit-time validation (the meat).🤖 Generated with Claude Code