Skip to content

feat(engine): Phase 11.2 logical clock + active-tx registry (SQLR-22)#123

Merged
joaoh82 merged 1 commit intomainfrom
worktree-phase-11-2-logical-clock
May 10, 2026
Merged

feat(engine): Phase 11.2 logical clock + active-tx registry (SQLR-22)#123
joaoh82 merged 1 commit intomainfrom
worktree-phase-11-2-logical-clock

Conversation

@joaoh82
Copy link
Copy Markdown
Owner

@joaoh82 joaoh82 commented May 10, 2026

Summary

Second slice of Phase 11 — concurrent writes via MVCC + BEGIN CONCURRENT (SQLR-22). Adds the MVCC primitives that 11.3 will plug an in-memory version index into and 11.4 will hand begin / commit timestamps to.

New sqlrite::mvcc module

  • MvccClock — process-wide monotonic u64 over AtomicU64. tick(), now(), observe(value). The observe-CAS-loop guarantees the clock never moves backwards under contention; used at WAL replay to bring the in-memory clock up to the persisted high-water mark.
  • ActiveTxRegistryMutex<BTreeMap> over in-flight TxId → begin_ts mappings. register(&clock) returns a RAII TxHandle that unregisters on drop. min_active_begin_ts() answers Phase 11.6 GC's "what's still possibly visible" question in O(log N).
  • TxId newtype + TxTimestampOrId tagged union, defined now so 11.4 can plug in without re-litigating the type shape.

WAL format bumps v1 → v2

  • Bytes 24..32 of the WAL header (previously reserved-zero) now carry the persisted clock_high_water: u64.
  • v1 WALs open cleanly — those zero bytes read as "clock never advanced" — and the next checkpoint rewrites the header at v2. No offline upgrade step.
  • New WAL_FORMAT_VERSION_MIN_SUPPORTED = 1 keeps the version-range check explicit. Forward versions (e.g. v3) reject with a clean error rather than misinterpreting bytes.
  • Wal::clock_high_water() / Wal::set_clock_high_water(value) accessors. The setter rejects regressions with a typed error (same value is a no-op).

Scope honesty

Not yet wired into the executor — that's 11.3. The clock is standalone today; only the mvcc module's own tests tick it. The plumbing is in place so 11.3 can read Wal::clock_high_water() at open time and feed the in-memory clock into a Database field.

Test plan

  • cargo build --workspace --exclude sqlrite-desktop --exclude sqlrite-python --exclude sqlrite-nodejs --exclude sqlrite-benchmarks --all-targets — clean
  • cargo test --workspace --exclude sqlrite-desktop --exclude sqlrite-python --exclude sqlrite-nodejs --exclude sqlrite-benchmarks — 599/599 pass (21 new)
  • cargo clippy --workspace --exclude sqlrite-desktop --exclude sqlrite-python --exclude sqlrite-nodejs --exclude sqlrite-benchmarks --all-targets — no new warnings on changed files
  • cargo fmt --all -- --check — clean
  • cargo doc --workspace --exclude sqlrite-desktop --exclude sqlrite-python --exclude sqlrite-nodejs --exclude sqlrite-benchmarks --no-deps — no new warnings on changed files
  • v1 WAL → v2 reader round-trip verified by hand-built v1 header in v1_wal_opens_with_zero_clock

New tests (21 total)

  • 8 in mvcc::clock::tests — seed, default, monotonicity, observe, contention (8 threads × 250 ticks unique), observe-under-contention.
  • 7 in mvcc::registry::tests — empty-min, register-advances-clock, unique-ids, unregister-arbitrary-order, Send+Sync compile check, concurrent registrations, TxId display, TxTimestampOrId round-trip.
  • 6 in sql::pager::wal::tests — fresh-zero, round-trip-through-truncate, monotonic-persistence-across-truncates, regression-rejected, v1-opens-with-zero-clock, unknown-future-version-rejected.

Followups (next sub-phases, separate PRs)

  • 11.3 — MvStore skeleton + PRAGMA journal_mode = mvcc snapshot-isolated reads. Wires MvccClock into Database; reads Wal::clock_high_water() at open.
  • 11.4 — BEGIN CONCURRENT writes + commit-time validation (the meat).
  • 11.5 — Checkpoint integration + crash recovery.
  • 11.6 — GC (per-commit + background).

🤖 Generated with Claude Code

Second slice of Phase 11 (concurrent writes via MVCC + BEGIN
CONCURRENT). Adds the MVCC primitives that 11.3 will plug an
in-memory version index into and 11.4 will hand begin/commit
timestamps to.

New `sqlrite::mvcc` module:
- MvccClock — process-wide monotonic u64 over AtomicU64. tick(),
  now(), observe(value). The observe-CAS-loop guarantees the clock
  never moves backwards under contention; used at WAL replay to
  bring the in-memory clock up to the persisted high-water mark.
- ActiveTxRegistry — Mutex<BTreeMap> over in-flight TxId → begin_ts
  mappings. register(&clock) returns a RAII TxHandle that
  unregisters on drop. min_active_begin_ts() answers Phase 11.6 GC's
  "what's still possibly visible" question in O(log N).
- TxId newtype + TxTimestampOrId tagged union, defined now so 11.4
  can plug in without re-litigating the type shape.

WAL format bumps v1 → v2:
- Bytes 24..32 of the WAL header (previously reserved-zero) now
  carry the persisted clock_high_water u64.
- v1 WALs open cleanly — those zero bytes read as "clock never
  advanced" — and the next checkpoint rewrites the header at v2. No
  offline upgrade step.
- New WAL_FORMAT_VERSION_MIN_SUPPORTED = 1 keeps the version-range
  check explicit. Forward versions (e.g. v3) reject with a clean
  error rather than misinterpreting bytes.
- Wal::clock_high_water() / Wal::set_clock_high_water(value)
  accessors. The setter rejects regressions with a typed error
  (same value is a no-op).

Not yet wired into the executor — that's 11.3. The clock is
standalone today; only the mvcc module's own tests tick it. The
plumbing is in place so 11.3 can read Wal::clock_high_water() at
open time and feed the in-memory clock into a Database field.

21 new tests:
- 8 in mvcc::clock::tests — seed, default, monotonicity, observe,
  contention (8 threads × 250 ticks unique).
- 7 in mvcc::registry::tests — empty-min, register-advances-clock,
  unique-ids, unregister-arbitrary-order, Send+Sync compile check,
  concurrent registrations.
- 6 in sql::pager::wal::tests — fresh-zero, round-trip-through-
  truncate, monotonic-persistence-across-truncates,
  regression-rejected, v1-opens-with-zero-clock,
  unknown-future-version-rejected.

599/599 workspace tests pass. fmt + clippy clean. cargo doc clean
on changed files.

Docs: docs/file-format.md WAL header diagram updated for v2 + the
v1→v2 compatibility note. docs/roadmap.md marks 11.1 ✅ and 11.2 🚧.
docs/design-decisions.md gets a 12b entry on the clock + WAL
persistence design. docs/_index.md project-state line picks up the
11.2 summary.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@vercel
Copy link
Copy Markdown

vercel Bot commented May 10, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
rust-sqlite Ready Ready Preview, Comment May 10, 2026 10:45am

Request Review

@joaoh82 joaoh82 merged commit 6838bff into main May 10, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant