feat(engine): Phase 11.3 MvStore skeleton + PRAGMA journal_mode (SQLR-22)#124
Merged
feat(engine): Phase 11.3 MvStore skeleton + PRAGMA journal_mode (SQLR-22)#124
Conversation
…-22)
Third slice of Phase 11 (concurrent writes via MVCC + BEGIN
CONCURRENT). Adds the in-memory version index (`MvStore`) and the
per-database `journal_mode` toggle so 11.4 can wire the executor's
read/write paths through a typed switch rather than a lazy-init
dance.
New `sqlrite::mvcc::store`:
- `RowID = (table, rowid)`, `VersionPayload = Present(cols) |
Tombstone`, `RowVersion { begin, end, payload }`,
`RowVersionChain = Vec<RowVersion>` behind `Arc<RwLock<…>>`.
- `MvStore` itself: `Mutex<HashMap<RowID, Arc<RwLock<…>>>>` plus a
shared `MvccClock` and `ActiveTxRegistry`. The outer mutex
guards the map's shape; the per-chain RwLock means readers
walking different chains don't fight.
- `MvStore::read(row, begin_ts)` implements the textbook
snapshot-isolation rule (`begin <= T < end`). Pure helper
`visible_at(version, begin_ts)` exposed for tests + 11.6's GC.
- `push_committed` validates monotonic begin + caps the previous
latest's `end` (validate-then-mutate; equal-begin retries
surface as NonMonotonicBegin instead of leaving a half-capped
chain). `push_in_flight` for tests + 11.4's BEGIN CONCURRENT
writes; in-flight versions invisible to other readers.
- `MvStoreError` enum: `NotCommitted`, `PreviousAlreadyCapped`,
`PreviousCappedByInFlight`, `NonMonotonicBegin`.
New `JournalMode { Wal, Mvcc }` enum (default Wal). `Database`
gains `journal_mode`, `mvcc_clock: Arc<MvccClock>`, `mv_store:
MvStore` fields — all allocated on `Database::new` even in Wal
mode so the toggle is cheap. `set_journal_mode` rejects
`Mvcc → Wal` while the store carries committed versions (would
silently strand them; v0 keeps this strict).
`PRAGMA journal_mode;` (read) renders the current mode; `PRAGMA
journal_mode = wal | mvcc;` (set) toggles. Case-insensitive on
both pragma name and value; numeric values rejected; unknown
modes return a typed error without disturbing the existing
setting. `Connection::journal_mode()` is the public read
accessor — every `Connection::connect` sibling sees the same
value (per-database, not per-connection; see `concurrent-writes-
plan.md` §8 for the open question).
Scope honesty: the executor doesn't consult `MvStore` yet.
Routing reads through MvStore in this PR would surface as wrong
rows because nothing publishes versions to it — the legacy
commit path mutates `Database.tables` directly. 11.4 ships the
read-path wiring AND the commit-path mirroring together because
they're coupled. This PR delivers the data structure + toggle
that 11.4 plugs into.
Test breakdown (24 new):
- mvcc::store::tests (12): visibility rule, cap-on-push,
tombstone semantics, in-flight invisibility, monotonic-begin
+ in-flight rejection, tracked-rows + total-versions counts,
Send+Sync compile check, concurrent reads consistency,
shared-clock pointer identity.
- mvcc::mod::tests (3): JournalMode default, str round-trip,
display.
- connection::tests (5): defaults_to_wal,
set_to_mvcc_propagates_to_siblings, case_insensitive,
rejects_unknown_value, rejects_numeric_value.
Two existing tests updated: they used `PRAGMA journal_mode` as
their canary unknown pragma; flipped to `PRAGMA synchronous`
since journal_mode is now recognised.
619/619 workspace tests pass. fmt + clippy + doc clean on
changed files. No file-format change (file or WAL).
Docs: roadmap.md marks 11.2 ✅ and 11.3 🚧. design-decisions.md
gets a 12c entry on the data-structure-before-wiring decision +
the per-database journal-mode trade-off. supported-sql.md picks
up the PRAGMA journal_mode reference. _index.md project-state
line picks up the 11.3 summary.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
CI's `cargo fmt --all -- --check` flagged two files my local check missed across earlier verification runs. Same toolchain version (rustfmt 1.8.0, Rust 1.94.0), so this is just rerunning fmt and committing the result. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Third slice of Phase 11 — concurrent writes via MVCC +
BEGIN CONCURRENT(SQLR-22). Adds the in-memory version index (MvStore) and the per-databasejournal_modetoggle so 11.4 can wire the executor's read/write paths through a typed switch.New
sqlrite::mvcc::storeRowID = (table, rowid),VersionPayload = Present(cols) | Tombstone,RowVersion { begin, end, payload },RowVersionChain = Vec<RowVersion>behindArc<RwLock<…>>.MvStoreitself:Mutex<HashMap<RowID, Arc<RwLock<…>>>>plus a sharedMvccClockandActiveTxRegistry. The outer mutex guards the map's shape; the per-chainRwLockmeans readers walking different chains don't fight.MvStore::read(row, begin_ts)implements the textbook snapshot-isolation rule (begin <= T < end). Pure helpervisible_at(version, begin_ts)exposed for tests + 11.6's GC.push_committedvalidates monotonic begin + caps the previous latest'send(validate-then-mutate; equal-begin retries surface asNonMonotonicBeginrather than leaving a half-capped chain).push_in_flightfor tests + 11.4'sBEGIN CONCURRENTwrites; in-flight versions are invisible to other readers.MvStoreErrorenum coversNotCommitted,PreviousAlreadyCapped,PreviousCappedByInFlight,NonMonotonicBegin.JournalMode+PRAGMA journal_modeJournalMode { Wal, Mvcc }enum, defaultWal.Databasegainsjournal_mode,mvcc_clock: Arc<MvccClock>,mv_store: MvStorefields — all allocated onDatabase::neweven inWalmode so the toggle is cheap.set_journal_moderejectsMvcc → Walwhile the store carries committed versions (would silently strand them; v0 keeps this strict).PRAGMA journal_mode;(read) renders the current mode;PRAGMA journal_mode = wal | mvcc;(set) toggles. Case-insensitive on both pragma name and value; numeric values rejected; unknown modes return a typed error without disturbing the existing setting.Connection::journal_mode()is the public read accessor — everyConnection::connectsibling sees the same value (per-database, not per-connection; seeconcurrent-writes-plan.md§8 for the open question).Scope honesty
The executor doesn't consult
MvStoreyet. Routing reads throughMvStorein this PR would surface as wrong rows because nothing publishes versions to it — the legacy commit path mutatesDatabase.tablesdirectly. 11.4 ships the read-path wiring AND the commit-path mirroring together because they're coupled. This PR delivers the data structure + toggle that 11.4 plugs into.Test plan
cargo build --workspace --exclude sqlrite-desktop --exclude sqlrite-python --exclude sqlrite-nodejs --exclude sqlrite-benchmarks --all-targets— cleancargo test --workspace --exclude sqlrite-desktop --exclude sqlrite-python --exclude sqlrite-nodejs --exclude sqlrite-benchmarks— 619/619 pass (24 new + 2 existing tests updated to usePRAGMA synchronousas the canary unknown-pragma name now thatjournal_modeis recognised)cargo clippy --workspace --exclude sqlrite-desktop --exclude sqlrite-python --exclude sqlrite-nodejs --exclude sqlrite-benchmarks --all-targets— no new warnings on changed filescargo fmt --all -- --check— cleancargo doc --workspace --exclude sqlrite-desktop --exclude sqlrite-python --exclude sqlrite-nodejs --exclude sqlrite-benchmarks --no-deps— no new warnings on changed filesNew tests (24)
mvcc::store::tests(12): visibility rule, cap-on-push, tombstone semantics, in-flight invisibility, monotonic-begin + in-flight rejection, tracked-rows + total-versions counts,Send + Synccompile check, concurrent-reads consistency, shared-clock pointer identity.mvcc::tests(3):JournalModedefault, str round-trip, display.connection::tests(5):defaults_to_wal,set_to_mvcc_propagates_to_siblings, case-insensitive, rejects-unknown-value, rejects-numeric-value.Followups
BEGIN CONCURRENTwrites, commit-time validation (write-write conflict →Busy/BusySnapshot), executor read path consultsMvStorewhenjournal_mode == Mvcc, legacy commit mirrors writes intoMvStore.🤖 Generated with Claude Code