fix(signals): add a CRDT collaborative lore draft behind phase-141 - #364
Merged
Darkvader-ship-it merged 1 commit intoSep 30, 2026
Merged
Darkvader-ship-it merged 1 commit into
Darkvader-ship-it merged 1 commit into
Conversation
Issue PHASE-STELLAR#207 reports that concurrent edits to one signal clobber each other. The existing version+CAS contract is correct but answers the wrong question when 50 people type at once — it keeps one text and rejects the rest. The spike in lib/__tests__/signal-crdt-benchmark.test.ts measures it: Yjs CRDT (phase-141) 50/50 50 updates folded, 0 rejected version + CAS (current) 1/50 1 committed, 49 rejected with 409 So the two compose rather than compete. A Yjs CRDT draft in SQLite merges concurrent proposals automatically and never writes the signals row; the merged result is promoted to committed lore only through PUT with from_draft: true, which reuses the same If-Match guard and signal_versions snapshot as a manual edit. The guard is not removed, it moves to the one place a decision has to be made: if a colleague committed while the draft was open, PUT still answers 409 with current_version. - lib/signal-crdt.ts: lore document (title/body Y.Text), minimal span-diff edits so two appends stay two inserts, state-vector helpers, convergence checks. The client imports it dynamically so Yjs stays out of the bundle while the flag is off. - lib/signal-crdt-store.ts: merge under BEGIN IMMEDIATE, update tail folded into the snapshot every 64 merges, contributor attribution, 256KB cap. - PUT /api/signals/[id] (If-Match required, full title+body) plus GET/POST /api/signals/[id]/crdt for HTTP state-vector sync. HTTP rather than WebSocket because the app deploys to Vercel serverless. - signal_version_conflicts is counted once, in the store, so a rejected CAS is not double-counted by every layer that observes it. Counters are aggregate only: no signal id, wallet, or text reaches a label. - Fixes diffLoreVersions, which was exported as @ts-nocheck referenced but never defined, breaking the narrative versions route. 22 new tests, including 50-writer no-lost-writes across two SQLite connections. Docs updated in PROJECT_ARCHITECTURE.md and docs/TECHNICAL.md.
|
@Olumide-01 Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits. You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #207
What this does
Concurrent edits to one signal clobber each other today. The existing
version+ compare-and-swap contract is correct — it never lets a writersilently overwrite a peer's row — but when many people type into the same
signal at once it answers the wrong question: one save wins and the rest
come back
409.lib/__tests__/signal-crdt-benchmark.test.tsruns the workload the issuespecifies (50 writers, each editing from the same starting state without
seeing the others) against both strategies:
version+ CAS (current)409, text clobberedSo this layers a CRDT draft under the existing guard instead of
replacing it. The two solve different problems:
authoritative, and it cannot write revertible history.
pre-edit text so the revision stays revertible.
The commit path is still guarded
signal_crdt_docs/signal_crdt_updatesare scratch state and neverwrite the
signalsrow. A merged draft is promoted to committed lore onlythrough
PUT /api/signals/[id]withfrom_draft: true, which lands in thesame
editSignalCAS path as a manual edit and snapshots intosignal_versionsas before.Conflicts are still possible and still visible at that boundary: if a
colleague committed while your draft was open,
PUTstill answers409with
current_versionand a freshETagso the client can rebase. TheCRDT removes lost updates; it does not remove the need for a guard.
Changes
lib/signal-crdt.tsloreY.Map oftitle/bodyY.Text. Edits apply as a minimal prefix/suffix span diff, so two appends become two independent inserts instead of one overwriting the other.lib/signal-crdt-store.tsBEGIN IMMEDIATE; the update tail folds into the snapshot every 64 merges so it cannot grow unbounded. 256KB update cap.lib/sqlite-db.tsapp/api/signals/[id]/route.tsPUT(mandatoryIf-Match, requires bothtitleandbodyso a revision is never half-applied). PATCH conflicts now carrycurrent_version+ETaglike the upvote path.app/api/signals/[id]/crdt/route.tsGETwith?state_vector=returns only the operations that client lacks;POSTmerges an update.app/signals/[id]/use-signal-crdt.tsawait import()s the CRDT module, so Yjs stays out of the initial bundle while the flag is off.app/signals/[id]/signal-detail-client.tsxlib/signal-version-metrics.tssignal_version_conflictssplit byedit/put/upvote/reply, flagged separately when a CAS retry was attempted, plus CRDT merge and commit counters.lib/signal-store.tsDesign decisions worth reviewing
HTTP, not WebSocket. The issue suggests prototyping Yjs over a
WebSocket. Yjs is transport-agnostic and its state-vector sync is a plain
request/response, so
GET-with-state-vector carries the same wire format asocket would without holding a connection open — and this app deploys to
Vercel serverless, where a long-lived socket per reader is not available.
The client posts on a short debounce and polls while the panel is open.
The store is the only thing that counts conflicts. An earlier draft
also incremented the metric in the route handler, which counted every
rejected CAS twice. Counters are aggregate only — no signal id, wallet, or
text ever reaches a label.
Flags.
phase-141, defaulting off. UnsetNEXT_PUBLIC_FEATURE_PHASE_141and the CRDT routes return
404, the draft panel is hidden, andPUTstillworks on its own as an
If-Match-guarded full replacement. Thesignal_crdt_*rows stay on disk as inert scratch state.Testing
lib/__tests__/signal-crdt.test.ts, including 50concurrent writers with zero lost writes, reversed arrival order,
idempotent re-submission, state-vector catch-up for a late replica, and
two independent SQLite connections racing on the same file.
npx tsc --noEmit: no new diagnostics. This also fixes a pre-existingbreak —
diffLoreVersionswas imported by the narrative versions routebut never actually exported, because the file carried
@ts-nocheck.nft-index-subscribe[SECURITY — WEBHOOK LEAK] Creator (100k NFTs) toevil.comand Exfiltrates AlltransferEvents Forever #228,world-roles-rbac[SECURITY — RBAC]world/[collection_id]/rolesPOST { wallet, role: 'admin' }Sansrequire_auth#291) were confirmed pre-existing on a stashed treeand are untouched by this PR.
Note on the issue title
This issue's title is about Soroban fee-bump and surge pricing, while the
body, evidence, and task list are about concurrent signal lore editing.
This PR addresses the body. Flagging in case the two should be split.
🤖 Generated with Claude Code