Skip to content

fix(signals): add a CRDT collaborative lore draft behind phase-141 - #364

Merged
Darkvader-ship-it merged 1 commit into
PHASE-STELLAR:mainfrom
Olumide-01:fix/p1-signal-crdt-concurrency
Sep 30, 2026
Merged

Darkvader-ship-it merged 1 commit into
PHASE-STELLAR:mainfrom
Olumide-01:fix/p1-signal-crdt-concurrency

Conversation

@Olumide-01

Copy link
Copy Markdown
Contributor

Closes #207

What this does

Concurrent edits to one signal clobber each other today. The existing
version + compare-and-swap contract is correct — it never lets a writer
silently overwrite a peer's row — but when many people type into the same
signal at once it answers the wrong question: one save wins and the rest
come back 409.

lib/__tests__/signal-crdt-benchmark.test.ts runs the workload the issue
specifies (50 writers, each editing from the same starting state without
seeing the others) against both strategies:

Strategy Preserved Outcome
version + CAS (current) 1/50 1 committed, 49 rejected with 409, text clobbered
Yjs CRDT draft (this PR) 50/50 50 updates folded into one snapshot, 0 rejected

So this layers a CRDT draft under the existing guard instead of
replacing it. The two solve different problems:

  • A CRDT merges concurrent proposals. It cannot decide which text is
    authoritative, and it cannot write revertible history.
  • The version guard decides what is authoritative, once, and records the
    pre-edit text so the revision stays revertible.

The commit path is still guarded

signal_crdt_docs / signal_crdt_updates are scratch state and never
write the signals row. A merged draft is promoted to committed lore only
through PUT /api/signals/[id] with from_draft: true, which lands in the
same editSignal CAS path as a manual edit and snapshots into
signal_versions as before.

Conflicts are still possible and still visible at that boundary: if a
colleague committed while your draft was open, PUT still answers 409
with current_version and a fresh ETag so the client can rebase. The
CRDT removes lost updates; it does not remove the need for a guard.

Changes

Area File Notes
Document model lib/signal-crdt.ts lore Y.Map of title/body Y.Text. Edits apply as a minimal prefix/suffix span diff, so two appends become two independent inserts instead of one overwriting the other.
Persistence lib/signal-crdt-store.ts Merges inside BEGIN IMMEDIATE; the update tail folds into the snapshot every 64 merges so it cannot grow unbounded. 256KB update cap.
Schema lib/sqlite-db.ts Two new tables plus an index. Purely additive, no migration.
API app/api/signals/[id]/route.ts Adds PUT (mandatory If-Match, requires both title and body so a revision is never half-applied). PATCH conflicts now carry current_version + ETag like the upvote path.
API app/api/signals/[id]/crdt/route.ts GET with ?state_vector= returns only the operations that client lacks; POST merges an update.
Client app/signals/[id]/use-signal-crdt.ts The hook await import()s the CRDT module, so Yjs stays out of the initial bundle while the flag is off.
UI app/signals/[id]/signal-detail-client.tsx Collaborative draft panel with sync state, contributor avatars, and a commit button.
Metrics lib/signal-version-metrics.ts signal_version_conflicts split by edit/put/upvote/reply, flagged separately when a CAS retry was attempted, plus CRDT merge and commit counters.
Store lib/signal-store.ts Emits the conflict metric at each existing guard.

Design decisions worth reviewing

HTTP, not WebSocket. The issue suggests prototyping Yjs over a
WebSocket. Yjs is transport-agnostic and its state-vector sync is a plain
request/response, so GET-with-state-vector carries the same wire format a
socket would without holding a connection open — and this app deploys to
Vercel serverless, where a long-lived socket per reader is not available.
The client posts on a short debounce and polls while the panel is open.

The store is the only thing that counts conflicts. An earlier draft
also incremented the metric in the route handler, which counted every
rejected CAS twice. Counters are aggregate only — no signal id, wallet, or
text ever reaches a label.

Flags. phase-141, defaulting off. Unset NEXT_PUBLIC_FEATURE_PHASE_141
and the CRDT routes return 404, the draft panel is hidden, and PUT still
works on its own as an If-Match-guarded full replacement. The
signal_crdt_* rows stay on disk as inert scratch state.

Testing

Note on the issue title

This issue's title is about Soroban fee-bump and surge pricing, while the
body, evidence, and task list are about concurrent signal lore editing.
This PR addresses the body. Flagging in case the two should be split.

🤖 Generated with Claude Code

Issue PHASE-STELLAR#207 reports that concurrent edits to one signal clobber each other.
The existing version+CAS contract is correct but answers the wrong question
when 50 people type at once — it keeps one text and rejects the rest. The
spike in lib/__tests__/signal-crdt-benchmark.test.ts measures it:

  Yjs CRDT (phase-141)      50/50   50 updates folded, 0 rejected
  version + CAS (current)    1/50   1 committed, 49 rejected with 409

So the two compose rather than compete. A Yjs CRDT draft in SQLite merges
concurrent proposals automatically and never writes the signals row; the
merged result is promoted to committed lore only through PUT with
from_draft: true, which reuses the same If-Match guard and signal_versions
snapshot as a manual edit. The guard is not removed, it moves to the one
place a decision has to be made: if a colleague committed while the draft
was open, PUT still answers 409 with current_version.

- lib/signal-crdt.ts: lore document (title/body Y.Text), minimal span-diff
  edits so two appends stay two inserts, state-vector helpers, convergence
  checks. The client imports it dynamically so Yjs stays out of the bundle
  while the flag is off.
- lib/signal-crdt-store.ts: merge under BEGIN IMMEDIATE, update tail folded
  into the snapshot every 64 merges, contributor attribution, 256KB cap.
- PUT /api/signals/[id] (If-Match required, full title+body) plus
  GET/POST /api/signals/[id]/crdt for HTTP state-vector sync. HTTP rather
  than WebSocket because the app deploys to Vercel serverless.
- signal_version_conflicts is counted once, in the store, so a rejected CAS
  is not double-counted by every layer that observes it. Counters are
  aggregate only: no signal id, wallet, or text reaches a label.
- Fixes diffLoreVersions, which was exported as @ts-nocheck referenced but
  never defined, breaking the narrative versions route.

22 new tests, including 50-writer no-lost-writes across two SQLite
connections. Docs updated in PROJECT_ARCHITECTURE.md and docs/TECHNICAL.md.
@drips-wave

drips-wave Bot commented Sep 28, 2026

Copy link
Copy Markdown

@Olumide-01 Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

@Darkvader-ship-it
Darkvader-ship-it merged commit 92c1f02 into PHASE-STELLAR:main Sep 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[INTEGRATION] Stellar Soroban Fee-Bump & Surge Pricing Blindness: Static 100 stroops

2 participants