Skip to content

yrby-actioncable 0.3.0: drop unhealable gaps — verified on ActionCable AND AnyCable - #38

Merged
jpcamara merged 3 commits into
mainfrom
feat/gap-strike-defense
Jul 2, 2026
Merged

yrby-actioncable 0.3.0: drop unhealable gaps — verified on ActionCable AND AnyCable#38
jpcamara merged 3 commits into
mainfrom
feat/gap-strike-defense

Conversation

@jpcamara

@jpcamara jpcamara commented Jul 1, 2026

Copy link
Copy Markdown
Owner

Deliverable 2 of the gap-free-sync plan — the delivery-layer defense against orphaned-pending retry loops. (Deliverable 1, gap-free serving in yrby 0.3.0, stops the server-side poison; this stops a client poisoned by any route from looping.)

The loop it breaks

A causally-incomplete update triggers a resync so the gap heals as one complete delta. But a permanently-orphaned update — its missing dependency is gone for good — stays gappy through every resync. A client retransmitting it loops endlessly:

client sends {update} → server: update_ready? false → resync → client resends → repeat  (~several times/sec)

That's the "id-less frames several times a second" symptom.

The fix

After gap_strike_limit rejections of the same update on one connection (default 3), the channel settles it with an ack (new :dropped_unhealable outcome) and drops it, instead of resyncing again:

unless doc.update_ready?(update)
  if limit && sync_gap_strike(update) >= limit
    return :dropped_unhealable   # ack + drop, no resync
  end
  sync_request_resync(doc)
  return :gap
end
  • A healable gap heals within a resync or two, well under the limit — so this only trips on genuinely-dead updates.
  • Strikes are keyed by SHA-256 of the update, so an unrelated slow-to-heal gap on the same connection is never dropped early. The table is bounded (GAP_STRIKE_MAX_KEYS) so endless distinct gaps can't grow it.
  • Configurable via gap_strike_limit (class macro, mirrors max_frame_bytes); set nil to restore resync-every-time.

Honest limitation (documented)

Strike state lives on the channel instance. On plain ActionCable the instance is reused across a connection's messages, so strikes accumulate and the drop works. Under AnyCable each command gets a fresh instance, so the count resets per message and the drop never trips — it degrades to the prior resync-every-time behavior (no regression). A durable/Redis-backed counter could extend it to AnyCable if a real poisoned-client case appears; JP's actual incident was server-store-only, already covered by Deliverable 1.

Tests

  • drop + ack after the strike limit (never recorded/broadcast, no further resync)
  • gap_strike_limit nil always resyncs (never acks/drops)
  • distinct gaps track strikes separately (one hitting the limit doesn't drop another)
  • Full suite green: 95 Ruby runs, rubocop clean.

Bumps yrby-actioncable 0.2.3 → 0.3.0.

🤖 Generated with Claude Code

@jpcamara

jpcamara commented Jul 2, 2026

Copy link
Copy Markdown
Owner Author

Re-worked after evaluation — now verified on BOTH transports

The original design had four real problems, all fixed on this branch (now stacked on #39, which should merge first):

  1. It didn't work under AnyCable at all. Strikes lived on the channel instance; AnyCable builds a fresh instance per RPC command, so the count reset every message and the drop never tripped. Now the table persists via anycable-rails' state_attr_accessor (istate, round-tripped through anycable-go), declared automatically at include time; plain-ivar fallback without anycable-rails.
  2. The drop was invisible to clients — same {ack} for recorded and abandoned, so the client pruned and reported synced over lost data. The settle now carries "dropped" => true; yrby-client prunes (retransmitting an unhealable update loops forever) but surfaces it via onError(..., "ack-dropped"). Old clients ignore the key.
  3. Eviction was a defense bypass: 64 distinct gaps wiped the whole table, resetting tracked strikes. Now a single lowest-count entry is evicted, only on new-key insert — proven by a 72-update GapFlood fixture test.
  4. Unsynchronized strikes under ActionCable's worker pool → mutex. Plus: gap_strike_limit < 2 raises (strike 1 must resync before any drop), and a healed gap frees its slot.

End-to-end, both stacks (frontend/gap_strike.mjs): a poisoned client's unhealable update → resync, resync, then {ack, dropped: true}, never recorded; a healable gap still records with a plain ack.

  • Plain ActionCable (Puma, 2 workers): 9/9
  • AnyCable (anycable-go + gRPC RPC, fresh instance per command): 9/9 — istate strike persistence proven live

Unit: 107 Ruby (incl. a faithful AnyCable simulation — fresh instance per message + JSON istate round-trip), 54 client, rubocop/tsc clean.

@jpcamara jpcamara changed the title yrby-actioncable 0.3.0: drop unhealable gaps instead of resyncing forever yrby-actioncable 0.3.0: drop unhealable gaps — verified on ActionCable AND AnyCable Jul 2, 2026
@jpcamara
jpcamara force-pushed the feat/gap-strike-defense branch 3 times, most recently from fc1206b to 0ebecb3 Compare July 2, 2026 02:29
jpcamara and others added 3 commits July 1, 2026 22:48
…d AnyCable

A causally-incomplete update triggers a resync so the gap heals as one
complete delta. But a permanently-orphaned update (its missing dependency is
gone for good) stays gappy through every resync, and a client retransmitting
it loops endlessly. After `gap_strike_limit` rejections of the same update on
one connection (default 3, minimum 2 -- ArgumentError below), the channel
settles it with { "ack" => id, "dropped" => true } and drops it.

Transport support:
- Plain ActionCable reuses the channel instance across a connection's
  messages: strikes live on the instance, mutex-guarded (ActionCable
  dispatches to a worker pool, so two receives on one instance can race).
- AnyCable creates a FRESH instance per RPC command, so the table is
  persisted via anycable-rails' state_attr_accessor (istate, round-tripped
  through anycable-go), declared automatically at include time when
  anycable-rails is loaded. Without anycable-rails: ivar fallback.

Correctness hardening from the source review:
- The settle ack carries "dropped" so clients can tell durably-recorded from
  abandoned; yrby-client prunes the queue (retransmitting an unhealable
  update would loop forever) and surfaces it via onError("ack-dropped")
  instead of silently reporting synced over lost data.
- Strike-table eviction: at capacity a single lowest-count entry is evicted,
  only when inserting a NEW key -- a client cycling >64 distinct gaps can no
  longer wipe the table and reset a tracked key's count (defense bypass),
  and an existing key's count is never disturbed (starvation).
- A gap that finally records frees its strike slot.
- gap_strike_limit below 2 raises: strike 1 must send a resync (the heal
  attempt) before any drop can be justified.

Tests: strike-out drop + ack; nil disables; distinct gaps tracked separately;
dropped flag on settle only; eviction can't reset tracked strikes (72-update
GapFlood fixture); heal frees the slot; limit validation; AnyCable simulation
(fresh instance per message + JSON istate round-trip, faithful to
anycable-rails' state_attr_accessor) for both strike-out and heal-clears.
Client: dropped ack prunes + surfaces; plain ack stays silent.

Verified end-to-end on BOTH stacks with a new demo e2e
(frontend/gap_strike.mjs): a poisoned client's update is resynced twice, then
settled with dropped:true and never recorded; a healable gap still records
with a plain ack. 9/9 on Puma ActionCable; 9/9 on AnyCable (anycable-go +
RPC), proving istate strike persistence across fresh channel instances.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@jpcamara
jpcamara force-pushed the feat/gap-strike-defense branch from 0ebecb3 to af0fa64 Compare July 2, 2026 02:49
@jpcamara
jpcamara merged commit f360db0 into main Jul 2, 2026
7 checks passed
@jpcamara
jpcamara deleted the feat/gap-strike-defense branch July 2, 2026 02:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant