Skip to content

Avoid gRPC crash when a sync is cancelled - #75

Open
j0ntz wants to merge 2 commits into
masterfrom
jon/fix-grpc-stream-cancel-crash
Open

j0ntz wants to merge 2 commits into
masterfrom
jon/fix-grpc-stream-cancel-crash

Conversation

@j0ntz

@j0ntz j0ntz commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

CHANGELOG

Does this branch warrant an entry to the CHANGELOG?

  • Yes
  • No

Dependencies

none

Description

Asana task

Fixes the iOS crash in LightWalletGRPCService.blockStream when a ZEC wallet's sync is stopped (pause, resync, logout) while blocks are downloading.

Root cause. gRPC-Swift 1.8.0 (the newest version on CocoaPods) holds _lock in PassthroughMessageSource.consumeNextElement() while it registers a task cancellation handler. If the reading task is already cancelled, Swift runs that handler immediately on the same thread, and the handler takes _lock again. NIO's lock is error-checking in debug builds: the relock returns EDEADLK and the precondition at lock.swift:74 traps. Release builds can deadlock or trap. The crash report's faulting thread shows this chain (consumeNextElement -> swift_task_addCancellationHandler -> the onCancel closure -> Lock.withLock). A channel or call freed under the iterator would not produce that stack. Upstream replaced this code in 1.11.0 (grpc/grpc-swift#1477), which ships only through SwiftPM.

Fix. ios/CancellationSafeStream.swift adds cancellationSafeResponses() on GRPCAsyncServerStreamingCall. An uncancelled task drains gRPC's stream into an AsyncThrowingStream. When the consumer is cancelled or drops the stream, the RPC is cancelled instead, and gRPC's cancellation handler never runs. A consumer cancelled mid-stream still gets CancellationError, which keeps the SDK's existing "sync cancelled" handling. scripts/updateSources.ts rewrites all six server-streaming call sites in LightWalletGRPCService.swift and fails if an SDK update changes the shape it matches, so a bump cannot drop the fix unnoticed.

Tests. ios-tests/ is a SwiftPM package pinned to the gRPC-Swift and SwiftNIO versions the app links. It is outside the npm package and not run in CI; run swift test from ios-tests.

  • CancelRepro raw reproduces the field crash (same precondition, SIGTRAP). The guarded variant ends with CancellationError.
  • A stress run of the SDK's blockStream shape with random cancellation crashed 5 of 5 runs unguarded and ran 20,000 rounds clean guarded.
  • 8 unit tests cover cancellation, element and error pass-through, and RPC cancellation on cancel or drop.

In-app (iOS sim, edge-react-gui). Stop loops drove a syncing ZEC wallet through resync and pause/unpause while blocks were downloading. The SDK logs confirm the stops landed mid-download (Sync failed with error: CancellationError()).

  • Unmodified gRPC: 48 stop cycles on each build, no crash on either. gRPC checks Task.isCancelled just before the vulnerable call, so the in-app window is narrow.
  • To hit the window reliably, a local test-only edit added a 50 ms wait (without the lock) at the top of gRPC's consumeNextElement(), before it registers the cancellation handler. Both builds used the same edited Pods source.
    • Unfixed 0.13.4: crashed in round 2 with EXC_BREAKPOINT (SIGTRAP). The faulting stack matches the field crash: Lock.withLock <- PassthroughMessageSource.consumeNextElement() onCancel closure <- swift_task_addCancellationHandler <- GRPCAsyncResponseStream.Iterator.next <- LightWalletGRPCService.blockStream.
    • This branch: 18 resync + pause rounds and 40 fast pause/unpause cycles, no crash, and sync resumed each time.
  • The gRPC edit was reverted and is not part of this PR.

react-native-piratechain vendors an SDK with the same gRPC stream pattern and likely has the same exposure. It is not changed here.


Note

Medium Risk
Changes iOS sync cancellation and all gRPC streaming RPC consumption paths; incorrect wrapping could affect stop/resync behavior, though tests target the failure mode.

Overview
Fixes an iOS crash when stopping wallet sync (pause, resync, logout) while block download is in progress. gRPC-Swift 1.8.0 (CocoaPods) can SIGTRAP/deadlock if the task reading a response stream is cancelled while PassthroughMessageSource holds its lock during cancellation handler registration.

Adds CancellationSafeStream and cancellationSafeResponses() on GRPCAsyncServerStreamingCall: an uncancelled task drains the gRPC stream; when the consumer cancels or drops the stream, the RPC is cancelled instead so gRPC never hits the bad path. Cancelled consumers still see CancellationError.

updateSources.ts rewrites all server-streaming sites in vendored LightWalletGRPCService.swift to use the wrapper and throws if an SDK bump changes the expected stream shape. The helper is included in the podspec. ios-tests/ (SwiftPM, not in CI) reproduces the crash and tests the guarded behavior.

Reviewed by Cursor Bugbot for commit 9858cd9. Bugbot is set up for automated code reviews on this repo. Configure here.

Test evidence

9858cd9
Test the gRPC stream cancellation guard
2026-09-25
🪓 added a 50 ms wait in gRPC-Swift PassthroughMessageSource.consumeNextElement before cancellation-handler registration to widen the race window

1. zec wallet menu pause resync mid sync

2. zec wallet alive after 48 stop cycles

3. 🪓 zec wallet alive after forced race cycles

gRPC-Swift 1.8.0, the newest release on CocoaPods, holds a lock while it registers a task cancellation handler in PassthroughMessageSource.consumeNextElement(). If the consuming task is already cancelled, the handler runs immediately and takes the same lock again, which traps (or deadlocks in release builds). Stopping a synchronizer cancels the block download task mid-stream, so iOS crashed in LightWalletGRPCService.blockStream.

update-sources now routes every response stream in LightWalletGRPCService through cancellationSafeResponses(): an uncancelled task drains the gRPC stream, and a cancelled consumer cancels the RPC instead of reaching gRPC's cancellation handler. The script fails if an SDK update changes the call shape it rewrites.
ios-tests is a SwiftPM package pinned to the gRPC-Swift and SwiftNIO versions the CocoaPods build links. CancelRepro reproduces the crash (the same lock.swift precondition as the field report) by reading gRPC's message stream from a cancelled task, and the tests check that the guarded stream throws CancellationError there instead, survives repeated cancellation of the SDK's blockStream shape, passes elements and errors through, and cancels the RPC when its reader is cancelled or dropped. Run with swift test from ios-tests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant