Skip to content

seqlock-fix-4: the optimistic descent must not dereference a null child - #1872

Merged
swapnilpaliwal-sd merged 1 commit into
apps/integration-0.1.9from
fix/speculative-fetch-guard
Oct 9, 2026
Merged

swapnilpaliwal-sd merged 1 commit into
apps/integration-0.1.9from
fix/speculative-fetch-guard

Conversation

@swapnilpaliwal-sd

Copy link
Copy Markdown
Contributor

Closes #1869 — the parallel-solve crash is root-caused, captured, fixed and validated against a live reproduction.

The capture (lldb, java -j8, darwin-arm64, under a souffle -g memory-load loop — the missing repro ingredient):

thread #5, EXC_BAD_ACCESS (code=1, address=0x8)
frame #0: souffle::detail::btree<...>::insert(...) + 704
->  ldapr  w8, [x8]        ; acquire-load of a seqlock version at null+8

insert's descent reads next = cur->getChild(idx) under a still-unvalidated optimistic lease and immediately dereferences it (next->lock.start_read()). A concurrent split can expose a null child slot; x86's ordering keeps the window closed, arm64 under memory pressure opens it. This explains every observation: javascript 3/10 and java 2/4 crash rates, a different rule each time, arm64-only, pressure-gated, serial always clean.

The fix (overlay seqlock-fix-4, engine-id salted): a null child is treated as a failed validation and the insert restarts — the same contract the descent already follows for torn reads. Applied to BTree.h and BTreeDelete.h. Defense in depth in the same overlay: the flyweight fetch tolerates speculative indices (static empty value) and generic record unpack returns zeroed storage rather than an empty record's null data().

Validation, 1.2M-LOC java subject, interleaved runs under the identical load loop:

engine segfaults
fix-3 (unfixed), -j8 8/10
fix-4, -j8 0/10

and fix-4 serial AND parallel outputs are sorted-identical to the fix-3 serial reference across every relation. The engine-package stub test carries the new anchors (green), and build-engines.yml applies the same awk-extracted patch with per-file assertions — its awk anchor is also repaired (the heredoc's import line had drifted under it, which would have failed the next release build loudly).

With this in, the parallel-by-default gate from #1861 stands safely for 0.1.9.

The parallel crash finally captured (java -j8, darwin-arm64, under souffle -g memory
load): btree::insert's descent reads the child pointer under a still-unvalidated
optimistic lease and immediately dereferences it for the next lease — ldapr of the
child's seqlock version at address null+8. A concurrent split can expose a null slot;
on x86's strong ordering the window effectively never opens, on arm64 under memory
pressure it does (js 3/10 and java 2/4 observed). The guard treats a null child as a
failed validation and restarts the insert — the same contract every other torn read
in the descent already follows. Applied to BTree.h and its BTreeDelete.h twin.

Defense in depth, same overlay: the flyweight fetch returns a static empty value for a
speculative (out-of-range or unassigned) index instead of dereferencing the slot, and
the generic record unpack returns zeroed storage instead of an empty record's null
data(). Engine id salt moves to +seqlock-fix-4; the engine-package stub carries the
new anchors, and build-engines.yml applies and asserts the same extracted patch (its
awk anchor also repaired: the heredoc's import line had changed under it).

Validation on a 1.2M-LOC java subject, keycloak-sized facts, identical and interleaved
conditions under a souffle -g load loop: unfixed engine 8/10 segfaults, fix-4 engine
0/10, and every output relation of fix-4 serial AND parallel sorted-identical to the
fix-3 serial reference.

Co-authored-by: axiomcode-bot[bot] <334110751+axiomcode-bot[bot]@users.noreply.github.com>
@swapnilpaliwal-sd

Copy link
Copy Markdown
Contributor Author

The javascript leg repeats the result: under the same interleaved load loop the unfixed engine segfaulted 3/10 while fix-4 ran 0/10, and the fix-4 parallel output is sorted-identical to the unfixed serial reference across every relation. Both crashing languages now validate the same way.

@swapnilpaliwal-sd
swapnilpaliwal-sd merged commit b690a5d into apps/integration-0.1.9 Oct 9, 2026
12 checks passed
@swapnilpaliwal-sd
swapnilpaliwal-sd deleted the fix/speculative-fetch-guard branch October 9, 2026 19:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant