Skip to content

fix(engine): keep shared files open when a caller's thread is interrupted - #87

Merged
venkat1701 merged 6 commits into
mainfrom
fix/interrupt-safe-io
Oct 12, 2026
Merged

venkat1701 merged 6 commits into
mainfrom
fix/interrupt-safe-io

Conversation

@venkat1701

Copy link
Copy Markdown
Collaborator

Closes #86.

Why

The JDK closes a FileChannel for every thread when one thread is interrupted during I/O on it. The engine shares its channels between threads, so one interrupted commit closed the write-ahead log. After that, every commit and the shutdown checkpoint failed until the process restarted. Segment files, the change feed, pages.dir and pages.ref had the same problem.

What changes

  • page/FileHandle wraps a channel that several threads share:
    • It clears the caller's interrupt flag for the call and sets it again afterwards, so the caller still sees its interrupt.
    • If the channel was closed anyway, it reopens the file and runs the call again. That covers an interrupt arriving in the middle of the call, or an interrupt on another thread. Every call is positional, so running it again reads or writes the same bytes.
    • Once the handle itself is closed, it stays closed.
  • Shared channels:
    • SegmentFile and MappedLongs (behind pages.dir and pages.ref) use it.
    • So do the active write-ahead log segment and the change feed's segments.
  • Checkpoints and shutdown:
    • A checkpoint runs with the caller's interrupt flag cleared, since it also writes short-lived files (the catalog, the reference journal).
    • Shutdown does the same. Before, join on the maintenance thread returned straight away on an interrupted thread, so shutdown could close files while maintenance was still using them.
  • docs/storage/pages.md explains all this.

The WAL reader that recovery uses keeps its own channel per reader. An interrupt there only fails that reader, which is what an interrupt should do.

Testing

  • Full suite: 157 tests pass, and each commit builds and passes on its own.
  • FileHandleTest:
    • a flag that was set beforehand;
    • an interrupt in the middle of a call;
    • a channel closed by another thread;
    • a closed handle.
  • PageStoreTest: an interrupted thread writes, reads and syncs pages in the growing segment, then another thread keeps using the same segment.
  • PageDirectoryTest: an interrupted thread grows the directory past its first 8 MiB chunk.
  • InterruptTest:
    • commits, reads, replays the feed and checkpoints from an interrupted thread;
    • then commits 200 times from another thread;
    • then closes from an interrupted thread, reopens, and checks the data and the reference counts.
  • Each test catches its bug: I put each changed file back to its version on main, one at a time: SegmentFile, MappedLongs, WriteAheadLog, ChangeFeed and Checkpointer. Every time, at least one of the new tests fails.
  • Not covered by a test: the shutdown change. With it reverted, close still succeeds in the test, because the maintenance thread happens to be idle.
  • Native image: the build passes, and:
    • after loading examples/clinical-claims.hql, hstore check verifies 105 pages and the reference counts match;
    • after a SIGKILL partway through a 20,000-insert load, reopening replays 467 commits and the counts match;
    • COMPACT; runs and check passes afterwards.

A FileChannel closes itself for every thread when one thread is
interrupted during I/O on it. FileHandle wraps a channel that several
threads share: it clears the caller's interrupt flag for the call and
sets it again afterwards, and if the channel was closed anyway, by an
interrupt in the middle of the call or on another thread, it reopens
the file and runs the call again. Calls are expected to be positional,
so running one twice reads or writes the same bytes.
…d callers

A commit or read on an interrupted thread closed the segment file it
touched, and every thread using that segment failed with
ClosedChannelException from then on, including background compaction
and the checkpoint at shutdown. Growing pages.dir or pages.ref on an
interrupted thread did the same to those files. Do their I/O through
FileHandle.
…mmitters

Committing on an interrupted thread closed the active write-ahead log
segment, so that commit and every later one failed with "write-ahead
log append failed" until the process restarted. The change feed's
segments had the same problem for commits and for subscribers reading
them. Do their I/O through FileHandle.
…read

A checkpoint writes short-lived files, the catalog and the reference
journal, that an interrupt would fail. Shutdown waited for the
maintenance thread with join, which returns at once on an interrupted
thread, so files could be closed while maintenance was still using
them. Run both with the caller's interrupt flag cleared and set it
again at the end.

The new test commits, reads, replays the feed and checkpoints from an
interrupted thread, keeps committing from another one, closes from an
interrupted thread and checks everything is there after reopening.
@venkat1701
venkat1701 merged commit 459b8be into main Oct 12, 2026
@venkat1701
venkat1701 deleted the fix/interrupt-safe-io branch October 12, 2026 08:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

An interrupted thread closes the engine's files for every thread

1 participant