Skip to content

fix(engine): keep at most a fixed number of segment files open - #94

Merged
venkat1701 merged 3 commits into
mainfrom
fix/segment-file-pool
Oct 14, 2026
Merged

venkat1701 merged 3 commits into
mainfrom
fix/segment-file-pool

Conversation

@venkat1701

Copy link
Copy Markdown
Collaborator

Closes #68

PageStore opened every segment file at startup and kept a FileChannel for each one until the engine closed. The open file count grew with the database, and smaller segments (#52) would have made that much worse.

Segment files now go through a FilePool that keeps at most max_open_segment_files of them open (256 by default).

How it works:

  • Opening the store only lists the segment files and reads their sizes with stat. A file is opened the first time a write, pread, map or sync needs it.
  • Each of those calls borrows the file's FileHandle for its own length. A handle that a call is using is never closed. When the pool is full, the handle unused for the longest time is closed to make room, and if every handle is in use the caller waits for one to come back.
  • A file closed to make room is forced first if it has writes that no sync has covered.
  • Each file counts its writes and only marks them durable once a force has returned. If a sync runs while another thread is still forcing a file it is about to close, the sync forces that file again instead of treating it as clean, so a checkpoint never goes ahead of its pages.
  • Writes put their file in an unsynced set, and sync() forces only the files in that set. Clean files are not touched, open or not.
  • Deleting a segment closes its handle once no call is using it, and the entry refuses to open again, so a late sync can't recreate a deleted file.

Sealed segments are still read through the read-only memory maps from #77. A map stays valid after its file is closed, so a mapped segment needs no handle for reads, and its handle is closed like any other once it is the least recently used. The pool builds on FileHandle from #87, so interrupted callers still can't close a shared channel.

The setting is EngineOptions.maxOpenSegmentFiles, and the server reads it from hstore.conf, HSTORE_MAX_OPEN_SEGMENT_FILES or --max-open-segment-files. Values below 1 or above Integer.MAX_VALUE fail at startup with a message that names the setting.

Open file count

Same probe on both sides (PageStore with 4 KiB pages and 16-page segments, 16,000 pages written, so 1,000 segments; descriptors counted from /dev/fd relative to before the store was opened):

main this branch
after writing, store open +1,002 +258
right after reopening +1,002 +2
after reading every page +1,002 +258
after close +0 +0

The +2 is pages.dir and pages.ref.

Tests

  • FilePoolTest: least recently used order, a dirty file is forced before it is closed to make room, sync() forces only files written since the last sync, eight threads on a pool of 3 across 16 files never see their channel closed mid-call and never see more than 3 open, and a closed entry is not reopened or recreated.
  • PageStoreTest: more than 100 segments with a pool of 4 stay within the pool (checked with the pool counter and with /dev/fd) before and after a reopen; a segment closed to make room is forced first and the next sync skips it; four readers and a writer share a pool of 2 without seeing a closed file.
  • ServerTest: the setting from the file, the environment and a flag, and the range check.

I checked that the tests catch the bug by breaking the code three ways: an unbounded pool fails 3 PageStoreTest tests, closing without forcing fails 2 tests, and letting the pool close a handle that is in use fails the contention test with a ClosedChannelException.

PageStore opened every segment file at startup and kept its FileChannel
until the engine closed, so the number of open files grew with the
database. Segment files now go through a FilePool that holds at most
EngineOptions.maxOpenSegmentFiles handles (256 by default).

A file is opened when a write, pread, map or sync needs it, and each of
those calls borrows the handle for its own length. When the pool is full
the least recently used handle that no call is borrowing is closed,
after forcing it if it has unsynced writes. Writes add their file to an
unsynced set and sync() forces only the files in it.

Sealed segments are still read through their memory map, which does not
need the file to stay open, and opening the store reads segment sizes
with stat instead of opening every file.
It sets EngineOptions.maxOpenSegmentFiles from hstore.conf,
HSTORE_MAX_OPEN_SEGMENT_FILES or --max-open-segment-files. Values below
1 or above Integer.MAX_VALUE fail at startup with a message that names
the setting.
@venkat1701
venkat1701 force-pushed the fix/segment-file-pool branch from 7218b7b to b01b196 Compare October 14, 2026 13:27
@venkat1701
venkat1701 merged commit a1ebd41 into main Oct 14, 2026
@venkat1701
venkat1701 deleted the fix/segment-file-pool branch October 14, 2026 13:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Every segment file stays open for the life of the engine

1 participant