Skip to content

fix!: Stop counting unflushed file cache as available memory under cgroups - #16

Merged
Mantisus merged 2 commits into
masterfrom
fix-cache
Sep 17, 2026
Merged

Mantisus merged 2 commits into
masterfrom
fix-cache

Conversation

@Mantisus

@Mantisus Mantisus commented Sep 16, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Under cgroups, available counted the whole inactive file cache as free. Right after an unsynced write, that cache holds dirty and writeback pages. The kernel cannot drop them until they reach the disk. A process that sized its allocations from available alone could therefore be OOM-killed.

Changed:

  • Each cgroup level now credits only the inactive file cache less the dirty and writeback pages. Cgroup v1 reads the total_* keys.
  • Under cgroups, used rises and available falls while writes are unflushed. Both return as the data reaches the disk.
  • A kernel that reports neither key keeps its budget, with the whole inactive cache credited as before.
  • memory.stat is parsed once per level, after the usage.

Added:

  • Description.raw_memory_unflushed_cache: the pages waiting to be written to disk at the level that set available. None where the kernel does not report them and on Windows.

Windows job objects are unaffected.

Breaking changes

  • The meaning of used and available changes under cgroups. used matches docker stats only while nothing is waiting to be written.
  • Description has a new required field. Code that builds it by position breaks.

Testing

  • Unit tests for the new credit and the new field.
  • A Docker e2e test writes an unsynced burst and checks that used and raw_memory_unflushed_cache rise with it. It fails on the old formula.

@Mantisus Mantisus self-assigned this Sep 16, 2026
@Mantisus
Mantisus requested a review from Pijukatel September 16, 2026 15:50

@Pijukatel Pijukatel left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have just some minor readability comments.

Comment thread src/proclimits/_cgroup.py Outdated
Comment thread src/proclimits/_cgroup.py Outdated
Comment thread src/proclimits/_cgroup.py Outdated
@Mantisus
Mantisus requested a review from Pijukatel September 17, 2026 12:11
@Mantisus
Mantisus merged commit 530890d into master Sep 17, 2026
31 checks passed
@Mantisus
Mantisus deleted the fix-cache branch September 17, 2026 14:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants