improve performance of reading sandbox stats (oom, peak memory)

[syphar]

after each command in the build we see (and need) this:

most exec cat calls as <100 ms, but I regularly see calls between 2 and 10 seconds. Which is quite a waste, when we see that we currently do 4 calls after each container command (two for oom, two for memory peak).

First optimization was that we remember which cgroup version we have, so then we can save one call in the later checks.

Then I discovered that we also can get the same information directly in special files from the host. So I added this. From what I see, the locations likely speficic to the used setup / engine, so I left the exec cat approach in. I checked that the "fast" way would be used by docs.rs.

I don't like the added complexity, but the performance impact will likely be noticeable.

While we're on it I extracted the bigger cgroup stats code into a new module, container lifetime stuff stays in Container

Generally: I think most randomly show things are i/o bound, and increasing the bandwidth / iops will hopefully help.

[GuillaumeGomez]

Instead of implementing all that, why not using crates with this feature like sysinfo?

[syphar]

I did check two crates, they either didn't provide these specific metrics, or you couldn't get them for existing cgroups (only when the library also created them).

I'll check once more for crates. Sysinfo wasn't on the initial list.

[syphar]

• cgroups-rs has the needed functionality, is badly documented, and has compile errors when building on non-linux (inconsistent gating of the linux specific logic). could probably make it work when gating it to linux in rustwide itself ( annoying when doing this on my machine). Or trying to fix the library itself, using a git fork in rustwide, or adding a patch if it's small.
• in sysinfo the struct is called CgroupLimits, but the names (and the used stats) don't look like limits, but the current values? In any case, the memory-peak is missing, and the oom counter too. Is that something you would see there?

what do you think?

I feel like with a stable cgroup

[syphar]

checking more libraries later, you can point me to the points you had in mind

[GuillaumeGomez]

Seems like sysinfo will need an update. For now let's go. :)