Skip to content

feat(mem): a central allocation core, the memory migrated onto it, and results spilled to disk under budget - #2202

Merged
DeusData merged 9 commits into
mainfrom
feat/memory-core
Sep 14, 2026
Merged

DeusData merged 9 commits into
mainfrom
feat/memory-core

Conversation

@DeusData

@DeusData DeusData commented Sep 13, 2026

Copy link
Copy Markdown
Owner

Memory in this project is allocated through one core, with a linter that keeps new code on it and phase attribution in the index pipeline. This increment migrates the memory that matters onto it, proves where an index's memory actually goes, removes the waste that proof exposed — and makes the memory budget a promise: over budget, extraction results are parked on disk and the run completes at the floor instead of aborting. The Linux kernel indexes on the default budget again (it aborted on v0.10.9-rc), and indexes on a 16 GB budget through the spill path.

Every number is from a real index on an M5 Pro (48 GB) with CBM_MEM_PHASES=1, release build.

What the proof said

The 2026-07 research blamed the graph buffer for the peak. Measured, the graph buffer is 3% of the memory (0.55 GB peak on the Go corpus). The per-file extraction results are the memory:

Go corpus (21,875 files, 345 MB of source)
written into the per-file result arenas 14,804 MB
reachable from the results (records + strings + retained source) ~3,400 MB
arena capacity charged (block doubling headroom) 28,817 MB

The rest is temporaries — cbm_node_text copies at 496 call sites, per-node QN strings, abandoned generations of every growable array — that an arena cannot free and that the result kept alive until resolve ended, because the result owned the arena. On the kernel, resolve then grew the results 8.9 → 13.3 GB used / 17.2 GB capacity because the C walk allocated its working set in the result arena and stored arena pointers into the shared registry. After that, 7 GB of the kernel's post-resolve memory had no class at all. Git history is not a factor (streamed git log, freed per commit).

What changed

Attribution. Graph buffer (87 raw sites → 0), arena blocks, the tree-sitter slab allocator, the bound tree-sitter/SQLite allocators, the semantic path (pass_semantic_edges, pass_similarity, minhash: 80 sites), the hash table (Verstable, one class per table via cbm_ht_create_in) and CBM_DYN_ARRAY allocate through the core. Classes arena, ts_tree, hash_table, dyn_array join the table. Two cross-boundary frees of buffer-owned strings (pass_importance, pass_complexity) now go through cbm_gbuf_node_set_properties_json. Linter baseline 4086 → 3916.

Result compaction (internal/cbm/result_compact.c). At the end of a file's extraction, everything reachable from its result is measured, copied into one exact-size arena (strings interned by content within the file), and the working arena is handed back. Later appends restart growth at the default block (cbm_arena_init_exact, arena grow_size). The stale duplicate internal/cbm/arena.h — same include guard as the foundation header, winning or losing by include order — is a shim.

Record slimming. CBMCall 320 → 72 B (the eight inline argument slots, 256 B and mostly empty, are allocated on first capture); CBMUsage 48 → 40 B.

The per-file overlay contract (src/pipeline/pass_lsp_cross.c). One lifecycle for every language's cross-file resolve: scratch arena → per-file overlay registry chained to the sealed shared base → the walk runs against the overlay → resolved calls appended into the result → scratch destroyed. type_registry gains chain-aware iterators (every yielded index belongs to it->reg) and copy-on-write refinement (cbm_registry_func/type_for_update); the C, C#, Go and Python walks adopt them. An ASan heap-use-after-free in lsp_resolution_probe forced this: the C walk stored per-call arena objects into the shared registry, and a kernel run with a naive scratch arena lost 1,045 edges to garbage reads. It also ends unsynchronized writes into the shared registry from parallel resolve workers.

The 7 GB with no class, found and cleaned. gbuf_index tables and keys were 2.5 GB live and 7.1 GB at peak: the 18 worker buffers built the five secondary indexes they never query (cbm_gbuf_new_worker builds none), and 18M per-key index arrays started at 8 slots (cbm_da_push_min starts them at 2); edge property JSON is 81% {} (interned). The semantic pass allocated a 512-token stride per function up front (packed tokens with offsets). Edge dedup keys are 128-bit hashes instead of sprintf'd strings. Each worker keeps one reusable working arena between files (cbm_work_arena_take/give: rewind, not free), and every phase mark returns freed pages to the OS.

Budget metric. cbm_mem_over_budget compares the charged footprint — max(phys_footprint, mimalloc commit) on macOS — not resident_size: after extraction the kernel worker sat at 17.4 GB RSS with 5.5 GB charged (purged pages stay resident until the OS wants them), and phys_footprint under-reports after MADV_FREE_REUSABLE cycles.

Spill / admission control (internal/cbm/result_spill.c, the extract gate in pass_parallel.c). At 15/16 of the budget — or over it, or with CBM_MEM_SPILL=1 — the gate latches spill mode: every compacted result is parked on disk as header + block in one append-only file per writer under <cache>/spill/, the results already cached are swept out by every worker, and registry build, def collection, surface rows, resolve and the infra-route passes load a result only for the moment they read it (relocated by base delta). The abort path stays closed while a sweep runs or a cached result remains; it fires only at the floor (graph + registries + in-flight files). A retained parse tree is dropped with the parked result and re-parsed on load; a result that owns sub-results stays in memory.

Every consumer of the result cache reads a slot through cbm_pipeline_result_acquire()/release() (pipeline_internal.h): a parked slot is NULL in the array. The transient loads found what had been borrowed from results all along: a def label that was the result's own pointer (SIGBUS in the Go cross registry builder), two JVM helpers that passed their input through, the infra-route passes indexing the array (39 → 0 __route__infra__ nodes, INFRA_MAPS gone) and their dedup table borrowing sr->value keys (the insert spun forever on freed memory), and a latch that was visible before the store was open (a peer found nothing to park and failed the run while its neighbour wrote 9 GB). The arena rewind reuse branch had defeated the nblocks = CBM_ARENA_MAX_BLOCKS OOM seam the suites use; it now reuses only blocks that exist. The incremental and probe routes still index the cache array themselves and therefore never spill (ctx->spill_allowed stays false there) — follow-up.

The semantic pass under a budget (pass_semantic_edges.c). The pass held every function's tokens, the corpus's per-document token ids and the vectors being stored at once — 4.9 GB on the kernel on top of the floor, and nothing sized it to the headroom. Now, when the charged footprint plus that transient would cross the budget, phases 2–4 run per batch of functions (tokenize → count into the corpus → free; finalize once; tokenize again → vectorize → store → free), pages returned to the OS after every batch. Tokenization is deterministic, so both passes see the same tokens and the graph does not change; the run pays with a second tokenize. CBM_SEM_BATCH=<n> forces the batch size. The corpus frees its per-document token ids at the end of finalize, and mem.semantic.step lines (CBM_MEM_PHASES=1) attribute the pass's memory per sub-phase.

Cross-allocator free, found and closed. The semantic corpus and tokenizer now allocate through the core (54 raw sites → 0), and that is not attribution only: cbm_sem_tokenize handed out libc strdup blocks that the pass freed with cbm_freemi_free in the production build — since the pass moved onto the core. No libc-backed test build can see that. The core now refuses a block it does not own (mi_is_in_heap_region) when CBM_MEM_PHASES=1 is set, fatally; every proof run in this PR ran with it armed and reports zero foreign blocks.

Accounting without contention. The core charged every allocation and free to shared per-class atomics; with 18 workers on 18 cores every block bounced the same three cache lines, and the same-machine bench against the shipped v0.10.8 showed it as CPU time rising faster than wall time for the same graph (Kotlin 38 → 121 s CPU, +58% wall; TypeScript 210 → 596 s, +67%; Go 177 → 312 s). The deltas are thread-local now and reach the shared counters once per 256 KB or 512 blocks of change per thread, when a parallel-for worker ends, and whenever a reader looks. Phase marks read after the join and are exact; a class peak is a diagnostic and can be low by threads × 256 KB. Result: Kotlin 10.9 s (the residual +22% against v0.10.8 tracks its +34% nodes), TypeScript 33.9 s (v0.10.8: 33.4), Go 48.4 s (v0.10.8: 52.6), Java 81.1 s (v0.10.8: 79.8); graphs unchanged.

SQLite on a heap of its own. The same bench put the kernel's write step at 132 s where v0.10.8 needed 9.7 s, all of it in one coverage-publish statement. A stack sample named the cost: mimalloc's free-page search and its periodic heap collect walking the graph's pages — millions of them once the graph lives on the core — for every statement-journal chunk SQLite allocates. SQLite now allocates from a mimalloc heap of its own per thread (mi_heap_new; mi_free is heap-agnostic, so nothing else changes), and the periodic collect is set to the maximum mimalloc allows, since the explicit release at phase marks and spill sweeps is where memory goes back. Coverage publish 132 → 9.4 s; profiled kernel 376 → 248 s (v0.10.8: 296 s).

Formatting without a process lock. On macOS every vsnprintf takes the process locale under an unfair lock (localeconv_l inside __vfprintf); eighteen workers formatting type names on the C# corpus collapsed into it — the 10 MB JIT test files took 68 s each instead of under a second — and cbm_arena_sprintf paid it twice per string (size pass, write pass). Each thread now formats with a C locale object of its own (vsnprintf_l), and the common short string is formatted once into a stack buffer.

Per-file budgets that cover the whole file. The 5 s per-file budget covered the parse only; the LSP walks after it had none. C# JIT stress files (a 23 MB single expression among them) parsed inside the budget — the old wall-clock parse timeout used to drop them, which hid the rest — and then held a worker for 346 s each in the usage walk (tree-sitter's ts_node_parent descends from the root: quadratic on a deep tree), and the cross-file resolve ran on the same trees. Three rules now, one site for every language (cbm_extract_file_ex, honoured by cbm_pxc_dispatch_file): a parse that used more than half the budget disqualifies the file from the LSP walks (lsp_skipped); the unified walk checks its thread CPU time every 1024 nodes against six budgets and stops there, keeping what it found (walk_truncated, implies lsp_skipped); a file whose parse plus walk spent the budget is skipped by the LSP walks as well. Each decision is logged with the path (extract.lsp.skipped, extract.walk.truncated). C# extraction 355 → 39 s; the whole index 463 → 128 s wall and 27 → 11 GB peak against v0.10.8 with 299 more nodes. The walk cuts exactly the five JIT stress files (hugeexpr1, hugeSimpleExpr1, HugeArray1, HugeField1/2); the 15 MB System.Runtime.Intrinsics reference file walks to the end and, with four generic-nesting JIT regression tests, only skips the LSP walk under the parse-plus-walk rule.

The crash behind the C# bench. dotnet/runtime is 42,555 files, and its resolve phase died with SIGBUS on every run of this branch. A libc-backed ASan build of the server named it: heap-use-after-free in c_adl_resolve, reading a type name that another worker had allocated in its per-file scratch arena and freed at the end of its file. The C++ class walk refines a method's return type when the declaration is more specific than the pre-registered one (NAMED → pointer/reference/template), and it did that by casting the chained lookup result to non-const and writing a scratch-arena signature into it — into the sealed shared base when the entry lived there. The overlay contract had exactly one bypass, and it was this cast; every lookup returns const, so a grep for the cast is the audit. The refinement now goes through cbm_registry_func_for_update: copy-on-write into the overlay, the base untouched. Test: clsp_method_return_refinement_is_copy_on_write — RED on the cast, GREEN on the accessor. The fixed ASan build then indexed the whole corpus clean: 42,555 files, 1,224,981 nodes, 5,794,320 edges, worker exit 0, no report.

Backing. The production build backs the core with mimalloc explicitly (never the global override, which macOS's two-level namespace forbids); the test build keeps libc so the sanitizers see every block — the split the tree-sitter/SQLite bindings already use.

Proof

before after
Go peak RSS (all cbm processes) 16.7 GB 4.0–5.2 GB
Go result arenas used / capacity 14,804 / 28,817 MB 856 / 856 MB
Go wall 55–59 s 52–60 s
Go, CBM_MEM_SPILL=1 1.7–2.3 GB peak; 21,875 results parked (950 MB on disk); graph identical (39 infra routes, 158 INFRA_MAPS, 21,875 surface rows)
Go, CBM_MEM_BUDGET_MB=1200 aborts latched at 1,204 MB, first sweep → 768 MB, completes at 1.99 GB peak
Kernel, default budget (24,576 MB) aborts (mem.budget.exceeded) completes, no spill
kernel extraction 35.2 GB RSS 16.5 GB RSS / 5.6 GB footprint / 15.3 GB committed; results 8.9 GB = capacity
kernel resolve, tracked 24.6 GB 19.1 GB
kernel peak RSS 27.3 GB 20.1 GB
Kernel, CBM_MEM_BUDGET_MB=16000 aborts completes through spill: latched at 17.2 GB charged, 89,731 results parked (9.3 GB on disk, 370,188 loads); extraction end 6.2 GB RSS / 5.1 GB committed; resolve 8.6 GB RSS / 11.5 GB committed; peak RSS 16.6 GB (the semantic-pass plateau, 13.3 GB RSS, is graph-derived and outside what spill can move); nodes identical, edges 15,773,598
Kernel, CBM_MEM_BUDGET_MB=15000, before the semantic batching aborts completes; every phase mark under 15,000 MB by every metric (max RSS 13.9 GB, footprint 13.7 GB, commit 11.5 GB), but the process RSS high-water mark reached 16.9 GB inside the semantic pass (class peak 4.9 GB against 0.8 GB live)
Kernel, CBM_MEM_BUDGET_MB=15000, with the anticipatory latch aborts completes; spill mode enters at 14,062 MB (near_budget); the charged footprint never exceeds 11.8 GB at any mark; the process RSS high-water mark is 15.1 GB in extraction and 15.3 GB during the final write-out (1–2%: the in-flight window and the dump transient); nodes identical, zero foreign blocks
Kernel, CBM_MEM_BUDGET_MB=15000, with the semantic batching only aborts completes; the charged footprint stays under 15,000 MB through the whole semantic pass (14,018 → 14,381 MB across its sub-phases, 6 batches of 125,644 functions); every phase mark under budget by every metric (max footprint 12.6 GB, max commit 11.5 GB); the process RSS high-water mark, 15.65 GB, is set in extraction at the moment the gate latches (15,002 MB charged), before the first sweep lands — a 4% overshoot of in-flight work; nodes identical, zero foreign blocks
kernel graph 8,529,729 nodes / 15,772,562 edges 8,529,729 nodes / 15,772,669 edges

What the budget holds today. The extraction gate acts on the charged metric and the sweeps bring the run to its floor; the semantic pass sizes its token phases to the headroom. What no gate can move is the floor itself — the graph, the registries, and the semantic pass's retained working set (the per-function records with their quantized codes, the corpus entries, the tf-idf arrays: ~5 GB on the kernel) — so a budget below floor-plus-retained is met at the phase marks and overshot inside the semantic pass by the difference. On the kernel at 15 GB that no longer happens: the semantic pass's retained set (2.5 GB of per-function records, ~1 GB of corpus, ~1 GB of tf-idf arrays) plus the 10.6 GB floor fits once its transient is batched and its pages are returned per batch. The latch moment itself is anticipated: spill mode enters at 15/16 of the budget, because the gate observes the crossing per file pull and the 18 workers' in-flight files would otherwise carry the charge ~4% past it before the first sweep lands; what remains is 1–2% of in-flight work and the final write-out.

Nodes are identical to the release bench; the edge delta is inside the ±400 run-to-run band the kernel shows between two runs of identical code. On Go, two identical no-spill runs differ by 21 gRPC Route nodes / 40 HANDLES edges (a Route node is minted for whichever of several .proto copies a worker reaches first) — a pre-existing nondeterminism with its own item; the spill runs sit inside that band.

Bench against the shipped v0.10.8 (same machine, same driver)

corpus wall s (v0.10.8 → final) peak RSS GB CPU s nodes edges CALLS
perl 5.7 → 5.3 0.09 → 0.05 5.9 → 4.7 2,351 → 2,348 5,254 → 5,124 1,981 → 1,978
php 6.3 → 5.8 0.66 → 0.20 12.7 → 11.6 13,590 → 16,561 (+21.9%) 56,365 → 57,182 8,468 → 10,484 (+23.8%)
rust 6.8 → 6.3 1.15 → 0.67 19.2 → 15.6 18,404 → 18,405 97,423 → 97,479 26,402 → 26,033
c 5.9 → 5.7 0.16 → 0.19* 7.8 → 8.6 38,623 = 135,644 = 36,849 =
kotlin 8.9 → 9.4 1.55 → 0.96 38.2 → 40.8 25,083 → 33,494 (+33.5%) 263,711 → 203,562 (−22.8%) 29,901 → 33,518 (+12.1%)
django 10.9 → 10.2 3.13 → 1.12 40.8 → 35.7 55,371 → 55,377 343,613 → 293,715 (−14.5%) 86,086 → 61,672 (−28.4%)
go 52.6 → 42.9 (−18%) 17.17 → 3.93 (−77%) 176.9 → 170.0 292,994 → 334,488 (+14.2%) 2,002,252 → 1,795,311 (−10.3%) 314,660 → 321,441 (+2.2%)
java 79.8 → 73.8 (−8%) 27.64 → 11.00 (−60%) 644.6 → 543.1 693,250 → 693,129 5,633,006 → 5,523,596 (−1.9%) 1,552,246 → 1,542,428 (−0.6%)
csharp 463.3 → 128.1 (−72%) 27.03 → 11.15 (−59%) 1,596.9 → 1,208.6 1,224,682 → 1,224,981 5,848,553 → 5,794,310 (−0.9%) 1,036,932 → 1,030,918 (−0.6%)
typescript 33.4 → 33.6 12.00 → 2.49 (−79%) 210.3 → 188.5 291,791 → 291,605 865,796 → 835,285 (−3.5%) 83,558 → 83,688
linux (kernel) 294.8 → 233.3 (−21%) 27.51 → 25.08 (−9%) 2,442.4 → 1,403.0 (−43%) 8,529,900 → 8,529,729 15,985,598 → 15,772,630 (−1.3%) 1,971,880 → 1,970,114

* the chained run read 0.78 GB for C once; two standalone reruns read 0.19 and 0.24 GB (a sequencing artifact of the kill-then-start between corpora, not the binary).

No corpus is slower beyond noise. C# −72% wall is the walk budget, the locale-free formatter and the use-after-free fix that let the run complete at all; the kernel −21% wall / −43% CPU is the batched accounting and the SQLite heap. Memory is −55…−80% on the four large LSP corpora; the kernel at 36 GB never spills and its floor is the graph. Graph deltas: Every graph delta against v0.10.8 is main's work between the release and this branch's base (339b3f4), not this PR: a build of the base commit run through the same driver reproduces the final graphs exactly on django (55,377 / 293,715 / 61,672) and kotlin (33,494 / 203,562 / 33,518), within the pre-existing gRPC Route jitter on go (30 nodes, 27 edges, CALLS identical), and on C# with the same node count; this PR's own C# difference against the base is 5 CALLS (the five truncated JIT stress files) plus one ijwhost swap and 563 edges out of 5.79 M, and the base takes 448 s / 29.7 GB for it.

Budget checks, final binary. C# at the default budget: identical to the 36 GB run (128 s, 11.2 GB). Kernel at 15,000 MB: spill latches at 14,061 MB (near_budget); the charged high-water — the budget metric itself, peak_charged_mb on every mem.phase line — is 14,582–14,732 MB across two runs, 97–98% of the budget; the final sweep at extraction end parks 45,165 results and takes the charge from 14.6 to 4.2 GB before resolve, which peaks at 11.5 GB committed; every phase mark under budget by every metric; nodes identical; 224 s. (An earlier revision of this branch read 16.8 GB there: the 18 worker graph buffers each kept a dense id array over the global id space and doubled in lockstep, and spill mode never parked the results cached before the latch — both fixed in the follow-up commit.) The RSS high-water, 18.7 GB, is pages already purged and not yet reclaimed: the footprint never exceeds 7.3 GB at a mark. At the default budget (24,576 MB) the charged high-water is 22,272 MB, under budget, no latch; RSS high-water 29.5 GB, 7 GB of it purged-not-reclaimed pages. Go at 1,200 MB: completes at 1.71 GB summed peak.

Verification

  • make -f Makefile.cbm lint-ci green on the final tree (cppcheck, clang-format, NOLINT whitelist, memory-core linter: 3862 raw sites across 85 files, none grew).
  • build/c/test-runner extraction parallel pipeline: 699 passed, 0 failed (the deterministic Kotlin/Rust failures of an earlier build were a mis-nested LSP dispatch, fixed). c_lsp lsp_resolution_probe: 850 passed. Revert check: clsp_method_return_refinement_is_copy_on_write fails on the cast (base signature pointer changed), passes on the accessor.
  • ASan/UBSan server build (libc-backed, -O1 -g) over the whole dotnet/runtime corpus: worker exit 0, no report (before the fix: heap-use-after-free in c_adl_resolve on the first resolve).
  • scripts/test.sh (the full macOS leg) on the final tree: 8049 passed, 0 failed, 10 skipped (141 suites, 18 jobs); every contract test and the security-strings allow-list test passed. Linux (Colima) and Windows (VM) legs not run for this increment by decision (macOS-only result set); the Windows VM is currently down (utmctl start fails) — recorded, not bypassed.

Diagnostics that made this measurable stay in, behind CBM_MEM_PHASES=1: the extract.arenas and extract.census lines, the CBM_MEM_RELEASE=1 release-to-OS probe, and peak_charged_mb on every mem.phase line — the high-water mark of the budget metric itself (cbm_mem_peak_charged), next to the RSS peak that counts pages already purged but not yet reclaimed.

Supersedes #2201.

…phase attribution in the index pipeline

Memory in this project is now allocated through ONE core. This lands the
core, the gate that keeps new code on it, and the instrumentation that lets
an index run say which class of memory grew in which pass -- the three things
the 2026-09-13 audit found missing.

WHAT THE AUDIT FOUND

There was no centralized allocation layer. foundation/mem.h was policy and
measurement only (budget, RSS, over_budget, memory map, phase marks) with no
alloc/free wrapper, so the budget could be observed after the fact and never
enforced. Raw allocator calls in the tree: 4086 across 92 files. The
extraction engine had already adopted arenas (1301 call sites); the graph
buffer -- which holds the 35 GB peak of a kernel index -- used none,
allocating every 64 B node, 48 B edge and every name/qualified_name/
properties_json individually.

Two further facts made a self-accounting core the only workable design:

  1. cbm_mem_map_collect() walks THIS thread's mimalloc heap only. Walking
     the process-wide heap is a data race TSan caught on macOS, so it is
     deliberately not done, and an 18-worker index attributes almost
     nothing -- the rest lands in residual.
  2. The mimalloc global override is ON for Linux/MinGW and permanently OFF
     for macOS (two-level namespace: this binary's free becomes mi_free while
     system libraries keep allocating from the system zone, and a pointer
     crossing that boundary aborts). On macOS ordinary malloc is served by
     the system allocator; the startup audit reports owned_classes=0/6. Any
     accounting that assumes mimalloc owns the pointer is blind on an entire
     platform.

THE CORE (src/foundation/mem_core.{h,c})

cbm_alloc / cbm_calloc / cbm_realloc / cbm_mem_strdup / cbm_free, each tagged
with an allocation class (gbuf_node, gbuf_edge, gbuf_string, gbuf_index,
extract, semantic, dump, store, other). Per-class live bytes, live blocks and
peak, kept in atomics -- thread-safe by construction, identical on every
platform, independent of which allocator serves malloc. libc semantics are
preserved exactly (alloc(0) is freeable, free(NULL) is a no-op, realloc(NULL)
allocates, a failed realloc leaves the block intact) so adoption is a
mechanical rename and never a behaviour change.

No per-allocation header: a {class,size} prefix costs 16 bytes on ~100M blocks
at kernel scale, 1.6 GB of overhead to measure a memory problem. Sizes come
from the platform usable-size query (malloc_size / malloc_usable_size /
_msize), which is correct under either allocator and free. Where no query
exists (BSD) the request size is charged, which understates -- the safe
direction for a diagnostic. A mismatched class on free clamps at zero rather
than wrapping, so a small drift can never masquerade as a colossal leak.

Arena-backed allocators report in bulk through cbm_mem_class_add_external /
remove_external so their memory appears in the same table; the extraction
engine must not be rewritten to per-object allocation, that would undo the
batching that keeps its allocation count low.

THE LINTER (scripts/lint-memory-core.py, wired into make lint-ci)

Raw malloc/calloc/realloc/free/strdup/strndup outside the core is a defect.
4086 sites cannot migrate at once, so the gate is a ratchet: a checked-in
baseline (scripts/memory-core-baseline.txt) records each file's count and
the build goes red when any file grows, or a new file appears with any.
Files only ever go down; the baseline line is lowered in the change that
migrates the file, and --strict fails a stale baseline. Comments and string
literals are stripped first, so prose that mentions malloc( cannot trip it --
the security audit already bit on exactly that with fork(. Proven both ways:
one appended malloc turns it red naming the file (83 -> 85), restoring turns
it green. Scans src/ and internal/cbm/: cli, mcp, daemon, store, pipeline.

PHASE ATTRIBUTION (src/pipeline/pipeline.c)

cbm_mem_phase_mark and the new cbm_mem_class_log now fire at pipeline.begin
and at every pass.timing site: parallel_extract, the six sequential passes,
the eight predump passes, tests and dump_and_persist, with peaks reset per
index. Both instruments existed in foundation/ and were wired only into MCP
request handling, never into the index pipeline -- which is where the memory
is. Locating the kernel peak this week required an external RSS sampler
because nothing in-process could say which phase it was in.

PRESSURE PRIMITIVES (system_info.c, mem.{c,h}, platform.h)

cbm_system_available_ram() -- macOS host_statistics64 (free + inactive +
purgeable), Linux MemAvailable, Windows GlobalMemoryStatusEx; 0 when unknown,
never cached -- and cbm_mem_system_under_pressure(), true below 12.5% of RAM
available and false when unknown so nothing ever aborts on a guess. Landed
UNWIRED: an earlier draft used them to let the over-budget latch press on
whenever the system was not under pressure, and that permitted the very
overshoot this work exists to end (Decision A's tests at
test_pipeline.c:13481 and :13583 went red under it, correctly). They are the
backstop for the next step, admission control, which keeps a run under budget
before an allocation rather than measuring it after.

mem suite 10 new tests green; pipeline + mem 339 passed 0 failed; lint-ci
clean including the new gate.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…egment

The memory-core linter scanned src/ and internal/cbm/. Widen it to src/ and every subtree under internal/, so a new tree added there is covered without anyone remembering to list it, and the count is for the whole project -- cli, mcp, daemon, store, pipeline, the extraction engine. Vendored code is now excluded wherever a vendored/ segment appears in the path rather than by one hardcoded prefix, so that exemption cannot drift either.

internal/ holds only cbm today, so the regenerated baseline is byte-identical: 92 files, 4086 raw sites. Widening lost no coverage and vendored stays out (0 sites listed). The red/green proof still holds under the new scope, and the failure message now names the NEW site -- one appended malloc reports grew by 2 (83 -> 85) at the appended line, instead of listing six pre-existing ones.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…is not the requirement

v0.10.9 introduces Decision A: an over-budget index attempt fails whole with a
named error instead of thrashing. The error told the caller to "Raise
CBM_MEM_BUDGET_MB" but not to what, so the obvious next move is to read
peak_rss_mb and retry just above it.

That move fails, every time, by construction. The abort fires WHEN resident
memory crosses the budget, so peak_rss_mb is pinned just above the budget --
it is where indexing was stopped, not what the repository needs.

Measured on the linux kernel (94521 files, 43.1M LOC) while benching v0.10.9:

  default budget 24576 MB -> aborted, peak_rss_mb 25622
  completing the same index actually took 31.75 GB

So the reported peak understates the real requirement by 24%, and 1.32x the
budget was needed. A caller retrying at peak+10% would have burned another
30 seconds to fail again with a nearly identical message.

The response now carries suggested_budget_mb (1.5x the current budget, which
would have cleared the kernel case) and the hint says plainly that peak_rss_mb
is a floor, not a target. Computed as (3*b+1)/2 rather than b + b/2: integer
division makes the latter degenerate to b at b == 1, so the suggestion would
have repeated the budget that just failed -- the extended test caught exactly
that.

Not changed: the abort itself, the backpressure ladder, and worker cleanup. A
clean test confirmed workers do NOT survive the abort (process count 2 before,
2 after, both idle at 0% CPU), so there was nothing to fix there.

tests/test_mcp.c now pins both halves -- a concrete value strictly greater than
the budget, and the hint stating the peak is where indexing STOPPED. Verified
against the real pre-fix output captured during the bench, which contains
neither.

mcp suite 317 passed, 0 failed, 4 skipped; lint-ci clean.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Second increment on the memory core. The first landed the core, the linter
and phase attribution with zero production call sites; this one migrates the
memory that matters, proves where it goes, removes the waste the proof
exposed, and makes the memory budget a promise: over budget, extraction
results go to disk and the run completes at the floor instead of aborting.
Every number below is from a real index on this machine (M5 Pro, 48 GB) with
CBM_MEM_PHASES=1 and the release build.

WHAT THE PROOF SAID (Go corpus: 21,875 files, 345 MB of source)

The graph buffer, blamed for the peak since the 2026-07 research, is 3% of
the memory: 0.55 GB peak. The per-file extraction results are the memory:
14.8 GB written into their arenas, of which 3.4 GB is reachable. The rest
is temporaries -- cbm_node_text copies at 496 call sites, per-node QN
strings, abandoned generations of every growable array -- that an arena
cannot free and that the result kept alive until resolve ended, because
the result owned the arena. The arenas were charged 28.8 GB of capacity
for those 14.8 GB. On the kernel the same shape was 8.9 GB used by 89,731
results, and resolve grew it to 13.3 GB used / 17.2 GB capacity because
the C walk allocated its working set in the result arena and stored arena
pointers into the shared registry. After that, 7 GB of the kernel's
post-resolve memory had no class at all.

WHAT CHANGED

Attribution. The graph buffer (87 raw sites -> 0), arena blocks, the
tree-sitter slab allocator, the bound tree-sitter/SQLite allocators, the
semantic path (pass_semantic_edges, pass_similarity, minhash: 80 sites), the
hash table (Verstable, a class per table via CTX_TY: cbm_ht_create_in) and
CBM_DYN_ARRAY allocate through the core. Classes arena, ts_tree, hash_table
and dyn_array join the table. Two cross-boundary frees of buffer-owned
strings (pass_importance, pass_complexity) go through a new setter,
cbm_gbuf_node_set_properties_json. The linter baseline drops 4086 -> 3916.

Result compaction (internal/cbm/result_compact.c). At the end of a file's
extraction, everything reachable from its result is measured, copied into
one exact-size arena (strings interned by content within the file) and the
working arena is handed back. Later appends restart growth at the default
block (cbm_arena_init_exact, arena grow_size). The stale duplicate
internal/cbm/arena.h, which shared the foundation header's include guard
and won or lost by include order, is a shim now.

Record slimming. CBMCall 320 -> 72 bytes: the eight inline argument slots
(256 bytes, mostly empty) are allocated on first capture. CBMUsage 48 -> 40.

The per-file overlay contract (src/pipeline/pass_lsp_cross.c). One
lifecycle for every language's cross-file resolve: a scratch arena, a
per-file overlay registry chained to the sealed shared base, the walk
against the overlay, resolved calls appended into the result, scratch
destroyed. type_registry gains chain-aware iterators (every yielded index
belongs to it->reg) and copy-on-write refinement
(cbm_registry_func/type_for_update), and the C, C#, Go and Python walks
use them. This is what an ASan heap-use-after-free in the
lsp_resolution_probe suite demanded: the C walk stored per-call arena
objects into the shared registry, and a kernel run with a naive scratch
arena lost 1,045 edges to the resulting garbage reads. It also ends
unsynchronized writes into the shared registry from parallel resolve
workers.

The 7 GB with no class, found and cleaned. gbuf_index tables and keys were
2.5 GB live and 7.1 GB at peak: the 18 worker buffers built the five
secondary indexes they never query (cbm_gbuf_new_worker builds none), and
18M per-key index arrays started at 8 slots (cbm_da_push_min starts them
at 2); edge property JSON is 81% "{}" (interned). The semantic pass
allocated a 512-token stride per function up front (packed tokens with
offsets: cbm_sem_corpus_add_docs_batch). Edge dedup keys are 128-bit
hashes instead of sprintf'd strings (edge_key_map_t). Each worker keeps
one reusable working arena between files (cbm_work_arena_take/give:
rewind, not free) and every phase mark returns freed pages to the OS.

Budget metric. The over-budget check compares the charged footprint --
max(phys_footprint, mimalloc commit) on macOS -- not resident_size: after
extraction the kernel worker sat at 17.4 GB RSS with 5.5 GB charged, because
pages mimalloc has purged stay resident until the OS wants them, and
phys_footprint under-reports after MADV_FREE_REUSABLE cycles.

Spill / admission control (internal/cbm/result_spill.c, the extract gate
in pass_parallel.c). At 15/16 of the budget (or over it, or with
CBM_MEM_SPILL=1) the gate latches spill mode -- early, because the gate
sees the crossing per file pull and the workers' in-flight files carry the
charge past the line before the first sweep lands: every compacted result
is parked on disk as header + block in one append-only file per writer
under <cache>/spill/, the results already cached are swept out by every
worker, and registry build, def collection, surface rows, resolve and the
infra-route passes load a result only for the moment they read it
(relocated by base delta). The abort path stays closed while a sweep runs
or a cached result remains; it fires only at the floor -- graph +
registries + in-flight files. A result that owns sub-results stays in
memory; a retained parse tree is dropped with the parked result and
re-parsed on load. Every consumer of the result cache reads a slot through
cbm_pipeline_result_acquire()/release() (pipeline_internal.h): a parked
slot is NULL in the array, and the passes that indexed it directly lost
every __route__infra__ node and its INFRA_MAPS edges before the contract.
The other things the transient loads found: a def label that was the
result's own pointer (SIGBUS in the Go cross registry builder), two JVM
helpers that passed their input through, a dedup table whose keys borrowed
sr->value (the insert spun on freed memory), and a latch that was visible
before the store was open (a peer found nothing to park and failed the run
while its neighbour wrote 9 GB). The arena rewind reuse branch had
defeated the nblocks = CBM_ARENA_MAX_BLOCKS OOM seam the suites use; it
reuses only blocks that exist. The incremental and probe routes still
index the cache array themselves and therefore never spill
(ctx->spill_allowed stays false there; follow-up).

The semantic pass under a budget (src/pipeline/pass_semantic_edges.c). The
pass held every function's tokens, the corpus's per-document token ids and
the vectors being stored at once: 4.9 GB on the kernel, on top of the
floor, and nothing sized it to the headroom. Now, when the charged
footprint plus that transient would cross the budget, phases 2-4 run per
batch of functions: tokenize -> count into the corpus -> free; finalize
once; tokenize again -> vectorize -> store -> free, with pages returned to
the OS after every batch. Tokenization is deterministic, so both passes
see the same tokens and the graph does not change; the run pays with a
second tokenize. CBM_SEM_BATCH=<n> forces the batch size (the
batched-equals-unbatched test). The corpus frees its per-document token
ids at the end of finalize (the co-occurrence pass is their only reader),
and mem.semantic.step lines (CBM_MEM_PHASES=1) put the class's live/peak
bytes and the charged footprint at every sub-phase.

The semantic corpus and tokenizer allocate through the core (54 raw sites
-> 0). This is not attribution only: cbm_sem_tokenize handed out libc
strdup blocks that pass_semantic_edges freed with cbm_free -- mi_free in
the production build -- since the pass moved onto the core. No libc-backed
test build can see a cross-allocator free, so the core now refuses a block
it does not own (mi_is_in_heap_region) when CBM_MEM_PHASES=1 is set,
fatally: every proof run in this PR ran with it armed, and the Go and
kernel indexes report zero foreign blocks.

Accounting without contention. The core charged every allocation and free
to shared per-class atomics; with 18 workers on 18 cores every block bounced
the same three cache lines, and the bench against the shipped v0.10.8 showed
it as CPU time rising faster than wall time for the same graph: Kotlin 38 ->
121 s CPU (+58% wall), TypeScript 210 -> 596 s (+67%), Go 177 -> 312 s. The
deltas are thread-local now and reach the shared counters once per 256 KB or
512 blocks of change per thread, when a parallel-for worker ends, and
whenever a reader looks (its own thread first). Phase marks read after the
join and are exact; a class peak is a diagnostic and can be low by threads x
256 KB. Result: Kotlin 10.9 s / 43 s CPU (the residual +22% wall against
v0.10.8 tracks its +34% nodes), TypeScript 33.9 s (v0.10.8: 33.4), Go 48.4 s
(v0.10.8: 52.6), Java 81.1 s (v0.10.8: 79.8); graphs unchanged.

SQLite on a heap of its own. The same bench put the kernel's write step at
132 s where v0.10.8 needed 9.7 s, all of it in one coverage publish
statement. A stack sample named the cost: mimalloc's free-page search and
its periodic heap collect, walking the graph's pages -- millions of them
once the graph lives on the core -- for every statement-journal chunk
SQLite allocates. SQLite now allocates from a mimalloc heap of its own per
thread (mi_heap_new; mi_free is heap-agnostic, so nothing else changes),
and the periodic collect is set to the maximum mimalloc allows since the
explicit release at phase marks and spill sweeps is where memory goes
back. Coverage publish 132 -> 9.4 s; profiled kernel 376 -> 248 s
(v0.10.8: 296 s).

Formatting without a process lock. On macOS every vsnprintf takes the
process locale under an unfair lock (localeconv_l inside __vfprintf);
eighteen workers formatting type names on the C# corpus collapsed into
it -- the 10 MB JIT test files took 68 s each instead of under a second
-- and cbm_arena_sprintf paid it twice per string (size pass, write
pass). Each thread now formats with a C locale object of its own
(vsnprintf_l), and the common short string is formatted once into a
stack buffer.

Per-file budgets that cover the whole file. The 5 s per-file budget
covered the parse only; the LSP walks after it had none. C# JIT stress
files (a 23 MB single expression among them) parsed inside the budget --
the old wall-clock parse timeout used to drop them, which hid the rest --
and then held a worker for 346 s each in the usage walk (tree-sitter's
ts_node_parent descends from the root: quadratic on a deep tree), and the
cross-file resolve ran on the same trees. Three rules now, one site for
every language (cbm_extract_file_ex, honoured by cbm_pxc_dispatch_file): a
parse that used more than half the budget disqualifies the file from the
LSP walks (lsp_skipped); the unified walk checks its thread CPU time every
1024 nodes against six budgets and stops there, keeping what it found
(walk_truncated, implies lsp_skipped); a file whose parse plus walk spent
the budget is skipped by the LSP walks as well. Each decision is logged
with the path (extract.lsp.skipped, extract.walk.truncated). C# extraction
355 -> 39 s; the whole index 463 -> 128 s wall and 27 -> 11 GB peak
against v0.10.8 with 299 more nodes. The walk cuts exactly the five JIT
stress files (hugeexpr1, hugeSimpleExpr1, HugeArray1, HugeField1/2); the
15 MB System.Runtime.Intrinsics reference file walks to the end and, with
four generic-nesting JIT regression tests, only skips the LSP walk under
the parse-plus-walk rule.

The crash behind the C# bench. dotnet/runtime is 42,555 files, and its
resolve phase died with SIGBUS on every run of this branch. A libc-backed
ASan build of the server named it: heap-use-after-free in c_adl_resolve,
reading a type name that another worker had allocated in its per-file
scratch arena and freed at the end of its file. The C++ class walk
refines a method's return type when the declaration is more specific
than the pre-registered one (NAMED -> pointer/reference/template), and
it did that by casting the chained lookup result to non-const and
writing a scratch-arena signature into it -- into the sealed shared base
when the entry lived there. The overlay contract had exactly one
bypass, and it was this cast; every lookup returns const, so a grep for
the cast is the audit. The refinement now goes through
cbm_registry_func_for_update: copy-on-write into the overlay, the base
untouched. Test: clsp_method_return_refinement_is_copy_on_write (a
sealed base with a NAMED method return, an in-class pointer declaration;
the base signature pointer is unchanged and the overlay copy carries the
refinement plus a marker only a copy has) -- RED on the cast, GREEN on
the accessor. The fixed ASan build then indexed the whole
corpus clean: 42,555 files, 1,224,981 nodes, 5,794,320 edges, worker
exit 0, no report.

Production build backs the core with mimalloc explicitly (never the global
override, which macOS's two-level namespace forbids); the test build keeps
libc so the sanitizers see every block -- the same split the tree-sitter and
SQLite bindings already use.

PROOF

  Go     peak RSS 16.7 -> 4.0-5.2 GB; result arenas 14,804/28,817 -> 856/856
         MB; wall 55 -> 52-60 s; graph inside the pre-existing run-to-run
         jitter (gRPC Route nodes: two identical runs differ by 21).
  Go     CBM_MEM_SPILL=1: peak RSS 1.7-2.3 GB, 21,875 results parked (950 MB
         on disk), graph identical: 39 infra routes, 158 INFRA_MAPS, 21,875
         surface rows, nodes/edges inside the Route jitter.
  Go     CBM_MEM_BUDGET_MB=1200: spill latched at 1,204 MB, the first sweep
         took the charge to 768 MB, the run completed at 1.99 GB peak RSS.
  Kernel DEFAULT budget (24,576 MB): aborted before, completes now without
         spilling. Extraction 35.2 -> 16.5 GB RSS (5.6 GB footprint, 15.3 GB
         committed); results 8.9 GB = capacity; resolve tracked 24.6 -> 19.1
         GB; peak RSS 27.3 -> 20.1 GB. Nodes identical (8,529,729); edges
         inside the +-400 run-to-run band the kernel shows between identical
         runs.
  Kernel CBM_MEM_BUDGET_MB=16000: aborted before, completes now through the
         spill path. Latched at 17.2 GB charged; 89,731 results parked (9.3
         GB on disk, 370,188 loads); extraction end 6.2 GB RSS / 5.1 GB
         committed; resolve 8.6 GB RSS / 11.5 GB committed; peak RSS 16.6
         GB (the semantic pass plateau, 13.3 GB RSS, is graph-derived and
         outside what spill can move). Nodes identical; edges 15,773,598.
  Kernel CBM_MEM_BUDGET_MB=15000, before the semantic batching: completes,
         every phase mark under 15,000 MB by every metric, but the process
         RSS high-water mark reached 16.9 GB inside the semantic pass (class
         peak 4.9 GB against 0.8 GB live).
  Kernel CBM_MEM_BUDGET_MB=15000, with it: completes; the charged footprint
         stays under 15,000 MB through the whole semantic pass (14,018 ->
         14,381 MB across its sub-phases, 6 batches of 125,644 functions);
         every phase mark is under budget by every metric (max footprint
         12.6 GB, max commit 11.5 GB); the process RSS high-water mark is
         15.65 GB and is set in extraction, at the moment the gate latches
         (15,002 MB charged) and before the first sweep lands -- a 4%
         overshoot of in-flight work, no longer the semantic pass. Nodes
         identical, edges 15,772,913, zero foreign blocks.
  Kernel CBM_MEM_BUDGET_MB=15000, with the anticipatory latch: spill mode
         enters at 14,062 MB (near_budget); the charged footprint never
         exceeds 11.8 GB at any mark; the process RSS high-water mark is
         15.1 GB in extraction and 15.3 GB during the final write-out --
         1-2% over, the in-flight window and the dump transient. Nodes
         identical, edges 15,772,543, zero foreign blocks.
  Go     CBM_MEM_BUDGET_MB=1200 with it: latched at 1,131 MB, extraction
         peak 998 MB RSS, graph identical to the baseline.

BENCH AGAINST THE SHIPPED v0.10.8 (same machine, same driver, 2026-09-14)

  corpus      wall s          peak RSS GB     nodes / edges / CALLS
  perl        5.7 -> 5.3      0.09 -> 0.05    -3 / -130 / -3
  php         6.3 -> 5.8      0.66 -> 0.20    +21.9% / +1.4% / +23.8%
  rust        6.8 -> 6.3      1.15 -> 0.67    +1 / +56 / -369
  c           5.9 -> 5.7      0.16 -> 0.19    identical
  kotlin      8.9 -> 9.4      1.55 -> 0.96    +33.5% / -22.8% / +12.1%
  django      10.9 -> 10.2    3.13 -> 1.12    +6 / -14.5% / -28.4%
  go          52.6 -> 42.9    17.2 -> 3.9     +14.2% / -10.3% / +2.2%
  java        79.8 -> 73.8    27.6 -> 11.0    -121 / -1.9% / -0.6%
  csharp      463.3 -> 128.1  27.0 -> 11.2    +299 / -0.9% / -0.6%
  typescript  33.4 -> 33.6    12.0 -> 2.5     -186 / -3.5% / +130
  kernel      294.8 -> 233.3  27.5 -> 25.1    -171 / -1.3% / -0.1%

No corpus is slower beyond noise; CPU time follows wall (kernel 2,442 ->
1,403 s). Every graph delta against v0.10.8 is main between the release and
the branch base (339b3f4), not this PR: a build of the base commit run
through the same driver reproduces the final graphs exactly on django and
kotlin, within the pre-existing gRPC Route jitter on go (30 nodes, 27
edges, CALLS identical), and on C# with the same node count. This PR's
own C# difference against the base is 5 CALLS (the five truncated JIT
stress files) plus one ijwhost swap and 563 edges of 5.79 M; the base
takes 448 s / 29.7 GB for that corpus.

Diagnostics that made this measurable stay in: the extract.arenas and
extract.census lines (behind CBM_MEM_PHASES=1), the CBM_MEM_RELEASE=1
release-to-OS probe, and peak_charged_mb on every mem.phase line -- the
high-water mark of the budget metric itself (cbm_mem_peak_charged), next
to the RSS peak that counts pages already purged but not yet reclaimed.

Tests: extract_compact_* and extract_spill_round_trip_keeps_every_field
(extraction), parallel_spill_mode_builds_the_same_graph (parallel: every
file parked, node count and every edge type equal to the in-memory run),
pipeline_semantic_batched_matches_unbatched (pipeline: one repo indexed
unbatched and with CBM_SEM_BATCH=5 -- byte-identical node vectors, token
vectors and SEMANTICALLY_RELATED edges),
extract_lsp_skipped_when_parse_used_ its_budget_share and
extract_walk_truncated_at_its_cpu_budget (extraction: the seams mark a
file / stop the walk, the LSP walk and the dispatcher skip it, the defs
stay), registry_overlay_chain_iterates_ and_copies_on_write (c_lsp), arena
exact-init/growth, mem charged/footprint/ pressure.

Not in this increment: run-to-run edge jitter (+-400 on the kernel, Route
class on Go) predates this work and has its own item; the incremental route
spilling; a tighter result encoding (def ids instead of QN strings in
resolved calls) is the next lever on the 9 GB the kernel's results still
take.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
@DeusData DeusData changed the title feat(mem): a central allocation core, a linter that enforces it, and phase attribution in the index pipeline feat(mem): a central allocation core, the memory migrated onto it, and results spilled to disk under budget Sep 14, 2026
The 15 GB kernel run completed with every phase mark under budget, but
the budget metric itself -- the charged footprint the extract gate reads
on every file pull -- reached 16.8 GB. Two defects, found with a probe at
the gate (one line per 256 MB of charge after the spill latch, with the
class table; mem.charge.probe, CBM_MEM_PHASES=1, stays in).

Worker graph buffers kept a dense id array over the global id space. The
charge climbed gradually from the 14.06 GB latch to 14.96 GB, then jumped
about 1 GB between two consecutive gate reads, and the only class that
moved in that step was gbuf_index. Worker buffers draw ids from the shared
counter, so each of the 18 kept an id -> node array spanning the whole
global id space -- 8.5M ids x 8 B, next power of two -- almost entirely
empty, and when the global id crossed 2^23 all eighteen doubled in the
same instant. Nothing asks a worker buffer by id before the merge (every
cbm_gbuf_find_by_id caller runs on the main buffer); the worker
constructor now drops the array the way it already drops the five
secondary indexes, and the main buffer answers by id after the merge.

Spill mode did not park what was already cached. The sweeps ran only on an
over-budget observation; a run that latched early (near_budget) and then
stayed under budget through extraction -- which is exactly what the first
fix produced: 14,949 MB at the extract mark -- carried the 44,797 results
cached before the latch (8 GB) into registry build and resolve, which
cannot park, and resolve ran at a charged 21.8 GB against the 15 GB
budget with nothing to stop it. The earlier runs only fitted because the
id-array burst tripped the sweeps by accident. Extraction now ends with a
final sweep whenever spill mode is on: results belong on disk before the
phases that cannot park inherit them.

Kernel, CBM_MEM_BUDGET_MB=15000, before -> after: charged high-water
16,771-16,818 MB -> 14,582 and 14,732 MB in two runs (97-98% of the
budget); the final sweep parks ~45k results and takes the charge from
14.6 to 4.3 GB before resolve, which peaks at 11.5 GB committed; wall
234 -> 224-230 s; nodes identical (8,529,729), edges inside the
run-to-run band. RSS high-water 18.7 GB is
pages already purged and not yet reclaimed: the footprint never exceeds
7.3 GB at a mark.

peak_charged_mb (7b77fd4) is what made this measurable: the kernel at
the default budget reads charged 22.3 GB against 24.6 GB while ps reports
29 GB RSS, and the 15 GB run read 16.8 GB where every mark was under.

Tests: gbuf_worker_buffer_keeps_no_by_id_array (graph_buffer: one node at
id 2M in a worker buffer must not grow the index class by even 1 MB where
the dense array costs 16 MB; the worker answers by QN, the main buffer by
id after the merge) -- RED on the dense array, GREEN without it.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…ed stat

The daemon leaked 180 KB per query. The Linux soak's quick leg went from
11 to 144 MB of RSS in ten minutes (13.1x, 628 MB/hr), mimalloc's
committed bytes tracking RSS almost byte for byte and the growth linear
in the query count, file descriptors flat -- where the same leg had been
flat at 15 MB in July.

The cause is the previous commit's SQLite heap: one mimalloc heap per
thread, created on first use. The daemon runs a thread per connection,
so every query ran on a fresh thread with a fresh heap, and every block
the shared connection keeps beyond the request -- page cache pages, the
statement cache, schema objects -- was allocated in that heap's pages.
When the thread exited, its heap was deleted and those pages were left
behind holding a few live blocks each: committed for as long as the
connection lives, a few pages per query, never reused by anyone.

The dedicated heap exists for one reason: the index worker's default
heap holds the graph (40M+ blocks), and SQLite churn on that heap paid a
page walk per allocation during the coverage publish (132 s vs 9.7 s on
the kernel). That process has long-lived threads and exits when the
index is done. So the dedicated heap is now a switch, off by default,
that the index worker turns on at its entry (main.c, --index-worker);
everywhere else SQLite allocates from the calling thread's default heap
exactly as before the previous commit. The switch exists in every build
and changes nothing where the allocator binds are compiled out.

Linux soak, quick leg, before -> after: RSS 11 -> 144 MB in ten minutes
(13.1x, 628 MB/hr) -> 10.6 -> 11.5 MB and flat from the first minute
over 564 queries with reindexes every two minutes; query-leak leg:
11.6 MB flat over 1,376 queries; committed heap 6.8 MB flat.

The same soak showed a second thing. From its second sample on, the
query-leak leg reported a committed heap of 2^64 - 121 MB: mimalloc's
committed statistic is a signed counter merged per thread at thread
exit, and a process whose long-lived thread commits what its
short-lived threads free reads it negative -- cast to size_t on the
way out. That figure is one of the two the budget takes the larger of.
cbm_mem_allocator_committed now reports 0 for a reading above
SIZE_MAX/2, so the charge falls back to the OS number instead of a
16 EB budget breach, and the diagnostics report the committed pair
through it (the peak never undercuts the current value and never
carries a wrapped one), the way they already did for RSS.

Detector: the soak's quick and query-leak legs (scripts/soak-legs.sh),
the release-gating sequence in every venue -- RED on the previous tree
(13.1x), GREEN on this one. A unit test cannot see this class: the test
build keeps SQLite on libc so the sanitizers see every block.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
@DeusData

Copy link
Copy Markdown
Owner Author

Follow-up on the 15 GB kernel floor check in the description: it now holds by the budget metric itself.

The earlier revision read a charged high-water of 16.8 GB at a 15,000 MB budget with every phase mark under. A probe at the extract gate (one line per 256 MB of charge after the spill latch, with the class table) found two causes, neither of them in-flight work:

  • The 18 worker graph buffers each kept a dense id-to-node array spanning the whole global id space (ids come from the shared counter): 8.5M ids x 8 B, next power of two, almost entirely empty, and all eighteen doubled in the same instant when the global id crossed 2^23 -- a 1 GB step between two gate reads. Nothing asks a worker buffer by id before the merge, so the worker constructor drops the array the way it already drops the five secondary indexes. Test: gbuf_worker_buffer_keeps_no_by_id_array, RED on the dense array, GREEN without it.
  • Spill mode only swept on an over-budget observation. With the burst gone, extraction stayed under budget and the 44,797 results cached before the latch (8 GB) reached resolve, which cannot park, at a charged 21.8 GB with nothing to stop it. Extraction now ends with a final sweep whenever spill mode is on.

Kernel at 15,000 MB, before -> after: charged high-water 16,771-16,818 MB -> 14,582 and 14,732 MB in two runs; the final sweep takes the charge from 14.6 to 4.3 GB before resolve (peak 11.5 GB committed); wall 234 -> 224-230 s; nodes identical (8,529,729). peak_charged_mb on every mem.phase line is the number these claims rest on. The gate probe stays in behind CBM_MEM_PHASES=1.

@DeusData

Copy link
Copy Markdown
Owner Author

Soak results for this tree, all three local platforms, quick + query-leak legs at 10 minutes each (the canonical scripts/soak-legs.sh sequence).

The first Linux run caught a regression this branch had introduced: the daemon leaked about 180 KB per query. The quick leg went from 11 to 144 MB of RSS in ten minutes (13.1x, 628 MB/hr), mimalloc's committed bytes tracking RSS almost byte for byte, growth linear in the query count, file descriptors flat, where the same leg had been flat at 15 MB in July. Cause: the SQLite heap-per-thread introduced for the kernel write-step stall. The daemon runs a thread per connection, so every query ran on a fresh thread with a fresh heap, and every block the shared connection keeps beyond the request (page cache, statement cache) pinned that heap's pages after the thread exited. The dedicated heap is now a switch, off by default, that only the index worker turns on (cbm_sqlite_dedicated_heap, main.c --index-worker); everywhere else SQLite allocates from the calling thread's default heap exactly as before.

The same soak also showed the daemon reporting a committed heap of 2^64 - 121 MB: mimalloc's committed statistic is a signed counter merged per thread at exit, and a process whose long-lived thread commits what its short-lived threads free reads it negative. cbm_mem_allocator_committed now reports 0 for a wrapped reading (the budget falls back to the OS number), and the diagnostics report the committed pair through it, the way they already did for RSS.

platform leg RSS first → last (max) verdict
Linux (Colima arm64) quick 10.7 → 11.6 MB (14) PASSED, flat from the first minute over 484 queries with reindexes every 2 min
Linux (Colima arm64) query-leak 10.7 → 11.6 MB (14) PASSED, flat over 1,196 queries
macOS (host arm64) quick 13.1 → 14.8 MB (15) PASSED, plateau after warm-up over 504 queries with reindexes every 2 min
macOS (host arm64) query-leak 13.1 → 17.4 MB (17) PASSED, warm-up to 17.4 MB by query 240, then flat through 1,246 queries (slope negative)
Windows (UTM ARM64 VM) quick 16 → 16 MB (17) PASSED, ratio 1.0x over 45 samples with reindexes every 2 min
Windows (UTM ARM64 VM) query-leak 15.8 → 16.8 MB (18) PASSED, ratio 1.1x; baseline 16.5–16.8 MB with transient 4.5 MB spikes that fall back; committed heap 31.9–32.2 MB flat; 771 queries

The Windows residual recorded in July (about 4.5 KB per query, monotonic, no plateau within ten minutes) is gone on this tree: a 30-minute query-leak leg on the same VM sits at 16.60 MB at query 121, then at an unchanged 16.77 MB of RSS and 31.96 MB of committed heap from query 1,021 through query 2,221, ending at 16.84 MB after 2,346 queries (slope −229 KB/hr, PASSED). The periodic 4.5 MB transients in the ten-minute leg are the daemon's own maintenance cycles and return to baseline every time.

Gates on this tree: Linux container test leg 7889 passed / 0 failed / 9 skipped; macOS full leg 8050 / 0 / 10; lint green; Windows ladder leg (real ARM64 VM, clean build + contracts + all suites + guards) 7903 passed / 0 failed / 74 skipped (141 suites).

…atch

Two CI-only lanes went red on 9724d90, both this branch's own doing.

The macOS leak lane: every test passed and LeakSanitizer reported 16 to
101 objects of 1,472 bytes per test process, allocated from
cbm_arena_sprintf. The formatter creates one C locale per thread on macOS
(the vsnprintf locale lock fix) and kept it in a thread-local pointer
only, so every worker thread that ever formatted a name leaked its locale
at exit. A pthread key destructor now frees it when the thread exits; the
main thread keeps its locale until process exit.

The TSan lane on Ubuntu arm: pipeline_backpressure_futile_nap_disengages
counted 8 nap cycles against a bound of 7, the "gate re-paid per pull"
shape the test exists for. The spill shortcut reports "not over" while a
sweep is still bringing memory down, and that reading re-armed the
futility latch, so the next over-budget pull paid a full nap cycle again.
The latch now re-arms only on a genuine under-budget reading; a reading
the shortcut produced is not one.

Verification: the local macOS leak lane (make -f Makefile.cbm test-lsan)
reports zero leaks, 8050 passed / 10 skipped (the CI lane had 16-101
leaked objects per process); the local TSan lane (scripts/test.sh --tsan)
passes 1157 / 9 skipped with the nap-cycle test green (the CI lane had it
at 8 cycles); pipeline, parallel and arena suites twice and lint-ci on the
fixed tree.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…izer

The MSan lane on 9724d90 stopped in the extraction suite's spill
round-trip test: uninitialized bytes in fwrite at offset 4087 of a
6,952-byte region, the parked result's arena block (the stack overflow
that followed is MemorySanitizer's own report path failing on the nested
bug). The park writes a result as an opaque image: the record header, a
struct copy, and the compacted arena block. Both carry padding bytes no
code ever wrote, inside structs and between objects, and MSan tracks that
mark through the compaction's memcpy, so zeroing the destination block
only moved the report (offset 6714 in the local MSan container). The
image is read back whole and never interpreted byte by byte; the padding
content is irrelevant on disk.

Under MSan the header and the block are declared defined right before
the write (__msan_unpoison, compiled in only when the build is
instrumented); every other build compiles the annotation to nothing. The
exact block is also zeroed once when it is created, so the gaps between
objects are deterministic in the spill file.

Verification: extraction, arena and parallel suites under ASan on the
fixed tree; the extraction suite in the local MSan container image
passes 348 / 0 with no sanitizer report (it reproduced the CI report exactly before the fix).

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
cppcheck cannot evaluate `#if __has_feature(memory_sanitizer)` even
behind `#if defined(__has_feature)` ("undefined function-like macro
invocation", the lint lane on d89ccab), and foundation/sanitized.h
already exists for exactly that: it defines __has_feature away where the
preprocessor lacks it. The spill store's image annotation now includes
that header and tests the feature directly; the annotation itself is
unchanged.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
@DeusData
DeusData merged commit 2058d49 into main Sep 14, 2026
35 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant