Skip to content

[JSC] Less memory per function that never runs, and code that can be dropped and decoded again - #628

Open
Jarred-Sumner wants to merge 18 commits into
mainfrom
claude/jsc-cold-code-memory
Open

[JSC] Less memory per function that never runs, and code that can be dropped and decoded again#628
Jarred-Sumner wants to merge 18 commits into
mainfrom
claude/jsc-cold-code-memory

Conversation

@Jarred-Sumner

@Jarred-Sumner Jarred-Sumner commented Sep 11, 2026

Copy link
Copy Markdown
Collaborator

15 commits on cf1b36ec8703. Each commit is one feature, builds on its own, and carries its
tests; they are ordered by dependency and can be reviewed (and reverted) one at a time.

Why

A large bundled CLI application (ESM, ~1,800 modules in ~900 chunks, standalone executable with an embedded bytecode cache)
sits at an idle prompt with ~100 MB of anonymous memory of which JSC code structures are the largest part, and at ~170 MB
after a session. Almost all of it belongs to code that never runs or ran once:

  • 30,000 top-level function declarations are instantiated at module link time; 83% are never read.
  • 12,700 LLInt-only CodeBlocks carry 15.3 MB of metadata for 4.2 MB of bytecode: 37% call link records of call sites that
    ran at most once, 23% value-profile predictions nobody reads.
  • Every UnlinkedCodeBlock carries value/array profile arrays that only the DFG ever benefits from.
  • Unlinked code decoded from the cache is never dropped, although decoding it again costs ~2.5k instructions per function
    against ~111k for a re-parse; module bodies keep their linked and unlinked code although they run once.

Nothing here changes behaviour observable from JavaScript. Every feature has an option (all default on unless stated) so it can
be switched off individually.

Commits

1. A ModuleProgramExecutable keeps one module environment symbol table for its whole life

A fix for the base, found while testing this series against module loaders (#522), and the invariant the rest relies on.

  • Bug: records of several loaders share an executable, its declarations' FunctionExecutables and their optimized code; that is
    sound only because every environment is made from the executable's ONE symbol table, so that the second environment
    invalidates SymbolTable::singleton() (the DFG folds the scope of resolve_scope closure variables through the table in the
    metadata, and of imports through topLevelExecutable->moduleEnvironmentSymbolTable()). ScriptExecutable::clearCode()
    cleared the table and getUnlinkedCodeBlock() made a new one (and a new declarations vector) when it generated the code
    again. A record that had made its environment but not run yet when all code was deleted (waiting on a dependency suspended
    at a top-level await) regenerates the code; the next loader adopts the executable with the new table; functions compiled
    after that bake the newest loader's exporting environment into code the earlier loaders run too: an imported binding read
    in loader a returns loader d's value (JSTests/stress/module-loaders-share-one-environment-symbol-table.js fails on
    cf1b36ec8703). With lazy declarations (commit 5) the same split would also hand a record executables specialised on
    another record's environment, hence a RELEASE_ASSERT(environment->symbolTable() == executable->moduleEnvironmentSymbolTable())
    on the lazy read path there.
  • Fix: the table and the declarations vector are made from the executable's first unlinked code and stay (an environment keeps
    its table alive anyway). Code fetched again is asked for in the code generation mode of the first (the environment's layout
    depends on it); getOrMakeExecutable only adopts an executable whose mode is the one the record would ask for. An executable
    whose code was deleted is still not adopted, and no longer sits in the clearable-code set for the sake of its table.
  • Audit of everything specialised on "the" module environment in shared code (closure-var and ModuleVar resolve_scope
    folding, tryGetConstantClosureVar, GetLazyClosureVar's fast case, FunctionExecutable::singleton()): all reduce to the
    one-table rule; details in the commit message.

2. Allocate an UnlinkedCodeBlock's value and array profiles when it first reaches the Baseline JIT

Options::useLazyUnlinkedValueAndArrayProfiles

  • What: m_valueProfiles/m_arrayProfiles (two FixedVectors) become one ButterflyArray (ValueAndArrayProfiles) allocated when a
    BaselineJITPlan is first created for a CodeBlock of the unlinked code. The number of value profiles is derived
    (numParameters + metadata.numValueProfiles), the number of array profiles is kept as a 32-bit count in existing padding.
    Builtins never get them. The bytecode cache no longer stores the value profile count (format revision 5 -> 6; this is the only
    bump in the series, commits 3 and 5-7 change the format too and stay on 6).
  • Concurrency: the arrays are allocated on the main thread; the pointer is published with a release store and the collector
    and compiler threads that fold profiles (CodeBlock::updateAll*Predictions) load it once per fold with acquire and skip
    folding when it is null. Freed only in ~UnlinkedCodeBlock.
  • Numbers: synthetic app (2,000 modules x 25 functions): 4.5 MB of the 6.0 MB the first three commits save.

3. Move UnlinkedCodeBlock's out-of-line jump targets into its RareData

An empty hash map per code block for a case few blocks have. With commit 2: sizeof(UnlinkedCodeBlock) 208 -> 192, one size
class down for all unlinked code blocks (static_assert in release, 64-bit, non-Windows builds). No new RareData on the
synthetic app (902/3009 before and after).

4. Index the bytecode cache's two-character atom table by identifier character class

512 KB per VM (zeroed, so all touched) -> 32 KB: 64 x 64 classes ($ 0-9 A-Z _ a-z); other pairs use the verified
direct-mapped cache the three-character names use.
Commits 2-4 together, synthetic: RssAnon 184.4 -> 178.4 MB; startup instructions -0.5..-0.8%, exercise -1.0..-1.6%.

5. Instantiate module function declarations on first read

Options::useLazyModuleFunctionDeclarations (predictFunctionForUnprofiledLazyClosureVarForTesting for tests)

  • What: InitializeEnvironment leaves the module environment slot of a heap-allocated function declaration empty. The function
    object and its FunctionExecutable are created when the binding is first read: get_from_scope with the new LazyClosureVar
    resolve type (LLInt, Baseline, LOL: load, test for empty, slow path; DFG GetLazyClosureVar, FTL lowering with a slow path
    call), or JSModuleEnvironment::readVariable() for by-name lookups, module namespace objects, WebAssembly imports and the
    debugger. ResolvedLazyClosureVar is what BytecodeGenerator emits for a module's own declarations; the linker turns
    ModuleVar and closure variables that are declaration slots into LazyClosureVar. An empty slot that is not a declaration
    is still TDZ. The type/control-flow profilers keep the eager path.
  • Module loaders ([JSC] Additional module loaders per global object, sharing linked module code between them #522): records that share a ModuleProgramExecutable share the declarations' FunctionExecutables
    (linkedFunctionDeclaration / linkFunctionDeclaration next to functionDeclaration); the record that reads a declaration
    first links it, each record fills its own environment with its own function objects. Whether a slot is a declaration slot
    depends only on the module's code, so every CodeBlock of an UnlinkedCodeBlock links a given get_from_scope the same way.
    Optimized code that one loader warmed up meets empty slots when the next loader runs it:
    JSTests/modules/module-loaders-lazy-function-declarations.js drives that through DFG/FTL code without the testing option.
  • Concurrency: slots are only filled by the mutator. A compiler thread that reads an empty slot through
    Graph::tryGetConstantClosureVar gets no constant. GetLazyClosureVar clobberizes as a read of the slot that can fire
    watchpoints (symbolTablePutTouchWatchpointSet on first instantiation) and allocate. The record's list of uninstantiated
    declarations is visited and released under cellLock().
  • Numbers (CLI application, 12 s after the prompt): FunctionExecutable 55.1k -> 30.0k, function objects 156k -> 131k, GC heap
    -6.3 MB, RssAnon -7.2 MB. Synthetic (2,000 modules x 46 declarations): FunctionExecutable 154k -> 66k, heap 72.6 -> 59.1 MB,
    startup instructions -6.5%.

6. LLInt get_from_scope: test for the closure variable resolve types first

Site mix at the prompt of the CLI application: ClosureVar 54.8%, LazyClosureVar 34.4%, GlobalProperty 9.8%, GlobalVar 1.0%.
Three compares fewer per closure variable read, two more per global one; pays for the empty check of commit 5.

7. Leave a module's uninstantiated function declarations in the bytecode cache payload

With commit 5, a module decoded from a payload that stays around does not create the UnlinkedFunctionExecutables of
declarations nobody read: m_functionDecls[i] stays null, ModuleFunctionDeclarationSlots remembers the Decoder and the record
array, UnlinkedCodeBlock::functionDecl() decodes on first request (mutator only; other threads use
functionDeclIfDecoded()). A never-read module function: 256 B of cells -> an 8-byte empty slot + a 4-byte table entry.
--help of the CLI application: 505 M -> 470 M user instructions.

8. jsc shell: ASAN builds keep locals on the real stack

detect_stack_use_after_return=0 as the shell's default ASAN option: the fake stack is not scanned conservatively, so every
multi-module test died under --collectContinuously / --slowPathAllocsBetweenGCs in ASAN builds.

9. Call sites in LLInt / Baseline metadata get their CallLinkInfo and ArrayProfile on their second execution

Options::useLazyLLIntCallLinkInfos

  • What: the metadata of the non-varargs call opcodes holds one pointer to a CallSiteData { DataOnlyCallLinkInfo, ArrayProfile }
    (96 B) allocated on the site's second execution; until then it points at one of two per-VM shared records ("never" /
    "once") that look like a polymorphic call to the new llint_unlinked_call thunk, so the LLInt fast path is the old one plus
    one load and no branch. sizeof(Metadata): op_call / call_ignore_result / tail_call 96 -> 8, construct 80 -> 8, super_construct
    88 -> 16, iterator_open 120 -> 48, iterator_next 136 -> 64.
  • Concurrency: a shared record is flagged and every mutator of CallLinkInfo RELEASE_ASSERTs it is not one. A site's own record
    is fully initialized before a store-store fence publishes it and lives as long as the MetadataTable. Baseline code never
    meets a shared record (setupWithUnlinkedBaselineCode allocates all of them first). Compiler threads and the collector go
    through forEachLLIntOrBaselineCallLinkInfo() / getArrayProfile(), which skip sites without their own record ("no
    information").

10. Keep the predictions of a MetadataTable's value profiles out of line, and only for code that has warmed up

Options::useLazyValueProfilePredictions, thresholdForValueProfilePredictions (40)

  • What: 8-byte buckets in front of the table; the predictions are an array off LinkingData, allocated by the first
    mergePrediction() with something to record. A still-interpreted CodeBlock below the threshold keeps samples in its buckets: GC
    end only clears buckets whose cell died.
  • Concurrency: the array is published with a compare-and-swap (mutator, marker and compiler threads may all get there first);
    readers without an array see SpecNone.

11. op_get_from_scope does not need the global variable's watchpoint set in its metadata

24 -> 16 bytes; the DFG parser finds the set in the symbol table entry under the table's lock. Field names kept.
Commits 9-11, synthetic (31.8k interpreter-only blocks): metadata 28.2 -> 15.8 MB (+0.2 out of line), 887 -> 496 B per block,
RssAnon at startup 127.2 -> 113.8 MB. Per call site: never run -73, first run +48, second run +445 instructions.

12. VMInspector::codeBlockCensus and $vm.codeBlockCensus()

Counts used by the tests of commits 13-14 (and by embedders that want to see where code memory goes).

13. Code decoded from a persistent bytecode cache can be dropped and decoded again

Options::useCodeRecoveryFromBytecodeCache

  • What: a code block decoded from a persistent payload remembers its payload (16-bit index into the per-VM
    PersistentBytecodePayloads) and record offset (32 bits), both in existing padding (sizeof unchanged).
    UnlinkedFunctionExecutable::returnCodeToCache() puts an executable back into its m_isCached state.
    VM::shrinkFootprintNow(flags) / shrinkFootprintWhenIdle: KeepCodeThatNeedsParsing, KeepCodeInUse,
    LeaveCollectionToCaller; Heap::deleteAllUnlinkedCodeBlocks takes which kinds to drop. VM::entryCountFromOutside() and
    Heap::lastActiveCollectionTime() let the embedder decide when the program is at rest.
  • The drop path never reads the payload (the embedder has usually paged it out by then): live child executables are remembered
    weakly under the parent's record offset and adopted by the block decoded from that record later, so closures made before and
    after a drop share code.
  • Identity and lifetime: a slot of PersistentBytecodePayloads is one tree of unlinked code = the payload's bytes as decoded for
    one SourceProvider (embedders wrap the same bytes in a new CachedBytecode + SourceProvider per load). Nothing remembered under one
    slot is handed to code decoded under another; code private to one executable (a module whose loader has its own module
    scope) is not registered, since its Baseline code assumes one resolution of that scope. Every Decoder and every code block of
    a slot hold it (retain/release, ~UnlinkedCodeBlock, ~Decoder); with the last one the payload/provider references go and the
    index is reused: reloading one file 50 times leaves 3 slots, not 52.
  • Contract: shrinkFootprintNow returns false (nothing done) under JS or from inside the collector; the Keep modes wait for a
    running collection, the flagless mode returns false instead; lastException is cleared (its stack kept dropped code alive twice);
    VM::deleteAllCode (flagless) now also deletes what a cache can hand back (heap after shrink + GC 5.4 MB instead of 31.7 MB on a
    synthetic application). A CodeCache entry decoded from a payload is committed and dropped in every mode: a provider that has
    the payload decodes again, one with equal source text but no payload parses.
  • Concurrency: all of it runs on the mutator. Heap::deleteAllUnlinkedCodeBlocks completes the compiler threads' ready plans
    (that allocates) BEFORE its HeapIterationScope and under DeferGC, then re-asserts that no collection runs, so nothing marks
    while returnCodeToCache turns an executable's two code block slots into {Decoder, offsets}; both directions publish with
    m_isCached in the middle (fences), and visitChildren reads flag, slots, flag. OnlyWithoutLinkedCode (restricts the
    recoverable kind) skips blocks anything links against; otherwise all plans are completed first, so no compiler thread holds a
    block that is dropped. Heap::lastActiveCollectionTime() is atomic (zero until the first collection that saw the mutator busy).
    Lean decoder only.
  • Numbers: re-decode 2.5k instructions per block vs 111k re-parse; synthetic: unlinked code blocks 19,252 -> 2,383, GC heap
    33 -> 23 MB.

14. Module and program code that has run is released right away

Options::useRunOnceCodeRelease; useSharedModuleFunctionExpressionExecutables (default off)

  • What: once JSModuleRecord::evaluate sees the body finished, the executable drops its linked code, and its unlinked code and
    CodeCache entry if they can be decoded again; the environment symbol table and the declarations' executables stay.
    Interpreter::executeProgram clears the ProgramExecutable it created.
  • Module loaders ([JSC] Additional module loaders per global object, sharing linked module code between them #522): the executable counts the records that adopted it and have not finished; the code goes with the last
    one (covers a loader suspended at a top-level await while another finishes). A later loader adopts an executable that only
    released recoverable code and has it decoded again (the symbol table and m_functionDeclarations are the executable's for
    life, commit 1; ClearCode::All withdraws a released executable from adoption). With commit 5/7 the module record no longer references
    releasable unlinked code for the sake of unread declarations (it decodes them from the payload).
  • Concurrency: linked code that left the interpreter or has a Baseline compile queued is left to age out
    (GlobalExecutable::canReleaseLinkedCodeNow).
  • Numbers: synthetic: ModuleProgramCodeBlock 2,001 -> 1, UnlinkedModuleProgramCodeBlock 2,564 -> 2, GC heap 39.4 -> 24.3 MB,
    extra memory 11.8 -> 1.5 MB, startup instructions -4.4%.

15. Heap: live size after the last collection and total bytes allocated, for embedders

Heap::sizeAfterLastCollection() and Heap::totalBytesAllocated() (running total kept in updateAllocationLimits() right before
the per-cycle counters are reset; mutator thread only). For an embedder's idle detection: a busy program whose heap does not grow
still allocates.

Not in this PR: dropping cold LLInt CodeBlocks in any collection and block sealing were written and measured; neither moved
resident memory on the application, both were off by default, so they were left out.

Numbers on the CLI application (RssAnon MB, RssFile 46-51 MB throughout; two to three runs each; rows 1-4 were measured with the series on the previous base dfd696443b, row 5 on this branch; runs of one round are comparable, rounds differ by up to ~8 MB)

fresh prompt (10 s) idle prompt (100 s) 105 s after a 20-turn session 240 s after
before 117-125 96-104 168-174 154-169
commits 2-8 108-117 86-96 156-160 150-155
commits 2-11 102-109 88-95 159-162 152-154
all (embedder calls shrinkFootprintWhenIdle at deep idle) 98 86 156 135-138 (131 dropping everything)
all, this branch on cf1b36ec8703 (same-run baseline 124-127 / 105-106 / 173 / 163-169) 106-108 93-94 158-163 135-140

CPU (instruction counts; wall clock on the measuring box is noise):

before after
--help, user instructions (like-for-like builds on the previous base) 505.7 M 471.2 M (-6.8%)
--help, this branch, same binary with every option of the series off / on 551 M 513 M (-6.9%)
20-turn session, whole process 19.01 G 18.83 G (-0.9%, noise ~1%)
interpreter-only micro loops (calls / property access / closure scope) -0.41% / -0.06% / -5.0%
module call loop: LLInt / Baseline only / DFG+FTL (useConcurrentJIT=0) -1.0% / +1.6% / identical to the instruction
global-variable-heavy loop, LLInt +1.9% (dispatch order of commit 6)
wake-up after a deep-idle shrink ~+20 M instructions once

Tests

New: JSTests/modules/lazy-function-declarations*.js (6, run under 32 option sets: tiers, eager thresholds, no-cjit validation,
collectContinuously, bytecode cache fill/hit incl. persistent payload), module-loaders-lazy-function-declarations.js;
JSTests/stress/bytecode-cache-unlinked-value-and-array-profiles.js, llint-lazy-call-link-info{,-async,-relink}.js,
value-profile-predictions-{on-demand,after-cold-collection}.js, get-from-scope-global-variable-watchpoint-lookup.js,
shrink-footprint-recovers-code-from-bytecode-cache.js, module-code-released-after-evaluation.js,
module-function-expression-executables-shared.js, module-lazy-function-declarations-outlive-released-code.js,
module-loaders-share-released-code.js, module-loaders-share-one-environment-symbol-table.js,
shrink-footprint-while-compilations-are-ready.js, shrink-footprint-decodes-code-again-repeatedly.js (also with --destroy-vm),
bytecode-cache-persistent-payloads-are-released.js.

Results on this branch, against a base list taken with jsc built from cf1b36ec8703 itself (release, and Debug+ASAN
where noted); every difference from base was re-run on both binaries:

suite this branch base (cf1b36ec8703)
every commit of the series builds (release jsc) and passes the tests it adds or changes, at that commit 15/15
lazy declaration tests (6) + module loader tests (5, incl. the new one) x 32 option sets, release 352/352
same, Debug+ASAN 352/352
JSTests/modules (122 files), default and no-cjit-validate 114 pass / 8 fail, both 113 / 9 (the 8 + the new loader test)
full JSTests/stress x {default, no-cjit-validate, no-JIT, collectContinuously, low thresholds, eager jettison, no concurrent JIT} 35 / 34 / 80(+1 timeout) / 36 / 36 / 34 / 33 failures of ~5,650 38 / 38 / 84(+1) / 39 / 40 / 40 / 37: the same tests plus the new tests that need this branch (and module-loaders-share-one-environment-symbol-table.js, which fails on the base because of the bug commit 1 fixes); the only others are three tests that flip between runs on both binaries (recursive-try-catch.js, int8-repeat-in-then-out-of-bounds.js, codeblock-aging-ftl-idle.js)
bytecode cache fill + forced hit: bytecode-cache-*.js and the new cache tests, all their option variants 47/47 (ASAN 44/44 over the same files without one ten-minute test) 19/15 (the new tests)
bytecode cache, stress list (1,357 runs) 1326 / 31 1325 / 32 (same 31 + one flaky)
bytecode cache, JSTests/modules, plain and with a persistent payload 113 / 9 both (ASAN 113 / 9) 112 / 10; the ninth is module-loaders-changed-dependency.js, which writes new source files on every run and so cannot pass a forced cache hit on either binary
test262 module-code, import, export, dynamic-import, import.meta, eval-code, function dirs (3,718) 16 failures the same 16
Debug+ASAN, 1,504 stress tests selected by name (call, profile, llint, osr, tier, inline, construct, spread, iterator, closure, scope, module, function, tdz, bytecode-cache ...) x 6 option sets 17 / 17 / 18 / 17 / 18 / 16 failures (default, eager jettison, low thresholds, no concurrent JIT, no JIT, collectContinuously) and 3 collectContinuously timeouts out of 1,521 files: exactly the (option set, test) pairs that failed in the same sweep before the rebase, each of which fails on the base release build too, is the 160 KB / 1.5 MB stack cap of a test meeting Debug+ASAN frames, or is a slow test under collectContinuously + ASAN on a loaded machine
aarch64: all of JavaScriptCore/WTF/bmalloc cross-compiled for darwin-aarch64 (offlineasm ARM64 LLInt, DFG, FTL) compiles; not run

Seen while testing, not caused by this series (reproduces on the base with no shrink): ASSERTION FAILED: addResult.isNewEntry in
CachedBytecode::copyLeafExecutables on the cache-filling run with --useJIT=0 --forceCodeBlockToJettisonDueToOldAge=1 when generated
unlinked code is regenerated after a collection (stale raw-pointer keys in m_leafExecutables).

LowLevelInterpreter32_64.asm is not updated for commits 5 and 9-11 (no 32-bit target is built from this repository).

@github-actions

github-actions Bot commented Sep 11, 2026

Copy link
Copy Markdown

Preview Builds

Commit Release Date
72ea768d autobuild-preview-pr-628-72ea768d 2026-09-13 01:52:29 UTC
affb84c2 autobuild-preview-pr-628-affb84c2 2026-09-12 07:17:09 UTC
b43ee532 autobuild-preview-pr-628-b43ee532 2026-09-11 21:00:46 UTC

@Jarred-Sumner
Jarred-Sumner force-pushed the claude/jsc-cold-code-memory branch from 209144a to affb84c Compare September 12, 2026 06:48
@Jarred-Sumner
Jarred-Sumner marked this pull request as ready for review September 12, 2026 08:17
Jarred-Sumner added a commit to oven-sh/bun that referenced this pull request Sep 12, 2026
Pins the preview release autobuild-preview-pr-628-affb84c2
(affb84c2ee68fe4d838fc819aef2830de30f4deb); to be replaced by the merged sha
before this lands.

What the new JavaScriptCore does differently: a module's function declarations
are instantiated when their binding is first read and stay in the embedded
bytecode until then; interpreter and Baseline call sites get their link record
on their second execution, in either tier (a tail call on its first), and the
collector is told about those records; value profile predictions and an unlinked code
block's value and array profiles are only allocated for code that warms up;
module and program code is released once it has run; code decoded from an
embedded bytecode cache can be dropped and decoded again
(JSC::VM::shrinkFootprintNow with flags); Heap::totalBytesAllocated() and
sizeAfterLastCollection() for embedders; the thunks of the call slow paths clear
the stack their C++ function's frame is going to occupy, so that what the last
callee left there is not kept alive by a later conservative scan. The bytecode cache format revision goes
from 5 to 6.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Findings marked 🟡 are optional suggestions and need no follow-up push.

Comment on lines +79 to +81
#if ASSERT_ENABLED
#define ASSERT_CALL_SLOW_PATH_RUNS_IN_CLEARED_STACK(calleeFrame) \
ASSERT(std::bit_cast<uintptr_t>(currentStackPointer()) + maxFrameExtentForSlowPathCall + stackBytesClearedForCallSlowPath >= std::bit_cast<uintptr_t>(calleeFrame))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 On C_LOOP debug builds this assertion compares the native C++ stack pointer against calleeFrame, which lives in the heap-allocated CLoopStack, so the inequality is between unrelated address spaces and can fire spuriously on the very first LLInt call slow path. Fix: guard the assertion (or make it a no-op) when ENABLE(C_LOOP), matching the if not C_LOOP guard around the stack-clearing loop in linkFor (LowLevelInterpreter.asm) that the assertion is meant to verify. Same pattern at 4 sites (llint/LLIntSlowPaths.cpp: llint_default_call, llint_unlinked_call, llint_virtual_call, llint_polymorphic_call).

Extended reasoning...

LowLevelInterpreter.asm's linkFor macro wraps the new stack-clearing loop in if not C_LOOP … end, so on the C loop the trampolines call llint_default_call/llint_unlinked_call/llint_virtual_call/llint_polymorphic_call without clearing anything. Each of those functions now does ASSERT_CALL_SLOW_PATH_RUNS_IN_CLEARED_STACK(calleeFrame), which expands to ASSERT(bit_cast<uintptr_t>(currentStackPointer()) + maxFrameExtentForSlowPathCall + stackBytesClearedForCallSlowPath >= bit_cast<uintptr_t>(calleeFrame)). With ENABLE(C_LOOP) the interpreter stack is a heap block (CLoopStack) and calleeFrame points into it, while currentStackPointer() (wtf/StackPointer.h) returns the native thread stack pointer; depending on where the heap block lands relative to the native stack, the comparison can be false and the ASSERT fires on the first JS call in a debug C_LOOP build (which the base handled fine because none of these asserts existed). The JIT thunks (ThunkGenerators.cpp emitCallSlowPath) are not affected because they are compiled only with ENABLE(JIT).

Verification: normal — On C_LOOP debug builds the new assertion compares unrelated address spaces and can fire on any call. The macro added by this diff (Source/JavaScriptCore/assembler/MaxFrameExtentForSlowPathCall.h:79-84) is: ``` #if ASSERT_ENABLED #define ASSERT_CALL_SLOW_PATH_RUNS_IN_CLEARED_STACK(calleeFrame) \ ASSERT(std::bit_cast<uintptr_t>(currentStackPointer()) + maxFrameExtentForSlowPathCall +…

noInline(callExported);
noInline(callPrivate);

for (let i = 0; i < 100000; ++i) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 nit (optional): new JSTests use hardcoded iteration counts (100000, 200000) instead of testLoopCount, violating JSTests/README.md rule 2 (referenced by JSTests/CLAUDE.md), so they cannot exit early in configs where tier-up doesn't matter and risk breaking the 200 ms rule. sweep:for \(let i = 0; i < [12]00000; Fix: replace hardcoded warm-up bounds with testLoopCount (e.g. for (let i = 0; i < testLoopCount; ++i)), keeping small fixed inner counts like 200 as-is.

Extended reasoning...

JSTests/README.md (imported by JSTests/CLAUDE.md) requires new tests to use testLoopCount/wasmTestLoopCount so the harness can scale iterations per configuration, and to run in under 200 ms in all configurations. At least a dozen new tests in this PR (JSTests/modules/lazy-function-declarations-dfg.js:77,97,100,111,114; JSTests/modules/lazy-function-declarations-dfg-slow-path.js; JSTests/stress/baseline-lazy-call-link-info.js; JSTests/stress/llint-lazy-call-link-info*.js; JSTests/stress/get-from-scope-global-variable-watchpoint-lookup.js; and others) hard-code 100000 or 200000 loop bounds. In slow configurations (no-cjit, eager, cloop, collect-continuously) these fixed counts can push individual tests well past 200 ms and cannot be scaled down by the runner, and in configs where the JIT is disabled the extra iterations serve no purpose. Base branch has no such tests; this is introduced by the diff. Structural fix: use testLoopCount for the tier-up warm-up loops across all new test files.

Verification: nit — JSTests/README.md:20 (imported by JSTests/CLAUDE.md via @ README.md) states rule 2: "Use testLoopCount or wasmTestLoopCount to control how many iterations a test runs. The jsc CLI sets these based on the configuration of the test, so tests iterate enough to tier up where that matters and exit early where it doesn't." At the candidate's location,… | nit — JSTests/README.md:20…

…able for its whole life

Records of several module loaders share a ModuleProgramExecutable, the
FunctionExecutables of its function declarations and with them their optimized
code, while each record has its own environment. That is sound because every
such environment is made from the executable's one symbol table: the optimizing
tiers treat the scope of a table that has only seen one environment as a
constant (SymbolTable::singleton(), for closure variables through the table in
resolve_scope's metadata, for imports through
topLevelExecutable->moduleEnvironmentSymbolTable()), and the second environment
made from the table invalidates that.

ScriptExecutable::clearCode() cleared the table, and getUnlinkedCodeBlock() made
a new one (and a new vector of declaration executables) whenever it generated
the code again. A record that had adopted the executable and made its
environment but not run yet (waiting for a dependency suspended at a top-level
await, or suspended itself) when all code is deleted generates the code again to
run; the executable then has unlinked code and is adopted by the next loader,
whose environment comes from the new table. Functions of the module compiled
after that fold the newest loader's exporting environment into the code that
the earlier loaders' functions run as well: an imported binding read in loader a
returns loader d's value.

The table and the vector are now made from the executable's first unlinked code
and stay (an environment keeps its table alive anyway). Code fetched again is
asked for in the code generation mode of the first, since the layout of the
module environment depends on it; JSModuleRecord::getOrMakeExecutable therefore
only lets a record adopt an executable whose mode is the one the record would
ask for. An executable whose code was deleted is still not adopted; it no
longer sits in the clearable-code set for the sake of its table.

* JSTests/stress/module-loaders-share-one-environment-symbol-table.js: Added.
…t first reaches the Baseline JIT

Every UnlinkedCodeBlock allocated one UnlinkedValueProfile per value profile
and one UnlinkedArrayProfile per array profile up front, so that its CodeBlocks
can fold their predictions into them at each collection and the next CodeBlock
linked from it starts from those. Only the DFG reads what ends up in a linked
profile that way, and nearly all functions never leave the LLInt.

Keep the number of array profiles on the UnlinkedCodeBlock, derive the number of
value profiles from the parameter count and the metadata table, and allocate the
two arrays as one ButterflyArray when a BaselineJITPlan is first created for one
of its CodeBlocks (on the main thread; the pointer is published with a release
store and the collector and compiler threads that fold profiles load it once,
with acquire). Until then folding finds nothing to fold into. Builtin functions,
which never fold, no longer get the arrays at all.

The bytecode cache no longer stores the number of value profiles
(cachedTypesFormatRevision 6, which also covers the two format changes that follow). Options::useLazyUnlinkedValueAndArrayProfiles()
(default true) restores the allocation at generation and decode when false.
…Data

Few code blocks have a jump whose target does not fit its operand, but each one
carried an empty hash map for them. With the previous change this brings
sizeof(UnlinkedCodeBlock) from 208 to 192, one size class down for every kind of
unlinked code block. The bytecode cache writes them with the rest of the rare
data (still cachedTypesFormatRevision 6). A static_assert keeps the size where it is in release builds.
…er character class

The table that resolves two-character inline strings had an entry for every
pair of Latin-1 characters: 512 KB per VM, all of it touched when it is zeroed.
Minified names only use $, _, digits and ASCII letters, which is 64 classes, so
a 64 x 64 table (32 KB) covers them. Any other pair shares the direct-mapped,
verified cache the three-character strings use.
InitializeEnvironment creates a JSFunction and a FunctionExecutable for every
heap allocated function declaration of a module before any of its code runs.
A large bundled CLI application declares ~30,000 top-level functions in the
chunks it evaluates before it is ready for input and has read ~17% of them by
then; each unread one costs a 128-byte FunctionExecutable, a 32-byte function
object and, when the code was parsed rather than decoded from the bytecode
cache, an UnlinkedFunctionExecutable.

With Options::useLazyModuleFunctionDeclarations() (default true)
InitializeEnvironment leaves the module environment slot of such a declaration
empty and hands the module record the list of them. The function object is
created when the binding is first read:
- get_from_scope with the new LazyClosureVar resolve type. BytecodeGenerator
  emits ResolvedLazyClosureVar for a module's own declarations; the linker
  turns ModuleVar and closure variables that are function declaration slots of
  a module environment into LazyClosureVar. LLInt, Baseline and LOL test the
  loaded value for empty and take the slow path; the DFG has GetLazyClosureVar
  (clobberizes as a read of the slot that can fire watchpoints and allocate),
  lowered by the FTL with a slow path call. A slot that holds a value is read
  exactly like ClosureVar.
- JSModuleEnvironment::readVariable() for everything else: by-name lookup on
  the environment, module namespace object property access, WebAssembly imports
  of the export, the debugger.
An empty slot that is not a function declaration is still a TDZ binding. Type
and control flow profilers keep the eager path (they want every function's
range up front).

Records of several module loaders that share a ModuleProgramExecutable share the
declarations' FunctionExecutables (ModuleProgramExecutable::functionDeclaration):
the record that reads a declaration first links it, every record fills its own
environment's slot with its own function object. Optimized code of one loader
therefore meets empty slots when the next loader runs it, which
JSTests/modules/module-loaders-lazy-function-declarations.js exercises in the
optimizing tiers without the testing option.

On that application, 12 s after the prompt: FunctionExecutable 55.1k -> 30.0k,
function objects 156k -> 131k, GC heap -6.3 MB, RssAnon -7.2 MB. CPU: LLInt
call-heavy loop -0.8%, Baseline-only +1.6% (test and branch per lazy read),
DFG/FTL unchanged to the instruction.

The bytecode cache stores the resolve type and the slot table: still
cachedTypesFormatRevision 6.
…s from the closure ones

Of the get_from_scope instructions linked by the time a large bundled ESM
application is ready for input, 89% are ClosureVar or LazyClosureVar and 11% are
global accesses (ClosureVar 54.8%, LazyClosureVar 34.4%, GlobalProperty 9.8%,
GlobalVar 1.0%); in a script it is the other way round. The chain of compares
tested GlobalProperty, GlobalVar and GlobalLexicalVar before ClosureVar, so a
closure variable read waited for three compares that a module never needs.

The resolve type is now read as the low byte of the GetPutInfo (one load instead
of a load and a mask; GetPutInfo.h asserts that it fits), and one compare sends
everything above GlobalLexicalVar past the three global types, the last of which
needs no compare of its own any more. Instructions up to the access itself:

  GlobalProperty 4 -> 5, GlobalVar 6 -> 7, GlobalLexicalVar 8 -> 7,
  ClosureVar 10 -> 5, LazyClosureVar (new, with its empty check) 7.

Interpreter-only instruction counts: closure-scope loops -4..-5%
(infer-one-time-closure-ten-vars, infer-closure-const-then-mov), a loop that calls
a global function and accumulates into a global: 45.11 G -> 45.01 G, global-heavy
microbenchmarks within +-1% (function-call +0.35%, array-prototype-indexOf-empty
+0.46%, array-of-contiguous-large -1.1%).
…tecode cache payload

With Options::useLazyModuleFunctionDeclarations(), a module decoded from a
bytecode cache payload that stays around (Decoder::canDeferIntoPayload()) does
not create the UnlinkedFunctionExecutables of the function declarations
InitializeEnvironment would have instantiated: their slots in
UnlinkedCodeBlock::m_functionDecls stay null and the module's
ModuleFunctionDeclarationSlots remembers the Decoder and the record array.
UnlinkedCodeBlock::functionDecl() decodes one when it is first asked for it
(mutator only; collector and compiler threads use functionDeclIfDecoded()),
which is when its binding is first read. The module record refers to the
unlinked code block instead of a vector of executables, and an empty slot is all
that says a declaration is uninstantiated.

A never-read module function goes from 256 bytes of cells (UnlinkedFunctionExecutable,
FunctionExecutable, JSFunction) to an 8-byte empty slot and a 4-byte table entry.
On a large bundled CLI application: UnlinkedFunctionExecutable 56.4k -> 31.3k at
the prompt; `--help` runs 505 M -> 470 M user instructions (-7%) since most of the
declarations of the chunks it loads are never touched.
detect_stack_use_after_return (on by default) moves address-taken locals to a heap-backed fake stack
that the conservative root scan does not visit, so a cell only such a local refers to is collected at
the next GC: with --slowPathAllocsBetweenGCs or --collectContinuously every test that imports more than
one module died in JSPromise::pipeFrom on the promise JSModuleLoader::hostLoadImportedModule had just
got back from fetch(). Same default as the embedder's binary.
…and ArrayProfile on their second execution

Most call sites of most functions run at most once, and most functions never
leave the LLInt, but every op_call carried an 80 byte DataOnlyCallLinkInfo and a
16 byte ArrayProfile in its metadata (96 bytes per call site).

The metadata of the call opcodes without varargs now holds a LazyCallLinkInfo,
which is a pointer to a CallSiteData { DataOnlyCallLinkInfo, ArrayProfile }.
Until a site has run twice it points at one of two CallSiteDatas that the VM
owns and all such sites share ("never executed" and "executed once"). Their
CallLinkInfo looks like a polymorphic call whose destination is a new thunk,
llint_unlinked_call, so the LLInt call fast path is what it was plus one load
of the pointer, and it needs no branch for the unallocated case. The thunk finds
the site through the caller's frame. On the first execution it calls the callee
without a CallLinkInfo (a temporary one on the stack for the error paths and for
callees that are not JS functions) and moves the site to "executed once". On the
second it allocates the site's own CallSiteData and hands it to linkFor(), which
links it, exactly when a CallLinkInfo used to be linked (the first slow path
trip only set the seen bit). op_tail_call has given up the caller's frame by
the time the thunk would run, so it checks for a CallLinkInfo without owner
and takes a slow path that allocates. The varargs opcodes keep their inline
DataOnlyCallLinkInfo; their slow path needed it on the first execution anyway.

The ArrayProfile of op_call / op_call_ignore_result / op_tail_call moves into the
CallSiteData. The LLInt stores |this|'s structure id through the pointer it has
already loaded, into the shared CallSiteData when the site has none of its own;
nobody reads the ArrayProfile of a shared one.

Baseline code loads the pointer as well and expects a CallSiteData that belongs
to the site: CodeBlock::setupWithUnlinkedBaselineCode() gives every site one
before it installs the code, so neither the Baseline nor the optimizing JITs
ever meet a shared CallSiteData. Compiler threads and the collector get at the
CallLinkInfos through CodeBlock::forEachLLIntOrBaselineCallLinkInfo() and at the
ArrayProfiles through CodeBlock::getArrayProfile(); both skip sites without a
CallSiteData of their own, which reads as "no information". A new CallSiteData
is initialized before a store-store fence publishes it and it lives as long as
the MetadataTable. Nothing may write to the CallLinkInfo of a shared
CallSiteData: it is flagged, and every mutator of CallLinkInfo asserts.

sizeof(Metadata): op_call, op_call_ignore_result, op_tail_call 96 -> 8, op_construct
80 -> 8, op_super_construct 88 -> 16, op_iterator_open 120 -> 48, op_iterator_next
136 -> 64. Options::useLazyLLIntCallLinkInfos() = false allocates every
CallSiteData when the CodeBlock is linked.
… line, and only for code that has warmed up

A ValueProfile in front of a MetadataTable was a bucket and a prediction, 16
bytes per profiled instruction. The LLInt and the Baseline JIT only ever store
to the bucket. The prediction is written when a collection or a compiler folds
the bucket into it, which for code that never tiers up buys nothing.

The table is now preceded by the buckets alone (8 bytes each; the LLInt's store
is indexed by the negated profile offset instead of multiplying it, the Baseline
JIT's is the same instruction with another displacement). The predictions are an
array that hangs off the table's LinkingData and is allocated by the first
ValueProfileRef::mergePrediction() that has something to record; it is published
with a compare-and-swap since the mutator, a marker thread and a compiler thread
can all get there first. ValueProfileRef is what CodeBlock::valueProfileForOffset()
and friends now return; without a predictions array it predicts SpecNone.

A collection used to fold every bucket of every live LLInt / Baseline CodeBlock
into its prediction, because nothing marks the cell in a bucket. So that this
does not allocate the predictions of everything that ever ran, a CodeBlock that
is still interpreted, has no predictions yet and whose LLInt execution counter is
below Options::thresholdForValueProfilePredictions() (40: it has been called
twice at most and has not looped much) keeps its samples in the buckets:
CodeBlock::reconcileWeakReferencesAtGCEnd() only clears the buckets whose cell
died, and visitChildren() leaves them alone. Once the code is warmer than that,
or has Baseline code, or a compiler looks at it as an inlining candidate,
predictions are computed as before, starting from whatever the buckets hold.
What is lost is the type of an object that died before its CodeBlock's third
call.

Options::useLazyValueProfilePredictions() = false allocates the predictions
with the table and always folds.
…t set in its metadata

The LLInt and the Baseline JIT never look at it. The DFG bytecode parser uses it
to constant-fold reads of global variables that have been written once; it can
find the set in the SymbolTableEntry of the global object's / global lexical
environment's symbol table, under the table's lock, which is where linking got
it from. With the StructureID moved out of the union into what was padding,
sizeof(OpGetFromScope::Metadata) goes from 24 to 16. m_getPutInfo, m_structureID
and m_operand keep their names, types and meaning.
Counts the live linked CodeBlocks by tier and kind, the unlinked code blocks by kind, the bytes of their metadata tables,
JIT code and instruction streams, the UnlinkedFunctionExecutables whose code is still in the bytecode cache they were
decoded from, the RegExps holding compiled code and the module executables that still hold code. For embedders that
want to see where code memory goes, and for tests about code lifetime.
…d decoded again

An UnlinkedFunctionExecutable decoded from a bytecode cache lets go of its Decoder and record offsets when its code is
first decoded, so the only way to get its unlinked code back after dropping it was to parse the function again, and
such code was therefore never dropped (Heap::deleteAllUnlinkedCodeBlocks does not even see it). In a process whose cache
payload stays mapped for its whole life a dropped block can instead be decoded again for a few thousand instructions.

- A code block decoded from a persistent payload remembers its payload (an index into the new per-VM
  PersistentBytecodePayloads, two bytes in existing padding) and its record's offset (four bytes in existing padding).
  sizeof(UnlinkedCodeBlock) and sizeof(UnlinkedFunctionExecutable) are unchanged.
- UnlinkedFunctionExecutable::returnCodeToCache() puts the executable back into its m_isCached state, naming the code
  blocks' records (negative offsets) and a Decoder for the payload: the payload's live one if there still is one, so
  that what it decoded stays shared. The live child executables of a dropped block are remembered weakly, in position
  order under the parent's record offset (which the block carries, so dropping never reads the payload: at deep idle
  the embedder has paged it out, and faulting it back cost ~28 MB of file pages for ~20 MB of heap released), and
  adopted by the block decoded from the same record later, so closures made before and after share their unlinked
  code. The registry prunes itself after each full collection like a WeakGCMap.
  Only with the lean decoder: the other one remembers cells by record offset and would hand out a dead one.
- Heap::deleteAllUnlinkedCodeBlocks takes which kinds of unlinked code to drop (generated, recoverable from a cache,
  only what nothing links against); Heap::deleteAllCodeBlocks and ScriptExecutable::clearCode can keep what would have
  to be parsed again.
  Compilations that are ready are finished (Heap::completeAllJITPlans allocates: DFG::LazyJSValue) and the set of unlinked
  code that linked code refers to is collected before the heap is prepared for iteration, as in deleteAllCodeBlocks().
- VM::shrinkFootprintNow / shrinkFootprintWhenIdle take flags: KeepCodeThatNeedsParsing (linked code, recoverable
  unlinked code, RegExp code and parser caches go; nothing that needs a re-parse, including the builtins),
  KeepCodeInUse (additionally keeps all linked and RegExp code and the unlinked code of every function that still has
  linked code) and LeaveCollectionToCaller. Without flags they behave as before.
- VM::entryCountFromOutside() and a public Heap::lastActiveCollectionTime() let an embedder tell whether the program
  is really at rest before it asks for any of this.

- A slot of PersistentBytecodePayloads stands for one tree of unlinked code: the payload's bytes as decoded for one
  SourceProvider (an embedder may wrap the same bytes in a new CachedBytecode and SourceProvider per load; what is decoded
  for a provider names it, e.g. as the provider of class sources). Children remembered under one slot are never handed to
  code decoded under another, and code that is private to one executable (a module whose loader has a module scope of its
  own) is not registered at all: its Baseline code assumes one resolution of that scope. Every Decoder of a slot and every
  code block that carries its index hold the slot; with the last of them the payload and provider references go and the
  index is used again, so a process that loads the same file over and over does not accumulate them (was: one entry per
  load until the VM died).
- shrinkFootprintNow also refuses when called from inside the collector, documents that the Keep modes wait for a running
  collection while the flagless mode returns false instead, clears lastException (its stack kept dropped code alive in a
  second generation), and clears the SourceProvider caches in every mode. VM::deleteAllCode (the flagless mode) now deletes
  the code a cache can hand back as well: on a synthetic application the heap after shrink + GC is 5.4 MB instead of 31.7.
- Heap::deleteAllUnlinkedCodeBlocks defers collection from before it finishes the compiler threads' plans until it is done:
  nothing marks while an executable's code block slots turn into {Decoder, offsets}. The two sides publish and read those
  slots with m_isCached in the middle and fences on both sides. Heap::lastActiveCollectionTime() is atomic.

Options::useCodeRecoveryFromBytecodeCache (default on) gates the bookkeeping and with it the recovery.

On a large bundled CLI application (idle 250 s after a 20-turn session): RssAnon 152-154 MB -> 135-138 MB with
KeepCodeInUse, 131 MB when everything is dropped, RssFile unchanged; decoding a dropped block again costs ~2.5k
instructions against ~111k for a re-parse. Synthetic (2000 modules): unlinked code blocks 19,252 -> 2,383, GC heap
33 -> 23 MB.
A module body runs once per module record, but its ModuleProgramCodeBlock,
UnlinkedModuleProgramCodeBlock and CodeCache entry stayed alive for as long as
any function the module created (through FunctionExecutable::topLevelExecutable
and the CodeCache's Strong), the linked code until a full collection found it
past its TTL. Once JSModuleRecord::evaluate sees the body finished,
ModuleProgramExecutable::didFinishEvaluation() drops the linked code, and the
unlinked code together with its CodeCache entry if it was decoded from a
persistent bytecode cache and can be decoded again; the environment's symbol
table and the function declarations' executables stay, and a later
getUnlinkedCodeBlock() keeps them, asks for the code generation mode of the
code it dropped and refuses code that did not come from the payload.

Records of several module loaders share one ModuleProgramExecutable, so the
executable counts the records that have adopted it and not finished yet
(JSModuleRecord::getOrMakeExecutable, didFinishEvaluation): the code goes when
the last one is done, which covers a loader suspended at a top-level await while
another runs the same module to its end. A loader that comes later adopts an
executable that has only let go of recoverable unlinked code and has it decoded
again. A body suspended at a top-level await is not finished and keeps its
unlinked code, also in VM::shrinkFootprintNow. Interpreter::executeProgram,
which creates its ProgramExecutable itself, clears it after the run. Code that
left the interpreter, or has a baseline compile queued, is left to age out: a
compiler thread may be looking at it (GlobalExecutable::canReleaseLinkedCodeNow).

With useLazyModuleFunctionDeclarations a module record referred to the module's
unlinked code so that a declaration nobody has read yet can be instantiated
later; since most never are, that alone kept nearly every module's unlinked code
alive. When the declarations were left in a persistent payload the record's
ModuleFunctionDeclarationSlots can decode any of them on its own, so in that
case the record does not reference the code block: a read links from the
executable's code block while it has one and from the payload otherwise.

Options::useRunOnceCodeRelease (default on).

Options::useSharedModuleFunctionExpressionExecutables (default off): the
FunctionExecutables of the function expressions and classes in a module's
top-level code belong to the ModuleProgramExecutable instead of to each linked
ModuleProgramCodeBlock, so their CodeBlocks and JIT code survive the module's own
linked code and are shared by every evaluation of the module.

Synthetic application of 2,000 modules: ModuleProgramCodeBlock 2,001 -> 1,
UnlinkedModuleProgramCodeBlock 2,564 -> 2, GC heap 39.4 -> 24.3 MB, extra memory
11.8 -> 1.5 MB, startup instructions -4.4%. A large bundled CLI application:
RssAnon 10 s after launch -3 MB.
…ated, for embedders

An embedder that wants to tell an idle program from a busy one that happens not
to grow its heap needs two numbers Heap keeps but does not hand out:
- sizeAfterLastCollection(): m_sizeAfterLastCollect, the live size (cells and
  extra memory) as of the last finished collection, eden or full.
- totalBytesAllocated(): everything the mutator has allocated, cells and
  reported extra memory, the current cycle included. The per-cycle counters are
  reset in updateAllocationLimits(); they are added to a running total right
  before that. Mutator thread only.
Both are read once per embedder GC timer tick.
…to the collector

Since call sites in LLInt / Baseline metadata keep their CallLinkInfo and
ArrayProfile out of line (LazyCallLinkInfo), the 100-odd bytes per call site that
the table used to hold inline, and report with its size, are separate allocations
the collector does not know about: for a function with 20,000 call sites that is
2.2 MB per CodeBlock that neither paces eden collections nor counts towards the
next full one.

MetadataTable::LinkingData counts the CallSiteDatas its call sites own (in what
is padding on 64-bit targets; only the mutator writes it, so a relaxed load and
store). On the allocation side one record is below what
Heap::reportExtraMemoryAllocated() takes note of, so LazyCallLinkInfo::ensureSlow()
reports them 32 at a time, against the owning CodeBlock, which also covers a
CodeBlock that is already marked. On the visiting side CodeBlock::visitChildren()
and estimatedSize() add them for the CodeBlock the table was linked for; the
optimized CodeBlocks that share the table of the one they replace do not, or a hot
function's records would be counted three or four times in every collection.

While here: MetadataTable::sizeInBytesForGC() took a reference on the
UnlinkedMetadataTable (an atomic increment and decrement) from every marker
thread for every CodeBlock; it reads it through the LinkingData now.
…+ function's frame is going to occupy

The slow paths for calls (llint_default_call, llint_virtual_call,
llint_polymorphic_call, llint_unlinked_call; operationDefaultCall,
operationVirtualCall, operationPolymorphicCall) run with the stack pointer at the
callee's frame: their own frame lies over whatever the last callee at that depth
left there, and sanitizeStackForVM(), which they call first thing, can only clear
what is below the function that calls it. What such a frame does not write stays
in reach of the conservative scan of a later, shallower callee's native frames.
How much stays depends on the size of that frame, which nothing controls:
llint_unlinked_call()'s is 216 bytes against llint_default_call()'s 120, and that
was enough to show.

Seen as: `rewrite(); rewrite(); Bun.gc(true)` at the top level of a script keeps
the HTMLRewriter that the last rewrite() made alive through that collection (Bun's
html-rewriter-leak test). The wrapper's address sits in an unwritten slot of the
host function's frame (bindgen_BunObject_jsGc), 176 bytes below the frame pointer
of rewrite()'s and Bun.gc's common caller; a heap snapshot taken at that point has
the wrapper with no incoming edge and no root; SlotVisitor::append(ConservativeRoots)
is what marks it; clearing that one word in a debugger lets it die. The first call
of the `pf()` site went through llint_unlinked_call(), whose frame covers that
address; through llint_default_call() the address is below the frame and cleared.

Rather than keep one of eight frames small, which only the optimizer decides (and
which a different ABI, or no optimization, decides differently), the thunks that
call these functions now clear the stackBytesClearedForCallSlowPath bytes that
the C++ frame is going to occupy before it exists: the linkFor macro in the LLInt,
and emitCallSlowPath() for the JIT's thunks, which is what the default, virtual and
polymorphic call thunks had four copies of. Nothing is written below the stack
pointer (no ABI we target promises that memory, and a signal handler's frame goes
there): the stack pointer moves down over the window, a multiple of 16 bytes, the
window is cleared above it, and it moves back. 256 bytes in a release build (the
largest of these frames is 152 bytes on x86_64; everything the function calls is
below what it clears itself), 32 straight-line stores, 16 stp on ARM64; 2 KB, in a
loop, with assertions or ASan, where ASSERT_CALL_SLOW_PATH_RUNS_IN_CLEARED_STACK()
checks in each slow path that its frame does not reach below the window. The
LLInt's linkFor clears 64 bytes per iteration of its loop.

That is 36 instructions per call that takes one of these slow paths with the JIT,
49 without. sanitizeStackForVM(), which each of them calls, spends about 55 on two
thread-local lookups to find the bounds of the current thread's stack and to check
that the thread holds the API lock; these slow paths run in the middle of JS, on
the thread that holds the lock, so sanitizeStackForVMInCallSlowPath() takes the
bounds from the lock's owner and makes the same two checks of lastStackTop against
them. A call site that takes a slow path on every call (3 M calls of Proxy
callables): 3.715 G -> 3.652 G user instructions with the JIT, 4.889 G -> 4.862 G
without; a plain call loop does not change.
…ns for the second time, as the LLInt does

A CodeBlock that was set up with Baseline code got a CallSiteData for every call
site that did not own one (CodeBlock::ensureCallLinkInfos), and that is every new
CodeBlock of a function whose Baseline code is shared through its
UnlinkedCodeBlock: a module or a CommonJS wrapper that is evaluated again, a
function linked in another realm. For exactly the functions the lazy records are
for, big ones whose call sites mostly run once per evaluation, that is a pass
over the instruction stream, an allocation, an initialization and a free per call
site per evaluation. A CommonJS module with 20,000 calls required 500 times:
7.2 G user instructions before call sites kept their records out of line, 10.8 G
since; resident memory of the loop swings by 100-150 MB with it (the records are
small allocations the allocator purges late, where one metadata allocation of
several MB used to be unmapped at once).

Nothing in Baseline code needs the record to be the site's own. It loads the
pointer from the metadata on every execution, and the CallLinkInfo the unowned
sites share looks like a polymorphic call whose destination is the unlinked call
thunk. That was the LLInt's trampoline; with the JIT it is now a thunk like the
default call thunk (LLInt::unlinkedCall() chooses, next to LLInt::defaultCall()),
calling operationUnlinkedCall(), which returns the exception-throwing stub when
the call threw. It and llint_unlinked_call() share LLInt::handleUnlinkedCall().

All the slow path gets is the shared CallLinkInfo, so the site is what the caller
left in its frame (CallFrame::bytecodeIndex(); the LLInt and Baseline code store it
the same way). handleUnlinkedCall() checks what that rests on: the caller's
CodeBlock is LLInt / Baseline code, the instruction is one of
FOR_EACH_OPCODE_WITH_LAZY_CALL_LINK_INFO (CodeBlock::lazyCallLinkInfoAt() crashes
otherwise), it is not a tail call, and the site does point at a shared
CallLinkInfo. A tail call has given up the caller's frame by then, so a tail call
site gets its own record before it runs for the first time. The LLInt tested for
that already (prepareCallSiteForTailCall). Baseline code pays nothing per tail
call for it: tail call sites start out with a third shared record, whose
CallLinkInfo looks unlinked rather than polymorphic, so their first execution
takes the path of the inline cache that an unlinked call takes anyway, before the
frame is given up, and that path now tests for a CallLinkInfo without an owner and
calls operationEnsureCallLinkInfoForTailCall(). A direct eval whose callee is not
eval makes a virtual call with the site's CallLinkInfo in Baseline code's slow
case, which does the same test first. Nothing looks at the whole instruction
stream any more.

What compiler threads see: more CodeBlocks now have sites that get their record
while an optimizing compile reads them. LazyCallLinkInfo::ownData() orders what
it reads through the pointer after the pointer (the publisher already has a
storeStoreFence), and CallLinkStatus::computeFor() looks the site up when the map
it was given predates the record, so that what DFG::InliningPlan saw and what the
parser sees agree. The ArrayProfile of |this|: both tiers note the structure in
whatever CallSiteData the site has, so the first two executions went into the
shared one, which nobody reads (arrayProfile() only returns a site's own); the
record is now seeded with the structure of |this| when it is made.

One CodeBlock::ensureCallLinkInfoAt() replaces four copies of the lookup, and
$vm.numberOfOwnCallLinkInfos(function) lets a test see which sites own a record.

User instructions, a standalone runtime built on this, 500 requires of a module
(before: the current WebKit main; then this series without and with this change):
- 20,000 plain calls in the CommonJS wrapper:        7.23 G   10.8 G   6.64 G
- 20,000 tail calls in a switch, 4 of them run:     10.7 G    15.0 G   8.37 G
- 2,000 strict functions, each with one tail call:   7.87 G    -       7.29 G
  (with a small heap, --smol: 7.18 G, 8.57 G, 7.71 G; the difference to the line
  above is collections that find more CodeBlocks alive, not these paths)
A large bundled CLI application: --help 554.6 M -> 494.4 M, a 20-request session
23.0-23.6 G -> 22.7-23.1 G.
@Jarred-Sumner
Jarred-Sumner force-pushed the claude/jsc-cold-code-memory branch from affb84c to 72ea768 Compare September 13, 2026 01:24

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review found no new issues

No new issues were found in this update; 2 findings from earlier reviews are still open above.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant