VEX regalloc: track occupied real registers in a bitmask

[codspeed-hq]

Problem

Profiling the benchmark suite shows that only ~2% of a Valgrind/Callgrind run is spent in JIT-generated code — the rest is Valgrind itself, and the VEX translation pipeline is ~30% of it. doRegisterAllocation_v3 is the hottest single function of that pipeline (~4–5% of total runtime).

Line-level profiling pinned ~30% of the register allocator on one loop: the post-instruction house-keeping at the end of Stage 4, which walks all allocatable real registers after every host instruction:

for (UInt r_idx = 0; r_idx < n_rregs; r_idx++) {
   RRegState* rreg = &rreg_state[r_idx];
   switch (rreg->disp) { case Free: break; ... }
}

On amd64/arm64 that is dozens of RRegState entries (spread over ~14 cache lines) inspected per instruction, while in practice only a handful are ever Bound/Reserved — everything else is Free and immediately skipped.

Change

Maintain ULong rregs_inuse, a bitmask where bit r_idx is set iff rreg_state[r_idx].disp != Free (the universe is already capped at 64 registers by STATIC_ASSERT(N_RREGUNIVERSE_REGS == 64), and the allocator already uses ULong register masks).

• FREE_RREG becomes a small free_rreg() helper that clears the bit; the three sites that bind or reserve an rreg set it.
• The post-instruction loop now iterates only over the set bits (ULong__minIndex + clear-lowest-bit), so it visits just the occupied registers.
• The invariant bit set <=> disp != Free is verified inside the allocator's existing periodic sanity-check loop, so a desynchronisation would be caught by the regression tests rather than silently corrupting allocation.

No allocation decisions change — this is purely a cheaper way to find the registers that need post-instruction attention.

Correctness

Both the unpatched (base) and patched (head) trees were built and their regression suites compared in the sandbox (x86_64, Debian 12):

• callgrind: 23/23 pass on both.
• none + memcheck: the set of failing tests is identical between base and head — 30 pre-existing, environment-related failures in this container (fdleak/, stackgrowth, sigstackgrowth, rlimitnofile, map_unmap, track*, xml-track-fds, getdents_filter, bigcode — the last one aborts on an aspacem sync check caused by the sandbox, not by code generation).

Register-allocation bugs produce broken generated code, so these suites are a strong check.

Performance

Measured locally with the exec harness in walltime mode (codspeed run), 8 representative benchmarks, 15 rounds + 1s warmup each. Base and head were measured twice, alternating, to separate the signal from machine drift:

Comparison	Overall impact
base #1 → head #1	+0.83% faster
base #2 → head #2	+0.88% faster
base #1 → base #2 (same build, noise floor)	+0.22%
head #1 → head #2 (same build, noise floor)	+0.27%
head #1 → base #2 (patched vs unpatched, reversed order)	−0.61%

Every individual benchmark moved in the same direction (faster) in both pairs, e.g. stress-ng --cpu 1, no-inline 704.6 ms → 692.7/691.4 ms, llsc_tzconvert_bench, full-with-inline 412.7 ms → 408.7/409.1 ms, python3 test.py, full-with-inline 2.22 s → 2.17 s. The win is modest but consistent (~3× the same-build noise floor) and architecture-independent: every translation on every target benefits. No regressions were observed.

Note: the sandbox is a shared x86_64 VM, so absolute numbers are less precise than the codspeed-macro runners used by CI — the CodSpeed run on this PR is the authoritative measurement.

[codspeed-hq]

Merging this PR will not alter performance

✅ 84 untouched benchmarks
⏩ 60 skipped benchmarks¹

---

_{Comparing codspeed-optim-vex-register-allocator-track-occupied-real-registe-1785647761728 (31db6c6) with master (ae6bf15)}

Footnotes

1. 60 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[greptile-apps]

Greptile Summary

The PR optimizes VEX register-allocation housekeeping by tracking occupied real registers in a 64-bit mask rather than scanning the full register universe after each instruction.

• Centralizes transitions to Free in free_rreg(), which also clears the occupancy bit.
• Sets occupancy bits when registers become bound or reserved.
• Iterates over a snapshot of occupied bits during post-instruction cleanup.
• Extends periodic allocator sanity checks to verify the mask/state invariant.

Confidence Score: 5/5

The PR appears safe to merge, with no actionable correctness or security defects identified.

Every transition between free and occupied register states updates the new mask, relocation cannot clear its destination bit because source and destination indices are distinct, and snapshot iteration visits each register occupied at the start of housekeeping exactly once.

Important Files Changed

Filename	Overview
VEX/priv/host_generic_reg_alloc3.c	Replaces full real-register housekeeping scans with synchronized occupancy-mask iteration; all register disposition transitions preserve the new invariant.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Register becomes Bound or Reserved] --> B[Set occupancy bit]
    B --> C[Execute host-instruction allocation]
    C --> D[Snapshot occupied-register mask]
    D --> E{Any set bits remain?}
    E -->|Yes| F[Select lowest occupied register]
    F --> G{Live range ended?}
    G -->|Yes| H[free_rreg clears state and bit]
    G -->|No| I[Keep register occupied]
    H --> E
    I --> E
    E -->|No| J[Continue to next instruction]

Loading

_{Reviews (1): Last reviewed commit: "VEX regalloc: track occupied real regist..." | Re-trigger Greptile}