Skip to content

perf: cache per-CPU slot pointer in per-arch registers and drop the percpu crate - #392

Draft
agicy wants to merge 8 commits into
devfrom
perf/per-cpu-register-cache
Draft

perf: cache per-CPU slot pointer in per-arch registers and drop the percpu crate#392
agicy wants to merge 8 commits into
devfrom
perf/per-cpu-register-cache

Conversation

@agicy

@agicy agicy commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Summary

Cache each physical CPU's PerCpu slot base in a per-arch privileged register so this_cpu_id() / this_cpu_data() become a register read + load instead of a CPU-id lookup plus table scan:

  • aarch64: TPIDR_EL2
  • x86_64: IA32_GS_BASE
  • riscv64: CSR_SSCRATCH (this is the original one, NOT changed)
  • loongarch64: root CSR SAVE0

Also removes the now-unnecessary percpu crate: its only two users were remote-indexed statics, now replaced by plain MAX_CPU_NUM arrays with repr(align(64)), deleting the .percpu link-time sections from all boards. ArchCpu::new (x86_64) is unshackled from the pre-cache this_cpu_id() cycle, and a dead per-VM-exit MPIDR_EL1 read + table scan is dropped from the aarch64 exit path.

Motivation

CPU-local access sits on every hot path: each log record, external-IRQ dispatch, IPI send, and the event loop. Before this change:

  • x86_64 paid a serializing CPUID leaf-1 + ACPI lookup per call;
  • aarch64 read MPIDR_EL1 and resolved the mapping per call;
  • loongarch64 re-read CPUNUM and re-derived the slot every time.

The slot base is a boot-time constant per CPU; re-deriving it per access is pure waste. The percpu crate had become dead weight: only its remote-indexed statics were used, so it contributed only link-time sections, per-CPU init calls, and a dependency for two structures that are now plain arrays.

Verification

  • Builds: make release for aarch64, riscv64, loongarch64, x86_64.
  • QEMU aarch64 microbenchmark (CNTPCT, single core before secondary wakeup, 2M calls * 5 rounds, dev vs this branch on identical environment):
    • Disassembly: this_cpu_id() = mrs TPIDR_EL2 + ldr (2 instructions) vs. mrs MPIDR_EL1 + addressing (~6-8) before.
    • Net cost per 2M calls vs. empty-loop baseline: this_cpu_id +28k cycles (branch) vs +558k cycles (dev), this_cpu_data().id +23k vs +645k. About 20–28x reduction is here.

Risks and Limitations

  1. New boot invariant: this_cpu_id() / this_cpu_data() are valid only after PerCpu::new has written the register cache on that CPU. All current paths are safe via the ENTERED_CPUS barrier. But any future pre-PerCpu::new log/panic path would crash where the old code returned a hardware-derived id. riscv64 already lived under this rule.
  2. x86_64 correctness relies on each VMCS host-state GS_BASE snapshot being taken by the pCPU that owns that VMCS (structurally guaranteed today: one VMCS per PerCpu slot); cross-pCPU VMCS reuse would require re-snapshotting.
  3. loongarch64 SAVE0 was chosen as a free root CSR after auditing SAVE3/SAVE4 (trap handoff) and the GCSR file (guest state); firmware sharing root SAVE0 would conflict.
  4. aarch64 assumes TPIDR_EL2 is untouched by EL3 firmware across PSCI calls.
  5. Perf gain is uneven: largest on x86_64, moderate on aarch64, neutral on riscv64/loongarch64, which mainly gain the unified abstraction and crate removal.

References

Related: hvisor-book #71.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant