perf: cache per-CPU slot pointer in per-arch registers and drop the percpu crate - #392
Draft
agicy wants to merge 8 commits into
Draft
perf: cache per-CPU slot pointer in per-arch registers and drop the percpu crate#392agicy wants to merge 8 commits into
agicy wants to merge 8 commits into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Cache each physical CPU's PerCpu slot base in a per-arch privileged register so
this_cpu_id()/this_cpu_data()become a register read + load instead of a CPU-id lookup plus table scan:aarch64:TPIDR_EL2x86_64:IA32_GS_BASEriscv64:CSR_SSCRATCH(this is the original one, NOT changed)loongarch64:root CSR SAVE0Also removes the now-unnecessary percpu crate: its only two users were remote-indexed statics, now replaced by plain
MAX_CPU_NUMarrays withrepr(align(64)), deleting the.percpulink-time sections from all boards.ArchCpu::new(x86_64) is unshackled from the pre-cachethis_cpu_id()cycle, and a dead per-VM-exitMPIDR_EL1read + table scan is dropped from the aarch64 exit path.Motivation
CPU-local access sits on every hot path: each log record, external-IRQ dispatch, IPI send, and the event loop. Before this change:
x86_64paid a serializing CPUID leaf-1 + ACPI lookup per call;aarch64readMPIDR_EL1and resolved the mapping per call;loongarch64re-readCPUNUMand re-derived the slot every time.The slot base is a boot-time constant per CPU; re-deriving it per access is pure waste. The percpu crate had become dead weight: only its remote-indexed statics were used, so it contributed only link-time sections, per-CPU init calls, and a dependency for two structures that are now plain arrays.
Verification
devvs this branch on identical environment):this_cpu_id()=mrs TPIDR_EL2+ldr(2 instructions) vs.mrs MPIDR_EL1+addressing(~6-8) before.this_cpu_id+28k cycles (branch) vs +558k cycles (dev),this_cpu_data().id+23k vs +645k. About 20–28x reduction is here.Risks and Limitations
this_cpu_id()/this_cpu_data()are valid only after PerCpu::new has written the register cache on that CPU. All current paths are safe via theENTERED_CPUSbarrier. But any futurepre-PerCpu::newlog/panic path would crash where the old code returned a hardware-derived id. riscv64 already lived under this rule.GS_BASEsnapshot being taken by the pCPU that owns that VMCS (structurally guaranteed today: one VMCS per PerCpu slot); cross-pCPU VMCS reuse would require re-snapshotting.SAVE0was chosen as a free root CSR after auditingSAVE3/SAVE4(trap handoff) and theGCSRfile (guest state); firmware sharing rootSAVE0would conflict.TPIDR_EL2is untouched by EL3 firmware acrossPSCIcalls.References
Related: hvisor-book #71.