Skip to content

Experiment with CPU and memory hotplug snapshots - #402

Draft
sjmiller609 wants to merge 1 commit into
hypeship/ch-dirty-snapshot-expfrom
hypeship/ch-hotplug-overlay-exp
Draft

Experiment with CPU and memory hotplug snapshots#402
sjmiller609 wants to merge 1 commit into
hypeship/ch-dirty-snapshot-expfrom
hypeship/ch-hotplug-overlay-exp

Conversation

@sjmiller609

Copy link
Copy Markdown
Collaborator

summary

  • configure a fixed Cloud Hypervisor vCPU hotplug envelope while preserving a smaller boot CPU count
  • back both boot and virtio-mem regions with one private file layout
  • preserve that layout through full and differential snapshots
  • allow kernel-paged restore and differential standby after CPU and memory resizing

validation

  • booted Chromium with 4 boot vCPUs, 8 max vCPUs, 4 GiB boot memory, and a 4 GiB virtio-mem aperture
  • grew the live VM to 8 vCPUs and 8 GiB in about 60 ms per resize request
  • fanned out the resized 8-vCPU/8-GiB snapshot 25/25: 308 ms average and 369 ms p95 fork-to-CDP
  • differential standby of a resized fork: 2.2s, 459 MB delta; restore API 92 ms with CDP immediately available
  • shrank a fork to 4 vCPUs/4 GiB, then completed differential standby and restore while preserving that shape
  • 25 simultaneous resized-VM standbys completed 25/25 in 42.1s; all 25 subsequently restored
  • Cloud Hypervisor: 56 VMM tests, 42 virtio-device tests, and cargo check -p cloud-hypervisor
  • targeted Hypeman Cloud Hypervisor and snapshot ownership tests

status

This is an isolated experiment stacked on the differential snapshot experiment. It requires the corresponding Cloud Hypervisor branch and is gated by HYPEMAN_EXPERIMENTAL_CH_HOTPLUG_OVERLAY and HYPEMAN_EXPERIMENTAL_CH_MAX_VCPUS.

The core direction works, including heterogeneous fork shapes and repeated standby/restore. The remaining major issue is burst standby latency: sparse delta publication still saturates storage around 40s at 25-way concurrency. The attempted direct-in-place writer was backed out because it made the VMM perform synchronous filesystem writes and timed out under concurrency rather than improving the result.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant