fix: discover DEV_DAX devices whose sysfs path has no region component - #81
Open
seohui-XCENA wants to merge 1 commit into
Open
fix: discover DEV_DAX devices whose sysfs path has no region component#81seohui-XCENA wants to merge 1 commit into
seohui-XCENA wants to merge 1 commit into
Conversation
DEV_DAX discovery derived the dax_region index by searching the sysfs symlink target for "region<N>". A device whose dax_region hangs off a platform hmem device resolves to ".../devices/platform/hmem.0/dax0.0", which has no region component, so scanDevices() skipped it and the daemon reported "no CXL/DAX devices found" and rescanned forever with no pool to allocate from. Derive the index from the device name instead: the kernel names every dev_dax "dax<dax_region id>.<dev_dax id>" whatever the provider is, so it does not depend on where the dax_region sits in the hierarchy. The link parse stays as a fallback, and pool ids are unchanged where discovery already worked -- the "region<N>" in a pmem or CXL path is the same region whose id names the device -- which matters because poolId keys pool_<id>.meta and the WAL allocation records. The FS_DAX branch of scanDevices() has had the same name-based fallback all along. Also log skipped devices; the silent skip made an undiscoverable device indistinguishable from an absent one.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤔 Background & Motivation (Why)
The resource manager finds no pool on a machine whose DAX device sits under a platform hmem device, even though the device is present and healthy (
daxctlreportsmode: devdax,/dev/dax0.0exists, the UUID header is written):With no pool there is nothing to allocate from, so
maru-serverandMaruHandlercannot do anything either — Maru is unusable on such a machine, single node included.scanDevices()derived the dax_region index by searching the sysfs symlink target forregion<N>. That works for pmem (.../ndbus0/region0/dax_region0/dax0.0) and for a CXL region, but an hmem-backed device has no region component in its path:so the lookup failed and the device was skipped with no log line — which is why the only visible symptom was "no devices found".
🏗️ Design Changes
region<N>, and that hierarchy differs per provider and per platform. The name does not —alloc_dax_regionanddevm_create_dev_daxare built into the kernel core rather than into any module, anddax_pmem,dax_cxlanddax_hmemall call into them, sodax<dax_region id>.<dev_dax id>is produced by one shared code path for every provider. Keeping the old parse as a fallback also means the set of discovered devices is a superset of what it was before, so a working setup cannot regress.parseRegionIndexFromDaxName(), declared inpool_manager.hso tests can link it.📝 Implementation Details
Only the region-id derivation changes. How the device path is found is untouched — it never went through the symlink:
/dev/+ entry name fromreaddir("/sys/bus/dax/devices")region<N>found anywhere in the sysfs link targetPool ids are unchanged
Worth a reviewer's eye, since
poolIdis a persistence key — it namespool_<id>.meta(metadata.cpp:33), is rejected on mismatch when loading (metadata.cpp:54), and is what WAL replay resolves a pool by (wal.cpp:125-127). A renumbering would orphan existing state.It does not happen here. Wherever discovery already worked, both sources yield the same number:
dax_region->idequals thend_regionid for pmem and the CXL region id for CXL, and that is precisely theregion<N>in the path. Where discovery failed there was no pool and no metadata to be compatible with.✅ Tests
Unit — 3 new gtest cases in
tests/test_dax_region_index.cpp;ctest19/19, clean under-Wall -Wextra -Wpedantic.ParsesRegionIndexFromDeviceNamedax0.0,dax0.1,dax3.7,dax12.0RejectsMalformedNamesdax,dax0,dax0.,dax.0,daxfoo,dax_region,pmem0,""ResolvesHmemDeviceWhoseLinkTargetHasNoRegionregioncomponent, and that the name still resolves — the regressionManual, on an hmem-backed devdax device: the rescan loop is gone and the pool registers at startup.
STATS_REQreports the pool, andALLOC_REQ→ mmap → write →FREE_REQcompletes. The full stack (resource manager +maru-server+ twoMaruHandlerclients storing and retrieving over several rounds) runs green, with the two clients receiving disjoint ranges.Not covered: no pmem or CXL-region machine was available, so the claim that those topologies are unaffected rests on the two arguments above rather than on a run. On such a machine it can be checked in a minute — for every device the first number of the name should equal
dax_region/id, andpool_<id>.metashould keep its existing filename.🔗 Related Issues (optional)
📦 Release Note (for auto-generation / write in English)
NEW
CHANGED
FIXED
region<N>component (e.g. adax_regionon a platform hmem device) were skipped during discovery, leaving the daemon with no pools and no way to allocate.IMPORTANT NOTES