feat(detector): support Hygon MIG detection - #22
Merged
Conversation
- ctypes binding over libhydmi_mig.so, loaded RTLD_LOCAL (the vendor exports NVML symbol names; global linkage would collide with libnvidia-ml.so), searched on the default linker path plus /opt/hyhal/lib and /opt/dtk/lib - struct layouts follow the vendor's dmi_mig.h v1.3.1, not pynvml - DMI-prefixed wrappers raise the package's own DMIError subclasses on any non-success return, never a bare OSError - array queries re-wrap handles at full pointer width; the vendor API has no init/shutdown entry point, so dmiInit only loads the library - smoke-validated on root@10.211.10.37 against libhydmi_mig.so.1.3: load, node-wide MIG mode, handle-by-BDF, memory reads on all 8 DCUs Task 1 of hygon-mig-detection. Signed-off-by: thxCode <thxcode0824@gmail.com>
- read the node-wide MIG mode via pydmi; when enabled, every card reachable through libhydmi_mig.so is marked mig: true and carries its live instances in appendix.mig_devices - enumerate instances by sweeping the node-global MIG index space, attributing each handle by its GI info's parent device, never by index ranges; per-instance failures are suppressed individually - instance identity comes from the vendor registry /etc/dmi_mig_config/ci/dev<N>gi<G>ci<C>.conf (reported as MIG-<uuid>), falling back to a synthetic MIG-<bdf>-gi<G>-ci<C> when unreadable - name/memory/cores come from the GI profile sweep; indexes are assigned by index_mig_devices; a missing library or an unreadable mode degrades to physical-only detection Task 2 of hygon-mig-detection. Signed-off-by: thxCode <thxcode0824@gmail.com>
- read each instance's memory and compute utilization through its own MIG device handle and merge by UUID via merge_devices_usage - a single unreadable instance is suppressed individually, never aborting the sweep; instances carry the card's temperature and power - the usage query re-reads the node-wide MIG mode and stays physical-only when the mode is off or the library is missing Task 3 of hygon-mig-detection. Signed-off-by: thxCode <thxcode0824@gmail.com>
- keep a MIG device's per-card discovery ordinal when an earlier instance's read fails, instead of renumbering the survivors - derive the physical core count from GPU-instance profiles while MIG is enabled, where HSA exposes only a partition's view - carry the card's memory-health verdict into its MIG entries - tolerate non-UTF-8 vendor strings in profile names and the conf registry instead of aborting the whole detection Signed-off-by: thxCode <thxcode0824@gmail.com>
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces Python bindings for the Hygon DMI Multi-Instance library (pydmi) and implements Multi-Instance GPU (MIG) detection and usage tracking for Hygon GPUs. It also adds comprehensive unit tests for both the pydmi bindings and the updated Hygon detector.
Feedback on the changes suggests refactoring duplicated code in hygon.py into helper functions, specifically for checking if MIG is enabled and for retrieving the DMI device handle and index. Additionally, it is recommended to use the with statement for managing threading.Lock in pydmi/__init__.py to ensure idiomatic and safe lock handling.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Align the Hygon detector with the operator's Hygon MIG support (gpustack-operator v0.8.6), the same way NVIDIA and T-Head MIG are already handled:
gpustack_runtime/detector/pydmi/— a vendored ctypes binding for the vendor'slibhydmi_mig.so(DMI MIG library). The vendor exports this API under NVML's symbol names while implementing something else, so every wrapper isdmi-prefixed and the library is loadedRTLD_LOCALto avoid colliding withlibnvidia-ml.soin a process that has both. Struct layouts follow the vendor'sdmi_mig.h(v1.3.1), not pynvml. The vendor API has no init/shutdown calls, sodmiInitonly loads and resolves the library.detect_info— when the node-wide MIG mode is on, each reachable card is markedmig: Trueand its live instances are enumerated intoappendix["mig_devices"]with the same inventory detail a plain device carries: conf-registry UUID (MIG-<uuid>, syntheticMIG-<bdf>-gi<G>-ci<C>fallback), profile-derived name/memory/cores, GI/CI ids, and placement. Numbering rides the sharedindex_mig_devicesper-card blocks. With MIG enabled HSA exposes only a partition's view of the card, so the physical core count is derived from the GPU-instance profiles. A missing library, an unreadable mode, or a DMI-unreachable card degrades to exactly today's physical-only behavior; one unreadable instance is suppressed individually and never aborts the sweep.detect_usage— each live instance's memory total/used and cores utilization are read through its own MIG device handle and merged by UUID viamerge_devices_usage.The detector stays strictly read-only: no create/destroy/set-mode symbols are bound,
hy-smiis never invoked, and no new dependencies are added.Test plan
make lintclean;uv run pytest tests/— 605 passed, 20 skipped (new:test_pydmi.pybinding/layout/error-path tests, Hygon MIG inventory/usage/degradation tests, CLImig_devicesJSON serialization case).hy-smi mig -lgi/-lciand/etc/dmi_mig_config/ci/*.confone-to-one on UUID, memory, GI/CI ids, and placement. The machine was restored afterwards (instances destroyed, MIG mode disabled).Notes
Known fail-safe gaps on non-standard configurations (heterogeneous MIG capacities, sub-GI compute profiles) and standalone docker/podman deployment of Hygon MIG partitions are deliberately out of scope here and tracked as follow-ups.