bf3 isolated mode fixes

[fantdl]

• mellanox: gate firmware reset for deselected NICs
  Preserve firmware VF settings by default when a NIC is removed from policy. Restore the previous behavior with the mellanoxFirmwareResetOnDeselect feature gate.

  Why: when a NIC leaves all policies (node deselected, policy deleted), the plugin staged NUM_OF_VFS=0 in firmware without a reboot. The staged value then applies at an arbitrary future reset - observed on a BF3 node as NICs coming up with zero VF capacity after an unrelated reboot. On BF3 isolated-mode substrates firmware is provider-owned and the write fails outright, turning this path into a sync-error loop. Firmware VF capacity is only a ceiling; leaving it in place after deselection has no functional cost.

• mellanox: preserve firmware VF capacity
  Only increase TotalVfs to satisfy policy and cancel staged reductions. Explicit numVfs=0 policies still disable SR-IOV.

  Why: the plugin aligned firmware NUM_OF_VFS exactly to the policy's numVfs, including downward. Reproduced on a BF3 host: a policy with numVfs=4 against firmware 8 staged a reduction and auto-rebooted the node; on hardware where a warm reboot does not reset the card the staged value never applies, so every boot re-staged and re-rebooted - an unbounded reboot loop broken only by deleting the policy. Reducing the firmware ceiling to match one policy is never functionally required.

• generic: reconcile VF drift on externally managed PFs
  Detect managed VF changes without treating externally owned PF settings as drift.

  Why: on BF3 isolated-mode hosts all PFs run externallyManaged: true (the host owns VF creation and PF config), and these interfaces were excluded from status-change detection entirely. Observed: VFs recreated out of band stayed bound to the default kernel driver instead of vfio-pci while the nodestate reported Succeeded - recoverable only by restarting the config daemon. Now driver drift triggers reconcile, and missing VFs also trigger it so validation surfaces a Failed status instead of silently running under capacity; settings the operator does not own on such PFs (extra VFs, PF MTU) are not treated as drift, to avoid reconcile loops.

[github-actions]

Thanks for your PR,
To run vendors CIs, Maintainers can use one of:

• /test-all: To run all tests for all vendors.
• /test-e2e-all: To run all E2E tests for all vendors.
• /test-e2e-nvidia-all: To run all E2E tests for NVIDIA vendor.

To skip the vendors CIs, Maintainers can use one of:

• /skip-all: To skip all tests for all vendors.
• /skip-e2e-all: To skip all E2E tests for all vendors.
• /skip-e2e-nvidia-all: To skip all E2E tests for NVIDIA vendor.
  Best regards.

[broly-code-security-scanner]

Broly Security Scan

Note

✅ Clean scan
No vulnerabilities detected in this PR.

Note

Re-scan this PR anytime with /broly scan — useful after /broly undismiss, or to refresh findings without a new push.

Broly — SAST (zai-org/GLM-5.2) · Secrets · SCA · IaC · GH Actions · Base Images · Supply Chain Threats · Exploit Chains · Adversarial Verification

We're continuously improving Broly's accuracy and finding quality — your feedback is valuable. False positives, missed findings, bugs, and feature requests all welcome.