Skip to content

Resolve unbounded NPU input shapes before compile - #4047

Open
pwolnows wants to merge 6 commits into
openvinotoolkit:masterfrom
pwolnows:upper_bound_dynamic_shapes_for_NPU
Open

Resolve unbounded NPU input shapes before compile#4047
pwolnows wants to merge 6 commits into
openvinotoolkit:masterfrom
pwolnows:upper_bound_dynamic_shapes_for_NPU

Conversation

@pwolnows

Copy link
Copy Markdown
Contributor

Resolve unbounded NPU input shapes before compilation

Summary

This change adds a bounded undefined-shape resolving policy for Accuracy Checker. The new policy bounds fully unbounded dynamic input dimensions before model compilation/loading and uses the default policy otherwise.

NPU can compile dynamic dimensions only when each dynamic dimension has an explicit upper bound. Models converted with inputs such as [-1, 3, -1, -1] expose unbounded batch, height, and width dimensions, which OpenVINO represents internally with INT64_MAX as the upper bound. The NPU plugin rejects these shapes during compilation because it cannot statically size buffers and tiling for an open-ended dimension.

The fix resolves unbounded input shapes from the first preprocessed input batch, or from the existing template shape range when available, before compiling/loading the model. NPU validation jobs can opt in to this behavior by setting --undefined_shapes_resolving_policy bounded.

Changes

  • Added bounded as a supported --undefined_shapes_resolving_policy value.
  • Added unbounded dynamic-shape handling in the OpenVINO launcher when bounded is selected.
  • Added the same handling in the legacy dlsdk launcher, since old configs still instantiate DLSDKLauncher directly.
  • Preserved existing default, dynamic, and static policy behavior.
  • Preserved dynamic-shape behavior for models that already have bounded dynamic dimensions.
  • Added focused tests for both launcher paths:
    • unbounded NPU dynamic shapes are reshaped before compilation/loading
    • bounded NPU dynamic shapes continue through the existing dynamic initialization path

Behavior

For an input like:

pixel_values: [-1, 3, -1, -1]

With --undefined_shapes_resolving_policy bounded, Accuracy Checker now resolves the shape before compilation using the shape of the actual preprocessed data. For example, an ImageNet ResNet config that resizes then crops to 224x224 resolves to:

pixel_values: [1, 3, 224, 224]

This avoids NPU compile failures such as:

Upper bounds are not specified ... bounds are [9223372036854775807, 3, 9223372036854775807, 9223372036854775807]

Validation

Run locally:

python3 -m compileall -q \
  tools/accuracy_checker/accuracy_checker/argparser.py \
  tools/accuracy_checker/accuracy_checker/launcher/dlsdk_launcher_config.py \
  tools/accuracy_checker/accuracy_checker/launcher/openvino_launcher.py \
  tools/accuracy_checker/accuracy_checker/launcher/dlsdk_launcher.py \
  tools/accuracy_checker/tests/test_openvino_launcher.py \
  tools/accuracy_checker/tests/test_dlsdk_launcher.py
git --no-pager diff --cached --check

Also attempted focused pytest coverage:

python3 -m pytest \
  tools/accuracy_checker/tests/test_openvino_launcher.py \
  tools/accuracy_checker/tests/test_dlsdk_launcher.py \
  -k 'bounded'

The focused pytest command was skipped in the local environment because OpenVINO Python packages are not installed there.

Notes

This is primarily an NPU compatibility fix, not a CPU/GPU performance optimization. CPU and GPU tolerate unbounded dynamic dimensions in this path, while NPU requires bounded or static dimensions before compilation. The behavior is opt-in through bounded so NPU jobs can enable it without changing the semantics of the existing dynamic policy.

@pwolnows
pwolnows enabled auto-merge (squash) August 21, 2026 07:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants