Summary
perf rejects the directory produced by a successful build
Priority: P1, confirmed by the release driver on 2026-09-23.
Bugbash reference: 2026-09-22 / BUG-003.
Status: Observed in the first test round; no fix or post-fix regression is recorded. This issue is based on retained test evidence, not a new test run.
Environment
- Test date: 2026-09-22; findings triaged on 2026-09-23.
- CLI: winml-cli 0.4.0, installed from a local wheel. The exact source commit/build provenance is not recorded in the test evidence.
- Hardware: Snapdragon X Elite X1E80100, Qualcomm Hexagon NPU.
- OS: Windows 11 26H1, build 28000.2956, ARM64.
- Python: 3.11.15, workspace virtual environment.
- EP: QNNExecutionProvider from WinML Catalog, package 2.2480.49.0.
- ONNX Runtime: 1.27.1.202607110137, Windows ML distribution.
- ONNX: 1.18.0; PyTorch: 2.14.0; Transformers: 4.57.6.
- Model: catalog microsoft/resnet-50, image-classification, pixel_values FP32 [1, 3, 224, 224].
- Quantization, where applicable: w8a16, uint8 weights / uint16 activations, 10 calibration samples.
Reproduction
Run in PowerShell with winml-cli 0.4.0 installed. Choose previously unused output paths.
$buildOutput = '.\repro-build-directory'
uv run winml build -m microsoft/resnet-50 --ep qnn --device npu --compile -o "$buildOutput"
uv run winml perf -m "$buildOutput" --ep qnn --device npu --warmup 1 --iterations 5 -o '.\repro-directory-perf.json'
The catalog build succeeds and creates model.onnx plus its compiled external context before the perf directory-input failure.
Expected result
The documented build-output-directory input resolves the generated model.onnx.
Actual result
perf exits with code 1:
Cannot resolve a concrete architecture from a model_type-less generic config.
The same build's model.onnx succeeds when passed directly, so this is an artifact-discovery/directory-input failure rather than evidence that the built model cannot execute.
Workaround
Pass the ONNX file instead of its parent directory:
uv run winml perf -m "$buildOutput\model.onnx" --ep qnn --device npu --warmup 1 --iterations 5 -o '.\repro-file-perf.json'
This direct-file path was verified successfully in the bugbash.
Scope and evidence
perf help advertises support for a build output directory. The observed failure breaks the documented build-to-perf directory handoff. This report does not establish whether other architectures or EPs are affected.
The following evidence is retained locally by the reporter; these filenames are an index, not uploaded attachments:
perf-build-directory.stderr.logperf-build-model.json
Summary
perf rejects the directory produced by a successful build
Priority: P1, confirmed by the release driver on 2026-09-23.
Bugbash reference: 2026-09-22 / BUG-003.
Status: Observed in the first test round; no fix or post-fix regression is recorded. This issue is based on retained test evidence, not a new test run.
Environment
Reproduction
Run in PowerShell with winml-cli 0.4.0 installed. Choose previously unused output paths.
The catalog build succeeds and creates model.onnx plus its compiled external context before the perf directory-input failure.
Expected result
The documented build-output-directory input resolves the generated model.onnx.
Actual result
perf exits with code 1:
The same build's model.onnx succeeds when passed directly, so this is an artifact-discovery/directory-input failure rather than evidence that the built model cannot execute.
Workaround
Pass the ONNX file instead of its parent directory:
This direct-file path was verified successfully in the bugbash.
Scope and evidence
perf help advertises support for a build output directory. The observed failure breaks the documented build-to-perf directory handoff. This report does not establish whether other architectures or EPs are affected.
The following evidence is retained locally by the reporter; these filenames are an index, not uploaded attachments:
perf-build-directory.stderr.logperf-build-model.json