Repository navigation
docs: benchmark the Qualcomm Dragonwing IQ-10 Hexagon NPU - #70
Merged
khanhnd61-vr merged 10 commits intoNov 2, 2026
Merged
Conversation
The Hexagon SDK has no arm64 Linux build, so on a Snapdragon Linux board such as IQ-10 vla.cpp is cross-built in Qualcomm's toolchain container. build_linux_snapdragon.sh runs that build and packages bin/ and lib/ with the HTP skels. With vla-server it installs Debian's arm64 protobuf and ZeroMQ dev packages in the container (multiarch); --no-server skips them. HTP_SKEL_POST runs a command on each skel, for boards that load only processed skels. Verified on IQ-10 (Linux, Hexagon v81): vla-server serves SmolVLA to eval/client/vla_cpp_client.py, and vla_predict_check matches the CPU F32 reference to 7.2e-4.
Document the IQ-10 under Linux: cross-building with scripts/build_linux_snapdragon.sh, installing the HTP skel (FastRPC there ignores ADSP_LIBRARY_PATH and loads only processed skels from /vendor/dsp/cdspN), running vla-server, the CONCAT refusal, and the fidelity of the six archs run so far. Add a CHANGELOG entry.
Flash attention is the default only on Hexagon, where the HMX kernel keeps scores and probabilities in F16. In the Qwen3-VL LM that moves the action chunk 5.4e-3 from the CPU F32 reference on an IQ-10 (v81); with the explicit path it is 2.0e-3, at the same latency. The vision tower and the VL self-attention keep flash attention, and --flash-attn 1 restores it for the LM. Other backends already default to the explicit path, so they are unchanged.
Note the HMX flash-attention precision limit and the N1.7 LM default next to the CONCAT note, update the IQ-10 fidelity table (N1.7 5.4e-3 -> 2.0e-3) and CHANGELOG, and use --flash-attn 0 for the CPU reference: a Hexagon build defaults to flash attention on the CPU too, which moved N1.6 from 1.3e-3 to 1.4e-3 and N1.7 by up to 6.4e-4.
ggml-hexagon claims a MUL_MAT whose weight has no batch dims while the activations do, and computes it wrong. GR00T N1.6's SigLIP2 tower takes two camera views that way ([1152,256,2] x [1152,1152]), which moved its action chunk 0.16 from the CPU F32 reference on an IQ-10 (v81); with these on the CPU it is 1.9e-3, and 25% faster. pi0.5 has one such attention matmul: 3.6e-4 -> 2.7e-4 and 14% faster. The other archs are unchanged.
Document the MUL_MAT the NPU computes wrong (a 2D weight against a batch; GR00T N1.6 with two views 0.16 -> 1.9e-3) next to the other Hexagon issues, update pi0.5 in the fidelity table, and add the serving check: vla-server on the board driven by eval/client with a fixed LIBERO observation, against a CPU F32 server. GR00T N1.5 and N1.6 are above the bar there, as the CPU is at F16. CHANGELOG follows.
The IQ-10 is a Qualcomm Dragonwing part. Rename scripts/build_linux_snapdragon.sh to build_linux_hexagon.sh, since it targets Qualcomm Linux boards with a Hexagon NPU, and its SNAPDRAGON_IMAGE override to TOOLCHAIN_IMAGE. Fix the board name in the script header, docs/backend/hexagon.md and CHANGELOG. The toolchain image name and the Snapdragon X laptop references are unchanged.
Bring in the Dragonwing IQ-10 naming fix and the script rename to build_linux_hexagon.sh. The CHANGELOG IQ-10 entry keeps this branch's text with the new names.
vla-bench under TEMPLATE.md's protocol at e8dfa34, for the six models whose GGUFs are on the board. The flag screen keeps F16 weights and varies only --flash-attn and --mm-prec: the NPU has no BF16 kernels, and F16 is the default. Memory adds the drop in system MemAvailable, since the weights sit in FastRPC buffers that peak RSS leaves out. The CPU section and the archs without GGUFs on the board are left for a later round.
Minimum latency at the defaults from qualcomm-iq10.md, measured at e8dfa34. Models whose GGUFs were not on the board are marked n/r; BitVLA is N/A off CUDA.
thanhdq1o2
added a commit
to thanhdq1o2/vla.cpp
that referenced
this pull request
Nov 3, 2026
VinRobotics#70 was squash-merged into main; its two commits are already on this branch, and the IQ-10 benchmark page and README row keep this branch's re-measured numbers.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the IQ-10 benchmark page and its row in the benchmark table.
Stacked on #68 and #69, which this branch contains. Until they merge, only the last two commits are new:
2a66f9faddsdocs/benchmark/qualcomm-iq10.mdcc3c2bbadds the IQ-10 row todocs/benchmark/README.mdWhat was measured
vla-benchunderTEMPLATE.md: 3 warmups + 20 timed reps, 3 processes, lowest mean reported, ate8dfa34, llama.cppb11223. F16 weights, the Hexagon build's default. Six models, whose GGUFs are on the board:Where it differs from the template
--flash-attn,--mm-precand--flash-attn 0.--weight-dtype bf16is left out because the NPU has no BF16 kernels (SmolVLA: 18.7 s against 0.82 s).--weight-dtype f16is the default.MemAvailablesampled every 0.5 s. Peak RSS leaves out the weights, which sit in FastRPC buffers.Not in this PR
n/r.These rows time the defaults and do not vouch for accuracy. GR00T N1.5 and N1.6 are above the 2.9e-3 bar on a served observation; see
docs/backend/hexagon.md.