Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .claude/skills/qualcomm/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,12 +83,12 @@ LD_LIBRARY_PATH=/path/to/executorch/build-x86/lib:/path/to/qnn_sdk/lib/x86_64-li
PYTHONPATH=$(dirname $EXECUTORCH_ROOT) \
python backends/qualcomm/tests/test_qnn_delegate.py \
TestQNNFloatingPointOperator.test_qnn_backend_abs \
-H $HOST -s $DEVICE_SERIAL -m SM8850 -b build-android -a /path/to/artifacts
--host $HOST --device $DEVICE_SERIAL --soc_model SM8850 --build_folder build-android -a /path/to/artifacts
```

> **Note (build from source):** Set `PYTHONPATH` to the parent directory of the executorch repo root. Required because `executorch.examples.qualcomm` lives in the source tree and is not installed into site-packages.

Required flags: `-m` (SoC model), `-b` (Android build dir). Optional: `-s` (device serial), `-H` (host), `-a` (artifact dir), `-c` (compile only), `-x` (run on x86_64).
Required flags: `--soc_model` (SoC model), `--build_folder` (Android build dir). Optional: `--device` (device serial), `--host` (host), `-a` (artifact dir), `--compile_only`, `--enable_x86_64`.

**Test classes:**

Expand Down
4 changes: 2 additions & 2 deletions .claude/skills/qualcomm/qnn_intermediate_debugger.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,12 +137,12 @@ Present the result as a fenced bash block, prefixed by a one-line note of what c
>
> ```bash
> python -m examples.qualcomm.util_scripts.my_model_debug \
> -H $HOST -s $DEVICE_SERIAL -m $SOC_MODEL -b build-android \
> --host $HOST --device $DEVICE_SERIAL --soc_model $SOC_MODEL --build_folder build-android \
> -d /path/to/dataset -a ./my_model_debug \
> --dump_intermediate_outputs
> ```

If the user did not give you a runnable original command in step 1 (e.g. they pasted only the script path), do **not** fabricate values for `-H` / `-s` / `-m` / `-b` / `-a` / `-d`. Stop and ask before printing — wrong device or SoC values waste a full export + on-device run.
If the user did not give you a runnable original command in step 1 (e.g. they pasted only the script path), do **not** fabricate values for `--host` / `--device` / `--soc_model` / `--build_folder` / `-a` / `-d`. Stop and ask before printing — wrong device or SoC values waste a full export + on-device run.

After the command runs, the artifact dir will contain SVG / CSV reports — green nodes pass, red nodes fail the comparator threshold. That's the first place to look for the layer that introduces the gap.

Expand Down
4 changes: 2 additions & 2 deletions .wiki/backends/qnn/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ There are two code paths for exporting LLMs to QNN. **Use `examples/qualcomm/oss
```bash
# CORRECT path for QNN LLM export
python examples/qualcomm/oss_scripts/llama/llama.py \
-b build-android -m SM8650 \
--build_folder build-android --soc_model SM8650 \
--decoder_model qwen3-0_6b \
--model_mode hybrid --max_seq_len 1024 --prefill_ar_len 128 \
--prompt "..." --tasks wikitext --limit 1 --compile_only
Expand Down Expand Up @@ -131,7 +131,7 @@ Use `--compile_only` to export, quantize, and compile a `.pte` without running o

```bash
python examples/qualcomm/oss_scripts/llama/llama.py \
-b build-android -m SM8650 --compile_only \
--build_folder build-android --soc_model SM8650 --compile_only \
--decoder_model qwen3-0_6b ...
```

Expand Down
2 changes: 1 addition & 1 deletion .wiki/quantization/debugging.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Quantized LLMs (especially 1B parameter models with 4-bit weights) frequently pr
```bash
# Use the QNN-specific script with better quantization
python examples/qualcomm/oss_scripts/llama/llama.py \
--compile_only -m SM8750 \
--compile_only --soc_model SM8750 \
--model_mode hybrid \
--decoder_model <model> \
...
Expand Down
2 changes: 1 addition & 1 deletion backends/qualcomm/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,7 @@ PRs are always welcome to help improve the codebase in a comprehensive manner. B
```bash
cd $PATH_TO_EXECUTORCH
# example usage of performing unit test
python backends/qualcomm/tests/test_qnn_delegate.py -k TestQNNQuantizedOperator.test_qnn_backend_layer_norm -s $DEVICE_SERIAL -m SM8650 -b build-android/ -a $PATH_TO_TEST_ARTIFACTS
python backends/qualcomm/tests/test_qnn_delegate.py -k TestQNNQuantizedOperator.test_qnn_backend_layer_norm --device $DEVICE_SERIAL --soc_model SM8650 --build_folder build-android/ -a $PATH_TO_TEST_ARTIFACTS
```
The test graph is expected to have 1 delegated node with only placeholders / output nodes being left. Check the execution report for more information.

Expand Down
6 changes: 3 additions & 3 deletions backends/qualcomm/debugger/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ pip install qairt-visualizer
## Quick start
This command launches an interactive GUI interface to visualize the `optrace` and `QHAS` results.
```
python -m examples.qualcomm.util_scripts.qairt_visualizer_demo -H ${host} -s {device} -m ${SOC_MODEL} -b build-android -a ${path_to_output_folder} --online_prepare
python -m examples.qualcomm.util_scripts.qairt_visualizer_demo --host ${host} --device {device} --soc_model ${SOC_MODEL} --build_folder build-android -a ${path_to_output_folder} --online_prepare
```
- If online prepare mode is `enabled`, the following artifacts will be generated:
- `model`.dlc
Expand Down Expand Up @@ -259,7 +259,7 @@ An Inception_V3 demo script is provided at [qnn_intermediate_debugger_demo.py](.
Before running, ensure the dataset is downloaded. An example dataset can be retrieved [here](https://www.kaggle.com/datasets/ifigotin/imagenetmini-1000).

```bash
python -m examples.qualcomm.util_scripts.qnn_intermediate_debugger_demo -b build-android -s $DEVICE_SERIAL -m $SOC_MODEL -d path/to/imagenet/val --dump_intermediate_outputs
python -m examples.qualcomm.util_scripts.qnn_intermediate_debugger_demo --build_folder build-android --device $DEVICE_SERIAL --soc_model $SOC_MODEL -d path/to/imagenet/val --dump_intermediate_outputs
```

## Limitations
Expand Down Expand Up @@ -290,7 +290,7 @@ The difference between the two values represents heap consumed during context ex
```bash
python backends/qualcomm/tests/test_qnn_delegate.py \
TestQNNQuantizedUtils.test_qnn_backend_runtime_option_heap_profile \
-b build-android -H ${HOST} -s ${SN} -m ${SOC_MODEL}
--build_folder build-android --host ${HOST} --device ${SN} --soc_model ${SOC_MODEL}
```

See [test_qnn_delegate.py](../tests/test_qnn_delegate.py) for the full test implementation.
Expand Down
6 changes: 3 additions & 3 deletions docs/source/backends-qualcomm.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,7 +149,7 @@ We use deeplab-v3-resnet101 as an example in this tutorial. Run below commands t
```bash
cd $EXECUTORCH_ROOT

python -m examples.qualcomm.scripts.deeplab_v3 -b build-android -m SM8550 --compile_only --download
python -m examples.qualcomm.scripts.deeplab_v3 --build_folder build-android --soc_model SM8550 --compile_only --download
```

You might see something like below:
Expand Down Expand Up @@ -287,9 +287,9 @@ The model is merely executed. If we want to feed real inputs and get model outpu
```bash
cd $EXECUTORCH_ROOT
# android
python -m examples.qualcomm.scripts.deeplab_v3 -b build-android -m SM8550 --download -s <device_serial>
python -m examples.qualcomm.scripts.deeplab_v3 --build_folder build-android --soc_model SM8550 --download --device <device_serial>
# (optional) linux embedded
python -m examples.qualcomm.scripts.deeplab_v3 -b build-oe-linux -m SXR1230P --download -s <device_serial> -t aarch64-oe-linux-gcc-9.3
python -m examples.qualcomm.scripts.deeplab_v3 --build_folder build-oe-linux --soc_model SXR1230P --download --device <device_serial> --target aarch64-oe-linux-gcc9.3
```
The `<device_serial>` can be found by `adb devices` command.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ To export with the Qualcomm AI Engine Direct Backend, ensure the following:

```bash
# export llama
python examples/qualcomm/oss_scripts/llama/llama.py -b build-android -s ${SERIAL_NUM} -m ${SOC_MODEL} --checkpoint consolidated.00.pth --params params.json --tokenizer_model tokenizer.model --decoder_model llama3_2-3b_instruct --model_mode kv --max_seq_len 1024 --prompt "I would like to learn python, could you teach me with a simple example?" --tasks wikitext --limit 1 --compile_only
python examples/qualcomm/oss_scripts/llama/llama.py --build_folder build-android --device ${SERIAL_NUM} --soc_model ${SOC_MODEL} --checkpoint consolidated.00.pth --params params.json --tokenizer_model tokenizer.model --decoder_model llama3_2-3b_instruct --model_mode kv --max_seq_len 1024 --prompt "I would like to learn python, could you teach me with a simple example?" --tasks wikitext --limit 1 --compile_only
```
Note: end-to-end [instructions](https://github.com/pytorch/executorch/blob/main/examples/qualcomm/oss_scripts/llama/README.md)

Expand All @@ -81,7 +81,7 @@ Note: end-to-end [instructions](https://github.com/pytorch/executorch/blob/main/
**3.3 Run model**
```bash
# Run llama
python examples/qualcomm/oss_scripts/llama/llama.py -b build-android -s ${SERIAL_NUM} -m ${SOC_MODEL} --checkpoint consolidated.00.pth --params params.json --tokenizer_model tokenizer.model --decoder_model llama3_2-3b_instruct --model_mode kv --max_seq_len 1024 --prompt "I would like to learn python, could you teach me with a simple example?" --tasks wikitext --limit 1 --pre_gen_pte ${PATH_TO_ARTIFACT}
python examples/qualcomm/oss_scripts/llama/llama.py --build_folder build-android --device ${SERIAL_NUM} --soc_model ${SOC_MODEL} --checkpoint consolidated.00.pth --params params.json --tokenizer_model tokenizer.model --decoder_model llama3_2-3b_instruct --model_mode kv --max_seq_len 1024 --prompt "I would like to learn python, could you teach me with a simple example?" --tasks wikitext --limit 1 --pre_gen_pte ${PATH_TO_ARTIFACT}
```

## What is coming?
Expand Down
16 changes: 8 additions & 8 deletions examples/qualcomm/custom_op/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -292,9 +292,9 @@ Registers `torch.ops.my_ops.mul3.default` (multiply by 3) and delegates it via `
```bash
python3 examples/qualcomm/custom_op/custom_ops_1.py \
--build_folder build-android \
-s <device_serial> \
-H <host> \
-m SM8650 \
--device <device_serial> \
--host <host> \
--soc_model SM8650 \
--op_package_dir examples/qualcomm/custom_op/example_op_package_htp/ExampleOpPackage \
--build_op_package
```
Expand All @@ -303,7 +303,7 @@ python3 examples/qualcomm/custom_op/custom_ops_1.py \
```bash
python3 examples/qualcomm/custom_op/custom_ops_1.py \
--build_folder build-x86 \
-m SM8650 \
--soc_model SM8650 \
--op_package_dir examples/qualcomm/custom_op/example_op_package_htp/ExampleOpPackage \
--build_op_package \
--enable_x86_64
Expand All @@ -317,9 +317,9 @@ Registers `torch.ops.my_ops.split_custom.default` (splits a tensor into two halv
```bash
python3 examples/qualcomm/custom_op/custom_ops_2.py \
--build_folder build-android \
-s <device_serial> \
-H <host> \
-m SM8650 \
--device <device_serial> \
--host <host> \
--soc_model SM8650 \
--op_package_dir examples/qualcomm/custom_op/example_op_package_htp_multi_output/SplitCustomOpPackage \
--build_op_package
```
Expand All @@ -328,7 +328,7 @@ python3 examples/qualcomm/custom_op/custom_ops_2.py \
```bash
python3 examples/qualcomm/custom_op/custom_ops_2.py \
--build_folder build-x86 \
-m SM8650 \
--soc_model SM8650 \
--op_package_dir examples/qualcomm/custom_op/example_op_package_htp_multi_output/SplitCustomOpPackage \
--build_op_package \
--enable_x86_64
Expand Down
4 changes: 2 additions & 2 deletions examples/qualcomm/qaihub_scripts/llama/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ Please refer to [Check context binary version](../../README.md#check-context-bin
#### Step4: Run default examples
```bash
# AIHUB_CONTEXT_BINARIES: ${PATH_TO_AIHUB_WORKSPACE}/build/llama_v2_7b_chat_quantized
python examples/qualcomm/qaihub_scripts/llama/llama2/qaihub_llama2_7b.py -b build-android -s ${SERIAL_NUM} -m ${SOC_MODEL} --context_binaries ${AIHUB_CONTEXT_BINARIES} --tokenizer_bin tokenizer.bin --prompt "What is Python?"
python examples/qualcomm/qaihub_scripts/llama/llama2/qaihub_llama2_7b.py --build_folder build-android --device ${SERIAL_NUM} --soc_model ${SOC_MODEL} --context_binaries ${AIHUB_CONTEXT_BINARIES} --tokenizer_bin tokenizer.bin --prompt "What is Python?"
```

## Llama-3-8b-chat-hf
Expand All @@ -53,5 +53,5 @@ Please refer to [Check context binary version](../../README.md#check-context-bin
#### Step4: Run default examples
```bash
# AIHUB_CONTEXT_BINARIES: ${PATH_TO_AIHUB_WORKSPACE}/build/llama_v3_8b_chat_quantized
python examples/qualcomm/qaihub_scripts/llama/llama3/qaihub_llama3_8b.py -b build-android -s ${SERIAL_NUM} -m ${SOC_MODEL} --context_binaries ${AIHUB_CONTEXT_BINARIES} --tokenizer_model tokenizer.model --prompt "What is baseball?"
python examples/qualcomm/qaihub_scripts/llama/llama3/qaihub_llama3_8b.py --build_folder build-android --device ${SERIAL_NUM} --soc_model ${SOC_MODEL} --context_binaries ${AIHUB_CONTEXT_BINARIES} --tokenizer_model tokenizer.model --prompt "What is baseball?"
```
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ Please refer to [Check context binary version](../../README.md#check-context-bin
#### Step5: Run default example
In this example, we execute the script for 20 time steps with the `prompt` 'a photo of an astronaut riding a horse on mars':
```bash
python examples/qualcomm/qaihub_scripts/stable_diffusion/qaihub_stable_diffusion.py -b build-android -m ${SOC_MODEL} --s ${SERIAL_NUM} --text_encoder_bin ${PATH_TO_TEXT_ENCODER_CONTEXT_BINARY} --unet_bin ${PATH_TO_UNET_CONTEXT_BINARY} --vae_bin ${PATH_TO_VAE_CONTEXT_BINARY} --vocab_json ${PATH_TO_VOCAB_JSON_FILE} --num_time_steps 20 --prompt "a photo of an astronaut riding a horse on mars"
python examples/qualcomm/qaihub_scripts/stable_diffusion/qaihub_stable_diffusion.py --build_folder build-android --soc_model ${SOC_MODEL} --device ${SERIAL_NUM} --text_encoder_bin ${PATH_TO_TEXT_ENCODER_CONTEXT_BINARY} --unet_bin ${PATH_TO_UNET_CONTEXT_BINARY} --vae_bin ${PATH_TO_VAE_CONTEXT_BINARY} --vocab_json ${PATH_TO_VOCAB_JSON_FILE} --num_time_steps 20 --prompt "a photo of an astronaut riding a horse on mars"
```
- Please replace `${PATH_TO_TEXT_ENCODER_CONTEXT_BINARY}`, `${PATH_TO_UNET_CONTEXT_BINARY}`, and `${PATH_TO_VAE_CONTEXT_BINARY}` with the actual paths to your AI Hub context binary files.
- Please replace `${PATH_TO_VOCAB_JSON_FILE}` with the actual path to your vocab.json file.
2 changes: 1 addition & 1 deletion examples/qualcomm/util_scripts/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ This section describes how to generate an ET record for a .pte program using the
```bash
# Example usage to generate ET record and inspect execution statistics
PYTHONPATH=.. python -m examples.qualcomm.util_scripts.gen_etrecord \
-b build-android \
--build_folder build-android \
--device $DEVICE_SERIAL \
--soc_model SM8750 \
```
Expand Down
Loading