feat: add x_attention_v2 operator use pypto as compile backend. - #2286
Open
Yangxu-China wants to merge 1 commit into
Open
feat: add x_attention_v2 operator use pypto as compile backend.#2286Yangxu-China wants to merge 1 commit into
Yangxu-China wants to merge 1 commit into
Conversation
Yangxu-China
requested review from
DongheJin,
DragonFive,
JimHsiung,
Kang-Meng,
liutongxuan,
xiao-yu-chen,
yingxudeng and
zhang-minchao
as code owners
August 29, 2026 02:55
Yangxu-China
force-pushed
the
x_attention_v2
branch
2 times, most recently
from
August 29, 2026 03:26
54927f2 to
5eb86b0
Compare
zhang-minchao
approved these changes
Aug 30, 2026
zhang-minchao
self-requested a review
September 1, 2026 05:22
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add the
x_attention_v2PyPTO Pro kernel to the xllm NPU kernel package, along with its JIT test suite.This PR introduces a new DSL sub-package
xllm/python/kernels_npu/pypto/(parallel to the existingtilelang/andtriton/sub-packages) containing thex_attention_v2operator — a beam-decode flash attention kernel for Ascend A5 (Ascend950), implementing shared KV + unshared KV dual-path merged attention with online softmax combine.The kernel is written in the PyPTO Pro DSL (
pypto_pro.language) and launched via the@pl.jitbracket-launch path. A Python host wrapper (x_attention_v2) computes tiling (core split) and launches the@pl.jitkernel (x_attention_v2_kernel).Key additions:
xllm/python/kernels_npu/pypto/__init__.py— pypto DSL sub-package entry (license + docstring only, not registered inkernels_npu/__init__.py, consistent withtilelang/andtriton/)xllm/python/kernels_npu/pypto/x_attention_v2.py—@pl.jitkernel + Python host wrapper (1892 lines, self-contained)tests/python/test_x_attention_v2_jit.py— 144 parametrized test cases (3 GQA configs × 2 batch sizes × 3 beam sizes × 4 KV lengths × 2 unshared lengths) with self-contained golden cache mechanism.gitignore— entries fortests/python/golden_cache/andtests/python/msprof*/Verification:
Related Issues
N/A — new operator integration, no tracking issue yet.
Change Type
Pull Request Checklist
Thank you for contributing to xLLM. Before requesting review, please make sure the following items are complete.
PR Title and Commit Messages
<type>: <subject>.Pre-commit Checks
pre-commitby runningpip install pre-commitor an equivalent command.pre-commit install.pre-commit run --all-filesand fixed any reported issues.Self Review
.agents/skills/code-review/references/custom-code-style.md, especially code written or assisted by AI.mainbranch.Build and Test Coverage
python setup.py build testhas passed on a CUDA machine.python setup.py build testhas passed on an NPU machine.python setup.py build testhas passed on an MLU machine.Reviewer Notes
pypto_prois a CANN-bundled package (installed at/home/developer/Ascend/cann-9.2.0/python/site-packages/pypto_pro/), not declared as a pip dependency — same model astilelang(conditionally installed via CANN setup). The kernel file imports it lazily inside the host wrapper, so the module is importable withoutpypto_proinstalled.golden_cache/directory (144.ptfiles) is gitignored — the test generates cache on first run if absent (~186s for all 144 CPU reference computations), subsequent runs load from disk (~14s).msprof*/directories are also gitignored — profiling results are not committed.x_attention_v2.py) is self-contained with zero external path dependencies — all imports are standard library,pypto_pro, ortorch..gitignorechange adds two entries for test artifacts.