Skip to content

pyhmmer: add build-pyhmmer.yml for riscv64 wheels - #1497

Open
luhenry wants to merge 1 commit into
mainfrom
pyhmmer
Open

pyhmmer: add build-pyhmmer.yml for riscv64 wheels#1497
luhenry wants to merge 1 commit into
mainfrom
pyhmmer

Conversation

@luhenry

@luhenry luhenry commented Sep 8, 2026

Copy link
Copy Markdown
Member

Compiles vendored HMMER3/Easel and Cython bindings for profile-HMM search. Upstream publishes no riscv64 wheel.

Mirrors upstream's package.yml (wheel-linux-x86_64/wheel-linux-aarch64 jobs).

Status: parked. Builds and links cleanly, but CI found a real riscv64-only correctness bug: p7_MSVFilter() returns wrong scores (hit counts of 0 where real hits are expected, or under-counts like 479 != 482). Root cause not isolated despite a thorough investigation (below); parking rather than merging a wheel that produces silently wrong search results.

Differs from upstream

  • No native SSE/NEON/VMX on riscv64 - SSE routed through SIMDe instead, via a build-time header shim.
  • esl_cpu.c's x86-only CPUID assembly is skipped on that path.

License: Wheel bundles vendor/hmmer (BSD-3-Clause) and vendor/easel (BSD-2-Clause); pyhmmer's own license-files omits them, so a patch adds both.

Investigation (real riscv64 hardware and real riscv64/GCC 14.3.1 via QEMU, not just the QEMU rehearsal used to validate the build):

  • Confirmed correct via HMMER's own unit-test drivers (each compares the SIMD filter against HMMER's independent Generic/portable reference): ViterbiFilter, the Forward filter, posterior decoding, optimal-accuracy alignment, null2 bias correction, and the vectorized expf/logf approximations in esl_sse.c (relative error ~6e-9 avg, ~1.2e-7 max - at float precision, matching arm64).
  • Confirmed broken, real riscv64/GCC 14.3.1 only (not arm64, same source, both Clang and GCC 15): p7_MSVFilter fails its own unit test with scores differ (-21.25, -10.86) - a ~10-nat divergence, not a rounding artifact.
  • Reproduces identically whether p7_SSVFilter's internal fast-path shortcut is active or force-disabled - rules out ssvfilter.c's control flow specifically.
  • Reproduces identically at -O1, -O2, -O3 - rules out an optimization-level-triggered miscompilation.
  • Every individual SIMD primitive the function uses checks out correct in isolation on real riscv64 (hundreds of random trials each): the saturating DP step (_mm_max_epu8/_mm_adds_epu8/_mm_subs_epu8), the byte-lane shift (_mm_slli_si128), the horizontal-max reduction chain, and the union { __m128i; uint8_t[16]; } type-pun pattern used to build om->rbv.
  • The Generic reference implementation (p7_GViterbi, pure portable C, no SIMD/SIMDe) passes its own internal self-consistency check (score vs. independently-reconstructed optimal-trace score) on real riscv64, making it unlikely to be the source of the divergence.
  • Conclusion: something in how the real p7_MSVFilter loop composes these (individually-correct) primitives across many iterations diverges specifically under riscv64/GCC - not isolated to an exact line. Likely needs a real riscv64 GCC debugger session or upstream HMMER/SIMDe maintainer input. Written up as gotcha 332 in the porting skill for the next attempt.

Patches

  • 0001-CMake-...patch - Inappropriate. Adds the SIMDe fallback since riscv64 has no native SIMD backend; riscv64-only.
  • 0002-easel-...patch - To upstream. Fixes esl_cpu.c's CPUID assembly breaking under the SIMDe path; riscv64-only.
  • 0003-package-the-licences-...patch - To upstream. Packages the vendored licences; same gap on every platform.

HMMER/Easel's fast profile-HMM filters only exist as hand-written
SSE/NEON/VMX kernels, so CMakeLists.txt hard-fails on riscv64; route
the SSE path through SIMDe instead, with a build-time header shim so
the vendored sources (and pyhmmer's own line-number-based patches
against them) stay untouched. Also fixes esl_cpu.c's x86-only CPUID
assembly and packages the vendored HMMER/Easel licences.
luhenry added a commit that referenced this pull request Sep 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant