Skip to content

DO NOT MERGE: experimental Windows clang-cl + TAILCALL VM lane (windows-x86_64-clang) - #45

Merged
luthermonson merged 8 commits into
mainfrom
windows-clang-tailcall-lane
Aug 20, 2026
Merged

DO NOT MERGE: experimental Windows clang-cl + TAILCALL VM lane (windows-x86_64-clang)#45
luthermonson merged 8 commits into
mainfrom
windows-clang-tailcall-lane

Conversation

@luthermonson

Copy link
Copy Markdown
Contributor

DO NOT MERGE — experimental Windows clang-cl + TAILCALL VM lane

Adds an additive-only build lane, platform=windows-x86_64-clang, producing
php-sdk-<ver>-windows-x86_64-clang.tar.gz: the same artifact layout and
extension set as the MSVC lane, but with PHP itself compiled by a pinned
LLVM release (22.1.8)
so PHP 8.5's new ZEND_VM_KIND_TAILCALL interpreter
is selected instead of the slow ZEND_VM_KIND_CALL that every MSVC build
gets. The existing windows-x86_64 lane is untouched.

Why

Zend/zend_vm_opcodes.h picks HYBRID only under __GNUC__ && HAVE_GCC_GLOBAL_REGS — never true on win32 — so Windows has always run the
CALL VM (upstream measured 1.77x slower than HYBRID). PHP 8.5 added TAILCALL
(musttail + preserve_none, clang-only), which recovers HYBRID-class
dispatch. HAVE_MUSTTAIL is header-detected; HAVE_PRESERVE_NONE only comes
from an autoconf run-test that never runs on Windows, so no Windows PHP has
ever selected TAILCALL. This lane hand-defines it.

Measured (PHP 8.5.7 CLI, Ryzen 9 5950X, ZTS, best-of-5 hrtime)

build VM kind int loop 1M mixed ref loop
MSVC 19.44 CALL (verified) 14.2 ms 4.42 ms
LLVM 22.1.8 clang-cl CALL (verified) 13.1 ms 4.31 ms
VS-bundled clang-cl 19.1.5 TAILCALL (verified) 22.6 ms 7.07 ms
LLVM 22.1.8 clang-cl TAILCALL (verified) 8.4 ms 2.72 ms

End to end through ePHPm (ephpm php, same loops, full extension set, both
binaries from the same ephpm commit, only PHP_SDK_PATH differing):

ePHPm build mixed ref loop int loop 1M
MSVC SDK (CALL) 4.43 ms 14.1 ms
clang SDK (TAILCALL) 2.73 ms (1.62x) 8.2 ms (1.72x)

Three findings that shape the design:

  1. The win is the TAILCALL VM, not clang per se: clang 22's CALL build is
    only at parity with MSVC (13.1 vs 14.2 ms); switching the VM kind delivers
    the 1.55–1.7x.
  2. The VS-bundled clang-cl 19 is a trap — its Zend codegen is ~45–90%
    slower than MSVC, enough to swamp the VM-kind win (verified CALL-vs-CALL,
    ZTS and NTS). Hence the pinned LLVM download with a hard version gate, not
    the VS Clang component.
  3. With LLVM 22, the mixed reference loop lands within ~12% of the same loop
    on ePHPm Linux (HYBRID) — the Windows interpreter gap is essentially closed.

How the lane works

  • Clones php-src at the release tag, patches win32/build/config.w32
    (toolset default vsclang; AC_DEFINE("HAVE_PRESERVE_NONE") in the
    CLANG_TOOLSET block, gated clang ≥ 19 + x64), and feeds it to stock
    spc via --custom-local "php-src:…" — spc's Windows target hardcodes its
    configure args, so no spc fork change is required. The patch step
    hard-fails if the anchor text drifted.
  • --dl-custom-local is repeated on spc build (the build phase runs its
    own downloader pass and otherwise silently replaces the patched tree with
    the stock tarball — found the hard way).
  • Deps stay MSVC-built; clang-cl is MSVC-ABI-compatible. Verified end to end:
    the clang php8embed.lib links into ePHPm's unchanged
    x86_64-pc-windows-msvc build (zero ePHPm source changes).
  • Consumer-header hygiene (both found building ePHPm, both fixed in the
    Package step with hard guards): the staged php_config.h/config.w32.h
    must NOT export HAVE_PRESERVE_NONE (rust-bindgen panics on libclang
    calling convention 20; MSVC consumers can't represent it either) nor the
    PHP_HAVE_BUILTIN_* defines (Zend's static-inline fast paths then emit
    __builtin_expect calls MSVC consumers cannot link). Stripping them gives
    every consumer the same portable CALL-view the MSVC-lane headers give,
    while the engine runs TAILCALL internally (zend_vm_kind() still says 5).
  • VM-kind gate: the workflow extracts zend_vm_execute.obj from the
    built php8embed.lib and requires zend_vm_kind to compile to
    mov eax,5 (TAILCALL). The lane cannot silently regress to CALL.
  • Never in platform=all, never in platforms_required, never gates a
    release (mirrors the -nts lanes' isolation).

Known blocker for running this in CI (pre-existing, affects the MSVC lane too)

spc v3.0.0-pgo-rc18's Windows passthru() gives children socket-type
stdout/stderr descriptors. msys git (any version — tested 2.49 and 2.55)
cannot dup socket handles as std fds: every git clone spc runs dies with
dup() in/out/err failed / cannot duplicate fd. The rc18 Windows leg has
never run in CI (fleet was down when rc18 landed) — when it does, the
existing MSVC lane will hit this too
on the git-sourced artifacts
(php-micro, gettext-win, libiconv-win, libaom, …). Local validation
worked around it with a git shim that re-execs git with file-backed std
handles; the real fix belongs in the spc fork (pipe descriptors for git, or
shipping such a shim). Filed as a first-class finding, not fixed here.

Local validation performed

  • go/no-go CLI experiment: MSVC vs clang19 vs clang22, CALL vs TAILCALL, ZTS
    and NTS — VM kinds proven via a zend_vm_kind() probe in each binary.
  • Full lane run locally with stock spc.exe rc18 + the exact workflow patch
    logic (plus the git-shim + short-path workarounds CI won't need).
  • ePHPm built against the resulting clang SDK with zero source changes;
    CLI + serve-mode benchmarks and correctness checks in the ePHPm repo
    (numbers in the lane's job comment and README).

DO NOT MERGE until: (a) the spc git/socket issue is fixed in the fork so the
lane can actually run in CI, and (b) the org decides the artifact should soak
as an opt-in lane.

luthermonson added a commit that referenced this pull request Aug 20, 2026
)

rc19 = rc18 + luthermonson/static-php-cli#9: spc's proc_open handed child
processes socket-type std handles on Windows, so msys git/patch died with
'dup() in/out/err failed' on every clone - the Windows leg has never run
green under rc18. rc19 gives children pipe/NUL handles.

Also lifts extensions.yml and mirror-deps.yml off rc17 so all three
workflows pin one spc release. Unblocks the MSVC Windows lane's CI and the
experimental clang/TAILCALL lane (#45).
…64-clang)

Every MSVC-compiled PHP gets the slow ZEND_VM_KIND_CALL interpreter
(zend_vm_opcodes.h selects HYBRID only under __GNUC__ +
HAVE_GCC_GLOBAL_REGS, never true on win32). PHP 8.5 added the TAILCALL
VM (musttail + preserve_none, clang-only); HAVE_PRESERVE_NONE is only
set by an autoconf run-test that never runs on Windows, so no Windows
build has ever selected it.

This adds an additive, explicitly-dispatched build lane
(platform=windows-x86_64-clang) that:

- clones php-src at the release tag and patches win32/build/config.w32
  (default toolset vs -> clang; AC_DEFINE HAVE_PRESERVE_NONE for
  clang >= 19 on x64), then feeds it to stock spc via --custom-local
  (spc's Windows target hardcodes its configure.bat args, so no spc
  fork change is needed);
- builds PHP itself with a PINNED LLVM release (22.1.8) while every
  dependency stays MSVC-built (clang-cl is MSVC-ABI-compatible);
- hard-fails unless the VM kind disassembled out of php8embed.lib is
  TAILCALL (5), so the lane can never silently regress to CALL;
- packages php-sdk-<ver>-windows-x86_64-clang.tar.gz with the exact
  layout of the MSVC lane.

Measured (PHP 8.5.7 CLI, Ryzen 9 5950X, ZTS, best-of-5 hrtime):
LLVM 22.1.8 TAILCALL int-loop 8.4ms vs MSVC CALL 14.2ms (1.7x);
mixed reference loop 2.72ms vs 4.42ms. The VS-BUNDLED clang-cl 19.1.5
is a trap: its Zend codegen is ~45-90% SLOWER than MSVC, which is why
the toolchain is a pinned LLVM download, not the VS Clang component.

Never part of platform=all, never gates a release, documented as
experimental in README and versions.json (platforms_experimental).
…nst a full local run

Fixes found by running the lane end to end locally with stock spc.exe
v3.0.0-pgo-rc18 (PHP 8.5.7, LLVM 22.1.8) and building ePHPm against the
resulting SDK:

- configure: force PHP_TOOLSET = clang in the config.w32 body instead of
  flipping the declaration default. spc passes --disable-all, and
  conf_process_args resets every unseen configure arg to no (-> vs),
  so a default flip silently produces an MSVC/CALL artifact. Also keep the
  injected comment free of the literal ARG_WITH/ARG_ENABLE strings:
  buildconf's preamble extractor greps for them and would copy comment
  text into configure.js as broken JS.
- spc build: repeat --dl-custom-local. The build phase runs its own
  downloader pass and otherwise re-resolves php-src, silently replacing
  the patched tree with the stock tarball.
- VM-kind gate: the probe member is Zend\zend_execute.obj
  (zend_vm_execute.h is #included by zend_execute.c; there is no
  zend_vm_execute.obj).
- Package: strip HAVE_PRESERVE_NONE and the PHP_HAVE_BUILTIN_* defines
  from BOTH staged config headers (php_config.h and config.w32.h -
  zend_config.h includes the latter). With them present, rust-bindgen
  panics on libclang calling convention 20 (preserve_none) and MSVC
  consumers emit unlinkable __builtin_expect calls. Stripped, consumers
  get the same portable CALL-view the MSVC-lane headers give while the
  engine runs TAILCALL internally - verified: ePHPm links and runs the
  clang SDK with zero source changes, 1.6-1.7x faster on CPU-bound PHP
  than the MSVC SDK build.
@luthermonson
luthermonson force-pushed the windows-clang-tailcall-lane branch from 2a010c7 to 30aaf4a Compare August 20, 2026 06:31
…tar lacks xz

The dispatch's 'Install pinned LLVM' step died with 'xz: Cannot exec': a bare
tar resolved to Git-for-Windows' GNU tar on the fleet runner, which execs an
external xz that Git's environment does not ship. Windows' bundled bsdtar
handles .tar.xz natively; pin its full path.
…hemerd's job guillotine

Root cause of attempts 3+4: both jobs died at job-age 2:09:14 and 2:09:13
(one second apart) - ephemerd's job_timeout (default 2h) destroys the VM
mid-step, so GitHub shows 'runner lost communication' with no logs. Nothing
in the lane or in spc was at fault: rc19's download is 78s cold / 2s warm,
89 MB peak RSS, sequential (validated locally). The old 50-attempt retry
loop with curl's 1-hour stall tolerance could legally run >1.5h on a
flaky-egress VM, pushing the job past the guillotine.

- Download PHP sources: timeout-minutes 15, retries 50 -> 3
- Install pinned LLVM: timeout-minutes 10
- Job: timeout-minutes 115 so GitHub, not ephemerd, ends over-long runs
  with a visible reason. Fleet-side ask (raise job_timeout to 4h) tracked
  in the ephemerd report.
… clone of winlibs/gettext hangs on fleet egress

Attempts 3-5 all stalled at artifact [22/30] gettext-win: spc does a FULL
git clone of winlibs/gettext, and a long single-stream transfer is exactly
what the fleet VM's flaky egress stalls (21 GitHub tarballs before it
downloaded in 3-5s each; the runner's own git clone of php-src succeeded
every attempt). Shallow --depth 1 --branch clones of gettext-win and
libiconv-win (the only git-type sources in this set) in a capped retry
step, handed to spc via --custom-local / --dl-custom-local exactly like
the patched php-src.
…ery step green and was cancelled mid-compile at 115m

The build itself needs more than the ~80m that remained after ~35m of
toolchain installs on the fleet VM. 200m is only reachable once ephemerd's
job_timeout (currently 2h) is raised fleet-side; until then this job cannot
finish regardless of workflow settings.
… under the 2h fleet timeout

Attempt 6 was cancelled by our OWN 115m lane cap at 115.7m job age (~35m
setup + 80.5m build still running), ~4m short of ephemerd's 120m cap - it
was never actually allowed to test the fleet limit. 119m sits just under
120m so an overrun fails visibly (with logs) instead of the silent
runner-death; if the build finishes under 119, no fleet change is needed.
@luthermonson
luthermonson marked this pull request as ready for review August 20, 2026 19:59
@luthermonson
luthermonson merged commit 10352d2 into main Aug 20, 2026
4 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant