Skip to content

The build and the fetch tooling reach for python3 by name too - #52

Merged
marcobambini merged 1 commit into
sqliteai:mainfrom
chongjiazhen:fix/windows-python3-alias-tooling
Aug 28, 2026
Merged

The build and the fetch tooling reach for python3 by name too#52
marcobambini merged 1 commit into
sqliteai:mainfrom
chongjiazhen:fix/windows-python3-alias-tooling

Conversation

@chongjiazhen

@chongjiazhen chongjiazhen commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Follow-up to #51, offered there and kept separate so each is reviewable on its
own. Same finding one layer out: on Windows python3 on PATH is usually the
Microsoft Store App Execution Alias, a zero-byte reparse point that exists,
exits 49, and prints an advert for the Store instead of running anything.

tests/run.sh can shim PATH for its own call sites. That does not reach a
recipe make runs itself, nor either tools script when run directly, so 11
call sites are still exposed:

file sites
Makefile 3: serve-check, fuzz, fuzz-asan
tools/fetch_weights.sh 4
tools/pipeline.sh 4

pipeline.sh is the one that misdiagnoses rather than merely failing. Its first
two probes end 2>/dev/null || echo 0, so an interpreter that is not one yields
0 and the script dies with download (no index at $SRC), naming a missing
index that is present and perfectly readable.

The change

The Makefile resolves $(PY) by running a candidate; the two scripts resolve
$PY the same way, and pipeline.sh exports it so the fetch_weights.sh it
drives inherits the answer instead of resolving once per stage.

Nothing skips here, deliberately. If no candidate answers, PY stays python3
and the recipe or script fails loudly at first use. That is the right behaviour
for a build target and for a 1.4 TB download: unlike the suite there is nothing
to skip, and a run that cannot read its own index must stop rather than carry
on.

fetch_weights.sh already knew it runs on Windows, at the tr -d '\r' comment
citing #36 gap 2. This is the same environment, one assumption earlier.

$(PY) is recursive rather than := deliberately. Immediate assignment runs
the probe on every make invocation, make clean and a no-op build included,
and where python3 is the alias the first spawn is the alias itself: measured
196 ms added per invocation, which nearly triples a no-op make. Lazy, it
runs only when a Python recipe expands it. The fallback sits inside the shell
for the same reason, since an ifeq on $(PY) would force the expansion at
parse time and undo it.

Verification

Windows 10, MinGW-w64 GCC 15.2, Python 3.13 working under its own name.
make -p reports PY := python.

make serve-check          -> OK (skipped=3)                              exit 0
make fuzz FUZZ_RUNS=20    -> 20 cases: 12 rejected, 8 still loaded,
                             0 crashed, 0 hung / FUZZ OK                 exit 0
no-op `make` x3           -> 0.664 s  (main: 0.678 s; with `:=` 1.277 s)
all three names exit 49   -> serve-check dies Error 49, loudly, no skip

The two tools scripts are not run end to end here, because that needs the
full source weights. They carry bash -n, review, and a three-state check of
the added resolver:

state result
Store alias first on PATH PY=python, runs 3.13.5
all three names shimmed to exit 49 PY=python3, fails loudly at first use (49)
PY already set in the environment respected, no re-resolution

Happy to fold this into #51 instead, or to change the shape: a single shared
resolver sourced by both scripts would also work, I kept them self-contained to
match how they read today.


Assisted by AI.

Follow-up to the suite fix in sqliteai#51, same finding one layer out: on Windows
`python3` on PATH is usually the Microsoft Store App Execution Alias, a
zero-byte reparse point that exists, exits 49, and prints an advert instead
of running anything. run.sh can shim PATH for its own call sites, but that
does not reach a recipe make runs itself, nor either tools script when run
directly. Eleven call sites: three recipes, four in fetch_weights.sh, four
in pipeline.sh.

Makefile: serve-check, fuzz and fuzz-asan get $(PY). fetch_weights.sh and
pipeline.sh get the same answer as $PY; pipeline.sh exports it so the
fetch_weights.sh it drives inherits rather than resolving per stage.

$(PY) is recursive rather than `:=` on purpose. Immediate assignment runs
the probe on every make invocation, `make clean` and a no-op build
included, and where python3 is the alias the first spawn is the alias
itself: measured 196 ms per invocation on this box, which nearly triples a
no-op make. Lazy, the probe runs only when a Python recipe expands it, and
a no-op make is back to 0.664 s against 0.678 s on main. The fallback sits
inside the shell for the same reason, since an ifeq on $(PY) would force
the expansion at parse time.

pipeline.sh is the one that misdiagnoses rather than merely failing: its
first two probes end `2>/dev/null || echo 0`, so an interpreter that is not
one yields 0 and the script dies "download (no index at $SRC)", naming a
missing index that is present and readable.

Nothing skips here. If no candidate answers, PY stays python3 and the
recipe or script fails loudly at first use, which is right for a build
target and for a 1.4 TB download: unlike the suite there is nothing to
skip, and a run that cannot read its own index must stop.

Verified on Windows 10, MinGW-w64 GCC 15.2. PY resolves to `python`;
`make serve-check` reports OK (skipped=3) and `make fuzz FUZZ_RUNS=20`
reports 20 cases, 0 crashed, 0 hung, both exit 0; with all three names
shimmed to exit 49, serve-check fails loudly with Error 49 rather than
skipping. The two tools scripts are not run end to end here, since that
needs the full source weights: they carry bash -n plus a three-state check
of the resolver.

Assisted by AI.
@chongjiazhen
chongjiazhen force-pushed the fix/windows-python3-alias-tooling branch from a866391 to 26ddf7c Compare August 26, 2026 04:45
@marcobambini
marcobambini merged commit 6d0b60b into sqliteai:main Aug 28, 2026
9 checks passed
marcobambini added a commit that referenced this pull request Aug 28, 2026
#51 fixed the case where python3 is present and is not an interpreter. It
did not finish the case where nothing answers, and the seven checks left
behind did the thing #51 was written to stop: they reported the engine,
the budget, the converter and the release's own preprocessing config
broken because no interpreter ran. Closes #54.

    PATH with python3, python and py all exiting 49, tests/run.sh /nonexistent

                        passed   failed   skipped
    before #51             7       14       14
    after #51 + #52        7        7       22
    now                    7        0       29

77 rather than a new number. tests/check_budget.sh already means
"could not measure, skip loudly" by it -- introduced for the Windows box
with no peak-RSS interface, #36 -- and both its call sites in run.sh
already act on it. One convention across the file beats a second one that
means the same thing.

Six places, eleven checks. Four helpers gained an early return 77 --
default_budget, info_rule, params_rule, vision_norm -- and each has two
call sites, the generic one and the K3 one; only the K3 twin failed in the
repro because the other skipped for want of a container. vision_norm
already had a three-way return, its rc=2 being "no vision tower", so a
missing interpreter would have been reported as a missing tower: it gets
its own arm rather than sharing that one.

check_budget.sh resolves the interpreter itself rather than trusting the
PATH shim. run.sh shims before it gets here, so it normally inherits an
answer, but the script is meant to be runnable standalone and a probe is
three lines. With none found it prints BUDGET UNMEASURABLE and exits 77,
and neither call site needed a change.

The guard at the convert.py GLM config check was still on the pattern #51
replaced. Not missed: it arrived with the GLM work in 0.7.0, three days
after #51 was opened, on that branch's blind side. XTML against the
release's own encoder was unguarded outright.

Verified on all three shapes rather than the one that motivated it: a
normal machine 61/0/9, the Store-alias decoy 61/0/9, nothing answering
7/0/29 with every skip naming the interpreter. And on real weights, where
these eight call sites actually fire and the synthetic run never reaches
them: K3 64 passed, 0 failed, 6 skipped, all seven exercised and green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants