ENVTESTPATH arch-selection is decided at Makefile parse time, picks amd64 on any cold bin/ regardless of host arch

[kaovilai]

Found while working on #2367 / #2368 (tool-binary caching reliability). Confirmed real, not cosmetic — tracking separately since it's pre-existing and independent of those PRs' fix, and has no CI/release-branch consequence (invisible on amd64 CI, only affects arm64 local dev).

The bug

ENVTEST := $(shell pwd)/bin/setup-envtest
ENVTESTPATH = $(shell $(ENVTEST) use $(ENVTEST_K8S_VERSION) -p path)
ifeq ($(shell $(ENVTEST) list | grep $(ENVTEST_K8S_VERSION)),)
	ENVTESTPATH = $(shell $(ENVTEST) --arch=amd64 use $(ENVTEST_K8S_VERSION) -p path)
endif

On any cold bin/ (fresh checkout, setup-envtest not yet installed), $(ENVTEST) list fails silently (binary doesn't exist), so the grep finds nothing, ifeq (...,) is true, and ENVTESTPATH gets redefined to force --arch=amd64 — regardless of the actual host architecture.

The ifeq directive itself runs at Makefile-parse time, which happens before any target's prerequisites (like envtest's own install/repair logic) have a chance to run. So even though setup-envtest gets correctly installed for the host's native arch by the time the test target's recipe actually executes, ENVTESTPATH's chosen text was already locked in as the amd64-forced variant earlier in that same make invocation, and isn't reconsidered.

Verified repro

Minimal Makefile, binary missing (cold bin/), run on an arm64 (Apple Silicon) host:

ENVTEST_K8S_VERSION = 1.29
ENVTEST := $(shell pwd)/bin/setup-envtest
ENVTESTPATH = $(shell $(ENVTEST) use $(ENVTEST_K8S_VERSION) -p path)
ifeq ($(shell $(ENVTEST) list | grep $(ENVTEST_K8S_VERSION)),)
	ENVTESTPATH = $(shell $(ENVTEST) --arch=amd64 use $(ENVTEST_K8S_VERSION) -p path)
endif

.PHONY: show-decision
show-decision:
	$(info ENVTESTPATH = $(value ENVTESTPATH))

$ rm -rf bin
$ gmake show-decision
ENVTESTPATH = $(shell $(ENVTEST) --arch=amd64 use $(ENVTEST_K8S_VERSION) -p path)

Confirms the amd64-forced form is chosen, on an arm64 host, purely because bin/ was cold at parse time.

Impact

• Invisible on amd64 CI (the forced arch happens to be correct there anyway) — no CI or release-branch risk.
• On an arm64 host (e.g. local dev on Apple Silicon) with a cold bin/, make test ends up using amd64 kubebuilder test assets (etcd, kube-apiserver) instead of native arm64 ones — the same class of arch-drift bug Make controller-gen/kustomize/golangci-lint/envtest tool-binary caching reliable #2367/[oadp-1.4] Make controller-gen/kustomize/golangci-lint/envtest tool-binary caching reliable #2368 fix for the tool binaries themselves, surviving here because this mechanism is separate (an ifeq-selected $(shell ...) variable, not a cached tool binary).

Suggested fix direction

Don't decide arch at parse time at all:

• Let setup-envtest use resolve the native arch by default, and only force --arch=amd64 explicitly when actually needed (e.g. via an opt-in variable), rather than probing $(ENVTEST) list at parse time as a proxy for "is this arch supported."
• Or, move the decision into a recipe that runs after setup-envtest is guaranteed to be installed, rather than in an ifeq directive evaluated unconditionally for every make invocation.

Note

Responses generated with Claude