[v1.0] Make the declared Base test command execute the full validation gate

[codeforester]

Goal

Ensure the repository's declared test entry point can fail on Python regressions.

Background

base_manifest.yaml declares ./tests/validate.sh as the project's test command:

base-cli/base_manifest.yaml

Lines 8 to 12 in 8a93d22

	python:
	manager: uv
	test:
	command: ./tests/validate.sh

. That script checks only repository files, badges, and release-strip text, then prints Repository baseline is present.:

base-cli/tests/validate.sh

Lines 1 to 82 in 8a93d22

	#!/usr/bin/env bash
	required_files=(
	README.md
	VERSION
	CHANGELOG.md
	SECURITY.md
	CONTRIBUTING.md
	.github/pull_request_template.md
	.github/base-project.yml
	.github/workflows/docs.yml
	LICENSE
	mkdocs.yml
	base_manifest.yaml
	.github/workflows/issue-branch-policy.yml
	.github/workflows/project-intake.yml
	.github/workflows/tests.yml
	.github/workflows/package.yml
	.github/workflows/examples.yml
	.github/workflows/compatibility.yml
	.github/workflows/dependency-matrix.yml
	.github/ISSUE_TEMPLATE/support.md
	docs/releasing.md
	docs/index.md
	docs/api-stability.md
	docs/api-reference.md
	docs/dependency-support.md
	docs/user-config-typing.md
	docs/migrations.md
	docs/security-threat-model.md
	docs/security-review.md
	docs/adopter-readiness.md
	docs/adoption-evidence.md
	docs/compatibility-dashboard.md
	MANIFEST.in
	scripts/validate_package_artifact.py
	scripts/validate_installed_package.py
	scripts/validate_docs.py
	scripts/generate_api_reference.py
	scripts/validate_examples.py
	scripts/validate_consumers.py
	scripts/generate_release_metadata.py
	scripts/validate_release_metadata.py
	scripts/record_compatibility_evidence.py
	scripts/generate_compatibility_dashboard.py
	scripts/benchmark_runtime.py
	tests/conftest.py
	compatibility/README.md
	compatibility/consumers/manifest.json
	)
	for file in "${required_files[@]}"; do
	[[ -f "$file" ]] || {
	printf 'Missing required file: %s\n' "$file" >&2
	exit 1
	}
	done
	version="$(head -n 1 VERSION | tr -d '\r')"
	readme_head="$(sed -n '1,18p' README.md | tr -d '\r')"
	if ! printf '%s\n' "$readme_head" | grep -F "[](https://github.com/basefoundry/base-cli/actions/workflows/tests.yml)" >/dev/null; then
	printf 'README.md is missing the main-branch tests health badge.\n' >&2
	exit 1
	fi
	if ! printf '%s\n' "$readme_head" | grep -F "[](https://github.com/basefoundry/base-cli/actions/workflows/compatibility.yml)" >/dev/null; then
	printf 'README.md is missing the reference-consumer health badge.\n' >&2
	exit 1
	fi
	if ! printf '%s\n' "$readme_head" | grep -F "[](https://pypi.org/project/base-cli/)" >/dev/null; then
	printf 'README.md is missing the PyPI release badge.\n' >&2
	exit 1
	fi
	if ! printf '%s\n' "$readme_head" | grep -F "[](https://pypi.org/project/base-cli/)" >/dev/null; then
	printf 'README.md is missing the supported Python versions badge.\n' >&2
	exit 1
	fi
	if ! printf '%s\n' "$readme_head" | grep -F "| \`$version\` | [Apache-2.0](LICENSE) | \`python -m pip install base-cli\` | [v$version](https://github.com/basefoundry/base-cli/releases/tag/v$version) |" >/dev/null; then
	printf 'README.md release strip does not match VERSION (%s), license, install command, and release link.\n' "$version" >&2
	exit 1
	fi
	printf 'Repository baseline is present.\n'

. It runs zero unit, type, lint, security, schema, documentation, example, compatibility, or performance tests.

The hosted workflows are healthy overall, but the repository-level Base contract can false-green after a code regression. The WSL job has the same gap: it runs tests/validate.sh and python3 --version, but never installs the package or executes the portable Python suite:

base-cli/.github/workflows/tests.yml

Lines 131 to 147 in 8a93d22

	wsl:
	name: Validate (WSL)
	runs-on: windows-latest
	timeout-minutes: 15
	steps:
	- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
	- name: Validate repository baseline inside WSL
	shell: pwsh
	run: |
	$distros = (wsl --list --quiet 2>$null | Out-String)
	if ($distros -notmatch "Ubuntu") {
	wsl --install --distribution Ubuntu --no-launch
	}
	$drive = $env:GITHUB_WORKSPACE.Substring(0, 1).ToLowerInvariant()
	$path = $env:GITHUB_WORKSPACE.Substring(2).Replace('\', '/')
	$linuxWorkspace = "/mnt/$drive$path"
	wsl --distribution Ubuntu --user root -- bash -lc "set -eu; cd '$linuxWorkspace'; sed -i 's/\r$//' tests/validate.sh; bash tests/validate.sh; python3 --version"

.

Scope

• Define one authoritative local/full validation entry point.
• Make the Base manifest invoke it.
• Keep a fast baseline-only check under an accurately named command if it remains useful.
• Document cache and optional-extra requirements.

Acceptance Criteria

• A deliberately failing unit test makes the declared test command fail.
• The command covers the agreed v1.0 local gates or delegates to a versioned orchestrator that does.
• The command works from a clean checkout with documented setup and writable caches.
• Hosted CI calls the same underlying gates or verifies parity.
• WSL installs the package and runs the portable lifecycle/persistence suite; only tests with documented POSIX or native-Windows prerequisites may be skipped.
• Fast and full variants are named and documented without ambiguity.

Validation

Run the manifest-declared command locally and in CI; include a fixture proving failure propagation.

Non-Goals

Do not require every OS matrix job to run on a developer laptop.

Project Fields

• Status: Backlog
• Priority: P1
• Area: CI
• Initiative: v1.0 Readiness
• Size: M

Ownership

• Assignee: @codeforester