Stop building a second QEMU and mirroring somebody else's kernel #108

Workflow file for this run

	name: CI

	on:
	push:
	pull_request:

	# One run per branch, not one per event. A push to a branch with an open pull request fires
	# both triggers; keying on `head_ref \|\| ref_name` gives both the same string, while
	# `github.ref` — the obvious choice — does not (`refs/pull/N/merge` vs
	# `refs/heads/my-branch`, two groups, nothing de-duplicated). `cancel-in-progress` then
	# makes a new push supersede its predecessor, on main as well.
	concurrency:
	group: ci-${{ github.head_ref \|\| github.ref_name }}
	cancel-in-progress: true

	# Least privilege by default: the gate reads the tree and nothing else. The jobs that
	# touch GitHub Packages raise it to `packages: read` themselves, at the job, so the
	# reason is next to the need.
	permissions:
	contents: read

	# ---------------------------------------------------------------------------------
	# Two jobs. It was five, because `task ci:full` could not run on one hosted runner: the
	# guest lanes needed the pinned QEMU, only executable inside the published runtime image
	# (ADR-0025), and a container job gets no Docker daemon for the TestContainers lanes.
	# Nothing in the merge gate needs QEMU now. The `gate` job below stays regardless — its
	# reason never depended on the split.
	# ---------------------------------------------------------------------------------

	jobs:
	# The merge gate, run as the one target a developer runs. It is `ci:full` and not a
	# list of steps for the reason the Taskfile exists: a second definition of what the gate
	# is, maintained in a workflow file, goes out of step with the first exactly when a lane
	# is added.
	ci:
	runs-on: ubuntu-latest
	# The gate starts a Volume Agent, and the Agent creates and inspects every qcow2
	# chain by running the pinned qemu-img — v6 §7 forbids a parser of our own — so it
	# refuses to start without one. `task machine` fetches the pinned spin-machine release,
	# which is minutes of download against the tens of minutes compiling QEMU here would
	# cost, and this repository no longer has a QEMU to compile.
	permissions:
	contents: read
	packages: read
	# A ceiling, not a budget — these workflows have never executed: enough for the race
	# suite, the coverage run and the Docker lanes (e2e allows itself 15m) on a two-core
	# runner, and far below the six-hour default a wedged TestContainer would hold.
	timeout-minutes: 60
	steps:
	- name: Checkout
	uses: actions/checkout@v6
	with:
	# The whole history, because `task lint` asks buf what this branch changed
	# about the wire contract and needs main to compare against. A shallow clone
	# makes that check error rather than pass, which is the right failure and
	# still a failure.
	fetch-depth: 0

	- name: Set up Go
	uses: actions/setup-go@v6
	with:
	go-version-file: go.mod
	cache: true

	- name: Set up Task
	uses: go-task/setup-task@v2

	# Every other tool (sqlc, pgschema, golangci-lint, buf and the proto plugins) is
	# pinned in the Taskfile and installed into ./.tools/bin by `task tools` — the same
	# command a developer runs locally, so CI and laptops cannot drift.
	- name: Cache pinned tools
	uses: actions/cache@v4
	with:
	path: .tools
	key: tools-${{ runner.os }}-${{ hashFiles('Taskfile.yml') }}

	# The gate's own targets declare these as `deps`, so this step is not what makes
	# them present; it is what makes a toolchain download fail under its own name and
	# populates the cache above.
	- name: Install pinned tools
	run: task tools

	# A read token for the published QEMU image. The pull below is the only thing in
	# this job that touches GitHub Packages, which is why the permission is raised at
	# the job rather than at the workflow.
	- name: Log in to GitHub Packages
	uses: docker/login-action@v3
	with:
	registry: ghcr.io
	username: ${{ github.actor }}
	password: ${{ secrets.GITHUB_TOKEN }}

	# Named as its own step so a missing or unpublished image fails here, under a name
	# that says what is wrong, rather than sixteen minutes into the gate as a lane that
	# cannot start an Agent.
	- name: The pinned machine (QEMU, qemu-img, the kernel)
	run: task machine

	- name: The merge gate (task ci:full)
	run: task ci:full

	# The guest lane: the four demos, each booting a real Linux kernel against a qcow2 this
	# system prepared, with the real binaries as processes.
	#
	# It is a job of its own and not part of `ci:full` because it needs two artefacts this
	# repository does not build — the pinned QEMU and the pinned kernel — and a lane that
	# silently skips when they are missing is the defect this whole file is shaped against.
	# Here their absence is a red job with a name that says which one.
	#
	# No KVM is asked for, and this is the lane the second QEMU binary exists for: with no
	# /dev/kvm, hack/demo-lib.sh runs qemu-system-x86_64-tcg and emulates — slower, and the
	# same code path through everything this lane is actually about. The production binary
	# has no TCG compiled in and would refuse here, which is the point of building two.
	# Measured under TCG on a developer machine: stage1 in 5s, all four well under a minute;
	# the timeout below is a ceiling for a two-core runner, not a budget.
	guest:
	runs-on: ubuntu-latest
	permissions:
	contents: read
	packages: read
	timeout-minutes: 30
	steps:
	- name: Checkout
	uses: actions/checkout@v6

	- name: Set up Go
	uses: actions/setup-go@v6
	with:
	go-version-file: go.mod
	cache: true

	- name: Set up Task
	uses: go-task/setup-task@v2

	- name: Cache pinned tools
	uses: actions/cache@v4
	with:
	path: .tools
	key: tools-${{ runner.os }}-${{ hashFiles('Taskfile.yml') }}

	- name: Install pinned tools
	run: task tools

	- name: Log in to GitHub Packages
	uses: docker/login-action@v3
	with:
	registry: ghcr.io
	username: ${{ github.actor }}
	password: ${{ secrets.GITHUB_TOKEN }}

	# Both QEMU binaries, qemu-img and the kernel, in one step — they are one artefact.
	#
	# This used to be two steps and a paragraph about loaders: the binaries were
	# dynamically linked against their build image, so a bare runner could not run them
	# and a wrapper had to carry the loader and the library closure beside them. They are
	# statically linked now, so an extracted tree runs wherever the kernel does and all
	# of that is gone.
	- name: The pinned machine (QEMU, qemu-img, the kernel)
	run: task machine

	# Asserts the lane's inputs before anything boots: the kernel is the one that was
	# pinned, is a PVH ELF, carries the four config options a boot depends on, and the
	# initramfs holds a ./init. Each of those is a boot-time symptom that reads like a
	# backend bug when it is not named here.
	- name: The guest lane's inputs
	run: task guest:verify

	# The demos run the real control-plane binary against a real Postgres, which is the
	# point of them — the catalog is where placement, epochs and the term guard live, and
	# a stage that stubbed it would be asserting on a stub. `task db:dev:up` starts the
	# same pinned container a developer runs, so there is one definition of it rather
	# than a `services:` block that would be a second.
	- name: The pinned development Postgres
	run: task db:dev:up

	- name: Stage 1 — a guest boots off a qcow2 this system made
	run: task demo:stage1

	- name: Stage 2 — the tip is sealed under a writing guest
	run: task demo:stage2

	- name: Stage 3 — what the guest wrote is in the bucket, sealed
	run: task demo:stage3

	- name: Stage 4 — the host is destroyed and the volume comes back
	run: task demo:stage4

	- name: Stage 5 — a snapshot is a name for a commit
	run: task demo:stage5

	- name: Stage 6 — a clone reads what its parent wrote
	run: task demo:stage6

	# The gate. This is the check to require on the branch.
	#
	# `needs` does not do it alone: a needed job that is skipped leaves the dependent job
	# free to run, and a workflow of green-or-grey jobs is reported as passing. `if:
	# always()` is required for the same reason — without it this job is itself skipped the
	# moment anything it needs fails, and a skipped gate never says no.
	gate:
	needs: [ci, guest]
	if: always()
	runs-on: ubuntu-latest
	timeout-minutes: 5
	steps:
	- name: Every lane ran, and every lane passed
	env:
	# The whole `needs` context, serialized, so `needs:` above is the only list of
	# lanes: a hand-written line per job is a second list, and a job missing from it
	# is a lane the gate reports green without checking. jq is preinstalled on hosted
	# Ubuntu runners and this is not a container job; if it ever is not, `set -e`
	# makes that a red gate.
	NEEDS: ${{ toJSON(needs) }}
	run: \|
	set -euo pipefail
	results=$(jq -r 'to_entries[] \| "\(.key)=\(.value.result)"' <<< "$NEEDS")
	# An enumeration that came back empty is the no-op that reports success — the
	# shape of every defect this gate was built to remove (a `-run` regex matching
	# no tests, a lane skipping itself, a target exiting 0 after printing SKIP).
	# A gate with no lanes has not passed; it has failed to run.
	if [ -z "$results" ]; then
	echo "::error::the gate enumerated no lanes — it would be green having checked nothing"
	exit 1
	fi
	failed=0
	{
	echo "## Gate"
	echo
	echo "\| lane \| result \|"
	echo "\| --- \| --- \|"
	} >> "$GITHUB_STEP_SUMMARY"
	while IFS='=' read -r job result; do
	[ -n "$job" ] \|\| continue
	echo "\| \`$job\` \| $result \|" >> "$GITHUB_STEP_SUMMARY"
	if [ "$result" != "success" ]; then
	failed=1
	# "skipped" is the case worth spelling out: it is not a neutral outcome
	# here, it is a proof this repository claims to make and did not.
	echo "::error::$job did not pass (${result:-no result}) — the gate is not green"
	fi
	done <<< "$results"
	if [ "$failed" -ne 0 ]; then
	echo >> "$GITHUB_STEP_SUMMARY"
	echo "A lane that was skipped counts as a failure: a run that did not make a" >> "$GITHUB_STEP_SUMMARY"
	echo "proof this repository claims has not passed the gate, it has failed to reach it." >> "$GITHUB_STEP_SUMMARY"
	exit 1
	fi
	echo "every lane ran and passed"

Navigation Menu

Uh oh!

Stop building a second QEMU and mirroring somebody else's kernel #108

Workflow file

Stop building a second QEMU and mirroring somebody else's kernel #108

Uh oh!

Workflow file for this run