Skip to content

Commit b2f8fd4

Browse files
committed
exercise and verify the cross-registry copy
Two gaps, one of them live. The dry run pushed only to a GHCR scratch repository and set publish_hub=false, so the Docker Hub copy was the one part of the publish no test could reach. A dry_run_hub input now resolves the Docker Hub name to a scratch repository too, so a dry run exercises imagetools create across registries without touching a tag any consumer pulls. It is refused unless dry_run is also set, because on its own it would take the real branch and read as though it had tested something. ⛔ The live gap: the publish job verified only the registry the digests were staged on. A Docker Hub tag that never arrived, or arrived without all four platforms, ended a green run. verify_index now runs against both. The four input combinations were checked before pushing: dry_run=false dry_run_hub=false ghcr.io/OWNER/archlinux pkgforge/archlinux publish_hub=true dry_run=false dry_run_hub=true refused dry_run=true dry_run_hub=false ghcr.io/OWNER/archlinux-ci pkgforge/archlinux-ci publish_hub=false dry_run=true dry_run_hub=true ghcr.io/OWNER/archlinux-ci pkgforge/archlinux-ci publish_hub=true A normal run resolves exactly what it did before. tests/static/35-publish-targets.sh guards the regression that would make this dangerous: a tag-creating step naming HUB_IMAGE directly keeps the scratch name on GHCR and writes real tags to Docker Hub, and nothing about the run looks wrong. Seen to fail four ways, recorded in HISTORY/tests-seen-to-fail.md. ⚠ The first dry_run_hub run creates pkgforge/archlinux-ci on Docker Hub, which does not exist yet. The GHCR counterpart already exists and holds 26 tags from the dry run before the first real publish.
1 parent f235f69 commit b2f8fd4

4 files changed

Lines changed: 219 additions & 34 deletions

File tree

‎.github/workflows/build-deploy.yml‎

Lines changed: 88 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,10 @@ on:
1818
description: Push to a scratch repository and create no public tag
1919
type: boolean
2020
default: false
21+
dry_run_hub:
22+
description: With dry_run, also copy to the Docker Hub scratch repository
23+
type: boolean
24+
default: false
2125
schedule:
2226
- cron: "30 05 * * *" # 08:30 PM UTC everyday
2327

@@ -37,6 +41,10 @@ env:
3741
GHCR_IMAGE: ghcr.io/${{ github.repository_owner }}/archlinux
3842
HUB_IMAGE: pkgforge/archlinux
3943
SCRATCH_IMAGE: ghcr.io/${{ github.repository_owner }}/archlinux-ci
44+
# The Docker Hub side of a dry run. Copying across registries is the one part
45+
# of the publish that a GHCR-only dry run cannot reach, so it needs a scratch
46+
# repository of its own on the other registry.
47+
HUB_SCRATCH_IMAGE: pkgforge/archlinux-ci
4048

4149
#------------------------------------------------------------------------------------#
4250
jobs:
@@ -53,6 +61,7 @@ jobs:
5361
build_date: ${{ steps.inputs.outputs.build_date }}
5462
anchors: ${{ steps.inputs.outputs.anchors }}
5563
target_image: ${{ steps.inputs.outputs.target_image }}
64+
hub_image: ${{ steps.inputs.outputs.hub_image }}
5665
publish_hub: ${{ steps.inputs.outputs.publish_hub }}
5766
steps:
5867
- name: Checkout
@@ -64,22 +73,44 @@ jobs:
6473
id: inputs
6574
env:
6675
DRY_RUN: ${{ inputs.dry_run }}
76+
DRY_RUN_HUB: ${{ inputs.dry_run_hub }}
6777
run: |
6878
set -euo pipefail
6979
version="$(date -u +'%Y.%m.%d')"
7080
build_date="$(date -u +'%Y-%m-%dT%H:%M:%SZ')"
71-
echo "version=$version" >> "$GITHUB_OUTPUT"
72-
echo "build_date=$build_date" >> "$GITHUB_OUTPUT"
81+
82+
# ⛔ dry_run_hub only makes sense inside a dry run. Without this guard
83+
# a run with dry_run_hub alone would take the real branch and read as
84+
# though it had tested something.
85+
if [ "${DRY_RUN_HUB:-false}" = "true" ] && [ "${DRY_RUN:-false}" != "true" ]; then
86+
echo "dry_run_hub needs dry_run. On its own it would publish to the real repositories." >&2
87+
exit 1
88+
fi
7389
7490
if [ "${DRY_RUN:-false}" = "true" ]; then
75-
echo "target_image=${SCRATCH_IMAGE}" >> "$GITHUB_OUTPUT"
76-
echo "publish_hub=false" >> "$GITHUB_OUTPUT"
77-
echo "::notice::dry run, building into ${SCRATCH_IMAGE} and creating no public tag"
91+
target_image="$SCRATCH_IMAGE"
92+
hub_image="$HUB_SCRATCH_IMAGE"
93+
if [ "${DRY_RUN_HUB:-false}" = "true" ]; then
94+
publish_hub=true
95+
echo "::notice::dry run into ${SCRATCH_IMAGE} and ${HUB_SCRATCH_IMAGE}, exercising the cross-registry copy, creating no public tag"
96+
else
97+
publish_hub=false
98+
echo "::notice::dry run, building into ${SCRATCH_IMAGE} and creating no public tag"
99+
fi
78100
else
79-
echo "target_image=${GHCR_IMAGE}" >> "$GITHUB_OUTPUT"
80-
echo "publish_hub=true" >> "$GITHUB_OUTPUT"
101+
target_image="$GHCR_IMAGE"
102+
hub_image="$HUB_IMAGE"
103+
publish_hub=true
81104
fi
82105
106+
{
107+
echo "version=$version"
108+
echo "build_date=$build_date"
109+
echo "target_image=$target_image"
110+
echo "hub_image=$hub_image"
111+
echo "publish_hub=$publish_hub"
112+
} >> "$GITHUB_OUTPUT"
113+
83114
# The anchor names the pinned tag family. The ports are not in
84115
# lockstep, so it is resolved per architecture.
85116
anchors='{'
@@ -282,14 +313,15 @@ jobs:
282313
- name: Create the per architecture tags
283314
env:
284315
TARGET_IMAGE: ${{ needs.resolve.outputs.target_image }}
316+
HUB_TARGET: ${{ needs.resolve.outputs.hub_image }}
285317
PUBLISH_HUB: ${{ needs.resolve.outputs.publish_hub }}
286318
VERSION: ${{ needs.resolve.outputs.version }}
287319
ANCHORS: ${{ needs.resolve.outputs.anchors }}
288320
run: |
289321
set -euo pipefail
290322
images=("$TARGET_IMAGE")
291323
if [ "$PUBLISH_HUB" = "true" ]; then
292-
images+=("$HUB_IMAGE")
324+
images+=("$HUB_TARGET")
293325
fi
294326
for arch in amd64 arm64 armv7 riscv64; do
295327
digest_file="/tmp/digests/$arch"
@@ -312,13 +344,14 @@ jobs:
312344
- name: Create the index tags
313345
env:
314346
TARGET_IMAGE: ${{ needs.resolve.outputs.target_image }}
347+
HUB_TARGET: ${{ needs.resolve.outputs.hub_image }}
315348
PUBLISH_HUB: ${{ needs.resolve.outputs.publish_hub }}
316349
VERSION: ${{ needs.resolve.outputs.version }}
317350
run: |
318351
set -euo pipefail
319352
images=("$TARGET_IMAGE")
320353
if [ "$PUBLISH_HUB" = "true" ]; then
321-
images+=("$HUB_IMAGE")
354+
images+=("$HUB_TARGET")
322355
fi
323356
sources=()
324357
for arch in amd64 arm64 armv7 riscv64; do
@@ -335,31 +368,52 @@ jobs:
335368
- name: Verify what was published
336369
env:
337370
TARGET_IMAGE: ${{ needs.resolve.outputs.target_image }}
371+
HUB_TARGET: ${{ needs.resolve.outputs.hub_image }}
372+
PUBLISH_HUB: ${{ needs.resolve.outputs.publish_hub }}
338373
VERSION: ${{ needs.resolve.outputs.version }}
339374
run: |
340375
set -euo pipefail
341-
# The dated index tag carries a leading v. scripts/tag-names is the
342-
# authority on the spelling, so it is asked rather than reconstructed:
343-
# inspecting a tag the publish job never created would fail every run.
344-
dated="$(scripts/tag-names index "$VERSION" "$TARGET_IMAGE" | grep -v ':latest$')"
345-
echo "verifying $dated"
346-
347-
# The index must carry all four platforms. An index with a hole in it
348-
# is the failure this whole topology exists to prevent.
349-
got="$(docker buildx imagetools inspect "$dated" --raw \
350-
| jq -r '[ .manifests[].platform
351-
| select(.os != "unknown")
352-
| .architecture + (if .variant then "/" + .variant else "" end) ]
353-
| sort | join(" ")')"
354-
echo "index platforms: $got"
355-
356-
# armv7 renders as arm/v7. A bare match on arm would also match arm64,
357-
# so the assertion would pass with armv7 missing and mean nothing.
358-
for expect in amd64 arm64 riscv64 arm/v7; do
359-
if ! grep -qw -- "$expect" <<< "$got"; then
360-
echo "the index for $dated is missing $expect" >&2
361-
echo "it carries: $got" >&2
362-
exit 1
363-
fi
364-
done
365-
echo "::notice::published $dated with platforms $got"
376+
377+
# ⛔ Both registries are verified, not just the one the digests were
378+
# staged on. Copying across registries is a separate operation that can
379+
# fail on its own, and until this step covered it a Docker Hub tag that
380+
# never arrived, or arrived without every platform, ended a green run.
381+
verify_index() {
382+
local image="$1" dated got expect
383+
# The dated index tag carries a leading v. scripts/tag-names is the
384+
# authority on the spelling, so it is asked rather than
385+
# reconstructed: inspecting a tag the publish job never created
386+
# would fail every run.
387+
dated="$(scripts/tag-names index "$VERSION" "$image" | grep -v ':latest$')"
388+
echo "verifying $dated"
389+
390+
# The index must carry all four platforms. An index with a hole in
391+
# it is the failure this whole topology exists to prevent.
392+
got="$(docker buildx imagetools inspect "$dated" --raw \
393+
| jq -r '[ .manifests[].platform
394+
| select(.os != "unknown")
395+
| .architecture + (if .variant then "/" + .variant else "" end) ]
396+
| sort | join(" ")')"
397+
echo "index platforms: $got"
398+
399+
# armv7 renders as arm/v7. A bare match on arm would also match
400+
# arm64, so the assertion would pass with armv7 missing and mean
401+
# nothing.
402+
for expect in amd64 arm64 riscv64 arm/v7; do
403+
if ! grep -qw -- "$expect" <<< "$got"; then
404+
echo "the index for $dated is missing $expect" >&2
405+
echo "it carries: $got" >&2
406+
echo "the digests were staged on ${TARGET_IMAGE}. If that index is" >&2
407+
echo "complete and this one is not, the cross-registry copy is what failed." >&2
408+
exit 1
409+
fi
410+
done
411+
echo "::notice::published $dated with platforms $got"
412+
}
413+
414+
verify_index "$TARGET_IMAGE"
415+
if [ "$PUBLISH_HUB" = "true" ]; then
416+
verify_index "$HUB_TARGET"
417+
else
418+
echo "::notice::publish_hub is false, so no Docker Hub tag was created or verified"
419+
fi

‎HISTORY/tests-seen-to-fail.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,10 @@ fault, and pointing `REPO_ROOT` at that directory.
3030
| `25` | the `grep -c` form restored at `scripts/gen-mirrorlist:278` | `not ok 1`, naming the file and line |
3131
| `25` | the `grep -c` form restored at `.github/workflows/freshness-mirrors.yml:47` | `not ok 2 - none of the 5 workflows uses a pipeline that fails on a normal result` |
3232
| `30-signature-checking-on.sh` | `SigLevel = Never` in `rootfs/riscv64/etc/pacman.conf` | `not ok 7 - does not disable signature checking`, `not ok 8 - sets SigLevel = Required` |
33+
| `35-publish-targets.sh` | a tag-creating step changed back to `images+=("$HUB_IMAGE")` | `not ok 4 - no tag-creating step names GHCR_IMAGE or HUB_IMAGE directly`, naming both lines |
34+
| `35` | `HUB_SCRATCH_IMAGE` set to `pkgforge/archlinux`, the real repository | `not ok 3 - HUB_SCRATCH_IMAGE names a different repository from HUB_IMAGE` |
35+
| `35` | the `dry_run_hub needs dry_run` guard deleted | `not ok 5 - the resolve step refuses dry_run_hub without dry_run` |
36+
| `35` | the `verify_index "$HUB_TARGET"` call deleted | `not ok 6 - the publish job verifies the index on both registries`, `matched 1 of the 2` |
3337
| `40-mirrors-reachable.sh` | the riscv64 list cut to one server, which is the shape that caused the outage | `not ok 11 - offers a fallback server`, `not ok 16 - keeps at least 2 reachable servers` |
3438
| `40` | ten of thirteen amd64 servers replaced with unresolvable hosts | `not ok 13 - keeps at least 7 reachable servers` |
3539
| `50-supply-chain.sh` | `actions/checkout@v7.0.1` at all three call sites | `not ok 1..3 - action is pinned to a commit hash at ...` |

‎tests/README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@ reproduces it. Exit status is non-zero when any assertion fails.
2323
| `20-no-swallowed-errors.sh` | no `continue-on-error`, `\|\| true`, `set +e` or `2>/dev/null` in the workflows, the Dockerfile or `scripts/`, outside `tests/policy/swallowed-errors.allow` |
2424
| `25-pipeline-traps.sh` | no script that sets `pipefail`, and no workflow `run` block, pipes into `head` or counts with a `grep` count flag. Both fail on a result that is normal, and both stop the script with no message |
2525
| `30-signature-checking-on.sh` | every shipped `pacman.conf` sets `SigLevel = Required` and none sets any SigLevel to `Never` |
26+
| `35-publish-targets.sh` | a dry run cannot reach a real repository: each scratch name differs from its real counterpart, no tag-creating step names `GHCR_IMAGE` or `HUB_IMAGE` directly, `dry_run_hub` is refused without `dry_run`, and the publish job verifies the index on both registries |
2627
| `40-mirrors-reachable.sh` | each list carries a generation date and is within the age bound, ships a fallback server and at least one over https, and keeps at least two reachable servers and at least half of what it ships. Every entry that does not answer 200 is named as a diagnostic |
2728
| `50-supply-chain.sh` | every action is pinned to a commit hash and names its version, nothing pipes a remote script into a shell, no opaque binary is fetched and made executable, no deprecated workflow command, every workflow declares least-privilege `permissions` |
2829
| `60-tag-families.sh` | `scripts/tag-names` emits every alias for each architecture on both registry names, each with a rolling, a dated and a pinned shape, plus the `latest` and `v<version>` index tags, with no alias claimed twice and an unknown architecture or empty version refused |

‎tests/static/35-publish-targets.sh‎

Lines changed: 126 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,126 @@
1+
#!/usr/bin/env bash
2+
#
3+
# A dry run must not be able to reach a real repository.
4+
#
5+
# The publish job creates tags from a resolved target name. A dry run resolves
6+
# that name to a scratch repository on each registry. ⛔ The failure this guards
7+
# is one edit away: if a tag-creating step goes back to naming HUB_IMAGE
8+
# directly instead of the resolved output, a dry run keeps its scratch name on
9+
# GHCR and writes real tags to Docker Hub. Nothing about the run would look
10+
# wrong, and the tags would be public.
11+
#
12+
# ⚠ This reads the workflow rather than running it. What it cannot check is
13+
# whether the resolve step's branches are correct; that is checked by running
14+
# the workflow with dry_run and dry_run_hub, recorded in
15+
# HISTORY/tests-seen-to-fail.md.
16+
set -euo pipefail
17+
# shellcheck source-path=SCRIPTDIR/..
18+
# shellcheck source=lib/harness.sh
19+
. "$REPO_ROOT/tests/lib/harness.sh"
20+
21+
WF="$REPO_ROOT/.github/workflows/build-deploy.yml"
22+
23+
if [ ! -f "$WF" ]; then
24+
fail "the build workflow exists" \
25+
"expected: $WF" \
26+
"reproduce: ls .github/workflows/build-deploy.yml"
27+
summary
28+
exit 1
29+
fi
30+
ok "the build workflow exists"
31+
32+
# value NAME -> the value of a top level env entry
33+
value() {
34+
awk -v k="$1" '$1 == k":" { sub(/^[^:]*:[[:space:]]*/, ""); print; exit }' "$WF"
35+
}
36+
37+
ghcr="$(value GHCR_IMAGE)"
38+
hub="$(value HUB_IMAGE)"
39+
scratch="$(value SCRATCH_IMAGE)"
40+
hub_scratch="$(value HUB_SCRATCH_IMAGE)"
41+
42+
#---------------------------------------------------------------------------#
43+
# 1. Four names, and each real one differs from its scratch counterpart.
44+
#---------------------------------------------------------------------------#
45+
for pair in "GHCR_IMAGE:$ghcr:SCRATCH_IMAGE:$scratch" "HUB_IMAGE:$hub:HUB_SCRATCH_IMAGE:$hub_scratch"; do
46+
IFS=: read -r rn rv sn sv <<< "$pair"
47+
if [ -z "$rv" ] || [ -z "$sv" ]; then
48+
fail "$rn and $sn are both set" \
49+
"$rn=${rv:-<unset>}, $sn=${sv:-<unset>}" \
50+
"reproduce: grep -nE '^ (GHCR|HUB|SCRATCH|HUB_SCRATCH)_IMAGE:' .github/workflows/build-deploy.yml"
51+
elif [ "$rv" = "$sv" ]; then
52+
fail "$sn names a different repository from $rn" \
53+
"both are $rv" \
54+
"a dry run would then write to the repository consumers pull" \
55+
"reproduce: grep -nE '^ (GHCR|HUB|SCRATCH|HUB_SCRATCH)_IMAGE:' .github/workflows/build-deploy.yml"
56+
else
57+
ok "$sn ($sv) is not $rn ($rv)"
58+
fi
59+
done
60+
61+
#---------------------------------------------------------------------------#
62+
# 2. No step that creates a tag names a real repository directly.
63+
#
64+
# The tag steps must build their image list from the resolved outputs. A
65+
# reference to HUB_IMAGE or GHCR_IMAGE inside one of them bypasses the dry run.
66+
#---------------------------------------------------------------------------#
67+
direct="$(awk '
68+
/^ - name: Create the (per architecture|index) tags$/ { instep = 1; next }
69+
/^ - name: / { instep = 0 }
70+
instep && /\$(\{)?(HUB_IMAGE|GHCR_IMAGE)/ { printf "%d: %s\n", NR, $0 }
71+
' "$WF")"
72+
73+
if [ -z "$direct" ]; then
74+
ok "no tag-creating step names GHCR_IMAGE or HUB_IMAGE directly"
75+
else
76+
fail "no tag-creating step names GHCR_IMAGE or HUB_IMAGE directly" \
77+
"found: $(printf '%s' "$direct" | tr '\n' ' ')" \
78+
"these steps must use the resolved target_image and hub_image outputs" \
79+
"naming the real repository here makes a dry run publish real tags" \
80+
"reproduce: grep -n 'HUB_IMAGE\|GHCR_IMAGE' .github/workflows/build-deploy.yml"
81+
fi
82+
83+
#---------------------------------------------------------------------------#
84+
# 3. dry_run_hub cannot be used on its own.
85+
#---------------------------------------------------------------------------#
86+
if grep -q 'dry_run_hub needs dry_run' "$WF"; then
87+
ok "the resolve step refuses dry_run_hub without dry_run"
88+
else
89+
fail "the resolve step refuses dry_run_hub without dry_run" \
90+
"the guard is not present" \
91+
"without it, dry_run_hub alone takes the real branch and looks like a test" \
92+
"reproduce: grep -n 'dry_run_hub' .github/workflows/build-deploy.yml"
93+
fi
94+
95+
#---------------------------------------------------------------------------#
96+
# 4. Both registries are verified after publishing.
97+
#
98+
# Copying across registries is its own operation and can fail on its own. A
99+
# verification that only inspects the staging registry passes while the other
100+
# one carries nothing.
101+
#---------------------------------------------------------------------------#
102+
verify_block="$(awk '
103+
/^ - name: Verify what was published$/ { instep = 1 }
104+
instep && /^ - name: / && !/Verify what was published/ { instep = 0 }
105+
instep { print }
106+
' "$WF")"
107+
108+
if [ -z "$verify_block" ]; then
109+
fail "the publish job verifies what it published" \
110+
"no 'Verify what was published' step found" \
111+
"reproduce: grep -n 'Verify what was published' .github/workflows/build-deploy.yml"
112+
else
113+
hits=0
114+
printf '%s\n' "$verify_block" | grep -q 'verify_index "\$TARGET_IMAGE"' && hits=$((hits + 1))
115+
printf '%s\n' "$verify_block" | grep -q 'verify_index "\$HUB_TARGET"' && hits=$((hits + 1))
116+
if [ "$hits" -eq 2 ]; then
117+
ok "the publish job verifies the index on both registries"
118+
else
119+
fail "the publish job verifies the index on both registries" \
120+
"matched $hits of the 2 expected verify_index calls" \
121+
"a run that copied nothing to the second registry would still be green" \
122+
"reproduce: sed -n '/Verify what was published/,/^\$/p' .github/workflows/build-deploy.yml"
123+
fi
124+
fi
125+
126+
summary

0 commit comments

Comments
 (0)