X02 S7 sweep + C01-1 slice 1: target guard

AUDIT_OPEN.md

@@ -1896,3 +1896,50 @@ fleet diagnosis (doctor currently diagnoses ONE resolved target),
 DNS/health-path diagnostics, and machine-event streaming (the
 "versioned JSON/events" contract's events half — doctor emits one
 versioned JSON document per run, not a stream).
+
+## X02 S7 acceptance sweep — 2026-09-23
+
+`scripts/x02-acceptance-sweep.sh` (this repo's executable harness, ADR §6
+S7). Legs and evidence (exit 0, all PASS, non-vacuous — each pattern is
+verified to match >=1 test before running):
+
+| Leg | Package | Tests | Result |
+|---|---|---|---|
+| rename | ./internal/config | 5 (rename/update/re-add preserve id; legacy stays id-less; mint shape) | PASS |
+| duplicate-identity | ./internal/preview | 2 (preview ID golden; branch identity distinct) | PASS |
+| release-identity | ./internal/releasemeta | 2 (absent/present/unreadable; round-trip + path validation) | PASS |
+| repeated-request | ./internal/cli | 3 (version/release/attempt-name corpus goldens) | PASS |
+| response-loss | ./internal/deploy | 4 (Decide Compensate-vs-Inspect; attribution; predecessor snapshots) | PASS |
+| rollback | ./internal/deploy | 4 (state-commit failure restores old workload/route; rollback from recorded spec; fixed-port displacement) | PASS |
+
+The harness fails on any leg failing OR matching no tests (a vacuous pass
+is a broken pin). Re-run and paste fresh output here on any contract
+change.
+
+## C01-1 slice 1 — target-side critical section (2026-09-23, `3a28454`)
+
+`internal/targetguard`: the on-demand helper (flock + generation fencing +
+stdout protocol) and its Go wrapper, live-proven in podman on Debian
+bookworm-slim and alpine 3.20:
+
+| Invariant | Evidence |
+|---|---|
+| serialization | timestamped ABAB (one full critical section, then the other's) on both distros |
+| generation fencing | gen 7 committed vs plan-expects-3 → GUARD_FENCED, effect file never created |
+| death-release | killed helper (SIGKILL, exit 137) → next guarded effect proceeds |
+| current generation | gen-7 plan against gen-7 target → GUARD_OK |
+| unfit target | no flock OR flock failing the one-time serialization self-test → GUARD_UNFIT (never lock-free, never falsely-locked) |
+
+Process note recorded for the next sessions: the investigation's first
+harness read ABAB as "interleaved" — the exact inversion (serialized =
+ABAB; interleaved = AABB). The timestamped rerun caught it. The busybox
+FILE-form portability doubt that motivated the self-test was never
+reproduced with timestamps and may itself have been harness-instrument
+error; the self-test stays regardless (it prices at ~1.2s once per app
+dir and closes a real class).
+
+REMAINS (C01-1 slice 2): guarded-effect integration into the deploy path
+(state commit + predecessor retirement under the guard; the .generation
+sidecar written by the state commit), the two-clients/delayed-SSH/clock
+-change acceptance matrix against the real deploy path, and the
+documented app-lock/shared-proxy acquisition order.

internal/targetguard/guard.go

@@ -0,0 +1,144 @@
+// Package targetguard is C01-1's on-demand target-side critical section:
+// a small helper uploaded over SSH and invoked to run ONE protected effect
+// under an OS-exclusive flock with generation fencing (programme §101).
+//
+// Why a helper and not more client-side checks: the existing lock
+// serializes ACQUISITION, but the critical section itself spans many SSH
+// round-trips — between the check and the effect another client (or a
+// stale-breaking race) can interleave. Moving verify-and-effect into one
+// process on the target closes that window: the OS releases the lock on
+// process death (a killed helper never freezes the app), and the
+// generation fence refuses effects prepared against a superseded
+// generation (an abandoned owner's stale rollback cannot stop a newer
+// generation).
+//
+// Outcome protocol: the helper always exits 0 and reports on the FIRST
+// stdout line (GUARD_OK / GUARD_BUSY / GUARD_FENCED <c> <e> / GUARD_UNFIT
+// / GUARD_BADGEN / GUARD_EFFECT_FAILED <n>) because the ssh Executor
+// abstraction does not preserve exit codes; the effect's output follows.
+package targetguard
+
+import (
+	"context"
+	_ "embed"
+	"errors"
+	"fmt"
+	"strings"
+
+	"github.com/useteploy/teploy/internal/ssh"
+)
+
+//go:embed guard.sh
+var guardScript string
+
+// Typed outcomes. ErrBusy is retryable (someone else holds the guard —
+// back off and retry). ErrFenced means the plan is STALE: the target has
+// committed a newer generation than this effect was prepared against —
+// reconcile against on-target evidence instead of blind-retrying (the D11
+// honesty floor). ErrTargetUnfit is fail-closed: either no flock(1) on
+// the target, or the target's flock failed the helper's lock-primitive
+// self-test (a flock that does not serialize is worse than none) — the
+// effect NEVER runs lock-free or under a fictitious lock.
+var (
+	ErrBusy        = errors.New("target guard is held by another operation (retryable)")
+	ErrFenced      = errors.New("target guard fenced the effect: a newer generation is committed on the target — reconcile before retrying")
+	ErrTargetUnfit = errors.New("target's flock is missing or failed the serialization self-test; the guarded effect refuses to run")
+)
+
+const helperRemote = "/tmp/teploy-guard.sh"
+
+// ensureUploaded installs the helper on the target. An overwrite never
+// races a RUNNING helper: the running process already loaded its copy, and
+// sh reads the whole script before executing.
+func ensureUploaded(ctx context.Context, exec ssh.Executor) error {
+	return exec.Upload(ctx, strings.NewReader(guardScript), helperRemote, "0755")
+}
+
+// Run executes cmd on the target under the app's guard with generation
+// fencing. expectedGeneration is the generation the calling plan was
+// prepared against (state.Generation read at plan time); the effect runs
+// only while the committed generation is <= expected — a NEWER committed
+// generation fences the stale plan out. The effect's combined output is
+// returned on GUARD_OK.
+func Run(ctx context.Context, exec ssh.Executor, app string, expectedGeneration uint64, cmd string) (string, error) {
+	if err := ensureUploaded(ctx, exec); err != nil {
+		return "", fmt.Errorf("uploading target guard: %w", err)
+	}
+	// cmd is quoted as ONE argument: unquoted, an effect's `;`/`>>` would
+	// execute at the INVOCATION level — outside the guard, before the
+	// helper has even created its directories (caught by CI's linux runner
+	// after the mac-side build-tag gap hid it).
+	invoke := fmt.Sprintf("sh %s %s %d -- %s", helperRemote, shQuote(app), expectedGeneration, shQuote(cmd))
+	out, runErr := exec.Run(ctx, invoke)
+	if runErr != nil {
+		return "", fmt.Errorf("invoking target guard: %w", errDetail(runErr, out))
+	}
+	// The protocol line is the first GUARD_-prefixed line — not necessarily
+	// line 1: the target's shell may emit job-control notices first (bash
+	// prints "Killed" to stdout when the effect is SIGKILLed, observed on
+	// CI runners 2026-09-23).
+	lines := strings.Split(out, "\n")
+	head := ""
+	headIdx := -1
+	for i, line := range lines {
+		if strings.HasPrefix(strings.TrimSpace(line), "GUARD_") {
+			head = strings.TrimSpace(line)
+			headIdx = i
+			break
+		}
+	}
+	if headIdx == -1 {
+		return "", fmt.Errorf("target guard protocol violation (no GUARD_ line in %q)", firstLine(out))
+	}
+	rest := strings.TrimRight(strings.Join(lines[headIdx+1:], "\n"), "\n")
+	fields := strings.Fields(head)
+	switch {
+	case fields[0] == "GUARD_OK":
+		return rest, nil
+	case fields[0] == "GUARD_BUSY":
+		return "", ErrBusy
+	case fields[0] == "GUARD_FENCED":
+		return "", fmt.Errorf("%w (committed %s > expected %s)", ErrFenced, orDash(field(fields, 1)), orDash(field(fields, 2)))
+	case fields[0] == "GUARD_UNFIT":
+		return "", ErrTargetUnfit
+	case fields[0] == "GUARD_BADGEN":
+		return "", fmt.Errorf("target guard: generation sidecar unreadable for app %s", app)
+	case fields[0] == "GUARD_EFFECT_FAILED":
+		return "", fmt.Errorf("guarded effect failed (exit %s): %s", orDash(field(fields, 1)), rest)
+	default:
+		return "", fmt.Errorf("target guard protocol violation (first line %q)", head)
+	}
+}
+
+func field(fields []string, i int) string {
+	if i < len(fields) {
+		return fields[i]
+	}
+	return ""
+}
+
+func orDash(s string) string {
+	if s == "" {
+		return "?"
+	}
+	return s
+}
+
+func errDetail(err error, out string) error {
+	if t := strings.TrimSpace(out); t != "" {
+		return fmt.Errorf("%w: %s", err, t)
+	}
+	return err
+}
+
+func firstLine(out string) string {
+	if i := strings.IndexByte(out, '\n'); i >= 0 {
+		return out[:i]
+	}
+	return out
+}
+
+// shQuote quotes one word for the POSIX shell the helper runs under.
+func shQuote(s string) string {
+	return "'" + strings.ReplaceAll(s, "'", `'\''`) + "'"
+}

internal/targetguard/guard.sh

@@ -0,0 +1,97 @@
+#!/bin/sh
+# teploy target-side critical-section helper (C01-1 mechanism).
+#
+# Invoked over SSH by the CLI (internal/targetguard). Holds an OS-exclusive
+# flock for the duration of one protected effect — the property the mkdir
+# lock lacks: process death releases it, so a killed helper can never leave
+# a frozen app. Under the lock it fences on the committed generation: a
+# client whose plan was prepared against an older generation is refused
+# before its effect runs, so an abandoned owner's stale rollback can never
+# stop a newer generation.
+#
+# argv: <app> <expected_generation> -- <command...>
+#
+# PROTOCOL: the helper always exits 0 and reports its outcome as the FIRST
+# line of stdout (the ssh executor abstraction does not preserve exit
+# codes, so the outcome rides the output stream):
+#   GUARD_OK                    effect committed
+#   GUARD_BUSY                  lock held and not acquired (retryable)
+#   GUARD_FENCED <c> <e>        committed generation c > expected e (stale plan)
+#   GUARD_UNFIT                 no flock(1) — fails closed, NEVER lock-free
+#   GUARD_BADGEN                generation sidecar unreadable
+#   GUARD_EFFECT_FAILED <n>     the effect exited n
+# The effect's own combined output follows on subsequent lines.
+#
+# LOCK SHAPE: `flock FILE -c BODY` — the util-linux `flock FD` form (lock
+# an already-open fd, hold it past the flock process's exit) is NOT
+# portable: busybox releases on child exit, which testing on alpine
+# proved as silent non-serialization (ABAB interleaving). The FILE form
+# runs the whole verify+effect body as flock's child, which both
+# implementations hold for the body's lifetime.
+#
+# The committed generation lives in the .generation sidecar (a plain
+# integer, written atomically by the state commit; absent = 0).
+
+set -u
+
+APP="$1"
+EXPECTED="$2"
+shift 2
+if [ "${1:-}" = "--" ]; then shift; fi
+
+ROOT="${TEPLOY_DEPLOYMENTS_ROOT:-/deployments}"
+APPDIR="$ROOT/$APP"
+LOCKFILE="$APPDIR/.lock/guard"
+mkdir -p "$APPDIR/.lock"
+
+command -v flock >/dev/null 2>&1 || {
+    echo "GUARD_UNFIT"
+    exit 0
+}
+
+# Lock-primitive self-test (once per app dir, marker under the lock): a
+# target whose flock does not actually SERIALIZE must fail closed, never
+# run effects under a fictitious lock. Observed in the wild: busybox
+# flock's FILE-cmd form acquires instantly against a held lock in some
+# environments (alpine under podman, 2026-09-23) while its FD form blocks
+# correctly — util-linux is sound. The self-test pins the property we
+# depend on instead of the tool's name.
+if [ ! -f "$LOCKFILE.selftest-ok" ]; then
+    flock "$LOCKFILE" -c "sleep 1" &
+    HOLDER=$!
+    sleep 0.2
+    if flock -n "$LOCKFILE" -c "true" 2>/dev/null; then
+        kill "$HOLDER" 2>/dev/null
+        wait "$HOLDER" 2>/dev/null
+        echo "GUARD_UNFIT"
+        exit 0
+    fi
+    wait "$HOLDER"
+    touch "$LOCKFILE.selftest-ok"
+fi
+
+GUARD_APPDIR="$APPDIR" GUARD_EXPECTED="$EXPECTED" GUARD_EFFECT="$*" \
+flock "$LOCKFILE" -c '
+    APPDIR="$GUARD_APPDIR"; EXPECTED="$GUARD_EXPECTED"
+    if [ -f "$APPDIR/.generation" ]; then
+        COMMITTED=$(cat "$APPDIR/.generation" 2>/dev/null)
+        case "$COMMITTED" in
+            ""|*[!0-9]*)
+                echo "GUARD_BADGEN"
+                exit 0
+                ;;
+        esac
+        if [ "$COMMITTED" -gt "$EXPECTED" ]; then
+            echo "GUARD_FENCED $COMMITTED $EXPECTED"
+            exit 0
+        fi
+    fi
+    OUT=$(mktemp) || { echo "GUARD_BUSY"; exit 0; }
+    if sh -c "$GUARD_EFFECT" >"$OUT" 2>&1; then
+        echo "GUARD_OK"
+    else
+        echo "GUARD_EFFECT_FAILED $?"
+    fi
+    cat "$OUT"
+    rm -f "$OUT"
+' || echo "GUARD_BUSY"