sync: CLI defect fixes (L4) + preview readiness probe (004743c)

AUDIT_OPEN.md

@@ -2550,3 +2550,31 @@ Gates: `go build ./...` && `go vet ./...` clean; `go test ./... -count=1`
 internal/network (touched this session) plus internal/backup and
 internal/openbao ok; gofmt clean on touched files (pre-existing strays
 untouched).
+
+## 2026-09-24 nineteenth-wave defect sweep (lane L4, cli-defects)
+
+The real defects the nineteenth wave's lanes found and left open, plus two
+more found while proving them. Each landed with a pin test in the same
+commit; live proofs on the colima fixture.
+
+| Item | Outcome | Commit | Evidence |
+|---|---|---|---|
+| `teploy log` does not resolve named servers (R02 docs lane) | **fixed** | 39197d0 | Root cause was shared, not log-specific: `config.ResolveServer`'s `--host` branch took the value verbatim, so every `--app --host <name>` command (log, status, logs, health, rollback --app, ...) dialed the literal name. A `--host` naming a servers.yml entry now resolves to it (`--user`/`--key` still win; unregistered values and a missing servers.yml stay raw). Pins: config + `runLog`. Live: `teploy log --app quickstart --host colima-vm` connected to the entry's `tyler@127.0.0.1:<port>`. No written workaround existed in this repo's docs; NEXT_SESSION's mention is the only record. |
+| tcp health gates pass against docker-proxy with a dead backend (C03 follow-up) | **fixed** | 4664094 | The probe connects and holds up to 1s: a byte or an open connection is ready; the proxy's immediate close (backend refused), a refused connect, or a bad host is not. `deploy.TCPProbeCommand` is shared by the deploy gate, auto's 404/3xx fallback, and heal. Live on colima: sleep-only container on a published port — old probe rc=0, new rc=1; the deploy gate refuses it; the httpd fixture passes. Pin runs the real command against accept-then-close / silent / banner / closed listeners (red on the old probe on Linux; skipped on bash<4, where a read timeout is indistinguishable from EOF, so it fails closed). |
+| (found proving the above) port-mismatch diagnosis blamed Docker's embedded DNS | **fixed** | 79b8b5c | A container with nothing listening was diagnosed as "the app is listening on port 46107": the 127.0.0.11 resolver inside every user-network container. Skipped now; the correct "nothing is listening" finding fires. The slow-boot hint named a nonexistent `health: { timeout: 90s }`; corrected to `timeout_seconds`. |
+| ID-created containers report the ID, not the tag (Ship wave-9) | **fixed**, not display-only | ff72cb2 | `docker.ResolveImageTags` (one batched `docker image inspect`, none when no ID-form image): `status` text/--json, `app status --json`, `server status --json` report image = first tag + image_id + image_tags. **Comparison impact found and fixed:** rollback overwrote the target release record's ImageRef with the container's docker ps image, so after a rollback the state carried an unpullable short ID (DR bundle restore resolves the image from ImageRef); the release record now wins, else the resolved tag. Prune's rmi and the predecessor snapshot keep the raw value by design. Corpus rev 7 (additive): optional image_id/image_tags. Live: quickstart asserts the status tag. |
+| `teploy env set` does not feed secret-backed vars (Ship wave-9) | **by design on precedence, guarded** | 65962bd | The server `.env` is the first env file; teploy.yml `env:` + decrypted secrets + resolved `secret:` refs ride the later attempt file and win. So `env set` on a secret-backed key was a silent no-op. It now refuses (nothing written) for keys in the secret store or `secret:` references in teploy.yml, naming the remedy; a plain teploy.yml `env:` key that shadows it gets a warning. README states the precedence. |
+| X05: no maintained deployable CLI fixture app | **fixed** | 4ed08f0 | `examples/quickstart/app` (landed by C09, acf475f) is the maintained fixture; `make quickstart` now covers deploy -> verify -> redeploy -> health -> status-by-tag -> rollback -> tcp-gate refusal of a never-listening qs3 with v1 still serving. Fixture fixes: the sh-wrapped httpd ignored SIGTERM as PID 1 (every retirement waited out the 10s kill: Exited 137, ~11.5s deploys -> TERM trap, ~1.5s, Exited 0); a real 200 `/health`. Green on colima. |
+
+**Recorded, not done here:** `internal/preview`'s `probeTCP` has the same
+connect-only shape as the old gate. Left alone because the L1 lane owned
+internal/preview this wave; the fix is to switch it to
+`deploy.TCPProbeCommand`. `teploy health` probes with the default auto
+mode and not the app's configured `health.mode`/path. That is a
+pre-existing inconsistency, now visible because the fixture has a real
+`/health`.
+
+Gates: `go test ./... -count=1` 26/26 packages ok (macOS);
+internal/deploy full suite PASS on Linux (colima, bash 5.2);
+`GOOS=linux go vet ./... && GOOS=linux go build ./...` clean; `make
+quickstart` green on colima.

README.md

@@ -217,7 +217,10 @@ processes:
 #   http — status-based only: GET path, 200 = ready. A 404/redirect FAILS
 #          (no fallback). Best when the app has a real health endpoint.
 #   tcp  — a TCP dial against the published port; nothing is fetched.
-#          For apps with no HTTP surface (game servers, TCP brokers).
+#          The connection is held ~1s: a listener that closes it at once
+#          (Docker's proxy does, when nothing in the container listens)
+#          is NOT ready. For apps with no HTTP surface (game servers, TCP
+#          brokers).
 #          Setting `path` alongside is rejected — nothing would fetch it.
 #   auto — compatibility default (also what an omitted mode means): HTTP
 #          GET first; a 404/3xx falls back to a TCP dial. The historical
@@ -385,6 +388,11 @@ teploy secret rm KEY               # delete an encrypted secret (local store)
 teploy secret get / list / rotate  # secret management
 ```
 
+At deploy, secrets (and `secret:` references in `env:`) override
+`teploy.yml` `env:`, which overrides the server `.env` that `env set`
+writes. `env set` therefore refuses a key that is a secret (use
+`secret set`) and warns when `teploy.yml` `env:` shadows the key.
+
 ### Fleet
 ```
 teploy server add <name> <host>    # add server to ~/.teploy/servers.yml

contracts/MANIFEST.md

@@ -10,6 +10,7 @@ Neutron/Nucleus dependency and a public mirror.
 
 | Corpus rev | Emitting CLI | Machine Interface | Notes |
 |---|---|---|---|
+| 7 | main (L4 cli-defects: ID-created container image resolution) | 2 | Additive. The container object in app-list-envelope and server-status-envelope gains optional `image_id` (full sha256) and `image_tags` (string array), emitted only for containers created by image ID (A52 creates web/worker containers from the immutable ID). For those, `image` now carries the first repo tag instead of the bare 12-hex short ID docker ps reports - restoring the pre-A52 meaning (Ship wave-9: ID-created containers never matched the artifact tag). Name-created containers and existing fixtures are unchanged (no fixture regenerated: the corpus fixtures use name-form images). No MI bump. |
 | 6 | main (tailnet preview mode, DELEGATED_DECISIONS §10) | 2 | Additive. preview-state schema gains optional record/list-row fields on both eras (`domain`, `url`, `base_domain`, `http_only`, `allow_ips`) with the invariant url scheme = `http://` iff `http_only` (else `https://`); two valid fixtures GENERATED from the real `preview list --json` row encoder (`previewListRows`, `contracts_golden_test.go`): canonical-list-row (default mode, no exposure keys, https url) and canonical-list-row-tailnet (base_domain + http_only + allow_ips, http url), each wrapped with the artifact's `era`/`app` classification keys (the wire row carries neither). The hand-authored identity fixtures (canonical, legacy, ambiguous) are unchanged. version-handshake gains the `preview-exposure` capability token (additive). No MI bump. |
 | 5 | main (X02 S2 tail: server-status fixtures + schema correction) | 2 | server-status-envelope fixtures landed (was "pending live capture"): valid x2 (full healthy observation, partial-caddy-unavailable — the class a target without a caddy container produces) + legacy pre-MI (machine_interface absent, the 42243e2-era shape). Encoder-derived: generated from the REAL `collectServerStatus` via a mock SSH executor (`contracts_golden_test.go`, TEPLOY_UPDATE_CONTRACTS) — synthetic values, real encoder and parse stages; the wire shape was verified against a live `server status --json` run before pinning. Defect fixed in the same commit: the schema had copied the appStatus root since its S2 draft (its own defect-fix commit 08cfb1b said so) and never described the actual serverStatusDTO wire format (server/host/uptime/load/memory/disks/docker/caddy) — rewritten to the real root with strict required-key coverage of the DTO's no-omitempty fields. Additive to consumers (a schema that matched nothing before now matches the wire); no MI bump. |
 | 4 | main (X02 S2 tail: server-list reshape) | 2 | **The MI 2 bump** (D8 non-additive): `server list --json` now emits the envelope `{machine_interface, servers[], observed_at}` carrying the per-server fields unchanged (name + id/host/user/role/tags/vpn_ip); the pre-reshape bare map-of-servers root is GONE on the wire and is pinned as the artifact's legacy class. New artifact server-list-envelope (schema + valid + legacy fixtures); version-handshake schema maximum 1→2 and its valid fixture renamed mi1→mi2 (app-list valid likewise — both envelopes now report MI 2). Capability tokens unchanged. Coordinated consumer: teploy-dash decodes both shapes during the transition (MaxSupportedMachineInterface 2). |

contracts/schema/app-list-envelope.schema.json

@@ -70,6 +70,16 @@
         "image": {
           "type": "string"
         },
+        "image_id": {
+          "type": "string",
+          "description": "Set only for a container created by image ID: the full sha256 image ID. image then carries the first repo tag (or the ID when untagged/unresolvable)."
+        },
+        "image_tags": {
+          "type": "array",
+          "items": {
+            "type": "string"
+          }
+        },
         "state": {
           "type": "string"
         },

contracts/schema/server-status-envelope.schema.json

@@ -187,6 +187,16 @@
         "image": {
           "type": "string"
         },
+        "image_id": {
+          "type": "string",
+          "description": "Set only for a container created by image ID: the full sha256 image ID. image then carries the first repo tag (or the ID when untagged/unresolvable)."
+        },
+        "image_tags": {
+          "type": "array",
+          "items": {
+            "type": "string"
+          }
+        },
         "state": {
           "type": "string"
         },

examples/quickstart/README.md

@@ -8,18 +8,25 @@ documentation, and it runs.
 make quickstart        # from the repo root
 ```
 
-`run.sh` deploys `app/` (the maintained fixture: a busybox httpd app with
-a `teploy.yml`) to the **local colima VM's own SSH endpoint**, then:
+`run.sh` deploys `app/` (the maintained CLI fixture app — busybox httpd
+serving `index.html` and a 200 `/health`, with a `teploy.yml`; X05's CLI
+fixture) to the **local colima VM's own SSH endpoint**, then:
 
 1. builds the CLI from this checkout,
 2. bootstraps the target once (`/deployments` directory; the only
    target-side setup, via the VM's passwordless sudo),
-3. deploys version `qs1` — build-on-target, health-gated start, host
+3. deploys version `qs1` — build-on-target, tcp-gated start, host
    ingress on `127.0.0.1:18080`,
 4. verifies the app answers with the v1 content,
 5. redeploys as `qs2` with changed content and verifies the switch,
-6. shows `teploy status`, then removes everything it created (app,
-   containers, images, its own known_hosts lines).
+6. runs `teploy health`, and checks `teploy status` names the image by
+   tag (`quickstart-build-qs2`), not the bare image ID,
+7. `teploy rollback` and verifies v1 is served again,
+8. deploys `qs3`, a container that runs but never listens, and verifies
+   the tcp readiness gate REFUSES it (docker-proxy accepts on the port
+   either way) while v1 keeps serving,
+9. removes everything it created (app, containers, images, its own
+   known_hosts lines).
 
 Requirements: docker CLI, a **running** colima VM (the script never
 starts one — `colima start` yourself), ssh/ssh-keyscan/curl/python3.

examples/quickstart/app/Dockerfile

@@ -1,5 +1,10 @@
 FROM busybox:1.37
 COPY index.html /www/index.html
+# /health answers 200 so `teploy health` (HTTP first) passes on the real
+# endpoint, not the 404->TCP fallback.
+RUN echo ok > /www/health
 # teploy injects PORT (the published port) as an env var; listen there so
-# the health gate and the published port see the same listener.
-CMD ["sh", "-c", "httpd -f -p ${PORT:-80} -h /www"]
+# the health gate and the published port see the same listener. PID 1 gets
+# no default signal handling, so the explicit TERM trap is what lets docker
+# stop retire the container at once instead of waiting out the kill timeout.
+CMD ["sh", "-c", "trap 'exit 0' TERM; httpd -f -p ${PORT:-80} -h /www & wait"]

examples/quickstart/run.sh

@@ -11,8 +11,10 @@
 #
 # What it proves: a new user path from `git clean` checkout to a
 # responding application — config in teploy.yml, build on the target,
-# health-gated start, published port, idempotent redeploy, status,
-# removal. Exit 0 only if every step held.
+# health-gated start, published port, on-demand health, redeploy, status
+# naming the image TAG, rollback, the tcp readiness gate refusing a
+# container that runs but never listens (with the served version left
+# alone), and removal. Exit 0 only if every step held.
 set -euo pipefail
 
 REPO_ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
@@ -33,7 +35,7 @@ cleanup() {
     "$TEPLOY_BIN" remove --purge --yes --app quickstart --host "$SSH_HOST" \
       --user "$SSH_USER" --key "$SSH_KEY" >/dev/null 2>&1
     ssh -i "$SSH_KEY" -p "$SSH_PORT" -o BatchMode=yes "$SSH_USER@$SSH_HOST" \
-      'docker rm -f quickstart-web-qs1 quickstart-web-qs2 >/dev/null 2>&1; docker rmi quickstart-build-qs1 quickstart-build-qs2 >/dev/null 2>&1; sudo rm -rf /deployments/quickstart' 2>/dev/null
+      'docker rm -f quickstart-web-qs1 quickstart-web-qs2 quickstart-web-qs3 >/dev/null 2>&1; docker rmi quickstart-build-qs1 quickstart-build-qs2 quickstart-build-qs3 >/dev/null 2>&1; sudo rm -rf /deployments/quickstart' 2>/dev/null
   fi
   if [ -n "$CONTAINER_KEY_FILE" ] && [ -f "$HOME/.ssh/known_hosts" ]; then
     # Remove only the lines this run appended.
@@ -117,9 +119,50 @@ BODY="$(curl -fsS -m 10 "http://127.0.0.1:$APP_PORT/")"
 grep -q "quickstart v2" <<<"$BODY" || { echo "FAIL: v2 content not served after redeploy: $BODY" >&2; exit 1; }
 log "verified: redeploy switched the served content to v2"
 
-# --- status ------------------------------------------------------------------
-log "teploy status sees the deployment"
-"${TEPLOY[@]}" status --app quickstart 2>&1 | sed 's/^/    /'
+serves() {
+  local body
+  body="$(curl -fsS -m 10 "http://127.0.0.1:$APP_PORT/")" || return 1
+  grep -q "$1" <<<"$body"
+}
 
-log "quickstart complete: deployed, verified, redeployed, verified again"
+# --- on-demand health ----------------------------------------------------------
+log "teploy health probes the live app"
+"${TEPLOY[@]}" health --app quickstart 2>&1 | sed 's/^/    /'
+
+# --- status names the image by tag -------------------------------------------
+log "teploy status sees the deployment (image reported by tag, not bare ID)"
+STATUS="$("${TEPLOY[@]}" status --app quickstart 2>&1)"
+sed 's/^/    /' <<<"$STATUS"
+grep -q "quickstart-build-qs2" <<<"$STATUS" \
+  || { echo "FAIL: status did not report the qs2 image tag" >&2; exit 1; }
+
+# --- rollback -----------------------------------------------------------------
+log "rolling back to qs1"
+"${TEPLOY[@]}" rollback --app quickstart 2>&1 | sed 's/^/    /'
+serves "quickstart v1" || { echo "FAIL: rollback did not restore the v1 content" >&2; exit 1; }
+log "verified: rollback serves v1 again"
+
+# --- tcp gate refuses a dead backend ----------------------------------------
+# A container that runs but never listens: docker-proxy still accepts on the
+# published port, so a connect-only probe would pass it.
+log "deploying qs3: runs, never listens — the tcp readiness gate must refuse it"
+cat >"$WORK_DIR/app/Dockerfile" <<'DEAD'
+FROM busybox:1.37
+CMD ["sleep", "3600"]
+DEAD
+# health: is the last block of the fixture's teploy.yml; a short deadline
+# keeps the refusal quick.
+printf '  timeout_seconds: 5\n' >>"$WORK_DIR/app/teploy.yml"
+tail -1 "$WORK_DIR/app/teploy.yml" | grep -q '^  timeout_seconds: 5$' || { echo "FAIL: could not set the qs3 deadline" >&2; exit 1; }
+if (cd "$WORK_DIR/app" && "${TEPLOY[@]}" deploy --version qs3) >"$WORK_DIR/qs3.log" 2>&1; then
+  sed 's/^/    /' "$WORK_DIR/qs3.log"
+  echo "FAIL: the tcp gate passed a container with nothing listening" >&2; exit 1
+fi
+sed 's/^/    /' "$WORK_DIR/qs3.log" | tail -6
+grep -q "mode tcp" "$WORK_DIR/qs3.log" || { echo "FAIL: qs3 did not fail at the tcp gate" >&2; exit 1; }
+log "verified: the tcp gate refused qs3"
+serves "quickstart v1" || { echo "FAIL: the refused deploy left v1 not serving" >&2; exit 1; }
+log "verified: v1 still serves after the refused deploy"
+
+log "quickstart complete: deploy, health, redeploy, status, rollback, dead-backend refusal"
 log "cleanup follows (teploy remove --purge, known_hosts lines, temp dir)"