update: bump lab pins to ePHPm v0.7.0, split off the removed-engine suites

README.md

@@ -8,18 +8,32 @@ A reproducible Kubernetes lab for people deciding whether ePHPm belongs in their
 
 ## Relationship to ePHPm v0.7.0
 
-> **Historical (pre-v0.7.0).** The database suites pin `ephpm/ephpm:v0.6.3` and
-> parts of them exercise machinery that **no longer exists upstream**. ePHPm
-> v0.7.0 removed the rusqlite engine (`[db.sqlite] engine = "sqlite"` is now a
-> **hard startup error**), the sqld sidecar, the `[db.sqlite.sqld]
-> write_permits` admission knob, and the `cdc_experimental` knob; Turso is the
-> only embedded engine and clustered replication runs over the in-process Turso
-> CDC path. The `engines`, `admission`, and sqld-cluster lanes in
-> [DB-BENCH.md](DB-BENCH.md) therefore run only against the pinned v0.6.3
-> image and **will not run against v0.7.0+ images**. Their recorded numbers are
-> retained as the parity evidence behind the engine switch — the same way
-> ePHPm's own [benchmarking results page](https://ephpm.dev/benchmarking/results/)
-> marks those sections historical.
+The lab's live pins are **`ephpm/ephpm:v0.7.0-php8.4`** (the Kubernetes suites)
+and **`ephpm/ephpm:v0.7.0-php8.5`** (the single-host database tier).
+
+Two database suites are the exception and stay **hard-pinned to v0.6.3**,
+because they measure machinery that **no longer exists upstream**. ePHPm
+v0.7.0 removed the rusqlite engine (`[db.sqlite] engine = "sqlite"` is now a
+**hard startup error**), the sqld sidecar, the `[db.sqlite.sqld]
+write_permits` admission knob, and the `cdc_experimental` knob; Turso is the
+only embedded engine and clustered replication runs over the in-process Turso
+CDC path. The `engines` and `admission` suites in [DB-BENCH.md](DB-BENCH.md)
+therefore **ignore `--image`** and run only against v0.6.3. Their recorded
+numbers are retained as the parity evidence behind the engine switch — the
+same way ePHPm's own
+[benchmarking results page](https://ephpm.dev/benchmarking/results/) marks
+those sections historical. The `bridge` and `wp-bridge` suites keep their
+rusqlite lanes on the same footing: opt-in, and pinned to v0.6.3 when enabled.
+
+> **The v0.7.0 bump crosses an engine change, so read database deltas
+> carefully.** Any v0.6.3-vs-v0.7.0 comparison on a database-touching path is
+> comparing *two different engines*, not two versions of one. Paths that never
+> reach the database (the `hello`/`cpu` runtime fixtures, the cache-heavy
+> Laravel/native-KV lanes) are like-for-like; `db.php`, the bridge suites and
+> WordPress-on-the-embedded-database are not. The
+> `k8s/runtimes-bench.yaml` `bench-ephpm-turso` lane, which existed purely to
+> A/B the engine knob, is retired at `replicas: 0` for the same reason: on
+> v0.7.0 it would be the `bench-ephpm` lane wearing a second label.
 
 ## The Numbers
 

RUNTIMES-BENCH.md

@@ -11,18 +11,28 @@ from the ePHPm-lab report's next-tests list.
 
 | Runtime | Image | PHP |
 |---------|-------|-----|
-| ePHPm v0.6.3 | `ephpm/ephpm:v0.6.3-php8.4` | 8.4 ZTS, glibc |
+| ePHPm v0.7.0 | `ephpm/ephpm:v0.7.0-php8.4` | 8.4 ZTS, glibc |
 | nginx + php-fpm | `nginx:1.27-alpine` + `php:8.4-fpm` (Debian) | 8.4 NTS, glibc |
 | FrankenPHP | `dunglas/frankenphp:latest` | 8.5 ZTS, glibc (image default; see caveat) |
 | Swoole | `phpswoole/swoole:php8.4` | 8.4 NTS, glibc |
 | RoadRunner | `php:8.4-cli-alpine` + `ghcr.io/roadrunner-server/roadrunner:2024` | 8.4 NTS, musl (see caveat) |
-| ePHPm v0.6.3 worker mode | `ephpm/ephpm:v0.6.3-php8.4` (`[php] mode = "worker"`) | 8.4 ZTS, glibc |
+| ePHPm v0.7.0 worker mode | `ephpm/ephpm:v0.7.0-php8.4` (`[php] mode = "worker"`) | 8.4 ZTS, glibc |
 
-The manifests pin **v0.6.3**, which carries the whole v0.4.x line
+The manifests pin **v0.7.0**, which carries the whole v0.4.x line
 (v0.4.1: 101x db.php latency fix + SHA-NI; v0.4.2: HTTP `TCP_NODELAY`
 -13% p99, worker dispatch fastpath, mimalloc/LTO), v0.5.0's
-**resource-aware autotuning**, and the v0.6.x database-path work
-(pool fixes, the `ephpm_db_*` bridge — see `DB-BENCH.md`).
+**resource-aware autotuning**, the v0.6.x database-path work
+(pool fixes, the `ephpm_db_*` bridge — see `DB-BENCH.md`), and v0.7.0's
+**engine swap**: the embedded database is now Turso only, the rusqlite
+engine and the sqld sidecar are gone.
+
+> **The v0.7.0 pin bump crosses an engine change.** Nothing in *this*
+> file's Class A / Class B fixtures (`hello`, `cpu`) touches the
+> database, so those lanes are a like-for-like v0.6.3-vs-v0.7.0
+> comparison. The `db.php` lane is **not**: on v0.6.3 it ran the
+> genuine-SQLite C engine and on v0.7.0 it runs Turso, so a delta there
+> is an engine delta, not a runtime delta. Label it that way or do not
+> report it.
 For the v0.4.0-vs-v0.4.1 before/after,
 see [docs/ephpm-0.4.1-retest.md](docs/ephpm-0.4.1-retest.md). The
 `db.php` lane (10 PDO queries on ePHPm's in-process SQLite) remains
@@ -42,24 +52,26 @@ the reproduction path for the database-latency number.
 > expected to benefit most; tiny-script lanes (`hello`) should be
 > unchanged. Operator config still overrides any derived value.
 
-## Experimental: Turso engine db lane (deployed, undriven)
-
-`k8s/runtimes-bench.yaml` carries a `bench-ephpm-turso` Deployment +
-Service — identical to the ePHPm db.php lane but with the
-**experimental** `[db.sqlite] engine = "turso"` knob (the Rust SQLite
-rewrite). On the v0.6.3 pin the knob exists, so the lane ships
-`replicas: 1` — but **no k6 Job drives it**: applying the manifest
-deploys the pod and then nothing measures it. To measure it, confirm
-the pod log shows the experimental-engine startup warning, then run the
-db.php profile by hand against `http://bench-ephpm-turso:8080/db.php`
-and compare with the `bench-ephpm` lane's db.php result. Context:
-Phase 1 microbenchmarks at the litewire seam measured 28x point-SELECT
-and 4x concurrent-writer throughput vs the C engine; this lane measures
-what survives of that through the full mysqlnd → MySQL-wire → engine
-path on a resource-limited pod. Note this lane collapses on any future
-v0.7.0+ pin bump: Turso becomes the only engine there, so the knob (and
-the lane's reason to exist) disappears — see the "Relationship to ePHPm
-v0.7.0" section in the README.
+## Retired: Turso engine db lane (`replicas: 0` on the v0.7.0 pin)
+
+`k8s/runtimes-bench.yaml` still carries the `bench-ephpm-turso`
+Deployment + Service, now scaled to **`replicas: 0`**. It existed to A/B
+the `[db.sqlite] engine = "turso"` knob against the genuine-SQLite C
+engine that was the v0.6.x default, and the v0.6.3 pin is the last one on
+which that A/B means anything. v0.7.0 removed the C engine — `"turso"` is
+the only value the knob accepts and also the default — so this Deployment
+and `bench-ephpm` would now select the **same engine**, leaving the
+comparison with no control arm. It is kept scaled to zero rather than
+deleted so the manifest records what the lane was; do not scale it back
+up and report it as an engine comparison, because it would be
+`bench-ephpm` wearing a second label.
+
+The question it was built to answer (Phase 1 microbenchmarks at the
+litewire seam measured 28x point-SELECT and 4x concurrent-writer
+throughput vs the C engine — how much survives the full mysqlnd →
+MySQL-wire → engine path?) is now answered only in the historical
+v0.6.3-pinned `engines` suite in [DB-BENCH.md](DB-BENCH.md). See the
+"Relationship to ePHPm v0.7.0" section in the README.
 
 ## Class A vs Class B
 
@@ -187,7 +199,8 @@ kubectl apply -f k8s/runtimes-bench.yaml
 kubectl delete job k6-bench-ephpm k6-bench-nginx-fpm k6-bench-frankenphp \
   k6-bench-swoole k6-bench-rr k6-bench-ephpm-worker \
   -n runtimes-bench --ignore-not-found
-# Wait for all deployments (bench-ephpm-turso comes up too, but no job drives it):
+# Wait for all deployments (bench-ephpm-turso is retired at replicas: 0, so
+# its rollout status returns immediately):
 for d in bench-ephpm bench-ephpm-turso bench-nginx-fpm bench-frankenphp \
          bench-swoole bench-rr bench-ephpm-worker; do
   kubectl rollout status deployment/$d -n runtimes-bench --timeout=300s
@@ -225,7 +238,8 @@ k8s/runtimes-bench.yaml    Single self-contained manifest:
                              - ConfigMaps (fixtures, configs, k6 script)
                              - 7 Deployments + 7 Services (five Class A/B
                                runtimes, the ePHPm worker-mode lane, and
-                               the undriven bench-ephpm-turso lane)
+                               the retired bench-ephpm-turso lane at
+                               replicas: 0)
                              - 6 k6 Jobs (one per runtime lane plus the
                                worker lane; bench-ephpm-turso has none)
 scripts/run-runtimes-bench.sh  Driver: apply, wait, run jobs, print summaries

db/bench-admission.sh

@@ -35,7 +35,22 @@
 # /usr/bin/grep explicitly. Raw oha output is kept per cell regardless.
 set -uo pipefail
 
-BASE_IMG="${EPHPM_IMAGE:-docker.io/ephpm/ephpm:v0.6.3-php8.5}"
+# ---------------------------------------------------------------------------
+# HISTORICAL SUITE -- HARD-PINNED TO v0.6.3. DO NOT BUMP.
+#
+# Every lane here sweeps `[db.sqlite.sqld] write_permits`, a knob ePHPm
+# REMOVED in v0.7.0 along with the sqld sidecar and the rusqlite engine
+# the cluster configs select. On a v0.7.0 image the configs fail at
+# startup (engine = "sqlite") and the knob no longer exists at all, so
+# there is nothing left for this sweep to sweep.
+#
+# The pin is hardcoded rather than read from EPHPM_IMAGE for the same
+# reason the suite already has a startup-log gate: the top-level driver
+# now defaults to a v0.7.0 image, and silently inheriting it would either
+# kill every lane or (worse, if a config ever became startable) benchmark
+# a lane wearing a label whose mechanism is gone.
+# ---------------------------------------------------------------------------
+BASE_IMG="${EPHPM_ADMISSION_BASE_IMAGE:-docker.io/ephpm/ephpm:v0.6.3-php8.5}"
 ADM_IMG="${EPHPM_ADMISSION_IMAGE:-$BASE_IMG}"
 OHA=ghcr.io/hatoo/oha:latest
 CURL=docker.io/curlimages/curl:latest

db/bench-bridge.sh

@@ -1,8 +1,11 @@
 #!/usr/bin/env bash
 # In-process bridge vs MySQL wire, same engine, same process.
 #
-#   A  sqlite  rusqlite in-process (production default)
-#   B  turso   Turso engine in-process (experimental)
+#   A  sqlite  rusqlite in-process -- REMOVED in ePHPm v0.7.0. Opt-in via
+#              BRIDGE_LEGACY_SQLITE=1, and then only on the pinned v0.6.3
+#              image; it is a historical row, not a lane of this run.
+#   B  turso   Turso engine in-process -- the only engine from v0.7.0 on,
+#              and the default lane of this suite.
 #
 # Each lane runs ONE container and measures six cells against it:
 #
@@ -33,7 +36,14 @@
 # measurement again.
 set -uo pipefail
 
-IMG="${EPHPM_IMAGE:-docker.io/ephpm/ephpm:v0.6.3-php8.5}"
+# The Turso lane runs on the current pin. The rusqlite lane CANNOT: v0.7.0
+# removed that engine and `engine = "sqlite"` is a hard startup error, so
+# lane A is opt-in (BRIDGE_LEGACY_SQLITE=1) and hard-pinned to the last
+# image that has the engine. Two lanes on two different ePHPm versions are
+# NOT an engine A/B -- they differ by a whole release. If you enable lane A,
+# report it as a separate historical row, never in one table with lane B.
+IMG="${EPHPM_IMAGE:-docker.io/ephpm/ephpm:v0.7.0-php8.5}"
+LEGACY_IMG="${EPHPM_LEGACY_IMAGE:-docker.io/ephpm/ephpm:v0.6.3-php8.5}"
 OHA=ghcr.io/hatoo/oha:latest
 CURL=docker.io/curlimages/curl:latest
 NET=dbbench-net
@@ -89,9 +99,22 @@ gate() {  # description urlpath expected-substring
   esac
 }
 
-run_lane() {  # lane cfg
-  LANE="$1"; local cfg="$2"
+run_lane() {  # lane cfg img
+  LANE="$1"; local cfg="$2" img="$3"
   echo ""; echo "############ LANE $LANE ($cfg, single-node, --cpus $CPUS) ############"
+  echo "   image: $img"
+  # Purge THIS lane's files before measuring. $OUT persists across runs, and a
+  # lane that is not run this time (lane A is opt-in) leaves its previous
+  # session's files sitting next to this session's -- parse.sh then prints one
+  # table mixing two runs on two images with nothing to distinguish them. That
+  # happened on 2026-08-19: an archived "v0.7.0" directory still carried
+  # v0.6.3 rusqlite rows. Stale results that look current are exactly the
+  # failure this suite's gates exist to prevent.
+  rm -f "$OUT/${LANE}-"*.txt
+  {
+    echo "lane=$LANE image=$img config=$cfg"
+    echo "recorded=$(date -u +%Y-%m-%dT%H:%M:%SZ) host=$(hostname) dur=$DUR reps=$REPS cpus=$CPUS"
+  } > "$OUT/RUN-INFO-${LANE}.txt"
   cleanup
   podman volume rm -f "dbv-$LANE" >/dev/null 2>&1 || true
   podman volume create "dbv-$LANE" >/dev/null
@@ -100,7 +123,7 @@ run_lane() {  # lane cfg
     -v "$HERE/fixtures/bridge:/var/www/html/bridge:ro" \
     -v "$HERE/configs/$cfg:/etc/ephpm/ephpm.toml:ro" \
     -v "dbv-$LANE:/data" \
-    "$IMG" >/dev/null
+    "$img" >/dev/null
   if ! wait_ready; then
     echo "!! $LANE never became ready:"; podman logs dbbench-c1 2>&1 | tail -40; return 1
   fi
@@ -129,8 +152,16 @@ run_lane() {  # lane cfg
 }
 
 FAILED=0
-run_lane A-sqlite single-sqlite.toml || FAILED=1
-run_lane B-turso  single-turso.toml  || FAILED=1
+# Lane A is the removed rusqlite engine: opt-in, and on its own pinned image.
+if [ "${BRIDGE_LEGACY_SQLITE:-0}" = 1 ]; then
+  if [ "$LEGACY_IMG" != "$IMG" ]; then
+    echo "!! lane A-sqlite runs on $LEGACY_IMG, lane B-turso on $IMG."
+    echo "!! Those differ by a whole ePHPm release, not just an engine: report"
+    echo "!! A as a separate historical row, never in one table with B."
+  fi
+  run_lane A-sqlite single-sqlite.toml "$LEGACY_IMG" || FAILED=1
+fi
+run_lane B-turso  single-turso.toml  "$IMG" || FAILED=1
 
 echo ""
 if [ "$FAILED" = 1 ]; then

db/bench-engines.sh

@@ -17,7 +17,30 @@
 # measurement again.
 set -uo pipefail
 
-IMG="${EPHPM_IMAGE:-docker.io/ephpm/ephpm:v0.6.3-php8.5}"
+# ---------------------------------------------------------------------------
+# HISTORICAL SUITE -- HARD-PINNED TO v0.6.3. DO NOT BUMP.
+#
+# Three of this matrix's four lanes exercise machinery ePHPm REMOVED in
+# v0.7.0, so this suite is not runnable on a current image:
+#   A (single-sqlite.toml)      engine = "sqlite"    -> hard startup error
+#   C (cluster-sqlite-*.toml)   engine = "sqlite" + sqld sidecar -> gone
+#   D (cluster-turso-*.toml)    replication.cdc_experimental -> knob removed;
+#                               ephpm-config does not reject unknown fields,
+#                               so on v0.7.0 that line is silently ignored
+#                               and lane D would be a DIFFERENT topology
+#                               wearing lane D's label.
+# Only lane B (Turso single-node) would survive, and a one-lane "engine
+# comparison" has no control arm.
+#
+# The pin is therefore hardcoded rather than read from EPHPM_IMAGE: the
+# top-level driver (scripts/run-db-bench.sh) now defaults to a v0.7.0
+# image, and inheriting it here would produce three dead lanes and one
+# mislabelled one. Replacing this suite for v0.7.0 means a NEW
+# Turso-single vs Turso-CDC-clustered matrix, not edits to these lanes.
+# EPHPM_ENGINES_IMAGE is the deliberate override if you know why you want
+# one.
+# ---------------------------------------------------------------------------
+IMG="${EPHPM_ENGINES_IMAGE:-docker.io/ephpm/ephpm:v0.6.3-php8.5}"
 OHA=ghcr.io/hatoo/oha:latest
 CURL=docker.io/curlimages/curl:latest
 NET=dbbench-net