diff --git a/DB-BENCH.md b/DB-BENCH.md index 1c8c4e4..a329491 100644 --- a/DB-BENCH.md +++ b/DB-BENCH.md @@ -12,27 +12,42 @@ different upstreams. ## Relationship to ePHPm v0.7.0 -> **Historical (pre-v0.7.0).** This file's suites pin `v0.6.3`, and three of -> them exercise machinery that was **removed upstream in v0.7.0**: the rusqlite -> engine (`[db.sqlite] engine = "sqlite"` is now a hard startup error with a -> migration message), the sqld sidecar, the `[db.sqlite.sqld] write_permits` -> knob, and `cdc_experimental`. Concretely, on a v0.7.0+ image: -> -> - the `engines` suite's rusqlite and sqld-cluster lanes fail at startup -> (`single-sqlite.toml`, `cluster-sqlite-*.toml`), and the rusqlite half of -> `bridge` fails the same way; -> - the entire `admission` suite sweeps a knob that no longer exists — the -> startup-log gate will correctly refuse every permit lane; -> - the Turso-vs-rusqlite comparison itself is no longer reproducible on any -> shippable image, because there is only one engine. -> -> These suites are retained **as the historical record against the pinned -> v0.6.3 image** — the parity evidence behind the v0.7.0 engine switch — the -> same way ePHPm's own +The v0.7.0 pin bump **split this file's suites in two**, because v0.7.0 +removed the rusqlite engine (`[db.sqlite] engine = "sqlite"` is now a hard +startup error with a migration message), the sqld sidecar, the +`[db.sqlite.sqld] write_permits` knob, and `cdc_experimental`. + +| Suite | Pin | Why | +| --- | --- | --- | +| `proxy` | **v0.7.0** | Measures the wire hop and the pool in front of litewire / `mysql:8` / `postgres:16`. Its litewire lanes now run the **Turso** engine on both releases, so the suite is comparable across the bump. | +| `bridge` | **v0.7.0** | The Turso lane is the whole suite now; the rusqlite lane is opt-in and separately pinned (below). | +| `wp-bridge` | **v0.7.0** | Same split as `bridge`. | +| `engines` | **v0.6.3, hard-pinned** | Three of four lanes are removed machinery. | +| `admission` | **v0.6.3, hard-pinned** | Sweeps a knob that no longer exists. | + +The two historical suites **ignore `--image` / `EPHPM_IMAGE`** and read +`EPHPM_ENGINES_IMAGE` / `EPHPM_ADMISSION_BASE_IMAGE` instead. That is +deliberate: `scripts/run-db-bench.sh` now defaults to a v0.7.0 image, and +letting these inherit it would give `engines` three dead lanes plus one +*silently mislabelled* one — `cluster-turso-primary.toml` sets +`replication.cdc_experimental = true`, `ephpm-config` does not reject unknown +fields, so on v0.7.0 that line is ignored and lane D would be a different +topology wearing lane D's name. + +The rusqlite halves of `bridge` and `wp-bridge` survive the same way: opt-in +via `BRIDGE_LEGACY_SQLITE=1` / `WP_BRIDGE_LEGACY_SQLITE=1`, and then hard-run +on the v0.6.3 image. **A v0.6.3 rusqlite lane and a v0.7.0 Turso lane differ +by a whole release, not by an engine** — the scripts print each lane's image +in its banner and warn on the legacy lane for exactly that reason. Never put +them in one table. + +> **Historical numbers stay historical.** Every recorded table below was taken +> on the v0.6.0/v0.6.1/v0.6.3 lines and is retained as the parity evidence +> behind the v0.7.0 engine switch — the same way ePHPm's own > [results page](https://ephpm.dev/benchmarking/results/) marks its pre-v0.7.0 -> engine and admission sections historical. A future v0.7.0 pin bump replaces -> them with a Turso-single vs Turso-CDC-clustered matrix rather than editing -> these lanes. +> engine and admission sections historical. Replacing the `engines` matrix for +> v0.7.0 means a **new** Turso-single vs Turso-CDC-clustered suite, not edits +> to these lanes. ## Why This Runs On Podman, Not Kubernetes @@ -51,30 +66,36 @@ the place for that, and the two tiers must never be put in the same table. > **Which image these numbers need.** The proxy results below require a build > with the v0.6.1 pool fixes (ePHPm main `bdc9861` or later). The harness now -> defaults to `ephpm/ephpm:v0.6.3-php8.5`, which contains those fixes, the -> `write_permits` admission knob, and the `ephpm_db_*` in-process bridge — so -> every suite in this file, including `bridge`, runs on the default image. On -> anything older than v0.6.1 the pooled lanes reproduce the two defects rather -> than the numbers, and on anything older than v0.6.3 the `bridge` suite fails -> its function-registration gate. +> defaults to `ephpm/ephpm:v0.7.0-php8.5`, which contains those fixes and the +> `ephpm_db_*` in-process bridge, so `proxy`, `bridge` and `wp-bridge` all run +> on the default image. On anything older than v0.6.1 the pooled lanes +> reproduce the two defects rather than the numbers, and on anything older than +> v0.6.3 the `bridge` suite fails its function-registration gate. The +> `engines` and `admission` suites do **not** follow this default — see +> "Relationship to ePHPm v0.7.0" above. ## Suites | Suite | Question | Runs on | | --- | --- | --- | -| `engines` | rusqlite vs Turso, single-node vs clustered sqld | Published image | -| `admission` | Does bounded write admission fix the clustered write collapse? | v0.6.1+ (knob merged in ephpm#222) | +| `engines` | rusqlite vs Turso, single-node vs clustered sqld | **v0.6.3 only** (historical; hard-pinned) | +| `admission` | Does bounded write admission fix the clustered write collapse? | **v0.6.3 only** (historical; hard-pinned) | | `proxy` | What does the DB proxy cost (a hop) and buy (pooling)? | v0.6.1+ (pool fixes in ephpm#221) | -| `bridge` | What does skipping the wire entirely buy? `ephpm_db_*` vs pdo_mysql, per engine | v0.6.3+ (bridge shipped in ephpm#257/#258) | +| `bridge` | What does skipping the wire entirely buy? `ephpm_db_*` vs pdo_mysql | v0.6.3+ (bridge shipped in ephpm#257/#258) | | `wp-bridge` | Does the bridge move a real app? WordPress with the db-wordpress drop-in vs mysqli | v0.6.3+ | ```bash -./scripts/run-db-bench.sh engines -./scripts/run-db-bench.sh admission # needs v0.6.1+ (default image is fine) +./scripts/run-db-bench.sh engines # historical, always v0.6.3 +./scripts/run-db-bench.sh admission # historical, always v0.6.3 ./scripts/run-db-bench.sh proxy -./scripts/run-db-bench.sh bridge # needs v0.6.3+ (ephpm_db_* functions) -./scripts/run-db-bench.sh wp-bridge # needs v0.6.3+ and network on first run -./scripts/run-db-bench.sh all --image docker.io/ephpm/ephpm:v0.6.3-php8.5 +./scripts/run-db-bench.sh bridge # Turso lane only by default +./scripts/run-db-bench.sh wp-bridge # needs network on first run +./scripts/run-db-bench.sh all --image docker.io/ephpm/ephpm:v0.7.0-php8.5 + +# Opt into the removed-engine lanes. These run on v0.6.3 no matter what +# --image says, and belong in their own table: +BRIDGE_LEGACY_SQLITE=1 ./scripts/run-db-bench.sh bridge +WP_BRIDGE_LEGACY_SQLITE=1 ./scripts/run-db-bench.sh wp-bridge ``` ## Fixtures @@ -189,23 +210,186 @@ relayed to the pooled backend, and a permit-accounting deadlock it was masking that nearly became a headline, is in [docs/ephpm-0.6.1-db-matrix.md](docs/ephpm-0.6.1-db-matrix.md). -## The Deliberately-Broken Config (proxy STEP 0) - -`db/configs/proxy-litewire-inprocess-BROKEN.toml` is broken **on purpose**, and -`db/bench-proxy.sh` runs it first ("STEP 0") on every invocation. It chains -`[db.mysql]` (the proxy) in front of the *same process's* in-process -`[db.sqlite]` litewire — a topology ePHPm cannot start: `start_db_proxies()` -awaits the proxy's backend connect inline and the litewire branch runs after -it, so the proxy spends its entire ~40 s ten-attempt backoff dialling a -listener that cannot exist yet, then gives up **non-fatally and nearly -silently** — the server goes on serving HTTP with nothing bound to the proxy -port and every database page returning `[2002] Connection refused`, while -liveness and readiness both look healthy (see the "Still true in v0.6.1" notes -on ePHPm's [results page](https://ephpm.dev/benchmarking/results/), which this -step reproduces). STEP 0 archives the evidence as -`db/results-proxy/FINDING-startup-order.log` each run. It is a gate in its own -right: it *proves* the proxy-vs-litewire lanes (B2/C2/J2) had to use a separate -litewire sidecar container, instead of leaving that as an assertion in prose. +### Re-recorded: litewire proxy lanes on `v0.6.3-php8.5`, Turso (2026-08-18) + +Same host and method as the `bridge` recording below. Only the litewire lanes +ran; the `mysql:8` / `postgres:16` lanes (D/E/F/G/H/I and `F24-pg-cliff`) were +**skipped** because their upstream containers were not running — see the +skip-message change in `bench-proxy.sh`. Mean of 2 × 15 s reps, RPS. + +| Lane | `db.php` c=1 | c=16 | `write.php` c=1 | c=16 | +| --- | ---: | ---: | ---: | ---: | +| A — litewire in-process, no proxy | 401.9 | 629.0 | 747.0 | 1239.2 | +| A2 — litewire sidecar, no proxy | 358.5 | 500.4 ⚠ | 346.5 ⚠ | 627.5 ⚠ | +| B2 — sidecar via proxy, pooled | 295.8 | 680.7 | 691.2 | 1235.1 | +| C2 — sidecar via proxy, no reuse | 230.7 | 334.1 ⚠ | 359.0 | 484.2 | + +The v0.6.1-era shape holds: at c=1 the proxy is a net loss against the direct +sidecar (B2 296 vs A2 358, −17 %), and pooling is what buys it back at c=16 +(B2 681 vs C2 334, **+104 %**). The hop itself still costs (A 402 → B2 296 at +c=1, though A is in-process and B2 crosses the podman bridge, so that pair is +not a clean hop measurement — A2 is the right control). + +⚠ **Two integrity problems in this run, reported rather than smoothed:** + +1. **A2 produced HTTP 500s.** `A2 write c=16 rep 2` returned **1454 × HTTP + 500** alongside 6867 × 200 — `db/parse.sh` flagged it `!!`. Per gate 5 that + cell is not a measurement, and the A2 write row above should be read as + suspect, not as a number. +2. **A2 is wildly unstable at c=1 on writes**: reps of 444.5 and 248.6 RPS + (56 % spread) — far outside this suite's "treat <20 % as unresolved" rule. + `C2 db c=16` (289.6 / 378.5) and `A2 db c=16` (403.0 / 597.8) are similarly + unstable. + +Whether that instability is the sidecar topology, the litewire frontend under +concurrent writes, or this host is **not established by this run**. It is +logged here as an open question, not as a v0.6.3 defect claim. + +**Both problems are now diagnosed — see the A2 subsection below. They are a +harness artifact, not an ePHPm defect.** + +### Re-recorded: litewire proxy lanes on `v0.7.0-php8.5` (2026-08-19) + +Same host, same method, same session as the `bridge` v0.7.0 recording (this run +occupied 21:32–21:45; load median 2.26 / p90 3.85 / max 7.57 — noisier than the +`bridge` windows because this suite runs two `--cpus 1` containers plus an +uncapped `oha`, see the load table above). Lanes D/E/H and F/G/I + +`F24-pg-cliff` **skipped** again — no `dbbench-mysql` / `dbbench-pg` upstreams +were running. Skipped is absence, not a measurement. +Mean of 2 × 15 s reps, RPS. The v0.6.3 column is the 2026-08-18 recording +(prior session — unlike the `bridge` table above, there is no same-session +control here). + +| Lane | fixture | c | v0.6.3 | v0.7.0 | Δ | +| --- | --- | ---: | ---: | ---: | ---: | +| A — litewire in-process, no proxy | db | 1 | 401.9 | 362.0 | −9.9 % | +| A — litewire in-process, no proxy | db | 16 | 629.0 | 447.9 | **−28.8 %** | +| A — litewire in-process, no proxy | write | 1 | 747.0 | 657.9 | −11.9 % | +| A — litewire in-process, no proxy | write | 16 | 1239.2 | 1042.3 | −15.9 % | +| A2 — litewire sidecar, no proxy | db | 1 | 358.5 ⚠ | 314.4 ⚠ | not a measurement | +| A2 — litewire sidecar, no proxy | db | 16 | 500.4 ⚠ | 385.8 ⚠ | not a measurement | +| A2 — litewire sidecar, no proxy | write | 1 | 346.5 ⚠ | 370.5 ⚠ | not a measurement | +| A2 — litewire sidecar, no proxy | write | 16 | 627.5 ⚠ | 579.7 ⚠ | not a measurement | +| B2 — sidecar via proxy, pooled | db | 1 | 295.8 | 275.6 | −6.8 % | +| B2 — sidecar via proxy, pooled | db | 16 | 680.7 | 630.7 | −7.3 % | +| B2 — sidecar via proxy, pooled | write | 1 | 691.2 | 641.8 | −7.2 % | +| B2 — sidecar via proxy, pooled | write | 16 | 1235.1 | 1224.4 | −0.9 % | +| C2 — sidecar via proxy, no reuse | db | 1 | 230.7 | 219.0 | −5.1 % | +| C2 — sidecar via proxy, no reuse | db | 16 | 334.1 ⚠ | 330.7 ⚠ | −1.0 % (both noisy) | +| C2 — sidecar via proxy, no reuse | write | 1 | 359.0 | 363.3 | +1.2 % | +| C2 — sidecar via proxy, no reuse | write | 16 | 484.2 | 466.6 | −3.6 % | + +The **shape** of the suite is unchanged across the release: the proxy is still +a net loss at c=1 and pooling still buys it back at c=16 (v0.7.0: B2 630.7 vs +C2 330.7 on `db` c=16, **+91 %**, against +104 % on v0.6.3). + +The **level** carries the same wire regression the `bridge` suite found, and +localises it further. Lane A is the in-process litewire wire path — the same +thing `bridge`'s wire cells measure — and it is down 9.9–28.8 %. Lanes B2/C2 +route through the proxy and move far less (−7.3 % to +1.2 %, mostly inside +noise). Pooling amortises whatever got more expensive; a per-request connect +pays it in full. See the `bridge` section for the litewire-0.2.0 hypothesis. + +⚠ **A2's HTTP 500s and instability: diagnosed, and it is the harness.** + +The A2 defect recurred on v0.7.0 — `A2 write c=16 rep 2` returned **581 × +HTTP 500** alongside 8145 × 200 (v0.6.3 rep 2: 1454 × 500). Same lane, same +fixture, same concurrency, same rep, two releases, two sessions. That +reproducibility made it worth chasing, so this run reproduced it under a +body-capturing probe instead of leaving it as an open question. The error is: + +``` +SQLSTATE[HY000] [2002] Cannot assign requested address +``` + +That is `EADDRNOTAVAIL` — **client-side ephemeral TCP port exhaustion**, not a +database error. Confirmed inside the A2 PHP container during sustained load: + +``` +/proc/sys/net/ipv4/ip_local_port_range = 32768 60999 (28 231 ports) +/proc/net/sockstat = TCP: ... tw 5807 +``` + +A2 is the one lane that opens a **fresh TCP connection to a remote host on +every request** with no reuse anywhere in the path. At ~500–800 req/s each +closed connection sits in `TIME_WAIT` for ~60 s, so steady-state demand is +~30 000–48 000 ports against a 28 231-port budget. `connect()` then fails, PHP +raises `PDOException`, and `write.php` returns its 500. + +This explains every previously-unexplained feature of the A2 rows: + +- **Why it is always rep 2.** `TIME_WAIT` accumulates across warmup + rep 1 + + rep 2. The budget is not exhausted until ~60 s of sustained load — which + lands in rep 2 every time. +- **Why the c=1 write cell is bimodal** (v0.6.3 444.5 / 248.6, 56 % spread; + v0.7.0 449.3 / 291.8, 42 %). At ~450 req/s × 60 s `TIME_WAIT` ≈ 27 000 + sockets, the lane runs *right at* the 28 231-port boundary: one rep clears + it, the next collapses. +- **Why only A2.** Lane A is in-process (no TCP). B2 pools. C2 has no reuse but + runs at ~220–470 req/s through a localhost proxy hop, below the threshold. +- **Why it is release-independent.** It reproduces identically on v0.6.3 and + v0.7.0 because it is a property of the topology, not of ePHPm. + +**Conclusion: A2 is not a valid lane as configured, and never was.** It is +measuring the host's ephemeral-port recycling as much as ePHPm. Its four cells +are struck from both releases' tables above rather than compared. This is *not* +an ePHPm bug — but it *is* a real-world caveat worth stating plainly: **any PHP +app that opens a fresh remote `pdo_mysql` connection per request, with no +persistent connections and no proxy, will hit this ceiling at a few hundred +requests per second.** That is precisely the cost the in-process bridge and the +pooling proxy exist to remove, and A2 accidentally demonstrates it. Fixing the +lane (rather than deleting it) needs connection reuse, a widened +`ip_local_port_range`, or `tcp_tw_reuse` — all of which change what it measures, +so the lane should be re-scoped or dropped rather than patched into silence. + +## The Formerly-Broken Config (proxy STEP 0) — **fixed upstream** + +`db/configs/proxy-litewire-inprocess-BROKEN.toml` chains `[db.mysql]` (the +proxy) in front of the *same process's* in-process `[db.sqlite]` litewire, and +`db/bench-proxy.sh` runs it first ("STEP 0") on every invocation. + +**Historically (v0.6.1 and earlier)** this was a topology ePHPm could not +start: `start_db_proxies()` awaited the proxy's backend connect inline and the +litewire branch ran after it, so the proxy spent its entire ~40 s ten-attempt +backoff dialling a listener that could not exist yet, then gave up +**non-fatally and nearly silently** — the server went on serving HTTP with +nothing bound to the proxy port and every database page returning +`[2002] Connection refused`, while liveness and readiness both looked healthy. +That is the behaviour recorded in the "Still true in v0.6.1" notes on ePHPm's +[results page](https://ephpm.dev/benchmarking/results/). + +**It no longer reproduces.** Re-run on `v0.6.3-php8.5` (2026-08-18), STEP 0 +produced a *working* chain. The proxy now binds first and resolves its upstream +asynchronously: + +``` +INFO ephpm_db::mysql: MySQL proxy listening (upstream connect continues in the + background) listen=127.0.0.1:3306 upstream=127.0.0.1:3307 +INFO ephpm_server: SQLite MySQL wire protocol enabled listen=127.0.0.1:3307 +WARN ephpm_db::health: database proxy upstream connect failed: Connection + refused (os error 111) ... failures=1 +INFO ephpm_db::mysql: backend connection established after retry attempt=2 +``` + +One refused attempt, then connected ~250 ms later; `db.php` returned a real SQL +error (`no such table: bench` — STEP 0 never seeds) instead of +`[2002] Connection refused`. The inline-await ordering defect is gone. + +Two consequences, both of which should be carried upstream: + +1. **ephpm.dev's results page is stale on this point** — the "Still true in + v0.6.1" note describes behaviour that a v0.6.3 image does not exhibit. +2. **STEP 0 is no longer a gate.** It used to *prove* that the + proxy-vs-litewire lanes (B2/C2) had to use a separate litewire sidecar + container. That proof is gone; the sidecar is now a deliberate isolation + choice (it keeps the proxy and the backend in separate `--cpus 1` cgroups, + matching the other container-to-container lanes) rather than a forced one. + +The step is retained because it still archives +`db/results-proxy/FINDING-startup-order.log` every run, which is what caught +the change. The config's `engine` was switched from `"sqlite"` to `"turso"` in +the v0.7.0 pin bump; the fix above is in proxy startup sequencing and is +engine-independent, but note the two changes landed in the same run. ## The Bridge Suites (`bridge`, `wp-bridge`) @@ -219,8 +403,10 @@ the MySQL wire frontend serves**. Same dialect translation, same resultset protocol. The `bridge` suite measures what that deletion is worth. One container per -engine (rusqlite and Turso, `single-sqlite.toml` / `single-turso.toml`), six -cells each: {point-select, insert, wide-select} × {wire, bridge}. Wire and +engine — on the v0.7.0 pin that is the Turso lane alone (`single-turso.toml`); +the rusqlite lane (`single-sqlite.toml`) is opt-in and runs on v0.6.3, see +above — six cells each: {point-select, insert, wide-select} × {wire, bridge}. +Wire and bridge cells run against the **same process**, so nothing differs but the path. The wire cells keep their per-request PDO connect deliberately — that is what a real PHP request pays without persistent connections, and removing it @@ -248,15 +434,195 @@ cells): `wpdb` for the wire cells, `Ephpm\Db\WordPress\Db` for the bridge cells. A fallen-back bridge cell fails the gate instead of benchmarking the wire path under the wrong label. -**No lab numbers yet.** These suites landed with the v0.6.3 pin bump and have -not been recorded with this harness. For scale, the ephpm-side development -benches this week (dev box, WSL, LTO off — *not* this harness, *not* the -published image, do not put them in a table with anything above): a bridge -point-select ran ~61 µs on rusqlite and ~3.4 µs on the Turso engine, against -roughly 200 µs for the same query over the wire path, and WordPress pages -rendered 10–16% faster with the drop-in. Treat those as the hypothesis this -suite exists to check on a published image, not as results. Reference numbers -will be recorded on `ephpm/ephpm:v0.6.3-php8.5` and added here. +### Recorded: `bridge` on `ephpm/ephpm:v0.6.3-php8.5` (2026-08-18) + +First recording of this suite with this harness. Host: Windows 11, podman +machine 32 vCPU / 62 GiB, ePHPm container `--cpus 1`, `oha`, 8 s warmup plus +**2 × 15 s** timed reps per cell, machine load average 0.00–0.96 throughout. +**Every cell below was 100 % HTTP 200.** Mean of the two reps, RPS. + +Both lanes ran on the **same image**, differing only in `[db.sqlite] engine`, +so this is a clean rusqlite-vs-Turso A/B — the last release on which that +comparison is possible at all. + +| Cell | rusqlite | Turso | Turso vs rusqlite | +| --- | ---: | ---: | ---: | +| wire-point c=1 | 384.6 | 403.6 | +4.9 % | +| wire-point c=16 | 629.3 | 560.1 | −11.0 % ⚠ | +| wire-write c=1 | 771.0 | 718.0 | −6.9 % | +| wire-write c=16 | 1333.4 | 1239.7 | −7.0 % | +| wire-wide c=1 | 704.9 | 655.4 | −7.0 % | +| wire-wide c=16 | 1214.7 | 1104.0 | −9.1 % | +| bridge-point c=1 | 663.7 | 915.7 | **+38.0 %** | +| bridge-point c=16 | 1108.5 | 1585.8 | **+43.1 %** | +| bridge-write c=1 | 1054.5 | 1037.3 | −1.6 % | +| bridge-write c=16 | 1543.2 | 1651.2 | +7.0 % | +| bridge-wide c=1 | 899.3 | 944.3 | +5.0 % | +| bridge-wide c=16 | 1617.3 | 1624.4 | +0.4 % | + +⚠ `wire-point c=16` on Turso is the one noisy cell in the run: reps of 599.7 +and 520.5 (13 % spread). Per this file's own two-reps caveat, treat it as +unresolved rather than as a −11 % result. + +**What the bridge is worth** (same lane, wire vs its bridge twin): + +| | rusqlite | Turso | +| --- | ---: | ---: | +| point-select c=1 | 1.73× | **2.27×** | +| point-select c=16 | 1.76× | **2.83×** | +| insert c=1 | 1.37× | 1.44× | +| insert c=16 | 1.16× | 1.33× | +| wide-select c=1 | 1.28× | 1.44× | +| wide-select c=16 | 1.33× | 1.47× | + +Deleting the wire is worth 1.2–1.8× on rusqlite and 1.3–2.8× on Turso. The +engine choice barely moves the **wire** path (it is dominated by connect and +protocol cost) but moves the **bridge** path a lot — which is the expected +shape: the bridge is the only path where engine time is a large share of the +request. + +For scale, the earlier ephpm-side development benches (dev box, WSL, LTO off — +*not* this harness, *not* the published image, do not table them with the +above) measured a bridge point-select at ~61 µs on rusqlite and ~3.4 µs on +Turso against ~200 µs over the wire. Those in-process microbench ratios do +**not** survive to the HTTP level: end to end the bridge is worth 2.3–2.8× on +Turso, not 60×, because a full request is mostly PHP and HTTP, not SQL. + +**`wp-bridge` has still not been recorded** with this harness. + +### Recorded: `bridge` on `ephpm/ephpm:v0.7.0-php8.5` (2026-08-19) — **wire-path regression** + +`v0.7.0` published 2026-08-19, tag commit `c84e3c6`. Images verified pullable +before measuring: `v0.7.0-php8.3`, `-php8.4`, `-php8.5`, `v0.7.0`, `latest` — +all five present as manifest lists. Measured on +`docker.io/ephpm/ephpm:v0.7.0-php8.5`, +digest `sha256:c40689f2a8c019922fc1ed8a601a794d7ea5ccae117de7120738536730b49db8` +(`org.opencontainers.image.version = v0.7.0+php8.5.7`, `revision = c84e3c6…`). + +**Both arms of this table were recorded in one session, back to back, on an +otherwise idle box.** The v0.6.3 arm is a *fresh control re-run* +(`v0.6.3-php8.5`, digest `sha256:2f93bfbb…`), not the 2026-08-18 recording +above — a −20 % cross-session delta is exactly the kind of claim that host +drift can manufacture, so the control removes drift as an explanation rather +than arguing about it. The control reproduced the 2026-08-18 baseline on every +cell (e.g. `wire-point c=1` 416.4 vs 403.6; `bridge-point c=16` 1662.1 vs +1585.8), which is also a small piece of evidence that this harness is stable +across sessions. + +Method identical to the recording above: `--cpus 1`, `oha`, 8 s warmup plus +2 × 15 s timed reps, mean RPS. **All 24 v0.7.0 cells and all 24 control cells +were 100 % HTTP 200** (`db/parse.sh` flagged nothing). + +Load average on the podman machine (32 vCPU), sampled every 30 s and sliced by +**actual run boundaries** rather than by which sampler happened to be running: + +| Window | samples | min | median | p90 | max | +| --- | ---: | ---: | ---: | ---: | ---: | +| `bridge` v0.7.0 (21:17–21:24) | 21 | 1.22 | 1.49 | 1.76 | **1.82** | +| `bridge` v0.6.3 control (21:24–21:32) | 32 | 0.94 | 1.15 | 1.37 | **1.60** | +| `proxy` v0.7.0 (21:32–21:45) | 35 | 0.82 | 2.26 | 3.85 | 7.57 | +| after all runs (repro + analysis) | 69 | 0.00 | 0.20 | 4.83 | 13.36 | + +The two arms of the table below — the ones carrying the regression — ran +**quiet and tightly matched**: max 1.82 vs 1.60 on a 32 vCPU box, all of it the +benchmark's own two containers. No other workload ran; `ephpm` and `oha` were +the only processes above 1 % CPU. The `proxy` suite is the noisier one (two +`--cpus 1` containers plus an uncapped `oha`), and the 13.36 peak is entirely +*after* measurement finished — it is the A2 bug-repro load test and analysis, +not a measurement window. + +> **Correction to an earlier draft of this section.** It first reported "v0.7.0 +> run median 1.37 / p90 2.57 / max 7.57" and used that to claim the 2026-08-18 +> recording's "load average 0.00–0.96 throughout" had been sampled too sparsely. +> Both statements were wrong, and wrong in the same way: the two samplers +> outlived the runs they were started for, so their numbers mixed the `bridge` +> windows with the noisier `proxy` window. Sliced correctly, the `bridge` runs +> peak at 1.82 and 1.60 — so the 2026-08-18 figure is **plausible for this +> suite after all**, and the retraction of it is itself retracted. The 5–7 +> excursions are real but belong to `proxy`. Attributing a load figure to the +> wrong window is the same class of error as attributing a measurement to the +> wrong image; it is recorded here rather than quietly fixed. + +| Cell | v0.6.3 (control) | v0.7.0 | Δ | rep spread (v0.6.3 / v0.7.0) | +| --- | ---: | ---: | ---: | --- | +| wire-point c=1 | 416.4 | 318.8 | **−23.4 %** | 1.7 % / 2.0 % | +| wire-point c=16 | 640.1 | 440.3 | **−31.2 %** | 0.2 % / 3.7 % | +| wire-write c=1 | 755.8 | 638.6 | −15.5 % | 0.1 % / 1.2 % | +| wire-write c=16 | 1235.3 | 1004.8 | −18.7 % | 2.1 % / 4.7 % | +| wire-wide c=1 | 671.8 | 575.2 | −14.4 % | 1.5 % / 10.0 % | +| wire-wide c=16 | 1108.5 | 878.3 | **−20.8 %** | 1.6 % / 5.3 % | +| bridge-point c=1 | 1052.9 | 1020.4 | −3.1 % | 2.8 % / 2.2 % | +| bridge-point c=16 | 1662.1 | 1647.8 | −0.9 % | 0.2 % / 0.4 % | +| bridge-write c=1 | 1046.9 | 1034.0 | −1.2 % | 2.7 % / 3.3 % | +| bridge-write c=16 | 1620.3 | 1541.3 | −4.9 % | 4.3 % / 9.3 % | +| bridge-wide c=1 | 989.9 | 988.5 | −0.1 % | 2.8 % / 0.2 % | +| bridge-wide c=16 | 1629.3 | 1664.6 | +2.2 % | 3.0 % / 0.3 % | + +**The result is a clean split.** Every one of the six **bridge** cells is +within ±5 % — flat, at or below this harness's own rep-to-rep noise. Every one +of the six **wire** cells is down, by 14–31 %. Three wire cells clear this +file's "treat <20 % as unresolved" bar on their own (`wire-point` at both +concurrencies, `wire-wide c=16`); the other three sit at 14–19 %, under that +bar individually. But the bar exists to stop a single noisy cell being read as +a result, and this is not one cell: it is **6 of 6 wire cells moving the same +direction, in a run whose intra-cell spreads are 0.1–10 %, against a +same-session control that itself reproduces a prior-session recording.** +Collectively the wire regression is resolved. Individually, `wire-write c=1`, +`wire-write c=16` and `wire-wide c=1` are not. + +**Where the cost is.** The wire cells and their bridge twins run in the *same +process* against the *same* backend instance. The only thing a wire cell does +that its bridge twin does not is open a `pdo_mysql` connection to litewire's +MySQL frontend and speak the protocol. The bridge cells did not move; the wire +cells did. So the regression is in the **per-request connect + MySQL frontend +path**, not in the Turso engine, not in PHP, and not in ePHPm's HTTP layer — +any of which would have moved both halves together. + +The `proxy` suite recorded the same day agrees independently: its `A-lite-inproc` +lane (the same in-process wire path, different suite) is down 9.9–28.8 %, while +its pooled proxy lane `B2` — which reuses backend connections and so pays the +frontend handshake once per pooled connection rather than once per request — is +down only 0.9–7.3 %. + +**Hypothesis, not a finding.** Between the two pins, litewire moved +`e34c63928ed9` → `10345a869d27` (0.2.0) and turso moved `=0.7.0` → `=0.7.2`. +litewire's MySQL frontend gained, over that range, a `ConnectionAuthenticator` +path with a random per-connection scramble, an `opensrv-mysql` TLS feature +fence, a server-side tenant-session SQL screen, and the litewire#28–#31 +wire-fidelity fixes. Per-connection handshake work is the shape that would +produce exactly this signature — a cost paid once per connection, invisible to +the bridge, amortised away by pooling. **This has not been bisected and is not +established by this run.** The decisive next step is to bisect the litewire pin +against a fixed ePHPm build. + +**This is a shipped regression.** v0.7.0 is published. Any deployment using +stock `pdo_mysql` against the embedded engine — which is the documented default +integration — gets 14–31 % less throughput than v0.6.3 on these fixtures. +Applications on the `ephpm_db_*` bridge are unaffected. + +**What the bridge is worth**, per release (same lane, wire vs its bridge twin): + +| | v0.6.3 (control) | v0.7.0 | +| --- | ---: | ---: | +| point-select c=1 | 2.53× | 3.20× | +| point-select c=16 | 2.60× | 3.74× | +| insert c=1 | 1.39× | 1.62× | +| insert c=16 | 1.31× | 1.53× | +| wide-select c=1 | 1.47× | 1.72× | +| wide-select c=16 | 1.47× | 1.90× | + +> **Do not quote the v0.7.0 column as an improvement to the bridge.** The +> bridge did not get faster — every bridge cell is flat within noise. The +> multiplier grew because its *denominator* shrank. "The bridge is now worth +> 3.7×" and "the wire path lost 31 %" are the same measurement, and only the +> second one is news. + +The ~60× in-process microbench ratio still does **not** survive to the HTTP +level, and the v0.7.0 numbers reinforce that: end to end the bridge is worth +1.5–3.7×, not 60×, because a full request is mostly PHP and HTTP, not SQL. The +2.3–2.8× recorded on 2026-08-18 and the 2.5–2.6× measured on the control here +are the honest figure for v0.6.3; 3.2–3.7× is the v0.7.0 figure and it is +inflated by a regression rather than earned. ## Caveats @@ -281,12 +647,21 @@ will be recorded on `ephpm/ephpm:v0.6.3-php8.5` and added here. pessimistic against a hypothetical true no-pool build. - **The `engines` and `admission` numbers were recorded on the v0.6.0/v0.6.1 - lines and the harness defaults have since moved to v0.6.3.** As with the - v0.5.0 autotuning note in `RUNTIMES-BENCH.md`, a version bump changes the - effective configuration, so numbers recorded across a bump are not directly - comparable. Re-record rather than assume. -- **The `admission` suite needs v0.6.1 or later** — satisfied by the default - image since the v0.6.3 pin bump. The `write_permits` knob it sweeps merged in + lines and cannot be re-recorded.** As with the v0.5.0 autotuning note in + `RUNTIMES-BENCH.md`, a version bump changes the effective configuration, so + numbers recorded across a bump are not directly comparable — and here the + bump deleted the mechanism outright, so there is no later version to + re-record on. They are frozen evidence, not a baseline to compare against. + +- **A v0.7.0 number and a v0.6.3 number on any database path are an engine + comparison.** v0.7.0 has one embedded engine (Turso); v0.6.3's default was + the genuine-SQLite C engine. A Turso-lane-to-Turso-lane comparison across + the two releases *is* meaningful (same engine, different release); a + v0.6.3 rusqlite lane against a v0.7.0 lane is not, and the harness keeps + them apart deliberately. Say which you are reporting. + +- **The `admission` suite needs v0.6.1 or later** — satisfied by its + hard-pinned v0.6.3 image. The `write_permits` knob it sweeps merged in [ephpm#222](https://github.com/ephpm/ephpm/pull/222) and is not in v0.6.0 or earlier; on an older image the `baseline` row is all you get — which on its own demonstrates the collapse that motivated the knob. The suite gates on the diff --git a/README.md b/README.md index 6e736eb..3224b49 100644 --- a/README.md +++ b/README.md @@ -8,18 +8,32 @@ A reproducible Kubernetes lab for people deciding whether ePHPm belongs in their ## Relationship to ePHPm v0.7.0 -> **Historical (pre-v0.7.0).** The database suites pin `ephpm/ephpm:v0.6.3` and -> parts of them exercise machinery that **no longer exists upstream**. ePHPm -> v0.7.0 removed the rusqlite engine (`[db.sqlite] engine = "sqlite"` is now a -> **hard startup error**), the sqld sidecar, the `[db.sqlite.sqld] -> write_permits` admission knob, and the `cdc_experimental` knob; Turso is the -> only embedded engine and clustered replication runs over the in-process Turso -> CDC path. The `engines`, `admission`, and sqld-cluster lanes in -> [DB-BENCH.md](DB-BENCH.md) therefore run only against the pinned v0.6.3 -> image and **will not run against v0.7.0+ images**. Their recorded numbers are -> retained as the parity evidence behind the engine switch — the same way -> ePHPm's own [benchmarking results page](https://ephpm.dev/benchmarking/results/) -> marks those sections historical. +The lab's live pins are **`ephpm/ephpm:v0.7.0-php8.4`** (the Kubernetes suites) +and **`ephpm/ephpm:v0.7.0-php8.5`** (the single-host database tier). + +Two database suites are the exception and stay **hard-pinned to v0.6.3**, +because they measure machinery that **no longer exists upstream**. ePHPm +v0.7.0 removed the rusqlite engine (`[db.sqlite] engine = "sqlite"` is now a +**hard startup error**), the sqld sidecar, the `[db.sqlite.sqld] +write_permits` admission knob, and the `cdc_experimental` knob; Turso is the +only embedded engine and clustered replication runs over the in-process Turso +CDC path. The `engines` and `admission` suites in [DB-BENCH.md](DB-BENCH.md) +therefore **ignore `--image`** and run only against v0.6.3. Their recorded +numbers are retained as the parity evidence behind the engine switch — the +same way ePHPm's own +[benchmarking results page](https://ephpm.dev/benchmarking/results/) marks +those sections historical. The `bridge` and `wp-bridge` suites keep their +rusqlite lanes on the same footing: opt-in, and pinned to v0.6.3 when enabled. + +> **The v0.7.0 bump crosses an engine change, so read database deltas +> carefully.** Any v0.6.3-vs-v0.7.0 comparison on a database-touching path is +> comparing *two different engines*, not two versions of one. Paths that never +> reach the database (the `hello`/`cpu` runtime fixtures, the cache-heavy +> Laravel/native-KV lanes) are like-for-like; `db.php`, the bridge suites and +> WordPress-on-the-embedded-database are not. The +> `k8s/runtimes-bench.yaml` `bench-ephpm-turso` lane, which existed purely to +> A/B the engine knob, is retired at `replicas: 0` for the same reason: on +> v0.7.0 it would be the `bench-ephpm` lane wearing a second label. ## The Numbers diff --git a/RUNTIMES-BENCH.md b/RUNTIMES-BENCH.md index 7d84bba..fb861fe 100644 --- a/RUNTIMES-BENCH.md +++ b/RUNTIMES-BENCH.md @@ -11,18 +11,28 @@ from the ePHPm-lab report's next-tests list. | Runtime | Image | PHP | |---------|-------|-----| -| ePHPm v0.6.3 | `ephpm/ephpm:v0.6.3-php8.4` | 8.4 ZTS, glibc | +| ePHPm v0.7.0 | `ephpm/ephpm:v0.7.0-php8.4` | 8.4 ZTS, glibc | | nginx + php-fpm | `nginx:1.27-alpine` + `php:8.4-fpm` (Debian) | 8.4 NTS, glibc | | FrankenPHP | `dunglas/frankenphp:latest` | 8.5 ZTS, glibc (image default; see caveat) | | Swoole | `phpswoole/swoole:php8.4` | 8.4 NTS, glibc | | RoadRunner | `php:8.4-cli-alpine` + `ghcr.io/roadrunner-server/roadrunner:2024` | 8.4 NTS, musl (see caveat) | -| ePHPm v0.6.3 worker mode | `ephpm/ephpm:v0.6.3-php8.4` (`[php] mode = "worker"`) | 8.4 ZTS, glibc | +| ePHPm v0.7.0 worker mode | `ephpm/ephpm:v0.7.0-php8.4` (`[php] mode = "worker"`) | 8.4 ZTS, glibc | -The manifests pin **v0.6.3**, which carries the whole v0.4.x line +The manifests pin **v0.7.0**, which carries the whole v0.4.x line (v0.4.1: 101x db.php latency fix + SHA-NI; v0.4.2: HTTP `TCP_NODELAY` -13% p99, worker dispatch fastpath, mimalloc/LTO), v0.5.0's -**resource-aware autotuning**, and the v0.6.x database-path work -(pool fixes, the `ephpm_db_*` bridge — see `DB-BENCH.md`). +**resource-aware autotuning**, the v0.6.x database-path work +(pool fixes, the `ephpm_db_*` bridge — see `DB-BENCH.md`), and v0.7.0's +**engine swap**: the embedded database is now Turso only, the rusqlite +engine and the sqld sidecar are gone. + +> **The v0.7.0 pin bump crosses an engine change.** Nothing in *this* +> file's Class A / Class B fixtures (`hello`, `cpu`) touches the +> database, so those lanes are a like-for-like v0.6.3-vs-v0.7.0 +> comparison. The `db.php` lane is **not**: on v0.6.3 it ran the +> genuine-SQLite C engine and on v0.7.0 it runs Turso, so a delta there +> is an engine delta, not a runtime delta. Label it that way or do not +> report it. For the v0.4.0-vs-v0.4.1 before/after, see [docs/ephpm-0.4.1-retest.md](docs/ephpm-0.4.1-retest.md). The `db.php` lane (10 PDO queries on ePHPm's in-process SQLite) remains @@ -42,24 +52,26 @@ the reproduction path for the database-latency number. > expected to benefit most; tiny-script lanes (`hello`) should be > unchanged. Operator config still overrides any derived value. -## Experimental: Turso engine db lane (deployed, undriven) - -`k8s/runtimes-bench.yaml` carries a `bench-ephpm-turso` Deployment + -Service — identical to the ePHPm db.php lane but with the -**experimental** `[db.sqlite] engine = "turso"` knob (the Rust SQLite -rewrite). On the v0.6.3 pin the knob exists, so the lane ships -`replicas: 1` — but **no k6 Job drives it**: applying the manifest -deploys the pod and then nothing measures it. To measure it, confirm -the pod log shows the experimental-engine startup warning, then run the -db.php profile by hand against `http://bench-ephpm-turso:8080/db.php` -and compare with the `bench-ephpm` lane's db.php result. Context: -Phase 1 microbenchmarks at the litewire seam measured 28x point-SELECT -and 4x concurrent-writer throughput vs the C engine; this lane measures -what survives of that through the full mysqlnd → MySQL-wire → engine -path on a resource-limited pod. Note this lane collapses on any future -v0.7.0+ pin bump: Turso becomes the only engine there, so the knob (and -the lane's reason to exist) disappears — see the "Relationship to ePHPm -v0.7.0" section in the README. +## Retired: Turso engine db lane (`replicas: 0` on the v0.7.0 pin) + +`k8s/runtimes-bench.yaml` still carries the `bench-ephpm-turso` +Deployment + Service, now scaled to **`replicas: 0`**. It existed to A/B +the `[db.sqlite] engine = "turso"` knob against the genuine-SQLite C +engine that was the v0.6.x default, and the v0.6.3 pin is the last one on +which that A/B means anything. v0.7.0 removed the C engine — `"turso"` is +the only value the knob accepts and also the default — so this Deployment +and `bench-ephpm` would now select the **same engine**, leaving the +comparison with no control arm. It is kept scaled to zero rather than +deleted so the manifest records what the lane was; do not scale it back +up and report it as an engine comparison, because it would be +`bench-ephpm` wearing a second label. + +The question it was built to answer (Phase 1 microbenchmarks at the +litewire seam measured 28x point-SELECT and 4x concurrent-writer +throughput vs the C engine — how much survives the full mysqlnd → +MySQL-wire → engine path?) is now answered only in the historical +v0.6.3-pinned `engines` suite in [DB-BENCH.md](DB-BENCH.md). See the +"Relationship to ePHPm v0.7.0" section in the README. ## Class A vs Class B @@ -187,7 +199,8 @@ kubectl apply -f k8s/runtimes-bench.yaml kubectl delete job k6-bench-ephpm k6-bench-nginx-fpm k6-bench-frankenphp \ k6-bench-swoole k6-bench-rr k6-bench-ephpm-worker \ -n runtimes-bench --ignore-not-found -# Wait for all deployments (bench-ephpm-turso comes up too, but no job drives it): +# Wait for all deployments (bench-ephpm-turso is retired at replicas: 0, so +# its rollout status returns immediately): for d in bench-ephpm bench-ephpm-turso bench-nginx-fpm bench-frankenphp \ bench-swoole bench-rr bench-ephpm-worker; do kubectl rollout status deployment/$d -n runtimes-bench --timeout=300s @@ -225,7 +238,8 @@ k8s/runtimes-bench.yaml Single self-contained manifest: - ConfigMaps (fixtures, configs, k6 script) - 7 Deployments + 7 Services (five Class A/B runtimes, the ePHPm worker-mode lane, and - the undriven bench-ephpm-turso lane) + the retired bench-ephpm-turso lane at + replicas: 0) - 6 k6 Jobs (one per runtime lane plus the worker lane; bench-ephpm-turso has none) scripts/run-runtimes-bench.sh Driver: apply, wait, run jobs, print summaries diff --git a/db/bench-admission.sh b/db/bench-admission.sh index d1156d4..175ce94 100644 --- a/db/bench-admission.sh +++ b/db/bench-admission.sh @@ -35,7 +35,22 @@ # /usr/bin/grep explicitly. Raw oha output is kept per cell regardless. set -uo pipefail -BASE_IMG="${EPHPM_IMAGE:-docker.io/ephpm/ephpm:v0.6.3-php8.5}" +# --------------------------------------------------------------------------- +# HISTORICAL SUITE -- HARD-PINNED TO v0.6.3. DO NOT BUMP. +# +# Every lane here sweeps `[db.sqlite.sqld] write_permits`, a knob ePHPm +# REMOVED in v0.7.0 along with the sqld sidecar and the rusqlite engine +# the cluster configs select. On a v0.7.0 image the configs fail at +# startup (engine = "sqlite") and the knob no longer exists at all, so +# there is nothing left for this sweep to sweep. +# +# The pin is hardcoded rather than read from EPHPM_IMAGE for the same +# reason the suite already has a startup-log gate: the top-level driver +# now defaults to a v0.7.0 image, and silently inheriting it would either +# kill every lane or (worse, if a config ever became startable) benchmark +# a lane wearing a label whose mechanism is gone. +# --------------------------------------------------------------------------- +BASE_IMG="${EPHPM_ADMISSION_BASE_IMAGE:-docker.io/ephpm/ephpm:v0.6.3-php8.5}" ADM_IMG="${EPHPM_ADMISSION_IMAGE:-$BASE_IMG}" OHA=ghcr.io/hatoo/oha:latest CURL=docker.io/curlimages/curl:latest diff --git a/db/bench-bridge.sh b/db/bench-bridge.sh index 2f5d798..93cd19b 100644 --- a/db/bench-bridge.sh +++ b/db/bench-bridge.sh @@ -1,8 +1,11 @@ #!/usr/bin/env bash # In-process bridge vs MySQL wire, same engine, same process. # -# A sqlite rusqlite in-process (production default) -# B turso Turso engine in-process (experimental) +# A sqlite rusqlite in-process -- REMOVED in ePHPm v0.7.0. Opt-in via +# BRIDGE_LEGACY_SQLITE=1, and then only on the pinned v0.6.3 +# image; it is a historical row, not a lane of this run. +# B turso Turso engine in-process -- the only engine from v0.7.0 on, +# and the default lane of this suite. # # Each lane runs ONE container and measures six cells against it: # @@ -33,7 +36,14 @@ # measurement again. set -uo pipefail -IMG="${EPHPM_IMAGE:-docker.io/ephpm/ephpm:v0.6.3-php8.5}" +# The Turso lane runs on the current pin. The rusqlite lane CANNOT: v0.7.0 +# removed that engine and `engine = "sqlite"` is a hard startup error, so +# lane A is opt-in (BRIDGE_LEGACY_SQLITE=1) and hard-pinned to the last +# image that has the engine. Two lanes on two different ePHPm versions are +# NOT an engine A/B -- they differ by a whole release. If you enable lane A, +# report it as a separate historical row, never in one table with lane B. +IMG="${EPHPM_IMAGE:-docker.io/ephpm/ephpm:v0.7.0-php8.5}" +LEGACY_IMG="${EPHPM_LEGACY_IMAGE:-docker.io/ephpm/ephpm:v0.6.3-php8.5}" OHA=ghcr.io/hatoo/oha:latest CURL=docker.io/curlimages/curl:latest NET=dbbench-net @@ -89,9 +99,22 @@ gate() { # description urlpath expected-substring esac } -run_lane() { # lane cfg - LANE="$1"; local cfg="$2" +run_lane() { # lane cfg img + LANE="$1"; local cfg="$2" img="$3" echo ""; echo "############ LANE $LANE ($cfg, single-node, --cpus $CPUS) ############" + echo " image: $img" + # Purge THIS lane's files before measuring. $OUT persists across runs, and a + # lane that is not run this time (lane A is opt-in) leaves its previous + # session's files sitting next to this session's -- parse.sh then prints one + # table mixing two runs on two images with nothing to distinguish them. That + # happened on 2026-08-19: an archived "v0.7.0" directory still carried + # v0.6.3 rusqlite rows. Stale results that look current are exactly the + # failure this suite's gates exist to prevent. + rm -f "$OUT/${LANE}-"*.txt + { + echo "lane=$LANE image=$img config=$cfg" + echo "recorded=$(date -u +%Y-%m-%dT%H:%M:%SZ) host=$(hostname) dur=$DUR reps=$REPS cpus=$CPUS" + } > "$OUT/RUN-INFO-${LANE}.txt" cleanup podman volume rm -f "dbv-$LANE" >/dev/null 2>&1 || true podman volume create "dbv-$LANE" >/dev/null @@ -100,7 +123,7 @@ run_lane() { # lane cfg -v "$HERE/fixtures/bridge:/var/www/html/bridge:ro" \ -v "$HERE/configs/$cfg:/etc/ephpm/ephpm.toml:ro" \ -v "dbv-$LANE:/data" \ - "$IMG" >/dev/null + "$img" >/dev/null if ! wait_ready; then echo "!! $LANE never became ready:"; podman logs dbbench-c1 2>&1 | tail -40; return 1 fi @@ -129,8 +152,16 @@ run_lane() { # lane cfg } FAILED=0 -run_lane A-sqlite single-sqlite.toml || FAILED=1 -run_lane B-turso single-turso.toml || FAILED=1 +# Lane A is the removed rusqlite engine: opt-in, and on its own pinned image. +if [ "${BRIDGE_LEGACY_SQLITE:-0}" = 1 ]; then + if [ "$LEGACY_IMG" != "$IMG" ]; then + echo "!! lane A-sqlite runs on $LEGACY_IMG, lane B-turso on $IMG." + echo "!! Those differ by a whole ePHPm release, not just an engine: report" + echo "!! A as a separate historical row, never in one table with B." + fi + run_lane A-sqlite single-sqlite.toml "$LEGACY_IMG" || FAILED=1 +fi +run_lane B-turso single-turso.toml "$IMG" || FAILED=1 echo "" if [ "$FAILED" = 1 ]; then diff --git a/db/bench-engines.sh b/db/bench-engines.sh index b574de7..8b5f0e1 100644 --- a/db/bench-engines.sh +++ b/db/bench-engines.sh @@ -17,7 +17,30 @@ # measurement again. set -uo pipefail -IMG="${EPHPM_IMAGE:-docker.io/ephpm/ephpm:v0.6.3-php8.5}" +# --------------------------------------------------------------------------- +# HISTORICAL SUITE -- HARD-PINNED TO v0.6.3. DO NOT BUMP. +# +# Three of this matrix's four lanes exercise machinery ePHPm REMOVED in +# v0.7.0, so this suite is not runnable on a current image: +# A (single-sqlite.toml) engine = "sqlite" -> hard startup error +# C (cluster-sqlite-*.toml) engine = "sqlite" + sqld sidecar -> gone +# D (cluster-turso-*.toml) replication.cdc_experimental -> knob removed; +# ephpm-config does not reject unknown fields, +# so on v0.7.0 that line is silently ignored +# and lane D would be a DIFFERENT topology +# wearing lane D's label. +# Only lane B (Turso single-node) would survive, and a one-lane "engine +# comparison" has no control arm. +# +# The pin is therefore hardcoded rather than read from EPHPM_IMAGE: the +# top-level driver (scripts/run-db-bench.sh) now defaults to a v0.7.0 +# image, and inheriting it here would produce three dead lanes and one +# mislabelled one. Replacing this suite for v0.7.0 means a NEW +# Turso-single vs Turso-CDC-clustered matrix, not edits to these lanes. +# EPHPM_ENGINES_IMAGE is the deliberate override if you know why you want +# one. +# --------------------------------------------------------------------------- +IMG="${EPHPM_ENGINES_IMAGE:-docker.io/ephpm/ephpm:v0.6.3-php8.5}" OHA=ghcr.io/hatoo/oha:latest CURL=docker.io/curlimages/curl:latest NET=dbbench-net diff --git a/db/bench-proxy.sh b/db/bench-proxy.sh index a33f270..fea8367 100644 --- a/db/bench-proxy.sh +++ b/db/bench-proxy.sh @@ -38,7 +38,7 @@ # --------------------------------------------------------------------------- set -uo pipefail -IMG="${EPHPM_IMAGE:-docker.io/ephpm/ephpm:v0.6.3-php8.5}" +IMG="${EPHPM_IMAGE:-docker.io/ephpm/ephpm:v0.7.0-php8.5}" OHA=ghcr.io/hatoo/oha:latest CURL=docker.io/curlimages/curl:latest NET=dbbench-net @@ -241,43 +241,80 @@ echo " db.php on the chained config: $(get http://dbbench-c1:8080/db.php)" cleanup # ---------------------------------------------------------------- lanes -run_lane A-lite-inproc single-sqlite.toml sqlite "" noleak "$LW" 'listen=127.0.0.1:3306' +# The litewire lanes run on the Turso engine. Before v0.7.0 they used +# rusqlite (single-sqlite.toml / litewire-sidecar-sqlite.toml) with a +# separate Turso lane bolted on the end; v0.7.0 removed rusqlite, so the +# whole group standardises on the one surviving engine. That is also what +# makes this suite comparable ACROSS the v0.6.3 -> v0.7.0 bump: both +# releases accept engine = "turso", so a delta here is a proxy/pool delta +# rather than an engine delta. +run_lane A-lite-inproc single-turso.toml sqlite "" noleak "$LW" 'listen=127.0.0.1:3306' -start_lw_node litewire-sidecar-sqlite.toml || exit 1 +start_lw_node litewire-sidecar-turso.toml || exit 1 run_lane A2-lite-remote proxy-none-direct.toml sqlite "" noleak "!$MYP" "!$PGP" "!$LW" -- \ DB_HOST=dbbench-lw DB_PORT=3306 run_lane B2-lite-proxy-pool proxy-litewire-pool.toml sqlite "" leak "$MYP" '!SQLite MySQL wire' run_lane C2-lite-proxy-nopool proxy-litewire-nopool.toml sqlite "" leak "$MYP" '!SQLite MySQL wire' stop_lw_node -run_lane D-mysql-proxy-pool proxy-mysql-pool.toml mysql verify_mysql_upstream leak "$MYP" -run_lane E-mysql-proxy-nopool proxy-mysql-nopool.toml mysql verify_mysql_upstream leak "$MYP" -run_lane F-pg-proxy-pool proxy-postgres-pool.toml postgres verify_pg_upstream noleak "$PGP" -run_lane G-pg-proxy-nopool proxy-postgres-nopool.toml postgres verify_pg_upstream noleak "$PGP" -run_lane H-mysql-direct proxy-none-direct.toml mysql verify_mysql_upstream noleak "!$MYP" "!$PGP" "!$LW" -- \ - DB_HOST=dbbench-mysql DB_PORT=3306 DB_NAME=bench DB_USER=root DB_PASSWORD= -run_lane I-pg-direct proxy-none-direct.toml postgres verify_pg_upstream noleak "!$MYP" "!$PGP" "!$LW" -- \ - DB_HOST=dbbench-pg DB_PORT=5432 DB_NAME=bench DB_USER=postgres DB_PASSWORD=bench +# The real-server lanes need the mysql:8 / postgres:16 upstreams to already +# be running on $NET -- this suite does not create them (see DB-BENCH.md). +# Check ONCE and say so, instead of letting six lanes each discover it +# separately and print "!! FIXTURE WRONG -- lane invalid". Those messages +# read like a product failure; the actual cause is a missing prerequisite, +# and the two must not look the same in a results log. +upstream_running() { podman container exists "$1" 2>/dev/null; } -start_lw_node litewire-sidecar-turso.toml && { - run_lane J2-turso-proxy-pool proxy-litewire-pool.toml sqlite "" leak "$MYP" - stop_lw_node -} +if upstream_running dbbench-mysql; then + run_lane D-mysql-proxy-pool proxy-mysql-pool.toml mysql verify_mysql_upstream leak "$MYP" + run_lane E-mysql-proxy-nopool proxy-mysql-nopool.toml mysql verify_mysql_upstream leak "$MYP" + run_lane H-mysql-direct proxy-none-direct.toml mysql verify_mysql_upstream noleak "!$MYP" "!$PGP" "!$LW" -- \ + DB_HOST=dbbench-mysql DB_PORT=3306 DB_NAME=bench DB_USER=root DB_PASSWORD= +else + echo "" + echo "== SKIPPING lanes D/E/H: no 'dbbench-mysql' container on $NET." + echo " These lanes need a real MySQL upstream; start it first, e.g." + echo " podman run -d --name dbbench-mysql --network $NET --cpus 4 \\" + echo " -e MYSQL_ALLOW_EMPTY_PASSWORD=1 -e MYSQL_DATABASE=bench \\" + echo " docker.io/library/mysql:8" + echo " SKIPPED is not a measurement and not a failure -- it is absence." +fi + +if upstream_running dbbench-pg; then + run_lane F-pg-proxy-pool proxy-postgres-pool.toml postgres verify_pg_upstream noleak "$PGP" + run_lane G-pg-proxy-nopool proxy-postgres-nopool.toml postgres verify_pg_upstream noleak "$PGP" + run_lane I-pg-direct proxy-none-direct.toml postgres verify_pg_upstream noleak "!$MYP" "!$PGP" "!$LW" -- \ + DB_HOST=dbbench-pg DB_PORT=5432 DB_NAME=bench DB_USER=postgres DB_PASSWORD=bench +else + echo "" + echo "== SKIPPING lanes F/G/I and F24-pg-cliff: no 'dbbench-pg' container on $NET." + echo " Start it first, e.g." + echo " podman run -d --name dbbench-pg --network $NET --cpus 4 \\" + echo " -e POSTGRES_PASSWORD=bench -e POSTGRES_DB=bench \\" + echo " docker.io/library/postgres:16" +fi + +# (The old J2-turso-proxy-pool lane lived here. It ran proxy-litewire-pool.toml +# against a Turso sidecar while B2 ran the same config against a rusqlite one. +# Now that the whole litewire group is Turso, J2 and B2 are the same lane, so +# J2 is gone rather than measured twice under two names.) # PG session-pinning cliff probe: pool ON at the SHIPPED default cap of 20 # backend connections, swept past it. A pinned-session proxy with a cap of # 20 cannot serve 24 concurrent PHP requests without queueing on # pool_timeout, so this is where the cliff would show if it exists. -banner "F24-pg-cliff (max_connections = 20, shipped default)" -start_node proxy-postgres-pool-default20.toml postgres -if wait_ready; then - check_log F24-pg-cliff "$PGP" - fixture_gate && { - measure F24-pg-cliff db.php "16 20 24 32" - measure F24-pg-cliff write.php "16 24" - } +if upstream_running dbbench-pg; then + banner "F24-pg-cliff (max_connections = 20, shipped default)" + start_node proxy-postgres-pool-default20.toml postgres + if wait_ready; then + check_log F24-pg-cliff "$PGP" + fixture_gate && { + measure F24-pg-cliff db.php "16 20 24 32" + measure F24-pg-cliff write.php "16 24" + } + fi + cleanup fi -cleanup echo "" echo "=== all lanes done; raw output in $OUT ===" diff --git a/db/bench-wordpress-bridge.sh b/db/bench-wordpress-bridge.sh index f400c80..817282c 100644 --- a/db/bench-wordpress-bridge.sh +++ b/db/bench-wordpress-bridge.sh @@ -2,8 +2,11 @@ # WordPress on the embedded database: mysqli wire vs the # ephpm/db-wordpress drop-in (wp-content/db.php), per engine. # -# wp-sqlite rusqlite engine (production default) -# wp-turso Turso engine (experimental) +# wp-sqlite rusqlite engine -- REMOVED in ePHPm v0.7.0. Opt-in via +# WP_BRIDGE_LEGACY_SQLITE=1, and then only on the pinned +# v0.6.3 image; a historical row, not a lane of this run. +# wp-turso Turso engine -- the only engine from v0.7.0 on, and the +# default lane of this suite. # # Each lane is ONE ePHPm container serving a real WordPress install from # the embedded database, measured twice on two pages: @@ -34,7 +37,11 @@ # /usr/bin/grep explicitly, and raw oha output is kept in results/. set -uo pipefail -IMG="${EPHPM_IMAGE:-docker.io/ephpm/ephpm:v0.6.3-php8.5}" +# See bench-bridge.sh: the rusqlite lane cannot run on v0.7.0+ (hard startup +# error), so it is opt-in and hard-pinned to the last image that has it. +# Two lanes on two ePHPm releases are not an engine A/B -- keep them apart. +IMG="${EPHPM_IMAGE:-docker.io/ephpm/ephpm:v0.7.0-php8.5}" +LEGACY_IMG="${EPHPM_LEGACY_IMAGE:-docker.io/ephpm/ephpm:v0.6.3-php8.5}" OHA=ghcr.io/hatoo/oha:latest CURL=docker.io/curlimages/curl:latest WPCLI=docker.io/library/wordpress:cli @@ -162,9 +169,10 @@ measure() { # cellname url done } -run_lane() { # lane cfg - LANE="$1"; local cfg="$2" +run_lane() { # lane cfg img + LANE="$1"; local cfg="$2" img="$3" echo ""; echo "############ LANE $LANE ($cfg, --cpus $CPUS) ############" + echo " image: $img" cleanup podman volume rm -f "dbv-$LANE" >/dev/null 2>&1 || true podman volume create "dbv-$LANE" >/dev/null @@ -172,7 +180,7 @@ run_lane() { # lane cfg -v "$HTMLVOL:/var/www/html" \ -v "$HERE/configs/$cfg:/etc/ephpm/ephpm.toml:ro" \ -v "dbv-$LANE:/data" \ - "$IMG" >/dev/null + "$img" >/dev/null if ! wait_db; then echo "!! $LANE MySQL frontend never became ready:"; podman logs wpbridge 2>&1 | tail -40; return 1 fi @@ -206,8 +214,14 @@ run_lane() { # lane cfg } FAILED=0 -run_lane wp-sqlite wp-bridge-sqlite.toml || FAILED=1 -run_lane wp-turso wp-bridge-turso.toml || FAILED=1 +if [ "${WP_BRIDGE_LEGACY_SQLITE:-0}" = 1 ]; then + if [ "$LEGACY_IMG" != "$IMG" ]; then + echo "!! lane wp-sqlite runs on $LEGACY_IMG, lane wp-turso on $IMG --" + echo "!! a whole release apart. Separate historical row, never one table." + fi + run_lane wp-sqlite wp-bridge-sqlite.toml "$LEGACY_IMG" || FAILED=1 +fi +run_lane wp-turso wp-bridge-turso.toml "$IMG" || FAILED=1 echo "" if [ "$FAILED" = 1 ]; then diff --git a/db/configs/proxy-litewire-inprocess-BROKEN.toml b/db/configs/proxy-litewire-inprocess-BROKEN.toml index 5038a1c..8bf273c 100644 --- a/db/configs/proxy-litewire-inprocess-BROKEN.toml +++ b/db/configs/proxy-litewire-inprocess-BROKEN.toml @@ -22,11 +22,15 @@ query_stats = false [db.sqlite] path = "/data/bench.db" -engine = "sqlite" +# Turso, not rusqlite: v0.7.0 removed the rusqlite engine and rejects +# engine = "sqlite" at startup. This lane must fail for the ORDERING +# reason it exists to demonstrate, not for an engine-validation reason, +# so it uses the engine that is valid on both v0.6.3 and v0.7.0. +engine = "turso" [db.sqlite.proxy] # litewire moves off 3306 so the ephpm-db proxy can own the port PHP -# connects to. PHP -> :3306 (proxy) -> :3307 (litewire) -> rusqlite. +# connects to. PHP -> :3306 (proxy) -> :3307 (litewire) -> Turso. mysql_listen = "127.0.0.1:3307" [db.mysql] diff --git a/db/probe-clean-vs-dirty.sh b/db/probe-clean-vs-dirty.sh index 680b74c..4624ebf 100644 --- a/db/probe-clean-vs-dirty.sh +++ b/db/probe-clean-vs-dirty.sh @@ -26,7 +26,7 @@ set -uo pipefail G=/usr/bin/grep NET=dbbench-net CURL=docker.io/curlimages/curl:latest -IMG="${EPHPM_IMAGE:-docker.io/ephpm/ephpm:v0.6.3-php8.5}" +IMG="${EPHPM_IMAGE:-docker.io/ephpm/ephpm:v0.7.0-php8.5}" HERE="$(cd "$(dirname "$0")" && pwd)" OUT="$HERE/results-proxy" get() { podman run --rm --network "$NET" "$CURL" -s --max-time 20 "$1" 2>/dev/null; } diff --git a/db/probe-pg.sh b/db/probe-pg.sh index 5d462c9..7366f9d 100644 --- a/db/probe-pg.sh +++ b/db/probe-pg.sh @@ -19,7 +19,7 @@ set -uo pipefail G=/usr/bin/grep NET=dbbench-net CURL=docker.io/curlimages/curl:latest -IMG="${EPHPM_IMAGE:-docker.io/ephpm/ephpm:v0.6.3-php8.5}" +IMG="${EPHPM_IMAGE:-docker.io/ephpm/ephpm:v0.7.0-php8.5}" HERE="$(cd "$(dirname "$0")" && pwd)" OUT="$HERE/results-proxy" get() { podman run --rm --network "$NET" "$CURL" -s --max-time 25 "$1" 2>/dev/null; } diff --git a/db/probe-reset.sh b/db/probe-reset.sh index fe518d5..c09a594 100644 --- a/db/probe-reset.sh +++ b/db/probe-reset.sh @@ -25,7 +25,7 @@ set -uo pipefail G=/usr/bin/grep NET=dbbench-net CURL=docker.io/curlimages/curl:latest -IMG="${EPHPM_IMAGE:-docker.io/ephpm/ephpm:v0.6.3-php8.5}" +IMG="${EPHPM_IMAGE:-docker.io/ephpm/ephpm:v0.7.0-php8.5}" HERE="$(cd "$(dirname "$0")" && pwd)" OUT="$HERE/results-proxy" TARGET="${1:-lite}" diff --git a/k8s/OPCACHE-CLUSTER.md b/k8s/OPCACHE-CLUSTER.md index d821b70..37a6372 100644 --- a/k8s/OPCACHE-CLUSTER.md +++ b/k8s/OPCACHE-CLUSTER.md @@ -5,7 +5,9 @@ invalidation feature and compare it to the php-fpm equivalent (rolling restart). Both require an image containing ePHPm >= **v0.4.0** (the release that -ships cluster invalidation). The manifests pin `ephpm/ephpm:v0.5.0-php8.4`. +ships cluster invalidation). The manifests pin `ephpm/ephpm:v0.7.0-php8.4`. +(This line read `v0.5.0-php8.4` while the manifest already pinned `v0.6.3`; +it is now derived from `opcache-cluster.yaml` rather than restated.) For a local kind cluster with a locally-loaded RC image, override the image used by the blip test with `EPHPM_IMAGE`: diff --git a/k8s/krayin-v3.yaml b/k8s/krayin-v3.yaml index 64aae64..e52d952 100644 --- a/k8s/krayin-v3.yaml +++ b/k8s/krayin-v3.yaml @@ -526,7 +526,7 @@ spec: # on PHP 8.4. mb_split()/mb_ereg*() (Laravel's Str helpers) are # also compiled in from this release on - the shim in # prepare-app.sh is no longer required. - image: ephpm/ephpm:v0.6.3-php8.4 + image: ephpm/ephpm:v0.7.0-php8.4 command: ["sh", "/scripts/ephpm-start.sh"] ports: - containerPort: 8080 @@ -607,7 +607,7 @@ spec: mountPath: /scripts containers: - name: ephpm-worker - image: ephpm/ephpm:v0.6.3-php8.4 + image: ephpm/ephpm:v0.7.0-php8.4 command: ["sh", "/scripts/ephpm-worker-start.sh"] ports: - containerPort: 8080 diff --git a/k8s/laravel-v4.yaml b/k8s/laravel-v4.yaml index 4ef2d7d..e542ba2 100644 --- a/k8s/laravel-v4.yaml +++ b/k8s/laravel-v4.yaml @@ -634,7 +634,7 @@ spec: # ePHPm >= 0.3.0 ships worker mode in the published image - no # source build needed. One binary per PHP minor; 8.4 matches the # PHP-FPM side of this lab. - image: ephpm/ephpm:v0.6.3-php8.4 + image: ephpm/ephpm:v0.7.0-php8.4 command: ["sh", "/scripts/ephpm-worker-start.sh"] env: - name: RUNTIME_NAME @@ -728,7 +728,7 @@ spec: # Earlier images compiled OPcache in but did not start it correctly # for the embedded runtime, which made request-mode framework tests # look much worse than they do now. - image: ephpm/ephpm:v0.6.3-php8.4 + image: ephpm/ephpm:v0.7.0-php8.4 command: ["sh", "/scripts/ephpm-start.sh"] env: - name: RUNTIME_NAME diff --git a/k8s/opcache-cluster.yaml b/k8s/opcache-cluster.yaml index 47c4880..410787f 100644 --- a/k8s/opcache-cluster.yaml +++ b/k8s/opcache-cluster.yaml @@ -171,7 +171,7 @@ spec: - name: ephpm # OPcache clustering ships in 0.4.0 (0.3.x has the watcher's # prerequisites — active OPcache — but not the invalidation). - image: ephpm/ephpm:v0.6.3-php8.4 + image: ephpm/ephpm:v0.7.0-php8.4 command: ["sh", "/files/start.sh"] env: - name: POD_IP diff --git a/k8s/php-benchmark-v2.yaml b/k8s/php-benchmark-v2.yaml index 4b2cf8b..0ae7a23 100644 --- a/k8s/php-benchmark-v2.yaml +++ b/k8s/php-benchmark-v2.yaml @@ -431,7 +431,7 @@ spec: spec: containers: - name: ephpm - image: ephpm/ephpm:v0.6.3-php8.4 + image: ephpm/ephpm:v0.7.0-php8.4 ports: - containerPort: 8080 resources: diff --git a/k8s/php-benchmark.yaml b/k8s/php-benchmark.yaml index c41b9fc..f4d0595 100644 --- a/k8s/php-benchmark.yaml +++ b/k8s/php-benchmark.yaml @@ -154,7 +154,7 @@ spec: spec: containers: - name: ephpm - image: ephpm/ephpm:v0.6.3-php8.4 + image: ephpm/ephpm:v0.7.0-php8.4 ports: - containerPort: 8080 resources: diff --git a/k8s/runtimes-bench.yaml b/k8s/runtimes-bench.yaml index ff4ad50..c188215 100644 --- a/k8s/runtimes-bench.yaml +++ b/k8s/runtimes-bench.yaml @@ -95,10 +95,12 @@ data: mysql_listen = "127.0.0.1:3306" # --------------------------------------------------------------------------- -# ePHPm config variant: EXPERIMENTAL Turso Database engine for the db lane. -# Identical to bench-ephpm-config except [db.sqlite] engine = "turso". -# The engine knob shipped in v0.6.0, which this manifest now pins, so the -# lane is live — see the bench-ephpm-turso Deployment comment. +# ePHPm config variant: Turso engine for the db lane. RETIRED on the v0.7.0 +# pin. Identical to bench-ephpm-config except the now-redundant +# [db.sqlite] engine = "turso" line: on v0.7.0 "turso" is the only value the +# knob accepts, so this ConfigMap and bench-ephpm-config select the same +# engine. Kept (with the Deployment scaled to 0) as the record of what the +# A/B lane was — see the bench-ephpm-turso Deployment comment. # --------------------------------------------------------------------------- --- apiVersion: v1 @@ -407,7 +409,7 @@ spec: spec: containers: - name: ephpm - image: ephpm/ephpm:v0.6.3-php8.4 + image: ephpm/ephpm:v0.7.0-php8.4 ports: - containerPort: 8080 resources: @@ -454,17 +456,16 @@ spec: targetPort: 8080 # --------------------------------------------------------------------------- -# 1b. ePHPm + EXPERIMENTAL Turso engine — db lane only. The `[db.sqlite] -# engine` knob shipped in v0.6.0 and this manifest pins v0.6.3, so the -# lane is enabled (replicas: 1). NOTE: no k6 Job drives this lane — -# it is deployed but undriven; measure it by hand as described below. -# Verify the engine actually engaged before trusting any number: the -# pod log must show the experimental-engine startup warning. Then run -# the db.php k6 profile against http://bench-ephpm-turso:8080/db.php -# and compare against the bench-ephpm lane's db.php result. -# The engine is still upstream-Beta and single-node only; ePHPm rejects -# `engine = "turso"` combined with clustered SQLite at startup. For the -# single-host measurement of the same question, see DB-BENCH.md. +# 1b. ePHPm + Turso engine — RETIRED (replicas: 0) on the v0.7.0 pin. +# This lane existed to A/B the `[db.sqlite] engine` knob against the +# genuine-SQLite C engine that was the v0.6.x default. v0.7.0 removed +# that engine: "turso" is now the only accepted value and the default, +# so this Deployment and `bench-ephpm` would run the SAME engine and the +# A/B has no control arm left. It is scaled to 0 rather than deleted so +# the manifest still records what the lane was; its recorded v0.6.x +# numbers stay in DB-BENCH.md as historical. +# Do NOT scale it back to 1 and report it as an engine comparison on a +# v0.7.0+ pin — it would be `bench-ephpm` wearing a second label. # --------------------------------------------------------------------------- --- apiVersion: apps/v1 @@ -476,7 +477,8 @@ metadata: bench-class: a runtime: ephpm-turso spec: - replicas: 1 + # RETIRED on the v0.7.0 pin - see the comment block above. + replicas: 0 selector: matchLabels: app: bench-ephpm-turso @@ -489,7 +491,7 @@ spec: spec: containers: - name: ephpm - image: ephpm/ephpm:v0.6.3-php8.4 + image: ephpm/ephpm:v0.7.0-php8.4 ports: - containerPort: 8080 resources: @@ -1242,7 +1244,7 @@ spec: spec: containers: - name: ephpm - image: ephpm/ephpm:v0.6.3-php8.4 + image: ephpm/ephpm:v0.7.0-php8.4 ports: - containerPort: 8080 resources: diff --git a/k8s/wordpress-v5.yaml b/k8s/wordpress-v5.yaml index c47de83..7e299a1 100644 --- a/k8s/wordpress-v5.yaml +++ b/k8s/wordpress-v5.yaml @@ -299,7 +299,7 @@ spec: mountPath: /scripts containers: - name: ephpm - image: ephpm/ephpm:v0.6.3-php8.4 + image: ephpm/ephpm:v0.7.0-php8.4 command: ["sh", "/scripts/ephpm-start.sh"] env: - name: CACHE_BACKEND @@ -400,7 +400,7 @@ spec: mountPath: /scripts containers: - name: ephpm-worker - image: ephpm/ephpm:v0.6.3-php8.4 + image: ephpm/ephpm:v0.7.0-php8.4 command: ["sh", "/scripts/ephpm-worker-start.sh"] env: - name: CACHE_BACKEND diff --git a/scripts/run-db-bench.sh b/scripts/run-db-bench.sh index 1e1ca19..ebdb1a8 100644 --- a/scripts/run-db-bench.sh +++ b/scripts/run-db-bench.sh @@ -5,13 +5,21 @@ # Usage: # ./scripts/run-db-bench.sh [--image IMG] [--dur 15s] [--reps 2] # -# suite = engines 4-lane SQLite/Turso matrix (single-node vs clustered) -# admission sqld write-admission sweep (write_permits 1/2/4/8) +# suite = engines HISTORICAL, v0.6.3-pinned. 4-lane SQLite/Turso matrix +# admission HISTORICAL, v0.6.3-pinned. sqld write_permits sweep # proxy DB-proxy cost/benefit matrix (hop vs pooling) -# bridge in-process ephpm_db_* vs MySQL wire, per engine +# bridge in-process ephpm_db_* vs MySQL wire (Turso) # wp-bridge WordPress: db-wordpress drop-in vs mysqli wire # all all five, in that order # +# `engines` and `admission` exercise the rusqlite engine, the sqld sidecar +# and the write_permits knob -- all REMOVED in ePHPm v0.7.0. They ignore +# --image/EPHPM_IMAGE and stay hard-pinned to v0.6.3 so that bumping the +# default below cannot turn them into dead or mislabelled lanes. Their +# recorded numbers are the historical parity evidence behind the engine +# switch; replacing them for v0.7.0 means a new Turso-single vs +# Turso-CDC-clustered matrix, not edits to those lanes. See DB-BENCH.md. +# # Unlike the k6/Kubernetes suites in k8s/, these run on ONE host under # podman. That is deliberate: the effects being measured (a wire-protocol # hop, a connection-pool checkout, a write-admission semaphore) are tens @@ -33,7 +41,7 @@ SUITE="${1:-}" [ -n "$SUITE" ] || { sed -n '2,20p' "${BASH_SOURCE[0]}"; exit 2; } shift -IMAGE="${EPHPM_IMAGE:-docker.io/ephpm/ephpm:v0.6.3-php8.5}" +IMAGE="${EPHPM_IMAGE:-docker.io/ephpm/ephpm:v0.7.0-php8.5}" DUR="${DUR:-15s}" REPS="${REPS:-2}" while [ $# -gt 0 ]; do @@ -47,10 +55,10 @@ done export EPHPM_IMAGE="$IMAGE" DUR REPS -run_suite() { # name script resultsdir +run_suite() { # name script resultsdir [image-label] echo "" echo "========================================================================" - echo " $1 -> image=$IMAGE dur=$DUR reps=$REPS" + echo " $1 -> image=${4:-$IMAGE} dur=$DUR reps=$REPS" echo "========================================================================" bash "${DB}/$2" || { echo "!! suite $1 failed"; return 1; } echo "" @@ -58,15 +66,19 @@ run_suite() { # name script resultsdir bash "${DB}/parse.sh" "$3" } +# The two historical suites keep their own v0.6.3 pin; label them as such +# rather than printing this run's $IMAGE over the top of it. +HIST="v0.6.3 (HARD-PINNED, historical -- ignores --image)" + case "$SUITE" in - engines) run_suite engines bench-engines.sh results-engines ;; - admission) run_suite admission bench-admission.sh results-admission ;; + engines) run_suite engines bench-engines.sh results-engines "$HIST" ;; + admission) run_suite admission bench-admission.sh results-admission "$HIST" ;; proxy) run_suite proxy bench-proxy.sh results-proxy ;; bridge) run_suite bridge bench-bridge.sh results-bridge ;; wp-bridge) run_suite wp-bridge bench-wordpress-bridge.sh results-wp-bridge ;; all) - run_suite engines bench-engines.sh results-engines - run_suite admission bench-admission.sh results-admission + run_suite engines bench-engines.sh results-engines "$HIST" + run_suite admission bench-admission.sh results-admission "$HIST" run_suite proxy bench-proxy.sh results-proxy run_suite bridge bench-bridge.sh results-bridge run_suite wp-bridge bench-wordpress-bridge.sh results-wp-bridge