fix(runtime): carve liveness out of the identity step so a configuration fault cannot restart a pod

.changeset/liveness-carved-out-of-the-identity-step.md

@@ -0,0 +1,13 @@
+---
+"@objectstack/runtime": minor
+---
+
+`GET /api/v1/health` answers 200 whenever the process can serve HTTP, even while a configuration fault is making every other route 503.
+
+The dispatcher resolves a per-request identity before any route handler runs, and that step reads the tenancy posture for every request — credentialed or not. Since a `tenancy` service that is registered and fails to build is (correctly) re-raised as a 503 rather than absorbed into "there is no posture", an uncredentialed liveness probe was answered 503 for the length of the outage. A liveness probe reads 503 as *restart me*; the service then fails to build again on the new pod. A restart cannot fix a service that cannot build, so the result was a restart loop that hid the very fault the 503 exists to make loud.
+
+**Liveness is now carved out of the identity step.** `GET /health` runs its handler directly: no identity resolution, no configuration read, no credential read — the payload it answers (`status`, `timestamp`, `version`, `uptime`) was already process-local. Wire it to `livenessProbe`.
+
+**Readiness is unchanged, deliberately.** `GET /ready` keeps the full identity step and its 503 body, so traffic is withheld until the fault is fixed and existing operator dashboards keep the signal they have. Wire it to `readinessProbe`. Nothing else about the 503 moved: every other route, and an environment-scoped `/environments/:id/health`, answers exactly as before.
+
+Which routes count as liveness is **derived from the dispatcher's own route table** rather than listed anywhere: a route declares `liveness: true` on its registry entry, and `DomainHandlerRegistry.resolveLiveness()` answers through the same matcher that picks the handler — so the set cannot drift from the routes that exist. `DomainRoute.liveness` and `resolveLiveness()` are additive public surface on `@objectstack/runtime`; a route that does not declare the flag is untouched.

content/docs/deployment/cli.mdx

@@ -225,6 +225,15 @@ os serve --preset minimal  # Skip auto-loaded auth/i18n/ui plugins
 - `--preset minimal | default | full` — Override the auto-registration
   tier (see below)
 
+**Probes.** A served process exposes `GET /api/v1/health` (liveness — process
+only, and deliberately blind to configuration and credentials, so a
+configuration fault never restarts the pod) and `GET /api/v1/ready` (readiness —
+the full request pipeline, answering `503` while booting, draining, or faulted).
+Wire the first to Kubernetes' `livenessProbe` and the second to
+`readinessProbe`, never the reverse — the field mapping and the reference
+manifest live in
+[Health checks & orchestration](/docs/deployment/self-hosting#health-checks--orchestration).
+
 **Tier presets**
 
 `os serve` decides which optional plugins to auto-register from a tier

content/docs/deployment/self-hosting.mdx

@@ -265,6 +265,20 @@ questions:
   inconclusive — no data engine at all, or the probe itself errors — the replica
   stays ready rather than black-holing a working deployment.
 
+**A configuration fault never restarts the pod.** The split above is enforced
+one step earlier than the two handlers: `/health` is served *before* the request
+pipeline resolves an identity, so it evaluates nothing that depends on
+configuration or credentials — a misconfigured or unbuildable security service
+leaves it answering `200` for as long as the process can serve HTTP. `/ready`
+keeps the whole pipeline and answers `503` for exactly that class of fault, with
+the body it has always returned. So wire `path: /api/v1/health` to
+`livenessProbe` and `path: /api/v1/ready` to `readinessProbe` — **never the
+other way round, and never the same path to both**. Crossed over, a
+configuration fault makes the orchestrator kill and recreate the pod; the new
+one reads the same configuration and fails the same way, and the operator sees
+`CrashLoopBackOff` instead of the fault. Withholding traffic is the response
+that helps; restarting is not.
+
 ### Kubernetes
 
 The same image works unchanged. A minimal reference Deployment — secrets from

packages/runtime/src/domain-handler-registry.ts

@@ -68,6 +68,27 @@ export interface DomainRoute {
     match?: 'prefix' | 'exact' | 'segment';
     /** Restrict to these UPPERCASE HTTP methods. Omit = all methods. */
     methods?: string[];
+    /**
+     * This route is a LIVENESS probe: `dispatch()` runs its handler WITHOUT the
+     * per-request identity step or the gates that follow it.
+     *
+     * Declared here, on the route itself, and nowhere else — that is the whole
+     * point of the field. "Which routes are liveness" is a question with exactly
+     * one honest source, the table `dispatch()` already routes on, so
+     * {@link DomainHandlerRegistry.resolveLiveness} answers it through the SAME
+     * matcher that picks the handler. A separate array of liveness paths would
+     * be a second list of routes, and this repo has measured what those cost:
+     * they drift from the thing they describe and the drift is silent.
+     *
+     * ⛔ Do not set this on a route whose body reads configuration, credentials
+     * or any service. A liveness handler may report process-local facts only
+     * (the process is executing code, the server is listening) — anything else
+     * puts a configuration fault back on the route whose consumer answers by
+     * restarting the pod, and a restart cannot fix a service that cannot build.
+     * Readiness is where a dependency check belongs; its failure mode (leave the
+     * load-balancer rotation) is the one that helps.
+     */
+    liveness?: boolean;
     handler: DomainHandler;
 }
 
@@ -302,6 +323,23 @@ export class DomainHandlerRegistry {
         return undefined;
     }
 
+    /**
+     * The route claiming `path` (+`method`) when — and only when — it declared
+     * itself a liveness probe ({@link DomainRoute.liveness}); otherwise
+     * `undefined`.
+     *
+     * DERIVED, not listed: it is {@link resolve} plus one field read, so the
+     * liveness set is a projection of the live route table and cannot name a
+     * route that is not registered, miss one that is, or disagree with the
+     * matcher about which route a path reaches. First-match-wins is inherited
+     * too — a non-liveness route registered earlier shadows here exactly as it
+     * shadows in `resolve`, because that is the route the request would get.
+     */
+    resolveLiveness(path: string, method: string): DomainRoute | undefined {
+        const route = this.resolve(path, method);
+        return route?.liveness ? route : undefined;
+    }
+
     private static matches(route: DomainRoute, path: string): boolean {
         switch (route.match) {
             case 'exact':