fix(objectql): assert the redactor's end-of-message head invariant at module load (#9359) (#9986)

The invariant that bounds #9275's head-anchored statement cut to over-redaction
-- only an end-of-message template may declare a head -- was held by prose plus
one behavioural case over the current rows. It is now held by the type
(head XOR tail) and by assertHeadBearingTemplatesAreEndAnchored(), called at
module load over VALUE_BEARING_TEMPLATES.

No redaction behaviour changes.


Claude-Session: https://claude.ai/code/session_019yDEhPBC3tcGkW9bkce1HM

Co-authored-by: Claude <noreply@anthropic.com>

Author: Elon Musk

.changeset/redactor-head-invariant-guard.md

@@ -0,0 +1,53 @@
+---
+"@objectstack/objectql": patch
+---
+
+fix(objectql): the redactor's end-of-message head invariant becomes a load-time guard and a type, not a doc comment (#9359)
+
+#9275 made the driver-fault statement cut **template-aware**: a separator standing
+immediately before a *measured* diagnostic head is the true cut point wherever it falls,
+so the head survives and the caller's value is dropped whole. That amendment is safe
+because of exactly one property:
+
+> **Only an end-of-message template may declare a head.**
+
+That property is what bounds a hostile value's influence to **over-redaction** — a value
+spelling a known head can suppress a real diagnostic and show a forged one, but it cannot
+make a value leak. Give a `head` to a family with a **right anchor** and the same cut keeps
+everything after that anchor, which on such a cut is statement, which is caller values.
+
+Until now the invariant was held by **prose plus one behavioural case** that forges the
+heads the table declares *today*. Nothing stopped a future author adding a head-bearing
+row whose `whole` is not end-anchored — the single shape that turns the amendment into a
+leak surface.
+
+**The argument for closing it structurally comes from this file's own history.** The head
+note once claimed leak-freedom rested on taking the LAST matching head as well as on
+end-of-message. Ablated: with the cut changed to take the FIRST head, all 50 cases in the
+suite stayed green — an end-of-message pattern matches only once, from its earliest
+position. A documented property about this very mechanism was wrong for weeks of reading
+and fell only to an ablation. A doc comment is not a guard.
+
+The invariant is now held in two places a future author cannot write past:
+
+- **The type.** `ValueBearingTemplate` is a union of `AnchoredTemplate` (`tail?`, and
+  `head?: never`) and `EndOfMessageTemplate` (`head`, and `tail?: never`), so a row
+  carrying both no longer compiles.
+- **`assertHeadBearingTemplatesAreEndAnchored()`**, called at module load over
+  `VALUE_BEARING_TEMPLATES` — the `assertMetaUrlSpellingsAgree()` shape. For every row
+  that declares a `head` it requires the `whole` to end with `$()`, to carry no `m` flag
+  (under `m`, `$` is end of LINE, so an "end-anchored" template would stop at the first
+  newline of a multi-line dump and leave the rest standing) and to have exactly two
+  capture groups (`redactDiagnosticValues` reads `whole[1]` and `whole[2]` by index).
+
+**No redaction behaviour changes.** The statement cut, the head set and the templates
+themselves are untouched; the guard only refuses tables that could not have been correct.
+The existing behavioural case that forges each head is kept — it is evidence, and the
+guard is additional rather than a replacement.
+
+The guard is proved to FIRE rather than merely to exist: eight new cases drive each shape
+it rejects (right-anchored `whole` with a head, the `m` flag, a wrong group count, a bad
+row behind a good one) and pin that it does **not** fire on the shipped rows or on
+right-anchored rows that correctly take a `tail`. Reverse-verified both legs — a
+head-bearing right-anchored row added to the shipped table makes every test in the package
+fail at module load, and a row carrying `head` and `tail` together fails `tsc`.

packages/objectql/src/driver-fault-redaction.test.ts

@@ -24,7 +24,12 @@
 
 import { describe, it, expect } from 'vitest';
 import { ObjectQL } from './engine.js';
-import { redactBoundStatement, redactStatementFromMessage } from './driver-fault-redaction.js';
+import {
+  VALUE_BEARING_TEMPLATES,
+  assertHeadBearingTemplatesAreEndAnchored,
+  redactBoundStatement,
+  redactStatementFromMessage,
+} from './driver-fault-redaction.js';
 
 /** The canaries the card planted, kept verbatim so a leak is unmistakable. */
 const SECRET = 'SENSITIVE-CANARY-9f3a2b';
@@ -594,6 +599,97 @@ describe('#9275 — a value containing " - " no longer eats the template head',
   });
 });
 
+describe('#9359 — the head invariant is a GUARD, not a note', () => {
+  // ⭐ Why this block exists at all. The invariant "only an end-of-message
+  // template may declare a head" is the WHOLE of the argument that #9275's
+  // head-anchored cut over-redacts rather than leaks. Until this card it was
+  // held by prose plus the behavioural case above, which forges the heads the
+  // table declares TODAY — nothing stopped a future author adding a
+  // head-bearing row with a right anchor, which is the single shape that turns
+  // the amendment into a leak surface.
+  //
+  // ⛔ The cautionary precedent is in this file's own history: the head note
+  // once claimed leak-freedom also rested on taking the LAST matching head, and
+  // that claim was FALSE — ablating to first-match left all 50 cases green. A
+  // documented property about this very mechanism read as true for weeks and
+  // fell only to an ablation. So these cases do not assert that the guard is
+  // present; they assert that it FIRES, on each shape it claims to reject.
+
+  /** A right-anchored `whole` — the shape that must never carry a `head`. */
+  const RIGHT_ANCHORED = /(incorrect \w+ value:\s+)'[\s\S]*'(\s+for column\s+'[^']*')/gi;
+  /** A well-formed end-of-message `whole`, for the cases isolating one fault. */
+  const END_ANCHORED = /(truncated incorrect \w+ value:\s+)'[\s\S]*$()/gi;
+  const SOME_HEAD = /truncated incorrect \w+ value:\s+'/gi;
+
+  it('does NOT fire on the shipped table — it must not be a false alarm on `main`', () => {
+    // The zero. Its positive control is every case below: the same function,
+    // called the same way, throws on a row the shipped table does not contain.
+    // The module-load call is the same assertion, so importing this file at all
+    // has already exercised it — this case only says so out loud.
+    expect(() => assertHeadBearingTemplatesAreEndAnchored(VALUE_BEARING_TEMPLATES)).not.toThrow();
+  });
+
+  it('the shipped table really does contain head-bearing rows — the guard has something to check', () => {
+    // ⛔ Anti-vacuity of the case above: a guard that loops over zero
+    // head-bearing rows also "does not throw". Two rows carry a `head` today
+    // (mysql/1292, pg/22P02) and the rest take a `tail`.
+    const headBearing = VALUE_BEARING_TEMPLATES.filter((t) => t.head !== undefined);
+    expect(headBearing.map((t) => t.id)).toEqual([
+      'mysql/1292 ER_TRUNCATED_WRONG_VALUE',
+      'pg/22P02 invalid_text_representation',
+    ]);
+  });
+
+  it('FIRES on a head-bearing row whose `whole` is right-anchored — the leak shape itself', () => {
+    // This is the one way #9275's amendment can leak: the head-anchored cut
+    // lands inside the statement, and a right anchor keeps everything after it.
+    expect(() => assertHeadBearingTemplatesAreEndAnchored([
+      { id: 'test/right-anchored-with-head', whole: RIGHT_ANCHORED, head: SOME_HEAD },
+    ])).toThrow(/does not end with/);
+  });
+
+  it('FIRES on an end-anchored `whole` that carries the `m` flag — `$` would mean end of LINE', () => {
+    // Spelled end-anchored, not end-anchored. Under `m` a multi-line dump stops
+    // at the first newline and the remainder — statement — survives.
+    expect(() => assertHeadBearingTemplatesAreEndAnchored([
+      { id: 'test/multiline-flag', whole: /(truncated incorrect \w+ value:\s+)'[\s\S]*$()/gim, head: SOME_HEAD },
+    ])).toThrow(/end of LINE/);
+  });
+
+  it('FIRES when the capture groups do not number exactly two', () => {
+    // `redactDiagnosticValues` reads `whole[1]` and `whole[2]` by index, so a
+    // third group silently reassigns what is kept after the value.
+    expect(() => assertHeadBearingTemplatesAreEndAnchored([
+      { id: 'test/three-groups', whole: /(a)(b)'[\s\S]*$()/gi, head: SOME_HEAD },
+    ])).toThrow(/capture group/);
+  });
+
+  it('names the offending row, so the throw is actionable rather than a riddle', () => {
+    expect(() => assertHeadBearingTemplatesAreEndAnchored([
+      { id: 'test/right-anchored-with-head', whole: RIGHT_ANCHORED, head: SOME_HEAD },
+    ])).toThrow(/test\/right-anchored-with-head/);
+  });
+
+  it('leaves ANCHORED rows alone — a right-anchored `whole` with no head is exactly correct', () => {
+    // ⛔ The control that stops this guard from being "throw on anything that
+    // is not end-anchored". Three of the five shipped families are right-
+    // anchored on purpose; the invariant is about rows that declare a HEAD.
+    expect(() => assertHeadBearingTemplatesAreEndAnchored([
+      { id: 'test/right-anchored-no-head', whole: RIGHT_ANCHORED, tail: /'(\s+for column\s+'[^']*')/gi },
+    ])).not.toThrow();
+    expect(() => assertHeadBearingTemplatesAreEndAnchored([
+      { id: 'test/end-anchored-with-head', whole: END_ANCHORED, head: SOME_HEAD },
+    ])).not.toThrow();
+  });
+
+  it('checks EVERY row, not just the first — a bad row cannot hide behind a good one', () => {
+    expect(() => assertHeadBearingTemplatesAreEndAnchored([
+      { id: 'test/end-anchored-with-head', whole: END_ANCHORED, head: SOME_HEAD },
+      { id: 'test/right-anchored-with-head', whole: RIGHT_ANCHORED, head: SOME_HEAD },
+    ])).toThrow(/test\/right-anchored-with-head/);
+  });
+});
+
 describe('redactBoundStatement', () => {
   it('redacts `stack` too — the statement opened it a second time', () => {
     const original = new Error(BOUND_INSERT);

packages/objectql/src/driver-fault-redaction.ts

@@ -190,7 +190,7 @@
  * that spells a known head makes the cut land inside the statement, at that
  * head — and ONE property, not several, makes what follows unleakable rather
  * than exposed: only a template whose value runs to END OF MESSAGE may declare
- * a `head` (the invariant on {@link ValueBearingTemplate.head}). Whatever the
+ * a `head` (the invariant on {@link EndOfMessageTemplate.head}). Whatever the
  * head-anchored cut leaves — the rest of the statement it cut into included —
  * is consumed whole by that template's own `whole` pattern.
  *
@@ -224,6 +224,14 @@
  * values. That is the one way this amendment could leak, and the invariant
  * above is what forecloses it.
  *
+ * [#9359] ⭐ And since the ablation recorded just above proves that a claim in
+ * this very note about this very mechanism can read as true for weeks, the
+ * invariant is no longer left to the note. It is held in two places that a
+ * future author cannot write past: the TYPE ({@link AnchoredTemplate.head} is
+ * `never`, so no row can carry a `head` and a `tail` at once) and
+ * {@link assertHeadBearingTemplatesAreEndAnchored}, which throws at module load
+ * on a head-bearing row whose `whole` is not end-anchored.
+ *
  * A driver dump with no separator carries no statement to cut (`UNIQUE
  * constraint failed: sys_user.email`, `SQLITE_CONSTRAINT_NOTNULL: …`) and is
  * returned untouched — there is nothing there but the diagnostic already.
@@ -444,13 +452,38 @@ const MYSQL_TRUNCATED_INCORRECT_VALUE_HEAD = /truncated incorrect \w+ value:\s+'
  * drops everything before the anchor, because an anchor is evidence about the
  * value, not licence to assert which template printed it.
  */
-interface ValueBearingTemplate {
+interface ValueBearingTemplateBase {
   /** Dialect and the server's own error code, as the probe raises it. */
   readonly id: string;
-  /** Head + value + anchor. */
+  /**
+   * Head + value + anchor. Group 1 is kept before the value, group 2 after —
+   * {@link redactDiagnosticValues} reads both by index, so every row needs
+   * exactly two.
+   */
   readonly whole: RegExp;
-  /** The head-gone residue, when the template has a right anchor to recover it. */
+}
+
+/**
+ * A family whose diagnostic continues PAST the value, through a right anchor.
+ *
+ * ⛔ Such a family may NOT declare a `head`, and `head?: never` is what makes
+ * that unrepresentable instead of merely forbidden: the head-anchored cut can
+ * land inside the STATEMENT (a hostile value may spell a head), and a right
+ * anchor would then keep whatever follows it — which on such a cut is
+ * statement, which is caller values. It takes {@link tail} instead, which
+ * recovers the same residue with no cut change at all.
+ */
+interface AnchoredTemplate extends ValueBearingTemplateBase {
+  /** The head-gone residue, recovered through the family's right anchor. */
   readonly tail?: RegExp;
+  readonly head?: never;
+}
+
+/**
+ * A family whose value runs to END OF MESSAGE, so it has no right anchor and no
+ * residue to recover — it takes a {@link head} instead.
+ */
+interface EndOfMessageTemplate extends ValueBearingTemplateBase {
   /**
    * [#9275] The template's head through the value's OPENING QUOTE — what the
    * statement cut looks for so a value containing ` - ` cannot eat it.
@@ -464,21 +497,127 @@ interface ValueBearingTemplate {
    * template with a right anchor would instead keep whatever follows that
    * anchor, which on such a cut is statement, which is caller values.
    *
-   * Families that DO have a right anchor take {@link tail} instead; it recovers
-   * the same residue with no cut change at all. Both are asserted structurally
-   * by `driver-fault-redaction.test.ts` rather than trusted from this note.
+   * [#9359] That invariant is no longer carried by this note. The `tail` half is
+   * held by the type ({@link AnchoredTemplate.head} is `never`); the anchoring
+   * half is held by
+   * {@link assertHeadBearingTemplatesAreEndAnchored}, which runs at module load
+   * over the table below. A doc comment is not a guard — a claim this very note
+   * once made about this very mechanism (that last-match was a second line of
+   * defence) read as true for weeks and fell only to an ablation.
    */
-  readonly head?: RegExp;
+  readonly head: RegExp;
+  readonly tail?: never;
 }
 
-const VALUE_BEARING_TEMPLATES: readonly ValueBearingTemplate[] = [
+type ValueBearingTemplate = AnchoredTemplate | EndOfMessageTemplate;
+
+export const VALUE_BEARING_TEMPLATES: readonly ValueBearingTemplate[] = [
   { id: 'mysql/1062 ER_DUP_ENTRY', whole: DUPLICATE_ENTRY, tail: DUPLICATE_ENTRY_TAIL },
   { id: 'mysql/1366 ER_TRUNCATED_WRONG_VALUE_FOR_FIELD', whole: MYSQL_INCORRECT_VALUE, tail: MYSQL_INCORRECT_VALUE_TAIL },
   { id: 'mysql/1292 ER_TRUNCATED_WRONG_VALUE', whole: MYSQL_TRUNCATED_INCORRECT_VALUE, head: MYSQL_TRUNCATED_INCORRECT_VALUE_HEAD },
   { id: 'pg/22P02 invalid_text_representation', whole: PG_INVALID_INPUT_SYNTAX, head: PG_INVALID_INPUT_SYNTAX_HEAD },
   { id: 'pg/22003 numeric_value_out_of_range', whole: PG_VALUE_OUT_OF_RANGE, tail: PG_VALUE_OUT_OF_RANGE_TAIL },
 ];
 
+/**
+ * [#9359] What a head-bearing `whole` must END with: `$` — end of INPUT, since
+ * JavaScript's `$` means end-of-line only under `m`, which is why the guard
+ * rejects that flag — immediately followed by the EMPTY group 2 that
+ * {@link redactDiagnosticValues} reads as "nothing is kept after the value".
+ */
+const END_OF_MESSAGE_ANCHOR = '$()';
+
+/**
+ * [#9359] The head invariant, ENFORCED at module load rather than described.
+ *
+ * ## What it forecloses
+ *
+ * Only an end-of-message template may declare a `head`. That single property is
+ * what bounds a hostile value's influence over the statement cut to
+ * OVER-REDACTION: a value that spells a known head makes the cut land inside
+ * the statement, and `whole` then swallows the remainder whole. Give a `head`
+ * to a family with a RIGHT ANCHOR and the same cut keeps everything after that
+ * anchor — statement, which is caller values. That is the one way #9275's
+ * amendment can leak, and this function is what stops it being written.
+ *
+ * ## Why a load-time throw is the right shape here
+ *
+ * The precedent is `assertMetaUrlSpellingsAgree()` in `packages/spec`, and the
+ * property that makes a boot-path throw safe is the same in both: this reads
+ * NOTHING but literals declared in this file. It cannot depend on install
+ * state, environment or import order, so it is deterministic per build — it
+ * fires on the author's first import, never at a deployment that imported the
+ * same bytes successfully yesterday. ⛔ If a future row is ever computed from
+ * anything outside this module, that reasoning lapses and this belongs in a
+ * gate script instead.
+ *
+ * ## The checks, and why each is the invariant rather than a proxy for it
+ *
+ * 1. **No `m` flag.** Under `m`, `$` is end-of-LINE, so an end-anchored
+ *    template would stop at the first newline of a multi-line dump and leave
+ *    the rest standing. Then `whole` is spelled end-anchored while not being
+ *    end-anchored — the exact silent shape this card exists to end.
+ * 2. **`whole.source` ends with {@link END_OF_MESSAGE_ANCHOR}.** This is the
+ *    invariant's own wording ("`$`-anchored and group 2 empty") in one token.
+ * 3. **Exactly two capture groups.** {@link redactDiagnosticValues} reads
+ *    `whole[1]` and `whole[2]` by index; a third group would silently reassign
+ *    what is kept after the value.
+ *
+ * Checks 2 and 3 are SYNTACTIC — they read the pattern, not its behaviour — and
+ * so they reject spellings they cannot prove end-anchored even where a human
+ * can see the pattern is fine. That is the safe direction: a novel spelling
+ * gets a loud, named throw and its author amends this guard deliberately,
+ * rather than the invariant quietly acquiring an exception.
+ *
+ * Exported for `driver-fault-redaction.test.ts`, which calls it with synthetic
+ * rows to prove it FIRES — a guard nobody has watched fail is the same prose
+ * this replaced.
+ *
+ * @param templates - the table to check; the shipped one is checked at load.
+ * @throws when a row declares a `head` whose `whole` is not end-anchored.
+ */
+export function assertHeadBearingTemplatesAreEndAnchored(
+  templates: readonly ValueBearingTemplate[],
+): void {
+  for (const template of templates) {
+    if (template.head === undefined) continue;
+    const { source, flags } = template.whole;
+
+    if (flags.includes('m')) {
+      throw new Error(
+        "[driver-fault-redaction] Template '" + template.id + "' declares a head, but its 'whole' carries the "
+        + "'m' flag, under which '$' matches end of LINE rather than end of MESSAGE. Only a template whose "
+        + 'value runs to end of message may declare a head.',
+      );
+    }
+
+    if (!source.endsWith(END_OF_MESSAGE_ANCHOR)) {
+      throw new Error(
+        "[driver-fault-redaction] Template '" + template.id + "' declares a head, but its 'whole' does not end "
+        + "with '" + END_OF_MESSAGE_ANCHOR + "', so its value does not run to end of message. Only an "
+        + 'end-of-message template may declare a head: the head-anchored statement cut can land inside the '
+        + "statement, and what bounds that to over-redaction is that 'whole' then swallows the remainder. A "
+        + 'template with a right anchor keeps whatever follows that anchor, which on such a cut is statement, '
+        + "which is caller values. Give this row a 'tail' instead.",
+      );
+    }
+
+    // `new RegExp(source + '|')` always matches the empty string through its
+    // empty alternative, so the result is the group count without needing an
+    // input the pattern accepts.
+    const probe = new RegExp(source + '|').exec('');
+    const groupCount = probe === null ? 0 : probe.length - 1;
+    if (groupCount !== 2) {
+      throw new Error(
+        "[driver-fault-redaction] Template '" + template.id + "' has " + groupCount + " capture group(s); "
+        + "exactly 2 are required, because redactDiagnosticValues reads 'whole[1]' and 'whole[2]' by index.",
+      );
+    }
+  }
+}
+
+assertHeadBearingTemplatesAreEndAnchored(VALUE_BEARING_TEMPLATES);
+
 /**
  * [#9275] Every known diagnostic head, as the separator that stands before it.
  *
@@ -488,7 +627,7 @@ const VALUE_BEARING_TEMPLATES: readonly ValueBearingTemplate[] = [
  * hand.
  */
 const HEAD_ANCHORED_CUTS: readonly RegExp[] = VALUE_BEARING_TEMPLATES
-  .filter((template): template is ValueBearingTemplate & { head: RegExp } => template.head !== undefined)
+  .filter((template): template is EndOfMessageTemplate => template.head !== undefined)
   .map((template) => new RegExp(`${STATEMENT_SEPARATOR}(?=${template.head.source})`, 'gi'));
 
 /**