diff --git a/content/docs/permissions/system-context.mdx b/content/docs/permissions/system-context.mdx index 460dac0c50..1227e7a146 100644 --- a/content/docs/permissions/system-context.mdx +++ b/content/docs/permissions/system-context.mdx @@ -27,14 +27,21 @@ context whenever one exists. **Every anchor, and every count about the census population, is checked by CI.** `scripts/check-system-context-census.mjs` re-runs the AST census over the whole -repo on each PR and refuses the page when an anchor points at a line that is no -longer the site it names, when a stated count about the population disagrees -with the census, or — the check that matters most — when the code holds an -elevation read that no row here anchors. Six raw text counts in +repo on each PR and refuses the page when an anchor names a symbol its file no +longer declares, when a stated count about the population disagrees with the +census, or — the check that matters most — when the code holds an elevation read +that no row here anchors. Six raw text counts in [Maintaining this table](#maintaining-this-table) are deliberately **not** -enforced, and say there when they were measured. Pure line rot is repaired by -`node scripts/check-system-context-census.mjs --fix`; a site that arrived or -vanished is deliberately left for a person. See +enforced, and say there when they were measured. + +⚠️ **What these anchors buy, and what they cost — read this before trusting a +row.** Every anchor here is a `path#symbol` citation. Nothing encodes a +position, so an unrelated edit above a site cannot rot one, and a repair is +never mechanical: there are no numbers to renumber. The price is granularity. +Several reads inside one function collapse onto one anchor, so **deleting a +whole symbol reds this page, and deleting one of several reads inside a symbol +that keeps at least one may not.** The gap is real, it is measured below, and it +is left open deliberately rather than hidden — see [Maintaining this table](#maintaining-this-table). @@ -46,10 +53,10 @@ nothing to do with elevation. | Declaration | What it is | This page? | |:---|:---|:---:| -| `ExecutionContext.isSystem` — `packages/spec/src/kernel/execution-context.zod.ts:269` | The elevation flag on an operation's context | ✅ | -| `Object.isSystem` — `packages/spec/src/data/object.zod.ts:1634` | Marks a **system object** (protected from deletion; defaults its org-wide sharing to `public` when no `sharingModel` is set) | ❌ | -| `EmailTemplate.isSystem` — `packages/spec/src/system/email-template.zod.ts:125` | Built-in template; tenants may override but should not delete | ❌ | -| `Environment.isSystem` — `packages/spec/src/cloud/environment.zod.ts:137` | Platform-infrastructure environment, not user data | ❌ | +| `ExecutionContext.isSystem` — `packages/spec/src/kernel/execution-context.zod.ts#isSystem` | The elevation flag on an operation's context | ✅ | +| `Object.isSystem` — `packages/spec/src/data/object.zod.ts#isSystem` | Marks a **system object** (protected from deletion; defaults its org-wide sharing to `public` when no `sharingModel` is set) | ❌ | +| `EmailTemplate.isSystem` — `packages/spec/src/system/email-template.zod.ts#isSystem` | Built-in template; tenants may override but should not delete | ❌ | +| `Environment.isSystem` — `packages/spec/src/cloud/environment.zod.ts#isSystem` | Platform-infrastructure environment, not user data | ❌ | The collision is a genuine hazard rather than a naming nit: `Object.isSystem` changes an object's **default sharing**, and `ExecutionContext.isSystem` changes @@ -57,22 +64,22 @@ whether **sharing grants are materialised** — so a search for "isSystem sharin returns both, and they are unrelated decisions. A fifth, closely-spelled family — `isSystemObjectName()` / -`isSystemObject()` in `packages/runtime/src/action-execution.ts:66`, -`packages/mcp/src/mcp-http-tools.ts:222` — keys on the `sys_` **name prefix**, +`isSystemObject()` in `packages/runtime/src/action-execution.ts#isSystemObjectName`, +`packages/mcp/src/mcp-http-tools.ts#isSystemObject` — keys on the `sys_` **name prefix**, not on any flag. ## How the flag is set `isSystem` is **server-constructed and never client-supplied**. Inbound HTTP -cannot set it (`packages/rest/src/rest-server.ts:1858`, `:1887`), and neither -can an action body (`packages/runtime/src/domains/actions.ts:414`). It is +cannot set it (`packages/rest/src/rest-server.ts#enforceAuth`), and neither +can an action body (`packages/runtime/src/domains/actions.ts#handleActionsRequest`). It is written by internal callers only, as an option on the engine call: ```ts await engine.insert('crm_account', row, { context: { isSystem: true } }); ``` -Its parse-time default is `false` (`execution-context.zod.ts:269`), so an absent +Its parse-time default is `false` (`packages/spec/src/kernel/execution-context.zod.ts#isSystem`), so an absent context is never elevated. --- @@ -87,40 +94,40 @@ that silently does not happen. | # | Behaviour when `isSystem` | Package | What you get / what you lose | Anchor | |:--|:---|:---|:---|:---| -| 1 | **The whole security middleware short-circuits** before any gate runs | plugin-security | Get: every CRUD/FLS/tenant/owner gate below skipped in one branch. Lose: all of rows 2–6 at once — this is the single largest behaviour on the page | `security-plugin.ts:1686` | -| 2 | **`owner_id` is not auto-stamped on INSERT** (the step 3.5 anchor guard is inside the block row 1 skips) | plugin-security | Lose: the row lands `owner_id = NULL`, so the default `owner_only_writes` policy hides it **from its own creator**. Get: nothing — this is a gap, not a capability | guard at `security-plugin.ts:2612` (the step 3.5 block), skipped by `:1686` | -| 3 | Row-level read filter resolves to "no filter" | plugin-security | Get: unscoped reads. Lose: row-level scoping entirely | `security-plugin.ts:4440` | -| 4 | Field-level security returns **all** fields | plugin-security | Get: every column readable. Lose: field masking | `security-plugin.ts:4591` | -| 5 | Export permission granted unconditionally | plugin-security | Get: `canExport` is `true` | `security-plugin.ts:4669` | -| 6 | Write bypass = `true`, effective write scope = `org` | plugin-security | Get: widest write scope without holding any capability | `security-plugin.ts:1513`, `:1535` | -| 7 | Metadata-plane schema masking exempt (ADR-0106 D4) | metadata-core | Get: unmasked object schema. Note: the exemption is a **caller** property — it short-circuits before the security service is consulted | `object-schema-fls.ts:228` | -| 8 | `explain()` may target a principal other than the caller | plugin-security | Get: no `manage_users` / delegated-admin check | `security-plugin.ts:3953` | -| 9 | Anonymous-deny treats the caller as authenticated | core | Get: passes the 401 seam with no `userId` | `anonymous-deny.ts:154` | -| 10 | Permission-set projection middleware skipped | plugin-security | Lose: projection of permission-set-derived columns | `permission-set-projection.ts:1015` | -| 11 | Session-resolution middleware skipped | plugin-auth | Get: no session lookup attempted | `auth-plugin.ts:1427` | -| 12 | Per-request performance timings disclosed | observability | Get: timing headers a normal caller cannot pull | `perf-timing.ts:474` | -| 13 | Permission-set **overlay discard** skips the tenant-admin assertion | plugin-security | Get: an overlay can be discarded with no authenticated tenant administrator | `permission-set-overlay-discard.ts:142` | -| 14 | MCP stdio bridge skips the object API-exposure gate | mcp | Get: the bridge reaches objects whose `apiEnabled` / `apiMethods` would refuse an external caller | `stdio-data-bridge.ts:250` | -| 15 | **Read-audit rows are not written** | plugin-audit | Lose: the "a person opened this record" trail. `sudo()` keeps the caller's `userId`, so this flag is the only thing separating a human read from a platform one | `read-audit.ts:556` | -| 16 | Approval snapshot payload redaction skipped | plugin-approvals | Get: the whole snapshot on `find` / `findOne` — the audit/replay channel. Lose: field-visibility redaction over approval payloads | `payload-redaction-middleware.ts:115` | -| 17 | REST anonymous-deny seam satisfied | rest | Get: `enforceAuth` passes with no `userId`. Not reachable from the wire — `isSystem` is never set on an inbound request | `rest-server.ts:1890` | +| 1 | **The whole security middleware short-circuits** before any gate runs | plugin-security | Get: every CRUD/FLS/tenant/owner gate below skipped in one branch. Lose: all of rows 2–6 at once — this is the single largest behaviour on the page | `packages/plugins/plugin-security/src/security-plugin.ts#start` | +| 2 | **`owner_id` is not auto-stamped on INSERT** (the step 3.5 anchor guard is inside the block row 1 skips) | plugin-security | Lose: the row lands `owner_id = NULL`, so the default `owner_only_writes` policy hides it **from its own creator**. Get: nothing — this is a gap, not a capability | the step 3.5 guard block and the short-circuit that skips it are both inside `packages/plugins/plugin-security/src/security-plugin.ts#start` | +| 3 | Row-level read filter resolves to "no filter" | plugin-security | Get: unscoped reads. Lose: row-level scoping entirely | `packages/plugins/plugin-security/src/security-plugin.ts#getReadFilter` | +| 4 | Field-level security returns **all** fields | plugin-security | Get: every column readable. Lose: field masking | `packages/plugins/plugin-security/src/security-plugin.ts#computeReadableFields` | +| 5 | Export permission granted unconditionally | plugin-security | Get: `canExport` is `true` | `packages/plugins/plugin-security/src/security-plugin.ts#canExport` | +| 6 | Write bypass = `true`, effective write scope = `org` | plugin-security | Get: widest write scope without holding any capability | `packages/plugins/plugin-security/src/security-plugin.ts#start` | +| 7 | Metadata-plane schema masking exempt (ADR-0106 D4) | metadata-core | Get: unmasked object schema. Note: the exemption is a **caller** property — it short-circuits before the security service is consulted | `packages/metadata-core/src/object-schema-fls.ts#isObjectSchemaMaskExempt` | +| 8 | `explain()` may target a principal other than the caller | plugin-security | Get: no `manage_users` / delegated-admin check | `packages/plugins/plugin-security/src/security-plugin.ts#explainAccessForCaller` | +| 9 | Anonymous-deny treats the caller as authenticated | core | Get: passes the 401 seam with no `userId` | `packages/core/src/security/anonymous-deny.ts#shouldDenyAnonymous` | +| 10 | Permission-set projection middleware skipped | plugin-security | Lose: projection of permission-set-derived columns | `packages/plugins/plugin-security/src/permission-set-projection.ts#createPermissionSetWriteThrough` | +| 11 | Session-resolution middleware skipped | plugin-auth | Get: no session lookup attempted | `packages/plugins/plugin-auth/src/auth-plugin.ts#start` | +| 12 | Per-request performance timings disclosed | observability | Get: timing headers a normal caller cannot pull | `packages/observability/src/perf-timing.ts#isPerfDisclosurePrincipal` | +| 13 | Permission-set **overlay discard** skips the tenant-admin assertion | plugin-security | Get: an overlay can be discarded with no authenticated tenant administrator | `packages/plugins/plugin-security/src/permission-set-overlay-discard.ts#assertTenantAdmin` | +| 14 | MCP stdio bridge skips the object API-exposure gate | mcp | Get: the bridge reaches objects whose `apiEnabled` / `apiMethods` would refuse an external caller | `packages/mcp/src/stdio-data-bridge.ts#enforceApiExposure` | +| 15 | **Read-audit rows are not written** | plugin-audit | Lose: the "a person opened this record" trail. `sudo()` keeps the caller's `userId`, so this flag is the only thing separating a human read from a platform one | `packages/plugins/plugin-audit/src/read-audit.ts#installReadAuditWriter` | +| 16 | Approval snapshot payload redaction skipped | plugin-approvals | Get: the whole snapshot on `find` / `findOne` — the audit/replay channel. Lose: field-visibility redaction over approval payloads | `packages/plugins/plugin-approvals/src/payload-redaction-middleware.ts#bindSnapshotRedactionMiddleware` | +| 17 | REST anonymous-deny seam satisfied | rest | Get: `enforceAuth` passes with no `userId`. Not reachable from the wire — `isSystem` is never set on an inbound request | `packages/rest/src/rest-server.ts#enforceAuth` | ### 2. Write pipeline and data integrity | # | Behaviour when `isSystem` | Package | What you get / what you lose | Anchor | |:--|:---|:---|:---|:---| -| 18 | **`readonly` strip bypassed — UPDATE, single row** | objectql | Get: a `readonly` field CAN be written. Lose: the protection that stops a caller seeding e.g. `approval_status` | `objectql/src/engine.ts:11767` | -| 19 | **`readonly` strip bypassed — UPDATE, bulk/predicate** | objectql | Same, on the multi-row path | `objectql/src/engine.ts:11950` | -| 20 | **`readonly` strip bypassed — INSERT** | objectql | Same, on create — one gate over BOTH create-side passes since the 2026-09-03 ruling moved the static-`readonly` strip in beside the runtime-owned one and deleted the DataProtocol ingress copy. `isSystem` is the **only** exemption on this path: `preserveAudit` is deliberately not read on create, so a non-system historical import is still stripped | `objectql/src/engine.ts:10415` | -| 21 | Strict-drop refusal never fires | objectql | Lose: a caller that opted into loud refusal gets **silence** — strict refuses exactly what the strip would have taken, and the strip took nothing | `objectql/src/engine.ts:10548`, `readonly-strict-errors.ts:66` | -| 22 | **Referential-integrity check skipped** | objectql | Get: writes proceed against unreachable/unresolvable targets. Lose: an `isSystem` caller can write a **dangling reference** | `objectql/src/engine.ts:6253` | -| 23 | Tenant-audit warning silenced; `bypassTenantAudit` threaded to the driver | objectql | Get: unscoped system writes stop warning. Lose: the signal that would flag a genuine user-path scoping bug | `objectql/src/engine.ts:3930`, `:3940`, `:3967` | -| 24 | Engine-owned / append-only write guard bypassed | plugin-security | Get: generic writes to `managedBy` engine-owned objects | `system-write-guard.ts:96`, `:120` | -| 25 | Identity write guard bypassed (ADR-0092) | plugin-auth | Get: direct writes to identity tables through the generic data path | `identity-write-guard.ts:99` | -| 26 | Search-companion column **kept** in a read's rows when it was explicitly requested | objectql | Get: the internal companion column is readable. Lose: nothing for app code — this is the engine reading its own index | `objectql/src/engine.ts:6952` | -| 27 | Dependent-count disclosure on a blocked delete | objectql | Get: the count of blocking children. Nothing was elevated past the caller, so nothing is withheld | `objectql/src/engine.ts:12569` | -| 28 | Reference-cleanup log attributes the write to `'system'` | objectql | Get: an honest actor label instead of `anonymous` when the context carries neither `userId` nor `actor` | `objectql/src/engine.ts:12498` | -| 29 | **Bulk data event `organizationId` OMITTED** — the batch is published "not asserted" | plugin-security | Get: nothing — the `data.records.*` event still publishes. Lose: the per-organization attribution: this exit is taken before the security middleware composes any tenant wall, so it records no Layer 0 verdict on the operation (`OperationContext.tenantLayer0Verdict`, #15813), and the engine's bulk producer — which reads that recorded verdict and nothing else — omits the key rather than filling it from the caller's `tenantId`; a tenant-scoped consumer then does not deliver the event inside an organization wall (#15225) | `security-plugin.ts:1686` | +| 18 | **`readonly` strip bypassed — UPDATE, single row** | objectql | Get: a `readonly` field CAN be written. Lose: the protection that stops a caller seeding e.g. `approval_status` | `packages/objectql/src/engine.ts#update` | +| 19 | **`readonly` strip bypassed — UPDATE, bulk/predicate** | objectql | Same, on the multi-row path | `packages/objectql/src/engine.ts#update` | +| 20 | **`readonly` strip bypassed — INSERT** | objectql | Same, on create — one gate over BOTH create-side passes since the 2026-09-03 ruling moved the static-`readonly` strip in beside the runtime-owned one and deleted the DataProtocol ingress copy. `isSystem` is the **only** exemption on this path: `preserveAudit` is deliberately not read on create, so a non-system historical import is still stripped | `packages/objectql/src/engine.ts#insert` | +| 21 | Strict-drop refusal never fires | objectql | Lose: a caller that opted into loud refusal gets **silence** — strict refuses exactly what the strip would have taken, and the strip took nothing | `packages/objectql/src/engine.ts#insert`, `packages/objectql/src/readonly-strict-errors.ts#READONLY_CLASS_REASONS` | +| 22 | **Referential-integrity check skipped** | objectql | Get: writes proceed against unreachable/unresolvable targets. Lose: an `isSystem` caller can write a **dangling reference** | `packages/objectql/src/engine.ts#assertReferencesResolve` | +| 23 | Tenant-audit warning silenced; `bypassTenantAudit` threaded to the driver | objectql | Get: unscoped system writes stop warning. Lose: the signal that would flag a genuine user-path scoping bug | `packages/objectql/src/engine.ts#buildDriverOptions` | +| 24 | Engine-owned / append-only write guard bypassed | plugin-security | Get: generic writes to `managedBy` engine-owned objects | `packages/plugins/plugin-security/src/system-write-guard.ts#isUserContextWrite`, `#assertEngineOwnedWriteAllowed` | +| 25 | Identity write guard bypassed (ADR-0092) | plugin-auth | Get: direct writes to identity tables through the generic data path | `packages/plugins/plugin-auth/src/identity-write-guard.ts#isUserContextWrite` | +| 26 | Search-companion column **kept** in a read's rows when it was explicitly requested | objectql | Get: the internal companion column is readable. Lose: nothing for app code — this is the engine reading its own index | `packages/objectql/src/engine.ts#stripSearchCompanionFromRead` | +| 27 | Dependent-count disclosure on a blocked delete | objectql | Get: the count of blocking children. Nothing was elevated past the caller, so nothing is withheld | `packages/objectql/src/engine.ts#dependentCountIsDisclosable` | +| 28 | Reference-cleanup log attributes the write to `'system'` | objectql | Get: an honest actor label instead of `anonymous` when the context carries neither `userId` nor `actor` | `packages/objectql/src/engine.ts#recordReferenceCheckElevation` | +| 29 | **Bulk data event `organizationId` OMITTED** — the batch is published "not asserted" | plugin-security | Get: nothing — the `data.records.*` event still publishes. Lose: the per-organization attribution: this exit is taken before the security middleware composes any tenant wall, so it records no Layer 0 verdict on the operation (`OperationContext.tenantLayer0Verdict`, #15813), and the engine's bulk producer — which reads that recorded verdict and nothing else — omits the key rather than filling it from the caller's `tenantId`; a tenant-scoped consumer then does not deliver the event inside an organization wall (#15225) | `packages/plugins/plugin-security/src/security-plugin.ts#start` | ### 3. Sharing (`plugin-sharing`) @@ -128,49 +135,49 @@ The largest single consumer — **17 of the 106 sites**. | # | Behaviour when `isSystem` | What you get / what you lose | Anchor | |:--|:---|:---|:---| -| 30 | **Sharing-rule REVOCATION is skipped on the record-`afterDelete` hook** — and on that hook only | Lose: nothing permanently — the revoke is **delivered, but deferred on the unbounded shape**. The payload belongs to another subscriber: `record-share-cascade.ts` binds on every sharing-capable object and stashes for system writes on its own account (#5103). When the deleted ids are enumerable it revokes inline; when they are not — a predicate delete whose row set the stash could not resolve — it hands the reclaim to a queued background orphan sweep instead, so the share rows outlive the deleted records until that sweep runs, with the boot orphan sweep behind it. No surviving record loses access either way, and a restart re-runs the same sweep. This is one subscriber declining work another owns, not elevation silencing a consequence. ⚠️ **Grant MATERIALISATION no longer asks** — the `afterInsert` / `afterUpdate` skips, and the `before*` stash skip that fed them, were removed by the 2026-08-31 ruling on #13533; a system write materialises exactly as a user write does | `rule-hooks.ts:292` | -| 31 | Sharing write verdict short-circuits to `allow` | Get: writes pass the sharing gate unconditionally | `plugin-sharing/src/sharing-service.ts:677` | -| 32 | Record visibility / manage-shares checks return true | Get: no ownership or Modify-All requirement | `plugin-sharing/src/sharing-service.ts:943`, `:1030`, `:1787` | -| 33 | `grant()` skips the enforcement + manage-shares assertions | Get: the rule evaluator can materialise through the public API. Note it is **not** a bare skip: the system branch asserts the grant is not *inert* instead (a grant on an object no verdict can consult is refused) | `plugin-sharing/src/sharing-service.ts:1238` | -| 34 | `revoke()` deletes directly, **before** the non-manual-source guard | Get: the evaluator can revoke its own grants. Lose: the `CONFLICT` guard that warns a rule-materialised grant will be silently re-granted on the next reconcile | `plugin-sharing/src/sharing-service.ts:1476` (guard at `:1501`) | -| 35 | `listShares()` skips the management gate | Get: full enumeration of who can see a record | `plugin-sharing/src/sharing-service.ts:1528` | -| 36 | `sys_record_share` reads are **not** self-scoped | Get: tenant-wide share listing without `manage_sharing` | `sharing-plugin.ts:1189` | -| 37 | Share-link policy `enabled` check bypassed; system callers re-enter under a system context | Get: link **creation** while the policy is off — resolution is **not** bypassed since #14033 (`publicSharing.enabled` is a standing policy held at every redemption): a link minted this way does not resolve until the block is enabled | `plugin-sharing/src/share-link-service.ts:459`, `:513`, `:517`, `:590`, `:620` | -| 38 | Sharing-rule provenance stamp skipped | Lose: the row is not marked as an admin customization — seeder / `defineRule` / boot reconcilers are "the package door" | `sharing-rule-provenance.ts:66` | -| 39 | Sharing-rule service write + delete paths return early | Lose: the manage-rules gate on the service surface, and the platform-global-rule delete guard | `sharing-rule-service.ts:279`, `:518` | +| 30 | **Sharing-rule REVOCATION is skipped on the record-`afterDelete` hook** — and on that hook only | Lose: nothing permanently — the revoke is **delivered, but deferred on the unbounded shape**. The payload belongs to another subscriber: `packages/plugins/plugin-sharing/src/record-share-cascade.ts` binds on every sharing-capable object and stashes for system writes on its own account (#5103). When the deleted ids are enumerable it revokes inline; when they are not — a predicate delete whose row set the stash could not resolve — it hands the reclaim to a queued background orphan sweep instead, so the share rows outlive the deleted records until that sweep runs, with the boot orphan sweep behind it. No surviving record loses access either way, and a restart re-runs the same sweep. This is one subscriber declining work another owns, not elevation silencing a consequence. ⚠️ **Grant MATERIALISATION no longer asks** — the `afterInsert` / `afterUpdate` skips, and the `before*` stash skip that fed them, were removed by the 2026-08-31 ruling on #13533; a system write materialises exactly as a user write does | `packages/plugins/plugin-sharing/src/rule-hooks.ts#bindRuleHooks` | +| 31 | Sharing write verdict short-circuits to `allow` | Get: writes pass the sharing gate unconditionally | `packages/plugins/plugin-sharing/src/sharing-service.ts#bypassVerdict` | +| 32 | Record visibility / manage-shares checks return true | Get: no ownership or Modify-All requirement | `packages/plugins/plugin-sharing/src/sharing-service.ts#canManageShares`, `#assertCanManageShares`, `#shouldBypass` | +| 33 | `grant()` skips the enforcement + manage-shares assertions | Get: the rule evaluator can materialise through the public API. Note it is **not** a bare skip: the system branch asserts the grant is not *inert* instead (a grant on an object no verdict can consult is refused) | `packages/plugins/plugin-sharing/src/sharing-service.ts#grant` | +| 34 | `revoke()` deletes directly, **before** the non-manual-source guard | Get: the evaluator can revoke its own grants. Lose: the `CONFLICT` guard that warns a rule-materialised grant will be silently re-granted on the next reconcile | `packages/plugins/plugin-sharing/src/sharing-service.ts#revoke` (the guard it deletes in front of is in the same function) | +| 35 | `listShares()` skips the management gate | Get: full enumeration of who can see a record | `packages/plugins/plugin-sharing/src/sharing-service.ts#listShares` | +| 36 | `sys_record_share` reads are **not** self-scoped | Get: tenant-wide share listing without `manage_sharing` | `packages/plugins/plugin-sharing/src/sharing-plugin.ts#buildSharingMiddleware` | +| 37 | Share-link policy `enabled` check bypassed; system callers re-enter under a system context | Get: link **creation** while the policy is off — resolution is **not** bypassed since #14033 (`publicSharing.enabled` is a standing policy held at every redemption): a link minted this way does not resolve until the block is enabled | `packages/plugins/plugin-sharing/src/share-link-service.ts#createLink`, `#revokeLink`, `#listLinks` | +| 38 | Sharing-rule provenance stamp skipped | Lose: the row is not marked as an admin customization — seeder / `defineRule` / boot reconcilers are "the package door" | `packages/plugins/plugin-sharing/src/sharing-rule-provenance.ts#bindRuleProvenanceStamp` | +| 39 | Sharing-rule service write + delete paths return early | Lose: the manage-rules gate on the service surface, and the platform-global-rule delete guard | `packages/plugins/plugin-sharing/src/sharing-rule-service.ts#assertCanManageRules`, `#assertCanDeletePlatformGlobalRule` | ### 4. Approvals, reports, attachments, comments, knowledge | # | Behaviour when `isSystem` | Package | What you get / what you lose | Anchor | |:--|:---|:---|:---|:---| -| 40 | **Approval record lock released** — a locked record is writable | plugin-approvals | Get: engine self-writes (the status mirror) pass. Lose: the lock that stops edits while an approval is live. Note there is deliberately **no admin exemption** here — only `isSystem` | `lifecycle-hooks.ts:347` | -| 41 | Delegation write guard bypassed | plugin-approvals | Get: service / seed / import may write delegation rows naming another delegator | `lifecycle-hooks.ts:570` | -| 42 | Approval actor / submitter / pending-approver checks bypassed (8 sites) | plugin-approvals | Get: approve, reject, recall, reassign without being a pending approver or the submitter | `plugin-approvals/src/approval-service.ts:1096`, `:1219`, `:3475`, `:3623`, `:3791`, `:3862`, `:4051`, `:4091` | -| 43 | Saved-report ownership is **assignable**, and an update may reassign it | plugin-reports | Get: `ownerId` from input is honoured. A non-system caller always owns what it creates and can never reassign | `plugin-reports/src/report-service.ts:404`, `:425` | -| 44 | Saved-report access / export / mutation gates bypassed | plugin-reports | Get: read, bulk-export and overwrite any report | `plugin-reports/src/report-service.ts:343`, `:372`, `:447`, `:684` | -| 45 | Attachment access hooks return early (insert + update + delete, and the read AST) | service-storage | Lose: attachment visibility scoping | `attachment-access-hooks.ts:300`, `:349`, `:448`, `:524` | -| 46 | Comment access hooks return early (insert + update + delete, and the read AST) | plugin-audit | Lose: comment visibility scoping | `comment-access-hooks.ts:322`, `:449`, `:488`, `:540` | -| 47 | Knowledge search returns hits unfiltered | service-knowledge | Lose: the permission filter over search results | `service-knowledge/src/knowledge-service.ts:316` | +| 40 | **Approval record lock released** — a locked record is writable | plugin-approvals | Get: engine self-writes (the status mirror) pass. Lose: the lock that stops edits while an approval is live. Note there is deliberately **no admin exemption** here — only `isSystem` | `packages/plugins/plugin-approvals/src/lifecycle-hooks.ts#bindApprovalLockHook` | +| 41 | Delegation write guard bypassed | plugin-approvals | Get: service / seed / import may write delegation rows naming another delegator | `packages/plugins/plugin-approvals/src/lifecycle-hooks.ts#bindDelegationWriteGuard` | +| 42 | Approval actor / submitter / pending-approver checks bypassed (8 sites) | plugin-approvals | Get: approve, reject, recall, reassign without being a pending approver or the submitter | `packages/plugins/plugin-approvals/src/approval-service.ts#isOverrideActor`, `#resolveActor`, `#sendBack`, `#resubmit`, `#reassign`, `#remind`, `#requestInfo`, `#comment` | +| 43 | Saved-report ownership is **assignable**, and an update may reassign it | plugin-reports | Get: `ownerId` from input is honoured. A non-system caller always owns what it creates and can never reassign | `packages/plugins/plugin-reports/src/report-service.ts#saveReport` | +| 44 | Saved-report access / export / mutation gates bypassed | plugin-reports | Get: read, bulk-export and overwrite any report | `packages/plugins/plugin-reports/src/report-service.ts#assertExportAllowed`, `#canAccessReport`, `#listReports`, `#listSchedules` | +| 45 | Attachment access hooks return early (insert + update + delete, and the read AST) | service-storage | Lose: attachment visibility scoping | `packages/services/service-storage/src/attachment-access-hooks.ts#installAttachmentAccessHooks`, `#installAttachmentReadVisibility` | +| 46 | Comment access hooks return early (insert + update + delete, and the read AST) | plugin-audit | Lose: comment visibility scoping | `packages/plugins/plugin-audit/src/comment-access-hooks.ts#installCommentAccessHooks`, `#installCommentReadVisibility` | +| 47 | Knowledge search returns hits unfiltered | service-knowledge | Lose: the permission filter over search results | `packages/services/service-knowledge/src/knowledge-service.ts#applyPermissionFilter` | ### 5. Actions, metadata plane, provenance, the organization wall | # | Behaviour when `isSystem` | Package | What you get / what you lose | Anchor | |:--|:---|:---|:---|:---| -| 48 | Object API-exposure gate bypassed (`apiEnabled` / `apiMethods`) | runtime | Get: internal self-writes ignore exposure declarations — these govern **external** exposure, not engine self-writes | `action-execution.ts:138` | -| 49 | Action `requiredPermissions` bypassed | runtime | Get: engine self-invocation runs any action | `action-execution.ts:401` | -| 50 | `manage_metadata` bypassed on metadata writes | runtime, rest | Get: schema writes without the capability | `domains/meta.ts:471`, `:874`, `rest-server.ts:5520`, `:6977`, `:7225`, `:7656`, `:7849` | -| 51 | The shared metadata-write verdict itself returns `allowed` | metadata-core | Get: the one function all of row 50's doors consult answers yes before any capability is examined | `meta-write-capability.ts:134` | -| 52 | Anonymous-deny seam satisfied on the domain dispatchers and the package/federation routes | runtime, rest | Get: passes with no `userId` | `domains/actions.ts:421`, `domains/ai.ts:60`, `domains/automation.ts:1079`, `domains/meta.ts:232`, `domains/security.ts:78`, `domains/packages.ts:552`, `external-datasource-routes.ts:302`, `package-routes.ts:97` | -| 53 | MCP principal check satisfied | runtime | Get: MCP surface reachable with no user | `domains/mcp.ts:61` | -| 54 | Package REST route capability gate bypassed | rest | Get: package read/write over REST without `manage_metadata` / `studio.access` / `setup.access` | `package-routes.ts:102` | -| 55 | Package domain capability gates bypassed | runtime | Get: package management and package-inventory reads without the capability | `domains/packages.ts:250`, `:283` | -| 56 | Activation write / authoring refusals do not fire | runtime | Get: activation artifacts writable and authorable without the activation-authoring capability | `activation-gate.ts:177`, `:268` | -| 57 | Automation run-state read, flow-authoring write and unrelated-screen read all pass | runtime | Get: run state, flow writes and screen reads with no grant | `domains/automation.ts:255`, `:546`, `:636` | -| 58 | Audience-binding suggestion recording skipped | plugin-security | Lose: install-time suggestions are not recorded for system callers | `suggested-audience-bindings.ts:703` | -| 59 | Email-template / webhook provenance stamps skipped | plugin-email, plugin-webhooks | Lose: the row is not marked as an admin customization | `email-template-provenance.ts:77`, `webhook-provenance.ts:68` | -| 60 | **Automation flow data nodes re-add the `owner_id` stamp** (the one place row 2's gap is compensated inline) | service-automation | Get: a flow-authored INSERT under system elevation still lands owned, when the run resolved a user. Fill-only — flow-authored values win | `runtime-identity.ts:279`, called from `builtin/crud-nodes.ts:319` | -| 61 | Inbox caller refusal names `isSystem` as what was carried | service-messaging | Get: nothing — the refusal still fires. The flag only shapes the diagnostic, because privilege is not an authorization subject | `inbox-caller.ts:148` | -| 62 | **`organization_id` is not auto-stamped on INSERT** — the organization-axis twin of the `owner_id` gap above | organizations | Get: an elevated write may name another organization deliberately, which is what the per-organization seed replay, the orphan-row claim, imports and migrations all rely on. Lose: the authoritative stamp, so an elevated insert that names no organization lands `organization_id = NULL` and the wall hides it. ⛔ This is why a forged `organization_id` is overwritten on the non-elevated path and not here: elevation is the seam the legitimate cross-organization writers use | `organizations-plugin.ts:302` | +| 48 | Object API-exposure gate bypassed (`apiEnabled` / `apiMethods`) | runtime | Get: internal self-writes ignore exposure declarations — these govern **external** exposure, not engine self-writes | `packages/runtime/src/action-execution.ts#callData` | +| 49 | Action `requiredPermissions` bypassed | runtime | Get: engine self-invocation runs any action | `packages/runtime/src/action-execution.ts#actionPermissionError` | +| 50 | `manage_metadata` bypassed on metadata writes | runtime, rest | Get: schema writes without the capability | `packages/runtime/src/domains/meta.ts#handleMetadataRequest`, `packages/rest/src/rest-server.ts#registerMetadataEndpointsInner` | +| 51 | The shared metadata-write verdict itself returns `allowed` | metadata-core | Get: the one function all of row 50's doors consult answers yes before any capability is examined | `packages/metadata-core/src/meta-write-capability.ts#metaWriteCapabilityVerdict` | +| 52 | Anonymous-deny seam satisfied on the domain dispatchers and the package/federation routes | runtime, rest | Get: passes with no `userId` | `packages/runtime/src/domains/actions.ts#handleActionsRequest`, `packages/runtime/src/domains/ai.ts#handleAIRequest`, `packages/runtime/src/domains/automation.ts#handleAutomationRequest`, `packages/runtime/src/domains/meta.ts#handleMetadataRequest`, `packages/runtime/src/domains/security.ts#handleSecurityRequest`, `packages/runtime/src/domains/packages.ts#handlePackagesRequest`, `packages/rest/src/external-datasource-routes.ts#registerExternalDatasourceRoutes`, `packages/rest/src/package-routes.ts#refusePackageRequest` | +| 53 | MCP principal check satisfied | runtime | Get: MCP surface reachable with no user | `packages/runtime/src/domains/mcp.ts#handleMcpRequest` | +| 54 | Package REST route capability gate bypassed | rest | Get: package read/write over REST without `manage_metadata` / `studio.access` / `setup.access` | `packages/rest/src/package-routes.ts#refusePackageRequest` | +| 55 | Package domain capability gates bypassed | runtime | Get: package management and package-inventory reads without the capability | `packages/runtime/src/domains/packages.ts#requireManageMetadata`, `#requireReadCapability` | +| 56 | Activation write / authoring refusals do not fire | runtime | Get: activation artifacts writable and authorable without the activation-authoring capability | `packages/runtime/src/domains/activation-gate.ts#refuseUngrantedActivationWrite`, `#refuseUngrantedActivationAuthoring` | +| 57 | Automation run-state read, flow-authoring write and unrelated-screen read all pass | runtime | Get: run state, flow writes and screen reads with no grant | `packages/runtime/src/domains/automation.ts#mayReadRunState`, `#refuseUngrantedFlowWrite`, `#refuseUnrelatedScreenRead` | +| 58 | Audience-binding suggestion recording skipped | plugin-security | Lose: install-time suggestions are not recorded for system callers | `packages/plugins/plugin-security/src/suggested-audience-bindings.ts#assertTenantAdmin` | +| 59 | Email-template / webhook provenance stamps skipped | plugin-email, plugin-webhooks | Lose: the row is not marked as an admin customization | `packages/plugins/plugin-email/src/email-template-provenance.ts#bindEmailTemplateProvenanceStamp`, `packages/plugins/plugin-webhooks/src/webhook-provenance.ts#bindWebhookProvenanceStamp` | +| 60 | **Automation flow data nodes re-add the `owner_id` stamp** (the one place row 2's gap is compensated inline) | service-automation | Get: a flow-authored INSERT under system elevation still lands owned, when the run resolved a user. Fill-only — flow-authored values win | `packages/services/service-automation/src/runtime-identity.ts#stampSystemInsertOwner`, called from `packages/services/service-automation/src/builtin/crud-nodes.ts#registerCrudNodes` | +| 61 | Inbox caller refusal names `isSystem` as what was carried | service-messaging | Get: nothing — the refusal still fires. The flag only shapes the diagnostic, because privilege is not an authorization subject | `packages/services/service-messaging/src/inbox-caller.ts#resolveInboxRecipient` | +| 62 | **`organization_id` is not auto-stamped on INSERT** — the organization-axis twin of the `owner_id` gap above | organizations | Get: an elevated write may name another organization deliberately, which is what the per-organization seed replay, the orphan-row claim, imports and migrations all rely on. Lose: the authoritative stamp, so an elevated insert that names no organization lands `organization_id = NULL` and the wall hides it. ⛔ This is why a forged `organization_id` is overwritten on the non-elevated path and not here: elevation is the seam the legitimate cross-organization writers use | `packages/plugins/organizations/src/organizations-plugin.ts#start` | ### 6. Reads that only carry the flag onward @@ -180,10 +187,10 @@ a reader tracing where elevation travels needs them. | # | Site | Package | What it does | |:--|:---|:---|:---| -| 63 | `objectql/src/engine.ts:3737` | objectql | Propagates `isSystem` into the hook session so hooks can tell engine self-writes from user writes | -| 64 | `objectql/src/engine.ts:15016` | objectql | `ScopedContext.isSystem` getter — re-exposes the underlying execution context's flag | -| 65 | `plugin-reports/src/report-service.ts:556` | plugin-reports | Threads the flag into the engine call that runs a report | -| 66 | `body-runner.ts:279` | runtime | Rebuilds an `ExecutionContext` from a hook session, carrying the flag across | +| 63 | `packages/objectql/src/engine.ts#buildSession` | objectql | Propagates `isSystem` into the hook session so hooks can tell engine self-writes from user writes | +| 64 | `packages/objectql/src/engine.ts#isSystem` | objectql | `ScopedContext.isSystem` getter — re-exposes the underlying execution context's flag | +| 65 | `packages/plugins/plugin-reports/src/report-service.ts#executeReport` | plugin-reports | Threads the flag into the engine call that runs a report | +| 66 | `packages/runtime/src/sandbox/body-runner.ts#executionContextFromHook` | runtime | Rebuilds an `ExecutionContext` from a hook session, carrying the flag across | --- @@ -194,13 +201,13 @@ assuming `isSystem` covers it is a documented source of bugs. | Assumption | Reality | Anchor | |:---|:---|:---| -| "It suppresses triggers / record-change automation" | **No.** Only `skipTriggers` does. A bare `{ isSystem: true }` on a seed write re-fired automation on freshly seeded rows and wedged first boot | `metadata-protocol/src/seed-loader.ts:2032` (rationale at `:1942`–`1944`, #3760), `flow.zod.ts:743` | -| "It skips the state machine" | **No.** That is `skipStateMachine`, carried by seed replay and by `treatAsHistorical` imports | `objectql/src/engine.ts` FSM gate; see [State Machine](/docs/protocol/objectql/state-machine) | -| "It skips validation rules" | **No.** Field shape, `format`, `script` and the rest still run. The `readonly` strip runs *before* validation precisely so a discarded value is not judged | `objectql/src/engine.ts:10398`–`10415` | -| "It preserves a supplied `updated_at` / `updated_by`" | **No.** That is `preserveAudit`, a separate opt-in — and an UPDATE-path exemption only | `field.zod.ts:1590` (#3493 / #6640) | -| "It stamps `created_by`" | **No.** Audit stamping reads `userId` from the context. A user-less system write stamps nothing — that is today's behaviour, not an error | `runtime-identity.ts:280`–`281` | -| "It bypasses every guard" | **No.** The last-admin guard applies to **every** context, `isSystem` included — the deprovision path that actually locks an org out is the system one | `last-admin-guard.ts:299` | -| "A client can request it" | **No.** Never settable from inbound HTTP or from an action body | `rest-server.ts:1858`, `:1887`; `domains/actions.ts:414` | +| "It suppresses triggers / record-change automation" | **No.** Only `skipTriggers` does. A bare `{ isSystem: true }` on a seed write re-fired automation on freshly seeded rows and wedged first boot | `packages/metadata-protocol/src/seed-loader.ts#SEED_OPTIONS` (rationale at `#writeDeferredReference`, #3760), `packages/spec/src/automation/flow.zod.ts#runAs` | +| "It skips the state machine" | **No.** That is `skipStateMachine`, carried by seed replay and by `treatAsHistorical` imports | `packages/objectql/src/engine.ts` FSM gate; see [State Machine](/docs/protocol/objectql/state-machine) | +| "It skips validation rules" | **No.** Field shape, `format`, `script` and the rest still run. The `readonly` strip runs *before* validation precisely so a discarded value is not judged | `packages/objectql/src/engine.ts#insert` | +| "It preserves a supplied `updated_at` / `updated_by`" | **No.** That is `preserveAudit`, a separate opt-in — and an UPDATE-path exemption only | `packages/spec/src/data/field.zod.ts#readonly` (#3493 / #6640) | +| "It stamps `created_by`" | **No.** Audit stamping reads `userId` from the context. A user-less system write stamps nothing — that is today's behaviour, not an error | `packages/services/service-automation/src/runtime-identity.ts#stampSystemInsertOwner` | +| "It bypasses every guard" | **No.** The last-admin guard applies to **every** context, `isSystem` included — the deprovision path that actually locks an org out is the system one | `packages/plugins/plugin-auth/src/last-admin-guard.ts` | +| "A client can request it" | **No.** Never settable from inbound HTTP or from an action body | `packages/rest/src/rest-server.ts#enforceAuth`; `packages/runtime/src/domains/actions.ts#handleActionsRequest` | --- @@ -212,10 +219,10 @@ should recognise it instead of re-deriving it. 1. **The `owner_id` gap has two independent compensations and no shared mechanism.** Row 2 is a real gap; the platform repairs it twice, in unrelated places — inline for automation flow writes - (`runtime-identity.ts:279`, whose own comment states the reason: "the + (`packages/services/service-automation/src/runtime-identity.ts#stampSystemInsertOwner`, whose own comment states the reason: "the security middleware that stamps it short-circuits on `isSystem` — so the writer fills it here"), and as a boot-time sweep for seeded rows - (`plugin-security/src/claim-seed-ownership.ts`). Any *third* system write + (`packages/plugins/plugin-security/src/claim-seed-ownership.ts`). Any *third* system write path gets neither. If you add one, stamp ownership yourself. 2. **Sharing materialisation is no longer skipped — the rough edge is closed, @@ -254,7 +261,7 @@ should recognise it instead of re-deriving it. rule-materialised grant that the next reconcile silently restores. 5. **`applySystemFields` does not read this flag.** It is named as if it did. - `packages/objectql/src/registry.ts:475` is **schema-side column + `packages/objectql/src/registry.ts#applySystemFields` is **schema-side column provisioning** — which columns an object carries — and consumes `ExecutionContext.isSystem` zero times. The write-time ownership behaviour people attribute to it is row 2, in `plugin-security`. @@ -307,7 +314,7 @@ one ends: it returns prose inside comments and strings, the three unrelated metadata fields, and the `isSystemObject` / `isSystemObjectName` / `isSystemLedgerObject` name helpers. Worse, it loses real sites — a regex pass over this same corpus silently dropped **6** reads in -`plugin-reports/src/report-service.ts` to a quoting desync and **11** more to +`packages/plugins/plugin-reports/src/report-service.ts` to a quoting desync and **11** more to `(ctx?.session as any)?.isSystem` casts. So the census walks the TypeScript AST and classifies each appearance of the identifier by where the parser puts it: a **read** (a table row here), a **declaration**, an object-literal or type @@ -336,6 +343,8 @@ still holds equal to the census on every pull request: | — carry the flag onward only (rows 63–66 above) | 4 | ✅ | | Packages containing at least one elevation read | **20** | ✅ | | Files containing at least one elevation read | 45 | ✅ | +| — the distinct symbols those reads live in — what this page anchors | 89 | ✅ | +| — of those files, the ones holding more than one read in one symbol | 9 | ✅ | The six rows marked — are a **dated decomposition, not a live claim**: they were measured on 2026-08-29 at `ca1965f2b5` and CI does not re-derive them. They count @@ -367,49 +376,68 @@ moves this count and moves nothing else on this page — the census's read population, the anchored rows above, and the packages and files totals all stay where they are. The most recent arrival is the scoped seed context threaded into the org-admin permission-set lookup in -`plugins/plugin-security/src/auto-org-admin-grant.ts`, so that read resolves +`packages/plugins/plugin-security/src/auto-org-admin-grant.ts`, so that read resolves against the granting organization's own catalog row rather than an -organization-less one (#11670). ⛔ Cited without a line number deliberately: an -anchor here would be refused, and rightly — this page anchors elevation -**reads**, and a declaration is not one. +organization-less one (#11670). ⛔ Cited as a FILE deliberately, with no +`#symbol`: this page's symbol anchors name the enclosing declaration of an +elevation **read**, and a declaration of the shape is not one — so there is no +symbol here for a row to name, and inventing one would put a name on the page +that no rename could ever red. Counting by hand is what made the previous edition wrong in two independent ways, so both are worth naming. Its headline said "80 distinct sites across 18 packages" while its own tables anchored **77** — the number never matched the page it described. And a `grep -c` for anchor-shaped text over the previous edition answered **64**, because it counted *lines carrying an anchor*, not anchors: that page's -real anchor population was **111** once continuation anchors (`` `:1409` ``) and -range ends are counted. -Decompose a text count before comparing it to anything. +real anchor population was **111** once continuation anchors and range ends are +counted. Decompose a text count before comparing it to anything. ### What CI holds, and why it is the population and not just the anchors A line-number anchor rots on every unrelated edit to the same file, silently: -re-resolving all 111 anchors of the previous edition found **101 pointing at a -line that no longer held what the row named**, while only **10** were still +re-resolving all 111 anchors of the edition before last found **101 pointing at +a line that no longer held what the row named**, while only **10** were still correct — and **41 of them named a basename that matches two files**, so they -could not be placed without reading the row's Package column. This edition -spells an ambiguous basename far enough to be unique -(`objectql/src/engine.ts`, not `engine.ts`), which is what makes an anchor -mechanically resolvable at all. +could not be placed without reading the row's Package column. Neither failure is +reachable from this edition: it carries no line numbers at all, and it spells +every path in full from the repository root. Each anchor is +`packages/…/file.ts#symbol`, resolved by the shared symbol-anchor resolver +(`scripts/symbol-anchors.mjs`) against that file's own declaration sites — the +same resolver, and the same registration shape, that holds `docs/adr/**`. +Renaming a symbol is now a loud red instead of a silent misdirection. + +⚠️ **The precision that costs, priced here rather than buried.** A symbol anchor +cannot say WHICH read inside a function it means, and **9** of the **45** +anchored files hold more than one read inside a single symbol. So the population +check runs per file at symbol granularity: every file the census finds a read in +must be anchored, and the set of symbols this page cites into that file must +equal the set of symbols the census finds reads in. Two consequences, and the +second one is a hole: + +- **Delete a whole symbol and this page reds** — the anchor stops resolving and + the symbol set stops matching, in the same run. +- **Delete one of several reads inside a symbol that keeps at least one, and the + symbol set does not move, so it may not red.** The previous edition's line + numbers did close this one: both reads lost in the last drift lived inside + `callerContext()` helpers that still exist under the same names. Closing it + again means a span-aware resolver and per-read disambiguation in those nine + files; that is its own card, and it is deliberately not folded in here. Resolving anchors is nevertheless the *second* check, not the first. ⭐ **A gate that only checks what the page already says can never find what the page failed -to say.** Re-resolving every anchor of the previous edition would have passed +to say.** Re-resolving every anchor of the edition before last would have passed while it was missing 32 sites and its headline was 29 too low. So the load-bearing direction runs census → page: every elevation read in the code must be anchored here, and a site that vanishes from the code takes the census total -with it, which is what makes a row describing a protection that no longer exists -fail. That matters because the two reads deleted during the last drift lived -inside `callerContext()` helpers that still exist under the same names — **a -symbol-name anchor would have resolved, and would have stayed green.** +with it — the site, package and file counts above are census-derived — which is +what makes a row describing a protection that no longer exists fail. Four checks run, in `scripts/check-system-context-census.mjs`: | Check | What fails | |:---|:---| -| **Population** | an elevation read in the code with no anchor here | -| **Resolution** | an anchor whose spelling matches no tracked file, matches two, or names a line the file does not have | +| **Population** | a file holding an elevation read with no anchor here, or a symbol holding one that no anchor here names | +| **Resolution** | an anchor naming no tracked file, or naming a symbol that file does not declare — and any surviving line number, which is no longer an anchor form | | **Counts** | any **census-derived** number above that disagrees with the census — including the count-sentence wording, so the check cannot go quietly vacuous. The six raw text counts are exempt by design, but their rows must still be present and dated | | **Classification** | an anchor that is not a read site and is not a declared non-read citation | diff --git a/scripts/check-system-context-census.mjs b/scripts/check-system-context-census.mjs index 5ec795c864..2e1c7815ae 100644 --- a/scripts/check-system-context-census.mjs +++ b/scripts/check-system-context-census.mjs @@ -7,7 +7,7 @@ * * node scripts/check-system-context-census.mjs * node scripts/check-system-context-census.mjs --self-test - * node scripts/check-system-context-census.mjs --fix # re-anchor rotted lines + * node scripts/check-system-context-census.mjs --fix # nothing to repair -- see below * * That page declares itself "the authority" for every platform behaviour keyed off * `ExecutionContext.isSystem`, and says it is "built by census over the whole repo, @@ -33,29 +33,69 @@ * carry an anchor. The other direction (PAGE -> CENSUS) is worth having and cheap, * but it is the second gate, not the first. * - * The two deletions are the reason a symbol-name anchor is not sufficient either. - * Both were the `isSystem` propagation inside a `callerContext()` helper; both - * helpers still exist under the same name. **A symbol anchor would still resolve - * and would still be green** while the protection the row described was gone. * Deletions are caught here by the counts, which are census-derived: lose a site - * and the page's declared 109 stops being true. + * and the page's declared total stops being true. + * + * ## ⭐ Why the anchors are `path#symbol` and no longer `path:line` (#15921) + * + * A line number is not an anchor form anywhere in this repo any more. The + * `docs/adr/**` migration measured 243 of 337 live line anchors broken -- 72.1%, + * a one-way lower bound -- and ruled the whole class out; this page joins that + * ruling as a CORPUS REGISTRATION against the same resolver, never a second + * implementation of it. `CORPUS` below is a `defineCorpus` call and nothing else; + * the grammar, the extractor and the resolution rule live in + * `scripts/symbol-anchors.mjs`, whose header is authoritative. + * + * ⛔ The resolver is deliberately NOT widened to understand spans. That was the + * other option on the ruling (a span-aware resolver plus per-read disambiguation + * in the colliding files) and it is its own card, to be taken if the gap below is + * ever measured to have let a deletion through. + * + * ## ⚠️ What that costs, measured rather than asserted + * + * A symbol anchor cannot say WHICH read inside a symbol it means. Measured on the + * tree this migration ran against: 106 read sites live in 89 distinct symbols + * across 45 files, and 9 of those files hold more than one read inside a single + * symbol (`packages/objectql/src/engine.ts` and `packages/rest/src/rest-server.ts` + * are the widest, at 10 + * reads in 9 symbols and 6 reads in 2). So: + * + * ⭐ Delete a whole symbol and this gate REDS -- twice over: the anchor stops + * resolving, and the census's symbol set for that file stops matching the + * page's. + * ⚠️ Delete ONE of several reads inside a symbol that keeps at least one, and + * the symbol set does not move, so this gate may NOT red. + * + * That second line is the precision the line numbers had and these anchors do + * not. It is the reason the page carries the same warning where a reader meets + * the anchors: a gap stated on the instrument and not on the artifact is a gap + * only the instrument's author knows about. ⛔ It is NOT closed by adding a + * count of reads per file -- a count of reads cannot be satisfied by a page whose + * anchors are symbols, which is precisely why the population rule is per file and + * per symbol. * * ## The four checks * - * A RESOLUTION every anchor resolves to exactly one tracked file, at a line - * that file has. Ambiguity is an error, never a guess: the - * previous edition had 41 of 111 anchors whose bare basename - * matched two files and could only be placed by reading the - * row's prose. - * B POPULATION every elevation read site the census finds is anchored at its - * exact `file:line`. Zero omissions. ⭐ This is the mandatory one. + * A RESOLUTION delegated WHOLE to `sweepCorpus` over the `CORPUS` + * registration below: every anchor names a tracked file, every + * `#symbol` has a declaration site in it, and a surviving line + * number is a hard finding. ⛔ This gate re-implements none of + * that -- a sweep that could not run is a refusal here, never a + * skip. + * B POPULATION per FILE, at SYMBOL granularity: every file the census finds + * a read in carries at least one anchor here, and the set of + * symbols this page cites into that file EQUALS the set the + * census plus `NON_READ_ANCHORS` require -- so the two counts + * are equal by construction and a difference names the symbol + * rather than only the number. ⭐ This is the mandatory one. * C COUNTS every CENSUS-DERIVED number the page states equals the census. * A pattern that matches NOTHING is an error, so a reworded page * cannot silently stop being checked. The page's whole-corpus * TEXT counts are deliberately NOT compared -- see the next * section -- but they are still required to be present and dated. - * D CLASSIFICATION an anchor that is not a read site must be a declared - * `NON_READ_ANCHORS` row, and that row must still locate the line. + * D CLASSIFICATION a symbol anchor the census does not call an elevation read + * must be a declared `NON_READ_ANCHORS` row, and that row's + * symbol must still be declared by its file. * * ## ⭐ What is enforced, and why the text decomposition is NOT * @@ -83,42 +123,50 @@ * the page does not certify -- and whose churn, measured, was blocking the page * from ever landing. * - * ## Why `NON_READ_ANCHORS` carries needles instead of line numbers + * ## Why `NON_READ_ANCHORS` is keyed by SYMBOL * - * 28 of the page's anchors are deliberately not read sites: the four unrelated - * `isSystem` declarations, the `sys_`-prefix name helpers, a guard block a row - * cites as the thing being skipped, and the prose targets in the "what it does NOT - * do" table. They need an allow-list -- and an allow-list of LINE NUMBERS would rot + * Some of the page's anchors are deliberately not read sites: the four unrelated + * `isSystem` declarations, the `sys_`-prefix name helpers, a guard a row cites as + * the thing being skipped, and the prose targets in the "what it does NOT do" + * table. They need an allow-list -- and an allow-list of LINE NUMBERS would rot * exactly like the anchors this gate exists to stop rotting, silently, because a - * stale row still excuses an anchor. - * - * So each row carries a `needle`: a literal that must appear on exactly one line of - * the file. The gate LOCATES the line and requires the page's anchor to name it. - * That makes every anchor on the page enforced and mechanically repairable, and it - * makes the ledger self-retiring -- a needle that matches zero lines, or more than - * one, is an error naming the row. - * - * ## `--fix` repairs rot and REFUSES to repair population - * - * Per file, when the page's DISTINCT anchor count equals the number of lines the - * file offers to be anchored -- its census read sites AND its `NON_READ_ANCHORS` - * citations, as one union -- the two are mapped in line order and the numbers - * rewritten: that is a pure shift, the shape an unrelated edit produces. When the - * counts differ, the population changed -- a site arrived or vanished -- and no - * mechanical mapping is honest. `--fix` leaves those alone and the gate stays red - * until a human writes the row. - * - * ⭐ The union is load-bearing, not tidiness: subtracting the ledger by LINE - * compares a pre-shift page with a post-shift ledger and reports a POPULATION - * change over a population that never moved (#13490). `fixAnchors` carries the two - * measured occurrences and why the union is the safer shape. - * - * ⇒ So the repair for a line-shift red is `--fix` and never a hand-edited line - * number, and the repairing PR should state that `--fix` REFUSED ZERO files. That - * sentence is what separates a pure re-anchor from a population change that - * happened to be shifted at the same time: the refusal is the gate's only signal - * that a site arrived or vanished, and a `--fix` run reporting refusals leaves - * rows a human still has to write. + * stale row still excuses an anchor. So did the `needle` this ledger used before + * #15921: a literal of source text, which every reformatting moved. + * + * Each row now names `{ file, symbol }`, the same pair the page writes, and the + * gate asks the SHARED resolver whether that file still declares that symbol. A + * row whose symbol is gone is an error naming the row, so the ledger stays + * self-retiring; and there is nothing left in it that a whitespace change can + * break. + * + * `symbol: null` is the FILE-LEVEL row and it is honest, not a shrug: the citation + * lands in a module docblock with no declaration around it, and a file-level + * anchor is what the grammar provides for exactly that. One row is like this today + * (`plugin-auth/src/last-admin-guard.ts`). + * + * ⭐ `collapsesOntoRead` is the declaration this migration made necessary. Under + * symbol granularity a citation can share its symbol with a census read site -- the + * `owner_id` guard block and the short-circuit that skips it are both inside + * `packages/plugins/plugin-security/src/security-plugin.ts#start` -- so the row + * stops EXCUSING anything while its `why` + * and its `rowSeams` are still worth keeping. The field says so, and the gate + * refuses when the declaration and the census disagree in EITHER direction: an + * undeclared overlap reads as a row that excuses an anchor when it does not, and a + * declared overlap that has ended is a row nobody re-examined. + * + * ## ⛔ `--fix` no longer rewrites anything, and that is the point + * + * It used to re-anchor a pure line shift, which was the common repair: a file grew + * an import, every anchor into it moved by one, and a mechanical remap was both + * safe and necessary. Symbol anchors do not shift, so that repair has no subject. + * ⛔ The flag is NOT silently accepted -- a `--fix` that writes nothing and exits 0 + * reads exactly like a repair that worked. It prints what it did not do and why, + * and then returns this gate's ordinary verdict, so `gen:system-context-census` + * stays wired and stays honest. + * + * ⇒ Every red here is now a HUMAN edit: a symbol was renamed (update the anchor), + * a read arrived or vanished (write or delete the row), or a citation moved out of + * the symbol that held it. * * ## Refusals, never quiet passes (#4690) * @@ -159,15 +207,22 @@ */ import { createHash } from 'node:crypto'; -import { mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'; +import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'; import { execFileSync } from 'node:child_process'; import { tmpdir } from 'node:os'; import { join } from 'node:path'; import { fileURLToPath } from 'node:url'; import { isEntrypoint } from './invoked-as.mjs'; -import { CORPUS_ROOTS, runCensus, siteKeys } from './isystem-census.mjs'; -import { extractLineAnchors, extractPathCitations, resolveAnchorFile } from './doc-line-anchors.mjs'; +import { CORPUS_ROOTS, runCensus, symbolPopulation } from './isystem-census.mjs'; +import { + ANCHOR_GRAMMAR, + defineCorpus, + extractAnchors, + formatFindings, + symbolResolutionClass, + sweepCorpus, +} from './symbol-anchors.mjs'; // ── The self-test's own battery roster and floor (#13489) ────────────────── // @@ -187,20 +242,20 @@ import { extractLineAnchors, extractPathCitations, resolveAnchorFile } from './d // remedy is to find what stopped registering. const SELF_TEST_BATTERIES = Object.freeze({ 'the GREEN control: a page that is correct': 2, - '⭐ the RED that matters: a site the page never mentions': 1, + '⭐ the RED that matters: a site the page never mentions': 2, 'the deletion shape: the row stands, the site is gone': 3, - 'resolution': 3, - 'ledger': 3, + '⭐ RESOLUTION is delegated, and a sweep that did not run is a REFUSAL': 3, + 'ledger': 5, 'counts': 6, '⭐ CRITERION: enforced means CENSUS-DERIVED, pinned over the REAL lists': 2, 'the same criterion, behaviourally, on one page': 3, '⛔ and the half that must NOT have moved: the contract still reds': 2, 'absence is loud': 1, - '--fix': 2, - '⭐ #13490: the incident shape -- reads AND ledger citations BOTH shift': 2, - '⛔ the dangerous direction: the citation crosses onto a read anchor\'s line ─': 1, - '⭐ and the safety property, on the shape that now ACCEPTS': 2, - 'the refusal has to SHOW its work (both counts, both classes, the diff)': 2, + '⛔ --fix rewrites NOTHING, and says so': 2, + '⭐ THE RULED RED-FIRST PAIR: a symbol rename REDS, a pure line move does NOT': 5, + '⭐ the precision this trades away, pinned so nobody rediscovers it as a bug': 2, + 'the refusal has to SHOW its work (both counts, the symbol, the file)': 2, + '⭐ CORPUS REGISTRATION: one resolver, not a second implementation': 3, 'WIRING: this gate, and its self-test, really run in CI': 2, 'POPULATION DECLARATION: what the dispatch derivation is told this gate reads': 6, '⭐ ROW REFERENCES: held by seam, and the insertion that was silent (#15869)': 23, @@ -217,6 +272,46 @@ const UNATTRIBUTED_BATTERY = '(no battery open)'; const ROOT = join(fileURLToPath(new URL('.', import.meta.url)), '..'); export const PAGE = 'content/docs/permissions/system-context.mdx'; +const PAGE_DIR = 'content/docs/permissions'; +const PAGE_FILE = /^system-context\.mdx$/; + +/** + * ── The corpus registration (#15921) ──────────────────────────────────────── + * + * ⚠️ THE MECHANISM IS NOT HERE. This is a `defineCorpus` call and nothing else, + * exactly like `scripts/check-adr-symbol-anchors.mjs#CORPUS`: the grammar, the + * extractor and the resolution rule are `scripts/symbol-anchors.mjs`'s, shared + * with `docs/adr/**` and with the `scripts/**` gate-header corpus. The ruling + * that put line anchors out of this repo said 「共享同一个 resolver,⛔ 不造第 + * 二套」, and a corpus is how a body of documents joins it. + * + * `docPattern` names ONE file rather than the directory: the other 22 pages under + * `content/docs/permissions` are hand-written prose that nobody has migrated, and + * sweeping them here would red this gate for citations it was never given the + * ledger to explain. Widening the pattern is its own decision with its own + * cleanup, not a side effect of this one. + * + * `checkBarePaths` is ON, which `docs/adr/**` cannot afford (1,056 findings there) + * and this page can: it carries 45 anchored files and a handful of prose + * citations, every one of them spelled in full from the repository root, so a + * bare path that resolves to nothing is a real finding and not a corpus-wide + * cleanup. + * + * ⚠️ For anyone registering the NEXT corpus: `sweepCorpus` resolves through + * `git ls-files` and passes no environment of its own, so a sweep of a SYNTHETIC + * root inherits whatever `GIT_DIR` / `GIT_INDEX_FILE` the caller was launched + * with. In-repo callers are unaffected (the inherited values name this + * repository, which is the right answer); a test that builds a throwaway tree is + * not. This gate's self-test detaches from those variables before its first case + * — see `buildRedFirstCorpus`, which carries the measured incident. + */ +export const CORPUS = defineCorpus({ + id: 'system-context', + label: 'content/docs/permissions/system-context.mdx (the isSystem census page)', + docRoots: [PAGE_DIR], + docPattern: PAGE_FILE, + checkBarePaths: true, +}); /** * ── The population this gate READS, declared where the dispatch tool looks ─── @@ -278,156 +373,146 @@ const ROOT_DIR_WATCH_HINTS = ['packages/**', 'examples/**']; /** * ⛔ SHRINK-ONLY. Anchors the page writes that are deliberately NOT elevation read - * sites. `needle` must appear on exactly ONE line of `file`; that line is where the - * page's anchor has to point. + * sites, keyed the way the page writes them: `{ file, symbol }`. + * + * `symbol` must have a declaration site in `file` by the SHARED resolver's rule + * (`scripts/symbol-anchors.mjs#symbolResolutionClass`), and the page must anchor + * exactly that pair. `symbol: null` is a FILE-LEVEL row -- the citation lands + * somewhere no declaration encloses, and the page anchors the bare path. + * + * `collapsesOntoRead: true` declares that this row's symbol is ALSO a symbol the + * census finds an elevation read in, so the row no longer excuses an anchor and is + * kept for its `why` and its `rowSeams`. The gate holds the declaration to the + * census in both directions. */ export const NON_READ_ANCHORS = [ // ── The four declarations that share the identifier ────────────────────────── { file: 'packages/spec/src/kernel/execution-context.zod.ts', - needle: 'isSystem: z.boolean().default(false),', + symbol: 'isSystem', why: 'the elevation flag itself -- a declaration, not a read', }, { file: 'packages/spec/src/data/object.zod.ts', - needle: "isSystem: z.boolean().optional().default(false).describe('Is system object", + symbol: 'isSystem', why: 'Object.isSystem -- an unrelated metadata field the page names to defuse the collision', }, { file: 'packages/spec/src/system/email-template.zod.ts', - needle: 'isSystem: z.boolean().default(false),', + symbol: 'isSystem', why: 'EmailTemplate.isSystem -- unrelated metadata field', }, { file: 'packages/spec/src/cloud/environment.zod.ts', - needle: "isSystem: z.boolean().default(false).describe('Whether this is a system environment", + symbol: 'isSystem', why: 'Environment.isSystem -- unrelated metadata field', }, // ── The `sys_` name-prefix family, cited to keep it apart from the flag ────── { file: 'packages/runtime/src/action-execution.ts', - needle: 'export function isSystemObjectName(name: string): boolean {', + symbol: 'isSystemObjectName', why: 'keys on the `sys_` NAME PREFIX, not on any flag', }, { file: 'packages/mcp/src/mcp-http-tools.ts', - needle: 'function isSystemObject(name: string): boolean {', + symbol: 'isSystemObject', why: 'the same name-prefix helper, MCP side', }, // ── Constructs a table row deliberately cites alongside its read ───────────── { file: 'packages/plugins/plugin-security/src/security-plugin.ts', - needle: '3.5. [#3004]', - why: 'row 2 -- the `owner_id` guard block that the row-1 short-circuit skips', + symbol: 'start', + collapsesOntoRead: true, + why: 'row 2 -- the `owner_id` guard block that the row-1 short-circuit skips; both are inside `start`', rowSeams: ['`owner_id` is not auto-stamped on INSERT', 'The whole security middleware short-circuits'], }, { file: 'packages/objectql/src/engine.ts', - needle: 'if (!hasTx && !hasTenant && !isSystem && !hasTz && !preserveAudit) return base;', - why: 'row 23 -- the early return the tenant-audit read feeds', - rowSeams: ['Tenant-audit warning silenced'], - }, - { - file: 'packages/objectql/src/engine.ts', - needle: 'if (isSystem && opts.bypassTenantAudit === undefined && !isTenantAuditInScope) {', - why: 'row 23 -- where `bypassTenantAudit` is threaded to the driver', + symbol: 'buildDriverOptions', + collapsesOntoRead: true, + why: 'row 23 -- the early return the tenant-audit read feeds, and where `bypassTenantAudit` is threaded to the driver', rowSeams: ['Tenant-audit warning silenced'], }, { file: 'packages/objectql/src/engine.ts', - needle: 'if (options?.strictReadonlyWrites === true) {', - why: 'row 21 -- the strict-drop refusal that never fires under elevation', + symbol: 'insert', + collapsesOntoRead: true, + why: 'row 21 -- the strict-drop refusal that never fires under elevation, and the strip-before-validation block the validation row cites', rowSeams: ['Strict-drop refusal never fires'], }, { file: 'packages/objectql/src/readonly-strict-errors.ts', - needle: 'const READONLY_CLASS_REASONS', + symbol: 'READONLY_CLASS_REASONS', why: 'row 21 -- the reason set the silent refusal would have used', rowSeams: ['Strict-drop refusal never fires'], }, { file: 'packages/plugins/plugin-security/src/system-write-guard.ts', - needle: 'if (!isUserContextWrite(context)) return;', + symbol: 'assertEngineOwnedWriteAllowed', why: 'row 24 -- the bypass expressed through a helper rather than a direct read', rowSeams: ['append-only write guard bypassed'], }, { file: 'packages/plugins/plugin-sharing/src/sharing-service.ts', - needle: "if (row.source != null && row.source !== 'manual') {", - why: 'row 34 -- the CONFLICT guard `revoke()` deletes in front of', + symbol: 'revoke', + collapsesOntoRead: true, + why: 'row 34 -- the CONFLICT guard `revoke()` deletes in front of, in the same function', rowSeams: ['`revoke()` deletes directly'], }, { file: 'packages/services/service-automation/src/builtin/crud-nodes.ts', - needle: 'stampSystemInsertOwner(fields, dataCtx, data, objectName);', + symbol: 'registerCrudNodes', why: 'row 60 -- the call site of the compensating owner stamp', rowSeams: ['Automation flow data nodes re-add the `owner_id` stamp'], }, { file: 'packages/objectql/src/registry.ts', - needle: 'export function applySystemFields(', + symbol: 'applySystemFields', why: 'rough edge 5 -- named as if it read the flag; it reads it zero times', }, // ── Prose targets: "what `isSystem` does NOT do", and the rough edges ──────── { file: 'packages/metadata-protocol/src/seed-loader.ts', - needle: 'so it must carry `skipTriggers` too.', - why: 'the rationale comment the triggers row cites', + symbol: 'writeDeferredReference', + why: 'the rationale comment the triggers row cites -- `isSystem` does NOT suppress trigger dispatch', }, { file: 'packages/metadata-protocol/src/seed-loader.ts', - needle: 'does NOT suppress trigger dispatch, only `skipTriggers` does', - why: 'end of that rationale comment', - }, - { - file: 'packages/metadata-protocol/src/seed-loader.ts', - needle: 'SEED_OPTIONS = { context: { isSystem: true, skipTriggers: true', + symbol: 'SEED_OPTIONS', why: 'the seed options that carry BOTH flags -- a producer, not a read', }, { file: 'packages/spec/src/automation/flow.zod.ts', - needle: 'Declare `system` to make the elevation explicit.', + symbol: 'runAs', why: 'the flow-side declaration of the same distinction', }, - { - file: 'packages/objectql/src/engine.ts', - needle: '// Runs BEFORE validation on purpose: a value the caller was never', - why: 'start of the strip-before-validation block the validation row cites', - }, { file: 'packages/spec/src/data/field.zod.ts', - needle: "readonly: z.boolean().default(false).describe(", + symbol: 'readonly', why: '`preserveAudit` is the separate opt-in -- this is the `readonly` declaration', }, { file: 'packages/services/service-automation/src/runtime-identity.ts', - needle: 'const userId = (dataCtx as RunIdentityContext).userId;', - why: 'audit stamping reads `userId`, not the flag', - }, - { - file: 'packages/services/service-automation/src/runtime-identity.ts', - needle: 'if (!userId) return;', - why: 'the user-less system write that stamps nothing', + symbol: 'stampSystemInsertOwner', + collapsesOntoRead: true, + why: 'audit stamping reads `userId`, not the flag, and the user-less system write stamps nothing', }, { file: 'packages/plugins/plugin-auth/src/last-admin-guard.ts', - needle: 'applies to EVERY context, `isSystem` included', - why: 'the guard that is NOT bypassed -- cited to refute "it bypasses every guard"', + symbol: null, + why: 'the guard that is NOT bypassed -- cited to refute "it bypasses every guard". The claim lives in the module docblock, which no declaration encloses, so this is the one FILE-LEVEL row', }, { file: 'packages/rest/src/rest-server.ts', - needle: '"authenticated". `isSystem` flags are never set on inbound HTTP', - why: 'inbound HTTP cannot set the flag', - }, - { - file: 'packages/rest/src/rest-server.ts', - needle: '`isSystem` is never set on inbound HTTP, so it cannot bypass.', - why: 'the second inbound seam', + symbol: 'enforceAuth', + collapsesOntoRead: true, + why: 'inbound HTTP cannot set the flag -- stated in the docblock and again inside the seam', }, { file: 'packages/runtime/src/domains/actions.ts', - needle: '`isSystem` is never settable from the wire; internal', - why: 'an action body cannot set the flag', + symbol: 'handleActionsRequest', + collapsesOntoRead: true, + why: 'nor can an action body', }, ]; @@ -794,7 +879,7 @@ export function checkRowReferences({ pageText, ledger = NON_READ_ANCHORS, pageRe const seams = row.rowSeams ?? []; if (mentions.length !== seams.length) { problems.push( - `[why-row-unkeyed] NON_READ_ANCHORS row for ${row.file} (needle \`${row.needle}\`) writes ` + + `[why-row-unkeyed] NON_READ_ANCHORS row for ${row.file} (\`#${row.symbol ?? ''}\`) writes ` + `${mentions.length} row reference(s) in its \`why\` (${row.why}) but declares ${seams.length} ` + '`rowSeams`. Every `row N` in a `why` needs the seam it is about, in the order it is ' + 'written -- an unkeyed number is held by nothing and reads as current forever.' @@ -848,6 +933,30 @@ export function checkRowReferences({ pageText, ledger = NON_READ_ANCHORS, pageRe * reworded out from under the check, which is how a counts gate goes quietly * vacuous. */ +/** + * How many distinct symbols the census's read sites live in — the population the + * page can actually ANCHOR, which is smaller than the site count wherever several + * reads share a symbol. + */ +function distinctSymbolCount(census) { + let total = 0; + for (const entry of symbolPopulation(census).values()) total += entry.symbols.size; + return total; +} + +/** + * How many files hold more than one read inside a single symbol — the size of the + * precision this page trades away, kept enforced so the sentence that prices it + * cannot quietly stop being true in either direction. + */ +function collapsingFileCount(census) { + let files = 0; + for (const entry of symbolPopulation(census).values()) { + if (entry.symbols.size + (entry.fileLevel ? 1 : 0) !== entry.sites) files += 1; + } + return files; +} + export const DECLARED_COUNTS = [ { id: 'headline-sites', @@ -927,6 +1036,30 @@ export const DECLARED_COUNTS = [ value: (c) => c.files.length, why: 'the decomposition table: file count', }, + { + id: 'table-symbols', + pattern: /\| — the distinct symbols those reads live in — what this page anchors \|\s*(\d+) \|/, + value: (c) => distinctSymbolCount(c), + why: 'the decomposition table: what this page can actually anchor, after the collapse', + }, + { + id: 'table-collapsing-files', + pattern: /\| — of those files, the ones holding more than one read in one symbol \|\s*(\d+) \|/, + value: (c) => collapsingFileCount(c), + why: 'the decomposition table: the size of the declared precision loss', + }, + { + id: 'precision-collapsing-files', + pattern: /\*\*(\d+)\*\* of the \*\*\d+\*\*\s*\n?\s*anchored files hold more than one read/, + value: (c) => collapsingFileCount(c), + why: 'the prose that prices the precision loss where a reader meets the anchors', + }, + { + id: 'precision-anchored-files', + pattern: /\*\*\d+\*\* of the \*\*(\d+)\*\*\s*\n?\s*anchored files hold more than one read/, + value: (c) => c.files.length, + why: 'the denominator of that same sentence', + }, { id: 'ruling-sites', pattern: /`isSystem` is a published contract with (\d+) read sites/, @@ -1064,25 +1197,23 @@ export function carryOnwardRowCount(pageText) { return rows.length; } -/** Tracked files, for anchor resolution. */ -export function trackedFiles(root = ROOT) { - const files = execFileSync('git', ['-C', root, 'ls-files'], { - encoding: 'utf8', - maxBuffer: 1 << 28, - }) - .split('\n') - .filter(Boolean); - if (files.length === 0) throw new Error('check-system-context-census: `git ls-files` listed nothing'); - return files; -} - /** - * Locate every `NON_READ_ANCHORS` row by its needle. - * - * @returns {{ located: Map, problems: string[] }} keyed `file:line` + * Resolve every `NON_READ_ANCHORS` row against its file, through the SHARED + * resolution rule. + * + * A row is stale when its file cannot be read, or when the file no longer + * declares its symbol -- the same predicate `sweepCorpus` applies to the page's + * own anchors, so the ledger and the page can never mean different things by + * "the symbol is there". A `symbol: null` row only requires its file to exist. + * + * @param {{ file: string, symbol: string|null, why: string, collapsesOntoRead?: boolean }[]} rows + * @param {(relPath: string) => string} readFile + * @param {Map }>} population the census, by file + * @returns {{ declared: Map, problems: string[] }} keyed `file#symbol`, or `file` */ -export function locateNonReadAnchors(rows, readFile) { - const located = new Map(); +export function resolveNonReadAnchors(rows, readFile, population = new Map()) { + /** @type {Map} */ + const declared = new Map(); const problems = []; for (const row of rows) { let body; @@ -1095,27 +1226,43 @@ export function locateNonReadAnchors(rows, readFile) { ); continue; } - const hits = []; - body.split('\n').forEach((line, i) => { - if (line.includes(row.needle)) hits.push(i + 1); - }); - if (hits.length === 0) { - problems.push( - `[ledger-stale] NON_READ_ANCHORS row for ${row.file} no longer finds its needle ` + - `\`${row.needle}\` -- the construct it excuses is gone or reworded (${row.why}).` - ); - continue; - } - if (hits.length > 1) { - problems.push( - `[ledger-ambiguous] NON_READ_ANCHORS needle \`${row.needle}\` matches ${hits.length} ` + - `lines of ${row.file} (${hits.join(', ')}) -- lengthen it until it is unique.` - ); - continue; + if (row.symbol !== null && row.symbol !== undefined) { + if (!symbolResolutionClass(body, row.file, row.symbol)) { + problems.push( + `[ledger-stale] NON_READ_ANCHORS row for ${row.file} names \`#${row.symbol}\`, which that ` + + `file no longer declares -- the construct it excuses was renamed or removed (${row.why}).` + ); + continue; + } + /* ⭐ The overlap is DECLARED, never inferred. A row whose symbol is also a + * census read symbol excuses nothing (POPULATION already requires that + * anchor); saying so in the row is what stops the next reader from taking + * it for a live exclusion, and holding the declaration to the census in + * BOTH directions is what stops the declaration itself from rotting. */ + const collapses = population.get(row.file)?.symbols.has(row.symbol) === true; + if (collapses && row.collapsesOntoRead !== true) { + problems.push( + `[ledger-undeclared-collapse] NON_READ_ANCHORS row for ${row.file}#${row.symbol} shares its ` + + 'symbol with a census elevation read, so it no longer excuses an anchor. Declare ' + + '`collapsesOntoRead: true` on the row, or re-key it to the symbol it is really about.' + ); + continue; + } + if (!collapses && row.collapsesOntoRead === true) { + problems.push( + `[ledger-stale-collapse] NON_READ_ANCHORS row for ${row.file}#${row.symbol} declares ` + + '`collapsesOntoRead`, but the census finds no elevation read in that symbol any more -- ' + + 'the read moved or was deleted, and this row is excusing an anchor again without anyone ' + + 'having re-read it.' + ); + continue; + } } - located.set(`${row.file}:${hits[0]}`, row); + const key = row.symbol === null || row.symbol === undefined ? row.file : `${row.file}#${row.symbol}`; + if (!declared.has(key)) declared.set(key, []); + declared.get(key).push(row); } - return { located, problems }; + return { declared, problems }; } /** @@ -1126,8 +1273,8 @@ export function locateNonReadAnchors(rows, readFile) { export function evaluate({ pageText, census, - tracked, readFile, + sweep, ledger = NON_READ_ANCHORS, declaredCounts = DECLARED_COUNTS, unenforcedCounts = UNENFORCED_TEXT_COUNTS, @@ -1136,10 +1283,10 @@ export function evaluate({ }) { const problems = []; - const anchors = extractLineAnchors(pageText); + const { anchors } = extractAnchors(pageText); if (anchors.length === 0) { problems.push( - '[no-anchors] the page yielded ZERO `file:line` anchors -- the reader stopped ' + + '[no-anchors] the page yielded ZERO anchors -- the reader stopped ' + 'recognising the page rather than the page being clean.' ); return { problems, stats: { anchors: 0 } }; @@ -1155,86 +1302,125 @@ export function evaluate({ ); } - // ── A. RESOLUTION ─────────────────────────────────────────────────────────── - /** @type {Map} `file:line` -> anchors pointing there */ - const anchored = new Map(); - const fileLengths = new Map(); + // ── A. RESOLUTION — delegated whole to the shared resolver ────────────────── + // + // ⭐ Not re-implemented here, and not optional either. `sweepCorpus` over + // `CORPUS` is what decides that a path is tracked, that a `#symbol` has a + // declaration site, and that a surviving line number is a finding. A sweep this + // gate could not run is a REFUSAL: "could not check" reported as "checked and + // clean" is the silently-degrading verifier this repo refuses on principle. + if (!sweep || !Array.isArray(sweep.findings)) { + problems.push( + '[no-sweep] the shared symbol-anchor resolver was not run over this page, so NOTHING here ' + + 'resolved an anchor. Run `sweepCorpus(CORPUS, root)` and pass its result -- a missing sweep ' + + 'is a failure, never a skip.' + ); + return { problems, stats: { anchors: anchors.length } }; + } + for (const finding of sweep.findings) { + if (finding.soft) continue; + problems.push(`[${finding.kind}] ${finding.doc}:${finding.line} ${finding.raw} -- ${finding.detail}`); + } + + // ── the page's own citations, as SETS ─────────────────────────────────────── + /** @type {Map>} path -> the symbols the page cites into it */ + const pageSymbols = new Map(); + /** @type {Set} paths the page cites with NO symbol -- file-level anchors */ + const pageFileLevel = new Set(); for (const anchor of anchors) { - const resolved = resolveAnchorFile(anchor.spelling, tracked); - if ('error' in resolved) { - problems.push( - resolved.error === 'ambiguous' - ? `[ambiguous-anchor] ${PAGE}:${anchor.docLine} spells \`${anchor.spelling}\`, which ` + - `matches ${resolved.matches.length} tracked files (${resolved.matches.join(', ')}) -- ` + - 'lengthen the spelling until it is unique.' - : `[unresolved-anchor] ${PAGE}:${anchor.docLine} spells \`${anchor.spelling}\`, which ` + - 'matches no tracked file -- the file moved or was deleted.' - ); + if (anchor.repo) continue; // cross-repo: reported by the sweep, resolved nowhere here + if (!anchor.symbol) { + pageFileLevel.add(anchor.path); continue; } - const path = resolved.path; - if (!fileLengths.has(path)) { - try { - fileLengths.set(path, readFile(path).split('\n').length); - } catch { - fileLengths.set(path, -1); - } - } - const length = fileLengths.get(path); - if (length === -1) { - problems.push(`[unreadable-anchor-target] ${path} cannot be read (anchored at ${PAGE}:${anchor.docLine}).`); + if (!pageSymbols.has(anchor.path)) pageSymbols.set(anchor.path, new Set()); + pageSymbols.get(anchor.path).add(anchor.symbol); + } + + const population = symbolPopulation(census); + const { declared, problems: ledgerProblems } = resolveNonReadAnchors(ledger, readFile, population); + problems.push(...ledgerProblems); + + /** The symbols the page is REQUIRED to cite, per file: census ∪ ledger. */ + /** @type {Map>} */ + const required = new Map(); + const requireSymbol = (file, symbol) => { + if (!required.has(file)) required.set(file, new Set()); + required.get(file).add(symbol); + }; + for (const [file, entry] of population) for (const symbol of entry.symbols) requireSymbol(file, symbol); + for (const key of declared.keys()) { + const at = key.indexOf('#'); + if (at !== -1) requireSymbol(key.slice(0, at), key.slice(at + 1)); + } + + // ── B. POPULATION — ⭐ the mandatory direction, per FILE ───────────────────── + // + // Two halves, and the second is what makes the first more than "the file is + // mentioned somewhere": every file with a read must be anchored, and the SET of + // symbols cited into it must equal the set required. Reporting the set + // difference rather than only the counts is deliberate -- the counts are equal + // exactly when the sets are, and a count alone cannot tell an author WHICH + // symbol to write. + const missing = []; + for (const [file, entry] of [...population].sort()) { + const cited = pageSymbols.get(file) ?? new Set(); + const need = required.get(file) ?? new Set(); + if (cited.size === 0 && !pageFileLevel.has(file)) { + missing.push(file); + problems.push( + `[file-without-a-row] ${file} holds ${entry.sites} elevation read site(s) in ` + + `${entry.symbols.size} symbol(s) (${[...entry.symbols].join(', ') || 'none nameable'}) and NO ` + + 'anchor on the page points into it at all. Either the page is missing this file entirely, ' + + 'or every row that cited it rotted off.' + ); continue; } - if (anchor.line < 1 || anchor.line > length) { + for (const symbol of [...entry.symbols].sort()) { + if (cited.has(symbol)) continue; + const sites = census.sites.filter((site) => site.file === file && site.symbol === symbol); + missing.push(`${file}#${symbol}`); problems.push( - `[out-of-range-anchor] ${PAGE}:${anchor.docLine} anchors ${path}:${anchor.line}, ` + - `but that file has ${length} lines.` + `[site-without-a-row] ${file}#${symbol} holds ${sites.length} elevation read(s) ` + + `(\`${(sites[0]?.text ?? '').slice(0, 90)}\`) and no row on the page anchors it. ` + + `This file cites ${cited.size} symbol(s), the census and the ledger require ${need.size}. ` + + 'Either the page is missing this elevation behaviour, or a row rotted off it.' ); - continue; } - const key = `${path}:${anchor.line}`; - if (!anchored.has(key)) anchored.set(key, []); - anchored.get(key).push(anchor); - } - - for (const citation of extractPathCitations(pageText)) { - const resolved = resolveAnchorFile(citation.spelling, tracked); - if ('error' in resolved) { + /* A read with no nameable enclosing declaration is anchored at FILE level -- + * the grammar's own fallback, and the only honest anchor for it. Today the + * census produces none of these; the branch is here so that the first one to + * arrive is a named refusal rather than a shape nothing considered. */ + if (entry.fileLevel && !pageFileLevel.has(file)) { + missing.push(file); problems.push( - `[unresolved-citation] ${PAGE}:${citation.docLine} cites \`${citation.spelling}\`, ` + - `which ${resolved.error === 'ambiguous' ? 'matches several tracked files' : 'matches no tracked file'}.` + `[site-without-a-file-anchor] ${file} holds an elevation read inside no nameable declaration, ` + + 'so it needs a FILE-LEVEL anchor here (the bare path, no `#symbol`) and the page carries none.' ); } } - // ── B. POPULATION — ⭐ the mandatory direction ─────────────────────────────── - const sites = siteKeys(census); - const missing = [...sites].filter((key) => !anchored.has(key)).sort(); - for (const key of missing) { - const site = census.sites.find((s) => `${s.file}:${s.line}` === key); - problems.push( - `[site-without-a-row] ${key} reads \`${site.receiver}.isSystem\` and NO row on the page ` + - `anchors it — \`${site.text.slice(0, 90)}\`. ` + - 'Either the page is missing this elevation behaviour, or an existing row rotted off it.' - ); - } - // ── D. CLASSIFICATION ─────────────────────────────────────────────────────── - const { located, problems: ledgerProblems } = locateNonReadAnchors(ledger, readFile); - problems.push(...ledgerProblems); - const unexplained = [...anchored.keys()].filter((key) => !sites.has(key) && !located.has(key)).sort(); - for (const key of unexplained) { - problems.push( - `[anchor-is-not-a-read-site] the page anchors ${key}, which the census does not call an ` + - 'elevation read and NON_READ_ANCHORS does not declare. Either the line rotted, or the ' + - 'citation is deliberate and needs a ledger row with a needle.' - ); + const unexplained = []; + for (const [file, cited] of [...pageSymbols].sort()) { + const need = required.get(file) ?? new Set(); + for (const symbol of [...cited].sort()) { + if (need.has(symbol)) continue; + unexplained.push(`${file}#${symbol}`); + problems.push( + `[anchor-is-not-a-read-site] the page anchors ${file}#${symbol}, which the census does not ` + + 'call an elevation read and NON_READ_ANCHORS does not declare. Either the symbol was ' + + 'renamed under the row, or the citation is deliberate and needs a ledger row.' + ); + } } - const unusedLedger = [...located.entries()].filter(([key]) => !anchored.has(key)); - for (const [key, row] of unusedLedger) { + for (const [key, rows] of declared) { + const at = key.indexOf('#'); + const used = at === -1 ? pageFileLevel.has(key) : pageSymbols.get(key.slice(0, at))?.has(key.slice(at + 1)); + if (used) continue; problems.push( - `[ledger-row-unused] NON_READ_ANCHORS excuses ${key} (${row.why}) but no anchor on the page ` + - 'points there -- the row outlived the citation, or the anchor rotted off it.' + `[ledger-row-unused] NON_READ_ANCHORS excuses ${key} (${rows.map((r) => r.why).join('; ')}) but no ` + + `anchor on the page points there -- the row outlived the citation, or the anchor was re-keyed.` ); } @@ -1294,202 +1480,65 @@ export function evaluate({ const rowRefs = checkRowReferences({ pageText, ledger, pageRefs: pageRowReferences }); problems.push(...rowRefs.problems); + let citedSymbols = 0; + for (const cited of pageSymbols.values()) citedSymbols += cited.size; + let requiredSymbols = 0; + for (const need of required.values()) requiredSymbols += need.size; + let censusSymbols = 0; + let collapsingFiles = 0; + for (const entry of population.values()) { + censusSymbols += entry.symbols.size; + if (entry.symbols.size + (entry.fileLevel ? 1 : 0) !== entry.sites) collapsingFiles += 1; + } + return { problems, stats: { anchors: anchors.length, - anchorTargets: anchored.size, - sites: sites.size, + citedSymbols, + requiredSymbols, + censusSymbols, + collapsingFiles, + fileLevelAnchors: pageFileLevel.size, + sites: census.sites.length, packages: census.packages.length, files: census.files.length, - nonReadAnchors: located.size, + nonReadAnchors: declared.size, missing: missing.length, + unexplained: unexplained.length, rowRefsHeld: rowRefs.held.page + rowRefs.held.why, rowRefsUnheld: rowRefs.held.unheld, }, }; } -/** A line list for a refusal message, capped so one bad file cannot flood the log. */ -function fmtLines(lines, cap = 14) { - if (lines.length === 0) return '(none)'; - const shown = lines.slice(0, cap).join(', '); - return lines.length > cap ? `${shown}, … (+${lines.length - cap} more)` : shown; +function readFileAt(root) { + return (relPath) => readFileSync(join(root, relPath), 'utf8'); } /** - * The refusal, with everything it compared -- BOTH counts, BOTH target classes, - * and the set difference. - * - * ⭐ Why the sets and not just the counts. Twice now this refusal has been read as - * "your diff added or removed an elevation read site" when nothing of the sort had - * happened, and the output gave the author no way to tell which case they were in - * short of running `isystem-census.mjs --json` in two trees by hand. The last line - * settles it mechanically: if NOTHING is already anchored the page is uniformly - * displaced and some citation is unaccounted for; if everything but one target is - * already anchored, that one target is the site that arrived. + * ⛔ `--fix` has nothing to repair, and says so instead of exiting 0 in silence. + * + * Before #15921 this rewrote line numbers after a pure shift, which was the + * common repair and a real one. Symbol anchors encode no position, so the shift + * that repair existed for cannot happen: an edit above a site moves nothing this + * page writes. Every red is now a human edit -- a rename, an arrived read, a + * vanished one -- and none of them is mechanically derivable from the tree. + * + * ⭐ The flag stays RECOGNISED on purpose. `gen:system-context-census` and + * `scripts/regen-artifacts.mjs` both name it, and a flag that silently became a + * no-op would leave both reading as a working regeneration path. This prints what + * it did not do, then returns the ordinary verdict. */ -function describeRefusal({ path, pageLines, censusLines, ledgerLines, targets, located }) { - const anchoredSet = new Set(pageLines); - const targetSet = new Set(targets); - const alreadyAnchored = targets.filter((line) => anchoredSet.has(line)); - const unanchored = targets.filter((line) => !anchoredSet.has(line)); - const stray = pageLines.filter((line) => !targetSet.has(line)); - const why = ledgerLines - .map((line) => `${line} (${located.get(`${path}:${line}`)?.why ?? 'declared non-read'})`) - .join('; '); - return ( - `${path}: the page anchors ${pageLines.length} distinct line(s) into this file, but the tree ` + - `holds ${targets.length} anchorable line(s) -- ${censusLines.length} census read site(s) plus ` + - `${ledgerLines.length} NON_READ_ANCHORS citation(s). The POPULATION changed, this is not a ` + - 'shift. A row has to be written or deleted by hand.\n' + - ` page anchors ......... ${fmtLines(pageLines)}\n` + - ` census read sites .... ${fmtLines(censusLines)}\n` + - ` ledger-excused ....... ${why || '(none)'}\n` + - ` already anchored ..... ${alreadyAnchored.length} of ${targets.length} target(s)\n` + - ` target, NO anchor .... ${fmtLines(unanchored)}\n` + - ` anchor, NO target .... ${fmtLines(stray)}` +function reportNoFix() { + process.stdout.write( + 'check-system-context-census --fix: nothing to rewrite — this page carries no line numbers.\n' + + ' Anchors are `path#symbol` (scripts/symbol-anchors.mjs), so an unrelated edit above a site\n' + + ' cannot rot one and there is no mechanical repair to apply. A red below is a human edit:\n' + + ' a renamed symbol, a read that arrived, or a read that vanished.\n' ); } -/** - * Rewrite rotted read-site anchors and ledger anchors in place. - * - * Only pure shifts. Per file the page's DISTINCT anchor lines are compared with the - * union of the two classes of line this page is allowed to anchor -- the census's - * read sites and the `NON_READ_ANCHORS` citations -- and rewritten by order when - * the two counts agree. A population change is left for a human. - * - * ## ⛔ Why the ledger cannot be subtracted by LINE (#13490) - * - * The obvious partition -- "a page anchor is a read anchor unless it sits on a - * ledger line" -- compares two DIFFERENT coordinate systems. The page's anchors are - * pre-shift, by construction: rot is the only reason `--fix` is running. The ledger - * lines are post-shift, because a row locates itself by NEEDLE in the current tree. - * So a file whose ledger-excused citation also moved has that citation counted as a - * read anchor, and the gate reports a POPULATION change over a population that - * never moved. Measured twice, in two lanes, on two different files: - * - * security-plugin.ts 7 read sites + 1 ledger citation, all displaced +20/+19 by - * an unrelated bootstrap edit; zero `isSystem` lines added or - * removed. Refusal: "page anchors 8 distinct read line(s), - * census finds 7". (PR #13514, cost a patch round.) - * rest-server.ts 6 read sites + 2 ledger citations, displaced +3/+11 by a - * merge. Refusal: "page anchors 7 ... census finds 6", with - * the contradicting `[ledger-row-unused]` line in the SAME - * run's output. - * - * ⚠️ And it is wrong in the other direction too, which is the dangerous one: a - * stale READ anchor that happens to land on a line the ledger now occupies was - * SUBTRACTED, so the counts could agree by cancellation and the rewrite would map - * the surviving anchors onto each other's rows -- a page that is wrong and GREEN, - * because both classes stay covered. That crossing is real: on the second - * occurrence `rest-server.ts:1267` was simultaneously the second inbound seam's new - * home and a read row's stale anchor. - * - * ⭐ Comparing the UNION removes both directions at once, and buys a postcondition - * the per-class comparison cannot state: the rewrite is a BIJECTION from the page's - * distinct anchor lines onto the file's anchorable lines, so every census site is - * anchored, every ledger row is used and no anchor is unexplained -- for every file - * `--fix` touches, `evaluate` is clean by construction. That is why the union is - * the safer of the two shapes, and it is the one taken: it also refuses when a - * ledger citation was added or dropped without the page following, which comparing - * reads alone would have rewritten straight past. - * - * ⛔ What it still cannot see, stated rather than papered over: alignment is by - * ORDER, so a pure displacement is reconstructed exactly, but a REORDERING that - * moves a cited construct past another one inside the same file is indistinguishable - * from a shift on line numbers alone. No line-only tool can tell those apart -- and - * `evaluate` cannot either, since both classes stay covered. Rows are matched to - * lines by a human there, as they always were. - * - * @returns {{ text: string, rewrites: string[], refused: string[] }} - */ -export function fixAnchors({ pageText, census, tracked, readFile, ledger = NON_READ_ANCHORS }) { - const anchors = extractLineAnchors(pageText); - const { located } = locateNonReadAnchors(ledger, readFile); - /** ledger target lines, per file */ - const ledgerByFile = new Map(); - for (const key of located.keys()) { - const at = key.lastIndexOf(':'); - const file = key.slice(0, at); - if (!ledgerByFile.has(file)) ledgerByFile.set(file, []); - ledgerByFile.get(file).push(Number(key.slice(at + 1))); - } - - /** @type {Map} anchor -> resolved path */ - const paths = new Map(); - for (const anchor of anchors) { - const resolved = resolveAnchorFile(anchor.spelling, tracked); - if ('path' in resolved) paths.set(anchor, resolved.path); - } - - /** @type {Map} anchor -> new line */ - const newLine = new Map(); - const refused = []; - const byFile = new Map(); - for (const anchor of anchors) { - const path = paths.get(anchor); - if (!path) continue; - if (!byFile.has(path)) byFile.set(path, []); - byFile.get(path).push(anchor); - } - for (const [path, fileAnchors] of byFile) { - const ledgerLines = [...new Set(ledgerByFile.get(path) ?? [])].sort((a, b) => a - b); - const censusLines = [...new Set(census.sites.filter((s) => s.file === path).map((s) => s.line))].sort( - (a, b) => a - b - ); - // ⭐ The comparison is against the UNION of both target classes, in one pass. - // A row cites the same line more than once (`:274` appears in the table AND in - // the rough edges), so the comparable unit is a DISTINCT line, not an anchor. - const targets = [...new Set([...censusLines, ...ledgerLines])].sort((a, b) => a - b); - const pageLines = [...new Set(fileAnchors.map((a) => a.line))].sort((a, b) => a - b); - if (pageLines.length !== targets.length) { - refused.push(describeRefusal({ path, pageLines, censusLines, ledgerLines, targets, located })); - continue; - } - const shift = new Map(pageLines.map((line, i) => [line, targets[i]])); - for (const anchor of fileAnchors) { - const to = shift.get(anchor.line); - if (to !== undefined && to !== anchor.line) newLine.set(anchor, to); - } - } - - // Apply, latest anchor first, so earlier offsets stay valid. - const rewrites = []; - let text = pageText; - const ordered = [...newLine.keys()].sort((a, b) => b.docLine - a.docLine || b.raw.length - a.raw.length); - for (const anchor of ordered) { - const to = newLine.get(anchor); - const from = anchor.raw; - const replacement = - anchor.kind === 'full' ? `${anchor.spelling}:${to}` : anchor.kind === 'continuation' ? `:${to}` : `${to}`; - const needle = `\`${from}\``; - const at = text.indexOf(needle, offsetOfDocLine(text, anchor.docLine)); - if (at === -1) { - refused.push(`could not re-find \`${from}\` at ${PAGE}:${anchor.docLine}`); - continue; - } - text = `${text.slice(0, at)}\`${replacement}\`${text.slice(at + needle.length)}`; - rewrites.push(`${PAGE}:${anchor.docLine} \`${from}\` -> \`${replacement}\``); - } - return { text, rewrites, refused }; -} - -function offsetOfDocLine(text, docLine) { - let offset = 0; - for (let n = 1; n < docLine; n += 1) { - const at = text.indexOf('\n', offset); - if (at === -1) return offset; - offset = at + 1; - } - return offset; -} - -function readFileAt(root) { - return (relPath) => readFileSync(join(root, relPath), 'utf8'); -} - function run({ fix = false } = {}) { const readFile = readFileAt(ROOT); let pageText; @@ -1499,34 +1548,40 @@ function run({ fix = false } = {}) { process.stderr.write(`::error::[unreadable-page] ${PAGE} could not be read -- ${error.message}\n`); return 1; } + if (fix) reportNoFix(); + const census = runCensus({ root: ROOT }); - const tracked = trackedFiles(ROOT); - - if (fix) { - const { text, rewrites, refused } = fixAnchors({ pageText, census, tracked, readFile }); - if (rewrites.length > 0) writeFileSync(join(ROOT, PAGE), text); - for (const line of rewrites) process.stdout.write(` re-anchored ${line}\n`); - for (const line of refused) process.stdout.write(` ⛔ NOT fixable: ${line}\n`); - process.stdout.write(`check-system-context-census --fix: ${rewrites.length} anchor(s) rewritten\n`); - pageText = text; + /* RESOLUTION is the shared resolver's, run once, over this gate's corpus + * registration. `evaluate` reports what it found and re-decides none of it. */ + const sweep = sweepCorpus(CORPUS, ROOT); + if (sweep.counts.docs === 0) { + process.stderr.write( + `::error::[corpus-empty] the \`${CORPUS.id}\` corpus swept ZERO documents -- ${PAGE} moved out ` + + 'from under this gate, which would otherwise report a clean sweep over nothing.\n' + ); + return 1; } - const { problems, stats } = evaluate({ pageText, census, tracked, readFile }); + const { problems, stats } = evaluate({ pageText, census, readFile, sweep }); for (const problem of problems) process.stderr.write(`::error::${problem}\n`); if (problems.length > 0) { process.stderr.write( `\ncheck-system-context-census: ${problems.length} problem(s) over ${stats.anchors} anchors ` + `and ${stats.sites} census sites.\n` + - `Re-run the census with \`node scripts/isystem-census.mjs --json\`; pure line rot is ` + - `repaired by \`node scripts/check-system-context-census.mjs --fix\`.\n` + 'Re-run the census with `node scripts/isystem-census.mjs --json`. ⛔ There is no mechanical ' + + 'repair: every red here is a human edit.\n' + + `\nThe anchor grammar:\n ${ANCHOR_GRAMMAR}\n` ); return 1; } process.stdout.write( `check-system-context-census: OK — ${stats.sites} elevation read sites in ${stats.packages} ` + - `packages across ${stats.files} files, all anchored; ${stats.anchors} anchors resolve, ` + - `${stats.nonReadAnchors} declared non-read; ${stats.rowRefsHeld} row reference(s) resolve to ` + - `their keyed row, ${stats.rowRefsUnheld} declared unheld.\n` + `packages across ${stats.files} files, living in ${stats.censusSymbols} symbol(s); the page ` + + `cites ${stats.citedSymbols} symbol(s) against ${stats.requiredSymbols} required, over ` + + `${stats.anchors} anchors and ${stats.fileLevelAnchors} file-level citation(s); ` + + `${stats.nonReadAnchors} declared non-read; ${stats.collapsingFiles} file(s) hold more than one ` + + `read in one symbol (the declared precision loss); ${stats.rowRefsHeld} row reference(s) resolve ` + + `to their keyed row, ${stats.rowRefsUnheld} declared unheld.\n` ); return 0; } @@ -1541,10 +1596,13 @@ const FIXTURE_SOURCE = [ '}', // 5 '// the sys_ prefix helper lives here', // 6 'export function isSystemObjectName(name: string) { return name.startsWith("sys_"); }', // 7 + 'export function unrelated() { return 1; }', // 8 ].join('\n'); const FIXTURE_CENSUS = { - sites: [{ file: 'pkg/a.ts', line: 2, receiver: 'ctx', package: 'pkg', text: 'if (ctx.isSystem) return ALLOW;' }], + sites: [ + { file: 'pkg/a.ts', line: 2, receiver: 'ctx', package: 'pkg', symbol: 'handler', text: 'if (ctx.isSystem) return ALLOW;' }, + ], nonElevationReads: [{ file: 'pkg/a.ts', line: 3, receiver: 'obj', field: 'Object.isSystem' }], roleCounts: { read: 2, declaration: 0, key: 0, other: 0 }, packages: ['pkg'], @@ -1555,70 +1613,23 @@ const FIXTURE_CENSUS = { }; const FIXTURE_LEDGER = [ - { file: 'pkg/a.ts', needle: 'export function isSystemObjectName', why: 'name-prefix helper, not a read' }, -]; - -/** - * ⭐ The CROSSING fixture (#13490). `pkg/a.ts` puts its read ABOVE its ledger - * citation, which is the easy order: a stale read anchor can never land on the - * ledger's line. Here the citation sits BELOW the read site, so a displacement - * walks the citation onto ground a read anchor used to hold -- and the ledger was - * subtracted by LINE, so that read anchor was subtracted with it. The counts then - * agreed by cancellation and the rewrite mapped the two surviving anchors onto - * each other's rows: a page that is WRONG and GREEN, because both classes stay - * covered and nothing downstream compares a row to its meaning. That crossing is - * not hypothetical -- `rest-server.ts:1267` was simultaneously the second inbound - * seam's new home and a read row's stale anchor on the second occurrence. - */ -const CROSSING_SOURCE = [ - 'export function guard(ctx: ExecutionContext) {', // 1 - ' const kind = classify(ctx);', // 2 - ' if (isSystemObjectName(ctx.objectName)) return SKIP;', // 3 <- ledger needle - ' audit(kind);', // 4 - ' if (ctx.isSystem) return ALLOW;', // 5 <- the elevation read - ' return DENY;', // 6 - '}', // 7 -].join('\n'); - -const CROSSING_CENSUS = { - ...FIXTURE_CENSUS, - sites: [{ file: 'pkg/b.ts', line: 5, receiver: 'ctx', package: 'pkg', text: 'if (ctx.isSystem) return ALLOW;' }], - files: ['pkg/b.ts'], -}; - -const CROSSING_LEDGER = [ - { - file: 'pkg/b.ts', - needle: 'isSystemObjectName(ctx.objectName)', - why: 'the sys_ name-prefix helper call, not a read', - }, + { file: 'pkg/a.ts', symbol: 'isSystemObjectName', why: 'name-prefix helper, not a read' }, ]; function fixtureRead(relPath) { if (relPath === 'pkg/a.ts') return FIXTURE_SOURCE; - if (relPath === 'pkg/b.ts') return CROSSING_SOURCE; throw new Error(`no fixture for ${relPath}`); } -const FIXTURE_TRACKED = ['pkg/a.ts', 'other/a.ts', 'pkg/b.ts']; - /** - * One anchor per line, so the two slots can be told apart AFTER a rewrite: the - * question these cases ask is not "are both lines covered" -- the buggy fixer - * covered both -- but "did each ROW keep its own line". + * A sweep result that found nothing. RESOLUTION is delegated to `sweepCorpus`, + * which walks a real tree; the fixtures below drive POPULATION, CLASSIFICATION and + * COUNTS, and hand `evaluate` the shape a clean sweep returns. + * + * ⭐ It is passed EXPLICITLY, never defaulted: `evaluate` refuses a missing sweep, + * and a default would let that refusal be forgotten by every caller at once. */ -function crossingPage({ read = 'pkg/b.ts:5', helper = 'pkg/b.ts:3' } = {}) { - return [ - '---', - 'title: crossing fixture', - '---', - '', - 'the elevation read at `' + read + '`.', - '', - 'the name helper at `' + helper + '`.', - '', - ].join('\n'); -} +const CLEAN_SWEEP = { findings: [], counts: { docs: 1, anchors: 2 } }; /** A one-row stand-in for `DECLARED_COUNTS`, so the fixtures need one sentence. */ const FIXTURE_COUNTS = [ @@ -1630,7 +1641,7 @@ const FIXTURE_COUNTS = [ }, ]; -function fixturePage({ anchor = 'pkg/a.ts:2', helper = 'pkg/a.ts:7' } = {}) { +function fixturePage({ anchor = 'pkg/a.ts#handler', helper = 'pkg/a.ts#isSystemObjectName' } = {}) { return [ '---', 'title: fixture', @@ -1639,12 +1650,75 @@ function fixturePage({ anchor = 'pkg/a.ts:2', helper = 'pkg/a.ts:7' } = {}) { 'read at `' + anchor + '` and the name helper at `' + helper + '`.', '', '```bash', - 'grep -rn "isSystem" packages # `pkg/a.ts:999` inside a fence is not an anchor', + 'grep -rn "isSystem" packages # not an anchor: fenced material is quoted, not cited', '```', '', ].join('\n'); } +/** + * ── The RED-FIRST corpus, as a real git tree ──────────────────────────────── + * + * ⭐ The ruled headline pair — a symbol rename REDS, a pure line move does NOT — + * cannot be shown on an in-memory fixture: `sweepCorpus` resolves against + * `git ls-files` and reads the target from disk, so a fixture that skipped either + * would be pinning something other than the gate. This builds a two-file tree, + * `git init`s it and stages it, which is the whole of what the resolver needs. + * + * ⛔ Nothing here ever touches the real repository. Every case reads back what it + * wrote and the temp dir is removed in a `finally`. + * + * ## ⛔ Why every `git` call below runs with a STRIPPED environment (measured) + * + * This self-test shells out to `git` -- here, and one frame down inside + * `sweepCorpus`, which asks `git ls-files` what a corpus's tracked files are. + * From a plain shell that is harmless. From INSIDE A GIT HOOK it is not: git + * exports `GIT_DIR`, `GIT_WORK_TREE` and `GIT_INDEX_FILE`, every child `git` + * inherits them, and then the throwaway corpus's `git init` creates nothing + * while its `git add -A` writes THE REPOSITORY'S INDEX. + * + * ⭐ That is not hypothetical and it is not a rare path: `check-regen-pending` + * runs this gate from `pre-commit`, which is precisely where an os-regen merge + * lap lands. Measured once, on this file's own branch, during exactly that lap: + * 8,190 paths staged as deleted and the fixture's own `pkg/a.ts` staged into the + * real index, from a self-test whose every case still printed `ok`. + * + * ⛔ The failure is SILENT in the direction that matters -- the self-test passes, + * and the damage is to a tree nobody was looking at. So the environment is + * stripped for the duration of the self-test AND passed stripped to each child + * here, rather than relying on either one alone. + * + * @param {{ symbol?: string, pad?: number }} shape + * @returns {{ dir: string, sourceLine: number }} + */ +function gitFreeEnv() { + const env = { ...process.env }; + for (const key of Object.keys(env)) if (key.startsWith('GIT_')) delete env[key]; + return env; +} + +function buildRedFirstCorpus({ symbol = 'handler', pad = 0 } = {}) { + const dir = mkdtempSync(join(tmpdir(), 'system-context-corpus-')); + mkdirSync(join(dir, 'content', 'docs', 'permissions'), { recursive: true }); + mkdirSync(join(dir, 'pkg'), { recursive: true }); + const head = Array.from({ length: pad }, (_, i) => `// padding line ${i + 1}`); + const body = [ + `export function ${symbol}(ctx: ExecutionContext) {`, + ' if (ctx.isSystem) return ALLOW;', + ' return DENY;', + '}', + ]; + writeFileSync(join(dir, 'pkg', 'a.ts'), [...head, ...body].join('\n')); + writeFileSync( + join(dir, 'content', 'docs', 'permissions', 'system-context.mdx'), + ['---', 'title: red-first fixture', '---', '', 'the elevation read lives at `pkg/a.ts#handler`.', ''].join('\n') + ); + const env = gitFreeEnv(); + execFileSync('git', ['init', '-q'], { cwd: dir, env }); + execFileSync('git', ['add', '-A'], { cwd: dir, env }); + return { dir, sourceLine: pad + 2 }; +} + /** * ── The ROW REFERENCE fixture (#15869) ────────────────────────────────────── * @@ -1665,16 +1739,16 @@ const ROW_FIXTURE_PAGE = [ '', '| # | Behaviour | Anchor |', '|:--|:---|:---|', - '| 1 | **The short-circuit** runs first | `pkg/a.ts:2` |', - '| 2 | **`owner_id` is not stamped** on INSERT | `pkg/a.ts:3` |', - '| 3 | `revoke()` deletes directly, before the guard | `pkg/a.ts:4` |', + '| 1 | **The short-circuit** runs first | `pkg/a.ts#handler` |', + '| 2 | **`owner_id` is not stamped** on INSERT | `pkg/a.ts#stamp` |', + '| 3 | `revoke()` deletes directly, before the guard | `pkg/a.ts#revoke` |', '', '### 6. Reads that only carry the flag onward', '', '| # | Site | What it does |', '|:--|:---|:---|', - '| 4 | `pkg/b.ts:5` | Propagates the flag onward |', - '| 5 | `pkg/b.ts:6` | Rebuilds the context |', + '| 4 | `pkg/b.ts#carry` | Propagates the flag onward |', + '| 5 | `pkg/b.ts#rebuild` | Rebuilds the context |', '', '1. **`revoke()` skips its own conflict guard.** Row 3 is correct for the rule.', "2. The shared verdict is the one function all of row 2's doors consult.", @@ -1705,17 +1779,17 @@ const ROW_FIXTURE_REFS = [ const ROW_FIXTURE_LEDGER = [ { file: 'pkg/a.ts', - needle: 'a', + symbol: 'handler', why: 'row 3 -- the guard `revoke()` deletes in front of', rowSeams: ['`revoke()` deletes directly'], }, { file: 'pkg/a.ts', - needle: 'b', + symbol: 'isSystemObjectName', why: 'row 2 -- the stamp guard the row-1 short-circuit skips', rowSeams: ['**`owner_id` is not stamped**', '**The short-circuit** runs first'], }, - { file: 'pkg/a.ts', needle: 'c', why: 'rough edge 5 -- names no row at all', rowSeams: [] }, + { file: 'pkg/a.ts', symbol: 'unrelated', why: 'rough edge 5 -- names no row at all', rowSeams: [] }, ]; /** @@ -1775,6 +1849,17 @@ function fixtureUnenforcedTable({ linesTotal = 6, dropTestsRow = false, dated = let selfTestReachedVerdict = false; function selfTest() { + /* ⛔ Detach from any inherited git environment BEFORE the first case. See + * `buildRedFirstCorpus`'s header for the measured incident: under `pre-commit` + * this function's children would otherwise write the REPOSITORY's index. This + * covers `sweepCorpus`'s own `git ls-files` too, which lives in the shared + * resolver and is not this gate's to change. Restored before the return, so an + * in-process caller gets its environment back. */ + const savedGitEnv = Object.fromEntries( + Object.keys(process.env).filter((key) => key.startsWith('GIT_')).map((key) => [key, process.env[key]]) + ); + for (const key of Object.keys(savedGitEnv)) delete process.env[key]; + // The battery ledger this self-test's floor is evaluated against (#13489). // `battery()` opens a battery; every assertion below is attributed to the one // most recently opened, so a section that stops running stops registering and @@ -1795,12 +1880,12 @@ function selfTest() { if (!ok) failures += 1; process.stdout.write(`${ok ? ' ok ' : ' FAIL'} ${name}${detail ? ` -- ${detail}` : ''}\n`); }; - const run = (page, census = FIXTURE_CENSUS, declaredCounts = [], unenforcedCounts = []) => + const run = (page, census = FIXTURE_CENSUS, declaredCounts = [], unenforcedCounts = [], extra = {}) => evaluate({ pageText: page, census, - tracked: FIXTURE_TRACKED, readFile: fixtureRead, + sweep: CLEAN_SWEEP, ledger: FIXTURE_LEDGER, declaredCounts, unenforcedCounts, @@ -1809,24 +1894,43 @@ function selfTest() { // carry no table at all. The battery below drives that check on its own // fixtures and on the real page. pageRowReferences: [], + ...extra, }); // ── the GREEN control: a page that is correct ─────────────────────────────── battery('the GREEN control: a page that is correct'); const green = run(fixturePage()); t('green control: a correct page reports nothing', green.problems.length === 0, green.problems.join(' | ')); - t('green control: the fenced `pkg/a.ts:999` is not read as an anchor', green.stats.anchors === 2); + t( + 'green control: the page cites exactly the symbols the census and the ledger require', + green.stats.citedSymbols === 2 && green.stats.requiredSymbols === 2 && green.stats.censusSymbols === 1, + JSON.stringify(green.stats) + ); // ── ⭐ the RED that matters: a site the page never mentions ────────────────── battery('⭐ the RED that matters: a site the page never mentions'); const arrived = { ...FIXTURE_CENSUS, - sites: [...FIXTURE_CENSUS.sites, { file: 'pkg/a.ts', line: 4, receiver: 'ctx', package: 'pkg', text: 'return DENY;' }], + sites: [ + ...FIXTURE_CENSUS.sites, + { file: 'pkg/a.ts', line: 8, receiver: 'ctx', package: 'pkg', symbol: 'unrelated', text: 'export function unrelated() { return 1; }' }, + ], }; const missing = run(fixturePage(), arrived); t( - 'POPULATION: a read site with no row is a finding', - missing.problems.some((p) => p.startsWith('[site-without-a-row] pkg/a.ts:4')) + 'POPULATION: a read in a symbol no row anchors is a finding, naming the symbol', + missing.problems.some((p) => p.startsWith('[site-without-a-row] pkg/a.ts#unrelated')), + missing.problems.join(' | ') + ); + const wholeFileMissing = run(fixturePage({ anchor: 'pkg/a.ts#unrelated', helper: 'pkg/a.ts#isSystemObjectName' }), { + ...FIXTURE_CENSUS, + sites: [{ file: 'pkg/b.ts', line: 1, receiver: 'ctx', package: 'pkg', symbol: 'guard', text: 'x' }], + files: ['pkg/b.ts'], + }); + t( + 'POPULATION: a whole FILE with reads and no anchor at all is its own finding', + wholeFileMissing.problems.some((p) => p.startsWith('[file-without-a-row] pkg/b.ts')), + wholeFileMissing.problems.join(' | ') ); // ── the deletion shape: the row stands, the site is gone ──────────────────── @@ -1837,67 +1941,104 @@ function selfTest() { const shrunk = { ...FIXTURE_CENSUS, - sites: [{ file: 'pkg/a.ts', line: 4, receiver: 'ctx', package: 'pkg', text: 'return DENY;' }], + sites: [{ file: 'pkg/a.ts', line: 8, receiver: 'ctx', package: 'pkg', symbol: 'unrelated', text: 'export function unrelated() { return 1; }' }], }; const stale = run(fixturePage(), shrunk); t( - 'DELETION: a row anchoring a line that is no longer a read site is a finding', - stale.problems.some((p) => p.startsWith('[anchor-is-not-a-read-site]') && p.includes('pkg/a.ts:2')) + 'DELETION: a row anchoring a symbol that is no longer a read site is a finding', + stale.problems.some((p) => p.startsWith('[anchor-is-not-a-read-site]') && p.includes('pkg/a.ts#handler')), + stale.problems.join(' | ') + ); + t( + 'DELETION: and the arrived symbol is named on the other side in the same run', + stale.problems.some((p) => p.startsWith('[site-without-a-row] pkg/a.ts#unrelated')), + stale.problems.join(' | ') ); - // ── rot ──────────────────────────────────────────────────────────────────── - const rotted = run(fixturePage({ anchor: 'pkg/a.ts:4' })); - t('ROT: a shifted anchor is caught from both sides', rotted.problems.length === 2, rotted.problems.join(' | ')); - - // ── resolution ───────────────────────────────────────────────────────────── - battery('resolution'); - const ambiguous = run(fixturePage({ anchor: 'a.ts:2' })); - t('RESOLUTION: a bare basename matching two files is refused', ambiguous.problems.some((p) => p.startsWith('[ambiguous-anchor]'))); - const gonefile = run(fixturePage({ anchor: 'pkg/nope.ts:2' })); - t('RESOLUTION: an anchor to a file that does not exist is refused', gonefile.problems.some((p) => p.startsWith('[unresolved-anchor]'))); - const overrun = run(fixturePage({ anchor: 'pkg/a.ts:999' })); - t('RESOLUTION: a line past end of file is refused', overrun.problems.some((p) => p.startsWith('[out-of-range-anchor]'))); + // ── ⭐ RESOLUTION is DELEGATED, and a sweep that did not run is a REFUSAL ──── + // + // ⛔ This gate must not re-decide what "the symbol is in that file" means -- + // two copies of that rule drift silently, each green on its own corpus. So the + // only thing asserted here is the delegation itself: the sweep's findings are + // surfaced verbatim, and its ABSENCE is a refusal rather than a quiet pass. + battery('⭐ RESOLUTION is delegated, and a sweep that did not run is a REFUSAL'); + const noSweep = run(fixturePage(), FIXTURE_CENSUS, [], [], { sweep: undefined }); + t( + 'RESOLUTION: a missing sweep REFUSES -- "could not check" never reports as "checked and clean"', + noSweep.problems.some((p) => p.startsWith('[no-sweep]')), + noSweep.problems.join(' | ') + ); + const brokenSweep = run(fixturePage(), FIXTURE_CENSUS, [], [], { sweep: { counts: {} } }); + t( + 'RESOLUTION: a sweep object with no findings ARRAY is refused too, not read as zero findings', + brokenSweep.problems.some((p) => p.startsWith('[no-sweep]')), + brokenSweep.problems.join(' | ') + ); + const sweptRed = run(fixturePage(), FIXTURE_CENSUS, [], [], { + sweep: { + findings: [ + { kind: 'unresolved-symbol', doc: PAGE, line: 5, raw: '`pkg/a.ts#handler`', detail: '`handler` has no declaration site' }, + { kind: 'cross-repo-skipped', doc: PAGE, line: 5, raw: '`objectui:x.ts`', detail: 'no checkout', soft: true }, + ], + }, + }); + t( + "RESOLUTION: the sweep's hard findings are surfaced verbatim and its soft ones are not", + sweptRed.problems.some((p) => p.startsWith('[unresolved-symbol]')) && + !sweptRed.problems.some((p) => p.includes('cross-repo-skipped')), + sweptRed.problems.join(' | ') + ); // ── ledger ───────────────────────────────────────────────────────────────── battery('ledger'); - const ledgerStale = evaluate({ - pageRowReferences: [], - pageText: fixturePage(), - census: FIXTURE_CENSUS, - tracked: FIXTURE_TRACKED, - readFile: fixtureRead, - ledger: [{ file: 'pkg/a.ts', needle: 'no such text anywhere', why: 'x' }], - unenforcedCounts: [], - declaredCounts: [], - }); - t('LEDGER: a needle that matches nothing is a finding', ledgerStale.problems.some((p) => p.startsWith('[ledger-stale]'))); - const ledgerAmbig = evaluate({ - pageRowReferences: [], - pageText: fixturePage(), - census: FIXTURE_CENSUS, - tracked: FIXTURE_TRACKED, - readFile: fixtureRead, - ledger: [{ file: 'pkg/a.ts', needle: 'return', why: 'x' }], - unenforcedCounts: [], - declaredCounts: [], - }); - t('LEDGER: a needle matching two lines is a finding', ledgerAmbig.problems.some((p) => p.startsWith('[ledger-ambiguous]'))); - const ledgerUnused = evaluate({ - pageRowReferences: [], - pageText: fixturePage({ helper: 'pkg/a.ts:2' }), - census: FIXTURE_CENSUS, - tracked: FIXTURE_TRACKED, - readFile: fixtureRead, - ledger: FIXTURE_LEDGER, - unenforcedCounts: [], - declaredCounts: [], - }); - t('LEDGER: a row no anchor uses is a finding', ledgerUnused.problems.some((p) => p.startsWith('[ledger-row-unused]'))); + const ledgerRun = (ledger, page = fixturePage()) => + evaluate({ + pageRowReferences: [], + pageText: page, + census: FIXTURE_CENSUS, + readFile: fixtureRead, + sweep: CLEAN_SWEEP, + ledger, + unenforcedCounts: [], + declaredCounts: [], + }); + t( + 'LEDGER: a symbol the file no longer declares is a finding', + ledgerRun([{ file: 'pkg/a.ts', symbol: 'noSuchSymbol', why: 'x' }]).problems.some((p) => + p.startsWith('[ledger-stale]') + ) + ); + t( + 'LEDGER: a file that cannot be read is a finding, not a skipped row', + ledgerRun([{ file: 'pkg/gone.ts', symbol: 'x', why: 'x' }]).problems.some((p) => + p.startsWith('[ledger-unreadable]') + ) + ); + t( + 'LEDGER: a row no anchor uses is a finding', + ledgerRun(FIXTURE_LEDGER, fixturePage({ helper: 'pkg/a.ts#handler' })).problems.some((p) => + p.startsWith('[ledger-row-unused]') + ) + ); + t( + '⭐ LEDGER: an UNDECLARED overlap with a census read symbol is a finding — a row that excuses ' + + 'nothing must say so', + ledgerRun([{ file: 'pkg/a.ts', symbol: 'handler', why: 'x' }]).problems.some((p) => + p.startsWith('[ledger-undeclared-collapse]') + ) + ); + t( + '⭐ LEDGER: and a DECLARED overlap that has ended is a finding too — the declaration cannot rot ' + + 'in the safe direction either', + ledgerRun([{ file: 'pkg/a.ts', symbol: 'isSystemObjectName', collapsesOntoRead: true, why: 'x' }]).problems.some( + (p) => p.startsWith('[ledger-stale-collapse]') + ) + ); // ── counts ───────────────────────────────────────────────────────────────── battery('counts'); - const countPage = - fixturePage() + '\nit is a single boolean read at **1\ndistinct sites across 1 packages**.\n'; + const countSentence = '\nit is a single boolean read at **1\ndistinct sites across 1 packages**.\n'; + const countPage = fixturePage() + countSentence; const countsOk = run(countPage, FIXTURE_CENSUS, FIXTURE_COUNTS); t( 'COUNTS: a matching declared count is silent', @@ -1922,8 +2063,8 @@ function selfTest() { '', '| # | Site |', '|:--|:---|', - '| 62 | `pkg/a.ts:2` |', - '| 63 | `pkg/a.ts:2` |', + '| 62 | `pkg/a.ts#handler` |', + '| 63 | `pkg/a.ts#handler` |', '', '---', '', @@ -1935,12 +2076,12 @@ function selfTest() { pageRowReferences: [], pageText: fixturePage(), census: FIXTURE_CENSUS, - tracked: FIXTURE_TRACKED, readFile: fixtureRead, + sweep: CLEAN_SWEEP, ledger: FIXTURE_LEDGER, unenforcedCounts: [], declaredCounts: [ - { id: 'x', pattern: /helper at `pkg\/a\.ts:(\d+)`/, value: () => carryOnwardRowCount('gone'), why: 'fixture' }, + { id: 'x', pattern: /helper at `pkg\/a\.ts#(\w+)`/, value: () => carryOnwardRowCount('gone'), why: 'fixture' }, ], }); t( @@ -1957,8 +2098,7 @@ function selfTest() { // // ⭐ These two cases run over `DECLARED_COUNTS` and `UNENFORCED_TEXT_COUNTS` // THEMSELVES, not over a fixture stand-in. That is the point: move a text count - // back into the enforced list and the first case names it by id. A criterion - // change with nothing watching it is how the next reader undoes it. + // back into the enforced list and the first case names it by id. battery('⭐ CRITERION: enforced means CENSUS-DERIVED, pinned over the REAL lists'); const textDrifted = { ...FIXTURE_CENSUS, @@ -1985,7 +2125,6 @@ function selfTest() { // ── the same criterion, behaviourally, on one page ────────────────────────── battery('the same criterion, behaviourally, on one page'); - const countSentence = '\nit is a single boolean read at **1\ndistinct sites across 1 packages**.\n'; const okPage = fixturePage() + countSentence; const staleText = run( okPage + fixtureUnenforcedTable({ linesTotal: 999 }), @@ -1998,7 +2137,7 @@ function selfTest() { staleText.problems.length === 0, staleText.problems.join(' | ') ); - const rowGone = run( + const rowGoneCase = run( okPage + fixtureUnenforcedTable({ dropTestsRow: true }), textDrifted, FIXTURE_COUNTS, @@ -2006,8 +2145,8 @@ function selfTest() { ); t( 'CRITERION: an unenforced row reworded off the page IS a finding', - rowGone.problems.some((p) => p.startsWith('[unenforced-count-missing]') && p.includes('`table-lines-tests`')), - rowGone.problems.join(' | ') + rowGoneCase.problems.some((p) => p.startsWith('[unenforced-count-missing]') && p.includes('`table-lines-tests`')), + rowGoneCase.problems.join(' | ') ); const undated = run( okPage + fixtureUnenforcedTable({ dated: false }), @@ -2024,13 +2163,13 @@ function selfTest() { // ── ⛔ and the half that must NOT have moved: the contract still reds ──────── battery('⛔ and the half that must NOT have moved: the contract still reds'); const rottedToo = run( - fixturePage({ anchor: 'pkg/a.ts:4' }) + countSentence + fixtureUnenforcedTable({ linesTotal: 999 }), + fixturePage({ anchor: 'pkg/a.ts#unrelated' }) + countSentence + fixtureUnenforcedTable({ linesTotal: 999 }), FIXTURE_CENSUS, FIXTURE_COUNTS, UNENFORCED_TEXT_COUNTS ); t( - 'CRITERION: a rotted ANCHOR still reds on the very page whose text counts are stale', + 'CRITERION: a re-keyed ANCHOR still reds on the very page whose text counts are stale', rottedToo.problems.some((p) => p.startsWith('[site-without-a-row]')) && rottedToo.problems.some((p) => p.startsWith('[anchor-is-not-a-read-site]')), rottedToo.problems.join(' | ') @@ -2043,7 +2182,7 @@ function selfTest() { ); t( 'CRITERION: a POPULATION change still reds on that same page', - grewToo.problems.some((p) => p.startsWith('[site-without-a-row] pkg/a.ts:4')) && + grewToo.problems.some((p) => p.startsWith('[site-without-a-row] pkg/a.ts#unrelated')) && grewToo.problems.some((p) => p.includes('`headline-sites` says 1, the census says 2')), grewToo.problems.join(' | ') ); @@ -2053,169 +2192,188 @@ function selfTest() { const noAnchors = run('---\ntitle: x\n---\n\nnothing here.\n'); t('ABSENCE: a page with no anchors refuses', noAnchors.problems.some((p) => p.startsWith('[no-anchors]'))); - // ── --fix ────────────────────────────────────────────────────────────────── - battery('--fix'); - const fixed = fixAnchors({ - pageText: fixturePage({ anchor: 'pkg/a.ts:4' }), - census: FIXTURE_CENSUS, - tracked: FIXTURE_TRACKED, - readFile: fixtureRead, - ledger: FIXTURE_LEDGER, - }); - t('FIX: a pure shift is rewritten', fixed.text.includes('`pkg/a.ts:2`'), fixed.rewrites.join(' | ')); - const refusedFix = fixAnchors({ - pageText: fixturePage(), - census: arrived, - tracked: FIXTURE_TRACKED, - readFile: fixtureRead, - ledger: FIXTURE_LEDGER, - }); - t( - 'FIX: a population change is REFUSED, never guessed', - refusedFix.rewrites.length === 0 && refusedFix.refused.length === 1, - JSON.stringify(refusedFix.refused) - ); + // ── ⛔ --fix rewrites NOTHING, and says so ────────────────────────────────── + // + // ⭐ The failure this pins is the QUIET one. `gen:system-context-census` and + // `scripts/regen-artifacts.mjs` both invoke `--fix`; a flag that silently became + // a no-op leaves both of them reading as a working regeneration path, and the + // first person to hit a red would run it, see exit 0, and conclude the red was + // spurious. + battery('⛔ --fix rewrites NOTHING, and says so'); + let ownSourceForFix = null; + try { + ownSourceForFix = readFileSync(join(ROOT, 'scripts/check-system-context-census.mjs'), 'utf8'); + } catch (err) { + t('--fix: this gate can read its own source', false, err.code ?? err.message); + } + if (ownSourceForFix !== null) { + t( + '--fix: no anchor-rewriting code survives -- nothing in this gate writes the page', + !/writeFileSync\(\s*join\(ROOT, PAGE\)/.test(ownSourceForFix) && !/\bfixAnchors\b/.test(ownSourceForFix), + 'a rewriter is back in this file' + ); + t( + '--fix: the flag is still RECOGNISED and still explains itself, so the wiring cannot go quiet', + /argv\.includes\('--fix'\)/.test(ownSourceForFix) && /nothing to rewrite/.test(ownSourceForFix) + ); + } - // ── ⭐ #13490: the incident shape -- reads AND ledger citations BOTH shift ──── + // ── ⭐ THE RULED RED-FIRST PAIR, on a real tree ───────────────────────────── // - // The pre-existing case above shifts the read only, which is why it never caught - // this: a page anchor is pre-shift by construction, a ledger line is located by - // needle in the CURRENT tree, and subtracting one from the other counts the - // displaced citation as a read anchor. Measured on two files in two lanes -- - // `security-plugin.ts` (7 reads + 1 citation, all +20/+19, zero `isSystem` lines - // added or removed) refused with "page anchors 8 distinct read line(s), census - // finds 7", and `rest-server.ts` (6 + 2) with "7 ... finds 6". - battery('⭐ #13490: the incident shape -- reads AND ledger citations BOTH shift'); - const bothShifted = fixAnchors({ - pageText: fixturePage({ anchor: 'pkg/a.ts:1', helper: 'pkg/a.ts:6' }), - census: FIXTURE_CENSUS, - tracked: FIXTURE_TRACKED, - readFile: fixtureRead, - ledger: FIXTURE_LEDGER, - }); - t( - 'FIX #13490: a shift that moves the LEDGER citation too is a shift, not a population change', - bothShifted.refused.length === 0 && - bothShifted.text.includes('`pkg/a.ts:2`') && - bothShifted.text.includes('`pkg/a.ts:7`'), - `refused=${JSON.stringify(bothShifted.refused)} rewrites=${JSON.stringify(bothShifted.rewrites)}` - ); - - // ⭐ The postcondition the union buys: the rewrite is a BIJECTION from the page's - // distinct anchor lines onto the file's anchorable lines, so a file `--fix` - // touched cannot come back with a missing site, an unexplained anchor or an - // unused ledger row. Pinned behaviourally rather than argued in a comment. - const afterFix = evaluate({ - pageRowReferences: [], - pageText: bothShifted.text, - census: FIXTURE_CENSUS, - tracked: FIXTURE_TRACKED, - readFile: fixtureRead, - ledger: FIXTURE_LEDGER, - declaredCounts: [], - unenforcedCounts: [], - }); - t( - 'FIX #13490: what --fix rewrote evaluates clean -- every site anchored, every ledger row used', - afterFix.problems.length === 0, - afterFix.problems.join(' | ') - ); + // The two headline behaviours the migration was ruled on, proved rather than + // asserted: a SYMBOL RENAME reds, a PURE LINE MOVE does not. Both run the real + // `sweepCorpus` over a real `git` tree, because that is the only place the + // resolver's inputs -- `git ls-files` and the target's own bytes -- exist. + battery('⭐ THE RULED RED-FIRST PAIR: a symbol rename REDS, a pure line move does NOT'); + const corpora = []; + try { + const control = buildRedFirstCorpus(); + corpora.push(control.dir); + const controlSweep = sweepCorpus(CORPUS, control.dir); + t( + 'RED-FIRST control: the unmutated tree sweeps clean and really found the anchor', + controlSweep.findings.filter((f) => !f.soft).length === 0 && controlSweep.counts.symbol === 1, + JSON.stringify(controlSweep.counts) + ' ' + formatFindings(controlSweep.findings) + ); + + const renamed = buildRedFirstCorpus({ symbol: 'handleRequest' }); + corpora.push(renamed.dir); + t( + 'RED-FIRST: the rename really reached disk -- the old name is gone from the target', + !readFileSync(join(renamed.dir, 'pkg', 'a.ts'), 'utf8').includes('function handler(') && + readFileSync(join(renamed.dir, 'pkg', 'a.ts'), 'utf8').includes('function handleRequest(') + ); + const renamedSweep = sweepCorpus(CORPUS, renamed.dir); + t( + '⭐ RED-FIRST: a RENAMED symbol turns the sweep RED, naming the anchor that no longer resolves', + renamedSweep.findings.some((f) => f.kind === 'unresolved-symbol' && f.raw.includes('#handler')), + formatFindings(renamedSweep.findings) + ); + + const moved = buildRedFirstCorpus({ pad: 40 }); + corpora.push(moved.dir); + const movedSweep = sweepCorpus(CORPUS, moved.dir); + t( + '⭐ RED-FIRST: a PURE LINE MOVE (the read is 40 lines lower) is NOT red -- the property the ' + + 'line numbers did not have', + moved.sourceLine === 42 && + control.sourceLine === 2 && + movedSweep.findings.filter((f) => !f.soft).length === 0, + `read moved ${control.sourceLine} -> ${moved.sourceLine}; ` + formatFindings(movedSweep.findings) + ); + /* ⛔ THE REGRESSION PIN for the measured incident above, and it pins the WRITE + * side, which is the side that did the damage: a hook's exported `GIT_DIR` + * must not reach `git init` / `git add -A`, or the throwaway corpus is never + * created and the REPOSITORY's index is written instead. + * + * A bogus `GIT_DIR` is injected, the corpus is built under it, and the temp + * tree is then read back with a stripped environment: two files staged, in + * ITS OWN repository. If the builder had leaked, `git init` would have + * created nothing there and `git ls-files` would answer with someone else's + * tree — or with nothing at all. + * + * ⚠️ Deliberately NOT sweeping under the injected variable. `sweepCorpus` + * asks `git ls-files` through the shared resolver, which passes no + * environment of its own, so a sweep is protected by the process-level strip + * at the top of this function rather than by anything here — and that strip + * is what the second half of this case asserts. Hardening the shared + * resolver is not this gate's to do. */ + const priorGitDir = process.env.GIT_DIR; + process.env.GIT_DIR = join(tmpdir(), 'a-git-dir-that-does-not-exist'); + let staged = null; + try { + const guarded = buildRedFirstCorpus(); + corpora.push(guarded.dir); + staged = execFileSync('git', ['ls-files'], { cwd: guarded.dir, encoding: 'utf8', env: gitFreeEnv() }) + .split('\n') + .filter(Boolean) + .sort(); + } catch (err) { + staged = err; + } finally { + if (priorGitDir === undefined) delete process.env.GIT_DIR; + else process.env.GIT_DIR = priorGitDir; + } + t( + '⛔ ENV LEAK: an inherited GIT_DIR (what `pre-commit` exports) never reaches the corpus builder, ' + + 'and the self-test itself runs detached — measured incident: 8,190 paths staged as deleted in ' + + 'the REAL index by a self-test whose every case still printed ok', + Array.isArray(staged) && + staged.join(' ') === 'content/docs/permissions/system-context.mdx pkg/a.ts' && + Object.keys(process.env).filter((key) => key.startsWith('GIT_')).length === 0, + staged instanceof Error ? String(staged.message).slice(0, 200) : JSON.stringify(staged) + ); + } finally { + for (const dir of corpora) rmSync(dir, { recursive: true, force: true }); + } - // ── ⛔ the dangerous direction: the citation crosses onto a read anchor's line ─ + // ── ⭐ the precision this trades away, pinned so nobody rediscovers it as a bug ─ // - // Subtracting the ledger by LINE removed the stale READ anchor here (it sits on - // `:3`, the citation's new home), the counts agreed by cancellation, and the one - // surviving anchor was mapped onto the read site -- leaving the page GREEN with - // the two rows pointing at each other's lines. Both spellings survive either - // way, so this case asserts which ROW holds which line. - battery('⛔ the dangerous direction: the citation crosses onto a read anchor\'s line ─'); - const crossed = fixAnchors({ - pageText: crossingPage({ read: 'pkg/b.ts:3', helper: 'pkg/b.ts:2' }), - census: CROSSING_CENSUS, - tracked: FIXTURE_TRACKED, - readFile: fixtureRead, - ledger: CROSSING_LEDGER, - }); + // ⛔ Written as a PASSING case on purpose. The gap is real and declared -- on the + // page and in this file's header -- and a self-test that quietly omitted it + // would leave the next reader to find it as a defect and "fix" it by weakening + // something else. Two reads inside ONE symbol, one of them deleted: the symbol + // set does not move, and this gate does not red. + battery('⭐ the precision this trades away, pinned so nobody rediscovers it as a bug'); + const twoInOne = { + ...FIXTURE_CENSUS, + sites: [ + ...FIXTURE_CENSUS.sites, + { file: 'pkg/a.ts', line: 4, receiver: 'ctx', package: 'pkg', symbol: 'handler', text: 'return DENY;' }, + ], + }; + const bothPresent = run(fixturePage(), twoInOne); + const oneDeleted = run(fixturePage(), FIXTURE_CENSUS); t( - 'FIX #13490: a citation crossing a read anchor keeps each ROW on its own line, not merely covered', - crossed.refused.length === 0 && - crossed.text.includes('the elevation read at `pkg/b.ts:5`') && - crossed.text.includes('the name helper at `pkg/b.ts:3`'), - `refused=${JSON.stringify(crossed.refused)} rewrites=${JSON.stringify(crossed.rewrites)}` + '⛔ THE DECLARED GAP: deleting one of two reads inside one symbol does NOT red -- stated here ' + + 'rather than discovered later', + bothPresent.problems.length === 0 && oneDeleted.problems.length === 0, + `${bothPresent.problems.join(' | ')} // ${oneDeleted.problems.join(' | ')}` ); - - // ── ⭐ and the safety property, on the shape that now ACCEPTS ──────────────── - // - // ⛔ The fix must not buy acceptance with the refusal. A read site ARRIVES while - // the ledger citation shifts: the old counting arm and the new one both refuse - // here, and that must stay true, or #13490 was closed by deleting the guard. - battery('⭐ and the safety property, on the shape that now ACCEPTS'); - const grewWhileShifting = fixAnchors({ - pageText: fixturePage({ anchor: 'pkg/a.ts:2', helper: 'pkg/a.ts:6' }), - census: arrived, - tracked: FIXTURE_TRACKED, - readFile: fixtureRead, - ledger: FIXTURE_LEDGER, - }); t( - 'FIX #13490: a site that ARRIVES while the citation shifts is still REFUSED', - grewWhileShifting.rewrites.length === 0 && grewWhileShifting.refused.length === 1, - JSON.stringify(grewWhileShifting.refused) + '⭐ and the half that DOES hold: deleting the whole symbol reds, so the gap is bounded', + run(fixturePage(), { ...FIXTURE_CENSUS, sites: [{ file: 'pkg/a.ts', line: 8, receiver: 'ctx', package: 'pkg', symbol: 'unrelated', text: 'x' }] }) + .problems.some((p) => p.startsWith('[anchor-is-not-a-read-site]') && p.includes('#handler')) ); - const vanished = fixAnchors({ - pageText: crossingPage(), - census: { ...CROSSING_CENSUS, sites: [...CROSSING_CENSUS.sites, { file: 'pkg/b.ts', line: 6, receiver: 'ctx', package: 'pkg', text: 'return DENY;' }] }, - tracked: FIXTURE_TRACKED, - readFile: fixtureRead, - ledger: CROSSING_LEDGER, - }); + // ── the refusal has to SHOW its work (both counts, the symbol, the file) ───── + battery('the refusal has to SHOW its work (both counts, the symbol, the file)'); + const refusal = missing.problems.find((p) => p.startsWith('[site-without-a-row]')) ?? ''; + t( + 'REFUSAL: the population refusal names the file, the symbol and how many reads live in it', + refusal.includes('pkg/a.ts#unrelated') && refusal.includes('1 elevation read(s)'), + refusal + ); t( - 'FIX #13490: an unanchored site in the crossing file is REFUSED too', - vanished.rewrites.length === 0 && vanished.refused.length === 1, - JSON.stringify(vanished.refused) + 'REFUSAL: and BOTH counts it compared, so the reader is not sent to run the census by hand', + refusal.includes('cites 2 symbol(s), the census and the ledger require 3'), + refusal ); - // ── the refusal has to SHOW its work (both counts, both classes, the diff) ──── - // - // Twice this refusal was read as "your diff added or removed an elevation read - // site" when nothing had, and the output gave no way to tell which case you were - // in short of running the census in two trees by hand. `already anchored 8 of 9` - // + one named target settles it; `0 of 9` says uniform displacement. - battery('the refusal has to SHOW its work (both counts, both classes, the diff)'); - const refusalText = grewWhileShifting.refused[0] ?? ''; + // ── ⭐ CORPUS REGISTRATION: one resolver, not a second implementation ──────── + battery('⭐ CORPUS REGISTRATION: one resolver, not a second implementation'); t( - 'FIX #13490: the refusal states BOTH counts it compared and the ledger it set aside', - refusalText.includes('the page anchors 2 distinct line(s)') && - refusalText.includes('holds 3 anchorable line(s)') && - refusalText.includes('2 census read site(s)') && - refusalText.includes('1 NON_READ_ANCHORS citation(s)') && - refusalText.includes('name-prefix helper, not a read'), - refusalText + 'CORPUS: the registration names THIS page and only this page', + CORPUS.docRoots.length === 1 && + PAGE.startsWith(`${CORPUS.docRoots[0]}/`) && + CORPUS.docPattern.test(PAGE.slice(CORPUS.docRoots[0].length + 1)) && + !CORPUS.docPattern.test('access-matrix.mdx'), + JSON.stringify({ docRoots: CORPUS.docRoots, docPattern: String(CORPUS.docPattern) }) ); t( - 'FIX #13490: the refusal names the set difference, not just a count', - refusalText.includes('already anchored') && - refusalText.includes('target, NO anchor') && - refusalText.includes('anchor, NO target'), - refusalText + 'CORPUS: bare paths ARE judged here -- this page spells every path in full', + CORPUS.checkBarePaths === true ); + if (ownSourceForFix !== null) { + t( + '⛔ CORPUS: the resolution rule is imported, never restated -- no local symbol matcher, no ' + + 'second grammar', + /import \{[\s\S]*?\} from '\.\/symbol-anchors\.mjs';/.test(ownSourceForFix) && + !/function\s+symbolResolutionClass\b/.test(ownSourceForFix) && + (ownSourceForFix.match(/defineCorpus\(/g) ?? []).length === 1 + ); + } - // ── WIRING: this gate, and its self-test, really run in CI ────────────────── - // - // ⭐ The half a clean tree cannot show, and the reason this block exists. Every - // other case above judges the RULES; this one judges whether anything runs them. - // `check-self-test-wired` is conditional in the wrong direction for that -- it - // requires "if CI runs the script, CI runs its --self-test too", so deleting BOTH - // lines from `lint.yml` leaves it green and silently retires the only instrument - // that catches a stale anchor. Measured: the census is what reddens when a cited - // file moves underneath a page nobody edited, so its scheduling is load-bearing, - // not incidental. - // - // Asserted against the workflow TEXT, following the precedent `check-doc-frontmatter`, - // `check-aggregator-roster` and `check-ci-filter-parity` set -- and, like the second - // docs root that gate added, this needed NO workflow edit: `lint.yml` already invokes - // both legs, and it is the repo's busiest file. // ── ⭐ ROW REFERENCES: held by SEAM, and the insertion that used to be silent ─ // // The shape this battery exists for (#15869): a row INSERTED into the behaviour @@ -2275,7 +2433,7 @@ function selfTest() { ); // ── the incident shape, on the fixture: ONE insertion, several falsehoods ─── - const inserted = insertRowAbove(ROW_FIXTURE_PAGE, 3, ' **An inserted row** | `pkg/a.ts:9` |'); + const inserted = insertRowAbove(ROW_FIXTURE_PAGE, 3, ' **An inserted row** | `pkg/a.ts#inserted` |'); const afterInsert = rowRefs(inserted.text); t( '⭐ THE INCIDENT SHAPE: inserting one row above row 3 falsifies every reference below it, ' + @@ -2297,8 +2455,8 @@ function selfTest() { 'a key that resolves to TWO rows refuses and names both candidates', (() => { const twice = ROW_FIXTURE_PAGE.replace( - '| 2 | **`owner_id` is not stamped** on INSERT | `pkg/a.ts:3` |', - '| 2 | **`owner_id` is not stamped** and `revoke()` deletes directly | `pkg/a.ts:3` |' + '| 2 | **`owner_id` is not stamped** on INSERT | `pkg/a.ts#stamp` |', + '| 2 | **`owner_id` is not stamped** and `revoke()` deletes directly | `pkg/a.ts#stamp` |' ); const result = rowRefs(twice); return result.problems.some((p) => p.startsWith('[row-ref-key-ambiguous]') && p.includes('rows 2, 3')); @@ -2363,7 +2521,7 @@ function selfTest() { t( 'a `why:` that names a row and declares NO seam is refused -- an unkeyed number reads as current forever', (() => { - const unkeyed = [{ file: 'pkg/a.ts', needle: 'n', why: 'row 3 -- unkeyed' }]; + const unkeyed = [{ file: 'pkg/a.ts', symbol: 'handler', why: 'row 3 -- unkeyed' }]; return codes(rowRefs(ROW_FIXTURE_PAGE, ROW_FIXTURE_REFS, unkeyed)).includes('[why-row-unkeyed]'); })() ); @@ -2383,8 +2541,8 @@ function selfTest() { realResult.problems.join(' | ') ); t( - 'and the real run really resolved them (10 page references + 9 `why:` mentions, 2 declared unheld)', - realResult.held.page === 10 && realResult.held.why === 9 && realResult.held.unheld === 2, + 'and the real run really resolved them (10 page references + 8 `why:` mentions, 2 declared unheld)', + realResult.held.page === 10 && realResult.held.why === 8 && realResult.held.unheld === 2, JSON.stringify(realResult.held) ); @@ -2396,7 +2554,8 @@ function selfTest() { const mutated = insertRowAbove( realPage, 34, - ' **An inserted row, for the ablation** | plugin-sharing | Get: nothing | `sharing-service.ts:1` |' + ' **An inserted row, for the ablation** | plugin-sharing | Get: nothing | ' + + '`packages/plugins/plugin-sharing/src/sharing-service.ts#grant` |' ); const copy = join(dir, 'system-context.mdx'); writeFileSync(copy, mutated.text); @@ -2508,6 +2667,7 @@ function selfTest() { ); } + Object.assign(process.env, savedGitEnv); process.stdout.write( failures === 0 ? '\ncheck-system-context-census --self-test: all cases passed\n' diff --git a/scripts/isystem-census.mjs b/scripts/isystem-census.mjs index 69cb3d34c2..39e241fa01 100644 --- a/scripts/isystem-census.mjs +++ b/scripts/isystem-census.mjs @@ -79,6 +79,7 @@ import { requireDefaultExport } from './import-prerequisite.mjs'; const ts = await requireDefaultExport('typescript', () => import('typescript'), import.meta.url); import { isEntrypoint } from './invoked-as.mjs'; +import { symbolResolutionClass } from './symbol-anchors.mjs'; import { parseSourceFile } from './ts-parse.mjs'; export const ROOT = join(fileURLToPath(new URL('.', import.meta.url)), '..'); @@ -183,20 +184,142 @@ export function packageOf(relPath, root = ROOT) { return null; } +/** + * ── Where a read LIVES, named so an anchor can survive a line shift (#15921) ── + * + * The page used to anchor a read by `file:line`, and a line number rots on every + * unrelated edit above it. It now anchors `path#symbol`, so the census has to + * answer a second question about each site: WHICH DECLARATION encloses it. + * + * ## The rule, and why it is the OUTERMOST function-like scope + * + * A site sits inside a stack of named things -- a local arrow, the method that + * built it, the class the method is on. The innermost name is the most precise + * and the WORST anchor: locals are called `handler`, `context` and `isSystem`, + * they are renamed by refactors that change no behaviour, and several of them + * per file are indistinguishable to a reader who opens the file looking for the + * row. Measured over this corpus, the innermost rule picked `handler`, + * `session`, `permitted`, `context` and -- for the getter on the engine's + * context wrapper -- the string `isSystem` itself. + * + * So the answer is the OUTERMOST function-like scope: the module-level function, + * or the class member (a class is not function-like, so a method stops the walk + * at itself rather than collapsing every method onto the class name). That is the + * declaration a reader greps for, and the one a rename has to move. + * + * ⭐ The chosen name is only accepted when the SHARED resolver would resolve it + * -- `symbolResolutionClass(...) === 'declaration'`, the same predicate + * `scripts/check-adr-symbol-anchors.mjs` sweeps with. A census that named a + * symbol the gate's resolver cannot bind would publish a population the page can + * never satisfy, which is the one failure mode a population check cannot survive. + * + * ## The fallbacks, in order, and the honest bottom + * + * 1. the outermost function-like named scope that resolves; + * 2. failing that, the innermost enclosing named declaration that resolves + * (a class, an interface, a `const` binding -- a read at module top level); + * 3. failing that, `null` -- and `null` is NOT an error and NOT a guess. It + * means no declaration in that file can be named, and the page anchors the + * FILE. A file-level anchor stays checked (the file must exist) and it is + * the one honest answer when there is no symbol; ⛔ inventing one would put + * a name in the page that no rename can ever red. + * + * ⚠️ What this costs, stated where it is derived: several sites inside ONE symbol + * collapse onto ONE anchor. `check-system-context-census.mjs` carries the + * measurement and the consequence for what the gate can and cannot catch. + */ +function isFunctionLike(node) { + return ( + ts.isFunctionDeclaration(node) || + ts.isMethodDeclaration(node) || + ts.isGetAccessorDeclaration(node) || + ts.isSetAccessorDeclaration(node) || + ts.isConstructorDeclaration(node) || + ts.isFunctionExpression(node) || + ts.isArrowFunction(node) + ); +} + +/** The name a node declares, or `null` when it declares none this census can cite. */ +function declaredName(node, sourceFile) { + if ( + ts.isFunctionDeclaration(node) || + ts.isClassDeclaration(node) || + ts.isInterfaceDeclaration(node) || + ts.isEnumDeclaration(node) || + ts.isTypeAliasDeclaration(node) || + ts.isModuleDeclaration(node) + ) { + return node.name && ts.isIdentifier(node.name) ? node.name.text : null; + } + if (ts.isMethodDeclaration(node) || ts.isGetAccessorDeclaration(node) || ts.isSetAccessorDeclaration(node)) { + const name = node.name; + return name && (ts.isIdentifier(name) || ts.isStringLiteral(name)) ? name.text : null; + } + if (ts.isConstructorDeclaration(node)) return 'constructor'; + if (ts.isVariableDeclaration(node) || ts.isPropertyAssignment(node) || ts.isPropertyDeclaration(node)) { + const name = node.name; + return name && (ts.isIdentifier(name) || ts.isStringLiteral(name)) ? name.text : null; + } + return null; +} + +/** + * The symbol a page anchor should name for a node, by the rule documented above. + * + * @param {import('typescript').Node} node + * @param {import('typescript').SourceFile} sourceFile + * @param {string} relPath + * @param {string} text the target's own source, for the resolver + * @returns {string|null} + */ +export function enclosingSymbol(node, sourceFile, relPath, text) { + const functionLike = []; + const anyNamed = []; + for (let p = node.parent; p; p = p.parent) { + const name = declaredName(p, sourceFile); + if (name === null) continue; + anyNamed.push(name); + /* A function or arrow bound to a name is function-like scope under the name + * it is bound to -- so the walk records the BINDING's name, not the anonymous + * expression's absence of one. */ + if (isFunctionLike(p) || (p.initializer !== undefined && p.initializer !== null && isFunctionLike(p.initializer))) { + functionLike.push(name); + } + } + const resolves = (name) => symbolResolutionClass(text, relPath, name) === 'declaration'; + for (let i = functionLike.length - 1; i >= 0; i -= 1) { + if (resolves(functionLike[i])) return functionLike[i]; + } + for (const name of anyNamed) { + if (resolves(name)) return name; + } + return null; +} + /** * Every syntactic role the identifier takes in one parsed source. * - * @returns {{ role: string, line: number, receiver: string|null, text: string }[]} + * @returns {{ role: string, line: number, receiver: string|null, text: string, + * symbol: string|null }[]} */ export function classifyFile(relPath, text) { const sourceFile = parseSourceFile(relPath, text); const lines = text.split('\n'); - /** @type {{ role: string, line: number, receiver: string|null, text: string }[]} */ + /** @type {{ role: string, line: number, receiver: string|null, text: string, + * symbol: string|null }[]} */ const found = []; const record = (node, role, receiver) => { const { line } = sourceFile.getLineAndCharacterOfPosition(node.getStart(sourceFile)); - found.push({ role, line: line + 1, receiver, text: (lines[line] ?? '').trim() }); + found.push({ + role, + line: line + 1, + receiver, + text: (lines[line] ?? '').trim(), + /* Only a READ is ever anchored, so only a read pays the ancestor walk. */ + symbol: role === 'read' ? enclosingSymbol(node, sourceFile, relPath, text) : null, + }); }; const visit = (node) => { @@ -236,7 +359,8 @@ function nonElevationRowFor(relPath, receiver) { * Run the census. * * @returns {{ - * sites: { file: string, line: number, receiver: string, package: string|null }[], + * sites: { file: string, line: number, receiver: string, package: string|null, + * symbol: string|null }[], * nonElevationReads: { file: string, line: number, receiver: string, field: string }[], * roleCounts: Record, * packages: string[], @@ -282,6 +406,7 @@ export function runCensus({ root = ROOT } = {}) { receiver: hit.receiver, package: packageOf(relPath, root), text: hit.text, + symbol: hit.symbol, }); } } @@ -333,11 +458,40 @@ export function countText(root = ROOT) { }; } -/** `file:line` keys for the elevation sites -- the census's comparable form. */ +/** `file:line` keys for the elevation sites -- the census's positional form. */ export function siteKeys(census) { return new Set(census.sites.map((s) => `${s.file}:${s.line}`)); } +/** + * The census's ANCHORABLE form: per file, the distinct symbols its read sites live + * in, and whether any of them has no nameable symbol at all. + * + * ⭐ This is the population `check-system-context-census.mjs` holds the page to, + * and it is deliberately smaller than `siteKeys` above: several sites inside one + * symbol collapse to one entry. The two are both kept because they answer + * different questions -- `siteKeys` is what the census COUNTS, this is what the + * page can CITE without encoding a position. + * + * @param {{ sites: { file: string, symbol: string|null }[] }} census + * @returns {Map, fileLevel: boolean, sites: number }>} + */ +export function symbolPopulation(census) { + /** @type {Map, fileLevel: boolean, sites: number }>} */ + const byFile = new Map(); + for (const site of census.sites) { + let entry = byFile.get(site.file); + if (!entry) { + entry = { symbols: new Set(), fileLevel: false, sites: 0 }; + byFile.set(site.file, entry); + } + entry.sites += 1; + if (site.symbol === null) entry.fileLevel = true; + else entry.symbols.add(site.symbol); + } + return byFile; +} + function main(argv) { const census = runCensus(); if (argv.includes('--json')) {