feat(spec): gate the shipped llms.txt against the real package surface (#11688)

`packages/spec/llms.txt` ships in the npm tarball as context for AI consumers
and is hand-kept with no generator, so nothing ever re-derived what it asserts.
It had drifted: eleven advertised symbols existed in no entry point, two
advertised packages did not exist, the schema-inventory heading disagreed with
the sum of its own table, and the package heading claimed 19 against a real 68.

Add `check:llms-txt`, which re-derives every checkable claim: symbols against
the checked-in api-surface/ shards, subpaths against the manifest `exports`,
per-domain counts against src/<domain>/**/*.zod.ts, and the package table
against the workspace. Symbol claims resolve at the strictness their position
earns — namespace bullets and fenced imports name an entry point and must
resolve from it, the architecture overview resolves against the union.

Repair the claims the gate was otherwise born red on. Deliberately no
`gen:llms-txt`: the numbers are not the claim, the prose beside them is.


Claude-Session: https://claude.ai/code/session_01Rxnd8cyFnoU8V5y21PaTsy

Co-authored-by: Claude <noreply@anthropic.com>

Author: os-warren

.changeset/llms-txt-freshness-gate.md

@@ -0,0 +1,28 @@
+---
+'@objectstack/spec': patch
+---
+
+Gate the shipped `llms.txt` against the real package, and repair the claims that had rotted.
+
+`packages/spec/llms.txt` ships in the npm tarball as context for AI consumers and is
+hand-kept with no generator, so nothing ever re-derived what it asserts. It had drifted
+badly: eleven advertised symbols existed in no entry point (`IUIService` — removed in
+v11 — plus `ThemeSchema`, `IdentitySchema`, `PolicySchema`, `ContractSchema`,
+`EndpointSchema`, `RAGPipelineSchema`, `MCPSchema`, `FilterSchema`, `AnalyticsSchema`,
+`FormSchema`), two advertised packages did not exist (`@objectstack/nextjs`,
+`@objectstack/nestjs`), the schema-inventory heading disagreed with the sum of its own
+table (171 vs 170) and with the tree (207), and the package heading claimed 19 against a
+real 68. An agent reading the file wrote imports that do not resolve.
+
+New gate `check:llms-txt` re-derives every checkable claim on every PR: advertised
+symbols against the checked-in `api-surface/` shards, `@objectstack/spec/x` subpaths
+against the manifest `exports`, the per-domain schema counts against
+`src/<domain>/**/*.zod.ts`, and the package table against the workspace. Symbol claims
+are resolved at the strictness their position earns — namespace bullets and fenced
+imports name an entry point and must resolve from it, while the architecture overview
+resolves against the union. Prose, code-fence bodies and `N+` lower-bound figures are
+out of population and the script header says why.
+
+There is deliberately no `gen:llms-txt`: the numbers are not the claim, the prose beside
+them is, and restamping a count without re-reading its row would turn a loud staleness
+into a silent lie.

.github/workflows/lint.yml

@@ -3015,6 +3015,23 @@ jobs:
       - name: Check spec pin citations still name the pin this repo builds against
         run: pnpm --filter @objectstack/spec check:objectui-pin-citations
 
+      # #11344. `packages/spec/llms.txt` is hand-kept, has no generator, and
+      # SHIPS in the published tarball as context for AI consumers — the
+      # audience least able to notice staleness and most likely to turn a claim
+      # into code. When this landed the file advertised eleven symbols that
+      # existed in no entry point (`IUIService` among them, removed in v11), two
+      # packages that did not exist, and a schema table whose heading disagreed
+      # with the sum of its own rows.
+      #
+      # Reads `src/`, the checked-in `api-surface/` shards, the package manifest
+      # and pnpm-workspace.yaml — no build — so it belongs in this pre-build
+      # source-audit group beside its nearest sibling above. Unfiltered on
+      # purpose: the counts are measured against the whole workspace and against
+      # every schema domain, so a PR that adds a package or a schema anywhere
+      # can falsify this file without touching packages/spec/**.
+      - name: Check the shipped llms.txt still describes this package
+        run: pnpm --filter @objectstack/spec check:llms-txt
+
   # Lane 2 of 4 behind the required `TypeScript Type Check` context. The
   # aggregator job at the bottom of this file explains the split, holds the
   # contract, and is the thing the merge queue actually requires — read it

packages/spec/llms.txt

@@ -14,14 +14,15 @@ It is divided into three layers, reflected in the import paths:
 - **`FieldSchema`**: Defines columns with 46+ types (`text`, `number`, `lookup`, `formula`, `vector`, etc.).
 - **`QuerySchema`**: A JSON-based AST for querying data (replaces SQL).
 - **`IDataDriver`**: The authoritative contract for database adapters (SQL, NoSQL, Memory).
-- **`AnalyticsSchema`**: OLAP cubes, measures, and dimensions.
+  Exported from `@objectstack/spec/contracts` — see §6.
+- **`CubeSchema`**: OLAP cubes, measures, and dimensions.
 
 ### Layer 2: ObjectOS (`@objectstack/spec/system` & `@objectstack/spec/api`)
 **The Runtime Kernel**. Defines "How System Operates".
 - **`ManifestSchema`**: `objectstack.config.ts` configuration.
-- **`IdentitySchema`**: Users, Roles, Organizations, SCIM.
+- **`OrganizationSchema`**: Organizations, members, positions, SCIM provisioning.
 - **`EventSchema`**: System bus, DLQ, and Webhooks (6 sub-modules).
-- **`EndpointSchema`**: API Gateway configuration.
+- **`ApiEndpointSchema`**: API Gateway configuration.
 - **`PluginSchema`**: Module lifecycle, security, registry, loading.
 
 ### Layer 3: ObjectUI (`@objectstack/spec/ui`)
@@ -30,14 +31,13 @@ It is divided into three layers, reflected in the import paths:
 - **`ViewSchema`**: Layouts for data (Grid, Kanban, Calendar, Gantt).
 - **`ActionSchema`**: Buttons and triggers.
 - **`DashboardSchema`**: Widget composition.
-- **`ThemeSchema`**: Design tokens and theming.
 
 ### Layer 4: ObjectAI (`@objectstack/spec/ai`)
 **The Intelligence Layer**. Defines AI Agents and Pipelines.
 - **`AgentSchema`**: Autonomous actors with tools and permissions.
-- **`RAGPipelineSchema`**: Retrieval Augmented Generation pipelines.
+- **`KnowledgeSourceSchema`**: Retrieval sources backing RAG grounding.
 - **`ModelRegistrySchema`**: LLM configuration and routing.
-- **`MCPSchema`**: Model Context Protocol integration.
+- **`MCPServerRefSchema`**: Model Context Protocol integration.
 
 ---
 
@@ -77,53 +77,58 @@ const query = {
 
 ---
 
-## 3. Schema Inventory by Domain (171 schemas)
+## 3. Schema Inventory by Domain (207 schemas)
+
+Counted as `*.zod.ts` modules under `packages/spec/src/<domain>/` — the sources
+that ship in this tarball (`files` includes `src/**/*.zod.ts`), so every number
+here is verifiable from the installed package.
 
 | Domain | Count | Key Schemas |
 |--------|-------|-------------|
+| system | 36 | Auth, Cache, Compliance, Encryption, HTTP Server, License, Logging, Metrics |
 | kernel | 32 | Plugin, Manifest, Events (6 sub-modules), Feature, Context, Package Registry |
-| system | 27 | Auth, Cache, Compliance, Encryption, HTTP Server, License, Logging, Metrics |
-| api | 25 | Contract, Endpoint, REST Server, Discovery, GraphQL, OData, Batch, WebSocket |
-| data | 19 | Object, Field, Query, Filter, Driver (SQL/NoSQL/Memory/Mongo/Postgres), Hook |
-| ui | 18 | View, App, Action, Dashboard, Page, Chart, Component, Theme, Animation |
-| ai | 14 | Agent, RAG Pipeline, Model Registry, MCP, Orchestration, NLQ, Predictive |
-| automation | 8 | Flow, Workflow, Trigger, Approval, ETL, State Machine, Webhook |
-| integration | 7 | Connector (Database, File Storage, GitHub, MQ, SaaS, Vercel) |
-| shared | 6 | Enums, HTTP, Identifiers, Mapping, Metadata Types, Connector Auth |
-| security | 5 | Permission, Policy, RLS, Sharing, Territory |
-| identity | 4 | Identity, Organization, Role, SCIM |
-| cloud | 4 | Marketplace, Developer Portal, App Store, Marketplace Admin |
-| studio | 1 | Studio Plugin Manifest |
+| data | 30 | Object, Field, Query, Filter, Driver (SQL/NoSQL/Memory/Mongo/Postgres), Cube |
+| api | 29 | Endpoint, REST Server, Discovery, OData, Batch, WebSocket, Response Envelope |
+| ui | 18 | View, App, Action, Dashboard, Page, Chart, Component, Animation |
+| automation | 13 | Flow, Approval, BPMN Interop, Control Flow, State Machine, Webhook |
+| shared | 13 | Enums, HTTP, Identifiers, Mapping, Metadata Types, Connector Auth, Retry Policy |
+| ai | 11 | Agent, Conversation, Knowledge Source/Document, Model Registry, MCP, Skill, Tool |
+| cloud | 11 | Marketplace, Developer Portal, App Store, Environment, Package, Tenant |
+| identity | 5 | Identity, Organization, Position, SCIM, Eval User |
+| security | 4 | Permission, RLS, Sharing, Explain |
+| studio | 3 | Flow Builder, Object Designer, Studio Plugin |
+| integration | 1 | Connector |
+| qa | 1 | Testing |
 
 ---
 
 ## 4. Key Exports by Namespace
 
 ### `import * as Data from '@objectstack/spec/data'`
 - `ObjectSchema`, `FieldSchema`: Logic & Storage definition.
-- `QuerySchema`, `FilterSchema`: Data retrieval AST.
-- `IDataDriver`, `DatasourceSchema`: Database connectivity.
-- `AnalyticsSchema`: OLAP cubes and metrics.
+- `QuerySchema`, `FilterArraySchema`: Data retrieval AST.
+- `DatasourceSchema`, `DriverInterfaceSchema`: Database connectivity.
+- `CubeSchema`: OLAP cubes and metrics.
 
 ### `import * as UI from '@objectstack/spec/ui'`
 - `ViewSchema`: `type: 'grid' | 'kanban' | 'calendar'`.
-- `FormSchema`: `layout: 'simple' | 'tabbed'`.
+- `FormViewSchema`: Form layout and sections.
 - `DashboardSchema`: Widget composition.
-- `ThemeSchema`: Design tokens.
+- `AppSchema`, `ActionSchema`: Navigation and triggers.
 
-### `import * as System from '@objectstack/spec/system'`
-- `PluginSchema`: Module lifecycle.
+### `import * as Kernel from '@objectstack/spec/kernel'`
+- `PluginSchema`, `ManifestSchema`: Module lifecycle and configuration.
 - `EventSchema`: Pub/Sub definitions.
-- `PolicySchema`: Security rules.
+- `KernelSecurityPolicySchema`: Plugin security rules.
 
 ### `import * as AI from '@objectstack/spec/ai'`
 - `AgentSchema`: AI agent configuration.
-- `RAGPipelineSchema`: Retrieval pipelines.
+- `KnowledgeSourceSchema`, `KnowledgeDocumentSchema`: Retrieval sources.
 - `ModelRegistrySchema`: LLM routing.
 
 ### `import * as API from '@objectstack/spec/api'`
-- `EndpointSchema`: REST/GraphQL endpoints.
-- `ContractSchema`: Request/Response envelopes.
+- `ApiEndpointSchema`: REST endpoints.
+- `ResponseEnvelopeConfigSchema`, `ApiErrorSchema`: Request/Response envelopes.
 - `DiscoverySchema`: Service discovery.
 
 ---
@@ -165,11 +170,14 @@ function registerObject(rawConfig: unknown) {
 | `IAnalyticsService` | query, aggregate, timeSeries |
 | `IAuthService` | authenticate, authorize, validateToken |
 | `IAutomationService` | executeFlow, triggerWorkflow |
-| `IUIService` | **DEPRECATED** — use IMetadataService.getView(), .listViews(), .getEffective('view', name, { userId }) |
 
 ---
 
-## 7. Package Ecosystem (19 packages)
+## 7. Package Ecosystem (68 packages)
+
+The workspace publishes 68 packages under the `@objectstack` scope. The table
+below is a curated entry-point list, not the full set — drivers, connectors,
+triggers, plugins and kernel-managed services each form their own family.
 
 | Package | Description |
 |---------|-------------|
@@ -183,9 +191,9 @@ function registerObject(rawConfig: unknown) {
 | `@objectstack/client-react` | React hooks for client |
 | `@objectstack/cli` | Command-line interface |
 | `@objectstack/hono` | Hono adapter |
-| `@objectstack/nextjs` | Next.js adapter |
-| `@objectstack/nestjs` | NestJS adapter |
 | `@objectstack/driver-memory` | In-memory database driver |
+| `@objectstack/driver-sql` | SQL database driver |
+| `@objectstack/types` | Shared TypeScript utilities |
 
 ---
 

packages/spec/package.json

@@ -268,6 +268,7 @@
     "check:dual-source-exports": "tsx scripts/check-dual-source-exports.ts --self-test && tsx scripts/check-dual-source-exports.ts",
     "check:browser-reachable-entries": "tsx scripts/check-browser-reachable-entries.ts --self-test && tsx scripts/check-browser-reachable-entries.ts",
     "check:objectui-pin-citations": "tsx scripts/check-objectui-pin-citations.ts --self-test && tsx scripts/check-objectui-pin-citations.ts",
+    "check:llms-txt": "tsx scripts/check-llms-txt.ts --self-test && tsx scripts/check-llms-txt.ts",
     "check:authorable-surface": "OS_EAGER_SCHEMAS=1 tsx scripts/build-schemas.ts --check",
     "gen:migration-registry": "tsx scripts/build-migration-registry.ts",
     "check:migration-registry": "tsx scripts/build-migration-registry.ts --self-test --check",

packages/spec/scripts/check-generated.ts

@@ -278,6 +278,19 @@ const NO_GENERATOR: ReadonlyArray<{ check: string; why: string }> = [
     check: 'check:objectui-pin-citations',
     why: 'audits spec source prose: a citation in the asserting spelling (`.objectui-sha` = `<sha>`) must equal the root pin file — no artifact, and deliberately no `gen:`, because rewriting the sha without re-measuring the anchors beside it is the failure mode (#10274)',
   },
+  // #11344. The nearest sibling to `check:objectui-pin-citations` above, and
+  // classified NO_GENERATOR for the same reason rather than for a bookkeeping
+  // one. `llms.txt` is hand-kept prose that SHIPS in the tarball (`files`), and
+  // the obvious `gen:` — restamp each count, drop each dead symbol — is the one
+  // operation that must never be offered: the number is not the claim, the
+  // sentence beside it is. Restamping `| integration | 7 | Connector (Database,
+  // File Storage, GitHub, MQ, SaaS, Vercel) |` to `1` would leave a freshly
+  // dated row listing six connectors that do not exist, converting a loud
+  // staleness into a silent lie. A failure here is a re-read of the section.
+  {
+    check: 'check:llms-txt',
+    why: 'audits the shipped llms.txt: every advertised symbol must resolve against api-surface/, every `@objectstack/spec/x` against the manifest `exports`, and every declared count against src/ and the workspace — no artifact, and deliberately no `gen:`, because restamping a count without re-reading the prose beside it is the failure mode (#11344)',
+  },
 ];
 
 /**