From 078d67a9885658be524bf30d95d4a82a1a93d706 Mon Sep 17 00:00:00 2001 From: Maneek21 <208369276+Maneek21@users.noreply.github.com> Date: Mon, 31 Aug 2026 10:09:50 +0530 Subject: [PATCH] docs(apps): seal Phase 3 release and plan Phase 4 --- ...8-30-app-platform-phase-3-certification.md | 68 +-- ...-platform-phase-3-release-certification.md | 185 ++++++++ .../2026-08-29-full-surface-app-platform.md | 29 +- ...-30-app-platform-phase-3-closeout-loops.md | 43 +- ...full-surface-app-platform-delivery-plan.md | 90 ++-- ...rm-phase-4-resource-participation-loops.md | 420 ++++++++++++++++++ 6 files changed, 708 insertions(+), 127 deletions(-) create mode 100644 docs/superpowers/audits/2026-08-31-app-platform-phase-3-release-certification.md create mode 100644 docs/superpowers/plans/2026-08-31-app-platform-phase-4-resource-participation-loops.md diff --git a/docs/superpowers/audits/2026-08-30-app-platform-phase-3-certification.md b/docs/superpowers/audits/2026-08-30-app-platform-phase-3-certification.md index a1c92bba..966ea00c 100644 --- a/docs/superpowers/audits/2026-08-30-app-platform-phase-3-certification.md +++ b/docs/superpowers/audits/2026-08-30-app-platform-phase-3-certification.md @@ -1,14 +1,20 @@ # App Platform Phase 3 certification checkpoints -- Date: 2026-08-30; consolidated PR D gate 2026-08-31 +- Date: 2026-08-30; consolidated PR D and immutable release gates 2026-08-31 - Historical pre-split baseline: `5050b0f1` with C5 source checkpoint `944b265f`; useful review evidence, superseded for rollout certification - Current PR D certification checkpoint: `fe66113f` on PR C merge `9c8e9ac9`; split-control source checkpoint `5cf88c51` -- Current result: consolidated local PR D gate passed; remote PR checks and the - immutable release-artifact gate remain -- Rollout decision: off/off by default; merge alone does not make the legacy MCP - canary supported +- Released baseline: `v0.3.0-preview.14` at `6d39e0e`, image digest + `sha256:e565cc64ee22b5b9f6f99973e3762b639c27e026dc8824852145035acdacf788` +- Current result: PASS. PR D, fix-forward App compatibility, remote CI/security, + immutable publishing, upgrade, recovery, rotation, drain, quiescence, and + rollback gates passed. +- Rollout decision: off/off remains the default. The legacy MCP canary is + supported only as an explicit self-host opt-in using the documented split + controls. App origin remains disabled until Phase 5 grants and bindings. +- Detailed release record: + `2026-08-31-app-platform-phase-3-release-certification.md` ## Current split-control delta @@ -88,30 +94,28 @@ explicit release-candidate gates. The one production source build is delegated to normal PR CI against the exact pushed tip; run it locally only if that CI job does not execute. -## Remaining PR D remote gate - -- Inspect the committed audit delta and open PR D from the exact checkpoint. -- Require normal PR CI, including the production source build, API tests, - typecheck, upgrade/image jobs, and security checks, to pass. -- Keep the branch unmerged until human review; passing PR D does not satisfy the - separate release-artifact gate below. - -## Remaining release-artifact gates - -- Build and identify the immutable merged candidate image and its supported - predecessor; record both digests and migration ledgers. -- Prove a current pgvector-backed fresh install and supported upgrade through - `.21` on release-capable infrastructure. -- Run deterministic-provider canary and approval/browser smoke without claiming - a public Run UI. -- Back up and restore database, uploads/App artifacts where applicable, and all - Run keyrings at consistent points; exercise rotation and referenced-key - retirement refusal. -- Disable intake, drain and quiesce governed work, then perform the actual - supported image rollback drill. -- Record the first released image containing the final split-control contract as - the operational rollback floor. - -Until those gates pass, App origin remains disabled, the legacy MCP canary stays -default-off and unsupported for production widening, and no local source hash -is an operational rollback floor. +## PR D remote gate — complete + +- PR D #274 merged after the required typecheck, API, upgrade, production-image, + browser, Hermes, dependency, and CodeQL checks passed. +- The Hello Workspace release gate then exposed a package-byte versus parsed- + manifest digest mismatch. The smallest boundary repair shipped in #275 with + a deterministic red/green regression and the same full remote check set. +- The merged fix-forward revision is `6d39e0e`; no Run state machine, migration, + token, connector, UI, or rollout-control contract changed. + +## Release-artifact gates — complete + +- The supported predecessor is `v0.3.0-preview.12` at `23694ef8`, digest + `sha256:34b306e53e5c959468a973a50aaeb3b59235c56e631d67003f7780d18002d24b`. +- The target is `v0.3.0-preview.14` at `6d39e0e`, digest + `sha256:e565cc64ee22b5b9f6f99973e3762b639c27e026dc8824852145035acdacf788`. +- Upgrade through `.21`, deterministic-provider canary, App lifecycle, + matched backup/restore, three-purpose key rotation, retirement refusal, + pause/drain/resume, exact quiescence, and immutable image rollback passed. +- This target image is the first operational rollback floor containing the + final split-control and durable job-pause contract. + +App origin remains disabled. The legacy MCP canary may be enabled only through +the documented self-host controls after a matched backup and canary; the +release does not authorize default-on production widening. diff --git a/docs/superpowers/audits/2026-08-31-app-platform-phase-3-release-certification.md b/docs/superpowers/audits/2026-08-31-app-platform-phase-3-release-certification.md new file mode 100644 index 00000000..39f69ca1 --- /dev/null +++ b/docs/superpowers/audits/2026-08-31-app-platform-phase-3-release-certification.md @@ -0,0 +1,185 @@ +# App Platform Phase 3 — Immutable release certification + +| Field | Certified value | +|---|---| +| Result | PASS, with one explicitly recorded visual-tool limitation | +| Release | `v0.3.0-preview.14` | +| Commit | `6d39e0e0413c82d36c9481849ae582fdf805d1a6` | +| Image | `ghcr.io/maneek21/deft@sha256:e565cc64ee22b5b9f6f99973e3762b639c27e026dc8824852145035acdacf788` | +| Supported predecessor | `v0.3.0-preview.12` at `23694ef832bc11b6e06a704bf9af234697955d80` | +| Predecessor image | `ghcr.io/maneek21/deft@sha256:34b306e53e5c959468a973a50aaeb3b59235c56e631d67003f7780d18002d24b` | +| Release workflow | GitHub Actions run `33355373750` | +| Merged-revision CI | GitHub Actions run `33354900483` | +| Merged-revision security | GitHub Actions run `33354900548` | +| Certification host | Isolated loopback-only Docker/pgvector host; Compose project `deft_phase3_preview13_cert` | + +## Decision + +`v0.3.0-preview.14` is the Phase 3 operational rollback floor. The separate Run +engine/drain and legacy MCP intake contract is release-supported with these +limits: + +- `DEFT_APP_RUNS_ENABLED=false` and + `DEFT_APP_RUN_LEGACY_MCP_CUTOVER_ENABLED=false` remain the default. +- A self-host operator may opt into the legacy MCP canary only after preserving + a matched database/keyring recovery point and following + `docs/app-run-operations.md`. +- Engine-on/intake-off is the drain and recovery state. Intake-on/engine-off is + invalid and fails closed. +- App-origin execution, App grants, App connector bindings, public Run APIs, + automation origins, and default-on rollout remain unsupported. + +## Merge and release provenance + +PR #274 merged the guarded runtime and split controls. Its first immutable +candidate, `v0.3.0-preview.13` at `4bad79d8`, passed publishing and Run gates but +failed the Hello Workspace install gate. Artifact verification correctly bound +the package digest to exact canonical package bytes; the API then incorrectly +compared that digest with a second digest computed after Module parsing inserted +an optional default. + +PR #275 removed only that redundant semantic-digest comparison and added a +regression where a valid Module omits the optional `required` property. It did +not change UI, migrations, tokens, connector ciphertext, App Run state, +provider dispatch, or rollout controls. All ten PR checks and the merged +revision's CI/security workflows passed before `v0.3.0-preview.14` was tagged. + +The release workflow passed its two clean-state Hermes certification runs, +built the amd64 image, signed it keylessly, attached GitHub build provenance, +generated an SPDX SBOM, archived the exact source and Hermes bundle, and +published the release manifest. All twelve assets covered by `SHA256SUMS` +matched. `gh attestation verify` passed for the digest above. The exact source +archive checksum is +`0920f15173885ea4ea43b44c031f324c60fa47a78c9374103dcb0fa48f959235`. + +## Installation and supported upgrade + +The release-capable host preserved an existing, unrelated Compose project named +`app`; only the isolated certification project was operated. + +The predecessor fixture contained: + +- one organization and owner; +- one streamable-HTTP MCP connection with an encrypted API-key envelope; +- one assigned conservative employee; +- one previously approved action and one pending action; +- one legacy receipt; and +- one deterministic provider effect. + +The supported release upgrade wrapper stopped writes, produced a backup, and +upgraded the same database from `v0.3.0-preview.12` through the current ledger. +Migrations `.17`–`.21` were present with their recorded checksums; the +`preview.14` fix-forward had zero additional migrations. Doctor and self-host +smoke passed on the exact target image. + +The connector ID, employee ID, assignments, action states, receipt, and +credential envelope survived. The envelope SHA-256 stayed +`addaa126502cf85e83b055bc1e7458b18fa4bf2821757dba74a50017c7b70289`; +no connector ciphertext rewrite occurred. + +The merged revision's production-image/browser job also proved a fresh +pgvector-backed schema, self-host Agent Channel/MCP smoke, production browser +smoke, and critical-image vulnerability gate against the exact release commit. + +## Rollout controls and governed canary + +All four flag combinations were exercised: + +| Engine | Legacy intake | Result | +|---|---|---| +| Off | Off | Healthy default; legacy execution | +| On | Off | Healthy drain-only state | +| On | On | Healthy governed canary | +| Off | On | API startup refused; no healthy API was exposed | + +The pending predecessor action was approved under the governed canary. It made +one provider call and produced one succeeded Run, one succeeded attempt, one +budget reservation, encrypted input and output rows, seven ordered events, and +signed terminal evidence. Replay returned the existing approval/result and did +not call the provider again. + +Across the full gate the deterministic provider recorded exactly three effects: + +1. the approved predecessor legacy fixture; +2. the governed post-upgrade canary; and +3. a new legacy call while the image was actually rolled back to the supported + predecessor. + +The released-image rollout-transition test independently passed engine-off +approval refusal, approval in drain-only mode, engine-off durable job deferral +with zero retry debit, engine-on completion, and registered-handler replay with +one stubbed provider dispatch. + +## Declarative App compatibility + +The public App Kit checked and built the unchanged Hello Workspace fixture at +package digest +`sha256:20e62afea91161e503dcecf547ad806c1419797834f6550f781d78bf3bb20884`. +A fresh single-use owner pairing installed and activated it on the released +image. `/api/apps` returned the active installation; `/api/apps/navigation` +returned its `Greetings` entry; and +`/modules/hello-workspace/greetings` returned HTTP 200. Disabling advanced the +lifecycle epoch from 1 to 2 and removed the navigation entry. + +The in-app browser controller failed to initialize its local runtime assets, so +an additional interactive visual inspection of that specific route was not +obtained. This is not represented as visual evidence. The exact merged revision +did pass the normal production browser smoke, the release fix changed no web +code, and the App-specific package, API, route, activation, and disable evidence +above passed. + +## Backup, restore, and key rotation + +A matched recovery point was taken with app writes stopped. It contains the +PostgreSQL dump, uploads archive, App Run keyring, and deployment secrets under +restricted permissions. Representative checksums were: + +- database: `c09d08c3579d46d3bc984569e5595dca3633be38cabac54b3be0202091b4aac8`; +- uploads: `de03d92de7b8138d6588d094a3ef65fdbcadab7265d8de22e4953e18c18197ba`; +- App Run keyring: `be9d9a2e598ac361f5863b49d191df1fd5b9ebb10f89fd45e0172a5803072359`. + +The database and uploads were restored into a separate Compose project. That +copy booted first with the engine off, then with the matched keyring in +engine-on/intake-off mode. Stable continuity projections before and after +restore had the same canonical SHA-256. An uploads marker survived. + +Encryption, receipt-signing, and fingerprint current IDs advanced to `enc-v2`, +`sig-v2`, and `fp-v2` while retaining the three `v1` entries. Before and after +rotation, retained input and output decrypted to identical digests and the +stored terminal receipt verified. Three separate non-serving boots removed one +referenced `v1` key at a time; each exited nonzero with the referenced-key +unavailable error. The healthy rotated configuration was restored after the +test. The disposable restore project and both of its volumes were then removed. + +## Quiescence and immutable image rollback + +Before rollback, the frozen runbook queries returned: + +- zero nonterminal Runs; +- zero pending `app_run_invoke` approvals; and +- zero pending or running `app-run-attempt` jobs. + +With intake and engine off, the exact predecessor digest was deployed against +the upgraded database without down-migration. Health reported +`v0.3.0-preview.12` at its recorded commit. The preserved connector executed +one new legacy approval successfully. The exact `preview.14` digest was then +restored, engine-on/intake-off passed the retained-key inventory and payload/ +receipt reads, and the project returned to the default off/off state. + +Final quiescence again reported zero for all three runbook queries. The only App +Run was succeeded, the deterministic provider count was three, the credential +envelope hash remained unchanged, and the final container used the certified +target digest. + +## Cleanup and remaining non-claims + +The temporary restore project, primary certification project, their isolated +volumes/networks, and the SSH browser tunnel were removed after the final +evidence snapshot. The on-disk evidence and matched backups were retained under +restricted operator access. The unrelated `app` project remained healthy +throughout. + +This evidence does not claim App-origin authority, connected App grants, +ResourceRef, custom UI, automation, external runtimes, sync, public ingress, +hosted KMS, marketplace, billing, or SaaS operations. Those remain owned by +later phases. diff --git a/docs/superpowers/plans/2026-08-29-full-surface-app-platform.md b/docs/superpowers/plans/2026-08-29-full-surface-app-platform.md index 20ab6c3b..0007843a 100644 --- a/docs/superpowers/plans/2026-08-29-full-surface-app-platform.md +++ b/docs/superpowers/plans/2026-08-29-full-surface-app-platform.md @@ -2,9 +2,9 @@ | Field | Value | |---|---| -| **Status** | Canonical capability map and threat model; Phase 3 PR C merged and guarded PR D closeout in progress; release gate pending | -| **Date** | 2026-08-29; execution rebaseline 2026-08-30 | -| **Baseline inspected** | `origin/master` at PR C merge `9c8e9ac9`; PR D is replayed from that exact merge and remains unreleased | +| **Status** | Canonical capability map and threat model; Phases 0–3 released and certified; Phase 4 planning handoff ready | +| **Date** | 2026-08-29; execution rebaseline 2026-08-30; Phase 3 release certification 2026-08-31 | +| **Baseline inspected** | `v0.3.0-preview.14` at `6d39e0e`, immutable image digest `sha256:e565cc64ee22b5b9f6f99973e3762b639c27e026dc8824852145035acdacf788` | | **North star** | A Deft user can ask Codex to build a useful application, install it into their Deft workspace, and have it participate natively in the human UI, search and knowledge, tasks and chat, Defty, employee agents, human MCP, automation, approvals, runs, receipts, and audit. | | **First delivery principle** | Build one safe participation protocol in stages; do not turn `deft.module.json` into an arbitrary plugin runtime. | | **Relationship to earlier work** | Supersedes the provisional App Protocol v2 implementation sequence while retaining its certified Module, Capability, App Run, and approval-boundary decisions as design input. Absorbs the useful resource-graph and solution-composition ideas from the modular work-management proposal. | @@ -778,15 +778,16 @@ and the staged secret/rollback decision. Phase 3 is not one big-bang PR; migrati identifiers must be re-confirmed at the implementation boundary even though the planning branch is now rebased onto the Phase 2 merge. -**Implementation status (2026-08-30):** PRs A and B merged the frozen contracts, +**Implementation status (2026-08-31):** PRs A and B merged the frozen contracts, purpose-separated key service, additive schema, lifecycle, and fake-provider engine. PR C (#273) merged release/budget evidence, live authority, approval compatibility, ancestry, receipts, Attention, and internal repair invariants. -PR D now contains the pinned worker runtime plus separate exact default-off Run -engine/drain and legacy MCP intake controls. Its local consolidated gate and -cross-process approval/drain transition passed at `fe66113f`; remote PR and -immutable release-artifact gates remain. App origin remains disabled; existing -connector ciphertext and legacy receipt writers are unchanged. +PR D (#274) merged the pinned worker runtime plus separate exact default-off Run +engine/drain and legacy MCP intake controls. The immutable release gate found +and fixed one unrelated App-package digest compatibility defect in #275, then +certified `v0.3.0-preview.14` at `6d39e0e`. The legacy MCP canary is supported +only as an explicit self-host opt-in; App origin remains disabled. See the +[Phase 3 release evidence](../audits/2026-08-31-app-platform-phase-3-release-certification.md). - [x] Add a versioned Secret Service with random nonces, AAD-bound ciphertext, current and decrypt-only keys, rotation, and explicit safe projections. - [x] Add minimum-input rules, strict Run payload/blob limits, retention classes, terminal-state purge, and sanitized audit residue; permission widening cannot silently extend retention. @@ -800,7 +801,7 @@ connector ciphertext and legacy receipt writers are unchanged. - [x] Enforce the actor/App-grant intersection for App origins and the exact current legacy policy for compatibility origins before approval and again immediately before execution. - [x] Recheck membership, employee health/budget, token scope, connector, schema digest, provider, assignment, grants/policy source, and bound authorization versions at execution. - [x] Add sanitized receipts, metrics, Attention, internal bounded inspection/repair primitives, and compatibility adapters without rewriting existing action receipts or claiming a public Operations surface. -- [x] Separate disabled-by-default Run engine/drain and legacy MCP intake controls, reject intake without the engine, and retain a bounded drain-first rollback path; immutable release certification remains the D2 gate. +- [x] Separate disabled-by-default Run engine/drain and legacy MCP intake controls, reject intake without the engine, and retain a bounded drain-first rollback path; immutable release certification passed at `v0.3.0-preview.14`. **Acceptance evidence:** auto and reviewed calls create one Run and at most one provider call; capability cycles stop before a second call and child Runs cannot reset budgets; no pseudo-App grant exists; legacy Runs are not App-discoverable; `always` review cannot be bypassed by an Autonomous employee; payload limits and terminal purge preserve only the declared sanitized residue; restart/replay, encryption/signer/fingerprint rotation, supported-image rollback, low-entropy guessing resistance, unknown-outcome, and ciphertext-leakage tests pass. @@ -978,7 +979,13 @@ Do not begin with custom UI, runtime hosting, or Email Lite. The first shippable This milestone creates a narrow but real user loop while adding no executable App code, new connector authority, background execution, or custom network surface. It also prevents the protocol and tooling from drifting apart. -The next internal milestone extracts CapabilityService with behavior parity. App Runs and the Secret Service follow; Resource Service/privacy follows that. Only after those gates pass does the same package expand into connected grants and Proof A. Governed automation comes next, followed by custom experiences, runtimes, sync, and public ingress as separately revocable planes. +That first milestone, CapabilityService, and governed App Runs are now delivered +through `v0.3.0-preview.14`. The next internal milestone is the Phase 4 Resource +Service/privacy seam described in the current delivery plan and the exact +[Phase 4 loop handoff](2026-08-31-app-platform-phase-4-resource-participation-loops.md). +Only after that gate passes does the package expand into connected grants and +Proof A. Governed automation follows, then custom experiences, runtimes, sync, +and public ingress as separately revocable planes. ## Migration and rollback strategy diff --git a/docs/superpowers/plans/2026-08-30-app-platform-phase-3-closeout-loops.md b/docs/superpowers/plans/2026-08-30-app-platform-phase-3-closeout-loops.md index d5250bf2..4cd30ce1 100644 --- a/docs/superpowers/plans/2026-08-30-app-platform-phase-3-closeout-loops.md +++ b/docs/superpowers/plans/2026-08-30-app-platform-phase-3-closeout-loops.md @@ -2,13 +2,13 @@ | Field | Value | |---|---| -| **Status** | In execution; Loops 0–5 complete, Loop 6 consolidated delta certification next | -| **Date** | 2026-08-30 | +| **Status** | Complete; Loops 0–9 merged, released, and certified | +| **Date** | 2026-08-30; completed 2026-08-31 | | **Architecture source** | `docs/superpowers/plans/2026-08-29-full-surface-app-platform.md` | | **Delivery source** | `docs/superpowers/plans/2026-08-30-full-surface-app-platform-delivery-plan.md` | -| **Merged base** | PR C merge on `origin/master` at `9c8e9ac9` | -| **Local closeout baseline** | PR D C4–D2 replay at `ee513a35`, plus split-control checkpoint `5cf88c51`; local-only and unreleased | -| **Outcome** | Merge and release the guarded Phase 3 execution substrate with Run draining separated from legacy MCP intake, then hand Phase 4 one immutable supported baseline. | +| **Merged base** | PR C #273 at `9c8e9ac9`; PR D #274 at `4bad79d8`; fix-forward #275 at `6d39e0e` | +| **Released closeout baseline** | `v0.3.0-preview.14` at `6d39e0e`, digest `sha256:e565cc64ee22b5b9f6f99973e3762b639c27e026dc8824852145035acdacf788` | +| **Outcome** | Complete. The guarded Phase 3 execution substrate is released at `v0.3.0-preview.14`, Run draining remains separate from legacy MCP intake, and Phase 4 has one immutable supported baseline. | ## Decision @@ -27,11 +27,11 @@ Do not redesign the completed engine. Do not begin ResourceRef, App grants, App-origin execution, connected App UI, or Authoring Kit beta work inside these trains. -## Current immutable facts +## Final immutable facts -- `origin/master` includes the Phase 3 foundation, engine, and PR C trust/data - completion through merge `9c8e9ac9`. -- The original Phase 3 source worktree carried this historical local chain: +- `origin/master` includes the Phase 3 foundation, engine, PR C trust/data + completion, guarded PR D runtime, and the release-gate compatibility repair. +- The original Phase 3 source worktree carried this historical review chain: - `4c4f48c1` — C0 engine/cutover hardening; - `e6274c41` — C1 live authority and budget evidence; - `c3670207` — C2 approval compatibility projection; @@ -39,15 +39,10 @@ trains. - `ffdab418` — C4 runtime/worker composition; - `944b265f` — C5 legacy capability cutover; - `5050b0f1` — local certification documentation. -- PR C replayed C0–C3 and merged as #273. Original hashes remain historical; - PR D was rebuilt from the merge as `1e4ac5a7` (C4), `bd28ff6f` (C5), and - `ee513a35` (historical certification), then added split controls at - `5cf88c51`. No PR D hash is released or an operational rollback floor. -- Before Loop 0, the revised delivery and closeout plans plus the intended - canonical-plan header/pointer edits existed only in the dirty main worktree. - The dirty main copy of the canonical plan predates newer Phase 1–3 evidence, - so only its reviewed header/pointer intent may be applied to the current - tracked body; the dirty originals and unrelated Hermes work remain untouched. +- PR C replayed C0–C3 and merged as #273. PR D was rebuilt from that merge, + merged as #274, and certified with the narrow #275 fix-forward. Original + source hashes remain historical review evidence; the operational floor is the + immutable release above. - Additive migrations `.19`–`.21` are implemented and tested. Preserve them; rewriting their history provides no functional value and invalidates useful evidence. @@ -55,9 +50,9 @@ trains. `DEFT_APP_RUN_LEGACY_MCP_CUTOVER_ENABLED` independently controls new legacy MCP admission. The invalid off/on state is rejected at startup. - App origin, system origin, and automation origin remain fail-closed. -- The current local host lacks pgvector and a running Docker release - environment. Do not retry those known local limitations; run their gates once - on release-capable infrastructure. +- The release-capable host passed pgvector upgrade, recovery, rotation, drain, + quiescence, and immutable rollback gates. The detailed result is + `docs/superpowers/audits/2026-08-31-app-platform-phase-3-release-certification.md`. ## Frozen rollout-control contract @@ -508,7 +503,11 @@ Create a read-only Phase 4 loop plan against the exact released baseline. Freeze - focused acceptance matrix, release environment, exclusions, and stop conditions. -No Phase 4 code begins before the Phase 3 release evidence record exists. +No Phase 4 code begins before the Phase 3 release evidence record exists. That +record now exists at +`docs/superpowers/audits/2026-08-31-app-platform-phase-3-release-certification.md`; +the read-only handoff is +`docs/superpowers/plans/2026-08-31-app-platform-phase-4-resource-participation-loops.md`. ## Phase 3 closeout exclusions diff --git a/docs/superpowers/plans/2026-08-30-full-surface-app-platform-delivery-plan.md b/docs/superpowers/plans/2026-08-30-full-surface-app-platform-delivery-plan.md index 12bc88b9..7e8f8bc8 100644 --- a/docs/superpowers/plans/2026-08-30-full-surface-app-platform-delivery-plan.md +++ b/docs/superpowers/plans/2026-08-30-full-surface-app-platform-delivery-plan.md @@ -2,10 +2,10 @@ | Field | Value | |---|---| -| **Status** | Active execution plan; Phases 0–2 and Phase 3 PR C merged, guarded PR D closeout in progress | -| **Date** | 2026-08-30 | +| **Status** | Active execution plan; Phases 0–3 released and certified; Phase 4 is next | +| **Date** | 2026-08-30; Phase 3 release baseline frozen 2026-08-31 | | **Architecture source** | `docs/superpowers/plans/2026-08-29-full-surface-app-platform.md` | -| **Execution baseline** | Phase 1 `bdb137ee`; Phase 2 `9ba7b7c8`; Phase 3 PR C merge `9c8e9ac9`; PR D replay `ee513a35` plus split-control checkpoint `5cf88c51` (unreleased) | +| **Execution baseline** | Phase 1 `bdb137ee`; Phase 2 `9ba7b7c8`; Phase 3 release `v0.3.0-preview.14` at `6d39e0e`, digest `sha256:e565cc64ee22b5b9f6f99973e3762b639c27e026dc8824852145035acdacf788` | | **North star** | A community member can ask Codex to build any feasible Deft App, install it on an ordinary self-hosted workspace, and have it participate through governed resources, knowledge, agents, capabilities, experiences, automation, runtimes, sync, and public ingress as required by that App. | ## Decision @@ -84,70 +84,36 @@ trusted API, web, or worker processes. preserving additive `.19`–`.21`, release/budget evidence, approval ownership, ancestry, receipts, Attention, and internal repair invariants. -The remaining PR D chain is local-only and unreleased: `1e4ac5a7` (C4 replay) --> `bd28ff6f` (C5 replay) -> `ee513a35` (historical certification replay) -> -`5cf88c51` (split controls) -> `a6f2eed2` (closeout contract) -> `fe66113f` -(restart-transition certification). Original C0–D2 hashes remain historical -review evidence. None of these hashes is a supported rollback floor; the floor -is the eventual immutable released image revision recorded by the release gate. +- **Phase 3 PR D (#274):** guarded runtime, split controls, rollout transition, + and operations contract merged at `4bad79d8`. +- **Phase 3 fix-forward (#275):** the release gate repaired the Hello Workspace + package-byte/parsed-manifest digest mismatch without changing Run, migration, + token, connector, UI, or control contracts. The immutable certified baseline + is `v0.3.0-preview.14` at `6d39e0e`, image digest + `sha256:e565cc64ee22b5b9f6f99973e3762b639c27e026dc8824852145035acdacf788`. No completed phase is restarted or redesigned. Later work must consume these deep services rather than create parallel execution, package, or authorization paths. -### Phase 3 closeout gate - -The historical C0–D2 work through `5050b0f1` is preserved as review evidence, -not treated as proof of the newer split-control contract or as an App-facing -feature. PR D's new checkpoint and consolidated matrix own closeout evidence. - -Before it merges: - -1. Record the dual-ledger exit: App Runs are canonical execution state; - `agent_actions` is the current approval projection and legacy receipts remain - historical compatibility. After native App approval/Run UI and Phase 6 - parity are proven, no new App path may depend on the compatibility ledger. -2. Preserve the already tested additive `.19`–`.21` migrations by default. - Consolidate them only if repository policy requires one unreleased train to - equal one migration, no supported or retained environment has recorded - their checksums, disposable environments are reset, and the resulting - upgrade/engine evidence is rerun. -3. Describe `AppRunOperationsService` as an internal repair primitive with a - runbook/SQL operator path. Do not claim a public operations surface until - one exists. -4. Keep `origin = app`, system/automation origins, and broad rollout fail-closed - until later phases provide their host-owned authority sources. -5. Keep the exact legacy MCP intake path default-off, with no shadow provider - call and no fallback after a governed attempt begins. -6. Keep Run runtime/key availability separate from legacy-connector cutover; - reject intake-on/engine-off and document the three safe combinations. Add an - App-origin control only with Phase 5's real grants/bindings entrance. - Enabling community Apps must not implicitly migrate all existing MCP traffic. -7. Split-control checkpoint `5cf88c51` supersedes `5050b0f1` for rollout - evidence. Complete its flag-combination, startup, queue-deferral, keyring, - rollback, and compatibility delta before PR D is certified. - -Recommended review shape: - -- **PR C — trust and data completion:** C0–C3 release fencing, live authority, - budget evidence, approval adapter, selected ancestry constraints, receipts, - Attention, and repair invariants. -- **PR D — guarded runtime and certification:** C4–C5 composition, exact - default-off legacy canary, rollback/key runbook, and the certification record. - -PR D may stack on PR C only while commit ancestry is preserved. If PR C is -squash-merged or rewritten, rebuild/rebase PR D on the resulting default branch -and recertify the changed delta rather than treating old hashes as evidence. - -Merge evidence is limited to changed boundaries plus one consolidated delta -matrix: shared contracts, final upgrade representation, App Run database suite, -engine-off/intake-off, engine-on/intake-off, and engine-on/intake-on modes in one -disposable-database fixture, focused -approval/trust/connector/worker/crypto/architecture tests, repository typecheck, -and one production source build. Browser testing is required only if web code -changes. Production-image, current-schema, backup/restore, rollback, and -deterministic-provider browser drills remain rollout gates and run once on the -release candidate. +### Phase 3 closeout result + +The closeout gate passed. App Runs are canonical execution state; +`agent_actions` remains the compatibility approval projection and legacy +receipts remain historical. Migrations `.17`–`.21` are additive and recorded. +`AppRunOperationsService` remains internal. App/system/automation origins remain +closed. The legacy MCP canary is default-off, has no shadow call or governed-to- +legacy fallback, and is supported only as a documented self-host opt-in. + +The release gate covered immutable source/image provenance, pgvector install and +upgrade, connector ciphertext continuity, all four rollout states, +deterministic-provider replay, Hello Workspace lifecycle, matched backup/ +restore, three-purpose key rotation and retirement refusal, pause/drain/resume, +quiescence, and actual rollback to the supported predecessor. See the +[immutable release certification](../audits/2026-08-31-app-platform-phase-3-release-certification.md). + +The exact Phase 4 implementation handoff is +[Resource participation loops](2026-08-31-app-platform-phase-4-resource-participation-loops.md). ## Revised delivery graph diff --git a/docs/superpowers/plans/2026-08-31-app-platform-phase-4-resource-participation-loops.md b/docs/superpowers/plans/2026-08-31-app-platform-phase-4-resource-participation-loops.md new file mode 100644 index 00000000..a42d84f2 --- /dev/null +++ b/docs/superpowers/plans/2026-08-31-app-platform-phase-4-resource-participation-loops.md @@ -0,0 +1,420 @@ +# App Platform Phase 4 — Resource participation loops + +| Field | Value | +|---|---| +| Status | Read-only implementation handoff; no Phase 4 code started | +| Released baseline | `v0.3.0-preview.14` at `6d39e0e0413c82d36c9481849ae582fdf805d1a6` | +| Baseline image | `sha256:e565cc64ee22b5b9f6f99973e3762b639c27e026dc8824852145035acdacf788` | +| Architecture source | `2026-08-29-full-surface-app-platform.md` | +| Delivery source | `2026-08-30-full-surface-app-platform-delivery-plan.md` | +| Phase 3 evidence | `../audits/2026-08-31-app-platform-phase-3-release-certification.md` | +| Outcome | Two independently installed declarative Apps can relate live-authorized resources without copying them; the same seam works for one core resource without weakening its current authorization. | + +## Why this is the next slice + +Phase 3 supplied durable governed effects. It deliberately did not give Apps +authority. Phase 4 supplies the other half of native participation: a stable way +to address, resolve, relate, and search resources while the host remains the +only authorization authority. + +This phase is not a universal rewrite of Deft data. It extracts the smallest +resource seam needed by two independent Apps, then proves that seam against a +heterogeneous core resource. Existing Module APIs, data, same-installation +relations, search results, and agent behavior remain valid throughout. + +## Architecture gate decision + +The user problem is cross-App composition without copied records or duplicated +authorization. The required outcome is one small host-owned seam that can serve +Module records and a real core resource while preserving tenant isolation, +current owner authorization, Module v1 bytes, and additive rollback. + +Three implementation shapes are credible: + +| Option | Strength | Why it loses or wins | +|---|---|---| +| Extend only `module_record_relations` across installations | Smallest immediate schema change | Loses: it embeds Module identity/authorization in every caller, cannot prove Task reuse, and makes the later Resource Service a second migration. | +| Closed ResourceAuthorizationService plus host-bound ResourceRef and adapters | Small public contract; current owners retain authorization; additive relation store supports heterogeneous endpoints | Wins: it is the smallest seam that satisfies both cross-App and Task proofs, is reversible behind existing callers, and keeps provider registration code-owned. | +| Universal registry and immediate conversion of all core resources | Maximum apparent generality | Loses: speculative adapters, broad migrations, large privacy blast radius, difficult rollback, and no evidence that the first proof needs most of it. | + +Choose the closed service and adapter option. Removing it would force relation, +picker, search, context, agent, and later App-grant callers to repeat provider +routing and live authorization, so it passes the deletion test: the complexity +is real and belongs behind one interface. Keep the interface deep by exposing +resolve/mutate/search semantics rather than adapter storage details. + +The main risks are a prematurely broad ResourceRef, generic edges without +cross-table foreign keys, temporary dual relation stores, and authorization +drift. Mitigate them with a closed provider union, no generalized public route +or token scope, host-derived organization, endpoint locks and live owner +authorization, a v1 compatibility adapter, deterministic parity shadows, and +release rollback that retains additive tables. The acceptance matrix at the end +of this plan is the evidence required before the decision is considered proven. + +## Frozen proof and scope + +### Independent artifacts + +Build two independent App v0 fixtures using only the public App Kit: + +- **Contacts** owns a Contacts Module with organization-visible contact + records. +- **Campaigns** owns a Campaigns Module whose records can reference Contacts + from the separately installed Contacts App. + +The Apps have distinct canonical lineages, packages, installations, Module +installations, and lifecycle epochs. Neither may assume the other's database +IDs or copy the other's records. Phase 4 does not add an App dependency grant; +Phase 5 binds exact App dependencies and capabilities. + +Freeze a deterministic sandbox-email MCP provider alongside the fixtures, but +do not invoke it in Phase 4. It constrains the future Campaign action and proves +that `update campaign` cannot hide a network effect. Sending email begins in +Phase 5 through CapabilityService and an App Run. + +### Exact operations + +The first proof needs only: + +- resolve one Contact or Campaign by ResourceRef; +- list safe references for an authorized picker; +- create, update, and archive through the owning Module service; +- link and unlink a Campaign-to-Contact relation with an idempotency key and + optimistic-concurrency precondition; +- render the resolved Contact label in the existing generic Module UI; +- include authorized Contact/Campaign results in the existing Module search and + agent context paths; and +- stop resolution immediately when the actor loses access or either owning + installation is disabled. + +It does not need arbitrary graph traversal, bulk mutation, files/blobs, +specialized provider resources, App tokens, capability grants, or email send. + +### Caller surfaces + +The Phase 4 seam serves these existing authenticated callers: + +1. human Module REST/UI reads and mutations; +2. current Module agent tools under their existing employee trust and + installation policy; +3. existing Module search/context/citation assembly; and +4. internal conformance tests using explicit human and employee actors. + +Do not add a generalized public `/api/resources` surface or new OAuth/MCP scope +in this phase. Phase 5 exposes App-scoped discovery and invocation after grants +exist. Personal MCP clients continue through their current Module tools. + +## Frozen ResourceRef contract + +The shared contract is a strict, bounded discriminated object: + +```ts +type ResourceRefV1 = { + schema_version: 'deft.resource_ref.v1'; + provider: + | { kind: 'module'; provider_instance_id: string } + | { kind: 'core'; provider_instance_id: 'tasks' }; + resource_type: string; + resource_id: string; +}; +``` + +For a Module record, `provider_instance_id` is the stable Module installation +ID, `resource_type` is the collection key, and `resource_id` is the opaque +record ID. For a Task, `provider_instance_id` is exactly `tasks`, +`resource_type` is exactly `task`, and `resource_id` is the current task ID. + +The serialized client contract contains no organization ID. The host binds the +reference to the authenticated organization before lookup. Persisted relation +rows carry a host-derived `org_id`; callers cannot use a reference to select or +override it. Unknown provider combinations fail with a stable structured error +before any adapter call. + +Adding a future provider kind is a versioned shared-contract change plus a +reviewed adapter registration. It is not a database row or App-manifest string +that dynamically loads code. + +## Authorization and data-flow boundary + +`ResourceAuthorizationService` is the only cross-provider resolve/mutation +entry point. It performs this order: + +```text +authenticated actor + host org + -> parse bounded ResourceRef + -> select closed code-owned adapter + -> resolve current owning resource + -> delegate to the owner's current authorization + -> return a safe projection or perform the authorized mutation +``` + +The service does not duplicate Module or Task access rules. The Module adapter +uses Module installation/record authorization. The Task adapter uses the +current Task/project authorization. Cached labels, search documents, relation +rows, provider claims, and App metadata may deny or assist lookup; none may +allow access. + +Pure canonical state changes may use Resource Service only when the host owns +the state, schema validation, authorization, audit, idempotency, and concurrency +rule. Any remote, network, connector, or independently operated effect must use +CapabilityService and an App Run. An adapter must reject an `update` request +that would conceal such an effect. + +## Relation and lifecycle contract + +Reserve the next migration identifier only at Loop 0 after re-reading the +current upgrade manifest. At the released baseline, the next apparent slot is +`0.3.0-preview.22`; it is not assigned until that check passes. + +The additive relation table stores: + +- organization; +- a source ResourceRef tuple; +- relation type and stable relation ID; +- a target ResourceRef tuple; +- position where ordered cardinality needs it; +- idempotency key and actor/audit metadata; and +- creation/update timestamps plus a soft-deleted marker. + +Both endpoints are resolved and authorized in one transaction before insert or +replacement. Database uniqueness prevents duplicate active edges and +cross-organization queries always include `org_id`. Because heterogeneous +targets cannot share a useful foreign key, service authorization and +tenant-scoped lookup are mandatory; relation rows never prove endpoint +existence or access. + +Lifecycle behavior is fixed: + +- **Module/App disabled:** retain edges, hide ordinary resolution/picker/search, + and restore visibility only after re-enable plus live authorization. +- **Record archived:** retain edges; ordinary resolve returns unavailable while + an authorized management projection may report `archived` without leaking + fields. +- **Record soft-deleted:** retain a dangling safe reference with no cached label + disclosure. Do not silently retarget or cascade-delete the opposite record. +- **Compatible Module upgrade:** stable installation and record IDs preserve + edges. +- **Incompatible staged upgrade:** active pointers and edges remain unchanged + when compatibility validation rejects activation. +- **Source unavailable:** ordinary relation reads return no target data. An + authorized management path may count dangling edges without disclosing the + inaccessible endpoint. + +Existing `module_record_relations` remain the optimized same-installation v1 +implementation. Phase 4 must not reinterpret them. A compatibility adapter may +project them as ResourceRefs; only new cross-installation fields use the new +relation store. + +## Manifest compatibility decision + +Module manifest schema `1`, its parser, canonical bytes, and digests are frozen. +Cross-installation reference fields require an additive Module manifest schema +version, provisionally `2`, with an explicit parser and upgrade compatibility +rule. Do not add optional keys to the v1 parser that change its canonical +output. + +The v2 reference field declares a bounded target interface, cardinality, and +display behavior. It does not contain tenant IDs, grants, connector IDs, +provider URLs, executable hooks, or authorization rules. The installed host +resolves compatible target installations and the actor's access live. + +## Task decision + +Task is the immediately following heterogeneous fixture, not a blocker for the +first Campaign-to-Contact proof. It must land before Phase 4 is declared +complete. This ordering allows the first relation to validate the Module/App +composition need, then forces the seam to generalize beyond Module internals. + +The Task adapter initially supports resolve and safe projection only. Task +mutation remains on the current Task service unless a later, separately tested +change proves exact parity. Task links are shadow-projected as ResourceRefs; +existing task relation tables and APIs remain source of truth. + +## Delivery loops + +### Loop 0 — Reconcile and freeze + +- Re-read current schema, upgrade manifest, Module v1 contracts, Module + relation service, Task authorization, search/context callers, and all dirty + worktree state. +- Confirm the released Phase 3 baseline and next available migration ID. +- Materialize the two independent App/Module fixtures and deterministic sandbox + provider without adding product behavior. +- Write the ResourceRef/authorization ADR with error shapes, data flow, + lifecycle table, rollback, and rejected alternatives. +- Freeze the focused acceptance matrix and PR boundaries below. + +**Stop condition:** if current Task/Module authorization cannot be delegated +without widening or if a caller needs a new token scope, stop and move that +caller to Phase 5 rather than weakening the boundary. + +### Loop 1 — Shared contract and closed service seam + +- Add strict shared ResourceRef schemas, limits, structured errors, safe + projection types, and adapter interfaces. +- Add `ResourceAuthorizationService` with an empty/closed registry and explicit + host context. +- Test malformed refs, unknown provider combinations, tenant spoof attempts, + adapter error normalization, and denial-before-disclosure. +- Add architecture tests preventing routes, App artifacts, and providers from + bypassing the service once they adopt it. + +This loop changes no schema, UI, App manifest, token, connector, or provider +execution path. + +### Loop 2 — Module adapter and parity shadow + +- Implement resolve/list-safe-projection and owner-service mutation delegation + for Module records. +- Project existing same-installation v1 relations as ResourceRefs without moving + or rewriting rows. +- Run byte/state parity against the existing Module API for human and employee + actors, disabled installations, deleted records, manifest mismatch, + optimistic concurrency, and idempotency. +- Keep production reads on the old path while collecting deterministic shadow + comparisons in tests and the certification fixture. + +**Stop condition:** any authorized-result difference is resolved before a new +relation schema or UI caller is added. + +### Loop 3 — Additive cross-installation relations and Module v2 + +- Add the confirmed versioned migration and Drizzle schema for generic relation + edges. +- Add the independent Module v2 parser/JSON schema and prove Module v1 parsing, + canonicalization, and digest fixtures are byte-identical. +- Implement link, unlink/replace, list, and safe dangling-state behavior with + endpoint locks, optimistic concurrency, idempotency, and audit. +- Test cross-org IDs, valid IDs from another installation, disabled endpoints, + archive/delete races, duplicate/reordered writes, rollback, and failed + activation. + +Migration rollback is image rollback with additive tables retained; no +down-migration or v1 row rewrite is permitted. + +### Loop 4 — Contacts/Campaigns compound proof + +- Independently build/check/package/install Contacts and Campaigns. +- Create Contacts and Campaigns through the existing generic Module UI/API. +- Link Campaigns to Contacts through Resource Service and the new relation + contract. +- Add the bounded reference picker/rendering behavior needed by the generic + Module form and detail view. UI completion requires desktop and mobile visual + inspection. +- Prove disable/re-enable, archive, delete/dangling, compatible upgrade, and + failed incompatible upgrade behavior without copied records or domain code in + core. +- Prove current employee Module tools see only the same authorized projections. + +The sandbox-email provider remains unused and its call count must remain zero. + +### Loop 5 — Task heterogeneous proof + +- Register the closed `core/tasks` adapter. +- Resolve safe Task projections through current Task/project authorization. +- Shadow-project existing task links as ResourceRefs without changing their + source tables or public behavior. +- Add a bounded Campaign-to-Task or Contact-to-Task conformance fixture only if + it improves the heterogeneous proof; do not add CRM-specific UI. +- Test project membership loss, assignment changes, private/restricted Task + visibility, deleted Tasks, stale refs, and cross-org IDs. + +Task mutation and universal core-resource conversion remain out of scope. + +### Loop 6 — Search, context, and citation cutover + +- Adapt only Module results used by the two proof Apps and Task results used by + the heterogeneous fixture. +- Compare old and Resource Service results for authorized IDs, safe fields, + ranking inputs, snippets, and citations. A projection mismatch cannot be + waived by a higher result count. +- Resolve and authorize live immediately before returning content; stale search + documents can only nominate candidates. +- Add revocation tests where membership, installation state, record state, or + Task visibility changes after indexing. +- Use one bounded default-off rollout control if runtime shadowing is required; + document it and its removal criterion. Do not add parallel permanent search + architectures. + +### Loop 7 — Consolidate, release, and hand Phase 5 exact inputs + +- Run shared ResourceRef and authorization contracts. +- Run Module v1 byte/digest compatibility and Module v2 tests. +- Run relation database, tenant, lifecycle, idempotency, concurrency, and + malicious-provider tests. +- Run Module human/employee parity, Task authorization parity, search/context + shadow/cutover, and architecture tests. +- Run API/shared/web/App Kit typecheck, one production build, and focused UI + visual checks for the generic picker/detail behavior. +- Prove fresh pgvector schema, supported upgrade from `preview.14`, matched + backup/restore, image rollback reads, and independent Contacts/Campaigns App + installation on the release candidate. +- Record exact released commit/image, evidence, remaining adapters, and the + Phase 5 sandbox-email interface inputs. Do not begin Phase 5 code in this + loop. + +## PR and review shape + +Use additive merge trains rather than one large PR: + +1. **PR A — contracts and authorization seam:** Loop 0 ADR plus Loop 1. +2. **PR B — Module adapter and relation substrate:** Loops 2–3, including the + migration and Module v2 compatibility proof. +3. **PR C — proof callers and heterogeneous adapter:** Loops 4–6. +4. **PR D only if needed — release-only repair:** bounded fixes revealed by the + release gate; do not mix Phase 5 grants or capability work into closeout. + +Each PR gets focused checks while developing and one consolidated relevant +validation pass. Release infrastructure, fresh schema, restore, and rollback +run once against the merged candidate unless an earlier diff directly changes +them. + +## Acceptance matrix + +Phase 4 passes only when all are true: + +- Contacts and Campaigns are independently built and installed; a Campaign + references a Contact without copying it. +- No ResourceRef value can select an organization or dynamically load an + adapter. +- Cross-org and unauthorized endpoint resolution fails before labels, snippets, + or fields are disclosed. +- Module/App disable, archive, delete, membership loss, and Task visibility + changes take effect on the next live resolve/search/context request. +- Existing Module v1 bytes/digests and same-installation relations are unchanged. +- Relation writes are tenant-bound, idempotent, concurrency-checked, audited, + and deterministic through disable/upgrade/archive/delete. +- Task authorization parity proves the seam is not Module-specific. +- Stale search projections and malicious provider access claims never + authorize. +- The sandbox-email provider receives zero calls. +- App origin, grants, connector bindings, and new token scopes remain absent. +- Fresh install, supported upgrade, backup/restore, and image rollback evidence + pass on a release-capable host. + +## Explicit exclusions + +- App manifest capability/resource requirements, dependency grants, connector + bindings, AppActionService, App-origin Runs, App Run UI, and App token scopes + belong to Phase 5. +- Email send and the connected CRM/Marketing beta belong to Phases 5–6. +- Messages, wiki, notes, files, calendar, people, teams, specialized provider + resources, sync, automation, custom UI, runtimes, and public ingress remain + evidence-led later adapters/tracks. +- No domain-specific Contact/Campaign table, route, service, agent tool, or core + branch is allowed. +- No marketplace, billing, hosted KMS, SaaS control plane, or mandatory hosted + dependency enters Phase 4. + +## Completion and pause rule + +Phase 4 is complete only after the released evidence record proves the compound +App relation and heterogeneous Task seam on an ordinary self-host stack. A +contract merge without live authorization, lifecycle, search/context, and +rollback evidence is infrastructure, not completion. + +Pause and reopen the architecture gate if implementation would require a new +tenant selector, token scope, App grant, dynamic provider registration, +in-process third-party code, remote effect hidden as resource mutation, v1 +canonicalization change, down-migration, or authorization rule duplicated +outside its owning service.