Skip to content

feat: propagate canonical identity through harness API v2 and clients - #2407

Draft
jbeda wants to merge 3 commits into
impl/2117-identity-adaptersfrom
impl/2117-identity-host
Draft

jbeda wants to merge 3 commits into
impl/2117-identity-adaptersfrom
impl/2117-identity-host

Conversation

@jbeda

@jbeda jbeda commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

Current engine prerequisite and review status

The engine prerequisite is now #2404 at 80ddd21af, based on 55b980d34. This host branch has not been refreshed onto that engine checkpoint. Scope/file counts below describe the original split; refresh stacked comparisons and qualify the assembled current-main stack before advancing delivery. Source callers of removed eventcodec.Project/Projection must use eventsource.Project/Projection; no compatibility wrappers remain.

The verification receipts below are historical 2026-10-18 evidence, not qualification of the latest engine plus this downstream stack. Historical plan comments may describe superseded hold/continuation policy; the contract below and #2404's current source supersede it. No new tests or source/pin changes were made for this description update.

Scope

Review stack: #2404 engine → #2405 providers → #2406 adapters → #2407 host. Historical delivery-plan archive; migration policy superseded below.

Fourth draft in the #2117 canonical-identity review stack. Base: impl/2117-identity-adapters. This is the existing coordinated consumer implementation, extracted into its own review diff (318 files), not a new TUI/presentation feature round.

  • Propagate canonical engine invocation identity through host execution/control, authorization, hooks, audit, session debug/ACP, token accounting and learning projections.
  • Keep tentative provider replay inside resilience/retry boundaries; copy history and compact/provider-switch without guessed pairing.
  • Expose approved Harness API major 2; remove/reserve public opaque archive reasoning fields, retain readable text/reasoning/media/phase and discovery routes.
  • Regenerate protobuf/TypeScript/Studio contracts via owning generators (preserved from the qualified checkpoint), and update required SDK/client/BFF identity propagation.
  • Render unprovable legacy activity as standalone content without guessed tool/approval/delegation links or execution authority.
  • Retain owning compatibility/upgrade documentation and domain/architecture corrections.

No new TUI layout/correlation redesign, guardrail lifecycle feature, general legacy reconciliation or changes to the separate mecatl-3 checkout. This does not close all of #2117.

API compatibility

Approved breaking Harness API major 2, including engine-originated canonical-ID semantics. Servers and first-party clients upgrade together; existing discovery/routes stay reachable for actionable major-skew refusal. No /v2 route/package rename solely to match the advertised major. Remove/reserve public ConversationMessage.reasoning and reasoning_item_id; readable reasoning and phase/media remain. Private provider replay stays in persistence/internal LLM boundaries, not public history/archive/evidence projections. SDK package semver remains independently 0.5.0 in this source; it is not changed to 2.0 merely for wire major 2.

Go: coordinated callers consume the already-approved breaking pre-v1 engine minor and changed provider/adapter module APIs. Pins must be reconciled against actual released prerequisites before merge/publication; no invented versions or permanent replacements.

Stored data: new writers use sessnap-json/2 and eventlog-json/2; upgraded readers support old/new subject to validation. Drain old readers/writers, upgrade both decoding snapshot/event drivers, then harness/compatible clients. No mixed writers or downgrade after new writes. Preserve nonpending legacy snapshot history/evidence, but reject legacy awaiting/authorizing snapshots even with absent/null pending payloads, and reject any non-null legacy pending approval/private-authorization payload in any state. Resolve parked work before upgrade or start fresh; no silent idle fallback. Any identity_hold key, including null/schema 2, rejects; direct schema-zero Snapshot inputs are unsupported. Valid current /2 controls retain effective arguments and deferred order. Complete-event reconstruction separately preserves provable legacy controls and abandons unprovable continuations before later canonical work with an unknown-effects notice. Sparse history and copied IDs never grant execution/learning authority. Do not delete durable data or reset log cursors to hide presentation skew.

ACP boundary: passive hydration is effect-free. Explicit editor reconnect may attach placement, mount supplied MCP servers, discover commands/capabilities and report setup errors; it cannot execute/resume tools, grant permission or publish learning. A separate prompt follows normal admission. No additional ACP protocol change is authorized by the engine cleanups.

Source identity and verification

Historical host head da27200c3d341262d6697d6c3d4a8822011959c0 has tree 3763ffe56e1ba21bab363d3ad416c5e4a50077d7, byte-identical to the retained assembled candidate (2dc6d6f9a8526e5be702187bd8dd438d0981fc94). The split adds no implementation delta beyond three new provider composition tests and their stronger event assertions.

Fresh engine-only standalone build/test, engine race and API snapshot freshness passed. New real-engine/real-provider HTTP round trips pass normal and race for Responses, Anthropic and Chat: repeated provider key, unique canonical identities, matching tool/events/history/results and correct third-request pairing/private replay. Four-axis incremental test review is clear after fixes. Original complete implementation/rebase reviews and SDK/Studio evidence are retained.

Historical complete assembled qualification passed on 2026-10-18, Go 1.27.2, Linux/amd64, with no uncommitted source changes:

  • task --concurrency 3 test:workspace:race — all 12 modules plus offline microvm_dev proof.
  • task lint — all module checks, 0 issues.
  • task docs and task site:build; generated references unchanged.
  • GOWORK=<assembled-checkout>/go.work go run ./cmd/mecademo — Read/result, Write ask/approval/result, team and background flows.
  • task test:module-metadata and clean tracked tree.

Workspace selection uses that checkout's absolute go.work, existing unchanged dependency metadata and exactly the source tree above. These receipts do not establish released dependency closure or a complete browser/image pass.

Explicit remaining gates

  • Real engine/provider/adapter releases and dependent pin updates/tidy; then all workspace-disabled and external-consumer/publication proofs and complete normal task lint && task test:race must pass with published closure. Workspace passes do not substitute for these.
  • Current-main integration qualification: the historical host checkpoint used base f9a3297c191df9f2a0d7e573f4e7dbba32c12695; engine feat(engine): establish canonical tool-call identity and private replay #2404 has since rebased onto 55b980d34 and advanced to 80ddd21af. Propagate/review the current engine and relevant main changes across the downstream stack, then rerun assembled and publication gates. The preserved host tree and older receipts do not qualify that integration.
  • Browser journeys and fresh-image verification remain outstanding. Chromium-full CI now provides concrete evidence: two tests pass; multimodal permission flow completes successfully but its final assertion still expects provider token browser-multimodal-write inside an opaque/canonical ask ID (sdk/typescript/e2e/browser/chromium.e2e.test.ts:361); the following cancel/steer/reattach test times out. The first is a confirmed stale assertion; the second is not yet runtime-diagnosed (worker restart/global script cursor is a source-supported hypothesis, not a confirmed cause). These are separate host-browser blockers, not expected unpublished-pin failures or an engine-standalone failure. They are recorded for the host follow-up rather than swept into the engine PR. No complete browser/image/deployed pass claimed.
  • Website installation also reports 65 audit findings in the existing unchanged dependency closure (2 low, 19 moderate, 25 high, 19 critical). No dependency changes or security-audit/remediation claim in this split.
  • Expected compile/test/lint/publication failures from old prerequisite pins are not a required-check waiver. Investigate other failures normally; explicit maintainer decision required for any red intermediate merge.

Release order

Review all coordinated source before the engine merge. Human merge/main-reachable engine release (separate authorization) → real engine pins and standalone checks in all changed provider modules and adapters → their independent releases → real host/integration pins, tidy and full standalone/publication/client gates → coordinated deployment only with separate authorization. The source drafts minimize preparation time during the red-main window but cannot eliminate release latency.

Draft for review and integration preparation, not merge/release-ready. No releases, deployments, CI bypasses or merges performed.

@jbeda

jbeda commented Oct 18, 2026

Copy link
Copy Markdown
Contributor Author

Approved engine-first delivery plan

Archived: 2026-10-18. Applies to review stack #2404 → #2405 → #2406 → #2407; exact branch/head/baseline identities below. Local plan paths are provenance only, not remote links. This is the full selected split/qualification plan; verification tasks below describe intended checks, not implicit pass claims. Actual results and remaining gates are recorded separately in the PR bodies.

Engine-first identity delivery

Slug: 2117-engine-identity
Plan path: .scratch/2117-engine-split/plan.md
Original implementation specification: .scratch/2117-tool-call-identity-plan.md (already implemented and reviewed; not reimplementation scope).

Immutable delivery baseline and branches

Base branch: main; recorded start commit f9a3297. Remote main observed at c4a043f (#2402, TUI/docs only); no automatic rebase. Host requires current-main merge qualification before merge.

Review branch Base branch Source head
impl/2117-engine-identity main 5d830f8
impl/2117-identity-providers impl/2117-engine-identity c669ec8
impl/2117-identity-adapters impl/2117-identity-providers 20f60fd
impl/2117-identity-host impl/2117-identity-adapters da27200

These bases organize review, not new module dependencies: providers and adapters both depend directly on engine. The host source tree exactly equals the retained assembled qualification candidate (tree 3763ffe56e1ba21bab363d3ad416c5e4a50077d7). The root checkout remains on the engine review branch; assembled/preparation worktrees stay under .scratch/2117-engine-split/.

Approved outcome and authority

Joe approved on 2026-10-18: prepare, commit, push and open four draft PRs: engine; changed providers; persistence adapters; host/contracts/required client integration. Humans alone merge. No tags/releases, deployment, invented dependency pins, CI bypasses or new TUI presentation work. Preserve the assembled candidate in a local worktree under .scratch and keep the engine review branch at the main checkout root.

Source boundary and contracts

  • Engine PR: engine/** and the necessary .golangci.yml depguard allowance, including engine-owned tests, migration fixtures, generated API snapshots, changelog and compatibility documentation.
  • Provider PR: existing changes under provider/openai, provider/anthropic and provider/openaichat, plus narrow offline real-engine/provider HTTP composition regressions. These modules release independently.
  • Persistence PR: adapters/**, including its changelog and driver format/skew coverage.
  • Host PR: remaining existing coordinated source, generated wire/SDK/Studio contracts, required client propagation and owning documentation. No new presentation work and no changes to mecatl-3.
  • Engine-minted canonical ToolCallID is distinct from provider pairing/item IDs; whole-batch validation before effects; copied history preserves occurrence IDs. Session owns opaque replay and bindings. Ambiguous legacy state is preserved but inspection-only before authority consumption/execution/learning.
  • Preserve existing provider fidelity, not a fuller replay redesign. Preserve allow/deny/ask, isolation, scrubbing and argument ownership behavior.

API compatibility

This is the already-approved breaking pre-v1 Go API minor, not additive-only and not /v2 import migration. Engine APIs change invocation identity, provider-neutral DTOs and private replay ownership. The engine PR does not itself change protobuf fields/server major, but carries engine-originated observable semantics that the final host PR exposes as approved Harness API major 2. Public opaque archive reasoning fields are removed/reserved in that final coordinated host/client change; readable reasoning/text/media/phase remain. Snapshot/event writers advance to sessnap-json/2 and eventlog-json/2; upgraded readers accept /1 and /2. Drain old readers/writers, upgrade both decoding drivers, then harness/clients; no mixed writers or downgrade after new writes. Exact module release versions remain unselected and publication requires separate authority.

Tasks and evidence

  1. Preserve assembled checkpoint; extract engine-only source in the root checkout and document exact base/HEAD identities.
  2. Verify existing engine standalone API/layering/migration tests. Capture engine-only root/provider compilation failures rather than guessing expected failures. Do not suppress unrelated defects.
  3. Reuse provider HTTP/SSE fixtures in one narrow real engine + real adapter + trivial tool test per changed provider. Assert canonical call/result ownership and next-request provider pairing/replay including completed provider-key reuse. No server, UI, deployment or paid/live calls. Keep engine dependencies inward.
  4. Run focused normal/race composition checks and assembled workspace/race, lint, docs and offline demo at the owning candidate; preserve original historical reviews and perform incremental review of new tests and split boundaries, not another full historical panel.
  5. Prepare stacked draft branches in dependency order, with exact source boundaries and matching assembled engine tree. Preserve a fully assembled stack tip. No fabricated PR-head module versions or permanent replacements.
  6. Publish four draft PRs with API compatibility statements, exact evidence, expected intermediate CI failures, dependency/release checklist and plan archive comments. Read back published bodies/comments and record their URLs/heads.

Merge/release readiness

Reviewable source is not merge/release-ready. Before the engine merge, dependent source must be reviewable, assembled behavior qualified and release/pin order explicit. Real engine publication precedes provider/adapter pins and independent releases, then final host/integration pins and standalone/publication qualification. Existing required checks are not waived: Joe accepted review drafts with accounted-for consumer failures; any intermediate merge still needs a separate explicit maintainer decision. Release latency cannot be eliminated. Browser/image qualification remains unclaimed unless actually completed; final host draft must disclose inherited limitations.

Verification ownership

The directing agent owns split, integration, PR publication and evidence. A focused writable worker adds provider composition tests only in the retained assembled worktree and runs narrow checks. New tests get independent incremental panel review. Full Go gates use 1800-second outer limits; other full gates 600 seconds. Heavy suites run serially. No HANDOFF update.

jbeda and others added 3 commits October 29, 2026 15:45
Integrate the engine, provider and persistence foundation with host controls, projections, learning, generated contracts, SDK and required historical client rendering. Preserve readable content while keeping opaque provider replay private.

Co-authored-by: Mecatl <noreply@mecatl.dev>
…learning

Keep client history regular, preserve honest debugger completeness, and enforce authoritative learning eligibility across admission, recovery, and publication. Retain current-run evidence validation and current delegation observability.

Co-authored-by: Mecatl <noreply@mecatl.dev>
…cleanup

Retain canonical call correlation without rejecting reset continuation sequences. Enforce inactive candidate retirement in durable activation, cover persisted recovery checkpoints, simplify ACP and evidence projections, and require usable snapshot fallback. Preserve existing engine sequencing, active skills, and accepted legacy-event skipping.

Co-authored-by: Mecatl <noreply@mecatl.dev>

This branch was successfully deployed

1 active deployment
Preview — 87f566fe Deployed Oct 29, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant