Skip to content

feat(providers): preserve pairing and replay behind canonical identity - #2405

Draft
jbeda wants to merge 5 commits into
impl/2117-engine-identityfrom
impl/2117-identity-providers
Draft

jbeda wants to merge 5 commits into
impl/2117-engine-identityfrom
impl/2117-identity-providers

Conversation

@jbeda

@jbeda jbeda commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

Current engine prerequisite and review status

The engine prerequisite is now #2404 at 80ddd21af, based on 55b980d34. This provider branch has not been refreshed onto that engine checkpoint. Scope/file counts below describe the original split; the live stacked comparison needs refreshing before further integration qualification.

The engine now rejects unsupported legacy snapshot continuations, normalizes supported snapshot versions before Restore, and owns complete-history reconstruction in eventsource rather than eventcodec. These are engine/persistence policies, not new provider behavior. Historical plan comments may describe superseded migration policy; use #2404's current description and source.

The verification receipts below are historical 2026-10-18 evidence, not qualification of the latest engine plus this downstream stack. No new tests or source/pin changes were made for this description update.

Scope

Review stack: #2404 engine → #2405 providers → #2406 adapters → #2407 host. Historical delivery-plan archive; migration policy superseded below.

Second draft in the #2117 canonical-identity review stack. Base: impl/2117-engine-identity. This diff contains only provider/openai, provider/anthropic, and provider/openaichat (56 files). The three modules release independently after the engine prerequisite.

  • Move request/stream conversion to provider-facing LLM DTOs, keeping provider pairing/item IDs distinct from engine identity.
  • Keep opaque Responses reasoning/item replay and Anthropic signed/redacted thinking adapter-owned; preserve current fidelity, not a fuller replay redesign.
  • Retain fragmented/index-based Chat Completions assembly, missing/duplicate-key admission semantics and fail-closed truncated streams.
  • Add narrow real-engine + real-provider + executable-tool regressions across three local HTTP/SSE requests. Two completed invocations reuse one provider key; execution/events/history use distinct canonical IDs and the third request retains each occurrence's exact provider pairing/private replay.

No server, TUI, deployment, live providers, production credentials, new dependencies or pin changes.

API compatibility

Consumes the already-approved breaking pre-v1 engine minor: provider-facing DTOs/replay replace domain reasoning/item fields and chunks. This is not additive-only and not a /v2 Go import migration. Updated provider source requires a real released engine version before standalone qualification/publication; versions are deliberately not fabricated here.

No Harness protobuf/HTTP/advertised-major change in this diff. The final coordinated host/client PR exposes approved Harness API major 2. Provider-private replay stays private. Stored session/event formats belong to the engine/persistence follow-ons; providers do not create a separate replay store. Existing replay fidelity and stateless full-history behavior remain the contract.

Verification

Go 1.27.2, Linux/amd64, 2026-10-18. Fresh normal and race, -count=1, passed:

GOWORK=<assembled-checkout>/go.work go test ./provider/openai/... ./provider/anthropic/... ./provider/openaichat/... -run 'Test(Responses|Messages|Chat)EngineCanonicalIdentityRoundTrip$' -count=1
GOWORK=<assembled-checkout>/go.work go test -race ./provider/openai/... ./provider/anthropic/... ./provider/openaichat/... -run 'Test(Responses|Messages|Chat)EngineCanonicalIdentityRoundTrip$' -count=1
  • Responses: distinct provider item IDs, encrypted reasoning, phase and repeated pairing key.
  • Anthropic: occurrence-specific signed/redacted thinking and repeated tool-use token.
  • Chat: fragmented arguments, repeated provider key and no fabricated opaque replay.
  • All: tool execution, events and canonical history/results agree. Returning the first invocation ID for the second result made all three tests fail; mutation reverted.

The new tests received four-axis incremental review; a test-adequacy advisory about event content was fixed and rechecked. No remaining findings. No production source changed relative to the previously reviewed assembled implementation.

Historical complete assembled qualification passed on 2026-10-18, Go 1.27.2, Linux/amd64, with no uncommitted source changes:

  • task --concurrency 3 test:workspace:race — all 12 modules plus offline microvm_dev proof.
  • task lint — all module checks, 0 issues.
  • task docs, task site:build, offline go run ./cmd/mecademo (tool/result, ask/approval/result), and task test:module-metadata; generated/tracked files unchanged.

Workspace selection uses the assembled checkout's absolute go.work and existing unchanged dependency metadata. The tested tree 3763ffe56e1ba21bab363d3ad416c5e4a50077d7 equals public host-stack head da27200c3, with the provider/engine source at that historical checkpoint (not the latest #2404 engine). Workspace evidence is not standalone/publication evidence.

Publication and expected red CI

This draft intentionally retains existing published engine pins until an authorized engine release exists. GOWORK=off tests cannot compile new provider APIs against those old pins. Passing workspace tests proves current engine/provider composition, not published dependency closure. Root consumer compilation also remains broken until the final host source/pins land. Do not waive unrelated failures or disable checks.

Before merging/publishing: human-reviewed engine merge → separately authorized main-reachable engine tag → update all three providers to that real engine version → tidy only the affected modules and run standalone build/test/race/publication checks → publish each changed provider module → update final host/integration pins. No permanent replacements or PR-head pseudo-versions. Intermediate required-check handling still requires a separate explicit maintainer decision.

Draft for source review, not merge/release-ready. The review stack's adapter PR follows this branch for readable diffs, but adapters do not depend on providers. No release/deployment/merge performed; no new TUI presentation work.

@jbeda

jbeda commented Oct 18, 2026

Copy link
Copy Markdown
Contributor Author

Approved engine-first delivery plan

Archived: 2026-10-18. Applies to review stack #2404 → #2405 → #2406 → #2407; exact branch/head/baseline identities below. Local plan paths are provenance only, not remote links. This is the full selected split/qualification plan; verification tasks below describe intended checks, not implicit pass claims. Actual results and remaining gates are recorded separately in the PR bodies.

Engine-first identity delivery

Slug: 2117-engine-identity
Plan path: .scratch/2117-engine-split/plan.md
Original implementation specification: .scratch/2117-tool-call-identity-plan.md (already implemented and reviewed; not reimplementation scope).

Immutable delivery baseline and branches

Base branch: main; recorded start commit f9a3297. Remote main observed at c4a043f (#2402, TUI/docs only); no automatic rebase. Host requires current-main merge qualification before merge.

Review branch Base branch Source head
impl/2117-engine-identity main 5d830f8
impl/2117-identity-providers impl/2117-engine-identity c669ec8
impl/2117-identity-adapters impl/2117-identity-providers 20f60fd
impl/2117-identity-host impl/2117-identity-adapters da27200

These bases organize review, not new module dependencies: providers and adapters both depend directly on engine. The host source tree exactly equals the retained assembled qualification candidate (tree 3763ffe56e1ba21bab363d3ad416c5e4a50077d7). The root checkout remains on the engine review branch; assembled/preparation worktrees stay under .scratch/2117-engine-split/.

Approved outcome and authority

Joe approved on 2026-10-18: prepare, commit, push and open four draft PRs: engine; changed providers; persistence adapters; host/contracts/required client integration. Humans alone merge. No tags/releases, deployment, invented dependency pins, CI bypasses or new TUI presentation work. Preserve the assembled candidate in a local worktree under .scratch and keep the engine review branch at the main checkout root.

Source boundary and contracts

  • Engine PR: engine/** and the necessary .golangci.yml depguard allowance, including engine-owned tests, migration fixtures, generated API snapshots, changelog and compatibility documentation.
  • Provider PR: existing changes under provider/openai, provider/anthropic and provider/openaichat, plus narrow offline real-engine/provider HTTP composition regressions. These modules release independently.
  • Persistence PR: adapters/**, including its changelog and driver format/skew coverage.
  • Host PR: remaining existing coordinated source, generated wire/SDK/Studio contracts, required client propagation and owning documentation. No new presentation work and no changes to mecatl-3.
  • Engine-minted canonical ToolCallID is distinct from provider pairing/item IDs; whole-batch validation before effects; copied history preserves occurrence IDs. Session owns opaque replay and bindings. Ambiguous legacy state is preserved but inspection-only before authority consumption/execution/learning.
  • Preserve existing provider fidelity, not a fuller replay redesign. Preserve allow/deny/ask, isolation, scrubbing and argument ownership behavior.

API compatibility

This is the already-approved breaking pre-v1 Go API minor, not additive-only and not /v2 import migration. Engine APIs change invocation identity, provider-neutral DTOs and private replay ownership. The engine PR does not itself change protobuf fields/server major, but carries engine-originated observable semantics that the final host PR exposes as approved Harness API major 2. Public opaque archive reasoning fields are removed/reserved in that final coordinated host/client change; readable reasoning/text/media/phase remain. Snapshot/event writers advance to sessnap-json/2 and eventlog-json/2; upgraded readers accept /1 and /2. Drain old readers/writers, upgrade both decoding drivers, then harness/clients; no mixed writers or downgrade after new writes. Exact module release versions remain unselected and publication requires separate authority.

Tasks and evidence

  1. Preserve assembled checkpoint; extract engine-only source in the root checkout and document exact base/HEAD identities.
  2. Verify existing engine standalone API/layering/migration tests. Capture engine-only root/provider compilation failures rather than guessing expected failures. Do not suppress unrelated defects.
  3. Reuse provider HTTP/SSE fixtures in one narrow real engine + real adapter + trivial tool test per changed provider. Assert canonical call/result ownership and next-request provider pairing/replay including completed provider-key reuse. No server, UI, deployment or paid/live calls. Keep engine dependencies inward.
  4. Run focused normal/race composition checks and assembled workspace/race, lint, docs and offline demo at the owning candidate; preserve original historical reviews and perform incremental review of new tests and split boundaries, not another full historical panel.
  5. Prepare stacked draft branches in dependency order, with exact source boundaries and matching assembled engine tree. Preserve a fully assembled stack tip. No fabricated PR-head module versions or permanent replacements.
  6. Publish four draft PRs with API compatibility statements, exact evidence, expected intermediate CI failures, dependency/release checklist and plan archive comments. Read back published bodies/comments and record their URLs/heads.

Merge/release readiness

Reviewable source is not merge/release-ready. Before the engine merge, dependent source must be reviewable, assembled behavior qualified and release/pin order explicit. Real engine publication precedes provider/adapter pins and independent releases, then final host/integration pins and standalone/publication qualification. Existing required checks are not waived: Joe accepted review drafts with accounted-for consumer failures; any intermediate merge still needs a separate explicit maintainer decision. Release latency cannot be eliminated. Browser/image qualification remains unclaimed unless actually completed; final host draft must disclose inherited limitations.

Verification ownership

The directing agent owns split, integration, PR publication and evidence. A focused writable worker adds provider composition tests only in the retained assembled worktree and runs narrow checks. New tests get independent incremental panel review. Full Go gates use 1800-second outer limits; other full gates 600 seconds. Heavy suites run serially. No HANDOFF update.

jbeda and others added 3 commits October 29, 2026 09:59
Migrate Responses, Anthropic and Chat Completions to provider-neutral LLM DTOs and qualify each real adapter through the engine with offline HTTP fixtures.

Co-authored-by: Mecatl <noreply@mecatl.dev>
Synchronize the reviewed engine follow-up without merging unrelated host-main source. Exercise exact signed and redacted replay, reused provider pairing IDs, and cache marker placement through real HTTP requests.

Co-authored-by: Mecatl <noreply@mecatl.dev>
Preserve the reviewed replay assertions when restacking onto the engine branch and its newer main baseline. Use the existing role-union content helper.

Co-authored-by: Mecatl <noreply@mecatl.dev>
@jbeda
jbeda force-pushed the impl/2117-identity-providers branch from 6bdb6a2 to 3e4541f Compare October 29, 2026 16:59
jbeda and others added 2 commits October 29, 2026 10:29
Fail closed when Messages SSE ends before message_stop, preserving cancellation and original transport failures. Add real HTTP and engine regressions proving tentative tool calls cannot execute or enter accepted history, and correct the text fixture's terminal event framing.

Co-authored-by: Mecatl <noreply@mecatl.dev>
Clarify replay envelopes, legacy decoding, phase metadata, and buffering ownership without changing runtime behavior. Exercise Responses wire IDs through parser and engine admission, and Chat completion through real HTTP/SDK streams.

Co-authored-by: Mecatl <noreply@mecatl.dev>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant