Skip to content

docs(drafts): design committed session history and execution kernel - #2179

Merged
jbeda merged 20 commits into
mainfrom
design/committed-session-history
Oct 30, 2026
Merged

jbeda merged 20 commits into
mainfrom
design/committed-session-history

Conversation

@jbeda

@jbeda jbeda commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Overview

Design committed session history and a focused execution kernel for server-owned sessions that clients can leave, reload, and share without losing the facts needed to understand or safely continue the work.

Today, continuation snapshots, optional activity history, and live streams have different guarantees. The proposal makes private committed records authoritative, with continuation state, retained history, and authorized client views derived from accepted facts. Required commits gate execution so recovery can distinguish established outcomes from work whose effects are unknown without automatically repeating it.

This is a documentation-only discussion draft. It contains no kernel implementation, production activation, or approval to cut over. The first-slice implementation plan is shared in a PR comment, rather than checked in as a permanent backlog.

Review guide

For the proposed kernel boundaries and near-term work, start with the execution-kernel note and the first-slice plan. The tracked ownership note covers stable snapshots and detached data; the components note distinguishes capability extensions from durable-record handlers and discusses learning and compaction boundaries. Concrete proposed Go paths and the task breakdown live in the plan comment.

For the wider session design, read the overview, then choose a review packet. There is no required six-packet reading sequence.

Packet Central review question
Execution and recovery What must commit before work advances, and what can recovery honestly establish?
Cancellation and results How can execution stop while produced output still awaits settlement or release?
Authorization and hooks Which exact action is authorized, and when can hook work be reused or retried?
Model continuation What must survive to reconstruct the effective model request without incidental drift?
Commands and lifecycle How do shared controls, bounded retries, and catalog decisions remain unambiguous?
Storage and observation What does acceptance mean, and how does a client load a complete view and follow it?

The reading map links the shared vocabulary, architecture and contract references, YAML-backed domain models, and optional prior-art research. Packet boundaries are review topics, not code packages or implementation phases. This continues the design independently of the earlier implementation attempt in #2078.

Kernel and ownership direction

The kernel is coupled execution, coordination, and session-state machinery. It is not a separate service, OS sandbox, or custom-coordinator plugin framework. The proposed package boundaries give each responsibility a narrow contract and private representation:

  • Loop: drives model turns and tool execution, prepares transitions, waits at commit gates, and rechecks admission.
  • Coordinator: orders transitions and live controls, arranges acceptance, resolves uncertainty, and manages session obligations beyond any one run.
  • Session: a new minimal aggregate that interprets accepted records, enforces legal transitions and pairing, and reconstructs accepted state. It does not wrap the existing broad session.Session aggregate.
  • History: the opinionated record and persistence contract, including identity, positions, expected-head checks, append acceptance, and resolution. Storage adapters implement the contract; they do not own session legality.

Use records for authoritative committed facts, distinct from the existing activity events. Check a transition against accepted state, establish acceptance, then reflect it in memory. Initially allow one unresolved append batch per history. Storage waits must not block prompt cancellation or prevent already-admitted work from returning. Concurrent read-only tools retain independent progress: a ready result can commit while a slow sibling is still executing, subject to all prerequisite gates and mutation barriers.

Detached invocation data and independently owned output prevent later mutation from changing the accepted action or result. Execution metadata stays separate from response content. Cancellation before final admission prevents invocation; cancellation afterward requests interruption and proves neither cessation nor absence of effects. Run termination does not end the kernel's responsibility to collect, settle, or explicitly abandon outstanding obligations, and abandonment does not prove resource release or append rejection.

Trusted capability drivers consume specific extension points. Delegation can own durable records and operation state; learning can consume eligible completion/evidence and memory/skill interfaces without becoming a second history authority. Broad private capability bundles and a universal extension interface are not the goal.

First executable slice

Start with an isolated, growing harness around the real new kernel. The early phases are an additive side experiment alongside the existing runtime, with no major production changes required. Production entry points keep using the current engine; the experiment can change direction without requiring project-wide adoption.

The plan proposes engine/kernel with public history/storage contracts, focused internal loop/coordinator/session packages, and cmd/mecakernel for executable composition. README and AGENTS guidance will distinguish this work from the current runtime. Tight package boundaries and mechanical dependency checks are intentional; internal file conventions alone are insufficient.

The planned sequence is:

  1. Establish record/storage contracts, the minimal aggregate, ownership guarantees, and package checks.
  2. Exercise a complete scripted model-to-tools-to-model cycle through the actual kernel.
  3. Connect a real LLM immediately afterward through the same loop, using explicitly enabled, bounded live execution and harmless fixture tools.
  4. Expand uncertainty, cancellation, session-lifetime cleanup, concurrent-read, and reconstruction tests.

Implementation uses an integrated main baseline after the canonical identity stack beginning with #2404. Reuse canonical identities and replay semantics, but copy small structures or algorithms where reuse would force a bad boundary. The experiment does not require production storage, server/client integration, full authorization/hooks/disclosure machinery, or capability extraction. In-memory acceptance tests prove protocol behavior, not physical durability or distributed fencing.

This is an early project. A later clean integration may deliberately break APIs or temporarily remove major functionality, including learning, without first implementing a replacement. That flexibility is not an instruction to disrupt the current runtime during the experiment or to compromise safety and truthful outcomes.

Broader design and later integration

The wider proposal covers shared API controls and retry receipts, catalogs and lifecycle authority, complete logical model-input capture and private provider replay, retained history through compaction, authorized reload/feed continuity, forks, deletion, and backend-enforced ownership fencing. Potentially sensitive unreleased results remain memory-only; loss is explicit and does not justify rerunning a tool. Existing guardrail decision semantics remain unchanged.

Production integration and cutover are separate decisions. Any runtime-selection mechanism belongs there, not in the initial harness. The proposed first cutover includes required commit gates, trustworthy reload, shared controls, registered in-process reconstruction for Subagent, Parallel, and Team, and explicit interrupted-run recovery. Legacy bytes remain untouched, with no migration, fallback, or mixed-version rollout.

Compatible-replica handoff that preserves the same active run and delegation tree is a later phase. Autonomous continuation, cross-run background work, shared catalogs, and search also remain later capabilities. Whether predecessor-result closure and successor admission must share one atomic commit remains open; planning readiness is scoped to the slice being built.

Compatibility and validation

This PR changes design documentation and its terminology rule, not shipped Go, wire, or storage behavior. The plan's proposed APIs and formats require their own compatibility assessment during implementation; justified breaking changes are acceptable but must identify affected callers and upgrade implications.

The first-slice plan received independent design critique and a targeted correction recheck with no remaining identified blockers. That is planning evidence, not runtime qualification. Earlier work recorded documentation and model lint/render checks; those checks were not rerun for this title/description update. No production test-suite, race-suite, live-provider, durability, or fencing pass is claimed by this PR description.

@JAORMX

JAORMX commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

I see where you're coming from and I like the direction as far as I can tell. The initial implementation did have some limitations and we are definitely starting to see the wrinkles. My main ask is that these constructs stay behind pluggable storage drivers, with a portable history format so we can move sessions between deployment types without changing their semantics.

In regards to these being breaking changes, I'm fine with that. We're in alpha stage, so if there's a time to break things, it's now.

I did take a look at what other harnesses do. OpenCode and OpenHands provide useful implementation references. In OpenCode, durable events and registered projections are written in the same database transaction, then readers are notified after commit. Readers catch up from persisted sequence numbers. A notification tells you to check storage. OpenHands V1 separates streaming progress from persisted, sequenced events. Its reconnect path subscribes, buffers live events, replays through a captured boundary, and deduplicates the buffer by sequence.

I mentioned ActiveGraph the other day and it seems to me like it's a good reference for commit-before-projection and provenance, but as you identified earlier, it's not a great fit for all of our needs. Its event log is authoritative and ordinary reconstruction does not rerun behaviors. Both are worth preserving here. Its operator documentation explicitly excludes a distributed runtime, though, and expected-head checks are not the same contract as checking the head, ownership epoch, and unexpired claim atomically.

If we were to take some learnings from this, I'd borrow its causal references and inspection ideas without bringing in its graph ontology, which is consistent with the draft's stated scope. Run IDs, call IDs, parent/child relationships, and fork positions should give us useful answers to “where did this come from?” without another domain model for beliefs and reactive edges.

@jbeda

jbeda commented Oct 8, 2026

Copy link
Copy Markdown
Contributor Author

Before we commit to anything, I'll do a survey of prior art for sure.

If we go this direction, there will be some big refactors and it'll be painful. I do plan to keep pluggable implementations and, I think, this points the way for a more comprehensive idea of an extension to the main engine/loop in a way we don't have now.

Just like CRDs can extend k8s, extensions here should be able to write events and manage long running tasks, etc. I see some parallels there.

The first test would be to move subagents out of the core and make the recursive usage of the engine go through an extension. That feels like a really big test of the extension points.

I'm not happy with the doc as it is so will keep working on it and will probably write an implementation plan too that breaks down the modules and interfaces and starts to map out how we get there.

@jbeda

jbeda commented Oct 12, 2026

Copy link
Copy Markdown
Contributor Author

@JAORMX I did a prior-art pass and added a companion doc to this PR. There are useful ideas to draw from the other harnesses, but nothing that fundamentally changes the direction.

I didn’t find a complete equivalent to the graceful server-side handoff during upgrade that we’re aiming toward. OpenHands is the closest operational reference, but the implementations surveyed don’t establish the whole coordinated drain, durable park, and fenced takeover flow. That remains a later phase of our proposal, rather than part of the first cut.

There are useful precedents for canonical event histories and for extension-owned persisted records, though those capabilities differ across systems. In particular, storing custom events doesn’t necessarily mean their continuation state can be reconstructed or safely transferred to another server.

jbeda and others added 15 commits October 27, 2026 11:46
Capture the session-history design independently of the earlier implementation work, with a focused Modelith companion and shared-client steer semantics.

Co-authored-by: Mecatl <noreply@mecatl.dev>
Use sentence-per-line formatting, split dense paragraphs, and highlight domain concepts without changing the design semantics.

Co-authored-by: Mecatl <noreply@mecatl.dev>
Define run lifecycle, vocabulary, derived views, and component responsibilities against current code. Preserve the discovery-only prior-art shortlist while deeper research waits on search credentials.

Co-authored-by: Mecatl <noreply@mecatl.dev>
Co-authored-by: Mecatl <noreply@mecatl.dev>
Co-authored-by: Mecatl <noreply@mecatl.dev>
Capture short-PR staging, cancellation and memory-only result settlement, bounded optional request receipts, and single-provider session catalogs. Split the catalog and command protocol model from session history, document commit costs, and extend the prior-art comparisons.

Co-authored-by: Mecatl <noreply@mecatl.dev>
Define lifecycle phases, commit gates, recovery evidence, and cancellation transitions. Add durable-execution research and cache-stable reload requirements while preserving the narrow in-harness design scope.

Co-authored-by: Mecatl <noreply@mecatl.dev>
Co-authored-by: Mecatl <noreply@mecatl.dev>
Add a short overview, shared vocabulary, and six independently reviewable topics. Retain detailed contracts as lookup references and preserve open design decisions.

Co-authored-by: Mecatl <noreply@mecatl.dev>
Explain safe pause points and same-run continuation for compatible-replica maintenance. Distinguish automatic client reconnection from transport migration, preserve crash and first-cutover limits, and add the later handoff acceptance scenario.

Co-authored-by: Mecatl <noreply@mecatl.dev>
Clarify execution, replay, and command terminology across the session-history design and regenerated domain model. Record the naming convention in the shared vocabulary and a narrowly scoped project rule.

Co-authored-by: Mecatl <noreply@mecatl.dev>
Explain execution checks, tool-specific adapters, output disclosure, and subsequent model input in plain language before the detailed authorization rules.

Co-authored-by: Mecatl <noreply@mecatl.dev>
Explain actors, problems, and examples before detailed constraints. Preserve existing requirements, qualifications, links, diagrams, and open decisions in the review packets.

Co-authored-by: Mecatl <noreply@mecatl.dev>
@jbeda
jbeda force-pushed the design/committed-session-history branch from 5eae6b4 to ce95d1f Compare October 27, 2026 18:53
jbeda and others added 5 commits October 29, 2026 09:08
Add implementation considerations for stable snapshots, explicit cloning and borrowing contracts, and candidate enforceable representation boundaries without changing the core design.

Co-authored-by: Mecatl <noreply@mecatl.dev>
Capture extraction considerations from identity work while preserving execution authority and the existing committed-history design. Treat storage observations as diagnostic input rather than a canonical decomposition.

Co-authored-by: Mecatl <noreply@mecatl.dev>
Describe delegation and learning as composed capabilities using different integration seams, with optional agent tools and client commands. Preserve the narrower durable-record protocol and existing delivery scope.

Co-authored-by: Mecatl <noreply@mecatl.dev>
Capture source-based capability extraction candidates, kernel-retained invariants, and staged preparation before storage integration. Preserve existing cutover scope and separate Go, stored-data, and wire compatibility assessments.

Co-authored-by: Mecatl <noreply@mecatl.dev>
Record coupled loop/coordinator/session vocabulary, stable invocation ownership, admission and cancellation semantics, and serialized history acceptance. Define the isolated first kernel harness while preserving concurrent read-only tools and later runtime integration.

Co-authored-by: Mecatl <noreply@mecatl.dev>
@jbeda

jbeda commented Oct 30, 2026

Copy link
Copy Markdown
Contributor Author

First committed-history kernel slice

Slug: session-history-kernel
Implementation branch: impl/session-history-kernel
Base branch: main, after the identity stack is merged and its consumers are integrated

Planning proposal only; no implementation or cutover is authorized by this document. The plan is shared as a conversation comment on design PR #2179 at Joe's request, not checked into the repository. That PR remains documentation-only. Implementation starts only on a separate request. Joe confirmed the package map, first-slice scope and session-lifetime ownership on 2026-10-29. Independent critique and targeted correction recheck completed with no remaining identified blockers; integrated-baseline prerequisites still apply.

Goal and decisions

Build the actual new kernel behind a small composition API. Demonstrate a complete model → tools → model cycle in an isolated, growing harness; connect a real LLM immediately after the first scripted cycle works, before completing the broader fault matrix.

The early phases are an additive, isolated experiment alongside the current runtime. Production entry points keep using the existing engine. Shared changes should be small and justified: build/architecture/API checks and genuinely useful primitives, not feature extraction or runtime activation. We can change direction or discard the experiment without requiring project-wide adoption. Production cutover is a later explicit decision; permission for disruptive changes is not an instruction to disrupt the existing project during the experiment.

Joe's architectural priority is tight, focused packages with compiler-enforced representation boundaries. Do not rely on file conventions inside a large package. Use a new minimal session aggregate, not an embedded/wrapped legacy session.Session. The project is early: compatibility is an assessed cost, not a veto on a cleaner design. Justified source/API changes and disruptive simplifications are allowed in the proposed approach; identify affected consumers and coordinate them rather than building legacy compatibility scaffolding. Copy small structures or algorithms when reuse would force inappropriate coupling. Preserve canonical identity semantics; copying is not permission to create a competing invocation identity/replay scheme.

Feature continuity is also not a constraint. Joe explicitly accepts temporarily breaking or removing major functionality, including learning, with only a coarse direction to rebuild it as an extension. No complete replacement design, bridge implementation, dual-runtime feature parity, or restoration date is required before making a clean boundary. Do not bend the kernel to keep every legacy capability working. This is permission to simplify where the slice needs it, not a task to proactively dismantle unrelated features. Identify deliberately unavailable functionality plainly; missing functionality must not masquerade as successful execution. Safety, truthful outcomes, accepted-fact preservation and the selected slice's contracts remain requirements.

Settled scope:

  • Accepted/rejected/unresolved commitment gates model requests and tool invocations.
  • Cancellation is ordered locally against final admission, remains responsive during storage waits, and does not manufacture nonexecution evidence.
  • Independently owned input/output, distinct safe execution metadata and response content.
  • Accepted records reconstruct equivalent accepted state without rerunning effects.
  • Concurrent reads and mutation/parent-mutation barriers; ready results can commit before slow sibling reads finish.
  • Early opt-in real-provider demonstration through exactly the same loop as scripted acceptance.

Excluded: production storage or fencing, server/client wiring or protocol changes, runtime-selection flags, legacy migration, full capability extraction, extension loader, delegation/learning/compaction implementation, production authorization/hooks/disclosure workflows, remote jobs, and same-run crash resume. The fixture has only explicit harmless tools and fixed inputs; omitted production controls are not implemented or qualified. Memory acceptance proves protocol behavior, not physical durability.

Grounding and baseline

  • Owning design: docs/drafts/session-history/implementation-kernel.md, implementation-ownership.md, vocabulary.md; execution gates in session-commit-history.md:1402–1427 and batching at :1489–1519.
  • docs(drafts): design committed session history and execution kernel #2179 is open/draft at c877499 when checked. No implementation belongs in that PR; humans merge it after fresh review/CI.
  • feat(engine): establish canonical tool-call identity and private replay #2404 was inspected at actual API head 3aa9bf4, not a newer SHA mentioned in its description. Reinspect the merged source before implementation. Its dependent provider/persistence/host work must be integrated, not merely the engine source merged with broken callers.
  • Identity foundation supplies canonical session.ToolCallID, occurrence-bound MessageReplay/ToolCallBinding, deep ownership helpers, provider-neutral port.LLMMessage/tool DTOs, and port.ProjectMessages. Run IDs remain strings; turn indices are not durable commit positions. Do not invent a replacement run namespace or treat event sequence as a history head.
  • Legacy Session mixes many concerns (engine/session/session.go:337) and mutates before persistence. Current EventLog.Append has neither the deduplication nor resolution contract required here (engine/port/eventlog.go:35); snapshot stores are also not append authority.
  • Reuse tool.Tool, Environment, and dispatch classification (engine/tool/tool.go:54, engine/agent/dispatch.go:23–83). Do not reuse the existing dispatch result-drain barrier: it waits for slow siblings.
  • mockllm provides scripted provider responses and request observation. cmd/mecademo/main.go illustrates explicit live provider selection, but its scenario drives the old engine and is not reused.

Implementation prerequisite: choose a main-reachable integrated baseline after the identity stack, recheck these seams, and capture the implementation branch's actual start commit. If APIs materially differ, update this plan rather than silently substituting a different architecture. No pins to invented versions or copied identity-branch files.

Package map and ownership

All new paths below are proposed. The kernel remains in the existing engine Go module.

Call authoritative committed facts records, not events, to distinguish them from the existing activity events being replaced. history defines the opinionated persistence contract, not its backend implementation; session interprets accepted records together and enforces domain transitions. Storage checks whether a batch can extend a head; session checks whether its transitions are legal; coordinator establishes acceptance before reflecting those facts.

Package/path Owns Must not own
engine/kernel Thin construction/run/control facade, opaque handle, public input/outcome views Execution algorithms, mutable state, replaceable-coordinator API
engine/kernel/history Detached record values, transition/batch identity and head vocabulary, append/resolve/read contracts Execution, aggregate mutation, physical storage, provider SDKs
engine/kernel/internal/session Minimal accepted-state aggregate; record legality/application/reconstruction; stable outward snapshots Legacy aggregate, storage I/O, live cancellation, driver state, extension policy
engine/kernel/internal/coordinator One-history append ordering/resolution, accepted-state publication, local admission/cancellation ordering, bounded ready facts, session-lifetime obligation settlement beyond run termination Model/provider behavior, tool implementations, authorization rubrics
engine/kernel/internal/loop Model stream assembly/projection, gate waits, short-tool dispatch, result preparation Direct history writes, aggregate representation, extension implementations
cmd/mecakernel Isolated executable composition, volatile store and harmless fixture tools, explicit scripted/live provider selection Production app factory, server/client activation, unrestricted operator workspace tools

Import direction (additional direct inward imports are allowed only where used):

kernel facade -> loop -> coordinator -> session -> history
                   \--------> history       \------> history
cmd/mecakernel -> kernel + kernel/history + existing provider/reference adapters

history may reuse canonical legacy session value types, never *session.Session or legacy snapshot/event adapters. The new session package depends only inward on history and necessary canonical values. Existing provider/tool DTOs are consumed at the loop boundary. History owns its exact effective-model-input description without importing port, tool, or prompt-building behavior; copy small data shapes and convert explicitly where necessary. Keep canonical tool identities/replay values rather than parallel identity types. Define new public storage contracts in history, not aliases to hidden internal contracts.

Drivers remain existing externally composed model/tool implementations. The first slice installs no universal driver framework, process registry, or private capability bundle. Tool execution receives detached action data plus separately live context/environment capabilities. If a current tool contract cannot express an observation, kernel-owned metadata supplements it without changing every existing tool signature.

Use a narrow session-owned preparation/application contract: validate against accepted state, establish commitment, then reflect accepted facts. Do not mutate published accepted state speculatively, clone the entire legacy aggregate per append, or introduce generic transactions/rollback. Reuse/extract an appropriate canonical prefix validator where that is clean. If doing so would force legacy aggregate coupling or an awkward public API, copy the small algorithm into the new owner and add shared behavioral/parity cases for canonical occurrence/replay invariants. Preserve one canonical identity meaning, not necessarily one implementation of every helper. Classify source API changes for any shared seam separately; do not add compatibility wrappers merely to avoid a justified break.

Mechanical boundaries and coexistence guidance

  • Add strict depguard allowlists and architecture direction checks for every new package. Negative checks must reject reverse dependencies and forbidden legacy aggregate use, not just detect cycles. Because canonical value imports also expose the legacy aggregate, add focused symbol-resolution checks for forbidden aggregate references, including type aliases; an import-only check is insufficient. No new kernel production code imports agent, reference adapters, host internals, SDKs, gRPC, or OS APIs.
  • Add kernel and kernel/history to arch.CorePackages, API baselines, compatibility documentation and Added changelog. Separate internal layering coverage from public API coverage: assert exact package-set coverage, not only counts, while retaining exact equality between the public API package set and arch.CorePackages. Internal exports do not become public compatibility promises. Do not expose internals simply to obtain architecture checks.
  • Add focused engine/kernel/README.md and AGENTS.md, plus cmd/mecakernel/README.md and AGENTS.md. Use the repository's plural AGENTS.md convention. README explains status, ownership/import map, harness usage and explicit nonclaims. Scoped instructions prohibit legacy-runtime shortcuts, unrelated feature growth and production activation; distinguish the new harness from mecademo/agent.Engine. Keep guidance short and nonduplicative.
  • Put integration tests at engine/kernel (external-package tests through its facade). Keep controllable storage/fixtures test-local initially. Unit tests live with their owning internals. No separate simulation or permanent duplicate test kernel.

Material contracts

Committed records and accepted state. Start with the records required for run/input acceptance, effective model input, completed model response, effective tool intent, canonical tool result, cancellation/terminal settlement. Version the record vocabulary; no production serialization format is promised. Model request identity/turn anchor is scoped by existing session incarnation and run identity plus an explicit turn index; it is not provider identity or commit position. Transition IDs identify proposed facts; positions are assigned by acceptance. Reconstruction checks the whole ordered prefix and never invokes a provider, tool, hook, or adapter to recreate missing facts. Unknown/malformed records or broken pairing fail closed.

Capture the complete effective logical model input used by this slice before each provider call: system layers, actual projected messages and private replay, tool definitions, model selector and the continuation basis. Identity transformations have explicit no-rewrite evidence; omitted hooks are not fabricated transformation success. The submitted provider request must be detached from the committed input. Preserve argument byte representation, bindings, phase and opaque replay; do not round-trip raw arguments through map normalization or assume JSON serialization preserves bytes. A live smoke that loses required provider replay is a failed milestone.

Storage seam. Append carries history incarnation, expected accepted head, stable batch/transition identities and detached records. Acceptance returns authoritative assigned positions/head; rejection definitively means this attempt did not establish those facts; unresolved means acceptance is not known. Identity is scoped to the history incarnation and bound to the exact ordered batch contents and original expected-head basis. Repeating the same accepted batch returns its original acceptance even after the head advances: duplicate recognition precedes ordinary expected-head conflict handling. Reusing a batch or transition identity with different contents fails explicitly, never attaches to the old acceptance. Resolve/lookup addresses that exact batch identity and expected-head basis, returning accepted, definitively not accepted, or still unresolved. A missing lookup result alone is not proof of rejection while an append may still complete. Do not automatically resubmit under a new identity. Read returns an owned contiguous accepted prefix through a selected head for reconstruction. Backend durability/claim/fencing implementations remain deferred, not falsely represented by the volatile adapter.

Coordination. At most one unresolved batch per history. Verify a candidate against accepted state; append without holding a control lock; reflect accepted facts before the next append. Bounded ready facts may group only when no gate separates them. Already-admitted work can return while a write waits. Cancellation closes admission promptly in the same local ordering domain as final admission; it does not wait for append I/O or reserve another append slot. Late intent acceptance cannot undo earlier cancellation. A stalled history does not stall another history.

Execution and outcomes. Commit effective input before inference; completed model response before tool preparation; effective intent before final tool admission; canonical permitted results before next dependent model request. Canonical invocation IDs are minted once at accepted model-response conversion, not taken from provider tokens. Invalid/failed/cancelled model streams do not admit tentative tool calls. No implicit model/tool retry after ambiguous effects.

For each invocation, separate closed kernel observations (admitted, interruption requested, returned) from produced content. No arbitrary error strings become safe durable metadata; a context error proves neither nonexecution nor cessation. Handed-off input and returned output have independently owned nested storage. The fixture accepts only its explicitly supported processing path; unsupported authorization/disclosure continuations fail closed instead of being invoked as ordinary tools.

Cancellation before admission prevents invocation even if intent later resolves accepted. After admission, signal interruption and process returned evidence normally, including successful completion by a non-interruptible tool. Use an injected settlement deadline. A bounded local stop must distinguish pending commitment/unresolved external activity from a committed terminal outcome; do not publish fictitious closure. After the settlement boundary, fence late advancement by invocation/run identity. No successor run starts until required predecessor pairing/settlement is accepted. Reconstructing an active prefix establishes interrupted/unknown work, never automatic replay of effects; any recovery close-out is a new accepted fact, not invented historical data.

Session lifetime and obligations. The kernel owns the session lifetime, which is longer than any run. Its per-history handle retains coordination and outstanding obligations after run termination or a bounded wait returns. The coordinator owns append resolution and session-level settlement; the loop reports invocation returns without requiring a live model loop. A run ending does not dispose of its tool/environment resources or erase pending work.

Collect and settle obligations when evidence arrives, or explicitly mark them abandoned under their original identities with safe uncertainty evidence. Abandonment closes eligibility for stale run advancement; it is not proof that a tool stopped, a cleanup action succeeded, resources were released, or an append was rejected. Where abandonment is a canonical decision it must be accepted through the same commit path; unavailable storage leaves that decision pending, not confirmed. Late authoritative append acceptance is always reconciled/reflected even after run termination or an abandonment decision, without reopening execution. Late tool output cannot replace an already accepted canonical close-out.

The handle keeps live invocation references and a bounded collection path until they return, or reports still-outstanding activity if they cannot settle; no generic supervisor or driver lifecycle framework is required in this slice. Caller-supplied store/provider/environment resources must remain valid while obligations can still use them. Bounded shutdown stops admission and reports unsettled/abandoned obligations separately from actual quiescence. Safe resource teardown requires actual quiescence or substantiated release evidence; logical abandonment alone does not establish it. The command's volatile store makes process exit an explicitly non-durable boundary, not proof of settled external effects.

Read batching follows actual ReadOnly, DispatchSerial, and MutatesParent classification. Mutations do not overlap prior reads or other mutations, and later reads do not cross a mutation barrier. Unsupported authorization requesters are not treated as ordinary reads. Each ready result can commit independently of slow sibling execution; provider continuation still waits for all results it depends on. Commit order follows readiness, not original call-list order; identities preserve pairing. Derive any order-sensitive counters from a documented semantic ordering, not goroutine scheduling; legacy consecutive-failure policy is not part of this slice.

Dependency-ordered implementation tasks

These are small reviewable outcomes, not permission to implement or a requirement for one PR per package. Agree coherent PR boundaries when implementation begins; never merge nonbuilding consumers.

  1. Package/contract foundation and guardrails. Add public history values/storage contracts, minimal aggregate and owned snapshots, architecture/API guards and coexistence READMEs/instructions. Reuse identity foundation deliberately and add only the validator seam actually needed. Prove legal accepted-prefix reconstruction and nested ownership, including outstanding siblings and duplicate/mismatched results. Done when the new domain can reconstruct a small accepted history without old aggregate/adapters and forbidden dependency controls have negative coverage.
  2. Thin executable spine. Add coordinator and loop, facade, scripted provider integration and cmd/mecakernel offline scenario. Use a simple accepting volatile store and explicit fake tools; provide a valid isolated in-memory environment/ledger rather than ambient capabilities or invalid zero values. Prove full accepted model input → response → intent → tool → result → next model input → final response/terminal path, including request observation and basic replay/usage preservation. Add Taskfile build target/output in bin/; do not use bare root builds. Establish session-handle ownership beyond the run and focused cancellation/teardown checks that prevent an unsafe live milestone. Done when both facade integration test and offline executable drive the real kernel, not agent.Engine.
  3. Early real-provider milestone, immediately after task 2. Add explicit opt-in live selection using the existing provider adapter and normal secret handling, copying the small provider-selection pattern instead of creating a new registry. Root composition keeps SDKs out of engine dependencies. Same harmless fixture tools, bounded turns/output/deadline, no ambient project prompt or tools, no arbitrary file/shell execution. Start with one provider supported by the integrated baseline; additional providers are not prerequisites. Mock remains default. Report whether the model actually requested a tool and completed the second model request; refusal/no tool call is not a successful tool-cycle demonstration. Never run paid probes without explicit operator authorization; if authorization is unavailable, deliver the runnable path and mark live qualification pending. Do not delay this task behind the full fault matrix.
  4. Uncertainty and live control. Grow the storage double with deterministic append gates and lost-ack resolution; exercise coordinator rejection/unresolved states, both cancel/admission orderings, cancellation during storage waits and non-interruptible return. Test bounded settlement, stale returns and honest pending-commitment outcomes through the facade. Separately prove late append acceptance is reflected after run termination, late tool evidence is collected without resurrecting a closed run/result, abandonment is explicit and safe, and resources are not falsely released while still used. Done when dependent model/tool work never crosses an unaccepted gate, cancellation remains prompt, resolution uses the same identity and no external invocation repeats.
  5. Concurrent reads, reconstruction and integration qualification. Add channel-gated read/read → mutation → read tests, independent ready-result acceptance, blocked history versus independent history, pending-output bounds, ownership mutation regressions at every boundary, crash-after-acceptance-before-reflection and full accepted-history reconstruction equivalence. Reconstruction never invokes doubles for effects. Verify cancellation/terminal recovery and canonical pairing at incomplete prefixes. Finish focused review and integrated gates; update owning design note with the stable map only as separately agreed for docs(drafts): design committed session history and execution kernel #2179, not by adding implementation to it.

Observable acceptance and test schedules

Use channel/barrier handshakes and controlled clocks, not sleep-based timing claims. Exercise the real facade, coordinator and aggregate; storage doubles replace dependencies only.

  • For each model-input/response/intent/result gate, block or reject append and assert no prohibited dependent call. Inject accepted-with-lost-ack and later same-identity resolution; no duplicate transition, accounting, model request or tool invocation. Repeat an identical batch after the head advances and recover its original acceptance; conflicting identity/content reuse rejects without mutation.
  • Cancel while intent is unresolved, then confirm acceptance: zero invocation. Admit first, then cancel: signal observed, successful output from a non-interruptible tool is not rewritten as not-started. Deadline expiration remains truthful and rejects stale advancement.
  • Hold read A, complete read B: B's result is accepted while A is running. Hold B's append acknowledgement: at most one batch unresolved, cancellation remains live, A may return into a bounded buffer. Mutation begins only after preceding read execution and required gates settle; later reads wait.
  • Terminate a run with an outstanding append: the session handle remains responsible and reflects later authoritative acceptance exactly once. Separately terminate with a non-interruptible tool: late return is collected or explicitly abandoned but cannot reopen the run or replace its accepted result. Assert that logical abandonment is not reported as physical cessation or safe dependency disposal, and resource teardown occurs only after actual quiescence/release evidence.
  • Mutate caller/provider/tool source buffers, returned snapshot collections and storage-returned slices; previously accepted input/output and old snapshots do not change. Include nested replay and representation-sensitive argument bytes.
  • Reconstruct fresh state from copied accepted records, including a crash after acceptance before reflection. Compare canonical state/head/usage/pairing with live accepted state; observe zero model/tool calls during replay. Missing or invalid facts reject, not normalize into success.
  • Default executable and all tests are offline, test-owned and independent of HOME/XDG, learned skills, app composition or operator stores. Live opt-in is explicit, bounded, and excludes credential-bearing values from output/history. No request dump of private replay or raw SDK errors.

Compatibility and verification

API compatibility: the expected change is additive public engine Go packages, but additive-only is not a design constraint. Prefer a justified clean break over compatibility machinery. Regenerate API snapshots and classify additions/breaks under the current pre-v1 engine policy; identify and update affected callers together. Existing production behavior need not be redesigned for this isolated slice. If a retained provider/tool contract obstructs a necessary boundary, bring the specific change and consumer impact back for agreement rather than silently weakening the boundary or introducing an adapter framework. The new record protocol is not a legacy snapshot/event migration and promises no interchangeability. No remote API behavior, protobuf, feature identifier or harness major changes are expected in this slice; assess separately and obtain specific approval before a discovered wire break. Later production cutover remains a separate incompatible slice.

Focused iterations: cd engine && go test ./kernel/... (and targeted race runs); root go test ./cmd/mecakernel/...; engine architecture/API checks; offline executable. Use injected clock/diagnostics and explicit test-owned paths. Add kernel command build through Taskfile, keeping output in bin/.

Integrated-change checks owned by the implementing agent: run task test once, standalone engine, task lint, task test:race, offline existing go run ./cmd/mecademo (tool/result plus ask/approval/result), and the new offline kernel executable. These are evidence, not a hidden requirement to preserve every old feature. If approved simplification intentionally removes legacy behavior, update/remove its obsolete assertions and document the missing capability and coarse extension destination; do not build bridges or require a replacement solely to make the legacy demonstration pass. Report remaining failures by intentional removal versus unintended regression, and obtain an explicit maintainer decision for any unresolved required gate rather than call it a pass or silently waive it. The retained kernel contracts must have passing focused verification. Because root/provider/engine composition spans modules, follow docs/module-development.md, run task test:workspace:race, and retain standalone checks; workspace success is not dependency-publication evidence. Do not invent module pins or alter unrelated sums. Complete test suites get a 1800-second outer timeout; other full gates get 600 seconds.

Use tech-writer when authoring the implementation's READMEs and run task docs for tracked Markdown changes. Generated changes use owning tasks, never hand edits. This scratch-only planning session does not claim runtime or documentation gates ran.

Review completed implementation with panel-review against the recorded implementation start commit, not moving main. Incremental fixes use the last reviewed candidate; rebase requires separate authorization. This plan is shared on #2179 for design discussion, not as implementation in that PR. If an implementation PR is later requested, preserve its latest full publishable plan in one identifiable conversation comment with privacy/identity/readback checks. Keep comment URLs/IDs in ignored publication metadata rather than in the published plan body. Humans alone merge. No automatic commits, push, release, or live monitoring.

Open items and risks

  • Fresh-context general critique found no architectural blocker, but two important contract gaps: append idempotency/content-conflict handling and lifetime ownership after bounded run return. Both are now specified above; Joe confirmed that the session-managing kernel owns collection/abandonment and cleanup beyond run termination. The general reviewer rechecked the revised contracts and tests and found both findings resolved, with no remaining concrete blockers. Focused architecture review found no blocker in the supplied architecture excerpts; symbol-level legacy exclusions, exact layering/API coverage and deep ownership tests were incorporated. Reviewers inspected the current checkout, not future merged identity source; the correction recheck covered supplied revised sections, not the entire updated plan. Neither ran tests or live calls.
  • Exact private signatures and record encoding are implementation choices constrained by the contracts above, not permission to invent a framework.
  • Merged identity source may differ from the inspected draft. Confirm canonical prefix validation/projected-request fidelity and integrated provider availability before task 1.
  • A real-model smoke is deliberately early but nondeterministic and paid; absence of authorization or provider readiness is reported, never replaced with a claimed live pass.
  • Growing the new public surface creates Go compatibility obligations. Keep facade and storage types minimal; internal exports are layering contracts, not public API.
  • Full hooks/disclosure/authority, durable backend fencing, extension record protocols and production cutover remain future slices. Nothing in this first harness proves those features.

@jbeda jbeda changed the title docs(drafts): committed session history and shared interaction design docs(drafts): design committed session history and execution kernel Oct 30, 2026
@jbeda
jbeda marked this pull request as ready for review October 30, 2026 01:00
@jbeda
jbeda merged commit 37e6a94 into main Oct 30, 2026
69 checks passed
@jbeda
jbeda deleted the design/committed-session-history branch October 30, 2026 15:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants