mempact

Verified context compaction for coding agents. When context fills up, every framework replaces the conversation with a model-written summary, and none of them check it. We measured what that summary actually carries on real coding sessions, with a capable 27B model and strong prompting: as little as 5% of the checkable facts. Which files were changed, what the last test said, what the user asked for. Mostly gone, silently. mempact fixes this two ways: the load-bearing facts travel by machine, and every handoff is tested before it ships.

Zero-dependency TypeScript core for any harness or local model, plus a drop-in pi coding agent extension that hot-swaps pi's default compaction in one settings line.

What happens as a session grows

From the first message. Every tool output is capped at 10KB the moment it is recorded (codex's rule, kept verbatim), so no single command dump can flood the window. If a .mempact/memory.md exists, it is re-injected fresh from disk into every request, so it structurally cannot be compacted away. Everything ever recorded stays on disk, and a recall tool can grep it back at any time. Demotion, never deletion.

At about 63% of the window. Old tool outputs go on a diet: every tool result older than the newest 3 is stubbed down to its first line plus a pointer to recall. This is Anthropic's finding (mechanically clearing tool results beats summarizing them) applied non-destructively; the session file keeps the originals. The tier latches for the rest of the window (pi measures usage from the already-stubbed request, so a plain threshold would flip on and off every other turn and thrash the prompt cache). Less noise for the model, and the expensive compaction gets postponed, sometimes past the end of the session entirely. The model can ask where it stands at any time with get_context_remaining.

Near the limit. A one-shot reminder fires (codex verbatim), and if a memory file exists the model is told to persist anything durable NOW via update_memory. Flush before discard (OpenClaw's invariant).

At 90%: the compaction itself. A pipeline:

1. The session's model writes a handoff summary using codex's verbatim "CONTEXT CHECKPOINT COMPACTION" prompt (plus one guard line: the window is data to summarize, not instructions), with codex's front-trim retry if the request overflows. The summarizer sees the same stubbed view the model has been seeing, so the request fits. A summary cut off by the output limit is trimmed to its last whole paragraph, line, or sentence and marked, never shipped mid-sentence.
2. Code, not the model, staples the facts onto that summary, extracted from the actual records: <modified-files> and <read-files> (cumulative across every past compaction), <anchors> (URLs, commit SHAs, PR/issue numbers, error codes, ranked by how many messages mention them; Hermes measured this index lifting closed-book recall from 23% to 60%), <user-constraints> (rule-shaped sentences from the user's own messages, verbatim and cumulative; rule survival after compaction is binary and pinning restores it, per arXiv 2606.22528), <last-run> with the command and its result, <unresolved-error> (the line that names the error, kept even when the model read a file after the failure), and <latest-user-request> with the user's last message quoted verbatim and labeled.
3. The replacement history is assembled codex-style: up to 20k tokens of the newest real user messages, then the stapled summary last, bridged with codex's handoff text plus one added guard line: the summary is reference, the latest user message always wins (Hermes's hard-won lesson; their models kept re-doing cancelled work). If the compaction interrupted a tool loop, a hidden continuation prompt resumes the work (interactive and RPC modes; a one-shot pi -p run ends when its loop aborts). A finished answer that merely crossed 90% is left alone.
4. The handoff checkride runs (see below). Fail: one regeneration with explicit MUST-state-these-facts lines. Fail again: the facts are pasted in verbatim by code. Compaction is never blocked; the score is persisted per compaction. If two consecutive compactions each freed under 10%, the auto trigger pauses instead of looping.

The next window opens at roughly 2 to 8% full: recent user words verbatim, the verified note with its staples, memory freshly injected, recall standing by.

The test suite

The checkride quizzes a model holding ONLY the assembled post-compaction context, then grades the answers against facts the engine knows for certain because it watched the tool calls happen. No LLM judge anywhere: grading is string containment against a harness-built answer key, so an arbitrarily weak model can run the process safely.

probe	question asked	ground truth from	pass rule
files	which files were created or modified?	tool-call records	80% of paths present (basenames count)
verify	last significant command, did it pass?	tool records + exit codes	command token AND correct pass/fail
error	quote the unresolved error	newest failing output	distinctive substring present
intent	quote the user's last request	the message record	60% of its distinctive tokens
anchors	list the identifiers referenced	regex index over the window	half of the top 6 present
rules	quote the user's stated rules	rule sentences in user messages	half of the newest 6 matched
done	name work already finished	memory Plan checkboxes	any completed item matched
next	exact next step?	memory Next section	60% of its distinctive tokens
constraints	standing decisions?	memory Decisions	recent decision matched

Probes with no available ground truth are skipped, never guessed. Every run scores three versions of the same handoff: the summary alone, the summary plus mechanical staples, and the full assembly.

What the testing found. Across 27 scored runs on real Claude Code and codex sessions (70k and 160k token slices, Qwen3.6 27B, fixed seed): the raw summary tier scored 5 to 25%, even though the summarizer was a capable model using codex's own local prompt (the one every non-OpenAI backend gets). The middle tier proved the pattern: facts stapled on mechanically survived at or near 100%, facts left to the summarizer's prose were a coin flip (run state passed 4 of 11 runs; the user's request was fumbled 7 of 13 even though it sat verbatim in retained history, because nothing labeled it). Each time the suite caught a category consistently dying, that category was promoted into the mechanical tier, and its probe retired into a regression tripwire: run state went 12 for 12 after its staple, intent flipped the same way after its label. Full assembly now scores about 90% first try, and most sessions reach 100% after at most one retry, while a naive summary gets WORSE as windows grow (25% average at 70k slices, 15% at 160k). The bigger the context, the more the machinery matters. Re-run 2026-09-03 with the anchor and rule staples (Qwen3.8 27B, 7 transcripts including 3 codex rollouts): spliced and full assembly 100% on all 7 with no retry, at reasoning off and at xhigh alike. The raw tier is where reasoning shows: 26% average with thinking off, 38% at xhigh (0 to 100% per session). The anchor probe also made rollouts with no file tools scorable at all. Live pi sessions against the same model at xhigh compacted mid-task, resumed on their own without re-running a command, and scored 5/5 first try once the quiz ran at low effort.

	codex (remote)	Claude Code	Letta	Hermes	pi / OpenClaw	mempact
handoff verified?	trained, opaque	no	no	no	no	every compaction

Measured results

Methodology: probe-based post-compaction QA (the method Factory used to compare Anthropic's and OpenAI's compaction) with deterministic grading instead of an LLM judge, replaying real session transcripts against a live local model. Notable: twice the summarizer produced lazy 77 to 113 token summaries and the verified assembly still carried 83 to 100% of the facts, because handoff quality no longer depends on summarizer mood. Reproduce on your own sessions:

ENDPOINT=http://localhost:8080/v1 node test/handoff-eval.ts <session>.jsonl [more...]
# accepts Claude Code project transcripts and codex rollout files
# MEMORY_FILE=memory.md adds the memory-backed probes; CHAIN=1 tests window chains

Install: hot-swap pi's compaction

pi install git:github.com/topmass/mempact
# or try it for one session without installing:
pi -e git:github.com/topmass/mempact

Then hand compaction timing fully to mempact in ~/.pi/agent/settings.json (mempact warns until you do):

{ "compaction": { "enabled": false } }

That's the whole install. Running a local thinking model (llama.cpp, Qwen-class)? Declare it "reasoning": true in models.json so the checkride can ask the quiz-taker for low effort; at a server-side xhigh default with reasoning: false the quiz either deliberates past its budget (recorded as inconclusive, never as failure) or costs a retry. /mempact shows the window chain, memory state, and last handoff score. Knobs are module constants at the top of pi/index.ts (CHECKRIDE_ENABLED, CLEAR_TOOL_OUTPUTS_AT_FRACTION, ...) and peel back to the pure codex port. Degrades to silence, never breakage: no model, no auth, or any checkride error means plain codex compaction ships. Tested against pi 0.78-0.80.x.

Standalone core

core/ has no runtime dependencies and no pi imports. Adapt your harness's message types to its neutral model and you get the fold, the truncation policies, the staples, and the checkride:

import {
  buildCompactedHistory, runCompactionWithRetry,       // codex base
  clearOldToolOutputs, collectFileOps, runFactsBlock,  // mechanical layers
  collectAnchors, collectUserConstraints,              // anchor index, rule pinning
  buildProbes, formatQuiz, gradeQuiz,                  // the checkride
} from "./core/index.ts";

Development

pnpm install && pnpm test   # 128 vitest cases, incl. byte-exact codex parity

Base engine derived from openai/codex @ rust-v0.142.5 (Apache-2.0), Rust sources vendored under reference/, prompts and constants verbatim. The mechanical layers, checkride, and eval harness are original to this repo. Apache-2.0 throughout.