Skip to content

fix(harness): keep compacted prompt ending on a user turn#81

Open
adolfo-tamayo wants to merge 1 commit into
vercel:mainfrom
adolfo-tamayo:fix/compaction-trailing-assistant
Open

fix(harness): keep compacted prompt ending on a user turn#81
adolfo-tamayo wants to merge 1 commit into
vercel:mainfrom
adolfo-tamayo:fix/compaction-trailing-assistant

Conversation

@adolfo-tamayo

Copy link
Copy Markdown

Fixes #80.

Problem

After compaction, the next model call can fail terminally on Anthropic models with "This model does not support assistant message prefill. The conversation must end with a user message."

The existing trailing-assistant guard only fires when the kept recent window trails with an assistant message (recent.length > 0 && recent.at(-1)?.role === "assistant"). It never covers the case where recent is empty — a single oversized tool-result turn, or keep decaying to 0 — so the summary's own assistant block becomes the final message, the guard is skipped, and Anthropic rejects the prompt. The turn cannot recover, since each retry re-compacts to the same assistant-terminal shape.

Fix

Move the guard off recent and onto the fully-assembled compacted array: append the synthetic "Continue." user message whenever the final message is an assistant turn, however it got there.

const compacted: ModelMessage[] = [
  { content: "Summary of our conversation so far:", role: "user" },
  { content: result.text, role: "assistant" },
  ...recent,
];

if (compacted.at(-1)?.role === "assistant") {
  compacted.push({ role: "user", content: "Continue." });
}

Tests

  • Adds a regression test: recentWindowSize: 0 forces an empty recent window, and the compacted result must still end on a user turn.
  • Updates the existing folds oversized recent tool results into the summary... test, which had asserted the buggy assistant-terminal output; it now expects the trailing "Continue." user turn.

pnpm run test:unit -- compaction → all green (26 passing).

@vercel

vercel Bot commented Jun 18, 2026

Copy link
Copy Markdown

@adolfo-tamayo is attempting to deploy a commit to the Vercel Team on Vercel.

A member of the Team first needs to authorize it.

@AndrewBarba

AndrewBarba commented Jun 18, 2026

Copy link
Copy Markdown
Collaborator

Thanks this looks great. We're finishing some setup on our side to be able to accept contributions. Will get this merged in the next day or two

@adolfo-tamayo adolfo-tamayo force-pushed the fix/compaction-trailing-assistant branch from 0766f20 to 6cb419d Compare June 19, 2026 08:27
The trailing-assistant guard was gated on `recent.length > 0`, so it
only fired when the kept recent window trailed with an assistant
message. When `recent` is empty — a single oversized turn, or `keep`
decaying to 0 — the assembled prompt ends on the summary's assistant
block. Providers without assistant-prefill support (e.g. Anthropic)
reject a prompt that doesn't end on a user turn, so the turn fails
immediately after compaction.

Move the guard onto the fully-assembled `compacted` array: append the
synthetic "Continue." user message whenever the final message is an
assistant turn, regardless of how it got there.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Adolfo Tamayo <atamayobr@gmail.com>
@adolfo-tamayo adolfo-tamayo force-pushed the fix/compaction-trailing-assistant branch from 6cb419d to db09f70 Compare June 19, 2026 08:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Compaction can emit a prompt ending on an assistant turn → Anthropic rejects the next model call

2 participants