Skip to content

🔄 fix: resume Anthropic context-window stops - #547

Draft
upman wants to merge 3 commits into
mainfrom
rlazar/search-context-resume
Draft

upman wants to merge 3 commits into
mainfrom
rlazar/search-context-resume

Conversation

@upman

@upman upman commented Sep 16, 2026

Copy link
Copy Markdown
Collaborator

Summary

Resume once per agent/run after Anthropic's model_context_window_exceeded stop, preserving partial output and encrypted server-search results. Keep the raw streaming stop reason; pending client tools retain their normal route. A repeated stop sets the existing output_truncated halt reason.

context stop → retain history + resume prompt → model
                                               ├─ answer → finish
                                               └─ context stop → halt

Claude Code behavior

Local Claude Code 2.1.268 tests showed top-level context stops trigger a resume request with unchanged history, not compaction. Its nested WebSearch path instead returns links to the outer model. This follows the top-level continuation pattern, with a one-continuation safety bound and search history retained.

Validation

Check Result
Regression tests 292 passed, including streaming and tracing
Local tarball in LibreChat Normal response: 1 call; context stop: 2 calls, persisted answer and aggregate usage; repeated stop: bounded at 2
Gateway + Bedrock Replayed a real context-stopped search response; LibreChat continued with encrypted history and received a coherent cited answer without another search

The Bedrock continuation used a full-capacity model context: this verifies replay, not compaction of a still-full window. The tested LibreChat build does not yet map output_truncated to its persisted unfinished flag.

@upman upman changed the title fix: resume Anthropic context-window stops 🔄 fix: resume Anthropic context-window stops Sep 16, 2026
@upman

upman commented Sep 16, 2026

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 16, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-16T21:19:04.213554Z e9dc340 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 976536a381

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/graphs/Graph.ts
Comment thread src/graphs/Graph.ts
@upman

upman commented Sep 16, 2026

Copy link
Copy Markdown
Collaborator Author

@codex re-review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1a99832eff

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/langfuseTraceShaping.ts Outdated
Comment thread src/langfuseTraceShaping.ts Outdated
@upman

upman commented Sep 16, 2026

Copy link
Copy Markdown
Collaborator Author

@codex re-review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e9dc340af1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/graphs/Graph.ts
*/
outputTruncatedIncomplete = false;
/** One automatic context-stop continuation per agent per run bounds billable retries. */
private contextStopContinued = new Map<string, string>();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Persist context-stop provenance across durable resumes

When the continuation produces an approval-gated tool call and the host recreates Run before resuming its checkpoint, this in-memory map is empty because it is absent from RunStepResumeState. The post-tool assistant message therefore loses contextStopContinuationOf, causing the Langfuse root output to omit the pre-interrupt segments; another context-window stop can also receive a second billable retry despite the one-per-run contract. Persist and restore this map with the other process-safe resume state.

AGENTS.md reference: AGENTS.md:L144-L144

Useful? React with 👍 / 👎.

Comment thread src/graphs/Graph.ts
Comment on lines +5023 to +5024
!this.contextStopContinued.has(agentId) &&
!this.preemptIncomplete &&

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Scope the preemption guard to the current agent

In a parallel MultiAgentGraph, if one agent first seals without a boundary injection, it sets the graph-wide preemptIncomplete flag; a different agent that then returns model_context_window_exceeded fails this condition and never receives its continuation. The result depends on branch completion order, so this guard should check whether the current response/agent was preempted rather than the shared aggregate flag.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant