[timbrinded]

Problem

Running workflows on openrouter/meta/muse-spark-1.3 (no workflow-models.json, so every profile uses the host model) failed most subagents mid-run with:

404: {"message":"Provider returned error","code":404,"metadata":{"raw":"{\"error\":{\"code\":\"model_not_found\",...}}","provider_name":"Meta","is_byok":false,...}}

The failing agents had already made 11–20 successful tool calls.

Root cause

1. Upstream flake. Direct curl probes with the same key showed OpenRouter's Meta endpoint returning 404 model_not_found on about 12% of requests (3/24), for any reasoning.effort. A few minutes later it was 0/72. Over a 13-turn agent, the chance of zero failures is 0.875¹³ ≈ 18%.
2. No turn-level recovery. pi classifies the error as retryable (provider.?returned.?error), but openAgentSession turned pi's in-session auto-retry off, and --agent-retries defaults to 0. When opted in, a run-level retry restarts the whole agent and discards its completed turns.
3. Persistence bug. AgentSession.setAutoRetryEnabled(false) is not session-scoped: it calls SettingsManager.setRetryEnabled() → save(), which writes "retry": {"enabled": false} into the user's global ~/.pi/agent/settings.json. Every workflow run has been turning off auto-retry for the host pi session and every later session too.

Ruled out: model-ref parsing / provider sync. With no profile config, children receive ctx.model as an object (provider: "openrouter", id: "meta/muse-spark-1.3"), so parseAgentModelRef is never called, and the auth sync keys off openrouter. The provider_name: "Meta" / is_byok: false fields are OpenRouter's own upstream metadata, so the request did reach OpenRouter with the user's key.

Fix

• Stop calling setAutoRetryEnabled(false). Subagents inherit pi's in-session auto-retry from the user's pi retry settings (default: enabled, 3 retries, 2s/4s/8s), so a failed turn is retried in place and earlier tool work is kept.
• Log pi's auto_retry_start events to the workflow progress log (<label>: transient provider failure (<error, truncated>); retrying turn n/N in Xms) and count them as agent.turn_retry.
• Record usage of failed turns that pi retried in place. pi drops the failed assistant message from session.messages before retrying (0.80.10 slices it off; 1.0.4 omits it through a context edit), so it is captured from message_end and included in usage and budget totals.
• Warn once per run when pi auto-retry is disabled, since 0.12.0 and 0.13.0 persisted retry.enabled: false.
• agentRetries tool-schema and completion text now describe it as the outer whole-agent restart.
• --agent-retries keeps its meaning as the outer whole-agent restart, now only after pi's turn retries are exhausted.
• Drop setAutoRetryEnabled from AgentRunnerSession, so the call can't come back without a type change. Add the pi trap to the AGENTS.md critical facts.

Verification

• Deterministic end-to-end repro in pi (tmux). A local proxy in front of OpenRouter fails request Reconcile npm, git tags, and GitHub Releases for v0.6.x #3 of every x-session-id with the exact OpenRouter 404 body. A drop-in workflow runs 5 parallel agents, each making 8 sequential bash calls on muse-spark-1.3.
  • master: 0/5 ok. Every agent died on the injected 404.
  • This branch: 5/5 ok. Each injected 404 was followed ~2.0s later by request Post PR inline comments to the base repository for forked PRs #4 in the same session (an in-place turn retry, not a restart), and ~/.pi/agent/settings.json was byte-identical before and after.
• tests/agent-session-services.test.ts: on the real createAgentSessionServices path, seeded user retry settings (enabled: true, maxRetries: 5) are inherited by the child and still unchanged on disk after settingsManager.flush(). With retry.enabled: false, the warning is logged exactly once across two agents. Mutation-checked: reintroducing setAutoRetryEnabled(false) or removing the warning fails it.
• tests/agent-retry.test.ts: a new case asserts that pi's auto_retry_start reaches the progress log with its reason, and that the dropped failed attempt's output tokens are counted (mutation-checked). The obsolete autoRetrySettings == [false, false] assertion is removed.
• CI isolation fix (7c9d008, pre-existing on master): package-recognition.test.ts loads the extension through pi's DefaultResourceLoader (jiti), which evicts index.ts from Bun's shared module cache. In CI's file order, workflow-tool.test.ts then imported a fresh index.ts while the cached fixture still drove the old instance, so getLastWorkflowInspection() returned undefined. Reproduced locally with three files and on master. The fixture now re-exports the getter from its own binding.
• bun run typecheck is clean; bun run test shows 523 pass, 0 fail, in both the default and CI file order.

Notes for release

• Users who ran v0.12.0/v0.13.0 workflows probably have "retry": {"enabled": false} in ~/.pi/agent/settings.json, written by this bug. While it's there, neither the host nor subagents auto-retry. Release notes should tell them to remove it. It can't be safely auto-repaired, because a deliberate opt-out looks identical.
• Behavior change: retryable rate-limit and usage-limit errors are now retried in-session (up to ~14s with defaults) before a background run's usage-limit pause sees them. That path only inspects the last assistant message, so it still fires once retries are exhausted. Aborting pi's retry early isn't safe: pi has already removed the failed message, so the turn would look successful. In-place retries, like an agent's other turns, are not budget-checked (documented).
• Not adopted from review: replacing user-settings inheritance with a session-scoped engine override (applyOverrides). This was a deliberate choice to have subagents behave like the host.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
🔒 Security Review	✅ Completed 2026-10-06T09:05:30.006719Z	c19c8e0	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[Copilot]

Copilot review overview

🟡 Changes recommended

Retry notices are written to the workflow-wide log rather than the promised agent progress row.

Review effort: Balanced
Findings: 1

Open (1)

• Attach retry notices to the correct agent progress row · New

What changed in this PR

Stops workflow subagents from disabling pi’s global retry setting and exposes inherited turn retries through progress telemetry.

Changes:

• Removes subagent auto-retry mutation.
• Logs and counts in-session retries.
• Updates tests and retry documentation.

File	Description
USAGE.md	Documents layered retry behavior.
AGENTS.md	Records the global-settings persistence hazard.
src/​agent-session.ts	Preserves inherited retries and handles retry events.
src/​agent-runner-types.ts	Removes retry mutation from the session interface.
tests/​agent-session-services.test.ts	Verifies settings remain enabled and unwritten.
tests/​agent-runner-fixtures.ts	Updates the session fixture.
tests/​agent-retry.test.ts	Covers retry-event logging.
tests/​agent-limits.test.ts	Updates the session stub.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.