From 67737f7707d1d68c23c069e1986652bbea34d740 Mon Sep 17 00:00:00 2001 From: Relayflow Date: Mon, 21 Sep 2026 22:27:45 +0000 Subject: [PATCH 1/3] flows logs: render Codex transcripts, not frame placeholders MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `flows logs --step ` rendered Claude's `stream-json` and printed one `frame … (not rendered here — see --raw)` line per frame for a Codex step. The software-factory preset runs its reviewers on Codex, so the verdict that matters most in a run was unreadable without `--raw` and hand parsing. `cloud-transcript.ts` now dispatches per frame rather than per provider, and `cloud-transcript-codex.ts` reads the `codex exec --json` vocabulary: `thread.started`, `turn.started`/`turn.completed`/`turn.failed`, a top-level `error`, and `item.started`/`item.updated`/`item.completed` for `agent_message`, `reasoning`, `command_execution`, `file_change`, `mcp_tool_call` and `error`. - Calls are numbered within their attempt and carry the result size, the exit code (zero included), the item's status and a bounded output excerpt (10 lines / 1,000 characters), so a failed command keeps its size, its exit and its failure on one line rather than collapsing to `ERROR`. - Agent messages print in full: the last one is the step's answer. - `reasoning` is a character count and never its text, as Claude's `thinking` already is. - Item lifecycles are matched locally and in order -- within an attempt, thread and turn, by id *and* item type -- so a reused id never reaches across a boundary and attempt 1's unfinished command survives attempt 2 completing the same id. An unfinished call is shown at its last snapshot and marked as never completed. - Nothing is dropped: `web_search`, `todo_list`, an unknown item type and a malformed known item all keep the placeholder line, now naming the item type, and a malformed item never spends a call number. - Every provider string is redacted whole before it is bounded, and every newly rendered string goes through the control-character filter. Claude entries are unchanged, field for field and line for line; everything Codex adds is a new union member or an optional field. The fixtures are captured, not written: `codex-exec-json.jsonl` is one real `codex exec --json` run of codex-cli 0.155.1 with its thread id and working directory normalised, and `codex-exec-json-failures.jsonl` is frames selected from four more captured runs. Co-Authored-By: Claude Opus 5 --- docs/CLOUD.md | 41 ++ packages/sdk/src/cloud-transcript-codex.ts | 379 ++++++++++++++ packages/sdk/src/cloud-transcript-types.ts | 214 ++++++++ packages/sdk/src/cloud-transcript.ts | 180 ++----- packages/sdk/tests/cloud-read.test.ts | 39 +- .../sdk/tests/cloud-transcript-codex.test.ts | 478 ++++++++++++++++++ .../fixtures/codex-exec-json-failures.jsonl | 10 + .../sdk/tests/fixtures/codex-exec-json.jsonl | 11 + 8 files changed, 1215 insertions(+), 137 deletions(-) create mode 100644 packages/sdk/src/cloud-transcript-codex.ts create mode 100644 packages/sdk/src/cloud-transcript-types.ts create mode 100644 packages/sdk/tests/cloud-transcript-codex.test.ts create mode 100644 packages/sdk/tests/fixtures/codex-exec-json-failures.jsonl create mode 100644 packages/sdk/tests/fixtures/codex-exec-json.jsonl diff --git a/docs/CLOUD.md b/docs/CLOUD.md index 1fc6bc839..194fa7ebe 100644 --- a/docs/CLOUD.md +++ b/docs/CLOUD.md @@ -187,6 +187,47 @@ assistant: ── result success · 6.2s · 2 turns · $0.108098 · 4 in / 248 out · 25,402 cache read · 25,638 cache write ── ``` +Both harness vocabularies render. A Codex step writes `codex exec --json`, +and its frames are read too: `thread.started` as the session header, +`turn.started`/`turn.completed` as separators with the turn's usage, +`agent_message` as prose — **in full**, because the last one is the step's +answer — `reasoning` as a character count and never its text, and +`command_execution` and `mcp_tool_call` as **numbered** call lines carrying the +result size, the exit code (zero included), the item's status, and a bounded +excerpt of the output: ten lines or a thousand characters, whichever comes +first, with the full size still on the call line and the whole of it under +`--raw`. An `apply_patch` is file activity rather than a call, so it is listed +by path and change kind and takes no number. A `turn.failed`, a top-level +`error` frame and an `error` item print their message rather than anything that +reads like success. Dispatch is per frame, so a log that mixes vocabularies +renders each in its own shape, and an item type this vocabulary does not read — +`web_search`, `todo_list`, anything newer — still gets the placeholder line +naming it. + +```text +LOG f92bf832-0d26-4f17-9c50-2b4b2e2f77d5 step agent-17 1,967 bytes complete +session codex · thread 01a0c600-0000-7a10-a68c-000000000000 +── turn ──────────────────────────────────────────────────────────────── +assistant: + I’ll perform the requested review steps in order. + tool 1 command_execution /bin/bash -lc 'cat src/pricing.ts' → 110 chars · exit 0 · completed + export function total(cents: number, taxRate: number): number { + return Math.round(cents * (1 + taxRate)); + } + tool 2 mcp_tool_call demo/echo_shout {"text":"p2","options":{"mode":"loud"}} → 29 chars · completed + files 1 change · completed + add /project/review.md +assistant: + Verdict: changes_requested + + The rounding in `total` truncates before the tax is applied, so a + 0.5-cent remainder is lost on every line item. + + One P2 remains and the gate artifact `review.clean` was not created. +── turn complete · 72,669 in / 244 out · 69,376 cache read · 0 cache write · 0 reasoning out ── +``` + + `--raw` prints the JSONL unrendered. It does **not** print it unredacted: every string that reaches the terminal — rendered, raw, or `--json` — goes through `redact.ts`, the redactor the local `flows status` uses. That is a diff --git a/packages/sdk/src/cloud-transcript-codex.ts b/packages/sdk/src/cloud-transcript-codex.ts new file mode 100644 index 000000000..c989e3312 --- /dev/null +++ b/packages/sdk/src/cloud-transcript-codex.ts @@ -0,0 +1,379 @@ +// The Codex half of `flows logs --step`: `codex exec --json` JSONL. +// +// The vocabulary was captured, not guessed. Every frame shape read here came +// out of a real `codex exec --json` run of codex-cli 0.155.1 on 2026-09-21; +// the captures are the two fixtures under `tests/fixtures/`, and the event and +// item names are the ones the binary's own enums carry (`thread.started`, +// `turn.started`, `turn.completed`, `turn.failed`, `item.started`, +// `item.updated`, `item.completed`, `error`; items `agent_message`, +// `reasoning`, `command_execution`, `file_change`, `mcp_tool_call`, +// `web_search`, `todo_list`, `error`). `web_search` and `todo_list` keep the +// unknown-frame placeholder: naming an item type is not the same as knowing +// which of its fields carry what, and a placeholder is honest. +// +// The two rules `cloud-transcript.ts` states hold here too. Nothing is +// dropped: an item this module cannot read becomes the placeholder line rather +// than vanishing or being reported as a success. And every provider string is +// redacted whole before it is bounded, because a truncated credential still +// matches nothing and is still most of a credential. + +import { + ERROR_MAX_CHARS, OUTPUT_MAX_CHARS, OUTPUT_MAX_LINES, PATH_MAX_CHARS, TARGET_MAX_CHARS, + isRecord, num, oneLine, safe, separator, str, thousands, + type TranscriptEntry, type TranscriptToolCodex, +} from './cloud-transcript-types.js'; + +/** Frame types that make a log a transcript this module can render. */ +export const CODEX_FRAME_TYPES: ReadonlySet = new Set([ + 'thread.started', 'turn.started', 'turn.completed', 'turn.failed', + 'item.started', 'item.updated', 'item.completed', 'error', +]); + +/** Item types whose lifecycles are matched and whose fields are read. */ +const SUPPORTED_ITEMS: ReadonlySet = new Set([ + 'agent_message', 'reasoning', 'command_execution', 'file_change', 'mcp_tool_call', 'error', +]); + +const ITEM_FRAMES: ReadonlySet = new Set(['item.started', 'item.updated', 'item.completed']); + +/** Calls are numbered; an `apply_patch` is file activity, not a call. */ +const NUMBERED_ITEMS: ReadonlySet = new Set(['command_execution', 'mcp_tool_call']); + +interface Lifecycle { + snapshots: Array>; + complete: boolean; + /** The index of the last frame of this lifecycle: where the entry is emitted. */ + at: number; +} + +export interface CodexIndex { + /** Every frame index that belongs to a matched lifecycle. */ + matched: Set; + /** The lifecycle to emit at this frame index. */ + emit: Map; +} + +/** + * Match item lifecycles locally and in order, rather than by a global set of + * completed ids. + * + * `partition` changes at every attempt marker, thread and turn boundary, so a + * reused id never reaches across one: an attempt that was killed mid-command + * keeps its unfinished call even when the next attempt completes the same id. + * Within a partition a lifecycle is keyed by id *and* item type -- two items + * that merely share an id are two items -- and a start that arrives after that + * key already completed opens a new one. + */ +export function indexCodexItems(frames: ReadonlyArray<{ frame: unknown }>): CodexIndex { + const matched = new Set(); + const emit = new Map(); + const open = new Map(); + let partition = 0; + for (let index = 0; index < frames.length; index += 1) { + const frame = frames[index]!.frame; + if (!isRecord(frame)) continue; + const type = str(frame['type']); + if (type === null) continue; + if (type.startsWith('relayflow.attempt') || type === 'thread.started' || type.startsWith('turn.')) { + partition += 1; + open.clear(); + continue; + } + if (!ITEM_FRAMES.has(type)) continue; + const item = isRecord(frame['item']) ? frame['item'] : null; + if (item === null) continue; + const id = str(item['id']); + const itemType = str(item['type']); + // No id is not a correlation key, and an unsupported item is not one this + // module can merge. Either way the frame stands where it is. + if (id === null || itemType === null || !SUPPORTED_ITEMS.has(itemType)) continue; + const key = `${partition}\u0000${id}\u0000${itemType}`; + let lifecycle = open.get(key); + if (lifecycle === undefined || lifecycle.complete) { + lifecycle = { snapshots: [], complete: false, at: index }; + open.set(key, lifecycle); + } + lifecycle.snapshots.push(item); + lifecycle.complete ||= type === 'item.completed'; + emit.delete(lifecycle.at); + lifecycle.at = index; + emit.set(index, lifecycle); + matched.add(index); + } + return { matched, emit }; +} + +/** Where the numbering lives; the parse loop resets it at an attempt boundary. */ +export interface CodexState { seq: number } + +interface Context { + clean: (text: string) => string; + state: CodexState; + /** The whole source line, for a placeholder's size. */ + line: string; + type: string; +} + +function bounded(text: string, clean: (t: string) => string, limit = TARGET_MAX_CHARS): string { + return oneLine(clean(text), limit); +} + +/** A result's size, never its content. */ +function textChars(content: unknown): number { + if (typeof content === 'string') return content.length; + if (!Array.isArray(content)) return 0; + let total = 0; + for (const block of content) if (isRecord(block)) total += str(block['text'])?.length ?? 0; + return total; +} + +/** + * Redact the whole output, then cut it. Both bounds are applied and either one + * sets the flag, so the reader is told the excerpt is short of the size the + * call reported. + */ +function excerpt(output: string, clean: (text: string) => string): { text: string; truncated: boolean } { + const cleaned = clean(output).replace(/\n+$/u, ''); + const lines = cleaned.split('\n'); + let truncated = lines.length > OUTPUT_MAX_LINES; + let text = lines.slice(0, OUTPUT_MAX_LINES).join('\n'); + if (text.length > OUTPUT_MAX_CHARS) { text = text.slice(0, OUTPUT_MAX_CHARS); truncated = true; } + return { text, truncated }; +} + +function placeholder(context: Context, itemType: string | null): TranscriptEntry { + const label = itemType === null ? context.type : `${context.type}/${itemType}`; + return { kind: 'unknown', type: context.clean(label), chars: context.line.length }; +} + +function tool(name: string, target: string, resultChars: number | null, isError: boolean, + codex: TranscriptToolCodex): TranscriptEntry { + return { kind: 'tool', name, target, result_chars: resultChars, is_error: isError, nested: false, codex }; +} + +function commandEntry(item: Record, complete: boolean, context: Context, + seq: number): TranscriptEntry | null { + const command = item['command']; + if (typeof command !== 'string') return null; + const output = typeof item['aggregated_output'] === 'string' ? item['aggregated_output'] : null; + const cut = output === null ? null : excerpt(output, context.clean); + const exitCode = num(item['exit_code']); + const status = str(item['status']); + return tool('command_execution', bounded(command, context.clean), output === null ? null : output.length, + status === 'failed' || (exitCode !== null && exitCode !== 0), { + seq, status: status === null ? null : bounded(status, context.clean, 40), + exit_code: exitCode, output_excerpt: cut === null ? null : cut.text, + output_truncated: cut?.truncated === true, complete, error: null, + }); +} + +/** `{"message": "..."}` is the shape both `turn.failed` and a failed call use. */ +function errorText(value: unknown): string | null { + if (typeof value === 'string') return value; + return isRecord(value) ? str(value['message']) : null; +} + +function mcpEntry(item: Record, complete: boolean, context: Context, + seq: number): TranscriptEntry | null { + const server = str(item['server']); + const name = str(item['tool']); + if (server === null && name === null) return null; + const args = item['arguments']; + // Serialized with its field names intact: `redact` recognises a credential + // field by its name, so flattening the object first would hide it. + const rendered = args === undefined || args === null ? '' : ` ${JSON.stringify(args)}`; + const result = isRecord(item['result']) ? item['result'] : null; + const structured = result === null ? null : result['structured_content']; + const failure = errorText(item['error']); + const status = str(item['status']); + return tool('mcp_tool_call', bounded(`${server ?? '?'}/${name ?? '?'}${rendered}`, context.clean), + result === null ? null : textChars(result['content']) + (structured === undefined || structured === null + ? 0 : JSON.stringify(structured).length), + status === 'failed' || failure !== null, { + seq, status: status === null ? null : bounded(status, context.clean, 40), + exit_code: null, output_excerpt: null, output_truncated: false, complete, + error: failure === null ? null : bounded(failure, context.clean, ERROR_MAX_CHARS), + }); +} + +function fileChangeEntry(item: Record, complete: boolean, context: Context): TranscriptEntry | null { + const raw = item['changes']; + if (!Array.isArray(raw)) return null; + const changes: Array<{ path: string | null; change: string | null }> = []; + let malformed = 0; + for (const change of raw) { + if (!isRecord(change)) { malformed += 1; continue; } + const path = str(change['path']); + const kind = str(change['kind']); + if (path === null && kind === null) { malformed += 1; continue; } + changes.push({ + path: path === null ? null : bounded(path, context.clean, PATH_MAX_CHARS), + change: kind === null ? null : bounded(kind, context.clean, 40), + }); + } + const status = str(item['status']); + return { + kind: 'file_change', changes, malformed, + status: status === null ? null : bounded(status, context.clean, 40), complete, + }; +} + +function itemEntries(item: Record, complete: boolean, context: Context): TranscriptEntry[] { + const itemType = str(item['type']); + if (itemType === 'agent_message') { + const text = item['text']; + if (typeof text !== 'string') return [placeholder(context, itemType)]; + return [{ kind: 'message', role: 'assistant', text: context.clean(text), nested: false, + ...(complete ? {} : { complete: false }) }]; + } + if (itemType === 'reasoning') { + const text = item['text']; + // The count, never the text -- the one thing Claude's renderer refuses to + // print, and Codex is not the place that starts printing it. + if (typeof text !== 'string') return [placeholder(context, itemType)]; + return [{ kind: 'thinking', chars: text.length, nested: false }]; + } + if (itemType === 'error') { + const message = errorText(item['message']); + if (message === null) return [placeholder(context, itemType)]; + return [{ kind: 'error', source: 'item', message: bounded(message, context.clean, ERROR_MAX_CHARS) }]; + } + if (itemType === 'command_execution' || itemType === 'mcp_tool_call') { + // The number is only spent on a call this module could actually read, so a + // malformed item leaves no gap in the sequence. + const seq = context.state.seq + 1; + const entry = itemType === 'command_execution' + ? commandEntry(item, complete, context, seq) : mcpEntry(item, complete, context, seq); + if (entry === null) return [placeholder(context, itemType)]; + context.state.seq = seq; + return [entry]; + } + if (itemType === 'file_change') { + const entry = fileChangeEntry(item, complete, context); + return [entry ?? placeholder(context, itemType)]; + } + return [placeholder(context, itemType)]; +} + +const NO_USAGE = { + tokens_in: null, tokens_out: null, cache_read: null, cache_creation: null, reasoning_out: null, +} as const; + +/** + * One Codex frame's entries, or undefined when the frame belongs to a + * lifecycle that is reported at a later frame. + */ +export function codexFrame( + frame: Record, type: string, index: number, index_: CodexIndex, + state: CodexState, clean: (text: string) => string, line: string, +): TranscriptEntry[] | undefined { + const context: Context = { clean, state, line, type }; + if (type === 'thread.started') { + const id = str(frame['thread_id']); + return [{ kind: 'thread', thread_id: id === null ? null : bounded(id, clean) }]; + } + if (type === 'turn.started') return [{ kind: 'turn', phase: 'started', ...NO_USAGE }]; + if (type === 'turn.completed') { + const usage = isRecord(frame['usage']) ? frame['usage'] : null; + return [{ + kind: 'turn', phase: 'completed', + tokens_in: usage === null ? null : num(usage['input_tokens']), + tokens_out: usage === null ? null : num(usage['output_tokens']), + cache_read: usage === null ? null : num(usage['cached_input_tokens']), + cache_creation: usage === null ? null : num(usage['cache_write_input_tokens']), + reasoning_out: usage === null ? null : num(usage['reasoning_output_tokens']), + }]; + } + if (type === 'turn.failed' || type === 'error') { + const message = errorText(type === 'error' ? frame['message'] : frame['error']); + // A failure with no readable message is still a failure; it must not fall + // through to anything that reads like success. + const entry: TranscriptEntry = { + kind: 'error', source: type, + message: message === null ? '(no message)' : bounded(message, clean, ERROR_MAX_CHARS), + }; + return type === 'turn.failed' ? [{ kind: 'turn', phase: 'failed', ...NO_USAGE }, entry] : [entry]; + } + // Everything left in `CODEX_FRAME_TYPES` is an `item.*` frame. + const lifecycle = index_.emit.get(index); + if (lifecycle === undefined) { + // Part of a lifecycle reported later, or an item this module does not + // correlate -- no id, an unsupported type, a malformed frame. + if (index_.matched.has(index)) return []; + const item = isRecord(frame['item']) ? frame['item'] : null; + if (item === null) return [placeholder(context, null)]; + return itemEntries(item, type === 'item.completed', context); + } + // The latest snapshot wins field by field, so a completion's null exit code + // is not overwritten by the zero-length output its start reported. + const merged = Object.assign({}, ...lifecycle.snapshots) as Record; + return itemEntries(merged, lifecycle.complete, context); +} + +function indented(text: string): string[] { + return text.split('\n').map((line) => ` ${safe(line)}`); +} + +/** The Codex entry kinds. Undefined for everything the Claude renderer owns. */ +export function renderCodexEntry(entry: TranscriptEntry): string[] | undefined { + switch (entry.kind) { + case 'thread': + return [`session codex${entry.thread_id === null ? '' : ` · thread ${safe(entry.thread_id)}`}`]; + case 'turn': { + if (entry.phase === 'started') return [separator('turn')]; + if (entry.phase === 'failed') return [separator('turn failed')]; + const facts = [ + entry.tokens_in === null && entry.tokens_out === null ? null + : `${entry.tokens_in === null ? '?' : thousands(entry.tokens_in)} in` + + ` / ${entry.tokens_out === null ? '?' : thousands(entry.tokens_out)} out`, + entry.cache_read === null ? null : `${thousands(entry.cache_read)} cache read`, + entry.cache_creation === null ? null : `${thousands(entry.cache_creation)} cache write`, + entry.reasoning_out === null ? null : `${thousands(entry.reasoning_out)} reasoning out`, + ].filter((fact): fact is string => fact !== null); + return [separator(`turn complete${facts.length === 0 ? '' : ` · ${facts.join(' · ')}`}`)]; + } + case 'error': + return [entry.source === 'error' ? 'error:' : `error (${safe(entry.source)}):`, ...entry.message.split('\n').map((line) => ` ${safe(line)}`)]; + case 'file_change': { + const total = entry.changes.length + entry.malformed; + const state = [ + entry.status === null ? null : safe(entry.status), + entry.complete ? null : 'no completion frame', + ].filter((fact): fact is string => fact !== null); + const lines = [` files ${total} change${total === 1 ? '' : 's'}` + + `${state.length === 0 ? '' : ` · ${state.join(' · ')}`}`]; + for (const change of entry.changes) { + lines.push(` ${safe(change.change ?? '?')} ${safe(change.path ?? '(no path)')}`); + } + if (entry.malformed > 0) { + lines.push(` ${entry.malformed} change${entry.malformed === 1 ? '' : 's'}` + + ' not rendered here — see --raw'); + } + return lines; + } + case 'tool': { + if (entry.codex === undefined) return undefined; + const codex = entry.codex; + // The size stays on the line whatever happened: a reviewer reading a + // failed step needs the exit code, the status and the size together. + const facts = [ + entry.result_chars === null ? 'no result' : `${thousands(entry.result_chars)} chars`, + codex.exit_code === null ? null : `exit ${codex.exit_code}`, + codex.status === null ? null : safe(codex.status), + codex.complete ? null : 'no completion frame', + ].filter((fact): fact is string => fact !== null); + const lines = [` tool ${codex.seq} ${safe(entry.name)} ${safe(entry.target ?? '')} → ${facts.join(' · ')}`]; + if (codex.error !== null) lines.push(...indented(`error ${codex.error}`)); + if (codex.output_excerpt !== null && codex.output_excerpt.length > 0) { + lines.push(...indented(codex.output_excerpt)); + } + if (codex.output_truncated) { + lines.push(` … excerpt cut at ${thousands(OUTPUT_MAX_LINES)} lines` + + ` / ${thousands(OUTPUT_MAX_CHARS)} chars — see --raw`); + } + return lines; + } + default: + return undefined; + } +} diff --git a/packages/sdk/src/cloud-transcript-types.ts b/packages/sdk/src/cloud-transcript-types.ts new file mode 100644 index 000000000..3ca3114cb --- /dev/null +++ b/packages/sdk/src/cloud-transcript-types.ts @@ -0,0 +1,214 @@ +// The entry vocabulary `flows logs --step` renders, and the pure helpers both +// provider parsers need. +// +// This module is a leaf: it imports nothing, so `cloud-transcript.ts` (the +// public entry point and the Claude vocabulary) and `cloud-transcript-codex.ts` +// can both depend on it without either depending on the other. +// +// The entry union is a public TypeScript API — `index.ts` re-exports it and +// `flows logs --json` emits it. Everything added for Codex is either a new +// member of the union or an optional field, so a Claude entry serializes to +// exactly the bytes it serialized to before. A consumer switching +// exhaustively over `kind` must still add the new members. + +export interface TranscriptAttempt { + kind: 'attempt'; + attempt: number | null; + bytes: number | null; + truncated: boolean; +} + +export interface TranscriptAttemptOmitted { + kind: 'attempt_omitted'; + attempt: number | null; + bytes: number | null; +} + +export interface TranscriptInit { + kind: 'init'; + model: string | null; + version: string | null; + permission_mode: string | null; + tools: number | null; + mcp_servers: number | null; + session_id: string | null; +} + +export interface TranscriptMessage { + kind: 'message'; + role: 'assistant' | 'user'; + text: string; + /** A frame carrying `parent_tool_use_id`: a subagent's turn, not the main one. */ + nested: boolean; + /** + * Codex only, and only when false: the message was read off an + * `item.started`/`item.updated` snapshot that never completed, so the text + * is what had been written so far and not the whole of it. + */ + complete?: boolean; +} + +export interface TranscriptThinking { + kind: 'thinking'; + /** The count only. A thinking block's text and signature never reach the page. */ + chars: number; + nested: boolean; +} + +/** + * The Codex half of a tool call. Present on an entry parsed from a Codex + * `command_execution` or `mcp_tool_call` item and absent on every Claude + * entry, which is what the renderer discriminates on: a Codex failure has to + * keep its exit code, its status and its result size on the same line, where + * Claude's renderer replaces the size with `ERROR`. + */ +export interface TranscriptToolCodex { + /** The call's position among the calls of its attempt, counted from 1. */ + seq: number; + /** `in_progress`, `completed` or `failed`, as the item reported it. */ + status: string | null; + /** Null when the item carried no exit code — not the same fact as exit 0. */ + exit_code: number | null; + /** Redacted, then bounded. Null when the item carried no output field at all. */ + output_excerpt: string | null; + /** True when a character or line bound cut the excerpt. */ + output_truncated: boolean; + /** False when no `item.completed` ever arrived for this call. */ + complete: boolean; + /** Redacted, then bounded. The item's own failure text. */ + error: string | null; +} + +export interface TranscriptTool { + kind: 'tool'; + name: string; + /** What the call was aimed at: a path, a command, a pattern. Null when the input had no string. */ + target: string | null; + /** Null when no `tool_result` answered it -- not the same fact as a zero-byte result. */ + result_chars: number | null; + is_error: boolean; + nested: boolean; + /** Codex only. Its presence is what tells the renderer which shape to print. */ + codex?: TranscriptToolCodex; +} + +/** One `apply_patch` item: file activity, not a tool call, so it is not numbered. */ +export interface TranscriptFileChange { + kind: 'file_change'; + changes: Array<{ path: string | null; change: string | null }>; + /** Change elements that were not objects. Counted rather than dropped. */ + malformed: number; + status: string | null; + complete: boolean; +} + +export interface TranscriptThread { + kind: 'thread'; + thread_id: string | null; +} + +export interface TranscriptTurn { + kind: 'turn'; + phase: 'started' | 'completed' | 'failed'; + tokens_in: number | null; + tokens_out: number | null; + cache_read: number | null; + cache_creation: number | null; + reasoning_out: number | null; +} + +/** A `turn.failed`, a top-level `error` frame, or an `error` item. */ +export interface TranscriptError { + kind: 'error'; + /** The frame this came from, so the reader knows what failed. */ + source: string; + message: string; +} + +export interface TranscriptResult { + kind: 'result'; + is_error: boolean; + subtype: string | null; + duration_ms: number | null; + num_turns: number | null; + total_cost_usd: number | null; + tokens_in: number | null; + tokens_out: number | null; + cache_read: number | null; + cache_creation: number | null; +} + +export interface TranscriptUnknown { + kind: 'unknown'; + type: string; + chars: number; +} + +export interface TranscriptUnparsed { + kind: 'unparsed'; + text: string; +} + +export type TranscriptEntry = + | TranscriptAttempt | TranscriptAttemptOmitted | TranscriptInit | TranscriptMessage + | TranscriptThinking | TranscriptTool | TranscriptFileChange | TranscriptThread + | TranscriptTurn | TranscriptError | TranscriptResult | TranscriptUnknown | TranscriptUnparsed; + +export interface ParsedTranscript { + entries: TranscriptEntry[]; + /** How many attempts said their head was cut to fit the log cap. */ + truncated_attempts: number; + /** How many attempts were dropped whole to fit the log cap. */ + omitted_attempts: number; + /** + * False when the log carried no frame either vocabulary this module knows -- + * a v1 plain-terminal sandbox log, say. The caller prints it raw rather than + * claiming an empty transcript. + */ + stream_json: boolean; +} + +/** A command, an argument list or a `server/tool` pair on one line. */ +export const TARGET_MAX_CHARS = 160; + +/** A command's or a tool's output excerpt: enough to read, bounded either way. */ +export const OUTPUT_MAX_CHARS = 1000; +export const OUTPUT_MAX_LINES = 10; + +/** Provider-supplied failure text. Prose from the agent is the only uncapped string. */ +export const ERROR_MAX_CHARS = 1000; + +/** A path stays identifiable, so it is bounded far above a one-line target. */ +export const PATH_MAX_CHARS = 200; + +export function isRecord(value: unknown): value is Record { + return typeof value === 'object' && value !== null && !Array.isArray(value); +} + +export function str(value: unknown): string | null { + return typeof value === 'string' && value.length > 0 ? value : null; +} + +export function num(value: unknown): number | null { + return typeof value === 'number' && Number.isFinite(value) ? value : null; +} + +/** Collapse to one line and bound it; a tool input can be a whole file body. */ +export function oneLine(text: string, limit = TARGET_MAX_CHARS): string { + const flat = text.replace(/\s+/gu, ' ').trim(); + return flat.length <= limit ? flat : `${flat.slice(0, limit)}…`; +} + +export function thousands(value: number): string { + return String(value).replace(/\B(?=(\d{3})+(?!\d))/gu, ','); +} + +/** Terminal control characters never reach the page, wherever the text came from. */ +export function safe(text: string): string { + return text.replace(/[\u0000-\u0009\u000B-\u001F\u007F-\u009F]/gu, '?'); +} + +/** One rule with a label, the way an attempt boundary reads. */ +export function separator(label: string): string { + return `── ${label} ${'─'.repeat(Math.max(2, 68 - label.length))}`; +} diff --git a/packages/sdk/src/cloud-transcript.ts b/packages/sdk/src/cloud-transcript.ts index 7763304f9..02981538b 100644 --- a/packages/sdk/src/cloud-transcript.ts +++ b/packages/sdk/src/cloud-transcript.ts @@ -1,13 +1,16 @@ // Rendering an agent step's transcript JSONL for a terminal. // -// The frames are what the harness wrote (Claude Code's `stream-json`), wrapped -// by the `relayflow.attempt` markers the v2 executor interleaves when it -// assembles one log out of several attempts. The frame vocabulary read here is -// the one Cloud's dashboard renderer reads (cloud#3847, -// `packages/web/lib/workflows/agent-transcript.ts`) -- the same shapes, so the -// CLI and the dashboard agree about what a run did. The implementation is not -// shared: this package cannot import from the Cloud app, and a CLI that had to -// be deployed in step with a dashboard would be worse than one that does not. +// The frames are what the harness wrote -- Claude Code's `stream-json`, or +// `codex exec --json` -- wrapped by the `relayflow.attempt` markers the v2 +// executor interleaves when it assembles one log out of several attempts. +// The Claude frame vocabulary read here is the one Cloud's dashboard renderer +// reads (cloud#3847, `packages/web/lib/workflows/agent-transcript.ts`) -- the +// same shapes, so the CLI and the dashboard agree about what a run did. The +// implementation is not shared: this package cannot import from the Cloud app, +// and a CLI that had to be deployed in step with a dashboard would be worse +// than one that does not. The Codex vocabulary lives in +// `cloud-transcript-codex.ts`; dispatch is per frame, not per provider, so a +// log that mixes them stays honest rather than being forced into a guess. // // Two rules the shapes do not give you: // @@ -19,98 +22,21 @@ // the redactor `flows status` uses. Transcript text is whatever the agent // printed, including anything it read out of its own environment. +import { + CODEX_FRAME_TYPES, codexFrame, indexCodexItems, renderCodexEntry, type CodexState, +} from './cloud-transcript-codex.js'; +import { + TARGET_MAX_CHARS, isRecord, num, oneLine, safe, separator, str, thousands, + type ParsedTranscript, type TranscriptEntry, +} from './cloud-transcript-types.js'; import { redact } from './redact.js'; -export interface TranscriptAttempt { - kind: 'attempt'; - attempt: number | null; - bytes: number | null; - truncated: boolean; -} - -export interface TranscriptAttemptOmitted { - kind: 'attempt_omitted'; - attempt: number | null; - bytes: number | null; -} - -export interface TranscriptInit { - kind: 'init'; - model: string | null; - version: string | null; - permission_mode: string | null; - tools: number | null; - mcp_servers: number | null; - session_id: string | null; -} - -export interface TranscriptMessage { - kind: 'message'; - role: 'assistant' | 'user'; - text: string; - /** A frame carrying `parent_tool_use_id`: a subagent's turn, not the main one. */ - nested: boolean; -} - -export interface TranscriptThinking { - kind: 'thinking'; - /** The count only. A thinking block's text and signature never reach the page. */ - chars: number; - nested: boolean; -} - -export interface TranscriptTool { - kind: 'tool'; - name: string; - /** What the call was aimed at: a path, a command, a pattern. Null when the input had no string. */ - target: string | null; - /** Null when no `tool_result` answered it -- not the same fact as a zero-byte result. */ - result_chars: number | null; - is_error: boolean; - nested: boolean; -} - -export interface TranscriptResult { - kind: 'result'; - is_error: boolean; - subtype: string | null; - duration_ms: number | null; - num_turns: number | null; - total_cost_usd: number | null; - tokens_in: number | null; - tokens_out: number | null; - cache_read: number | null; - cache_creation: number | null; -} - -export interface TranscriptUnknown { - kind: 'unknown'; - type: string; - chars: number; -} - -export interface TranscriptUnparsed { - kind: 'unparsed'; - text: string; -} - -export type TranscriptEntry = - | TranscriptAttempt | TranscriptAttemptOmitted | TranscriptInit | TranscriptMessage - | TranscriptThinking | TranscriptTool | TranscriptResult | TranscriptUnknown | TranscriptUnparsed; - -export interface ParsedTranscript { - entries: TranscriptEntry[]; - /** How many attempts said their head was cut to fit the log cap. */ - truncated_attempts: number; - /** How many attempts were dropped whole to fit the log cap. */ - omitted_attempts: number; - /** - * False when the log carried no frame this vocabulary knows -- a v1 - * plain-terminal sandbox log, say. The caller prints it raw rather than - * claiming an empty transcript. - */ - stream_json: boolean; -} +export type { + ParsedTranscript, TranscriptEntry, TranscriptAttempt, TranscriptAttemptOmitted, TranscriptError, + TranscriptFileChange, TranscriptInit, TranscriptMessage, TranscriptResult, TranscriptThinking, + TranscriptThread, TranscriptTool, TranscriptToolCodex, TranscriptTurn, TranscriptUnknown, + TranscriptUnparsed, +} from './cloud-transcript-types.js'; /** * Which input field names a tool call. Order matters: the first present @@ -122,26 +48,6 @@ const TARGET_KEYS = [ 'prompt', 'description', 'subagent_type', ] as const; -const TARGET_MAX_CHARS = 160; - -function isRecord(value: unknown): value is Record { - return typeof value === 'object' && value !== null && !Array.isArray(value); -} - -function str(value: unknown): string | null { - return typeof value === 'string' && value.length > 0 ? value : null; -} - -function num(value: unknown): number | null { - return typeof value === 'number' && Number.isFinite(value) ? value : null; -} - -/** Collapse to one line and bound it; a tool input can be a whole file body. */ -function oneLine(text: string, limit = TARGET_MAX_CHARS): string { - const flat = text.replace(/\s+/gu, ' ').trim(); - return flat.length <= limit ? flat : `${flat.slice(0, limit)}…`; -} - /** * `clean` runs before `oneLine`, not after, and the order is load-bearing. * @@ -154,12 +60,12 @@ function toolTarget(input: unknown, clean: (text: string) => string): string | n if (!isRecord(input)) return null; for (const key of TARGET_KEYS) { const value = str(input[key]); - if (value !== null) return oneLine(clean(value)); + if (value !== null) return oneLine(clean(value), TARGET_MAX_CHARS); } // An unfamiliar tool still shows something rather than nothing. for (const value of Object.values(input)) { const text = str(value); - if (text !== null) return oneLine(clean(text)); + if (text !== null) return oneLine(clean(text), TARGET_MAX_CHARS); } return null; } @@ -219,13 +125,15 @@ export function parseAgentTranscript(content: string, env: NodeJS.ProcessEnv = p } } const results = indexToolResults(frames); + const codexIndex = indexCodexItems(raw); + const codexState: CodexState = { seq: 0 }; const entries: TranscriptEntry[] = []; let truncatedAttempts = 0; let omittedAttempts = 0; let streamJson = false; const clean = (text: string): string => redact(text, env); - for (const { frame, line } of raw) { + for (const [index, { frame, line }] of raw.entries()) { if (frame === undefined || !isRecord(frame)) { entries.push({ kind: 'unparsed', text: clean(line) }); continue; @@ -233,6 +141,8 @@ export function parseAgentTranscript(content: string, env: NodeJS.ProcessEnv = p const type = str(frame['type']); if (type === 'relayflow.attempt') { streamJson = true; + // Calls are numbered within an attempt, so a second attempt starts over. + codexState.seq = 0; const truncated = frame['truncated'] === true; if (truncated) truncatedAttempts += 1; entries.push({ kind: 'attempt', attempt: num(frame['attempt']), bytes: num(frame['bytes']), truncated }); @@ -240,10 +150,16 @@ export function parseAgentTranscript(content: string, env: NodeJS.ProcessEnv = p } if (type === 'relayflow.attempt.omitted') { streamJson = true; + codexState.seq = 0; omittedAttempts += 1; entries.push({ kind: 'attempt_omitted', attempt: num(frame['attempt']), bytes: num(frame['bytes']) }); continue; } + if (type !== null && CODEX_FRAME_TYPES.has(type)) { + streamJson = true; + entries.push(...(codexFrame(frame, type, index, codexIndex, codexState, clean, line) ?? [])); + continue; + } if (type === 'system' && frame['subtype'] === 'init') { streamJson = true; const tools = frame['tools']; @@ -327,28 +243,14 @@ export function parseAgentTranscript(content: string, env: NodeJS.ProcessEnv = p return { entries, truncated_attempts: truncatedAttempts, omitted_attempts: omittedAttempts, stream_json: streamJson }; } -function thousands(value: number): string { - return String(value).replace(/\B(?=(\d{3})+(?!\d))/gu, ','); -} - function seconds(ms: number): string { return ms < 1000 ? `${ms}ms` : `${(ms / 1000).toFixed(1)}s`; } -/** Terminal control characters never reach the page, wherever the text came from. */ -function safe(text: string): string { - return text.replace(/[\u0000-\u0009\u000B-\u001F\u007F-\u009F]/gu, '?'); -} - function dollars(value: number): string { return `$${value.toFixed(6).replace(/0+$/u, '').replace(/\.$/u, '')}`; } -/** One rule with a label, the way an attempt boundary reads. */ -function separator(label: string): string { - return `── ${label} ${'─'.repeat(Math.max(2, 68 - label.length))}`; -} - /** * Render a parsed transcript as terminal lines. * @@ -359,6 +261,11 @@ function separator(label: string): string { export function renderAgentTranscript(parsed: ParsedTranscript): string[] { const lines: string[] = []; for (const entry of parsed.entries) { + // A Codex tool call carries its exit code, its status and its result size + // on one line, which the Claude shape below has no room for; everything + // the Codex vocabulary owns is rendered there. + const codex = renderCodexEntry(entry); + if (codex !== undefined) { lines.push(...codex); continue; } switch (entry.kind) { case 'attempt': { const size = entry.bytes === null ? '' : ` · ${thousands(entry.bytes)} bytes`; @@ -380,7 +287,8 @@ export function renderAgentTranscript(parsed: ParsedTranscript): string[] { break; } case 'message': { - const who = `${entry.role}${entry.nested ? ' (subagent)' : ''}`; + const state = entry.complete === false ? ' (incomplete)' : ''; + const who = `${entry.role}${entry.nested ? ' (subagent)' : ''}${state}`; const body = entry.text.split('\n'); lines.push(`${who}:`); for (const line of body) lines.push(` ${safe(line)}`); diff --git a/packages/sdk/tests/cloud-read.test.ts b/packages/sdk/tests/cloud-read.test.ts index 548772530..3bdfe82ea 100644 --- a/packages/sdk/tests/cloud-read.test.ts +++ b/packages/sdk/tests/cloud-read.test.ts @@ -5,7 +5,7 @@ // step is an agent with a seven-frame transcript. Anything these tests assert // about a field name is a field name the real routes emit. -import { mkdtempSync, rmSync, writeFileSync } from 'node:fs'; +import { mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'; import { tmpdir } from 'node:os'; import { join } from 'node:path'; import { afterEach, describe, expect, it, vi } from 'vitest'; @@ -332,6 +332,43 @@ describe('flows logs', () => { } }); + // The reviewer steps of the software-factory preset run on Codex, so the + // verdict that matters most in a run arrives in this vocabulary. A parser + // unit test does not prove that `flows logs --step` reaches the renderer. + const CODEX = readFileSync(join(import.meta.dirname, 'fixtures/codex-exec-json.jsonl'), 'utf8'); + + it('renders a Codex step’s transcript, verdict and all', async () => { + wholeCloud({ transcript: CODEX }); + const out = io(); + expect(await runCloudLogsCli(parseLogsArgs([RUN, '--step', 'agent-2'])!, out.io, CONNECTION)).toBe(0); + const rendered = out.stdout.join('\n'); + expect(rendered).toContain('session codex · thread 01a0c600-0000-7a10-a68c-000000000000'); + expect(rendered).toContain(" tool 1 command_execution /bin/bash -lc 'cat src/pricing.ts'" + + ' → 110 chars · exit 0 · completed'); + expect(rendered).toContain(' add /project/review.md'); + expect(rendered).toContain(' One P2 remains and the gate artifact `review.clean` was not created.'); + expect(rendered).toContain('── turn complete · 72,669 in / 244 out · 69,376 cache read'); + // The old behaviour: every frame a placeholder and the verdict unreadable. + expect(rendered).not.toContain('not rendered here — see --raw'); + }); + + it('gives a Codex step’s entries to --json, and the JSONL to --raw', async () => { + for (const argv of [[RUN, '--step', 'agent-2', '--json'], [RUN, '--step', 'agent-2', '--raw'], + [RUN, '--step', 'agent-2', '--raw', '--json']]) { + wholeCloud({ transcript: CODEX }); + const out = io(); + expect(await runCloudLogsCli(parseLogsArgs(argv)!, out.io, CONNECTION)).toBe(0); + expect(out.stdout.join('\n')).toContain('review.clean'); + } + wholeCloud({ transcript: CODEX }); + const json = io(); + expect(await runCloudLogsCli(parseLogsArgs([RUN, '--step', 'agent-2', '--json'])!, json.io, CONNECTION)).toBe(0); + const body = JSON.parse(json.stdout[0]!) as { entries: Array> }; + expect(body.entries[0]).toEqual({ kind: 'thread', thread_id: '01a0c600-0000-7a10-a68c-000000000000' }); + const call = body.entries.find((entry) => entry['kind'] === 'tool')!; + expect(call['codex']).toMatchObject({ seq: 1, status: 'completed', exit_code: 0, complete: true }); + }); + it('propagates a step-list failure instead of calling it a missing transcript', async () => { // The empty log body alone cannot tell an unknown step from a step that // has written nothing; the step list is the evidence for either. When that diff --git a/packages/sdk/tests/cloud-transcript-codex.test.ts b/packages/sdk/tests/cloud-transcript-codex.test.ts new file mode 100644 index 000000000..688b41acc --- /dev/null +++ b/packages/sdk/tests/cloud-transcript-codex.test.ts @@ -0,0 +1,478 @@ +// Rendering a Codex transcript in `flows logs --step `. +// +// Provenance. Every frame shape asserted here was emitted by a real +// `codex exec --json` run of codex-cli 0.155.1 on 2026-09-21, captured on this +// machine. The two fixtures are those captures: +// +// fixtures/codex-exec-json.jsonl one run, verbatim except that the +// `thread_id` and the `/tmp/...` working directory were replaced by stable +// placeholders. A reviewer-shaped run: a shell command, an MCP tool call, +// an `apply_patch`, and a multi-line verdict as its final message. +// fixtures/codex-exec-json-failures.jsonl frames selected from four further +// captured runs -- a `reasoning` item, a command that exited 1, an MCP +// call the server refused, and a run against an unknown model, which is +// what produced the `error` item, the top-level `error` frame and +// `turn.failed`. Item ids were renumbered so the concatenation reads as +// one thread; nothing else was edited. +// +// The JSONL built inline below is synthetic. It exercises lifecycle, bound, +// fallback and redaction branches that a captured run does not reach; the item +// and event names it uses are the ones the codex-cli binary's own enums carry. +// `item.updated` in particular is in that enum and was not observed in any +// capture, so its handling is pinned here and labelled for what it is. + +import { readFileSync } from 'node:fs'; +import { join } from 'node:path'; +import { describe, expect, it } from 'vitest'; +import { parseAgentTranscript, renderAgentTranscript, type TranscriptEntry } from '../src/cloud-transcript.js'; + +const FIXTURES = join(import.meta.dirname, 'fixtures'); + +function fixture(name: string): string { + return readFileSync(join(FIXTURES, name), 'utf8'); +} + +/** The env is always explicit, so no assertion depends on this machine. */ +function parse(jsonl: string, env: NodeJS.ProcessEnv = {}): TranscriptEntry[] { + return parseAgentTranscript(jsonl, env).entries; +} + +function render(jsonl: string, env: NodeJS.ProcessEnv = {}): string { + return renderAgentTranscript(parseAgentTranscript(jsonl, env)).join('\n'); +} + +function frames(...lines: object[]): string { + return `${lines.map((line) => JSON.stringify(line)).join('\n')}\n`; +} + +function command(id: string, overrides: Record = {}): Record { + return { id, type: 'command_execution', command: 'pytest -q', ...overrides }; +} + +describe('the captured transcripts', () => { + const REVIEW = fixture('codex-exec-json.jsonl'); + + it('renders the reviewer run: thread, turn, numbered calls, file change, usage', () => { + const rendered = render(REVIEW); + expect(rendered).toContain('session codex · thread 01a0c600-0000-7a10-a68c-000000000000'); + expect(rendered).toContain('── turn ─'); + expect(rendered).toContain(" tool 1 command_execution /bin/bash -lc 'cat src/pricing.ts'" + + ' → 110 chars · exit 0 · completed'); + expect(rendered).toContain(' tool 2 mcp_tool_call demo/echo_shout {"text":"p2","options":{"mode":"loud"}}' + + ' → 29 chars · completed'); + expect(rendered).toContain(' files 1 change · completed'); + expect(rendered).toContain(' add /project/review.md'); + expect(rendered).toContain('── turn complete · 72,669 in / 244 out · 69,376 cache read' + + ' · 0 cache write · 0 reasoning out'); + }); + + it('prints the command output as a bounded excerpt, not only its size', () => { + const rendered = render(REVIEW); + expect(rendered).toContain(' export function total(cents: number, taxRate: number): number {'); + expect(rendered).toContain(' return Math.round(cents * (1 + taxRate));'); + }); + + it('prints the final agent message in full — it is the step’s answer', () => { + const rendered = render(REVIEW); + expect(rendered).toContain([ + 'assistant:', + ' Verdict: changes_requested', + ' ', + ' The rounding in `total` truncates before the tax is applied, so a', + ' 0.5-cent remainder is lost on every line item.', + ' ', + ' One P2 remains and the gate artifact `review.clean` was not created.', + ].join('\n')); + }); + + it('detects a Codex log with no relayflow.attempt wrapper as a transcript', () => { + const parsed = parseAgentTranscript(REVIEW, {}); + expect(parsed.stream_json).toBe(true); + expect(parsed.truncated_attempts).toBe(0); + expect(parsed.omitted_attempts).toBe(0); + }); + + it('renders the failure run: reasoning as a count, failures keeping size and exit', () => { + const rendered = render(fixture('codex-exec-json-failures.jsonl')); + expect(rendered).toContain(' thinking 19 chars (not shown)'); + expect(rendered).not.toContain('Reviewing notes'); + expect(rendered).toContain(" tool 1 command_execution /bin/bash -lc 'cat no-such-file.txt'" + + ' → 49 chars · exit 1 · failed'); + expect(rendered).toContain(' cat: no-such-file.txt: No such file or directory'); + expect(rendered).toContain(' tool 2 mcp_tool_call demo/echo_shout {"text":"boom"} → no result · failed'); + expect(rendered).toContain(' error tool call error: tool call failed for `demo/echo_shout`'); + expect(rendered).toContain('error (item):'); + expect(rendered).toContain(' Model metadata for `no-such-model-xyz` not found.'); + expect(rendered).toContain('── turn failed ─'); + expect(rendered).toContain('error (turn.failed):'); + expect(rendered).toContain("The 'no-such-model-xyz' model is not supported"); + }); +}); + +describe('item lifecycles', () => { + it('reports a started/completed pair once, at its completion', () => { + const rendered = render(frames( + { type: 'item.started', item: command('i1', { aggregated_output: '', exit_code: null, status: 'in_progress' }) }, + { type: 'item.completed', item: command('i1', { aggregated_output: 'ok\n', exit_code: 0, status: 'completed' }) }, + { type: 'item.completed', item: { id: 'i2', type: 'agent_message', text: 'done' } }, + )); + expect(rendered.match(/tool 1 {2}command_execution/gu)).toHaveLength(1); + expect(rendered).toBe([ + ' tool 1 command_execution pytest -q → 3 chars · exit 0 · completed', + ' ok', + 'assistant:', + ' done', + ].join('\n')); + }); + + it('folds started/updated/completed into one call', () => { + // `item.updated` is in codex-cli's event enum and was not observed in any + // capture on this machine; this frame is synthetic. + const rendered = render(frames( + { type: 'item.started', item: command('i1', { aggregated_output: '', status: 'in_progress' }) }, + { type: 'item.updated', item: command('i1', { aggregated_output: 'partial', status: 'in_progress' }) }, + { type: 'item.completed', item: command('i1', { aggregated_output: 'whole', exit_code: 0, status: 'completed' }) }, + )); + expect(rendered.match(/tool \d {2}command_execution/gu)).toHaveLength(1); + expect(rendered).toContain('→ 5 chars · exit 0 · completed'); + expect(rendered).not.toContain('partial'); + }); + + it('keeps an unfinished call, at its last snapshot, and says it never completed', () => { + const rendered = render(frames( + { type: 'item.started', item: command('i1', { aggregated_output: '', status: 'in_progress' }) }, + { type: 'item.updated', item: command('i1', { aggregated_output: 'half', status: 'in_progress' }) }, + { type: 'item.completed', item: { id: 'i2', type: 'agent_message', text: 'after' } }, + )); + expect(rendered).toBe([ + ' tool 1 command_execution pytest -q → 4 chars · in_progress · no completion frame', + ' half', + 'assistant:', + ' after', + ].join('\n')); + }); + + it('renders a completion with no start, which is what a head-cut log looks like', () => { + const rendered = render(frames( + { type: 'item.completed', item: command('i1', { aggregated_output: 'ok', exit_code: 0, status: 'completed' }) }, + )); + expect(rendered).toBe(' tool 1 command_execution pytest -q → 2 chars · exit 0 · completed\n ok'); + }); + + it('never correlates items with no id', () => { + const rendered = render(frames( + { type: 'item.started', item: { type: 'command_execution', command: 'a', status: 'in_progress' } }, + { type: 'item.completed', item: { type: 'command_execution', command: 'a', exit_code: 0, status: 'completed' } }, + )); + expect(rendered.match(/command_execution/gu)).toHaveLength(2); + expect(rendered).toContain(' tool 1 command_execution a → no result · in_progress · no completion frame'); + expect(rendered).toContain(' tool 2 command_execution a → no result · exit 0 · completed'); + }); + + it('never correlates two item types that merely share an id', () => { + const rendered = render(frames( + { type: 'item.started', item: command('i1', { status: 'in_progress' }) }, + { type: 'item.completed', item: { id: 'i1', type: 'mcp_tool_call', server: 's', tool: 't', status: 'completed' } }, + )); + expect(rendered).toContain(' tool 1 command_execution pytest -q → no result · in_progress · no completion frame'); + expect(rendered).toContain(' tool 2 mcp_tool_call s/t → no result · completed'); + }); + + it('starts a new call when a start follows a completion of the same id', () => { + const rendered = render(frames( + { type: 'item.completed', item: command('i1', { aggregated_output: 'one', exit_code: 0, status: 'completed' }) }, + { type: 'item.started', item: command('i1', { aggregated_output: '', status: 'in_progress' }) }, + )); + expect(rendered).toContain(' tool 1 command_execution pytest -q → 3 chars · exit 0 · completed'); + expect(rendered).toContain(' tool 2 command_execution pytest -q → 0 chars · in_progress · no completion frame'); + }); + + it('keeps attempt 1’s unfinished call when attempt 2 completes the same id, and renumbers', () => { + const rendered = render(frames( + { type: 'relayflow.attempt', attempt: 1, bytes: 90, truncated: false }, + { type: 'item.started', item: command('i1', { aggregated_output: '', status: 'in_progress' }) }, + { type: 'relayflow.attempt', attempt: 2, bytes: 90, truncated: false }, + { type: 'item.completed', item: command('i1', { aggregated_output: 'ok', exit_code: 0, status: 'completed' }) }, + )); + expect(rendered).toBe([ + '── attempt 1 · 90 bytes ────────────────────────────────────────────────', + ' tool 1 command_execution pytest -q → 0 chars · in_progress · no completion frame', + '── attempt 2 · 90 bytes ────────────────────────────────────────────────', + ' tool 1 command_execution pytest -q → 2 chars · exit 0 · completed', + ' ok', + ].join('\n')); + }); + + it('does not correlate across a turn or a thread boundary', () => { + const rendered = render(frames( + { type: 'item.started', item: command('i1', { status: 'in_progress' }) }, + { type: 'turn.completed' }, + { type: 'item.completed', item: command('i1', { aggregated_output: 'ok', exit_code: 0, status: 'completed' }) }, + { type: 'thread.started', thread_id: 'th-2' }, + { type: 'item.completed', item: command('i1', { aggregated_output: 'ok', exit_code: 0, status: 'completed' }) }, + )); + expect(rendered.match(/tool \d {2}command_execution/gu)).toHaveLength(3); + // Numbering runs on across turns and threads; only an attempt resets it. + expect(rendered).toContain(' tool 3 command_execution'); + }); +}); + +describe('commands and MCP calls', () => { + it('tells an empty output apart from a missing one', () => { + const rendered = render(frames( + { type: 'item.completed', item: command('i1', { aggregated_output: '', exit_code: 0, status: 'completed' }) }, + { type: 'item.completed', item: command('i2', { exit_code: 0, status: 'completed' }) }, + )); + expect(rendered).toContain(' tool 1 command_execution pytest -q → 0 chars · exit 0 · completed'); + expect(rendered).toContain(' tool 2 command_execution pytest -q → no result · exit 0 · completed'); + }); + + it('keeps exit 0 visible, and a failed status with no exit code', () => { + const rendered = render(frames( + { type: 'item.completed', item: command('i1', { aggregated_output: 'x', exit_code: 0, status: 'completed' }) }, + { type: 'item.completed', item: command('i2', { aggregated_output: 'x', exit_code: null, status: 'failed' }) }, + )); + expect(rendered).toContain('→ 1 chars · exit 0 · completed'); + expect(rendered).toContain(' tool 2 command_execution pytest -q → 1 chars · failed'); + }); + + it('bounds an oversized output to ten lines and keeps the full size on the line', () => { + const output = `${Array.from({ length: 40 }, (_, index) => `line ${index}`).join('\n')}\n`; + const rendered = render(frames( + { type: 'item.completed', item: command('i1', { aggregated_output: output, exit_code: 0, status: 'completed' }) }, + )); + expect(rendered).toContain(`→ ${output.length} chars · exit 0 · completed`); + expect(rendered).toContain(' line 9'); + expect(rendered).not.toContain(' line 10'); + expect(rendered).toContain(' … excerpt cut at 10 lines / 1,000 chars — see --raw'); + }); + + it('bounds an oversized single line of output by characters', () => { + const output = 'z'.repeat(4000); + const rendered = render(frames( + { type: 'item.completed', item: command('i1', { aggregated_output: output, exit_code: 0, status: 'completed' }) }, + )); + expect(rendered).toContain('→ 4,000 chars · exit 0 · completed'); + expect(rendered).toContain(` ${'z'.repeat(1000)}`); + expect(rendered).not.toContain('z'.repeat(1001)); + expect(rendered).toContain('… excerpt cut at 10 lines / 1,000 chars — see --raw'); + }); + + it('serializes nested MCP arguments and sizes a structured-only result', () => { + const rendered = render(frames( + { type: 'item.completed', item: { + id: 'i1', type: 'mcp_tool_call', server: 'demo', tool: 'shout', + arguments: { text: 'hi', options: { mode: 'loud', retries: 2 } }, + result: { structured_content: { shouted: 'HI' } }, error: null, status: 'completed', + } }, + )); + expect(rendered).toBe(' tool 1 mcp_tool_call demo/shout' + + ' {"text":"hi","options":{"mode":"loud","retries":2}} → 16 chars · completed'); + }); + + it('keeps a failed MCP call’s result size next to its error', () => { + const rendered = render(frames( + { type: 'item.completed', item: { + id: 'i1', type: 'mcp_tool_call', server: 'demo', tool: 'shout', arguments: {}, + result: { content: [{ type: 'text', text: 'partial output' }] }, + error: { message: 'server refused' }, status: 'failed', + } }, + )); + expect(rendered).toContain(' tool 1 mcp_tool_call demo/shout {} → 14 chars · failed'); + expect(rendered).toContain(' error server refused'); + }); +}); + +describe('fallbacks', () => { + it('names the item type in the placeholder, on every lifecycle event', () => { + const rendered = render(frames( + { type: 'item.started', item: { id: 'i1', type: 'todo_list', items: [] } }, + { type: 'item.updated', item: { id: 'i1', type: 'todo_list', items: [] } }, + { type: 'item.completed', item: { id: 'i1', type: 'web_search', query: 'relayflow' } }, + )); + expect(rendered).toContain(' frame item.started/todo_list ('); + expect(rendered).toContain(' frame item.updated/todo_list ('); + expect(rendered).toContain(' frame item.completed/web_search ('); + expect(rendered).not.toContain('relayflow'); + }); + + it('falls back rather than inventing success for a malformed known item', () => { + const rendered = render(frames( + { type: 'item.completed', item: { id: 'i1', type: 'command_execution', exit_code: 0, status: 'completed' } }, + { type: 'item.completed', item: { id: 'i2', type: 'agent_message' } }, + { type: 'item.completed', item: { id: 'i3', type: 'reasoning', text: 42 } }, + { type: 'item.completed', item: { id: 'i4', type: 'file_change', changes: 'nope' } }, + { type: 'item.completed', item: { id: 'i5', type: 'mcp_tool_call', status: 'completed' } }, + { type: 'item.completed', item: 'not an object' }, + )); + for (const label of ['item.completed/command_execution', 'item.completed/agent_message', + 'item.completed/reasoning', 'item.completed/file_change', 'item.completed/mcp_tool_call']) { + expect(rendered, label).toContain(` frame ${label} (`); + } + expect(rendered).toContain(' frame item.completed ('); + expect(rendered).not.toContain('→ '); + // A malformed item must never spend a call number. + expect(rendered).not.toContain('tool 1'); + }); + + it('counts malformed file changes instead of dropping them', () => { + const rendered = render(frames( + { type: 'item.completed', item: { id: 'i1', type: 'file_change', status: 'completed', changes: [ + { path: '/project/a.ts', kind: 'update' }, 'nope', { kind: 'delete' }, {}, + ] } }, + )); + expect(rendered).toBe([ + ' files 4 changes · completed', + ' update /project/a.ts', + ' delete (no path)', + ' 2 changes not rendered here — see --raw', + ].join('\n')); + }); + + it('says an unfinished file change never completed', () => { + const rendered = render(frames( + { type: 'item.started', item: { id: 'i1', type: 'file_change', status: 'in_progress', changes: [ + { path: '/project/a.ts', kind: 'add' }, + ] } }, + )); + expect(rendered).toContain(' files 1 change · in_progress · no completion frame'); + }); + + it('keeps non-JSON, non-object and cut lines visible', () => { + const jsonl = `${frames({ type: 'turn.started' }).trimEnd()}\n` + + '"just a string"\n[1,2,3]\nnot json at all\n{"type":"item.completed","item":{"id":"i1"\n'; + const rendered = render(jsonl); + expect(rendered).toContain(' unparsed not json at all'); + expect(rendered).toContain(' unparsed "just a string"'); + expect(rendered).toContain(' unparsed [1,2,3]'); + expect(rendered).toContain(' unparsed {"type":"item.completed","item":{"id":"i1"'); + }); + + it('marks a turn.failed with no readable message as a failure anyway', () => { + const rendered = render(frames({ type: 'turn.failed' })); + expect(rendered).toBe('── turn failed ─────────────────────────────────────────────────────────\n' + + 'error (turn.failed):\n (no message)'); + }); +}); + +describe('redaction', () => { + it('scrubs a token shape out of every string it renders', () => { + const jsonl = frames( + { type: 'thread.started', thread_id: 'rk_live_THREADLEAK1' }, + { type: 'item.completed', item: { id: 'i1', type: 'agent_message', text: 'exported rk_live_MESSAGELEAK1' } }, + { type: 'item.completed', item: { id: 'i2', type: 'command_execution', command: 'echo rk_live_COMMANDLEAK1', + aggregated_output: 'printed rk_live_OUTPUTLEAK1', exit_code: 1, status: 'rk_live_STATUSLEAK1' } }, + { type: 'item.completed', item: { id: 'i3', type: 'mcp_tool_call', server: 'rk_live_SERVERLEAK1', + tool: 'rk_live_TOOLLEAK1', arguments: { note: 'rk_live_ARGLEAK1' }, result: null, + error: { message: 'rk_live_ERRORLEAK1' }, status: 'failed' } }, + { type: 'item.completed', item: { id: 'i4', type: 'file_change', status: 'completed', + changes: [{ path: '/project/rk_live_PATHLEAK1.ts', kind: 'rk_live_KINDLEAK1' }] } }, + { type: 'item.completed', item: { id: 'i5', type: 'rk_live_ITEMTYPELEAK1' } }, + { type: 'item.completed', item: { id: 'i6', type: 'error', message: 'rk_live_ITEMERRORLEAK1' } }, + { type: 'error', message: 'rk_live_FRAMEERRORLEAK1' }, + { type: 'turn.failed', error: { message: 'rk_live_TURNERRORLEAK1' } }, + ); + const leaks = ['THREADLEAK1', 'MESSAGELEAK1', 'COMMANDLEAK1', 'OUTPUTLEAK1', 'STATUSLEAK1', 'SERVERLEAK1', + 'TOOLLEAK1', 'ARGLEAK1', 'ERRORLEAK1', 'PATHLEAK1', 'KINDLEAK1', 'ITEMTYPELEAK1', 'ITEMERRORLEAK1', + 'FRAMEERRORLEAK1', 'TURNERRORLEAK1']; + // Both the rendered page and the `--json` entries, which bypass the renderer. + for (const text of [render(jsonl), JSON.stringify(parse(jsonl))]) { + for (const leak of leaks) expect(text, `${leak} reached the page`).not.toContain(leak); + expect(text).toContain('[redacted]'); + } + }); + + it('redacts a credential field nested in MCP arguments', () => { + const jsonl = frames({ type: 'item.completed', item: { + id: 'i1', type: 'mcp_tool_call', server: 'demo', tool: 'deploy', + arguments: { config: { apiKey: 'opaque-value-nobody-exported' } }, result: null, error: null, + status: 'completed', + } }); + const rendered = render(jsonl); + expect(rendered).not.toContain('opaque-value-nobody-exported'); + expect(rendered).toContain('"apiKey":"[redacted]"'); + }); + + it('redacts a secret longer than each display cap before bounding it', () => { + // `redact` matches an env value whole. Bounding first would leave the + // first 160 (or 200, or 1,000) characters of the secret on the page. + const secret = `secret-${'x'.repeat(1200)}-tail`; + const env = { DEPLOY_TOKEN: secret }; + const jsonl = frames( + { type: 'item.completed', item: { id: 'i1', type: 'command_execution', command: `deploy --key ${secret}`, + aggregated_output: `sent ${secret}`, exit_code: 0, status: 'completed' } }, + { type: 'item.completed', item: { id: 'i2', type: 'file_change', status: 'completed', + changes: [{ path: `/project/${secret}.ts`, kind: 'add' }] } }, + { type: 'turn.failed', error: { message: `failed with ${secret}` } }, + ); + for (const text of [render(jsonl, env), JSON.stringify(parse(jsonl, env))]) { + expect(text).toContain('[redacted:DEPLOY_TOKEN]'); + expect(text).not.toContain(secret.slice(0, 160)); + } + }); + + it('replaces terminal control characters in every new rendering branch', () => { + const bell = '\u0007'; + const rendered = render(frames( + { type: 'thread.started', thread_id: `th${bell}1` }, + { type: 'item.completed', item: { id: 'i1', type: 'command_execution', command: `echo${bell}`, + aggregated_output: `out${bell}`, exit_code: 0, status: `done${bell}` } }, + { type: 'item.completed', item: { id: 'i2', type: 'file_change', status: 'completed', + changes: [{ path: `/p${bell}.ts`, kind: `add${bell}` }] } }, + { type: 'item.completed', item: { id: 'i3', type: `weird${bell}` } }, + { type: 'error', message: `boom${bell}` }, + )); + expect(rendered).not.toContain(bell); + expect(rendered).toContain('session codex · thread th?1'); + expect(rendered).toContain('echo?'); + expect(rendered).toContain(' out?'); + expect(rendered).toContain(' add? /p?.ts'); + expect(rendered).toContain('weird?'); + expect(rendered).toContain(' boom?'); + }); +}); + +describe('the Claude vocabulary is untouched', () => { + const CLAUDE = frames( + { type: 'relayflow.attempt', attempt: 1, bytes: 400, truncated: false }, + { type: 'system', subtype: 'init', model: 'claude-opus-5', claude_code_version: '2.1.19', + permissionMode: 'bypassPermissions', session_id: '4076fae9', tools_count: 18, mcp_servers_count: 0 }, + { type: 'assistant', message: { role: 'assistant', content: [ + { type: 'tool_use', id: 'toolu_1', name: 'Read', input: { file_path: '/project/notes.txt' } }, + ] } }, + { type: 'user', message: { role: 'user', content: [ + { tool_use_id: 'toolu_1', type: 'tool_result', content: 'alpha', is_error: true }, + ] } }, + { type: 'result', subtype: 'success', is_error: false, num_turns: 2 }, + ); + + it('renders unnumbered Claude tool lines and carries no Codex fields', () => { + expect(render(CLAUDE)).toContain(' tool Read /project/notes.txt → ERROR'); + expect(JSON.stringify(parse(CLAUDE))).not.toContain('codex'); + expect(parse(CLAUDE).filter((entry) => entry.kind === 'tool')).toEqual([ + { kind: 'tool', name: 'Read', target: '/project/notes.txt', result_chars: 5, is_error: true, nested: false }, + ]); + }); + + it('renders a log that mixes both vocabularies, each in its own shape', () => { + const rendered = render(CLAUDE + frames( + { type: 'item.completed', item: command('i1', { aggregated_output: 'ok', exit_code: 0, status: 'completed' }) }, + )); + expect(rendered).toContain(' tool Read /project/notes.txt → ERROR'); + expect(rendered).toContain(' tool 1 command_execution pytest -q → 2 chars · exit 0 · completed'); + }); + + it('still counts truncated and omitted attempts around Codex frames', () => { + const parsed = parseAgentTranscript(frames( + { type: 'relayflow.attempt.omitted', attempt: 1, bytes: 1048576 }, + { type: 'relayflow.attempt', attempt: 2, bytes: 900, truncated: true }, + { type: 'item.completed', item: command('i1', { aggregated_output: 'ok', exit_code: 0, status: 'completed' }) }, + ), {}); + expect(parsed.truncated_attempts).toBe(1); + expect(parsed.omitted_attempts).toBe(1); + }); + + it('reports a log in neither vocabulary as not a transcript', () => { + expect(parseAgentTranscript('plain terminal output\n', {}).stream_json).toBe(false); + }); +}); diff --git a/packages/sdk/tests/fixtures/codex-exec-json-failures.jsonl b/packages/sdk/tests/fixtures/codex-exec-json-failures.jsonl new file mode 100644 index 000000000..11d950401 --- /dev/null +++ b/packages/sdk/tests/fixtures/codex-exec-json-failures.jsonl @@ -0,0 +1,10 @@ +{"type":"thread.started","thread_id":"01a0c5ff-0000-7800-9fce-000000000000"} +{"type":"turn.started"} +{"type":"item.completed","item":{"id":"item_0","type":"reasoning","text":"**Reviewing notes**"}} +{"type":"item.started","item":{"id":"item_1","type":"command_execution","command":"/bin/bash -lc 'cat no-such-file.txt'","aggregated_output":"","exit_code":null,"status":"in_progress"}} +{"type":"item.completed","item":{"id":"item_1","type":"command_execution","command":"/bin/bash -lc 'cat no-such-file.txt'","aggregated_output":"cat: no-such-file.txt: No such file or directory\n","exit_code":1,"status":"failed"}} +{"type":"item.started","item":{"id":"item_2","type":"mcp_tool_call","server":"demo","tool":"echo_shout","arguments":{"text":"boom"},"result":null,"error":null,"status":"in_progress"}} +{"type":"item.completed","item":{"id":"item_2","type":"mcp_tool_call","server":"demo","tool":"echo_shout","arguments":{"text":"boom"},"result":null,"error":{"message":"tool call error: tool call failed for `demo/echo_shout`\n\nCaused by:\n Mcp error: -32000: echo_shout refused: boom is not allowed"},"status":"failed"}} +{"type":"item.completed","item":{"id":"item_3","type":"error","message":"Model metadata for `no-such-model-xyz` not found. Defaulting to fallback metadata; this can degrade performance and cause issues."}} +{"type":"error","message":"{\"type\":\"error\",\"status\":400,\"error\":{\"type\":\"invalid_request_error\",\"message\":\"The 'no-such-model-xyz' model is not supported when using Codex with a ChatGPT account.\"}}"} +{"type":"turn.failed","error":{"message":"{\"type\":\"error\",\"status\":400,\"error\":{\"type\":\"invalid_request_error\",\"message\":\"The 'no-such-model-xyz' model is not supported when using Codex with a ChatGPT account.\"}}"}} diff --git a/packages/sdk/tests/fixtures/codex-exec-json.jsonl b/packages/sdk/tests/fixtures/codex-exec-json.jsonl new file mode 100644 index 000000000..2c37c4663 --- /dev/null +++ b/packages/sdk/tests/fixtures/codex-exec-json.jsonl @@ -0,0 +1,11 @@ +{"type":"thread.started","thread_id":"01a0c600-0000-7a10-a68c-000000000000"} +{"type":"turn.started"} +{"type":"item.completed","item":{"id":"item_0","type":"agent_message","text":"I’ll perform the requested review steps in order.\n"}} +{"type":"item.started","item":{"id":"item_1","type":"command_execution","command":"/bin/bash -lc 'cat src/pricing.ts'","aggregated_output":"","exit_code":null,"status":"in_progress"}} +{"type":"item.completed","item":{"id":"item_1","type":"command_execution","command":"/bin/bash -lc 'cat src/pricing.ts'","aggregated_output":"export function total(cents: number, taxRate: number): number {\n return Math.round(cents * (1 + taxRate));\n}\n","exit_code":0,"status":"completed"}} +{"type":"item.started","item":{"id":"item_2","type":"mcp_tool_call","server":"demo","tool":"echo_shout","arguments":{"text":"p2","options":{"mode":"loud"}},"result":null,"error":null,"status":"in_progress"}} +{"type":"item.completed","item":{"id":"item_2","type":"mcp_tool_call","server":"demo","tool":"echo_shout","arguments":{"text":"p2","options":{"mode":"loud"}},"result":{"content":[{"type":"text","text":"P2"}],"structured_content":{"shouted":"P2","length":2}},"error":null,"status":"completed"}} +{"type":"item.started","item":{"id":"item_3","type":"file_change","changes":[{"path":"/project/review.md","kind":"add"}],"status":"in_progress"}} +{"type":"item.completed","item":{"id":"item_3","type":"file_change","changes":[{"path":"/project/review.md","kind":"add"}],"status":"completed"}} +{"type":"item.completed","item":{"id":"item_4","type":"agent_message","text":"Verdict: changes_requested\n\nThe rounding in `total` truncates before the tax is applied, so a\n0.5-cent remainder is lost on every line item.\n\nOne P2 remains and the gate artifact `review.clean` was not created."}} +{"type":"turn.completed","usage":{"input_tokens":72669,"cached_input_tokens":69376,"cache_write_input_tokens":0,"output_tokens":244,"reasoning_output_tokens":0}} From 349eeb69e34632cbf34b847571a44d82bf67ae2d Mon Sep 17 00:00:00 2001 From: Relayflow Date: Mon, 21 Sep 2026 22:38:10 +0000 Subject: [PATCH 2/3] flows logs: redact MCP arguments before JSON escaping hides them `mcpEntry` serialized an MCP call's arguments and redacted the serialization. `redact` matches an environment value literally, and `JSON.stringify` escapes a quote, backslash, newline or tab inside one -- so a secret carrying any of them no longer matched its own value once serialized, and reached the tool line and the `--json` entry escaped but complete. The credential-field-by-name rule does not cover it: the value sits in an ordinary field such as `text` or `content`. Rendering MCP arguments is what newly exposed this; the frame was a placeholder before. Arguments are now redacted as decoded leaves -- values and keys, at every depth, through arrays and a bare string argument -- and the serialized form is still redacted again by the `bounded` call, which is what the credential-field rule needs, since a leaf standing alone has no field name left to recognise. The line is bounded only after both passes. Co-Authored-By: Claude Opus 5 --- packages/sdk/src/cloud-transcript-codex.ts | 29 +++++++++++++++-- .../sdk/tests/cloud-transcript-codex.test.ts | 31 +++++++++++++++++++ 2 files changed, 57 insertions(+), 3 deletions(-) diff --git a/packages/sdk/src/cloud-transcript-codex.ts b/packages/sdk/src/cloud-transcript-codex.ts index c989e3312..21a212ae8 100644 --- a/packages/sdk/src/cloud-transcript-codex.ts +++ b/packages/sdk/src/cloud-transcript-codex.ts @@ -173,15 +173,38 @@ function errorText(value: unknown): string | null { return isRecord(value) ? str(value['message']) : null; } +/** + * Redact every string an argument value carries, while each one is still the + * string the provider decoded. + * + * `redact` matches an environment value literally, and `JSON.stringify` escapes + * a quote, backslash, newline or tab inside one -- so a secret containing any + * of them no longer matches its own value once serialized, and reached the page + * escaped but complete. Keys are provider strings too: an argument object built + * out of an environment dump carries the value in the name. + */ +function redactLeaves(value: unknown, clean: (text: string) => string): unknown { + if (typeof value === 'string') return clean(value); + if (Array.isArray(value)) return value.map((element) => redactLeaves(element, clean)); + if (!isRecord(value)) return value; + const out: Record = {}; + for (const [key, element] of Object.entries(value)) out[clean(key)] = redactLeaves(element, clean); + return out; +} + function mcpEntry(item: Record, complete: boolean, context: Context, seq: number): TranscriptEntry | null { const server = str(item['server']); const name = str(item['tool']); if (server === null && name === null) return null; const args = item['arguments']; - // Serialized with its field names intact: `redact` recognises a credential - // field by its name, so flattening the object first would hide it. - const rendered = args === undefined || args === null ? '' : ` ${JSON.stringify(args)}`; + // Redacted twice, bounded once, and in that order. The leaves go first, + // before serialization can escape a secret out of its own match; the + // serialized form is redacted again by the `bounded` call below, because + // `redact` recognises a credential field by its name and a leaf standing on + // its own has no name left to recognise. Only then is the line cut. + const rendered = args === undefined || args === null + ? '' : ` ${JSON.stringify(redactLeaves(args, context.clean))}`; const result = isRecord(item['result']) ? item['result'] : null; const structured = result === null ? null : result['structured_content']; const failure = errorText(item['error']); diff --git a/packages/sdk/tests/cloud-transcript-codex.test.ts b/packages/sdk/tests/cloud-transcript-codex.test.ts index 688b41acc..b3c86f89e 100644 --- a/packages/sdk/tests/cloud-transcript-codex.test.ts +++ b/packages/sdk/tests/cloud-transcript-codex.test.ts @@ -393,6 +393,37 @@ describe('redaction', () => { expect(rendered).toContain('"apiKey":"[redacted]"'); }); + it('redacts an MCP argument secret that JSON escaping would hide, at every depth', () => { + // `redact` matches an env value literally, and `JSON.stringify` escapes a + // quote, backslash, newline or tab inside it. Serializing before redacting + // left the whole credential on the line -- escaped, and recoverable by + // anyone who can call `JSON.parse`. The field names here are ordinary + // (`text`, `note`, `items`), so the credential-field rule cannot help. + const secret = 'opaque"review\\secret\nvalue\ttail'; + const env = { DEPLOY_TOKEN: secret }; + const escaped = JSON.stringify(secret).slice(1, -1); + const call = (id: string, args: unknown): Record => ({ + type: 'item.completed', + item: { id, type: 'mcp_tool_call', server: 'demo', tool: 'echo', arguments: args, + result: null, error: null, status: 'completed' }, + }); + const jsonl = frames( + call('i1', { text: secret, nested: { note: { deep: secret } } }), + call('i2', [secret, { items: [secret] }]), + call('i3', secret), + call('i4', { [secret]: 'in the name, not the value' }), + ); + // The rendered page and the `--json` entries, which bypass the renderer. + for (const text of [render(jsonl, env), JSON.stringify(parse(jsonl, env))]) { + expect(text, 'the secret reached the page').not.toContain(secret); + expect(text, 'the escaped secret reached the page').not.toContain(escaped); + expect(text).toContain('[redacted:DEPLOY_TOKEN]'); + } + expect(render(jsonl, env)).toContain(' tool 1 mcp_tool_call demo/echo' + + ' {"text":"[redacted:DEPLOY_TOKEN]","nested":{"note":{"deep":"[redacted:DEPLOY_TOKEN]"}}}' + + ' → no result · completed'); + }); + it('redacts a secret longer than each display cap before bounding it', () => { // `redact` matches an env value whole. Bounding first would leave the // first 160 (or 200, or 1,000) characters of the secret on the page. From 3f19fda9c0f97586eae9c24e42014dcb9683c76d Mon Sep 17 00:00:00 2001 From: Proactive Runtime Bot Date: Tue, 22 Sep 2026 12:11:21 -0700 Subject: [PATCH 3/3] fix(sdk): render MCP result excerpts instead of discarding them `mcpEntry` measured an mcp_tool_call's result and threw the text away, so a reader got a size and had to re-fetch with --raw. It now bounds and redacts the result exactly as `commandEntry` bounds command output: text content, the MCP content-block array, or a structured-only result serialized. Addresses the review finding on cloud-transcript-codex.ts:218. Co-Authored-By: Claude Opus 5 --- packages/sdk/src/cloud-transcript-codex.ts | 29 ++++++++- .../sdk/tests/cloud-transcript-codex.test.ts | 60 ++++++++++++++++++- 2 files changed, 86 insertions(+), 3 deletions(-) diff --git a/packages/sdk/src/cloud-transcript-codex.ts b/packages/sdk/src/cloud-transcript-codex.ts index 21a212ae8..b83b26957 100644 --- a/packages/sdk/src/cloud-transcript-codex.ts +++ b/packages/sdk/src/cloud-transcript-codex.ts @@ -118,6 +118,28 @@ function bounded(text: string, clean: (t: string) => string, limit = TARGET_MAX_ return oneLine(clean(text), limit); } +/** + * The textual content of an MCP result, as one string. `content` is a string + * or the MCP content-block array; a block that carries no text (an image, say) + * contributes nothing, exactly as `textChars` counts nothing for it. + */ +function mcpResultText(result: Record): string | null { + const content = result['content']; + if (typeof content === 'string') return content.length === 0 ? null : content; + const parts: string[] = []; + if (Array.isArray(content)) { + for (const block of content) { + if (!isRecord(block)) continue; + const text = str(block['text']); + if (text !== null && text.length > 0) parts.push(text); + } + } + if (parts.length > 0) return parts.join('\n'); + // A result can be structured only; showing it beats reporting a size alone. + const structured = result['structured_content']; + return structured === undefined || structured === null ? null : JSON.stringify(structured); +} + /** A result's size, never its content. */ function textChars(content: unknown): number { if (typeof content === 'string') return content.length; @@ -209,12 +231,17 @@ function mcpEntry(item: Record, complete: boolean, context: Con const structured = result === null ? null : result['structured_content']; const failure = errorText(item['error']); const status = str(item['status']); + // The result is evidence: bound and redact it exactly as command output is, + // instead of reporting a size the reader then has to fetch with `--raw`. + const text = result === null ? null : mcpResultText(result); + const cut = text === null ? null : excerpt(text, context.clean); return tool('mcp_tool_call', bounded(`${server ?? '?'}/${name ?? '?'}${rendered}`, context.clean), result === null ? null : textChars(result['content']) + (structured === undefined || structured === null ? 0 : JSON.stringify(structured).length), status === 'failed' || failure !== null, { seq, status: status === null ? null : bounded(status, context.clean, 40), - exit_code: null, output_excerpt: null, output_truncated: false, complete, + exit_code: null, output_excerpt: cut === null ? null : cut.text, + output_truncated: cut?.truncated === true, complete, error: failure === null ? null : bounded(failure, context.clean, ERROR_MAX_CHARS), }); } diff --git a/packages/sdk/tests/cloud-transcript-codex.test.ts b/packages/sdk/tests/cloud-transcript-codex.test.ts index b3c86f89e..fcd439a3c 100644 --- a/packages/sdk/tests/cloud-transcript-codex.test.ts +++ b/packages/sdk/tests/cloud-transcript-codex.test.ts @@ -258,7 +258,7 @@ describe('commands and MCP calls', () => { expect(rendered).toContain('… excerpt cut at 10 lines / 1,000 chars — see --raw'); }); - it('serializes nested MCP arguments and sizes a structured-only result', () => { + it('serializes nested MCP arguments and shows a structured-only result', () => { const rendered = render(frames( { type: 'item.completed', item: { id: 'i1', type: 'mcp_tool_call', server: 'demo', tool: 'shout', @@ -267,7 +267,62 @@ describe('commands and MCP calls', () => { } }, )); expect(rendered).toBe(' tool 1 mcp_tool_call demo/shout' - + ' {"text":"hi","options":{"mode":"loud","retries":2}} → 16 chars · completed'); + + ' {"text":"hi","options":{"mode":"loud","retries":2}} → 16 chars · completed' + + '\n {"shouted":"HI"}'); + }); + + it('shows an MCP result\u2019s text rather than only its size', () => { + const rendered = render(frames( + { type: 'item.completed', item: { + id: 'i1', type: 'mcp_tool_call', server: 'demo', tool: 'shout', arguments: {}, + result: { content: [{ type: 'text', text: 'first block' }, { type: 'text', text: 'second block' }] }, + error: null, status: 'completed', + } }, + )); + expect(rendered).toContain(' first block'); + expect(rendered).toContain(' second block'); + }); + + it('carries a string MCP result through, and skips a block with no text', () => { + const rendered = render(frames( + { type: 'item.completed', item: { + id: 'i1', type: 'mcp_tool_call', server: 'demo', tool: 'a', arguments: {}, + result: { content: 'plain string result' }, error: null, status: 'completed', + } }, + { type: 'item.completed', item: { + id: 'i2', type: 'mcp_tool_call', server: 'demo', tool: 'b', arguments: {}, + result: { content: [{ type: 'image', data: 'AAAA' }, { type: 'text', text: 'only text' }] }, + error: null, status: 'completed', + } }, + )); + expect(rendered).toContain(' plain string result'); + expect(rendered).toContain(' only text'); + expect(rendered).not.toContain('AAAA'); + }); + + it('bounds an oversized MCP result the same way command output is bounded', () => { + const text = `${Array.from({ length: 40 }, (_, index) => `mcp ${index}`).join('\n')}`; + const rendered = render(frames( + { type: 'item.completed', item: { + id: 'i1', type: 'mcp_tool_call', server: 'demo', tool: 'shout', arguments: {}, + result: { content: [{ type: 'text', text }] }, error: null, status: 'completed', + } }, + )); + expect(rendered).toContain(' mcp 9'); + expect(rendered).not.toContain(' mcp 10'); + expect(rendered).toContain('\u2026 excerpt cut at 10 lines / 1,000 chars \u2014 see --raw'); + }); + + it('redacts an MCP result exactly as it redacts command output', () => { + const rendered = render(frames( + { type: 'item.completed', item: { + id: 'i1', type: 'mcp_tool_call', server: 'demo', tool: 'shout', arguments: {}, + result: { content: [{ type: 'text', text: 'token sk-secret-value' }] }, + error: null, status: 'completed', + } }, + ), { DEPLOY_TOKEN: 'sk-secret-value' }); + expect(rendered).not.toContain('sk-secret-value'); + expect(rendered).toContain('[redacted:DEPLOY_TOKEN]'); }); it('keeps a failed MCP call’s result size next to its error', () => { @@ -279,6 +334,7 @@ describe('commands and MCP calls', () => { } }, )); expect(rendered).toContain(' tool 1 mcp_tool_call demo/shout {} → 14 chars · failed'); + expect(rendered).toContain(' partial output'); expect(rendered).toContain(' error server refused'); }); });