Skip to content

feat(headless): add --output=json to opencli run + JSON schema contract eval#239

Open
zjshen14 wants to merge 1 commit into
mainfrom
feat/issue-234-json-output
Open

feat(headless): add --output=json to opencli run + JSON schema contract eval#239
zjshen14 wants to merge 1 commit into
mainfrom
feat/issue-234-json-output

Conversation

@zjshen14

@zjshen14 zjshen14 commented Jun 9, 2026

Copy link
Copy Markdown
Owner

Summary

  • Adds --output <text|json> flag to opencli run; in json mode every AgentEvent is written as a newline-terminated JSON line (NDJSON) to stdout instead of the human-readable text rendering
  • Introduces src/cli/json-output.ts with the stable JsonOutputEvent type (the public schema) and toJsonLine() which converts any AgentEvent → JSON line, explicitly stripping the provider-internal thoughtSignature from tool_call events
  • Adds src/eval/headless/json-schema.test.ts — 33 contract-eval tests asserting parseable NDJSON, newline termination, required fields per event type, thoughtSignature exclusion, and exhaustive schema coverage across all 7 AgentEvent types

Related issue

Part of #234 (D2 follow-up: headless --output=json contract eval sub-item)

Test plan

  • npm run typecheck && npm run lint && npm run format:check && npm test — all pass (742 tests, 33 new)
  • toJsonLine() unit-tested for all 7 AgentEvent types
  • thoughtSignature stripping verified in contract eval
  • Schema coverage test enforces exhaustiveness — will fail if a new AgentEvent type is added without updating the eval

https://claude.ai/code/session_01TakF9J9FMjjQd6okE7kczu


Generated by Claude Code

…ct eval

Implements the headless --output=json sub-item of D2 (#234):
- src/cli/json-output.ts: defines JsonOutputEvent (stable public schema)
  and toJsonLine() which serialises every AgentEvent to a newline-terminated
  JSON line, stripping the provider-internal thoughtSignature from tool_call.
- src/cli/index.ts: adds --output <text|json> to opencli run; in json mode
  every AgentEvent is written as an NDJSON line to stdout instead of the
  human-readable text rendering.
- src/eval/headless/json-schema.test.ts: contract eval with 33 tests
  asserting parseable JSON, newline termination, required fields per event
  type, thoughtSignature exclusion, and exhaustive schema coverage.

Part of #234
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@zjshen14

zjshen14 commented Jun 9, 2026

Copy link
Copy Markdown
Owner Author

Review: feat(headless): add --output=json to opencli run + JSON schema contract eval

What's good: Clean, well-layered addition. json-output.ts correctly lives in cli/ as a presentation-layer concern and imports only from core/. The explicit object construction in toJsonLine's tool_call case ({type, name, args} — no spread) is the right way to strip thoughtSignature. The forward-compat default: return null guard is sensible. Tests are thorough: all 7 AgentEvent types covered, thoughtSignature stripping verified, NDJSON termination checked, and the exhaustiveness test will catch any future AgentEvent additions that miss a toJsonLine case. CI is green, mergeable_state is clean.


Notes

1. error routing differs between modes (index.ts:261)

Text mode writes Error: ... to process.stderr; JSON mode emits {"type":"error","message":"..."} to process.stdout. The asymmetry is intentional — structured output should keep all events on one stream — but it's non-obvious from reading the function. Consumers expecting stderr-based error detection across both modes will be surprised. No action required, but worth a note in public-facing docs when --output=json is documented.

2. text accumulation in json mode

if (event.type === "text") text += event.text runs in json mode, feeding stream()'s return value. Presumably this feeds session logging (which is still correct behavior in json mode). Not a bug — just noting that the return value is not dead in json mode.


Recommendation: Merge.


Generated by Claude Code

@zjshen14

Copy link
Copy Markdown
Owner Author

Review: feat(headless): add --output=json to opencli run

What's good: Clean, well-bounded feature. toJsonLine() is a pure function that maps all 7 AgentEvent types exhaustively, strips thoughtSignature (correctly — it's a provider-internal Gemini field with no public contract), and returns null for unknown types as a forward-compat guard. The JSON/text paths in runSingle() are clearly separated with no shared mutable state. The contract test suite is a standout: the exhaustiveness check (covers every AgentEvent type) will fail immediately if a new event type is added without updating the schema — exactly the right invariant to enforce.

Notes

1. --output validation could use Commander's .choices()

The manual if (output !== undefined && output !== "text" && output !== "json") check works, but .option("--output <format>", "...", ...).choices(["text", "json"]) (Commander v12+) gives consistent CLI error formatting. Minor; not a blocker.

2. error events route to stdout in JSON mode vs. stderr in text mode

In JSON mode all events — including error — go to stdout as NDJSON lines. This is the correct design for programmatic consumers (single parseable stream), but the current option description ("Output format: text | json (default: text)") doesn't hint at this. Consider extending it to "Output format: text (human-readable, errors → stderr) | json (NDJSON, all events → stdout)" so callers know not to filter stderr in JSON mode.

Recommendation: Merge. Note 2 is a docs gap, not a correctness issue. Both notes are optional improvements.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants