Skip to content

Harden parser + backend: structured status, turn attribution, Postgres CLI - #2

Merged
kkrlstrm merged 1 commit into
mainfrom
harden-parser-postgres-turns
Jul 8, 2026
Merged

kkrlstrm merged 1 commit into
mainfrom
harden-parser-postgres-turns

Conversation

@kkrlstrm

@kkrlstrm kkrlstrm commented Jul 6, 2026

Copy link
Copy Markdown
Owner

Acts on a round of technical review. I verified every claim against the real Codex rollout data in ~/.codex/sessions before changing anything — a few review points turned out to be raw-GitHub-view rendering artifacts, and the fixes for the real ones are grounded in what the rollouts actually contain.

Real bugs fixed

1. Postgres CLI was broken. sessions / inspect / stats hard-coded the SQLite table names (FROM sessions, FROM tool_calls), but the Postgres backend stores codex_* tables and query() only translated placeholders — never the table names. So every read command failed against Postgres (ingest worked, which is why it wasn't obvious). Fixed with a store.table() indirection the CLI uses to resolve the physical name per backend.

I deliberately did not use the reviewer's "create unprefixed PG views" option — an unprefixed sessions view reintroduces the exact name collision the codex_ prefix exists to avoid when sharing a warehouse with cc-logger. The table() method is the lighter version of their "backend methods" option.

2. Tool-call status was shell-centric. Success/failure was scraped from the string Process exited with code N, which non-shell tools don't emit — so on real data apply_patch was 9/9 unknown. The rollouts already carry authoritative outcomes the parser ignored: exec_command_end.exit_code and patch_apply_end.success. Consuming those:

tool before after (real data)
apply_patch 9 unknown 8 success, 1 failure
exec_command 27 unknown 7 unknown (rest resolved by exit_code)

The string scrape is now a fallback that never downgrades a structured result.

3. No turn-level attribution. Added turn_id to tool_calls and messages (parser + both schemas + upserts). 289/289 tool calls on real data now carry their turn, enabling "which prompt caused the failed patch / which subagent turn burned the most tokens."

4. launchd was non-portable. The committed plist hard-coded /usr/bin/python3 and a stale WorkingDirectory (kai-gtm-agents/codex-logger, which no longer exists). Replaced with a generated install-launchd command — writes a plist wired to the real interpreter + repo path, plutil-valid — plus a placeholder template in launchd/.

Docs tightened to match the code

  • Softened "captures everything"; the event table now states exactly what is and isn't resolved.
  • Fixed a docstring implying event_msg agent_message/user_message were parsed — they're streamed duplicates of response_item/message (which already captures all 164 assistant + 76 user messages on real data), so re-ingesting them would double-count.
  • Added a Privacy & security section (this stores prompts, tool output, and secrets that scroll through a terminal).
  • Noted the CLI now works against Postgres; kept the honest "no live-DB CI yet" caveat.

Verified as non-issues (not changed)

The reviewer read the GitHub raw view and several claims were rendering artifacts, confirmed against source: the Python files are normally formatted (not "one long physical line"), pyproject.toml parses as valid TOML, and the plist was valid XML.

Tests

Added coverage for status-resolved-from-*_end-events, per-turn attribution, and turn_id round-trip. Full suite green; also validated pip install -e ., the codex-logger console script, and install-launchd output (plutil -lint: OK).

🤖 Generated with Claude Code

…ostgres CLI

Acts on a technical review. Verified each claim against the real Codex
rollout data in ~/.codex/sessions before changing anything.

Real fixes:
- Postgres CLI was broken: sessions/inspect/stats hard-coded the SQLite
  table names (FROM sessions), but the PG backend uses codex_* tables and
  query() never rewrote them. Added a store.table() indirection so the CLI
  resolves the right physical name per backend. (Chose this over unprefixed
  PG views, which would reintroduce the exact name collision the codex_
  prefix exists to avoid in a shared cc-logger warehouse.)
- Tool-call status was shell-centric (scraped 'Process exited with code N'),
  leaving apply_patch 9/9 'unknown' on real data. Now resolved from the
  structured events Codex already emits: exec_command_end.exit_code and
  patch_apply_end.success. On real history this turns apply_patch into 8
  success / 1 failure and cuts exec_command unknowns 27 -> 7. The scrape is
  now a fallback that never downgrades a structured result.
- Added turn_id to tool_calls and messages (parser + both schemas + upserts),
  so every call/message is attributable to its turn (289/289 populated on
  real data). Enables turn-level questions: which prompt caused a failed
  patch, which subagent turn burned the most tokens.
- launchd: the committed plist hard-coded /usr/bin/python3 and a stale
  WorkingDirectory. Replaced with a generated install-launchd command
  (real interpreter + repo path, plutil-valid) plus a placeholder template.

Docs/claims tightened to match the code:
- Softened 'captures everything'; status/message rows now state exactly what
  is and isn't resolved. Fixed a docstring that implied event_msg messages
  were parsed (they're duplicates of response_item/message, which is already
  complete - 164 assistant + 76 user on real data).
- Added a Privacy & security section (this stores prompts/output/secrets).
- Noted the CLI now works against Postgres; kept the honest 'no live-DB CI
  yet' caveat.

Not changed (review points that were raw-GitHub-view rendering artifacts,
verified against source): files are normally formatted (not one long line),
pyproject parses as valid TOML, the plist was valid XML.

Tests: added coverage for status-from-*_end events, per-turn attribution,
and turn_id round-trip. Full suite green on 3.10-3.13.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@kkrlstrm
kkrlstrm merged commit b0c4821 into main Jul 8, 2026
4 checks passed
@kkrlstrm
kkrlstrm deleted the harden-parser-postgres-turns branch July 8, 2026 18:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant