Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -490,10 +490,10 @@ On Windows / PowerShell, the equivalent cache clear is `Remove-Item -Recurse -Fo

Every session agentmemory records is replayable. Open the viewer, pick the **Replay** tab, and scrub through the timeline: prompts, tool calls, tool results, and responses render as discrete events with play/pause, speed control (0.5×–4×), and keyboard shortcuts (space to toggle, arrows to step).

Already have older Claude Code JSONL transcripts you want to bring in?
Already have older Claude Code or Hermes Agent JSONL transcripts you want to bring in?

```bash
# Import everything under the default ~/.claude/projects
# Import everything under the default Claude Code ~/.claude/projects
npx @agentmemory/agentmemory import-jsonl

# Or import a single file
Expand Down
158 changes: 158 additions & 0 deletions docs/todos/2026-06-19-issue-297-import-hermes-jsonl/plan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,158 @@
# Hermes JSONL Import Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Import Hermes Agent JSONL transcripts through the existing `import-jsonl` replay path.

**Architecture:** Extend the existing JSONL parser with flat Hermes `role` branches and small local helpers. Keep the import function, CLI, REST endpoint, session model, and observation model unchanged.

**Tech Stack:** TypeScript ESM, Vitest, existing `src/replay/jsonl-parser.ts` and `src/functions/replay.ts` replay import harness.

---

## Sprint Contract

Goal: make `import-jsonl` import Hermes Agent JSONL transcripts into replay sessions using the existing replay observation model.

Scope: modify `src/replay/jsonl-parser.ts`, add Hermes fixtures, add parser/import tests, update Claude-only import copy, and update task-state notes.

Non-goals: no new CLI flags, endpoint changes, MCP tools, schema changes, migrations, dependencies, or Hermes live-hook/provider changes.

Acceptance criteria:
- Hermes user rows import as `prompt_submit`.
- Hermes assistant text imports as `stop`.
- Hermes assistant `tool_calls` import as `pre_tool_use`.
- Hermes tool rows import as `post_tool_use` or `post_tool_failure` when explicit failure markers are present.
- Valid JSON argument strings parse to structured values; invalid strings are preserved.
- `session_meta` rows do not create observations.
- Existing Claude JSONL tests still pass.
- Replay import writes a Hermes session and observations.

Intended verification:
- `corepack pnpm exec vitest run test/replay.test.ts`
- `corepack pnpm exec vitest run test/replay.test.ts test/replay-import-key.test.ts`
- `corepack pnpm run lint`
- `corepack pnpm run build`
- `corepack pnpm test`
- Required security gates before commit/PR prep as documented in the task record.

## Files

- Modify: `src/replay/jsonl-parser.ts`
- Modify: `src/cli.ts`
- Modify: `src/viewer/index.html`
- Modify: `README.md`
- Modify: `test/replay.test.ts`
- Modify: `test/replay-import-key.test.ts`
- Create: `test/fixtures/jsonl/hermes.jsonl`
- Update: `docs/todos/2026-06-19-issue-297-import-hermes-jsonl/todo.md`

## Feature / Verification Matrix

| Change | Verification method | Status |
| --- | --- | --- |
| Hermes parser rows | `test/replay.test.ts` Hermes tests | Done |
| Existing Claude parser behavior | Existing `test/replay.test.ts` tests | Done |
| Replay import storage | `test/replay-import-key.test.ts` Hermes import test | Done |
| User-facing import copy | README/CLI/viewer source inspection plus lint/build | Done |
| Type/build integrity | lint/build/full test | Done |

## Task 1: Add Failing Parser And Import Tests

**Files:**
- Create: `test/fixtures/jsonl/hermes.jsonl`
- Modify: `test/replay.test.ts`
- Modify: `test/replay-import-key.test.ts`

- [x] **Step 1: Create a Hermes fixture**

Add `test/fixtures/jsonl/hermes.jsonl` with session metadata, text blocks, successful and failed tool rows, an omitted tool result name, and invalid JSON arguments.

- [x] **Step 2: Add parser assertions**

Add a test in `test/replay.test.ts` that calls `parseJsonlText(fx("hermes.jsonl"), "hermes-fallback")` and asserts the expected hook sequence, text extraction, parsed tool input, invalid argument preservation, inferred tool result name, and failure hook.

- [x] **Step 3: Add import-flow assertions**

Add a test in `test/replay-import-key.test.ts` that writes a Hermes JSONL file, triggers `mem::replay::import-jsonl`, and asserts one imported session and nonzero observations.

- [x] **Step 4: Verify RED**

Run:

```bash
corepack pnpm exec vitest run test/replay.test.ts test/replay-import-key.test.ts
```

Expected before implementation: Hermes tests fail because current parser emits zero observations for Hermes rows.

## Task 2: Implement Hermes Parser Support

**Files:**
- Modify: `src/replay/jsonl-parser.ts`

- [x] **Step 1: Add Hermes entry fields**

Extend `JsonlEntry` with optional top-level `role`, `content`, `tool_calls`, `tool_call_id`, `name`, `is_error`, `isError`, `status`, and `error` fields.

- [x] **Step 2: Add local helper functions**

Add helpers for record checks, text block detection, JSON argument parsing, Hermes tool-call extraction, explicit failure detection, and tool-result output normalization.

- [x] **Step 3: Add Hermes role branches**

Inside `parseJsonlText`, keep the current timestamp/session loop, then handle `role: "session_meta"`, `role: "user"`, `role: "assistant"`, and `role: "tool"` before falling back to Claude Code branches.

- [x] **Step 4: Verify GREEN**

Run:

```bash
corepack pnpm exec vitest run test/replay.test.ts test/replay-import-key.test.ts
```

Expected: targeted tests pass.

## Task 3: Cleanup, Review, And Broad Verification

**Files:**
- Modify only task-owned files listed above.

- [x] **Step 1: Focused simplification pass**

Reread `src/replay/jsonl-parser.ts` and remove unnecessary helpers or duplicated branches while preserving parser behavior.

- [x] **Step 2: Run repo-native checks**

Run:

```bash
corepack pnpm run lint
corepack pnpm run build
corepack pnpm test
```

Expected: all pass, aside from any clearly unrelated pre-existing or flaky failure recorded with evidence.

- [x] **Step 3: Run required security gates**

For this parser/deserializer change, run Semgrep or repo-defined equivalent. After staging intended files, run `gitleaks protect --staged --redact`.

- [x] **Step 4: Update task state**

Record arena synthesis, verification evidence, review notes, caveats, and final Feature / Verification Matrix status in `todo.md`.

## Arena Synthesis

Base: candidate 1.

Grafts:
- From candidate 2: remember Hermes tool-call names by id so unnamed tool result rows still get `toolName`.
- From candidate 2: test an unnamed tool result row.
- From candidate 3: test assistant rows with empty content and tool calls emit no empty `stop`.
- From candidate 3: normalize text-block tool result arrays into useful text output.

Rejected:
- Raw array/object `toolOutput` when text extraction is possible.
- Failure detection based on `error != null` or output string contents.
- New `--format` flag or public import option.
58 changes: 58 additions & 0 deletions docs/todos/2026-06-19-issue-297-import-hermes-jsonl/spec.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# Issue 297 Hermes JSONL Import Spec

## Design

Support Hermes Agent JSONL in the existing `parseJsonlText` flow by detecting
top-level `role` rows alongside the current Claude Code `type` rows. This keeps
the public import command, REST endpoint, iii function, persisted session model,
and replay observation model unchanged.

Hermes row mapping:
- `role: "session_meta"` is metadata only. Its timestamp contributes to
transcript `startedAt`/`endedAt`, but it creates no observation.
- `role: "user"` imports text content as `prompt_submit`.
- `role: "assistant"` imports non-empty text content as `stop`.
- `role: "assistant"` `tool_calls[]` imports one `pre_tool_use` observation per
call.
- `role: "tool"` imports as `post_tool_use` or `post_tool_failure` when the row
carries explicit failure markers.

Content handling:
- String content is used directly.
- Array content may include strings or text-like blocks with `type: "text"`,
`type: "input_text"`, `type: "output_text"`, or no type and a string `text`
field.
- Non-text blocks such as images are ignored by text extraction.
- Tool result arrays use the same text extraction when possible; otherwise the
original content is preserved.

Tool argument handling:
- Hermes `function.arguments` strings are parsed as JSON only when valid.
- Invalid argument strings are preserved unchanged.
- Tool-call names are read from `function.name` first, then top-level `name`,
then `"unknown"`.
- Tool result names use `entry.name` when present and otherwise fall back to the
prior assistant `tool_calls` name by `tool_call_id`.

## Alternatives

Recommended approach: inline Hermes branches in `src/replay/jsonl-parser.ts`
using small helpers. This matches current parser style and keeps the change
bounded.

Rejected alternatives:
- Add `--format=hermes` or a REST/API format option. The row shape is
distinguishable and a public option would expand the contract unnecessarily.
- Convert Hermes rows to fake Claude rows. That would obscure raw Hermes fields
and add an intermediate format without reducing complexity.
- Infer tool failures from content strings. Only explicit failure markers should
mark `post_tool_failure` to avoid false positives.

## Acceptance

- Existing Claude JSONL fixtures continue to parse unchanged.
- Hermes user, assistant, tool-call, tool-result, text-block, invalid argument,
and failure rows parse into existing replay observations.
- A Hermes JSONL file imported through `mem::replay::import-jsonl` writes one
session with nonzero observations.
- No dependency, schema, migration, endpoint, MCP tool, or CLI flag change.
Loading
Loading