Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions skills/build-trace-analyst/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,19 @@ Use `runAnalystBenchmark` with `registryBenchmarkRunner` from `@tangle-network/a
Use AgentRx or CodeTraceBench labels when they match the finding.
Compare the candidate with a no-findings baseline and relevant maintained or external analysts on identical cases and limits.

Two shipped engines are available as comparison arms.
`--analyzer prime` inlines the whole span projection into one prompt and needs no REPL.
`--analyzer halo` and `--analyzer hodoscope` drill into the trace with paged tools.
Compare an inline arm with a paged arm to separate the prompt from the navigation.

```bash
traces analyze --last 1 --analyzer prime
```

Assert the served model, not the requested id.
A gateway can answer one id with another model, so a panel can report families it never ran.
Use `assertServedModel` and `assertCrossFamilyServed` from `@tangle-network/agent-eval` before you trust a per-model or cross-family number.

Use at least 20 independent cases before comparative statistics.
Retain every case row and report exact-step recall, precision, F1, root-step accuracy when labeled, citation coverage and validity, clean-case false positives, repeat agreement, failures, latency, calls, all token fields, and known or missing cost.

Expand Down
15 changes: 15 additions & 0 deletions skills/inspect-agent-traces/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,21 @@ traces improve --harness codex --current --latest-turn --workflow \
`improve` writes findings, evidence, a report, and spans.
It does not edit an agent, repository, memory store, or knowledge base.

Write one session's durable evidence directory when a later reader must cite the session:

```bash
traces bundle --harness claude-code --session <id-or-path> --out .traces/bundle
```

`bundle` copies the transcript, the derived report and spans, the `.evolve` ledger rows inside the session window, and a `manifest.json` with a SHA-256 per file.
It spends no model call.
Use it when the live stores can rotate before the reader arrives.
A missing transcript stops the assembly.
An absent optional input is recorded in `manifest.absent` with the probed path.

The bundle holds the whole session transcript.
Do not give a bundle to a writer that must not see an earlier conclusion, because the transcript holds the text of every file the session wrote.

## Report

- State the source (`--otlp <path>` or the harness), selected task boundary, session and span counts, and integrity warnings.
Expand Down
Loading