Skip to content

trail init learns from PR review comments - #614

Closed
anirudhkumar-nanonets wants to merge 1 commit into
fix/trail-init-extension-claudefrom
feat/trail-init-pr-reviews
Closed

anirudhkumar-nanonets wants to merge 1 commit into
fix/trail-init-extension-claudefrom
feat/trail-init-pr-reviews

Conversation

@anirudhkumar-nanonets

@anirudhkumar-nanonets anirudhkumar-nanonets commented Oct 15, 2026 •

Copy link
Copy Markdown
Collaborator

Moves: learnings_committed, clones whose committed learnings went up in the last 7 days (PostHog Postgres, DB 11, posthog_event_full_view; query in daily/project.md). I couldn't read today's value because this session has no Metabase access. The last recorded value is 0 on 2026-10-13, before the event shipped.

What

trail init gets a second source for its first-run learnings: review comments on the repo's last 50 merged PRs. This was rushabh's 13 Oct ask for init's opening, e.g. "PR reviews keep flagging missing error handling in /api".

  • Reading. One gh api graphql call fetches review summaries and inline comments. It runs only when the repo has a GitHub remote and gh auth token succeeds; otherwise it's skipped quietly, the same way init stays quiet when it can't find claude.
  • Cleaning. Bot comments, the PR author's own replies, quoted lines and HTML comments are dropped. scrub() runs on every comment before it reaches Claude, and no logins are sent.
  • Asking Claude. A second headless Claude call (same flags and model as the session read) runs in parallel with the session read. A finding is kept only when its quotes really appear on 2 or more PRs. The quote matcher is now shared as placeQuotes, which verify() uses too.
  • Picking. Review findings go through trail init: find extension claude, tick learnings #612's keep step. One picker lists session and review findings together, and review rows say "flagged in review on N PRs". Cancelling saves none and leaves both reads unmarked, so the next init asks again. Without a terminal, or with --yes, all are kept.
  • Saving. Kept findings become standing team learnings in .trail/learnings/, with the line _Flagged in review on N PRs here (#41, #38), most recently on Oct 10._
  • Once only. It runs once per repo per machine (pr-reviews.json). It's also skipped when .trail/learnings/ already holds review learnings, i.e. a teammate already ran it, so the same reviews don't create duplicates.
  • Switched repos. It also runs in _trail-past-read, for repos that move from graft to trail at session start.

Init output, with both sources:

· reading your 2 past Claude sessions here for what you keep telling Claude
· reading review comments on this repo's merged PRs for what reviewers keep flagging
…
✓ learned 1 thing you told Claude more than once, from your last 2 sessions here
    for your team · told Claude in 2 sessions: Run go test with -race before pushing
✓ learned 1 thing reviewers flagged on more than one PR, from the last 4 merged PRs here
    for your team · flagged in review on 2 PRs: Return a 500 when db.query fails in src/api
· Claude follows them from your next session
🌱 Commit what you learned and give your teammates' agents a head start

Telemetry

  • cold_start gains source (sessions/reviews) and reviews_bucket (merged PRs with a reviewer's comment, bucketed).
  • New outcomes: no_reviews and no_gh (and cancelled, from trail init: find extension claude, tick learnings #612, applies to both sources).
  • Session-read events now carry source: sessions. An event without source comes from a build before this change.
  • TELEMETRY.md and the README's "What stays on your machine" section are updated.

Tested

  • New test/trail-pr-reviews.test.ts:
    • comment filtering and scrubbing, and the digest format
    • verifyReviews: a finding on 2 PRs is kept, a misfiled quote still counts, made-up and single-PR findings are dropped
    • end-to-end trail init with a fake gh and claude on PATH: both learning blocks, once per repo, skipped on a teammate's machine, and what Claude was sent
    • a gh that isn't signed in, no GitHub remote, and gh missing
    • keeping some review findings, and cancelling (nothing saved, not marked read)
  • test/cli-picker.test.ts: a review row in the same list as session rows.
  • Real run on assign (nothing saved): 50 merged PRs, 13 with reviewer comments, 1 finding verified across 2 PRs, 14 s, $0.04.
  • npm test: 1407 tests, 0 failures, 1 skipped (the existing Node 24 skip). tsc --noEmit is clean.

Stacked on #612

The base is fix/trail-init-extension-claude, because #612 changes the same init code (the claudeBin() lookup, the session counts and the keep picker). Once #612 merges, GitHub retargets this PR to main.

🤖 Generated with Claude Code

@trailhq-graft

trailhq-graft Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

🌱 graft blast radius

3 areas changed → 4 areas can be affected. 11 dependent symbols, depth 2.
Tests: 2 areas updated their tests.
Tag: @shhdwi — 4 of 7 areas · @afeddersen — Telemetry Tracking

flowchart TB
  A0(("Session State Management<br/>7 symbols"))
  A1(("Telemetry Tracking<br/>2 symbols"))
  A2(("Cloud Nudges<br/>1 symbol"))
  A3(("MCP Tools<br/>1 symbol"))
  classDef reached fill:#D9EDF3,stroke:#3AA7C9,stroke-width:1.5px,color:#0E313C;
  class A0,A1,A2,A3 reached;
Loading
Can be affected Symbols Nearest hop Reached from
Session State Management 7 src/claude/hooks.ts:L1-L662 hooks.ts — imports, depth 1 Telemetry Contract
Telemetry Tracking 2 src/telemetry/sessions.ts:L1-L113 sessions.ts — imports, depth 1 Telemetry Contract
Cloud Nudges 1 src/cloud/nudge.ts:L1-L330 nudge.ts — imports, depth 1 Telemetry Contract
MCP Tools 1 src/mcp/tools.ts:L1-L331 tools.ts — imports, depth 2 Telemetry Contract
Who knows this code — 2 people across 7 areas
Area Who knows it
Learning Picker CLI · changed @shhdwi — 25 commits, last yesterday
Session State Management · affected @shhdwi — 21 commits, last yesterday
Telemetry Tracking · affected @afeddersen — 1 commit, last 2mo ago
Cloud Nudges · affected @shhdwi — 1 commit, last yesterday
MCP Tools · affected @shhdwi — 15 commits, last yesterday
Telemetry Contract · changed only you — nobody else has touched these files
Notes Read Tracking · changed only you — nobody else has touched these files

Ownership is git history over each area's own files, weighted towards recent work (120-day half-life). Merge commits and bots are dropped, and you are dropped from your own PR. A name with no @ has no GitHub handle in its commit email — tag them by hand, or add a .mailmap entry. A suggestion from history, not a CODEOWNERS rule.

All 11 dependent symbols, grouped by area

Session State Management — 7 symbols in 7 files

  • src/claude/hooks.ts:L1-L662 — hooks.ts (imports, depth 1)
    72: * strictly inside whatever budget this repo really has.
  • src/claude/session-metrics.ts:L1-L215 — session-metrics.ts (imports, depth 1)
    74: return GRAFT_MCP_TOOL_NAMES.has(bare);
  • src/claude/state.ts:L1-L112 — state.ts (imports, depth 1)
    54: * at Stop once the turn has been counted. Transient, not a total. */
  • src/claude/format.ts:L1-L347 — format.ts (imports, depth 2)
    41: // Dollars only once a turn has actually been billed — see context/price.ts.
  • src/claude/stats.ts:L1-L27 — stats.ts (imports, depth 2)
  • src/claude/statusline.ts:L1-L39 — statusline.ts (imports, depth 2)
  • src/claude/sync-run.ts:L1-L42 — sync-run.ts (imports, depth 2)

Telemetry Tracking — 2 symbols in 2 files

  • src/telemetry/sessions.ts:L1-L113 — sessions.ts (imports, depth 1)
    13: * because Claude Code has no reliable end-of-session hook — `Stop` fires per
  • src/telemetry/track.ts:L1-L161 — track.ts (imports, depth 1)
    39: /** Repo root, when the call site has one — supplies `repo_id`. */

Cloud Nudges — 1 symbol in 1 file

  • src/cloud/nudge.ts:L1-L330 — nudge.ts (imports, depth 1)
    4: * Nobody is asked to type `trail login`. Once a person has a learning worth

MCP Tools — 1 symbol in 1 file

  • src/mcp/tools.ts:L1-L331 — tools.ts (imports, depth 2)
    226: // Freshness first: an answer that cites file:line has to be about the code as
Test signal per changed area — 2 ✓ · 1 –

Reached = a node under a test path has a resolved edge into the changed symbol. It undercounts anything called indirectly — through a CLI, a spawned process or a dynamic import — so read a low ratio as “look here”, never as a coverage gate.

  • – Telemetry Contract — 1 test file reaches it, none changed here
  • ✓ Notes Read Tracking — 7 of 18 reached · 1 test file changed here: test/trail-pr-reviews.test.ts
    • not reached: askClaude, norm, has, beginPastRead, beginReviewRead, fetchMergedPRs, findReviewLearnings, isBot, …3 more
  • ✓ Learning Picker CLI — 1 of 4 reached · 1 test file changed here: test/cli-picker.test.ts
    • not reached: pickLearnings, runInitCommand, keepFrom
19 test suites also reference this code

19 symbols, kept out of the diagram and the table so they cannot crowd out the areas a reviewer has to look at.

  • test/claude-format.test.ts
  • test/claude-hooks.test.ts
  • test/claude-state.test.ts
  • test/claude-statusline.test.ts
  • test/claude-tally.test.ts
  • test/hosts-codex-hooks.test.ts
  • test/session-metrics.test.ts
  • test/telemetry-contract.test.ts
  • test/telemetry-gate.test.ts
  • test/telemetry-install.test.ts
  • test/telemetry-sessions.test.ts
  • test/trail-cold-start.test.ts
  • test/trail-learnings-committed.test.ts
  • test/trail-name.test.ts
  • test/trail-note-ask.test.ts
  • test/trail-nudge.test.ts
  • test/trail-prompt-notes.test.ts
  • test/trail-spelling.test.ts
  • test/upkeep-hooks.test.ts

⚠️ 2 changed files not in the graph (README.md, TELEMETRY.md) — no parser claims the extension, or the index predates the file.

graft blast · origin/fix/trail-init-extension-claude...HEAD · depth 2 · 9 changed files

Open the interactive graph → — click an area to see its dependent symbols at file:line.

github-actions Bot added a commit that referenced this pull request Oct 15, 2026
@anirudhkumar-nanonets
anirudhkumar-nanonets changed the base branch from main to fix/trail-init-extension-claude October 15, 2026 09:49
github-actions Bot added a commit that referenced this pull request Oct 15, 2026
@anirudhkumar-nanonets

Copy link
Copy Markdown
Collaborator Author

Folded into #611, so all the trail CLI work for 0.22.1 ships as one PR. Same commit (516fcf8), on top of #611's branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant