Skip to content

Extract harness-independent hook protocol - #55

Closed
TensorTemplar wants to merge 1 commit into
mainfrom
feat/hook-protocol
Closed

TensorTemplar wants to merge 1 commit into
mainfrom
feat/hook-protocol

Conversation

@TensorTemplar

Copy link
Copy Markdown
Owner

Closes #50
Addresses the ingestion question in #46

What this does

Implements the stable, abstract hook protocol requested in #50 — independent of Claude Code and OpenCode — so third-party agent tools can feed session events into slopometry without emulating Claude Code hook payloads or patching core.

Protocol design

Following the envelope proposed by @Necmttn in #46, with the closed-source caveat resolved:

  • core/protocol/schema.py — EventEnvelope (v1): source, schema_version (validated, fails fast on unsupported), session_id, event_id (required, so backfills are always idempotent), parent_session_id, seq, occurred_at, kind, tool_name, duration_ms, exit_code, error_message, raw.
  • core/protocol/kinds.py: canonical EventKind taxonomy (closed, validated: tool_call, tool_result, notification, stop, subagent_stop, subagent_start, todo_updated, message_updated) + open KnownSource constants. The source dimension is an open string — third parties pass any source name without touching core. Kinds are added only when a real adapter needs them (extension policy documented on the enum).
  • core/protocol/adapters/: harness-name → canonical-kind maps for Claude (PreToolUse → tool_call, …) and OpenCode (pre_tool_use → tool_call, …). Harness names never reach storage or downstream consumers.
  • core/protocol/ingest.py + slopometry ingest --source <name> [--file] [--working-directory]: JSONL envelopes from stdin/file, atomic validate-then-insert, loud failure (exit 2) on invalid lines or source/--source mismatch. Dedupe on (source, event_id) via unique partial index — re-ingesting a trace is a no-op. Sequence numbers continue after the highest stored/batched seq when envelopes omit seq.
  • Shared capture_event factory (core/event_capture.py): owns event assembly (sequencing, git-state rule, project detection, tool classification, persistence) for the Claude, OpenCode, and envelope paths — no more near-duplicated construction blocks in the two handlers.

Storage changes

  • HookEventType → EventKind everywhere (canonical snake_case values); EventSource closed enum → open str source; HookEvent.event_id added.
  • Migration 019: remaps legacy PascalCase wire names and the started/completed dialect from a parallel implementation onto canonical kinds, backfills NULL sources, adds event_id + unique partial index. Numbered 019 because a parallel tree already claimed 016–018 on live databases.

Testing

  • New: test_protocol_schema.py, test_ingest.py (envelope validation, idempotency, seq continuation, CLI incl. failure paths), migration-019 test incl. index integrity.
  • Full suite: 855 passed standalone on this branch (ruff clean, mypy no new errors).
  • Verified live: installed per README, migration 019 applied cleanly to a 910k-event production DB, ingest + re-ingest + solo ls/solo show all correct on migrated data.

For reviewers from #46

@botbotfromuk — this answers your question: ingest is an explicit import call (slopometry ingest), not a watched path. A collector like your SlopometryCollector can now write envelope JSONL and pipe it in; source + event_id makes retroactive convert_trace_to_slopometry() backfills safe to re-run. Mapping tick_start/decision/action → notification is the pragmatic choice; keep finer distinctions in raw until real consumers need first-class kinds (see the extension policy in kinds.py).

@Necmttn — the closed AbstractEventSource caveat from your review is resolved: source is an open validated string, KnownSource is constants-only, nothing gates on membership.

Closes #50, addresses #46.

- New core/protocol package: canonical EventKind taxonomy (validated,
  closed), open KnownSource constants, and the stable EventEnvelope
  schema (v1) for third-party agent tools; event_id is required so
  backfills are always idempotent.
- New `slopometry ingest --source <name>` CLI: JSONL envelopes from
  stdin/file, loud failure on invalid lines or source mismatch.
- EventSource closed enum replaced by open source strings; HookEventType
  replaced by EventKind with snake_case canonical values across storage,
  handlers, display, and solo commands.
- Shared capture_event factory (core/event_capture.py) now owns event
  assembly for the Claude, OpenCode, and envelope ingest paths.
- Migration019: remap legacy PascalCase and parallel-implementation
  dialect event types onto canonical kinds, backfill source, add
  event_id with a unique partial index on (source, event_id).
  Numbered 019 because a parallel tree already claimed 016-018 on live
  databases.

Tool name mapping (get_tool_type) moved from hook_handler into
core/models/hook.py.
@TensorTemplar

Copy link
Copy Markdown
Owner Author

Closing: superseded by a delta approach. Since this branched, #54 landed the abstract protocol on main (AbstractHookEvent, AbstractEventType, adapters, dispatch, emit-event) — this PR unknowingly re-implemented it with a divergent taxonomy and is CONFLICTING as a result. The genuinely additive parts (open source dimension, stable external envelope with required event_id, idempotent JSONL backfill) will be re-submitted as a small PR on top of current main instead. Apologies for the churn — and to reviewers from #46: the ingest contract question will be answered by the follow-up PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement a stable abstract hook protocol instead of harness-specific mappings

1 participant