Latency Metrics (end-to-end) — re-implemented on new architecture

[angrave]

Supersedes #67. The original latency-metrics PR was built against the pre-refactor architecture and could not be conflict-merged — staging had rewritten the node-server, deleted the transcription-service-client lib, split webapp into client-webapp/kiosk-webapp, and refactored the Python worker pool, and the new pipeline carried no chunk-id/timestamp to hang latency off of. This PR rebuilds the feature on the current architecture.

What it does

• Kiosk frames each audio chunk in a versioned, self-describing binary envelope (@scribear/audio-frame-protocol, "SAFP" v1: magic + version + TLV metadata + CRC-32) carrying a chunkId and a clock-corrected sentAt.
• Node server decodes frames, keeps a bounded per-session chunkId → {recvMono, sentAt} map, forwards audio upstream, and on each transcript correlates the echoed chunk_ids to emit two latencies:
  • pipelineMs — measured on the node's monotonic clock (skew-free);
  • e2eMs — capture→display, using a clock-synced sentAt; null until synced.
• Clock sync via timeSyncPing/timeSyncPong (Cristian's algorithm, min-RTT sample).
• Client webapp shows a 60-sample moving average of both metrics (final / interim).
• Python threads chunkId through the provider/job and echoes final/in_progress chunk_ids on the transcript message.

Design notes

• Replaces PR Latency Metrics - End-to-End Latency Tracker #67's hard-coded 44-byte header with a versioned TLV+CRC envelope so kiosk/node/python can evolve independently. The TS and Python codecs produce byte-identical frames.
• The monotonic pipelineMs is the trustworthy metric for alerting; e2eMs is indicative (depends on clock-offset estimation).

Verification

• TS: full build, all-workspace lint + format, node-server 78 tests (incl. a real transcription-service container integration proving SAFP round-trips end-to-end and audio actually flows), audio-frame-protocol 19 tests (codec + CRC known-answer + ClockSync).
• Python: 180 tests, pylint 10/10, black/isort clean.
• Hardened a latent weak integration test that passed even when audio was dropped (debug provider emits Processed 0.0000 seconds on its tick regardless) — it now sends a real SAFP frame and asserts a non-zero processed duration.

Follow-ups

See TOBEREVIEWED.md for deliberately-deferred items (VAD-config exposure, Python stage-timing diagnostics) and production concerns (clock-skew reliability, decode-drop observability, unauthenticated/unthrottled timeSyncPing, pendingChunks cap, CRC cost).

🤖 Generated with Claude Code

[github-actions]

Node Server - Unit Tests Coverage Report

Package	Line Rate	Branch Rate	Complexity	Health
app-config	100%	100%	0	✔
server	0%	0%	0	❌
server.dependency-injection	0%	0%	0	❌
server.features.probes	0%	0%	0	❌
server.features.transcription-stream	63%	50%	0	➖
server.features.transcription-stream.events	100%	100%	0	✔
server.plugins	0%	100%	0	❌
server.shared.services	98%	94%	0	✔
Summary	61% (245 / 399)	50% (81 / 163)	0	➖

[github-actions]

Node Server - Integration Tests Coverage Report

Package	Line Rate	Branch Rate	Complexity	Health
app-config	0%	0%	0	❌
server	100%	100%	0	✔
server.dependency-injection	89%	75%	0	✔
server.features.probes	100%	100%	0	✔
server.features.transcription-stream	72%	46%	0	➖
server.features.transcription-stream.events	100%	100%	0	✔
server.plugins	100%	100%	0	✔
server.shared.services	93%	67%	0	✔
Summary	77% (306 / 399)	52% (85 / 163)	0	✔