feat(audio): provider-neutral speech-to-text seam

[hung12ct]

Summary

Adds pkg/audio, a provider-neutral speech-to-text seam, and closes a silent-failure hole in how all three LLM adapters handle history.MediaPart.

Of the three providers here, only Gemini accepts audio in a chat message: Anthropic has no audio content block, and the vendored OpenAI Chat Completions client cannot express one (ChatMessagePartType has only text and image_url). Transcribing ahead of the loop rather than inside the message makes the capability portable — what reaches the model is ordinary text, so every provider can drive an audio-fed agent.

It is also the cheaper shape. History is re-sent on every LLM call in a session, so audio carried as a message part is re-uploaded on every subsequent turn: a one-hour recording transcribed once costs one upload, while the same recording as message parts is billed again on each turn of the conversation about it.

Changes

pkg/audio — new stdlib-only leaf package

• Transcriber interface plus Clip, Transcript, Segment, Options value types.
• Three sentinels rather than one error, because a live-capture pipeline responds to each differently: ErrTooLarge → re-cut into shorter chunks, ErrUnsupportedFormat → re-encode, ErrNoAudio → caller bug. Route with errors.Is, not message matching.
• Transcript.Segments is nil when the backend emits no timing — a normal result, not a failure, so callers needing timestamps must check rather than assume.
• Ext strips MIME parameters and matches case-insensitively: browsers' MediaRecorder reports audio/webm;codecs=opus, and some emit the video/webm spelling for an audio-only recording. Rejecting either would fail a clip the backend decodes fine.
• Takes []byte, not io.Reader: the OpenAI client assembles its multipart body into a bytes.Buffer and genai.Blob needs []byte, so a Reader would be a streaming API that does not stream.

Two implementations

• openai.NewTranscriber — populates Segments, Language, Duration. Requests verbose_json only for whisper models, since the gpt-4o-transcribe family rejects that format outright rather than degrading; asking for it everywhere would fail every request instead of merely losing timings. Matched as a substring, because compatible endpoints name the same weights differently (Systran/faster-whisper-large-v3). Oversized clips are rejected against the 25 MB limit before the upload. Honours WithBaseURL, so a self-hosted transcription server is a supported target.
• gemini.NewTranscriber — no dedicated transcription endpoint exists, so it constrains the generation API with a system instruction; without one the model opens with a preamble or summarizes instead of transcribing, and both corrupt a transcript appended verbatim. Segments is always nil, stated on the type rather than discovered at run time.

agent.ErrUnrenderablePart — media parts no longer vanish silently

All three adapters converted MediaPart with a switch that fell through for anything unexpected, and one documented the omission as deliberate. Dropping a part does not degrade the call — it silently changes what the question was. The model receives the caption alone and answers it fluently, and nothing distinguishes that from success: not the response, not the logs, not a schema check, because a well-formed answer is exactly what success looks like.

Four shapes now fail:

• an unknown part type,
• an image with neither URL nor Data,
• a parts slice that yields no content at all,
• media parts on a non-user role — previously ignored wholesale by every adapter (OpenAI behind an explicit role guard, Anthropic and Gemini by rendering media only under their user branch).

Empty text parts are still skipped; they carry nothing to lose. isRetryable treats the sentinel as terminal — the same bytes fail identically on every attempt.

Type of Change

• New feature
• Bug fix
• Refactoring
• Documentation
• Performance improvement
• Breaking change

Breaking: a message carrying a malformed or unsupported media part now fails where it previously went through silently. That is the intent, and the reason for the minor bump to v0.41.0.

Blast radius was checked before the change rather than assumed: tools.Result.Parts has zero consumers in pkg/agent (declared but never wired), no internal path builds a non-user message with Parts, and examples/media_chat/main.go:371 uses Role: "user". Nothing in-tree breaks.

Testing

• Existing tests pass
• New tests added
• Manual testing completed

gofmt -l . clean · go vet ok · go build ok · golangci-lint 0 issues · go test ./... 24 packages ok, 0 failures · go test -race ./... clean · go mod tidy produces no dependency changes.

New coverage:

• pkg/audio — MIME parsing with codec parameters and the video/webm spelling, clip validation per sentinel, sentinel distinctness, single-package-prefix error messages.
• OpenAI transcriber — segment/duration mapping with sub-second precision, filename extension derived from MIME, language and prompt forwarding, verbose_json selected only for whisper, oversized clip rejected without contacting the API (asserted via a server that records whether it was called).
• Gemini transcriber — transcriptFromResponse extracted as a pure function so its order-sensitive checks are testable without a client: clean stop, safety block, truncation, no-candidate.
• All three adapters — empty text dropped, unrenderable parts rejected, Parts on system/assistant/tool roles rejected.

Notes

One issue found and deliberately not fixed here: ErrLLMAuth's doc states auth failures are deterministic, but isRetryable (pkg/agent/retry.go:62) never excluded it, so auth failures are retried 3× today. Changing retry semantics for auth deserves its own decision rather than riding along in an audio release.