feat(sdk): add hexgate.messages wire contract and content caps

[victorludvig]

Wire contract only — no emitter, no platform change; nothing sends this scope yet.

What is Changing

• hexgate/tracing/semconv.py — SCOPE_MESSAGES = "hexgate.messages", the official GenAI content names gen_ai.input.messages / gen_ai.output.messages / gen_ai.system_instructions (verbatim, never coined), and the Hexgate-specific sec_ai.message_seq / turn_key / resynced / truncated. The module docstring gains the message rules: JSON-string content, one event per LLM call carrying only the messages new to that call, message_seq per turn_key for gap detection.
• hexgate/audit.py — MAX_INPUT_MESSAGES_BYTES (256 KiB), MAX_OUTPUT_MESSAGES_BYTES (8 KiB), MAX_SYSTEM_INSTRUCTIONS_BYTES (8 KiB) beside the decision caps, plus truncate_head_tail() and cap_json_head_tail(). The latter shrinks every string leaf head+tail to one shared byte allowance — the largest that makes the serialized JSON fit, found by binary search — so roles and message boundaries survive and only the middle of oversized text is lost. Falls back to the existing truncate_json preview wrapper when the JSON structure alone exceeds the cap. Pure; measures with json.dumps(default=str) like the platform.

Why is this change necessary?

semconv.py is the one file both packages import, so the wire contract lands first and the platform and SDK lanes can then proceed in parallel.

Why the caps and helpers live in the SDK. The SDK applies the cap before export — the design's blast-radius argument (one oversized span must not take its batch down) only holds upstream of the batch — and the platform enricher will import the same functions as enforcement, exactly as enforcement.py imports truncate_json today.

Why the input cap is 256 KiB and not the 32 KiB first proposed. 32 KiB is ~7,000 tokens of ASCII (~5,400 CJK characters, since JSON escapes each to 6 bytes), which 20 retrieved chunks of 500 tokens already exceed — the cap would have fired on exactly the RAG calls the message log exists to explain. What bounds it is the OTLP record size rather than storage, and the message path's topic is raised to max.message.bytes=8 MiB anyway, so a quarter-megabyte field costs nothing operationally.

Why one shared allowance rather than a per-leaf budget. Deriving each leaf's target from the document's overage is wrong twice over: the overage is measured on escaped JSON while a leaf is measured in UTF-8, so escape-heavy content over-cuts to the floor (70k CJK characters kept 42 of a 32 KiB cap); and charging one leaf for the whole overage makes the outcome depend on which message happens to be biggest (two equal messages came out 64 / 8,004). A single allowance is order-independent and leaves no cap unspent.

Tests

• tests/tracing/test_semconv.py — new scope and names in the prefix / collision / official-name invariants.
• tests/audit/test_message_caps.py — head+tail on ASCII and multibyte text, exact omitted-byte count, largest-leaf-first, both-leaves-cut with balance and utilization bounds, escape-heavy content, tuple-shaped message lists (LangChain's [("system", …), ("human", …)]), caps below the preview wrapper's headroom, no input mutation, structure-only overflow fallback, bare string, CJK under the JSON measure.
• Full SDK suite: 2389 passed, 13 skipped.

Known gap

Inline base64 image parts are capped as ordinary string leaves, so a large data:image/... URL survives as a useless fragment eating most of the input budget. Tracked in #198 — image parts should be placeholder-substituted before the cap runs.

Design

LLM message logging design · implementation spec

🤖 Generated with Claude Code

[codecov]

Codecov Report

❌ Patch coverage is 92.85714% with 5 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
hexgate/audit.py	91.93%	4 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

[guillaume-hexamind]

Review: two issues on the new message caps

1. MAX_INPUT_MESSAGES_BYTES = 256 KiB is above what the pipeline can actually carry (hexgate/audit.py:71)

The comment justifies the number with "the message path's topic carries max.message.bytes=8 MiB". That topic and that setting don't exist in this repo, and three separate limits sit below 256 KiB × batch size:

• No max.message.bytes anywhere. platform/redpanda/init/create-topics.sh:66-79 sets only --partitions 3 --replicas 1 -c retention.ms=…. There is also no separate "message path" topic — the Collector exports every scope to the single hexgate.otlp.raw (platform/collector/config.yaml:74).
• The Kafka exporter caps records producer-side, so a topic setting wouldn't help. configkafka@v0.158.0 defaults to MaxMessageBytes: 1000000 and Compression: "none" (config.go:328,331); our exporter block overrides neither. The exporter rejects the record before Redpanda ever sees it.
• One Kafka record per Collector batch, and we can't split it. partition_traces_by_id is mutually exclusive with message_key_from_metadata_key (kafkaexporter@v0.158.0/config.go:34,194), which we need for project keying (config.yaml:78). With send_batch_size: 512 and no send_batch_max_size, four max-size message spans in one batch exceed 1 MB and fail the whole record — taking the decision-audit spans batched alongside them down with it. That's exactly the blast radius the module comment says the SDK-side caps exist to bound.
• The earlier failure is the OTLP receiver. confighttp@v0.158.0/server.go:35 defaults max_request_body_size to 20 MiB and config.yaml doesn't override it. The SDK exports over OTLP/HTTP with compression defaulting to None and BatchSpanProcessor defaults intact (512 spans per POST, _senders.py:143), so ~80 max-size spans in one export = 20 MiB and the entire POST is rejected. memory_limiter is limit_mib: 512.
• OTel won't clip the attribute for us either: _SPAN_LIMITS sets max_attribute_length / max_span_attribute_length to SpanLimits.UNSET (-1 → unlimited), so a 256 KiB attribute goes out at full size. Worst-case resident queue is 2048 × 256 KiB ≈ 512 MiB.

The reasoning for rejecting 32 KiB is sound — a 20-chunk RAG call would trip it, which is the wrong call to lose. But the ceiling needs to come from the batch/broker/receiver limits rather than from a topic config that isn't there. Either land that config (exporter max_message_bytes, send_batch_max_size, receiver body size, topic max.message.bytes) in this PR, or derive the constant from the limits as they currently stand.

2. copy.deepcopy on the under-cap fast path breaks the function's own tolerance contract (hexgate/audit.py:263)

_cap_json_head_tail is deliberately built to survive arbitrary framework objects — it measures with json.dumps(default=str) throughout, and the truncation path rebuilds a fresh structure via _cap_leaves. deepcopy doesn't share that tolerance:

cap_json_head_tail([{"role": "user", "lock": threading.Lock()}], cap=8192)
# TypeError: cannot pickle '_thread.lock' object

The same input on the over-cap path returns cleanly. So the fast path is strictly less robust than the slow one, for message lists carrying a lock, socket, or generator — plausible in framework-owned message objects. Once an emitter is wired, span_attributes() runs inside AuditSender.emit()'s blanket except Exception (_senders.py:233), so this surfaces as a silently dropped event plus a log line.

It also isn't buying anything: the caller serializes the result immediately and the truncation path already returns a fresh structure, so purity never depended on the deepcopy. (It isn't a perf issue either — deepcopy returns identity for str, measured at 0.02 ms on a 250 KiB list.) Returning value unchanged, or shallow-copying the container to match _truncate_json's dict(payload), fixes it.

---

For what it's worth, the algorithm itself held up under fuzzing: truncate_head_tail / cap_json_head_tail across ASCII / Latin-1 / CJK / escape-heavy inputs at caps from 64 B to 256 KiB produced zero cap violations, and the binary search matched a brute-force scan of every allowance in 400/400 randomized trials. The preview_bytes floor in _truncate_json fixes a real pre-existing non-termination bug.

[guillaume-hexamind]

2 Issues to investigate above; otherwise I approve the design and this first step

[victorludvig]

• 2 (deepcopy): fixed in 7e66a3f — the under-cap path now rebuilds via _cap_leaves(value, limit=cap), which cuts nothing and copies the containers. Regression test for a lock-carrying message on both paths.
• 1 (256 KiB): you're right, the comment was wrong — corrected in the same commit. It now names the topic/exporter/batch/receiver limits as an undeployed prerequisite instead of existing config. Constant stays at 256 KiB: nothing emits hexgate.messages until PR 5 ships, so it's inert until then.
• Your two extra finds are in the plan now: no separate "message path" (one batch processor, one topic), so PR 5 uses send_batch_max_size: 24 on the shared processor, and it grows from four limits to six — adding exporter compression and the receiver's max_request_body_size. PR 5 is marked as blocking PR 8–13. implementation spec §III · design §V
• Thanks for the fuzzing pass.

[guillaume-hexamind]

Rebase before merge, but good to me