Add cross-chunk redaction safety for streaming Frames

[dgenio]

Summary

Harden Firewall.apply_stream so a sensitive pattern that is split across two
streamed chunks is still detected, rather than evading every per-chunk regex.

Why this matters

apply_stream wraps each chunk in a synthetic RawResult and redacts it
independently. A secret like a JWT or connection string delivered as "...eyJabc" then
"def.ghi..." matches no single-chunk pattern and reaches the LLM. As streaming becomes
the default UX for agent tool output, this boundary gap matters.

Current evidence

• firewall/transform.py: apply_stream() iterates chunks and calls self.transform(...) per chunk; redaction state does not carry across chunks.
• kernel/_stream.py: _stream_chunks delegates per-chunk to apply_stream; docstring notes the firewall is stateless across chunks.
• firewall/redaction.py operates on one value at a time.

External context

Streaming tokenized output is standard; split-token evasion is a recognized class of
filter bypass.

Proposed implementation

1. Maintain a small carry-over tail buffer of the last N characters of string content
   across chunks for the string-scanning patterns, scanning tail + chunk.
2. Only re-emit the newly safe portion; hold back a bounded suffix until the next chunk
   (or until the final chunk) so a pattern straddling the boundary is caught.
3. Keep this opt-in/bounded so streaming latency and memory stay predictable; document
   the residual risk if a user disables it.

AI-agent execution notes

• Inspect first: firewall/transform.py (apply_stream), kernel/_stream.py, tests/test_firewall_stream.py.
• Determinism is required (AGENTS.md) — buffering must be deterministic.
• Edge cases: final chunk must flush the buffer; binary/non-string payloads unaffected.

Acceptance criteria

• A fake secret split across two chunks is redacted in the streamed Frames.
• The final chunk flushes any held-back content.
• Per-chunk latency stays bounded (buffer size capped).

Test plan

Add split-pattern cases to tests/test_firewall_stream.py. Run make ci.

Documentation plan

Document streaming redaction semantics and any residual limitation in
docs/context_firewall.md.

Migration and compatibility notes

Streaming consumers may observe a one-chunk delay on the trailing buffer. Document the
behavior. Not expected to require migration.

Risks and tradeoffs

Adds bounded state to a deliberately stateless component; keep the buffer small and the
contract documented to avoid surprising latency.

Suggested labels

security, reliability, testing

[dgenio]

Closing as fixed by merged PR #229 (merged 2026-06-17).

A new stateful StreamRedactor (holdback + overlap window) provides cross-chunk redaction safety: Firewall.apply_stream() carries a per-field redactor (_apply_stream_redactors) so a secret whose characters span two chunks is reassembled and redacted before either half is emitted; stream warnings are surfaced on the Frame. Verified in the current tree at src/weaver_kernel/firewall/redaction.py (class StreamRedactor) and src/weaver_kernel/firewall/transform.py (apply_stream / _apply_stream_redactors). Regression coverage: tests/test_firewall_canary.py::test_canary_split_across_chunks_is_redacted and ::test_stream_redactor_preserves_full_text_minus_secret.

Documented residual limits (deliberate safety/latency and memory trades) are noted in docs/security.md: a pattern with internal whitespace split exactly at the held boundary, and a single contiguous secret longer than the overlap * 4 buffer, may still evade detection.

---

Generated by Claude Code