feat: replay-trace tokens + /v1/replay endpoint (Phase 3.1)#19
Merged
Conversation
Every CostStrategy result now carries a deterministic trace token —
sha256(doc_id | doc_version | model | system_prompt_version |
sorted(selected_ids)) hex-encoded. Same inputs always produce the
same 64-char hex string, regardless of reasoning path; permuted ID
order is invariant.
ComputeTraceToken is the canonical helper. SinglePass, ChunkedTree,
and AgenticStrategy each call it before returning. The Cached
wrapper re-derives the token on cache hits so the trace survives
the cache layer (the token is a pure function of cached inputs).
SystemPromptVersion ("v1") is bumped whenever a retrieval system
prompt changes in a way that should invalidate replay; the constant
is asserted in tests so the bump is a deliberate decision.
A future phase will replace the placeholder doc_version "1" with
real per-document versioning — the parameter is in the signature
already so that's a one-line change.
LRUReplayStore is a thin facade over pkg/cache.LRU that maps trace tokens to ReplayEntry values (DocumentID + Query + Model + SelectedIDs + raw ResponseJSON bytes + CreatedAt). The store is safe for concurrent Put/Get, bounds itself by MaxEntries (default 1024) and expires entries past TTL (default 24h). ReplayEntry.ResponseJSON is the literal bytes of the original response — replay returns these verbatim so the byte-exact guarantee holds regardless of how the response is constructed. Go's encoding/json already sorts map keys lexicographically, but storing raw bytes removes any future doubt about determinism. retrieval.replay config block ships with Enabled=true: replay is the moat versus stateless vector RAG and should be on by default. Operators can opt out via retrieval.replay.enabled=false or VLE_RETRIEVAL_REPLAY_ENABLED=false. Capacity / TTL tune via VLE_RETRIEVAL_REPLAY_MAX_ENTRIES and VLE_RETRIEVAL_REPLAY_TTL_SECONDS. Tests cover Put/Get, miss, empty-token safety, LRU eviction at capacity, TTL expiry, in-place update, byte-exactness on tricky payloads (whitespace + unicode), default-TTL non-zero, and a parallel hammer that surfaces races under go test -race. Not durable across process restarts — Phase 3.2 will replace this with persistent storage + per-document versioning. The interface abstraction (ReplayStore) lets that swap happen without touching handlers.
handleQuery and handleAnswer now stamp a deterministic trace_token
into the response body. The exact bytes sent on the wire are also
stored in retrieval.ReplayStore under that token. POST /v1/replay
with {trace_token, query, document_id} returns those bytes
verbatim — same wire bytes, same Content-Type, same trailing
newline.
The byte-exactness chain:
1. The response map is marshalled once with json.Marshal.
Go's encoding/json sorts map[string]any keys lexicographically,
so the same Go value always produces the same []byte.
2. marshalJSONForReplay appends the same trailing newline that
json.Encoder.Encode would, so the wire format is unchanged
from the pre-3.1 behaviour and existing clients see no diff.
3. writeJSONWithReplay writes those bytes to the response AND
hands them to the replay store in lock-step — a single
[]byte, two writes.
4. The replay handler returns store.Get(token).ResponseJSON
verbatim. No re-marshalling, no normalisation.
Replay validation: missing/empty fields → 400, unknown token →
404, mismatched document_id → 409 with details=document_id
differs, mismatched query → 409 with details=query differs. The
order matters: document_id is checked first because it's the
highest-cardinality identifier and surfaces the most useful
"you're pointing at the wrong document" signal.
cmd/engine/main.go wires LRUReplayStore when
retrieval.replay.enabled is true (the default). When disabled,
Deps.Replay is nil — handlers skip the per-response Put and
/v1/replay returns 501.
OpenAPI adds:
- trace_token field on QueryResponse + AnswerResponse
- ReplayRequest schema (all three fields required)
- /v1/replay path with 200/400/404/409/501 documented
- 200's body is oneOf [QueryResponse, AnswerResponse] so the
spec encodes the actual replayed shape
config.example.yaml gets the retrieval.replay block with
inline guidance (opt-out semantics, in-memory v1 caveats,
forward pointer to Phase 3.2).
The internal/api/replay_test.go suite covers byte-exact replay,
unknown token, both mismatch flavours, disabled-store 501,
required-fields 400, malformed JSON 400, unicode/whitespace
preservation, and an end-to-end test that re-marshals the same
map twice and asserts encoding/json is deterministic over the
shape the engine actually emits.
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
|
Warning Review limit reached
More reviews will be available in 16 minutes and 55 seconds. Learn how PR review limits work. Your organization has run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (17)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Phase 3.1 of the engine plan: every retrieval response now carries a deterministic
trace_token, and a newPOST /v1/replayendpoint returns the byte-identical response when given that token plus the original query + document_id. This turns the whitepaper's "every answer is reproducible" claim into a working surface.Design
Trace token shape
trace_token = sha256(doc_id | doc_version | model | system_prompt_version | sorted(selected_ids).joined("\0")), hex-encoded lowercase. 64 chars."a,b" + "c"vs"a" + "b,c").doc_versionis a parameter so Phase 3.2's per-document versioning is a one-line change; today every call passes"1".SystemPromptVersion("v1") is a build-time constant. A test pins it so bumping it is a deliberate, replay-invalidating decision.Byte-exact replay
The chain:
handleQuery/handleAnswerbuild amap[string]anyresponse.marshalJSONForReplaycallsjson.Marshal(which sorts map keys lexicographically) and appends a trailing newline to match the pre-3.1json.Encoder.Encodewire format. Same Go value → same[]byte, always.writeJSONWithReplaywrites those exact bytes to the wire AND hands them to the replay store in lock-step.handleReplayreturnsentry.ResponseJSONverbatim. No re-marshalling, no normalisation.Replay validation
document_idmismatch → 409 withdetails: "document_id differs from original".querymismatch → 409 withdetails: "query differs from original".retrieval.replay.enabled=falseorDeps.Replay == nil) → 501.document_id is checked first because it's the highest-cardinality identifier and surfaces the most useful "you're pointing at the wrong document" signal.
Config
retrieval.replayblock underRetrievalConfig:Env:
VLE_RETRIEVAL_REPLAY_{ENABLED,MAX_ENTRIES,TTL_SECONDS}.Opt-out
Set
retrieval.replay.enabled=false(orVLE_RETRIEVAL_REPLAY_ENABLED=false).Deps.Replaybecomes nil; handlers skip the per-response Put and/v1/replayreturns 501. Thetrace_tokenfield on the response is still computed (it's free) but no replay entry is stored.Test plan
pkg/retrieval/trace_test.go—ComputeTraceToken: hex shape, determinism, sort invariance, no-mutation, input sensitivity (per component), delimiter collision avoidance, empty selection, SystemPromptVersion pinning.pkg/retrieval/replay_test.go— LRUReplayStore: basic Put/Get, miss, empty-token rejection, LRU eviction at capacity, TTL expiry, in-place update, byte-exactness on tricky payloads (unicode + whitespace), default-TTL non-zero, parallel hammer for race detection.pkg/retrieval/retrieval_test.go— extensions: SinglePass and ChunkedTree stamp 64-char hex TraceToken; strategy token matches ComputeTraceToken externally.pkg/config/config_test.go— defaults (replay enabled, 1024, 86400), env overrides (enable, disable, max, ttl), bad-env rejection, negative-value validation.internal/api/replay_test.go— byte-exact replay, unknown token 404, both mismatch flavours 409, disabled-store 501, required-fields 400, malformed JSON 400, unicode/whitespace preservation, end-to-end byte-exactness through marshalJSONForReplay → store → handler.go test ./...).go build ./...andgo vet ./...clean.Known limitation
The replay store is in-memory only — not durable across process restarts. This is the documented v1 limitation; Phase 3.2 will swap
LRUReplayStorefor a persistent store + per-document versioning behind the sameretrieval.ReplayStoreinterface (so handlers don't change).