From f05c0b81f0d8a98712ab67c042844fa041dcebeb Mon Sep 17 00:00:00 2001 From: mnoah1 Date: Wed, 26 Aug 2026 16:41:14 +0000 Subject: [PATCH 1/4] docs(stovepipe): propose request history API MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Summary: Intent: - Expose Stovepipe's retained request history through the same request-ID and URI selector coverage offered by SubmitQueue. - Keep current-state reads owned by operational entities instead of introducing a competing materialized projection. Changes: - Define the representative protobuf contract, public projection, ordering, completeness, authorization, and retention semantics. - Explain why Stovepipe reads append-only entries directly while SubmitQueue materializes gateway-owned request summaries. This PR builds on the request event-history RFC in the parent PR. --- Generated by the 🪄 [pr-create](https://sg.uberinternal.com/code.uber.internal/uber-code/devexp-agent-marketplace/-/blob/claude-code/plugins/dev/uber-dev/skills/pr-create/SKILL.md) skill in devexp-agent-marketplace --- doc/rfc/index.md | 1 + doc/rfc/stovepipe/request-history-api.md | 192 +++++++++++++++++++++++ 2 files changed, 193 insertions(+) create mode 100644 doc/rfc/stovepipe/request-history-api.md diff --git a/doc/rfc/index.md b/doc/rfc/index.md index b03fc856..22e710d9 100644 --- a/doc/rfc/index.md +++ b/doc/rfc/index.md @@ -30,6 +30,7 @@ Design documents and technical proposals, grouped by scope. Shared/cross-cutting - [Buildsignal stage](stovepipe/steps/buildsignal.md) - Build polling, terminal status persistence, and the handoff to record - [Record stage](stovepipe/steps/record.md) - Immutable validation facts keyed by `(queue, uri, project)`, monotonic last-green bookmark advancement and ref promotion, and the deferred hook-event and analyze handoffs - [Request Event History](stovepipe/request-event-history.md) - Append-only request lifecycle model, durable source context, idempotent storage, and reliable write and repair paths +- [Request History API](stovepipe/request-history-api.md) - Queue-scoped request-ID and URI lookup, public projection, materialization decision, ordering, and completeness ## Runway diff --git a/doc/rfc/stovepipe/request-history-api.md b/doc/rfc/stovepipe/request-history-api.md new file mode 100644 index 00000000..d3a41e0c --- /dev/null +++ b/doc/rfc/stovepipe/request-history-api.md @@ -0,0 +1,192 @@ +# Stovepipe Request History API + +## RFC Status + +Proposed. + +## Summary + +Stovepipe exposes retained request history through queue-scoped point lookups by request ID and exact URI. Both selectors return the same ordered history and explicit coverage metadata. + +The API reads the append-only model defined by [Stovepipe Request Event History](request-event-history.md) directly. It does not replay history into current state or introduce a second persisted history projection. Current commit status remains a separate read concern derived from operational entities rather than request history. + +## API Coverage + +The API supports the same selectors as [SubmitQueue Gateway Request History APIs](../submitqueue/history-api.md): + +1. request ID, for callers retaining the receipt returned by `Ingest`; +2. exact URI, for callers starting from a commit identity. + +Both methods require a queue because Stovepipe storage and authorization are queue-scoped. A selector belonging to another queue is not found rather than resolved across shards. + +SubmitQueue's URI method returns several histories because the same change may be submitted repeatedly. Stovepipe ingest permanently deduplicates `(queue, URI)` to one request, so its URI method returns exactly one history. Relaxing that invariant later would require a new plural method rather than changing this method's cardinality in place. + +## Representative Contract + +The final protobuf receives a separate compatibility review before implementation. Its representative shape is: + +```proto +message GetRequestHistoryByRequestIDRequest { + string queue = 1; + string request_id = 2; +} + +message GetRequestHistoryByURIRequest { + string queue = 1; + string uri = 2; +} + +message ValidationResult { + double degree = 1; +} + +message HistoryEntry { + string id = 1; + int64 timestamp_ms = 2; + oneof occurrence { + string request_state = 3; + string event = 4; + } + string superseded_by_request_id = 5; + string build_id = 6; + string outcome_reason = 7; + ValidationResult result = 8; +} + +message RequestHistory { + string request_id = 1; + string uri = 2; + string build_strategy = 3; + string base_uri = 4; + repeated HistoryEntry entries = 5; + bool complete = 6; + int64 complete_from_ms = 7; +} + +message GetRequestHistoryByRequestIDResponse { + RequestHistory history = 1; +} + +message GetRequestHistoryByURIResponse { + RequestHistory history = 1; +} + +service Stovepipe { + rpc GetRequestHistoryByRequestID(GetRequestHistoryByRequestIDRequest) returns (GetRequestHistoryByRequestIDResponse) {} + rpc GetRequestHistoryByURI(GetRequestHistoryByURIRequest) returns (GetRequestHistoryByURIResponse) {} +} +``` + +Entry IDs are opaque. Clients may compare them but never parse their format. Request-state and event values are strings so clients can tolerate additive vocabulary changes. Request version remains persisted for internal transition ordering but is not part of the public entry, matching SubmitQueue's API boundary. + +## Selection Flow + +Request-ID lookup validates the queue and ID, loads the queue's `Request`, and lists its `RequestHistoryStore` entries. Loading the Request distinguishes an unknown ID from an existing pre-cutover request with no retained entries. + +URI lookup resolves the existing `RequestURIStore` primary key and delegates to the same request-ID path. It does not scan history by an entry's `URI` field and requires no new storage index. A missing mapping is not found; a mapping whose Request is missing is an internal consistency error. + +The request-URI mapping must be repaired and retained with its Request and history. Otherwise URI lookup could lose coverage while request-ID lookup still succeeds. The loaded Request supplies URI, build strategy, and base URI once on the `RequestHistory` wrapper; these immutable values are not duplicated on every entry. + +## Public Projection + +Each stored `RequestHistoryEntry` maps to exactly one public `HistoryEntry`. The API does not deduplicate, coalesce, or infer missing occurrences. + +State entries set `request_state` and preserve each durable transition. The public vocabulary is: + +| Public request state | Meaning | +|---|---| +| `accepted` | The request was durably admitted. | +| `processing` | The request was admitted to validation with a selected strategy. | +| `superseded` | A newer request replaced this request before validation completed. | +| `succeeded` | The request's validation work completed successfully. | +| `failed` | The request's validation work failed or could not continue. | +| `cancelled` | The request's validation work was cancelled. | + +These strings intentionally match the current domain states one-to-one, but they are a stable public history vocabulary: an internal refactor cannot rename or reinterpret an existing wire value. In particular, `succeeded` and `failed` remain distinct rather than collapsing into a derived snapshot phase such as `finalizing`. + +Build events set `event` and `build_id`. `validation_fact_recorded` supplies `ValidationResult`; result presence distinguishes a recorded degree of zero from no verdict. The `occurrence` oneof makes state and event mutually exclusive without a redundant type field. A terminal Request state entry never substitutes for the fact event. + +Raw dependency errors, credentials, stack traces, and unrestricted metadata are not exposed. `outcome_reason` uses a bounded public vocabulary. + +## Materialization + +SubmitQueue's `PersistLog` operation performs two jobs after receiving a log message: + +1. append the `RequestLog` row that is itself returned as history; +2. consider status rows for a gateway-owned `RequestSummary`, resolve out-of-order candidates using request version, terminal precedence, and timestamp, then update URI and queue-list projections. + +Event rows remain in SubmitQueue history but never participate in current-status materialization. The materializer exists because the gateway owns public reads but cannot read the orchestrator's mutable Request store. + +Stovepipe has no equivalent ownership gap. The same service owns the queue-scoped `Request`, `ValidationFact`, request-URI mapping, and history stores. Operational reads use their owning entities, while request history reads retained entries directly. + +Stovepipe therefore does not add `RequestSummary`, replay history to determine current state, or materialize another history table. The controller performs only an in-memory wire projection from stored entries to protobuf messages. This avoids a second winner-selection algorithm competing with Request CAS state. + +## Ordering and Consistency + +Entries are returned by `(timestamp_ms ASC, event_id ASC)`. Timestamps are display order, not conflict resolution. Persisted Request versions provide internal causal ordering for state transitions, while Build identity and write-once terminal status ensure one triggered and one finished occurrence per build. + +History may briefly lag a source entity between the source write and history creation. The pipeline blocks its dependent handoff during that window, and retry or reconciliation repairs the missing entry. The read API never fabricates an occurrence from current state. + +Request-ID and URI lookup return the same stored entries, ordering, and completeness metadata. A later repair may insert an older occurrence into its correct chronological position. + +## Historical Completeness + +Existing mutable rows cannot reproduce all transitions that occurred before history writers were enabled. Each response therefore carries coverage metadata: + +- `complete=true` means all occurrences since request acceptance are retained; `complete_from_ms` is the accepted-entry time. +- `complete=false` means earlier occurrences may be missing; `complete_from_ms` is the deployment cutover time. + +Completeness describes coverage, not terminality. An in-progress post-cutover request can be complete even though more entries may arrive. + +An empty history for a known post-cutover request is an internal consistency error. An existing pre-cutover request may return an empty, incomplete history. Rollout metadata must let the controller distinguish those cases without guessing from the latest Request state. + +## Pagination and Retention + +The initial methods return all retained entries. The bounded event vocabulary should keep one request history small; if production sizes disprove that assumption, a separately named paginated method preserves the original return-all contract. + +History, `Request`, and request-URI mapping retention must support the same advertised lookup period. The API does not promise a lifetime longer than every record required by its selector. + +## Errors and Authorization + +- Empty queue or selector is invalid. +- An unknown request ID or URI, including one scoped to the wrong queue, is not found. +- A URI mapping whose Request is missing and a post-cutover Request whose required history is missing are internal consistency errors. +- Retryable storage failures are unavailable; context cancellation and deadline errors retain their canonical codes. + +Authorization follows the same queue policy as other Stovepipe reads. Possession of a request ID alone does not bypass queue authorization. + +## Testing + +Contract and controller tests cover: + +- identical request-ID and URI responses; +- deterministic equal-timestamp ordering; +- state, build-event, and fact-event public mapping; +- queue isolation and cross-queue not found; +- unknown selectors versus dangling mappings; +- complete post-cutover and incomplete pre-cutover histories; +- a repaired older occurrence appearing in chronological position; +- unknown future request-state and event strings remaining readable. + +## Alternatives Considered + +### Materialize current status from history + +Rejected because `Request` and `ValidationFact` already own current operational state and verdicts. Replaying history would add another reconciliation path without enabling either selector. + +### Query history directly by URI + +Rejected because it would add query-by-attribute capability to `RequestHistoryStore`. The existing exact request-URI mapping resolves the primary request key for any backend. + +### Return several histories by URI + +Rejected while ingest enforces one request per `(queue, URI)`. A repeated field would imply cardinality the domain does not permit and make a future deduplication change silently alter existing responses. + +### Paginate immediately + +Rejected until retained sizes demonstrate a need. Adding cursors now would complicate a bounded point read without evidence that one response is unsafe. + +## Open Questions + +1. What retained build count or response size should trigger a paginated method? +2. What common Request, request-URI mapping, and history retention period should both selectors promise? From bdf40ee043fd9fd0a58b75832b1dd34de6688fc2 Mon Sep 17 00:00:00 2001 From: mnoah1 Date: Wed, 26 Aug 2026 17:01:54 +0000 Subject: [PATCH 2/4] docs(stovepipe): paginate request history API --- doc/rfc/stovepipe/request-history-api.md | 40 ++++++++++++++++-------- 1 file changed, 27 insertions(+), 13 deletions(-) diff --git a/doc/rfc/stovepipe/request-history-api.md b/doc/rfc/stovepipe/request-history-api.md index d3a41e0c..4dd7b132 100644 --- a/doc/rfc/stovepipe/request-history-api.md +++ b/doc/rfc/stovepipe/request-history-api.md @@ -6,7 +6,7 @@ Proposed. ## Summary -Stovepipe exposes retained request history through queue-scoped point lookups by request ID and exact URI. Both selectors return the same ordered history and explicit coverage metadata. +Stovepipe exposes retained request history through queue-scoped point lookups by request ID and exact URI. Both selectors return the same ordered, cursor-paginated history and explicit coverage metadata. The API reads the append-only model defined by [Stovepipe Request Event History](request-event-history.md) directly. It does not replay history into current state or introduce a second persisted history projection. Current commit status remains a separate read concern derived from operational entities rather than request history. @@ -29,11 +29,15 @@ The final protobuf receives a separate compatibility review before implementatio message GetRequestHistoryByRequestIDRequest { string queue = 1; string request_id = 2; + int32 page_size = 3; + string page_token = 4; } message GetRequestHistoryByURIRequest { string queue = 1; string uri = 2; + int32 page_size = 3; + string page_token = 4; } message ValidationResult { @@ -65,10 +69,12 @@ message RequestHistory { message GetRequestHistoryByRequestIDResponse { RequestHistory history = 1; + string next_page_token = 2; } message GetRequestHistoryByURIResponse { RequestHistory history = 1; + string next_page_token = 2; } service Stovepipe { @@ -77,19 +83,19 @@ service Stovepipe { } ``` -Entry IDs are opaque. Clients may compare them but never parse their format. Request-state and event values are strings so clients can tolerate additive vocabulary changes. Request version remains persisted for internal transition ordering but is not part of the public entry, matching SubmitQueue's API boundary. +Entry IDs and page tokens are opaque. Clients may compare entry IDs and pass page tokens back to the method that issued them but never parse either format. Request-state and event values are strings so clients can tolerate additive vocabulary changes. Request version remains persisted for internal transition ordering but is not part of the public entry, matching SubmitQueue's API boundary. ## Selection Flow -Request-ID lookup validates the queue and ID, loads the queue's `Request`, and lists its `RequestHistoryStore` entries. Loading the Request distinguishes an unknown ID from an existing pre-cutover request with no retained entries. +Request-ID lookup validates the queue, ID, page size, and optional token; loads the queue's `Request`; and lists one bounded page of its `RequestHistoryStore` entries. Loading the Request distinguishes an unknown ID from an existing pre-cutover request with no retained entries. -URI lookup resolves the existing `RequestURIStore` primary key and delegates to the same request-ID path. It does not scan history by an entry's `URI` field and requires no new storage index. A missing mapping is not found; a mapping whose Request is missing is an internal consistency error. +URI lookup resolves the existing `RequestURIStore` primary key and delegates to the same paginated request-ID path. It does not scan history by an entry's `URI` field and requires no new storage index. A missing mapping is not found; a mapping whose Request is missing is an internal consistency error. The request-URI mapping must be repaired and retained with its Request and history. Otherwise URI lookup could lose coverage while request-ID lookup still succeeds. The loaded Request supplies URI, build strategy, and base URI once on the `RequestHistory` wrapper; these immutable values are not duplicated on every entry. ## Public Projection -Each stored `RequestHistoryEntry` maps to exactly one public `HistoryEntry`. The API does not deduplicate, coalesce, or infer missing occurrences. +Each stored `RequestHistoryEntry` in the selected page maps to exactly one public `HistoryEntry`. The API does not deduplicate, coalesce, or infer missing occurrences. State entries set `request_state` and preserve each durable transition. The public vocabulary is: @@ -123,11 +129,11 @@ Stovepipe therefore does not add `RequestSummary`, replay history to determine c ## Ordering and Consistency -Entries are returned by `(timestamp_ms ASC, event_id ASC)`. Timestamps are display order, not conflict resolution. Persisted Request versions provide internal causal ordering for state transitions, while Build identity and write-once terminal status ensure one triggered and one finished occurrence per build. +Entries are returned across pages by `(timestamp_ms ASC, event_id ASC)`. Timestamps are display order, not conflict resolution. Persisted Request versions provide internal causal ordering for state transitions, while Build identity and write-once terminal status ensure one triggered and one finished occurrence per build. History may briefly lag a source entity between the source write and history creation. The pipeline blocks its dependent handoff during that window, and retry or reconciliation repairs the missing entry. The read API never fabricates an occurrence from current state. -Request-ID and URI lookup return the same stored entries, ordering, and completeness metadata. A later repair may insert an older occurrence into its correct chronological position. +Request-ID and URI lookup return the same stored entries, ordering, pagination semantics, and completeness metadata. A later repair may insert an older occurrence into its correct chronological position. ## Historical Completeness @@ -136,19 +142,24 @@ Existing mutable rows cannot reproduce all transitions that occurred before hist - `complete=true` means all occurrences since request acceptance are retained; `complete_from_ms` is the accepted-entry time. - `complete=false` means earlier occurrences may be missing; `complete_from_ms` is the deployment cutover time. -Completeness describes coverage, not terminality. An in-progress post-cutover request can be complete even though more entries may arrive. +Completeness describes historical coverage, not terminality or pagination. An in-progress post-cutover request can be complete even though more entries may arrive, and every page repeats the same completeness metadata. Only `next_page_token` indicates whether more entries remain in the current traversal. An empty history for a known post-cutover request is an internal consistency error. An existing pre-cutover request may return an empty, incomplete history. Rollout metadata must let the controller distinguish those cases without guessing from the latest Request state. ## Pagination and Retention -The initial methods return all retained entries. The bounded event vocabulary should keep one request history small; if production sizes disprove that assumption, a separately named paginated method preserves the original return-all contract. +The public shape follows SubmitQueue's queue `List` convention: an empty token selects the first page, zero page size selects the server default, and the response returns an opaque `next_page_token` that is empty on the last page. The initial default is 50 entries and the maximum is 200. The controller requests one more entry than the effective page size, returns only the requested page, and issues a token only when the extra entry proves that another page exists. + +Pagination uses the immutable keyset `(timestamp_ms ASC, event_id ASC)`. The versioned token carries the original selector kind and value, queue, resolved request ID, and last returned ordering tuple. Reusing a token with another method, queue, request ID, or URI is invalid. Page size is not bound into the token, so a caller may change it between pages within the server maximum. The controller decodes the public token into the storage contract's typed exclusive cursor; storage implementations never parse wire tokens. + +Tokens are traversal cursors rather than snapshot handles. A new occurrence ordered after the cursor can appear on a later page. A repair that inserts an older occurrence at or before an already-consumed cursor may be observed only by starting a fresh traversal. This is the paginated form of the API's existing eventual-consistency guarantee: a token prevents duplicates from stable retained entries but does not freeze history while pipeline writers and repair are active. History, `Request`, and request-URI mapping retention must support the same advertised lookup period. The API does not promise a lifetime longer than every record required by its selector. ## Errors and Authorization - Empty queue or selector is invalid. +- Negative page sizes, page sizes above the server maximum, malformed tokens, unsupported token versions, and tokens reused with another query are invalid. - An unknown request ID or URI, including one scoped to the wrong queue, is not found. - A URI mapping whose Request is missing and a post-cutover Request whose required history is missing are internal consistency errors. - Retryable storage failures are unavailable; context cancellation and deadline errors retain their canonical codes. @@ -160,6 +171,10 @@ Authorization follows the same queue policy as other Stovepipe reads. Possession Contract and controller tests cover: - identical request-ID and URI responses; +- first, middle, and final pages through both selectors; +- default, changed, maximum, and invalid page sizes; +- malformed, unsupported-version, and cross-query page tokens; +- equal-timestamp keyset boundaries and no duplicates across stable pages; - deterministic equal-timestamp ordering; - state, build-event, and fact-event public mapping; - queue isolation and cross-queue not found; @@ -182,11 +197,10 @@ Rejected because it would add query-by-attribute capability to `RequestHistorySt Rejected while ingest enforces one request per `(queue, URI)`. A repeated field would imply cardinality the domain does not permit and make a future deduplication change silently alter existing responses. -### Paginate immediately +### Return every entry in one response -Rejected until retained sizes demonstrate a need. Adding cursors now would complicate a bounded point read without evidence that one response is unsafe. +Rejected even though SubmitQueue's request-history methods currently do this. Stovepipe permits several builds for one request and anticipates project-scoped builds, so a bounded event vocabulary does not bound entry cardinality. Establishing page semantics now avoids later changing an existing return-all method into a partial response that older clients would silently misread as complete. ## Open Questions -1. What retained build count or response size should trigger a paginated method? -2. What common Request, request-URI mapping, and history retention period should both selectors promise? +1. What common Request, request-URI mapping, and history retention period should both selectors promise? From b948c2cd5334c356476a57f75a7f88499b562017 Mon Sep 17 00:00:00 2001 From: mnoah1 Date: Wed, 26 Aug 2026 17:20:26 +0000 Subject: [PATCH 3/4] docs(stovepipe): clarify page token binding --- doc/rfc/stovepipe/request-history-api.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/rfc/stovepipe/request-history-api.md b/doc/rfc/stovepipe/request-history-api.md index 4dd7b132..337a7609 100644 --- a/doc/rfc/stovepipe/request-history-api.md +++ b/doc/rfc/stovepipe/request-history-api.md @@ -129,7 +129,7 @@ Stovepipe therefore does not add `RequestSummary`, replay history to determine c ## Ordering and Consistency -Entries are returned across pages by `(timestamp_ms ASC, event_id ASC)`. Timestamps are display order, not conflict resolution. Persisted Request versions provide internal causal ordering for state transitions, while Build identity and write-once terminal status ensure one triggered and one finished occurrence per build. +Entries are returned across pages by `(timestamp_ms ASC, entry_id ASC)`. Timestamps are display order, not conflict resolution. Persisted Request versions provide internal causal ordering for state transitions, while Build identity and write-once terminal status ensure one triggered and one finished occurrence per build. History may briefly lag a source entity between the source write and history creation. The pipeline blocks its dependent handoff during that window, and retry or reconciliation repairs the missing entry. The read API never fabricates an occurrence from current state. @@ -150,7 +150,7 @@ An empty history for a known post-cutover request is an internal consistency err The public shape follows SubmitQueue's queue `List` convention: an empty token selects the first page, zero page size selects the server default, and the response returns an opaque `next_page_token` that is empty on the last page. The initial default is 50 entries and the maximum is 200. The controller requests one more entry than the effective page size, returns only the requested page, and issues a token only when the extra entry proves that another page exists. -Pagination uses the immutable keyset `(timestamp_ms ASC, event_id ASC)`. The versioned token carries the original selector kind and value, queue, resolved request ID, and last returned ordering tuple. Reusing a token with another method, queue, request ID, or URI is invalid. Page size is not bound into the token, so a caller may change it between pages within the server maximum. The controller decodes the public token into the storage contract's typed exclusive cursor; storage implementations never parse wire tokens. +Pagination uses the immutable keyset `(timestamp_ms ASC, entry_id ASC)`. The versioned token represents the last returned ordering tuple and is bound to the original selector kind and value, queue, and resolved request ID. The binding does not prescribe whether those values are embedded or fingerprinted in the token. Reusing a token with another method, queue, request ID, or URI is invalid. Page size is not bound into the token, so a caller may change it between pages within the server maximum. The controller decodes the public token into the storage contract's typed exclusive cursor; storage implementations never parse wire tokens. Tokens are traversal cursors rather than snapshot handles. A new occurrence ordered after the cursor can appear on a later page. A repair that inserts an older occurrence at or before an already-consumed cursor may be observed only by starting a fresh traversal. This is the paginated form of the API's existing eventual-consistency guarantee: a token prevents duplicates from stable retained entries but does not freeze history while pipeline writers and repair are active. From 3eec2071bc16af8bc4c612b3c7850400b2f0969a Mon Sep 17 00:00:00 2001 From: mnoah1 Date: Wed, 26 Aug 2026 17:31:38 +0000 Subject: [PATCH 4/4] docs(stovepipe): simplify history API contract --- doc/rfc/stovepipe/request-history-api.md | 32 ++++-------------------- 1 file changed, 5 insertions(+), 27 deletions(-) diff --git a/doc/rfc/stovepipe/request-history-api.md b/doc/rfc/stovepipe/request-history-api.md index 337a7609..babb6081 100644 --- a/doc/rfc/stovepipe/request-history-api.md +++ b/doc/rfc/stovepipe/request-history-api.md @@ -1,12 +1,8 @@ # Stovepipe Request History API -## RFC Status - -Proposed. - ## Summary -Stovepipe exposes retained request history through queue-scoped point lookups by request ID and exact URI. Both selectors return the same ordered, cursor-paginated history and explicit coverage metadata. +Stovepipe exposes retained request history through queue-scoped point lookups by request ID and exact URI. Both selectors return the same ordered, cursor-paginated history. The API reads the append-only model defined by [Stovepipe Request Event History](request-event-history.md) directly. It does not replay history into current state or introduce a second persisted history projection. Current commit status remains a separate read concern derived from operational entities rather than request history. @@ -63,8 +59,6 @@ message RequestHistory { string build_strategy = 3; string base_uri = 4; repeated HistoryEntry entries = 5; - bool complete = 6; - int64 complete_from_ms = 7; } message GetRequestHistoryByRequestIDResponse { @@ -87,7 +81,7 @@ Entry IDs and page tokens are opaque. Clients may compare entry IDs and pass pag ## Selection Flow -Request-ID lookup validates the queue, ID, page size, and optional token; loads the queue's `Request`; and lists one bounded page of its `RequestHistoryStore` entries. Loading the Request distinguishes an unknown ID from an existing pre-cutover request with no retained entries. +Request-ID lookup validates the queue, ID, page size, and optional token; loads the queue's `Request` to validate the selector and supply immutable wrapper context; and lists one bounded page of its `RequestHistoryStore` entries. URI lookup resolves the existing `RequestURIStore` primary key and delegates to the same paginated request-ID path. It does not scan history by an entry's `URI` field and requires no new storage index. A missing mapping is not found; a mapping whose Request is missing is an internal consistency error. @@ -133,18 +127,7 @@ Entries are returned across pages by `(timestamp_ms ASC, entry_id ASC)`. Timesta History may briefly lag a source entity between the source write and history creation. The pipeline blocks its dependent handoff during that window, and retry or reconciliation repairs the missing entry. The read API never fabricates an occurrence from current state. -Request-ID and URI lookup return the same stored entries, ordering, pagination semantics, and completeness metadata. A later repair may insert an older occurrence into its correct chronological position. - -## Historical Completeness - -Existing mutable rows cannot reproduce all transitions that occurred before history writers were enabled. Each response therefore carries coverage metadata: - -- `complete=true` means all occurrences since request acceptance are retained; `complete_from_ms` is the accepted-entry time. -- `complete=false` means earlier occurrences may be missing; `complete_from_ms` is the deployment cutover time. - -Completeness describes historical coverage, not terminality or pagination. An in-progress post-cutover request can be complete even though more entries may arrive, and every page repeats the same completeness metadata. Only `next_page_token` indicates whether more entries remain in the current traversal. - -An empty history for a known post-cutover request is an internal consistency error. An existing pre-cutover request may return an empty, incomplete history. Rollout metadata must let the controller distinguish those cases without guessing from the latest Request state. +Request-ID and URI lookup return the same stored entries, ordering, and pagination semantics. A later repair may insert an older occurrence into its correct chronological position. ## Pagination and Retention @@ -154,14 +137,14 @@ Pagination uses the immutable keyset `(timestamp_ms ASC, entry_id ASC)`. The ver Tokens are traversal cursors rather than snapshot handles. A new occurrence ordered after the cursor can appear on a later page. A repair that inserts an older occurrence at or before an already-consumed cursor may be observed only by starting a fresh traversal. This is the paginated form of the API's existing eventual-consistency guarantee: a token prevents duplicates from stable retained entries but does not freeze history while pipeline writers and repair are active. -History, `Request`, and request-URI mapping retention must support the same advertised lookup period. The API does not promise a lifetime longer than every record required by its selector. +History, `Request`, and request-URI mapping retention must support the same advertised lookup period. The API does not promise a lifetime longer than every record required by its selector. The initial rollout exposes only requests accepted after all history writers and repair paths are active; older requests are outside the lookup period and are not backfilled. ## Errors and Authorization - Empty queue or selector is invalid. - Negative page sizes, page sizes above the server maximum, malformed tokens, unsupported token versions, and tokens reused with another query are invalid. - An unknown request ID or URI, including one scoped to the wrong queue, is not found. -- A URI mapping whose Request is missing and a post-cutover Request whose required history is missing are internal consistency errors. +- A URI mapping whose Request is missing and a request within the advertised history lookup period whose required history is missing are internal consistency errors. - Retryable storage failures are unavailable; context cancellation and deadline errors retain their canonical codes. Authorization follows the same queue policy as other Stovepipe reads. Possession of a request ID alone does not bypass queue authorization. @@ -179,7 +162,6 @@ Contract and controller tests cover: - state, build-event, and fact-event public mapping; - queue isolation and cross-queue not found; - unknown selectors versus dangling mappings; -- complete post-cutover and incomplete pre-cutover histories; - a repaired older occurrence appearing in chronological position; - unknown future request-state and event strings remaining readable. @@ -200,7 +182,3 @@ Rejected while ingest enforces one request per `(queue, URI)`. A repeated field ### Return every entry in one response Rejected even though SubmitQueue's request-history methods currently do this. Stovepipe permits several builds for one request and anticipates project-scoped builds, so a bounded event vocabulary does not bound entry cardinality. Establishing page semantics now avoids later changing an existing return-all method into a partial response that older clients would silently misread as complete. - -## Open Questions - -1. What common Request, request-URI mapping, and history retention period should both selectors promise?