feat(speculation): add enumerator and selector extensions#232
Draft
behinddwalls wants to merge 3 commits into
Draft
feat(speculation): add enumerator and selector extensions#232behinddwalls wants to merge 3 commits into
behinddwalls wants to merge 3 commits into
Conversation
This was referenced Jun 9, 2026
behinddwalls
force-pushed
the
preetam/ext/speculation-extensions
branch
from
6ff6d6c to
5087796
Compare
behinddwalls
force-pushed
the
preetam/ext/speculation-store
branch
from
927e240 to
6bc8426
Compare
behinddwalls
force-pushed
the
preetam/ext/speculation-extensions
branch
from
5087796 to
5064a60
Compare
behinddwalls
force-pushed
the
preetam/ext/speculation-store
branch
from
6bc8426 to
7b49586
Compare
Capture how SubmitQueue speculates before the interfaces land: the problem (dependent batches serialize behind multi-minute builds), the Base/Head path-and-tree model, where speculation sits in the orchestrator pipeline, and the two pluggable seams it splits into — tree enumeration and path selection. Also records the status-vs-action ownership split and speculation depth. Design only; the entities, store, and extension interfaces follow in later PRs in this stack. Adds doc/rfc/submitqueue/speculation.md and links it from the RFC index.
Reshape the speculation tree data model around the Base/Head path that the build stage actually consumes, and align its store.
entity: SpeculationPath{Base, Head} becomes the unit; SpeculationPathInfo carries the path plus its enumerator Score, controller-owned Status (candidate/selected/building/passed/failed/cancelled), and BuildID. Adds SpeculationPathAction and SpeculationPathDecision for the selector seam. SpeculationTree.Speculations becomes Paths. The Build entity drops its own local SpeculationPathInfo and uses the shared SpeculationPath (Head = the batch under verification), with the build controller updated to match.
storage: SpeculationTreeStore.UpdateSpeculations(batchID, []SpeculationInfo) becomes Update(ctx, SpeculationTree) — symmetric with Create, keyed by tree.BatchID. MySQL impl and mock updated.
Add the two pluggable seams from the speculation RFC, as vendor-agnostic extension interfaces under submitqueue/extension/speculation/. enumerator: given a batch and its dependency batches (carrying per-batch Score), mechanically lists the candidate Base/Head paths and scores each — pure, deterministic, status-free. selector: given a speculation tree with controller-stamped status, returns a per-path action (Build/Cancel) — the policy seam; reads status, emits actions, never writes status. Each follows the repo extension contract (conflict.Analyzer reference shape): Factory.For(Config) (T, error) with Config carrying only QueueName; behavioral knobs are integrator-injected at construction. Includes READMEs, gomock packages, and Makefile mock-gen wiring. Interfaces only; concrete impls and controller wiring are deferred.
behinddwalls
force-pushed
the
preetam/ext/speculation-extensions
branch
from
5064a60 to
d42228b
Compare
behinddwalls
added a commit
that referenced
this pull request
Jul 7, 2026
## Summary Capture how SubmitQueue speculates before the interfaces land. Speculation is modeled as two layers: **decision seams** — enumeration and **scoring** *describe* the tree of possible bets, selection and prioritization *act* on it — and **signal-driven limit policies** that decide *how much* to allow at each step (dependency limit, selection limit, prioritization limit), scaling with the build system's resources. A path's score is dynamic — recomputed as dependencies land, fail, or have their builds pass — not frozen at enumeration. Covers: the problem (dependent batches serialize behind multi-minute builds), the Base/Head path-and-tree model, where speculation sits in the orchestrator pipeline, the status-vs-action ownership split, the dependency limit as an eligibility gate (batches wait for predecessors to land rather than trimming their base), dynamic rescoring, and how prioritization rations the queue's builds against its budget. Design only; entities, store, and extension interfaces follow in later PRs in this stack. Adds doc/rfc/submitqueue/speculation.md and links it from the RFC index. ### Changes since review Reframed from two seams (enumeration + selection) to describe/act decisions + limits, addressing @sbalabanov's review: - selector resource-awareness → the new **prioritization** layer owns the queue-wide build budget; the selector stays a pure per-batch policy - "variable depth, not a single knob" → the **dependency limit** is a signal-driven policy, not a constant - clarified the dependency limit is an **eligibility gate** on active in-flight dependencies (a batch waits until predecessors land), not a base trim - **dynamic scoring**: split scoring out of enumeration into a **scorer** that recomputes each path's score as dependencies land/fail/pass — enumeration is now structure-only - entire tree persisted (the selector reads all path statuses + scores, so it can cancel a path it earlier asked to build); the redundant fallback path is cancelled on a winning bet; slash-free example names; trimmed impl-detail bullets - recorded as open: extending rescoring to weigh how long a batch has waited ## Test Plan Doc-only. Rendered locally; ASCII diagrams and tables verified; no hard-wrapped prose. ## Issues ## Stack 1. @ #230 1. #231 1. #232
behinddwalls
force-pushed
the
preetam/ext/speculation-store
branch
from
7b49586 to
4606cc7
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add the two pluggable seams from the speculation RFC, as vendor-agnostic extension interfaces under submitqueue/extension/speculation/.
enumerator: given a batch and its dependency batches (carrying per-batch Score), mechanically lists the candidate Base/Head paths and scores each — pure, deterministic, status-free.
selector: given a speculation tree with controller-stamped status, returns a per-path action (Build/Cancel) — the policy seam; reads status, emits actions, never writes status.
Each follows the repo extension contract (conflict.Analyzer reference shape): Factory.For(Config) (T, error) with Config carrying only QueueName; behavioral knobs are integrator-injected at construction. Includes READMEs, gomock packages, and Makefile mock-gen wiring. Interfaces only; concrete impls and controller wiring are deferred.
Test Plan
Issues
Stack