Skip to content

Commit 607704c

Browse files
committed
feat(speculation): add selector + selection-limit extensions
Add the selector seam and its limit counterpart from the speculation RFC, as vendor-agnostic extension interfaces under submitqueue/extension/speculation/. selector: the controller hands it the batch's speculation tree; it returns a per-path decision (Promote/Cancel) for the paths it chooses to act on. It is the policy seam — it reads the tree it is given and emits decisions only, never writing status, and does not decide merging. The controller maps each decision to a guarded status transition (Promote → Selected, Cancel → Cancelling/Cancelled) and persists it, staying the single writer of tree state. selectionlimit: the "how much" policy bounding how many paths a batch may build in parallel. It is the selector's companion — the selector decides which paths, the limit decides how many. Injected into the selector at construction and called by it, keeping the selector interface limit-free; the value is dynamic, not a fixed constant. Each follows the repo extension contract: Factory.For(Config) (T, error) with Config carrying only QueueName. Includes READMEs, gomock packages, and programmable fakes. Interfaces only; concrete impls and controller wiring are deferred.
1 parent ab06623 commit 607704c

17 files changed

Lines changed: 630 additions & 1 deletion

File tree

Makefile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -364,7 +364,7 @@ local-stovepipe-stop: ## Stop the Stovepipe service
364364

365365
mocks: ## Generate mock files using mockgen
366366
@echo "Generating mocks..."
367-
@$(BAZEL) run @rules_go//go -- generate ./submitqueue/extension/storage/... ./submitqueue/extension/buildrunner/... ./submitqueue/extension/changeprovider/... ./platform/extension/counter/... ./platform/extension/messagequeue/... ./submitqueue/extension/queueconfig/... ./submitqueue/extension/mergechecker/... ./submitqueue/extension/pusher/... ./submitqueue/extension/scorer/... ./submitqueue/extension/conflict/... ./submitqueue/extension/speculation/enumerator/... ./submitqueue/extension/speculation/dependencylimit/... ./submitqueue/extension/speculation/pathscorer/... ./submitqueue/extension/validator/... ./platform/consumer/... ./submitqueue/core/changeset/... ./stovepipe/extension/storage/... ./stovepipe/extension/sourcecontrol/...
367+
@$(BAZEL) run @rules_go//go -- generate ./submitqueue/extension/storage/... ./submitqueue/extension/buildrunner/... ./submitqueue/extension/changeprovider/... ./platform/extension/counter/... ./platform/extension/messagequeue/... ./submitqueue/extension/queueconfig/... ./submitqueue/extension/mergechecker/... ./submitqueue/extension/pusher/... ./submitqueue/extension/scorer/... ./submitqueue/extension/conflict/... ./submitqueue/extension/speculation/enumerator/... ./submitqueue/extension/speculation/dependencylimit/... ./submitqueue/extension/speculation/selector/... ./submitqueue/extension/speculation/selectionlimit/... ./submitqueue/extension/validator/... ./submitqueue/extension/speculation/pathscorer/... ./platform/consumer/... ./submitqueue/core/changeset/... ./stovepipe/extension/storage/... ./stovepipe/extension/sourcecontrol/...
368368
@echo "Mocks generated successfully!"
369369

370370
proto: ## Generate protobuf files from .proto definitions
Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
load("@rules_go//go:def.bzl", "go_library")
2+
3+
go_library(
4+
name = "go_default_library",
5+
srcs = ["selectionlimit.go"],
6+
importpath = "github.com/uber/submitqueue/submitqueue/extension/speculation/selectionlimit",
7+
visibility = ["//visibility:public"],
8+
)
Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
# Speculation Selection Limit
2+
3+
Vendor-agnostic "how much" policy that bounds how many paths a batch may build in parallel.
4+
5+
See the [Speculation RFC](/doc/rfc/submitqueue/speculation.md) for the end-to-end design and how limits fit into the two-layer speculation model.
6+
7+
## Selection Limit
8+
9+
The selection limit is the [selector](../selector)'s companion. The selector decides **which** of a batch's paths are worth building — its ranking over the tree; the selection limit decides **how many** of them may run at once. Keeping "which" and "how much" separate keeps selector logic free of resource accounting and lets the bound scale with build resources without touching that logic.
10+
11+
The value is **signal-driven**, not a fixed constant. Its primary input is the build system's available capacity, but a policy may also weigh historical pass rates, cost budgets, time of day, or an experiment toggle.
12+
13+
Unlike the dependency limit — which the controller holds and applies as an eligibility gate — the selection limit is **injected into the seam that uses it**: the selector is constructed with it and calls it itself, never receiving it as a method parameter. This follows the repo's extension-contract pattern (dependencies injected at the `Factory`), keeps the selector interface limit-free and stable, and lets the limit be swapped independently of selector logic.
14+
15+
## Factory
16+
17+
A per-queue factory returns the limit policy for a queue, following the repo's extension contract. It is handed only the queue identity; the signals a policy weighs — a capacity feed, historical metrics, config — are injected at construction by the integrator in the wiring layer, which is also where the limit is handed to the selector. Computing the limit itself takes no further inputs.
Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
load("@rules_go//go:def.bzl", "go_library", "go_test")
2+
3+
go_library(
4+
name = "go_default_library",
5+
srcs = ["fake.go"],
6+
importpath = "github.com/uber/submitqueue/submitqueue/extension/speculation/selectionlimit/fake",
7+
visibility = ["//visibility:public"],
8+
deps = ["//submitqueue/extension/speculation/selectionlimit:go_default_library"],
9+
)
10+
11+
go_test(
12+
name = "go_default_test",
13+
srcs = ["fake_test.go"],
14+
embed = [":go_default_library"],
15+
deps = [
16+
"@com_github_stretchr_testify//assert:go_default_library",
17+
"@com_github_stretchr_testify//require:go_default_library",
18+
],
19+
)
Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
// Copyright (c) 2025 Uber Technologies, Inc.
2+
//
3+
// Licensed under the Apache License, Version 2.0 (the "License");
4+
// you may not use this file except in compliance with the License.
5+
// You may obtain a copy of the License at
6+
//
7+
// http://www.apache.org/licenses/LICENSE-2.0
8+
//
9+
// Unless required by applicable law or agreed to in writing, software
10+
// distributed under the License is distributed on an "AS IS" BASIS,
11+
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
// See the License for the specific language governing permissions and
13+
// limitations under the License.
14+
15+
// Package fake provides a programmable selectionlimit.SelectionLimit for tests
16+
// and examples. New sets the value returned by Limit; FailWith injects an error
17+
// on every call. It is intended for examples and tests only, never production.
18+
package fake
19+
20+
import (
21+
"context"
22+
23+
"github.com/uber/submitqueue/submitqueue/extension/speculation/selectionlimit"
24+
)
25+
26+
// SelectionLimit is a programmable selectionlimit.SelectionLimit.
27+
type SelectionLimit struct {
28+
limit int
29+
err error
30+
}
31+
32+
// New returns a fake SelectionLimit whose Limit returns the given value.
33+
func New(limit int) *SelectionLimit {
34+
return &SelectionLimit{limit: limit}
35+
}
36+
37+
// FailWith makes every Limit call return err.
38+
func (l *SelectionLimit) FailWith(err error) *SelectionLimit {
39+
l.err = err
40+
return l
41+
}
42+
43+
// Limit returns the configured value, or the injected error if FailWith was set.
44+
func (l *SelectionLimit) Limit(_ context.Context) (int, error) {
45+
if l.err != nil {
46+
return 0, l.err
47+
}
48+
return l.limit, nil
49+
}
50+
51+
// ensure the fake satisfies the interface.
52+
var _ selectionlimit.SelectionLimit = (*SelectionLimit)(nil)
Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
// Copyright (c) 2025 Uber Technologies, Inc.
2+
//
3+
// Licensed under the Apache License, Version 2.0 (the "License");
4+
// you may not use this file except in compliance with the License.
5+
// You may obtain a copy of the License at
6+
//
7+
// http://www.apache.org/licenses/LICENSE-2.0
8+
//
9+
// Unless required by applicable law or agreed to in writing, software
10+
// distributed under the License is distributed on an "AS IS" BASIS,
11+
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
// See the License for the specific language governing permissions and
13+
// limitations under the License.
14+
15+
package fake
16+
17+
import (
18+
"context"
19+
"errors"
20+
"testing"
21+
22+
"github.com/stretchr/testify/assert"
23+
"github.com/stretchr/testify/require"
24+
)
25+
26+
func TestLimit_ReturnsConfiguredValue(t *testing.T) {
27+
got, err := New(2).Limit(context.Background())
28+
require.NoError(t, err)
29+
assert.Equal(t, 2, got)
30+
}
31+
32+
func TestLimit_FailWith(t *testing.T) {
33+
sentinel := errors.New("boom")
34+
_, err := New(2).FailWith(sentinel).Limit(context.Background())
35+
require.ErrorIs(t, err, sentinel)
36+
}
Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
load("@rules_go//go:def.bzl", "go_library")
2+
3+
go_library(
4+
name = "go_default_library",
5+
srcs = ["selectionlimit_mock.go"],
6+
importpath = "github.com/uber/submitqueue/submitqueue/extension/speculation/selectionlimit/mock",
7+
visibility = ["//visibility:public"],
8+
deps = [
9+
"//submitqueue/extension/speculation/selectionlimit:go_default_library",
10+
"@org_uber_go_mock//gomock:go_default_library",
11+
],
12+
)

submitqueue/extension/speculation/selectionlimit/mock/selectionlimit_mock.go

Lines changed: 96 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
// Copyright (c) 2025 Uber Technologies, Inc.
2+
//
3+
// Licensed under the Apache License, Version 2.0 (the "License");
4+
// you may not use this file except in compliance with the License.
5+
// You may obtain a copy of the License at
6+
//
7+
// http://www.apache.org/licenses/LICENSE-2.0
8+
//
9+
// Unless required by applicable law or agreed to in writing, software
10+
// distributed under the License is distributed on an "AS IS" BASIS,
11+
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
// See the License for the specific language governing permissions and
13+
// limitations under the License.
14+
15+
package selectionlimit
16+
17+
//go:generate mockgen -source=selectionlimit.go -destination=mock/selectionlimit_mock.go -package=mock
18+
19+
import "context"
20+
21+
// SelectionLimit is the "how much" policy that bounds how many paths a batch may
22+
// build in parallel.
23+
//
24+
// It is the selector's companion: the selector decides *which* of a batch's
25+
// paths are worth building (its ranking); the selection limit decides *how many*
26+
// of them may run at once. Separating the two keeps selector logic free of
27+
// resource accounting and lets the bound scale with build resources without
28+
// touching that logic.
29+
//
30+
// The value is dynamic: it may change between calls, so the selector reads it
31+
// each pass rather than caching it.
32+
//
33+
// Unlike the dependency limit, this limit is injected into the seam that uses it
34+
// — the selector is constructed with it and calls it itself — never passed as a
35+
// method parameter, keeping the selector interface limit-free and stable.
36+
type SelectionLimit interface {
37+
// Limit returns the current maximum number of paths a batch may build in
38+
// parallel. The selector caps its Promote decisions at this. It takes no
39+
// parameters; anything an implementation needs is injected at construction.
40+
Limit(ctx context.Context) (int, error)
41+
}
42+
43+
// Config carries the per-queue identity handed to a Factory. The system knows
44+
// only the queue name; everything a policy needs to compute the limit (a
45+
// capacity feed, historical metrics, config) is injected at construction by the
46+
// integrator.
47+
type Config struct {
48+
// QueueName identifies the queue this SelectionLimit serves.
49+
QueueName string
50+
}
51+
52+
// Factory builds the SelectionLimit for a queue. Implementations are provided by
53+
// integrators (and tests) and inject whatever signals they need at construction.
54+
type Factory interface {
55+
// For returns the SelectionLimit for the given queue.
56+
For(cfg Config) (SelectionLimit, error)
57+
}
Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
load("@rules_go//go:def.bzl", "go_library")
2+
3+
go_library(
4+
name = "go_default_library",
5+
srcs = ["selector.go"],
6+
importpath = "github.com/uber/submitqueue/submitqueue/extension/speculation/selector",
7+
visibility = ["//visibility:public"],
8+
deps = ["//submitqueue/entity:go_default_library"],
9+
)

0 commit comments

Comments
 (0)