You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Requires Docker and Docker Compose. See [Development Setup](doc/howto/DEVELOPMENT.md) for full prerequisites.
18
+
Land a change and watch it reach `landed`. Requires Docker and Docker Compose, and nothing else — no repository, no account, no token. See [Development Setup](doc/howto/DEVELOPMENT.md) for full prerequisites.
19
19
20
20
```bash
21
-
# Build everything
22
-
make build
23
-
24
-
# Run unit tests
25
-
make test
26
-
27
-
# Start full stack locally (Gateway + Orchestrator + MySQL via Docker Compose)
21
+
# Start the full stack (Gateway + Orchestrator + Runway + MySQL)
make land-status QUEUE=test-queue SQID=test-queue/1
32
32
33
33
# Stop services
34
34
make local-stop
35
35
```
36
36
37
-
See [service/README.md](service/README.md) for more examples including running individual services and clients.
37
+
Every integration at the edges is faked — the change provider, CI, and the merge itself — so the run is free and finishes in seconds. The queue's own logic is real: validation, batching, conflict analysis, and speculation all run, and the request log records the full trail from `accepted` to `landed`. Nothing is pushed to any repository.
38
+
39
+
[Quickstart](doc/howto/QUICKSTART.md) explains the change URI, how to make a change fail on demand, and what this does and does not prove. From there, `make e2e-git-test` adds a real git merge (still no credentials), and [PROVIDER-E2E.md](doc/howto/PROVIDER-E2E.md) adds a live provider. See [service/README.md](service/README.md) for running individual services and clients.
38
40
39
41
## Documentation
40
42
41
43
| Document | Description |
42
44
|----------|-------------|
45
+
|[Quickstart](doc/howto/QUICKSTART.md)| Land a change locally with no credentials |
43
46
|[Development Setup](doc/howto/DEVELOPMENT.md)| Prerequisites, build, environment, IDE setup |
44
47
|[Contributing](CONTRIBUTING.md)| How to contribute, workflow, guidelines |
45
48
|[Testing Guide](doc/howto/TESTING.md)| Unit, integration, and E2E testing patterns |
49
+
|[Landing real changes](doc/howto/PROVIDER-E2E.md)| Running the pipeline against a live provider |
make land-status QUEUE=test-queue SQID=test-queue/1
68
71
69
72
# 5. Stop services
70
73
make local-stop
71
74
```
72
75
76
+
[QUICKSTART.md](QUICKSTART.md) walks through the same run in detail — what the change URI has to look like, how to make a change fail on demand, and which parts of the pipeline are faked.
77
+
73
78
If any step fails, see [Troubleshooting](#troubleshooting) below.
Copy file name to clipboardExpand all lines: doc/howto/PROVIDER-E2E.md
+43-4Lines changed: 43 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -97,27 +97,32 @@ Opening pull requests by hand gets old fast. `demo-pr` creates them, enqueues th
97
97
make demo-pr # 3 independent PRs, each enqueued as it is created
98
98
make demo-pr COUNT=8 # more traffic
99
99
make demo-pr FILES=8 # wider changes, more files per PR
100
+
make demo-pr CONCURRENCY=1 # create them one at a time
100
101
make demo-pr STACKED=true # one stack, enqueued as a single request
101
102
make demo-pr LAND=false # create only, print the land command
102
103
```
103
104
104
105
Each pull request is enqueued the moment it exists, so the queue is already working on the first while the last is still being opened. That overlap is the point: a queue holding one request at a time never batches, never analyzes a conflict against another batch, and never speculates. Nothing is awaited until every request is in.
105
106
107
+
Independent pull requests are created **five at a time** by default (`CONCURRENCY`). Opening one is several round trips — a branch, a commit per file, the pull request itself — so creating them serially was most of what a large run spent its time on, and it delayed the overlap the demo exists to show. A stack ignores the setting: each of its changes is based on the branch before it, so the next cannot be cut until the previous head exists. Lower it if the provider starts refusing bursts.
108
+
106
109
The table is there from the start — one row per land request, drawn before the first pull request exists and filled in as the run proceeds. Whatever is happening right now is a single line underneath it, so creating and enqueuing does not scroll the table away:
demo-queue/13 #32 31s accepted → started → validating → validated
113
118
demo-queue/14 #33 28s accepted → started
114
119
115
120
▸ 1 of 3 settled
116
121
```
117
122
118
123
Each row shows the states its request passed through, not just the one it is in. That comes from the gateway's history API rather than from sampling the current status, so a transition between two polls is not missed. `CHANGES` links to the pull request: on a terminal `#31` is clickable, and in a redirected run it is written out as a full URL instead. `ELAPSED` runs from the moment the gateway accepted the request and stops when it settles, so a finished row keeps the time it took rather than counting on.
119
124
120
-
The trail is only as detailed as what the pipeline reports, which today is `accepted`, `started`, `validated`, `batched`and then a terminal `landed`, `error` or `cancelled`. The finer-grained statuses the API defines — `speculating`, `building`, `landing` and the rest — are never published, so a request sits on `batched` for the whole of its active life even while its batch is speculating and building. Do not read that as the request being stuck.
125
+
The trail is as detailed as what the pipeline reports, which is the full walk: `accepted`, `started`, `validating`, `validated`, `batched`, `speculating`, `speculated`, `landing`, and then a terminal `landed`, `error` or `cancelled`. `building` and `built` are recorded alongside as events rather than statuses. A long pause on `speculating` is the batch waiting on its build, not a stuck request.
121
126
122
127
`STACKED=true` is the exception to the overlap: one request carries the whole chain, so it can only go in once every pull request in it exists. That is the atomic-stack path — the whole set reaches `main` in a single push, and the table shows it as the single row it is.
123
128
@@ -129,6 +134,40 @@ The command exits non-zero if any request settles anywhere other than `landed`,
129
134
130
135
## Watching it work
131
136
137
+
The queue itself is readable without creating any traffic:
138
+
139
+
```bash
140
+
make land-list # a table of recent requests
141
+
make land-list SINCE=24h LIMIT=200 # a wider window
142
+
make land-watch # follow them until they settle
143
+
```
144
+
145
+
Both draw the same table `make demo-pr` does — the demo tool and the CLI share it — but against whatever the queue already holds, so watching a queue no longer means adding to it. `land-watch` fixes its set when it starts and exits non-zero if any request in that set finishes anywhere other than `landed`, which makes it usable from a script. A request accepted after the watch begins is not picked up: a watch that grew as the queue did would never finish.
146
+
147
+
Under the hood these are `client list` and `client watch`, which take a queue and reach any gateway:
148
+
149
+
```bash
150
+
bazel run //service/submitqueue/gateway/client:gateway -- \
151
+
-addr sq.example.com:443 -tls list -queue my-queue -since 1h
152
+
```
153
+
154
+
`-addr`is passed to the dialler untouched, so `dns:///host:port` and `unix:///path.sock` work as well as a plain `host:port`. Transport security is a separate flag rather than part of the address, because gRPC keeps target resolution and credentials apart — there is no `grpcs://` to write.
155
+
156
+
A listing of a busy queue is mostly `speculating` rows, since that is where a request spends most of its active life — waiting on the build its batch was admitted for.
157
+
158
+
### Authentication
159
+
160
+
The gateway admits every caller. It is a sandbox stack, and nothing in it checks a credential.
161
+
162
+
The client can still present one, for a gateway reached through something that does — a proxy, a mesh sidecar, an ingress that terminates auth ahead of the service. It reads `SQ_TOKEN` by default and sends it as `Authorization: Bearer …`; `-token-env` names a different variable, and an unset one sends nothing rather than failing, which is how it stays usable against a stack that wants no credential.
163
+
164
+
```bash
165
+
SQ_TOKEN=$(cat ~/.sq-token) bazel run //service/submitqueue/gateway/client:gateway -- \
166
+
-addr sq.example.com:443 -tls list -queue my-queue
0 commit comments