Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -78,15 +78,15 @@ Compass ships as two products over one shared core:
boundary.
All of this record's `go/internal/*` and `agent-image/*` citations are
public-repo paths.
- **Managed Compass (private monorepo, dual AGPL + commercial license).** The
- **Managed Compass (dual AGPL + commercial license), built out of tree.** The
hosted multi-tenant service, which *reuses* the OSS core rather than forking
it.

**This record designs a change to the OSS core.** All three axes and all seven
tasks land in the public repo — that is why the record lives here and every
code citation resolves against `RigelBuild/compass`. Nothing in the managed
control plane (tenant orchestration, billing, the hosted control surface, all
of which live in the private monorepo) is designed here.
of which are built out of tree) is designed here.

**The motivation is primarily the managed service.** Hosted-scale density (pack
many tenants, suspend the idle ones), durability across eviction, and a
Expand Down Expand Up @@ -890,8 +890,8 @@ Nothing pinned in the Approach is re-opened here.
as a microVM OCI runtime (krun/libkrun or kata via podman `--runtime`), so
the seam is expected to hold; the image/boot/egress plumbing above it is
the real work I1 owns. Through Dogfood + trusted-tenant Beta the rootless
container remains the running boundary; I1 lands the microVM before the
first external multi-tenant tenant.
container remains the running boundary; I1 lands the microVM before it
becomes the sole runtime.
6. **[non-load-bearing] Cold-idle object-store archive — thresholds.** D4
archives an idle session's volume to the object store past a deeper idle
threshold (freeing local disk; a cold-idle session costs object storage
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -192,7 +192,7 @@ the leg; a re-run clears it. If that proves frequent, the Woodpecker fallback
If hosted-runner KVM proves unreliable in practice — frequent reds from
withheld `/dev/kvm`, or GitHub tightening access — the fallback is to **enroll
the compass repo itself on the self-hosted Woodpecker fleet** and run the KVM
leg there. This is explicitly *compass's own* CI, not the private monorepo's:
leg there. This is explicitly *compass's own* CI, not the managed service's:
compass features are tested in compass. The fleet has a registered self-hosted
Linux CI agent (x86_64, KVM-capable, docker backend) already, so the hardware
is in place; what is missing is a compass pipeline (no `.woodpecker*` config
Expand Down Expand Up @@ -220,11 +220,11 @@ same env + tag contract E-D2's GHA step uses, so adopting the fallback is a
lane swap, not a redesign. It also realizes V8's "CI job (KVM-labeled runner)"
(`microvm-runner.md:606-608`).

*Separate concern — the managed service:* the managed Compass will run from the
private monorepo, and that monorepo will need its own KVM CI when the managed
buildout starts. That is deferred to the managed-service control-plane work
(the fleet control plane, RIG-2485), not this record — this record keeps
compass's own tests in compass.
*Separate concern — the managed service:* the managed service is built out of
tree and will need its own KVM CI when its buildout starts. That is
deferred to the managed-service control-plane work (the fleet control
plane, RIG-2485), not this record — this record keeps compass's own
tests in compass.

### E-D4 — guest image: derivations beside the agent image, kernel from nixpkgs, a project binary cache (F4)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -78,8 +78,8 @@ timeline:

- `design.md:892-894` (OQ-5, the resolved inter-tenant-boundary decision):
"Through Dogfood + trusted-tenant Beta the rootless container remains the
running boundary; I1 lands the microVM before the first external multi-tenant
tenant." This stays exactly as frozen, and `microvm-runner.md:60-65` restates
running boundary; I1 lands the microVM before it becomes the sole
runtime." This stays exactly as frozen, and `microvm-runner.md:60-65` restates
it: the container is the running boundary **through Beta**, then removed. The
container is a transitional bootstrap, not a KVM-absent fallback — the two are
different roles, and only the fallback role is retired here.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -59,10 +59,10 @@ inter-tenant boundary IS a hardware-virtualized microVM, design.md:881-894;
the virtio-fs no-copy invariant, design.md:586-587) and replaces only the
falsified implementation mechanism. Per design.md:892-894, "Through Dogfood +
trusted-tenant Beta the rootless container remains the running boundary" — so
this work runs in parallel with M0/S1/P2/C3 and blocks nothing before the
first external multi-tenant tenant. Beyond that point the container path is
**removed** and microVM becomes the sole runtime (D2); the container backend
is a transitional bootstrap, not a permanent second runtime.
this work runs in parallel with M0/S1/P2/C3 and is off the critical path. Once
microVM is proven the container path is **removed** and microVM becomes the
sole runtime (D2); the container backend is a transitional bootstrap, not a
permanent second runtime.

## Approach

Expand Down Expand Up @@ -403,8 +403,8 @@ demand via cloud-hypervisor hotplug rather than reserving peak RAM (D5).
- **Transitional container path, then microVM-only (D2).** The rootless
container remains the running boundary through Dogfood + trusted-tenant Beta
(design.md:892-894) and is then **removed**: microVM is the sole runtime.
This work runs in parallel with M0/S1/P2/C3 and gates nothing before the
first external multi-tenant tenant.
This work runs in parallel with M0/S1/P2/C3 and is off the critical
path.

## Plan

Expand Down Expand Up @@ -676,20 +676,20 @@ each is kept so the executor sees *why*, not just *what*.
second permanent runtime roughly *doubles* the production support and
bugfix surface, which for a solo maintainer dominates the one-time build
cost — and microVM must be built regardless (it is the reason the parent
design exists). Reach is preserved without a second runtime: the managed
service runs cloud-hypervisor on **elastic, hourly, autoscaling bare-metal**
instances (AWS `*.metal` in standard ASGs; GCP `c3-*-metal` "consumed and
managed in the same way as VM instances") — no nesting, so **no ~10%
nested-virtualization performance tax**; self-hosters provision a
KVM-capable box *for* Compass (an owned/homelab box, Hetzner-class
bare-metal, or a nested-virt-enabled hyperscaler instance — GCP any Linux
VM, Azure Dv3/Ev3+, AWS C8i/M8i/R8i or `.metal`). **Native-macOS-embedded
is dropped**: macOS has no KVM, Compass is an always-on/overnight workload
ill-suited to a personal Mac anyway, and macOS users use the managed
service or point the app at a remote KVM Runner. The ~10% tax therefore
only ever applies to a self-hoster who *chooses* a nested cloud VM over
bare-metal — a self-inflicted, clearly-documented tradeoff, never the
managed path.
design exists). Reach is preserved without a second runtime:
cloud-hypervisor needs only a KVM-capable host, and running
it directly on bare metal means no nesting and therefore **no
~10% nested-virtualization performance tax**. Self-hosters
provision a KVM-capable box *for* Compass (an owned/homelab
box, Hetzner-class bare-metal, or a nested-virt-enabled
hyperscaler instance — GCP any Linux VM, Azure Dv3/Ev3+,
AWS C8i/M8i/R8i or `.metal`). **Native-macOS-embedded is
dropped**: macOS has no KVM, Compass is an always-on/overnight
workload ill-suited to a personal Mac anyway, and macOS
users use the managed service or point the app at a remote
KVM Runner. The ~10% tax therefore only ever applies
to a self-hoster who *chooses* a nested cloud VM over
bare-metal — a self-inflicted, clearly-documented tradeoff.
3. **D3 — KVM absent ⇒ hard-fail, loudly.** With no container fallback (D2),
a box without microVM support cannot run Compass. `VerifyMicroVMSupport`
(V5) fails Runner startup with an error naming the missing capability and
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -412,8 +412,8 @@ in V2a-concrete form.
hard-fail under `COMPASS_REQUIRE_MICROVM=1` (`microvmtest.go:35-41`). The
test process needs the `kvm` supplementary group.
- **External-reference gate.** This record and every artifact it plans are
Compass tracked files: no references to the private monorepo's name, no
private hostnames, no private-tracker issue slugs (RIG-NNN is fine).
Compass tracked files: never name or point at the private repo, no private
hostnames, no private-tracker issue slugs (RIG-NNN is fine).
- **Egress prerequisites ride along.** The rootfs keeps `nft`/`getent`/`awk`
and the writable regular-file `/etc/resolv.conf`
(`guest-image/default.nix:141-156`; `egress.go:76-77`) — V3 depends on
Expand Down
30 changes: 15 additions & 15 deletions docs/designs/infra/runtime/compass-managed-multitenancy/design.md
Original file line number Diff line number Diff line change
Expand Up @@ -416,13 +416,13 @@ non-sticky wake on any Runner —
RIG-2485 is the control plane those rulings defer to) is a hard input to Q2:
the topology chosen here is what makes any-Runner wake routable.

**Repo boundary (RIG-1717):** the tenancy substrate — schema, RLS, and the
fabric seams and implementations (T1-T6) — lands in the OSS
`RigelBuild/compass` core, because the OSS core must run it
single-tenant-degenerate; the managed-service orchestration (tenant
provisioning, billing, box lifecycle) is the private control-plane layer
(RIG-2485 and the managed control plane), consistent with RIG-1717's
one-architecture-two-products split
**Repo boundary (RIG-1717):** the tenancy substrate — schema,
RLS, and the fabric seams and implementations (T1-T6) — lands
in the OSS `RigelBuild/compass` core, because the OSS core
must run it single-tenant-degenerate; the managed-service
orchestration (tenant provisioning, billing, box lifecycle)
is the out-of-tree managed control plane (RIG-2485),
consistent with RIG-1717's one-architecture-two-products split
(`docs/designs/infra/runtime/compass-elastic-session-runtime/design.md:71-101`).
The where-does-work-land convention this split implies is formalized in
`docs/designs/meta/oss-core-managed-boundary/design.md`.
Expand All @@ -438,11 +438,11 @@ pod on a nested-virt Kubernetes node pool. So the choice of managed box
substrate does not change any Q1/Q2/Q3 answer, interface, or task in this
record.

That substrate choice is nonetheless a live managed-control-plane fork.
RIG-2394's D2 froze the *managed* path as elastic **bare-metal** (AWS
`*.metal`, GCP `c3-metal`) specifically to dodge the nested-virtualization
tax, and called a nested cloud VM "a self-inflicted tradeoff, never the
managed path"
That substrate choice is nonetheless a live managed-control-plane
fork. RIG-2394's D2 prefers a KVM-capable **bare-metal** host — running
cloud-hypervisor directly on host VT-x — specifically to dodge the
nested-virtualization tax, and calls a nested cloud VM "a self-inflicted,
clearly-documented tradeoff"
(`docs/designs/infra/runtime/compass-elastic-session-runtime/microvm-runner.md:671-692`).
Two later facts stale that premise: AWS enabled nested
virtualization on **non-metal** EC2 (C8i/M8i/R8i, no surcharge, 2026-02), and
Expand All @@ -462,9 +462,9 @@ target, i.e. proportionally more or bigger instances for the same load.
virtualization level). Our agent workload is largely I/O-bound waiting on LLM
calls, so the real overhead is likely under the rule-of-thumb and is worth
measuring on the actual workload before treating 10% as a planning fact.
Because that fork spans this record's topology and RIG-2485/RIG-2394-D2's box
lifecycle, its home is the private monorepo's AWS-EKS substrate
design (RIG-2878), not this record; it gates only the box layer *beneath*
Because that fork spans this record's topology and RIG-2485/RIG-2394-D2's
box lifecycle, it is an out-of-tree managed-control-plane concern, not
this record; it gates only the box layer *beneath*
this substrate-agnostic topology.

## Alternatives considered
Expand Down
Loading