diff --git a/docs/designs/infra/runtime/compass-elastic-session-runtime/design.md b/docs/designs/infra/runtime/compass-elastic-session-runtime/design.md index bb4bb759..4369ef12 100644 --- a/docs/designs/infra/runtime/compass-elastic-session-runtime/design.md +++ b/docs/designs/infra/runtime/compass-elastic-session-runtime/design.md @@ -78,7 +78,7 @@ Compass ships as two products over one shared core: boundary. All of this record's `go/internal/*` and `agent-image/*` citations are public-repo paths. -- **Managed Compass (private monorepo, dual AGPL + commercial license).** The +- **Managed Compass (dual AGPL + commercial license), built out of tree.** The hosted multi-tenant service, which *reuses* the OSS core rather than forking it. @@ -86,7 +86,7 @@ Compass ships as two products over one shared core: tasks land in the public repo — that is why the record lives here and every code citation resolves against `RigelBuild/compass`. Nothing in the managed control plane (tenant orchestration, billing, the hosted control surface, all -of which live in the private monorepo) is designed here. +of which are built out of tree) is designed here. **The motivation is primarily the managed service.** Hosted-scale density (pack many tenants, suspend the idle ones), durability across eviction, and a @@ -890,8 +890,8 @@ Nothing pinned in the Approach is re-opened here. as a microVM OCI runtime (krun/libkrun or kata via podman `--runtime`), so the seam is expected to hold; the image/boot/egress plumbing above it is the real work I1 owns. Through Dogfood + trusted-tenant Beta the rootless - container remains the running boundary; I1 lands the microVM before the - first external multi-tenant tenant. + container remains the running boundary; I1 lands the microVM before it + becomes the sole runtime. 6. **[non-load-bearing] Cold-idle object-store archive — thresholds.** D4 archives an idle session's volume to the object store past a deeper idle threshold (freeing local disk; a cold-idle session costs object storage diff --git a/docs/designs/infra/runtime/compass-elastic-session-runtime/microvm-ci-dev-enablement.md b/docs/designs/infra/runtime/compass-elastic-session-runtime/microvm-ci-dev-enablement.md index 147925f9..de060487 100644 --- a/docs/designs/infra/runtime/compass-elastic-session-runtime/microvm-ci-dev-enablement.md +++ b/docs/designs/infra/runtime/compass-elastic-session-runtime/microvm-ci-dev-enablement.md @@ -192,7 +192,7 @@ the leg; a re-run clears it. If that proves frequent, the Woodpecker fallback If hosted-runner KVM proves unreliable in practice — frequent reds from withheld `/dev/kvm`, or GitHub tightening access — the fallback is to **enroll the compass repo itself on the self-hosted Woodpecker fleet** and run the KVM -leg there. This is explicitly *compass's own* CI, not the private monorepo's: +leg there. This is explicitly *compass's own* CI, not the managed service's: compass features are tested in compass. The fleet has a registered self-hosted Linux CI agent (x86_64, KVM-capable, docker backend) already, so the hardware is in place; what is missing is a compass pipeline (no `.woodpecker*` config @@ -220,11 +220,11 @@ same env + tag contract E-D2's GHA step uses, so adopting the fallback is a lane swap, not a redesign. It also realizes V8's "CI job (KVM-labeled runner)" (`microvm-runner.md:606-608`). -*Separate concern — the managed service:* the managed Compass will run from the -private monorepo, and that monorepo will need its own KVM CI when the managed -buildout starts. That is deferred to the managed-service control-plane work -(the fleet control plane, RIG-2485), not this record — this record keeps -compass's own tests in compass. +*Separate concern — the managed service:* the managed service is built out of +tree and will need its own KVM CI when its buildout starts. That is +deferred to the managed-service control-plane work (the fleet control +plane, RIG-2485), not this record — this record keeps compass's own +tests in compass. ### E-D4 — guest image: derivations beside the agent image, kernel from nixpkgs, a project binary cache (F4) diff --git a/docs/designs/infra/runtime/compass-elastic-session-runtime/microvm-kvm-only-amendment.md b/docs/designs/infra/runtime/compass-elastic-session-runtime/microvm-kvm-only-amendment.md index f51d50de..bb2ab6e6 100644 --- a/docs/designs/infra/runtime/compass-elastic-session-runtime/microvm-kvm-only-amendment.md +++ b/docs/designs/infra/runtime/compass-elastic-session-runtime/microvm-kvm-only-amendment.md @@ -78,8 +78,8 @@ timeline: - `design.md:892-894` (OQ-5, the resolved inter-tenant-boundary decision): "Through Dogfood + trusted-tenant Beta the rootless container remains the - running boundary; I1 lands the microVM before the first external multi-tenant - tenant." This stays exactly as frozen, and `microvm-runner.md:60-65` restates + running boundary; I1 lands the microVM before it becomes the sole + runtime." This stays exactly as frozen, and `microvm-runner.md:60-65` restates it: the container is the running boundary **through Beta**, then removed. The container is a transitional bootstrap, not a KVM-absent fallback — the two are different roles, and only the fallback role is retired here. diff --git a/docs/designs/infra/runtime/compass-elastic-session-runtime/microvm-runner.md b/docs/designs/infra/runtime/compass-elastic-session-runtime/microvm-runner.md index 554a81b7..9706e747 100644 --- a/docs/designs/infra/runtime/compass-elastic-session-runtime/microvm-runner.md +++ b/docs/designs/infra/runtime/compass-elastic-session-runtime/microvm-runner.md @@ -59,10 +59,10 @@ inter-tenant boundary IS a hardware-virtualized microVM, design.md:881-894; the virtio-fs no-copy invariant, design.md:586-587) and replaces only the falsified implementation mechanism. Per design.md:892-894, "Through Dogfood + trusted-tenant Beta the rootless container remains the running boundary" — so -this work runs in parallel with M0/S1/P2/C3 and blocks nothing before the -first external multi-tenant tenant. Beyond that point the container path is -**removed** and microVM becomes the sole runtime (D2); the container backend -is a transitional bootstrap, not a permanent second runtime. +this work runs in parallel with M0/S1/P2/C3 and is off the critical path. Once +microVM is proven the container path is **removed** and microVM becomes the +sole runtime (D2); the container backend is a transitional bootstrap, not a +permanent second runtime. ## Approach @@ -403,8 +403,8 @@ demand via cloud-hypervisor hotplug rather than reserving peak RAM (D5). - **Transitional container path, then microVM-only (D2).** The rootless container remains the running boundary through Dogfood + trusted-tenant Beta (design.md:892-894) and is then **removed**: microVM is the sole runtime. - This work runs in parallel with M0/S1/P2/C3 and gates nothing before the - first external multi-tenant tenant. + This work runs in parallel with M0/S1/P2/C3 and is off the critical + path. ## Plan @@ -676,20 +676,20 @@ each is kept so the executor sees *why*, not just *what*. second permanent runtime roughly *doubles* the production support and bugfix surface, which for a solo maintainer dominates the one-time build cost — and microVM must be built regardless (it is the reason the parent - design exists). Reach is preserved without a second runtime: the managed - service runs cloud-hypervisor on **elastic, hourly, autoscaling bare-metal** - instances (AWS `*.metal` in standard ASGs; GCP `c3-*-metal` "consumed and - managed in the same way as VM instances") — no nesting, so **no ~10% - nested-virtualization performance tax**; self-hosters provision a - KVM-capable box *for* Compass (an owned/homelab box, Hetzner-class - bare-metal, or a nested-virt-enabled hyperscaler instance — GCP any Linux - VM, Azure Dv3/Ev3+, AWS C8i/M8i/R8i or `.metal`). **Native-macOS-embedded - is dropped**: macOS has no KVM, Compass is an always-on/overnight workload - ill-suited to a personal Mac anyway, and macOS users use the managed - service or point the app at a remote KVM Runner. The ~10% tax therefore - only ever applies to a self-hoster who *chooses* a nested cloud VM over - bare-metal — a self-inflicted, clearly-documented tradeoff, never the - managed path. + design exists). Reach is preserved without a second runtime: + cloud-hypervisor needs only a KVM-capable host, and running + it directly on bare metal means no nesting and therefore **no + ~10% nested-virtualization performance tax**. Self-hosters + provision a KVM-capable box *for* Compass (an owned/homelab + box, Hetzner-class bare-metal, or a nested-virt-enabled + hyperscaler instance — GCP any Linux VM, Azure Dv3/Ev3+, + AWS C8i/M8i/R8i or `.metal`). **Native-macOS-embedded is + dropped**: macOS has no KVM, Compass is an always-on/overnight + workload ill-suited to a personal Mac anyway, and macOS + users use the managed service or point the app at a remote + KVM Runner. The ~10% tax therefore only ever applies + to a self-hoster who *chooses* a nested cloud VM over + bare-metal — a self-inflicted, clearly-documented tradeoff. 3. **D3 — KVM absent ⇒ hard-fail, loudly.** With no container fallback (D2), a box without microVM support cannot run Compass. `VerifyMicroVMSupport` (V5) fails Runner startup with an error naming the missing capability and diff --git a/docs/designs/infra/runtime/compass-elastic-session-runtime/microvm-v2a-guest-image-boot-spike.md b/docs/designs/infra/runtime/compass-elastic-session-runtime/microvm-v2a-guest-image-boot-spike.md index 220cb602..2d5ccb7d 100644 --- a/docs/designs/infra/runtime/compass-elastic-session-runtime/microvm-v2a-guest-image-boot-spike.md +++ b/docs/designs/infra/runtime/compass-elastic-session-runtime/microvm-v2a-guest-image-boot-spike.md @@ -412,8 +412,8 @@ in V2a-concrete form. hard-fail under `COMPASS_REQUIRE_MICROVM=1` (`microvmtest.go:35-41`). The test process needs the `kvm` supplementary group. - **External-reference gate.** This record and every artifact it plans are - Compass tracked files: no references to the private monorepo's name, no - private hostnames, no private-tracker issue slugs (RIG-NNN is fine). + Compass tracked files: never name or point at the private repo, no private + hostnames, no private-tracker issue slugs (RIG-NNN is fine). - **Egress prerequisites ride along.** The rootfs keeps `nft`/`getent`/`awk` and the writable regular-file `/etc/resolv.conf` (`guest-image/default.nix:141-156`; `egress.go:76-77`) — V3 depends on diff --git a/docs/designs/infra/runtime/compass-managed-multitenancy/design.md b/docs/designs/infra/runtime/compass-managed-multitenancy/design.md index 1401d676..f26b0de7 100644 --- a/docs/designs/infra/runtime/compass-managed-multitenancy/design.md +++ b/docs/designs/infra/runtime/compass-managed-multitenancy/design.md @@ -416,13 +416,13 @@ non-sticky wake on any Runner — RIG-2485 is the control plane those rulings defer to) is a hard input to Q2: the topology chosen here is what makes any-Runner wake routable. -**Repo boundary (RIG-1717):** the tenancy substrate — schema, RLS, and the -fabric seams and implementations (T1-T6) — lands in the OSS -`RigelBuild/compass` core, because the OSS core must run it -single-tenant-degenerate; the managed-service orchestration (tenant -provisioning, billing, box lifecycle) is the private control-plane layer -(RIG-2485 and the managed control plane), consistent with RIG-1717's -one-architecture-two-products split +**Repo boundary (RIG-1717):** the tenancy substrate — schema, +RLS, and the fabric seams and implementations (T1-T6) — lands +in the OSS `RigelBuild/compass` core, because the OSS core +must run it single-tenant-degenerate; the managed-service +orchestration (tenant provisioning, billing, box lifecycle) +is the out-of-tree managed control plane (RIG-2485), +consistent with RIG-1717's one-architecture-two-products split (`docs/designs/infra/runtime/compass-elastic-session-runtime/design.md:71-101`). The where-does-work-land convention this split implies is formalized in `docs/designs/meta/oss-core-managed-boundary/design.md`. @@ -438,11 +438,11 @@ pod on a nested-virt Kubernetes node pool. So the choice of managed box substrate does not change any Q1/Q2/Q3 answer, interface, or task in this record. -That substrate choice is nonetheless a live managed-control-plane fork. -RIG-2394's D2 froze the *managed* path as elastic **bare-metal** (AWS -`*.metal`, GCP `c3-metal`) specifically to dodge the nested-virtualization -tax, and called a nested cloud VM "a self-inflicted tradeoff, never the -managed path" +That substrate choice is nonetheless a live managed-control-plane +fork. RIG-2394's D2 prefers a KVM-capable **bare-metal** host — running +cloud-hypervisor directly on host VT-x — specifically to dodge the +nested-virtualization tax, and calls a nested cloud VM "a self-inflicted, +clearly-documented tradeoff" (`docs/designs/infra/runtime/compass-elastic-session-runtime/microvm-runner.md:671-692`). Two later facts stale that premise: AWS enabled nested virtualization on **non-metal** EC2 (C8i/M8i/R8i, no surcharge, 2026-02), and @@ -462,9 +462,9 @@ target, i.e. proportionally more or bigger instances for the same load. virtualization level). Our agent workload is largely I/O-bound waiting on LLM calls, so the real overhead is likely under the rule-of-thumb and is worth measuring on the actual workload before treating 10% as a planning fact. -Because that fork spans this record's topology and RIG-2485/RIG-2394-D2's box -lifecycle, its home is the private monorepo's AWS-EKS substrate -design (RIG-2878), not this record; it gates only the box layer *beneath* +Because that fork spans this record's topology and RIG-2485/RIG-2394-D2's +box lifecycle, it is an out-of-tree managed-control-plane concern, not +this record; it gates only the box layer *beneath* this substrate-agnostic topology. ## Alternatives considered