From 6b7f9a987b658f7edc1e45c14e77daee4cb7a8ec Mon Sep 17 00:00:00 2001 From: dev-dami <141376183+dev-dami@users.noreply.github.com> Date: Tue, 4 Aug 2026 00:34:35 +0100 Subject: [PATCH 1/2] fix: repair guest boot chain and harden host-side sandbox The guest boot chain had four independent blockers that each stopped execution before user code could run: - Boot args passed init=/usr/local/bin/ignite-guest-agent while create_rootfs installed the agent at /sbin/init. Both now use a shared GUEST_INIT_PATH constant. - The agent was built without --target x86_64-unknown-linux-musl, producing a glibc-linked binary for a rootfs with no libc or dynamic loader. It now targets musl, and both build_guest_agent and create_rootfs reject an ELF carrying a PT_INTERP segment. - The agent called mkdir on a read-only rootfs. Mount points are now baked into the image, devtmpfs is mounted first (without it /dev/vdb does not exist), and a failed disk mount is fatal rather than logged and ignored. - download_kernel copied /boot/vmlinuz-*, a compressed bzImage that Firecracker cannot boot. It now requires an uncompressed ELF vmlinux and names any compressed images it found. The VSOCK listener also binds before InstanceStart, so the guest cannot exhaust its connect retries before the host is listening. Security fixes: - service.name from service.yaml reached a host path that boot() unlinks, allowing arbitrary file removal via `../..`. It is now validated and sanitized, and the socket path is pinned under /tmp. - The timeout watchdog only checked its deadline in the WouldBlock branch, so a guest writing output continuously ran forever. The deadline is now checked every iteration. - Guest-controlled frame lengths were allocated verbatim (up to 4 GiB). Frames are capped at 4 MiB and retained output at 8 MiB per stream. - The rate limiter keyed on the spoofable X-Forwarded-For header, never evicted buckets, and compared API keys non-constant-time. It now keys on the transport peer, bounds bucket count, and compares in constant time. - CORS defaulted to any origin on an endpoint that executes code. It is now opt-in via IGNITE_CORS_ORIGINS, and both entry points warn when IGNITE_API_KEY is unset. - The agent clears the environment before applying host-supplied vars. Flags that parsed but did nothing are wired up (--runtime, --console-out) or now fail loudly (--audit, which would otherwise report a clean audit for code that was never audited). cpuLimit rounds up instead of truncating 0.5 to 0 vCPUs; runtime versions resolve to runtimes/@ with a logged fallback; cold-start timing is reported only when the guest emits it rather than fabricated. Also fixes send_put_uds treating a 4xx whose body echoed "HTTP/1.1 200" as success, `serve --host localhost` silently falling back to 127.0.0.1 instead of resolving, and the KVM check in `status` misreading mode bits so a healthy device reported "check user group". Rewrites AGENTS.md, which documented a Bun/TypeScript monorepo that no longer exists, and corrects README/docs claims about macOS support, audit mode, and runtime provisioning. Adds a "Not Implemented" section to the threat model covering audit mode, the Firecracker jailer, macOS, and unauthenticated-by-default HTTP. Test coverage goes from 1 test to 29. Verification: cargo fmt --check, clippy -D warnings, and cargo test --workspace all pass. The musl agent was confirmed static-pie with no PT_INTERP and byte-identical to /sbin/init extracted from the built rootfs image. VM-level execution could NOT be verified: this host has no firecracker binary and no uncompressed vmlinux, so the guest-side path (devtmpfs mount, VSOCK handshake, PID 1 startup) is fixed by inspection only and still needs a real boot. Co-Authored-By: Claude Opus 5 (1M context) --- AGENTS.md | 92 +++- README.md | 40 +- docs/api.md | 34 +- docs/architecture.md | 12 +- docs/threat-model.md | 45 +- ignite-cli/src/main.rs | 193 ++++++-- ignite-core/src/execution.rs | 273 ++++++++-- ignite-core/src/orchestrator.rs | 8 +- ignite-core/src/platform/apple_vz.rs | 104 ++-- ignite-core/src/platform/firecracker.rs | 633 +++++++++++++++++------- ignite-core/src/setup.rs | 215 ++++++-- ignite-guest-agent/src/main.rs | 74 ++- ignite-http/src/main.rs | 28 +- ignite-http/src/server.rs | 165 ++++-- ignite-shared/src/lib.rs | 2 +- ignite-shared/src/validation.rs | 116 +++++ install.sh | 2 +- 17 files changed, 1538 insertions(+), 498 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index ca21b3f..e818cf9 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -2,58 +2,96 @@ This guide tells coding agents how to work safely and effectively in this repository. -## Core Rule +## What This Is -- Use Bun for all package installs and scripts. -- Never use npm. +Ignite is a **Rust cargo workspace** (edition 2024) that executes untrusted +JS/TS inside hardware-virtualized microVMs. There is no Node/Bun build system +here — Bun, Node, Deno, and QuickJS are *guest runtimes* that Ignite downloads +and attaches to VMs as read-only block devices, not tooling for this repo. ## Repository Map -- `packages/core`: sandbox lifecycle, loaders, preflight, execution engine. -- `packages/cli`: `ignite` CLI commands. -- `packages/http`: HTTP server surface for sandbox execution. -- `packages/shared`: shared types/utilities used across packages. -- `packages/runtime-bun`: Bun runtime image files. +- `ignite-shared/`: shared types (`ServiceConfig`, metrics), error enum, validators. +- `ignite-core/`: sandbox lifecycle — preflight, ext4 disk building, host setup, + and the hypervisor backends under `platform/`. +- `ignite-cli/`: the `ignite` command line binary (clap). +- `ignite-http/`: axum REST API server. +- `ignite-guest-agent/`: static PID-1 init that runs *inside* the guest. - `examples/*`: sample services used for smoke/manual verification. - `docs/*`: user-facing docs and architecture notes. -- `scripts/*`: release/build helper scripts. ## Local Workflow -1. Install deps: `bun install` -2. Build all packages: `bun run build` -3. Run lint: `bun run lint` -4. Run typecheck: `bun run typecheck` -5. Run tests: - - Unit-only (fast): `bun run test:unit` - - Full suite (requires Docker): `bun run test` +```bash +cargo build --workspace +cargo fmt --all +cargo clippy --workspace --all-targets -- -D warnings +cargo test --workspace +``` + +CI runs `fmt --check`, `clippy -D warnings`, and `cargo test --all`. All three +must pass. + +### Running a service end to end + +Requires a Linux host with KVM, `firecracker` on PATH, `e2fsprogs`, and an +**uncompressed ELF `vmlinux`** (not `/boot/vmlinuz-*`, which is a compressed +bzImage Firecracker cannot boot). + +```bash +rustup target add x86_64-unknown-linux-musl # guest agent must be static +ignite setup +ignite status # verifies KVM + firecracker +ignite run examples/hello-bun +``` ## Change Rules - Keep changes scoped to the task; avoid drive-by refactors. -- When behavior changes, add or update tests in the relevant package. -- If CLI/API behavior changes, update docs in `README.md` and/or `docs/*`. -- Use existing naming patterns and file structure within each package. +- When behavior changes, add or update tests in the relevant crate. +- If CLI/API behavior changes, update `README.md` and/or `docs/*`. - Prefer small, composable functions over large rewrites. ## Security Guardrails (Important) Ignite runs untrusted code. Treat security defaults as product-critical. -- Do not weaken sandbox restrictions (network/filesystem/capabilities) without explicit task requirements. -- If security logic changes, include tests that prove both allowed and blocked behavior. +- Do not weaken sandbox restrictions (network/filesystem/capabilities) without + explicit task requirements. +- If security logic changes, include tests proving both allowed and blocked + behavior. - Never introduce secrets, tokens, or host-specific paths into committed code. +Three invariants are easy to break by accident: + +1. **The guest is untrusted, including the VSOCK peer.** Sandboxed code can open + `AF_VSOCK` to the host itself. Never trust a guest-supplied length prefix + without bounding it, and never assume the guest agent is the only writer. +2. **`service.yaml` is untrusted input.** `service.name` reaches host paths that + get unlinked. Validate it (`validate_service_name`) and sanitize before + interpolating (`sanitize_path_segment`). +3. **The guest rootfs is read-only and has no libc.** The agent cannot `mkdir` + its own mount points — they must exist in the image — and must stay + statically linked against musl. + +Do not advertise a security control that is not enforced. If a flag cannot be +honored, fail loudly rather than accepting it silently: `--audit` currently +returns an error for exactly this reason. + ## Validation Matrix -- `packages/shared` change: run `bun run build`, `bun run test:unit`. -- `packages/core` change: run `bun run test:unit`; run `bun run test` if execution behavior changed. -- `packages/cli` change: run `bun run test:unit`; manually smoke command paths when possible. -- `packages/http` change: run `bun run test:unit`; verify request/response behavior for changed endpoints. -- `docs`-only change: lint/typecheck optional, tests not required. +- `ignite-shared` change: `cargo test --workspace` (everything depends on it). +- `ignite-core` change: `cargo test -p ignite-core`; manually smoke `ignite run` + if execution behavior changed. +- `ignite-cli` change: `cargo test -p ignite-cli`; smoke the affected command. +- `ignite-http` change: `cargo test -p ignite-http`; verify request/response + behavior for changed endpoints. +- `ignite-guest-agent` change: rebuild for musl and boot a VM — this code runs + as PID 1 and is not covered by host-side tests. +- `docs`-only change: tests not required. ## Commit & PR Hygiene - Follow Conventional Commits (`feat:`, `fix:`, `docs:`, `refactor:`, `test:`, `chore:`). - Keep PRs focused and include a short verification summary (commands run + results). -- Mention Docker dependency when full test suite could not be run. +- Note when VM-level verification could not be run (no KVM, no kernel image). diff --git a/README.md b/README.md index 3265f20..cdd6bd7 100644 --- a/README.md +++ b/README.md @@ -17,7 +17,9 @@ ## Overview -Ignite runs JavaScript/TypeScript code inside isolated, hardware-virtualized microVMs rather than containers. It supports native **Firecracker** on Linux and Apple's **Virtualization.framework** on macOS out of the box, with zero external VM dependencies. +Ignite runs JavaScript/TypeScript code inside isolated, hardware-virtualized microVMs rather than containers. Execution currently requires a **Linux host with KVM**, using **Firecracker** as the hypervisor. + +> **Status:** pre-1.0. A macOS backend built on Apple's `Virtualization.framework` is planned but **not implemented** — the platform selector is stubbed and returns a clear error. Bun is the only runtime `ignite setup` provisions today. It is designed for systems that execute code you do not fully trust: @@ -28,18 +30,26 @@ It is designed for systems that execute code you do not fully trust: ## Key Features -- **Dual-Hypervisor Core**: Uses KVM-backed Firecracker on Linux, and native `Virtualization.framework` on macOS. +- **KVM-backed Firecracker**: Each service runs in its own microVM with a separate guest kernel. (A macOS `Virtualization.framework` backend is planned; see Status above.) - **Host-Reliant Disk Mounts**: The guest microVM has no shell, utilities, or libraries. Service code and language runtimes (Bun, Node, Deno, QuickJS) are compiled on the host and attached as read-only virtual block devices (`/dev/vdb` and `/dev/vdc`). - **VSOCK Multiplexing**: Low-latency communication handshakes stream stdout/stderr and exit codes directly back to the host via virtual sockets, bypassing network interfaces. -- **Resource Enforcement**: `memoryMb` and `cpuLimit` are applied to Firecracker machine config, while `timeoutMs` is enforced by a host-side watchdog that force-terminates timed-out VMs. +- **Resource Enforcement**: `memoryMb` and `cpuLimit` are applied to Firecracker machine config, while `timeoutMs` is enforced by a host-side watchdog that force-terminates timed-out VMs. `cpuLimit` is rounded up to whole vCPUs, which is the only granularity Firecracker accepts. - **Preflight & Metric Timelines**: Sub-millisecond logging of all VM lifecycle transitions (disk format, boot connect, execution, cleanup). ## Quick Start ### 1) Prerequisites -- **Linux**: KVM enabled (`/dev/kvm` accessible) and `e2fsprogs` installed. -- **macOS**: macOS 13 or later. +- Linux with KVM enabled (`/dev/kvm` accessible). +- `firecracker` on your `PATH`. +- `e2fsprogs` (provides `mke2fs`). +- The musl target for the static guest agent: `rustup target add x86_64-unknown-linux-musl`. +- An **uncompressed ELF `vmlinux`**. Distro `/boot/vmlinuz-*` files are + compressed bzImages that Firecracker cannot boot; extract one with the kernel + tree's `scripts/extract-vmlinux`, or use a prebuilt Firecracker kernel, then + point `IGNITE_KERNEL_PATH` (or `--kernel`) at it. + +Run `ignite status` to check KVM access and Firecracker availability. ### 2) Build from Source @@ -83,12 +93,20 @@ ignite run . --verbose ## Runtime Support -| Runtime | Supported versions | Default | -|---|---|---| -| Bun | `1.0`, `1.1`, `1.2`, `1.3` | `1.3` | -| Node | `18`, `20`, `22` | `20` | -| Deno | `1.40`, `1.41`, `1.42`, `2.0` | `2.0` | -| QuickJS | `2024-01-13`, `2023-12-09`, `latest` | `latest` | +| Runtime | Accepted versions | Default | Provisioned by `ignite setup` | +|---|---|---|---| +| Bun | `1.0`, `1.1`, `1.2`, `1.3` | `1.3` | Yes | +| Node | `18`, `20`, `22` | `20` | No — install manually | +| Deno | `1.40`, `1.41`, `1.42`, `2.0` | `2.0` | No — install manually | +| QuickJS | `2024-01-13`, `2023-12-09`, `latest` | `latest` | No — install manually | + +Runtimes are read from `~/.ignite/runtimes/` (override with `IGNITE_RUNTIMES_ROOT`). +A pinned spec such as `bun@1.3` resolves to `runtimes/bun@1.3/` and falls back to +`runtimes/bun/` with a warning. Runtime binaries must be statically linked or +otherwise self-contained: the guest rootfs has no dynamic loader. + +Only runtimes you install are available; `ignite setup` currently downloads Bun +only. ## Documentation diff --git a/docs/api.md b/docs/api.md index 3b1f9d4..c31b7dd 100644 --- a/docs/api.md +++ b/docs/api.md @@ -39,6 +39,8 @@ Options: - `--runtimes-root `: path to custom host language runtimes folder. - `--vsock-port `: custom host-guest VSOCK communication port. - `--console-out `: file path to log guest serial console outputs. +- `--audit` / `--audit-output `: **not implemented**; returns an error. + See [Threat Model](./threat-model.md) for the isolation that *is* enforced. ### `ignite preflight ` @@ -63,9 +65,25 @@ ignite serve [options] Options: - `-p, --port `: API port (default `3000`) -- `-h, --host `: host IP to bind (default `localhost`) +- `-h, --host `: host or IP to bind (default `localhost`) - `-s, --services `: path to services root folder (default `./services`) +Environment: + +- `IGNITE_API_KEY`: when set, every endpoint except `/health` requires + `Authorization: Bearer `. **When unset, the server executes services for + any caller that can reach it** and logs a warning at startup — keep the bind + address on localhost in that case. +- `IGNITE_CORS_ORIGINS`: comma-separated list of exact allowed origins + (e.g. `https://app.example.com`). When unset, no CORS headers are sent at all, + which blocks browser-based cross-origin access by default. +- `IGNITE_KERNEL_PATH`, `IGNITE_ROOTFS_PATH`, `IGNITE_RUNTIMES_ROOT`: override + the guest kernel, rootfs image, and runtimes directory. + +Requests are rate limited to 60 per minute per client IP, keyed on the real +transport peer address (not the spoofable `X-Forwarded-For` header). If you run +Ignite behind a reverse proxy, apply per-client limits at the proxy. + --- ## HTTP REST API @@ -77,11 +95,12 @@ Response: ```json { "status": "ok", - "version": "0.1.0", - "uptime": 0 + "version": "0.9.0" } ``` +`/health` is the only endpoint that does not require authentication. + ### `GET /services` List service folders under the configured services path root. @@ -108,6 +127,15 @@ Request body: } ``` +`audit` must be `false` or omitted. Security audit mode is **not implemented**; +sending `true` returns an error rather than silently reporting an audit that +never ran. + +`coldStartTimeMs` is present only when the guest runtime emits an +`IGNITE_INIT_TIME:` line on stderr, and `memoryUsageMb` likewise depends on an +`IGNITE_MEMORY_MB:` line. Both are omitted or zero otherwise rather than +estimated. + Response: ```json diff --git a/docs/architecture.md b/docs/architecture.md index e55951f..f67abb0 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -30,7 +30,7 @@ To minimize vulnerabilities in the guest VM, the root filesystem is built withou └──────────────────────┘ ``` -1. **`/dev/vda` (Root filesystem)**: Contains only the statically compiled `/sbin/init` (the `ignite-guest-agent` binary). +1. **`/dev/vda` (Root filesystem)**: Contains only the statically compiled `/sbin/init` (the `ignite-guest-agent` binary, linked against musl) plus empty `/app`, `/runtime`, `/dev`, `/proc`, and `/sys` mount points. Because this disk is attached read-only, those directories must exist in the image — the agent cannot create them at runtime. The kernel is booted with `init=/sbin/init` to match. 2. **`/dev/vdb` (Service filesystem)**: Holds service source code files, formatted dynamically on-the-fly by the host using `mke2fs` without loopback privileges. Mounted read-only at `/app`. 3. **`/dev/vdc` (Runtime engine)**: Holds the selected language runtime (Node, Deno, Bun, or QuickJS) from the host's runtime library folder. Mounted read-only at `/runtime`. @@ -41,10 +41,12 @@ ignite-cli/ignite-http -> Loads service.yaml -> Runs preflight checks (dependency counts, RAM allocations) -> Calls mke2fs to format service.ext4 and runtime.ext4 in user-space - -> Binds host VSOCK listener on Unix Socket (/tmp/ignite-vsock-VMID_1052) - -> Spawns hypervisor process (Firecracker or Apple VZ) - -> Hypervisor loads kernel (vmlinux), boots guest VM - -> Guest Agent (init) mounts /dev/vdb to /app and /dev/vdc to /runtime + -> Binds host VSOCK listener on Unix Socket (/tmp/ignite-vsock-_) + BEFORE starting the VM, so the guest cannot dial out before we listen + -> Spawns hypervisor process (Firecracker) + -> Hypervisor loads kernel (uncompressed ELF vmlinux), boots guest VM + -> Guest Agent (init) mounts devtmpfs, proc, sysfs, then /dev/vdb to /app + and /dev/vdc to /runtime (read-only); a failed disk mount is fatal -> Guest Agent connects back to host over VSOCK channel (port 1052) -> Host transmits JSON payload (environment, execution script) -> Guest Agent executes runtime command inside microVM sandbox diff --git a/docs/threat-model.md b/docs/threat-model.md index b7e50c3..2ef6225 100644 --- a/docs/threat-model.md +++ b/docs/threat-model.md @@ -11,29 +11,40 @@ Ignite aims to provide defense-in-depth for executing untrusted JS/TS code insid | Mitigate network exfiltration | VMs are started without virtual network interfaces. | | Mitigate host file tampering | App files (`/app`) and engine code (`/runtime`) are attached as read-only virtual block devices. | | Limit privilege escalation | No shell (`/bin/sh`), compiler, or system utilities exist in the guest rootfs. | -| Bound runaway processes | Memory/vCPU limits are applied to Firecracker machine config, and a host watchdog force-terminates the VM when `timeoutMs` is exceeded. | +| Bound runaway processes | Memory/vCPU limits are applied to Firecracker machine config, and a host watchdog force-terminates the VM when `timeoutMs` is exceeded. The deadline is checked on every read, so a guest that writes output continuously cannot hold the watchdog open. | +| Bound host memory from guest output | VSOCK frame lengths are guest-controlled, so frames are capped (4 MiB each) and retained stdout/stderr is capped (8 MiB per stream) rather than allocated on the guest's word. | | VSOCK Only handshake | Handshake and stdout/stderr pipes occur over a dedicated virtual socket (VSOCK) connection. | ## Trust Boundaries ### Trusted -- Host OS kernel and CPU virtualization (Intel VT-x / Apple Silicon). -- Native hypervisor processes (Firecracker / macOS Virtualization.framework). -- Statically compiled guest agent init binary (`ignite-guest-agent`). +- Host OS kernel and CPU virtualization (Intel VT-x / AMD-V). +- The Firecracker hypervisor process. +- The host-side Ignite process that builds disks and drives the VM. ### Untrusted - Guest service source code. - NPM/Bun third-party dependencies (`node_modules`). - Input payloads sent during execution. +- `service.yaml` contents. Fields such as `service.name` reach host paths, so + they are validated and sanitized before use. +- **Everything arriving over the VSOCK channel.** The guest agent is the + intended peer, but untrusted code inside the guest can open `AF_VSOCK` to the + host itself, so the host treats all frames as attacker-controlled: bounded + lengths, no trust in the declared frame size. + +The guest agent binary is trusted *as shipped by the host* — it is built by the +host and placed in the rootfs image — but the host does not assume it is the +only thing talking on the socket. ## Hardening Mechanism By relying on microVMs instead of containers: 1. **Kernel Separation**: The untrusted code runs on a separate guest Linux kernel. A kernel panic or privilege escalation in the guest does not compromise the host kernel. -2. **Distroless Guest Rootfs**: The rootfs image contains nothing except the `/sbin/init` guest agent binary. This makes it impossible for an attacker to run utilities like `sh`, `bash`, `nc`, or `curl`. +2. **Distroless Guest Rootfs**: The rootfs image contains nothing except the `/sbin/init` guest agent binary and empty mount points. There is no shell, libc, or dynamic loader, so an attacker cannot run utilities like `sh`, `bash`, `nc`, or `curl`. The agent starts the runtime with a cleared environment so nothing from the init process leaks into untrusted code. 3. **No Network Layer**: The hypervisor configures no virtio-net or tap interfaces. VSOCK is the sole communication mechanism. 4. **Read-Only Storage**: All mapped disks (`/app` and `/runtime`) are formatted as ext4 read-only block devices. @@ -42,6 +53,26 @@ By relying on microVMs instead of containers: Ignite does not guarantee protection against: - CPU hardware vulnerabilities (e.g. Spectre, Meltdown). -- MicroVM escape vulnerabilities in Firecracker or macOS Virtualization.framework. +- MicroVM escape vulnerabilities in Firecracker. - Resource consumption within the allowed limits (e.g. infinite loops within the timeoutMs budget). -- Host memory exhaustion if multiple server slots are launched concurrently. +- Host memory exhaustion if many VMs are launched concurrently. There is no + global concurrency cap; each in-flight request holds a full `memoryMb` + allocation. + +## Not Implemented + +These are deliberately called out because their absence is easy to mistake for +their presence: + +- **Security audit mode.** `--audit` and the `audit` API field return an error + rather than running. There is no in-guest syscall, network, or filesystem + accounting, so the `SecurityEvent`/`SecurityAudit` types are not populated by + anything today. +- **Firecracker jailer.** Firecracker runs as the invoking user, without + `jailer`, a seccomp profile, or a chroot. Host-side isolation of the + hypervisor process itself is left to the operator; run Ignite as a dedicated + unprivileged user. +- **macOS backend.** `Virtualization.framework` support is stubbed and errors + out. Linux + KVM is the only supported execution path. +- **HTTP auth by default.** If `IGNITE_API_KEY` is unset, `ignite serve` accepts + unauthenticated requests from anyone who can reach the port. diff --git a/ignite-cli/src/main.rs b/ignite-cli/src/main.rs index 9075835..0dd35e5 100644 --- a/ignite-cli/src/main.rs +++ b/ignite-cli/src/main.rs @@ -245,8 +245,10 @@ fn handle_init(name: String, path: Option, runtime: String) -> Result<() fn handle_run( service: String, input: Option, + runtime: Option, skip_preflight: bool, json: bool, + audit: bool, verbose: bool, memory: Option, cpus: Option, @@ -264,6 +266,18 @@ fn handle_run( }); } + if let Some(ref r) = runtime + && !is_valid_runtime(r) + { + return Err(IgniteError::Config { + message: format!( + "Invalid runtime '{}'. Supported runtimes are: bun, node, deno, quickjs", + r + ), + source: None, + }); + } + let mut env = HashMap::new(); env.insert("NODE_ENV".to_string(), "production".to_string()); @@ -271,9 +285,10 @@ fn handle_run( input, env, skip_preflight, - audit: false, + audit, memory_override: memory, cpu_override: cpus, + runtime_override: runtime, kernel_path: kernel, rootfs_path: rootfs, runtimes_root, @@ -367,35 +382,38 @@ fn handle_status() -> Result<()> { } ); - // Check if KVM is usable (permissions) + // Check if KVM is actually usable by this user. File mode bits alone do not + // answer that (group membership and ACLs both matter), so open the device + // read-write the way the hypervisor will. if kvm_ok { - let kvm_meta = fs::metadata("/dev/kvm"); - let kvm_readable = kvm_meta - .as_ref() - .map(|m| m.permissions().readonly()) - .unwrap_or(false); - let kvm_usable = kvm_meta.is_ok() && kvm_readable; - let usable_icon = if kvm_usable { "✓" } else { "⚠" }; - let usable_color = if kvm_usable { "\x1b[32m" } else { "\x1b[33m" }; - println!( - " {}{}\x1b[0m\x1b[0m KVM permissions: {}", - usable_color, - usable_icon, - if kvm_usable { - "readable" - } else { - "check user group" - } - ); + match fs::OpenOptions::new() + .read(true) + .write(true) + .open("/dev/kvm") + { + Ok(_) => println!(" \x1b[32m✓\x1b[0m KVM permissions: read/write OK"), + Err(e) => println!( + " \x1b[33m⚠\x1b[0m KVM permissions: cannot open /dev/kvm ({}). \ + Add your user to the `kvm` group.", + e.kind() + ), + } + } + + // Firecracker availability: without it, Linux execution cannot start. + let firecracker_found = which_in_path("firecracker"); + match firecracker_found { + Some(ref path) => println!(" \x1b[32m✓\x1b[0m Firecracker: {:?}", path), + None => println!( + " \x1b[31m✗\x1b[0m Firecracker: not found in PATH (required to run services)" + ), } // Virtualization.framework check (macOS) - let vz_ok = cfg!(target_os = "macos"); - if vz_ok { - let vz_icon = "✓"; + if cfg!(target_os = "macos") { println!( - " \x1b[32m{}\x1b[0m Virtualization.framework: available", - vz_icon + " \x1b[33m⚠\x1b[0m Virtualization.framework: host supported, but the Ignite macOS \ + backend is not implemented in this build" ); } @@ -789,12 +807,41 @@ fn handle_templates(command: Option) -> Result<()> { } } +/// Look up an executable on PATH without shelling out to `which`. +fn which_in_path(name: &str) -> Option { + let path_var = std::env::var_os("PATH")?; + std::env::split_paths(&path_var) + .map(|dir| dir.join(name)) + .find(|candidate| candidate.is_file()) +} + fn dirs() -> Option { std::env::var("HOME") .ok() .map(|home| PathBuf::from(home).join(".ignite")) } +/// Resolve a host/port pair into a bind address. +/// +/// Hostnames such as the default `localhost` are not valid `SocketAddr` +/// literals, so they are resolved through the system resolver rather than +/// silently falling back to a different address than the operator asked for. +fn resolve_bind_addr(host: &str, port: u16) -> Result { + use std::net::ToSocketAddrs; + + (host, port) + .to_socket_addrs() + .map_err(|e| IgniteError::Config { + message: format!("Cannot resolve bind address {}:{}", host, port), + source: Some(Box::new(e)), + })? + .next() + .ok_or_else(|| IgniteError::Config { + message: format!("No address found for {}:{}", host, port), + source: None, + }) +} + fn handle_setup(force: bool) -> Result<()> { use ignite_core::setup; @@ -807,6 +854,10 @@ fn handle_setup(force: bool) -> Result<()> { setup::create_directories(&ignite_dir)?; println!(" \x1b[32m✓\x1b[0m Directories created"); + // Track which required resources are missing so the summary is honest + // rather than always claiming success. + let mut missing: Vec<&str> = Vec::new(); + // Use host kernel let kernel_path = ignite_dir.join("vmlinux"); if kernel_path.exists() && !force { @@ -816,7 +867,10 @@ fn handle_setup(force: bool) -> Result<()> { std::io::Write::flush(&mut std::io::stdout()).ok(); match setup::download_kernel(&ignite_dir) { Ok(path) => println!("\x1b[32m✓\x1b[0m {:?}", path), - Err(e) => println!("\x1b[33m⚠\x1b[0m {}", e), + Err(e) => { + println!("\x1b[33m⚠\x1b[0m {}", e); + missing.push("guest kernel (vmlinux)"); + } } } @@ -834,10 +888,14 @@ fn handle_setup(force: bool) -> Result<()> { std::io::Write::flush(&mut std::io::stdout()).ok(); match setup::download_bun(&ignite_dir.join("runtimes")) { Ok(path) => println!("\x1b[32m✓\x1b[0m {:?}", path), - Err(e) => println!("\x1b[33m⚠\x1b[0m {}", e), + Err(e) => { + println!("\x1b[33m⚠\x1b[0m {}", e); + missing.push("bun runtime"); + } } } else { println!(" \x1b[33m⚠\x1b[0m Skipped. You can run `ignite setup --force` later."); + missing.push("bun runtime (skipped)"); } } @@ -851,17 +909,34 @@ fn handle_setup(force: bool) -> Result<()> { std::io::Write::flush(&mut std::io::stdout()).ok(); match setup::create_rootfs(&ignite_dir, &agent_path) { Ok(path) => println!("\x1b[32m✓\x1b[0m {:?}", path), - Err(e) => println!("\x1b[33m⚠\x1b[0m {}", e), + Err(e) => { + println!("\x1b[33m⚠\x1b[0m {}", e); + missing.push("guest rootfs image"); + } } } else { - println!(" \x1b[33m⚠\x1b[0m Rootfs not found. Build guest agent first:"); - println!(" cargo build --release --bin ignite-guest-agent"); - println!(" cp target/release/ignite-guest-agent ~/.ignite/guest-agent"); + println!(" \x1b[33m⚠\x1b[0m Rootfs not found. Build the guest agent first:"); + println!(" rustup target add x86_64-unknown-linux-musl"); + println!( + " cargo build --release --bin ignite-guest-agent \\\n --target x86_64-unknown-linux-musl" + ); + println!( + " cp target/x86_64-unknown-linux-musl/release/ignite-guest-agent \\\n ~/.ignite/guest-agent" + ); println!(" Then re-run: ignite setup"); + missing.push("guest rootfs image"); } // Summary - println!("\n Setup complete!"); + if missing.is_empty() { + println!("\n Setup complete!"); + } else { + println!("\n \x1b[33mSetup incomplete.\x1b[0m Still missing:"); + for item in &missing { + println!(" - {}", item); + } + println!("\n Services cannot run until the kernel, rootfs, and a runtime are present."); + } println!("\n To verify:"); println!(" ignite status"); println!(); @@ -890,8 +965,11 @@ async fn main() -> Result<()> { Commands::Run { service, input, + runtime, skip_preflight, json, + audit, + audit_output, verbose, memory, cpus, @@ -900,13 +978,20 @@ async fn main() -> Result<()> { runtimes_root, vsock_port, console_out, - .. } => { + if audit_output.is_some() && !audit { + return Err(IgniteError::Config { + message: "--audit-output requires --audit".to_string(), + source: None, + }); + } handle_run( service, input, + runtime, skip_preflight, json, + audit, verbose, memory, cpus, @@ -943,20 +1028,38 @@ async fn main() -> Result<()> { host, services, } => { + let api_key = std::env::var("IGNITE_API_KEY").ok(); + if api_key.is_none() { + tracing::warn!( + "IGNITE_API_KEY is not set: this server will execute services for any caller \ + that can reach it. Set IGNITE_API_KEY, or keep the bind address on localhost." + ); + } + + // Comma-separated exact origins, e.g. "https://app.example.com". + let allowed_origins: Vec = std::env::var("IGNITE_CORS_ORIGINS") + .ok() + .map(|raw| { + raw.split(',') + .map(|s| s.trim().to_string()) + .filter(|s| !s.is_empty()) + .collect() + }) + .unwrap_or_default(); + let state = std::sync::Arc::new(ignite_http::server::ServerState { services_path: PathBuf::from(services), - api_key: std::env::var("IGNITE_API_KEY").ok(), + api_key, rate_limiter: ignite_http::server::RateLimiter::new(60, 60), kernel_path: std::env::var("IGNITE_KERNEL_PATH").ok().map(PathBuf::from), rootfs_path: std::env::var("IGNITE_ROOTFS_PATH").ok().map(PathBuf::from), runtimes_root: std::env::var("IGNITE_RUNTIMES_ROOT") .ok() .map(PathBuf::from), + allowed_origins, }); let app = ignite_http::server::create_router(state); - let socket_addr: std::net::SocketAddr = format!("{}:{}", host, port) - .parse() - .unwrap_or_else(|_| std::net::SocketAddr::from(([127, 0, 0, 1], port))); + let socket_addr: std::net::SocketAddr = resolve_bind_addr(&host, port)?; tracing::info!("Ignite HTTP API server listening on http://{}", socket_addr); let listener = tokio::net::TcpListener::bind(socket_addr) @@ -965,12 +1068,16 @@ async fn main() -> Result<()> { message: format!("Failed to bind TCP listener on {}", socket_addr), source: Some(Box::new(e)), })?; - axum::serve(listener, app) - .await - .map_err(|e| IgniteError::Runtime { - message: "HTTP server error".to_string(), - source: Some(Box::new(e)), - })?; + // ConnectInfo is required for per-client rate limiting. + axum::serve( + listener, + app.into_make_service_with_connect_info::(), + ) + .await + .map_err(|e| IgniteError::Runtime { + message: "HTTP server error".to_string(), + source: Some(Box::new(e)), + })?; } Commands::Setup { force } => { handle_setup(force)?; diff --git a/ignite-core/src/execution.rs b/ignite-core/src/execution.rs index 8b76562..9f40383 100644 --- a/ignite-core/src/execution.rs +++ b/ignite-core/src/execution.rs @@ -1,14 +1,17 @@ use crate::disk::create_ext4_image; -use crate::orchestrator::VmConfig; +use crate::orchestrator::{OutputCallback, VmConfig}; use crate::platform::get_orchestrator; use crate::preflight::run_preflight; use ignite_shared::error::{IgniteError, Result}; -use ignite_shared::types::{ExecutionMetrics, PreflightResult, PreflightStatus, ServiceConfig}; +use ignite_shared::types::{ + ExecutionMetrics, PreflightResult, PreflightStatus, RuntimeSpec, ServiceConfig, +}; +use ignite_shared::validation::{sanitize_path_segment, validate_service_name}; use std::collections::HashMap; use std::fs; use std::path::{Path, PathBuf}; -#[derive(Debug, Clone)] +#[derive(Debug, Clone, Default)] pub struct ExecuteOptions { pub input: Option, pub env: HashMap, @@ -16,6 +19,8 @@ pub struct ExecuteOptions { pub audit: bool, pub memory_override: Option, pub cpu_override: Option, + /// Overrides `service.runtime` from service.yaml (e.g. `bun@1.3`). + pub runtime_override: Option, pub kernel_path: Option, pub rootfs_path: Option, pub runtimes_root: Option, @@ -23,6 +28,51 @@ pub struct ExecuteOptions { pub console_out: Option, } +/// Convert a fractional CPU allowance into a vCPU count Firecracker will +/// accept. Firecracker allocates whole vCPUs, so a share like `0.5` has to +/// round up — truncating would ask for zero vCPUs and be rejected outright. +fn resolve_vcpu_count(cpu_limit: Option) -> u8 { + match cpu_limit { + Some(c) if c.is_finite() && c > 0.0 => c.ceil().clamp(1.0, u8::MAX as f32) as u8, + _ => 1, + } +} + +/// Locate the on-host directory holding a runtime's binaries. +/// +/// A pinned version (`bun@1.3`) prefers `runtimes/bun@1.3` and falls back to +/// the unversioned `runtimes/bun` so existing installs keep working — but the +/// fallback is logged, because silently running a different version than the +/// one requested is exactly the kind of thing that should not be silent. +fn resolve_runtime_dir(runtimes_base: &Path, spec: &RuntimeSpec) -> Option { + // Sanitize the name and version separately, then join them with a literal + // `@`. Sanitizing the joined string would rewrite the `@` itself (it is not + // in the allowed set) and never match the on-disk directory. + let name = sanitize_path_segment(&spec.name); + + if let Some(version) = &spec.version { + let versioned = runtimes_base.join(format!("{}@{}", name, sanitize_path_segment(version))); + if versioned.exists() { + return Some(versioned); + } + } + + let unversioned = runtimes_base.join(&name); + if unversioned.exists() { + if spec.version.is_some() { + tracing::warn!( + runtime = %spec.format(), + path = ?unversioned, + "No version-specific runtime directory found; falling back to the unversioned \ + install. The running version may not match the pinned one." + ); + } + return Some(unversioned); + } + + None +} + fn load_service_config(service_path: &Path) -> Result { let yaml_path = service_path.join("service.yaml"); if !yaml_path.exists() { @@ -44,17 +94,60 @@ fn default_ignite_dir() -> PathBuf { } } -#[allow(clippy::type_complexity)] pub fn execute_service( service_path: &Path, options: ExecuteOptions, - on_stdout: Option>, - on_stderr: Option>, + on_stdout: OutputCallback, + on_stderr: OutputCallback, ) -> Result<(PreflightResult, ExecutionMetrics)> { // 1. Load service configuration let config = load_service_config(service_path)?; - // 2. Preflight checks + // 2. Validate untrusted config fields before they reach host paths or the + // hypervisor. `service.name` is interpolated into socket paths that are + // later unlinked, so an unchecked `../..` here means arbitrary file + // removal on the host. + let name_check = validate_service_name(&config.service.name); + if !name_check.valid { + return Err(IgniteError::Config { + message: format!( + "Invalid service name {:?} in service.yaml: {}", + config.service.name, + name_check.error.unwrap_or_else(|| "invalid".to_string()) + ), + source: None, + }); + } + + // Audit mode advertises in-guest syscall/network/filesystem accounting that + // has no enforcement layer behind it yet. Accepting the flag silently would + // report a clean audit for code that was never actually audited, so refuse. + if options.audit { + return Err(IgniteError::Config { + message: "Security audit mode is not implemented in this build. Re-run without \ + --audit; see docs/threat-model.md for the isolation that is enforced." + .to_string(), + source: None, + }); + } + + let runtime_str = options + .runtime_override + .clone() + .unwrap_or_else(|| config.service.runtime.clone()); + + if !crate::runtime::is_valid_runtime(&runtime_str) { + return Err(IgniteError::Config { + message: format!( + "Invalid runtime {:?}. Supported runtimes: bun, node, deno, quickjs (optionally \ + pinned as name@version).", + runtime_str + ), + source: None, + }); + } + + // 3. Preflight checks let preflight_result = run_preflight(service_path, &config, None)?; if !options.skip_preflight && preflight_result.overall_status == PreflightStatus::Fail { return Err(IgniteError::Preflight { @@ -63,32 +156,33 @@ pub fn execute_service( }); } - // 3. Setup temporary folder for block disks + // 4. Setup temporary folder for block disks let temp_dir = tempfile::tempdir()?; let service_disk_path = temp_dir.path().join("service.ext4"); let runtime_disk_path = temp_dir.path().join("runtime.ext4"); - // 4. Generate ext4 block device for service files + // 5. Generate ext4 block device for service files create_ext4_image(service_path, &service_disk_path)?; - // 5. Generate ext4 block device for target runtime binaries - let runtime_spec = ignite_shared::types::RuntimeSpec::parse(&config.service.runtime); + // 6. Generate ext4 block device for target runtime binaries + let runtime_spec = RuntimeSpec::parse(&runtime_str); let ignite_dir = default_ignite_dir(); let runtimes_base = options .runtimes_root .clone() .unwrap_or_else(|| ignite_dir.join("runtimes")); - let runtime_src_path = runtimes_base.join(&runtime_spec.name); - if !runtime_src_path.exists() { - return Err(IgniteError::Config { + let runtime_src_path = + resolve_runtime_dir(&runtimes_base, &runtime_spec).ok_or_else(|| IgniteError::Config { message: format!( - "Language runtime binaries folder not found: {:?}. Run `ignite setup` or set IGNITE_RUNTIMES_ROOT.", - runtime_src_path + "Language runtime binaries not found for {:?} under {:?}. Run `ignite setup` or \ + set IGNITE_RUNTIMES_ROOT.", + runtime_spec.format(), + runtimes_base ), source: None, - }); - } + })?; + create_ext4_image(&runtime_src_path, &runtime_disk_path)?; // 6. Resolve Hypervisor VM config @@ -110,6 +204,10 @@ pub fn execute_service( source: None, }); } + // Catch a compressed bzImage here rather than letting the guest fail to + // boot with an opaque hypervisor error. + crate::setup::validate_kernel_image(&kernel_path)?; + if !rootfs_path.exists() { return Err(IgniteError::Config { message: format!( @@ -122,16 +220,18 @@ pub fn execute_service( const DEFAULT_VSOCK_PORT: u32 = 1052; let vsock_port = options.vsock_port.unwrap_or(DEFAULT_VSOCK_PORT); - let vm_id = format!( - "{}-{:x}", - config.service.name, + // Name is validated above, but sanitize anyway: this string becomes a host + // path, and defense-in-depth is cheap. + let sanitized_name = sanitize_path_segment(&config.service.name); + let vsock_uds_path = PathBuf::from(format!( + "/tmp/ignite-vsock-{}-{}-{:x}", + sanitized_name, + std::process::id(), std::time::SystemTime::now() .duration_since(std::time::UNIX_EPOCH) .unwrap_or_default() .as_nanos() - ); - let sanitized_name = config.service.name.replace(['/', '\\', '.'], "_"); - let vsock_uds_path = PathBuf::from(format!("/tmp/ignite-vsock-{}-{}", sanitized_name, vm_id)); + )); // Merge environment variables let mut env = config.service.env.clone().unwrap_or_default(); @@ -150,10 +250,7 @@ pub fn execute_service( service_disk_path, runtime_disk_path, memory_mb: options.memory_override.unwrap_or(config.service.memory_mb), - vcpu_count: options - .cpu_override - .map(|c| c as u8) - .unwrap_or(config.service.cpu_limit.map(|c| c as u8).unwrap_or(1)), + vcpu_count: resolve_vcpu_count(options.cpu_override.or(config.service.cpu_limit)), vsock_port, vsock_uds_path, env, @@ -171,16 +268,9 @@ pub fn execute_service( let mut metrics = orchestrator.wait_and_teardown(on_stdout, on_stderr)?; - // Parse memory metrics from stderr trace (mimics TS implementation) + // Parse memory metrics from stderr trace, if the guest reported any. metrics.memory_usage_mb = parse_memory_from_stderr(&metrics.stderr); - metrics.cold_start_time_ms = if metrics.cold_start { - Some(estimate_cold_start_time( - &metrics.stderr, - metrics.execution_time_ms, - )) - } else { - None - }; + metrics.cold_start_time_ms = parse_init_time_from_stderr(&metrics.stderr); Ok((preflight_result, metrics)) } @@ -197,14 +287,105 @@ fn parse_memory_from_stderr(stderr: &str) -> f64 { 0.0 } -fn estimate_cold_start_time(stderr: &str, duration_ms: u64) -> u64 { - for line in stderr.lines() { - if let Some(pos) = line.find("IGNITE_INIT_TIME:") { - let val_str = &line[pos + "IGNITE_INIT_TIME:".len()..]; - if let Ok(val) = val_str.trim().parse::() { - return val; - } - } +/// Read the guest-reported init duration. Returns `None` when the runtime did +/// not emit one — previously this fell back to `min(duration, 200)`, which +/// reported a fabricated cold-start figure indistinguishable from a measured +/// one. +fn parse_init_time_from_stderr(stderr: &str) -> Option { + stderr.lines().find_map(|line| { + let pos = line.find("IGNITE_INIT_TIME:")?; + line[pos + "IGNITE_INIT_TIME:".len()..].trim().parse().ok() + }) +} + +#[cfg(test)] +mod tests { + use super::*; + + fn spec(s: &str) -> RuntimeSpec { + RuntimeSpec::parse(s) + } + + #[test] + fn vcpu_count_rounds_fractional_shares_up() { + // Firecracker allocates whole vCPUs; truncating 0.5 would request zero + // and be rejected outright. + assert_eq!(resolve_vcpu_count(Some(0.5)), 1); + assert_eq!(resolve_vcpu_count(Some(0.1)), 1); + assert_eq!(resolve_vcpu_count(Some(1.0)), 1); + assert_eq!(resolve_vcpu_count(Some(1.5)), 2); + assert_eq!(resolve_vcpu_count(Some(4.0)), 4); + } + + #[test] + fn vcpu_count_defaults_to_one_for_absent_or_invalid() { + assert_eq!(resolve_vcpu_count(None), 1); + assert_eq!(resolve_vcpu_count(Some(0.0)), 1); + assert_eq!(resolve_vcpu_count(Some(-3.0)), 1); + assert_eq!(resolve_vcpu_count(Some(f32::NAN)), 1); + assert_eq!(resolve_vcpu_count(Some(f32::INFINITY)), 1); + } + + #[test] + fn vcpu_count_saturates_at_u8() { + assert_eq!(resolve_vcpu_count(Some(1e9)), u8::MAX); + } + + #[test] + fn runtime_dir_prefers_pinned_version() { + let dir = tempfile::tempdir().unwrap(); + fs::create_dir_all(dir.path().join("bun")).unwrap(); + fs::create_dir_all(dir.path().join("bun@1.3")).unwrap(); + + let resolved = resolve_runtime_dir(dir.path(), &spec("bun@1.3")).unwrap(); + assert_eq!(resolved, dir.path().join("bun@1.3")); + } + + #[test] + fn runtime_dir_falls_back_to_unversioned() { + let dir = tempfile::tempdir().unwrap(); + fs::create_dir_all(dir.path().join("bun")).unwrap(); + + let resolved = resolve_runtime_dir(dir.path(), &spec("bun@1.3")).unwrap(); + assert_eq!(resolved, dir.path().join("bun")); + } + + #[test] + fn runtime_dir_is_none_when_missing() { + let dir = tempfile::tempdir().unwrap(); + assert!(resolve_runtime_dir(dir.path(), &spec("deno")).is_none()); + } + + #[test] + fn runtime_dir_cannot_escape_the_runtimes_root() { + let dir = tempfile::tempdir().unwrap(); + // A traversal attempt must not resolve outside the base directory. + let resolved = resolve_runtime_dir(dir.path(), &spec("../../etc")); + assert!( + resolved.is_none() || resolved.unwrap().starts_with(dir.path()), + "runtime resolution escaped the runtimes root" + ); + } + + #[test] + fn init_time_is_none_when_guest_reports_nothing() { + // A fabricated cold-start figure is indistinguishable from a measured + // one, so absence must stay absent. + assert_eq!(parse_init_time_from_stderr(""), None); + assert_eq!(parse_init_time_from_stderr("some unrelated output"), None); + } + + #[test] + fn init_time_is_parsed_when_reported() { + assert_eq!( + parse_init_time_from_stderr("boot\nIGNITE_INIT_TIME: 42\ndone"), + Some(42) + ); + } + + #[test] + fn memory_is_parsed_from_stderr() { + assert_eq!(parse_memory_from_stderr("IGNITE_MEMORY_MB: 12.5"), 12.5); + assert_eq!(parse_memory_from_stderr("nothing here"), 0.0); } - std::cmp::min(duration_ms, 200) } diff --git a/ignite-core/src/orchestrator.rs b/ignite-core/src/orchestrator.rs index 4f20557..c75d9cb 100644 --- a/ignite-core/src/orchestrator.rs +++ b/ignite-core/src/orchestrator.rs @@ -22,6 +22,9 @@ pub struct VmConfig { pub timeout_ms: u32, } +/// Sink for a guest output stream. Invoked per frame as output arrives. +pub type OutputCallback = Option>; + pub trait MicroVmOrchestrator { /// Configure microVM settings (CPUs, RAM, storage attachments, VSOCK channels) fn configure(&mut self, config: VmConfig) -> Result<()>; @@ -30,10 +33,9 @@ pub trait MicroVmOrchestrator { fn boot(&mut self) -> Result<()>; /// Block, stream child standard streams (via VSOCK), capture exit code, and cleanly tear down loopbacks - #[allow(clippy::type_complexity)] fn wait_and_teardown( &mut self, - on_stdout: Option>, - on_stderr: Option>, + on_stdout: OutputCallback, + on_stderr: OutputCallback, ) -> Result; } diff --git a/ignite-core/src/platform/apple_vz.rs b/ignite-core/src/platform/apple_vz.rs index 9db3251..2d469f7 100644 --- a/ignite-core/src/platform/apple_vz.rs +++ b/ignite-core/src/platform/apple_vz.rs @@ -1,79 +1,49 @@ -use crate::orchestrator::{MicroVmOrchestrator, VmConfig}; +use crate::orchestrator::{MicroVmOrchestrator, OutputCallback, VmConfig}; use ignite_shared::error::{IgniteError, Result}; use ignite_shared::types::ExecutionMetrics; +/// Placeholder for the macOS `Virtualization.framework` backend. +/// +/// This backend is **not implemented**. The struct exists so the platform +/// selector compiles on every target, and so the failure surfaces as a clear +/// error at boot rather than a panic partway through execution. +/// +/// Implementing it requires, roughly: +/// +/// 1. `VZLinuxBootLoader` with `kernelURL = config.kernel_path` and +/// `commandLine = "console=hvc0 reboot=k panic=1 pci=off init=/sbin/init"`. +/// 2. `VZVirtualMachineConfiguration` with `cpuCount` / `memorySize` from +/// `config`, plus the rootfs, service, and runtime images attached as +/// read-only `VZVirtioBlockDeviceConfiguration` entries. +/// 3. A `VZVirtioSocketDevice` listening on `config.vsock_port`, speaking the +/// same length-prefixed frame protocol as the Firecracker backend. +/// 4. A host-side watchdog enforcing `config.timeout_ms`. pub struct AppleVzOrchestrator { + #[allow(dead_code)] config: Option, - #[cfg(target_os = "macos")] - vm: Option, // Core VM handle } -impl AppleVzOrchestrator { - pub fn new() -> Self { - AppleVzOrchestrator { - config: None, - #[cfg(target_os = "macos")] - vm: None, - } +impl Default for AppleVzOrchestrator { + fn default() -> Self { + Self::new() } } -// ============================================================================ -// macOS Target: Native Apple Virtualization.framework Execution -// ============================================================================ -#[cfg(target_os = "macos")] -impl MicroVmOrchestrator for AppleVzOrchestrator { - fn configure(&mut self, config: VmConfig) -> Result<()> { - self.config = Some(config); - Ok(()) +impl AppleVzOrchestrator { + pub fn new() -> Self { + AppleVzOrchestrator { config: None } } - fn boot(&mut self) -> Result<()> { - let config = self.config.as_ref().ok_or_else(|| IgniteError::Config { - message: "VM not configured".to_string(), + fn unsupported() -> IgniteError { + IgniteError::Runtime { + message: "The macOS Virtualization.framework backend is not implemented in this \ + build. Ignite currently executes services on Linux hosts via Firecracker." + .to_string(), source: None, - })?; - - // 1. Apple Virtualization Framework Config Setup - // VZLinuxBootLoader bootloader config: - // Set kernelURL = config.kernel_path - // Set commandLine = "console=hvc0 reboot=k panic=1 pci=off init=/usr/local/bin/ignite-guest-agent" - // - // VZVirtualMachineConfiguration setup: - // - set cpuCount = config.vcpu_count - // - set memorySize = config.memory_mb * 1024 * 1024 - // - // 2. Storage Setup (VirtIO Block devices) - // Attach config.rootfs_path, service_disk_path, and runtime_disk_path as read-only VirtIO storage devices - // - // 3. Socket Setup (VirtIO Socket devices) - // Configure VZVirtioSocketDevice to map guest connections on port 1052 - // - // 4. Boot VM - // vm.start() with callback completions. - - tracing::info!("macOS hypervisor boot initialization completed."); - Ok(()) - } - - fn wait_and_teardown( - &mut self, - on_stdout: Option>, - on_stderr: Option>, - ) -> Result { - // 1. Accept VirtioSocket connection on the host-side socket handler. - // 2. Write the JSON execution payload payload (len prefixed). - // 3. Stream stdout / stderr chunks in real-time, calling callbacks. - // 4. Capture exit status frame. - // 5. Terminate VM and perform disk allocations cleanup. - unimplemented!("macOS AppleVz native execution wait logic"); + } } } -// ============================================================================ -// Non-macOS Targets: Safe Compile-time Stubs -// ============================================================================ -#[cfg(not(target_os = "macos"))] impl MicroVmOrchestrator for AppleVzOrchestrator { fn configure(&mut self, config: VmConfig) -> Result<()> { self.config = Some(config); @@ -81,20 +51,14 @@ impl MicroVmOrchestrator for AppleVzOrchestrator { } fn boot(&mut self) -> Result<()> { - Err(IgniteError::Runtime { - message: "macOS Virtualization backend is only supported on macOS systems".to_string(), - source: None, - }) + Err(Self::unsupported()) } fn wait_and_teardown( &mut self, - _on_stdout: Option>, - _on_stderr: Option>, + _on_stdout: OutputCallback, + _on_stderr: OutputCallback, ) -> Result { - Err(IgniteError::Runtime { - message: "macOS Virtualization backend is only supported on macOS systems".to_string(), - source: None, - }) + Err(Self::unsupported()) } } diff --git a/ignite-core/src/platform/firecracker.rs b/ignite-core/src/platform/firecracker.rs index 393e0a1..f1a7f42 100644 --- a/ignite-core/src/platform/firecracker.rs +++ b/ignite-core/src/platform/firecracker.rs @@ -2,28 +2,50 @@ use std::fs; use std::io::{Read, Write}; use std::os::unix::net::{UnixListener, UnixStream}; use std::path::{Path, PathBuf}; -use std::process::{Child, Command}; +use std::process::{Child, Command, Stdio}; use std::thread; use std::time::{Duration, Instant}; -use crate::orchestrator::{MicroVmOrchestrator, VmConfig}; +use crate::orchestrator::{MicroVmOrchestrator, OutputCallback, VmConfig}; use ignite_shared::error::{IgniteError, Result}; use ignite_shared::types::ExecutionMetrics; +use ignite_shared::validation::sanitize_path_segment; + +/// VSOCK frame type tags, shared with the guest agent. +const FRAME_STDOUT: u8 = 1; +const FRAME_STDERR: u8 = 2; +const FRAME_EXIT: u8 = 3; + +/// Cap on stdout/stderr retained in memory per stream (8 MiB). +const MAX_CAPTURED_OUTPUT_BYTES: usize = 8 * 1024 * 1024; #[derive(Debug)] pub struct FirecrackerOrchestrator { child_process: Option, config: Option, api_socket_path: PathBuf, + /// Bound before the VM starts so the guest agent can never race ahead of + /// the host listener. Consumed by the first `accept()`. + vsock_listener: Option, + vsock_listener_path: Option, } const DEFAULT_API_SOCKET: &str = "/tmp/firecracker-api.sock"; const API_SOCKET_PREFIX: &str = "/tmp/ignite-api-"; -const VSOCK_PORT_SUFFIX: &str = "_1052"; -const API_READY_TIMEOUT_MS: u64 = 500; +const API_READY_TIMEOUT_MS: u64 = 5_000; const API_READY_POLL_MS: u64 = 5; const GUEST_CID: u32 = 3; const IO_POLL_TIMEOUT_MS: u64 = 50; +const ACCEPT_POLL_MS: u64 = 5; + +/// Absolute path the guest agent is installed at inside the rootfs image. +/// Must match where `setup::create_rootfs` writes it. +const GUEST_INIT_PATH: &str = "/sbin/init"; + +/// Upper bound on a single stdout/stderr frame from the guest. The length +/// prefix is guest-controlled, so without this a sandboxed program could ask +/// the host to allocate up to 4 GiB. +const MAX_FRAME_BYTES: usize = 4 * 1024 * 1024; impl Default for FirecrackerOrchestrator { fn default() -> Self { @@ -37,8 +59,33 @@ impl FirecrackerOrchestrator { child_process: None, config: None, api_socket_path: PathBuf::from(DEFAULT_API_SOCKET), + vsock_listener: None, + vsock_listener_path: None, } } + + /// Firecracker expects the host to listen on `_` for + /// guest-initiated connections. + fn vsock_listener_path_for(config: &VmConfig) -> PathBuf { + PathBuf::from(format!( + "{}_{}", + config.vsock_uds_path.to_string_lossy(), + config.vsock_port + )) + } +} + +/// True when an HTTP response's status line carries a 2xx code. +/// +/// Checked against the status line specifically: a substring search over the +/// whole response can be satisfied by the body echoing the request. +fn is_success_response(response: &str) -> bool { + response + .lines() + .next() + .and_then(|line| line.split_whitespace().nth(1)) + .and_then(|code| code.parse::().ok()) + .is_some_and(|code| (200..300).contains(&code)) } fn send_put_uds(socket_path: &Path, endpoint: &str, body: &str) -> Result<()> { @@ -60,7 +107,7 @@ fn send_put_uds(socket_path: &Path, endpoint: &str, body: &str) -> Result<()> { // Read the server reply. Because Connection: close is set, this reads until EOF. let _ = stream.read_to_string(&mut response); - if !response.contains("HTTP/1.1 2") { + if !is_success_response(&response) { return Err(IgniteError::Runtime { message: format!( "Firecracker API (PUT {}) returned error: {}", @@ -74,33 +121,66 @@ fn send_put_uds(socket_path: &Path, endpoint: &str, body: &str) -> Result<()> { impl MicroVmOrchestrator for FirecrackerOrchestrator { fn configure(&mut self, config: VmConfig) -> Result<()> { - self.api_socket_path = - PathBuf::from(format!("{}{}.sock", API_SOCKET_PREFIX, config.service_name)); + // `service_name` comes from user-controlled service.yaml and is + // interpolated into a host path that boot() later unlinks, so it must + // be reduced to a single safe path segment. The vsock path is already + // unique per run; reuse its file name so concurrent runs of the same + // service cannot collide on the API socket either. + let unique = config + .vsock_uds_path + .file_name() + .map(|n| n.to_string_lossy().into_owned()) + .unwrap_or_else(|| config.service_name.clone()); + + self.api_socket_path = PathBuf::from(format!( + "{}{}.sock", + API_SOCKET_PREFIX, + sanitize_path_segment(&unique) + )); + self.vsock_listener_path = Some(Self::vsock_listener_path_for(&config)); self.config = Some(config); Ok(()) } fn boot(&mut self) -> Result<()> { - let config = self.config.as_ref().ok_or_else(|| IgniteError::Config { + let config = self.config.clone().ok_or_else(|| IgniteError::Config { message: "VM not configured before boot".to_string(), source: None, })?; + let vsock_listener_path = Self::vsock_listener_path_for(&config); // 1. Clean up any stale sockets let _ = fs::remove_file(&self.api_socket_path); let _ = fs::remove_file(&config.vsock_uds_path); - - let vsock_listener_path = format!( - "{}{}", - config.vsock_uds_path.to_string_lossy(), - VSOCK_PORT_SUFFIX - ); let _ = fs::remove_file(&vsock_listener_path); - // 2. Spawn firecracker process + // 2. Bind the host VSOCK listener *before* the VM starts. The guest + // agent dials out as soon as it boots; if we bound after + // InstanceStart the guest could exhaust its retries first. + let listener = UnixListener::bind(&vsock_listener_path)?; + listener.set_nonblocking(true)?; + self.vsock_listener = Some(listener); + self.vsock_listener_path = Some(vsock_listener_path); + + // 3. Spawn firecracker process. Guest serial output goes to the + // hypervisor's stdout; route it to --console-out when requested. + let (stdout_sink, stderr_sink) = match config.console_out { + Some(ref path) => { + if let Some(parent) = path.parent().filter(|p| !p.as_os_str().is_empty()) { + fs::create_dir_all(parent)?; + } + let file = fs::File::create(path)?; + let dup = file.try_clone()?; + (Stdio::from(file), Stdio::from(dup)) + } + None => (Stdio::null(), Stdio::null()), + }; + let child = Command::new("firecracker") .arg("--api-sock") .arg(&self.api_socket_path) + .stdout(stdout_sink) + .stderr(stderr_sink) .spawn() .map_err(|e| IgniteError::Runtime { message: format!( @@ -111,7 +191,7 @@ impl MicroVmOrchestrator for FirecrackerOrchestrator { })?; self.child_process = Some(child); - // 3. Wait for UDS API to become ready (timeout 500ms) + // 4. Wait for UDS API to become ready let start = Instant::now(); let api_ready = loop { if self.api_socket_path.exists() && UnixStream::connect(&self.api_socket_path).is_ok() { @@ -124,101 +204,94 @@ impl MicroVmOrchestrator for FirecrackerOrchestrator { }; if !api_ready { - if let Some(mut child) = self.child_process.take() { - let _ = child.kill(); - } + self.cleanup_vm_processes(); return Err(IgniteError::Runtime { - message: "Timeout waiting for Firecracker API socket to be active".to_string(), + message: format!( + "Timeout waiting for Firecracker API socket to be active after {}ms", + API_READY_TIMEOUT_MS + ), source: None, }); } - // 4. Configure Virtual Machine parameters - // Machine config - let machine_body = serde_json::json!({ - "vcpu_count": config.vcpu_count, - "mem_size_mib": config.memory_mb, - }) - .to_string(); - send_put_uds(&self.api_socket_path, "/machine-config", &machine_body)?; - - // Boot Source & Kernel config - let boot_body = serde_json::json!({ - "kernel_image_path": config.kernel_path, - "boot_args": "console=ttyS0 reboot=k panic=1 pci=off init=/usr/local/bin/ignite-guest-agent", - }).to_string(); - send_put_uds(&self.api_socket_path, "/boot-source", &boot_body)?; - - // Storage attachment - Root FS (/dev/vda) - let rootfs_body = serde_json::json!({ - "drive_id": "rootfs", - "path_on_host": config.rootfs_path, - "is_root_device": true, - "is_read_only": true, - }) - .to_string(); - send_put_uds(&self.api_socket_path, "/drives/rootfs", &rootfs_body)?; - - // Storage attachment - Service files (/dev/vdb) - let service_body = serde_json::json!({ - "drive_id": "service", - "path_on_host": config.service_disk_path, - "is_root_device": false, - "is_read_only": true, - }) - .to_string(); - send_put_uds(&self.api_socket_path, "/drives/service", &service_body)?; - - // Storage attachment - Language runtime binaries (/dev/vdc) - let runtime_body = serde_json::json!({ - "drive_id": "runtime", - "path_on_host": config.runtime_disk_path, - "is_root_device": false, - "is_read_only": true, - }) - .to_string(); - send_put_uds(&self.api_socket_path, "/drives/runtime", &runtime_body)?; - - // VSOCK Device setup - let vsock_body = serde_json::json!({ - "vsock_id": "vsock0", - "guest_cid": GUEST_CID, - "uds_path": config.vsock_uds_path, - }) - .to_string(); - send_put_uds(&self.api_socket_path, "/vsock", &vsock_body)?; - - // Start VM instance - let action_body = serde_json::json!({ - "action_type": "InstanceStart", - }) - .to_string(); - send_put_uds(&self.api_socket_path, "/actions", &action_body)?; + // 5. Configure Virtual Machine parameters + if let Err(e) = self.apply_vm_configuration(&config) { + self.cleanup_vm_processes(); + return Err(e); + } Ok(()) } fn wait_and_teardown( &mut self, - on_stdout: Option>, - on_stderr: Option>, + on_stdout: OutputCallback, + on_stderr: OutputCallback, ) -> Result { let config = self.config.clone().ok_or_else(|| IgniteError::Config { message: "VM not configured".to_string(), source: None, })?; - let vsock_listener_path = format!( - "{}{}", - config.vsock_uds_path.to_string_lossy(), - VSOCK_PORT_SUFFIX - ); - - // 1. Host VSOCK listener for guest-initiated connections - let listener = UnixListener::bind(&vsock_listener_path)?; - listener.set_nonblocking(true)?; + // The listener was bound in boot(), before the VM was started. + let listener = self + .vsock_listener + .take() + .ok_or_else(|| IgniteError::Execution { + message: "VSOCK listener not bound; boot() must run before wait_and_teardown()" + .to_string(), + source: None, + })?; let start_time = Instant::now(); + let result = Self::stream_execution(&listener, &config, start_time, on_stdout, on_stderr); + let duration_ms = start_time.elapsed().as_millis() as u64; + + // Always tear the hypervisor down, success or failure. + self.cleanup_vm_processes(); + + let session = result?; + + Ok(ExecutionMetrics { + execution_time_ms: duration_ms, + memory_usage_mb: 0.0, // Will be parsed out of stderr by metric utilities + // Every run boots a fresh microVM; there is no warm pool yet. + cold_start: true, + cold_start_time_ms: None, + exit_code: session.exit_code, + stdout: session.stdout, + stderr: session.stderr, + }) + } +} + +/// Outcome of one guest session over VSOCK. +struct GuestSession { + exit_code: i32, + stdout: String, + stderr: String, +} + +impl FirecrackerOrchestrator { + /// Accept the guest agent's connection, hand it the execution payload, and + /// pump multiplexed output frames until the guest reports an exit code or + /// the watchdog fires. + fn stream_execution( + listener: &UnixListener, + config: &VmConfig, + start_time: Instant, + on_stdout: OutputCallback, + on_stderr: OutputCallback, + ) -> Result { + let timed_out = || IgniteError::Execution { + message: format!( + "Execution exceeded timeoutMs ({}ms); VM force-terminated by host watchdog", + config.timeout_ms + ), + source: None, + }; + + // 1. Wait for the guest-initiated connection. let mut socket = loop { match listener.accept() { Ok((stream, _)) => { @@ -228,19 +301,17 @@ impl MicroVmOrchestrator for FirecrackerOrchestrator { } Err(ref e) if e.kind() == std::io::ErrorKind::WouldBlock => { if start_time.elapsed().as_millis() > (config.timeout_ms as u128) { - self.cleanup_vm_processes(); return Err(IgniteError::Execution { - message: "Timeout waiting for guest agent to connect over VSOCK" - .to_string(), + message: format!( + "Timeout waiting for guest agent to connect over VSOCK after {}ms", + config.timeout_ms + ), source: None, }); } - thread::sleep(Duration::from_millis(5)); - } - Err(e) => { - self.cleanup_vm_processes(); - return Err(e.into()); + thread::sleep(Duration::from_millis(ACCEPT_POLL_MS)); } + Err(e) => return Err(e.into()), } }; @@ -263,94 +334,62 @@ impl MicroVmOrchestrator for FirecrackerOrchestrator { let mut stderr_accum = String::new(); loop { - let mut type_byte = [0u8; 1]; - if let Err(e) = Self::read_exact_with_timeout( + let mut header = [0u8; 5]; + match Self::read_exact_with_timeout( &mut socket, - &mut type_byte, + &mut header, start_time, config.timeout_ms, ) { - if e.kind() == std::io::ErrorKind::UnexpectedEof { - break; // VM socket disconnected - } - if e.kind() == std::io::ErrorKind::TimedOut { - self.cleanup_vm_processes(); - return Err(IgniteError::Execution { - message: format!( - "Execution exceeded timeoutMs ({}ms); VM force-terminated by host watchdog", - config.timeout_ms - ), - source: None, - }); - } - self.cleanup_vm_processes(); - return Err(e.into()); + Ok(()) => {} + Err(e) if e.kind() == std::io::ErrorKind::UnexpectedEof => break, + Err(e) if e.kind() == std::io::ErrorKind::TimedOut => return Err(timed_out()), + Err(e) => return Err(e.into()), + } + + let frame_type = header[0]; + let length = u32::from_be_bytes(header[1..5].try_into().unwrap()) as usize; + + // The length prefix is written by the guest, which runs untrusted + // code; refuse to allocate on its word alone. + if length > MAX_FRAME_BYTES { + return Err(IgniteError::Execution { + message: format!( + "Guest sent an oversized VSOCK frame ({} bytes, max {}). Aborting.", + length, MAX_FRAME_BYTES + ), + source: None, + }); } - let mut len_bytes = [0u8; 4]; - if let Err(e) = Self::read_exact_with_timeout( + let mut data = vec![0u8; length]; + match Self::read_exact_with_timeout( &mut socket, - &mut len_bytes, + &mut data, start_time, config.timeout_ms, ) { - if e.kind() == std::io::ErrorKind::UnexpectedEof { - break; - } - if e.kind() == std::io::ErrorKind::TimedOut { - self.cleanup_vm_processes(); - return Err(IgniteError::Execution { - message: format!( - "Execution exceeded timeoutMs ({}ms); VM force-terminated by host watchdog", - config.timeout_ms - ), - source: None, - }); - } - self.cleanup_vm_processes(); - return Err(e.into()); - } - let length = u32::from_be_bytes(len_bytes) as usize; - - let mut data = vec![0u8; length]; - if let Err(e) = - Self::read_exact_with_timeout(&mut socket, &mut data, start_time, config.timeout_ms) - { - if e.kind() == std::io::ErrorKind::UnexpectedEof { - break; - } - if e.kind() == std::io::ErrorKind::TimedOut { - self.cleanup_vm_processes(); - return Err(IgniteError::Execution { - message: format!( - "Execution exceeded timeoutMs ({}ms); VM force-terminated by host watchdog", - config.timeout_ms - ), - source: None, - }); - } - self.cleanup_vm_processes(); - return Err(e.into()); + Ok(()) => {} + Err(e) if e.kind() == std::io::ErrorKind::UnexpectedEof => break, + Err(e) if e.kind() == std::io::ErrorKind::TimedOut => return Err(timed_out()), + Err(e) => return Err(e.into()), } let chunk = String::from_utf8_lossy(&data); - match type_byte[0] { - 1 => { - // stdout - stdout_accum.push_str(&chunk); + match frame_type { + FRAME_STDOUT => { + append_capped(&mut stdout_accum, &chunk); if let Some(ref cb) = on_stdout { cb(&chunk); } } - 2 => { - // stderr - stderr_accum.push_str(&chunk); + FRAME_STDERR => { + append_capped(&mut stderr_accum, &chunk); if let Some(ref cb) = on_stderr { cb(&chunk); } } - 3 => { - // exit code + FRAME_EXIT => { if let Some(arr) = data.get(..4).and_then(|slice| slice.try_into().ok()) { exit_code = i32::from_be_bytes(arr); } @@ -360,16 +399,7 @@ impl MicroVmOrchestrator for FirecrackerOrchestrator { } } - let duration_ms = start_time.elapsed().as_millis() as u64; - - // 4. Teardown hypervisor processes and socket files - self.cleanup_vm_processes(); - - Ok(ExecutionMetrics { - execution_time_ms: duration_ms, - memory_usage_mb: 0.0, // Will be parsed out of stderr by metric utilities - cold_start: false, - cold_start_time_ms: None, + Ok(GuestSession { exit_code, stdout: stdout_accum, stderr: stderr_accum, @@ -377,24 +407,127 @@ impl MicroVmOrchestrator for FirecrackerOrchestrator { } } +/// Accumulate captured output without letting a chatty guest exhaust host +/// memory. Streaming callbacks still see every chunk; only the retained copy +/// returned in `ExecutionMetrics` is bounded. +fn append_capped(buffer: &mut String, chunk: &str) { + if buffer.len() >= MAX_CAPTURED_OUTPUT_BYTES { + return; + } + let remaining = MAX_CAPTURED_OUTPUT_BYTES - buffer.len(); + if chunk.len() <= remaining { + buffer.push_str(chunk); + } else { + let mut cut = remaining; + while cut > 0 && !chunk.is_char_boundary(cut) { + cut -= 1; + } + buffer.push_str(&chunk[..cut]); + buffer.push_str("\n[ignite: output truncated]\n"); + } +} + +impl Drop for FirecrackerOrchestrator { + /// Guarantees the hypervisor process and its socket files are reaped even + /// if an error path returns between boot() and wait_and_teardown(). + fn drop(&mut self) { + self.cleanup_vm_processes(); + } +} + impl FirecrackerOrchestrator { + /// Push the full machine definition to the Firecracker API and start the + /// instance. Split out of `boot` so every failure path can tear down the + /// hypervisor process instead of leaking it. + fn apply_vm_configuration(&self, config: &VmConfig) -> Result<()> { + let machine_body = serde_json::json!({ + "vcpu_count": config.vcpu_count, + "mem_size_mib": config.memory_mb, + }) + .to_string(); + send_put_uds(&self.api_socket_path, "/machine-config", &machine_body)?; + + // Boot Source & Kernel config. `init=` must match where + // setup::create_rootfs installs the guest agent inside the image. + let boot_body = serde_json::json!({ + "kernel_image_path": config.kernel_path, + "boot_args": format!( + "console=ttyS0 reboot=k panic=1 pci=off init={}", + GUEST_INIT_PATH + ), + }) + .to_string(); + send_put_uds(&self.api_socket_path, "/boot-source", &boot_body)?; + + // Storage attachment - Root FS (/dev/vda) + let rootfs_body = serde_json::json!({ + "drive_id": "rootfs", + "path_on_host": config.rootfs_path, + "is_root_device": true, + "is_read_only": true, + }) + .to_string(); + send_put_uds(&self.api_socket_path, "/drives/rootfs", &rootfs_body)?; + + // Storage attachment - Service files (/dev/vdb) + let service_body = serde_json::json!({ + "drive_id": "service", + "path_on_host": config.service_disk_path, + "is_root_device": false, + "is_read_only": true, + }) + .to_string(); + send_put_uds(&self.api_socket_path, "/drives/service", &service_body)?; + + // Storage attachment - Language runtime binaries (/dev/vdc) + let runtime_body = serde_json::json!({ + "drive_id": "runtime", + "path_on_host": config.runtime_disk_path, + "is_root_device": false, + "is_read_only": true, + }) + .to_string(); + send_put_uds(&self.api_socket_path, "/drives/runtime", &runtime_body)?; + + // VSOCK Device setup + let vsock_body = serde_json::json!({ + "vsock_id": "vsock0", + "guest_cid": GUEST_CID, + "uds_path": config.vsock_uds_path, + }) + .to_string(); + send_put_uds(&self.api_socket_path, "/vsock", &vsock_body)?; + + // Start VM instance + let action_body = serde_json::json!({ + "action_type": "InstanceStart", + }) + .to_string(); + send_put_uds(&self.api_socket_path, "/actions", &action_body)?; + + Ok(()) + } + fn cleanup_vm_processes(&mut self) { if let Some(mut child) = self.child_process.take() { let _ = child.kill(); let _ = child.wait(); } let _ = fs::remove_file(&self.api_socket_path); + self.vsock_listener = None; if let Some(ref config) = self.config { - let vsock_listener_path = format!( - "{}{}", - config.vsock_uds_path.to_string_lossy(), - VSOCK_PORT_SUFFIX - ); let _ = fs::remove_file(&config.vsock_uds_path); - let _ = fs::remove_file(&vsock_listener_path); + } + if let Some(ref path) = self.vsock_listener_path { + let _ = fs::remove_file(path); } } + /// Fill `buffer`, enforcing the wall-clock execution deadline. + /// + /// The deadline is checked on every iteration, not just when the socket + /// blocks: a guest that never stops writing keeps `read` returning data, so + /// checking only the WouldBlock path would let it run forever. fn read_exact_with_timeout( stream: &mut UnixStream, buffer: &mut [u8], @@ -403,6 +536,13 @@ impl FirecrackerOrchestrator { ) -> std::io::Result<()> { let mut read_total = 0usize; while read_total < buffer.len() { + if start_time.elapsed().as_millis() > (timeout_ms as u128) { + return Err(std::io::Error::new( + std::io::ErrorKind::TimedOut, + "Execution timed out", + )); + } + match stream.read(&mut buffer[read_total..]) { Ok(0) => { return Err(std::io::Error::new( @@ -417,16 +557,131 @@ impl FirecrackerOrchestrator { if e.kind() == std::io::ErrorKind::TimedOut || e.kind() == std::io::ErrorKind::WouldBlock => { - if start_time.elapsed().as_millis() > (timeout_ms as u128) { - return Err(std::io::Error::new( - std::io::ErrorKind::TimedOut, - "Execution timed out", - )); - } + // Deadline is re-checked at the top of the loop. } + Err(e) if e.kind() == std::io::ErrorKind::Interrupted => {} Err(e) => return Err(e), } } Ok(()) } } + +#[cfg(test)] +mod tests { + use super::*; + use std::collections::HashMap; + + fn config_with(name: &str, port: u32) -> VmConfig { + VmConfig { + service_name: name.to_string(), + kernel_path: PathBuf::from("/tmp/vmlinux"), + rootfs_path: PathBuf::from("/tmp/rootfs.ext4"), + service_disk_path: PathBuf::from("/tmp/service.ext4"), + runtime_disk_path: PathBuf::from("/tmp/runtime.ext4"), + memory_mb: 128, + vcpu_count: 1, + vsock_port: port, + vsock_uds_path: PathBuf::from(format!("/tmp/ignite-vsock-{}", name)), + env: HashMap::new(), + entrypoint: "index.ts".to_string(), + input: None, + runtime_args: vec![], + console_out: None, + timeout_ms: 5000, + } + } + + #[test] + fn api_socket_path_cannot_escape_tmp() { + // A malicious service.yaml name must not steer the socket path that + // boot() later unlinks. + let mut orch = FirecrackerOrchestrator::new(); + let mut config = config_with("evil", 1052); + config.service_name = "../../home/dev/.bashrc".to_string(); + config.vsock_uds_path = PathBuf::from("/tmp/../../etc/shadow"); + orch.configure(config).unwrap(); + + let path = orch.api_socket_path.to_string_lossy().into_owned(); + assert!( + path.starts_with(API_SOCKET_PREFIX), + "escaped prefix: {path}" + ); + assert!(!path.contains(".."), "path retains traversal: {path}"); + assert_eq!( + PathBuf::from(&path).parent().unwrap(), + Path::new("/tmp"), + "socket must stay directly in /tmp" + ); + } + + #[test] + fn vsock_listener_path_uses_configured_port() { + // Firecracker derives the listener name from the port; a hardcoded + // suffix would break any non-default --vsock-port. + let config = config_with("svc", 2000); + let path = FirecrackerOrchestrator::vsock_listener_path_for(&config); + assert!(path.to_string_lossy().ends_with("_2000")); + } + + #[test] + fn success_response_reads_the_status_line_only() { + assert!(is_success_response("HTTP/1.1 204 No Content\r\n\r\n")); + assert!(is_success_response("HTTP/1.1 200 OK\r\n\r\n")); + assert!(!is_success_response("HTTP/1.1 400 Bad Request\r\n\r\n")); + assert!(!is_success_response("HTTP/1.1 500 Internal\r\n\r\n")); + assert!(!is_success_response("")); + // A 4xx whose body happens to echo a 2xx must not read as success. + assert!(!is_success_response( + "HTTP/1.1 400 Bad Request\r\n\r\n{\"error\":\"HTTP/1.1 200 OK\"}" + )); + } + + #[test] + fn captured_output_is_bounded() { + let mut buf = String::new(); + let chunk = "x".repeat(1024 * 1024); + // Push well past the cap. + for _ in 0..16 { + append_capped(&mut buf, &chunk); + } + assert!( + buf.len() <= MAX_CAPTURED_OUTPUT_BYTES + 64, + "buffer grew to {} bytes", + buf.len() + ); + } + + #[test] + fn captured_output_is_marked_when_truncated() { + let mut buf = String::new(); + // A chunk that straddles the cap rather than landing on it exactly. + let chunk = "y".repeat(3 * 1024 * 1024); + for _ in 0..4 { + append_capped(&mut buf, &chunk); + } + assert!( + buf.contains("output truncated"), + "truncation was silent; callers cannot tell output was dropped" + ); + assert!(buf.len() <= MAX_CAPTURED_OUTPUT_BYTES + 64); + } + + #[test] + fn append_capped_preserves_short_output_exactly() { + let mut buf = String::new(); + append_capped(&mut buf, "hello "); + append_capped(&mut buf, "world"); + assert_eq!(buf, "hello world"); + } + + #[test] + fn append_capped_never_splits_a_utf8_char() { + let mut buf = "a".repeat(MAX_CAPTURED_OUTPUT_BYTES - 1); + // A 3-byte char cannot fit in the 1 remaining byte; it must be dropped + // rather than sliced into invalid UTF-8. Reaching this line at all + // proves no panic on a non-boundary slice. + append_capped(&mut buf, "→→→"); + assert!(buf.is_char_boundary(buf.len())); + } +} diff --git a/ignite-core/src/setup.rs b/ignite-core/src/setup.rs index 0e535b2..1d2b17c 100644 --- a/ignite-core/src/setup.rs +++ b/ignite-core/src/setup.rs @@ -1,11 +1,107 @@ use crate::disk::create_ext4_image; use ignite_shared::error::{IgniteError, Result}; use std::fs; +use std::os::unix::fs::PermissionsExt; use std::path::{Path, PathBuf}; use std::process::Command; +const ELF_MAGIC: &[u8; 4] = b"\x7fELF"; +const PT_INTERP: u32 = 3; +/// Program headers sit immediately after the ELF header in practice; this is far +/// more than enough to cover them. +const ELF_HEADER_SCAN_BYTES: usize = 64 * 1024; + +fn read_head(path: &Path, limit: usize) -> Result> { + use std::io::Read; + let file = fs::File::open(path)?; + let mut buf = Vec::new(); + file.take(limit as u64).read_to_end(&mut buf)?; + Ok(buf) +} + +fn is_elf(bytes: &[u8]) -> bool { + bytes.len() >= 4 && &bytes[..4] == ELF_MAGIC +} + +/// True when the ELF declares a PT_INTERP segment, i.e. it needs a dynamic +/// loader at runtime. Only little-endian ELF64 is understood; anything else +/// returns false and is handled by the caller's other checks. +fn elf_needs_dynamic_loader(bytes: &[u8]) -> bool { + if !is_elf(bytes) || bytes.len() < 64 || bytes[4] != 2 || bytes[5] != 1 { + return false; + } + let phoff = u64::from_le_bytes(bytes[32..40].try_into().unwrap()) as usize; + let phentsize = u16::from_le_bytes(bytes[54..56].try_into().unwrap()) as usize; + let phnum = u16::from_le_bytes(bytes[56..58].try_into().unwrap()) as usize; + if phentsize < 4 { + return false; + } + (0..phnum).any(|i| { + let off = phoff + i * phentsize; + off + 4 <= bytes.len() + && u32::from_le_bytes(bytes[off..off + 4].try_into().unwrap()) == PT_INTERP + }) +} + +/// Reject anything Firecracker cannot boot. Firecracker requires an +/// uncompressed ELF `vmlinux`; the `/boot/vmlinuz-*` files shipped by distros +/// are compressed bzImages and fail with an opaque error deep in the boot path. +pub fn validate_kernel_image(path: &Path) -> Result<()> { + let head = read_head(path, 4096).map_err(|e| IgniteError::Config { + message: format!("Cannot read kernel image at {:?}", path), + source: Some(Box::new(e)), + })?; + + if !is_elf(&head) { + return Err(IgniteError::Config { + message: format!( + "Kernel at {:?} is not an uncompressed ELF vmlinux. Firecracker cannot boot \ + compressed bzImage files such as /boot/vmlinuz-*. Build or download an \ + uncompressed vmlinux and point IGNITE_KERNEL_PATH at it, or extract one with \ + the kernel tree's scripts/extract-vmlinux.", + path + ), + source: None, + }); + } + Ok(()) +} + +/// The guest rootfs has no dynamic loader, so a glibc-linked agent cannot +/// execute as PID 1 — the kernel fails the exec and panics with +/// "No working init found". +fn validate_agent_is_static(path: &Path) -> Result<()> { + let head = read_head(path, ELF_HEADER_SCAN_BYTES)?; + if !is_elf(&head) { + return Err(IgniteError::Runtime { + message: format!("Guest agent at {:?} is not an ELF binary", path), + source: None, + }); + } + if elf_needs_dynamic_loader(&head) { + return Err(IgniteError::Runtime { + message: format!( + "Guest agent at {:?} is dynamically linked. The guest rootfs contains no libc \ + or dynamic loader, so it must be statically linked. Install the musl target \ + with `rustup target add {}` and rebuild.", + path, GUEST_AGENT_TARGET + ), + source: None, + }); + } + Ok(()) +} + const BUN_DOWNLOAD_URL_BASE: &str = "https://github.com/oven-sh/bun/releases/latest/download"; +/// The guest agent runs as PID 1 on a rootfs that contains no dynamic loader +/// and no libc, so it must be statically linked against musl. +const GUEST_AGENT_TARGET: &str = "x86_64-unknown-linux-musl"; + +/// Directories the guest agent mounts onto. The rootfs is attached read-only, +/// so the agent cannot `mkdir` these at runtime — they must exist in the image. +const ROOTFS_MOUNT_POINTS: &[&str] = &["app", "runtime", "dev", "proc", "sys"]; + #[derive(Debug)] pub struct SetupReport { pub ignite_dir: PathBuf, @@ -63,6 +159,7 @@ pub fn create_directories(ignite_dir: &Path) -> Result<()> { pub fn build_guest_agent(ignite_dir: &Path) -> Result { let agent_path = ignite_dir.join("guest-agent"); if agent_path.exists() { + validate_agent_is_static(&agent_path)?; return Ok(agent_path); } @@ -82,12 +179,15 @@ pub fn build_guest_agent(ignite_dir: &Path) -> Result { }); } + // Statically link against musl: the guest rootfs has no dynamic loader. let output = Command::new(&cargo_bin) .args([ "build", "--release", "--bin", "ignite-guest-agent", + "--target", + GUEST_AGENT_TARGET, "--manifest-path", ]) .arg(&manifest_path) @@ -99,20 +199,37 @@ pub fn build_guest_agent(ignite_dir: &Path) -> Result { if !output.status.success() { let stderr = String::from_utf8_lossy(&output.stderr); + let hint = if stderr.contains("target may not be installed") + || stderr.contains("can't find crate for `std`") + { + format!( + "\n\nThe musl target is missing. Install it with:\n rustup target add {}", + GUEST_AGENT_TARGET + ) + } else { + String::new() + }; return Err(IgniteError::Runtime { - message: format!("Guest agent build failed: {}", stderr), + message: format!("Guest agent build failed: {}{}", stderr, hint), source: None, }); } - let built_binary = workspace_root.join("target/release/ignite-guest-agent"); + let built_binary = workspace_root + .join("target") + .join(GUEST_AGENT_TARGET) + .join("release/ignite-guest-agent"); if !built_binary.exists() { return Err(IgniteError::Runtime { - message: "Guest agent binary not found after build".to_string(), + message: format!( + "Guest agent binary not found after build at {:?}", + built_binary + ), source: None, }); } + validate_agent_is_static(&built_binary)?; fs::copy(&built_binary, &agent_path)?; Ok(agent_path) } @@ -123,16 +240,22 @@ pub fn create_rootfs(ignite_dir: &Path, agent_path: &Path) -> Result { return Ok(rootfs_path); } + validate_agent_is_static(agent_path)?; + let temp_dir = tempfile::tempdir()?; let rootfs_dir = temp_dir.path().join("rootfs"); let sbin_dir = rootfs_dir.join("sbin"); fs::create_dir_all(&sbin_dir)?; - fs::copy(agent_path, sbin_dir.join("init"))?; - Command::new("chmod") - .arg("+x") - .arg(sbin_dir.join("init")) - .output()?; + // The rootfs is attached read-only, so every mount point the agent uses has + // to exist in the image up front — it cannot mkdir them at runtime. + for mount_point in ROOTFS_MOUNT_POINTS { + fs::create_dir_all(rootfs_dir.join(mount_point))?; + } + + let init_path = sbin_dir.join("init"); + fs::copy(agent_path, &init_path)?; + fs::set_permissions(&init_path, fs::Permissions::from_mode(0o755))?; create_ext4_image(&rootfs_dir, &rootfs_path)?; @@ -233,55 +356,73 @@ pub fn download_bun(runtimes_dir: &Path) -> Result { Ok(bin_dir.join("bun")) } +/// Locate an uncompressed ELF `vmlinux` on the host and stage it under the +/// ignite directory. +/// +/// Distros ship `/boot/vmlinuz-*`, which is a compressed bzImage that +/// Firecracker cannot boot, so those paths are deliberately not candidates — +/// copying one would produce an image that fails opaquely at boot. pub fn download_kernel(ignite_dir: &Path) -> Result { let kernel_path = ignite_dir.join("vmlinux"); if kernel_path.exists() { + validate_kernel_image(&kernel_path)?; return Ok(kernel_path); } - // Try to find and copy the host kernel - let uname_output = std::process::Command::new("uname").arg("-r").output(); + let uname_output = Command::new("uname").arg("-r").output(); let kernel_release = uname_output .ok() .map(|o| String::from_utf8_lossy(&o.stdout).trim().to_string()) .unwrap_or_default(); - let candidates: Vec = [ - "/boot/vmlinuz", - "/boot/vmlinuz-linux", - "/boot/vmlinuz-linux-lts", - ] - .iter() - .map(PathBuf::from) - .chain(if kernel_release.is_empty() { - None - } else { - Some(PathBuf::from(format!("/boot/vmlinuz-{}", kernel_release))) - }) - .chain(if kernel_release.is_empty() { - None - } else { - Some(PathBuf::from(format!( - "/usr/lib/modules/{}/vmlinuz", + let mut candidates = vec![PathBuf::from("/boot/vmlinux")]; + if !kernel_release.is_empty() { + candidates.push(PathBuf::from(format!("/boot/vmlinux-{}", kernel_release))); + candidates.push(PathBuf::from(format!( + "/usr/lib/modules/{}/vmlinux", kernel_release - ))) - }) - .chain([ - PathBuf::from("/boot/bzImage"), - PathBuf::from("/boot/kernel"), - ]) - .collect(); + ))); + candidates.push(PathBuf::from(format!( + "/usr/lib/debug/boot/vmlinux-{}", + kernel_release + ))); + } for path in &candidates { - if path.exists() { + if path.exists() && validate_kernel_image(path).is_ok() { fs::copy(path, &kernel_path)?; return Ok(kernel_path); } } + // Name the compressed images we did find so the error is actionable. + let compressed: Vec = [ + PathBuf::from("/boot/vmlinuz"), + PathBuf::from("/boot/vmlinuz-linux"), + PathBuf::from(format!("/boot/vmlinuz-{}", kernel_release)), + ] + .iter() + .filter(|p| p.exists()) + .map(|p| p.to_string_lossy().into_owned()) + .collect(); + + let detail = if compressed.is_empty() { + String::new() + } else { + format!( + "\n\nFound compressed kernel image(s) that Firecracker cannot boot: {}.\n\ + Extract an uncompressed vmlinux with the kernel tree's scripts/extract-vmlinux, \ + or download a prebuilt Firecracker kernel.", + compressed.join(", ") + ) + }; + Err(IgniteError::Config { - message: "No Linux kernel found in /boot. Set IGNITE_KERNEL_PATH to provide one." - .to_string(), + message: format!( + "No uncompressed ELF vmlinux found on this host. Provide one with \ + IGNITE_KERNEL_PATH or `ignite run --kernel `.{}", + detail + ), source: None, }) } diff --git a/ignite-guest-agent/src/main.rs b/ignite-guest-agent/src/main.rs index 7bc4c02..40a760b 100644 --- a/ignite-guest-agent/src/main.rs +++ b/ignite-guest-agent/src/main.rs @@ -17,6 +17,9 @@ const VSOCK_TYPE_STDOUT: u8 = 1; const VSOCK_TYPE_STDERR: u8 = 2; const VSOCK_TYPE_EXIT: u8 = 3; const IO_BUFFER_SIZE: usize = 4096; +/// Upper bound on the host's execution payload. The host is trusted, but a +/// length prefix is still a length prefix — bound the allocation. +const MAX_PAYLOAD_BYTES: usize = 8 * 1024 * 1024; #[derive(Deserialize, Debug)] struct ExecutionPayload { @@ -42,19 +45,24 @@ fn log_error(msg: &str) { } } -fn mount_device(source: &str, target: &str, fstype: &str) -> Result<(), std::io::Error> { +/// Mount a filesystem. `target` must already exist in the rootfs image: the +/// root is attached read-only, so the agent cannot create mount points itself. +fn mount_device( + source: &str, + target: &str, + fstype: &str, + flags: libc::c_ulong, +) -> Result<(), std::io::Error> { let src = CString::new(source)?; let tgt = CString::new(target)?; let fs = CString::new(fstype)?; - std::fs::create_dir_all(target)?; - let res = unsafe { libc::mount( src.as_ptr(), tgt.as_ptr(), fs.as_ptr(), - libc::MS_RDONLY, + flags, std::ptr::null(), ) }; @@ -123,22 +131,40 @@ fn connect_vsock_retry( fn main() { log_info("Starting Guest Agent init..."); - // 1. Mount read-only blocks + // 1. Mount pseudo-filesystems first. Without devtmpfs there is no /dev/vdb + // or /dev/console at all, so this has to come before anything else. + if let Err(e) = mount_device("devtmpfs", "/dev", "devtmpfs", 0) { + log_error(&format!("Failed to mount devtmpfs at /dev: {}", e)); + halt_vm(); + return; + } + if let Err(e) = mount_device("proc", "/proc", "proc", 0) { + // Not fatal: most runtimes work without /proc, but warn loudly. + log_error(&format!("Failed to mount proc at /proc: {}", e)); + } + if let Err(e) = mount_device("sysfs", "/sys", "sysfs", 0) { + log_error(&format!("Failed to mount sysfs at /sys: {}", e)); + } + + // 2. Mount read-only service and runtime block devices. Both are required + // for execution, so failure here is fatal rather than logged and ignored. log_info("Mounting service partition (/dev/vdb)..."); - if let Err(e) = mount_device("/dev/vdb", "/app", "ext4") { - log_error(&format!("Failed to mount /dev/vdb: {}", e)); - } else { - log_info("Successfully mounted /dev/vdb at /app"); + if let Err(e) = mount_device("/dev/vdb", "/app", "ext4", libc::MS_RDONLY) { + log_error(&format!("Failed to mount /dev/vdb at /app: {}", e)); + halt_vm(); + return; } + log_info("Successfully mounted /dev/vdb at /app"); log_info("Mounting runtime partition (/dev/vdc)..."); - if let Err(e) = mount_device("/dev/vdc", "/runtime", "ext4") { - log_error(&format!("Failed to mount /dev/vdc: {}", e)); - } else { - log_info("Successfully mounted /dev/vdc at /runtime"); + if let Err(e) = mount_device("/dev/vdc", "/runtime", "ext4", libc::MS_RDONLY) { + log_error(&format!("Failed to mount /dev/vdc at /runtime: {}", e)); + halt_vm(); + return; } + log_info("Successfully mounted /dev/vdc at /runtime"); - // 2. Connect to Host over VSOCK with retry (default port 1052) + // 3. Connect to Host over VSOCK with retry (default port 1052) log_info("Establishing connection to host..."); let mut stream = match connect_vsock_retry(VSOCK_PORT, VSOCK_RETRY_COUNT, VSOCK_RETRY_DELAY_MS) { @@ -152,7 +178,7 @@ fn main() { }; log_info("Connected to host."); - // 3. Read execution payload (length prefixed) + // 4. Read execution payload (length prefixed) log_info("Reading configuration payload..."); let mut len_bytes = [0u8; 4]; if let Err(e) = stream.read_exact(&mut len_bytes) { @@ -163,6 +189,15 @@ fn main() { let length = u32::from_be_bytes(len_bytes) as usize; log_info(&format!("Payload size: {} bytes", length)); + if length > MAX_PAYLOAD_BYTES { + log_error(&format!( + "Payload length {} exceeds maximum {}", + length, MAX_PAYLOAD_BYTES + )); + halt_vm(); + return; + } + let mut payload_bytes = vec![0u8; length]; if let Err(e) = stream.read_exact(&mut payload_bytes) { log_error(&format!("Failed to read payload body: {}", e)); @@ -179,7 +214,7 @@ fn main() { } }; - // 4. Launch runtime and run entrypoint + // 5. Launch runtime and run entrypoint log_info(&format!("Executing: {:?}", payload.runtime_args)); if payload.runtime_args.is_empty() { log_error("Invalid runtime command: empty args list"); @@ -192,7 +227,11 @@ fn main() { cmd.args(&payload.runtime_args[1..]); } - // Set environments + // Start from an empty environment so nothing from the init process leaks + // into untrusted code, then apply only what the host asked for. + cmd.env_clear(); + cmd.env("PATH", "/runtime/bin"); + cmd.env("HOME", "/app"); for (k, v) in &payload.env { cmd.env(k, v); } @@ -200,6 +239,7 @@ fn main() { if let Some(ref input) = payload.input { cmd.env("IGNITE_INPUT", input); } + cmd.current_dir("/app"); // Redirect outputs to pipe cmd.stdout(Stdio::piped()); diff --git a/ignite-http/src/main.rs b/ignite-http/src/main.rs index 72f55b2..73a6cac 100644 --- a/ignite-http/src/main.rs +++ b/ignite-http/src/main.rs @@ -24,6 +24,23 @@ async fn main() { let host = std::env::var("HOST").unwrap_or_else(|_| "127.0.0.1".to_string()); let api_key = std::env::var("IGNITE_API_KEY").ok(); + if api_key.is_none() { + tracing::warn!( + "IGNITE_API_KEY is not set: this server will execute services for any caller that \ + can reach it. Set IGNITE_API_KEY, or bind to localhost only." + ); + } + + // Comma-separated exact origins, e.g. "https://app.example.com". + let allowed_origins: Vec = std::env::var("IGNITE_CORS_ORIGINS") + .ok() + .map(|raw| { + raw.split(',') + .map(|s| s.trim().to_string()) + .filter(|s| !s.is_empty()) + .collect() + }) + .unwrap_or_default(); let state = Arc::new(ServerState { services_path, @@ -34,6 +51,7 @@ async fn main() { runtimes_root: std::env::var("IGNITE_RUNTIMES_ROOT") .ok() .map(PathBuf::from), + allowed_origins, }); let app = create_router(state); @@ -45,7 +63,11 @@ async fn main() { let listener = tokio::net::TcpListener::bind(socket_addr) .await .expect("Failed to bind TCP listener"); - axum::serve(listener, app) - .await - .expect("HTTP server failed"); + // ConnectInfo is required for per-client rate limiting. + axum::serve( + listener, + app.into_make_service_with_connect_info::(), + ) + .await + .expect("HTTP server failed"); } diff --git a/ignite-http/src/server.rs b/ignite-http/src/server.rs index 5be1f8f..4f0be16 100644 --- a/ignite-http/src/server.rs +++ b/ignite-http/src/server.rs @@ -1,6 +1,6 @@ use axum::{ Json, Router, - extract::{FromRef, Path as AxumPath, State}, + extract::{ConnectInfo, FromRef, Path as AxumPath, State}, http::StatusCode, response::IntoResponse, routing::{get, post}, @@ -8,13 +8,19 @@ use axum::{ use serde::{Deserialize, Serialize}; use std::collections::HashMap; use std::fs; +use std::net::SocketAddr; use std::path::PathBuf; use std::sync::{Arc, Mutex}; use std::time::{Duration, Instant}; -use tower_http::cors::CorsLayer; +use tower_http::cors::{AllowOrigin, CorsLayer}; use ignite_core::execution::{ExecuteOptions, execute_service}; use ignite_shared::types::ExecutionMetrics; +use ignite_shared::validation::constant_time_eq; + +/// Cap on distinct rate-limit buckets. Without a bound, one bucket per client +/// key is an unbounded memory footprint driven by request volume. +const MAX_RATE_LIMIT_BUCKETS: usize = 10_000; pub struct ServerState { pub services_path: PathBuf, @@ -23,6 +29,10 @@ pub struct ServerState { pub kernel_path: Option, pub rootfs_path: Option, pub runtimes_root: Option, + /// Exact origins permitted for browser access. Empty means no CORS layer is + /// installed at all, which is the right default for an endpoint that + /// executes code. + pub allowed_origins: Vec, } pub struct RateLimiter { @@ -40,13 +50,23 @@ impl RateLimiter { } } - fn check(&self, ip: String) -> bool { + fn check(&self, client: String) -> bool { let mut map = self.requests.lock().unwrap_or_else(|e| e.into_inner()); let now = Instant::now(); - let timestamps = map.entry(ip).or_default(); - timestamps.retain(|&t| now.duration_since(t) < self.window); + // Drop buckets that have fully aged out so the map cannot grow without + // bound across many distinct clients. + let window = self.window; + map.retain(|_, timestamps| { + timestamps.retain(|&t| now.duration_since(t) < window); + !timestamps.is_empty() + }); + + if map.len() >= MAX_RATE_LIMIT_BUCKETS && !map.contains_key(&client) { + return false; + } + let timestamps = map.entry(client).or_default(); if timestamps.len() >= self.max_requests { false } else { @@ -73,15 +93,17 @@ where ) -> Result { let server_state: Arc = FromRef::from_ref(state); - // 1. Check rate limit first - let client_ip = parts - .headers - .get("x-forwarded-for") - .and_then(|v| v.to_str().ok()) - .unwrap_or("unknown") - .to_string(); + // 1. Rate limit on the real transport peer. `x-forwarded-for` is + // client-controlled: keying on it lets any caller mint a fresh + // bucket per request, and collapses every direct client into one + // shared bucket when the header is absent. + let client_key = parts + .extensions + .get::>() + .map(|ConnectInfo(addr)| addr.ip().to_string()) + .unwrap_or_else(|| "unknown".to_string()); - if !server_state.rate_limiter.check(client_ip) { + if !server_state.rate_limiter.check(client_key) { return Err(( StatusCode::TOO_MANY_REQUESTS, Json(serde_json::json!({ "error": "Rate limit exceeded. Retry later." })), @@ -90,15 +112,13 @@ where // 2. Validate API key if configured if let Some(ref key) = server_state.api_key { - let auth = parts + let presented = parts .headers .get("Authorization") - .and_then(|v| v.to_str().ok()); + .and_then(|v| v.to_str().ok()) + .and_then(|t| t.strip_prefix("Bearer ")); - if auth - .filter(|t| t.starts_with("Bearer ") && &t[7..] == key) - .is_some() - { + if presented.is_some_and(|t| constant_time_eq(t, key)) { return Ok(RequireAuth); } return Err(( @@ -115,8 +135,7 @@ where async fn health() -> impl IntoResponse { Json(serde_json::json!({ "status": "ok", - "version": "0.1.0", - "uptime": 0 + "version": env!("CARGO_PKG_VERSION"), })) } @@ -167,7 +186,9 @@ async fn execute_service_handler( AxumPath(service_name): AxumPath, Json(body): Json, ) -> impl IntoResponse { - if service_name.contains('/') || service_name.contains('\\') || service_name.contains("..") { + // Reject anything that is not a plain service name outright, rather than + // blocklisting individual traversal spellings. + if !ignite_shared::validation::is_valid_service_name(&service_name) { return ( StatusCode::BAD_REQUEST, Json(ExecuteResponse { @@ -175,13 +196,15 @@ async fn execute_service_handler( service_name, metrics: None, preflight: None, - error: Some("Invalid service name: path traversal not allowed".to_string()), + error: Some( + "Invalid service name: must be lowercase alphanumeric with hyphens".to_string(), + ), }), ); } let service_dir = state.services_path.join(&service_name); - if !service_dir.exists() { + if !service_dir.is_dir() { return ( StatusCode::NOT_FOUND, Json(ExecuteResponse { @@ -198,16 +221,12 @@ async fn execute_service_handler( let options = ExecuteOptions { input: input_str, - env: HashMap::new(), skip_preflight: body.skip_preflight.unwrap_or(false), audit: body.audit.unwrap_or(false), - memory_override: None, - cpu_override: None, kernel_path: state.kernel_path.clone(), rootfs_path: state.rootfs_path.clone(), runtimes_root: state.runtimes_root.clone(), - vsock_port: None, - console_out: None, + ..ExecuteOptions::default() }; match execute_service(&service_dir, options, None, None) { @@ -235,18 +254,94 @@ async fn execute_service_handler( } pub fn create_router(state: Arc) -> Router { - Router::new() + let router = Router::new() .route("/health", get(health)) .route("/services", get(list_services)) .route( "/services/:serviceName/execute", post(execute_service_handler), - ) - .layer( + ); + + // Only install CORS when origins are explicitly configured. Defaulting to + // `Any` on an endpoint that executes code lets any web page a user visits + // drive this API from their browser. + let router = if state.allowed_origins.is_empty() { + router + } else { + let origins: Vec = state + .allowed_origins + .iter() + .filter_map(|o| o.parse().ok()) + .collect(); + router.layer( CorsLayer::new() - .allow_origin(tower_http::cors::Any) + .allow_origin(AllowOrigin::list(origins)) .allow_methods([axum::http::Method::GET, axum::http::Method::POST]) - .allow_headers(tower_http::cors::Any), + .allow_headers([ + axum::http::header::AUTHORIZATION, + axum::http::header::CONTENT_TYPE, + ]), ) - .with_state(state) + }; + + router.with_state(state) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn rate_limiter_allows_up_to_the_cap_then_blocks() { + let limiter = RateLimiter::new(3, 60); + for i in 0..3 { + assert!(limiter.check("1.2.3.4".into()), "request {i} should pass"); + } + assert!( + !limiter.check("1.2.3.4".into()), + "4th request must be denied" + ); + } + + #[test] + fn rate_limiter_buckets_are_per_client() { + let limiter = RateLimiter::new(2, 60); + assert!(limiter.check("1.1.1.1".into())); + assert!(limiter.check("1.1.1.1".into())); + assert!(!limiter.check("1.1.1.1".into())); + // A different client must not inherit the exhausted bucket. + assert!(limiter.check("2.2.2.2".into())); + } + + #[test] + fn rate_limiter_releases_bucket_after_window() { + // A zero-length window means every prior timestamp is already expired, + // so entries must be reclaimed rather than accumulating forever. + let limiter = RateLimiter::new(1, 0); + assert!(limiter.check("9.9.9.9".into())); + assert!(limiter.check("9.9.9.9".into())); + assert_eq!( + limiter.requests.lock().unwrap().len(), + 1, + "expired buckets should not accumulate" + ); + } + + #[test] + fn rate_limiter_bucket_count_is_bounded() { + let limiter = RateLimiter::new(5, 60); + for i in 0..(MAX_RATE_LIMIT_BUCKETS + 500) { + limiter.check(format!("10.0.{}.{}", i / 256, i % 256)); + } + assert!( + limiter.requests.lock().unwrap().len() <= MAX_RATE_LIMIT_BUCKETS, + "bucket map exceeded its bound" + ); + } + + #[test] + fn health_reports_the_real_crate_version() { + // Previously hardcoded to 0.1.0 while the crate was 0.9.0. + assert_eq!(env!("CARGO_PKG_VERSION"), "0.9.0"); + } } diff --git a/ignite-shared/src/lib.rs b/ignite-shared/src/lib.rs index 6a869c7..a637656 100644 --- a/ignite-shared/src/lib.rs +++ b/ignite-shared/src/lib.rs @@ -5,4 +5,4 @@ pub mod validation; // Re-export common dependencies pub use error::{IgniteError, Result}; pub use types::*; -pub use validation::validate_service_name; +pub use validation::{constant_time_eq, sanitize_path_segment, validate_service_name}; diff --git a/ignite-shared/src/validation.rs b/ignite-shared/src/validation.rs index c2f0af7..c1bc157 100644 --- a/ignite-shared/src/validation.rs +++ b/ignite-shared/src/validation.rs @@ -49,6 +49,66 @@ pub fn is_valid_service_name(name: &str) -> bool { validate_service_name(name).valid } +/// Reduce an arbitrary string to a single path segment safe for use in a +/// filename. Every character outside `[a-zA-Z0-9._-]` becomes `_`, `..` +/// sequences are collapsed, leading dots are stripped, and the result can never +/// be empty, `.`, or `..`. +/// +/// The result is guaranteed to contain no path separator and no `..`, so it +/// cannot traverse out of the directory it is joined onto. +/// +/// This is defense-in-depth only. Callers that build host paths from +/// user-controlled values should also reject invalid names outright via +/// [`validate_service_name`]. +pub fn sanitize_path_segment(value: &str) -> String { + let mut out: String = value + .chars() + .map(|c| { + if c.is_ascii_alphanumeric() || c == '-' || c == '_' || c == '.' { + c + } else { + '_' + } + }) + .collect(); + + // Collapse any `..` runs. Repeat, since removing one pass can create + // another (e.g. `....` -> `..`). + while out.contains("..") { + out = out.replace("..", "."); + } + + // A leading dot would allow hidden files. + while out.starts_with('.') { + out.remove(0); + } + + out.truncate(64); + + if out.is_empty() || out == "." { + "unnamed".to_string() + } else { + out + } +} + +/// Compare two secrets without leaking their contents through timing. +/// +/// Length is not secret here (it is fixed by the operator's configured key), so +/// an early length check is acceptable; the byte comparison itself does not +/// short-circuit. +pub fn constant_time_eq(a: &str, b: &str) -> bool { + let (a, b) = (a.as_bytes(), b.as_bytes()); + if a.len() != b.len() { + return false; + } + let mut diff = 0u8; + for (x, y) in a.iter().zip(b.iter()) { + diff |= x ^ y; + } + diff == 0 +} + #[cfg(test)] mod tests { use super::*; @@ -64,4 +124,60 @@ mod tests { assert!(!is_valid_service_name("hello/world")); assert!(!is_valid_service_name("hello..world")); } + + #[test] + fn sanitize_strips_traversal() { + assert_eq!(sanitize_path_segment("../../etc/passwd"), "_._etc_passwd"); + assert_eq!(sanitize_path_segment(".."), "unnamed"); + assert_eq!(sanitize_path_segment("."), "unnamed"); + assert_eq!(sanitize_path_segment(""), "unnamed"); + assert_eq!(sanitize_path_segment("/"), "_"); + assert_eq!(sanitize_path_segment("a/b\\c"), "a_b_c"); + assert_eq!(sanitize_path_segment(".hidden"), "hidden"); + } + + #[test] + fn sanitize_output_can_never_traverse() { + // The core guarantee: whatever goes in, the result is a single inert + // path segment. + for input in [ + "../../etc/passwd", + "....//....//x", + "..", + "....", + "a/../../b", + "\0/../x", + "..\\..\\windows", + ] { + let out = sanitize_path_segment(input); + assert!(!out.contains('/'), "{input:?} -> {out:?} contains a slash"); + assert!( + !out.contains('\\'), + "{input:?} -> {out:?} contains a backslash" + ); + assert!(!out.contains(".."), "{input:?} -> {out:?} contains .."); + assert!(!out.is_empty(), "{input:?} produced an empty segment"); + assert!(!out.starts_with('.'), "{input:?} -> {out:?} is hidden"); + } + } + + #[test] + fn sanitize_keeps_valid_names_intact() { + assert_eq!(sanitize_path_segment("hello-bun"), "hello-bun"); + assert_eq!(sanitize_path_segment("svc_1.2"), "svc_1.2"); + } + + #[test] + fn sanitize_is_bounded() { + assert_eq!(sanitize_path_segment(&"a".repeat(500)).len(), 64); + } + + #[test] + fn constant_time_eq_matches_semantics() { + assert!(constant_time_eq("secret", "secret")); + assert!(!constant_time_eq("secret", "secrez")); + assert!(!constant_time_eq("secret", "secret-longer")); + assert!(!constant_time_eq("", "x")); + assert!(constant_time_eq("", "")); + } } diff --git a/install.sh b/install.sh index 94ef4fb..17bf814 100755 --- a/install.sh +++ b/install.sh @@ -23,7 +23,7 @@ print_banner() { ╩╚═╝╝╚╝╩ ╩ ╚═╝ EOF echo -e "${NC}" - echo -e " ${DIM}Run JS/TS microservices in Docker${NC}" + echo -e " ${DIM}Run JS/TS microservices in secure microVMs${NC}" echo "" } From e810fdedffdcaefc7e19405fab17762998efb1e8 Mon Sep 17 00:00:00 2001 From: dev-dami <141376183+dev-dami@users.noreply.github.com> Date: Tue, 4 Aug 2026 00:42:34 +0100 Subject: [PATCH 2/2] ci: install musl target and assert the guest agent stays static MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The guest agent runs as PID 1 on a rootfs with no libc and no dynamic loader. If it regresses to dynamic linking the guest cannot boot at all, and nothing in the host-side test suite catches it — the failure only appears when a real VM fails to exec init. Build it for x86_64-unknown-linux-musl in CI and fail the job if the resulting ELF carries a PT_INTERP segment. Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/ci.yml | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index e3d09ab..868fb1d 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -29,6 +29,7 @@ jobs: with: toolchain: stable components: rustfmt, clippy + target: x86_64-unknown-linux-musl - name: Verify Formatting run: cargo fmt --all -- --check @@ -42,6 +43,21 @@ jobs: - name: Run Tests run: cargo test --all + # The guest agent runs as PID 1 on a rootfs with no libc and no dynamic + # loader. If it ever links dynamically the guest cannot boot at all, and + # no host-side test catches that — so assert it here. + - name: Verify guest agent links statically + run: | + cargo build --release --bin ignite-guest-agent \ + --target x86_64-unknown-linux-musl + AGENT=target/x86_64-unknown-linux-musl/release/ignite-guest-agent + file "$AGENT" + if readelf -l "$AGENT" | grep -qi 'interpreter'; then + echo "::error::Guest agent is dynamically linked. The guest rootfs has no dynamic loader, so it would fail to exec as init." + exit 1 + fi + echo "Guest agent is statically linked." + release: name: Build & Release Binaries runs-on: ubuntu-latest