Skip to content

feat(linux): run the firecracker provider without sudo - #11

Merged
celrenheit merged 1 commit into
mainfrom
feat/linux-zero-sudo
Aug 1, 2026
Merged

feat(linux): run the firecracker provider without sudo#11
celrenheit merged 1 commit into
mainfrom
feat/linux-zero-sudo

Conversation

@celrenheit

@celrenheit celrenheit commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Booting a sandbox on Linux took privilege twice per boot, and #9 was right that both asks were too large:

  • Creating the bridge and TAPs shelled out to sudo ip, where a NOPASSWD entry hands over the host's entire network configuration to save one password prompt.
  • Injecting the worktree loop-mounted the rootfs through sudo clawk __loop-mount, a root helper that passed its argv straight to syscall.Mount — the widest privilege in the tree, since anyone who could run it could mount any image anywhere.

Neither is necessary.

Networking

The kernel hands out CAP_NET_ADMIN for free inside a user namespace you own. The daemon forks an anchor into CLONE_NEWUSER|CLONE_NEWNET with a single-uid mapping (no /etc/subuid, no setuid helper — clawk wants capabilities over its own network, never over other users' files), builds br0 + tap0 + gv0 there with netlink, and passes the gvproxy-side TAP fd back over SCM_RIGHTS.

gvproxy stays in the host namespace, where its egress sockets belong; only the VM's NIC moves, entered via nsenter because setns into a netns also demands CAP_SYS_ADMIN in the owning userns. Such a boot performs no privileged operation at all and leaves the host's interface list empty — and the VM loses its view of host networking, which is a real isolation gain. The anchor blocks on a pipe the daemon holds, so the namespace and every device in it dies with the VM: nothing to leak, nothing to clean up after a crash.

Worktree

It becomes its own ext4 disk, built in userspace by the same writer that already builds every rootfs, and clawk-init mounts it from /dev/vdc via an additive Block/FSType transport on manifest mounts. The guest still sees /workspace/<name>, so nothing downstream changes, and the loop-mount helper is deleted outright.

Fallback

Hosts that forbid unprivileged user namespaces (Ubuntu 24.04+ via AppArmor, or a zeroed sysctl) fall back to bridge mode. The CLI provisions those devices in the foreground, while it still has a terminal to authenticate on — the daemon never has one — so sudo prompts at most once per sandbox and never per boot. The mode is decided once and pinned for the daemon, with the reason travelling alongside it so the daemon log names the real cause. clawk doctor reports which mode a host will use and the setting to change; CLAWK_NET_MODE pins either.

Also here

  • One L2 segment and guest MAC per sandbox, so concurrent VMs sharing gvproxy's 192.168.127.2 stop colliding.
  • Every host device name scoped by uid, so two users' identically-named sandboxes cannot touch each other's devices.
  • The in-guest binaries shipped prebuilt, so an installed clawk boots a sandbox with no Go toolchain present.
  • The vm ( kernel … ) override honoured instead of silently dropped.
  • A Linux quick start, and CI publishing linux/amd64 + arm64 binaries.

Upgrading

Every host device name now carries the invoking uid, TAPs included. Sandboxes created by an earlier clawk have differently-named devices, so the first clawk up after upgrading reports the expected device as missing — clawk down && clawk up re-provisions it. The old devices are no longer named by anything and outlive clawk destroy; remove them with sudo ip link del clawk<hash> (listed by ip link | grep clawk).

A worktree disk that fails to mount now fails the boot rather than coming up with an empty workspace and reporting success.

Testing

Both modules build, vet and test clean on linux/arm64 and linux/amd64. Beyond unit tests, this was exercised on a bare-metal KVM host (Ubuntu 24.04, x86_64, firecracker v1.12), each behaviour change A/B'd against a build of the parent commit:

  • Rootless and bridge boots, snapshot/resume, worktree ownership, egress filtering, concurrent sandboxes.
  • A privilege matrix across four users — no sudo at all, NOPASSWD for ip only, and password-sudo — covering doctor's OK/WARN/FAIL verdicts and the sudo-classification paths that only misbehave on an ip-only sudoers.
  • Two users with identically-named sandboxes, which previously collided on one TAP.
  • A user with no sudo rights whatsoever boots rootless: no clawk devices on the host, one /dev/net/tun fd in the daemon, working egress from the guest.

macOS is validated by this PR's CI run, not locally — the vz provider is cgo + Virtualization.framework and cannot be cross-compiled, so the macos-14 job is the first real compile of this branch on Darwin. Worth a look before merging.

Closes #9

Booting a sandbox on Linux took privilege twice per boot, and issue #9 was
right that both asks were too large. Creating the bridge and TAPs shelled out
to `sudo ip`, where a NOPASSWD entry hands over the host's entire network
configuration to save one password prompt. Injecting the worktree loop-mounted
the rootfs through `sudo clawk __loop-mount`, a root helper that passed its
argv straight to syscall.Mount — the widest privilege in the tree, since
anyone who could run it could mount any image anywhere.

Neither is necessary.

Networking: the kernel hands out CAP_NET_ADMIN for free inside a user
namespace you own. The daemon forks an anchor into CLONE_NEWUSER|CLONE_NEWNET
with a single-uid mapping (no /etc/subuid, no setuid helper — clawk wants
capabilities over its own network, never over other users' files), builds
br0 + tap0 + gv0 there with netlink, and passes the gvproxy-side TAP fd back
over SCM_RIGHTS. gvproxy stays in the host namespace, where its egress sockets
belong; only the VM's NIC moves, entered via nsenter because setns into a
netns also demands CAP_SYS_ADMIN in the owning userns. Such a boot performs no
privileged operation at all and leaves the host's interface list empty, and
the VM loses its view of host networking — a real isolation gain. The anchor
blocks on a pipe the daemon holds, so the namespace and every device in it
dies with the VM: nothing to leak, nothing to clean up after a crash.

Worktree: it becomes its own ext4 disk, built in userspace by the same writer
that already builds every rootfs, and clawk-init mounts it from /dev/vdc via
an additive Block/FSType transport on manifest mounts. The guest still sees
/workspace/<name>, so nothing downstream changes, and the loop-mount helper is
deleted outright.

Hosts that forbid unprivileged user namespaces (Ubuntu 24.04+ via AppArmor, or
a zeroed sysctl) fall back to bridge mode. The CLI provisions those devices in
the foreground, while it still has a terminal to authenticate on — the daemon
never has one — so sudo prompts at most once per sandbox and never per boot.
The mode is decided once and pinned for the daemon, with the reason travelling
alongside it so the daemon log names the real cause. doctor reports which mode
a host will use and the setting to change; CLAWK_NET_MODE pins either.

Also here: one L2 segment and guest MAC per sandbox, so concurrent VMs sharing
gvproxy's 192.168.127.2 stop colliding; every host device name scoped by uid,
so two users' identically-named sandboxes cannot touch each other's devices;
the in-guest binaries shipped prebuilt, so an installed clawk boots a sandbox
with no Go toolchain present; the `vm ( kernel … )` override honoured instead
of silently dropped; a Linux quick start; and CI publishing linux/amd64 and
arm64 binaries.

Verified on an x86_64 KVM host: rootless and bridge boots, snapshot/resume,
worktree ownership, egress filtering, concurrent sandboxes, and a privilege
matrix across four users — no sudo, NOPASSWD for ip only, and password sudo.
A user with no sudo rights at all boots rootless with no host devices.
@celrenheit
celrenheit merged commit 1bf2a49 into main Aug 1, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Linux firecracker: sudo password requirement blocks first boot

1 participant