Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,27 @@ jobs:
go vet ./...
go test ./...

# The release path, which the tests above can't reach: a plain checkout
# embeds no guest binaries, so the prebuilt tests skip themselves. Run
# the generator, then assert a sandbox can be prepared with no Go
# toolchain — the whole point of shipping them prebuilt.
- name: prebuilt guest binaries — generate + use without a toolchain
run: |
make guestbin
go test ./internal/guestbuild/ -run 'TestBuildUsesPrebuilt|TestPrebuiltRejectsStaleSources' -v

# Cross-build both release artifacts, so a tag can't be the first time we
# discover the Linux release doesn't compile.
- name: cross-build linux release artifacts
run: |
for arch in amd64 arm64; do
make clean-guestbin
make guestbin GUEST_ARCH="$arch"
CGO_ENABLED=0 GOOS=linux GOARCH="$arch" \
go build -trimpath -o "/tmp/clawk-linux-$arch" ./cmd/clawk
ls -l "/tmp/clawk-linux-$arch"
done

macos:
runs-on: macos-14 # Apple Silicon — required for the vz provider
steps:
Expand Down
77 changes: 73 additions & 4 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -1,8 +1,15 @@
name: release

# Cut a release by pushing a tag: `git tag v0.1.0 && git push origin v0.1.0`.
# Builds the macOS/arm64 binary, publishes it as a GitHub release, and (if
# HOMEBREW_TAP_TOKEN is set) bumps the Homebrew tap formula to match.
# Builds the macOS/arm64 and Linux (amd64 + arm64) binaries, publishes them as
# one GitHub release, and (if HOMEBREW_TAP_TOKEN is set) bumps the Homebrew tap
# formula to the macOS tarball.
#
# Every artifact runs `make guestbin` first, embedding the in-guest binaries so
# an installed clawk needs no Go toolchain to boot a sandbox. GUEST_ARCH is the
# artifact's OWN architecture: hardware virtualization can't cross
# architectures, so a linux/amd64 clawk only ever boots amd64 guests and needs
# exactly one set.
on:
push:
tags:
Expand All @@ -12,9 +19,69 @@ permissions:
contents: write

jobs:
# Create the release once, before anything uploads into it.
#
# The three artifact builds below (two Linux matrix legs plus macOS) stay
# deliberately independent of each other — a failure in one shouldn't
# withhold the others' binaries — but they cannot each CREATE the release: on
# a tag with no release yet they race to POST /releases, and the loser fails
# with 422 already_exists, dropping an architecture's tarball from a release
# that built fine. Doing it here also pins the generated notes to one
# deterministic run instead of whichever job happened to win.
create-release:
runs-on: ubuntu-latest
steps:
- uses: softprops/action-gh-release@v2
with:
generate_release_notes: true

# The Linux artifacts the firecracker provider needs. Pure Go — no cgo, no
# signing — so both architectures cross-compile from one runner.
linux:
needs: create-release
runs-on: ubuntu-latest
strategy:
matrix:
goarch: [amd64, arm64]
steps:
- uses: actions/checkout@v4

- uses: actions/setup-go@v5
with:
go-version: "1.26"

- name: Prebuild guest binaries (linux/${{ matrix.goarch }})
run: make guestbin GUEST_ARCH=${{ matrix.goarch }}

- name: Build clawk (linux/${{ matrix.goarch }})
env:
CGO_ENABLED: "0"
GOOS: linux
GOARCH: ${{ matrix.goarch }}
run: go build -trimpath -o clawk ./cmd/clawk

- name: Package tarball
id: pkg
run: |
tarball="clawk-${GITHUB_REF_NAME}-linux-${{ matrix.goarch }}.tar.gz"
# No clawk.entitlements: codesigning is macOS-only.
tar czf "$tarball" clawk LICENSE
sha=$(sha256sum "$tarball" | awk '{print $1}')
echo "tarball=$tarball" >> "$GITHUB_OUTPUT"
{
echo "### clawk ${GITHUB_REF_NAME} (linux/${{ matrix.goarch }})"
echo "\`sha256: ${sha}\`"
} >> "$GITHUB_STEP_SUMMARY"

- name: Upload release asset
uses: softprops/action-gh-release@v2
with:
files: ${{ steps.pkg.outputs.tarball }}

release:
# Virtualization.framework is cgo/Objective-C: the binary MUST be built
# on macOS arm64 — it cannot be cross-compiled from Linux.
needs: create-release
runs-on: macos-14
env:
TAP_TOKEN: ${{ secrets.HOMEBREW_TAP_TOKEN }}
Expand All @@ -25,6 +92,9 @@ jobs:
with:
go-version: "1.26"

- name: Prebuild guest binaries (linux/arm64)
run: make guestbin GUEST_ARCH=arm64

- name: Build clawk (darwin/arm64)
run: go build -trimpath -o clawk ./cmd/clawk

Expand All @@ -44,11 +114,10 @@ jobs:
echo "\`sha256: ${sha}\`"
} >> "$GITHUB_STEP_SUMMARY"

- name: Publish GitHub release
- name: Upload release asset
uses: softprops/action-gh-release@v2
with:
files: ${{ steps.pkg.outputs.tarball }}
generate_release_notes: true

# Rewrites version/url/sha256 in the tap's Formula/clawk.rb and pushes.
# Skipped automatically when HOMEBREW_TAP_TOKEN isn't configured — in
Expand Down
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -22,3 +22,8 @@ debug-logs/
# Local Claude Code settings and skills (personal, not shared)
.claude/settings.local.json
.claude/skills/

# Working notes: implementation plans, scratch write-ups. Kept out of the
# repo because they age into fiction the moment the code lands — the design
# rationale that matters belongs in the code and in docs/.
.plans/
5 changes: 3 additions & 2 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,8 +105,9 @@ because it pins a vendored `gvisor-tap-vsock` fork; everything clawk-specific
|--|----|-------------|
| Host | macOS (Apple silicon) | Linux |
| Hypervisor | Virtualization.framework (cgo, Code-Hex/vz) | `firecracker` + `/dev/kvm` |
| NIC | file-handle NIC speaking gvproxy's datagram protocol | host TAP, bridged to gvproxy by the frame pump |
| Worktree | virtio-fs live mount (`/home/agent/workspace`) | baked into the rootfs at create (`/workspace`); host edits don't propagate |
| NIC | file-handle NIC speaking gvproxy's datagram protocol | TAP, bridged to gvproxy by the frame pump |
| Network devices | none on the host | bridge + 2 TAPs, in a per-sandbox unprivileged user+netns (rootless, default) or on the host via sudo (fallback) |
| Worktree | virtio-fs live mount (`/home/agent/workspace`) | its own ext4 disk built in userspace, mounted at `/workspace`; host edits don't propagate |

The split is `//go:build`-gated and total: a Linux checkout can't compile the vz
code and vice versa. Most of clawk is shared; only the hypervisor glue is
Expand Down
70 changes: 70 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,35 @@ tagged.

### Added

- **Linux firecracker sandboxes boot without sudo.** Each sandbox's network
now lives in its own unprivileged user + network namespace, where clawk has
`CAP_NET_ADMIN` over its own bridge and TAPs without asking the host for
anything — so a boot performs no privileged operation, and no clawk
interfaces appear on the host at all. gvproxy stays in the host namespace
(that is where its egress sockets belong); only the VM's NIC moves, which
also means the VM no longer sees host networking.

Hosts that forbid unprivileged user namespaces fall back to the previous
sudo-based bridge mode. `clawk doctor` reports which mode is active under
`host: network mode` and names the setting to change (on Ubuntu 24.04+,
AppArmor's `kernel.apparmor_restrict_unprivileged_userns`).
`CLAWK_NET_MODE=rootless|bridge` pins one.

- **Release binaries no longer need a Go toolchain.** Booting a sandbox
compiled the three in-guest binaries on the host first, which made `go` a
hard prerequisite on every platform — `clawk doctor` failed without it — and
put a module download in the first-boot path. Release artifacts now embed
those binaries (`make guestbin`), so an installed clawk just unpacks them.
Building from source still compiles them on first boot, which is what a
contributor editing the guest sources wants; `doctor` reports the toolchain as
required only in that case. Costs ~7 MiB of binary size per artifact.

- **Linux release binaries.** `linux/amd64` and `linux/arm64` tarballs are built
and published alongside the macOS one, so the firecracker provider no longer
requires building from source. Each artifact embeds the in-guest binaries for
its own architecture — guest arch always equals host arch, since hardware
virtualization can't cross architectures.

- **`env ( … )` gains aliases, defaults, and literals.** Entries now use
shell / docker-compose parameter-expansion syntax: `NAME = ${HOST}` aliases
a differently-named host variable, `${HOST:-default}` / `${HOST-default}`
Expand All @@ -17,8 +46,49 @@ tagged.
or quoted right-hand side (`EDITOR = vim`) sets a literal constant. Bare
`NAME` passthrough is unchanged.

### Changed

- **The worktree rides in on its own disk instead of being copied into the
rootfs.** Staging it used to loop-mount the rootfs as root — six privileged
operations per boot, plus a `sudo clawk __loop-mount` helper that mounted
whatever it was told. It is now built as a separate ext4 disk in userspace
and mounted by the guest from `/dev/vdc`. Same semantics (host edits still
don't propagate into a running guest); no privilege, and the root mount
helper is gone.

### Fixed

- **Two firecracker sandboxes can run at once.** Every guest NIC used the same
hardcoded MAC and every TAP hung off one shared host bridge, so two guests
claimed one L2 identity on one segment — the second sandbox logged an IPv6
duplicate-address error and the bridge's forwarding table decided which
guest received what. Guest MACs are now derived per VM and each sandbox gets
its own bridge (keyed on uid, so users don't share segments either).

**Upgrading:** every host device name now carries the invoking uid, TAPs
included. Sandboxes created by an earlier clawk have differently-named
devices, so the first `clawk up` after upgrading reports the expected device
as missing — `clawk down && clawk up` re-provisions it. The old devices are
no longer named by anything and outlive `clawk destroy`; remove them with
`sudo ip link del clawk<hash>` (they are listed by `ip link | grep clawk`).
- **A worktree disk that fails to mount now fails the boot.** The guest logged
the error to its console and carried on, so a sandbox whose disk was
unreadable came up with an empty directory where the repo should be,
answered the agent, and reported success — an agent could then run a whole
session against nothing. Boots that would have been silently empty now stop
with `clawk-init: FATAL: mount /dev/vdc …` on the console.
- **First-boot failures on Linux say what went wrong.** The network setup ran
inside the detached VM daemon, which has no terminal — so on a host where
sudo needs a password it could never succeed, and its error went to a log
the CLI then deleted during rollback, leaving only "agent did not become
ready". Setup moved to the CLI (one prompt, in front of you), `clawk doctor`
gained a check for it, and boot failures now quote the daemon log.
([#9](https://github.com/clawkwork/clawk/issues/9))
- **A snapshot no longer risks a corrupted filesystem on restore.** `clawk up`
re-materialized the rootfs on every boot, including boots that were about to
restore a suspend state onto it — pairing a saved memory image with a disk
that had just been reset. Disks are left alone when a restorable state is
present.
- **Forwarded `env ( … )` vars now reach the agent user.** The generated
`/etc/profile.d/99-clawk-env.sh` was written `0600 root:root`, so the
agent's login shells silently skipped it and the variables never arrived.
Expand Down
24 changes: 22 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ BIN ?= ./bin/clawk
ENTITLEMENTS := clawk.entitlements
UNAME_S := $(shell uname -s)

.PHONY: install build sign test clean
.PHONY: install build sign test clean guestbin clean-guestbin

install:
go install ./cmd/clawk
Expand All @@ -47,5 +47,25 @@ endif
test:
go test ./...

clean:
# Cross-compile the in-guest binaries into internal/agentembed/prebuilt/ so the
# next `go build` embeds them and the resulting clawk needs no Go toolchain to
# boot a sandbox. Release builds run this first; source builds skip it and
# compile the guest binaries on first boot instead.
#
# GUEST_ARCH must match the architecture the artifact will RUN on: hardware
# virtualization can't cross architectures, so an arm64 clawk only boots arm64
# guests. Defaults to this machine's.
GUEST_ARCH ?= $(shell go env GOARCH)

guestbin:
go run ./internal/guestbuild/cmd/gen-prebuilt -arch $(GUEST_ARCH)

# Drop the embedded payload, restoring a plain source build.
clean-guestbin:
rm -f $(PREBUILT_DIR)/clawk-init $(PREBUILT_DIR)/clawk-pty-agent \
$(PREBUILT_DIR)/clawk-time-sync $(PREBUILT_DIR)/manifest.json

PREBUILT_DIR := internal/agentembed/prebuilt

clean: clean-guestbin
rm -rf ./bin
12 changes: 7 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,9 +122,9 @@ not a sandboxed process."
## Install

Requires macOS 14+ on Apple silicon. (Linux is supported via firecracker and
currently experimental; see
[VM providers](docs/commands.md#vm-providers) for the gaps. This README is
macOS-first.)
currently experimental — start with
**[docs/linux-quickstart.md](docs/linux-quickstart.md)**, which covers setup,
the workflow, and the gaps. This README is macOS-first.)

```sh
brew install clawkwork/tap/clawk
Expand All @@ -138,8 +138,10 @@ make install
```

Either way there's no extra host tooling: no Docker, no qemu, no sudo. The
hypervisor is Apple's Virtualization.framework, linked into the binary. First
run probes for anything missing and offers to fix it.
hypervisor is Apple's Virtualization.framework, linked into the binary, and the
release binaries carry the in-guest agent prebuilt — so a Go toolchain is only
needed if you build from source, in which case you have one. First run probes
for anything missing and offers to fix it.

**Uninstall:** `clawk destroy` your sandboxes, `rm -rf ~/.clawk`, then remove
the binary with `brew uninstall clawk` (or delete it from `$GOBIN` for a
Expand Down
7 changes: 4 additions & 3 deletions cmd/clawk/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,10 @@ import (
)

func main() {
// Privileged mount/umount re-exec path — runs before cobra because these
// hidden subcommands exist only for `sudo $self __loop-mount/-unmount`.
sandbox.InitRootHelpers()
// Network-namespace helper re-execs (rootless mode) — before cobra,
// because these are the clawk binary acting as its own helper, not CLI
// verbs, and they must not touch the config store or flag parsing.
sandbox.InitNetNSHelpers()
err := cli.Execute()
if err == nil {
return
Expand Down
56 changes: 48 additions & 8 deletions docs/commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -131,14 +131,18 @@ without notice.
| Provider | Host | Notes |
|--------------------|-------|-----------------------------------------------------------------------|
| `vz` (default) | macOS | Apple Virtualization.framework; no sudo. Live-mounts your worktree. |
| `firecracker` (experimental) | Linux | KVM microVM. Bakes the worktree into the rootfs (host edits don't propagate live), and skips host-file push, ssh-agent forwarding, and per-phase hooks today. |
| `firecracker` (experimental) | Linux | KVM microVM; no sudo on hosts that allow unprivileged user namespaces. Carries the worktree on its own disk (host edits don't propagate live), and skips host-file push, ssh-agent forwarding, and per-phase hooks today. |

Pick one with `--provider`; the choice persists with the sandbox. Both run the
same OCI rootfs, vsock agent, and egress allow-list — see
[ARCHITECTURE.md](../ARCHITECTURE.md) for how they differ under the hood.

### Linux prerequisites (firecracker)

> New to clawk on Linux? **[linux-quickstart.md](linux-quickstart.md)** walks
> the whole path — setup, first run, the workflow, and what isn't there yet.
> This section is the reference detail.

The firecracker provider needs three things on the host; `clawk doctor`
checks all of them:

Expand All @@ -153,10 +157,46 @@ checks all of them:
Without this, boots fail with firecracker's opaque
`Permission denied (os error 13) ... /dev/kvm file's ACL`; `clawk up`
now surfaces the kvm-group fix directly.
- **A Go toolchain** — used to cross-compile the tiny in-guest binaries on
first boot. Any `go` ≥ 1.21 works (older toolchains auto-download the one
the guest modules pin via `GOTOOLCHAIN=auto`).

`clawk` shells out to `sudo` for the host bridge/TAP plumbing (`ip link`,
`ip tuntap`) and to stage the worktree into the rootfs at create — never to
run the VM itself, which firecracker does unprivileged against `/dev/kvm`.
- **`nsenter`** (from `util-linux`, already installed on mainstream distros) —
used by rootless mode to launch the VM inside its network namespace. Missing
it isn't fatal: clawk falls back to bridge mode and says so.

A **Go toolchain** is not among them. A release binary carries the tiny
in-guest binaries prebuilt, and `clawk doctor` says so
(`host: go toolchain — not needed`). It's a prerequisite only for a clawk built
from source, which cross-compiles them on first boot; any `go` ≥ 1.21 works
there (older toolchains auto-download the one the guest modules pin via
`GOTOOLCHAIN=auto`).

#### Privileges: rootless by default, sudo only as a fallback

Creating a VM's network devices (a bridge and two TAPs) needs
`CAP_NET_ADMIN`. clawk gets it without sudo by running each sandbox's network
in its own **unprivileged user + network namespace**: inside a namespace you
own, you are root over your own network. Nothing else needs privilege —
firecracker itself runs unprivileged against `/dev/kvm`, and the worktree disk
is built in userspace. So on a host that allows unprivileged user namespaces,
a sandbox boots with **no privileged operation at all**, and no clawk
interfaces appear on the host.

`clawk doctor` reports which mode is in use under `host: network mode`:

- **`rootless`** — the default. Nothing to configure; the namespace (and every
device in it) is created and destroyed with the VM.
- **`bridge mode via sudo`** — the fallback for hosts that forbid unprivileged
user namespaces. Ubuntu 24.04+ is the common case: it blocks them via
AppArmor. Getting rootless mode there takes one root action, once:
`sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0` (persist it
in `/etc/sysctl.d/` if you want it to survive a reboot) — verified. The
narrower alternative is an AppArmor profile for the clawk binary granting
`userns,`, which keeps the restriction on for everything else; that route is
documented by Ubuntu but has not been tested here. In bridge mode clawk
creates host devices with
`sudo ip`: it prompts at most **once per sandbox** (never per boot — later
boots find the devices configured and touch no privilege), and it prompts in
the foreground rather than inside the background VM daemon, which has no
terminal to authenticate on. `NOPASSWD` for `ip` avoids the prompt entirely,
at the cost of granting network reconfiguration.

Pin a mode with `CLAWK_NET_MODE=rootless|bridge` — useful to assert that no
sudo will ever be attempted (`rootless` fails loudly instead of falling back).
Loading
Loading