Skip to content

split-runner.md

Latest commit

 

History

History
447 lines (338 loc) · 23.6 KB

split-runner.md

File metadata and controls

447 lines (338 loc) · 23.6 KB
Raw

Split mode: a ledger the agent cannot reach

On a single-user machine the ledger is a SQLite file owned by the account that runs tdd, which is the account the agent runs as. The agent cannot reach it by editing its worktree, but it can reach it by looking: sqlite3 ~/.local/share/tdd-cli/*.sqlite3 opens it, and nothing in the file says afterwards that anyone did. tdd doctor reports this as mode: single and says so in its ledger isolation check.

Split mode moves the ledger, and the process that observes the tests, to a second account:

  • A runner account owns the ledger, in a directory only it can open (mode 700).
  • The agent's tdd does nothing itself except docs and init. For every other verb it asks the runner, through sudo.
  • The runner runs every suite, gate, artifact hook and git command as the agent, in the agent's worktree, and reads the results from outside. The agent's code never runs as the account that owns the ledger.

The command surface is unchanged. No verb accepts a phase, a cycle number or an identity on either side of the boundary.

This is one machine's arrangement. Each machine has its own runner and its own ledger.

What it does and does not protect

It stops the agent's account from reading or writing the ledger, and from running code as the account that can.

It does not stop:

  • A forged test report. The suite runs as the agent, so a test can write whatever report it likes. The runner parses reports as data and never executes them, but it cannot know a passing report is honest. This is the same on a single-user machine.
  • A client-side downgrade. An agent can point its own tdd at a config of its own (TDD_RUNNER_CONFIG), or install a second copy of tdd-cli, and record a run in a ledger it owns. Nothing is forged by that: the run is simply absent from the runner's ledger, which is the only one to trust. Read runs from the runner's ledger.

Setup

You need root for this. Examples use tdd-runner for the runner account and agent1 for the agent's.

1. The runner account

Create a system account with a home directory and no login shell you hand out. On Linux: useradd --system --create-home --shell /usr/sbin/nologin tdd-runner. On macOS: sysadminctl -addUser tdd-runner -home /var/tdd-runner (then hide it if you wish).

2. Install tdd-cli where the agent cannot write

A runner whose code the agent can edit is the agent. Install for the runner as root:

python3 -m venv /opt/tdd-cli
/opt/tdd-cli/bin/pip install tdd-cli

It must be readable and executable by the agent's account: the runner runs small helpers from this install as the agent (python -m tddcli.agentfs, python -m tddcli.identity). It must not be writable by it.

The agent's own tdd can be this same binary or a separate install; both read the same config and become a client.

3. sudoers

Two lines, in a file under /etc/sudoers.d/ (edit with visudo -f):

agent1      ALL=(tdd-runner) NOPASSWD: /opt/tdd-cli/bin/tdd
tdd-runner  ALL=(agent1)     NOPASSWD:SETENV: ALL

The first lets the agent ask the runner, and only through that one binary. The second lets the runner act as the agent. Add one pair per agent account.

Four things to get right:

  • SETENV: is required on the second line. The runner passes the agent's own environment to the agent's commands with sudo -E; without the tag sudo answers "sorry, you are not allowed to preserve the environment" and tdd doctor fails its runs as the agent check. The agent's umask travels the same way: the runner sets it in each command it runs as the agent, after sudo, so the umask sudo's PAM session leaves (002 under pam_umask on Ubuntu) never reaches the suites.
  • NOPASSWD is required on both. tdd always passes -n, so a rule that would prompt fails at once instead of hanging an unattended agent.
  • Leave env_reset on (it is sudo's default). It is what strips the agent's TDD_LEDGER_HOME, TDD_RUNNER_CONFIG and PYTHONPATH on the way to the runner.
  • Nothing to configure for HOME or the working directory. The client passes -H, because macOS sudo would otherwise leave HOME pointing at the agent's and the runner's ledger home would follow it. sudo keeps the caller's working directory, which is how the runner knows which worktree it was asked about.

4. The config file

/etc/tdd-cli/runner.toml, owned by root, mode 644:

[runner]
user        = "tdd-runner"
command     = "/opt/tdd-cli/bin/tdd"   # the path the first sudoers line names
# sudo      = "/usr/bin/sudo"          # the default
# ledger_home = "/var/lib/tdd-cli"     # default: ~tdd-runner/.local/share/tdd-cli

[executor]                             # optional, see "Executor identity"
agent1 = "claude-fable-5-1"

A process whose account is user is the runner. Every other process on the machine is a client. With no file, the machine is single-user and nothing changes.

The file is trusted only if it is owned by root or by the account reading it, and is not group- or world-writable. Anything else is refused. That is what stops an agent handing the runner a config of its own: even if a sudoers rule let TDD_RUNNER_CONFIG through, the file it named would belong to the agent, and the runner refuses it.

5. Let the runner read the worktree

The runner reads the worktree as itself: tdd.toml, the test files it globs for, JUnit XML under build/. The agent's worktree, and the directories above it, must be readable and traversable by the runner account (chmod -R go+rX, or a shared group). On macOS that rules out ~/Desktop, ~/Documents and ~/Downloads, which are mode 700.

The runner never writes there as itself. Commits, the friction log, sensitivity-check restores and temporary probe worktrees are all made as the agent.

If a file is unreadable the runner says which, with reason: "worktree_unreadable".

Verify it

Run these as the agent's account, in a worktree:

  1. tdd doctor reports mode: "split", healthy: true, and these three checks ok:
    • runs as the agent: id -u, asked through sudo, answered with the agent's uid.
    • ledger out of the agent's reach: test -r <ledger>, as the agent, was refused.
    • install not writable by the agent: test -w <install>, as the agent, was refused.
  2. cat ~tdd-runner/.local/share/tdd-cli/*.sqlite3 (or your ledger_home) is denied.
  3. Start a run and close a cycle. git log -1 --format='%an %cn' shows the agent as author and committer, and ls -l on the committed files shows the agent as owner.
  4. sudo -u tdd-runner /opt/tdd-cli/bin/tdd status, run by hand from the agent's account, works; the same command naming any other binary is denied.

The project's own test suite cannot do this for you. It runs without root, so it uses a stand-in for sudo that does everything except change uid. The three doctor checks are the only place the uid change itself is verified, and they run against your real sudo.

Executor identity

tdd run start records which model did the work. Everything it reads on a single-user machine (the session id, the transcript, TDD_EXECUTOR_MODEL) is the agent's to write, so in split mode the runner trusts none of it:

  • If the calling account is in [executor], the run records that model with executor_source: "operator". One account per agent slot makes this exact.
  • Otherwise the ordinary resolution runs on the agent's side and the run records what it found with executor_source: "claimed". Leave claimed runs out of any comparison between models that has to be defended.

--executor, the human label, is a claim like any other in split mode.

Existing ledgers

A ledger written before the split was reachable by the agent the whole time. The runner can keep it, marked as such. From a root shell (so that sudo reports uid 0), or logged in as the runner with no sudo at all:

sudo -u tdd-runner /opt/tdd-cli/bin/tdd runner import /home/agent1/.local/share/tdd-cli/<repo>.sqlite3

The runner must be able to read that file. It copies the ledger into the runner's home under the same name, migrates it to the current schema, and records pre_split_import with the source path, the time, and last_run_id. tdd metrics reports that marker, so anyone comparing runs can see which predate the split.

It refuses when:

  • the machine is not split, or the caller is not the runner (not_split);
  • an agent's account is the caller (operator_only): an agent that could import could hand the runner a history it wrote itself;
  • the runner already holds a ledger of that name (ledger_exists). There is no merge. The runner's copy may hold runs recorded out of the agent's reach, and a second import would trade them for ones that were not.

Afterwards, remove the agent's copy so that nothing reads it by mistake.

Fleet, leases and other details

  • Worker leases move with the runner: they live under the runner's ~/.cache/tdd-cli, so one budget covers every agent account on the machine.
  • tdd fleet, tdd metrics and tdd log render are forwarded like everything else and read the runner's ledger.
  • A runner invoked directly, with no sudo in between, acts for nobody: every verb except runner import and ledger is refused with reason: "no_agent". It never falls back to running suites as itself.

Across a VM boundary

Everything above is one machine. Once agents run in disposable VMs or containers, the runner has to be in the guest, because that is where the suites run. Its ledger must not be: it would be deleted with the guest, two guests' files could never be merged, and an agent with root in the guest (a docker group membership is enough) could rewrite it whatever the file modes say.

So the runner in the guest keeps no ledger. It reads and writes the host's through a unix socket, served by a ledger service on the host. The service owns the SQLite files and is their only writer. The runner still runs every suite, gate and git command as the agent, in the agent's worktree, exactly as above.

host                                         guest (one per run)
┌──────────────────────────────┐             ┌──────────────────────────────┐
│ tdd ledger serve             │             │ agent ─sudo─▶ runner          │
│   admin.sock   (operator)    │             │                ledger_socket │
│   vm-1.sock ───forwarded──────────────────▶│ /run/tdd-cli/ledger.sock     │
│   <home>/<repo>.sqlite3      │             │ (no ledger file here)        │
└──────────────────────────────┘             └──────────────────────────────┘

A run's source is the socket it arrived on. The operator names it when they create the socket, and the protocol has no field in which a guest could name it instead.

The service

/etc/tdd-cli/ledger.toml (or TDD_LEDGER_SERVICE_CONFIG), owned by root or the service account and not group- or world-writable, the same trust rule as runner.toml:

[service]
admin_socket = "/run/tdd-cli/admin.sock"
# home       = "/var/lib/tdd-cli"     # default: the service account's ~/.local/share/tdd-cli

Run tdd ledger serve under a dedicated account that owns home. It runs until SIGTERM and exits 0. A systemd unit:

[Unit]
Description=tdd-cli ledger service

[Service]
User=tdd-ledger
Group=tdd-guests
RuntimeDirectory=tdd-cli
ExecStart=/opt/tdd-cli/bin/tdd ledger serve
KillSignal=SIGTERM

[Install]
WantedBy=multi-user.target

Host ledgers are <home>/<slug>.sqlite3, one per repository, where the slug is the guest's repository path with / replaced by -, as on a single machine. Every guest that checks a repository out at the same path shares that file, kept apart by source.

One source per guest

The operator adds a source for each guest, from the host:

tdd ledger add-source vm-1 --socket /run/tdd-cli/vm-1.sock --executor claude-fable-5-1
tdd ledger bind vm-1 --executor claude-opus-5-5     # the model for its next runs
tdd ledger unbind vm-1                               # its next runs record the guest's claim again
tdd ledger sources                                   # every source, socket and binding
tdd ledger remove-source vm-1                        # closes the socket; its runs stay

Sources are kept in <home>/sources.json, so a restarted service listens on all of them again. The admin socket is mode 600, the service account's alone. A guest's socket is mode 660: give its group to whatever forwards it into the guest, and nothing else.

tdd ledger verbs are always answered on the machine you run them on, never forwarded to a runner. With no service config they fail with reason: "no_service".

A bind-only socket for an orchestrator

The process that knows which model a guest runs is usually the one that starts it: a VM or container orchestrator under its own account. It needs bind and unbind, and nothing else the admin socket answers. Give it a bind socket:

[service]
admin_socket = "/run/tdd-cli/admin.sock"
bind_socket  = "/run/tdd-cli/bind.sock"
bind_group   = "guest-starters"
  • The two keys go together. A config with one and not the other is refused, and so is a bind_socket that is the admin_socket. A bind_group that is not a group on the host is refused when the config loads.
  • The socket is mode 660 and belongs to bind_group. The service account must be a member of that group: if it is not, tdd ledger serve fails to start. Put the orchestrator's account in the group, and nothing else. The admin socket stays mode 600.
  • It answers bind and unbind alone, for a source that already exists. Every other method, ping included, is refused with refusal: "method_not_allowed". An unknown source is unknown_source and an empty executor is bad_executor, as on the admin socket. It takes no identity from the request: its authority is its mode and group.
  • Which socket the CLI uses. tdd ledger bind and tdd ledger unbind take --socket PATH and use that socket alone. Without it they connect to admin_socket, and when that connect fails (no permission, no socket) and a bind_socket is configured, they send the request there instead. A request the admin socket took is never resent. Every other tdd ledger verb uses the admin socket only.
  • The orchestrator still reads ledger.toml. Every tdd ledger verb loads the service config first, held to the runner's trust rule: owned by root or by the reading account, not group- or world-writable. So the file must be owned by root and readable by the orchestrator's account. One owned by the service account is refused for every other account. --socket does not skip the config.
  • A binding belongs to the source, not to a run. A run abandoned after its source was rebound for the next run is stamped with the binding in force when it is abandoned.

Forwarding the socket into the guest

The forwarding itself is the operator's, and tdd-cli does not set it up:

  • ssh: ssh -R /run/tdd-cli/ledger.sock:/run/tdd-cli/vm-1.sock guest, with StreamLocalBindUnlink yes in the guest's sshd_config so a reconnect can replace a stale socket, and StreamLocalBindMask 0117 so the forwarded socket is not world-open.
  • containers: bind-mount the source's socket, -v /run/tdd-cli/vm-1.sock:/run/tdd-cli/ledger.sock. Mount the one socket, never the host's ledger directory.
  • vsock: forward the source's socket to a vsock port with a relay such as socat, and relay it back to a unix socket in the guest.

The guest

The guest's runner.toml names the forwarded socket instead of a ledger home:

[runner]
user          = "tdd-runner"
command       = "/opt/tdd-cli/bin/tdd"
ledger_socket = "/run/tdd-cli/ledger.sock"   # instead of ledger_home
# core_budget = 4   # caps this guest's worker lease; defaults to the CPUs the runner may use

Naming both ledger_socket and ledger_home is refused. In the guest, tdd doctor's ledger reachable check pings the service and names the source it reached: socket /run/tdd-cli/ledger.sock (source vm-1).

It fails closed. If the socket is missing or the service hangs up, every verb that needs the ledger fails with reason: "ledger_unreachable". Nothing in the guest ever falls back to a ledger of its own.

What it does and does not protect

The service enforces this for every request, so a modified client cannot get around it:

  • A guest reads and writes only its own source's runs. A run, cycle, plan contract or suite invocation of another source is refused with refusal: "foreign_run", and so is one that does not exist, so a guest cannot probe for other sources' ids. Claims and the baseline cache are per source: one guest can neither block another's run start at the same path nor hand it a forged cached baseline.
  • A closed run stays closed. Writes to a complete or abandoned run are refused with refusal: "run_closed". tdd note is the one exception, so a closing narrative can still be added. A blocked run is not closed: resume --unblock still works.
  • A guest reaches only named domain methods. The wire carries a method name and its arguments, never SQL. Methods meant for the host alone (set_meta, the generic row helpers) are refused with refusal: "method_not_allowed", and a repository path that is not absolute is refused with refusal: "bad_repo".

Refusals reach the guest's envelope as reason: "ledger_refused" with the refusal code.

What a compromised guest can still do is record runs under its own source, and lie in them, which is no worse than a forged test report on a single machine.

Executor identity is set on the host. A guest's [executor] table is guest root's to edit, so it is a claim. When the operator binds an executor to a source, the service records that model with executor_source: "operator" on every run the source starts or abandons, whatever the guest sent. With no binding, a guest that says operator is recorded as claimed. tdd ledger unbind <source> clears a binding, so the source's next runs record the guest's claim again; bind refuses an empty or blank executor with reason: "bad_executor".

One history per host

On the host, the operator reads every source together through the admin socket:

tdd ledger metrics [--repo <slug>] [--since <time>]   # `tdd metrics` over every source, per repository
tdd ledger fleet                                      # active runs of every source, and the host's suites
tdd ledger log --repo <slug> --run <id> [--out FILE]

Every run in them carries its source. Inside a guest, tdd metrics, tdd fleet, tdd log render and tdd invocation show still work, and see only that guest's source.

--since is for a poller: it keeps only the runs whose figures can have changed since its last poll, those still open and those that ended at or after the given time. A resumed run is open again, so it comes back. The time is ISO 8601 with a UTC offset (2026-09-01T10:20:00+00:00, 2026-09-01T10:20:00Z, 2026-09-01T11:20:00+01:00); one with no offset, a bare date or anything else is refused with reason: "bad_since", never read as local time. Every run in metrics, here and in tdd metrics, carries time.cycles: each cycle's ordinal, opened_at, closed_at, wall_clock_s, suite_s and suite runs, with closed_at and wall_clock_s null while the cycle is open.

A read-only report socket for dashboards

A dashboard that charts cycle and suite time across the host needs those three reads and nothing else. The admin socket would also give it add-source, remove-source, import and bind. Give it a report socket instead:

[service]
admin_socket  = "/run/tdd-cli/admin.sock"
report_socket = "/var/lib/tdd-cli/report/report.sock"

With no report_socket there is no report socket, and nothing changes. It must not be admin_socket's path: the service refuses that config, because the report listener would replace the admin socket.

  • It answers metrics, fleet and log only, with the same results the admin socket gives. Every other request, ping included, is refused with method_not_allowed.
  • It opens every ledger read-only (SQLite's mode=ro), so no request on it can write, nor migrate an old ledger.
  • It is mode 660, like a source's, with the group a new file gets. The unit's Group= is the guests' group, so the mode alone does not keep it to one account.

Whose it is. It is the dashboard's account's, and nobody else's. Put it in a directory of its own that only the service account and that account can traverse, and admit the account with an inherited (default) ACL on the directory, so the socket the service recreates on every start gets it too:

install -d -o tdd-ledger -g tdd-ledger -m 700 /var/lib/tdd-cli/report
setfacl -m u:orchestrator:rx,d:u:orchestrator:rw /var/lib/tdd-cli/report

tdd ledger metrics, fleet and log fall back to the report socket when the admin socket cannot be opened, so the dashboard's account runs the same commands as the operator. Only a failed connect falls back; every other tdd ledger verb still needs the admin socket. That account must be able to read /etc/tdd-cli/ledger.toml, which stays owned by root or the service account.

A client that speaks the wire itself sends one JSON line per request, and reads one back:

{"method": "metrics", "args": [<slug or null>], "kwargs": {"since": <time>}}
{"method": "fleet"}
{"method": "log", "args": [<slug>, <run id>]}

Worker leases are the host's. A socket-mode runner takes each suite's worker lease from the service, which splits the host's cores across every guest's running suites. A lease ends when its connection closes, which is also what a dead VM does, and one older than the stale limit is not counted. Named leases stay in the guest: they guard resources the guest's suites run against.

The guest caps that share at its own budget: a suite gets min(the service's share, the guest's budget) workers. The guest's budget is core_budget in its runner.toml's [runner], otherwise the CPUs the runner may run on (len(os.sched_getaffinity(0))), or os.cpu_count() where the platform has no affinity. So a 6-core, 4 GB guest on a 10-core host no longer runs -n 10. core_budget replaces the affinity rather than adding to it, and it only ever lowers the share. Runner.toml is root's, so the operator sets it and the agent cannot; TDD_CORE_BUDGET is never read in a guest. A core_budget that is not a positive integer is refused. Only a runner with ledger_socket reads it: a runner that keeps its own ledger splits its own cores as before. tdd fleet still shows the host's split.

Existing ledgers

tdd runner import is refused on a socket-mode runner (reason: "ledger_remote"): it has no ledger to import into. Bring an existing ledger in on the host instead, as one source:

tdd ledger import /home/agent1/.local/share/tdd-cli/<repo>.sqlite3 --source legacy

It copies the file into the service's home under the same name, migrates it, retags every run and plan contract it recorded to the named source, drops its claims, and marks it as pre_split_import, which tdd ledger metrics reports. It refuses a file the host already holds (reason: "ledger_exists"). There is no merge.

Ledgers from before schema v13 gain a source on their first open: every existing run, contract, claim and cached baseline becomes local, the source of single-user mode and of a split runner on one machine.