From 4490b8ab6514dbdad1a176a5b9796baea954adfe Mon Sep 17 00:00:00 2001 From: makseq Date: Tue, 11 Aug 2026 01:16:22 +0300 Subject: [PATCH] docs: the setup block is two commands, and the audit it needed is gone MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit OPERATIONS.md still told an operator to make the agent's state directory world-writable (`chmod 1777`) and then handed them forty lines of shell to work out whether that was exploitable on their machine. Neither is what the orchestrator prints any more. The agent now starts with `--user "$(id -u):$(id -g)"`, so it runs as the person who pasted the command rather than as the service account its image declares. Its state directory can therefore be `0700` and owned by them, nothing on the machine is shared with anybody, and the audit has nothing left to audit: every branch of it existed to make a world-writable parent safe, and it had to GUESS which uid the agent would run as (`a=10001`, printed before any agent existed) — a guess an operator can neither confirm nor act on. Its `UNSAFE` and `INCOMPLETE` verdicts went with it, so the paragraph that explained them is gone too. `--group-add` is unaffected, and the bullet now says why: it grants a supplementary GROUP, which is independent of the uid, so overriding the user does not take the socket away. A reader who does not know that assumes the two flags fight. What `--user` does not fix is stated rather than checked. `0700` protects the directory, not its NAME: an account that can write a directory ABOVE `$HOME/lspo-agent` can rename it away and leave its own under that name, the running agent never notices because its mount stays on the original inode, and the docker daemon resolves the replacement on the host and as root when it mounts a job's credentials. No checker for that ships anywhere — not in the printed block, not in the agent's log. Several shipped and were withdrawn, all wrong in the same family of ways, because a short check cannot soundly resolve a chain of symbolic links: canonicalising a path is exactly what discards the directory that HOLDS each link. The two sentences the orchestrator prints in their place are reproduced here word for word, so an operator reads the same thing in both places. The labels move with the facts. The `mkdir` line is BEHAVIOUR — skip it and the daemon creates the mount source as root, and the agent exits at startup with "Operation not permitted" — rather than the RECOMMENDATION it used to carry; and the gap is BEHAVIOUR for what nothing checks plus a RECOMMENDATION for what to do about it, never a RULE, since nothing refuses anything. Taken from `noderegistry/services.py::docker_run_command` at 7dcf95c5 rather than retyped: both the command and the two sentences were compared against that commit mechanically. `python -m pytest` is 128 passed, 1 skipped — the expected skip. Not touched, because it needs its own change: everything here that describes a job's credentials directory as `0700` with a `0600` file and tells a node author to build their image as uid 10001. The platform replaced that the day before (`0444` file inside a `0711` directory, re-applied on every write, precisely so an image running as its own user can read its own credentials), and this repository mentions neither mode. It spans PROTOCOL.md, AUTHORING.md, the uid-coupling section of OPERATIONS.md, the Dockerfile comment, `conformance/contract.py` and one platform test, so correcting half of it here would leave a document contradicting itself. Run with `LSPO_ORCHESTRATOR_SRC` set and the verbatim-citation check names it: `test_platform_rules.py:277` quotes a sentence `agent/creds.py` no longer contains. Co-Authored-By: Claude Fable 5 --- docs/OPERATIONS.md | 94 +++++++++++++++++++++------------------------- 1 file changed, 43 insertions(+), 51 deletions(-) diff --git a/docs/OPERATIONS.md b/docs/OPERATIONS.md index ec622c2..b5cef4d 100644 --- a/docs/OPERATIONS.md +++ b/docs/OPERATIONS.md @@ -149,9 +149,10 @@ prefer either of those, because they carry the real pool name and, when that reg minted one, the real token. The general form: ```bash -mkdir -p "$HOME/lspo-agent" && chmod 1777 "$HOME/lspo-agent" +mkdir -p "$HOME/lspo-agent" && chmod 700 "$HOME/lspo-agent" docker run -d --name lspo-agent \ + --user "$(id -u):$(id -g)" \ -v /var/run/docker.sock:/var/run/docker.sock \ --group-add $(getent group docker | cut -d: -f3) \ -v "$HOME/lspo-agent:$HOME/lspo-agent" \ @@ -168,55 +169,44 @@ Three of those values are placeholders and the printed command has them filled i orchestrator's address, the pool name, and the token — see the registration section above for when the reply carries a token and when it does not. -**RECOMMENDATION, and it is the other half of a safety check the agent cannot make.** The -first line above is a prerequisite rather than decoration: the agent runs as a non-root -user and creates its own state directory inside the one you make, which requires owning -the parent. Skip it and the container exits at startup with "Operation not permitted", -which names nothing anybody can act on. - -The agent checks the state directory's permissions from **inside** its own container, -where it can prove almost nothing about the host: the directories above the mount are its -own image, and even for the mount it cannot show that the name it reads is the name the -docker daemon resolves. The block below runs that check where the answer is. **It only -reads; it changes nothing.** Anything it prints as `UNSAFE` is a directory another account -on that machine could use to replace the agent's state — the directory the daemon hands -your jobs their credentials out of — so fix those before starting the agent. On a fresh -machine it ends `INCOMPLETE`, because the state directory does not exist until the agent -has started once; run it again afterwards. It has to be told which uid the agent runs as, -and before there is an agent that can only be the uid the image declares (10001); the agent -prints this same block in its own startup log with the uid it really has. - -```bash -t="$HOME/lspo-agent/state"; u=$(id -u); a=10001; g=1; bad=; here=; leaned= -lean() { if [ "$1" = "$a" ] && [ "$1" != "$u" ] && [ "$1" != 0 ]; then leaned=1; fi; } -seen() { m=${1%% *}; r=${1#* }; o=${r%% *}; k=${r##* }; } -now=$(stat -c "%a %u %f" "$t") || now=; seen "$now"; why= -if [ -n "$now" ]; then here=1 - lean "$o" - if [ "$o" != "$u" ] && [ "$o" != 0 ] && [ "$o" != "$a" ]; then leaned=1; why="the docker daemon hands your jobs their credentials out of this directory, and whoever owns it decides what is in it -- uid $o is neither you, nor root, nor the uid this block was told the agent runs as ($a)" - elif [ $((0$m&022)) -ne 0 ]; then why="any account on this host can write into the directory the daemon hands your jobs their credentials out of, so any account can leave a symlink there for the daemon to follow AS ROOT -- chmod g-w,o-w '$t' (a symbolic link in its place reports mode 777 here, and is the same finding)" - fi - if [ -n "$why" ]; then bad=1; echo "UNSAFE $t (mode $m, owner uid $o): $why"; fi -fi -p="$t" -while [ "$p" != / ]; do d=$(dirname "$p"); why= - now=$(stat -c "%a %u %f" "$p") || now=; seen "$now"; e=$o; el= - if [ -n "$k" ] && [ $((0x$k&0xf000)) -eq $((0xa000)) ]; then el=1; fi - now=$(stat -Lc "%a %u %f" "$d") || now=; seen "$now" - lean "$o" - if [ -n "$el" ] && [ $((0$m&022)) -ne 0 ] && [ $((0$m&01000)) -ne 0 ]; then lean "$e"; fi - if [ "$o" != "$u" ] && [ "$o" != 0 ] && [ "$o" != "$a" ]; then leaned=1; why="its owner (uid $o) may rename $p away whatever the mode says, and a sticky bit does not restrain a directory's OWNER -- that is neither you, nor root, nor the uid this block was told the agent runs as ($a); chmod cannot fix it, move the agent's state directory out from under it" - elif [ $((0$m&022)) -ne 0 ] && [ $((0$m&01000)) -eq 0 ]; then why="any account on this host can rename $p away -- chmod g-w,o-w '$d', or chmod +t '$d' if it must stay shared" - elif [ $((0$m&022)) -ne 0 ] && [ -n "$e" ] && [ "$e" != "$u" ] && [ "$e" != 0 ] && [ "$e" != "$a" ]; then leaned=1; why="its sticky bit stops everyone but the owner of $p, and that entry belongs to uid $e -- neither you, nor root, nor the uid this block was told the agent runs as ($a); chown it to yourself, or move the agent's state directory elsewhere" - fi - if [ -n "$why" ]; then bad=1; echo "UNSAFE $d (mode $m, owner uid $o): $why"; fi - p="$d" -done -if [ -n "$bad" ]; then echo "fix every UNSAFE directory above -- each line says how; until then another account here can put its own directory in the agent's place"; fi -if [ -n "$leaned" ] && [ -n "$g" ]; then echo "ASSUMED this block was TOLD the agent runs as uid $a and did not check it, and at least one directory above was judged on that -- ACCEPTED because it belongs to $a, or reported because it does not. If the number is wrong (a --user override, an image of your own, version skew) then an acceptance here is worth nothing and a report here is a mismatch rather than an intruder. Set a= at the top and run this again, or use the copy the agent prints in its own startup log, which carries the uid it really has."; fi -if [ -z "$here" ]; then echo "INCOMPLETE $t does not exist yet, so the directory the daemon will hand your jobs their credentials out of was NOT checked -- and a sticky bit protects the entries in a directory, not the vacant names, so any account that can write to a shared parent can create it before the agent does. Run this block again after the agent's first start." -elif [ -z "$bad" ]; then echo "OK: nothing but you, root or uid $a can replace $t or any directory on the way to it"; fi -``` +**BEHAVIOUR.** The first line is a prerequisite rather than decoration, and skipping it does +not fail where the mistake is. `docker run` needs the mount source to exist, and when it does +not, the daemon creates it — **as root**. The agent then cannot create its own state directory +inside it, which it has to do itself because it makes that directory owner-only and a `chmod` +needs ownership rather than write permission, and the container exits at startup with +"Operation not permitted", which names nothing anybody can act on. + +**BEHAVIOUR.** `--user "$(id -u):$(id -g)"` runs the agent as the account that pasted the +command rather than as the service account its image declares, and that flag is what makes +`chmod 700` enough. Everything the agent keeps on the machine — its identity file, every +running job's credentials — then belongs to that account, and nothing here is shared with any +other account on the host. Earlier releases had no such flag: the agent ran as a uid that +belongs to nobody on the host, so the parent directory had to be world-writable (`chmod 1777`) +before the agent could create anything inside it, and a world-writable directory then had to be +policed, which is what the long host-side audit that used to be printed here existed for. +Running as the operator removes the world-writable directory, and the audit went with it. + +**BEHAVIOUR, and it is what `--user` does not fix.** `0700` protects the directory; it does +not protect its **name**. If any directory above `$HOME/lspo-agent` can be written by other +accounts — a home directory at `0775` with a shared group, which some sites ship — a member of +that group can rename `lspo-agent` away and leave their own directory under the same name. The +running agent never notices: its bind mount stays attached to the original inode, so its +startup checks and every recheck go on passing, while the docker daemon resolves the +replacement **on the host and as root** every time it mounts a job's credentials. Nothing +checks that, and nothing prints a checker for you to run — not the block above, not the agent's +own startup log. One shipped for several releases and was withdrawn: every version of it was +wrong in the same family of ways, because a short check cannot soundly resolve a chain of +symbolic links — resolving a path to its canonical form is exactly what discards the directory +that *holds* each link — and the platform's position is that a checker it cannot vouch for is +worse than none (`agent/README.md`, "Run it"). + +**BEHAVIOUR for the first sentence below, RECOMMENDATION for the second.** They are the two +the orchestrator prints with the block, word for word: + +> On a machine other accounts use, nothing here checks who can replace the agent's state +> directory or anything on the path to it — including the directories holding any symbolic +> links in that path. Satisfy yourself of that before you start, and prefer a machine you do +> not share. **BEHAVIOUR, for all four bullets below.** Four parts of that command are load-bearing, and each one fails in its own way when it is wrong (`agent/README.md`, "Run it"). None of them @@ -225,7 +215,9 @@ what visibly breaks: * **`--group-add`** with the host's docker group id. The agent image runs as a non-root user and has no access to the docker socket without it. The agent refuses to start - rather than failing on its first job. + rather than failing on its first job. It grants a supplementary **group**, and a group is + independent of the uid, so `--user` above does not take the socket away — the two flags do + not fight, they are answering different questions. * **The state bind mount, with the host path and the container path spelled the same**, and the workdir pointing at a **child** of it. This is the one that silently destroys jobs when it is wrong. The agent asks the host's docker daemon to bind-mount each job's