Hardening: close the residues recorded through layer 8 - #184
Open
EiffL wants to merge 9 commits into
Open
Conversation
… uv.toml The ambient half (#179): child_env() drops every UV_* variable outside a closed plumbing allowlist — cache location, timeouts, TLS, air-gap mode, index credentials, uv's own recursion guard — so an exported UV_NO_BINARY or UV_PYTHON can no longer steer what a sync installs while env_version reports nothing moved. The run verbs (materialize, lc run) name any non-empty variable the scrub dropped, from the same predicate, so the report cannot disagree with the scrub. The config-file half (#176, the advisory option): uv merges user- and system-level uv.toml underneath the project's settings, and list settings concatenate across levels — so the scan now checks the two documented paths per platform for audited install-settings keys and reports a hit beside sdist_built. Reported, never hashed: machine state in env_version would make one commit answer differently on two hosts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016CRERrB5zWdVXD6uPv2BRa
…the OCI wrap The containerized backend was the one mechanism restricting the network, which made "containerized" carry a promise the direct mechanisms never made — and the direction of the recorded decision is symmetry: nothing pretends to a control it does not apply. The wrap now emits no --network flag, the attestation says `allowed` like landlock and seatbelt, and the `denied` literal stays reserved for a mechanism that genuinely emits a denial flag. This also retires the Perlmutter spike item about `--network none` hanging on compute nodes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016CRERrB5zWdVXD6uPv2BRa
Closes the cross-write residue: a concurrent task landing bytes in a sibling's output directory before the sibling hashed produced a manifest that was self-consistent and wrong — undetectable by any checksum, so prevention is the only fix. exec_policy gains one keyword (output_dir), handed down from the worker's task; a probe has no output id and keeps results/ whole, which is now the one probe/recipe asymmetry: the probe→recipe promise excludes exactly the commands that write outside their own output directory. All three mechanisms express the narrower shape natively (the same nested-writable-directory form, one level deeper), and the OCI mount table inherits it from the write set with no backend change. Integrity-answering stays data_version's job alone. The enforcement test is mutation-checked: the same cross-write through Unavailable() succeeds (verified rc 0, bytes replaced). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016CRERrB5zWdVXD6uPv2BRa
The engine-closure decision's concrete loss was that nothing recorded which uv converged the environment — the one tool between the lock and the installed artifacts. The driver probes `uv --version` once per run and hands it down (the HEAD discipline; the rerun entry point probes its own), and the worker records it beside lc_version. Attestation, never identity: defaulted empty, outside both hashes, never read by classify, and an unparseable probe records "" rather than failing a run. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016CRERrB5zWdVXD6uPv2BRa
… as sha256 Every file in an output directory now appears in the crate as a File under its dataset's hasPart, carrying sha256 and contentSize parsed from its SHA256E annex key — the raw digest `sha256sum` can verify after a `git archive` deposit, available with none of the bytes fetched because keys are repository state (dataset.annex_keys; `--include=*` is load-bearing — bare `find` lists only present files). A non-SHA-256 backend key yields size and no digest, never a wrong one; git-carried files (the lock, universes, manifests) hash their own bytes. The key map is injected into render like the writer, so the builder stays git-free and the render pure. This also fixes a latent honesty bug: an out-of-tree declared input's recorded input_versions digest is lc's *framed* hash, and publishing it under the workflow-run `sha256` term claimed a checksum nothing could verify. Externals now publish no digest — the manifests keep the full story, the layer's stated weaker promise. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016CRERrB5zWdVXD6uPv2BRa
The rerun-lag residue made visible. The render pins datePublished to the newest manifest finished_at, so status can read lag off the document itself — a content comparison against the manifests the walk already read, with no git call and no rocrate import (the crate stays the one materialize-only dependency on status's path; license_of and CRATE_FILENAME move to project.py for the same reason). Four states: not maintained / will be created / up to date / behind, plus an honest "unreadable" for a corrupted file — and the line lands in --json through as_dict like the rest of the header. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016CRERrB5zWdVXD6uPv2BRa
The mid-run edit hole, surfaced: the dirty check runs at start of run while manifests are written per-output later, so an edit in between left manifests whose git_sha no longer described the code that ran, silently. The run now ends with one dataset.status call — the tree started clean and save/restore keeps results/ clean, so any dirt appeared mid-run — and warns with the edited paths. Still a warning, never a manifest field: the spec's git_dirty stays unwritten by decision. Also closes the recorded review item: a metadata test now pins that no [project.scripts] entry ever targets lightcone.engine or the sandbox shim — every entry point is the CLI or a mirrored git-annex executable. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016CRERrB5zWdVXD6uPv2BRa
The uv identity holes (scrub landed, machine-config advisory), the crate's raw-digest rule and the injected key map, the status crate line, and the launcher-scrub note resolved. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016CRERrB5zWdVXD6uPv2BRa
The sweep: dataset.save now commits with the same pathspec it stages —
a partial commit, built from HEAD plus its own paths alone — so work the
user staged while a graph ran stays staged and warned-about, never swept
into an output or crate commit. The nothing-to-commit probe is scoped
the same way, and the annex per-add config rides on the commit too,
since a partial commit takes the paths through the clean filter again.
The stronger move — a frozen execution worktree — is recorded as
deferred to the venue era, not rejected.
The crate's byte fallback re-checks the annex pointer shape before
hashing: annex_keys answers empty for the whole repository when
git-annex cannot answer at all, and a pointer file reads perfectly well,
so one failed `git annex find` would otherwise publish a well-formed
digest of the pointer text for every output file. The key parse splits
from the last tab (git-annex emits ${file} unescaped; keys never contain
tabs), and _file stops hashing the license file twice.
The scrub allowlist keeps four more plumbing variables —
UV_PYTHON_INSTALL_DIR (the interpreter store has no project-level
spelling, so scrubbing it shipped a remedy that does not exist),
UV_LINK_MODE, UV_PYTHON_INSTALL_MIRROR, UV_KEYRING_PROVIDER — and the
warning is composed once (project.uv_scrub_warning) and surfaced by
every uv-acting verb: init and build now say it too, instead of leaving
a corporate-mirror user with uv's raw resolution error. The machine
config probe covers XDG_CONFIG_DIRS, keeping its "complete" claim true.
Honesty edges: the crate status line says "up to date with the outputs"
/ "behind the outputs" — the exact scope of its datePublished proxy,
correct in both directions (a dropped output regresses the newest stamp
as a rerun advances it); the write-denial remedy names the recipe's own
output directory instead of results/; the Landlock/Seatbelt divergence
the narrowing exposed (`rm -rf $OUT` is EACCES on Linux, fine on macOS)
is recorded with its fix — delete the redundant prelude, the worker
resets the directory; the deposit-vs-consumed meaning of a File's sha256
is recorded where the old conflict rule used to be; and a comment that
referenced the spec stands on its own now.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016CRERrB5zWdVXD6uPv2BRa
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The rebuild is functionally complete through layer 8; this PR closes the
recorded residues that do not need Perlmutter access, one commit per
decision. Each choice below was settled explicitly before implementation.
What lands
UV_*scrub (closes Scrub the ambient UV_* namespace before invoking uv #179):project.child_envdrops everyUV_*variable outside a closed plumbing allowlist (cache dir,timeouts, TLS, air-gap mode, index credentials, uv's recursion guard),
so an exported
UV_NO_BINARYorUV_PYTHONcan no longer steer a syncwhile
env_versionreports nothing moved. The run verbs warn with thenames of any non-empty variable dropped, from the same predicate.
uv's two documented user/system config paths for audited
install-settings keys — key presence, not file presence, because list
settings concatenate across levels — and reports a hit beside
sdist_built. Never hashed.--network none, its one restriction, so all three mechanisms attestnetwork: allowedsymmetrically and no consumer can read a promiseinto "containerized".
deniedstays in the type for a mechanism thatgenuinely emits a denial flag. Retires the Perlmutter spike item about
--network nonehanging on compute nodes.the recorded "not narrowed" decision): closes the cross-write residue —
a concurrent task landing bytes in a sibling's directory before the
sibling hashed produced a manifest that was self-consistent and wrong,
undetectable by any checksum, so prevention is the only fix. A probe
keeps
results/whole (no output id) — the one probe/recipe asymmetry,and it excludes exactly the accident being prevented. The new
enforcement test is mutation-checked through
Unavailable().uv_versionmanifest attestation: the engine-closure decision'sconcrete loss, recorded — probed once per run by the driver and handed
down (the HEAD discipline), beside
lc_version. Attestation, neveridentity.
a
Fileunder its dataset'shasPartwithsha256/contentSizeparsed from its SHA256E key — verifiable by
sha256sumafter agit archivedeposit, available bytes-free because keys are repositorystate (
--include=*is load-bearing). Non-SHA-256 keys yield size andno digest; git-carried files hash their bytes. Also fixes a latent
honesty bug: out-of-tree inputs published lc's framed digest under
the workflow-run
sha256term; they now publish none. Validator floorunchanged at five.
crate:line inlc status: the rerun-lag residue made visible bycomparing the document's own
datePublishedagainst the newestmanifest
finished_atthe walk already read — no git, no rocrateimport on status's path.
git_dirtywindow) is now named in the report; still no manifestfield, by decision.
[project.scripts]absence pinned: the recorded review item — ametadata test asserts the worker and the shim never become console
scripts.
Re-examined and deliberately left (recorded in CLAUDE.md)
adversaries, and a copied subject is already deliberate.
publication-metadata config surface.
Housekeeping
Closed stale pre-rebuild issues #166, #128, #153 with pointers;
commented triage on #152 and the advisory landing on #176.
Verification
Full suite: 582 passed (sandbox enforcement real on Linux; crate smoke
with the official rocrate-validator, REQUIRED clean and the floor
unchanged). ruff and mypy clean. Cross-write denial mutation-checked
through
Unavailable()(rc 0, bytes replaced, unsandboxed).Closes #179.
🤖 Generated with Claude Code
https://claude.ai/code/session_016CRERrB5zWdVXD6uPv2BRa