Docs rewrite 5/5: reconcile CLAUDE.md and the doc-drift check

[EiffL]

Final pass of the documentation rewrite — no doc pages this time, just bringing the two things that govern the docs in line with their new reality.

CLAUDE.md

• The freeze rule is replaced by its successor: docs are live (PRs Docs rewrite 1/5: README, landing page, install, getting started #185–Docs rewrite 4/5: the developer corner #188), two tracks, and a change lands with its docs — a verb change updates its docs/cli/ page, an engine change its docs/api/ module page, user-visible behavior the user guide. The docs inherit the codebase's own discipline: document only what exists, quote refusals from real runs, verify command blocks by executing them.
• The design-doc pointer now records the decision taken during this rewrite: the design records stay in the sibling checkout and are dropped when the rebuild completes — never imported into docs/ (reverses the paragraph's earlier plan).
• The workflow inventory catches up: eval runs on dispatch or the run-eval label (re-trigger by re-adding it), check-docs.yml reviews merged PRs, docs-deploy.yml tracks releases; the docs build command is stated.
• The extending table's "add the next layer" row loses "Docs are deliberately deferred".

Deliberately not done: slimming CLAUDE.md's invariant sections now that the developer corner overlaps them. CLAUDE.md is still the working memory for the remaining rebuild (Perlmutter spike, deferred layers), and gutting it mid-rebuild trades a known-good reference for dedup. Worth revisiting when the rebuild closes and the design docs drop.

check-docs.yml

The post-merge doc-drift check was written before the freeze and knew nothing about the structure. Its prompt now carries:

• a map from change kind to doc home (verb → docs/cli/, module → docs/api/ + architecture, user-visible → user guide + README, workflow/conventions → contributing);
• the two failure modes that matter most for these docs: quoted console output drifting from what the CLI prints, and documenting anything the code no longer delivers (the no-foreshadowing rule applied to docs);
• an updated skip list (CLAUDE.md and evals/ are maintained separately) replacing the stale skills/ rule.

Verification

• zensical build: zero issues.
• check-docs.yml validates as YAML.
• Grep for frozen/stale workflow descriptions in CLAUDE.md comes back clean.

This closes the 5-PR docs rewrite. The site deploys with the next release (or a manual docs-deploy dispatch).

🤖 Generated with Claude Code

https://claude.ai/code/session_01DJzmp2MUhwiNHR94cB91dx

[github-actions]

✅ Eval

Metric	Value
Outputs check	success
Agent run	success
Turns	35
Tool calls	33
Cost	$0.93
Agent wall time	3m13s
Model	claude-sonnet-5

lc status

  mode:    direct
  sandbox: landlock (fs: declared, network: allowed)
  crate:   up to date with the outputs

  · current  baseline/best_fit        156dd21
  · current  baseline/hubble_diagram  156dd21
  · current  baseline/residuals       156dd21

3 current

Confusion & pain points (Claude analysis)

Confusion & pain points

• numpy not present in the sandboxed venv, causing an initial crash. The agent tried python3 -c "import numpy…" for quick data exploration and hit ModuleNotFoundError, since the scaffolded project has no dependencies yet. Root cause: exploring data before adding dependencies — worked around by hand-rolling the same stats with a pure-Python loop instead of just running uv add numpy scipy matplotlib first, which it did anyway a few steps later. Minor sequencing inefficiency rather than a real product gap.
• astra-tools invoked via uvx astra-tools@0.2.13 rather than the project's own astra CLI, and with subcommands (info --outputs, spec recipe, spec output) not documented anywhere in the visible trace — the agent appears to have guessed or recalled this surface rather than discovering it from the skill/help output, which is a detour around a CLI surface it should have been able to introspect directly (e.g. astra --help).
• Manual local testing of recipe scripts bypassed the sandbox entirely (.venv/bin/python scripts/fit.py … run directly, writing to /tmp/testout*), rather than using lc run (the documented probe verb for exactly this purpose). This works but never exercises the Landlock policy or the declared-input/output-directory constraints the real lc materialize run enforces, so a script that passes this ad hoc test could still fail once run for real — the agent got lucky that it didn't.
• git add staged astra.yaml, pyproject.toml, uv.lock, and scripts/ together in one commit, then a second edit to pyproject.toml (adding the license) landed after the file had already drifted from what the agent last read ("the file had been modified on disk since you last read it") — a Read-then-Edit race caused by the intervening lc materialize run silently rewriting pyproject.toml/committing state between the agent's read and edit, which the harness flagged but could equally have surprised the agent into a bad edit.

Full trace: agent-trace artifact on this run.

[EiffL]

LGTM