Eval trace follow-up: lc run syntax note, clean harness env

[EiffL]

Follow-up to the PR #190 eval trace, which surfaced three harness-attributable frictions (the other findings were agent-side noise).

The lc run convention question, settled

Checked the ecosystem before touching anything: uv run "python -V" fails with Failed to spawn: `python -V` (verified live), docker run img "echo hi" fails with exec: "echo hi": executable file not found — argv with no shell parsing is the convention across docker run, uv run, kubectl exec, pipx run, poetry run, srun. lc run already matches it, so the behavior stands unchanged and the syntax gets one line in the two places it was missing:

• evals/prompt.md: "Argv style, like docker run or uv run: lc run python scripts/fit.py --output /tmp/x, never a single quoted shell string; for shell syntax use lc run bash -c '...'."
• docs/cli/run.md: same sentence, in the synopsis.

In the trace, the missing line cost one round trip through a genuinely cryptic failure (/usr/bin/env: '<whole string>': No such file or directory + the sandbox trailer insinuating a denial).

Harness environment

• UV_PYTHON noise: setup-uv's python-version input exports UV_PYTHON, and the Hardening: close the residues recorded through layer 8 #184 install-settings scrub then (correctly) warned ignored ambient UV_PYTHON — … on every lc run/lc materialize — a dozen-plus unactionable lines the agent ended up grep -v'ing to read its own output. The input is dropped; determinism is kept by pinning --python 3.12 on both tool installs. Same category as the VIRTUAL_ENV cleanup earlier.
• Truthful dev versions: the shallow checkout had no tags, so hatch-vcs stamped 0.1.dev1+g<sha> into the installed engine — and into every manifest's lc_version. fetch-depth: 0 fixes it.

Validation: label this PR run-eval to see the updated prompt and clean environment in action.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DJzmp2MUhwiNHR94cB91dx

[github-actions]

✅ Eval

Metric	Value
Outputs check	success
Agent run	success
Turns	40
Tool calls	38
Cost	$1.17
Agent wall time	4m01s
Model	claude-sonnet-5

lc status

  mode:    direct
  sandbox: landlock (fs: declared, network: allowed)
  crate:   up to date with the outputs

  · current  baseline/best_fit        d2f7ac8
  · current  baseline/hubble_diagram  d2f7ac8
  · current  baseline/residuals       d2f7ac8

3 current

Confusion & pain points (Claude analysis)

Confusion & pain points

• astra-tools spec metric failed with "Unknown term" — the agent guessed at a schema term (metric) that doesn't exist in ASTRA's type list, wasting a round-trip before falling back to spec recipe and grepping the full guide dump. The CLI's error at least listed valid terms, which let the agent recover, but it suggests the agent had no reliable way to discover valid spec term names up front (e.g., a spec --list or completion) short of triggering a failure first.
• No documented convention for materialized metric/JSON output shape — the agent spent several tool calls (reading results/README.md, dumping the full guide, grepping for "metric|astra:value|.json") trying to find how a metric output's JSON should be structured for {astra:value} references, ultimately proceeding on inference rather than a confirmed spec answer. This points to a gap in the ASTRA guide/skill content rather than in lightcone-cli itself.
• Ambiguous data column semantics required manual reverse-engineering — the agent ran several ad hoc awk/python3 statistics passes over data/SCPUnion2.1_mu_vs_z.txt to figure out what column 5 represented, since the task description and data didn't unambiguously distinguish "host-mass probability" from "systematic error." It made a judgment call (column 5 as an added-in-quadrature systematic) and flagged it as unverified rather than being able to confirm it against ground truth — a data/task-description gap in the eval seed, not a tool issue.
• Otherwise the run was clean: astra validate, uv add, lc materialize, and lc status --json all worked on the first try, and the license/RO-Crate publication flow (declaring license in pyproject.toml then re-running lc materialize) worked exactly as documented with no detours.

Full trace: agent-trace artifact on this run.