Land PR #2 on main, remove dead code, rewrite README for customers - #3
Merged
Merged
Conversation
- Correct the openai-compatible patch hunk so the upstream anthropic branch of call_llm keeps its return statement. - Reject a dirty vendored upstream checkout at run time, not only in sync-upstream, so shipped validator code always matches the pin. - Fix the result-archive traversal guard: normalize the joined path so ".." members are rejected lexically (filter="data" remains). - Drop gemini-cli from --agent; it speaks the Gemini API, not the OpenAI-compatible transport the adapter wires, and failed silently. - Declare bedrock-mantle.*.api.aws in both packaged policies and match model hosts as patterns so --bedrock-region works outside us-west-2. - Copy the hardened install_codex.sh unconditionally so a mistyped --remote-install-codex path errors instead of shipping upstream's unhardened installer. - Document that network-locked.json needs preloaded agent tooling that this release does not ship. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- Regenerate openai-compatible.patch with three lines of context and apply it without --unidiff-zero, so a misplaced hunk fails loudly instead of applying silently at the wrong line. - Refuse agent runs under a policy whose runtime allowlist is only the model host; agent tooling is installed in the task container and such runs previously failed and were graded as eligible model failures. - Drop symlink and hardlink members when extracting result archives so a stray agent symlink neither escapes the artifact root nor fails collection of a completed run. - Validate --network-policy, --patch-file, --remote-smoke, and --remote-install-codex once at CLI entry instead of once per task. - Point the dirty-checkout error at a remedy that works; sync-upstream also refuses dirty trees. - Type RunOptions.agent as codex|openhands, remove the stale gemini test assertion and comment, document supported agents, and assert the two packaged policies share their host lists. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- The dirty-checkout remedy ran `git clean -fd` without -C, which would clean the caller's repository; both steps now target the vendored path. - Only scripts/ and projects/ are checked for local changes; those are the trees shipped to the sandbox, and IDE metadata elsewhere is harmless. - Result extraction keeps in-tree hardlinks and symlinks and drops only members the tarfile data filter rejects. - batch evaluates the network policy once before resolving or submitting any task, matching preflight and run. - Type the run kind as a Literal and update the README to say the locked policy is refused up front for agent runs while smoke still works. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
main() requires E2B_API_KEY before dispatching to batch. The test passed locally only because the key was set in the developer shell; set a dummy value so CI exercises the policy gate instead of the key check. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Docker Hub allows about 100 anonymous manifest requests per hour per address, so a lock that must resolve all 506 mutable tags before any task can run never completes without an authenticated Docker Hub plan. - `images lock --task <project>/<task>` (repeatable) resolves only the images those tasks need and merges into the existing lock for the same upstream inputs, so the inventory can be walked in slices. - The lock loader no longer rejects a partial lock; every task is still refused at resolution time if its image is absent, with the exact command to add it. Extra, stale, mutable, and provenance-free entries are rejected as before. - The Buildx resolver backs off and retries briefly on a 429 and reports the registry's error text instead of a bare exit status. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Issues are disabled on this repository, so the confirmed reliability findings from the release review that PR #2 leaves alone are tracked in-tree. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Fix release review findings
…nto cleanup/land-review-fixes
PR #2 was merged into the already-merged feature branch instead of main, so main still carried the broken patch hunk and the full-inventory-only image lock. This branch merges that work and cleans up on top of it. Runtime (closes FOLLOW-UPS 1-5, 8): - One _enforces_allowlist predicate shared by _network, _network_eligibility, _require_runnable_policy, and _execution_context. - Post-workload observation is one best-effort helper; a bad sample or a hung metrics command can no longer prevent trajectory collection. - Resource monitor starts after the bundle upload that wipes its directory. - Workload polling fails fast after 12 consecutive unreachable polls. - 600 s reserved for collection; evaluation_timeout validated in preflight. - Host-side DNS for FFmpeg rsync hosts runs only under an allowlist, with a clear per-task error on failure. - _resource_summary drops legacy-format heuristics and skips truncated lines. Dead code and debt: - Remove --image-map raw-dict compatibility path and --resume alias. - Remove resolved_asdict, duplicate template-name and E2B-id patterns, a no-op dict copy, and the double requirements validation. - Public-tree policy and .gitignore no longer name files from the private extraction (analyze_profiles.py, *-tasks.txt, campaigns/, reports/, ...). - e2b dependency widened to >=2.37,<3 (identical API surface verified on 2.46.4); uv.lock keeps 2.37.0. Version 0.3.0. Docs: - README rewritten around a quickstart, an honest deviations-from-upstream table, and known limitations (no claude-code/Anthropic path, no plain OpenAI provider, Docker Hub anonymous limit). - FOLLOW-UPS trimmed to what is still open. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This was referenced Sep 8, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
PR #2 (
fix/review-findings) was merged intocodex/cybergym-public-release, which had itself already been merged intomainby PR #1.maintherefore never received the review fixes: the compatibility patch hunk that left upstream's Anthropiccall_llmbranch unreachable, the task-scopedimages lock --task(the only lock a customer without a paid Docker Hub login can complete), the locked-policy refusal for agent runs, and the archive traversal fix. This branch merges that work intomainand cleans up on top.Runtime fixes (FOLLOW-UPS 1–5, 8)
_enforces_allowlistpredicate replaces the egress checks duplicated across_network,_network_eligibility,_require_runnable_policy.rm -rfs its output directory._shell_runfails fast after 12 consecutive unreachable polls instead of pinning a batch worker until the full evaluation timeout.evaluation_timeoutis validated up front (also inpreflight)._resource_summarydrops heuristics for sample formats that never shipped and tolerates a truncated last line.Dead code and debt
--image-mapraw-dict compatibility path and--resumealias removed; the lock format is the only accepted input.resolved_asdict, duplicate template-name and E2B-id regexes, a no-op dict copy in_ensure_image, and a second requirements-lock validation removed.scripts/check_public_tree.pyand.gitignoreno longer name files from the private extraction (analyze_profiles.py,*-tasks.txt,campaigns/,reports/,docs/validation/).e2bdependency widened from<2.38to>=2.37,<3. TheSandbox.create,update_network,pause,Template.build, andTemplate.get_tagssignatures are identical on 2.37.0 and 2.46.4;uv.lockstill resolves 2.37.0. Version bumped to 0.3.0.Docs
openaiprovider,claude-code/Anthropic support).Verification
uv run pytest -q: 73 passed (70 before; new tests cover poll liveness, the collection reserve, allowlist-only DNS, the observation helper, and the lock-only loader)ruff check,ruff format --check,scripts/check_public_tree.py,actionlint,uv build,twine check: passsmoke curl/arvo_66012before merge would confirm the collection path end to end.🤖 Generated with Claude Code