Skip to content

Validate quoted shell test commands without false path rejections - #765

Open
wwind123 wants to merge 1 commit into
mainfrom
codex/quoted-shell-test-reports
Open

Validate quoted shell test commands without false path rejections#765
wwind123 wants to merge 1 commit into
mainfrom
codex/quoted-shell-test-reports

Conversation

@wwind123

Copy link
Copy Markdown
Owner

Summary

Validate explicit quoted shell command strings as commands, rather than treating the whole string as one filesystem path.

Reproduction

The historical llm-dialectic #1136 / PR #1137 run pushed its implementation and then failed test-report validation on:

bash -lc '/home/wwind123/llm-dialectic/.venv/bin/pytest tests/test_grafana_dashboards.py tests/test_production_topology_artifacts.py -q'

Reproduced against current main 4b46054, importing the guard explicitly from this checkout with PYTHONPATH=src. The error names the entire quoted command, including test arguments, as an outside-checkout path. The external virtualenv executable should retain the existing interpreter exemption, while test paths and working-directory changes must still be checked independently.

Related to #493. This does not close that broader environment-prose issue or change #762's chained managed-wrapper contract.

Changes

  • Recognize explicit sh, bash, and zsh command strings using -c and a conservative set of no-operand option combinations such as -lc.
  • Split and validate the nested executable text through the existing clause/path/URL checks; retain checks on the launcher, prefix environment, and positional arguments.
  • Preserve rejection of outside test paths, outside working directories, live URLs, and unknown outside scripts. Unsupported shell option forms receive no special handling.
  • Bound nested shell expansion to eight levels and reject malformed command-string quoting.
  • Cover the historical example, wrapper prefixes, nested shells/managed wrappers, trailing clauses, and negative cases. Document the behavior and limits.

This validates reported command text. It neither executes shell strings nor evaluates substitutions nor supplies a shell sandbox. No live service tests or provider calls are involved.

Validation

  • Focused guard/runtime/rendering suites: 365 passed using PYTHONPATH=src and isolated writable runtime-lock storage.
  • Full suite: 2,898 passed in 146.61 seconds with env PYTHONPATH=src XDG_RUNTIME_DIR=/tmp/quoted-shell-test-runtime python3 -m pytest tests/ -q --tb=short. Existing tests require user-cache log writes, so the full suite ran with that permission.
  • git diff --check: passed.
  • An initial focused run had four runtime-test failures because the sandbox could not write lock files under /run/user/1001. With XDG_RUNTIME_DIR=/tmp/quoted-shell-test-runtime, all focused tests passed; no live locks were removed or runtime-lock implementation changed.
  • An initial new test mistakenly created the same fixture directory twice; corrected the test to reuse the fixture path.

-- OpenAI Codex

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant