Skip to content

fix(workspace): fail closed and report delegated test execution state - #2613

Merged
codeforester merged 4 commits into
mainfrom
bug/2612-20261018-fix-workspace-fail-closed-and-report-execution-state-for-tru
Oct 19, 2026
Merged

codeforester merged 4 commits into
mainfrom
bug/2612-20261018-fix-workspace-fail-closed-and-report-execution-state-for-tru

Conversation

@codeforester

Copy link
Copy Markdown
Collaborator

Summary

  • Preflight manifest-command trust for every selected workspace test project before starting any delegated test.
  • If trust preflight is blocked, start zero project test commands and report blocked versus not-run projects explicitly.
  • Add text progress for preflight, start, completion, elapsed time, and dry-run planning.
  • Publish workspace-test JSON schema version 2 with explicit preflight, execution, blocked, and not-run state.

Issue

Closes #2612

Validation

  • env -u BASE_HOME BASE_TEST_PYTHON=/Users/rameshhp/.base.d/base/.venv/bin/python BASE_BASH_LIBS_DIR=/Users/rameshhp/work/base-bash-libs/lib/bash ./bin/base-test — 1,826 Python tests and 1,052 Bats/integration tests passed.
  • Focused workspace/trust/schema tests: 41 passed, 18 subtests passed.
  • Live blocked-workspace reproduction reported 0/7 project test commands started and marked trusted projects NOT RUN.
  • Stable compatibility check passed.

Notes

The PR is intentionally not merged by this train; merge requires explicit approval.

@codeforester
codeforester requested a review from a team as a code owner October 18, 2026 12:03
@codeforester

Copy link
Copy Markdown
Collaborator Author

Automated review findings

  1. --dry-run can report false success for a selection guaranteed to fail (cli/python/base_projects/workspace_test.py ~line 452): dry_run_workspace_test_results reports every action=="skip" target as status="skipped" unconditionally, ignoring target.fatal. The real execution loop a few lines away computes status = "failed" if target.fatal else "skipped" for the identical case. Confirmed directly against both code paths. Concretely: a workspace manifest selects a required repo that's missing/outside the workspace (inspect_workspace_repo returns a fatal skip state). basectl workspace test --dry-run --format json reports that project as "skipped" and exits 0, because the dry-run exit code (FAILURE if (preflight_blocked or preflight_failed) else SUCCESS) never looks at per-target fatal status. Dropping --dry-run for the identical selection makes counts.failed nonzero and the command exit 1 - the dry run promises success for a selection that's guaranteed to fail, defeating its purpose of previewing the real outcome.

  2. Redundant manifest re-parsing (~line 379): workspace_test_preflight re-reads and re-parses every selected project's base_manifest.yaml, which was already parsed moments earlier by inspect_workspace_repo during target construction - and the manifest is parsed a third time inside each delegated basectl test subprocess. For a workspace with N selected test projects, this is 2-3x the manifest I/O/YAML parsing needed, scaling linearly with workspace size for no behavioral benefit. Passing the already-parsed manifest through WorkspaceTestTarget (or caching it) would avoid the duplicate work.

  3. Reuse gap: workspace_test_preflight reimplements the trust-identity-lookup logic that already exists in base_projects/test_preflight.py::project_test_preflight, instead of extracting a shared, batch-capable helper. The trust-identity/trust-status computation now has two independent call sites with different detail-message formatting (this one's "manifest command trust is blocked (...)" doesn't match the wording in manifest_trust_guidance.print_blocked_command_text) - a future change to trust semantics has to be applied in both places and risks drifting further.

Posted via Claude Code

@codeforester

Copy link
Copy Markdown
Collaborator Author

Addressed the automated review findings in ed66295.

  • Dry-run now preserves fatal required-project selection failures as status=failed and exits non-zero; added a regression test for a missing required repository.
  • WorkspaceTestTarget now carries the parsed manifest from workspace inspection, so workspace preflight no longer reparses base_manifest.yaml.
  • Extracted project_test_trust_status() and reused it from both project test execution and workspace preflight, with one trust-store instance for the workspace batch.

Validation: full local gate passed (1,827 Python tests and 1,052 BATS tests), stable compatibility passed, targeted pylint passed, and git diff --check passed. PR remains open and unmerged.

@codeforester
codeforester merged commit 214f0b9 into main Oct 19, 2026
25 checks passed
@codeforester
codeforester deleted the bug/2612-20261018-fix-workspace-fail-closed-and-report-execution-state-for-tru branch October 19, 2026 15:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(workspace): fail closed and report execution state for trusted workspace tests

1 participant