Skip to content

Add dedicated --allow-testing-features gate for builtinTestServer#542

Open
MGudgin wants to merge 2 commits into
mainfrom
user/gudge/allow-testing-features-gate
Open

Add dedicated --allow-testing-features gate for builtinTestServer#542
MGudgin wants to merge 2 commits into
mainfrom
user/gudge/allow-testing-features-gate

Conversation

@MGudgin

@MGudgin MGudgin commented Jun 19, 2026

Copy link
Copy Markdown
Member

Summary

This PR fixes a fail-open parity gap where the builtin test proxy
(network.proxy.builtinTestServer) could be activated on the Windows
process-container backends (AppContainer and BaseContainer) without any gate,
while bubblewrap gated it behind the overloaded --experimental flag. It
introduces a dedicated testing-only axis, --allow-testing-features, enforced
uniformly across all backends.

The root cause was that --experimental is a single overloaded axis (backend
admission + experimental features + testing-only helpers) — on the Windows
process-container family it is even self-contradictory as a gate, since it also
flips AppContainer to BaseContainer. The fix decouples "not-for-production
testing scaffolding" into its own flag.

Details

  • Add ExecutionRequest.testing_features_enabled, wired from a new
    --allow-testing-features flag on wxc-exec, lxc-exec, and mxc-exec-mac.
  • Enforce the gate centrally in validate_common (called by ScriptRunner::run
    for every backend): reject network.proxy.builtinTestServer unless the flag
    is set. This is a distinct axis from --experimental ("unstable/new") versus
    "not-for-production testing scaffolding".
  • Remove bubblewrap's now-redundant local --experimental gate; the central
    check covers it.
  • The SDK forwards --allow-testing-features automatically when a one-shot
    policy sets builtinTestServer, preserving SDK ergonomics while the direct
    CLI/wxc-exec surface stays fail-closed.
  • wxc-test-driver auto-passes the flag for configs that use
    builtinTestServer; the bwrap proxy test script is updated.
  • Docs updated: bubblewrap backend, examples, schema, policy v1, and SDK type
    JSDoc.

Tests

  • cargo fmt --check, cargo check --workspace --all-targets, and
    cargo clippy --workspace --all-targets -- -D warnings all pass.
  • Rust unit tests pass, including two new validate_common tests (reject
    without the flag / accept with it) and an updated bwrap test (wxc_common 340
    passed; plus wxc, lxc, appcontainer_common, wxc_test_driver).
  • SDK: tsc build clean and npm test 178 passed, including two new
    flag-forwarding tests.
  • macOS mxc-exec-mac compiles on Windows but its runtime path was not
    exercised on this host.
Microsoft Reviewers: Open in CodeFlow

This PR fixes a fail-open parity gap where the builtin test proxy
(network.proxy.builtinTestServer) could be activated on the Windows
process-container backends (AppContainer and BaseContainer) without any
gate, while bubblewrap gated it behind the overloaded --experimental
flag. It introduces a dedicated testing-only axis,
--allow-testing-features, enforced uniformly across all backends.

Details

* Add ExecutionRequest.testing_features_enabled, wired from a new
  --allow-testing-features flag on wxc-exec, lxc-exec, and mxc-exec-mac.
* Enforce the gate centrally in validate_common (called by
  ScriptRunner::run for every backend): reject
  network.proxy.builtinTestServer unless the flag is set. This is a
  distinct axis from --experimental ("unstable/new") versus
  "not-for-production testing scaffolding".
* Remove bubblewrap's now-redundant local --experimental gate; the
  central check covers it.
* The SDK forwards --allow-testing-features automatically when a
  one-shot policy sets builtinTestServer, preserving SDK ergonomics
  while the direct CLI/wxc-exec surface stays fail-closed.
* wxc-test-driver auto-passes the flag for configs that use
  builtinTestServer; the bwrap proxy test script is updated.
* Docs updated: bubblewrap backend, examples, schema, policy v1, and
  SDK type JSDoc.

Tests

* cargo fmt --check, cargo check --workspace --all-targets, and
  cargo clippy --workspace --all-targets -- -D warnings all pass.
* Rust unit tests pass, including two new validate_common tests (reject
  without the flag / accept with it) and an updated bwrap test
  (wxc_common 340 passed; plus wxc, lxc, appcontainer_common,
  wxc_test_driver).
* SDK: tsc build clean and npm test 178 passed, including two new
  flag-forwarding tests.
* macOS mxc-exec-mac compiles on Windows but its runtime path was not
  exercised on this host.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings June 19, 2026 19:52
@MGudgin MGudgin requested a review from a team as a code owner June 19, 2026 19:52

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a dedicated --allow-testing-features CLI flag and a corresponding ExecutionRequest.testing_features_enabled bit to fail-closed gate the testing-only network.proxy.builtinTestServer feature across one-shot runners, replacing the prior ad-hoc/overloaded gating (notably Bubblewrap’s --experimental-based check).

Changes:

  • Add --allow-testing-features to native executors and plumb it into ExecutionRequest.testing_features_enabled.
  • Enforce network.proxy.builtinTestServer gating centrally in wxc_common::validator::validate_common, and remove Bubblewrap’s redundant local gate.
  • Forward the flag automatically from the SDK (and update docs/tests/scripts to reflect the new gating).
Show a summary per file
File Description
tests/scripts/run_bwrap_network_proxy_test.sh Updates Bubblewrap proxy test invocation to pass --allow-testing-features.
src/testing/wxc_test_driver/src/main.rs Auto-adds --allow-testing-features when configs appear to opt into builtinTestServer.
src/core/wxc/src/main.rs Adds --allow-testing-features CLI flag and plumbs into one-shot ExecutionRequest.
src/core/wxc_common/src/validator.rs Centralized validation rejects builtinTestServer unless testing features are enabled; adds unit tests.
src/core/wxc_common/src/models.rs Adds testing_features_enabled to ExecutionRequest with documentation.
src/core/wxc_common/src/config_parser.rs Defaults testing_features_enabled to false in parsed requests.
src/core/mxc_darwin/src/main.rs Adds --allow-testing-features flag and plumbs into request.
src/core/lxc/src/main.rs Adds --allow-testing-features flag and plumbs into request.
src/backends/bubblewrap/common/src/bwrap_runner.rs Removes backend-local builtinTestServer gating and adjusts runner test accordingly.
sdk/tests/unit/sandbox.test.ts Adds unit coverage ensuring the SDK forwards --allow-testing-features when needed.
sdk/src/types.ts Updates type/JSDoc to document the new gate and SDK auto-forwarding.
sdk/src/helper.ts Implements SDK auto-forwarding of --allow-testing-features when builtinTestServer is set.
docs/schema.md Documents builtinTestServer as requiring --allow-testing-features.
docs/sandbox-policy/v1/policy.md Updates policy docs to mention the new gate for builtinTestServer.
docs/examples.md Expands examples/docs to explain the new testing-only flag and its rationale.
docs/bwrap-support/bubblewrap-backend.md Updates Bubblewrap docs to reflect the new flag gating.

Copilot's findings

  • Files reviewed: 16/16 changed files
  • Comments generated: 2

Comment on lines +63 to +67
// builtinTestServer is testing-only scaffolding gated behind
// --allow-testing-features — pass it when the config opts in.
if content.contains("\"builtinTestServer\"") {
cmd.arg("--allow-testing-features");
}
Comment thread docs/sandbox-policy/v1/policy.md Outdated
| `allowedHosts` | When set, ONLY these outbound hosts are reachable. Error if `allowOutbound` is not set. |
| `blockedHosts` | Hosts to block even when outbound is allowed. Error if `allowOutbound` is not set. |
| `proxy` | `{ builtinTestServer: true }` or `{ url: "..." }`. Routes all traffic through this proxy. Cannot be combined with other network flags. |
| `proxy` | `{ builtinTestServer: true }` or `{ url: "..." }`. Routes all traffic through this proxy. Cannot be combined with other network flags. `builtinTestServer` is testing-only and requires the `--allow-testing-features` flag (the SDK forwards it automatically when the policy uses it). |
…host proxy

* wxc-test-driver now matches the concrete `"builtinTestServer": true` JSON
  pattern (both spacings) instead of the bare key, avoiding false positives
  (e.g. the string appearing inside process.commandLine) that could mask
  fail-closed coverage.
* policy v1 doc lists `{ localhost: <port> }` as a proxy option in both the
  type and the network table, matching the schema/SDK wire format.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@MGudgin

MGudgin commented Jun 19, 2026

Copy link
Copy Markdown
Member Author

Addressed both review comments in f726343:

  • wxc-test-driver: narrowed the match from the bare "builtinTestServer" substring to the concrete "builtinTestServer": true JSON pattern (both spacings), so it can't false-positive on the string appearing in process.commandLine and mask fail-closed coverage.
  • policy v1 docs: added { localhost: <port> } as a proxy option in both the type definition and the network table to match the schema/SDK wire format.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants