github · lpcox · Sep 10, 2026 · Sep 9, 2026 · Sep 9, 2026 · Sep 9, 2026
diff --git a/.github/agents/self-hosted-runner-doctor.md b/.github/agents/self-hosted-runner-doctor.md
@@ -82,6 +82,7 @@ Prefer the narrowest match. Examples:
 - `copilot: command not found` inside `--container-runtime sbx` when binary is at `~/.local/bin/copilot` → D10 (`bash -lc` login init resets injected PATH in sbx; fixed by wrapping the executed command with `export PATH="$HOME/.local/bin${PATH:+:$PATH}"` in github/gh-aw-firewall#6407)
 - `SIGABRT` / `signal=SIGABRT duration=0s stdout=0B` for Copilot CLI all retries under `--container-runtime gvisor`; or exit 139 / `Segmentation fault` on bash wrapper, often before any model or tool call → D11 (Node.js v22 V8 ESM decode assertion under gVisor; one-shot restart mitigation in github/gh-aw-firewall#6514; underlying Node/gVisor incompatibility unresolved in github/gh-aw-firewall#6558)
 - `Model "auto" has no AI credits pricing and no default pricing is configured` together with `awf-reflect: request failed: fetch failed` under `--container-runtime gvisor` or `sbx` → D12 (isolated runtime cannot reach `/reflect` to pre-resolve `auto`; AI-credits guard rejected sentinel `auto`; fixed in github/gh-aw-firewall#6811)
+- `Direct sbx egress reached 1.1.1.1 without proxy environment variables` (or a similar denied-destination reach) despite Squid healthchecks passing → D13 (`sbx daemon` restarted without `DOCKER_SANDBOXES_PROXY`; workaround/CI fix in github/gh-aw-firewall#8252)
 - `TCP_DENIED` in Squid access log for a topology peer or `difcProxyHost` during agent run; in-session MCP/HTTP calls to those hosts fail in network-isolation mode → B13 (topology peer hostnames and `difcProxyHost` not added to `NO_PROXY`; fixed in github/gh-aw-firewall#6189 and github/gh-aw-firewall#6438; if block report still flags topology peer after github/gh-aw-firewall#6473, treat as audit/policy-manifest reporting false positive tracked in github/gh-aw-firewall#6652 / github/gh-aw-firewall#6658 — runtime traffic is not blocked)
 - `⚠️ Firewall blocked N domain(s)` warning lists `awmgmcpg` or `172.30.0.x` as a blocked domain on every run, even with no actual external blocks → B13 (internal MCP gateway traffic counted by log aggregator as denied; fixed in github/gh-aw-firewall#6689 with `isInternalAwfDomain()` filter)
 - `--network-isolation is not yet supported with --enable-host-access` → B15 (compiler auto-emits both flags when `localhost` in allowlist + topology; fixed in github/gh-aw-firewall#6657)
@@ -204,6 +205,8 @@ D11 / github/gh-aw-firewall#6558 — gVisor + Node.js v22 V8 ESM startup crash r
 
 D12 / github/gh-aw-firewall#6810, github/gh-aw-firewall#6811 — Copilot runs using `model: auto` under isolated runtimes (`--container-runtime gvisor` or `sbx`) could fail before agent start with `awf-reflect: request failed: fetch failed` plus `Model "auto" has no AI credits pricing and no default pricing is configured` when `apiProxy.maxAiCredits` was enabled. **Fixed in AWF (PR github/gh-aw-firewall#6811, merged 2026-08-01):** api-proxy `checkUnknownModelRejection` now allows Copilot `auto` through pre-flight (`provider === 'copilot' && model.toLowerCase() === 'auto'`) and charges credits from the response's resolved model metadata. Non-Copilot providers still reject unresolved `auto`. Workaround (older AWF): pin a concrete priced model (for example `model: claude-sonnet-4.6`).
 
+D13 / github/gh-aw-firewall#8250, github/gh-aw-firewall#8252 — With `--container-runtime sbx` and `network.verifySbxEgress`/`--verify-sbx-egress` enabled, AWF can fail closed before agent startup with `Direct sbx egress reached 1.1.1.1 without proxy environment variables` despite healthy Squid checks. The long-lived `sbx daemon` was started or restarted without `DOCKER_SANDBOXES_PROXY` pointing to AWF's published Squid endpoint, so its own sandboxed egress bypasses Squid. This is caller/workflow daemon lifecycle, not an AWF core-code defect: start/restart the daemon with `DOCKER_SANDBOXES_PROXY=http://host.docker.internal:3128` (or the appropriate Squid gateway) before AWF creates the sandbox. This repository's smoke-workflow post-processing final restart was fixed in github/gh-aw-firewall#8252; see `docs/sbx-integration.md` for the daemon-level contract. Probe with `sbx daemon status` and inspect its environment for `DOCKER_SANDBOXES_PROXY`.
+
 C7 / #5615 — DIFC proxy enterprise-host awareness for `*.ghe.com` data-residency is not yet implemented in the companion projects; AWF ≥ v0.27.12 provides improved diagnostics (HTTP status + targeted hint) but the underlying cause remains unresolved.
 
 C8 / github/gh-aw-firewall#5872, github/gh-aw-firewall#6237 — Copilot Business `token` prefix short-circuit on GHEC is **fixed** in AWF version including github/gh-aw-firewall#5872. **Additional fix (github/gh-aw-firewall#6237):** `gh-aw`'s offline mode sets `COPILOT_PROVIDER_API_KEY=dummy-byok-key-for-offline-mode` as a sentinel. In AWF before github/gh-aw-firewall#6237, this sentinel was treated as a real BYOK key, suppressing the GitHub-token auth path and producing `400` on Business/Enterprise targets. Fixed by treating `dummy-byok-key-for-offline-mode` as a non-credential sentinel (same class as AWF placeholder tokens).
@@ -356,6 +359,7 @@ Establish these facts before matching a failure mode:
 | D10 | On `--container-runtime sbx`, Copilot CLI installed via `install_copilot_cli.sh --rootless` (redirects install to `~/.local/bin`) is **not found** even though `~/.local` is mounted into the microVM; `copilot: command not found` or `ENOENT` at agent startup | sbx executes agent commands via `bash -lc` (login shell), whose profile initialization can reset PATH and discard `--env PATH=...`. The binary is present in the VM but unreachable by name unless PATH is fixed after login init. | **Fixed in AWF (PR github/gh-aw-firewall#6407, merged 2026-07-19):** sbx wraps the executed command with `export PATH="$HOME/.local/bin${PATH:+:$PATH}"` after login initialization (`withLocalBinOnPath()` in `sbx-manager.ts`). Upgrade to the AWF version including github/gh-aw-firewall#6407. Workaround (older AWF): invoke `$HOME/.local/bin/copilot` directly, or prefix the agent command with `export PATH="$HOME/.local/bin${PATH:+:$PATH}"; ...`. | Inside the sbx agent: `which copilot` or `ls ~/.local/bin/copilot` confirms binary presence; on patched AWF, inspect the executed command wrapper in sbx logs and verify it prepends `~/.local/bin` before invoking the agent command | github/gh-aw-firewall#6407 |
 | D11 | Copilot CLI agent starts under `--container-runtime gvisor` but exits immediately with **exit code 139** (`SIGSEGV`) or `SIGABRT` (exit 1); `[copilot-harness]` log shows all retry attempts crashing within ~90 ms (`tokenCount=0`, `stdout=0B`); the outer `bash` wrapper can also segfault, often before any model or tool call is issued. Affects `sandbox.agent.runtime: gvisor` (compose-managed gVisor) at ~8% failure rate; identical workloads on `runc` and `sbx` are unaffected. | Node.js v22 (bundled in the Copilot CLI) can trigger a V8 native assertion (`StringBytes::Encode ... Assertion failed: (written) == (u16size)`) during ESM module translation under gVisor's userspace netstack; gVisor's `mmap`/`madvise` emulation can return unexpected buffer contents to V8's UTF-8 decode path. The `SIGSEGV`/exit 139 variant can also take down the outer `bash` wrapper so the Copilot harness cannot retry. Root cause (gVisor ↔ Node.js v22 incompatibility) is **unresolved** — tracked in github/gh-aw-firewall#6558. | **Mitigated in AWF (PR github/gh-aw-firewall#6514, merged 2026-07-23):** `runAgentCommand()` detects gVisor runtime and automatically retries the agent container **once** (via `docker start awf-agent`) when the exit code is `134` (SIGABRT) or `139` (SIGSEGV) **within the first 30 s** (`GVISOR_STARTUP_CRASH_WINDOW_MS = 30_000`); `MAX_GVISOR_AGENT_RETRIES = 1`. Log reattachment uses `docker logs --since <restart-time> -f awf-agent` to avoid replaying the crashed attempt. Non-gVisor runtimes are unaffected. Root cause remains open; upgrade AWF to include github/gh-aw-firewall#6514 to get the retry mitigation. | Agent logs show `signal=SIGABRT duration=0s stdout=0B` on attempt 1; with github/gh-aw-firewall#6514, a second `docker start awf-agent` log block appears and usually succeeds. Without the fix, all harness retries fail identically. Confirm `containerRuntime: gvisor` in the resolved `docker-compose.redacted.yml`. | github/gh-aw-firewall#6513, github/gh-aw-firewall#6514, github/gh-aw-firewall#6558 |
 | D12 | Copilot workflow with `model: auto` (or no explicit top-level `model`, where gh-aw v0.84.1+ emits `auto`) fails before the agent starts under `--container-runtime gvisor` or `sbx`; harness logs `awf-reflect: fetching (apiproxy/redacted)` then `request failed: fetch failed`, followed by `400 ... Model "auto" has no AI credits pricing and no default pricing is configured`; retries fail identically with zero tokens consumed. Same workflow succeeds under default (non-isolated) AWF runtime. | Isolated agent runtimes (gVisor, sbx) may not reach `(apiproxy/redacted)`, which the harness uses to pre-resolve `auto` to a concrete priced model. Without that resolution, api-proxy `maxAiCredits` pre-flight guard (`checkUnknownModelRejection` in `guards/ai-credits-guard.js`) had no pricing for literal `auto` and rejected the request with HTTP 400, even though Copilot resolves `auto` server-side and returns priced resolved-model metadata post-response. | **Fixed in AWF (PR github/gh-aw-firewall#6811, merged 2026-08-01):** `checkUnknownModelRejection` now allows `provider === 'copilot' && model.toLowerCase() === 'auto'` to pass pre-flight; AI-credit accounting then uses the response's resolved model. Non-Copilot providers still reject unresolved `auto`. Upgrade AWF to include github/gh-aw-firewall#6811. Workaround (older AWF): pin a concrete priced model in workflow frontmatter (for example `model: claude-sonnet-4.6`) to avoid catalog-based `auto` resolution under isolated runtimes. | Confirm `sandbox.agent.runtime: gvisor` or `sbx` in resolved AWF config; check api-proxy logs for `400 ... Model "auto" has no AI credits pricing` alongside harness `awf-reflect: request failed: fetch failed`; verify whether `apiProxy.maxAiCredits` is enabled (guard only fires when enabled) | github/gh-aw-firewall#6810, github/gh-aw-firewall#6811 |
+| D13 | Under `--container-runtime sbx` with `network.verifySbxEgress`/`--verify-sbx-egress` enabled, AWF reports `Direct sbx egress reached 1.1.1.1 without proxy environment variables` and aborts before the agent starts, even though Squid is healthy (`squid host.docker.internal:3128 -> 200`). | The long-lived `sbx daemon` was started or restarted without `DOCKER_SANDBOXES_PROXY` pointing to AWF's published Squid endpoint, allowing its sandboxed egress to bypass Squid. AWF correctly detects this and fails closed. | The caller/workflow owns daemon lifecycle: start or restart `sbx daemon` with `DOCKER_SANDBOXES_PROXY=http://host.docker.internal:3128` (or the appropriate Squid gateway) before AWF creates the sandbox. PR github/gh-aw-firewall#8252 fixes this repository's smoke-workflow post-processing final daemon restart. See `docs/sbx-integration.md` for the general contract. | Run `sbx daemon status` and inspect its environment for `DOCKER_SANDBOXES_PROXY`; restart it without that variable and confirm AWF's direct-egress probe reaches `1.1.1.1` or a denied destination without proxy variables. | github/gh-aw-firewall#8250, github/gh-aw-firewall#8252 |
 
 ## Error-string quick lookup
 
@@ -398,6 +402,7 @@ Establish these facts before matching a failure mode:
 | `copilot: command not found` inside sbx microVM when binary is at `~/.local/bin/copilot` | D10 (`bash -lc` login init resets injected PATH; fixed by command wrapper `export PATH="$HOME/.local/bin${PATH:+:$PATH}"` in github/gh-aw-firewall#6407) |
 | `SIGABRT` / `signal=SIGABRT duration=0s stdout=0B` for Copilot CLI all retries under `--container-runtime gvisor`; or exit code 139 with `Segmentation fault` on bash wrapper | D11 (Node.js v22 V8 ESM decode assertion under gVisor; one-shot restart mitigation in github/gh-aw-firewall#6514; underlying Node/gVisor incompatibility unresolved in github/gh-aw-firewall#6558) |
 | `Model "auto" has no AI credits pricing and no default pricing is configured` together with `awf-reflect: request failed: fetch failed` under `--container-runtime gvisor` or `sbx` | D12 (isolated runtime cannot reach `/reflect` to pre-resolve `auto`; AI-credits guard rejected sentinel `auto`; fixed in github/gh-aw-firewall#6811) |
+| `Direct sbx egress reached 1.1.1.1 without proxy environment variables` (or a similar denied-destination reach) despite Squid healthchecks passing | D13 (`sbx daemon` restarted without `DOCKER_SANDBOXES_PROXY`; workaround/CI fix in github/gh-aw-firewall#8252) |
 | `TCP_DENIED` in Squid access log for topology peer or DIFC proxy host during agent run; MCP tool calls silently fail or return connection errors in network-isolation mode | B13 (topology peers and `difcProxyHost` missing from `NO_PROXY`; fixed in github/gh-aw-firewall#6189 and github/gh-aw-firewall#6438; if block report still flags topology peer after github/gh-aw-firewall#6473, treat as audit/policy-manifest reporting false positive tracked in github/gh-aw-firewall#6652 / github/gh-aw-firewall#6658 — runtime traffic is not blocked) |
 | `⚠️ Firewall blocked N domain(s)` warning lists `awmgmcpg` or `172.30.0.x` as a blocked domain on every run, even with no actual external blocks | B13 (internal MCP gateway traffic counted by log aggregator as denied; fixed in github/gh-aw-firewall#6689 with `isInternalAwfDomain()` filter) |
 | `--network-isolation is not yet supported with --enable-host-access` | B15 (compiler auto-emits both flags when `localhost` in allowlist + topology; fixed in github/gh-aw-firewall#6657) |

diff --git a/.github/workflows/self-hosted-runner-doctor.md b/.github/workflows/self-hosted-runner-doctor.md
@@ -111,6 +111,7 @@ Prefer the narrowest match. Examples:
 - `copilot: command not found` inside `--container-runtime sbx` when binary is at `~/.local/bin/copilot` → D10 (`bash -lc` login init resets injected PATH in sbx; fixed by wrapping the executed command with `export PATH="$HOME/.local/bin${PATH:+:$PATH}"` in github/gh-aw-firewall#6407)
 - `SIGABRT` / `signal=SIGABRT duration=0s stdout=0B` for Copilot CLI all retries under `--container-runtime gvisor`; or exit 139 / `Segmentation fault` on bash wrapper, often before any model or tool call → D11 (Node.js v22 V8 ESM decode assertion under gVisor; one-shot restart mitigation in github/gh-aw-firewall#6514; underlying Node/gVisor incompatibility unresolved in github/gh-aw-firewall#6558)
 - `Model "auto" has no AI credits pricing and no default pricing is configured` together with `awf-reflect: request failed: fetch failed` under `--container-runtime gvisor` or `sbx` → D12 (isolated runtime cannot reach `/reflect` to pre-resolve `auto`; AI-credits guard rejected sentinel `auto`; fixed in github/gh-aw-firewall#6811)
+- `Direct sbx egress reached 1.1.1.1 without proxy environment variables` (or a similar denied-destination reach) despite Squid healthchecks passing → D13 (`sbx daemon` restarted without `DOCKER_SANDBOXES_PROXY`; workaround/CI fix in github/gh-aw-firewall#8252)
 - `TCP_DENIED` in Squid access log for a topology peer or `difcProxyHost` during agent run; in-session MCP/HTTP calls to those hosts fail in network-isolation mode → B13 (topology peer hostnames and `difcProxyHost` not added to `NO_PROXY`; fixed in github/gh-aw-firewall#6189 and github/gh-aw-firewall#6438; if block report still flags topology peer after github/gh-aw-firewall#6473, treat as audit/policy-manifest reporting false positive tracked in github/gh-aw-firewall#6652 / github/gh-aw-firewall#6658 — runtime traffic is not blocked)
 - `⚠️ Firewall blocked N domain(s)` warning lists `awmgmcpg` or `172.30.0.x` as a blocked domain on every run, even with no actual external blocks → B13 (internal MCP gateway traffic counted by log aggregator as denied; fixed in github/gh-aw-firewall#6689 with `isInternalAwfDomain()` filter)
 - `--network-isolation is not yet supported with --enable-host-access` → B15 (compiler auto-emits both flags when `localhost` in allowlist + topology; fixed in github/gh-aw-firewall#6657)
@@ -227,6 +228,8 @@ D10 / github/gh-aw-firewall#6407 — On `--container-runtime sbx`, the Copilot C
 
 D12 / github/gh-aw-firewall#6810, github/gh-aw-firewall#6811 — Copilot runs using `model: auto` under isolated runtimes (`--container-runtime gvisor` or `sbx`) could fail before agent start with `awf-reflect: request failed: fetch failed` plus `Model "auto" has no AI credits pricing and no default pricing is configured` when `apiProxy.maxAiCredits` was enabled. **Fixed in AWF (PR github/gh-aw-firewall#6811, merged 2026-08-01):** api-proxy `checkUnknownModelRejection` now allows Copilot `auto` through pre-flight (`provider === 'copilot' && model.toLowerCase() === 'auto'`) and charges credits from the response's resolved model metadata. Non-Copilot providers still reject unresolved `auto`. Workaround (older AWF): pin a concrete priced model (for example `model: claude-sonnet-4.6`).
 
+D13 / github/gh-aw-firewall#8250, github/gh-aw-firewall#8252 — With `--container-runtime sbx` and `network.verifySbxEgress`/`--verify-sbx-egress` enabled, AWF can fail closed before agent startup with `Direct sbx egress reached 1.1.1.1 without proxy environment variables` despite healthy Squid checks. The long-lived `sbx daemon` was started or restarted without `DOCKER_SANDBOXES_PROXY` pointing to AWF's published Squid endpoint, so its own sandboxed egress bypasses Squid. This is caller/workflow daemon lifecycle, not an AWF core-code defect: start/restart the daemon with `DOCKER_SANDBOXES_PROXY=http://host.docker.internal:3128` (or the appropriate Squid gateway) before AWF creates the sandbox. This repository's smoke-workflow post-processing final restart was fixed in github/gh-aw-firewall#8252; see `docs/sbx-integration.md` for the daemon-level contract. Probe with `sbx daemon status` and inspect its environment for `DOCKER_SANDBOXES_PROXY`.
+
 C7 / #5615 — DIFC proxy enterprise-host awareness for `*.ghe.com` data-residency is not yet implemented in the companion projects; AWF ≥ v0.27.12 provides improved diagnostics (HTTP status + targeted hint) but the underlying cause remains unresolved.
 
 C8 / github/gh-aw-firewall#5872 — Copilot Business `token` prefix short-circuit on GHEC is **fixed** in AWF version including github/gh-aw-firewall#5872. **Additional fix (github/gh-aw-firewall#6237):** `gh-aw`'s offline mode sets `COPILOT_PROVIDER_API_KEY=dummy-byok-key-for-offline-mode` as a sentinel. In AWF before github/gh-aw-firewall#6237, this sentinel was treated as a real BYOK key, suppressing the GitHub-token auth path and producing `400` on Business/Enterprise targets. Fixed by treating `dummy-byok-key-for-offline-mode` as a non-credential sentinel (same class as AWF placeholder tokens).