Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -255,6 +255,11 @@ jobs:
- name: Fast-path contract tests
run: python3 scripts/test_fast_path.py

# E2E off the pull request: the `CI_E2E_IN_CI` switch and `mode: e2e`
# (the post-merge / nightly run), over the real job graph.
- name: E2E mode contract tests
run: python3 scripts/test_e2e_mode.py

# The pull-request E2E subset (`e2e-pr-shards`/`e2e-pr-args`). Two
# halves: that leaving both unset is byte-for-byte today's behaviour on
# every event — every pinned adopter passes neither — and that `e2e`
Expand Down
225 changes: 177 additions & 48 deletions .github/workflows/nuxt-cloudflare.yml

Large diffs are not rendered by default.

12 changes: 9 additions & 3 deletions .github/workflows/red-main-listener.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,16 +34,22 @@ name: Reusable Red-Main Listener
# types: [completed]
# jobs:
# red-main:
# # `event == 'push' || event == 'workflow_dispatch'` matters: CI also
# # `event == 'push' || event == 'workflow_dispatch' || event ==
# # 'schedule'` matters: CI also
# # runs on `pull_request`, whose `head_branch` is the PR's OWN
# # branch, not `main` -- a branch that happens to be named `main`
# # would otherwise pass the filter below. `workflow_dispatch` is
# # included because a manually-dispatched CI run against the default
# # branch is just as real a "main is red" signal as a push; drop it
# # from this list if a given caller's `workflow_dispatch` runs never
# # target the default branch as CI.
# # target the default branch as CI. `schedule` is included because a
# # scheduled run always runs the default branch, so a red nightly
# # (the `E2E` workflow's safety net, see README "E2E off the pull
# # request") must open or refresh the issue too. List the post-merge
# # workflow beside CI: `workflows: ["CI", "E2E"]`; each workflow gets
# # its own "main is red: <name>" issue.
# if: |
# (github.event.workflow_run.event == 'push' || github.event.workflow_run.event == 'workflow_dispatch') &&
# (github.event.workflow_run.event == 'push' || github.event.workflow_run.event == 'workflow_dispatch' || github.event.workflow_run.event == 'schedule') &&
# github.event.workflow_run.head_branch == github.event.repository.default_branch
# uses: narduk-enterprises/workflows/.github/workflows/red-main-listener.yml@<sha> # v1
# permissions:
Expand Down
7 changes: 6 additions & 1 deletion .github/workflows/red-main-self.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,13 @@ jobs:
# is exactly as real a "main is red" signal as a red push -- the
# `head_branch == default_branch` check still excludes a manual dispatch
# against any other branch or ref.
# `schedule` is included (fast-CI program, 2026-09-29): a nightly run --
# the post-merge `E2E` workflow's safety net -- runs the default branch, so
# a red nightly is a "main is red" signal too. This repository's own CI has
# no schedule trigger; the clause keeps the reference adopter identical to
# the caller shape README documents for adopters that do.
if: |
(github.event.workflow_run.event == 'push' || github.event.workflow_run.event == 'workflow_dispatch') &&
(github.event.workflow_run.event == 'push' || github.event.workflow_run.event == 'workflow_dispatch' || github.event.workflow_run.event == 'schedule') &&
github.event.workflow_run.head_branch == github.event.repository.default_branch
uses: ./.github/workflows/red-main-listener.yml
permissions:
Expand Down
120 changes: 120 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1038,6 +1038,20 @@ That last row is the same fail-closed rule `require-scripts` exists for: "the
audit did not run" must never be indistinguishable from "the audit found
nothing".

**A pull request only warns (Logan, 2026-09-29, "Main + nightly").** An
advisory is published on its own schedule, not the pull request's, so a fixable
high/critical finding used to block an unrelated PR the day it appeared. On
`pull_request` and `pull_request_target` every `::error::` row above is
re-labelled `::warning::` (the same advisory list, still in the job summary)
and the step passes; the closing line says the finding will fail the default
branch. On a `push` to the default branch, a `schedule` run and a
`workflow_dispatch` the step fails exactly as before, so a red audit is caught
by the merge that follows and by the nightly run (see
[E2E off the pull request](#e2e-off-the-pull-request-mode-e2e-and-ci_e2e_in_ci),
whose nightly caller is not the place for it: the audit runs in the `ci`
workflow, so add `schedule:` to that caller too if the audit should be checked
nightly).

Both report shapes are parsed, and the `package-manager` input selects the
*command*, never the parser — npm changed this format once already, and a
parser keyed on the input would silently read zero advisories the next time it
Expand Down Expand Up @@ -1089,6 +1103,105 @@ it is a security bar rather than an optional lane. It is still a new gate that
can turn an existing adopter red, so the `v1` tag must not move onto it until
the adopters have been checked — see [Versioning policy](#versioning-policy).

#### E2E off the pull request (`mode: e2e` and `CI_E2E_IN_CI`)

Playwright is the slowest lane and it almost never catches a bug that the
merge-time run would not (Logan, 2026-09-29: "skip on PRs; run after each merge,
newest wins, plus nightly; one org switch"). Two pieces, both additive: with
neither in play a caller behaves exactly as before.

**1. The org switch: `vars.CI_E2E_IN_CI`.** When the organization (or one repo,
which overrides the org value) sets `CI_E2E_IN_CI` to `false` (GitHub compares strings case-insensitively, so `False` and `FALSE` switch it off too), `E2E plan`, `E2E`
and `E2E quarantine` are skipped in an ordinary CI run on **every** event, and
`Required` reads that skip as success (a lane that runs anyway still fails
it). Build then skips packing and uploading the prebuilt application for the E2E
jobs, and a `checks-in-build` caller with no other lane folds `Required` into
`Build` (one job, no extra queue hop). A pull request that skipped E2E this way mints no `Required` proof (the proof key does not
include the variable, so a reusable proof would let a later push, after the variable is removed
or overridden, go green without E2E ever having run). Unset, empty or any other value changes
nothing. The switch deliberately does **not** override two promises to run
browsers: `e2e-full-paths` (a caller that sets it wants protected-path escalation,
which needs `E2E plan` to decide, so acre-oracle keeps browsers on auth/payment
changes) and `expected-candidate-sha` (explicit release validation cannot skip
configured browser coverage).

**2. The post-merge and nightly run: `mode: e2e`.** With `mode: e2e` the
callable runs only Build -> E2E plan -> E2E (and the quarantine lane) and a
`Required` verdict. Every other lane, the Caller lint and the dependency audit
are skipped, the `CI_E2E_IN_CI` variable is ignored, and the run never
path-skips (a newer push cancels this run, so no diff can prove a skip).
`Required` is red if the mode is not `ci` or `e2e`, if `run-e2e` is not `true`,
if `journey-smoke-url` is set, or if any E2E lane fails, is cancelled, or was
skipped without the plan's say-so. The mode has its own workflow because an
app's `promote.yml` fires on `workflow_run` completion of the **whole** CI
workflow: E2E left in `ci.yml` would delay every promotion.

Name the caller workflow **`E2E`** (the reaper and the red-main listener look for
that name) and stamp it exactly, with the same `with:` values as the app's `ci`
job for the E2E inputs and the same runner inputs:

```yaml
name: E2E

# Post-merge and nightly Playwright. The newest merge wins: a push cancels the
# run still in flight, and every run tests the whole suite.
on:
push:
branches: [main]
schedule:
- cron: '17 8 * * *' # 08:17 UTC = 3:17 AM CT
workflow_dispatch:

concurrency:
group: e2e-${{ github.repository }}
cancel-in-progress: true

permissions:
contents: read

jobs:
ci:
permissions:
contents: read
packages: read
actions: read
pull-requests: write
uses: narduk-enterprises/workflows/.github/workflows/nuxt-cloudflare.yml@<sha> # v2
secrets: inherit
with:
mode: e2e
run-e2e: true
# ...the app's ci.yml E2E and runner inputs, verbatim (e2e-runner,
# e2e-shards, e2e-args, e2e-build-artifact-path, e2e-quarantine-args,
# install-script, node-version, working-directory, ...)
```

Format the stamped file with the app's own Prettier config: apps that run `format:check` over `.github/workflows` (cloudflarestat-us, single quotes) fail a double-quoted cron.

The calling job id stays `ci` so the composed context reads `E2E / ci / Required`.
`concurrency` sits in the caller, never in the callable (R6). Pass the same
`permissions:` block the app's `ci` job grants: a job in the callable may only
use what its caller granted (R12).

A red post-merge or nightly run reaches the same `red-main` issue flow as CI:
add the workflow to the app's red-main listener, and accept `schedule`.

```yaml
on:
workflow_run:
workflows: ["CI", "E2E"]
types: [completed]
jobs:
red-main:
if: |
(github.event.workflow_run.event == 'push' || github.event.workflow_run.event == 'workflow_dispatch' || github.event.workflow_run.event == 'schedule') &&
github.event.workflow_run.head_branch == github.event.repository.default_branch
```

Each workflow keeps its own "main is red: `<name>`" issue, opened by the first
red run and closed by the next green one. A run cancelled by a newer merge is
not a verdict and does nothing.

#### Quality level (`quality-level`, `quality-opt-out`)

The estate has web quality tools that gated nothing on most apps, because each
Expand Down Expand Up @@ -2070,6 +2183,13 @@ P3-C2 / O-D8 rather than kept as a dead compatibility surface.
callers who never asked for one, which is a breaking change dressed as an
additive input. Getting the *default* wrong is how an "additive" change
breaks people.
- `nuxt-cloudflare.yml`'s `mode` input and `vars.CI_E2E_IN_CI` switch
([E2E off the pull request](#e2e-off-the-pull-request-mode-e2e-and-ci_e2e_in_ci))
are within-major on the same rule: one optional input defaulting to `ci`, no
new job, no new permission, and an unset variable reproduces every
adopter's behaviour exactly. The `Dependency audit` step's pull-request
downgrade (warn on `pull_request`, block on push, schedule and dispatch) only
*loosens* a gate on one event, so no adopter turns red because of it.
- `nuxt-cloudflare.yml`'s `foundation-check` / `foundation-check-tool-version`
(company-hq docs/WEB-FOUNDATION-CHECK.md, D-WEBFOUND-2 Q5/Q9 (a),
D-WEBFOUND-3) are within-major on the same rule — two optional inputs, no
Expand Down
100 changes: 99 additions & 1 deletion scripts/test_dependency_audit.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,10 @@
* a stale `audit-ignore` entry warns so it gets removed
* a missing, empty, non-JSON or unrecognised report is a HARD FAILURE --
"the audit did not run" must never look like "the audit found nothing"
* on `pull_request` / `pull_request_target` EVERY finding above becomes a
`::warning::` carrying the same advisory list and the step passes; the
default branch (`push`), `schedule` and `workflow_dispatch` keep failing
(fast-CI program, Logan 2026-09-29 "Main + nightly")

Run: python3 scripts/test_dependency_audit.py
"""
Expand Down Expand Up @@ -139,7 +143,8 @@ def npm7_vuln(name: str, severity: str, fix_available, ghsa: str, source: int =


# --- harness --------------------------------------------------------------
def run_case(script: str, *, pm: str, stdout: str, exit_code: int, audit_ignore: str = "") -> tuple[int, str, str]:
def run_case(script: str, *, pm: str, stdout: str, exit_code: int, audit_ignore: str = "",
event_name: str | None = None) -> tuple[int, str, str]:
"""Execute the shipped gate text with a fake package manager on PATH."""
tmp = tempfile.mkdtemp(prefix="dependency-audit-")
try:
Expand All @@ -164,6 +169,8 @@ def run_case(script: str, *, pm: str, stdout: str, exit_code: int, audit_ignore:
"GITHUB_STEP_SUMMARY": str(summary),
"HOME": tmp,
}
if event_name is not None:
env["EVENT_NAME"] = event_name
proc = subprocess.run(
["bash", "-c", script], cwd=tmp, env=env, capture_output=True, text=True
)
Expand Down Expand Up @@ -320,6 +327,97 @@ def run_case(script: str, *, pm: str, stdout: str, exit_code: int, audit_ignore:
]


# The same findings, on the event that decides whether they block. A pull
# request only warns; every other event (unset included) fails as before.
PR_EVENTS = ("pull_request", "pull_request_target")
BLOCKING_EVENTS = ("push", "schedule", "workflow_dispatch")
for _event in PR_EVENTS:
CASES += [
(
f"{_event}: fixable high warns and passes, listing the advisory",
{"pm": "pnpm", "stdout": FIXABLE_HIGH, "exit_code": 1, "event_name": _event},
0,
["::warning::high in left-pad", "FIX AVAILABLE (>=1.2.3)", "GHSA-AAAA-BBBB-CCCC",
"do not fail this pull request"],
["::error::"],
),
(
f"{_event}: mixed tree warns on both and passes",
{"pm": "pnpm", "stdout": MIXED, "exit_code": 1, "event_name": _event},
0,
["::warning::high in left-pad", "::warning::critical in tar", "1 fixable, 1 unfixable"],
["::error::"],
),
(
f"{_event}: an audit-ignore entry without a reason warns and passes",
{"pm": "pnpm", "stdout": FIXABLE_HIGH, "exit_code": 1, "audit_ignore": "GHSA-aaaa-bbbb-cccc",
"event_name": _event},
0,
["::warning::audit-ignore entry", "has no reason"],
["::error::"],
),
(
f"{_event}: an empty report (audit did not run) warns and passes",
{"pm": "pnpm", "stdout": "", "exit_code": 0, "event_name": _event},
0,
["::warning::dependency audit produced no report", "did not run on this pull request"],
["::error::"],
),
(
f"{_event}: a non-JSON report warns and passes",
{"pm": "pnpm", "stdout": "ERR_PNPM_AUDIT_ENDPOINT_UNAVAILABLE\n", "exit_code": 1, "event_name": _event},
0,
["::warning::dependency audit report is not JSON"],
["::error::"],
),
(
f"{_event}: an unrecognised report shape warns and passes",
{"pm": "pnpm", "stdout": UNKNOWN_SHAPE, "exit_code": 0, "event_name": _event},
0,
["::warning::dependency audit report has neither"],
["::error::"],
),
(
f"{_event}: npm 7+ fixable high warns and passes",
{"pm": "npm", "stdout": NPM7_FIXABLE_HIGH, "exit_code": 1, "event_name": _event},
0,
["::warning::high in axios", "FIX AVAILABLE (axios@1.7.4)"],
["::error::"],
),
(
f"{_event}: a clean report still says nothing alarming",
{"pm": "pnpm", "stdout": CLEAN_PNPM, "exit_code": 0, "event_name": _event},
0,
["0 fixable, 0 unfixable, 0 suppressed"],
["::error::", "::warning::"],
),
]
for _event in BLOCKING_EVENTS:
CASES += [
(
f"{_event}: fixable high still fails",
{"pm": "pnpm", "stdout": FIXABLE_HIGH, "exit_code": 1, "event_name": _event},
1,
["::error::high in left-pad", "FIX AVAILABLE (>=1.2.3)"],
["do not fail this pull request"],
),
(
f"{_event}: an empty report (audit did not run) still fails",
{"pm": "pnpm", "stdout": "", "exit_code": 0, "event_name": _event},
1,
["::error::dependency audit produced no report"],
[],
),
(
f"{_event}: a non-JSON report still fails",
{"pm": "pnpm", "stdout": "ERR_PNPM_AUDIT_ENDPOINT_UNAVAILABLE\n", "exit_code": 1, "event_name": _event},
1,
["::error::dependency audit report is not JSON"],
[],
),
]


def check_shipped_flags(script: str) -> list[str]:
"""The gate must ask for JSON and must not let the tool's own exit status decide."""
problems = []
Expand Down
Loading
Loading