From 595d884364e09c107183994e5889fd80dc6ed890 Mon Sep 17 00:00:00 2001 From: Felipe Fujiy Pessoto Date: Sat, 15 Aug 2026 01:56:58 +0000 Subject: [PATCH 1/9] [GLUTEN-12743][CI] Let contributors trigger the Delta Spark UT with a /delta-test PR comment The Delta Spark UT pipeline (#12388) only runs per-PR when the `paths:` filter matches. A Velox/core/native change does not match it but can still break Delta offload, and there was no way for a contributor to force a run. A `run-delta-ci` label was considered and dropped: changing labels needs write/triage permission, so a PR author working from a fork -- exactly the person who needs it -- cannot apply one, and a label cannot be expressed as a `paths:` filter, so it would need a gate job on every PR. Use an `issue_comment` slash command instead, as velox_backend_ansi.yml already does for `/ansi-test`. Anyone can comment, so the PR author can opt their own PR in; authorised commenters are the PR author or anyone with write access. It is an ADDITIONAL `on:` key, so the `paths:` filter is untouched and PRs that do not ask for a run still cost zero jobs. `startsWith`, not `contains`, so quoting the command while discussing it does not spend ~11 job-hours. An `issue_comment` run is created against the default branch, so `env. DELTA_CHECKOUT_REF` resolves the PR's merge ref and every job checks it out; it is empty on every other event, which is exactly actions/checkout's default, so their behaviour is unchanged. Such a run is also not attached to the PR's Checks tab, so the gate job replies with a link. `update_baseline` remains reachable only from `workflow_dispatch`, so no comment can rewrite the committed baseline. Cache writes are scoped to the run's GITHUB_REF, which for `issue_comment` is the default branch rather than the PR. Since these jobs build and execute code from the PR, comment-triggered runs now restore the ccache / Maven / sbt caches but never save to them, so a PR cannot plant an entry that the nightly on main later restores. Workflow permissions are pinned to `contents: read`, with `pull-requests: write` granted only to the gate job, which never checks out PR code. Generated-by: GitHub Copilot CLI Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/workflows/delta_spark_ut.yml | 96 +++++++++++++++++-- .../workflows/util/delta-spark-ut/README.md | 16 +++- 2 files changed, 105 insertions(+), 7 deletions(-) diff --git a/.github/workflows/delta_spark_ut.yml b/.github/workflows/delta_spark_ut.yml index c0fc1211a5..3dfcaab7bd 100644 --- a/.github/workflows/delta_spark_ut.yml +++ b/.github/workflows/delta_spark_ut.yml @@ -60,6 +60,16 @@ on: # Negated patterns are evaluated in order and override the positives # above, so this must stay last. - '!.github/workflows/util/delta-spark-ut/**/*.md' + # Escape hatch from the `paths:` filter: comment `/delta-test` on a PR to force + # a full run. A Velox/core/native change does not match the filter but can + # still break Delta offload, and the nightly is not always soon enough. + # + # A comment rather than a label, because labelling needs write/triage + # permission -- a PR author on a fork, the person who most needs this, cannot + # apply one. And an ADDITIONAL `on:` key rather than a change to the trigger + # above, so PRs that do not ask for a run still cost zero jobs. + issue_comment: + types: [created] workflow_dispatch: inputs: delta_ref: @@ -92,10 +102,23 @@ on: schedule: - cron: '0 5 * * *' +# Unlike `pull_request`, an `issue_comment` run is a base-repo run with a +# base-repo token, and the jobs below build and execute code from the PR. Nothing +# here needs write access, so pin the default to read-only. +permissions: + contents: read + env: ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION: true MVN_CMD: 'build/mvn -ntp' CCACHE_DIR: "${{ github.workspace }}/.ccache" + # What every job below checks out. Empty on all events except `issue_comment`, + # whose run is created against the DEFAULT BRANCH -- so without this a + # `/delta-test` comment would test main, not the PR. An empty `ref` is exactly + # what actions/checkout does by default, so the other events are unaffected. + # The merge ref is what `pull_request` tests too; a conflicted PR has none, and + # checkout then fails with a clear git error. + DELTA_CHECKOUT_REF: ${{ github.event.issue.number && format('refs/pull/{0}/merge', github.event.issue.number) || '' }} # Gluten profile / bundle naming for the build-gluten-bundle and # delta-spark-test jobs. `spark_version` is the single source of truth for the # Spark version: it drives the Gluten bundle profile (-Pspark-), the bundle @@ -139,19 +162,63 @@ env: # Now that this is a standalone workflow (not called by velox_backend_x86.yml), # `github.workflow` resolves to THIS workflow, so a concurrency group is both safe # and necessary: without it every push to a PR branch stacks another full ~2.5 h -# Delta run instead of superseding the previous one. Keyed on the branch for PRs -# and the sha otherwise, matching velox_backend_x86.yml's group. +# Delta run instead of superseding the previous one. Keyed on the PR number when +# there is one, so a `pull_request` run and a `/delta-test` run on the same PR +# supersede each other -- `github.head_ref` is empty on `issue_comment`, which +# would otherwise lump every commented-on PR into one shared group. concurrency: - group: ${{ github.repository }}-${{ github.head_ref || github.sha }}-${{ github.workflow }} + group: ${{ github.repository }}-${{ github.event.pull_request.number || github.event.issue.number || github.head_ref || github.sha }}-${{ github.workflow }} cancel-in-progress: true jobs: + # Gate + acknowledgement for `/delta-test`, and nothing else: the ref to check + # out is computed in `env.DELTA_CHECKOUT_REF` above, so this job has no outputs + # and the rest of the pipeline just hangs off it via `needs`. + # + # Every other event passes straight through (the `if` short-circuits), leaving + # their behaviour unchanged. + # + # Kept as its own job so that `pull-requests: write` -- needed to comment back + # -- is never granted to a job that builds and runs the PR's code. + delta-test-requested: + # The PR author (the whole point: a fork author cannot label their own PR, + # but can always comment on it) or anyone with write access. `startsWith`, + # not `contains`, so quoting the command while discussing it does not spend + # ~11 job-hours. + if: >- + github.event_name != 'issue_comment' || + (github.event.issue.pull_request && + startsWith(github.event.comment.body, '/delta-test') && + (github.event.comment.user.login == github.event.issue.user.login || + contains(fromJSON('["OWNER","MEMBER","COLLABORATOR"]'), github.event.comment.author_association))) + runs-on: ubuntu-22.04 + permissions: + pull-requests: write + steps: + # An `issue_comment` run belongs to the default branch, so GitHub cannot + # attach it to the PR's Checks tab. Leave a link, or the contributor sees + # nothing happen for ~2.5 h. + - name: Acknowledge the request + if: ${{ github.event_name == 'issue_comment' }} + env: + GH_TOKEN: ${{ github.token }} + REPO: ${{ github.repository }} + PR_NUMBER: ${{ github.event.issue.number }} + TRIGGER_USER: ${{ github.event.comment.user.login }} + RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + run: | + gh pr comment "$PR_NUMBER" --repo "$REPO" --body \ + "🔄 Delta Spark UT started by @${TRIGGER_USER} (~2.5 h). [View run](${RUN_URL})" + build-native-lib-centos-7: + needs: delta-test-requested # This workflow always builds its own native lib -- there is no caller to # provide one, and the `paths:` filter already decided whether the run happens. runs-on: ubuntu-22.04 steps: - uses: actions/checkout@v4 + with: + ref: ${{ env.DELTA_CHECKOUT_REF }} - name: Get Ccache from Apache Stash uses: apache/infrastructure-actions/stash/restore@0ba14156c9f4c3cfbe4b0c9f36339ab0f8d81e53 with: @@ -172,7 +239,15 @@ jobs: ccache -s " - name: Save Ccache to Apache Stash - if: always() + # A stash is saved as an artifact named `-`, and is + # restored by matching `head_branch` + `head_repository_id` -- so the + # scope is the branch, exactly as with actions/cache. On `issue_comment` + # that branch is the DEFAULT BRANCH, not the PR, and these jobs execute + # the PR's code; with the save action's `overwrite: true` default a + # comment-triggered run would *replace* main's stash, which the nightly + # then restores. So comment runs restore but never save. Same for the + # Maven and sbt stashes below. + if: ${{ always() && github.event_name != 'issue_comment' }} uses: apache/infrastructure-actions/stash/save@0ba14156c9f4c3cfbe4b0c9f36339ab0f8d81e53 with: path: '${{ env.CCACHE_DIR }}' @@ -190,6 +265,8 @@ jobs: container: apache/gluten:centos-9-jdk17 steps: - uses: actions/checkout@v4 + with: + ref: ${{ env.DELTA_CHECKOUT_REF }} - name: Download native artifacts uses: actions/download-artifact@v4 with: @@ -232,6 +309,8 @@ jobs: -Pbackends-velox -Pdelta \ -DskipTests -Dmaven.compiler.release=17 - name: Save Maven repository to Apache Stash + # Trusted runs only -- see the Save Ccache step above. + if: ${{ github.event_name != 'issue_comment' }} uses: apache/infrastructure-actions/stash/save@0ba14156c9f4c3cfbe4b0c9f36339ab0f8d81e53 with: path: /root/.m2/repository @@ -282,6 +361,8 @@ jobs: SHARD_ID: ${{ matrix.shard }} steps: - uses: actions/checkout@v4 + with: + ref: ${{ env.DELTA_CHECKOUT_REF }} - name: Resolve workflow inputs id: resolve @@ -394,8 +475,9 @@ jobs: - name: Save sbt / Ivy / Coursier to Apache Stash # All shards have the same dependencies; one writer avoids matrix jobs - # overwriting the same stash artifact. - if: ${{ success() && matrix.shard == 0 }} + # overwriting the same stash artifact. Trusted runs only -- see the Save + # Ccache step in build-native-lib-centos-7. + if: ${{ success() && matrix.shard == 0 && github.event_name != 'issue_comment' }} uses: apache/infrastructure-actions/stash/save@0ba14156c9f4c3cfbe4b0c9f36339ab0f8d81e53 with: path: | @@ -478,6 +560,8 @@ jobs: runs-on: ubuntu-22.04 steps: - uses: actions/checkout@v4 + with: + ref: ${{ env.DELTA_CHECKOUT_REF }} - name: Download per-shard gate lists uses: actions/download-artifact@v4 continue-on-error: true diff --git a/.github/workflows/util/delta-spark-ut/README.md b/.github/workflows/util/delta-spark-ut/README.md index b3119d7d39..aadd6d3108 100644 --- a/.github/workflows/util/delta-spark-ut/README.md +++ b/.github/workflows/util/delta-spark-ut/README.md @@ -97,12 +97,26 @@ longer shared between them, those PRs pay for the centos-7 native build twice filter before creating the run, so an unrelated PR costs nothing at all. Changes to general Velox/core/native code can also affect Delta offload, but they're touched on most PRs, so per-PR they skip the suite — the nightly run is - the safety net. + the safety net, and `/delta-test` below forces a run on any PR the filter + skipped. - **Nightly** — the **full** suite runs against the latest default branch on a `schedule` (05:00 UTC), so regressions from general Velox/core changes are still caught daily. The nightly run enforces the baseline **and** fails on now-passing tests (`fail_on_fixed=true`), so baseline drift surfaces as a red nightly — the signal to refresh `known-failures.txt`. +- **On demand, from a PR** — comment **`/delta-test`** (as the first thing in the + comment) to force a full run on a PR the `paths:` filter skipped. The **PR + author** — including from a fork, which is why this is a comment and not a + label — or anyone with write access can use it. It runs the PR's merge ref with + the default settings and the baseline **enforced**; `update_baseline` stays + reachable only from `workflow_dispatch`, so no comment can rewrite the + baseline. The workflow replies with a link to the run, because an + `issue_comment` run belongs to the default branch and so cannot appear in the + PR's Checks tab. Two consequences: `/delta-test` reads the workflow file from + the default branch (so it cannot test changes to this pipeline itself — those + match the `paths:` filter anyway), and comment-triggered runs **restore but + never save** the ccache/Maven/sbt caches, since their cache scope is the + default branch and they execute the PR's code. - **Manually** — **Actions → Delta Spark UT (Gluten) → Run workflow** (`workflow_dispatch`), e.g. to refresh the baseline (see below). This is also how you validate a Velox/core change against Delta before merging: run it on From 4e2b7d14c230674c0c0ce7b1282a5e1a73e69a00 Mon Sep 17 00:00:00 2001 From: Felipe Fujiy Pessoto Date: Sat, 15 Aug 2026 02:24:12 +0000 Subject: [PATCH 2/9] [GLUTEN-12743][CI] Restrict /delta-test to the PR author Addresses review feedback that the `author_association` allow-list is not equivalent to write access. On an ASF repo it is worse than imprecise: write access comes from Gitbox, not from GitHub org/collaborator membership, so `COLLABORATOR` matches nobody on apache/gluten and 16 of the 18 committers sampled from recent PR review comments report `CONTRIBUTOR`. An OWNER/MEMBER/COLLABORATOR list would therefore have rejected nearly every maintainer while accepting any member of the apache org. Drop the list rather than reword it. `/delta-test` now requires the commenter to be the PR author, which is precisely the gap this closes -- a fork author cannot label their own PR but can always comment on it -- and is exactly describable, so the workflow comment and README no longer overstate who may trigger a run. A maintainer who wants a run on someone else's PR asks the author to comment. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/workflows/delta_spark_ut.yml | 27 ++++++++++++------- .../workflows/util/delta-spark-ut/README.md | 25 ++++++++--------- 2 files changed, 31 insertions(+), 21 deletions(-) diff --git a/.github/workflows/delta_spark_ut.yml b/.github/workflows/delta_spark_ut.yml index 3dfcaab7bd..25bd63a0a5 100644 --- a/.github/workflows/delta_spark_ut.yml +++ b/.github/workflows/delta_spark_ut.yml @@ -60,9 +60,9 @@ on: # Negated patterns are evaluated in order and override the positives # above, so this must stay last. - '!.github/workflows/util/delta-spark-ut/**/*.md' - # Escape hatch from the `paths:` filter: comment `/delta-test` on a PR to force - # a full run. A Velox/core/native change does not match the filter but can - # still break Delta offload, and the nightly is not always soon enough. + # Escape hatch from the `paths:` filter: the PR author comments `/delta-test` + # to force a full run. A Velox/core/native change does not match the filter but + # can still break Delta offload, and the nightly is not always soon enough. # # A comment rather than a label, because labelling needs write/triage # permission -- a PR author on a fork, the person who most needs this, cannot @@ -181,16 +181,25 @@ jobs: # Kept as its own job so that `pull-requests: write` -- needed to comment back # -- is never granted to a job that builds and runs the PR's code. delta-test-requested: - # The PR author (the whole point: a fork author cannot label their own PR, - # but can always comment on it) or anyone with write access. `startsWith`, - # not `contains`, so quoting the command while discussing it does not spend - # ~11 job-hours. + # Only the PR author may spend a run on their own PR -- which is exactly the + # gap this closes: a fork author cannot label their own PR, but can always + # comment on it. + # + # Deliberately NOT an `author_association` allow-list for maintainers: on an + # ASF repo write access comes from Gitbox, not from GitHub org/collaborator + # membership, so `COLLABORATOR` matches nobody here and all but two of the + # Gluten committers report `CONTRIBUTOR` -- an OWNER/MEMBER/COLLABORATOR list + # would reject nearly every maintainer while accepting any apache org member. + # A maintainer who wants a run on someone else's PR asks the author to + # comment. + # + # `startsWith`, not `contains`, so quoting the command while discussing it + # does not spend ~11 job-hours. if: >- github.event_name != 'issue_comment' || (github.event.issue.pull_request && startsWith(github.event.comment.body, '/delta-test') && - (github.event.comment.user.login == github.event.issue.user.login || - contains(fromJSON('["OWNER","MEMBER","COLLABORATOR"]'), github.event.comment.author_association))) + github.event.comment.user.login == github.event.issue.user.login) runs-on: ubuntu-22.04 permissions: pull-requests: write diff --git a/.github/workflows/util/delta-spark-ut/README.md b/.github/workflows/util/delta-spark-ut/README.md index aadd6d3108..c7ffc42c62 100644 --- a/.github/workflows/util/delta-spark-ut/README.md +++ b/.github/workflows/util/delta-spark-ut/README.md @@ -104,18 +104,19 @@ longer shared between them, those PRs pay for the centos-7 native build twice still caught daily. The nightly run enforces the baseline **and** fails on now-passing tests (`fail_on_fixed=true`), so baseline drift surfaces as a red nightly — the signal to refresh `known-failures.txt`. -- **On demand, from a PR** — comment **`/delta-test`** (as the first thing in the - comment) to force a full run on a PR the `paths:` filter skipped. The **PR - author** — including from a fork, which is why this is a comment and not a - label — or anyone with write access can use it. It runs the PR's merge ref with - the default settings and the baseline **enforced**; `update_baseline` stays - reachable only from `workflow_dispatch`, so no comment can rewrite the - baseline. The workflow replies with a link to the run, because an - `issue_comment` run belongs to the default branch and so cannot appear in the - PR's Checks tab. Two consequences: `/delta-test` reads the workflow file from - the default branch (so it cannot test changes to this pipeline itself — those - match the `paths:` filter anyway), and comment-triggered runs **restore but - never save** the ccache/Maven/sbt caches, since their cache scope is the +- **On demand, from a PR** — the **PR author** comments **`/delta-test`** (as the + first thing in the comment) to force a full run on a PR the `paths:` filter + skipped. Restricted to the author, who is the one this exists for: a fork + author cannot label their own PR, but can always comment on it. A maintainer + who wants a run on someone else's PR asks the author to comment. It runs the + PR's merge ref with the default settings and the baseline **enforced**; + `update_baseline` stays reachable only from `workflow_dispatch`, so no comment + can rewrite the baseline. The workflow replies with a link to the run, because + an `issue_comment` run belongs to the default branch and so cannot appear in + the PR's Checks tab. Two consequences: `/delta-test` reads the workflow file + from the default branch (so it cannot test changes to this pipeline itself — + those match the `paths:` filter anyway), and comment-triggered runs **restore + but never save** the ccache/Maven/sbt caches, since their cache scope is the default branch and they execute the PR's code. - **Manually** — **Actions → Delta Spark UT (Gluten) → Run workflow** (`workflow_dispatch`), e.g. to refresh the baseline (see below). This is also From 248f4f5df48cf55e4e3a29c19af47a75cc955a7b Mon Sep 17 00:00:00 2001 From: Felipe Fujiy Pessoto Date: Sat, 15 Aug 2026 03:55:56 +0000 Subject: [PATCH 3/9] [GLUTEN-12743][CI] Don't let the /delta-test acknowledgement fail the run The whole pipeline hangs off `delta-test-requested` via `needs`, so a transient `gh pr comment` failure in its acknowledgement step would fail the gate job and skip every downstream job -- silently dropping an authorised run. The comment is informational only, so mark the step `continue-on-error: true`. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/workflows/delta_spark_ut.yml | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/.github/workflows/delta_spark_ut.yml b/.github/workflows/delta_spark_ut.yml index 25bd63a0a5..b2373cdde7 100644 --- a/.github/workflows/delta_spark_ut.yml +++ b/.github/workflows/delta_spark_ut.yml @@ -209,6 +209,10 @@ jobs: # nothing happen for ~2.5 h. - name: Acknowledge the request if: ${{ github.event_name == 'issue_comment' }} + # Purely informational: this job is what the whole pipeline hangs off, so + # a transient API failure here must not fail it and silently skip an + # authorised ~2.5 h run. + continue-on-error: true env: GH_TOKEN: ${{ github.token }} REPO: ${{ github.repository }} From 8911f45fe6607e94dd503931ef94e217c1e9fbe3 Mon Sep 17 00:00:00 2001 From: Felipe Fujiy Pessoto Date: Wed, 19 Aug 2026 17:06:38 +0000 Subject: [PATCH 4/9] [GLUTEN-12743][CI] Adapt /delta-test to the Apache Stash cache migration Follow-up to rebasing onto #12820, which replaced actions/cache with Apache Stash. Two things the migration changes for this feature: 1. `permissions:` must grant `actions: read`. Declaring the block at all sets every unlisted scope to `none`, and the Stash restore action reads caches through the artifacts REST API (`gh api repos/.../actions/artifacts` and `gh run download`), which 403s without it. The other Stash-using workflows declare no `permissions:` block and inherit the repo default, so they never had to say this. Without this the Stash restores would fail on every event, not just on comment runs. 2. The "restore but never save" guards still apply, and matter more. A stash is an artifact named `-`, restored by matching `head_branch` + `head_repository_id` -- the same branch scoping actions/cache has. On `issue_comment` that branch is the default branch, so a save from a run executing PR code would land on `main`, and Stash's `overwrite: true` default means it replaces the existing entry rather than merely competing with it. #12820 also moved the ccache to the key `ccache-centos7-release-default-${{ hashFiles('ep/build-velox/src/**') }}`, which velox_backend_x86.yml restores from as well, so an unguarded save would reach beyond this pipeline. Reads are deliberately left unguarded: a comment run restores main's stashes and is therefore no slower than any other run. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/workflows/delta_spark_ut.yml | 8 ++++++++ .github/workflows/util/delta-spark-ut/README.md | 9 +++++++-- 2 files changed, 15 insertions(+), 2 deletions(-) diff --git a/.github/workflows/delta_spark_ut.yml b/.github/workflows/delta_spark_ut.yml index b2373cdde7..a168421e9c 100644 --- a/.github/workflows/delta_spark_ut.yml +++ b/.github/workflows/delta_spark_ut.yml @@ -105,8 +105,16 @@ on: # Unlike `pull_request`, an `issue_comment` run is a base-repo run with a # base-repo token, and the jobs below build and execute code from the PR. Nothing # here needs write access, so pin the default to read-only. +# +# `actions: read` is required, not optional: setting this block at all sets every +# unlisted scope to `none`, and the Apache Stash restore action reads the caches +# through the artifacts REST API (`gh api repos/.../actions/artifacts` plus +# `gh run download`), which 403s without it. The other Stash-using workflows here +# declare no `permissions:` block and so inherit the repo default, which is why +# they do not need to say this. permissions: contents: read + actions: read env: ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION: true diff --git a/.github/workflows/util/delta-spark-ut/README.md b/.github/workflows/util/delta-spark-ut/README.md index c7ffc42c62..b6f105351b 100644 --- a/.github/workflows/util/delta-spark-ut/README.md +++ b/.github/workflows/util/delta-spark-ut/README.md @@ -116,8 +116,13 @@ longer shared between them, those PRs pay for the centos-7 native build twice the PR's Checks tab. Two consequences: `/delta-test` reads the workflow file from the default branch (so it cannot test changes to this pipeline itself — those match the `paths:` filter anyway), and comment-triggered runs **restore - but never save** the ccache/Maven/sbt caches, since their cache scope is the - default branch and they execute the PR's code. + but never save** the ccache/Maven/sbt stashes. A stash is stored as an + artifact named `-` and looked up by branch, and for an + `issue_comment` run that branch is the **default branch**, not the PR — so a + save would overwrite (Stash defaults to `overwrite: true`) a stash that the + nightly restores. The ccache key is shared with `velox_backend_x86.yml`, so + that would reach beyond this pipeline. Reads are unaffected: a comment run + still restores `main`'s stashes, so it is no slower. - **Manually** — **Actions → Delta Spark UT (Gluten) → Run workflow** (`workflow_dispatch`), e.g. to refresh the baseline (see below). This is also how you validate a Velox/core change against Delta before merging: run it on From dfe2e66cdf7d3109ab25d3f4a80d577fa307cb11 Mon Sep 17 00:00:00 2001 From: Felipe Fujiy Pessoto Date: Wed, 19 Aug 2026 19:42:17 +0000 Subject: [PATCH 5/9] [GLUTEN-12743][CI] Require /delta-test to be an exact command token Addresses review feedback: `startsWith(body, '/delta-test')` also matches `/delta-test-arm` and `/delta-testers`, so a longer command that merely starts the same way -- or a future `/delta-test-` -- would spend ~11 job-hours on this pipeline by accident. GHA expressions have no regex, so spell out the ways the token can legally end: end-of-body, a space, or a newline. `fromJSON('"\r"')` / `fromJSON('"\n"')` is the only way to express a control character in an expression; GitHub stores comment bodies with CRLF and the API can deliver bare LF, so both are matched. This keeps multi-line comments working (`/delta-test` on its own first line) while rejecting the longer-prefix cases. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/workflows/delta_spark_ut.yml | 15 ++++++++++++--- .github/workflows/util/delta-spark-ut/README.md | 9 ++++++--- 2 files changed, 18 insertions(+), 6 deletions(-) diff --git a/.github/workflows/delta_spark_ut.yml b/.github/workflows/delta_spark_ut.yml index a168421e9c..5b02822c16 100644 --- a/.github/workflows/delta_spark_ut.yml +++ b/.github/workflows/delta_spark_ut.yml @@ -201,12 +201,21 @@ jobs: # A maintainer who wants a run on someone else's PR asks the author to # comment. # - # `startsWith`, not `contains`, so quoting the command while discussing it - # does not spend ~11 job-hours. + # The command must be the FIRST TOKEN of the comment, so neither quoting it + # while discussing it (`contains` would match) nor a longer command that + # merely starts the same way (`/delta-test-arm`, `/delta-testers` -- plain + # `startsWith` would match both) can spend ~11 job-hours. GHA expressions + # have no regex, so spell out the four ways the token can legally end: + # end-of-body, a space, or a newline (`fromJSON` is the only way to write a + # control character in an expression; GitHub stores comment bodies CRLF, and + # bare LF arrives via the API). if: >- github.event_name != 'issue_comment' || (github.event.issue.pull_request && - startsWith(github.event.comment.body, '/delta-test') && + (github.event.comment.body == '/delta-test' || + startsWith(github.event.comment.body, '/delta-test ') || + startsWith(github.event.comment.body, format('/delta-test{0}', fromJSON('"\r"'))) || + startsWith(github.event.comment.body, format('/delta-test{0}', fromJSON('"\n"')))) && github.event.comment.user.login == github.event.issue.user.login) runs-on: ubuntu-22.04 permissions: diff --git a/.github/workflows/util/delta-spark-ut/README.md b/.github/workflows/util/delta-spark-ut/README.md index b6f105351b..59d536e48f 100644 --- a/.github/workflows/util/delta-spark-ut/README.md +++ b/.github/workflows/util/delta-spark-ut/README.md @@ -104,9 +104,12 @@ longer shared between them, those PRs pay for the centos-7 native build twice still caught daily. The nightly run enforces the baseline **and** fails on now-passing tests (`fail_on_fixed=true`), so baseline drift surfaces as a red nightly — the signal to refresh `known-failures.txt`. -- **On demand, from a PR** — the **PR author** comments **`/delta-test`** (as the - first thing in the comment) to force a full run on a PR the `paths:` filter - skipped. Restricted to the author, who is the one this exists for: a fork +- **On demand, from a PR** — the **PR author** comments **`/delta-test`** to + force a full run on a PR the `paths:` filter skipped. The command must be the + comment's first token — it may be followed by a space or a newline and then + any text, but `/delta-test-arm` or `/delta-testers` will *not* match, so a + future command that starts the same way cannot fire this one by accident. + Restricted to the author, who is the one this exists for: a fork author cannot label their own PR, but can always comment on it. A maintainer who wants a run on someone else's PR asks the author to comment. It runs the PR's merge ref with the default settings and the baseline **enforced**; From 239c929043a579726609168682905f1a9351fc5c Mon Sep 17 00:00:00 2001 From: Felipe Fujiy Pessoto Date: Wed, 19 Aug 2026 23:39:35 +0000 Subject: [PATCH 6/9] [GLUTEN-12743][CI] Document why /delta-test is author-scoped Comments only; no logic change. Records why this trigger is stricter than the repo's other comment triggers (velox_backend_ansi.yml's /ansi-test and take.yml gate on nothing at all), and why the accurate maintainer check -- collaborators/{user}/permission -- is deliberately absent: maintainers can already start the suite via workflow_dispatch, so the PR author is the only party that had no way in, which is the gap #12743 describes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> EOF --- .github/workflows/delta_spark_ut.yml | 14 ++++++++++++-- .github/workflows/util/delta-spark-ut/README.md | 6 ++++-- 2 files changed, 16 insertions(+), 4 deletions(-) diff --git a/.github/workflows/delta_spark_ut.yml b/.github/workflows/delta_spark_ut.yml index 5b02822c16..ea1cc1a37e 100644 --- a/.github/workflows/delta_spark_ut.yml +++ b/.github/workflows/delta_spark_ut.yml @@ -198,8 +198,18 @@ jobs: # membership, so `COLLABORATOR` matches nobody here and all but two of the # Gluten committers report `CONTRIBUTOR` -- an OWNER/MEMBER/COLLABORATOR list # would reject nearly every maintainer while accepting any apache org member. - # A maintainer who wants a run on someone else's PR asks the author to - # comment. + # + # Nor is there a `collaborators/{user}/permission` lookup, which WOULD be + # accurate: maintainers do not need one. They can already start this suite + # from Actions -> Run workflow (`workflow_dispatch`), which is precisely why + # the trigger is author-scoped -- the author is the only party that had no + # way in (labelling a PR needs write/triage, so a fork author cannot). A + # maintainer who wants a run on someone else's PR uses workflow_dispatch or + # asks the author to comment. + # + # This is stricter than the repo's other comment triggers -- velox_backend_ansi.yml + # (`/ansi-test`) and take.yml gate on nothing at all -- which is deliberate, + # given a run here costs ~11 job-hours. # # The command must be the FIRST TOKEN of the comment, so neither quoting it # while discussing it (`contains` would match) nor a longer command that diff --git a/.github/workflows/util/delta-spark-ut/README.md b/.github/workflows/util/delta-spark-ut/README.md index 59d536e48f..f44318d761 100644 --- a/.github/workflows/util/delta-spark-ut/README.md +++ b/.github/workflows/util/delta-spark-ut/README.md @@ -110,8 +110,10 @@ longer shared between them, those PRs pay for the centos-7 native build twice any text, but `/delta-test-arm` or `/delta-testers` will *not* match, so a future command that starts the same way cannot fire this one by accident. Restricted to the author, who is the one this exists for: a fork - author cannot label their own PR, but can always comment on it. A maintainer - who wants a run on someone else's PR asks the author to comment. It runs the + author cannot label their own PR, but can always comment on it, whereas a + maintainer can already start the suite from **Run workflow** + (`workflow_dispatch`). A maintainer who wants a run on someone else's PR uses + that, or asks the author to comment. It runs the PR's merge ref with the default settings and the baseline **enforced**; `update_baseline` stays reachable only from `workflow_dispatch`, so no comment can rewrite the baseline. The workflow replies with a link to the run, because From ea6e9ef200ec9e9309a15ecff774275e72c615ff Mon Sep 17 00:00:00 2001 From: Felipe Fujiy Pessoto Date: Thu, 20 Aug 2026 00:46:02 +0000 Subject: [PATCH 7/9] [GLUTEN-12743][CI] Let anyone trigger /delta-test, matching /ansi-test Drop the PR-author restriction so this trigger behaves like the repo's existing comment triggers: velox_backend_ansi.yml (`/ansi-test`, `/ansi-analyze`) and take.yml both gate on the command alone, with no author or permission check. Being the only workflow in the repo with an authorization gate was inconsistent, and it blocked the case where a reviewer wants the Delta suite run against a contributor's PR before merging. The exact-token match is deliberately kept rather than following `/ansi-test`'s `contains()`: matching a mention mid-sentence, or a longer command such as a future `/delta-test-arm`, was raised in review and would spend ~11 job-hours by accident. This also promotes the "restore but never save" cache guards from belt-and-braces to load bearing, since any user can now start a run that builds and executes the PR's code. They were already in place and are unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/workflows/delta_spark_ut.yml | 57 ++++++++----------- .../workflows/util/delta-spark-ut/README.md | 10 ++-- 2 files changed, 27 insertions(+), 40 deletions(-) diff --git a/.github/workflows/delta_spark_ut.yml b/.github/workflows/delta_spark_ut.yml index ea1cc1a37e..17895c9747 100644 --- a/.github/workflows/delta_spark_ut.yml +++ b/.github/workflows/delta_spark_ut.yml @@ -60,14 +60,15 @@ on: # Negated patterns are evaluated in order and override the positives # above, so this must stay last. - '!.github/workflows/util/delta-spark-ut/**/*.md' - # Escape hatch from the `paths:` filter: the PR author comments `/delta-test` - # to force a full run. A Velox/core/native change does not match the filter but + # Escape hatch from the `paths:` filter: comment `/delta-test` on a PR to force + # a full run. A Velox/core/native change does not match the filter but # can still break Delta offload, and the nightly is not always soon enough. # # A comment rather than a label, because labelling needs write/triage # permission -- a PR author on a fork, the person who most needs this, cannot - # apply one. And an ADDITIONAL `on:` key rather than a change to the trigger - # above, so PRs that do not ask for a run still cost zero jobs. + # apply one. Same mechanism as velox_backend_ansi.yml's `/ansi-test`. And an + # ADDITIONAL `on:` key rather than a change to the trigger above, so PRs that + # do not ask for a run still cost zero jobs. issue_comment: types: [created] workflow_dispatch: @@ -189,44 +190,32 @@ jobs: # Kept as its own job so that `pull-requests: write` -- needed to comment back # -- is never granted to a job that builds and runs the PR's code. delta-test-requested: - # Only the PR author may spend a run on their own PR -- which is exactly the - # gap this closes: a fork author cannot label their own PR, but can always - # comment on it. + # Anyone may ask for a run, matching velox_backend_ansi.yml's `/ansi-test` + # (and take.yml), which are the repo's existing comment triggers and gate on + # the command alone. The PR author is the party this exists for -- labelling + # a PR needs write/triage, so a fork author cannot opt their own PR in -- + # but there is no reason to be stricter here than the pipeline next door. # - # Deliberately NOT an `author_association` allow-list for maintainers: on an - # ASF repo write access comes from Gitbox, not from GitHub org/collaborator - # membership, so `COLLABORATOR` matches nobody here and all but two of the - # Gluten committers report `CONTRIBUTOR` -- an OWNER/MEMBER/COLLABORATOR list - # would reject nearly every maintainer while accepting any apache org member. + # Note this makes the "restore but never save" cache guards below load + # bearing rather than belt-and-braces: any user can now start a run that + # builds and executes the PR's code. # - # Nor is there a `collaborators/{user}/permission` lookup, which WOULD be - # accurate: maintainers do not need one. They can already start this suite - # from Actions -> Run workflow (`workflow_dispatch`), which is precisely why - # the trigger is author-scoped -- the author is the only party that had no - # way in (labelling a PR needs write/triage, so a fork author cannot). A - # maintainer who wants a run on someone else's PR uses workflow_dispatch or - # asks the author to comment. - # - # This is stricter than the repo's other comment triggers -- velox_backend_ansi.yml - # (`/ansi-test`) and take.yml gate on nothing at all -- which is deliberate, - # given a run here costs ~11 job-hours. - # - # The command must be the FIRST TOKEN of the comment, so neither quoting it - # while discussing it (`contains` would match) nor a longer command that - # merely starts the same way (`/delta-test-arm`, `/delta-testers` -- plain - # `startsWith` would match both) can spend ~11 job-hours. GHA expressions - # have no regex, so spell out the four ways the token can legally end: - # end-of-body, a space, or a newline (`fromJSON` is the only way to write a - # control character in an expression; GitHub stores comment bodies CRLF, and - # bare LF arrives via the API). + # The command must still be the FIRST TOKEN of the comment, so neither + # quoting it while discussing it (`/ansi-test` uses `contains`, which does + # match a mention mid-sentence) nor a longer command that merely starts the + # same way (`/delta-test-arm`, `/delta-testers` -- plain `startsWith` would + # match both) can spend ~11 job-hours by accident. GHA expressions have no + # regex, so spell out the ways the token can legally end: end-of-body, a + # space, or a newline (`fromJSON` is the only way to write a control + # character in an expression; GitHub stores comment bodies CRLF, and bare LF + # arrives via the API). if: >- github.event_name != 'issue_comment' || (github.event.issue.pull_request && (github.event.comment.body == '/delta-test' || startsWith(github.event.comment.body, '/delta-test ') || startsWith(github.event.comment.body, format('/delta-test{0}', fromJSON('"\r"'))) || - startsWith(github.event.comment.body, format('/delta-test{0}', fromJSON('"\n"')))) && - github.event.comment.user.login == github.event.issue.user.login) + startsWith(github.event.comment.body, format('/delta-test{0}', fromJSON('"\n"'))))) runs-on: ubuntu-22.04 permissions: pull-requests: write diff --git a/.github/workflows/util/delta-spark-ut/README.md b/.github/workflows/util/delta-spark-ut/README.md index f44318d761..5f242bec47 100644 --- a/.github/workflows/util/delta-spark-ut/README.md +++ b/.github/workflows/util/delta-spark-ut/README.md @@ -104,16 +104,14 @@ longer shared between them, those PRs pay for the centos-7 native build twice still caught daily. The nightly run enforces the baseline **and** fails on now-passing tests (`fail_on_fixed=true`), so baseline drift surfaces as a red nightly — the signal to refresh `known-failures.txt`. -- **On demand, from a PR** — the **PR author** comments **`/delta-test`** to +- **On demand, from a PR** — comment **`/delta-test`** to force a full run on a PR the `paths:` filter skipped. The command must be the comment's first token — it may be followed by a space or a newline and then any text, but `/delta-test-arm` or `/delta-testers` will *not* match, so a future command that starts the same way cannot fire this one by accident. - Restricted to the author, who is the one this exists for: a fork - author cannot label their own PR, but can always comment on it, whereas a - maintainer can already start the suite from **Run workflow** - (`workflow_dispatch`). A maintainer who wants a run on someone else's PR uses - that, or asks the author to comment. It runs the + Anyone can use it, as with `velox_backend_ansi.yml`'s `/ansi-test`; a comment + rather than a label because labelling needs write/triage permission, so a fork + author — the person who most needs this — cannot opt their own PR in. It runs the PR's merge ref with the default settings and the baseline **enforced**; `update_baseline` stays reachable only from `workflow_dispatch`, so no comment can rewrite the baseline. The workflow replies with a link to the run, because From b0ad8e7140a26ece7bab392d710a018ee3fcf572 Mon Sep 17 00:00:00 2001 From: Felipe Fujiy Pessoto Date: Thu, 20 Aug 2026 00:57:09 +0000 Subject: [PATCH 8/9] [GLUTEN-12743][CI] Clarify /delta-test documentation wording Apply the valid grammar suggestion from the suppressed review: use "opt into their own PR" consistently in the workflow comment and README. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/workflows/delta_spark_ut.yml | 2 +- .github/workflows/util/delta-spark-ut/README.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/delta_spark_ut.yml b/.github/workflows/delta_spark_ut.yml index 17895c9747..9783f692d3 100644 --- a/.github/workflows/delta_spark_ut.yml +++ b/.github/workflows/delta_spark_ut.yml @@ -193,7 +193,7 @@ jobs: # Anyone may ask for a run, matching velox_backend_ansi.yml's `/ansi-test` # (and take.yml), which are the repo's existing comment triggers and gate on # the command alone. The PR author is the party this exists for -- labelling - # a PR needs write/triage, so a fork author cannot opt their own PR in -- + # a PR needs write/triage, so a fork author cannot opt into their own PR -- # but there is no reason to be stricter here than the pipeline next door. # # Note this makes the "restore but never save" cache guards below load diff --git a/.github/workflows/util/delta-spark-ut/README.md b/.github/workflows/util/delta-spark-ut/README.md index 5f242bec47..8030f26861 100644 --- a/.github/workflows/util/delta-spark-ut/README.md +++ b/.github/workflows/util/delta-spark-ut/README.md @@ -111,7 +111,7 @@ longer shared between them, those PRs pay for the centos-7 native build twice future command that starts the same way cannot fire this one by accident. Anyone can use it, as with `velox_backend_ansi.yml`'s `/ansi-test`; a comment rather than a label because labelling needs write/triage permission, so a fork - author — the person who most needs this — cannot opt their own PR in. It runs the + author — the person who most needs this — cannot opt into their own PR. It runs the PR's merge ref with the default settings and the baseline **enforced**; `update_baseline` stays reachable only from `workflow_dispatch`, so no comment can rewrite the baseline. The workflow replies with a link to the run, because From 14cc60836dadc000ae34c22c0d953d3553ba8991 Mon Sep 17 00:00:00 2001 From: Felipe Fujiy Pessoto Date: Fri, 21 Aug 2026 23:34:53 +0000 Subject: [PATCH 9/9] [GLUTEN-12743][CI] Trim Delta comment-trigger documentation Keep only the non-obvious behavior and security invariants in the workflow and README; remove the historical rationale and repeated explanations. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/workflows/delta_spark_ut.yml | 85 ++----------------- .../workflows/util/delta-spark-ut/README.md | 31 ++----- 2 files changed, 15 insertions(+), 101 deletions(-) diff --git a/.github/workflows/delta_spark_ut.yml b/.github/workflows/delta_spark_ut.yml index 9783f692d3..97e897fcc2 100644 --- a/.github/workflows/delta_spark_ut.yml +++ b/.github/workflows/delta_spark_ut.yml @@ -60,15 +60,7 @@ on: # Negated patterns are evaluated in order and override the positives # above, so this must stay last. - '!.github/workflows/util/delta-spark-ut/**/*.md' - # Escape hatch from the `paths:` filter: comment `/delta-test` on a PR to force - # a full run. A Velox/core/native change does not match the filter but - # can still break Delta offload, and the nightly is not always soon enough. - # - # A comment rather than a label, because labelling needs write/triage - # permission -- a PR author on a fork, the person who most needs this, cannot - # apply one. Same mechanism as velox_backend_ansi.yml's `/ansi-test`. And an - # ADDITIONAL `on:` key rather than a change to the trigger above, so PRs that - # do not ask for a run still cost zero jobs. + # Force a run skipped by `paths:` without requiring label permissions. issue_comment: types: [created] workflow_dispatch: @@ -103,16 +95,7 @@ on: schedule: - cron: '0 5 * * *' -# Unlike `pull_request`, an `issue_comment` run is a base-repo run with a -# base-repo token, and the jobs below build and execute code from the PR. Nothing -# here needs write access, so pin the default to read-only. -# -# `actions: read` is required, not optional: setting this block at all sets every -# unlisted scope to `none`, and the Apache Stash restore action reads the caches -# through the artifacts REST API (`gh api repos/.../actions/artifacts` plus -# `gh run download`), which 403s without it. The other Stash-using workflows here -# declare no `permissions:` block and so inherit the repo default, which is why -# they do not need to say this. +# Comment runs execute PR code; keep the token read-only. Stash needs artifacts access. permissions: contents: read actions: read @@ -121,12 +104,7 @@ env: ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION: true MVN_CMD: 'build/mvn -ntp' CCACHE_DIR: "${{ github.workspace }}/.ccache" - # What every job below checks out. Empty on all events except `issue_comment`, - # whose run is created against the DEFAULT BRANCH -- so without this a - # `/delta-test` comment would test main, not the PR. An empty `ref` is exactly - # what actions/checkout does by default, so the other events are unaffected. - # The merge ref is what `pull_request` tests too; a conflicted PR has none, and - # checkout then fails with a clear git error. + # `issue_comment` runs on the default branch, so explicitly select the PR merge ref. DELTA_CHECKOUT_REF: ${{ github.event.issue.number && format('refs/pull/{0}/merge', github.event.issue.number) || '' }} # Gluten profile / bundle naming for the build-gluten-bundle and # delta-spark-test jobs. `spark_version` is the single source of truth for the @@ -168,47 +146,14 @@ env: # granularity for the split to stay balanced. DELTA_NUM_SHARDS: '8' -# Now that this is a standalone workflow (not called by velox_backend_x86.yml), -# `github.workflow` resolves to THIS workflow, so a concurrency group is both safe -# and necessary: without it every push to a PR branch stacks another full ~2.5 h -# Delta run instead of superseding the previous one. Keyed on the PR number when -# there is one, so a `pull_request` run and a `/delta-test` run on the same PR -# supersede each other -- `github.head_ref` is empty on `issue_comment`, which -# would otherwise lump every commented-on PR into one shared group. +# Cancel older `pull_request` and `/delta-test` runs for the same PR. concurrency: group: ${{ github.repository }}-${{ github.event.pull_request.number || github.event.issue.number || github.head_ref || github.sha }}-${{ github.workflow }} cancel-in-progress: true jobs: - # Gate + acknowledgement for `/delta-test`, and nothing else: the ref to check - # out is computed in `env.DELTA_CHECKOUT_REF` above, so this job has no outputs - # and the rest of the pipeline just hangs off it via `needs`. - # - # Every other event passes straight through (the `if` short-circuits), leaving - # their behaviour unchanged. - # - # Kept as its own job so that `pull-requests: write` -- needed to comment back - # -- is never granted to a job that builds and runs the PR's code. delta-test-requested: - # Anyone may ask for a run, matching velox_backend_ansi.yml's `/ansi-test` - # (and take.yml), which are the repo's existing comment triggers and gate on - # the command alone. The PR author is the party this exists for -- labelling - # a PR needs write/triage, so a fork author cannot opt into their own PR -- - # but there is no reason to be stricter here than the pipeline next door. - # - # Note this makes the "restore but never save" cache guards below load - # bearing rather than belt-and-braces: any user can now start a run that - # builds and executes the PR's code. - # - # The command must still be the FIRST TOKEN of the comment, so neither - # quoting it while discussing it (`/ansi-test` uses `contains`, which does - # match a mention mid-sentence) nor a longer command that merely starts the - # same way (`/delta-test-arm`, `/delta-testers` -- plain `startsWith` would - # match both) can spend ~11 job-hours by accident. GHA expressions have no - # regex, so spell out the ways the token can legally end: end-of-body, a - # space, or a newline (`fromJSON` is the only way to write a control - # character in an expression; GitHub stores comment bodies CRLF, and bare LF - # arrives via the API). + # Match `/delta-test` as the first token; pass other event types through. if: >- github.event_name != 'issue_comment' || (github.event.issue.pull_request && @@ -220,14 +165,9 @@ jobs: permissions: pull-requests: write steps: - # An `issue_comment` run belongs to the default branch, so GitHub cannot - # attach it to the PR's Checks tab. Leave a link, or the contributor sees - # nothing happen for ~2.5 h. - name: Acknowledge the request if: ${{ github.event_name == 'issue_comment' }} - # Purely informational: this job is what the whole pipeline hangs off, so - # a transient API failure here must not fail it and silently skip an - # authorised ~2.5 h run. + # The informational comment must not block the run. continue-on-error: true env: GH_TOKEN: ${{ github.token }} @@ -268,14 +208,7 @@ jobs: ccache -s " - name: Save Ccache to Apache Stash - # A stash is saved as an artifact named `-`, and is - # restored by matching `head_branch` + `head_repository_id` -- so the - # scope is the branch, exactly as with actions/cache. On `issue_comment` - # that branch is the DEFAULT BRANCH, not the PR, and these jobs execute - # the PR's code; with the save action's `overwrite: true` default a - # comment-triggered run would *replace* main's stash, which the nightly - # then restores. So comment runs restore but never save. Same for the - # Maven and sbt stashes below. + # Comment runs use main's Stash scope; never overwrite it with PR output. if: ${{ always() && github.event_name != 'issue_comment' }} uses: apache/infrastructure-actions/stash/save@0ba14156c9f4c3cfbe4b0c9f36339ab0f8d81e53 with: @@ -338,7 +271,6 @@ jobs: -Pbackends-velox -Pdelta \ -DskipTests -Dmaven.compiler.release=17 - name: Save Maven repository to Apache Stash - # Trusted runs only -- see the Save Ccache step above. if: ${{ github.event_name != 'issue_comment' }} uses: apache/infrastructure-actions/stash/save@0ba14156c9f4c3cfbe4b0c9f36339ab0f8d81e53 with: @@ -504,8 +436,7 @@ jobs: - name: Save sbt / Ivy / Coursier to Apache Stash # All shards have the same dependencies; one writer avoids matrix jobs - # overwriting the same stash artifact. Trusted runs only -- see the Save - # Ccache step in build-native-lib-centos-7. + # overwriting the same stash artifact. if: ${{ success() && matrix.shard == 0 && github.event_name != 'issue_comment' }} uses: apache/infrastructure-actions/stash/save@0ba14156c9f4c3cfbe4b0c9f36339ab0f8d81e53 with: diff --git a/.github/workflows/util/delta-spark-ut/README.md b/.github/workflows/util/delta-spark-ut/README.md index 8030f26861..c836672937 100644 --- a/.github/workflows/util/delta-spark-ut/README.md +++ b/.github/workflows/util/delta-spark-ut/README.md @@ -97,35 +97,18 @@ longer shared between them, those PRs pay for the centos-7 native build twice filter before creating the run, so an unrelated PR costs nothing at all. Changes to general Velox/core/native code can also affect Delta offload, but they're touched on most PRs, so per-PR they skip the suite — the nightly run is - the safety net, and `/delta-test` below forces a run on any PR the filter - skipped. + the safety net. - **Nightly** — the **full** suite runs against the latest default branch on a `schedule` (05:00 UTC), so regressions from general Velox/core changes are still caught daily. The nightly run enforces the baseline **and** fails on now-passing tests (`fail_on_fixed=true`), so baseline drift surfaces as a red nightly — the signal to refresh `known-failures.txt`. -- **On demand, from a PR** — comment **`/delta-test`** to - force a full run on a PR the `paths:` filter skipped. The command must be the - comment's first token — it may be followed by a space or a newline and then - any text, but `/delta-test-arm` or `/delta-testers` will *not* match, so a - future command that starts the same way cannot fire this one by accident. - Anyone can use it, as with `velox_backend_ansi.yml`'s `/ansi-test`; a comment - rather than a label because labelling needs write/triage permission, so a fork - author — the person who most needs this — cannot opt into their own PR. It runs the - PR's merge ref with the default settings and the baseline **enforced**; - `update_baseline` stays reachable only from `workflow_dispatch`, so no comment - can rewrite the baseline. The workflow replies with a link to the run, because - an `issue_comment` run belongs to the default branch and so cannot appear in - the PR's Checks tab. Two consequences: `/delta-test` reads the workflow file - from the default branch (so it cannot test changes to this pipeline itself — - those match the `paths:` filter anyway), and comment-triggered runs **restore - but never save** the ccache/Maven/sbt stashes. A stash is stored as an - artifact named `-` and looked up by branch, and for an - `issue_comment` run that branch is the **default branch**, not the PR — so a - save would overwrite (Stash defaults to `overwrite: true`) a stash that the - nightly restores. The ccache key is shared with `velox_backend_x86.yml`, so - that would reach beyond this pipeline. Reads are unaffected: a comment run - still restores `main`'s stashes, so it is no slower. +- **On demand, from a PR** — anyone can comment **`/delta-test`** as the first + token to run the default configuration against the PR merge ref. This covers + PRs skipped by `paths:` without requiring label permissions. The workflow + posts the run link on the PR and enforces the committed baseline. Comment runs + restore but never save main-scoped Stash caches; use `workflow_dispatch` for + custom inputs or baseline updates. - **Manually** — **Actions → Delta Spark UT (Gluten) → Run workflow** (`workflow_dispatch`), e.g. to refresh the baseline (see below). This is also how you validate a Velox/core change against Delta before merging: run it on