Skip to content

[GLUTEN-12743][CI] Let contributors trigger the Delta Spark UT with a /delta-test PR comment - #12781

Open
felipepessoto wants to merge 8 commits into
apache:mainfrom
felipepessoto:delta-ci-comment-trigger
Open

[GLUTEN-12743][CI] Let contributors trigger the Delta Spark UT with a /delta-test PR comment#12781
felipepessoto wants to merge 8 commits into
apache:mainfrom
felipepessoto:delta-ci-comment-trigger

Conversation

@felipepessoto

@felipepessoto felipepessoto commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

Part of #12743.

What changes are proposed in this pull request?

The Delta Spark UT pipeline added in #12388 runs per-PR only when its paths: filter matches (gluten-delta/**, backends-velox/src-delta*/**, and the pipeline's own files). A change to general Velox/core/native code does not match that filter but can still break Delta offload, and today there is no way for a contributor to force a run — workflow_dispatch needs write access to this repo, and the nightly is not always soon enough.

#12743 lists a run-delta-ci label as the candidate. That does not work for the people who need it most: changing labels requires write/triage permission, so a PR author working from a fork cannot opt their own PR in. A label also cannot be expressed as a paths: filter, so it would need a gate job running git diff on every PR.

This PR adds a /delta-test PR comment instead — the same mechanism velox_backend_ansi.yml already uses for /ansi-test:

  • Anyone can comment, so a fork author can trigger the suite on their own PR — which is the whole point, since labelling needs write/triage permission and a fork author has neither. The gate is the command alone, matching the repo's existing comment triggers (velox_backend_ansi.yml's /ansi-test and take.yml), so a reviewer can also run it against someone else's PR. No author_association check: it does not track write access on an ASF repo (write comes from Gitbox), so an OWNER/MEMBER/COLLABORATOR allow-list would reject 16 of the 18 apache/gluten maintainers sampled while accepting any apache org member.
  • It is an additional on: key, not a change to the pull_request trigger, so the paths: filter is untouched and PRs that don't ask for a run still cost zero jobs — no per-PR gate job.
  • startsWith, not contains, so quoting the command while discussing it doesn't spend ~11 job-hours.

How it works

An issue_comment run is created against the default branch, so github.ref points at main, not the PR. One workflow-level env resolves what to check out:

DELTA_CHECKOUT_REF: ${{ github.event.issue.number && format('refs/pull/{0}/merge', github.event.issue.number) || '' }}

Every job then checks out ref: ${{ env.DELTA_CHECKOUT_REF }}. It is empty on every other event, which is exactly what actions/checkout does by default, so pull_request, schedule and workflow_dispatch behave precisely as before. Using the PR's merge ref (what pull_request itself tests) avoids any API call, job outputs or SHA plumbing.

A small delta-test-requested job holds the authorisation if: and posts a link to the run — an issue_comment run belongs to the default branch, so GitHub cannot attach it to the PR's Checks tab, and without the link the contributor sees nothing happen for ~2.5 h. It is a separate job so that pull-requests: write is never granted to a job that builds and runs the PR's code. Everything else hangs off it via needs, so a comment that isn't authorised skips the whole pipeline.

update_baseline stays reachable only from workflow_dispatch, so no comment can rewrite the committed baseline.

Cache scoping (why the cache steps changed)

A cache write is scoped to the run's GITHUB_REF. For pull_request that is refs/pull/N/merge, isolated to the PR — but for issue_comment it is the default branch, shared with every trusted run. Since these jobs compile and execute the PR's code, saving there would let a PR plant a ccache/Maven/sbt entry that the nightly on main later restores (the prefixed restore-keys make it reachable), i.e. attacker-controlled compiler output in a trusted build.

So the three caches are split into restore + save, and comment-triggered runs restore but never save. They still read the shared caches, so they are no slower; they just don't contribute back. Workflow permissions are additionally pinned to contents: read.

Files

File Change
.github/workflows/delta_spark_ut.yml issue_comment trigger, DELTA_CHECKOUT_REF, delta-test-requested gate job, ref: on the 4 checkouts, cache restore/save split, permissions, PR-number concurrency key.
.github/workflows/util/delta-spark-ut/README.md Documents /delta-test under "When it runs".

58 functional lines (the rest of the diff is comments and docs).

How was this patch tested?

This change is CI, and the trigger can only be exercised once the workflow is on the default branch. Verified statically instead:

  • actionlint clean on delta_spark_ut.yml.
  • Trigger truth table, 12 cases: pull_request / schedule / workflow_dispatch all pass through; a comment on an issue (not a PR), the command quoted mid-sentence, a longer command (/delta-test-arm, /delta-testers), and ordinary PR chatter are all rejected; /delta-test is accepted from any user, bare, with trailing text, or as the first line of a multi-line comment (CRLF and LF).
  • DELTA_CHECKOUT_REF resolves to '' on pull_request / schedule / workflow_dispatch and to refs/pull/N/merge on issue_comment; confirmed against actions/checkout's source that an empty ref reproduces its default (github.context.ref + github.context.sha), so existing events are unaffected.
  • Skip propagation: with the gate skipped, all five downstream jobs skip (verified against each job's existing if:).
  • Cache read/write split asserted programmatically: every save is gated on issue_comment, every restore is not.
  • No untrusted data (github.event.comment.* / github.event.issue.*) is interpolated into any run: block; it is passed via env: only.
  • Confirmed actions/checkout's assertSafePrCheckout guard only fires on pull_request_target / workflow_run, so issue_comment needs no allow-unsafe-pr-checkout.

Once merged, /delta-test on any PR is the end-to-end test.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: GitHub Copilot CLI

Copilot AI lite review requested due to automatic review settings August 15, 2026 02:10

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an opt-in /delta-test PR comment trigger to run the Delta Spark UT workflow even when the existing paths: filter would otherwise skip it, using a gated issue_comment entrypoint and a shared checkout ref (refs/pull/N/merge) so the rest of the pipeline runs the PR code safely with read-only default permissions.

Changes:

  • Add issue_comment trigger and a delta-test-requested gate job to authorize /delta-test and post a run link back to the PR.
  • Introduce DELTA_CHECKOUT_REF and apply it to all checkouts so comment-triggered runs test the PR merge ref.
  • Split cache restore/save so comment-triggered runs can restore caches but never write to default-branch-scoped caches.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
.github/workflows/delta_spark_ut.yml Adds /delta-test comment trigger, authorization gate job, PR-merge-ref checkout, cache save gating, and tighter default permissions.
.github/workflows/util/delta-spark-ut/README.md Documents the new /delta-test “on demand” trigger and its behavior/constraints.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread .github/workflows/delta_spark_ut.yml Outdated
Comment thread .github/workflows/util/delta-spark-ut/README.md Outdated
Copilot AI review requested due to automatic review settings August 15, 2026 02:24

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Suppressed comments (2)

.github/workflows/delta_spark_ut.yml:186

  • The PR description says /delta-test can be triggered by the PR author or anyone with write access, but the implemented gate only allows the PR author (by design per prior discussion). Please update the PR description to match the current authorization behavior, or expand the gate to include the intended additional authorized users.
    # Only the PR author may spend a run on their own PR -- which is exactly the
    # gap this closes: a fork author cannot label their own PR, but can always
    # comment on it.

.github/workflows/delta_spark_ut.yml:220

  • The acknowledgement step is part of the gating job; if gh pr comment fails (e.g., transient API failure), the whole delta-test-requested job fails and the entire pipeline is blocked even though the request is authorized. Since this comment is informational only, it should not be able to fail the workflow run.
      - name: Acknowledge the request
        if: ${{ github.event_name == 'issue_comment' }}
        env:

Copilot AI review requested due to automatic review settings August 15, 2026 03:56
@felipepessoto

Copy link
Copy Markdown
Contributor Author

Thanks — surfacing the two suppressed comments from that review, since they aren't visible in the diff:

1. delta_spark_ut.yml:220 — the acknowledgement step can block an authorised run. Real bug, fixed in 625a3d7.

The whole pipeline hangs off delta-test-requested via needs, so a transient gh pr comment failure (API blip, rate limit) would fail the gate job and skip every downstream job — silently dropping a run the contributor legitimately asked for, with no obvious cause. The comment is informational only, so the step is now continue-on-error: true:

      - name: Acknowledge the request
        if: ${{ github.event_name == 'issue_comment' }}
        continue-on-error: true

2. delta_spark_ut.yml:186 — PR description out of sync with the gate. Already resolved; the review snapshot just predates the edit. The description was updated alongside 625a3d7's predecessor d36c9dc and now reads "The gate is the PR author only", with the verification section listing "a non-author (including a maintainer) … rejected". No remaining claim of write-access-based triggering.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

@felipepessoto
felipepessoto force-pushed the delta-ci-comment-trigger branch from 625a3d7 to 190e1ef Compare August 19, 2026 00:08
Copilot AI review requested due to automatic review settings August 19, 2026 00:08

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

felipepessoto and others added 4 commits August 19, 2026 17:03
… /delta-test PR comment

The Delta Spark UT pipeline (apache#12388) only runs per-PR when the `paths:` filter
matches. A Velox/core/native change does not match it but can still break Delta
offload, and there was no way for a contributor to force a run.

A `run-delta-ci` label was considered and dropped: changing labels needs
write/triage permission, so a PR author working from a fork -- exactly the person
who needs it -- cannot apply one, and a label cannot be expressed as a `paths:`
filter, so it would need a gate job on every PR.

Use an `issue_comment` slash command instead, as velox_backend_ansi.yml already
does for `/ansi-test`. Anyone can comment, so the PR author can opt their own PR
in; authorised commenters are the PR author or anyone with write access. It is an
ADDITIONAL `on:` key, so the `paths:` filter is untouched and PRs that do not ask
for a run still cost zero jobs. `startsWith`, not `contains`, so quoting the
command while discussing it does not spend ~11 job-hours.

An `issue_comment` run is created against the default branch, so `env.
DELTA_CHECKOUT_REF` resolves the PR's merge ref and every job checks it out; it
is empty on every other event, which is exactly actions/checkout's default, so
their behaviour is unchanged. Such a run is also not attached to the PR's Checks
tab, so the gate job replies with a link. `update_baseline` remains reachable
only from `workflow_dispatch`, so no comment can rewrite the committed baseline.

Cache writes are scoped to the run's GITHUB_REF, which for `issue_comment` is the
default branch rather than the PR. Since these jobs build and execute code from
the PR, comment-triggered runs now restore the ccache / Maven / sbt caches but
never save to them, so a PR cannot plant an entry that the nightly on main later
restores. Workflow permissions are pinned to `contents: read`, with
`pull-requests: write` granted only to the gate job, which never checks out PR
code.

Generated-by: GitHub Copilot CLI

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Addresses review feedback that the `author_association` allow-list is not
equivalent to write access. On an ASF repo it is worse than imprecise: write
access comes from Gitbox, not from GitHub org/collaborator membership, so
`COLLABORATOR` matches nobody on apache/gluten and 16 of the 18 committers
sampled from recent PR review comments report `CONTRIBUTOR`. An
OWNER/MEMBER/COLLABORATOR list would therefore have rejected nearly every
maintainer while accepting any member of the apache org.

Drop the list rather than reword it. `/delta-test` now requires the commenter to
be the PR author, which is precisely the gap this closes -- a fork author cannot
label their own PR but can always comment on it -- and is exactly describable, so
the workflow comment and README no longer overstate who may trigger a run. A
maintainer who wants a run on someone else's PR asks the author to comment.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… run

The whole pipeline hangs off `delta-test-requested` via `needs`, so a transient
`gh pr comment` failure in its acknowledgement step would fail the gate job and
skip every downstream job -- silently dropping an authorised run. The comment is
informational only, so mark the step `continue-on-error: true`.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Follow-up to rebasing onto apache#12820, which replaced actions/cache with Apache
Stash. Two things the migration changes for this feature:

1. `permissions:` must grant `actions: read`. Declaring the block at all sets
   every unlisted scope to `none`, and the Stash restore action reads caches
   through the artifacts REST API (`gh api repos/.../actions/artifacts` and
   `gh run download`), which 403s without it. The other Stash-using workflows
   declare no `permissions:` block and inherit the repo default, so they never
   had to say this. Without this the Stash restores would fail on every event,
   not just on comment runs.

2. The "restore but never save" guards still apply, and matter more. A stash is
   an artifact named `<key>-<github.ref_name>`, restored by matching
   `head_branch` + `head_repository_id` -- the same branch scoping actions/cache
   has. On `issue_comment` that branch is the default branch, so a save from a
   run executing PR code would land on `main`, and Stash's `overwrite: true`
   default means it replaces the existing entry rather than merely competing
   with it. apache#12820 also moved the ccache to the key
   `ccache-centos7-release-default-${{ hashFiles('ep/build-velox/src/**') }}`,
   which velox_backend_x86.yml restores from as well, so an unguarded save would
   reach beyond this pipeline.

Reads are deliberately left unguarded: a comment run restores main's stashes and
is therefore no slower than any other run.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@felipepessoto
felipepessoto force-pushed the delta-ci-comment-trigger branch from 190e1ef to 8911f45 Compare August 19, 2026 17:07
Copilot AI review requested due to automatic review settings August 19, 2026 17:07
@felipepessoto

Copy link
Copy Markdown
Contributor Author

Rebased onto main to pick up #12820 (Delta Spark UT caches moved to Apache Stash), which conflicted with this PR since both touch the cache steps.

The conflict itself was mostly a simplification. #12820 already splits every cache into stash/restore + stash/save, which this PR previously had to do by hand — so the restore/save split, the id:s and the cache-hit guards are all gone. The functional diff dropped from 58 lines to 42; what remains on top of main is the issue_comment trigger, DELTA_CHECKOUT_REF + four ref: pins, the gate job, and three if: guards.

It also required one change beyond conflict resolution (separate commit, 8911f45):

permissions: must grant actions: read. Declaring a permissions: block at all sets every unlisted scope to none, and Stash's restore reads caches through the artifacts REST API:

gh api repos/{repo}/actions/runs/{run_id}/artifacts
gh api repos/{repo}/actions/artifacts
gh run download

all of which 403 without actions: read. Every other Stash-using workflow here (velox_backend_x86.yml, velox_weekly.yml, velox_backend_arm.yml, …) declares no permissions: block and so inherits the repo default, which is why none of them says this. Left unfixed, the Stash restores would have failed on every event, not just comment runs — a cold ccache on the nightly.

And I re-verified the guards still have a job to do under Stash. They do, more so than before. A stash is an artifact named <key>-<github.ref_name>, restored by matching head_branch + head_repository_id — the same branch scoping actions/cache has (the action's README says so explicitly). On issue_comment that branch is the default branch, so a save from a run executing PR code lands on main:

event github.ref_name artifact written
pull_request my-branch <key>-my-branch
schedule (nightly) main <key>-main
issue_comment main <key>-main ← same name the nightly reads

Two things make it sharper than under actions/cache: Stash's save defaults to overwrite: true, so it replaces the existing entry rather than losing a race with it; and #12820 moved the ccache onto ccache-centos7-release-default-${{ hashFiles('ep/build-velox/src/**') }}, which velox_backend_x86.yml also restores from — so an unguarded save would reach beyond this pipeline. Restores stay unguarded, so a comment run still reads main's stashes and is no slower.

Verified: actionlint clean; authorization and DELTA_CHECKOUT_REF truth tables re-run against the rebased file; asserted 3/3 saves guarded and 3/3 restores unguarded, and that no job overrides permissions: in a way that would strip actions: read from a Stash step. README updated for Stash semantics.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Suppressed comments (4)

.github/workflows/delta_spark_ut.yml:72

  • Using on: issue_comment will still create a workflow run for every newly created comment across the repo (including non-command PR chatter and issue comments), even though the jobs will mostly be skipped by the gate. This can significantly clutter the Actions UI and create noisy 'skipped' runs. Consider moving the comment listener into a tiny dedicated workflow that only triggers a real run when the command matches (e.g., via a dispatch to this workflow), so non-matching comments do not create runs for the heavy workflow.
  issue_comment:
    types: [created]

.github/workflows/delta_spark_ut.yml:210

  • Using on: issue_comment will still create a workflow run for every newly created comment across the repo (including non-command PR chatter and issue comments), even though the jobs will mostly be skipped by the gate. This can significantly clutter the Actions UI and create noisy 'skipped' runs. Consider moving the comment listener into a tiny dedicated workflow that only triggers a real run when the command matches (e.g., via a dispatch to this workflow), so non-matching comments do not create runs for the heavy workflow.
    if: >-
      github.event_name != 'issue_comment' ||
      (github.event.issue.pull_request &&
       startsWith(github.event.comment.body, '/delta-test') &&
       github.event.comment.user.login == github.event.issue.user.login)

.github/workflows/delta_spark_ut.yml:209

  • startsWith(..., '/delta-test') will also match unintended prefixes like /delta-test-foo or /delta-testers, which can accidentally trigger a ~2.5h run. Tighten the condition to require an exact command token (e.g., /delta-test followed by end-of-string or whitespace/newline) while still avoiding contains.
       startsWith(github.event.comment.body, '/delta-test') &&

.github/workflows/delta_spark_ut.yml:334

  • Unlike the ccache save step (which uses always() for trusted runs), this Maven save will run only on success due to the default if: success() behavior. That means a trusted run that fails late (e.g., during tests) won’t save updated dependencies, potentially slowing subsequent reruns. If the intent is to keep dependency caching warm even when later steps fail, consider using an always()-style condition here as well (still excluding issue_comment).
      - name: Save Maven repository to Apache Stash
        # Trusted runs only -- see the Save Ccache step above.
        if: ${{ github.event_name != 'issue_comment' }}
        uses: apache/infrastructure-actions/stash/save@0ba14156c9f4c3cfbe4b0c9f36339ab0f8d81e53

Addresses review feedback: `startsWith(body, '/delta-test')` also matches
`/delta-test-arm` and `/delta-testers`, so a longer command that merely starts
the same way -- or a future `/delta-test-<something>` -- would spend ~11
job-hours on this pipeline by accident.

GHA expressions have no regex, so spell out the ways the token can legally end:
end-of-body, a space, or a newline. `fromJSON('"\r"')` / `fromJSON('"\n"')` is
the only way to express a control character in an expression; GitHub stores
comment bodies with CRLF and the API can deliver bare LF, so both are matched.
This keeps multi-line comments working (`/delta-test` on its own first line)
while rejecting the longer-prefix cases.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 19, 2026 19:42
@felipepessoto

Copy link
Copy Markdown
Contributor Author

Surfacing the 4 suppressed comments from that review (they're 3 distinct points — the first is duplicated at two lines). One was a real bug and is fixed; two I'm declining, with evidence.

1. startsWith matches /delta-test-foo / /delta-testers — valid, fixed in dfe2e66

Correct, and the forward-compat case is the one that worried me: add a /delta-test-arm later and it would fire this ~11 job-hour pipeline too. GHA expressions have no regex, so the token boundary is spelled out:

      (github.event.comment.body == '/delta-test' ||
       startsWith(github.event.comment.body, '/delta-test ') ||
       startsWith(github.event.comment.body, format('/delta-test{0}', fromJSON('"\r"'))) ||
       startsWith(github.event.comment.body, format('/delta-test{0}', fromJSON('"\n"'))))

fromJSON is the only way to write a control character in an expression. Both CR and LF are matched because GitHub stores comment bodies CRLF while the API can deliver bare LF — so /delta-test on its own first line of a multi-line comment still works. Truth table re-run, 13/13, including the new rejects (/delta-test-arm, /delta-testers) and accepts (CRLF and LF multi-line).

2. Maven save runs only on success, unlike the ccache save — declining, it's not a change this PR makes

This is upstream #12820's behaviour, preserved exactly. Per the expressions docs:

A default status check of success() is applied unless you include one of these functions.

if: ${{ github.event_name != 'issue_comment' }} contains no status-check function, so it evaluates as success() && github.event_name != 'issue_comment' — the same success-gating as #12820's no-if: version, plus my restriction. The ccache step differs only because #12820 wrote always() there and I kept it (always() && …), and the sbt step likewise keeps its explicit success() && matrix.shard == 0.

So the asymmetry is pre-existing and deliberate on #12820's side. Changing the Maven save to always() would alter behaviour unrelated to this PR (and would stash a partial .m2 from a failed build), so it belongs in its own PR if wanted.

3. on: issue_comment creates a skipped run for every comment repo-wide — real, but declining the split

The observation is correct and I measured it. Of the last 100 velox_backend_ansi.yml runs (the existing /ansi-test listener), 100/100 are issue_comment / skipped — its real runs are completely buried. So this is a genuine cost, not hypothetical.

I'm still not splitting it, for one specific reason: the suggested "listener dispatches the heavy workflow" design silently disables every security guard in this PR. workflow_dispatch from GITHUB_TOKEN does work (documented exception to the recursion rule), but the dispatched run arrives with github.event_name == 'workflow_dispatch', so:

  • the three github.event_name != 'issue_comment' stash guards stop firing, and the run — still executing fork code — starts writing main-scoped stashes, including the ccache key shared with velox_backend_x86.yml;
  • update_baseline, currently gated on github.event_name == 'workflow_dispatch', becomes reachable from a comment;
  • the untrusted-ness has to be re-encoded as an input and re-honoured in ~5 places, where forgetting one fails open and silently.

Trading a tidier Actions list for a set of fail-open guards isn't a good deal at ~11 job-hours and a shared ccache. The noise is also consistent with the existing /ansi-test precedent, so this PR doesn't make the situation worse than the pattern already in the repo.

If maintainers do want the run list cleaned up, the safe version is to keep the guards keyed on an explicit untrusted input rather than on event_name — happy to do that as a follow-up, and it would fix /ansi-test's noise at the same time. It seemed wrong to fold that into this PR.

@github-actions

Copy link
Copy Markdown

🔄 ANSI mode analysis started by @felipepessoto. View run

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Suppressed comments (1)

.github/workflows/delta_spark_ut.yml:223

  • delta-test-requested runs on all events (because the job-level if short-circuits true when github.event_name != 'issue_comment'), but it always receives pull-requests: write via job-level permissions. That grants an unnecessary write-scoped GITHUB_TOKEN to scheduled and ordinary pull_request runs even though the only step that uses the token is guarded to issue_comment.

Consider splitting this into (1) a no-permissions pass-through gate job for non-issue_comment events and (2) an issue_comment-only acknowledgement job that has pull-requests: write, so write permission is only present when it’s actually needed.

    if: >-
      github.event_name != 'issue_comment' ||
      (github.event.issue.pull_request &&
       (github.event.comment.body == '/delta-test' ||
        startsWith(github.event.comment.body, '/delta-test ') ||
        startsWith(github.event.comment.body, format('/delta-test{0}', fromJSON('"\r"'))) ||
        startsWith(github.event.comment.body, format('/delta-test{0}', fromJSON('"\n"')))) &&
       github.event.comment.user.login == github.event.issue.user.login)
    runs-on: ubuntu-22.04
    permissions:
      pull-requests: write
    steps:

@felipepessoto

Copy link
Copy Markdown
Contributor Author

Tested end-to-end before merge

issue_comment workflows are always read from the default branch, so /delta-test cannot be exercised from this PR's branch here. I ran it in a fork instead (fork main fast-forwarded to this branch, restored afterwards), on a throwaway PR that touches only .github/workflows/util/delta-spark-ut/README.md — a path the Delta filter deliberately excludes, i.e. exactly the "filter skipped it" case this feature exists for.

# Check Result
1 PR on a filter-excluded path creates no Delta run ✅ no run created
2 /delta-test-arm rejected (the dfe2e66 fix) ✅ all 5 jobs skipped
3 /delta-test by the PR author authorises delta-test-requested succeeded
4 Acknowledgement comment posted ✅ posted with run link
5 Checks out the PR, not main refs/pull/2/merge
6 Stash restore under actions: read ✅ succeeded, no 403
7 Cache save suppressed on comment runs Save Ccacheskipped

Two notes on the method:

#6 shows the actions: read addition is load-bearing. The Stash restore shells out to the artifacts REST API via run_checked(..., check=True), so a 403 aborts the step outright; it completed cleanly instead.

#7 is valid despite my cancelling the run. The step is always() && github.event_name != 'issue_comment', and always() still evaluates on cancellation — so skipped is the guard firing, not a by-product of the cancel. I cancelled once the native build started, since the remaining ~2.5 h only re-tests Delta results that the pull_request runs here already cover. Total cost ~10 runner-minutes rather than ~11 job-hours.

The run log also confirms empirically the branch-scoping argument made above:

No stash found for keys ccache-centos7-release-default-<hash>-main
                     or ccache-centos7-release-default-<hash>-main

On a comment run both the head and base lookups mung to -main — the exact artifact the nightly restores, and (via the shared ccache key) the one velox_backend_x86.yml restores too. An unguarded save would have overwritten it, which is what the three guards prevent.

The one thing that cannot be tested pre-merge is the workflow file itself being read from main — by construction, that only takes effect once this is merged.

@github-actions

Copy link
Copy Markdown

ANSI Mode Test Analysis Report (Spark 4.1)

Note

Expression-level ANSI mode offload coverage analysis.
Test config: spark.sql.ansi.enabled=true, spark.gluten.sql.ansiFallback.enabled=false.

  • Passed (🟢): Velox correctly handles ANSI semantics
  • Fallback (🔴): Expression falls back to Spark execution, needs ANSI support in Velox
  • Failed (🟡): Velox executes but ANSI error behavior differs from Spark, needs exception handling fix

ANSI Offload suites: 70 tests, 11648 records | Other suites: 3331 tests

ANSI Offload

Overview (ANSI Offload Expression Records)

Classification Count %
🟢 Passed 10944 94.0%
🟡 Failed 1 0.0%
🔴 Fallback 703 6.0%

Per-Suite Summary

Suite 🟢 Passed 🟡 Failed 🔴 Fallback
GlutenTryCastSuite 10926 (94%) 0 703
GlutenDecimalExpressionSuite 18 (95%) 1 0

Failure Cause Analysis (1 failures)

Cause Count Description
WRONG_EXCEPTION 1 Exception wrapped as SparkException

Other (28 failures)

Suite Failures
GlutenHiveTableScanSuite 10
ScalarFunctionsValidateSuite 9
MiscOperatorSuite Support multi-children count with row construct
Remainder with non-foldable right side
Cast string to date
MathFunctionsValidateSuite decimal arithmetic
decimal arithmetic respects allowPrecisionLoss captured at view analysis time
FallbackSuite fallback when nested loop join has unsupported expression
UDFPartialProjectSuite udf in agg simple
DateFunctionsValidateSuite make_date
GlutenComplexTypeSuite SPARK-33386: GetArrayItem ArrayIndexOutOfBoundsException

Comments only; no logic change. Records why this trigger is stricter than the
repo's other comment triggers (velox_backend_ansi.yml's /ansi-test and take.yml
gate on nothing at all), and why the accurate maintainer check --
collaborators/{user}/permission -- is deliberately absent: maintainers can
already start the suite via workflow_dispatch, so the PR author is the only
party that had no way in, which is the gap apache#12743 describes.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
EOF
Copilot AI review requested due to automatic review settings August 19, 2026 23:39
@github-actions

Copy link
Copy Markdown

🔄 ANSI mode analysis started by @felipepessoto. View run

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Drop the PR-author restriction so this trigger behaves like the repo's existing
comment triggers: velox_backend_ansi.yml (`/ansi-test`, `/ansi-analyze`) and
take.yml both gate on the command alone, with no author or permission check.

Being the only workflow in the repo with an authorization gate was inconsistent,
and it blocked the case where a reviewer wants the Delta suite run against a
contributor's PR before merging.

The exact-token match is deliberately kept rather than following `/ansi-test`'s
`contains()`: matching a mention mid-sentence, or a longer command such as a
future `/delta-test-arm`, was raised in review and would spend ~11 job-hours by
accident.

This also promotes the "restore but never save" cache guards from
belt-and-braces to load bearing, since any user can now start a run that builds
and executes the PR's code. They were already in place and are unchanged.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 20, 2026 00:46
@felipepessoto

Copy link
Copy Markdown
Contributor Author

Changed in ea6e9ef: dropped the PR-author restriction, so /delta-test now gates on the command alone — the same as the repo's existing comment triggers, velox_backend_ansi.yml (/ansi-test, /ansi-analyze) and take.yml, neither of which has an author or permission check.

This supersedes my earlier reasoning in this thread, where I argued for author-only on the grounds that maintainers already have workflow_dispatch. Consistency with the pipeline next door is the better default, and author-only did block a real case: a reviewer wanting the Delta suite run against a contributor's PR before merging.

One deliberate difference from /ansi-test retained: the exact-token match, rather than contains(). contains() fires on a mention mid-sentence — this very comment thread would trigger /ansi-test several times over — and would also match a future /delta-test-arm. That was raised in review and is worth keeping at ~11 job-hours a run.

So the gate is now:

    if: >-
      github.event_name != 'issue_comment' ||
      (github.event.issue.pull_request &&
       (github.event.comment.body == '/delta-test' ||
        startsWith(github.event.comment.body, '/delta-test ') ||
        startsWith(github.event.comment.body, format('/delta-test{0}', fromJSON('"\r"'))) ||
        startsWith(github.event.comment.body, format('/delta-test{0}', fromJSON('"\n"')))))

Worth flagging for reviewers: this promotes the three "restore but never save" cache guards from belt-and-braces to load bearing, since any user can now start a run that builds and executes the PR's code — and the ccache key is shared with velox_backend_x86.yml. Those guards were already in place and are unchanged; the e2e test above confirmed Save Ccache is skipped on comment-triggered runs.

Truth table re-run, 12/12: accepted from any user (bare, with trailing text, and as the first line of a CRLF or LF multi-line comment); still rejecting /delta-test-arm, /delta-testers, a mid-sentence mention, comments on non-PR issues, and ordinary chatter. actionlint clean. PR description updated to match.

@github-actions

Copy link
Copy Markdown

🔄 ANSI mode analysis started by @felipepessoto. View run

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Suppressed comments (2)

.github/workflows/delta_spark_ut.yml:221

  • The Acknowledge the request step posts a PR comment via gh pr comment, but this job-level permissions block only grants pull-requests: write. Because PR comments are created through the Issues API, the token may lack the issues: write scope and the acknowledgement can 403; continue-on-error: true would hide that failure and leave contributors without the run link.
    permissions:
      pull-requests: write

.github/workflows/util/delta-spark-ut/README.md:114

  • Minor grammar: “cannot opt their own PR in” is awkward; consider “cannot opt into their own PR”.
  author — the person who most needs this — cannot opt their own PR in. It runs the

Apply the valid grammar suggestion from the suppressed review: use "opt into
their own PR" consistently in the workflow comment and README.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 20, 2026 00:57
@felipepessoto

Copy link
Copy Markdown
Contributor Author

Addressing the 2 suppressed comments from review 4978060920:

  1. pull-requests: write may not permit gh pr comment — no change needed; the premise is false. GitHub's Create an issue comment endpoint accepts either Issues: write OR Pull requests: write. More importantly, the pre-merge e2e run already exercised this exact step with the current job permissions: github-actions successfully posted the acknowledgement and run link using only pull-requests: write. There was no 403. Keeping the narrower PR-specific permission is preferable to adding general issue-write access.

    continue-on-error remains intentional for genuine transient failures (API outage/rate limiting): the informational comment must not block an authorized ~11 job-hour run. It is not masking a missing permission, as the e2e result proves.

  2. Grammar — valid, fixed in b0ad8e7. Changed “cannot opt their own PR in” to “cannot opt into their own PR” in both the workflow comment and README.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Suppressed comments (2)

.github/workflows/delta_spark_ut.yml:221

  • gh pr comment posts an issue comment on the PR, but this job only grants pull-requests: write. With the workflow-level permissions: block, issues defaults to none, so this step is likely to 403 and (because of continue-on-error) you’ll silently lose the “reply with a link” UX that the PR description relies on.
    permissions:
      pull-requests: write

.github/workflows/delta_spark_ut.yml:216

  • This issue_comment trigger currently allows any GitHub user who can comment on the PR to start a full ~2.5h, multi-job run that builds/executes PR code (only gated on the command token). That increases exposure to CI resource abuse and widens the blast radius compared to limiting the trigger to the PR author and/or trusted users.
    if: >-
      github.event_name != 'issue_comment' ||
      (github.event.issue.pull_request &&
       (github.event.comment.body == '/delta-test' ||
        startsWith(github.event.comment.body, '/delta-test ') ||

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants