Skip to content

dev: include ASF status checks in the local lint suite - #25313

Open
kumarUjjawal wants to merge 2 commits into
apache:mainfrom
kumarUjjawal:dev/21048-local-asf-status-checks
Open

kumarUjjawal wants to merge 2 commits into
apache:mainfrom
kumarUjjawal:dev/21048-local-asf-status-checks

Conversation

@kumarUjjawal

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

.asf.yaml lists the GitHub Actions jobs that must pass before a PR can merge. If a job is renamed and .asf.yaml still requires the old name, PRs wait for a status that never arrives. CI runs ci/scripts/check_asf_yaml_status_checks.py in dev.yml to catch this, but dev/rust_lint.sh does not. A contributor can pass the local lint suite and then see the failure only in CI. This PR adds the existing validator to the local suite.

What changes are included in this PR?

  • dev/rust_lint.sh runs ci/scripts/check_asf_yaml_status_checks.py as a read-only step, after the workflow install check and before the Markdown link check. The --write and --allow-dirty flags never reach it.
  • The runner needs python3 on PATH with PyYAML installed. It checks both after argument parsing and before any tool install or formatter, and it reports a distinct error for each missing prerequisite. It does not install Python packages. Help output does not run these checks.
  • ci/scripts/check_asf_yaml_status_checks.py becomes executable, because the runner invokes each registered script directly. The script body, .asf.yaml, and the GitHub workflow do not change.
  • docs/source/contributor-guide/testing.md documents the purpose of the check, the Python setup, the standalone command, and the local lint integration.

What is the testing strategy for this PR?

  • The real validator passes on this branch through python3 ci/scripts/check_asf_yaml_status_checks.py and through direct invocation, with Python 3.14.7 and PyYAML 6.0.3.
  • Disposable configurations exercised the real validator. A valid configuration passes. A required name with no matching job fails with the existing diagnostic. A name served only by a path-filtered workflow fails, and an added unfiltered workflow makes it pass.
  • A disposable fixture with stubbed steps and tools exercised the runner. The validator runs once with no arguments in check mode and in both write modes. A missing python3 and a missing PyYAML each stop the suite with their own error before any tool install or formatter. Help output skips the checks. A validator failure stops the later steps.
  • A disposable clone with a bogus required name in .asf.yaml: ./dev/rust_lint.sh fails with the validator diagnostic before the link check, and passes again after the file is restored.
  • The full ./dev/rust_lint.sh passes on this branch.

Are there any user-facing changes?

No. The local lint suite gains the check and needs Python 3 with PyYAML. CI is unchanged.

🤖 Generated with Claude Code

`dev/rust_lint.sh` now runs `ci/scripts/check_asf_yaml_status_checks.py`
as a read-only step, after the workflow install check and before the
Markdown link check. The validator compares the required status names
in `.asf.yaml` with the jobs in `.github/workflows`, so a renamed job
that still has a stale requirement fails locally instead of leaving a
pull request waiting for a status that never arrives.

The validator runs with `python3` from PATH and imports PyYAML. The
runner checks both before it installs any tool or runs any formatter,
and reports a distinct, actionable error for each missing prerequisite.
It does not install Python packages. Help output stays independent of
these checks.

The validator becomes executable so the runner can invoke it directly,
as it does for every other registered script. Its body, `.asf.yaml`,
and the GitHub workflow are unchanged, and `--write` never reaches it.

Partial progress on apache#21048.
@github-actions github-actions Bot added documentation Improvements or additions to documentation development-process Related to development process of DataFusion labels Sep 15, 2026
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.92%. Comparing base (04ca5ac) to head (ef05edd).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff            @@
##             main   #25313    +/-   ##
========================================
  Coverage   81.91%   81.92%            
========================================
  Files        1134     1134            
  Lines      425655   425990   +335     
  Branches   425655   425990   +335     
========================================
+ Hits       348679   348989   +310     
  Misses      56300    56300            
- Partials    20676    20701    +25     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@2010YOUY01 2010YOUY01 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you. I've tested it locally and it works well.

Comment thread dev/rust_lint.sh
exit 1
fi
if ! python3 -c 'import yaml' &> /dev/null; then
echo "[${SCRIPT_NAME}] PyYAML is not installed for $(command -v python3). Install it in your active Python environment with: python3 -m pip install pyyaml" >&2

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: the error message could say something like 'please run through uv'

Now internal python scripts should be run through uv package manager, so

# Get dependency setup automatically
uv run ./dev/rust_lint.sh

Reference: #20414

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

development-process Related to development process of DataFusion documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants