EPIC: Add Local Scripts to Reproduce Full CI

[2010YOUY01]

Is your feature request related to a problem or challenge?

It would be great to have a local script to run full CI, for all functional and non-functional(e.g. lints) tests, this helps development process for human and AI agents.

./dev/ci.sh      # Run full CI locally

Here is a previous work that provides a single script to run some non-functional tests

• Add Local Scripts to Reproduce Full CI and Perform Auto-Fixes #19227

The remaining work is to

• Continue supporting more non-functional tests
• Create a similar script for all functional tests (rust tests, sqllogictests, extended tests, ...)

(might use xtask instead of scripts, see #23414)
### Tracker for non-functional tests

./dev/rust_lint.sh should fully reproduce non-functional CI locally.
./dev/rust_lint.sh --write --allow-dirty should do best-effort fixing where applicable.

Today the script covers cargo fmt, clippy, Cargo.toml formatting, license headers, typos, markdown prettier, and cargo doc.

Remaining CI checks that could be migrated similarly:

• Semver / breaking API detector: .github/workflows/breaking_changes_detector.yml, ci/scripts/changed_crates.sh
• Markdown link check: .github/workflows/dev.yml, ci/scripts/markdown_link_check.sh
• ASF required status check validation: .github/workflows/dev.yml, ci/scripts/check_asf_yaml_status_checks.py
• Security audit: .github/workflows/audit.yml, cargo audit
• Dependency checks: .github/workflows/dependencies.yml, dev/depcheck, cargo machete
• Large file check: .github/workflows/large_files.yml
• CodeQL for GitHub Actions: .github/workflows/codeql.yml
• Docs HTML build: .github/workflows/docs_pr.yaml, docs/build.sh
• Config/function docs freshness: .github/workflows/rust.yml, dev/update_config_docs.sh, dev/update_function_docs.sh
• Examples README freshness: .github/workflows/rust.yml, ci/scripts/check_examples_docs.sh

Potential Improvements to rust_lint.sh

• Better naming.
• Support running subsets, so developers and AI agents can run only checks likely affected by local changes.