Sync v3-3-stable with v3-3-test to release 3.3.1rc1 - #71197
Open
vatsrahul1001 wants to merge 297 commits into
Open
Sync v3-3-stable with v3-3-test to release 3.3.1rc1#71197vatsrahul1001 wants to merge 297 commits into
vatsrahul1001 wants to merge 297 commits into
Conversation
…n CI (#69411) (#69580) * Speed up lang-SDK k8s test by building Go/Java natively in CI The Multi-Lang KubernetesExecutor system test builds a Go bundle and a Java jar inside throwaway toolchain containers. On a dev host the module, Gradle-distribution and dependency caches persist between runs, but CI runners are ephemeral and nothing cached them, so every scheduled run re-pulled the golang/temurin images and re-downloaded the Gradle distribution and all dependencies — adding roughly ten minutes to each of the six KubernetesExecutor variants that run the test. Build the artifacts with the host toolchain in CI instead, provisioned and cached through actions/setup-go and actions/setup-java, so the toolchain image pulls and cold dependency downloads no longer happen on every run. Local runs keep the containerised build so a dev host still needs neither Go nor a JDK installed. * Centralize lang-SDK JDK version and add a bustable cache key in CI Follow-ups to the native lang-SDK build: - Move the JDK version (17) into a JAVA_SDK_VERSION breeze constant and surface it as the java-sdk-version selective-checks / build-info output, so the k8s workflow reads it instead of hardcoding the version in YAML. - Restore the Go module/build and Gradle caches with an explicit actions/cache keyed on a "-v1-" salt rather than the setup-* built-in caching, giving an in-repo knob to force-invalidate a poisoned cache (bump the salt) without waiting for a dependency change to rotate it. (cherry picked from commit 673183b) Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
) (#69604) Airflow 3 hid the conf column even though the API and column definition already existed. Operators triggering Dags with run conf need that context in the list view without toggling columns manually. (cherry picked from commit 50c44b5) closes: #53382 Co-authored-by: Ashutosh Shaha <61512480+ashutosh264@users.noreply.github.com>
… 6x per run (#69500) (#69587) The Multi-Lang KubernetesExecutor coordinator system test was gated on KubernetesExecutor + use-standard-naming==false, but that condition is true once per kubernetes-combo, so it ran redundantly across all six Python/K8s combos on ci-amd (and again on ci-arm) - adding ~4 minutes of Go/Java build and coordinator run to each of those six system-test jobs and pushing them to ~20 minutes, well above the rest of the matrix. The test is executor/naming/version-independent, so run it once instead: a dedicated tests-kubernetes-lang-sdk job on a single default Python-Kubernetes combo that provisions the native Go/Java toolchains (cached) and runs only the lang-SDK test (-k test_lang_sdk_combined_dag_succeeds), not the full suite or the upgrade path. The regular system-test matrix no longer runs it at all, so those six jobs drop back to ~15 minutes. Also scope the Go/Java build caches by runner.arch: the previous key used only runner.os (Linux), which would collide between amd64 and arm64 runners now that the dedicated job runs on both. (cherry picked from commit 586e658) Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
…9495) (#69620) The database session in the FastAPI layer is injected via the SessionDep dependency, which automatically commits the transaction when a request completes successfully. Explicitly calling session.commit() inside the route handler is redundant and circumvents the dependency's intended transaction lifecycle management. This brings the route handlers in line with the codebase guideline that prohibits functions accepting a session parameter from calling commit. (cherry picked from commit 5e23c6a) Co-authored-by: Kushagra Gupta <kushagrab424@gmail.com>
Only Dag pages set a page-specific browser tab title today, so every other page shows just the instance name and multiple open Airflow tabs cannot be told apart. Set a localized, page-aware title on the remaining pages so each tab identifies the page it is showing. (cherry picked from commit 493312b)
* Fix log line-number links highlighting * Add regression test for log line-number to visible index mapping (cherry picked from commit 9dec3eb) Co-authored-by: Guan-Ming Chiu <105915352+guan404ming@users.noreply.github.com>
… updates (#69725) Bumps the 3-3-auth-ui-package-updates group with 10 updates in the /airflow-core/src/airflow/api_fastapi/auth/managers/simple/ui directory: | Package | From | To | | --- | --- | --- | | [react-hook-form](https://github.com/react-hook-form/react-hook-form) | `7.80.0` | `7.81.0` | | [react-router-dom](https://github.com/remix-run/react-router/tree/HEAD/packages/react-router-dom) | `7.18.0` | `7.18.1` | | [@typescript-eslint/eslint-plugin](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/eslint-plugin) | `8.62.0` | `8.62.1` | | [@typescript-eslint/parser](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/parser) | `8.62.0` | `8.62.1` | | [@typescript-eslint/utils](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/utils) | `8.62.0` | `8.62.1` | | [@vitest/coverage-v8](https://github.com/vitest-dev/vitest/tree/HEAD/packages/coverage-v8) | `4.1.9` | `4.1.10` | | [prettier](https://github.com/prettier/prettier) | `3.9.3` | `3.9.4` | | [typescript-eslint](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/typescript-eslint) | `8.62.0` | `8.62.1` | | [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) | `8.1.0` | `8.1.3` | | [vitest](https://github.com/vitest-dev/vitest/tree/HEAD/packages/vitest) | `4.1.9` | `4.1.10` | Updates `react-hook-form` from 7.80.0 to 7.81.0 - [Release notes](https://github.com/react-hook-form/react-hook-form/releases) - [Changelog](https://github.com/react-hook-form/react-hook-form/blob/master/CHANGELOG.md) - [Commits](react-hook-form/react-hook-form@v7.80.0...v7.81.0) Updates `react-router-dom` from 7.18.0 to 7.18.1 - [Release notes](https://github.com/remix-run/react-router/releases) - [Changelog](https://github.com/remix-run/react-router/blob/react-router-dom@7.18.1/packages/react-router-dom/CHANGELOG.md) - [Commits](https://github.com/remix-run/react-router/commits/react-router-dom@7.18.1/packages/react-router-dom) Updates `@typescript-eslint/eslint-plugin` from 8.62.0 to 8.62.1 - [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases) - [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/eslint-plugin/CHANGELOG.md) - [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v8.62.1/packages/eslint-plugin) Updates `@typescript-eslint/parser` from 8.62.0 to 8.62.1 - [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases) - [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/parser/CHANGELOG.md) - [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v8.62.1/packages/parser) Updates `@typescript-eslint/utils` from 8.62.0 to 8.62.1 - [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases) - [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/utils/CHANGELOG.md) - [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v8.62.1/packages/utils) Updates `@vitest/coverage-v8` from 4.1.9 to 4.1.10 - [Release notes](https://github.com/vitest-dev/vitest/releases) - [Changelog](https://github.com/vitest-dev/vitest/blob/main/docs/releases.md) - [Commits](https://github.com/vitest-dev/vitest/commits/v4.1.10/packages/coverage-v8) Updates `prettier` from 3.9.3 to 3.9.4 - [Release notes](https://github.com/prettier/prettier/releases) - [Changelog](https://github.com/prettier/prettier/blob/main/CHANGELOG.md) - [Commits](prettier/prettier@3.9.3...3.9.4) Updates `typescript-eslint` from 8.62.0 to 8.62.1 - [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases) - [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/typescript-eslint/CHANGELOG.md) - [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v8.62.1/packages/typescript-eslint) Updates `vite` from 8.1.0 to 8.1.3 - [Release notes](https://github.com/vitejs/vite/releases) - [Changelog](https://github.com/vitejs/vite/blob/main/packages/vite/CHANGELOG.md) - [Commits](https://github.com/vitejs/vite/commits/v8.1.3/packages/vite) Updates `vitest` from 4.1.9 to 4.1.10 - [Release notes](https://github.com/vitest-dev/vitest/releases) - [Changelog](https://github.com/vitest-dev/vitest/blob/main/docs/releases.md) - [Commits](https://github.com/vitest-dev/vitest/commits/v4.1.10/packages/vitest) --- updated-dependencies: - dependency-name: react-hook-form dependency-version: 7.81.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: 3-3-auth-ui-package-updates - dependency-name: react-router-dom dependency-version: 7.18.1 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: 3-3-auth-ui-package-updates - dependency-name: "@typescript-eslint/eslint-plugin" dependency-version: 8.62.1 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: 3-3-auth-ui-package-updates - dependency-name: "@typescript-eslint/parser" dependency-version: 8.62.1 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: 3-3-auth-ui-package-updates - dependency-name: "@typescript-eslint/utils" dependency-version: 8.62.1 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: 3-3-auth-ui-package-updates - dependency-name: "@vitest/coverage-v8" dependency-version: 4.1.10 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: 3-3-auth-ui-package-updates - dependency-name: prettier dependency-version: 3.9.4 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: 3-3-auth-ui-package-updates - dependency-name: typescript-eslint dependency-version: 8.62.1 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: 3-3-auth-ui-package-updates - dependency-name: vite dependency-version: 8.1.3 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: 3-3-auth-ui-package-updates - dependency-name: vitest dependency-version: 4.1.10 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: 3-3-auth-ui-package-updates ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…69729) (#69756) Upgrading the pinned Flask-AppBuilder dependency is deceptively involved: the FAB provider vendors and subclasses large parts of FAB's security manager, so a bump must be reconciled against that vendored code, the alignment-test tripwire, and the generated provider README. Capturing the verified procedure as a skill — and a matching providers/fab/CONTRIBUTING.rst that points contributors at it and records past bumps — lets future FAB upgrades follow the same checklist instead of rediscovering it each time. The skill lives vendor-neutrally in .agents/skills with .claude and .github relays, matching the repository's existing skill layout. (cherry picked from commit 621192d) Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
--- updated-dependencies: - dependency-name: github/codeql-action/init dependency-version: 4.36.3 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: github-actions-updates - dependency-name: github/codeql-action/autobuild dependency-version: 4.36.3 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: github-actions-updates - dependency-name: github/codeql-action/analyze dependency-version: 4.36.3 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: github-actions-updates - dependency-name: astral-sh/setup-uv dependency-version: 8.3.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: github-actions-updates ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…updates (#69724) --- updated-dependencies: - dependency-name: "@xyflow/react" dependency-version: 12.11.2 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: 3-3-core-ui-package-updates - dependency-name: react-hook-form dependency-version: 7.81.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: 3-3-core-ui-package-updates - dependency-name: "@vitest/coverage-v8" dependency-version: 4.1.10 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: 3-3-core-ui-package-updates - dependency-name: vite dependency-version: 8.1.3 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: 3-3-core-ui-package-updates - dependency-name: vitest dependency-version: 4.1.10 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: 3-3-core-ui-package-updates ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…ommitters (#69417) (#69634) * Add framework for delegating providers release to non-PMC committers The provider release cycle currently assumes a PMC member drives every step, yet ASF policy only reserves the binding vote and dist/release publication to the PMC. Documenting a clear split lets a non-PMC committer run the bulk of the process while the PMC signs, publishes to PyPI, and casts the binding votes — both spreading the release-manager load and giving prospective PMC members a supervised on-ramp. Generated-by: Claude Code (Opus 4.8 1M context) * Merge the provider build step into the PMC-owned release block A PMC member who only signs artifacts built by someone else does not actually know what they are signing, per ASF release policy guidance on owned/controlled hardware. Move the build step into the PMC block so the same PMC member builds, signs, commits to dist/dev, and publishes the PyPI RC without handing control back to the Delegate. (cherry picked from commit 4d56e6c) Co-authored-by: Shahar Epstein <60007259+shahar1@users.noreply.github.com>
) While running through the steps today I noticed a few small things to fix: - Clarify wording provider release SVN check - Drop duplicate cd in verification steps (cherry picked from commit 478f25e) Co-authored-by: Niko Oliveira <onikolas@amazon.com>
…rom log format (#69402) (#69787) When a record reaches the percent formatter without callsite information, for example a stdlib warning routed through the logging bridge, the process and thread fields are absent. The formatter fell back to the "(unknown)" string for them, so a format string using the numeric "%(process)d" or "%(thread)d" specifiers raised "TypeError: %d format: a real number is required" and could take down the scheduler at startup. Give those two numeric callsite parameters a numeric fallback of 0, the same way lineno is already handled, so the format never receives a string where a number is expected. (cherry picked from commit fd7d535) Co-authored-by: Anas Khan <anxkhn28@gmail.com>
…#69763) (#69784) The multiprocessing pools in the providers-metadata generation flow used the platform-default start method (fork on Linux). Before the pools are created the parent process has already used GitPython — which opens persistent `git cat-file --batch` subprocesses and is not fork-safe — and holds open network sockets from the version and constraints downloads. Forking that state into the workers left them with broken inherited file descriptors, so the pool deadlocked and the command hung forever, most reliably when --refresh-constraints-and-airflow-releases forces the parent through git and the network before forking. Switching these pools to the spawn start method gives each worker a clean interpreter with no inherited git subprocesses or sockets. (cherry picked from commit 5833692) Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
…run trigger endpoint (#61550) (#69719) * feat: adding the option to add bundle version parameter on dag run trigger endpoint * removing code comments and adding the new parameter to the schema * feat: adding the new parameter to schema and types files * fix: fixing format and missing field in class * fix: fixing the usage of bundle version in orm dagrun creation * feat: added migration file * feat: adding unit test for bundle version parameter dagrun endpoint * fix: fixing the unit test by removing unecessary asserts * fix: removing duplicate dag version check and raising exception accordingly * fix: adding a raise exception when passing incorrect bundle version * fix: verify exception message to raise accordingly * fix: wiring to bundle version and moved to the correct version target * fix: missed adding bundle_version field in 2 unit tests response assertion * fix: adding missing bundle version field in tests * fix: handling dag and dag's tasks correctly when passing bundle version * feat: added new airflow exception to handle no dag version has found * fix: assigning run dag to dag version when only using bundle version * fix: fixing assertion equal failing * fix: removed unecessary init files after prek hook added them * fix: adding support for all the dag-level callbacks * feat: adding versioning test for bundle version parameter * fix: rebasing branch * feat: adding the option to add bundle version parameter on dag run trigger endpoint * removing code comments and adding the new parameter to the schema * feat: adding the new parameter to schema and types files * fix: fixing format and missing field in class * fix: fixing the usage of bundle version in orm dagrun creation * feat: added migration file * feat: adding unit test for bundle version parameter dagrun endpoint * fix: fixing the unit test by removing unecessary asserts * fix: removing duplicate dag version check and raising exception accordingly * fix: adding a raise exception when passing incorrect bundle version * fix: verify exception message to raise accordingly * fix: wiring to bundle version and moved to the correct version target * fix: missed adding bundle_version field in 2 unit tests response assertion * fix: adding missing bundle version field in tests * fix: handling dag and dag's tasks correctly when passing bundle version * feat: added new airflow exception to handle no dag version has found * fix: assigning run dag to dag version when only using bundle version * fix: fixing assertion equal failing * fix: removed unecessary init files after prek hook added them * fix: adding support for all the dag-level callbacks * fix: prek didn't updated datamodels for airflowctl * fix: adding bundle version support also in asset route, fixing execution api versioning, and also getting latest version of dag with bundle version * fix: duplicate import and duplicate methods * fix: remove duplicate DagVersion lookup and unintended dag swap for non-explicit triggers * fix: reverting uv lock due to unnecessary changes * fix: fixing test to resolve ci errors * fix: restore uv.lock to upstream/main * fix: remove double code, moving to new CalVer file, raising correct exception and add more callbacks support in test * fix: instead of always using latest dag using the correct dag if bundle version is passed * fix: missing fields in schema.json * Regenerate supervisor schema snapshot after rebase onto main The schema.json was stale: it contained legacy top-level $defs (DagRun, DagRunState, DagRunType, JsonValue, and nested asset reference types) that the generator no longer emits after upstream main restructured them. bundle_version also needed to land at the correct offset in the generated output. * fix: consolidate bundle version resolution, fix double deserialization, drop bundle_version from Execution API * restored provider_dependencies.json.sha256sum * Fix bundle_version overwritten by sync_dag_to_db and stale supervisor schema snapshot * revert accidently changes to uv.lock * Move AddPartitionDateField to 2026-06-30 version to match partition_date introduction timeline * Remove dead AirflowBadRequest handler and unintended execution API version split * Derive run context from requested bundle version on trigger * Validate partition_key and allowed_run_types against the requested bundle version * Revert accidental uv.lock * Regenerate private UI OpenAPI spec * Removing unnecessary callback attributes * Remove unnecessary test due to remove bundle version from execution api --------- (cherry picked from commit 1ce3f17) Co-authored-by: Itay Adler <35665981+itayweb@users.noreply.github.com> Co-authored-by: Itay Adler <itayadler@MacBook-Pro-sl-Itay.local> Co-authored-by: Itay Adler <itayadler@Itays-Mac-mini.lan>
…ml (#69473) (#69653) * refactor: Added static check for conn-fields defined in provider.yaml to make sure those correspond with the ones defined in the get_connection_form_widgets method of the corresponding hook * Update scripts/in_container/run_provider_yaml_files_check.py * refactor: Updated comment in _get_widget_keys method --------- (cherry picked from commit 53de1c1) Co-authored-by: David Blain <info@dabla.be> Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
boring-cyborg
Bot
added
area:logging
area:production-image
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
13 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Time for 3.3.1rc1
Release Notes
Version update
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.