Skip to content

Move CloudBatchSubmitJobOperator job normalization out of __init__ - #71201

Open
yunusakca wants to merge 1 commit into
apache:mainfrom
yunusakca:fix-cloud-batch-submit-job-operator-init
Open

Move CloudBatchSubmitJobOperator job normalization out of __init__#71201
yunusakca wants to merge 1 commit into
apache:mainfrom
yunusakca:fix-cloud-batch-submit-job-operator-init

Conversation

@yunusakca

Copy link
Copy Markdown

Moves CloudBatchSubmitJobOperator's Job protobuf → dict normalization out of
__init__, as part of the exemption-list burn-down tracked in #70296.

Template fields are rendered after the constructor runs, so reading a template
field's value inside __init__ operates on the un-rendered Jinja expression
rather than the real value. The job protobuf-to-dict conversion — added for
#37217 so the renderer can descend into nested fields such as a runnable's
container commands — is exactly that pattern.

The conversion has to run before Jinja rendering, not after: moving it to
execute() would silently stop nested Jinja expressions inside a Job
protobuf from ever being rendered. It now lives in prepare_template(), the
hook Airflow runs after construction but before rendering, matching the pattern
CloudBuildCreateBuildOperator already uses in this provider.

CloudBatchSubmitJobOperator's entry is removed from
validate_operators_init_exemptions.txt.

The existing rendering test already covers both a dict and a protobuf Job
input; a new test asserts the normalization happens in prepare_template()
rather than the constructor.

related: #70296


Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Sonnet 5 and Opus 5)

Generated-by: Claude Code (Sonnet 5 and Opus 5) following the guidelines

Template fields are rendered after the constructor runs, so any read of a
template-field value inside __init__ operates on the un-rendered Jinja
expression rather than the real value. CloudBatchSubmitJobOperator's job
protobuf-to-dict conversion (added for apache#37217, so the template renderer can
descend into nested fields) is exactly that pattern.

The conversion has to happen before Jinja rendering, not after, or nested
Jinja expressions inside a protobuf Job would never be discovered by the
renderer. Move it into prepare_template(), the hook Airflow already runs
after construction but before rendering — mirroring the same pattern used
by CloudBuildCreateBuildOperator in this provider.

Part of the exemption-list burn-down tracked in apache#70296.
@boring-cyborg

boring-cyborg Bot commented Aug 5, 2026

Copy link
Copy Markdown

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
Here are some useful points:

  • Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example Dag that shows how users should use it.
  • Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
  • Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:dev-tools area:providers backport-to-v3-3-test Backport to v3-3-test provider:google Google (including GCP) related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant