Eliminate trust policy race by migrating emr_eks test to Pod Identity - #71207
Open
ramitkataria wants to merge 1 commit into
Open
Eliminate trust policy race by migrating emr_eks test to Pod Identity#71207ramitkataria wants to merge 1 commit into
ramitkataria wants to merge 1 commit into
Conversation
The test shares one EMR job execution role across runs while each run creates its own EKS cluster. With IRSA, every run had to rewrite the shared role's trust policy to federate its own cluster's OIDC provider, and pruning old statements to stay under the trust policy size limit meant concurrent runs deleted each other's trust entries, failing jobs with AssumeRoleWithWebIdentity AccessDenied. IAM offers no compare-and-swap on trust policies, so this race cannot be fixed while runs mutate shared IAM state. EKS Pod Identity (supported by EMR on EKS since emr-7.3.0) removes the shared mutable state: the role keeps one static trust policy and each run creates Pod Identity associations scoped to its own cluster, also avoiding the 4096-char trust policy cap that limits IRSA to roughly a dozen clusters per role. Pod Identity is also AWS's recommended mechanism for pod credentials: > Unless you have specific usecases for IRSA, we recommend you use > EKS Pod Identities when using EKS. https://docs.aws.amazon.com/eks/latest/best-practices/identity-and-access-management.html#_eks_pod_identities_compared_to_irsa
ramitkataria
force-pushed
the
ramitkataria/eks-pod-identity
branch
from
August 5, 2026 23:24
39c4d48 to
b17b493
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The test shares one EMR job execution role across runs while each run creates its own EKS cluster. With IRSA, every run had to rewrite the shared role's trust policy to federate its own cluster's OIDC provider, and pruning old statements to stay under the trust policy size limit meant concurrent runs deleted each other's trust entries, failing jobs with AssumeRoleWithWebIdentity AccessDenied. IAM offers no compare-and-swap on trust policies, so this race cannot be fixed while runs mutate shared IAM state.
EKS Pod Identity (supported by EMR on EKS since emr-7.3.0) removes the shared mutable state: the role keeps one static trust policy and each run creates Pod Identity associations scoped to its own cluster, also avoiding the 4096-char trust policy cap that limits IRSA to roughly a dozen clusters per role.
Pod Identity is also AWS's recommended mechanism for pod credentials:
Was generative AI tooling used to co-author this PR?
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.