[Backport v1.29] [TON-743] Add EKS add-on credential sync resources

[dd-octo-sts]

Backport 5d82e4a from #3291.

---

What does this PR do?

Adds opt-in AWS Secrets Manager credential synchronization to the Datadog Operator EKS add-on chart.

When credentialsSecretSync.apiKeySecretId, credentialsSecretSync.appKeySecretId, or both are configured, the chart renders SecretProviderClass/datadog-credentials, which synchronizes the configured values to the api-key and app-key keys in Secret/<kubernetesSecretName> (datadog-secret by default).

To have the Operator use this Secret, set its name in datadog-operator.apiKeyExistingSecret and/or datadog-operator.appKeyExistingSecret.

To use this feature, a pod in the add-on namespace must mount a Secrets Store CSI volume referencing SecretProviderClass/datadog-credentials. The volume can be mounted to the Operator pod using datadog-operator.volumes and datadog-operator.volumeMounts (see QA example below). A future version of the Datadog Operator chart will provide a dedicated option for enabling this configuration. The pod service account must have an EKS Pod Identity association with an IAM role that can read the configured secrets. The cluster must have the EKS Pod Identity Agent and AWS Secrets Store CSI Driver Provider add-ons installed, with Kubernetes Secret sync enabled.

Motivation

Bootstraps Datadog API and application-key credentials from AWS Secrets Manager when installing the Datadog Operator EKS add-on.

Minimum Agent Versions

• Datadog Operator: N/A
• Datadog Operator chart: N/A
• Agent: N/A
• Cluster Agent: N/A

Describe your test plan

Automated/local validation performed:

• Linted and packaged the complete wrapper chart using the vendored datadog-operator dependency.
• Rendered API-key-only, application-key-only, and combined configurations.
• Verified the enabled configuration renders SecretProviderClass/datadog-credentials and the Operator Deployment can mount its CSI volume.
• Verified the chart does not create a credentials-sync Deployment or require a helper container image.
• Rendered the default configuration and verified that no credentials-sync resources are created.
• Validated the Marketplace configuration schema as JSON.
• Upgraded a self-managed installation on an EKS 1.35 cluster with the prerequisite add-ons. The Operator pod mounted both objects, SecretProviderClassPodStatus reported mounted=true, and SHA-256 comparisons confirmed both synchronized keys matched their Secrets Manager values.
• Ran git diff --check.

QA

1. On an EKS cluster with a Linux EC2 node, install the prerequisite add-ons and wait for them to become ACTIVE:

# If using EKS Auto Mode, omit installing this add-on: 
aws eks create-addon --region <REGION> --cluster-name <CLUSTER> \
 --addon-name eks-pod-identity-agent

aws eks create-addon --region <REGION> --cluster-name <CLUSTER> \
 --addon-name aws-secrets-store-csi-driver-provider \
 --configuration-values '{"secrets-store-csi-driver":{"syncSecret":{"enabled":true}}}'

2. In Secrets Manager, create separate API- and App-key secrets as raw plaintext values (not JSON key-value pairs). Use the default aws/secretsmanager encryption key. Record both secret ARNs.

   Create a Pod Identity IAM role using these policies, replacing the secret ARN placeholders:

   trust-policy.json:

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": {"Service": "pods.eks.amazonaws.com"},
    "Action": ["sts:AssumeRole", "sts:TagSession"],
    "Condition": {"StringEquals": {
      "aws:RequestTag/kubernetes-namespace": "datadog-addon-qa",
      "aws:RequestTag/kubernetes-service-account": "operator-eks-addon-datadog-operator"
    }}
  }]
}

permissions-policy.json:

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Action": ["secretsmanager:GetSecretValue", "secretsmanager:DescribeSecret"],
    "Resource": ["<API_KEY_SECRET_ARN>", "<APP_KEY_SECRET_ARN>"]
  }]
}

aws iam create-role --role-name datadog-addon-qa-secret-sync \
  --assume-role-policy-document file://<PATH_TO>/trust-policy.json
aws iam put-role-policy --role-name datadog-addon-qa-secret-sync \
  --policy-name ReadDatadogAddonCredentials \
  --policy-document file://<PATH_TO>/permissions-policy.json

ROLE_ARN="$(aws iam get-role --role-name datadog-addon-qa-secret-sync \
  --query 'Role.Arn' --output text)"
aws eks create-pod-identity-association --region <REGION> --cluster-name <CLUSTER> \
  --namespace datadog-addon-qa --service-account operator-eks-addon-datadog-operator \
  --role-arn "$ROLE_ARN"

3. Build the chart locally using marketplaces/charts/operator-eks-addon/README.md. Install it with values equivalent to:

datadog-operator:
  image:
    repository: gcr.io/datadoghq/operator
  apiKeyExistingSecret: datadog-secret
  appKeyExistingSecret: datadog-secret # Omit when not testing an application key.
  serviceAccount:
    create: true
    name: operator-eks-addon-datadog-operator
  volumes:
    - name: datadog-credentials
      csi:
        driver: secrets-store.csi.k8s.io
        readOnly: true
        volumeAttributes:
          secretProviderClass: datadog-credentials
  volumeMounts:
    - name: datadog-credentials
      mountPath: /mnt/datadog-credentials
      readOnly: true

credentialsSecretSync:
  kubernetesSecretName: datadog-secret
  apiKeySecretId: <API_KEY_SECRET_ARN>
  appKeySecretId: <APP_KEY_SECRET_ARN>

helm upgrade --install operator-eks-addon-qa ./operator-eks-addon-0.1.29.tgz \
  --namespace datadog-addon-qa \
  --create-namespace \
  --values ./qa-values.yaml

4. Verify synchronization:

kubectl --namespace datadog-addon-qa rollout status deployment/operator-eks-addon-qa-datadog-operator
kubectl --namespace datadog-addon-qa get secretproviderclass datadog-credentials
kubectl --namespace datadog-addon-qa get secretproviderclasspodstatus
kubectl --namespace datadog-addon-qa get secret datadog-secret
kubectl --namespace datadog-addon-qa get deployment datadog-ascp-secret-sync

Confirm the pod status reports mounted=true, datadog-secret contains api-key and app-key, and their SHA-256 hashes match the Secrets Manager values without printing them. The final command should report NotFound because there is no helper Deployment.

Also install or render with credentialsSecretSync: null and verify that SecretProviderClass/datadog-credentials is absent.

AWS references: ASCP EKS add-on and EKS Pod Identity roles.

Checklist

• PR has at least one valid label: bug, enhancement, refactoring, documentation, tooling, and/or dependencies
• PR has a milestone or the qa/skip-qa label
• All commits are signed (see: signing commits)

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a6091500ce

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

Add a pod mount to drive credential secret sync

When a user enables credentialsSecretSync and points datadog-operator.*ExistingSecret at this secret, this manifest still will not create the Kubernetes Secret by itself: the Secrets Store CSI Driver only syncs secretObjects after a pod mounts the SecretProviderClass, and this chart does not render any pod/volume referencing datadog-credentials. In that configuration the operator waits on a missing Secret unless users deploy an extra mounting pod outside the add-on, so the chart should render that mount/keeper pod or mount the SPC on the operator deployment together with this resource.

Useful? React with 👍 / 👎.

[fanny-jiang]

Users can mount the datadog-credentials volume to the operator pod via the Add-on values or to any pod of their choice running in the same namespace.

[datadog-datadog-us1-prod]

 

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 46.85% (+0.00%)

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: a609150 | Docs | Datadog PR Page | Give us feedback!}