Skip to content

ci: add AKS 4.20 e2e job and remove unsupported nodepool versions#80768

Open
celebdor wants to merge 2 commits into
openshift:mainfrom
celebdor:hypershift-aks-4-20-ci
Open

ci: add AKS 4.20 e2e job and remove unsupported nodepool versions#80768
celebdor wants to merge 2 commits into
openshift:mainfrom
celebdor:hypershift-aks-4-20-ci

Conversation

@celebdor

@celebdor celebdor commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add e2e-aks-4-20 job to test against a 4.20 hosted cluster on AKS from the main branch
  • Add initial-420 release ref needed by the new job
  • Remove sub-4.20 nodepool version references from e2e-aks and e2e-aks-4-21 since AKS does not support versions below 4.20

Why

ARO/HCP users are running OCP 4.20 on AKS, and the HyperShift operator runs from main. Without this job, features backported to 4.20 (like the Azure Workload Identity webhook in hypershift#8323) aren't exercised in our e2es at that version.

Test plan

  • Verify new e2e-aks-4-20 job appears in Prow and can be triggered
  • Verify e2e-aks and e2e-aks-4-21 still pass without the removed sub-4.20 nodepool versions (tests skip gracefully when release image is empty)

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Chores
    • Added support for OCP version 4.20 in the CI release matrix.
    • Updated Azure AKS E2E test job configurations to streamline dependencies and add testing for version 4.20.

@openshift-ci openshift-ci Bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 18, 2026
@openshift-ci openshift-ci Bot requested review from muraee and sjenning June 18, 2026 22:56
@openshift-ci

openshift-ci Bot commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: celebdor
Once this PR has been reviewed and has the lgtm label, please assign cblecker for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@coderabbitai

coderabbitai Bot commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Walkthrough

Adds a new initial-420 release candidate entry for OCP 4.20 in the CI config. Removes unused OCP_IMAGE_N2/N3/N4 dependency entries from the e2e-aks-4-21 Azure AKS E2E job. Introduces a new e2e-aks-4-20 job mirroring the 4.x AKS flow, referencing release:latest-420 and release:initial-420.

Changes

OCP 4.20 AKS E2E Job and Release Wiring

Layer / File(s) Summary
Release entry, dependency cleanup, and new e2e-aks-4-20 job
ci-operator/config/openshift/hypershift/openshift-hypershift-main.yaml
Adds initial-420 to the releases matrix (OCP ci stream, version 4.20). Removes OCP_IMAGE_N2/N3/N4 from e2e-aks-4-21 dependencies and adds OCP_IMAGE_PREVIOUS: release:initial-421. Introduces the new e2e-aks-4-20 job with OCP_IMAGE_LATEST: release:latest-420, OCP_IMAGE_PREVIOUS: release:initial-420, AKS env flags, and the hypershift-azure-aks-e2e workflow.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~5 minutes

Suggested labels

lgtm, approved, rehearsals-ack

Suggested reviewers

  • hector-vido
  • bear-redhat
  • muraee
  • bryan-cox
🚥 Pre-merge checks | ✅ 15
✅ Passed checks (15 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main changes: adding an AKS 4.20 e2e job and removing unsupported nodepool versions, which directly match the PR objectives and file modifications.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed PR modifies only YAML CI configuration (openshift-hypershift-main.yaml), not Ginkgo test definitions. Check for stable test names is not applicable.
Test Structure And Quality ✅ Passed PR modifies only CI configuration YAML (openshift-hypershift-main.yaml), not Ginkgo test code. The custom check for test structure/quality is not applicable to this PR.
Microshift Test Compatibility ✅ Passed This PR modifies CI configuration only; no new Ginkgo e2e tests are added. The check for MicroShift test compatibility is not applicable.
Single Node Openshift (Sno) Test Compatibility ✅ Passed PR modifies CI job configuration (YAML), not Ginkgo e2e test code. SNO compatibility check applies only to new Ginkgo tests (It(), Describe(), etc.), not CI configs.
Topology-Aware Scheduling Compatibility ✅ Passed The check targets deployment manifests, operator code, and controllers. This PR only modifies CI configuration (ci-operator test job definitions) and does not introduce any scheduling constraints i...
Ote Binary Stdout Contract ✅ Passed Custom check for OTE Binary Stdout Contract is not applicable; PR only modifies CI configuration YAML, contains no Go code, test code, or binary-level changes that could violate stdout contract.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed PR only adds CI job configuration (YAML), not new Ginkgo e2e tests. Check requires actual Ginkgo test code (It(), Describe(), etc.) to be added.
No-Weak-Crypto ✅ Passed No weak cryptography, custom crypto implementations, or non-constant-time secret comparisons found. The PR modifies only CI configuration files (YAML), not cryptographic code.
Container-Privileges ✅ Passed The custom check for container privileges does not apply to this PR. The modified file is a CI-Operator configuration file that defines CI/CD pipeline jobs, not Kubernetes/container manifests with...
No-Sensitive-Data-In-Logs ✅ Passed Pull request adds CI job configuration with only non-sensitive data: boolean flags, public regions, version references, no logging statements that expose passwords, tokens, API keys, or PII.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@celebdor, pj-rehearse: unable to determine affected jobs. This could be due to a branch that needs to be rebased. ERROR:

couldn't prepare candidate: couldn't rebase candidate onto 55271322141c4ee94f113bda8c45ef950c1ac767 due to conflicts
Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@ci-operator/config/openshift/hypershift/openshift-hypershift-main.yaml`:
- Around line 190-202: The e2e-aks-4-20 job is missing the explicit OCP_IMAGE_N1
dependency, causing it to default to the unversioned release:latest instead of a
4.20-compatible version. Add OCP_IMAGE_N1: release:latest-420 to the
dependencies section of the e2e-aks-4-20 job (alongside OCP_IMAGE_LATEST and
OCP_IMAGE_PREVIOUS) to ensure N-1 nodepool version compatibility testing uses a
supported 4.20 image, matching the pattern already established in the
e2e-aks-4-21 job.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 42d97b0d-0a58-49ca-94bb-2eb6d6a9c066

📥 Commits

Reviewing files that changed from the base of the PR and between 5527132 and 12631bd.

📒 Files selected for processing (1)
  • ci-operator/config/openshift/hypershift/openshift-hypershift-main.yaml

Comment on lines +190 to +202
- always_run: false
as: e2e-aks-4-20
pipeline_run_if_changed: ^(test/e2e|hypershift-operator)
steps:
cluster_profile: hypershift-aks
dependencies:
OCP_IMAGE_LATEST: release:latest-420
OCP_IMAGE_PREVIOUS: release:initial-420
env:
AUTH_THROUGH_CERTS: "true"
ENABLE_HYPERSHIFT_CERT_ROTATION_SCALE: "true"
HYPERSHIFT_AZURE_LOCATION: centralus
workflow: hypershift-azure-aks-e2e

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Description: Check how hypershift-azure-run-e2e handles OCP_IMAGE_N* variables
# and verify consistency across AKS E2E jobs

# Check the step commands script for OCP_IMAGE_N* usage
echo "=== Checking how OCP_IMAGE_N* variables are used in the step ==="
rg -A 5 -B 5 'OCP_IMAGE_N[1-4]' ci-operator/step-registry/hypershift/azure/run-e2e/hypershift-azure-run-e2e-commands.sh

# Check all e2e-aks* jobs in the config for dependency patterns
echo -e "\n=== Checking dependency patterns in e2e-aks* jobs ==="
ast-grep --pattern $'- $$$
  as: e2e-aks$_
  $$$
  dependencies:
    $$$' ci-operator/config/openshift/hypershift/openshift-hypershift-main.yaml

Repository: openshift/release

Length of output: 1331


🏁 Script executed:

# Find the step reference to see how OCP_IMAGE_N* variables are declared/defaulted
find ci-operator/step-registry -name "*hypershift-azure-run-e2e*ref.yaml" -exec cat -n {} \;

# Also check the step reference for dependency declarations
echo "=== Step reference for hypershift-azure-run-e2e ==="
find ci-operator/step-registry -path "*hypershift/azure/run-e2e/*" -type f -exec ls -la {} \;

Repository: openshift/release

Length of output: 3521


🏁 Script executed:

# Extract e2e-aks-4-21 job configuration to compare with e2e-aks-4-20
echo "=== Checking e2e-aks-4-21 job for comparison ==="
rg -A 15 'as: e2e-aks-4-21' ci-operator/config/openshift/hypershift/openshift-hypershift-main.yaml

# Also check if there's default handling in step-ref.yaml
echo -e "\n=== Looking for step reference metadata ==="
rg -B 5 -A 10 'env:' ci-operator/step-registry/hypershift/azure/run-e2e/ | head -60

Repository: openshift/release

Length of output: 4244


Provide explicit N1 dependency to match e2e-aks-4-21 pattern.

The e2e-aks-4-20 job omits explicit OCP_IMAGE_N1 and other N* dependencies, causing them to default to the unversioned release:latest from the step reference—which resolves to the current development release, not a 4.20-compatible image.

The e2e-aks-4-21 job correctly provides OCP_IMAGE_N1: release:latest-420, ensuring N-1 compatibility testing uses a supported version. For consistency and correctness, add the same to e2e-aks-4-20:

dependencies:
  OCP_IMAGE_LATEST: release:latest-420
  OCP_IMAGE_PREVIOUS: release:initial-420
  OCP_IMAGE_N1: release:latest-420

This prevents the step from passing an incompatible (unversioned latest) image to the test when checking N-1 nodepool version support, and aligns the configuration with the proven e2e-aks-4-21 pattern.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@ci-operator/config/openshift/hypershift/openshift-hypershift-main.yaml`
around lines 190 - 202, The e2e-aks-4-20 job is missing the explicit
OCP_IMAGE_N1 dependency, causing it to default to the unversioned release:latest
instead of a 4.20-compatible version. Add OCP_IMAGE_N1: release:latest-420 to
the dependencies section of the e2e-aks-4-20 job (alongside OCP_IMAGE_LATEST and
OCP_IMAGE_PREVIOUS) to ensure N-1 nodepool version compatibility testing uses a
supported 4.20 image, matching the pattern already established in the
e2e-aks-4-21 job.

ARO/HCP users are running OCP 4.20 on AKS, and the HyperShift operator
runs from main, but no AKS CI job on main tests against a 4.20 hosted
cluster. This gap means features backported to 4.20 (like the Azure
Workload Identity webhook, hypershift#8323) aren't exercised in our e2es.

Add an e2e-aks-4-20 job that creates a primary hosted cluster at 4.20.
Also remove sub-4.20 nodepool version references from e2e-aks and
e2e-aks-4-22 since AKS does not support versions below 4.20.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@celebdor celebdor force-pushed the hypershift-aks-4-20-ci branch from 12631bd to f0dd25b Compare June 18, 2026 23:06
@openshift-ci openshift-ci Bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 18, 2026
Set OCP_IMAGE_N1 to release:latest-420 to match the e2e-aks-4-21
pattern and prevent the step ref default (release:latest) from
injecting a mismatched version.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@celebdor

Copy link
Copy Markdown
Contributor Author

/pj-rehearse e2e-aks-4-20

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@celebdor: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

[REHEARSALNOTIFIER]
@celebdor: the pj-rehearse plugin accommodates running rehearsal tests for the changes in this PR. Expand 'Interacting with pj-rehearse' for usage details. The following rehearsable tests have been affected by this change:

Test name Repo Type Reason
pull-ci-openshift-hypershift-main-address-review-comments openshift/hypershift presubmit Ci-operator config changed
pull-ci-openshift-hypershift-main-agentic-qe-aws openshift/hypershift presubmit Ci-operator config changed
pull-ci-openshift-hypershift-main-e2e-agent-connected-ovn-ipv4-metal-backuprestore openshift/hypershift presubmit Ci-operator config changed
pull-ci-openshift-hypershift-main-e2e-aks openshift/hypershift presubmit Ci-operator config changed
pull-ci-openshift-hypershift-main-e2e-aks-4-22 openshift/hypershift presubmit Ci-operator config changed
pull-ci-openshift-hypershift-main-e2e-aks-override openshift/hypershift presubmit Ci-operator config changed
pull-ci-openshift-hypershift-main-e2e-aws openshift/hypershift presubmit Ci-operator config changed
pull-ci-openshift-hypershift-main-e2e-aws-4-22 openshift/hypershift presubmit Ci-operator config changed
pull-ci-openshift-hypershift-main-e2e-aws-autonode openshift/hypershift presubmit Ci-operator config changed
pull-ci-openshift-hypershift-main-e2e-aws-external-oidc-techpreview openshift/hypershift presubmit Ci-operator config changed
pull-ci-openshift-hypershift-main-e2e-aws-metrics openshift/hypershift presubmit Ci-operator config changed
pull-ci-openshift-hypershift-main-e2e-aws-minimal openshift/hypershift presubmit Ci-operator config changed
pull-ci-openshift-hypershift-main-e2e-aws-override openshift/hypershift presubmit Ci-operator config changed
pull-ci-openshift-hypershift-main-e2e-aws-ovn-conformance-ccm openshift/hypershift presubmit Ci-operator config changed
pull-ci-openshift-hypershift-main-e2e-aws-ovn-conformance-techpreview openshift/hypershift presubmit Ci-operator config changed
pull-ci-openshift-hypershift-main-e2e-aws-techpreview openshift/hypershift presubmit Ci-operator config changed
pull-ci-openshift-hypershift-main-e2e-aws-upgrade-hypershift-operator openshift/hypershift presubmit Ci-operator config changed
pull-ci-openshift-hypershift-main-e2e-azure-aks-external-oidc-techpreview openshift/hypershift presubmit Ci-operator config changed
pull-ci-openshift-hypershift-main-e2e-azure-aks-ovn-conformance openshift/hypershift presubmit Ci-operator config changed
pull-ci-openshift-hypershift-main-e2e-azure-kubevirt-ovn openshift/hypershift presubmit Ci-operator config changed
pull-ci-openshift-hypershift-main-e2e-azure-v2-self-managed openshift/hypershift presubmit Ci-operator config changed
pull-ci-openshift-hypershift-main-e2e-conformance openshift/hypershift presubmit Ci-operator config changed
pull-ci-openshift-hypershift-main-e2e-kubevirt-aws-ovn openshift/hypershift presubmit Ci-operator config changed
pull-ci-openshift-hypershift-main-e2e-kubevirt-aws-ovn-reduced openshift/hypershift presubmit Ci-operator config changed
pull-ci-openshift-hypershift-main-e2e-kubevirt-azure-ovn openshift/hypershift presubmit Ci-operator config changed

A total of 46 jobs have been affected by this change. The above listing is non-exhaustive and limited to 25 jobs.

A full list of affected jobs can be found here

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@celebdor: job(s): e2e-aks-4-20 either don't exist or were not found to be affected, and cannot be rehearsed

@openshift-ci

openshift-ci Bot commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

@celebdor: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/generated-config 53406ed link true /test generated-config

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant