NO-ISSUE: Increase greenboot and robot timeouts for release scenarios - #7319
NO-ISSUE: Increase greenboot and robot timeouts for release scenarios#7319agullon wants to merge 2 commits into
Conversation
Release scenarios running upgrade paths with LVMS workloads followed by full standard suites were hitting timeout limits under I/O contention on x86 (c5.metal, 4750 Mbps EBS) when many VMs boot and pull images in parallel. Increase greenboot healthcheck timeout from 600s to 1200s and robot framework timeout from 30m (CI-overridden to 45m) to 60m for release scenarios only. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> pre-commit.check-secrets: ENABLED
|
Pipeline controller notification For optional jobs, comment This repository is configured in: LGTM mode |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository YAML (base), Central YAML (inherited) Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (5)
💤 Files with no reviewable changes (5)
Included review availability: Your plan provides up to 12 included reviews per hour; 10 remain after this review. WalkthroughRelease scenarios now receive ChangesRelease scenario timeout configuration
Estimated code review effort: 1 (Trivial) | ~2 minutes Merge Risk: ⚪ Minimal · up to Release scenarios receive higher centralized timeout ceilings while non-release scenarios remain unchanged. No current merge-blocking risk is identified. Suggested labels: 🚥 Pre-merge checks | ✅ 15✅ Passed checks (15 passed)
Full details: Docstring CoverageExplanation No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 1 files. Full details: Stable And Deterministic Test NamesExplanation PASS — The pull request changes only CI/scenario shell scripts. The added lines export Full details: Test Structure And QualityExplanation The pull request changes six shell scripts only. The diff from the base commit contains no Go files, Ginkgo Full details: Microshift Test CompatibilityExplanation PASS — The pull request changes six shell scripts only. The diff adds timeout exports and removes redundant scenario overrides; it adds no Ginkgo tests or test cases using OpenShift APIs or unsupported MicroShift features. Full details: Single Node Openshift (Sno) Test CompatibilityExplanation PASS: The pull request changes only Full details: Topology-Aware Scheduling CompatibilityExplanation PASS: The pull request changes only CI/scenario shell scripts. It adds release-scoped timeout environment variables and removes redundant per-scenario timeout overrides. The available commit range contains no deployment manifests, operator/controller code, or topology scheduling constraints such as affinity, node selectors, tolerations, replica logic, topology spread, or PDBs. The topology-aware scheduling check is therefore not applicable. Full details: Ote Binary Stdout ContractExplanation PASS: The full PR range changes only six shell scenario/CI files. The only additions are Full details: Ipv6 And Disconnected Network Test CompatibilityExplanation PASS: The pull request changes only six shell scripts. It adds centralized timeout exports and removes redundant per-scenario assignments. It adds no Ginkgo tests, IPv4-specific logic, or external connectivity code, so this check is not applicable. Full details: No-Weak-CryptoExplanation PASS: The pull request adds only Full details: Container-PrivilegesExplanation PASS. The PR changes only timeout environment variables in Full details: No-Sensitive-Data-In-LogsExplanation PASS: The PR adds only release-scenario timeout exports and explanatory comments, and removes duplicate timeout assignments. The added values are ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: agullon The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
GREENBOOT_TIMEOUT=1200 and TEST_EXECUTION_TIMEOUT=60m are now set centrally in ci_phase_boot_and_test.sh for all release scenarios. Remove the redundant per-scenario overrides. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> pre-commit.check-secrets: ENABLED
|
@agullon: This pull request explicitly references no jira issue. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
@agullon: all tests passed! Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
|
Consolidated into #7326 |
Summary
ci_phase_boot_and_test.shContext
Release scenarios running upgrade paths with LVMS workloads followed by full standard suites
were hitting timeout limits under I/O contention on x86 (
c5.metal, 4750 Mbps EBS bandwidth)when many VMs boot and pull container images from the mirror registry in parallel. The ARM
instance (
m7g.metal, 20000 Mbps EBS) has ~4.2x more I/O bandwidth and doesn't hit these limits.Specific failures observed in PR #7297:
el96-y2@el98-lrel@lvms-standard1/2: greenboot healthcheck failed — pods not ready within 600sel98-y1@el98-lrel@lvms-standard: Robot Framework killed by SIGTERM at 45m (44/52 tests had passed)These timeouts are ceilings, not floors — greenboot polls and exits immediately when pods are ready,
and robot finishes whenever tests complete. The happy-path duration is unaffected.
The el10 lvms-standard scenarios and the
optionalscenario already set these values per-scenario.This change centralizes them for all release scenarios in
ci_phase_boot_and_test.sh.Counterpart of #7318 for release-5.0.
Test plan
🤖 Generated with Claude Code
Summary by CodeRabbit