oadp-analyze-e2e-failure: give Claude explicit source-clone instructions - #84048
Conversation
Follow-up to openshift#83957. The post-step runs on the generic claude-ai-helpers image, not oadp-operator's own build_root, so unlike the in-repo approach it replaced, the repo source isn't pre-checked-out on disk. Add an OADP_BRANCH env (set per config file, matching zz_generated_metadata.branch) and tell Claude explicitly, in the system prompt, to `git clone` both openshift/oadp-operator and openshift/velero at that branch if artifact-based evidence alone isn't enough -- rather than leaving it to infer the right repo/branch/URL on its own. Branch names are mirrored 1:1 between oadp-operator and velero, confirmed via https://github.com/oadp-rebasebot/oadp-rebase/blob/oadp-dev/repos.yaml (the canonical source of truth for OADP ecosystem repo/branch mappings). The claude-ai-helpers image (openshift-eng/ai-helpers) is built on the standard OCP golang builder base (has git) plus an explicit `gh` CLI install, so it's fully capable of this -- it just wasn't being told to. Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
|
/pj-rehearse Note Responses generated with Claude |
|
@kaovilai: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository YAML (base), Central YAML (inherited) Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (1)
Included review availability: Your plan provides up to 12 included reviews per hour; 8 remain after this review. WalkthroughOADP AWS E2E job configurations now set ChangesOADP branch-aware E2E testing
Estimated code review effort: 2 (Simple) | ~10 minutes Merge Risk: ⚪ Minimal · up to This change gives failure analysis access to the relevant OADP and Velero source branches without altering product runtime behavior; no actionable merge-blocking risk remains after normal checks and review. Suggested reviewers: 🚥 Pre-merge checks | ✅ 15✅ Passed checks (15 passed)
Full details: Docstring CoverageExplanation No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 1 files. Full details: Stable And Deterministic Test NamesExplanation PASS: The pull-request diff adds only static Full details: Test Structure And QualityExplanation PASS: The pull request does not add or modify Ginkgo test code. The diff contains only YAML environment entries and changes to the analysis shell step and its reference YAML. The diff contains no Ginkgo constructs such as Full details: Microshift Test CompatibilityExplanation PASS: The pull request adds no Ginkgo test declarations and no Go files. The diff only adds OADP_BRANCH environment entries and source-cloning guidance to CI configuration and a shell post-step. Therefore, the MicroShift Test Compatibility check is not applicable. Full details: Single Node Openshift (Sno) Test CompatibilityExplanation PASS — The pull request adds no new Ginkgo e2e tests. The diff contains only CI YAML environment entries, the failure-analysis shell script, and its reference YAML; it adds no Go/test-source paths or Ginkgo declarations such as It(), Describe(), Context(), or When(). Therefore the SNO compatibility conditions do not apply. Full details: Topology-Aware Scheduling CompatibilityExplanation PASS — The pull request changes only CI configuration environment variables, the failure-analysis shell prompt, and its step reference. The diff adds OADP_BRANCH values and source-clone guidance. It does not add or modify deployment manifests, operator code, controllers, replica settings, affinity, topology spread, node selectors, tolerations, or PDBs. The topology-aware scheduling check is therefore not applicable. Full details: Ote Binary Stdout ContractExplanation PASS: The complete PR diff contains only YAML configuration and one shell post-step change. It contains no Go or OTE binary source changes, and no added OTE stdout APIs or suite setup. The YAML additions only set Full details: Ipv6 And Disconnected Network Test CompatibilityExplanation PASS: The PR adds no new Ginkgo tests. The diff contains only CI YAML environment entries, step metadata, and source-clone guidance in the failure-analysis shell prompt. The added GitHub clone instructions are not test implementation, so the custom check does not apply. Full details: No-Weak-CryptoExplanation PASS: The pull-request diff adds OADP_BRANCH YAML values and source-cloning guidance only. The changed lines introduce no MD5, SHA-1, DES, 3DES, RC4, Blowfish, ECB, custom cryptography, or secret/token comparisons. Existing secret-redaction regexes are unchanged. Full details: Container-PrivilegesExplanation No container-privilege finding was introduced. The diff adds only OADP_BRANCH environment entries, prompt text, and documentation. The changed step manifest has no privileged, hostPID, hostNetwork, hostIPC, SYS_ADMIN, allowPrivilegeEscalation, or securityContext setting. It continues to reference the existing claude-ai-helpers image without changing its runtime user or privileges. Full details: No-Sensitive-Data-In-LogsExplanation PASS. The PR adds branch values and source-clone guidance for public GitHub repositories. It does not add logging of passwords, tokens, API keys, PII, or customer data. The Claude output redirection and redaction logic are unchanged from the base revision. Added ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
openshift-ci
Bot
added
the
approved
The prior commit only named openshift/oadp-operator and openshift/velero explicitly. Root causes can equally live in a cloud-provider plugin or other OADP ecosystem component (velero-plugin-for-aws, kubevirt-velero-plugin, hypershift-oadp-plugin, etc.) that the prompt gave no guidance for at all. Point Claude at oadp-rebasebot/oadp-rebase's repos.yaml -- the canonical org/repo/branch mapping for the whole OADP ecosystem -- instead of trying to enumerate every repo/branch exception (dev_branch overrides, etc.) inline in the prompt. Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
|
/pj-rehearse Note Responses generated with Claude |
|
@kaovilai: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
[REHEARSALNOTIFIER]
A total of 64 jobs have been affected by this change. The above listing is non-exhaustive and limited to 25 jobs. A full list of affected jobs can be found here Interacting with pj-rehearseComment: Once you are satisfied with the results of the rehearsals, comment: |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: kaovilai, shubham-pampattiwar, sseago The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
/pj-rehearse ack |
|
@kaovilai: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
openshift-merge-bot
Bot
added
the
rehearsals-ack
|
/pj-rehearse pull-ci-openshift-oadp-operator-oadp-1.6-4.23-e2e-test-kubevirt-aws The one rehearsal failure here was in Note Responses generated with Claude |
|
@kaovilai: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
32693bf
into
openshift:main
3 participants
Summary
Follow-up to #83957 (as promised there).
oadp-analyze-e2e-failureruns on the genericclaude-ai-helpersimage, not oadp-operator's ownbuild_root— so unlike the in-repo Claude CLI approach it replaced (which ran inside the same container as the test, with the full repo checked out viaCOPY ./ .), the new post-step's pod has no oadp-operator source on disk at all. It only had GCS test artifacts (JUnit, must-gather, pod logs) plus a hardcoded system-prompt description of OADP concepts.Two concrete pieces of context were being lost as a result:
tests/e2e/lib/flakes.go's known-flake patterns (referenced by path in the prompt, but with no way for Claude to actually read it).git cloneofopenshift/velero@oadp-devspecifically for this reason, see build/ci-Dockerfile#L36-L53).Fix
claude-ai-helpers(openshift-eng/ai-helpers) is built on the standard OCP golang builder base (shipsgit) plus an explicitghCLI install — it's fully capable of cloning source on demand, it just wasn't being told to.OADP_BRANCHenv to the step (defaultoadp-dev, overridden per config file to match that file's ownzz_generated_metadata.branch).git clone --depth 1 --branch ${OADP_BRANCH}bothopenshift/oadp-operatorandopenshift/veleroif artifact evidence alone isn't enough — rather than leaving it to infer the right repo/branch/URL on its own. Branch names are mirrored 1:1 between the two repos, confirmed viaoadp-rebasebot/oadp-rebase'srepos.yaml— the canonical source of truth for OADP ecosystem repo/branch mappings (used here purely as a reference for the branch-naming convention, no dependency added).Testing
make updateran clean (ci-operator-checkconfig passed as part of it).shellcheckclean on the updatedoadp-analyze-e2e-failure-commands.sh.Note
Responses generated with Claude
Summary by CodeRabbit
OADP_BRANCHfor OADP Operator AWS E2E jobs across supported release branches.oadp-analyze-e2e-failureto clone matching OADP Operator, Velero, cloud-provider, and other OADP ecosystem repositories when artifacts are insufficient.oadp-rebasebot/oadp-rebaserepository mappings to select source repositories and branches.