Skip to content

Fixed analysis when gather-extra fails in post phase#117

Closed
qu4rkn3t wants to merge 4 commits into
redhat-performance:mainfrom
qu4rkn3t:gather-extra
Closed

Fixed analysis when gather-extra fails in post phase#117
qu4rkn3t wants to merge 4 commits into
redhat-performance:mainfrom
qu4rkn3t:gather-extra

Conversation

@qu4rkn3t

Copy link
Copy Markdown
Contributor

Description

When gather-extra fails in the post phase, it doesn't create clusteroperators.json. BugZooka would just return a generic error with no deeper analysis.

Now, we first check if orion performance data exists. If not, we get relevant errors from build logs and then trigger LLM analysis of those errors.

Error link: https://prow.ci.openshift.org/view/gs/test-platform-results/logs/periodic-ci-openshift-eng-ocp-qe-perfscale-ci-main-aws-4.22-nightly-x86-payload-control-plane-6nodes/2064660673337495552

@coderabbitai

coderabbitai Bot commented Jun 12, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Summary by CodeRabbit

  • Documentation

    • Updated environment variable naming: INFERENCE_API_TIMEOUT replaces INFERENCE_API_TIMEOUT_SECONDS
  • New Features

    • Enhanced build artifact error detection with improved diagnostic fallback handling
  • Chores

    • Added shared inference library dependency to requirements

Walkthrough

This PR refactors BugZooka to use a shared inference library (py-commons) for client and configuration management, replacing local implementations with thin re-exports. Async execution is migrated from asyncio to anyio across analyzer modules. Error detection logic in the prow analyzer is updated to prioritize orion results when cluster operators data is missing. Documentation is updated to reflect environment variable naming changes.

Changes

Inference Library Refactoring and Async Migration

Layer / File(s) Summary
Shared library integration and inference client refactoring
requirements.txt, bugzooka/integrations/inference_client.py, bugzooka/core/config.py
Dependencies on py-commons>=0.3.0 and cryptography>=44.0.0 are added. inference_client.py is converted from a full HTTP/OpenAI client implementation into a thin re-export layer that imports InferenceClient, get_inference_client, analyze_with_agentic, and shared error types from commons.inference. Local exception classes are replaced with aliases of shared error types. config.py imports are updated to use the shared get_base_inference_config helper.
Configuration refactoring and constants cleanup
bugzooka/core/config.py, bugzooka/core/constants.py
get_inference_config() delegates inference configuration assembly to get_base_inference_config() from the shared library, then injects BugZooka-specific retry settings by reading INFERENCE_API_RETRY_MAX_ATTEMPTS, INFERENCE_API_RETRY_DELAY, INFERENCE_API_RETRY_BACKOFF_MULTIPLIER, and INFERENCE_API_RETRY_MAX_DELAY into config["retry"]. Constants INFERENCE_TEMPERATURE, INFERENCE_MAX_TOKENS, INFERENCE_MAX_TOOL_ITERATIONS, and INFERENCE_API_TIMEOUT_SECONDS are removed; retry/backoff constants are reordered and formatting is adjusted.
Log analyzer async migration to anyio
bugzooka/analysis/log_analyzer.py
Module now imports and uses anyio instead of asyncio. The _with_retry decorator pulls retry configuration from get_inference_config()["retry"]. Async execution in _run() is switched from asyncio.run(_run_async()) to anyio.run(_run_async()). Module imports are adjusted to include get_inference_config, analyze_prow_artifacts, and extract_job_details.
Slack listener async migration to anyio
bugzooka/integrations/slack_socket_listener.py
Module imports anyio and replaces asyncio event-loop management in three Slack mention handlers: PR analysis handler now uses anyio.run(analyze_pr_with_gemini, ...), nightly regression handler uses anyio.run(analyze_nightly_regression, ...), and performance summary handler uses anyio.run(analyze_performance, ...). Function argument passing is adjusted, especially for the channel parameter, to work with positional/keyword argument patterns of anyio.run().
Prow analyzer orion-first error detection
bugzooka/analysis/prow_analyzer.py
When clusteroperators.json is missing, analyze_prow_artifacts() now prioritizes scan_orion_jsons(). If orion preview is found, it returns those results with requires_llm=False and includes orion full results in the response. If no orion preview is available, it falls back to build-log-based error extraction via search_prow_errors() and sets requires_llm=True. Prior behavior that read a tail of the build log with a "couldn't find clusteroperators.json" message is removed.
Documentation updates for environment variable rename
CONTRIBUTING.md, README.md
Environment variable name is updated from INFERENCE_API_TIMEOUT_SECONDS to INFERENCE_API_TIMEOUT in the .env template and configuration documentation. README project structure comment is updated to describe bugzooka/integrations/inference_client.py as a re-export from py-commons rather than a unified OpenAI-compatible inference client.

🎯 3 (Moderate) | ⏱️ ~20 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title directly addresses the main fix: improved analysis handling when gather-extra fails in the post phase, matching the core objective of the PR.
Description check ✅ Passed The description clearly explains the problem, the solution (checking for Orion data first, then extracting build log errors before LLM analysis), and provides a concrete error example.
Docstring Coverage ✅ Passed Docstring coverage is 85.71% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@requirements.txt`:
- Around line 46-47: Update the vulnerable dependency line
"cryptography>=44.0.0" in requirements.txt to a safe minimum version (e.g.,
"cryptography>=44.0.1") so resolution cannot pick the affected 44.0.0 release;
after updating the constraint, regenerate any lockfile or dependency freeze (if
present) so CI uses the new minimum.
- Around line 18-19: The requirements entry "py-commons>=0.3.0" is not
resolvable from the public PyPI index and is blocking installs; either replace
that entry with the correct published package name and version (e.g., the actual
package you intended) or add the custom index/extra-index-url that hosts
"py-commons" so pip can find it; also tighten the cryptography bound by changing
"cryptography>=44.0.0" to "cryptography>=44.0.1" so the vulnerable 44.0.0 wheel
is excluded. Ensure you update the requirements.txt entries for "py-commons" and
"cryptography" accordingly.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Enterprise

Run ID: 65780694-3b2c-45dc-b715-8e573dfe0d68

📥 Commits

Reviewing files that changed from the base of the PR and between 180deb9 and 55f1f97.

📒 Files selected for processing (9)
  • CONTRIBUTING.md
  • README.md
  • bugzooka/analysis/log_analyzer.py
  • bugzooka/analysis/prow_analyzer.py
  • bugzooka/core/config.py
  • bugzooka/core/constants.py
  • bugzooka/integrations/inference_client.py
  • bugzooka/integrations/slack_socket_listener.py
  • requirements.txt

Comment thread requirements.txt
Comment on lines +18 to +19
# py-commons
py-commons>=0.3.0

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Locate and print requirements.txt with line numbers around the referenced lines.
if [ -f requirements.txt ]; then
  echo "== requirements.txt =="
  nl -ba requirements.txt | sed -n '1,120p'
else
  echo "requirements.txt not found at repo root" >&2
  exit 1
fi

Repository: redhat-performance/BugZooka

Length of output: 155


🏁 Script executed:

#!/bin/bash
set -euo pipefail

if [ -f requirements.txt ]; then
  echo "== requirements.txt =="
  cat -n requirements.txt | sed -n '1,120p'
else
  echo "requirements.txt not found at repo root" >&2
  exit 1
fi

Repository: redhat-performance/BugZooka

Length of output: 1275


🌐 Web query:

py-commons 0.3.0 PyPI

💡 Result:

As of June 12, 2026, there is no widely recognized or active package on the Python Package Index (PyPI) named exactly py-commons that corresponds to a version 0.3.0. The search results identify several distinct projects with similar names, none of which appear to have a version 0.3.0 in active circulation: 1. pycommons (without the hyphen): This is an active utility library maintained by Thomas Weise, currently on versions in the 0.8.x range (e.g., 0.8.98) [1][2][3]. 2. py-commons (Vaastav-Technologies): A GitHub repository that provides reusable utilities [4]. Its latest release is listed as v0.0.1dev3 [4]. 3. py-commons (piwheels): An older, separate package for data analysis utilities, with documented versions 0.1.0 and 0.1.2 released in 2016 [5]. If you are looking for a specific library, ensure you have the correct name, as the hyphenated py-commons and the non-hyphenated pycommons refer to different codebases [1][4][5].

Citations:


🌐 Web query:

cryptography 44.0.0 security advisory

💡 Result:

The cryptography library version 44.0.0 is affected by a security vulnerability related to the OpenSSL version bundled in its distributed wheels [1][2]. Vulnerability Details (CVE-2024-12797 / GHSA-79v4-65xg-pq4g): The wheels for cryptography version 44.0.0 include a statically linked copy of OpenSSL that contains a security vulnerability [1]. This issue affects versions 42.0.0 through 44.0.0 [1]. Remediation: The vulnerability was fixed in version 44.0.1 [1][3]. Users installing cryptography via wheels (e.g., from PyPI) are advised to upgrade to at least version 44.0.1 [1]. If you are building the library from the source distribution (sdist), you are responsible for managing and upgrading your own OpenSSL installation [1]. It is recommended to maintain the library at the latest stable release to ensure security patches and OpenSSL updates are applied [3][4]. As of June 2026, subsequent releases have addressed further vulnerabilities, such as those covered in later CVEs [5][6][7][8].

Citations:


Fix broken dependency resolution and tighten cryptography security bounds

  • requirements.txt line 19 (py-commons>=0.3.0) is not resolvable from the default/public package index (no released py-commons==0.3.0 available), blocking pip install -r requirements.txt; use the correct package name/version or configure the intended index.
  • requirements.txt line 47 (cryptography>=44.0.0) allows vulnerable cryptography==44.0.0 wheels (GHSA-79v4-65xg-pq4g); raise the lower bound to cryptography>=44.0.1 (or newer).
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@requirements.txt` around lines 18 - 19, The requirements entry
"py-commons>=0.3.0" is not resolvable from the public PyPI index and is blocking
installs; either replace that entry with the correct published package name and
version (e.g., the actual package you intended) or add the custom
index/extra-index-url that hosts "py-commons" so pip can find it; also tighten
the cryptography bound by changing "cryptography>=44.0.0" to
"cryptography>=44.0.1" so the vulnerable 44.0.0 wheel is excluded. Ensure you
update the requirements.txt entries for "py-commons" and "cryptography"
accordingly.

Source: Pipeline failures

Comment thread requirements.txt
Comment on lines +46 to +47
# Misc.
cryptography>=44.0.0

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail
ls -la
if [ -f requirements.txt ]; then
  echo "---- requirements.txt (tail/head around 40-60) ----"
  nl -ba requirements.txt | sed -n '35,60p'
else
  echo "requirements.txt not found at repo root"
  find . -maxdepth 3 -name requirements.txt -print
fi

Repository: redhat-performance/BugZooka

Length of output: 1728


🌐 Web query:

cryptography 44.0.0 security advisory CVE fixed version

💡 Result:

The cryptography library version 44.0.0 is affected by a security vulnerability related to the statically linked copy of OpenSSL included in its binary distributions (wheels) [1][2]. Security Advisory Details: - Vulnerability: Vulnerable OpenSSL included in cryptography wheels [1]. - CVE ID: CVE-2024-12797 [1][3]. - Affected Versions: >= 42.0.0 and <= 44.0.0 [1][2]. - Fixed Version: 44.0.1 or higher [1][2]. This vulnerability specifically impacts users installing pre-compiled wheels from PyPI [1][4]. Users who build the library from source (sdist) are responsible for managing their own OpenSSL version [1][2]. To mitigate this issue, users should upgrade to cryptography version 44.0.1 or later [1][2].

Citations:


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "---- requirements.txt: lines 40-55 ----"
if [ -f requirements.txt ]; then
  sed -n '40,55p' requirements.txt
else
  echo "requirements.txt not found at repo root"
  find . -maxdepth 4 -name requirements.txt -print
fi

Repository: redhat-performance/BugZooka

Length of output: 211


🌐 Web query:

GHSA-79v4-65xg-pq4g cryptography fixed version 44.0.1

💡 Result:

GHSA-79v4-65xg-pq4g is a security advisory for the Python cryptography library related to the inclusion of a vulnerable, statically linked version of OpenSSL [1][2]. This vulnerability is tracked as CVE-2024-12797 [1][3]. The issue affects cryptography library versions 42.0.0 through 44.0.0 [1][4]. Version 44.0.1 was released to fix this vulnerability by updating the bundled OpenSSL version [1][2]. Key details: - Impact: The vulnerability involves a missing report of an error condition during TLS/DTLS handshakes when using Raw Public Keys (RPKs), which could allow an attacker to impersonate a server in a Man-in-the-Middle (MitM) attack [3]. - Affected Users: This primarily affects users installing cryptography from pre-built wheels distributed on PyPI, as these contain the statically linked OpenSSL [1][5]. - Mitigation: Users should upgrade to cryptography version 44.0.1 or higher [1][6]. If building from source (sdist), users are responsible for ensuring the linked OpenSSL version is secure [1][2].

Citations:


Raise the cryptography minimum to avoid vulnerable 44.0.0
requirements.txt uses cryptography>=44.0.0, which still permits resolving to 44.0.0 affected by GHSA-79v4-65xg-pq4g / CVE-2024-12797; change the constraint to cryptography>=44.0.1 (or newer).

🧰 Tools
🪛 OSV Scanner (2.3.8)

[HIGH] 47-47: cryptography 44.0.0: undefined

(PYSEC-2026-35)


[HIGH] 47-47: cryptography 44.0.0: Vulnerable OpenSSL included in cryptography wheels

(GHSA-79v4-65xg-pq4g)


[HIGH] 47-47: cryptography 44.0.0: cryptography has incomplete DNS name constraint enforcement on peer names

(GHSA-m959-cc7f-wv43)


[HIGH] 47-47: cryptography 44.0.0: cryptography Vulnerable to a Subgroup Attack Due to Missing Subgroup Validation for SECT Curves

(GHSA-r6ph-v2qm-q3c2)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@requirements.txt` around lines 46 - 47, Update the vulnerable dependency line
"cryptography>=44.0.0" in requirements.txt to a safe minimum version (e.g.,
"cryptography>=44.0.1") so resolution cannot pick the affected 44.0.0 release;
after updating the constraint, regenerate any lockfile or dependency freeze (if
present) so CI uses the new minimum.

Source: Linters/SAST tools

@qu4rkn3t qu4rkn3t closed this Jun 12, 2026
@qu4rkn3t

Copy link
Copy Markdown
Contributor Author

Closed because it included all changes not relevant to this PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant