Skip to content

Improve reliability of Coverage reporting on CI#1337

Merged
fizyk merged 1 commit into
mainfrom
coverage-combine
May 13, 2026
Merged

Improve reliability of Coverage reporting on CI#1337
fizyk merged 1 commit into
mainfrom
coverage-combine

Conversation

@fizyk
Copy link
Copy Markdown
Member

@fizyk fizyk commented May 12, 2026

Chore that needs to be done:

  • Add newsfragment pipenv run towncrier create [issue_number].[type].rst

Types are defined in the pyproject.toml, issue_number either from issue tracker or the Pull request number

Summary by CodeRabbit

  • Chores
    • Enhanced CI coverage reporting with improved multiprocessing support for more reliable test coverage collection.
    • Optimised coverage collection across serial and parallel test execution modes.
    • Updated testing infrastructure dependencies.

Review Change Stack

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 12, 2026

📝 Walkthrough

Walkthrough

This PR refactors CI coverage reporting by replacing pytest-cov plugin integration with explicit coverage.py invocations. Coverage data is now collected separately for serial and xdist test modes, combined per mode, and exported to distinct XML files for Codecov upload. Local pytest configurations no longer use the --cov flag.

Changes

Coverage reporting refactoring

Layer / File(s) Summary
Coverage configuration and dependencies
.coveragerc, Pipfile
Coverage.py multiprocessing concurrency and parallel execution are enabled; pytest-cov dependency is replaced with coverage package (≥7.0).
Local pytest configuration cleanup
pyproject.toml, pyproject.oldest.toml
The --cov flag is removed from pytest addopts in both main and oldest-dependencies configurations.
Docker workflow coverage collection
.github/workflows/dockerised-postgres.yml
Docker-based tests now run under explicit coverage run with a dedicated .coverage.docker data file and new combine/export step that generates coverage-docker.xml for Codecov.
PostgreSQL workflow variants with serial and xdist coverage
.github/workflows/oldest-postgres.yml, .github/workflows/single-postgres-windows.yml, .github/workflows/single-postgres.yml
Oldest-dependencies, Windows, and main workflows are split into serial and xdist test phases, each using explicit coverage run with distinct data files (.coverage.serial and .coverage.xdist), followed by separate combine/export steps producing coverage-serial.xml and coverage-xdist.xml for Codecov upload.
Release notes
newsfragments/+4ed991c2.misc.rst
News fragment documents improved reliability of coverage reporting on CI.

🎯 2 (Simple) | ⏱️ ~12 minutes


🐰 A rabbit hops through CI lanes so bright,
Coverage data now splits left and right,
Serial and xdist, each on their own track,
Combined with purpose, no missing the pack!
Reliable reports from parallel test flight. 🚀

🚥 Pre-merge checks | ✅ 6
✅ Passed checks (6 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'Improve reliability of Coverage reporting on CI' directly reflects the main change: restructuring coverage collection across multiple CI workflows to use explicit coverage data files and combine/export steps instead of relying on pytest's built-in coverage reporting, which improves reliability.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Newsfragment Check ✅ Passed Newsfragment file +4ed991c2.misc.rst exists with valid 'misc' type, proper format, and content matching PR scope.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch coverage-combine

Tip

💬 Introducing Slack Agent: The best way for teams to turn conversations into code.

Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.

  • Generate code and open pull requests
  • Plan features and break down work
  • Investigate incidents and troubleshoot customer tickets together
  • Automate recurring tasks and respond to alerts with triggers
  • Summarize progress and report instantly

Built for teams:

  • Shared memory across your entire org—no repeating context
  • Per-thread sandboxes to safely plan and execute work
  • Governance built-in—scoped access, auditability, and budget controls

One agent for your entire SDLC. Right inside Slack.

👉 Get started


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 12, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
newsfragments/+4ed991c2.misc.rst (1)

1-1: 💤 Low value

Consider using the PR number instead of orphan format.

The PR objectives specify using the PR number (1337) for the newsfragment filename. Whilst the orphan format +[hash].[type].rst is acceptable when no dedicated issue exists, the objectives state: "issue_number should be taken from the issue tracker or the pull request number." Consider renaming to 1337.misc.rst to align with the stated convention.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@newsfragments/`+4ed991c2.misc.rst at line 1, The newsfragment filename uses
the orphan/hash format (+4ed991c2.misc.rst); rename this file to use the pull
request number as specified (e.g., 1337.misc.rst) so the issue_number is taken
from the PR instead of the orphan format; update any references to
+4ed991c2.misc.rst in the repo or changelog tooling to the new 1337.misc.rst
name.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@newsfragments/`+4ed991c2.misc.rst:
- Line 1: The newsfragment filename uses the orphan/hash format
(+4ed991c2.misc.rst); rename this file to use the pull request number as
specified (e.g., 1337.misc.rst) so the issue_number is taken from the PR instead
of the orphan format; update any references to +4ed991c2.misc.rst in the repo or
changelog tooling to the new 1337.misc.rst name.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 418b5556-7165-44a5-b2eb-b59139035e60

📥 Commits

Reviewing files that changed from the base of the PR and between d912c6a and 872f852.

📒 Files selected for processing (9)
  • .coveragerc
  • .github/workflows/dockerised-postgres.yml
  • .github/workflows/oldest-postgres.yml
  • .github/workflows/single-postgres-windows.yml
  • .github/workflows/single-postgres.yml
  • Pipfile
  • newsfragments/+4ed991c2.misc.rst
  • pyproject.oldest.toml
  • pyproject.toml

@fizyk fizyk merged commit 3645b1a into main May 13, 2026
55 checks passed
@fizyk fizyk deleted the coverage-combine branch May 13, 2026 15:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant