Skip to content

test-coverage: add examples/vignette coverage, a failing threshold, and Codecov upload hardening #334

Description

@d-morrison

We already have test-coverage (covr -> Codecov). Comparing it against IndrajeetPatil/workflows/.github/workflows/test-coverage.yaml (MIT) turns up four things ours does not do. They are independent, so this can be split if it grows.

Related: #234 (JUnit / Codecov Test Analytics upload), which is a different gap in the same workflow — worth checking for overlap before implementing.

1. A second coverage job for examples and vignettes

Upstream runs a separate examples-coverage job:

covr::package_coverage(
  type = c("examples", "vignettes"),
  commentDonttest = FALSE,
  commentDontrun = FALSE
)

Setting both comment* flags to FALSE means \donttest{} and \dontrun{} blocks actually execute. Those are exactly the code paths that rot unnoticed — they are skipped by R CMD check in its default mode and by the normal test suite, so nothing else runs them. This measures something genuinely different from unit-test coverage rather than re-reporting it.

2. A threshold that fails the job

Upstream fails below 100% in both jobs (stop() in one, cli::cli_abort() in the other). We report coverage and let Codecov's own status decide.

100% is the author's own bar and should not be inherited wholesale — but a configurable min-coverage input (default off, so current behavior is unchanged) would let consumers opt in. Note upstream is inconsistent with itself here: 100% for R, 95% for Python.

3. Codecov upload hardening

plugin: noop
disable_search: true
files: ./cobertura.xml

This stops the Codecov action from scanning the workspace and uploading whatever it finds. Worth doing regardless of the rest: it makes the upload exactly what we intend and nothing else.

4. Conditional fail_ci_if_error

fail_ci_if_error: ${{ github.event_name != 'pull_request' || secrets.CODECOV_TOKEN != '' }}

Strict on pushes and whenever a token is available, tolerant of tokenless uploads from forked PRs — where the upload legitimately cannot authenticate and failing CI would punish an outside contributor for something they cannot fix.

Reading a secrets context inside a with: expression is only legal in workflow_call context, which is exactly what we are, so this transfers cleanly.

Also worth taking

Upstream dumps testthat.Rout* on if: always() with a trailing || true, so a missing log never masks the real failure, and uploads the whole install path as an artifact on failure() for debugging. Our composite already does some of this — worth a line-by-line comparison rather than assuming parity.

Source: IndrajeetPatil/workflows, MIT.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions