Repository navigation
Integrate coverage testing #19
Description
Generate Coverage report
This should nearly at the end as it messes up the build repositories. It should be something similar to:
stage ('Test coverage') {
def repo_name = /python3 -c "import sys, json; print(json.load(sys.stdin)['repository']['name']).toupper()"/
switch (repo_name) {
case 'dmd':
sh 'mv src/dmd src/host_dmd'
sh 'make -j$N -C dmd/src -f posix.mak MODEL=$MODEL HOST_DMD=./host_dmd clean'
sh 'make -j$N -C dmd/src -f posix.mak MODEL=$MODEL HOST_DMD=./host_dmd dmd.conf'
sh 'make -j$N -C dmd/src -f posix.mak MODEL=$MODEL HOST_DMD=./host_dmd ENABLE_COVERAGE=1'
sh 'make -j$N -C dmd/test MODEL=$MODEL ARGS="-O -inline -release" DMD_TEST_COVERAGE=1'
sh 'mv src/host dmd src/dmd'
break;
case 'druntime':
sh 'TEST_COVERAGE="1" make -j$N -C . -f posix.mak MODEL=$MODEL unittest-debug'
break;
case 'phobos':
sh 'ENABLE_COVERAGE="1" make -f posix.mak MODEL=$MODEL unittest-debug'
// tests needs to be run separately due to https://issues.dlang.org/show_bug.cgi?id=16397
sh 'make -f posix.mak $(find std etc -name "*.d" | sed "s/[.]d$/.test")'
break;
default:
}
CodeCov
Assuming the CodeCov tokens (can be fetched at e.g. https://codecov.io/gh/dlang/phobos/settings) are loaded in the environment something as CODECOV_TOKEN_PHOBOS (and similarly for dmd and druntime) like this should work:
stage ("Upload coverage report") {
def repo_name = /python3 -c "import sys, json; print(json.load(sys.stdin)['repository']['name']).toupper()"/
sh 'curl -s https://codecov.io/bash | bash -s - -t CODECOV_TOKEN_${DMD}'
}
CodeCov uptime doesn't seem to be perfect, we might want to provide a fallback URL as well - see e.g.
dlang/phobos#5140
(though I am not sure what's the proper behavior as the CodeCov API is then probably down as well)
Activity
MartinNowak commentedon Feb 2, 2018
That might actually be some work because external testers usually require authentication with coverage services.
wilzbach commentedon Feb 2, 2018
Only the CodeCov token would be required.
However at the moment we don't run any of the DMD, druntime or Phobos testsuites and this requires quite some resources.
CircleCi does a good job at this and we even use three parallel runs at DMD now: https://github.com/dlang/dmd/blob/master/.circleci/config.yml (x86, x86_64, x86_64 - pic).
They are all required to improve the overall coverage (see dlang/dmd#7818).
If we ever want to unify this, it's probably easier to move the
.circleci/ci.shscript totoolsand fetch it from each repo.[-]integrate coverage testing[/-][+]Integrate coverage testing[/+]