The CODECHECK register management bot.
If the research paper has code, it must run.
@chekhovbot assists the repeated tasks around a CODECHECK: welcoming
codecheckers, assigning them, validating codecheck.yml early, and opening the
pull request against register.csv. It is controlled by mentions in the checks
issue in codecheckers/register,
following the convention established by the Open Journals bots: the command
goes on the first line of the comment, one command per comment.
@chekhovbot commands
@chekhovbot assign @user as codechecker
@chekhovbot suggest codecheckers
@chekhovbot check codecheck.yml
@chekhovbot registerTracking issue: codecheckers/register#209
Anton Chekhov (1860-1904): a pun on check, and on Chekhov's gun — if the gun
is on the wall in Act I it must fire in Act III; if the paper has code, it must
run. (Star Trek's Chekov has one h.)
The bot answers commands, hello and an unknown command on the testing
register, and validates a codecheck.yml when a comment names the repository
to read it from. Editors can announce a published certificate on Mastodon with
@chekhovbot announce <certificate>, which previews the toot, and
@chekhovbot announce <certificate> confirm, which posts it; in development
the toot is a direct message with every mention defused. @chekhovbot codecheckers says where the codechecker lists are, and an editor can ask
@chekhovbot suggest codecheckers for candidates, named without notifying
them. @chekhovbot next certificate says which certificate identifier comes
next, and @chekhovbot set certificate reserves it for the check. It runs
against
codecheckers/testing-dev-register
only; the production register is not configured anywhere. See
docs/deployment.md.
go run ./cmd/chekhov serve # the bot: /dispatch, /healthz, /nudges
echo '@chekhovbot commands' | go run ./cmd/chekhov comment - # the same answer, offline
echo '@chekhovbot announce 1970-001' | go run ./cmd/chekhov comment --as nuest --online - # an editor's previewgo build ./...
go run ./cmd/chekhov check path/to/codecheck.yml # the report, non-zero on failure
go run ./cmd/chekhov check --online path/to/codecheck.yml # also asks OpenAlex, ORCID, Zenodo
go run ./cmd/chekhov check github::codecheckers/Piccolo-2020 # read it from the repository
go run ./cmd/chekhov check codecheckers/Piccolo-2020 # the same, platform inferred
go run ./cmd/chekhov check 2020-001 # the certificate, looked up in the register
go run ./cmd/chekhov check bundle github::codecheckers/demo # one part of the catalogue
go run ./cmd/chekhov check repository codecheckers/demo # what the repository is, not a verdict
go run ./cmd/chekhov check --markdown path/to/codecheck.yml # the reply the bot posts
go run ./cmd/chekhov rules # the rules, and which are checked
go run ./cmd/chekhov rules --check # are they the register's current ones?The bot reports one version, internal/build.Version, and it is bumped by
hand in the commit that earns it - beside the CHANGELOG.md entry, never saved
up for a release. /healthz, @chekhovbot version and the development footer
all read that constant, so a deployment and a local build answer the same way.
It is semantic versioning applied to what somebody talking to the bot can see. A new command or a new thing a reply reports is a minor bump; a fix, a wording change or a rule refresh is a patch; a command removed or renamed, or a reply whose meaning changes under somebody already relying on it, is major.
Two things hold it together. A test compares the constant with the newest
heading in CHANGELOG.md, checks the headings are newest-first and dated, and
fails when anything is parked under Unreleased. And because that test reads
one tree rather than a diff - so a feature written under the existing heading
would pass - CI additionally fails a pull request that changes non-test Go
without moving the version, unless it carries the no-version-bump label.
The commit the build was made from is a separate question, answered only by the
stamp go build makes from the checkout since Go 1.24 - the one source nothing
can hand over wrongly. A development deployment, and a locally built binary
previewing a reply, name it and say when the tree had uncommitted changes; a
go run names nothing rather than guessing. The version itself is not taken
from git, because the deployment's builder may have no .git, and a version
that is sometimes semantic and sometimes a pseudo-version is worse than one
that is always the same shape.
The rules are maintained in the register, not here, one file per version of the
configuration file specification:
https://github.com/codecheckers/register/blob/master/RULES.md. Chekhov embeds
a copy, refreshed with scripts/update-rules.sh.
The same identifiers are used by the
codecheck R package, so the same
rule is CC-CFG-016 in both implementations, and the two can be compared by
grepping for an identifier. Which rules apply to a file follows from the
specification version it declares, and a rule's severity comes from the rule
file rather than from either codebase.
Everything is exercised against the testing register, never the real one:
https://github.com/codecheckers/testing-dev-register. See
docs/development.md for the full setup.
go test ./... # fast, offline
CHEKHOV_INTEGRATION=1 go test -run Integration ./internal/check/ # external servicesThe fast suite needs no network and covers every rule, including the ones that ask an external service: recorded cassettes replay what OpenAlex, ORCID, Zenodo, GitHub and the register really answered, and stubbed servers cover the answers a published CODECHECK never gives. CI runs it on every push and pull request.
The integration suite asks the live services the same questions, to catch the day one of them changes its answer. It reads only, and CI runs it weekly.
Code MIT (see LICENSE). Graphics in logo/ are CC BY 4.0, matching the rest
of the CODECHECK branding. The Chekhov quotes in
internal/command/data/quotes.yml come from
Wikiquote and are CC BY-SA 4.0;
every reply that uses one links back to the page.