Wisdom on GitHub autograding from Lawrence.

[dham]

A cloned assignment with autograding has the following .github structure:

.github/classroom/autograding.json <- this file enumerates the tests that will be run
.github/workflows/classroom.yml <- this file runs the tests

So e.g. for me, the classroom.yml is:

name: GitHub Classroom Workflow
on: [push]
jobs:
  build:
    name: Autograding
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-python@v2
        with:
          python-version: 3.8
      - name: Install numpy
        run: |
          pip install numpy
      - uses: education/autograding@v1

The magic - uses: education/autograding@v1 line the action to run that

The autograding.json is fully undocumented, but it looks like:

{
"tests": [
   {"name": "test 1", ...},
   {"name": "test 2", ...},
  ]
}

each test entry is like:

        {
            "name": "Compile code",
            "setup": "",
            "run": "gcc -std=c99 -fopenmp -Wall -Wextra -o simulation simulation.c -lm",
            "input": "",
            "output": "",
            "comparison": "exact",
            "timeout": 2,
            "points": null
        }

and then you have to read https://github.com/education/autograding/blob/master/src/runner.ts to figure out how these things are interpreted

runner.ts
import {spawn, ChildProcess} from 'child_process'
import kill from 'tree-kill'
import {v4 as uuidv4} from 'uuid'
import * as core from '@actions/core'
import {setCheckRunOutput} from './output'

TBH: I think the better thing is to just write a normal workflow run, ignoring github's autograder, that does some scoring assignment