Context and problem
Student project teams on GitLab leave a lot of evidence behind: issues, milestones, wiki edits and events. Judging a team’s process from that by hand is slow and inconsistent. Computing numbers straight from a live API has a worse problem: the same team can get a different number tomorrow, and nobody can say why.
What it measures
The tool does not grade. It measures five criteria from the course’s assessment spec and reports each as a value and a level from 0 to 4, with flags where a human should look.
Approach
- Two jobs with a hard boundary. The snapshot job is the only part that touches the network: it fetches the data and writes immutable snapshot files. The scoring job reads those files and nothing else. It has no I/O, no network, no randomness and no clock access. This is recorded as an architecture decision and enforced in review.
- Five criteria: sprint structure, issue quality, completion rate, contribution balance and process documentation. Each produces a value, a level and flags. There is no overall score.
- Time handling is explicit. All timestamps are converted from UTC to Europe/Berlin once, on ingest. Sprint windows come from the sprint calendar, not from GitLab milestone dates.
- Spec ambiguities are resolved in the direction most favourable to the team, and each resolution is logged in a decisions file with the spec section it belongs to. When the implementation and the spec disagree, the spec is the bug to fix.
- Configuration is layered: a roster file per course, optional global defaults, and CLI flags that override both. Course variants for 12 and 24 credit points are supported.
- Output is a Markdown readout for people and sorted-key JSON with fixed six-decimal floats for further processing.
Principle
Same snapshots and config produce byte-identical output. Every rule that could be read two ways has a written decision behind it, so a result can be traced and defended.
Status
Ongoing. All five criteria and the snapshot, score, readout, discovery and config commands are implemented. The suite covers unit, property-based (Hypothesis), integration with a mocked GitLab API, and hand-derived golden fixtures. The repository is private.
Transfer
Any measurement built on a live, changing API needs the same split: fetch once into immutable records, then compute from those records with no hidden inputs. Keeping a log of every interpretation makes the result auditable. This applies to KPI reporting, compliance evidence and audit trails in organisations.