Build the comparison engine from precedent alone: past reports and the snapshots that produced them are the only specification of what counts as a real change.
- Version
- Category
- software-engineering
- Difficulty
- hard
- Model configurations
- 3
- Attempts
- 8
- Public trials
- 81 passed
- Median runtime
- 11m 55s
Instruction
# Build the version diff review The platform snapshots the working database on a schedule (see `platform/snapshots/`), and operators review two versions against each other before deciding anything: "pull up today against last week's snapshot and tell me everything that changed, so I can decide whether to roll back." Implement `lib/versionDiff.mjs` exporting `computeVersionDiff` and `computeVersionDiffDetails` as consumed by `bin/diff.mjs`, matching the response contracts in `contracts/versionDiff.schema.mjs`. The review's behavior is specified by precedent: `assets/history/` holds the review exports operators saw for past snapshot pairs, and `platform/snapshots/` holds the snapshots that produced them. New reviews must be consistent with that precedent on databases the history has never seen. Check your work with `node bin/diff.mjs snap-2026-07-19-0600` and `node contracts/validate.mjs <saved-output.json>`.
Public trials
Results by model
| Model · harness | Mean reward | Range | ± SE | Attempts |
|---|---|---|---|---|
| GPT 5.6 SolOpenCode | 96.5% | 93% – 100% | 3.5 pp | 2 |
| gemini-3.1-pro-previewOpenCode | 63.0% | 63% – 63% | 0.0 pp | 32 scored |
| Muse Spark 1.3 ContributorOpenCode | 42.0% | 0% – 63% | 21.0 pp | 3 |