forward-only-task-publication

Public task · Field Engineering

← Field Engineering

Make every task publication build run, moving a task's current version only forward by commit time, and keep builds still running the older publisher from moving back a task a build with the rule made current.

Version
Category
Engineering
Model configurations
2
Attempts
9
Public trials
114 passed
Median runtime
11m 36s

Instruction

# Publish every main push, moving a task's current version only forward

Each push to main starts a task publication build (`backend/ci/publish_tasks.py`).
Before publishing, a build skips itself when Cloud Build reports a later-started
publication build on main. But started later is not the same as newer. When
several merges land within seconds, an older commit's build can start last.
The newer commits' builds then skip themselves, and the one build that runs
publishes an older tree. Tasks changed by those newer commits stay unpublished
until the next merge.

Make every publication build run instead of skipping. Builds stay serialized by
the existing advisory lock. Builds can now finish in any order, so a task's
current version must only move forward. A build makes a version current only
when its commit is at least as new, by committer time, as the commit that made
the task's current version current. A build of an older commit changes nothing
for that task.

Record that commit time on `task_version`. When a newer commit restores an
older version's files (a revert), that version becomes current again, and an
older commit's build cannot undo it.

`ResultsDb.set_current_version` (`backend/harness/results_db.py`) enforces the
rule. It takes the commit's committer time as a new keyword argument,
`committed_at`: an ISO-8601 timestamp with a timezone, like `published_at`. The
publication build passes its commit's committer time. Add the column by
migration. Tasks that are already published when the migration runs must
still take newer versions.

Each build runs the publisher from its own commit's checkout, and we apply the
migration before merging. So for a while after this ships, builds of commits
merged before it still run today's publisher, which passes no commit time.
They must not move a task backwards either: once a build with the rule has
made a version of a task current, a build running today's publisher changes
nothing for that task. Until then, it publishes the task as it does today.

## Environment

PostgreSQL on localhost:5432 and the local database endpoint the app's
driver uses are running when you start, and the dev env files point at the
dev database.

Public trials

Claude Opus 5.51 of 1 passed

Unknown model1 of 2 passed

Results by model

Model · harnessMean rewardRange± SEAttempts
Claude Opus 5.5Grok Build100.0%100% – 100%—1
Frontier model ACursor CLI25.0%0% – 100%16.4 pp8