Field Engineering

Can coding agents ship real production work in private codebases?

Field Engineering evaluates whether coding agents can complete real engineering work in nplus1's private production codebases. Tasks are reconstructed from work performed for actual customers and span data pipelines, failure reproduction, migrations, infrastructure, and more.

Each task keeps the customer's context and the engineer's direction from the original request; the hidden verifier checks the behavior the shipped change had to have.

software engineering · full stack

Tasks
1,745
Public tasks
24
Models evaluated
5
Published trials
34,482
Mean reward
Average of task means across the task versions that match the dataset’s rules.
± SE
Standard error across task means; directional when a dataset has few tasks.
Model · harnessMean reward± SETasksTrials
Claude Opus 5.5Claude Code, Grok Build84.0%Task means: 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 8.3%, 10.0%, 12.5%, 12.5%, 16.7%, 16.7%, 16.7%, 16.7%, 20.0%, 20.0%, 21.7%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 28.6%, 33.3%, 33.3%, 33.3%, 33.3%, 33.3%, 33.3%, 33.3%, 37.5%, 37.5%, 40.0%, 40.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 60.0%, 62.5%, 66.7%, 66.7%, 66.7%, 66.7%, 66.7%, 66.7%, 66.7%, 66.7%, 66.7%, 66.7%, 66.7%, 66.7%, 66.7%, 66.7%, 66.7%, 66.7%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 80.0%, 80.0%, 83.3%, 83.3%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%.1.1 pp761 / 17451,737
Frontier model ACursor CLI68.5%Task means: 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 5.0%, 5.3%, 6.3%, 6.3%, 6.3%, 10.0%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 18.2%, 18.8%, 18.8%, 18.8%, 18.8%, 18.8%, 18.8%, 18.8%, 20.8%, 22.7%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 29.4%, 31.3%, 31.3%, 31.3%, 31.3%, 31.3%, 31.3%, 31.3%, 31.3%, 31.3%, 31.3%, 34.4%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 38.5%, 39.1%, 39.1%, 40.0%, 40.0%, 42.9%, 42.9%, 42.9%, 43.8%, 43.8%, 43.8%, 43.8%, 43.8%, 43.8%, 43.8%, 43.8%, 43.8%, 43.8%, 43.8%, 43.8%, 43.8%, 43.8%, 43.8%, 43.8%, 44.4%, 45.8%, 45.8%, 46.7%, 47.1%, 47.1%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 53.1%, 54.2%, 54.2%, 55.0%, 55.0%, 55.6%, 55.6%, 55.6%, 55.6%, 56.3%, 56.3%, 56.3%, 56.3%, 56.3%, 56.3%, 56.3%, 56.3%, 56.3%, 56.3%, 56.3%, 56.3%, 56.3%, 56.3%, 56.3%, 56.3%, 56.3%, 56.3%, 56.3%, 57.1%, 57.1%, 57.1%, 57.1%, 58.3%, 58.3%, 58.3%, 59.1%, 59.4%, 59.4%, 59.4%, 60.0%, 60.7%, 61.1%, 61.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 63.0%, 63.2%, 65.0%, 65.6%, 66.7%, 66.7%, 66.7%, 66.7%, 66.7%, 66.7%, 66.7%, 66.7%, 66.7%, 68.8%, 68.8%, 68.8%, 68.8%, 68.8%, 68.8%, 68.8%, 68.8%, 68.8%, 68.8%, 68.8%, 68.8%, 68.8%, 68.8%, 68.8%, 68.8%, 68.8%, 68.8%, 68.8%, 68.8%, 68.8%, 68.8%, 68.8%, 68.8%, 68.8%, 68.8%, 68.8%, 68.8%, 68.8%, 68.8%, 68.8%, 68.8%, 68.8%, 68.8%, 68.8%, 68.8%, 68.8%, 68.8%, 70.0%, 70.0%, 70.0%, 70.0%, 70.0%, 70.0%, 70.0%, 70.8%, 70.8%, 70.8%, 70.8%, 70.8%, 70.8%, 70.8%, 70.8%, 70.8%, 70.8%, 70.8%, 70.8%, 70.8%, 70.8%, 70.8%, 70.8%, 71.0%, 71.0%, 71.4%, 71.9%, 71.9%, 71.9%, 71.9%, 71.9%, 73.4%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 76.0%, 76.1%, 76.2%, 76.6%, 76.7%, 77.1%, 77.1%, 77.1%, 77.5%, 78.1%, 78.1%, 78.6%, 79.2%, 79.2%, 79.2%, 79.2%, 79.2%, 79.2%, 79.2%, 79.2%, 79.4%, 79.5%, 79.7%, 80.0%, 80.0%, 80.6%, 81.3%, 81.3%, 81.3%, 81.3%, 81.3%, 81.3%, 81.3%, 81.3%, 82.1%, 82.5%, 82.5%, 82.6%, 83.3%, 83.3%, 83.3%, 83.9%, 84.1%, 84.2%, 84.4%, 84.4%, 84.4%, 84.4%, 85.0%, 85.0%, 85.0%, 85.1%, 85.4%, 85.4%, 85.7%, 86.7%, 87.2%, 87.5%, 87.5%, 87.5%, 87.5%, 87.5%, 87.5%, 87.5%, 87.5%, 87.5%, 87.5%, 87.5%, 87.5%, 87.5%, 87.5%, 87.5%, 87.5%, 87.5%, 87.5%, 87.5%, 87.5%, 87.5%, 87.5%, 87.5%, 87.5%, 87.5%, 87.5%, 87.5%, 87.5%, 87.5%, 87.5%, 87.5%, 89.6%, 89.6%, 90.0%, 90.3%, 90.6%, 90.6%, 90.6%, 91.7%, 91.7%, 91.7%, 91.7%, 91.7%, 91.7%, 91.7%, 91.7%, 91.7%, 91.7%, 91.7%, 91.7%, 91.7%, 91.7%, 91.7%, 93.1%, 93.3%, 93.8%, 93.8%, 93.8%, 93.8%, 93.8%, 93.8%, 93.8%, 93.8%, 93.8%, 93.8%, 93.8%, 93.8%, 93.8%, 93.8%, 93.8%, 93.8%, 93.8%, 93.8%, 93.8%, 93.8%, 93.8%, 93.8%, 93.8%, 93.8%, 93.8%, 93.8%, 93.8%, 93.8%, 93.8%, 93.8%, 93.8%, 93.8%, 93.8%, 95.8%, 95.8%, 95.8%, 95.8%, 95.8%, 95.8%, 95.8%, 95.8%, 95.8%, 95.8%, 95.8%, 95.8%, 95.8%, 95.8%, 95.8%, 95.8%, 95.8%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%.0.8 pp1730 / 174521,175
Frontier model BGrok Build53.4%Task means: 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 1.4%, 1.8%, 1.9%, 2.4%, 2.9%, 3.3%, 3.4%, 3.6%, 3.7%, 3.8%, 4.2%, 4.5%, 4.5%, 5.0%, 5.0%, 5.3%, 5.6%, 5.6%, 5.6%, 5.6%, 5.6%, 5.7%, 6.3%, 6.3%, 6.3%, 7.1%, 7.5%, 7.6%, 7.7%, 7.7%, 7.7%, 7.9%, 7.9%, 7.9%, 8.0%, 8.0%, 8.3%, 8.3%, 8.3%, 8.3%, 8.3%, 8.3%, 8.3%, 8.3%, 8.3%, 8.3%, 8.3%, 8.7%, 8.7%, 9.1%, 9.1%, 9.1%, 9.1%, 9.4%, 10.0%, 10.0%, 10.0%, 10.0%, 10.0%, 10.0%, 10.0%, 10.0%, 10.4%, 10.7%, 10.7%, 11.1%, 11.1%, 11.1%, 11.1%, 11.4%, 11.5%, 11.6%, 11.8%, 11.9%, 12.0%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 12.5%, 13.3%, 13.3%, 13.3%, 13.3%, 13.6%, 14.0%, 14.3%, 14.3%, 14.3%, 14.3%, 15.4%, 15.9%, 16.0%, 16.7%, 16.7%, 16.7%, 16.7%, 16.7%, 17.4%, 18.2%, 18.2%, 18.2%, 18.2%, 18.2%, 19.0%, 19.0%, 20.0%, 20.0%, 20.0%, 20.0%, 20.0%, 20.0%, 20.7%, 20.8%, 21.1%, 21.1%, 21.1%, 21.1%, 21.1%, 21.4%, 21.4%, 22.2%, 22.2%, 22.2%, 22.2%, 22.2%, 22.2%, 22.2%, 22.2%, 22.2%, 22.2%, 22.2%, 22.2%, 22.2%, 22.2%, 22.2%, 22.2%, 22.2%, 22.7%, 23.1%, 23.1%, 23.1%, 23.1%, 23.7%, 23.8%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 25.0%, 26.1%, 26.3%, 26.9%, 27.3%, 27.3%, 27.3%, 29.2%, 29.2%, 29.7%, 30.0%, 30.0%, 30.0%, 30.0%, 30.0%, 30.0%, 30.4%, 30.8%, 31.3%, 31.6%, 33.3%, 33.3%, 33.3%, 33.3%, 33.3%, 33.3%, 33.3%, 33.3%, 33.3%, 33.3%, 33.3%, 33.3%, 34.5%, 34.6%, 34.8%, 35.0%, 35.6%, 36.1%, 36.4%, 36.4%, 36.8%, 36.8%, 37.0%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 37.5%, 38.5%, 40.0%, 40.0%, 40.0%, 40.0%, 40.0%, 40.0%, 41.7%, 41.7%, 41.7%, 41.7%, 42.3%, 42.9%, 42.9%, 42.9%, 44.4%, 44.4%, 44.4%, 44.4%, 44.4%, 44.4%, 44.4%, 45.0%, 45.5%, 45.5%, 45.5%, 45.8%, 46.2%, 46.7%, 47.1%, 48.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 51.5%, 52.0%, 53.8%, 53.8%, 53.8%, 54.5%, 54.5%, 54.5%, 54.5%, 55.6%, 55.6%, 55.6%, 56.0%, 57.1%, 57.1%, 57.1%, 57.1%, 57.1%, 58.3%, 58.3%, 58.5%, 59.3%, 59.4%, 60.0%, 60.0%, 60.9%, 61.1%, 61.3%, 61.5%, 61.5%, 61.5%, 61.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 62.5%, 63.2%, 63.6%, 63.6%, 64.3%, 64.3%, 64.3%, 64.7%, 65.4%, 65.4%, 65.5%, 65.7%, 66.7%, 66.7%, 66.7%, 66.7%, 66.7%, 66.7%, 66.7%, 66.7%, 66.7%, 66.7%, 66.7%, 66.7%, 66.7%, 66.7%, 66.7%, 66.7%, 66.7%, 66.7%, 66.7%, 66.7%, 66.7%, 67.9%, 67.9%, 67.9%, 68.8%, 69.2%, 69.2%, 69.2%, 69.6%, 70.0%, 70.0%, 70.0%, 70.0%, 70.0%, 70.6%, 70.8%, 71.4%, 71.4%, 72.2%, 72.2%, 72.7%, 72.7%, 72.7%, 72.7%, 72.7%, 73.5%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 75.0%, 76.5%, 76.5%, 76.9%, 76.9%, 76.9%, 77.3%, 77.8%, 78.6%, 78.6%, 78.6%, 78.6%, 79.2%, 79.4%, 80.0%, 80.0%, 80.0%, 81.0%, 81.3%, 81.3%, 81.3%, 81.3%, 81.3%, 81.3%, 81.3%, 81.3%, 81.3%, 81.3%, 81.3%, 81.3%, 81.8%, 81.8%, 82.1%, 82.4%, 83.3%, 83.3%, 83.3%, 83.3%, 83.3%, 83.3%, 83.3%, 83.3%, 83.3%, 83.3%, 83.3%, 84.6%, 85.2%, 85.7%, 86.0%, 87.5%, 87.5%, 87.5%, 87.5%, 87.5%, 88.2%, 88.3%, 88.9%, 88.9%, 88.9%, 88.9%, 88.9%, 88.9%, 88.9%, 89.3%, 89.3%, 90.0%, 90.0%, 90.0%, 90.0%, 90.0%, 90.0%, 90.9%, 91.7%, 91.7%, 91.7%, 91.7%, 91.7%, 91.7%, 91.7%, 91.7%, 91.7%, 91.7%, 92.3%, 92.3%, 92.3%, 92.3%, 92.3%, 92.3%, 92.9%, 93.3%, 93.3%, 93.8%, 93.8%, 93.8%, 93.8%, 93.8%, 93.8%, 93.8%, 93.8%, 93.8%, 93.8%, 93.8%, 95.0%, 95.0%, 95.0%, 95.0%, 95.0%, 95.0%, 96.2%, 96.2%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%.1.2 pp736 / 174511,485
Qwen3.8 27BTinker bash48.8%Task means: 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 0.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 50.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%, 100.0%.6.8 pp40 / 174585
Pass rate per task
Tasks02004006000: 22010: 1041020: 1242030: 1613040: 914050: 1985060: 2306070: 1317080: 1538090: 11590100: 401100median 67%

Pass rate (%) · 63.3% of 34,481 scored trials pass

Agent steps per task
Tasks02004006000: 97050: 48750100: 429100150: 370150200: 194200250: 106250300: 31300350+: 16350+median 131

Median steps per trial · 1,730 tasks

Trial runtime
Trials010,00020,000<30m: 19495<30m30m–1h: 676130m–1h1–1.5h: 32301–1.5h1.5–2h: 18341.5–2h2–2.5h: 12662–2.5h2.5–3h: 5772.5–3h3h+: 13193h+median 25m

Elapsed runtime · 34,482 timed trials

24 of 24 tasks
Public trials
v3.1.255.6%18 attempts5 / 826m 32s
PASS24mPASS25mPASS26mPASS27mPASS28mFAIL22mFAIL28mFAIL29m
v3.1.357.1%28 attempts4 / 822m 38s
PASS18mPASS21mPASS22mPASS24mFAIL22mFAIL24mFAIL40mFAIL44m
v3.1.244.4%18 attempts4 / 818m 10s
PASS17mPASS19mPASS19mPASS20mFAIL15mFAIL18mFAIL18mFAIL18m
v3.3.230.0%20 attempts3 / 845m 19s
PASS44mPASS47mPASS55mFAIL40mFAIL42mFAIL43mFAIL50mFAIL56m
v3.1.452.9%34 attempts2 / 81h 31m
PASS1h 31mPASS1h 36mFAIL1h 27mFAIL1h 30mFAIL1h 31mFAIL1h 33mFAIL1h 35mFAIL1h 48m
v3.2.157.1%21 attempts3 / 820m 54s
PASS18mPASS23mPASS27mFAIL9mFAIL18mFAIL20mFAIL22mFAIL22m
v3.2.236.8%19 attempts3 / 840m 6s
PASS39mPASS44mPASS50mFAIL34mFAIL38mFAIL39mFAIL41mFAIL43m
v4.0.033.3%9 attempts3 / 812m 10s
PASS42mPASS10mPASS12mFAIL10mFAIL11mFAIL12mFAIL14mFAIL14m
v3.1.230.3%33 attempts4 / 817m 37s
PASS16mPASS16mPASS18mPASS22mFAIL18mFAIL18mFAIL20mFAIL22m
v3.1.247.4%19 attempts4 / 822m 35s
PASS21mPASS22mPASS24mPASS25mFAIL18mFAIL19mFAIL23mFAIL24m
v3.3.055.6%9 attempts5 / 814m 25s
PASS14mPASS14mPASS15mPASS15mPASS16mFAIL10mFAIL14mFAIL16m
v3.3.324.1%29 attempts4 / 849m 23s
PASS44mPASS46mPASS48mPASS52mFAIL43mFAIL51mFAIL55mFAIL56m
v3.2.673.3%60 attempts4 / 854m 47s
PASS50mPASS53mPASS1h 2mPASS1h 14mFAIL47mFAIL48mFAIL56mFAIL1h 4m
v3.1.242.9%21 attempts4 / 816m 9s
PASS13mPASS14mPASS14mPASS15mFAIL17mFAIL18mFAIL18mFAIL18m
v3.0.561.5%26 attempts4 / 811m 14s
PASS10mPASS11mPASS12mPASS12mFAIL9mFAIL10mFAIL12mFAIL12m
v3.3.250.0%18 attempts4 / 812m 50s
PASS11mPASS12mPASS13mPASS13mFAIL10mFAIL13mFAIL13mFAIL16m
v3.2.060.0%10 attempts4 / 817m 16s
PASS31mPASS15mPASS15mPASS17mFAIL15mFAIL17mFAIL18mFAIL20m
v3.1.047.1%17 attempts4 / 817m 50s
PASS37mPASS14mPASS18mPASS19mFAIL15mFAIL15mFAIL18mFAIL24m
v3.1.261.1%18 attempts4 / 89m 56s
PASS9mPASS10mPASS10mPASS10mFAIL8mFAIL10mFAIL10mFAIL12m
v3.2.332.5%40 attempts4 / 821m 24s
PASS20mPASS22mPASS24mPASS25mFAIL17mFAIL19mFAIL21mFAIL22m
v3.2.064.7%17 attempts5 / 87m 52s
PASS31mPASS7mPASS8mPASS8mPASS8mFAIL7mFAIL7mFAIL8m
v3.3.138.9%18 attempts4 / 820m 5s
PASS18mPASS19mPASS22mPASS23mFAIL19mFAIL19mFAIL21mFAIL22m
v3.2.230.8%26 attempts2 / 824m 19s
PASS21mPASS25mFAIL21mFAIL22mFAIL23mFAIL26mFAIL27mFAIL27m
v3.2.236.4%33 attempts5 / 839m 35s
PASS33mPASS36mPASS40mPASS42mPASS50mFAIL30mFAIL39mFAIL44m