COASTERBENCH
A benchmark in which language models design roller coasters that RollerCoaster Tycoon 2 builds, tests, and rates.
Each round a model submits a track program. OpenRCT2 builds it, runs a test train, and returns ratings and a screenshot; the reported score is excitement.
track program"] --> B["build + validate"] B --> T["test train"] T --> R["ratings + screenshot"] R --> M B -. rejected piece .-> M
Pieces are placed through the same validated action path the game uses for player input, so an impossible curve or an open circuit fails as it would in-game. Intensity above 10 suppresses excitement, a crash disqualifies the ride, and designs close to the stock library scale toward zero. Ratings come from the game, not from a model. Modes: design — design mode — models build from scratch. library — library mode — models may search the stock track design library, which measures retrieval and adaptation; copies score zero. open note — a modifier on either mode — the agent is also given the engine source it is scored by, so it can read the ratings code instead of guessing; scores are not comparable with black-box runs.
| run | setup | model | score | int / nau | tokens / cost | |
|---|---|---|---|---|---|---|
|
20260723-v2-library2026-07-23 | library · wooden · driver-api | 1st claude-opus-4-8 | 8.43round 6/6 | / | — |
|
20260723-v2-library2026-07-23 | library · wooden · driver-api | 2nd claude-fable-5 | 7.53round 6/6 | / | — |
|
20260725-opus5-opennote2026-07-25 | design + open note · steel twister · claude-code | 1st claude-opus-5 | 7.39round 6/6 | / | 286 in / 200k out · $7.91 |
|
20260724-opus5-twister2026-07-24 | design · steel twister · claude-code | 1st claude-opus-5 | 7.13round 4/6 | / | 403 in / 359k out · $12.67 |
|
20260725-opus5-bash-probe2026-07-25 | design · steel twister · claude-code | 1st claude-opus-5 | 7.06round 1/2 | / | 191 in / 150k out · $5.82 |
|
20260724-kimi-k3-twister2026-07-24 | design · steel twister · opencode | 1st openrouter_moonshotai_kimi-k3 | 7.05round 2/6 | / | 584k in / 321k out · $9.44 |
|
20260725-opus5-blind2026-07-25 | design · steel twister · claude-code | 1st claude-opus-5 | 7.02round 1/6 | / | 414 in / 334k out · $11.81 |
|
20260725-opus5-python-probe2026-07-25 | design + open note · steel twister · claude-code | 1st claude-opus-5 | 6.84round 1/2 | / | 165 in / 113k out · $6.55 |
|
20260723-v2-design2026-07-23 | design · wooden · driver-api | 1st claude-fable-5 | 6.73round 6/6 | / | — |
|
20260724-opus48-twister2026-07-24 | design · steel twister · claude-code | 1st claude-opus-4-8 | 6.67round 5/6 | / | 531 in / 382k out · $26.57 |
|
20260723-showdown2026-07-23 | design · wooden · driver-api | 1st claude-fable-5 | 6.59round 6/6 | / | — |
|
20260723-v2-design2026-07-23 | design · wooden · driver-api | 2nd claude-opus-4-8 | 6.38round 6/6 | / | — |
|
20260723-v2-library2026-07-23 | library · wooden · driver-api | 3rd claude-sonnet-5 | 6.30round 6/6 | / | — |
|
20260723-showdown2026-07-23 | design · wooden · driver-api | 2nd claude-sonnet-5 | 5.95round 6/6 | / | — |
|
20260723-showdown2026-07-23 | design · wooden · driver-api | 3rd claude-opus-4-8 | 5.78round 3/6 | / | — |
|
20260723-v2-design2026-07-23 | design · wooden · driver-api | 3rd claude-sonnet-5 | 5.71round 6/6 | / | — |
|
20260725-codex-kimi2026-07-25 | design + open note · steel twister · codex | 1st openrouter_moonshotai_kimi-k3 | 0.57round 1/1 | / | $1.23 |
|
20260723-v3-twister-design2026-07-23 | design · steel twister · driver-api | 1st claude-opus-4-6 | 0.24round 1/6 | / | — |
|
20260725-opencode-mini-control2026-07-25 | design + open note · steel twister · opencode | 1st openrouter_openai_gpt-5-mini | no successful coaster | 28k in / 8k out · $0.04 | |
|
20260725-codex-smoke22026-07-25 | design + open note · steel twister · codex | 1st openrouter_openai_gpt-5-mini | no successful coaster | 2.0M in / 13k out · $0.10 | |
|
20260725-codex-smoke2026-07-25 | design + open note · steel twister · codex | 1st openrouter_openai_gpt-5-nano | no successful coaster | 134k in / 4k out · $0.00 | |
|
20260725-codex-scored2026-07-25 | design + open note · steel twister · codex | 1st openrouter_openai_gpt-5.1 | no successful coaster | $1.56 | |
|
20260725-codex-oauth-probe2026-07-25 | design + open note · steel twister · codex | 1st gpt-5.6-sol | no successful coaster | $0.00 | |
|
20260723-v3-twister-design2026-07-23 | design · steel twister · driver-api | 2nd claude-haiku-4-5 | no successful coaster | — | |
|
20260723-v3-twister-design2026-07-23 | design · steel twister · driver-api | 3rd claude-sonnet-5 | no successful coaster | — | |
|
20260722-2248372026-07-22 | design · wooden · driver-api | 1st claude-opus-4-6 | no successful coaster | — | |
|
20260722-2248372026-07-22 | design · wooden · driver-api | 2nd claude-sonnet-5 | no successful coaster | — | |
not shown, incomplete or still running: 20260725-kimi-k3-fresh, 20260724-laguna-wooden, 20260723-sub-twister-1, 20260723-library