minecraftbench.com
$
worldgen macro per-block-class accuracy vs the real game, mean over post-freeze time-seeds
claude-opus-4-8 (effort max)
83.1%
grok-4.5 (high effort)
32.7%
gpt-5.5 (xhigh)
17.5%
baseline: all-stone
14.4%
baseline: superflat
9.1%
where they die per-block-class accuracy %; dark column = nobody has cracked it
| air | stone-family | bedrock | dirt/grass | sand/gravel | water | lava | ores | wood | leaves | vegetation | snow/ice | clay/terracotta | structure-ish | other | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| claude-opus-4-8 (effort max) | 100 | 99 | 100 | 99 | 97 | 97 | 98 | 97 | 61 | 31 | 73 | 91 | 97 | 26 | 89 |
| grok-4.5 (high effort) | 99 | 75 | 73 | 53 | 31 | 85 | 9 | · | · | 3 | 5 | 1 | · | · | 31 |
| gpt-5.5 (xhigh) | 97 | 75 | 57 | 3 | 4 | 13 | · | · | · | · | · | · | · | · | · |
| model | harness | prompt | date | built | worldgen | traj div | traj state | budget | cost | audit | note | trace |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| claude-opus-4-8 (effort max) | claude | v2 | 2026-07-09 | yes | 83.1% | - | - | 4.61h / 24h | $267 | clean | full-pack isolation run; self-terminated after ~4.6h of 24h; vendored MIT cubiomes + public-domain stb; implements --dump-world/--replay-tape and CUDA batch path; self-reported ~88.7 macro on its own 19 seeds vs official 10 time-seed protocol 83.11; residual gaps mostly decoration (big-oak static, jungle leaf/vine) per agent notes | - |
| grok-4.5 (high effort) | grok | v2 | 2026-07-08 | yes | 32.7% | - | - | 0.34h / 24h | - | clean | first run under the full pack (isolated harness + tape contract); vendored five open-source worldgen reimplementations (cubiomes, glowstone, pocketmine vanillagen, fluffy-parakeet, +1 - all clean-room, within the vendoring ruling) and implements --replay-tape; its earlier 74.75 predates per-run isolation and had grok cross-session memory of two prior same-day attempts, so that figure was iteration-assisted (see DEVLOG) - this 20 min run is the clean single-shot result | - |
| gpt-5.5 (xhigh) | codex | v2 | 2026-07-08 | yes | 17.5% | - | - | 0.19h / 24h | - | clean | first run under the full pack (isolated harness + tape contract); self-terminated at 11 min of 24h, sign-off concedes 'deterministic approximations, not vanilla-oracle matching', no CUDA; does implement --replay-tape, so the trajectory leg can score once the graded tape lands; worldgen consistent with its prior run (17.34) | - |
| baseline: all-stone | - | - | - | - | 14.4% | - | - | - | - | - | trivial floor: bedrock + stone to y63 + air; 20 time-seeds | - |
| baseline: superflat | - | - | - | - | 9.1% | - | - | - | - | - | trivial floor: vanilla flat preset; 20 time-seeds | - |