Slop code bench is unique among benchmarks in that it makes a model work against a progressively disclosed list of features (checkpoints) that it doesn’t know it advance, inheriting the same codebase through the lifecycle of a single multi-checkpoint challenge
I like this because it much more mirrors how real software dev happens, rather than most benchmarks with one off tasks where slop code doesn’t hinder future progress
The original post[1] and the SOL/fable follow up[2] went into a bit more detail on the benchmark and its challenges
1. https://news.ycombinator.com/item?id=49076391
2. https://github.com/humanlayer/advanced-context-engineering-f...","time":1792091374,"title":"SlopCodeBench 10/2026 – Opus 5.5, Astra wins overall, xhigh considered harmful","type":"story","url":"https://github.com/humanlayer/advanced-context-engineering-for-coding-agents/blob/main/benchmarking-new-models-on-slop-code-bench.md"}