We’ve updated how the Steerability score is calculated on the Agent Arena signal leaderboard. Previously, it only measured whether a model fixed a mistake right after being corrected by a user. This meant that the signal rewarded corrected turns, but it didn’t reward a model for needing fewer corrections in the first place. That made the signal not as accurate as it could be: a model that made more mistakes had more chances to "pass," while a model that got things right the first time earned no credit.
The newest frontier models require corrections less often than other models, but the old score didn’t capture that advantage. The updated methodology now evaluates every assistant turn in a conversation we can judge, so getting things right the first time counts. It also continues to weigh repeated corrections more heavily than one-off corrections that are resolved immediately.
We're sharing this update because we want the community and our customers to understand exactly how and why it's changing, not just see scores move. Backfilled scores under the new methodology will be available in our public HuggingFace datasets soon. Full technical breakdown of the new definition is available on the signal definitions page.
ideogram-4.5 has been added to the Image Edit leaderboard. …