What is Benchmark? Plain-English Definition

In more detail

A benchmark is a standardized exam for AI — a fixed set of questions in math, coding, reasoning or general knowledge that every model can be scored on, so rivals can be compared fairly. New model launches always arrive brandishing benchmark results.

It matters, with one caveat: benchmarks are the closest thing AI has to objective measurement, but models can effectively cram for well-known tests — sometimes the questions even leak into training data. Treat headline scores like school grades: a real signal, never the whole student.

📌 See it in action

Two models each claim to be the best at coding, citing wins on different benchmarks. One aces the test yet fumbles your actual project; the other, less decorated, just quietly works. Seasoned users read benchmarks the way sensible employers read report cards — and then run their own interview.

Goes with

Arena Elo →Model Card →Frontier Model →

Now put the vocabulary to work.

Meet the models →Which AI is mine?