AI evaluation: tests, benchmarks and LLM-as-judge

In short: How good a model is is measured with tests: fixed benchmarks, human comparisons and increasingly other models as raters. Each method has weaknesses.

In more detail: Language models can do many things, so there is no single number. Good evaluation combines several methods and own tests for your own purpose.

In detail

• Benchmarks: MMLU (knowledge), GSM8K/MATH (arithmetic), HumanEval/SWE-bench (code), GPQA (expert questions). Many are “saturated”: top models reach almost the maximum.
• Contamination: test questions can be in the training data. New, secret or continuously renewed tests help.
• Human arena: users compare two anonymous answers, an Elo ranking results (Chatbot Arena).
• LLM-as-judge: a strong model rates answers. Cheap and fast but with preferences (long answers, own style).
• Task evaluations: own test cases with expected results for your own application (“evals”), run before every change.
• Agents: evaluation by successfully solved tasks in environments.
• Measurement problems: prompt wording, sample size, chance (temperature).
• Practice: the best choice is the model that wins in your own tests for your own task.

See also: Benchmarks for language models, Metrics: accuracy, precision, recall and F1