LiveSPX7,620-1.28%NDX29,127-1.41%US10Y4.961%+3.23%BTC76,816-1.24%ETH2,468-2.07%GOLD4,320-1.02%NVDA210.96-8.42%MSFT505.41+1.14%GOOGL349.39+3.23%
Advertisement
AI glossary

Benchmark

A standard test — and an imperfect one.

Benchmarks score models on fixed question sets covering reasoning, coding, maths or knowledge, so releases can be compared on a common scale.

They decay: once a benchmark is public it leaks into training data, and scores rise without capability rising with them. This is called contamination.

Treat leaderboard position as a weak signal and your own evaluation on your own tasks as the strong one.

Related terms