---
title: Elo Rating — AI Glossary
description: An Elo rating is a relative skill score derived from head-to-head comparisons, borrowed from chess and applied to models. In systems like Chatbot Arena, human voters pick the better of two model responses, and Elo converts these pairwise wins and losses into a ranking that reflects overall preference.
canonical: https://prompeteer.ai/glossary/elo-rating
---

# Elo Rating

An Elo rating is a relative skill score derived from head-to-head comparisons, borrowed from chess and applied to models. In systems like Chatbot Arena, human voters pick the better of two model responses, and Elo converts these pairwise wins and losses into a ranking that reflects overall preference.

## Definition

Elo scores rank competitors by outcomes against one another rather than by an absolute test. Each comparison updates the participants' ratings based on who won and how surprising the result was, so a model that consistently beats strong opponents rises. This suits subjective quality, where no fixed answer key exists.

Chatbot Arena popularized Elo for language models by crowdsourcing blind pairwise votes between anonymized model outputs. The approach captures human preference at scale, but reflects the voter population and prompt distribution, so it complements rather than replaces task-specific benchmarks and evals.

## Sources

- [Chiang et al. (2024), Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference](https://arxiv.org/abs/2403.04132)
- [Elo, A. (1978), The Rating of Chessplayers, Past and Present](https://www.worldcat.org/title/rating-of-chessplayers-past-and-present/oclc/3789663)

## Related terms

- [benchmark](https://prompeteer.ai/glossary/benchmark)
- [evaluation](https://prompeteer.ai/glossary/evaluation)