Benchmark Almanac

LMSYS

We track 2 benchmarks that LMSYS made or helped make. Its models are on its lab page.

0 in use 1 with public code 1 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
MT-Bench 80 two-turn questions graded by GPT-4 as judge. Introduced LLM-as-judge methodology. Multi-turn conversation Jun 2023 9.35 Fading
LMSYS Chatbot Arena Human Preference Predictions Models predict which answer a user preferred in an anonymous pair. Kaggle scores the submitted probabilities by log loss. Overall chat quality May 2024 Not in cards