LLaMA-65B

LLaMA-65B ranks first on IndoMMLU.

Meta Released February 24th, 2023 Compare with others

Benchmarks with a score

Ranked first, out of five or more models

In the top three

Meta Feb 2023

Open weights

101.7 #214 of 256

Epoch Capabilities Index

Where it ranks

Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.

Benchmark	Score	Rank	Source
TriviaQA	86%	#5 of 38	Claim in Llama 2: Open Foundation and Fine-Tuned Chat Models, via Epoch AI
HellaSwag	84.2%	#12 of 63	Claim in LLaMA: Open and Efficient Foundation Language Models, via Epoch AI
IndoMMLU	35.8%	#1 of 5	Official leaderboard
BoolQ	86.6%	#13 of 61	Claim in Qwen Technical Report, via Epoch AI
LAMBADA	77.7%	#7 of 26	Claim in Qwen Technical Report, via Epoch AI
PIQA	82.8%	#16 of 58	Claim in Qwen Technical Report, via Epoch AI
ARC-Challenge	69.5%	#23 of 75	Claim in Qwen Technical Report, via Epoch AI
OpenBookQA	60.2%	#14 of 41	Claim in Llama 2: Open Foundation and Fine-Tuned Chat Models, via Epoch AI
WinoGrande	77%	#28 of 76	Claim in LLaMA: Open and Efficient Foundation Language Models, via Epoch AI
BIG-Bench Hard	58.4%	#17 of 45	Claim in Qwen Technical Report, via Epoch AI
MMLU	63.4%	#37 of 85	Claim in Llama 2: Open Foundation and Fine-Tuned Chat Models, via Epoch AI
CMMLU	39.8%	#11 of 11	Official leaderboard
C-Eval	38.8%	#3 of 3	Official leaderboard

Benchmark	Score		Rank	Source
GSM8K	46.6%		#5 of 24	Stanford HELM, via Epoch AI

Benchmark	Score		Rank	Source
MedQA	50.7%		#61 of 71	Official leaderboard

Benchmark	Score		Rank	Source
nocot-bench (NCRI)	37.5		#179 of 185	Official leaderboard

Benchmark	Score		Rank	Source
HELM Classic	90.8		#2 of 17	Official leaderboard
HELM Lite	34.5		#54 of 71	Official leaderboard

Meta's models in color, against every other model on the Epoch Capabilities Index, by release date.

Source: Epoch Capabilities Index, Epoch AI (CC BY 4.0)