Benchmark Almanac

GenAI-Arena

FLUX.1 [dev] 1,133
Not in model cards

Users compare outputs from two anonymous models for image generation, image editing and video generation, and vote. Votes give Elo ratings.

Published at NeurIPS 2024 Datasets and Benchmarks. The Hugging Face space shows a build error and was last changed on July 3rd, 2025.

Released
June 2024
Kind
Image generation and editing
Level
Everyday
Scored
Ranked by votes
Answers
Open-ended
Tasks
All public
Scores refreshed
November 12th, 2026

In the authors’ words

Generative AI has made remarkable strides to revolutionize fields such as image and video generation. These advancements are driven by innovative algorithms, architecture, and data. However, the rapid proliferation of generative models has highlighted a critical gap: the absence of trustworthy evaluation metrics. Current automatic assessments such as FID, CLIP, FVD, etc often fail to capture the nuanced quality and user satisfaction associated with generative outputs. This paper proposes an open platform GenAI-Arena to evaluate different image and video generative models, where users can actively participate in evaluating these models. By leveraging collective user feedback and votes, GenAI-Arena aims to provide a more democratic and accurate measure of model performance. It covers three tasks of text-to-image generation, text-to-video generation, and image editing respectively. Currently, we cover a total of 35 open-source generative models. GenAI-Arena has been operating for seven months, amassing over 9000 votes from the community. We describe our platform, analyze the data, and explain the statistical methods for ranking the models. To further promote the research in building model-based evaluation metrics, we release a cleaned version of our preference data for the three tasks, namely GenAI-Bench. We prompt the existing multi-modal models like Gemini, and GPT-4o to mimic human voting. We compute the accuracy by comparing the model voting with the human voting to understand their judging abilities. Our results show existing multimodal models are still lagging in assessing the generated visual content, even the best model GPT-4o only achieves an average accuracy of 49.19 across the three generative tasks. Open-source MLLMs perform even worse due to the lack of instruction-following and reasoning ability in complex vision scenarios.

Abstract of GenAI Arena: An Open Evaluation Platform for Generative Models (arXiv:2406.04485, CC0)
42
Entries on its own leaderboard, 1 model we track

FLUX.1 [dev] leads with 1,133

Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

42 entries. Scroll sideways for the rest. Bars start at 700.

  1. 1,133 FLUX.1 [dev]
  2. 1,126 CogVideoX-5B
  3. 1,121 Mochi1
  4. 1,117 PlayGround V2.5
  5. 1,116 StableVideoDiffusion
  6. 1,114 Pyramid Flow
  7. 1,098 MagicBrush
  8. 1,094 FLUX.1-schnell
  9. 1,072 PlayGround V2
  10. 1,068 UltraEdit
  11. 1,064 InfEdit
  12. 1,061 Kolors
  13. 1,056 CosXLEdit
  14. 1,055 T2V-Turbo
  15. 1,054 CogVideoX-2B
  16. 1,043 StableCascade
  17. 1,039 AnimateDiff
  18. 1,029 VideoCrafter2
  19. 1,029 InstructPix2Pix
  20. 1,022 HunyuanDiT
  21. 1,018 PixArtAlpha
  22. 1,017 PixArtSigma
  23. 1,014 SDXL-Lightning
  24. 1,006 SD3
  25. 1,002 Allegro
  26. 994 AuraFlow
  27. 993 PNP
  28. 984 Prompt2prompt
  29. 967 SDXL
  30. 964 LaVie
  31. 964 LTXVideo
  32. 936 CycleDiffusion
  33. 919 SDEdit
  34. 915 SDXLTurbo
  35. 905 LCM(v1.5/XL)
  36. 881 OpenSora
  37. 872 OpenSora v1.2
  38. 855 Pix2PixZero
  39. 835 AnimateDiff Turbo
  40. 832 OpenJourney
  41. 830 ModelScope
  42. 790 LCM

Bars start at 700.

  1. 1 FLUX.1 [dev] t2i_generation; anonymous votes 1,133
  2. 2 CogVideoX-5B video_generation; anonymous votes 1,126
  3. 3 Mochi1 video_generation; anonymous votes 1,121
  4. 4 PlayGround V2.5 t2i_generation; anonymous votes 1,117
  5. 5 StableVideoDiffusion video_generation; anonymous votes 1,116
  6. 6 Pyramid Flow video_generation; anonymous votes 1,114
  7. 7 MagicBrush image_editing; anonymous votes 1,098
  8. 8 FLUX.1-schnell t2i_generation; anonymous votes 1,094
  9. 9 PlayGround V2 t2i_generation; anonymous votes 1,072
  10. 10 UltraEdit image_editing; anonymous votes 1,068
  11. 11 InfEdit image_editing; anonymous votes 1,064
  12. 12 Kolors t2i_generation; anonymous votes 1,061
  13. 13 CosXLEdit image_editing; anonymous votes 1,056
  14. 14 T2V-Turbo video_generation; anonymous votes 1,055
  15. 15 CogVideoX-2B video_generation; anonymous votes 1,054
  16. 16 StableCascade t2i_generation; anonymous votes 1,043
  17. 17 AnimateDiff video_generation; anonymous votes 1,039
  18. 18 VideoCrafter2 video_generation; anonymous votes 1,029
  19. 19 InstructPix2Pix image_editing; anonymous votes 1,029
  20. 20 HunyuanDiT t2i_generation; anonymous votes 1,022
  21. 21 PixArtAlpha t2i_generation; anonymous votes 1,018
  22. 22 PixArtSigma t2i_generation; anonymous votes 1,017
  23. 23 SDXL-Lightning t2i_generation; anonymous votes 1,014
  24. 24 SD3 t2i_generation; anonymous votes 1,006
  25. 25 Allegro video_generation; anonymous votes 1,002
  26. 26 AuraFlow t2i_generation; anonymous votes 994
  27. 27 PNP image_editing; anonymous votes 993
  28. 28 Prompt2prompt image_editing; anonymous votes 984
  29. 29 SDXL t2i_generation; anonymous votes 967
  30. 30 LaVie video_generation; anonymous votes 964
  31. 31 LTXVideo video_generation; anonymous votes 964
  32. 32 CycleDiffusion image_editing; anonymous votes 936
  33. 33 SDEdit image_editing; anonymous votes 919
  34. 34 SDXLTurbo t2i_generation; anonymous votes 915
  35. 35 LCM(v1.5/XL) t2i_generation; anonymous votes 905
  36. 36 OpenSora video_generation; anonymous votes 881
  37. 37 OpenSora v1.2 video_generation; anonymous votes 872
  38. 38 Pix2PixZero image_editing; anonymous votes 855
  39. 39 AnimateDiff Turbo video_generation; anonymous votes 835
  40. 40 OpenJourney t2i_generation; anonymous votes 832
  41. 41 ModelScope video_generation; anonymous votes 830
  42. 42 LCM t2i_generation; anonymous votes 790

Source: Official GenAI-Arena leaderboard

#EntryScoreSettingDateSource
1 FLUX.1 [dev] 1,133 t2i_generation; anonymous votes Official leaderboard
2 CogVideoX-5B 1,126 video_generation; anonymous votes Official leaderboard
3 Mochi1 1,121 video_generation; anonymous votes Official leaderboard
4 PlayGround V2.5 1,117 t2i_generation; anonymous votes Official leaderboard
5 StableVideoDiffusion 1,116 video_generation; anonymous votes Official leaderboard
6 Pyramid Flow 1,114 video_generation; anonymous votes Official leaderboard
7 MagicBrush 1,098 image_editing; anonymous votes Official leaderboard
8 FLUX.1-schnell 1,094 t2i_generation; anonymous votes Official leaderboard
9 PlayGround V2 1,072 t2i_generation; anonymous votes Official leaderboard
10 UltraEdit 1,068 image_editing; anonymous votes Official leaderboard
11 InfEdit 1,064 image_editing; anonymous votes Official leaderboard
12 Kolors 1,061 t2i_generation; anonymous votes Official leaderboard
13 CosXLEdit 1,056 image_editing; anonymous votes Official leaderboard
14 T2V-Turbo 1,055 video_generation; anonymous votes Official leaderboard
15 CogVideoX-2B 1,054 video_generation; anonymous votes Official leaderboard
16 StableCascade 1,043 t2i_generation; anonymous votes Official leaderboard
17 AnimateDiff 1,039 video_generation; anonymous votes Official leaderboard
18 VideoCrafter2 1,029 video_generation; anonymous votes Official leaderboard
19 InstructPix2Pix 1,029 image_editing; anonymous votes Official leaderboard
20 HunyuanDiT 1,022 t2i_generation; anonymous votes Official leaderboard
21 PixArtAlpha 1,018 t2i_generation; anonymous votes Official leaderboard
22 PixArtSigma 1,017 t2i_generation; anonymous votes Official leaderboard
23 SDXL-Lightning 1,014 t2i_generation; anonymous votes Official leaderboard
24 SD3 1,006 t2i_generation; anonymous votes Official leaderboard
25 Allegro 1,002 video_generation; anonymous votes Official leaderboard
26 AuraFlow 994 t2i_generation; anonymous votes Official leaderboard
27 PNP 993 image_editing; anonymous votes Official leaderboard
28 Prompt2prompt 984 image_editing; anonymous votes Official leaderboard
29 SDXL 967 t2i_generation; anonymous votes Official leaderboard
30 LaVie 964 video_generation; anonymous votes Official leaderboard
31 LTXVideo 964 video_generation; anonymous votes Official leaderboard
32 CycleDiffusion 936 image_editing; anonymous votes Official leaderboard
33 SDEdit 919 image_editing; anonymous votes Official leaderboard
34 SDXLTurbo 915 t2i_generation; anonymous votes Official leaderboard
35 LCM(v1.5/XL) 905 t2i_generation; anonymous votes Official leaderboard
36 OpenSora 881 video_generation; anonymous votes Official leaderboard
37 OpenSora v1.2 872 video_generation; anonymous votes Official leaderboard
38 Pix2PixZero 855 image_editing; anonymous votes Official leaderboard
39 AnimateDiff Turbo 835 video_generation; anonymous votes Official leaderboard
40 OpenJourney 832 t2i_generation; anonymous votes Official leaderboard
41 ModelScope 830 video_generation; anonymous votes Official leaderboard
42 LCM 790 t2i_generation; anonymous votes Official leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

No dated values yet.

No lab model card reports it yet

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.

More in Vision

All 281 in Vision
  • MMMU In use 11.5K college-level multimodal questions across 30 subjects. Multiple-choice and open accuracy. GPT-5.1 85.4%
  • MMMU-Pro In use Harder MMMU: college-level multimodal questions with more options and vision-only inputs to remove text shortcuts. GPT-5.6 Sol 84.6%
  • CharXiv In use Chart understanding on real arXiv figures. Reasoning questions require synthesizing information from complex charts. Claude Fable 5 93.5%
  • MATH-Vision (MathVision) In use 3,040 competition math problems with visual context. Answer accuracy. Qwen 3.8 Max 97.7%
  • MMStar In use Models answer 1,500 human-curated image questions that cannot be solved without the image, across 6 core capabilities. Scored by accuracy. Qwen 3.8 Max 85.9%
  • ChartQA In use Models answer human-written and generated questions about chart images that need visual and arithmetic reasoning. Scored by relaxed accuracy with a 5 percent numeric tolerance. Command A Vision 90.9%