Skip to content

Google's AI won the people's vote. It came second on the exams.

Gemini 4 Carbon tops a public vote where people pick the answer they like, then slips to second on a stack of hard tests. Why the lists disagree, and why neither win is as solid as it looks.

By TodayAI4 min read
Silhouette of a hand dropping a paper ballot into a box against a white background
Silhouette of a hand dropping a paper ballot into a box against a white background· Photo by Element5 Digital on Unsplash

The Quick Read

20-Second Answer

Google's Gemini 4 Carbon is #1 on LMArena, where people vote for the answer they prefer. On Epoch AI's capability index, built from standard tests, it is #2 behind Claude Fable 5.2, which isn't in the vote at all. Both leads are a photo finish.

What Matters

  1. 01A vote measures which answer people would rather read. A test measures whether a model solves hard problems correctly.
  2. 02The two lists can disagree a lot: GPT-6 Astra is #3 on the tests but #36 in the vote, and Claude Opus 4.6 is #6 in the vote but #37 on the tests.
  3. 03Every score comes with a margin of error. Carbon's overlaps with its nearest rivals on both lists, so neither order is settled.

What Should I Do?

For everyday chat and writing help, lean on the vote. For hard maths, science or code, check Epoch's index and the single test closest to your task.

Picture two contests for the same crown. In the first, thousands of strangers read two unlabelled answers side by side and pick the one they like better. In the second, the contestants sit a stack of hard exams that are marked right or wrong.

Google's Gemini 4 Carbon wins the first contest. On the exams it comes a close second, behind a model that never entered the first.

So which one is the best? Neither contest can say on its own, because they count different things. Once you see what each counts, the disagreement becomes useful.

A taste test and a stack of exams

The first contest is LMArena's public voting leaderboard, and it works like a blind taste test. Someone types a question, two anonymous AI models answer, and the person votes for the reply they prefer. Enough votes add up to a score. Carbon's is 1526, the highest on the board.

A vote records which answer someone would rather get, not whether a grader marked it right. Our numbers come from the leaderboard's default view, as described on LMArena's dataset card.

The second contest is Epoch AI's capability index, and it's closer to a pile of final exams. It combines scores from over 50 benchmarks, standard tests made of fixed, hard problems. Anthropic's Claude Fable 5.2 tops it with 173. Carbon scores 170.

Epoch pins its scale so that OpenAI's GPT-5 scores 150, and says only the comparisons between models really count.

A taste test rewards answers that feel good to read. An exam rewards getting the hard problem right, even if the explanation is clunky. Most of the time those go together. Sometimes they really don't.

Where the two lists split

Only 119 models appear on both lists, and lining up their places turns up some big gaps.

OpenAI's GPT-6 Astra is #3 on the exams but #36 in the vote. Claude Sonnet 5.5 slides the same way, from #8 to #41.

Claude Haiku 5.5 falls furthest of these: #18 on the exams, #116 in the vote.

It runs the other way too. Claude Opus 4.6, released back in February, is #6 in the vote but only #37 on the exams. Meta's Muse Wonder is #3 by votes and #15 on the exams.

Slope chart linking each model's rank on Epoch AI's test-based index to its rank in LMArena's public vote: Gemini 4 Carbon #2 to #1, Muse Wonder #15 to #3, Claude Opus 4.6 #37 to #6, GPT-6 Astra #3 to #36, Claude Sonnet 5.5 #8 to #41, Claude Haiku 5.5 #18 to #116.
Slope chart linking each model's rank on Epoch AI's test-based index to its rank in LMArena's public vote: Gemini 4 Carbon #2 to #1, Muse Wonder #15 to #3, Claude Opus 4.6 #37 to #6, GPT-6 Astra #3 to #36, Claude Sonnet 5.5 #8 to #41, Claude Haiku 5.5 #18 to #116.

My read is that the taste test and the exams are each doing their job. A model can be excellent at hard maths and still write answers people find dry. Another can be a pleasure to read without cracking the toughest problems.

Some models show up on only one list. Claude Fable 5.2 has no score in the vote. Neither does OpenAI's GPT-6.1 Terra, #6 on the exams. And Gemini 4 Hydrogen, #13 in the vote, isn't on Epoch's index. That puts a real asterisk on Carbon's win: the model that beats it on the exams wasn't in the voting contest.

A photo finish at the top

Both rankings give each model a range, not a single exact score. That range is the margin of error. Think of a photo finish: if the gap between two runners is smaller than the blur in the photo, you can't call the winner.

In the vote, Carbon's range overlaps with Gemini 4 Argon's at #2 and Muse Wonder's at #3. So its #1 isn't settled. Carbon also has far fewer votes behind it: about 4,300, the fewest of any model in the top 20, against nearly 21,000 for Argon. Fewer votes, blurrier photo. Claude Opus 5.5, at #4, is the first model clearly behind.

The exams are just as tight. Fable 5.2's range overlaps Carbon's, and so does GPT-6 Astra's. "Second" here means "in a close pack at the top."

Epoch's individual tests show how that pack splits. On its maths test, FrontierMath, Fable 5.2 is #1 at 95% and Carbon is #6 at 91%.

On its science test, GPQA Diamond, Carbon is #1 with Argon a hair behind, a tie for practical purposes. Fable 5.2 is #17. Different exams, different winners.

Which ranking I'd trust, and for what

For everyday chat, writing help or having things explained, I'd lean on the vote. It's people choosing the answer they'd rather get, which is what you'll be doing.

For hard, checkable problems like maths, science or tricky code, I'd trust Epoch's index more. Better still, look at the one test closest to your task. You can see all of these side by side on our compare page.

Why care if you never read a leaderboard? In my view, because "the best AI" is a claim you'll keep hearing, and it means little until you know which contest was won, and by how much.

This week, for Gemini 4 Carbon, the answers are "being liked" and "not by much."

Sources and links

  1. LMArena's dataset cardhuggingface.co
  2. combines scores from over 50 benchmarksepoch.ai

Share this story

About the author

BS

TodayAI

Written and researched by the TodayAI team. We publish under this shared byline rather than crediting individual writers while the publication grows. Read our editorial policy.