Google launched Gemini 4 Argon at the end of September, and the chart looked great. It led on most of the 18 benchmarks Google picked. My first reaction wasn’t excitement. It was a flat “sure, okay.”
Gemini has been bad for me lately, really bad, and a model that keeps disappointing me in real use doesn’t get the benefit of the doubt just because a launch chart says otherwise. But I also wanted to know if that was a fair reaction or just a grudge. So I went and checked.
What I found
I should say this first: I have no proof Argon was trained on benchmark answers, and I didn’t find anyone who has shown that. This is about how much weight the chart deserves, not about catching anyone.
What made me squint was mostly who produced the numbers. Google picked the 18 benchmarks, and the methodology PDF says it ran several of them itself, including DeepSWE, Terminal-Bench 4.0 and LVBench. The competitor numbers are mostly what those labs reported about themselves. I couldn’t find anyone who had reproduced any of it at launch.
I also looked for a decontamination statement, meaning some line about how test data was kept out of training. I checked the launch blog, the PDF and the DeepMind page and found nothing. There’s no model card either. Maybe it exists somewhere I didn’t look, but it’s the first thing I’d want to read.
Then there’s the benchmarks themselves. Here are four from the chart that gave me pause:
| Benchmark | Argon | Why I hesitate |
|---|---|---|
| DeepSWE v1.1 | 77.9% (Opus 5.5: 74.2%) | Epoch AI rated it “Flawed” |
| Terminal-Bench 4.0 | 57.4% (Opus 5.5: 66.4%) | Argon trails here, and I only have secondary coverage for the number |
| GraphWalks, 256K–1M | OpenAI dataset | Public since April 2025, with a ground-truth bugfix in Feb 2026 |
| LVBench | 91.7% | From 2024, probably close to saturated |
The DeepSWE problem
DeepSWE is the headline coding number, so I read Epoch AI’s review of it. Epoch found at least 23 of 113 tasks (20.3%) had errors, and 18 of those 23 trace back to one grader quirk. If a model edits an existing test file, the verifier throws the edit away and restores the original. Sometimes that breaks the build, and a correct solution gets marked wrong.
Argon beats Opus 5.5 by 3.7 points on it. With a fifth of the tasks possibly mis-graded, I honestly don’t know what 3.7 points means.
It’s not just DeepSWE. In the same round, Epoch also marked SWE-bench Verified and Humanity’s Last Exam as flawed. On HLE, 22 of 48 sampled questions had errors that changed the answer. So the ruler is bent, and everyone is measuring with it.
To be fair
A grudge is easy to feed, so here’s the part that cuts the other way.
Argon loses on Terminal-Bench 4.0, and Artificial Analysis puts its Intelligence Index around 53, below Opus 5.5’s 57.6. A model tuned hard to win charts would probably win those too, so losing in public counts for something. Some of the wins also come from third-party leaderboards instead of Google’s own runs, like Zapier’s AutomationBench and the ones run by Vals.
The Gemini thing I half-remembered
I had a memory of someone accusing earlier Gemini versions of gaming benchmarks. When I went looking, the real story was smaller than my memory.
The closest match is a 2025 paper, The Leaderboard Illusion, from researchers at Cohere Labs, AI2, Princeton, Stanford and others. It found that Google and OpenAI each got roughly a fifth of all LMArena battle data, and that big providers could privately test many variants and publish only the best one. Google’s side said it sent only one variant for pre-release testing. But the paper is about LMArena only. It doesn’t say Gemini falls apart on held-out tests.
The other one is a LessWrong post from November 2025. The author got Gemini 3 to reproduce a benchmark canary string, which hints that benchmark data was in its training set. It’s suggestive, and the author doesn’t claim more than that.
So I can’t honestly say Gemini was caught benchmaxxing. What I found is a leaderboard that favored big labs, and one odd contamination hint. That’s enough to make me careful, but not enough to call it a verdict.
So, do I trust it?
Not yet. I don’t think launch charts are worthless, they just can’t be the whole story. These days I read the rows where the model loses first, and I check who ran each number before I look at the number.
Mostly I’ll wait for independent results, and then try it on my own work, which is the one test nobody has tuned for. Maybe Argon will surprise me. I’d like it to. But I’ll decide after I’ve used it, not after I’ve looked at the chart.