Gemini 4 Argon Tries Its Hand at StarCraft Bot Coding

AI Bots Battle in StarCraft

Google's new Gemini 4 Argon model placed third in a challenging test where artificial intelligence systems had to write bots for the classic strategy game StarCraft: Brood War.

The benchmark, called StarSkirmish Bench version 0.2, gave each AI model one hour to write a Protoss bot in C++ code. The bots were then tested against other AI bots and a roster of competitive bots written by humans.

Gemini 4 Argon scored between 37 and 39, placing it in a tight group with Claude Opus 5.5 and Claude Fable 5.1, all falling behind top performer GPT-6 Astra and second-place GPT-6.1 Sol.

The Rules of the Challenge

Each AI started with a basic bot skeleton that only gathered resources. They had access to demo bot code and two libraries — one for combat simulation and another for analyzing maps. The AIs had to write, compile and improve their bots during the full hour while playing practice games.

Games were played on three maps: Heartbreak Ridge, Benzene and Destination. A match ended when one side lost all its buildings. The practice opponents ranged from simple demo bots to strong human-written bots like Stardust, PurpleWave and BananaBrain.

"The strongest predictor of a model's score is how much it practiced against the 7 strongest bots on the roster," said the StarSkirmish Bench page.

Not Ready for Humans Yet

Even the best AI models still have a long way to go. GPT-6 Astra, which won the benchmark, only managed to win about 7% of its games against the top human-written bots. This shows the gap between current AI coding abilities and what experienced human programmers can produce for complex strategy games.

The benchmark revealed interesting differences in how models approached the task. The strongest performers like GPT-6 Astra, GPT-6.1 Sol and Gemini 4 Argon started playing against difficult bots early on, even while their own creations were still weak. Other models like Muse Wonder Full rarely or never practiced against these tough opponents.

Gemini 4 Argon showed notable inconsistency across runs. Its best and worst attempts differed by 18 points, while the variation was smaller for competing models at just 9 points for GPT-6 Astra and 8 for Claude Opus 5.5.

Cost and Performance Trade-offs

The benchmark also tracked how much each model cost to run. GPT-6.1 Sol stood out as offering the best value, scoring second place while costing just over a tenth of what GPT-6 Astra charged per run. This made it the most cost-effective option for this particular challenge.

Gemini 4 Argon ran at its highest reasoning setting, while every other model operated at maximum effort within their own tools. Each lab used its own command-line interface rather than a shared testing environment.

The results mark an update to the benchmark's first version, which featured older models like Gemini 3.8 Flash. Google and Meta's newer flagships replaced those earlier entries, with Gemini 4 Argon taking third place and Muse Wonder Full landing seventh.

Key facts

Gemini 4 Argon placed third in the StarSkirmish Bench v0.2 with a score between 37 and 39
GPT-6 Astra won the benchmark, followed by GPT-6.1 Sol in second place
Even the best AI models only win about 7% of games against top human-written StarCraft bots
Gemini 4 Argon showed more variation between runs than competing models from OpenAI and Anthropic

GPT-6.1 Sol delivered the best value, earning a second-place score at roughly one-tenth the price of GPT-6 Astra.