Top AI Models Battle in Kaggle Chess Tournament to Test Decision-Making

Some of the top AI models in the world will compete in a three-day chess tournament at Google's Kaggle Game Arena from August 5–7, 2025.The goal is to test how well large language models (LLMs) make decisions and reason under complex conditions. OpenAI's o3 and o4-mini, Google's Gemini 2.5 Pro and Gemini 2.5 Flash, Anthropic's Claude 4 Opus, xAI's Grok 4, DeepSeek's DeepSeek-R1, and Moonshot's Kimi 2-K2-Instruct are among the participating AI models.These models, which are typically used for tasks like coding or writing, are being tested in chess—a game known for its strategic depth and decision-making demands.

The competition follows a single-elimination style, with four games in each match.  In the quarterfinals, o4-mini defeated DeepSeek R1 4-0, Gemini 2.5 Pro defeated Claude 4 Opus 4-0, Grok 4 defeated Gemini 2.5 Flash 4-0, and OpenAI's o3 swept Kimi K2 with a 4-0 triumph.In the semifinals, o3 again had no difficulty defeating o4-mini with a dominant 4-0. Meanwhile, Grok 4 narrowly defeated Gemini 2.5 Pro after a tiebreaker, after they also finished with a match tied at 2.5-2.5. The final match of the tournament will see OpenAI's o3 against xAI's Grok 4, with Gemini 2.5 Pro and o4-mini wrestling for 3rd place.

While traditional chess engines have effectively mastered chess as a systematic process for brute-force computational power, this is not the case for the 8 models participating in this tournament. These LLMs are not chess engines and have not been trained for chess; they are expected to use general reasoning. The model tournament is intended to show how these models navigate structured decision-making and whether they are able to leverage the generalisation capabilities in a constrained set of rules like chess. Kaggle has indicated that it will leave a leaderboard that ranks model performances in a system similar to Elo ratings.

The matches are being reviewed by grandmasters (including Hikaru Nakamura), with live coverage of the tournament on his Twitch and YouTube channels. The final will take place on Thursday 7th August, and will be yet another examination of the evolving strategic capabilities of generalist AI models.