ChatGPT faced a significant challenge when it was pitted against the Atari 2600 in a chess match, resulting in a disappointing performance. This experiment revealed that, despite advancements in artificial intelligence, ChatGPT struggled to understand the game, leading to what observers described as it getting “absolutely wrecked” by the vintage console. Even with the Atari 2600 nearing its 50th anniversary, ChatGPT failed to navigate the chessboard effectively, barely making it through the match before ultimately forfeiting after 90 minutes of confusion. The difficulties of AI language models (LLMs) like ChatGPT extend beyond chess.
Users have experienced various issues, particularly with the phenomenon known as “hallucinations,” where AI generates incorrect information. In one instance, an apology for the game Lord of the Rings: Gollum was reportedly generated by ChatGPT without the consent of the development team, highlighting potential misuse of the technology. Robert Jr. Caruso, the engineer who conducted the chess match experiment, noted that ChatGPT itself suggested the challenge to display its skills. However, the AI frequently became confused about the placement of pieces and made questionable decisions, such as sacrificing knights for pawns.
Even after switching to standard chess notation, the AI continued to blunder, demonstrating significant limitations in its gameplay understanding. Similar struggles have been observed in other AI models. For example, a user attempted to engage OpenAI’s o3 model in Pokemon Red. While it has shown some progress, the AI’s speed in decision-making has proven to be much slower than that of human players, including young children.
After over 366 hours of gameplay, the AI still had not reached Victory Road. On a more positive note, some AI, such as Google’s Gemini, have fared better in gaming contexts, reportedly completing Pokemon Blue in 800 hours. This suggests that while some AIs face hurdles in gaming, others may be capable of performing better in these environments.