The classic clash of punches and kicks in the iconic game Street Fighter has taken a fascinating turn. Typically, players pit their skills against each other or an AI opponent. But in a recent experiment, it wasn’t human fingers flying across the buttons – it was artificial intelligence (AI) in control.
14 different large language models (LLMs) faced off in a whopping 314 Street Fighter III matches! For those new to the tech talk, LLMs are advanced AI systems that can understand and generate human-like text.
Think of them as super-powered chatbots that have analyzed massive amounts of written language.
So how did brainy chatbots become fighting game champions?
Amazon engineer Banjo Obayomi set up the ultimate AI showdown.
Here’s how he did it:
- The AI arena: Obayomi used open-source software called LLM Colosseum and an emulator to run the classic 1997 version of Street Fighter III.
- The contenders: 14 different LLMs stepped into the ring, ready to throw down some code.
- Brain vs. brawn: LLM Colosseum gathered information like a game commentator, keeping track of the LLMs’ health and match status. The LLMs then made strategic decisions on which moves to use, and the software executed them in the game.
Couldn’t you imagine it? Here is the Mistral 7B vs Mistral 7B Street Fighter III match from the GitHub page of the study:
And the winner is…
After the dust – or rather, the pixels – settled, a victor emerged. The language model dubbed Claude 3 Haiku rose to the top, demonstrating its unexpected Street Fighter prowess! It turns out, this AI isn’t just good at generating poetry.
Wondering how an LLM can pull off fighting game combos? Let’s break it down:
- Game knowledge is key: The LLMs likely learned a ton about Street Fighter III moves, techniques, and weaknesses by “reading” strategy guides and discussion forums.
- Practice makes perfect: Like any player, LLMs probably improve through trial and error and by analyzing their matches.
- Mind games matter: Even in a classic fighting game, the ability to anticipate an opponent’s moves and formulate counter-strategies can help secure victory.
Beyond the button-mashing
Sure, seeing AI duke it out in a video game is a blast. But this experiment goes beyond pure entertainment.
The Street Fighter matches show that LLMs aren’t just confined to chatting or writing text formats. They can potentially adapt to a range of tasks, even those that seem totally unrelated to language.
Instead of just answering questions or completing sentences, the LLMs had to apply their abilities in a dynamic, strategic environment. This helps researchers measure AI progress in new ways.
While AI-powered Street Fighter opponents might not hit your game console anytime soon, this experiment is a fascinating glimpse into AI potential. Who knows, maybe one day you’ll be teaming up with, or even facing off against, an AI partner in your favorite games.
And remember, sometimes the real winner isn’t the one on the victory screen, but the technology pushing boundaries behind the scenes.
Featured image credit: Ryan Quintal/Unsplash
