Microsoft recently launched Muse, a generative AI model for gameplay ideation. Built on the World and Human Action Model (WHAM), the model can generate game visuals, controller actions, or both.
In a recent interview with Dwarkesh Patel, Microsoft chief Satya Nadella expressed his excitement about the future of gaming. He revealed that Microsoft will soon have a catalogue of games wherein AI models will play a key role.
He explained that these models will either be trained to generate game content or will be directly integrated into gameplay. “We’re going to have a catalogue of games soon that we will start using these models for, or we’re going to train these models to generate and then start playing them,” he said.
When Xbox chief Phil Spencer first demoed Muse for him, Nadella saw the model take inputs from an Xbox controller and generate outputs that perfectly matched the game. “It was a massive moment of ‘wow’. It’s kind of like the first time we saw ChatGPT complete sentences, or DALL-E draw, or Sora (sic).”
Nadella shared an interesting fact about Microsoft’s history, mentioning that the company developed a game before even creating Windows.
“Flight Simulator was a Microsoft product long before we even built Windows. Gaming has a long history at the company, and we want to be in gaming for gaming’s sake,” he said.
Nadella described gaming data as more than just a resource for the gaming industry, suggesting it could serve as both a general action model and a world model. “It’s fantastic,” he said, drawing a comparison: “I think about gaming data as perhaps, you know, what YouTube is to Google, gaming data is to Microsoft.”
Google uses YouTube data to train its models, such as its video generation model Veo 2.
On the other hand, Muse was trained on human gameplay data from Bleeding Edge, a 4v4 online game by Ninja Theory. The dataset includes visuals and controller actions recorded with user consent. The model has been trained on over 1 billion images and actions, representing over seven years of continuous gameplay.
AI x Gaming
On October 13, 2023, Microsoft acquired Activision Blizzard for $75.4 billion. The acquisition included franchises such as Call of Duty, Warcraft, Diablo, Overwatch, and Candy Crush. In the recent quarter, Xbox content and services reported a 2% increase in revenue.
Speaking about Microsoft’s substantial investment in gaming, Nadella said cloud gaming is a natural area to invest in as it expands the TAM and allows people to play games everywhere. He added that the combination of AI and gaming could be the ‘CGI moment’ for the gaming industry.
Reportedly, Microsoft is not the only one betting on AI gaming. Elon Musk’s xAI has similar plans. While introducing xAI’s latest model, Grok-3, Musk said, “We’re launching an AI gaming studio at xAI. If you’re interested in joining us and building AI games, please join xAI.”
Since the launch of Grok-3, many developers have used it to create games.
“Grok-3 is an incredible AI coding assistant. After a few hours and over 1,000 lines of generated code, I now have a fully functional 2D vertical jumping game,” said Alvaro Cintas-Canto, assistant professor of cybersecurity at Marymount University. He added that the game features different heroes, monsters, platforms, difficulty levels, and lives.
“Grok 3 is so good at programming that it makes game creation feel more like an art project. I generated this themed, endless-runner arcade game in < 30 mins,” said Mickey Friedman, co-founder of Flair.ai.
Besides xAI, Google DeepMind last year introduced Genie 2, a large-scale foundation world model capable of generating diverse playable 3D environments.
It enables the development of embodied AI agents by transforming a single image into interactive virtual worlds, which can be explored by humans or AI using standard keyboard and mouse controls. “The world model is taking shape,” said Google DeepMind chief Demis Hassabis while announcing it.
Building on this trend, Decart AI launched Oasis, the world’s first real-time, generative AI-based playable world model. This fully interactive game generates each frame through a Transformer model that responds instantly to keyboard and mouse inputs, simulating physics, game mechanics, and graphics.
Meanwhile, Netflix recently appointed Mike Verdu as the VP of GenAI for games. “I am working on driving a once-in-a-generation inflexion point for game development and player experiences using generative AI. This transformational technology will accelerate the velocity of development and unlock truly novel game experiences that will surprise, delight, and inspire players,” said Verdu in a LinkedIn post.
