As podcaster Ryan Zhao put it on Bluesky, “The design course of has gone mistaken when what you should prototype is ‘what if there was an area.'”
Gotta go quick
When Google revealed the primary model of Genie earlier this 12 months, it additionally revealed an in depth analysis paper outlining the precise steps taken behind the scenes to coach the mannequin and the way that mannequin generated interactive movies. No such analysis paper has been revealed detailing Genie 2’s course of, leaving us guessing at some necessary particulars.
One of a very powerful of those particulars is mannequin pace. The first Genie mannequin generated its world at roughly one body per second, a fee that was orders of magnitude slower than can be tolerably playable in actual time. For Genie 2, Google solely says that “the samples on this weblog submit are generated by an undistilled base mannequin, to indicate what is feasible. We can play a distilled model in real-time with a discount in high quality of the outputs.”
Reading between the strains, it appears like the complete model of Genie 2 operates at one thing effectively beneath the real-time interactions implied by these flashy GIFs. It’s unclear how a lot “discount in high quality” is important to get a diluted model of the mannequin to real-time controls, however given the dearth of examples introduced by Google, we’ve to imagine that discount is critical.
Credit:
Oasis
Real-time, interactive AI video technology is not precisely a pipe dream. Earlier this 12 months, AI mannequin maker Decart and {hardware} maker Etched revealed the Oasis mannequin, displaying off a human-controllable, AI-generated video clone of Minecraft that runs at a full 20 frames per second. However, that 500 million parameter mannequin was educated on hundreds of thousands of hours of footage of a single, comparatively easy recreation, and centered completely on the restricted set of actions and environmental designs inherent to that recreation.
When Oasis launched, its creators totally admitted the mannequin “struggles with area generalization,” displaying how “sensible” beginning scenes needed to be lowered to simplistic Minecraft blocks to realize good outcomes. And even with these limitations, it is not arduous to seek out footage of Oasis degenerating into horrifying nightmare gas after just some minutes of play.
