Meta debuted a suite of artificial intelligence models called “Movie Gen” on Oct. 4 capable of generating photorealistic movies up to 16-seconds long complete with sound effects and backing music tracks.
Movie Gen isn’t the first multimodal AI model capable of generating video and audio from simple text prompts, but it appears to demonstrate state-of-the-art capabilities. The researchers responsible for the application’s development claim it outperformed rival systems in human testing.
A still image taken from a movie generated by Meta Movie Gen. Source: Meta
Movie Gen
According to a blog post from Meta, Movie Gen is currently capable of outputting movies up to 16-seconds long at a frame rate of 16 frames per second (FPS). To put this into perspective, Hollywood films prior to the digital age were traditionally shot at 24 FPS to achieve what’s called the “film look.”
While higher FPS rates are considered better in gaming and other graphical applications, Meta’s 16 FPS isn’t far off from what would be considered professional quality movie imagery.
The Movie Gen models can generate completely novel movies based on simple text prompts or modify existing images or videos to replace or modify objects and backgrounds.
Still images from an uploaded movie edited three different ways. Source: Meta
Its most advanced contribution, however, may be the AI suite’s ability to generate up to 45-seconds of audio featuring sound effects and background music. According to Meta, Movie Gen integrates and syncs the audio to the motion in the generated videos.
Related: Meta shows off Web3-to-reality bridge with ‘Hyperscape’ metaverse demo
Research only
Meta is keeping the foundation models behind Movie Gen under wraps for the time being. The company hasn’t given a timeframe for the product’s launch and says it will require further safety testing before deployment.
Per a research paper from Meta’s AI team:
“The Movie Gen cast of foundation models were developed for research purposes and need multiple improvements before deploying them … when we do deploy these models, we will incorporate safety models that can reject input prompts or generations that violate our policies to prevent misuse.”
Magazine: Anti-aging tycoon Bryan Johnson almost devoted his life to crypto