Adobe’s latest AI prototype can make still images dance

Adobe will reveal a prototype AI project later today at Adobe Max 2022 that can convert a still image of a person into an animated dancer. Adobe says that all you need to do is load a full-body picture into Project Motion Mix, and the system will turn that individual into an AI-controlled puppet, animating new dance moves. 

The system uses a combination of AI-based motion generation and what Adobe is calling “human rendering technologies” to create its animations. The software lets users select from different dance styles, tweak the background, and add multiple dancers into one frame. However, it’s still just a prototype, and Adobe says it isn’t sure if or when the system might be added to its user-facing services.

Motion Mix is an interesting project but not necessarily a new idea. Researchers have been using machine learning to create custom dances from still photographs since at least 2018, and the output of Project Motion Mix is not yet photorealistic. As with AI-powered image generators, the system seems to struggle with creating hands and limbs that aren’t blurry and misshapen.

It’s a fun feature, though — reminiscent of those customizable JibJab holiday cards from years past. And it’s also easy to see how the tech could be scaled up: helping content creators easily generate human animations. If Adobe were able to offer a large library of realistic movements for its model, it could help quickly generate content without the need for time-consuming animation or motion capture. 

Adobe says it can imagine similar tech being used on social media, generating TikTok videos for example, and could eventually make its way into more advanced projects.