Apple's Keyframer can animate simple drawings using text descriptions

Machine learning researchers at Apple have developed an application that can accept a simple drawing and a text description to animate the drawing in desired ways. Tiffany Tseng, Ruijia Cheng and Jeffrey Nichols have published a paper describing the new app, called Keyframer, on the arXiv preprint server.

AI researchers continue to find LLMs capable of tasks that may not at first appear applicable. In this new effort, the researchers at Apple found that the LLM GPT-4 is capable of accepting both a simple drawing (in SVG format) and a text prompt describing a desired animation for the drawing, and then producing an animation that carries out the instructions.

As an example, the application can accept a picture of a rocket sitting on a launch pad. The accompanying text may read "Make the rocket launch into the sky with a bunch of fire blowing out beneath it." The app will then attempt to animate the rocket to match the prompt.

The researchers explain that the LLM can be used to animate the drawing by first imagining what needs to be done and then generating CSS animation code. As a result, the animation can be ported to other devices and the code can be edited to add or remove additional animation manually—new prompts may be added, as well.

They note that Keyframe also supports iterative animation, where users can continue to add prompts to improve the finished product after each animation is generated—a means for producing a desired animated project in a short period of time.

In their paper, the researchers suggest that Keyframer could set the stage for transforming the animation landscape—if Apple makes it available on all its hardware platforms, for example, users would no longer need other animation applications, and some may use it to create professional products such as commercials. It also allows nonprofessionals to create professional-quality products with a limited amount of effort.