May 15, 2015 weblog
Microsoft Hyperlapse turns long video into distilled entertainment
Many can remember invitations from neighbors to view hours—and one hour could seem like hours—of footage of a family vacation, which would not be at all entertaining. It's still in many cases tedious to view unedited videos. The great thing about professional videos is that they deliver a smooth encapsulation of time that begins with a story intro leading into a "and then what happened" which flows into an ending. Microsoft Research has been working up technology that can do the same for everyone, pro or amateur. It can turn an otherwise tedious video – from a bumpy bicycle ride to a walk along a river—into a short, distilled version. On Thursday, Microsoft announced Microsoft Hyperlapse, which makes first-person videos not only fast but smooth.
Allison Linn at the Microsoft News Center said the technology is designed to glide quickly over times when nothing much is happening, such as stopping the vehicle at red lights or pausing for water. She told how professional photographer Nick Didlick recently took his handful of GoPros to a ski resort in Canada, where he made an action movie using Microsoft Hyperlapse. The result was a time-lapse movie showing two athletes flying down the mountains of British Columbia, first on skis, then on bikes, then leaping into the water.
What is Microsoft Hyperlapse? The name embraces a set of products designed for smooth, stabilized time lapses from first-person videos. There is the Mobile flavor for sharing videos with pals and family. There is Pro for professionals and video enthusiasts, which can create a hyperlapse using a Windows computer from video shot on a camera or other device. Linn announced that the Pro has been placed on free public preview. The third flavor is for developers, called Hyperlapse for Azure Media Services. This is where developers can integrate hyperlapse options into websites and apps. The latter was placed as available in limited free public preview.
The key advance that the research team has made with Microsoft Hyperlapse is for times when the camera is not stationery but in motion. When there is camera motion, the speed-up process may accentuate the motion, and this results in a jumble which is hard to view. The descriptive "smooth" is important here, because the "Hyperlapse" videos indicate camera-motion smoothing, or "stabilization."
Why did Microsoft get busy with Microsoft Hyperlapse? Researcher Johannes Kopf works with the Interactive Visual Media Group at Microsoft. He recalled how, in 2013, he and a team of other researchers in the Computational Photography Group knew they were dealing with a hard problem to solve. A first-person camera may be quite shaky and uneven if the user is walking, running, biking or otherwise in motion. Simply speeding up the video – for example, just showing every 10th frame –amplifies the shakes and jerks. Rather than stitching together various pieces of frames, one of the researchers, Neel Joshi, who works in Microsoft Research's Graphics group, tried another approach. He created technology that looks for entire frames with the most overlap with each other. "Instead of just arbitrarily choosing every 10th frame, for example, the system might pick out the first, 12th, 18th and 29th frame," said Linn. This means the tech can disregard a sudden jerk or jump and go instead for the video's most compatible parts.
Joshi said both pro and consumer versions rely on video stabilization technology which Microsoft researchers had previously developed, further reducing jerkiness often associated with first-person videos.
Overall, the technique, according to Linn, works like this: "The researchers came up with an algorithm that first creates an approximate 3D model of the landscape being filmed, and then identifies the dominant path that the camera took through that landscape. It then stitches together bits and pieces of different frames to create a smooth, stable hyperlapse that showcases the essence of the original video."
Kopf said he would be interested to see "tons of amateurs use this for everyday stuff. I want to see people doing hyperlapses of their work commute."
The Microsoft Hyperlapse page offers ample information on system requirements, supported devices, and SIGGRAPH papers about the technology and downloads.
© 2015 Tech Xplore