As a field of photography, Slow Motion video is well know for producing some of the most stunning, intriguing and illuminating visuals around. A new take on subjects from the mundane to the extraordinary can be found, all by increasing the frame rate of capture beyond the conventional 30 or 60 fps.
Capturing all video at high frame rates as a matter of course is the dream scenario, but sadly technological limitations prevent that for the time being. Not only is the sensor hardware relatively expensive, storage requirements swiftly balloon well beyond that available from conventional media.
Researchers at NVIDIA have however hit on a novel solution: exploiting the power of deep learning to create interpolated frames, transforming video captured at standard frame rates into slow motion by playing it back at normal speed. A research paper on the subject will be presented this week at the annual Computer Vision and Pattern Recognition (CVPR) conference, but the team have unveiled a short video showcasing the techniques in action.
The examples above show video transformed into slow motion at 1/8th the speed through the use of a neural network. When shown side-by-side with the source video slowed down and played at the same rate the effect is convincing, and easily enough to fool the casual viewer. Only minor tell-tale artefacts of the transformation - reminiscent of motion blur - give the game away, although even these can be mistook for artefacts generated by Youtube's transcoding process.
The neural network was trained on over 11,000 samples of everyday activities captured at 240fps, and validated against a separate dataset. NVIDIA's Tesla V100 GPU hardware accelerated a process which not all that long ago would be computationally untenable.
The technique can also be used to augment video already shot in slow motion, accentuating the effect. NVIDIA teamed with The SlowMo Guys to validate and demonstrate it.
More information, including links to the research paper, can be found at Developer.Nvidia.com, where you can also read more on other AI and Deep Learning innovations NVIDIA are bringing to CVPR this year. One wonders what else deep learning is capable of, and where it will be used in our future day-to-day lives.
SOURCE: NVIDIA Developer News
