Fraunhofer HHI presents MPEG OMAF viewport-dependent VR video streaming using HEVC tiles

Fraunhofer HHI just showed a new media profile for delivering Virtual Reality content at IBC 2017. MPEG’s ongoing work on Omnidirectional Media Application Format (OMAF) aims at standardizing storage and delivery formats for VR video content by the end of this year. One of the most promising techniques for streaming the 360-degree VR video considered in OMAF is viewport-dependent streaming based on HEVC tiles.

---
In a conventional service design, the whole 360-degree VR video content is streamed at a single resolution. Such an approach sacrifices video quality in the actual user viewport for pixels that are not even presented to the user most of the time. Fraunhofer HHI’s implementation of the OMAF viewport-dependent streaming profile is based on HEVC tiles and overcomes the challenges of ultra-high resolution content in context of limited bandwidth and decoder capabilities found on mobile devices.

High resolution, low storage requirements

“We divide the entire 360-degree video into tiles that are encoded independently of each other – that is why we call our technology tile-based streaming,” explains Robert Skupin, scientist at Fraunhofer HHI. “Within the user’s field of vision, the image is high resolution; behind him, it is low.” The major difference when compared to the state-of-the-art method is that the end device decides which tiles to download in high resolution and which in low resolution. It puts the tiles it needs together to form the required image in the right resolution instead of downloading one of several prepared videos. This means that, unlike the existing videos, each tile only needs to be stored on the server once. “The resolution is high, the operating costs are low, and, what’s more, the technology opens up several degrees of freedom in various areas such as dynamic adaptation of the transmission rates,” says Skupin.

In order to create virtual worlds in high definition using the tile-based streaming technology, one needs – on the one hand – a suitable video codec and – on the other – an efficient transmission standard. As far as the video codec goes, it already exists in the form of HEVC. “HEVC is the first video standard where all end devices support a tile-based approach and are therefore already prepared for tile-based streaming,” Skupin can advise. The necessary transmission standard, called “Omnidirectional Media Application Format” (OMAF), is currently being developed by the MPEG standardization committee; it should be ready by the end of 2017. The Fraunhofer HHI researchers are also involved: They are expanding the standard, so that it can allow tile-based streaming, in order to optimally supply current and future devices with high-definition video.

The researches from Fraunhofer HHI are also contributing the HEVC-based technology to the VR Industry Forum (VRIF), which was formed to further widespread availability of high quality audiovisual VR experiences. At IBC 2017 VRIF presented the first draft of its industry guidelines. The guidelines enable delivery of high quality VR experiences, emphasizing interoperability in an open ecosystem. Part of this guideline is the OMAF viewport-dependent VR video streaming using HEVC tiles. Fraunhofer HHI’s implementation of the MPEG OMAF viewport-dependent profile using HEVC tiles was presented at IBC.

Weitere Informationen:
- https://www.hhi.fraunhofer.de/en/press-media/press-releases.html

Quelle: idw