Intelligent Ultra High Definition Video Encoder Optimization for Future Versatile Video Coding
Recently, there has been an increasing demand of the ultra-high-definition (UHD) video services. The UHD video, which provides a realistic viewing experience with high volume of data, is revolutionizing the video ecosystem and driving the upgrade of the information industry. In this context, the contradiction between the exponential growth of ultra-high-definition video big data and limited storage space as well as transmission bandwidth has become increasingly prominent. In view of these challenges, the joint video experts team from ITU-T and MPEG has launched the project of next generation video coding standard-versatile video coding (VVC).
In VVC, new coding structures and a set of coding tools are adopted, which significantly boost the coding performance compared to the previous HEVC standard. Along with the standardization process, it is urgent and important to study the encoder optimization algorithms, which not only facilitate the implementation and deployment of the VVC standard in the future, but also provide useful information and guidance for the standardization process.
In this project, we will systematically study the VVC encoder optimization from the perspectives of high efficiency rate control and low complexity encoding by leveraging the artificial intelligence technologies. In particular, we first study the rate-distortion characteristics of UHD videos in VVC to regularize the coding bit rate. More specifically, the rate-quantization and distortion-quantization relationships are investigated with the new coding structure that incorporates both quadtree-binary tree as well as triple tree partitioning, and the intelligent rate control is achieved based on the recent development of reinforcement learning. As such, the number of coding bits for each frame and CTU is optimized to achieve high efficiency rate control.
Furthermore, we extend the rate-distortion modeling to the joint rate-distortion-complexity (RDC) optimization of VVC codec, aiming to achieve a good trade-off between rate-distortion performance and encoding complexity for low complexity encoding. In particular, by modelling the relationship between them with the advanced machine learning techniques, the encoder structure can be determined in the complexity constrained scenario. Furthermore, the decision of coding modes can be further optimized with the complexity allocation strategy. By intelligently modelling the local RDC relationship, the complexity allocation is optimized for each coding unit to achieve the overall optimal coding performance.
The research results are expected to be adopted by a diverse set of applications based on UHD video, including video surveillance, digital TV broadcasting, IPTV, video communication, and Internet video.