Today, video quality matters. Consumers expect the same high quality of experience on smartphones and tablets as they do on TVs. The challenge for service providers is how to deliver superior video quality on all screens in the most bandwidth-efficient way possible, with low latency.
Over the last 20 years, video compression technology has aided service providers in delivering the best video quality at the lowest bitrate. During that time, compression has evolved immensely, boosting performance and, in particular, video quality. This article will examine the broader impact and benefits of Artificial Intelligence (AI) on video compression, explaining possible implementation scenarios for broadcast and OTT applications.
Benefits of AI-Based Video Compression
Software-based video compression solutions are the norm today. While significant efficiencies are enabled in software, machine learning and artificial intelligence are several new technologies that are being used to further optimize video delivery. The primary benefit of using AI and ML for video compression is cost, but there are other advantages, including development savings. Video codecs are complex algorithms that take a significant amount of time to develop and fine-tune. By combining the capability to process a lot of data and the detection capability, AI systems help to make algorithms converge faster. With a human-designed algorithm, it can take months of work to fine-tune an algorithm for a particular need (i.e., face detection or dark scene processing).
ML and AI also improve the density of encoders, as certain ML implementations can take less computation than regular algorithms. Moreover, the use of ML algorithms reduces development time. When codec evolutions are needed, they can be tested quickly, allowing operators to experiment with new business models. This feature is particularly useful with the new deployment models on public or private cloud or SaaS, which also allow video services to be deployed very quickly.
Since the evolution of compression algorithms is faster and less manually driven, customisation is possible. Today, a compression encoder is generic, with some parameters that allow tuning. Usually operators will want to trade-off between no artifacts with a softer picture and a sharp picture with some artifacts. Customisation would allow more detailed tuning, such as processing differently depending on the content: for example, movies versus sports, or even processing various sports differently.
Possible AI/ML Use Cases
Ultimately the industry could greatly benefit from having an AI-based codec implementation that runs in real-time to process live video. This would enable better video quality than a regular (non-AI) algorithm and using fewer computation resources. Unfortunately, that type of implementation is not available yet, as a neural network that would cover all of the decisions in a codec would in fact use more CPU than a regular advanced algorithm. However, there are use cases that can benefit from these technologies.
During the pre-analysis encoding stage, an AI-based algorithm can be used to extract information about the content in order to drive decisions in the subsequent steps in the encoder. For example, ML is particularly good at detection and prediction, which might be applied toward skin tone detection. Another approach is to include AI for only part of the video algorithm. In this case, AI may be used for macroblock decisions (i.e., inter, intra coding). As the encoder constantly measures video quality versus the bitrate, the ML algorithm will try to make improvements.
In satellite use cases, ML compression algorithms have been shown to provide a 20 percent bandwidth savings compared encoding solutions. For OTT applications, Dynamic Resolution Selection, aided by AI, allows operators to improve the QoE because of an improved picture quality. It also enhances the density of the solution, as less profiles need to be used for OTT.
Conclusion
ML and AI technologies are driving better video delivery, improving reliability, latency and bandwidth availability for both broadcast and OTT implementations. In the future, we can expect compression standards to natively include AI in some areas of the framework, which will hopefully reduce compression standards development times.
The subject of AI and ML for video compression will be addressed in more detail in a NAB 2019 technology paper and speaking session. A presentation, ‘AI Technology is Changing the Future of Video Compression,’ will take place during the Broadcast Engineering and Information Technology Conference at NAB on Monday, April 8 from 10:40 to 11:00 a.m. in room N256 of the Las Vegas Convention Center.
