VPU, Video Encoding, Decoding and Transcoding

VPU, Video Encoding, Decoding and Transcoding

We are used to video playback and even its encoding being carried out by small coprocessors in SoCs and GPUs in charge of this task. The concept of the Video Processor Unit or VPU goes beyond that. A type of processor designed for larger-scale video processing.

The business of streaming content created by users has been with us for more than a decade, as has the broadcasting of series and movies over the internet. Behind all this there is a specialized hardware that is increasingly in the making, which is the VPU, and which means the return to specialized video hardware.

Encoding, decoding and transcoding

VPU Transcodificación

One of the most important elements is the transmission of video in real time, which means that there has to be the minimum possible time between the transmission of the event and its reception. In other words, there has to be a completely fluid retransmission and this requires a great computational capacity, since the processing speed we need is much higher as it requires coding in much less time.

To this complication, add the enormous number of devices connected to the network, which will demand the content at different resolutions and in different formats. This is something that analog television did not have to deal with at all, since all televisions all operated at the same resolution regardless of their size, so there was no need to generate a multitude of different resolutions.

From the point of view of the reproduction the thing is easier, since they only have to process a video signal at a specific bit rate, a file format and a specific resolution. To this we must add that most of the PCs that people have at home are already equipped to play several video streams at the same time, but encoding is a different issue and is much more complex when it comes to implementing it and in terms of at your computational cost.

A GPU is not suitable for transcoding

Road Map GPUs 2021

Graphic cards for years have had units that encode video, their job is to take the last successive frames of an application and create a video with them in a specific format and then store it or transmit it on the network. The encoding is done because if we had the file with the original information it would require a large amount of data and a very high upload speed.

But the hardware video codec inside the GPU does not have the speed to be able to transmit at different resolutions and in real time, if you look at it, the vast majority of people who stream have a lag time of several seconds and therefore hundreds frames.

So the GPUs, whether dedicated or integrated, are not good enough to be able to transmit content in real time and it is necessary to use a type of specialized hardware that not everyone has, it is at that point where they enter the scene. VPUs as a dedicated processor for video processing.

The need to handle a greater amount of data than the codecs included as standard in SoCs and in GPUs make VPUs the equivalent of a dedicated graphics card compared to simple integrated hardware codecs, so processors are just created specialized we call VPUs.

Architecture of a VPU

CPU NPU VPU Render Road Ma

A VPU like a GPU is a coprocessor for the main CPU, so its job is to free the CPU to do a specific job and to do it in better conditions and in less time than the CPU itself.

Its internal organization is very similar to a GPU, and therefore derived from it, but with some important changes:

  • You can have shader units, but these lack the textures unit and other elements for graphics such as the calculation unit for the intersection of rays in the scene that is used in Ray Tracing.
  • It also does not have the specialized units for graphics outside of the shader units, such as tessellation units, rasterization units, and render outputs.
  • They have a large number of hardware video codecs with the ability to handle multiple video streams at the same time. These codecs occupy a significant part of the chip compared to what they occupy in a normal GPU.
  • Some units use artificial intelligence hardware to execute algorithms such as changing frame resolution, style, interpolation, and extrapolation.

Compared to a GPU, its capabilities in transforming data from one format to another and from one resolution to another are much higher in terms of speed. So its encoders are much more powerful, in part because they achieve a high encoding, decoding and transcoding speed not only thanks to the hardware codecs, but also to the computing power of the units around them.

Where are the VPUs located?

Trust Iris VPU

So in general, a VPU is not a type of hardware that is usually found as standard in most computers, since very few people need video processing hardware with that level of power for their daily use.

VPUs can generally be found in the following types of hardware.

  • On the servers of video streaming platforms, whatever the type of content that is transmitted through them to users.
  • In specialized hardware used for transcoding, these are typically video player-like units with a series of video inputs and outputs. Some veteran television networks use this type of equipment to broadcast their old and analog-age content directly to digital.
  • In videoconferencing systems, where the arrival of teleworking has created the need to be able to carry out clear and fluid communication between the different parties.
  • In computer vision systems so that the system has eyes and can “see” what is around, not only in robots or smart cars, but also in smartphones and tablets.

So VPUs are always going to be seen when a much higher video processing is required than that provided by a conventional PC and little by little they have permeated in different markets given the importance of getting video streaming for several different clients that play low. different conditions.