Surely you wonder how it is possible that Artificial Intelligence helps developers to create better games, better images and ultimately, to make better graphics with more realism. The answer, if not all, is largely a very little known technology that comes from the hand of NVIDIA: NGX , which is one of the holy grails for the future of AI when working with GPUs.
The most basic question if we do not have much idea about any of these topics could be, how does a GPU work with Artificial Intelligence? Everything is much simpler, but at the same time somewhat complicated to explain if we take into account the RTX architectures of NVIDIA graphics cards, especially a specific unit for this purpose, the Tensor Cores.
Deep learning shapes NVIDIA NGX AI
As we surely already know, Tensor Cores units are used by NVIDIA in video games for Deep Learning Super Sampling, since it needs the inference of the AI to super-sample the pixels.
But this view of the Tensor Cores is incomplete, since we are only seeing what happens from the player’s side and not from the developer’s side, which creates those images that are then scaled. This is where NVIDIA NGX comes in and where it is based on its use to bring more lifelike images to life, as it is nothing more than a technology stack powered by Deep Learning that equips developers with AI functions to accelerate or improve graphics, images and even video processing in real time.
As we have already mentioned, in order to function it needs a specific unit that works from below and this is where the Tensor Cores of the RTX GPUs come in. The NVIDIA NGX SDK provides developers with a number of Artificial Intelligence functions at their disposal through networks that have been trained on the NVIDIA Saturn V supercomputer.
What can this technology do and how does it do it?
We could say that broadly there are 3 scenarios that encompass the rest: images, videos and games. For each of them there are different functions that developers can use for multiple purposes, since we are talking about a complete and very powerful SDK.
To include some examples, in an image we can delete content from it and create it through AI, which will perfectly replace the content as the neural network (InPainting) has been trained.
We can insert interpolated frames in a video, give life to any liquid in a series of images that did not contain it and at two different speeds, since the AI identifies the movement of the camera and allows us to add those FPS on the screen that did not exist.
In low resolution or faulty images, you can scale up to 8x the original quality by stretching existing pixels and filtering them between them thanks to AI.
It creates new pixels based on your training and places them where they are needed, improving image quality as well as sharpness. In games it is even more amazing how it works, since the function is included in the NVIDIA driver and as we have already guessed, it is used for DLSS through previous training on Saturn V.
Its operation is simple, since when the driver detects the call of the function it gives full access to NGX to make use of the hardware and rescale the frames in real time using DLSS.
