[Stereodude]

Originally Posted by nevcairiel

Its curious that people try to come up with new use-cases while we already had a Neural Network algorithm in madVR before, for image doubling: NNEDI3. Image scaling should be a breeze for the Tensor Cores, considering they are even using that as a gaming feature called DLSS - Deep Learning Super Sampling. Now their algorithm is not going to be quite as high quality as a madVR scaler might be, but it also needs to run on top of 3D rendering and whatnot, so it has to be ultra fast.

A new Neural Network based scaler with more complexity/quality then NNEDI3, but compensating for the low performance of the added complexity by using Tensor Cores sounds like a rather logical step.

madshi posted on AVSforum that NGU sharp is neural network based.

This is actually very near to how madVR's "NGU Sharp" algorithm was designed: It tries to undo/revert a 4K -> 2K downscale in the best possible way. There's zero artificial sharpening going on. The algo is just looking at the 2K downscale and then tries to take a best guess at how the original 4K image might have looked like, by throwing lots and lots of GLOPS on the task. The core part of the whole algo is a neural network (AI) which was carefully trained to "guess" the original 4K image, given only the 2K image. The training of such a neural network works by feeding it with both the downscaled 2K and the original 4K image, and then the training automatically analyzes what the neural network does and how much its output differs from the original 4K image, and then applies small corrections to the neural network to get nearer to the ideal results. This training is done hundreds of thousands of times, over and over again.

Sadly, if a video wasn't actually downscaled from 4K -> 2K, but is actually a native 2K source, the algorithm doesn't produce as good results as otherwise, but it's usually still noticably better than conventional upscaling algorithms.