ESRGAN - AI upscaler

[Terka]

can something like this be implemented to avisynth?
https://github.com/xinntao/Real-ESRGAN/blob/master/assets/teaser.jpg

[Reel.Deel]

There's Real-ESRGAN-ncnn-vulkan (https://github.com/xinntao/Real-ESRGAN-ncnn-vulkan) which uses a similar framework as the RIFE (https://github.com/Asd-g/AviSynthPlus-RIFE) and w2xncnnvk (https://github.com/Asd-g/AviSynthPlus-w2xncnnvk) plugins so there's hope.

[kedautinh12]

https://github.com/Kiyamou?tab=repositories
https://github.com/rlaphoenix/VSGAN

[kedautinh12]

Hope one of there resizer plugins will port to vapoursynth
One can do it, you can ask him here:
https://github.com/Asd-g/AviSynthPlus-Scripts/issues

[Dogway]

For video the state of the art is BasicVSR++ (https://arxiv.org/pdf/2111.12704v1.pdf), already in vapoursynth (https://github.com/HolyWu/vs-basicvsrpp), it's even better than BasicVSR, you can see BasicVSR vs Real-ESRGAN here (https://arxiv.org/pdf/2111.12704v1.pdf).

[Julek]

For VS there are several plugins, the most complete being MLRT (https://github.com/AmusementClub/vs-mlrt), with compatibility for CUDA/Vulkan and pure CPU.

[kedautinh12]

https://github.com/HolyWu?tab=repositories

[tormento]

How I wish some porting to AVS+.

[takla]

Why are all good models cuda only...

[kedautinh12]

Why are all good models cuda only...

No, some plugins only need Vulkan

[takla]

No, some plugins only need Vulkan

And those are all mediocre (with the exception of rife for interpolation)

[kedautinh12]

mediocre?? Your Real-ESRGAN sometime don't perfect like example. Each plugin can use for some cases. You can't do for all case with only one plugin. Please remember. If you think you can develop one plugin can do all for it, you can complain others is mediocre

[takla]

mediocre?? Your Real-ESRGAN sometime don't perfect like example. Each plugin can use for some cases. You can't do for all case with only one plugin. Please remember. If you think you can develop one plugin can do all for it, you can complain others is mediocre

Yes mediocre. All the models Real-ESRGAN ncnn Vulkan comes with are very mediocre.

Also, esrgan models can very much be the only "plugin" you need to be applied to a given source. I've seen some amazing models before, but again, only for cuda.

[real.finder]

I really hope the nnedi4 and nnsharpen https://forum.doom9.org/showthread.php?p=1577586#post1577586 was done and tritical still do his great work which is not resources hungry as these AI upscaler, maybe his nnedi4 or even nnedi5 will beat those python toys, what I really like about nnedi family it only upscale and do not change colors etc, and with nnsharpen the upscales of nnedi will be better

anyway, BasicVSR++ output seems ok, but I think it model are something like 1 GB? Not to mention the requirements for Python and rgb24 only and very fast gpu preferably nvidia for less than 1 fps encode speed

[kedautinh12]

I really hope the nnedi4 and nnsharpen https://forum.doom9.org/showthread.php?p=1577586#post1577586 was done and tritical still do his great work which is not resources hungry as these AI upscaler, maybe his nnedi4 or even nnedi5 will beat those python toys, what I really like about nnedi family it only upscale and do not change colors etc, and with nnsharpen the upscales of nnedi will be better

anyway, BasicVSR++ output seems ok, but I think it model are something like 1 GB? Not to mention the requirements for Python and rgb24 only and very fast gpu preferably nvidia for less than 1 fps encode speed

But, link of him died and i don't know he still work or not

[Dogway]

I really hope the nnedi4 and nnsharpen https://forum.doom9.org/showthread.php?p=1577586#post1577586 was done and tritical still do his great work which is not resources hungry as these AI upscaler, maybe his nnedi4 or even nnedi5 will beat those python toys, what I really like about nnedi family it only upscale and do not change colors etc, and with nnsharpen the upscales of nnedi will be better

There's RAVU (Rapid and Accurate Video Upscaling) also, it's based on RAISR (Rapid and Accurate Image Super Resolution), so it's on the fast side.
I checked and seems to be a little bit better than nnedi3, like less aliasing.
You can run it with:
libplacebo_Shader("ravu-r4.hook",width()*2,height()*2,filter="lanczos",radius=4)

Comparison (check offline with nearest scaling for comparison):

NNEDI3CL (blackman6 pixel shifted).........RAVU4
http://i.imgur.com/Tz4Wp9Bm.png (https://i.imgur.com/Tz4Wp9B.png)...http://i.imgur.com/oG8IOyhm.png (https://i.imgur.com/oG8IOyh.png)

[real.finder]

There's RAVU (Rapid and Accurate Video Upscaling) also, it's based on RAISR (Rapid and Accurate Image Super Resolution), so it's on the fast side.
I checked and seems to be a little bit better than nnedi3, like less aliasing.
You can run it with:
libplacebo_Shader("ravu-r4.hook",width()*2,height()*2,filter="lanczos",radius=4)

Comparison (check offline with nearest scaling for comparison):

NNEDI3CL (blackman6 pixel shifted).........RAVU4
http://i.imgur.com/Tz4Wp9Bm.png (https://i.imgur.com/Tz4Wp9B.png)...http://i.imgur.com/oG8IOyhm.png (https://i.imgur.com/oG8IOyh.png)

I note some aliasing artifacts in https://i.imgur.com/oG8IOyh.png in around the Chevrolet logo and the lights, also there are some blur/details lost in dark around Chevrolet logo, it only better in the manhole (sharper)

https://i.imgur.com/Tz4Wp9B.png win, which I think it's the nnedi3?