Topaz Video Enhance AI

[takla]

https://topazlabs.com/downloads/

Thoughts on this neat piece of software?

Personally, I think it outperforms all deblock/deblur/degrain and sharpening tools from avisynth/vapoursynth by a landslide.

Some benchmark:
Artemis High Quality v12 (1080p > 2160p) takes around 0.55 seconds per frame on my AMD RX 5700.

It is rather slow, but for the quality you get it is unmatched.

I also think all models but Artemis are redundant, because not only do they cause some artefacts (which artemis does not), they also are much slower (up to twice as slow). And just in general, scaling higher then 200% is pointtless.

[lansing]

For anime probably yes. But for live action, it'll be a hit or miss, with miss most of the time. And that processing time is just impractical. I'm not wasting a month just to upscale an one hour video, you are crazy.

[Zarxrax]

Haven't tried Video Enhance AI, but VSGAN for vapoursynth is basically the same thing, and there are lots of great user-generated models that can work with it.

[takla]

Originally Posted by lansing

For anime probably yes. But for live action, it'll be a hit or miss, with miss most of the time. And that processing time is just impractical. I'm not wasting a month just to upscale an one hour video, you are crazy.

<0.6 sec per frame is slightly less then 13h for 20 minutes of 1080p to 2160p (sd 540p to 1080p is 0.1sec per frame, very doable). And that speed is with a midrange gpu from 2019. You can expect that time to be cut in half with the next generation of gpus.

Originally Posted by Zarxrax

Haven't tried Video Enhance AI, but VSGAN for vapoursynth is basically the same thing, and there are lots of great user-generated models that can work with it.

esrgan and vsgan are cuda based, which means they only work on nvidia gpus. so not possible for me to test. Edit: in the vsgan thread someone posted it takes around 1.4sec per frame. that is much longer then topaz, but depending on the model might work out much better.

[Zarxrax]

Originally Posted by takla

esrgan and vsgan are cuda based, which means they only work on nvidia gpus. so not possible for me to test. Edit: in the vsgan thread someone posted it takes around 1.4sec per frame. that is much longer then topaz, but depending on the model might work out much better.

Yeah, its highly dependent on the model. Depending on the model I'm using, I have seen differences of more than 5x in the encoding time. Even with the fastest models, its still quite slow.

[kedautinh12]

Waiting for avs port of vsgan

[lollo2]

Personally, I think it outperforms all deblock/deblur/degrain and sharpening tools from avisynth/vapoursynth by a landslide.

Personally, on my s-vhs captures I experimented that it is not any better than a full AviSynth restoration + upscale flow:

https://forum.videohelp.com/threads/...ts#post2620598

https://forum.videohelp.com/threads/...e9#post2626723

I used it just as an upscaler after the excellent avisynth denoise + sharpening, so Topaz VEAI takes advantage of the avisynth restoration; otherwise in my case is not even worth of consideration!

[PatchWorKs]

I repost the question here:

Hi there, I recently found this Upscale "render" that has an interesting workflow:

Flowframes (framerate interpolation): https://nmkd.itch.io/flowframes
Topaz Video AI (resolution upscale): https://www.topazlabs.com/video-enhance-ai
Transition Chooser to clean up scene changes and artifacting: https://www.magentacloud.de/share/lg6gou41fc

Judge result yourself:
https://www.youtube.com/watch?v=iUPVczZ2Xq4

...how to implement it with open source softwares only ?

Hope that helps/inspires.

[kedautinh12]

Originally Posted by lollo2

Personally, on my s-vhs captures I experimented that it is not any better than a full AviSynth restoration + upscale flow:

https://forum.videohelp.com/threads/...ts#post2620598

https://forum.videohelp.com/threads/...e9#post2626723

I used it just as an upscaler after the excellent avisynth denoise + sharpening, so Topaz VEAI takes advantage of the avisynth restoration; otherwise in my case is not even worth of consideration!

Are you sure it's better VSGAN??
https://rlaphoenix.github.io/VSGAN/

[lollo2]

I don’t know yet. Selur addressed me to some AI filters in VapourSynth, but I still have to try (TecoGAN should be the best candidate).

In any case, he seems to indicate that the VapourSynth AI model approach is better than Topaz VEAI (and in my case I am sure it is )

[lollo2]

Originally Posted by PatchWorKs

I repost the question here:

You already had the answer: https://forum.videohelp.com/threads/...11#post2627997

Are you looking for something different?

[Selur]

side note: In theory TecoGan, iSeeBetter (and probably Topaz Video Enhance AI) are mainly interessting since they are designed to look at video and not single images and thus might produce better results since they also look at the adjustend frames and not a single frame.
Some folks first filter with Vapoursynth and then use Topaz Video Enhance AI, TecoGan, iSeeBetter in the hope to get a better results.

That said at least for Topaz Video Enhance AI on it's own I haven't seen anything so far that one couldn't archive or beat with Vapoursynth or Avisynth.

Problem with all the ML/AI stuff is simply the lack or control and ability to fine tune, Topaz tries to lessen this by:
a. providing different models
b. updating their models.
-> All this would be way more interessting if current graphic cards weren't so damn expensive. (and thus at least for me, who isn't earning money with it, it simply is not worth it)

Cu Selur

[lollo2]

That said at least for Topaz Video Enhance AI on it's own I haven't seen anything so far that one couldn't archive or beat with Vapoursynth or Avisynth.

I agree, as everybody who seriously tried both

[takla]

Originally Posted by lollo2

Personally, on my s-vhs captures I experimented that it is not any better than a full AviSynth restoration + upscale flow:

https://forum.videohelp.com/threads/...ts#post2620598

https://forum.videohelp.com/threads/...e9#post2626723

I used it just as an upscaler after the excellent avisynth denoise + sharpening, so Topaz VEAI takes advantage of the avisynth restoration; otherwise in my case is not even worth of consideration!

As I already said, people should use Artemis because it is the fasted and most tuned model. Furthermore, all AI models do denoising, so denoising with avisynth is pointless (will destroy too much fine detail if denoised twice). The same is true for sharpening (as long as your source is not too soft). Both of these things Artemis can do much better.

And personally, I'd save topaz output to prores (and use ffmpeg with more control for final encode)

Oh and for interlaced content I use bwdif because qtgmc makes the image look like a waxpainting.

[takla]

Originally Posted by PatchWorKs

I repost the question here:

Hi there, I recently found this Upscale "render" that has an interesting workflow:


Judge result yourself:
https://www.youtube.com/watch?v=iUPVczZ2Xq4

...how to implement it with open source softwares only ?

Hope that helps/inspires.

Your youtube video is in 24 fps...

[lollo2]

As I already said, people should use Artemis because it is the fasted and most tuned model. Furthermore, all AI models do denoising, so denoising with avisynth is pointless (will destroy too much fine detail if denoised twice). The same is true for sharpening (as long as your source is not too soft). Both of these things Artemis can do much better.

As I already said, in my experiments Topaz VEAI is far inferior to AviSynth processing for denoising and sharpening. I gave it a second chance as an upscaler, it failed as well to improve over my AviSynth flow. In my case Topaz VEAI is useless (I posted some comparison and some restoration work in the posts I linked), but YMMV and, of course, every opinion is respectable ;-)

[kedautinh12]

Originally Posted by takla

Furthermore, all AI models do denoising, so denoising with avisynth is pointless (will destroy too much fine detail if denoised twice).

I think you still don't use KNLMeansCL and BM3D. Two denoiser is very good and don't detroy too much fine detail. It's still low but if you use GPU ver, it will faster

[takla]

Originally Posted by kedautinh12

I think you still don't use KNLMeansCL and BM3D. Two denoiser is very good and don't detroy too much fine detail. It's still low but if you use GPU ver, it will faster

I never got knlmeansCL to work on my gpu, and bm3d uses cuda, so it only works with a nvidia gpu, so I cannot test it. And for the cpu version of nlmeans, quality-to-speed ratio is inferior to what I can achieve with mvtools2 (I posted my denoise script in the mvtools2 thread) still, it is inferior to what I achieve with topaz.

[kedautinh12]

Bm3d have cpu ver. If you have cpu with many core, you will use prefetch() and got fast fps too
https://github.com/WolframRhodium/Va...ment-886173346

[takla]

Here are some comparison between: Raw (Original + Spline36), MDegrain (Spline36 + Adaptive Sharpen), Topaz AI (Artemis High Quality v12)

MDegrain uses Delta=4, thSAD=150, blksize=16 & overlap=8.

https://drive.google.com/drive/folde...j_WGZd6JwxYUoH