Waifu2x-caffe

[HolyWu]

https://github.com/HomeOfVapourSynth...-Waifu2x-caffe

Another implementation of waifu2x, based on the waifu2x-caffe library. It can only run on NVIDIA GPU.

[HolyWu]

Update r6.

• Update waifu2x-caffe library to 1.1.6.
• Add noise reduction level 0 and make it the default level.

[HolyWu]

Update r7.

• Update upconv_7 models.
• Change the default model to upconv_7_anime_style_art_rgb.
• Update waifu2x-caffe library to 1.1.7.1.

[brucethemoose]

Out of curiosity, why does Waifu2x caffe support non-integer scaling when this version doesn't? 2.25x scaling would be perfect for 480p -> 1080p.

EDIT: Also, great filter BTW. This is by far the best animation scaling I've seen.

[HolyWu]

Originally Posted by brucethemoose

Out of curiosity, why does Waifu2x caffe support non-integer scaling when this version doesn't? 2.25x scaling would be perfect for 480p -> 1080p.

Internally the image is always enlarged by a power of 2. So even you specify a scale ratio of 2.25x, the program still enlarges the image to 4x first. It then downscales the 4x image to the final resolution afterwards. I simply don't want to hardcode the downscaling part in my filter. It's up to the users to decide which resizer to use for downscaling.

[HolyWu]

Update r8.

• Update waifu2x-caffe library to 1.1.8.3.
• Now uses cuDNN 6.0.
• Add configure script. It's compilable on Linux now. Thanks Are_ for testing.

[stax76]

I try to replicate an exception posted to the staxrip issue tracker, the user is getting E_ACCESSDENIED after staxrip calls TaskDialog to show an error returned from VapourSynth, I get following error:

Python exception: Waifu2x-caffe: failed open model file at initialization

Where can I find this model file?

import vapoursynth as vs
core = vs.get_core()
core.std.LoadPlugin(r"D:\Projekte\VS\VB\StaxRip\bin\Apps\Plugins\vs\fmtconv\fmtconv.dll")
core.std.LoadPlugin(r"D:\Projekte\VS\VB\StaxRip\bin\Apps\Plugins\vs\vslsmashsource\vslsmashsource.dll")
clip = core.lsmas.LWLibavSource(r"D:\Temp\staxrip\ash.mkv")
core.std.LoadPlugin(path=r'D:\Temp\Waifu2x-caffe-r8\Win64\Waifu2x-caffe.dll')
clip = core.fmtc.bitdepth(clip,bits=32)
clip = core.caffe.Waifu2x(clip, noise=1, scale=2, block_w=512, block_h=512, model=3, cudnn=True, processor=0, tta=False)
clip.set_output()

https://github.com/stax76/staxrip/issues/190

[HolyWu]

Originally Posted by stax76

Python exception: Waifu2x-caffe: failed open model file at initialization

Where can I find this model file?

They are at the GitHub page as well, just in previous release.

[stax76]

Thanks, got it working now.

[poisondeathray]

Thanks HolyWu,

I tested on some low to midrange cards, and this seems much faster than the vpy Waifu2x-w2xc implementation. Playing with the blocksize can make a big difference in the speed

Offhand do you know of any big differences quality wise ? or would there be any reason NOT to switch to using the caffe version if you had a compatible GPU ?

[HolyWu]

Originally Posted by poisondeathray

Offhand do you know of any big differences quality wise ? or would there be any reason NOT to switch to using the caffe version if you had a compatible GPU ?

The difference in quality is quite minor. Actually the models 0~2 in caffe version are the same as w2xc version. The major difference is that caffe version has upconv_7 models and cuDNN support while w2xc version doesn't. Both of them primarily concern speed. There is really no reason to use w2xc version when you have a compatible NVIDIA GPU though.

[lansing]

What is the recommended card for this? I'm using the default setting on a clip scaled to 1440x1020 after the filter, and I'm getting 0.66fps...

[HolyWu]

Originally Posted by lansing

What is the recommended card for this? I'm using the default setting on a clip scaled to 1440x1020 after the filter, and I'm getting 0.66fps...

There is no officially recommended card. The only requirement is a NVIDIA GPU of compute capability 3.0 or higher.

I guess the color family of your clip is YUV? You will get slightly faster speed if you convert to RGB first before invoking the filter. Furthermore, tweak block_w/block_h setting so that it can divide the width/height of the clip with no remainder. Larger value isn't necessarily better because it has to do with the card's capability and/or memory size. You have to do experiments with some different values to find out the optimal pair for the specific resolution.

[lansing]

Originally Posted by HolyWu

Furthermore, tweak block_w/block_h setting so that it can divide the width/height of the clip with no remainder. Larger value isn't necessarily better because it has to do with the card's capability and/or memory size. You have to do experiments with some different values to find out the optimal pair for the specific resolution.

Do you mean the width/height of the video before the filter or after?

[poisondeathray]

Originally Posted by lansing

Do you mean the width/height of the video before the filter or after?

no, the block_w , block_h are filter arguments

https://github.com/HomeOfVapourSynth...-Waifu2x-caffe

block_w: The horizontal block size for dividing the image during processing. Smaller value results in lower VRAM usage, while larger value may not necessarily give faster speed. The optimal value may vary according to different graphics card and image size.

block_h: The same as block_w but for vertical.

It can make a big difference in speed , play with different values

Don't complain, this is much faster than the waifu w2xc implementation . And that was much much faster than the avisynth waifu2x implementation

[HolyWu]

I guess lansing was asking whether he should divide the image of the original size or the upscaled size. The answer is the original size though.

[HolyWu]

Update r10.

• Update waifu2x-caffe library to 1.1.9.2.
• Add UpResNet10 model.
• Add batch parameter.
• Now uses CUDA Toolkit v10.0.130 and cuDNN v7.3.1.

[amayra]

so this work only in windows ?

[Blovesx]

Okay, where can I find any explanation what's UpResNet10? How much is it different from other models?

What's batch option for?

I'm using GUI version for Windows. How can I encode video using this program? You posted the link for vapoursynth (whatever it is) but I don't get it how to compile it. Can't you upload it as you do with caffe-gui releases?

[WolframRhodium]

Originally Posted by Blovesx

Okay, where can I find any explanation what's UpResNet10? How much is it different from other models?

Personally, UpResNet10 combines the advantages of residual block and squeeze-and-excitation block (the former one eases the training while the later one boosts the representation power):
The network is made up of 5 "residual SE blocks" with global skip connection. Each residual SE block includes 2 convolution layers (5 * 2 =10, which gives the name UpResNet10) and a channel attention mechanism.
Previous waifu2x models do not make use of any of these techniques.

Similar structure can be found in recent state-of-the-art super-resolution models like EDSR, RCAN, etc.