waifu2x for AviSynth 2.6

[Reel.Deel]

Just found out someone ported waifu2x to AviSynth: https://github.com/sunnyone/Waifu2xAvisynth

Just spreading the news just in case someone is not already aware. :)

[Sparktank]

Fascinating.

I saw the other thread and was hoping somoene might try to port it.
Thanks for the info.

[foxyshadis]

Laziness wins again! (I was going to attempt this.) There's a website, too: http://waifu2x-avisynth.sunnyone.org/

Translation of usage:
waifu2x(clip, int nr=0, int scale=2, string models=".", int jobs=0)

nr: 0, 1, or 2 for no/light/more noise reduction.
scale: A power of 2. 1 disables scaling (noise reduction only)
models: Location of models, defaults to plugin folder.
jobs: Threads, 0=# of cores in system.

[Reel.Deel]

It seems not all of the parameters are in order, here's the correct order (https://github.com/sunnyone/Waifu2xAvisynth/blob/master/Waifu2xAvisynth/waifu2xAvisynth.cpp#L254): waifu2x(clip, nr, scale, jobs, models)

Also only YV12, YV16, and YV24 are supported. I wonder why not Y8? Y8 is useful for processing RGB as individual planes. Sure one can use YV12 but it can be less efficient. I might ask the author if he can add support for it.

One question for those of you who can read code, I see that unlike the original that only scales up to 2x, this implementation can scale 4x, 6x, 8x...etc. How is that done? Is it done in one shot or is waifu2x called multiple times?

[foxyshadis]

It seems not all of the parameters are in order, here's the correct order (https://github.com/sunnyone/Waifu2xAvisynth/blob/master/Waifu2xAvisynth/waifu2xAvisynth.cpp#L254): waifu2x(clip, nr, scale, jobs, models)

Also only YV12, YV16, and YV24 are supported. I wonder why not Y8? Y8 is useful for processing RGB as individual planes. Sure one can use YV12 but it can be less efficient. I might ask the author if he can add support for it.

One question for those of you who can read code, I see that unlike the original that only scales up to 2x, this implementation can scale 4x, 6x, 8x...etc. How is that done? Is it done in one shot or is waifu2x called multiple times?

Good catch. It just runs the 2x filter multiple times, so 2, 4, 8, 16, etc. It could support 16-bit as well, the meat of the processing is done in (very unoptimized) 32-bit float.

[Reel.Deel]

Thanks for the info foxyshadis. I overlooked that scale has be a power of 2 not a multiple of 2. :o

I wonder how well it will perform with video? A good point was brought up in #avs-plus.
a lot of "smart" upscalers end up with awful shimmering due to not quite doing the same thing on each frame
Hopefully that's not the case here, otherwise it'll only be usable on single images for the time being.

(not at home so I haven't had a chance to try it out)

[kuchikirukia]

I wonder how well it will perform with video? A good point was brought up in #avs-plus.

Hopefully that's not the case here, otherwise it'll only be usable on single images for the time being.

From what I've seen it oil paintings realistic scenes. So it's likely only for still anime.

[Reel.Deel]

From what I've seen it oil paintings realistic scenes. So it's likely only for still anime.

It really only turns things into oil paintings when noise reduction is set to 2 (maybe even 1 for some sources) but that's not what I was referring to. waifu2x works well on single images but when you process video it might create shimmering artifacts due to interpolation differences between frames. Shimmering cannot be seen on a single frame but if you flip back between two frames sometimes you can seen lines and other details slightly jumping up and down (bobbing). Anyways I don't know if this is the case with waifu2x, I wanted to try and see but it's painfully slow (even slower than the online version) so I gave up.

[SEt]

My waifu can't have so strong halos! I think she needs some more training... Also she is quite hungry now – stand alone version ate my 12 GB of RAM like nothing and crashed on moderately sized image.

Noise reduction is nice though – does good job on re-compressed images, but can't cure quantized to zero chroma.

[foxyshadis]

This thing is absurdly slow. 50 seconds per frame for 480x270->960x540 on a Haswell! I wanted to redo an entire scene at 4x, but I didn't want to die of old age first.

4 second sample (https://dl.dropboxusercontent.com/u/54412753/doom9/possession%20coffeeshop%20upscale%20enc.mp4), shows no shimmering that I could see, so at least it's fairly stable.

[burfadel]

Maybe someone could make a temporal version of NNEDI3. Interpolation based on forward and backward frame data, as well as top and bottom fields. Sure, it might not be as good as this, but it would have to be faster!

[*.mp4 guy]

While doing some testing on waifu2x there was strong evidence that its neural network may be significantly overtrained for the purposes of digital video. There can be a change uf up to 1 db in psnr simply by switching between different downscaling algorithms, and it gets conflicting psnr results compared to all other upscalers I have compared with, it always prefers neutral bicubic downscaling.

In one test it prefered neutral bicubic to sharp bicubic by ~0.6 db, while all other scalers prefered sharp cubic downscaling to neutral. This would strongly indicate that it was trained almost exclusively on content scaled using a catmull-rom kernel. Unsurprisingly, it does produce very nice results when fed with input it has been trained for.

I will probably do some more rigorous testing on more varied content, but its speed makes testing it aggravating.

[colours]

Maybe someone could make a temporal version of NNEDI3. Interpolation based on forward and backward frame data, as well as top and bottom fields. Sure, it might not be as good as this, but it would have to be faster!

Sometimes people forget that we already have a super-resolution filter for Avisynth and it's called QTGMC. Motion-compensated deinterlacing is really just a special case of super-resolution, after all.

[CkJ]

*Original
http://0.t.imgbox.com/i4eViOMy.jpg (http://imgbox.com/i4eViOMy)

*Nnedi3
http://6.t.imgbox.com/s4wergxr.jpg (http://imgbox.com/s4wergxr)

*Waifu2x
http://6.t.imgbox.com/ao7AKjpu.jpg (http://imgbox.com/ao7AKjpu)

[mastrboy]

@CkJ

How about using a little more safe for work image site the next time, did not expect female genitalia popping up in my face when clicking those links :P

[CkJ]

@CkJ

How about using a little more safe for work image site the next time, did not expect female genitalia popping up in my face when clicking those links :P

sorry, I use adblock so don't see that popup :D

[mandarinka]

Take care what you link, even if it gets blocked by adblock (protip - not everybody has it active). There is a fullscreen extremely not safe for work ad, consider actually putting a warning in the post, because that thing could really get somebody in trouble if they browsed that at work.

BTW, imagebam is a bad host for images for another reason: it doesn't have a reasonable "show just this image 1:1" mode, it forces you to download when you want to see the original.

Try imgbox - it shouldn't have nasty ads if you specify you are uploading family-safe content. And there are probably other less annoying imagehosts.

[creaothceann]

e.g. imgur, which lets you link directly to the picture.

[mandarinka]

e.g. imgur, which lets you link directly to the picture.

Imgur converts your image to jpg if you upload a PNG that is too big (over 1 MB), so it isn't really suitable here.

[colours]

Imgur converts your image to jpg if you upload a PNG that is too big (over 1 MB), so it isn't really suitable here.

Unless you have an account, in which case the threshold for automatic JPEG conversion is a few megabytes higher.

[sl1pkn07]

try with wstaw. the PNG is PNG

[bxyhxyh]

It now supports Y8.