Waifu2x-w2xc

[HolyWu]

https://github.com/HomeOfVapourSynthEvolution/VapourSynth-Waifu2x-w2xc/

Another implementation of waifu2x, based on the w2xc (https://github.com/tanakamura/waifu2x-converter-cpp) library. It can run on either GPU or CPU.

On my GTX 660, it runs about 25~45% faster when the color family of input clip is RGB than YUV/YCoCg.

[feisty2]

tried photo=True on some RGB clips, and nnedi3 still works better on photo like clips imho.... (waifu2x looks like some stuff over warpsharped or whatever)

[MonoS]

Where did you get the photo model??

[Reel.Deel]

Here: https://github.com/nagadomi/waifu2x/tree/master/models/ukbench

[MonoS]

Here: https://github.com/nagadomi/waifu2x/tree/master/models/ukbench

I feel really stupid sometimes, thanks.

I'll make some tries with this models in the next weeks even if actually i was trying to train my own model for movies but was unable to find a good dataset

[HolyWu]

Update r3.


Update w2xc library to r314.
Add processor and list_proc parameters.


The new parameters are mainly to let users be able to specify which device to use when there are more than one GPUs in the system.

[HolyWu]

Update r4.


Update w2xc library to r419.
Update anime_style_art_rgb and photo models. The old ukbench folder isn't used anymore and you can delete it.
Now the W2XConv object is initialized and destroyed only once per instance.
Fix forget to free the node in one error.

[HolyWu]

Update r5.


Update w2xc library to r423 which fixes memory leaks.
Add "-w2xc" to error messages and some other places.
Put shared memory in instance data instead of allocating them every frame.

[Elegant]

Hi HolyWu, I was wondering if you could add the library/headers you're using to compile your project. Building the waifu2x-converter-cpp from tanakamura with CMake is not well suited for Windows.

[HolyWu]

Hi HolyWu, I was wondering if you could add the library/headers you're using to compile your project. Building the waifu2x-converter-cpp from tanakamura with CMake is not well suited for Windows.

His CMakeLists isn't well written to detect OpenCV on Windows, so you probably need to manually add a "HAVE_OPENCV" entry and set the path to OpenCV's directory in CMake before generating project files. Anyway, http://www.mediafire.com/download/535gmhtzxsnc374/waifu2x-converter-cpp.7z contains project files of MSVC 2013 x64 for your information.

[HolyWu]

Update r6.


Update models and add noise reduction level 3.

[brucethemoose]

Did you change the syntax?

For the life of me, I can't figure out how to call the function... I have all the plugins, but "w2xc.Waifu2x(clip)" just throws the exception

Python exception: name 'w2xc' is not defined


EDIT: Also, I KNOW all the DLLs are loaded. If I try to import in manually, it throws and error saying the plugin is already loaded.

[brucethemoose]

EDIT: Stupid syntax error on my part, my mistake.

[Are_]

That's not the proper error message for an incorrect namespace, can you post your full script?

EDIT: But let me do a wild guess, you didn't include de "core" portion in the call, did you?

[brucethemoose]

That's not the proper error message for an incorrect namespace, can you post your full script?

EDIT: But let me do a wild guess, you didn't include de "core" portion in the call, did you?

At first yes, and I figured that out. What got me stuck was a silly path typo though, thanks.

[HolyWu]

Update r7.


Now use DeadSix27's fork of w2xc.
Update models.
There is no 32 bit dll for windows now. 32 bit is super dead.

[amayra]

thanks but how about using this fork (https://github.com/yui0/waifu2x-ocl) i think this will give a better results

[Myrsloik]

Update r7.


Now use DeadSix27's fork of w2xc.
Update models.
There is no 32 bit dll for windows now. 32 bit is super dead.


Nice, you didn't use an ancient intel compiler that needs extra runtimes. I'll update it.

[Myrsloik]

Could you stop using different directory structures and separate/not separate models between releases? Updating every release in vsrepo is quite annoying.

[HolyWu]

Could you stop using different directory structures and separate/not separate models between releases? Updating every release in vsrepo is quite annoying.

OK. I'll unify the way of archiving the binaries.

[Selur]

Not totally sure whether I make a mistake in using Waifu2x or if this is a bug.
When I use limited range input and feed it to Waifu2x the histogram is compressed.
To illustrate the problem I created the following script:
# Imports
import vapoursynth as vs
core = vs.get_core()
import ctypes
# Loading Support Files
Dllref = ctypes.windll.LoadLibrary("I:/Hybrid/64bit/vsfilters/ResizeFilter/Waifu2x/w2xc.dll")
# Loading Plugins
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/ResizeFilter/Waifu2x/Waifu2x-w2xc.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/vslsmashsource.dll")
# Loading C:\Users\Selur\Desktop\test3.mkv using LWLibavSource
clip = core.lsmas.LWLibavSource(source="C:/Users/Selur/Desktop/test3.mkv", format="YUV420P10", cache=0, prefer_hw=0)

#limited
limited = clip
limited = core.resize.Point(limited, matrix_in_s="470bg",range_s="limited")
limited = core.std.AssumeFPS(limited, fpsnum=24000, fpsden=1001) # making sure frame rate is set to 23.976
limited = core.std.SetFrameProp(clip=limited, prop="_ColorRange", intval=1) # Setting color range to TV (limited) range.
limited = core.resize.Bicubic(clip=limited, format=vs.RGB48, matrix_in_s="470bg", range_s="limited") # adjusting color space from YUV420P10 to RGB48 for Waifu2x-w2xc
limited = core.resize.Bicubic(clip=limited, format=vs.YUV420P16, matrix_s="470bg", range_s="limited") # to YUV420P16
limited = core.hist.Classic(clip=limited) # add histogram

# limited with waifu2x
limitedw = clip
limitedw = core.resize.Point(limitedw, matrix_in_s="470bg",range_s="limited")
limitedw = core.std.AssumeFPS(limitedw, fpsnum=24000, fpsden=1001) # making sure frame rate is set to 23.976
limitedw = core.std.SetFrameProp(clip=limitedw, prop="_ColorRange", intval=1) # Setting color range to TV (limited) range.
limitedw = core.resize.Bicubic(clip=limitedw, format=vs.RGB48, matrix_in_s="470bg", range_s="limited") # adjusting color space from YUV420P10 to RGB48 for Waifu2x-w2xc
limitedw = core.fmtc.bitdepth(clip=limitedw,bits=32) # adjusting bit depth to 32bit
limitedw = core.w2xc.Waifu2x(clip=limitedw, scale=1) # using Waifu2x
limitedw = core.resize.Bicubic(clip=limitedw, format=vs.YUV420P16, matrix_s="470bg", range_s="limited") # to YUV420P16
limitedw = core.hist.Classic(clip=limitedw) # add histogram

# full
full = clip
full = core.resize.Point(full, matrix_in_s="470bg",range_s="full")
full = core.std.AssumeFPS(full, fpsnum=24000, fpsden=1001) # making sure frame rate is set to 23.976
full = core.std.SetFrameProp(clip=full, prop="_ColorRange", intval=0) # Setting color range to PC (full) range.
full = core.resize.Bicubic(clip=full, format=vs.YUV420P16, matrix_s="470bg", range_s="full") # 16bit
full = core.hist.Classic(clip=full) # add histogram


# full with waifu2x
fullw = clip
fullw = core.resize.Point(fullw, matrix_in_s="470bg",range_s="full")
fullw = core.std.AssumeFPS(fullw, fpsnum=24000, fpsden=1001) # making sure frame rate is set to 23.976
fullw = core.std.SetFrameProp(clip=fullw, prop="_ColorRange", intval=0) # Setting color range to PC (full) range.
fullw = core.resize.Bicubic(clip=fullw, format=vs.RGB48, matrix_in_s="470bg", range_s="full") # adjusting color space from YUV420P10 to RGB48 for Waifu2x-w2xc
fullw = core.fmtc.bitdepth(clip=fullw,bits=32) # adjusting bit depth to 32bit
fullw = core.w2xc.Waifu2x(clip=fullw, scale=1) # using Waifu2x
fullw = core.resize.Bicubic(clip=fullw, format=vs.YUV420P16, matrix_s="470bg", range_s="full") # 16bit
fullw = core.hist.Classic(clip=fullw) # add histogram

limitedstack = core.std.StackHorizontal([limited,limitedw])
fullstack = core.std.StackHorizontal([full,fullw])
clip = core.std.StackVertical([limitedstack,fullstack])

# Output
clip.set_output()
which compares what happens to the histogram if the source is interpreted as 'full' or 'limited', converted to RGB48 and then fed to Waifu2x.
https://i.ibb.co/KXdx4bP/Waifu2x-Test.png
I uploaded the script I used and the source to my GoogleDrive (https://drive.google.com/open?id=0B_WxUS1XGCPAUTlILW54VThMTFU) as test.mkv and Waifu2x_Test.vpy

So my question is:
Is this a bug or am I doing something wrong?

In case I'm doing something wrong it would be nice if someone could post how to properly handle limited sources with Waifu2x.
Thanks!

Cu Selur

[poisondeathray]

@Selur:

1) it works if you use core.resize.Bicubic for 32bit (format=vs.RGBS) instead of fmtc . Should be faster than 2 separate operations (resize and fmtc) also

Or,

2) fulls=False, fulld=True for core.fmtc.bitdepth

Not sure if that is intended behaviour for fmtc, since the immediate input was RGB48, but it seems to work


(Another observation is that w2xc.Waifu2x clamps to limited range - ie. superbrights/darks are clamped. If you bypass that step, just go RGBS back to YUV, they are still there)

[Selur]

Adjusted my test script to:
# Imports
import vapoursynth as vs
core = vs.get_core()
import ctypes
# Loading Support Files
Dllref = ctypes.windll.LoadLibrary("I:/Hybrid/64bit/vsfilters/ResizeFilter/Waifu2x/w2xc.dll")
# Loading Plugins
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/ResizeFilter/Waifu2x/Waifu2x-w2xc.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/vslsmashsource.dll")
# Loading C:\Users\Selur\Desktop\test3.mkv using LWLibavSource
clip = core.lsmas.LWLibavSource(source="C:/Users/Selur/Desktop/test3.mkv", format="YUV420P10", cache=0, prefer_hw=0)

#limited
limited = clip
limited = core.resize.Point(limited, matrix_in_s="470bg",range_s="limited")
limited = core.std.AssumeFPS(limited, fpsnum=24000, fpsden=1001) # making sure frame rate is set to 23.976
limited = core.std.SetFrameProp(clip=limited, prop="_ColorRange", intval=1) # Setting color range to TV (limited) range.
limited = core.resize.Bicubic(clip=limited, format=vs.YUV420P16, matrix_s="470bg", range_s="limited") # to YUV420P16
limited = core.hist.Classic(clip=limited) # add histogram

# limited with waifu2x
limitedw = clip
limitedw = core.resize.Point(limitedw, matrix_in_s="470bg",range_s="limited")
limitedw = core.std.AssumeFPS(limitedw, fpsnum=24000, fpsden=1001) # making sure frame rate is set to 23.976
limitedw = core.std.SetFrameProp(clip=limitedw, prop="_ColorRange", intval=1) # Setting color range to TV (limited) range.
limitedw = core.resize.Bicubic(clip=limitedw, format=vs.RGBS, matrix_in_s="470bg", range_s="limited") # adjusting color space from RGBS to RGB48 for Waifu2x-w2xc
limitedw = core.w2xc.Waifu2x(clip=limitedw, scale=1) # using Waifu2x
limitedw = core.resize.Bicubic(clip=limitedw, format=vs.YUV420P16, matrix_s="470bg", range_s="limited") # to YUV420P16
limitedw = core.hist.Classic(clip=limitedw) # add histogram

# full
full = clip
full = core.resize.Point(full, matrix_in_s="470bg",range_s="full")
full = core.std.AssumeFPS(full, fpsnum=24000, fpsden=1001) # making sure frame rate is set to 23.976
full = core.std.SetFrameProp(clip=full, prop="_ColorRange", intval=0) # Setting color range to PC (full) range.
full = core.resize.Bicubic(clip=full, format=vs.YUV420P16, matrix_s="470bg", range_s="full") # 16bit
full = core.hist.Classic(clip=full) # add histogram


# full with waifu2x
fullw = clip
fullw = core.resize.Point(fullw, matrix_in_s="470bg",range_s="full")
fullw = core.std.AssumeFPS(fullw, fpsnum=24000, fpsden=1001) # making sure frame rate is set to 23.976
fullw = core.std.SetFrameProp(clip=fullw, prop="_ColorRange", intval=0) # Setting color range to PC (full) range.
fullw = core.resize.Bicubic(clip=fullw, format=vs.RGBS, matrix_in_s="470bg", range_s="full") # adjusting color space from YUV420P10 to RGB48 for Waifu2x-w2xc
fullw = core.w2xc.Waifu2x(clip=fullw, scale=1) # using Waifu2x
fullw = core.resize.Bicubic(clip=fullw, format=vs.YUV420P16, matrix_s="470bg", range_s="full") # 16bit
fullw = core.hist.Classic(clip=fullw) # add histogram

limitedstack = core.std.StackHorizontal([limited,limitedw])
fullstack = core.std.StackHorizontal([full,fullw])
clip = core.std.StackVertical([limitedstack,fullstack])

# Output
clip.set_output()
using RGBS does seem to fix the problem! THANKS! :)

Cu Selur

[sl1pkn07]

Hi

yes, i know this plugin is set to deprecated, but:


[1/2] Compiling C++ object 'waifu2x-w2xc@sha/Waifu2x-w2xc_Waifu2x-w2xc.cpp.o'.
FAILED: waifu2x-w2xc@sha/Waifu2x-w2xc_Waifu2x-w2xc.cpp.o
c++ -Iwaifu2x-w2xc@sha -I. -I../waifu2x-w2xc -I/usr/include/vapoursynth -fvisibility=hidden -flto -fdiagnostics-color=always -DNDEBUG -pipe -D_FILE_OFFSET_BITS=64 -std=c++14 -ffast-math -mfpmath=sse -msse2 -march=native -O2 -fno-plt -D_FORTIFY_SOURCE=2 -fPIC -MD -MQ 'waifu2x-w2xc@sha/Waifu2x-w2xc_Waifu2x-w2xc.cpp.o' -MF 'waifu2x-w2xc@sha/Waifu2x-w2xc_Waifu2x-w2xc.cpp.o.d' -o 'waifu2x-w2xc@sha/Waifu2x-w2xc_Waifu2x-w2xc.cpp.o' -c ../waifu2x-w2xc/Waifu2x-w2xc/Waifu2x-w2xc.cpp
../waifu2x-w2xc/Waifu2x-w2xc/Waifu2x-w2xc.cpp: En la función ‘bool filter(const VSFrameRef*, VSFrameRef*, Waifu2xData*, const VSAPI*)’:
../waifu2x-w2xc/Waifu2x-w2xc/Waifu2x-w2xc.cpp:71:9: error: ‘w2xconv_convert_rgb_f32’ no se declaró en este ámbito
71 | if (w2xconv_convert_rgb_f32(d->conv,
| ^~~~~~~~~~~~~~~~~~~~~~~
ninja: build stopped: subcommand failed.


with waifu2x-converter-cpp from Deadsix27 fork (with opencv and cuda activated)

EDIT: fail build with this commit of waifu2x-converter-cpp https://github.com/DeadSix27/waifu2x-converter-cpp/commit/248797748d5a7410924a1d5f4ad1cc0d112549f4

greetings

[HolyWu]

EDIT: fail build with this commit of waifu2x-converter-cpp https://github.com/DeadSix27/waifu2x-converter-cpp/commit/248797748d5a7410924a1d5f4ad1cc0d112549f4

Fixed.