mxnet : vs_mxDnCNN function

[MonoS]

Originally Posted by WolframRhodium

This bug has been identified and fixed at previous commit 5bbb212.
I always post the code at specific commit rather than at latest status in this forum for clarity. In this case the latest code on GitHub has been fixed while link in this forum was not. Sorry for the inconvenience.

Then my apologies and thanks for your time

[cwk]

Does anyone have any guidance on compiling the contents of this repo on linux:

https://github.com/kice/vs_mxnet

I've modified a few of the include files within vsMXNet.cpp so their reference is local. E.g.

#include <VapourSynth/VapourSynth.h>

has been changed to:

#include "VapourSynth.h"

then I'm explicitly including paths with a command like this:
g++ -I /code-vsynth/vapoursynth/include -I /code-vsynth/opencv/modules/core/include/opencv2/ -I code-vsynth/opencv/modules/core/include/ vs_mxnet/vsMXNet.cpp -o libvsMXNet.so -lmxnet -std=gnu++11

But I'm getting this error:
In file included from vs_mxnet/MXDll.h:3:0,
from vs_mxnet/vsMXNet.cpp:24:
vs_mxnet/PDll.h:12:21: fatal error: windows.h: No such file or directory
#include <windows.h>
^
compilation terminated.

Given that I'm trying to compile on Ubuntu 16.04, that windows error leads me to believe this might be insurmountable. Can I include this dependency on Linux?

More generally, has anyone had success compiling this project on linux?

[sl1pkn07]

here problem with vs_mxDnCNN

  sed 's|VapourSynth/||g' -i vs_mxDnCNN/mxDnCNN.cpp
  echo "all:
	  g++ -c -fPIC ${CXXFLAGS} ${CPPFLAGS} -I. -I/usr/include/mxnet $(pkg-config --cflags vapoursynth) -o vs_mxDnCNN/mxDnCNN.o vs_mxDnCNN/mxDnCNN.cpp

	  g++ -shared -lmxnet -fPIC ${LDFLAGS} -o libvs${_plug}.so vs_mxDnCNN/mxDnCNN.o"> Makefile

and install the DnCNN-symbol.json, DnCNN88-0000.params and DnCNN125-0000.params in the dncnn folder in the same directory of the install the library

but i have a problem when try to load, for example, this script:

import vapoursynth as vs
import mxnet
core = vs.get_core()

clip = core.lsmas.LWLibavSource("sample.m2ts")
clip = core.fmtc.bitdepth(clip, bits=32)
clip = core.fmtc.resample (clip, css="444")

noise = core.mx.mxDNCNN(clip, param=88, ctx=2, dev_id=0)

res = core.std.MakeDiff(clip, noise)

# For multi-GPU processing (scales almost linearly)
even = core.mx.mxDNCNN(core.std.SelectEvery(clip, 2, 0), ctx=2, dev_id=0)
odd = core.mx.mxDNCNN(core.std.SelectEvery(clip, 2, 1), ctx=2, dev_id=1)

res = core.std.Interleave([even, odd])

clip=core.fmtc.bitdepth(clip, bits=8) 
clip.set_output()

i got

└───╼  vspipe --info mxnet.vpy -
[01:44:57] /tmp/makepkg/sl1-mxnet-git/src/incubator-mxnet/src/nnvm/legacy_json_util.cc:209: Loading symbol saved by previous version v1.0.0. Attempting to upgrade...
[01:44:57] /tmp/makepkg/sl1-mxnet-git/src/incubator-mxnet/src/nnvm/legacy_json_util.cc:217: Symbol successfully upgraded!
Script evaluation failed:
Python exception: mxDncnn: Create Predictor failed

Traceback (most recent call last):
  File "vapoursynth.pyx", line 1937, in vapoursynth.vpy_evaluateScript
  File "vapoursynth.pyx", line 1938, in vapoursynth.vpy_evaluateScript
  File "mxnet-test.vpy", line 15, in <module>
    odd = core.mx.mxDNCNN(core.std.SelectEvery(clip, 2, 1), ctx=2, dev_id=1)
  File "vapoursynth.pyx", line 1843, in vapoursynth.Function.__call__
vapoursynth.Error: mxDncnn: Create Predictor failed

i'm not sure i do wrong

linux and mxnet compiled from sources (1.4.0)

greetings

[poisondeathray]

can anyone get these to work on windows?

1) for vs_mxnet I get "Vapoursynth.Error: mxnet: Create MXNet Predictor failed" . I followed the instructions, visual studio (do you need a specific version or type of install), cuda, cuda_dnn, mxnet 1.5 with cuda

2) for muvsfunc - super_resolution_mxnet.vpy , I'm having problems with the syntax for loading both params and json file . I tried a bunch of different things , not working. It's probably something simple I'm overlooking

model_filename: Path to the pre-trained model.
This specifies the path prefix of saved model files.
You should have "model_filename-symbol.json", "model_filename-xxxx.params", where xxxx is the epoch number.

[WolframRhodium]

Originally Posted by poisondeathray

can anyone get these to work on windows?

1) for vs_mxnet I get "Vapoursynth.Error: mxnet: Create MXNet Predictor failed" . I followed the instructions, visual studio (do you need a specific version or type of install), cuda, cuda_dnn, mxnet 1.5 with cuda

2) for muvsfunc - super_resolution_mxnet.vpy , I'm having problems with the syntax for loading both params and json file . I tried a bunch of different things , not working. It's probably something simple I'm overlooking

For the later case, what code have you tried and what errors have arisen?

[poisondeathray]

Originally Posted by WolframRhodium

For the later case, what code have you tried and what errors have arisen?

I'm trying ESRGAN out . I copy/pasted the super_resolution_mxnet.vpy example, but the dict model_filename argument doesn't have 2 entries . So the usual error is

SyntaxError: positional argument follows keyword argument

or

vapoursynth.Error: mxnet: Cannot open symbol json file or param data file

I tried full paths with r' , using ' instead of " , enclosing everything in either '' , "" , parentheses . I tried copying json and params in the immediate directory path

elif sr_algorithm == 1:
# sr_args = dict(model_filename=r'VDSR\pytorch-vdsr@twtygqyy\VDSR',
# device_id=device_id, block_w=128, block_h=128,
# up_scale=2, is_rgb_model=False, pre_upscale=True)
sr_args = dict(model_filename="ESRGAN_4x-symbol.json", "ESRGAN_4x-0000.params", up_scale=4, is_rgb_model=True, pad=None, crop=None, pre_upscale=False)

[poisondeathray]

And I'm able to get vs_mxDnCNN denoiser working, so presumably the mxnet part is working
https://github.com/kice/vs_mxDnCNN

On the other hand, everything was bundled with that package separately including libmxnet.dll and the older cuda 9.0 , not relying on system installed stuff - so maybe not

[WolframRhodium]

Originally Posted by poisondeathray

sr_args = dict(model_filename="ESRGAN_4x-symbol.json", "ESRGAN_4x-0000.params", up_scale=4, is_rgb_model=True, pad=None, crop=None, pre_upscale=False)

should be

sr_args = dict(model_filename=r"path_to_parent_folder\ESRGAN_4x", up_scale=4, is_rgb_model=True, pad=None, crop=None, pre_upscale=False)

where files "ESRGAN_4x-symbol.json", "ESRGAN_4x-0000.params" can be found in "path_to_parent_folder".

[ChaosKing]

Originally Posted by sl1pkn07

here problem with vs_mxDnCNN

import vapoursynth as vs
import mxnet
core = vs.get_core()

clip = core.lsmas.LWLibavSource("sample.m2ts")
clip = core.fmtc.bitdepth(clip, bits=32)
clip = core.fmtc.resample (clip, css="444")

noise = core.mx.mxDNCNN(clip, param=88, ctx=2, dev_id=0)

res = core.std.MakeDiff(clip, noise)

# For multi-GPU processing (scales almost linearly)
even = core.mx.mxDNCNN(core.std.SelectEvery(clip, 2, 0), ctx=2, dev_id=0)
odd = core.mx.mxDNCNN(core.std.SelectEvery(clip, 2, 1), ctx=2, dev_id=1)

res = core.std.Interleave([even, odd])

clip=core.fmtc.bitdepth(clip, bits=8) 
clip.set_output()

I just installed it also and had the same error. You need to remove the code for multi-GPU processing as it uses also dev_id=1!

EDIT:

But the returned image is just black... Or can I only use it in conbination with MadeDiff? The makediff output has some banding like texture.

noise = core.mx.mxDNCNN(clip, param=88, ctx=2, dev_id=0)
res = core.std.MakeDiff(clip, noise)

[poisondeathray]

Originally Posted by WolframRhodium

should be

sr_args = dict(model_filename=r"path_to_parent_folder\ESRGAN_4x", up_scale=4, is_rgb_model=True, pad=None, crop=None, pre_upscale=False)

where files "ESRGAN_4x-symbol.json", "ESRGAN_4x-0000.params" can be found in "path_to_parent_folder".

Thanks, this fixed that syntax error , but now I get

"vapoursynth.Error: mxnet: Create MXNet Predictor failed" like vs_mxnet

[WolframRhodium]

Originally Posted by poisondeathray

Thanks, this fixed that syntax error , but now I get

"vapoursynth.Error: mxnet: Create MXNet Predictor failed" like vs_mxnet

Please try muvsfunc_numpy's super_resolution(). The vs_mxnet used in muvsfunc's super_resolution() does not provide much useful information.

[poisondeathray]

Originally Posted by WolframRhodium

Please try muvsfunc_numpy's super_resolution(). The vs_mxnet used in muvsfunc's super_resolution() does not provide much useful information.

Failed to evaluate the script:
Python exception: [08:33:01] c:\jenkins\workspace\mxnet\mxnet\3rdparty\dmlc-core\include\dmlc\./json.h:731: Check failed: ch == '{' (60 vs. {) : Error at Line 6, around ^`!DOCTYPE html>`, Expect '{' but get ' File "C:\Python37\lib\site-packages\mxnet\symbol\symbol.py", line 2714, in load
check_call(_LIB.MXSymbolCreateFromFile(c_str(fname), ctypes.byref(handle)))
File "C:\Python37\lib\site-packages\mxnet\base.py", line 254, in check_call
raise MXNetError(py_str(_LIB.MXGetLastError()))
mxnet.base.MXNetError: [08:33:01] c:\jenkins\workspace\mxnet\mxnet\3rdparty\dmlc-core\include\dmlc\./json.h:731: Check failed: ch == '{' (60 vs. {) : Error at Line 6, around ^`!DOCTYPE html>`, Expect '{' but get '

Who is this c:\jenkins ? Does it make a temp directory ?

[ChaosKing]

I'm also trying to get mxnet-gpu to work, but no luck so far.

2019-04-23 16:50:48.035
Failed to evaluate the script:
Python exception: simple_bind error. Arguments:
data: (1, 3, 128, 128)
[16:50:48] C:\Jenkins\workspace\mxnet-tag\mxnet\src\storage\storage.cc:143: Compile with USE_CUDA=1 to enable GPU usage

Traceback (most recent call last):
File "C:\Python37\lib\site-packages\mxnet\symbol\symbol.py", line 1523, in simple_bind
ctypes.byref(exe_handle)))
File "C:\Python37\lib\site-packages\mxnet\base.py", line 252, in check_call
raise MXNetError(py_str(_LIB.MXGetLastError()))
mxnet.base.MXNetError: [16:50:48] C:\Jenkins\workspace\mxnet-tag\mxnet\src\storage\storage.cc:143: Compile with USE_CUDA=1 to enable GPU usage

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "src\cython\vapoursynth.pyx", line 1927, in vapoursynth.vpy_evaluateScript
File "src\cython\vapoursynth.pyx", line 1928, in vapoursynth.vpy_evaluateScript
File "", line 21, in 
File "C:\Users\abcd\AppData\Roaming\Python\Python37\site-packages\muvsfunc_numpy.py", line 1692, in super_resolution
net.bind(for_training=False, data_shapes=[('data', data_shape)])
File "C:\Python37\lib\site-packages\mxnet\module\module.py", line 429, in bind
state_names=self._state_names)
File "C:\Python37\lib\site-packages\mxnet\module\executor_group.py", line 279, in __init__
self.bind_exec(data_shapes, label_shapes, shared_group)
File "C:\Python37\lib\site-packages\mxnet\module\executor_group.py", line 375, in bind_exec
shared_group))
File "C:\Python37\lib\site-packages\mxnet\module\executor_group.py", line 662, in _bind_ith_exec
shared_buffer=shared_data_arrays, **input_shapes)
File "C:\Python37\lib\site-packages\mxnet\symbol\symbol.py", line 1529, in simple_bind
raise RuntimeError(error_msg)
RuntimeError: simple_bind error. Arguments:
data: (1, 3, 128, 128)
[16:50:48] C:\Jenkins\workspace\mxnet-tag\mxnet\src\storage\storage.cc:143: Compile with USE_CUDA=1 to enable GPU usage

I think mxnet-gpu is NOT working right now.
My steps so far:
- pip install mxnet-cu92 --pre
Installation ok, but importing mxnet failed.
- installed CUDA from nvidia https://developer.nvidia.com/cuda-do..._type=exelocal
mxnet import failed
- pip install mxnet
This one works (as in import works), but it seems this is cpu only

I also uninstalled mxnet-cu92 and reinstalled via pip after I installed CUDA from nvidia, but te result is the same. What am I missing?

Maybe the cuda version from nvidia is too new?

[poisondeathray]

ck - you got that jenkins dude popping up too

did you mean the script works under cpu with - pip install mxnet ?

[poisondeathray]

Is it possible there is an issue or compatibility problem with 9.2 ? That compiled 9.0 cuda version , with 526MB libmxnet.dll distributed with vs_mxDnCNN_package worked for me for vs_mxDnCNN

[ChaosKing]

No only the import works or for example this check

python -c "import mxnet; print(mxnet.__version__)"

(result is 1.4.0)
vs_mxDnCNN works for me too.

[poisondeathray]

Originally Posted by ChaosKing

No only the import works or for example this check (result is 1.4.0)

I get 1.5.0

[ChaosKing]

Ok we need cuda 9.2!!! Google always showed me the newest version. It works now

Here is 9.2:
https://developer.nvidia.com/cuda-92..._type=exelocal

I removed mxnet via pip again

pip uninstall mxnet
also
pip uninstall mxnet-cu92
and re-installed again
pip install mxnet-cu92

(restart powershell)

python -c "import mxnet; print(mxnet.__version__)" returns 1.5.0 and this script works now too:

import muvsfunc_numpy as mufn
core = vs.core

clip = core.lsmas.LWLibavSource(source=r"D:\test.mkv")
clip = mufn.super_resolution(clip, model_filename = "C:/somefolder/waifu2x/upconv_7_anime_style_art_rgb/scale2.0x_model")

p.s. I installed only CUDA->Development and CUDA->Runtimes and unchecked everything else.


EDIT
With model_filename = "ESRGAN/ESRGAN_4x" i get this error message: could not broadcast input array from shape (1,3,512,512) into shape (1,3,256,256)
input is 1080p yuv420

EDIT2
you need to set this params for ESRGAN_4x (see info.md): up_scale=4, is_rgb_model=True, pad=None, crop=None, pre_upscale=False

[poisondeathray]

I already had cuda 9.2 and the matching mxnet-cu92

Like you I unchecked everything else for cuda

But I'll try uninstall / reinstall, maybe I have several mxnet conflicting versions

[ChaosKing]

I noticed that some denoising models produce large "blocks" in the video. Can this be avoided somehow? With pad and crop maybe? But in the info.md pad and crop is set to None and other values doesn't seem to work.

ex Super-Resolution-Zoo-master\MoePhoto\denoising
muf.super_resolution(clip, block_w=128, block_h=128, up_scale=1, is_rgb_model=False, pad=None, crop=None, pre_upscale=False, upscale_uv=False, model_filename = "C:/denoising/dn_lite15")