mxnet : vs_mxDnCNN function

[Cary Knoop]

Is there anyone who can help with getting this to work? I like to test it.

https://github.com/kice/vs_mxDnCNN

I installed the required data in the vs plugins folder and installed mxnet using pip.

I get an error saying no module named.....

I tried mx and mxnet, none worked.



def NeuralDenoise(c, strength=88):
even = core.mx.mxDNCNN(core.std.SelectEvery(c, 2, 0), param=strength, ctx=2, dev_id=0)
odd = core.mx.mxDNCNN(core.std.SelectEvery(c, 2, 1), param=strength, ctx=2, dev_id=1)
return core.std.Interleave([even, odd])

[Selur]

You do have a multi-device setup, yes?
Otherwise you should probably simply use:
clip = core.mx.mxDNCNN(clip, param=88, ctx=2, dev_id=0)
and adjust param to change the strength,...
(haven't tested the filter myself, but that seemed an obvious question)

[Cary Knoop]

You do have a multi-device setup, yes?
Otherwise you should probably simply use:
clip = core.mx.mxDNCNN(clip, param=88, ctx=2, dev_id=0)
and adjust param to change the strength,...
(haven't tested the filter myself, but that seemed an obvious question)
Yes, I have a multi-device setup.

Ah, found out that the actual plugin is separate:
https://github.com/kice/vs_mxDnCNN/releases

It "works" now, I don't get a denoised image though, but it seems to return some kind of noise, am I suppose to subtract it from the image?

[poisondeathray]

It "works" now, I don't get a denoised image though, but it seems to return some kind of noise, am I suppose to subtract it from the image?

Yes, MakeDiff - there is an example on the usage page

[Cary Knoop]

Yes, MakeDiff - there is an example on the usage page
Where is the usage page, I only see the readme?

[poisondeathray]

Where is the usage page, I only see the readme?

You linked to it in the first post :) There is a usage section

https://github.com/kice/vs_mxDnCNN

[poisondeathray]

But there is typo , it should be core.std.MakeDiff not std.core.MakeDiff

[Cary Knoop]

Thanks! It is working now.

88 is very strong, 125 starts to look better, but hf details are definitely touched.

Also, the resulting image gives me some artifacts near the top spanning the full width of the frame.

[poisondeathray]

Yes, it would be nicer if it had more settings , more than 2 param data. I wonder how difficult it would be to generate ? In the dncnn folder , I'm assuming DnCNN88-0000.params, and DnCNN125-0000.params refer to 88 and 125

[poisondeathray]

Also, the resulting image gives me some artifacts near the top spanning the full width of the frame.

attachments can take a while to get approved...sometimes days

people tend to use free 3rd party image hosting sites e.g https://postimages.org/

[Cary Knoop]

Ok, here is a dropbox of a fraction of a frame showing the issue:
https://www.dropbox.com/s/29s3qcm9jbgh828/complete-01.jpg?dl=0

I suppose it is avoidable by adding some extra black space to the top of the video frames.

[Wolfberry]

You can also try DnCNN (https://github.com/WolframRhodium/Super-Resolution-Zoo/tree/master/DnCNN) made by WolframRhodium. Usage is in info.md

Main function: muvsfunc_numpy.super_resolution()

[kice]

Try this out:

https://github.com/kice/vs_mxnet

[ChaosKing]

@kice vs_mxnet and vs_mxDnCNN have the same namespace "mx", maybe you could change one to avoid conflicts?

[WolframRhodium]

@kice vs_mxnet and vs_mxDnCNN have the same namespace "mx", maybe you could change one to avoid conflicts?

I believe vs_mxDnCNN has been deprecated. Use vs_mxnet instead.

I will try to make a comparison between vs_mxDnCNN and vs_mxnet:

The former one is too specific:
It tailored to the DnCNN models that were trained by him in early days, only works on YUV444 data, and it does not support models for super-resolution which enlarge images.

The later one is more general:
It currently has RGB and gray scale support, and can work on versatile models like super-resolution models. (waifu2x for example. See model-zoo (https://github.com/WolframRhodium/Super-Resolution-Zoo). And I have also written a wrapper function (https://github.com/WolframRhodium/muvsfunc/blob/9e6a67a0f64b7e76322204f230775647666dd348/muvsfunc.py#L4177) for it. You can go to demo (https://github.com/WolframRhodium/muvsfunc/blob/9e6a67a0f64b7e76322204f230775647666dd348/Collections/examples/super_resolution_mxnet.vpy) for more information)

[poisondeathray]

Thanks for all the new stuff to play with!

Does anyone have any comments / general impressions for any of the models , or any interesting examples outside of the pdf /original research papers ?

[MonoS]

Has anyone had this problem:
Cannot load MXNet. Please install MXNet when starting a script?

If i import mxnet from python's REPL it seems to work, but from this script
import vapoursynth as vs
import MFunc as mon
import havsfunc as has
import nnedi3_resample

core = vs.get_core()

src = mon.Source("file:///D:/Working/video_source (1).mkv", False)

srcf = nnedi3_resample.nnedi3_resample(src, 1920,1080, csp=vs.RGBS, fast=True, chromak_up="bicubic")
#srcf = core.fmtc.bitdepth(src, flt=True)

#denMV = mon.Denoise2(src, 500, blksize=16, fast=False)
denNN = core.mx.Predict(srcf, "DnCNN-S-15-symbol.json", "DnCNN-S-15-0000.params", ctx=2, scale=1)
#, patch_w, patch_h, scale, output_w, output_h, frame_w, frame_h, step_w, step_h, outstep_w, outstep_h, padding, boder_type, dev_id)

denNN = core.fmtc.bitdepth(denNN, bits=8)

srcf.set_output()
it give me that error.

Any suggestion?

[WolframRhodium]

Has anyone had this problem:
Cannot load MXNet. Please install MXNet when starting a script?

If i import mxnet from python's REPL it seems to work, but from this script

Any suggestion?

MXNet should be loaded explicitly via "import mxnet".

[MonoS]

MXNet should be loaded explicitly via "import mxnet".

Thank you, this solved the problem :)
I guess there are some bug in your muvsfunc regarding YUV clip, for example the ShufflePlanes at line 4380 (https://github.com/WolframRhodium/muvsfunc/blob/9e6a67a0f64b7e76322204f230775647666dd348/muvsfunc.py#L4380) should result in [0,0,0], not [0,1,2]

[WolframRhodium]

Thank you, this solved the problem :)
I guess there are some bug in your muvsfunc regarding YUV clip, for example the ShufflePlanes at line 4380 (https://github.com/WolframRhodium/muvsfunc/blob/9e6a67a0f64b7e76322204f230775647666dd348/muvsfunc.py#L4380) should result in [0,0,0], not [0,1,2]

:thanks:
This bug has been identified and fixed at previous commit 5bbb212 (https://github.com/WolframRhodium/muvsfunc/commit/5bbb2124d1de35cf1afb247dbbb6bd28692e2848).
I always post the code at specific commit rather than at latest status in this forum for clarity. In this case the latest code on GitHub has been fixed while link in this forum was not. Sorry for the inconvenience.

[MonoS]

:thanks:
This bug has been identified and fixed at previous commit 5bbb212 (https://github.com/WolframRhodium/muvsfunc/commit/5bbb2124d1de35cf1afb247dbbb6bd28692e2848).
I always post the code at specific commit rather than at latest status in this forum for clarity. In this case the latest code on GitHub has been fixed while link in this forum was not. Sorry for the inconvenience.

Then my apologies and thanks for your time :)

[cwk]

Does anyone have any guidance on compiling the contents of this repo on linux:

https://github.com/kice/vs_mxnet

I've modified a few of the include files within vsMXNet.cpp so their reference is local. E.g.

#include <VapourSynth/VapourSynth.h>

has been changed to:

#include "VapourSynth.h"

then I'm explicitly including paths with a command like this:
g++ -I /code-vsynth/vapoursynth/include -I /code-vsynth/opencv/modules/core/include/opencv2/ -I code-vsynth/opencv/modules/core/include/ vs_mxnet/vsMXNet.cpp -o libvsMXNet.so -lmxnet -std=gnu++11

But I'm getting this error:
In file included from vs_mxnet/MXDll.h:3:0,
from vs_mxnet/vsMXNet.cpp:24:
vs_mxnet/PDll.h:12:21: fatal error: windows.h: No such file or directory
#include <windows.h>
^
compilation terminated.

Given that I'm trying to compile on Ubuntu 16.04, that windows error leads me to believe this might be insurmountable. Can I include this dependency on Linux?

More generally, has anyone had success compiling this project on linux?

[sl1pkn07]

here problem with vs_mxDnCNN


sed 's|VapourSynth/||g' -i vs_mxDnCNN/mxDnCNN.cpp
echo "all:
g++ -c -fPIC ${CXXFLAGS} ${CPPFLAGS} -I. -I/usr/include/mxnet $(pkg-config --cflags vapoursynth) -o vs_mxDnCNN/mxDnCNN.o vs_mxDnCNN/mxDnCNN.cpp

g++ -shared -lmxnet -fPIC ${LDFLAGS} -o libvs${_plug}.so vs_mxDnCNN/mxDnCNN.o"> Makefile


and install the DnCNN-symbol.json, DnCNN88-0000.params and DnCNN125-0000.params in the dncnn folder in the same directory of the install the library

but i have a problem when try to load, for example, this script:


import vapoursynth as vs
import mxnet
core = vs.get_core()

clip = core.lsmas.LWLibavSource("sample.m2ts")
clip = core.fmtc.bitdepth(clip, bits=32)
clip = core.fmtc.resample (clip, css="444")

noise = core.mx.mxDNCNN(clip, param=88, ctx=2, dev_id=0)

res = core.std.MakeDiff(clip, noise)

# For multi-GPU processing (scales almost linearly)
even = core.mx.mxDNCNN(core.std.SelectEvery(clip, 2, 0), ctx=2, dev_id=0)
odd = core.mx.mxDNCNN(core.std.SelectEvery(clip, 2, 1), ctx=2, dev_id=1)

res = core.std.Interleave([even, odd])

clip=core.fmtc.bitdepth(clip, bits=8)
clip.set_output()


i got


└───╼ vspipe --info mxnet.vpy -
[01:44:57] /tmp/makepkg/sl1-mxnet-git/src/incubator-mxnet/src/nnvm/legacy_json_util.cc:209: Loading symbol saved by previous version v1.0.0. Attempting to upgrade...
[01:44:57] /tmp/makepkg/sl1-mxnet-git/src/incubator-mxnet/src/nnvm/legacy_json_util.cc:217: Symbol successfully upgraded!
Script evaluation failed:
Python exception: mxDncnn: Create Predictor failed

Traceback (most recent call last):
File "vapoursynth.pyx", line 1937, in vapoursynth.vpy_evaluateScript
File "vapoursynth.pyx", line 1938, in vapoursynth.vpy_evaluateScript
File "mxnet-test.vpy", line 15, in <module>
odd = core.mx.mxDNCNN(core.std.SelectEvery(clip, 2, 1), ctx=2, dev_id=1)
File "vapoursynth.pyx", line 1843, in vapoursynth.Function.__call__
vapoursynth.Error: mxDncnn: Create Predictor failed



i'm not sure i do wrong

linux and mxnet compiled from sources (1.4.0)

greetings

[poisondeathray]

can anyone get these to work on windows?

1) for vs_mxnet I get "Vapoursynth.Error: mxnet: Create MXNet Predictor failed" . I followed the instructions, visual studio (do you need a specific version or type of install), cuda, cuda_dnn, mxnet 1.5 with cuda

2) for muvsfunc - super_resolution_mxnet.vpy , I'm having problems with the syntax for loading both params and json file . I tried a bunch of different things , not working. It's probably something simple I'm overlooking


model_filename: Path to the pre-trained model.
This specifies the path prefix of saved model files.
You should have "model_filename-symbol.json", "model_filename-xxxx.params", where xxxx is the epoch number.

[WolframRhodium]

can anyone get these to work on windows?

1) for vs_mxnet I get "Vapoursynth.Error: mxnet: Create MXNet Predictor failed" . I followed the instructions, visual studio (do you need a specific version or type of install), cuda, cuda_dnn, mxnet 1.5 with cuda

2) for muvsfunc - super_resolution_mxnet.vpy , I'm having problems with the syntax for loading both params and json file . I tried a bunch of different things , not working. It's probably something simple I'm overlooking

For the later case, what code have you tried and what errors have arisen?

[poisondeathray]

For the later case, what code have you tried and what errors have arisen?


I'm trying ESRGAN out . I copy/pasted the super_resolution_mxnet.vpy example, but the dict model_filename argument doesn't have 2 entries . So the usual error is


SyntaxError: positional argument follows keyword argument


or


vapoursynth.Error: mxnet: Cannot open symbol json file or param data file


I tried full paths with r' , using ' instead of " , enclosing everything in either '' , "" , parentheses . I tried copying json and params in the immediate directory path

elif sr_algorithm == 1:
# sr_args = dict(model_filename=r'VDSR\pytorch-vdsr@twtygqyy\VDSR',
# device_id=device_id, block_w=128, block_h=128,
# up_scale=2, is_rgb_model=False, pre_upscale=True)
sr_args = dict(model_filename="ESRGAN_4x-symbol.json", "ESRGAN_4x-0000.params", up_scale=4, is_rgb_model=True, pad=None, crop=None, pre_upscale=False)

[poisondeathray]

And I'm able to get vs_mxDnCNN denoiser working, so presumably the mxnet part is working
https://github.com/kice/vs_mxDnCNN

On the other hand, everything was bundled with that package separately including libmxnet.dll and the older cuda 9.0 , not relying on system installed stuff - so maybe not

[WolframRhodium]

sr_args = dict(model_filename="ESRGAN_4x-symbol.json", "ESRGAN_4x-0000.params", up_scale=4, is_rgb_model=True, pad=None, crop=None, pre_upscale=False)

should be
sr_args = dict(model_filename=r"path_to_parent_folder\ESRGAN_4x", up_scale=4, is_rgb_model=True, pad=None, crop=None, pre_upscale=False)

where files "ESRGAN_4x-symbol.json", "ESRGAN_4x-0000.params" can be found in "path_to_parent_folder".

[ChaosKing]

here problem with vs_mxDnCNN


import vapoursynth as vs
import mxnet
core = vs.get_core()

clip = core.lsmas.LWLibavSource("sample.m2ts")
clip = core.fmtc.bitdepth(clip, bits=32)
clip = core.fmtc.resample (clip, css="444")

noise = core.mx.mxDNCNN(clip, param=88, ctx=2, dev_id=0)

res = core.std.MakeDiff(clip, noise)

# For multi-GPU processing (scales almost linearly)
even = core.mx.mxDNCNN(core.std.SelectEvery(clip, 2, 0), ctx=2, dev_id=0)
odd = core.mx.mxDNCNN(core.std.SelectEvery(clip, 2, 1), ctx=2, dev_id=1)

res = core.std.Interleave([even, odd])

clip=core.fmtc.bitdepth(clip, bits=8)
clip.set_output()



I just installed it also and had the same error. You need to remove the code for multi-GPU processing as it uses also dev_id=1!

EDIT:

But the returned image is just black... Or can I only use it in conbination with MadeDiff? The makediff output has some banding like texture.

noise = core.mx.mxDNCNN(clip, param=88, ctx=2, dev_id=0)
res = core.std.MakeDiff(clip, noise)

[poisondeathray]

should be
sr_args = dict(model_filename=r"path_to_parent_folder\ESRGAN_4x", up_scale=4, is_rgb_model=True, pad=None, crop=None, pre_upscale=False)

where files "ESRGAN_4x-symbol.json", "ESRGAN_4x-0000.params" can be found in "path_to_parent_folder".

Thanks, this fixed that syntax error , but now I get

"vapoursynth.Error: mxnet: Create MXNet Predictor failed" like vs_mxnet

[WolframRhodium]

Thanks, this fixed that syntax error , but now I get

"vapoursynth.Error: mxnet: Create MXNet Predictor failed" like vs_mxnet

Please try muvsfunc_numpy's super_resolution(). The vs_mxnet used in muvsfunc's super_resolution() does not provide much useful information.

[poisondeathray]

Please try muvsfunc_numpy's super_resolution(). The vs_mxnet used in muvsfunc's super_resolution() does not provide much useful information.



Failed to evaluate the script:
Python exception: [08:33:01] c:\jenkins\workspace\mxnet\mxnet\3rdparty\dmlc-core\include\dmlc\./json.h:731: Check failed: ch == '{' (60 vs. {) : Error at Line 6, around ^`!DOCTYPE html>`, Expect '{' but get ' File "C:\Python37\lib\site-packages\mxnet\symbol\symbol.py", line 2714, in load
check_call(_LIB.MXSymbolCreateFromFile(c_str(fname), ctypes.byref(handle)))
File "C:\Python37\lib\site-packages\mxnet\base.py", line 254, in check_call
raise MXNetError(py_str(_LIB.MXGetLastError()))
mxnet.base.MXNetError: [08:33:01] c:\jenkins\workspace\mxnet\mxnet\3rdparty\dmlc-core\include\dmlc\./json.h:731: Check failed: ch == '{' (60 vs. {) : Error at Line 6, around ^`!DOCTYPE html>`, Expect '{' but get '



Who is this c:\jenkins ? Does it make a temp directory ?

[ChaosKing]

I'm also trying to get mxnet-gpu to work, but no luck so far.


2019-04-23 16:50:48.035
Failed to evaluate the script:
Python exception: simple_bind error. Arguments:
data: (1, 3, 128, 128)
[16:50:48] C:\Jenkins\workspace\mxnet-tag\mxnet\src\storage\storage.cc:143: Compile with USE_CUDA=1 to enable GPU usage

Traceback (most recent call last):
File "C:\Python37\lib\site-packages\mxnet\symbol\symbol.py", line 1523, in simple_bind
ctypes.byref(exe_handle)))
File "C:\Python37\lib\site-packages\mxnet\base.py", line 252, in check_call
raise MXNetError(py_str(_LIB.MXGetLastError()))
mxnet.base.MXNetError: [16:50:48] C:\Jenkins\workspace\mxnet-tag\mxnet\src\storage\storage.cc:143: Compile with USE_CUDA=1 to enable GPU usage

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "src\cython\vapoursynth.pyx", line 1927, in vapoursynth.vpy_evaluateScript
File "src\cython\vapoursynth.pyx", line 1928, in vapoursynth.vpy_evaluateScript
File "", line 21, in
File "C:\Users\abcd\AppData\Roaming\Python\Python37\site-packages\muvsfunc_numpy.py", line 1692, in super_resolution
net.bind(for_training=False, data_shapes=[('data', data_shape)])
File "C:\Python37\lib\site-packages\mxnet\module\module.py", line 429, in bind
state_names=self._state_names)
File "C:\Python37\lib\site-packages\mxnet\module\executor_group.py", line 279, in __init__
self.bind_exec(data_shapes, label_shapes, shared_group)
File "C:\Python37\lib\site-packages\mxnet\module\executor_group.py", line 375, in bind_exec
shared_group))
File "C:\Python37\lib\site-packages\mxnet\module\executor_group.py", line 662, in _bind_ith_exec
shared_buffer=shared_data_arrays, **input_shapes)
File "C:\Python37\lib\site-packages\mxnet\symbol\symbol.py", line 1529, in simple_bind
raise RuntimeError(error_msg)
RuntimeError: simple_bind error. Arguments:
data: (1, 3, 128, 128)
[16:50:48] C:\Jenkins\workspace\mxnet-tag\mxnet\src\storage\storage.cc:143: Compile with USE_CUDA=1 to enable GPU usage

I think mxnet-gpu is NOT working right now.
My steps so far:
- pip install mxnet-cu92 --pre
Installation ok, but importing mxnet failed.
- installed CUDA from nvidia https://developer.nvidia.com/cuda-downloads?target_os=Windows&target_arch=x86_64&target_version=10&target_type=exelocal
mxnet import failed
- pip install mxnet
This one works (as in import works), but it seems this is cpu only

I also uninstalled mxnet-cu92 and reinstalled via pip after I installed CUDA from nvidia, but te result is the same. What am I missing?

Maybe the cuda version from nvidia is too new?

[poisondeathray]

ck - you got that jenkins dude popping up too :)

did you mean the script works under cpu with - pip install mxnet ?

[poisondeathray]

Is it possible there is an issue or compatibility problem with 9.2 ? That compiled 9.0 cuda version , with 526MB libmxnet.dll distributed with vs_mxDnCNN_package worked for me for vs_mxDnCNN

[ChaosKing]

No only the import works or for example this check python -c "import mxnet; print(mxnet.__version__)" (result is 1.4.0)
vs_mxDnCNN works for me too.

[poisondeathray]

No only the import works or for example this check (result is 1.4.0)



I get 1.5.0

[ChaosKing]

Ok we need cuda 9.2!!! Google always showed me the newest version. It works now :D

Here is 9.2:
https://developer.nvidia.com/cuda-92-download-archive?target_os=Windows&target_arch=x86_64&target_version=10&target_type=exelocal

I removed mxnet via pip again

pip uninstall mxnet
also
pip uninstall mxnet-cu92
and re-installed again
pip install mxnet-cu92

(restart powershell)

python -c "import mxnet; print(mxnet.__version__)" returns 1.5.0 and this script works now too:

import muvsfunc_numpy as mufn
core = vs.core

clip = core.lsmas.LWLibavSource(source=r"D:\test.mkv")
clip = mufn.super_resolution(clip, model_filename = "C:/somefolder/waifu2x/upconv_7_anime_style_art_rgb/scale2.0x_model")


p.s. I installed only CUDA->Development and CUDA->Runtimes and unchecked everything else.


EDIT
With model_filename = "ESRGAN/ESRGAN_4x" i get this error message: could not broadcast input array from shape (1,3,512,512) into shape (1,3,256,256)
input is 1080p yuv420

EDIT2
you need to set this params for ESRGAN_4x (see info.md): up_scale=4, is_rgb_model=True, pad=None, crop=None, pre_upscale=False

[poisondeathray]

I already had cuda 9.2 and the matching mxnet-cu92 :(

Like you I unchecked everything else for cuda

But I'll try uninstall / reinstall, maybe I have several mxnet conflicting versions

[ChaosKing]

I noticed that some denoising models produce large "blocks" in the video. Can this be avoided somehow? With pad and crop maybe? But in the info.md pad and crop is set to None and other values doesn't seem to work.

ex Super-Resolution-Zoo-master\MoePhoto\denoising
muf.super_resolution(clip, block_w=128, block_h=128, up_scale=1, is_rgb_model=False, pad=None, crop=None, pre_upscale=False, upscale_uv=False, model_filename = "C:/denoising/dn_lite15")

[poisondeathray]

EDIT2
you need to set this params for ESRGAN_4x (see info.md): up_scale=4, is_rgb_model=True, pad=None, crop=None, pre_upscale=False


I had that already above


I noticed that some denoising models produce large "blocks" in the video. Can this be avoided somehow? With pad and crop maybe? But in the info.md pad and crop is set to None and other values doesn't seem to work.


I would love to check and compare notes but I can't get it working :(

[ChaosKing]

I fixed it by setting block_w=clip.width, block_h=clip.height
But I must say that most denoising models are useless bcs they are just too strong and kill all fine details. As far as i can see there is no "strength" parameter. I can be only controlled by the model itself like this dn_lite5, dn_lite10, dn_lite15.

Oh and a BIG Thank You to WolframRhodium for converting all the models :thanks:


The ByNet upscaler is relatively fast and also looks good.

[WolframRhodium]

Failed to evaluate the script:
Python exception: [08:33:01] c:\jenkins\workspace\mxnet\mxnet\3rdparty\dmlc-core\include\dmlc\./json.h:731:
Check failed: ch == '{' (60 vs. {) : Error at Line 6, around ^`!DOCTYPE html>`,
Expect '{' but get ' File "C:\Python37\lib\site-packages\mxnet\symbol\symbol.py", line 2714, in load
check_call(_LIB.MXSymbolCreateFromFile(c_str(fname), ctypes.byref(handle)))
File "C:\Python37\lib\site-packages\mxnet\base.py", line 254, in check_call
raise MXNetError(py_str(_LIB.MXGetLastError()))
mxnet.base.MXNetError: [08:33:01] c:\jenkins\workspace\mxnet\mxnet\3rdparty\dmlc-core\include\dmlc\./json.h:731:
Check failed: ch == '{' (60 vs. {) : Error at Line 6, around ^`!DOCTYPE html>`, Expect '{' but get '



Who is this c:\jenkins ? Does it make a temp directory ?

It's a directory on the compilation machine.

Is it possible there is an issue or compatibility problem with 9.2 ? That compiled 9.0 cuda version , with 526MB libmxnet.dll distributed with vs_mxDnCNN_package worked for me for vs_mxDnCNN

I already had cuda 9.2 and the matching mxnet-cu92 :(

Like you I unchecked everything else for cuda

But I'll try uninstall / reinstall, maybe I have several mxnet conflicting versions

Please try an old cuDNN, e.g. 7.3 or older. Sorry I have no idea on that error message.

Ok we need cuda 9.2!!! Google always showed me the newest version. It works now :D

Here is 9.2:
https://developer.nvidia.com/cuda-92-download-archive?target_os=Windows&target_arch=x86_64&target_version=10&target_type=exelocal


The CUDA version should match that of MXNet package, e.g. (CUDA 9.2 + mxnet-cu92) or (CUDA 10.0 + mxnet-cu100).

I noticed that some denoising models produce large "blocks" in the video. Can this be avoided somehow? With pad and crop maybe? But in the info.md pad and crop is set to None and other values doesn't seem to work.

I fixed it by setting block_w=clip.width, block_h=clip.height

Besides increasing block_w/block_h, pad and crop do help to reduce such artifacts. In your case you can set "pad=(x, x, x, x)" and "crop=(4x, 4x, 4x, 4x)", where x is a positive integer, as stated in an example (https://github.com/WolframRhodium/muvsfunc/blob/97125d2ccbcdcaa139d021381a89ed0c4d13447d/Collections/examples/super_resolution_mxnet.vpy#L30-L55).


But I must say that most denoising models are useless bcs they are just too strong and kill all fine details.

Yes, that's why I still prefer KNLMeansCL and BM3D.


As far as i can see there is no "strength" parameter. I can be only controlled by the model itself like this dn_lite5, dn_lite10, dn_lite15.

Yes, that's one of the weaknesses of many DNN-based algos. Besides postprocessing (like FreqMerge (https://github.com/IFeelBloated/Oyster/blob/master/Oyster.py#L43-L46)), one way to solve it is a technique called network interpolation (https://github.com/WolframRhodium/Super-Resolution-Zoo/blob/master/ESRGAN/net_interp.py), which applies linear interpolation in the parameter space of two networks of same architecture (same .json). Providing a noise map as in FFDNet (https://github.com/cszn/ffdnet) is another solution, but it is not compatible with existing plugins and I have not convert it yet.

[ChaosKing]

@poisondeathray try to uninstall everything mxnet related via pip AND look if a mxnet* folder(s) is still preset in C:\Python37\Lib\site-packages\
After that "pip install mxnet-cu92 --pre"

I also installed mxnet-cu100 but nothing was working then anymore xD

[poisondeathray]

@poisondeathray try to uninstall everything mxnet related via pip AND look if a mxnet* folder(s) is still preset in C:\Python37\Lib\site-packages\
After that "pip install mxnet-cu92 --pre"


Yes, I tried that and the folders were removed as expected

I might try matching earlier versions as WR suggested, including cudnn and cuda.



When you installed cuda toolkit 9.2 (mine was for windows 8) , there was a separate patch 1 - did you install that? (I did)

[ChaosKing]

I forgot about the patch... installing.

Edit
Nothing's changed, still working.

[sl1pkn07]

oh! thanks @ChaosKing, now works!

[dipje]

If it's too strong, will a simple blend with the source not give what you want ?

[ChaosKing]

Maybe. But wouldn't you also add some noise back?

[Mystery Keeper]

Initialization is super slow. Takes VS few minutes to initialize the script.

[brucethemoose]

FYI cu100 works fine for me. There's a CUDA 10.1 mxnet package now too, but I haven't tried it yet.

Initialization is super slow. Takes VS few minutes to initialize the script.

It takes a couple of seconds for me, but not minutes.

[Mystery Keeper]

It is super slow when running on CPU. Fast on GPU, but crashes VSEdit on most of videos even with 8GB VRAM.

[brucethemoose]

It is super slow when running on CPU. Fast on GPU, but crashes VSEdit on most of videos even with 8GB VRAM.

Block size, resolution, and the model all have an influence on VRAM usage, which might be the root of the crashing.

CUDA 10.0 and cuDNN 7.3.1 seem stable on my 6GB GPU, but I just picked all this back up after a hiatus of a few months... So time will tell, but I've processed entire videos with MSRN and VSEdit last year. You might even want to give CUDA 10.1 a shot since you have a new GPU.


Pure CPU speed might be better with the Intel MKLDNN package, but I haven't gone down that rabbit hole yet.

[Mystery Keeper]

I got it working, but now every denoising model produces vertical stripes on some clips and works well on others. Doesn't matter if I pad the clip or change patch_w/h.

[brucethemoose]

I got it working, but now every denoising model produces vertical stripes on some clips and works well on others. Doesn't matter if I pad the clip or change patch_w/h.

Does changing block size change the artifacts?

Some of the NN upscalers will pick up almost imperceptible blocking artifacts and sharpen them like crazy. I'm not sure what your source is, but it's possible that they have some kind of artifact the neural net denoiser is sniffing out.

I initially played with the denoisers, but quickly gave up and blended BM3D and KNLMeansCL instead, as they do a better job with temporal noise anyway.

[ChaosKing]

Does changing block size change the artifacts?

Some of the NN upscalers will pick up almost imperceptible blocking artifacts and sharpen them like crazy. I'm not sure what your source is, but it's possible that they have some kind of artifact the neural net denoiser is sniffing out.

I initially played with the denoisers, but quickly gave up and blended BM3D and KNLMeansCL instead, as they do a better job with temporal noise anyway.

Yep, I would only concentrate on sharpeners as they do a much better job compared to denoisers.


Most of the "SuperRes Zoo models" worked for me but you need to use the provided parameters. Sometimes it helps to completely restart you editor if you're testing many different models.

[Mystery Keeper]

Changing patch_w and patch_h AND padding to multiples of 32 doesn't help. It looks not like blocking, but like vertical stripes of distortion. I would really like to get the denoising networks to work. They're not good on their own, but do WONDERS as motion estimation reference clips for pure temporal denoising.

[Mystery Keeper]

The problem vanished with VapourSynth 47.2.

[brucethemoose]

Anyone successfully load kice's plugin with the plain mxnet python package?

I can import and run the test in the python console just fine, but I get this error every time I try to load the plugin in a script:

Failed to evaluate the script:
Python exception: Failed to load MXNet/vs_mxnet.dll. GetLastError() returned 1114.

Traceback (most recent call last):
File "src\cython\vapoursynth.pyx", line 1946, in vapoursynth.vpy_evaluateScript
File "src\cython\vapoursynth.pyx", line 1947, in vapoursynth.vpy_evaluateScript
File "C:/Code/Python/VSSuperRes Helper/Test/VapourSynth64Portable/VapourSynthScripts/ProcessImagesAuto.vpy", line 7, in
core.std.LoadPlugin(r'MXNet/vs_mxnet.dll', altsearchpath=True)
File "src\cython\vapoursynth.pyx", line 1852, in vapoursynth.Function.__call__
vapoursynth.Error: Failed to load MXNet/vs_mxnet.dll. GetLastError() returned 1114.

Happened across 2 fresh VapourSynth installs (one portable, one installed), tried it with mxnet 1.4, 1.5, and 1.6.

"1114" seems to be (https://docs.microsoft.com/en-us/windows/win32/debug/system-error-codes--1000-1299-) "A dynamic link library (DLL) initialization routine failed" error. ChaosKing very helpfully pointed me towards something (https://github.com/brechtsanders/pedeps) for finding .dll dependencies, but it isn't finding anything at all for vs_mxnet.dll... I guess I could try rolling back VapourSynth to a early 2019 version?

[ChaosKing]

A quick google shows that error 1114 has to do something with "switchable dynamic graphics". Try to set it to max performance and see if it helps https://youtu.be/wHsLWvwSfgo?t=123

Another thing to try is to run your app as admin.

[brucethemoose]

A quick google shows that error 1114 has to do something with "switchable dynamic graphics". Try to set it to max performance and see if it helps https://youtu.be/wHsLWvwSfgo?t=123

Another thing to try is to run your app as admin.

Tried admin. And it's an AMD APU, nothing to switch. :(

Maybe CUDA is a hard requirement, even though it has a switch to change devices. I could've sworn I ran the plugin on the CPU before, but maybe I'm remembering the Waifu2x OpenCL plugin...

[Mystery Keeper]

Got myself RTX 3090 and cannot make vs_mxnet work anymore.
"Python exception: mxnet: Invalid MXNet Predictor: Please Try to Upgrade MXNet."
Windows 10.
Tried with mxnet-cu101 and cu102 ver 1.5 with respective CUDA. Then installed CUDA 11.2 and built mxnet 2.0 from sources. Result is the same.

[WolframRhodium]

Does "import mxnet" and following example code from vs_mxnet's test works?

RTX 3090 requires at least CUDA 11.1. There is pre-built binary for cu112 of MXNet.

[Mystery Keeper]

I have recompiled mxnet with CUDA 11.2.
Yes, the example works.
I couldn't find prebuilt cu112 for Windows.
The best pip gets is cu102, version 1.5.

upd: One little correction. Example works, but with this notice:
>>> a = mx.nd.ones((2,3), mx.gpu())
>>> [18:06:07] F:\git\incubator-mxnet\src\storage\storage.cc:199: Using Pooled (Naive) StorageManager for GPU