mxnet : vs_mxDnCNN function

[poisondeathray]

Originally Posted by ChaosKing

EDIT2
you need to set this params for ESRGAN_4x (see info.md): up_scale=4, is_rgb_model=True, pad=None, crop=None, pre_upscale=False

I had that already above

Originally Posted by ChaosKing

I noticed that some denoising models produce large "blocks" in the video. Can this be avoided somehow? With pad and crop maybe? But in the info.md pad and crop is set to None and other values doesn't seem to work.

I would love to check and compare notes but I can't get it working

[ChaosKing]

I fixed it by setting block_w=clip.width, block_h=clip.height
But I must say that most denoising models are useless bcs they are just too strong and kill all fine details. As far as i can see there is no "strength" parameter. I can be only controlled by the model itself like this dn_lite5, dn_lite10, dn_lite15.

Oh and a BIG Thank You to WolframRhodium for converting all the models


The ByNet upscaler is relatively fast and also looks good.

[WolframRhodium]

Originally Posted by poisondeathray

Failed to evaluate the script:
Python exception: [08:33:01] c:\jenkins\workspace\mxnet\mxnet\3rdparty\dmlc-core\include\dmlc\./json.h:731: 
Check failed: ch == '{' (60 vs. {) : Error at Line 6, around ^`!DOCTYPE html>`, 
Expect '{' but get ' File "C:\Python37\lib\site-packages\mxnet\symbol\symbol.py", line 2714, in load
check_call(_LIB.MXSymbolCreateFromFile(c_str(fname), ctypes.byref(handle)))
File "C:\Python37\lib\site-packages\mxnet\base.py", line 254, in check_call
raise MXNetError(py_str(_LIB.MXGetLastError()))
mxnet.base.MXNetError: [08:33:01] c:\jenkins\workspace\mxnet\mxnet\3rdparty\dmlc-core\include\dmlc\./json.h:731: 
Check failed: ch == '{' (60 vs. {) : Error at Line 6, around ^`!DOCTYPE html>`, Expect '{' but get '

Who is this c:\jenkins ? Does it make a temp directory ?

It's a directory on the compilation machine.

Originally Posted by poisondeathray

Is it possible there is an issue or compatibility problem with 9.2 ? That compiled 9.0 cuda version , with 526MB libmxnet.dll distributed with vs_mxDnCNN_package worked for me for vs_mxDnCNN

Originally Posted by poisondeathray

I already had cuda 9.2 and the matching mxnet-cu92

Like you I unchecked everything else for cuda

But I'll try uninstall / reinstall, maybe I have several mxnet conflicting versions

Please try an old cuDNN, e.g. 7.3 or older. Sorry I have no idea on that error message.

Originally Posted by ChaosKing

Ok we need cuda 9.2!!! Google always showed me the newest version. It works now

Here is 9.2:
https://developer.nvidia.com/cuda-92..._type=exelocal

The CUDA version should match that of MXNet package, e.g. (CUDA 9.2 + mxnet-cu92) or (CUDA 10.0 + mxnet-cu100).

Originally Posted by ChaosKing

I noticed that some denoising models produce large "blocks" in the video. Can this be avoided somehow? With pad and crop maybe? But in the info.md pad and crop is set to None and other values doesn't seem to work.

Originally Posted by ChaosKing

I fixed it by setting block_w=clip.width, block_h=clip.height

Besides increasing block_w/block_h, pad and crop do help to reduce such artifacts. In your case you can set "pad=(x, x, x, x)" and "crop=(4x, 4x, 4x, 4x)", where x is a positive integer, as stated in an example.

Originally Posted by ChaosKing

But I must say that most denoising models are useless bcs they are just too strong and kill all fine details.

Yes, that's why I still prefer KNLMeansCL and BM3D.

Originally Posted by ChaosKing

As far as i can see there is no "strength" parameter. I can be only controlled by the model itself like this dn_lite5, dn_lite10, dn_lite15.

Yes, that's one of the weaknesses of many DNN-based algos. Besides postprocessing (like FreqMerge), one way to solve it is a technique called network interpolation, which applies linear interpolation in the parameter space of two networks of same architecture (same .json). Providing a noise map as in FFDNet is another solution, but it is not compatible with existing plugins and I have not convert it yet.

[ChaosKing]

@poisondeathray try to uninstall everything mxnet related via pip AND look if a mxnet* folder(s) is still preset in C:\Python37\Lib\site-packages\
After that "pip install mxnet-cu92 --pre"

I also installed mxnet-cu100 but nothing was working then anymore xD

[poisondeathray]

Originally Posted by ChaosKing

@poisondeathray try to uninstall everything mxnet related via pip AND look if a mxnet* folder(s) is still preset in C:\Python37\Lib\site-packages\
After that "pip install mxnet-cu92 --pre"

Yes, I tried that and the folders were removed as expected

I might try matching earlier versions as WR suggested, including cudnn and cuda.



When you installed cuda toolkit 9.2 (mine was for windows 8) , there was a separate patch 1 - did you install that? (I did)

[ChaosKing]

I forgot about the patch... installing.

Edit
Nothing's changed, still working.

[sl1pkn07]

oh! thanks @ChaosKing, now works!

[dipje]

If it's too strong, will a simple blend with the source not give what you want ?

[ChaosKing]

Maybe. But wouldn't you also add some noise back?

[Mystery Keeper]

Initialization is super slow. Takes VS few minutes to initialize the script.

[brucethemoose]

FYI cu100 works fine for me. There's a CUDA 10.1 mxnet package now too, but I haven't tried it yet.

Originally Posted by Mystery Keeper

Initialization is super slow. Takes VS few minutes to initialize the script.

It takes a couple of seconds for me, but not minutes.

[Mystery Keeper]

It is super slow when running on CPU. Fast on GPU, but crashes VSEdit on most of videos even with 8GB VRAM.

[brucethemoose]

Originally Posted by Mystery Keeper

It is super slow when running on CPU. Fast on GPU, but crashes VSEdit on most of videos even with 8GB VRAM.

Block size, resolution, and the model all have an influence on VRAM usage, which might be the root of the crashing.

CUDA 10.0 and cuDNN 7.3.1 seem stable on my 6GB GPU, but I just picked all this back up after a hiatus of a few months... So time will tell, but I've processed entire videos with MSRN and VSEdit last year. You might even want to give CUDA 10.1 a shot since you have a new GPU.


Pure CPU speed might be better with the Intel MKLDNN package, but I haven't gone down that rabbit hole yet.

[Mystery Keeper]

I got it working, but now every denoising model produces vertical stripes on some clips and works well on others. Doesn't matter if I pad the clip or change patch_w/h.

[brucethemoose]

Originally Posted by Mystery Keeper

I got it working, but now every denoising model produces vertical stripes on some clips and works well on others. Doesn't matter if I pad the clip or change patch_w/h.

Does changing block size change the artifacts?

Some of the NN upscalers will pick up almost imperceptible blocking artifacts and sharpen them like crazy. I'm not sure what your source is, but it's possible that they have some kind of artifact the neural net denoiser is sniffing out.

I initially played with the denoisers, but quickly gave up and blended BM3D and KNLMeansCL instead, as they do a better job with temporal noise anyway.

[ChaosKing]

Originally Posted by brucethemoose

Does changing block size change the artifacts?

Some of the NN upscalers will pick up almost imperceptible blocking artifacts and sharpen them like crazy. I'm not sure what your source is, but it's possible that they have some kind of artifact the neural net denoiser is sniffing out.

I initially played with the denoisers, but quickly gave up and blended BM3D and KNLMeansCL instead, as they do a better job with temporal noise anyway.

Yep, I would only concentrate on sharpeners as they do a much better job compared to denoisers.


Most of the "SuperRes Zoo models" worked for me but you need to use the provided parameters. Sometimes it helps to completely restart you editor if you're testing many different models.

[Mystery Keeper]

Changing patch_w and patch_h AND padding to multiples of 32 doesn't help. It looks not like blocking, but like vertical stripes of distortion. I would really like to get the denoising networks to work. They're not good on their own, but do WONDERS as motion estimation reference clips for pure temporal denoising.

[Mystery Keeper]

The problem vanished with VapourSynth 47.2.

[brucethemoose]

Anyone successfully load kice's plugin with the plain mxnet python package?

I can import and run the test in the python console just fine, but I get this error every time I try to load the plugin in a script:

Failed to evaluate the script:
Python exception: Failed to load MXNet/vs_mxnet.dll. GetLastError() returned 1114.

Traceback (most recent call last):
File "src\cython\vapoursynth.pyx", line 1946, in vapoursynth.vpy_evaluateScript
File "src\cython\vapoursynth.pyx", line 1947, in vapoursynth.vpy_evaluateScript
File "C:/Code/Python/VSSuperRes Helper/Test/VapourSynth64Portable/VapourSynthScripts/ProcessImagesAuto.vpy", line 7, in
core.std.LoadPlugin(r'MXNet/vs_mxnet.dll', altsearchpath=True)
File "src\cython\vapoursynth.pyx", line 1852, in vapoursynth.Function.__call__
vapoursynth.Error: Failed to load MXNet/vs_mxnet.dll. GetLastError() returned 1114.

Happened across 2 fresh VapourSynth installs (one portable, one installed), tried it with mxnet 1.4, 1.5, and 1.6.

"1114" seems to be "A dynamic link library (DLL) initialization routine failed" error. ChaosKing very helpfully pointed me towards something for finding .dll dependencies, but it isn't finding anything at all for vs_mxnet.dll... I guess I could try rolling back VapourSynth to a early 2019 version?

[ChaosKing]

A quick google shows that error 1114 has to do something with "switchable dynamic graphics". Try to set it to max performance and see if it helps https://youtu.be/wHsLWvwSfgo?t=123

Another thing to try is to run your app as admin.