Log in

View Full Version : any idea how to port a keras model to a vaporsynth plugin


feisty2
3rd January 2018, 18:07
I'm still far far away from getting my super resolution neural net done, the full model has around 40 million parameters and I have yet trained 100k of them with just one iteration of 2.5 million image samples, it's still pretty early stage cuz it's just one iteration of backprop and I only trained the low level residual part of the full model yet, the results are very "raw" and far far away from completion (it's now just one iteration for 100k parameters, I'm planning on 10 ~ 15 iterations for 40m parameters), but it already outperformed stuff like nnedi3 and waifu2x, I just thought I might as well write a vaporsynth plugin for it but I don't know where to get started, the model was coded with keras, a high level tensorflow library, in Python, it has tons of dependencies like cudnn, tensorflow and installing tensorflow, especially the gpu version, is a real pain, how can make it like waifu2x, a single plugin without any dependencies?

also if ur interested in the early stage raw results
small image
https://s9.postimg.org/zelqdm89r/small.png
2x upscale with non-ringing spline
https://s9.postimg.org/4yftf923j/spline.png
2x upscale with nnedi3
https://s9.postimg.org/k75qtcvvj/nnedi3.png
2x upscale with waifu2x
https://s9.postimg.org/gss5qtcy7/waifu2x.png
early stage result of my model
https://s9.postimg.org/a2bohji4f/model.png

WolframRhodium
4th January 2018, 01:41
To the best of my knowledge, there seems no straightforward solution to write a single plugin of CNN without many dependencies.
I'm not sure whether the C++ support of TF on Windows is good or not, but for MXNet, a plugin (https://github.com/kice/vs_mxDnCNN) was released building upon the official C++ support, while the model was trained in Python. The dependencies of it are enormous. There are also many convertors (https://github.com/ysh329/deep-learning-model-convertor) to transform a model in one framework to another, but I'm not sure whether they work well.
Another naive solution is to directly call the model inside Python, but since std.ModifyFrame is a single thread plugin, it behaves like a performance bottleneck.

amayra
6th January 2018, 21:43
vapoursynth*

feisty2
7th January 2018, 13:12
vapoursynth*

and the commonwealth spelling should die

raffriff42
8th January 2018, 02:29
Sounds like a fascinating project! It looks like installing the CPU version of TensorFlow is not that hard, basically one Python command. This eliminates the need for cuDNN as well. Once your project is released, maybe someone will help by writing an install program for the GPU version. https://www.tensorflow.org/install/install_windows

If your system does not have a NVIDIAŽ GPU, you must install this [CPU] version. Note that this version of TensorFlow is typically much easier to install (typically, in 5 or 10 minutes), so even if you have an NVIDIA GPU, we recommend installing this version first.

https://www.tensorflow.org/install/install_windows#installing_with_native_pip
To install TensorFlow, start a [Python] terminal. Then issue the appropriate pip3 install command in that terminal. To install the CPU-only version of TensorFlow, enter the following command:

C:\> pip3 install --upgrade tensorflow

amayra
9th January 2018, 06:46
and the commonwealth spelling should die

commonwealth*

feisty2
9th January 2018, 09:27
commonwealth*

It's spelled "vapor" in Latin so "vapour" is simply incorrect
Now stop trolling and stay on topic

ChaosKing
29th January 2018, 12:39
Google released Tensorflow 1.5: easier integration / installation, built-in CUDA support, a lite version of TF. Maybe it's easier to distribute now as a plugin...

https://developers.googleblog.com/2018/01/announcing-tensorflow-15.html