VSGAN - VapourSynth GAN Implementation, based on ESRGAN's implementation

[PRAGMA]

https://github.com/rlaphoenix/VSGAN
docs: https://vsgan.phoeniix.dev

VSGAN as of 1.6.0 is a port of multiple kinds of machine learning GAN Networks and Architectures.
As of 1.6.0, ESRGAN (and its derivatives), and EGVSR is supported.

For much more information, getting started, more details on supported models, networks, architectures, check the docs.

[poisondeathray]

Thanks, it works well

Any lists for nice pre-trained models ? I found a bunch on Google , but does anyone have any "favorites" that work well in specific situations?
(including Pragma's own AD model)

https://upscale.wiki/wiki/Model_Database

[PRAGMA]

Thanks, it works well

Any lists for nice pre-trained models ? I found a bunch on Google , but does anyone have any "favorites" that work well in specific situations?
(including Pragma's own AD model)

https://upscale.wiki/wiki/Model_Database

Yep, take a look at:
https://github.com/WolframRhodium/Super-Resolution-Zoo

[poisondeathray]

Yep, take a look at:
https://github.com/WolframRhodium/Super-Resolution-Zoo

Yes, I remember this, from the other thread - and I could never get that working

But these are converted models (params , json ; not pth), are they drop in compatible ?

(And some of the links to the original projects and papers have pth models)

[ChaosKing]

Thanks, it works well

Any lists for nice pre-trained models ? I found a bunch on Google , but does anyone have any "favorites" that work well in specific situations?
(including Pragma's own AD model)

https://upscale.wiki/wiki/Model_Database

I tested some filters and the most usefull (and safe) one which also does not generate artifacts was 1x_DeSharpen.

The "spongebob model" looked also not bad for some older animes but had some blurry spots for some reason.

[ChaosKing]

I get out of memory errors for 1080p content using a GTX 1070. Does it really need what much ram? used model 1x_DeSharpen

CUDA out of memory. Tried to allocate 1.48 GiB (GPU 0; 8.00 GiB total capacity; 2.56 GiB already allocated; 72.05 MiB free; 3.65 GiB cached)

Restarting vsedit didn't help.

[poisondeathray]

I think so...I can only use 1x_Desharpen on smaller dimensions . You can look at GPU caps viewer or similar utilities and it looks like it's using all

I wonder if there is a way to share system memory with CUDA memory ? eg. Although it's slower, some 3D/CG renderers can offload graphics card memory to system memory when doing calculations (like a shared pool) enabling you to complete if the scene is too large

[brucethemoose]

I think so...I can only use 1x_Desharpen on smaller dimensions . You can look at GPU caps viewer or similar utilities and it looks like it's using all

I wonder if there is a way to share system memory with CUDA memory ? eg. Although it's slower, some 3D/CG renderers can offload graphics card memory to system memory when doing calculations (like a shared pool) enabling you to complete if the scene is too large

I believe graphics drivers automatically do this in 3D programs, but that might not be the case with CUDA.

PyTorch does have a "empty cache" function. Maybe you could call it every few frames with FrameEval (http://www.vapoursynth.com/doc/functions/frameeval.html).

https://pytorch.org/docs/stable/cuda.html#memory-management

[poisondeathray]

I believe graphics drivers automatically do this in 3D programs, but that might not be the case with CUDA.


Definitely CUDA for some 3D renderers like redshift . It's known as "out of core" rendering. I don't think it's an "automatic" feature for CUDA, some programmer figured it out. It's a big selling point for some types of renderers



PyTorch does have a "empty cache" function. Maybe you could call it every few frames with FrameEval (http://www.vapoursynth.com/doc/functions/frameeval.html).

https://pytorch.org/docs/stable/cuda.html#memory-management

Thanks, but I can't even do 1 1920x1080 frame, it probably won't help calling it periodically. It's only that 1x_DeSharpen, the other models work ok (I didn't test them all , just a few) . So maybe something is up with that model

[PRAGMA]

I get out of memory errors for 1080p content using a GTX 1070. Does it really need what much ram? used model 1x_DeSharpen

CUDA out of memory. Tried to allocate 1.48 GiB (GPU 0; 8.00 GiB total capacity; 2.56 GiB already allocated; 72.05 MiB free; 3.65 GiB cached)

Restarting vsedit didn't help.

I tested some filters and the most usefull (and safe) one which also does not generate artifacts was 1x_DeSharpen.

The "spongebob model" looked also not bad for some older animes but had some blurry spots for some reason.

This is an ESRGAN port, so it will generate results the exact same as ESRGAN, and use the same amount of memory as ESRGAN.
But yes, generally speaking x1080 resolution content require quite a bit of memory, i'm trying to work out a way to split up the input frame into multiple split images, to then run them all separately and then combine them back to 1 frame and return, that way using the same amount of memory, but less memory simultaneously, would be slower, but would be the only viable option for most GPU's.
I'm having a bit of trouble working it out right now but hopefully I or someone else, figures out a way to do so.

[WolframRhodium]

i'm trying to work out a way to split up the input frame into multiple split images, to then run them all separately and then combine them back to 1 frame and return

Example from EDSR: https://github.com/thstkdgus35/EDSR-PyTorch/blob/9d3bb0ec620ea2ac1b5e5e7a32b0133fbba66fd2/src/model/__init__.py#L105

[PRAGMA]

Example from EDSR: https://github.com/thstkdgus35/EDSR-PyTorch/blob/9d3bb0ec620ea2ac1b5e5e7a32b0133fbba66fd2/src/model/__init__.py#L105

I updated VSGAN with chunk support, it will chunk into 4 sub images, and render them separately instead of all at once, then re-compile it back into 1 image and returns to frame server.

I didn't end up using anything like EDSR though, instead I simply cropped the frame, and then ran them.

[poisondeathray]

Thanks for the update

Request: If you have time would it be possible to port EDVR ? Looks great and deals with video specific issues, temporal and motion blur

https://xinntao.github.io/projects/EDVR
https://github.com/xinntao/EDVR

[aegisofrime]

Hey PRAGMA,

Just wanted to let you know that the __init__.py file installed by pip and by the zip file provided by you in the manual installation steps have different file sizes. Is this intended?

[poisondeathray]

Request#2 , another pytorch implementation, PPON

https://github.com/Zheng222/PPON

Someone forked it and wrote some demo code , would it be possible to implement in vapoursynth ?
https://github.com/vlad3996/PPON/blob/master/inference_utils.py
https://github.com/vlad3996/PPON/blob/master/demo.ipynb

Thanks

[PRAGMA]

Hey PRAGMA,

Just wanted to let you know that the __init__.py file installed by pip and by the zip file provided by you in the manual installation steps have different file sizes. Is this intended?

I believe when uploaded/setup for PyPI is gets optimized/minified or something before being sent off to PyPI's repo, but I could be wrong, regardless, they are both up to date both the repo and PyPI pip package.

[aegisofrime]

I believe when uploaded/setup for PyPI is gets optimized/minified or something before being sent off to PyPI's repo, but I could be wrong, regardless, they are both up to date both the repo and PyPI pip package.

Well, maybe it's something silly I did somewhere, but I got an unable to import VSGAN error using pip, but was able to use it with the manual installation zip you provided.

As an aside, I wonder what speeds are you folks getting? With a GTX 1070, upscaling 480p content using the "PSNR_x4_DB.pth" model, I'm getting like... 1fps :/

[PRAGMA]

Well, maybe it's something silly I did somewhere, but I got an unable to import VSGAN error using pip, but was able to use it with the manual installation zip you provided.

As an aside, I wonder what speeds are you folks getting? With a GTX 1070, upscaling 480p content using the "PSNR_x4_DB.pth" model, I'm getting like... 1fps :/

Yeah on a GTX 1080ti it takes about 1.8s per frame for a 720x480 input image.
RTX 2080ti takes about 1.4s per frame for the same input.

[Cary Knoop]

Yeah on a GTX 1080ti it takes about 1.8s per frame for a 720x480 input image.
RTX 2080ti takes about 1.4s per frame for the same input.
That's strange, why would the 2080ti be slower than the 1080ti?

[poisondeathray]

That's strange, why would the 2080ti be slower than the 1080ti?

lower sec / frame means it 's faster

[Cary Knoop]

lower sec / frame means it 's faster
Oops, my mistake, I thought I read frames per second.

[Cary Knoop]

It would probably be useful to add installation instructions for portable Vapoursynth installs as well.

[AlphaAtlas]

It would probably be useful to add installation instructions for portable Vapoursynth installs as well.

It works the same in the VS Fatpack for me. Just open a console window wherever python.exe is, and then install all the dependencies with pip the same way.

Unfortunately CUDA isn't portable, so that part needs a system install.

[Cary Knoop]

It works the same in the VS Fatpack for me. Just open a console window wherever python.exe is, and then install all the dependencies with pip the same way.

It's looking for the Vapoursynth installation path.

[PRAGMA]

It's looking for the Vapoursynth installation path.

VSGAN doesn't care where or how you have VapourSynth installed. All it cares is you have installed it via pip and that you have all dependencies especially PyTorch installed.

[Cary Knoop]

VSGAN doesn't care where or how you have VapourSynth installed. All it cares is you have installed it via pip and that you have all dependencies especially PyTorch installed.
That was my whole point if you have an install (not through pip) it fails.

[brucethemoose]

That was my whole point if you have an install (not through pip) it fails.

AFAIK it's not really practical to install without pip, if that's what you mean.

To be clear, what I do with the vapoursynth fatpack is call "C:/pathtofatpack/VapourSynth64/python.exe -m pip install vsgan". You have to specify the portable python path, otherwise you'll call pip in a python setup that doesn't have access to VS.

On Linux, you can also set the PYTHONPATH environment variable in the terminal if you have multiple Python installs.

[Cary Knoop]

To be clear, what I do with the vapoursynth fatpack is call "C:/pathtofatpack/VapourSynth64/python.exe -m pip install vsgan". You have to specify the portable python path, otherwise you'll call pip in a python setup that doesn't have access to VS.

That would work.

I merely wanted to point out that not everybody will know this.

[Selur]

Here's what I did:

download lates VapourSynth64Portable
https://github.com/theChaosCoder/vapoursynth-portable-FATPACK/releases
upgrade pip
python.exe -m pip install --upgrade pip
installed torch (inside the Vapoursynth64 folder)
python.exe -m pip install torch===1.4.0 torchvision===0.5.0 -f https://download.pytorch.org/whl/torch_stable.html
installed vsgan:
python -m pip install vsgan


Then I used a script that worked before:
# Imports
import vapoursynth as vs
core = vs.get_core()
# Loading Plugins
from vsgan import VSGAN
vsgan_device = VSGAN("cuda")
# Loading C:/Users/Selur/Desktop/5000frames.mp4 using LWLibavSource
clip = core.lsmas.LWLibavSource(source="F:/TestClips&Co/files/5000frames.mp4", format="YUV420P10", cache=0, prefer_hw=0)
# making sure input color matrix is set as 470bg
clip = core.resize.Point(clip, matrix_in_s="470bg",range_s="limited")
# making sure frame rate is set to 25
clip = core.std.AssumeFPS(clip, fpsnum=25, fpsden=1)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)

vsgan_device.load_model(
model="I:/Hybrid/64bit/vsfilters/ResizeFilter/VSGRAN/models/4x_xbrz_90k.pth",
scale=2
)
clip = vsgan_device.run(clip=clip, chunk=True)

clip.set_output()
(see: https://github.com/rlaPHOENiX/VSGAN/issues/1)
which now gives me:

Python exception: Error(s) in loading state_dict for RRDBNet:
Missing key(s) in state_dict: "conv_first.weight", "conv_first.bias", "RRDB_trunk.0.RDB1.conv1.weight", "RRDB_trunk.0.RDB1.conv1.bias", "RRDB_trunk.0.RDB1.conv2.weight", "RRDB_trunk.0.RDB1.conv2.bias", "RRDB_trunk.0.RDB1.conv3.weight", "RRDB_trunk.0.RDB1.conv3.bias", "RRDB_trunk.0.RDB1.conv4.weight", "RRDB_trunk.0.RDB1.conv4.bias", "RRDB_trunk.0.RDB1.conv5.weight", "RRDB_trunk.0.RDB1.conv5.bias", "RRDB_trunk.0.RDB2.conv1.weight", "RRDB_trunk.0.RDB2.conv1.bias", "RRDB_trunk.0.RDB2.conv2.weight", "RRDB_trunk.0.RDB2.conv2.bias", "RRDB_trunk.0.RDB2.conv3.weight", "RRDB_trunk.0.RDB2.conv3.bias", "RRDB_trunk.0.RDB2.conv4.weight", "RRDB_trunk.0.RDB2.conv4.bias", "RRDB_trunk.0.RDB2.conv5.weight", "RRDB_trunk.0.RDB2.conv5.bias", "RRDB_trunk.0.RDB3.conv1.weight", "RRDB_trunk.0.RDB3.conv1.bias", "RRDB_trunk.0.RDB3.conv2.weight", "RRDB_trunk.0.RDB3.conv2.bias", "RRDB_trunk.0.RDB3.conv3.weight", "RRDB_trunk.0.RDB3.conv3.bias", "RRDB_trunk.0.RDB3.conv4.weight", "RRDB_trunk.0.RDB3.conv4.bias", "RRDB_trunk.0.RDB3.conv5.weight", "RRDB_trunk.0.RDB3.conv5.bias", "RRDB_trunk.1.RDB1.conv1.weight", "RRDB_trunk.1.RDB1.conv1.bias", "RRDB_trunk.1.RDB1.conv2.weight", "RRDB_trunk.1.RDB1.conv2.bias", "RRDB_trunk.1.RDB1.conv3.weight", "RRDB_trunk.1.RDB1.conv3.bias", "RRDB_trunk.1.RDB1.conv4.weight", "RRDB_trunk.1.RDB1.conv4.bias", "RRDB_trunk.1.RDB1.conv5.weight", "RRDB_trunk.1.RDB1.conv5.bias", "RRDB_trunk.1.RDB2.conv1.weight", "RRDB_trunk.1.RDB2.conv1.bias", "RRDB_trunk.1.RDB2.conv2.weight", "RRDB_trunk.1.RDB2.conv2.bias", "RRDB_trunk.1.RDB2.conv3.weight", "RRDB_trunk.1.RDB2.conv3.bias", "RRDB_trunk.1.RDB2.conv4.weight", "RRDB_trunk.1.RDB2.conv4.bias", "RRDB_trunk.1.RDB2.conv5.weight", "RRDB_trunk.1.RDB2.conv5.bias", "RRDB_trunk.1.RDB3.conv1.weight", "RRDB_trunk.1.RDB3.conv1.bias", "RRDB_trunk.1.RDB3.conv2.weight", "RRDB_trunk.1.RDB3.conv2.bias", "RRDB_trunk.1.RDB3.conv3.weight", "RRDB_trunk.1.RDB3.conv3.bias", "RRDB_trunk.1.RDB3.conv4.weight", "RRDB_trunk.1.RDB3.conv4.bias", "RRDB_trunk.1.RDB3.conv5.weight", "RRDB_trunk.1.RDB3.conv5.bias", "RRDB_trunk.2.RDB1.conv1.weight", "RRDB_trunk.2.RDB1.conv1.bias", "RRDB_trunk.2.RDB1.conv2.weight", "RRDB_trunk.2.RDB1.conv2.bias", "RRDB_trunk.2.RDB1.conv3.weight", "RRDB_trunk.2.RDB1.conv3.bias", "RRDB_trunk.2.RDB1.conv4.weight", "RRDB_trunk.2.RDB1.conv4.bias", "RRDB_trunk.2.RDB1.conv5.weight", "RRDB_trunk.2.RDB1.conv5.bias", "RRDB_trunk.2.RDB2.conv1.weight", "RRDB_trunk.2.RDB2.conv1.bias", "RRDB_trunk.2.RDB2.conv2.weight", "RRDB_trunk.2.RDB2.conv2.bias", "RRDB_trunk.2.RDB2.conv3.weight", "RRDB_trunk.2.RDB2.conv3.bias", "RRDB_trunk.2.RDB2.conv4.weight", "RRDB_trunk.2.RDB2.conv4.bias", "RRDB_trunk.2.RDB2.conv5.weight", "RRDB_trunk.2.RDB2.conv5.bias", "RRDB_trunk.2.RDB3.conv1.weight", "RRDB_trunk.2.RDB3.conv1.bias", "RRDB_trunk.2.RDB3.conv2.weight", "RRDB_trunk.2.RDB3.conv2.bias", "RRDB_trunk.2.RDB3.conv3.weight", "RRDB_trunk.2.RDB3.conv3.bias", "RRDB_trunk.2.RDB3.conv4.weight", "RRDB_trunk.2.RDB3.conv4.bias", "RRDB_trunk.2.RDB3.conv5.weight", "RRDB_trunk.2.RDB3.conv5.bias", "RRDB_trunk.3.RDB1.conv1.weight", "RRDB_trunk.3.RDB1.conv1.bias", "RRDB_trunk.3.RDB1.conv2.weight", "RRDB_trunk.3.RDB1.conv2.bias", "RRDB_trunk.3.RDB1.conv3.weight", "RRDB_trunk.3.RDB1.conv3.bias", "RRDB_trunk.3.RDB1.conv4.weight", "RRDB_trunk.3.RDB1.conv4.bias", "RRDB_trunk.3.RDB1.conv5.weight", "RRDB_trunk.3.RDB1.conv5.bias", "RRDB_trunk.3.RDB2.conv1.weight", "RRDB_trunk.3.RDB2.conv1.bias", "RRDB_trunk.3.RDB2.conv2.weight", "RRDB_trunk.3.RDB2.conv2.bias", "RRDB_trunk.3.RDB2.conv3.weight", "RRDB_trunk.3.RDB2.conv3.bias", "RRDB_trunk.3.RDB2.conv4.weight", "RRDB_trunk.3.RDB2.conv4.bias", "RRDB_trunk.3.RDB2.conv5.weight", "RRDB_trunk.3.RDB2.conv5.bias", "RRDB_trunk.3.RDB3.conv1.weight", "RRDB_trunk.3.RDB3.conv1.bias", "RRDB_trunk.3.RDB3.conv2.weight", "RRDB_trunk.3.RDB3.conv2.bias", "RRDB_trunk.3.RDB3.conv3.weight", "RRDB_trunk.3.RDB3.conv3.bias", "RRDB_trunk.3.RDB3.conv4.weight", "RRDB_trunk.3.RDB3.conv4.bias", "RRDB_trunk.3.RDB3.conv5.weight", "RRDB_trunk.3.RDB3.conv5.bias", "RRDB_trunk.4.RDB1.conv1.weight", "RRDB_trunk.4.RDB1.conv1.bias", "RRDB_trunk.4.RDB1.conv2.weight", "RRDB_trunk.4.RDB1.conv2.bias", "RRDB_trunk.4.RDB1.conv3.weight", "RRDB_trunk.4.RDB1.conv3.bias", "RRDB_trunk.4.RDB1.conv4.weight", "RRDB_trunk.4.RDB1.conv4.bias", "RRDB_trunk.4.RDB1.conv5.weight", "RRDB_trunk.4.RDB1.conv5.bias", "RRDB_trunk.4.RDB2.conv1.weight", "RRDB_trunk.4.RDB2.conv1.bias", "RRDB_trunk.4.RDB2.conv2.weight", "RRDB_trunk.4.RDB2.conv2.bias", "RRDB_trunk.4.RDB2.conv3.weight", "RRDB_trunk.4.RDB2.conv3.bias", "RRDB_trunk.4.RDB2.conv4.weight", "RRDB_trunk.4.RDB2.conv4.bias", "RRDB_trunk.4.RDB2.conv5.weight", "RRDB_trunk.4.RDB2.conv5.bias", "RRDB_trunk.4.RDB3.conv1.weight", "RRDB_trunk.4.RDB3.conv1.bias", "RRDB_trunk.4.RDB3.conv2.weight", "RRDB_trunk.4.RDB3.conv2.bias", "RRDB_trunk.4.RDB3.conv3.weight", "RRDB_trunk.4.RDB3.conv3.bias", "RRDB_trunk.4.RDB3.conv4.weight", "RRDB_trunk.4.RDB3.conv4.bias", "RRDB_trunk.4.RDB3.conv5.weight", "RRDB_trunk.4.RDB3.conv5.bias", "RRDB_trunk.5.RDB1.conv1.weight", "RRDB_trunk.5.RDB1.conv1.bias", "RRDB_trunk.5.RDB1.conv2.weight", "RRDB_trunk.5.RDB1.conv2.bias", "RRDB_trunk.5.RDB1.conv3.weight", "RRDB_trunk.5.RDB1.conv3.bias", "RRDB_trunk.5.RDB1.conv4.weight", "RRDB_trunk.5.RDB1.conv4.bias", "RRDB_trunk.5.RDB1.conv5.weight", "RRDB_trunk.5.RDB1.conv5.bias", "RRDB_trunk.5.RDB2.conv1.weight", "RRDB_trunk.5.RDB2.conv1.bias", "RRDB_trunk.5.RDB2.conv2.weight", "RRDB_trunk.5.RDB2.conv2.bias", "RRDB_trunk.5.RDB2.conv3.weight", "RRDB_trunk.5.RDB2.conv3.bias", "RRDB_trunk.5.RDB2.conv4.weight", "RRDB_trunk.5.RDB2.conv4.bias", "RRDB_trunk.5.RDB2.conv5.weight", "RRDB_trunk.5.RDB2.conv5.bias", "RRDB_trunk.5.RDB3.conv1.weight",
....
Traceback (most recent call last):
File "src\cython\vapoursynth.pyx", line 1956, in vapoursynth.vpy_evaluateScript
File "src\cython\vapoursynth.pyx", line 1957, in vapoursynth.vpy_evaluateScript
File "C:/Users/Selur/Desktop/vsgantest.vpy", line 16, in
vsgan_device.load_model(
File "C:\Users\Selur\Desktop\VapourSynth64Portable\VapourSynth64\Lib\site-packages\vsgan\__init__.py", line 34, in load_model
self.rrdb_net_model.load_state_dict(torch.load(self.model_file), strict=True)
File "C:\Users\Selur\Desktop\VapourSynth64Portable\VapourSynth64\Lib\site-packages\torch\nn\modules\module.py", line 829, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for RRDBNet:
Missing key(s) in state_dict: "conv_first.weight", "conv_first.bias", "RRDB_trunk.0.RDB1.conv1.weight", "RRDB_trunk.0.RDB1.conv1.bias", "RRDB_trunk.0.RDB1.conv2.weight", "RRDB_trunk.0.RDB1.conv2.bias", "RRDB_trunk.0.RDB1.conv3.weight", "RRDB_trunk.0.RDB1.conv3.bias", "RRDB_trunk.0.RDB1.conv4.weight", "RRDB_trunk.0.RDB1.conv4.bias", "RRDB_trunk.0.RDB1.conv5.weight", "RRDB_trunk.0.RDB1.conv5.bias", "RRDB_trunk.0.RDB2.conv1.weight", "RRDB_trunk.0.RDB2.conv1.bias", "RRDB_trunk.0.RDB2.conv2.weight", "RRDB_trunk.0.RDB2.conv2.bias", "RRDB_trunk.0.RDB2.conv3.weight", "RRDB_trunk.0.RDB2.conv3.bias", "RRDB_trunk.0.RDB2.conv4.weight", "RRDB_trunk.0.RDB2.conv4.bias", "RRDB_trunk.0.RDB2.conv5.weight", "RRDB_trunk.0.RDB2.conv5.bias", "RRDB_trunk.0.RDB3.conv1.weight", "RRDB_trunk.0.RDB3.conv1.bias", "RRDB_trunk.0.RDB3.conv2.weight", "RRDB_trunk.0.RDB3.conv2.bias", "RRDB_trunk.0.RDB3.conv3.weight", "RRDB_trunk.0.RDB3.conv3.bias", "RRDB_trunk.0.RDB3.conv4.weight", "RRDB_trunk.0.RDB3.conv4.bias", "RRDB_trunk.0.RDB3.conv5.weight", "RRDB_trunk.0.RDB3.conv5.bias", "RRDB_trunk.1.RDB1.conv1.weight", "RRDB_trunk.1.RDB1.conv1.bias", "RRDB_trunk.1.RDB1.conv2.weight", "RRDB_trunk.1.RDB1.conv2.bias", "RRDB_trunk.1.RDB1.conv3.weight", "RRDB_trunk.1.RDB1.conv3.bias", "RRDB_trunk.1.RDB1.conv4.weight", "RRDB_trunk.1.RDB1.conv4.bias", "RRDB_trunk.1.RDB1.conv5.weight", "RRDB_trunk.1.RDB1.conv5.bias", "RRDB_trunk.1.RDB2.conv1.weight", "RRDB_trunk.1.RDB2.conv1.bias", "RRDB_trunk.1.RDB2.conv2.weight", "RRDB_trunk.1.RDB2.conv2.bias", "RRDB_trunk.1.RDB2.conv3.weight", "RRDB_trunk.1.RDB2.conv3.bias", "RRDB_trunk.1.RDB2.conv4.weight", "RRDB_trunk.1.RDB2.conv4.bias", "RRDB_trunk.1.RDB2.conv5.weight", "RRDB_trunk.1.RDB2.conv5.bias", "RRDB_trunk.1.RDB3.conv1.weight", "RRDB_trunk.1.RDB3.conv1.bias", "RRDB_trunk.1.RDB3.conv2.weight", "RRDB_trunk.1.RDB3.conv2.bias", "RRDB_trunk.1.RDB3.conv3.weight", "RRDB_trunk.1.RDB3.conv3.bias", "RRDB_trunk.1.RDB3.conv4.weight", "RRDB_trunk.1.RDB3.conv4.bias", "RRDB_trunk.1.RDB3.conv5.weight", "RRDB_trunk.1.RDB3.conv5.bias", "RRDB_trunk.2.RDB1.conv1.weight", "RRDB_trunk.2.RDB1.conv1.bias", "RRDB_trunk.2.RDB1.conv2.weight", "RRDB_trunk.2.RDB1.conv2.bias", "RRDB_trunk.2.RDB1.conv3.weight", "RRDB_trunk.2.RDB1.conv3.bias", "RRDB_trunk.2.RDB1.conv4.weight", "RRDB_trunk.2.RDB1.conv4.bias", "RRDB_trunk.2.RDB1.conv5.weight", "RRDB_trunk.2.RDB1.conv5.bias", "RRDB_trunk.2.RDB2.conv1.weight", "RRDB_trunk.2.RDB2.conv1.bias", "RRDB_trunk.2.RDB2.conv2.weight", "RRDB_trunk.2.RDB2.conv2.bias", "RRDB_trunk.2.RDB2.conv3.weight", "RRDB_trunk.2.RDB2.conv3.bias", "RRDB_trunk.2.RDB2.conv4.weight", "RRDB_trunk.2.RDB2.conv4.bia
...


Okay, I thought since its 4 months since I last looked at this may be something changed that I'm not aware of so I looked at https://drive.google.com/drive/u/0/folders/17VYV_SoZZesU6mbxz2dMAIccSSlqLecY and tried the 'RRDB_ESRGAN_x4.pth'-model
using:
# Imports
import vapoursynth as vs
core = vs.get_core()
# Loading Plugins
from vsgan import VSGAN
vsgan_device = VSGAN("cuda")
# Loading C:/Users/Selur/Desktop/5000frames.mp4 using LWLibavSource
clip = core.lsmas.LWLibavSource(source="F:/TestClips&Co/files/5000frames.mp4", format="YUV420P10", cache=0, prefer_hw=0)
# making sure input color matrix is set as 470bg
clip = core.resize.Point(clip, matrix_in_s="470bg",range_s="limited")
# making sure frame rate is set to 25
clip = core.std.AssumeFPS(clip, fpsnum=25, fpsden=1)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)

vsgan_device.load_model(
model="I:/Hybrid/64bit/vsfilters/ResizeFilter/VSGRAN/models/RRDB_ESRGAN_x4.pth",
scale=2
)
clip = vsgan_device.run(clip=clip, chunk=True)

clip.set_output()
but that only gives me:

Error on frame 0 request:
error in LoadLibraryA
using RRDB_ESRGAN_x4_old_arch instead of RRDB_ESRGAN_x4 gives me the same errors as with 4x_xbrz_90k.

-> Does anyone know how to fix this?

Cu Selur

[ChaosKing]

You could also try https://github.com/AlphaAtlas/VapourSynth-Super-Resolution-Helper
I noticed that there are some modules that don't play nicely with the python embedded version (https://github.com/Irrational-Encoding-Wizardry/yuuno is one of such module). I think you can take a regular python intstallation and use it more or less as a "portable" version (install in a vm and then just copy the install folder to you hdd)

Maybe then it will work.

[Selur]

The strange thing is that the above did work a while back.
My main problem is I don't know why it isn't working now. :)

[poisondeathray]

You're using a 4x trained model , but specified scale=2 . Did you try scale=4 ?

[Selur]

Nope, since it worked before with 2, but I just did and still got the same error.
Error on frame 0 request:
error in LoadLibraryA

[ChaosKing]

Have you tried newer/older torch torchvision version?

[Selur]

Nope, didn't have the time to play with it that much. Only looked at it since it looked interesting and I thought about adding support for it in Hybrid. ;)
Also since that torch version worked before, I haven't really spend much thought about trying other versions.
Thought others might have played with it and directly knew what was going wrong. :)
(Don't know when I will look at this more actively since I'm quite busy with normal live atm.)

Cu Selur

[_Al_]

Installing pytorch using cpu only, which went well:
pip install torch==1.6.0+cpu torchvision==0.7.0+cpu -f https://download.pytorch.org/whl/torch_stable.html
then for final install:
pip install vsgan
I get error not having Microsoft Visual C++ v 14.0. But I have in PC 2008, 2013 and Microsoft Visual C++ 2017 which is 14.15.26706
Anyone what could go wrong?
C:\Users\xxx\AppData\Local\Programs\Python\Python37\Scripts>pip3 install vsgan
Collecting vsgan
Downloading https://files.pythonhosted.org/packages/1d/eb/369bc17433a3e8ab8bf7ab0a74ab7052f28b35ecb2de07683163d385eb2f/vsgan-1.0.8-py3-none-any.whl
Collecting vapoursynth (from vsgan)
Downloading https://files.pythonhosted.org/packages/41/a6/75c8e6c37e26641f73cd967f4c365655b206f279ddd52461f4a1b9bd1621/VapourSynth-51.zip (426kB)
100% |████████████████████████████████| 430kB 5.4MB/s
Requirement already satisfied: torch in c:\users\xxx\appdata\local\programs\python\python37\lib\site-packages (from vsgan) (1.6.0+cpu)
Requirement already satisfied: numpy in c:\users\xxx\appdata\local\programs\python\python37\lib\site-packages (from vsgan) (1.16.2)
Requirement already satisfied: future in c:\users\xxx\appdata\local\programs\python\python37\lib\site-packages (from torch->vsgan) (0.18.2)
Installing collected packages: vapoursynth, vsgan
Running setup.py install for vapoursynth ... error
Complete output from command c:\users\xxx\appdata\local\programs\python\python37\python.exe -u -c "import setuptools, tokenize;__file__='C:\\Users\
\xxx\\AppData\\Local\\Temp\\pip-install-s6avgaje\\vapoursynth\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n'
);f.close();exec(compile(code, __file__, 'exec'))" install --record C:\Users\xxx\AppData\Local\Temp\pip-record-ntdlen2q\install-record.txt --single-ver
sion-externally-managed --compile:
Found VapourSynth.dll at: C:\Program Files (x86)\VapourSynth\core64\vapoursynth.dll
running install
running build
running build_ext
skipping 'src\cython\vapoursynth.c' Cython extension (up-to-date)
building 'vapoursynth' extension
error: Microsoft Visual C++ 14.0 is required. Get it with "Microsoft Visual C++ Build Tools": https://visualstudio.microsoft.com/downloads/

----------------------------------------
Command "c:\users\xxx\appdata\local\programs\python\python37\python.exe -u -c "import setuptools, tokenize;__file__='C:\\Users\\xxxl\\AppData\\Local\\Tem
p\\pip-install-s6avgaje\\vapoursynth\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile
(code, __file__, 'exec'))" install --record C:\Users\xx\AppData\Local\Temp\pip-record-ntdlen2q\install-record.txt --single-version-externally-managed
--compile" failed with error code 1 in C:\Users\xxx\AppData\Local\Temp\pip-install-s6avgaje\vapoursynth\

[Cary Knoop]

Try to uninstall VC 2017 and then install VC 2015 Build Tools and then install VC 2017 again.

[Selur]

btw. has anyone compiled and tried https://github.com/Sg4Dylan/vapoursynth-fsrcnn-ncnn-vulkan ?

[ReinerSchweinlin]

No, I haven´t,

maybe this shader implementation can be used?
https://github.com/igv/FSRCNN-TensorFlow/releases

[Selur]

Since I was testing some stuff here's how on can create portable VSGAN (https://github.com/rlaPHOENiX/VSGAN) version:


Download latest Vapoursynth portabe Fatpack (https://github.com/theChaosCoder/vapoursynth-portable-FATPACK/releases) and extract it to a folder
Delete the links an folders inside the VapourSynth64 aside from VapourSynth64 and the Scripts folder (optional)
Open a Windows command prompt and change into the VapourSynth64/VapourSynth64-folder.
Call `python.exe -m pip install --upgrade pip" to update pip` to update pip
Call `python.exe -m pip install torch==1.8.1+cu111 torchvision==0.9.1+cu111 torchaudio===0.8.1 -f https://download.pytorch.org/whl/torch_stable.html` to install pytorch
For cuda10 use `python.exe -m pip install torch==1.8.1+cu102 torchvision==0.9.1+cu102 torchaudio===0.8.1 -f https://download.pytorch.org/whl/torch_stable.html` to install pytorch
Call `python.exe -m pip install pyd2v pymediainfo vsgan" to install vsgan`
Close Windows Command Prompt.

now you got a 6GB folder containing a portable Vapoursynth (~560MB) with VSGAN (~5.4GB).

In case you want to try out VSGAN:

Download some models
I went to https://upscale.wiki/wiki/Model_Database and then to Anime Oldies Alternative (https://drive.google.com/drive/folders/1ldwajXL50uC7PCS63B4Wato6Dnk-svNL) and downloaded the pth files and extract them somewhere.
Open VapourSynth64Portable\VapourSynth64\vsedit.exe and write a script like:

# Imports
import vapoursynth as vs
core = vs.get_core()
# Loading Plugins
from vsgan import VSGAN

# Loading F:\TestClips&Co\files/5000frames.mp4 using LWLibavSource
clip = core.lsmas.LWLibavSource(source="F:/TestClips&Co/files/5000frames.mp4", format="YUV420P10", cache=0, prefer_hw=0)
# making sure input color matrix is set as 470bg
clip = core.resize.Point(clip, matrix_in_s="470bg",range_s="limited")
# making sure frame rate is set to 25
clip = core.std.AssumeFPS(clip, fpsnum=25, fpsden=1)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)

# creating a VSGAN instance
vsgan = VSGAN("cuda")
# setting model
model = "C:/Users/Selur/Desktop/Vapoursynth64portable/vsgan_models/PSNR Pretrain Models/4xPSNR.pth"
vsgan.load_model(model)
# convert clip color space from YUV420P8 to RGB24 for vsgan
clip = core.resize.Bicubic(clip=clip, format=vs.RGB24, matrix_in_s="470bg", range_s="limited")
clip = vsgan.run(clip=clip)

clip.set_output()
(paths need to be adjusted to your setup)
start the preview.

Cu Selur

[ReinerSchweinlin]

:) I hope I didn´t talk you into too much work :) Really appreciate this addition, thanx !!

My amateur suspection is, that the ...GAN folders should be somehow reduceable in size (since some other similar implementations come in smaller packages), but my skills are far from doing that :)

[Selur]

Yeah, my guess too is that one probably can delete some of the large cuda files inside VapourSynth64\Lib\site-packages\torch\lib since probably not all are needed, but haven't tested and probably won't spend time on it.
If someone has the time and motivation to test which dlls are really needed let me know. :)

Cu Selur

[Selur]

Just wondering: Can the .params&.json files from https://github.com/WolframRhodium/Super-Resolution-Zoo
be used in VSGAN somehow?

Cu Selur

[Zarxrax]

I have installed VSGAN yesterday on windows, and the installation seems to have gone smoothly, but when I try the example script from the documentation, I receive this error:
raise EnvironmentError(f"VSGAN: Either NVIDIA CUDA or the device ({device}) isn't available.")
OSError: VSGAN: Either NVIDIA CUDA or the device (cuda) isn't available.

Any ideas what this error means? My video card is GTX 750 ti, which I know is quite old, but I am able to use other implementations of ESRGAN on it. I'm not sure if this is telling me my video card is not compatible, or something else.

[Selur]

Wild guess would be a driver or rights issue.
Here (https://forum.selur.net/thread-1858.html) is how I set it up.

[ChaosKing]

You can check cuda support with gpuz.

[Zarxrax]

Wild guess would be a driver or rights issue.
Here (https://forum.selur.net/thread-1858.html) is how I set it up.

It was the torch version. I uninstalled torch and then installed an older version using the command you provided, and it works now.

[takla]

Would anyone be so kind to post some benchmark numbers for 1080p to 2160p with one model for anime and one model for real life? Please name your gpu and time in seconds per frame. Thanks.

[Selur]

Can't do 1080p to 2160p with my Geforce GTX 1070ti (8GB VRAM) even when using a the 2x_PSNR model, when using one of the models with a 4x resize factor my guess it that you at least need a 30xx card with 12GB or more VRAM.

[poisondeathray]

Can't do 1080p to 2160p with my Geforce GTX 1070ti (8GB VRAM) even when using a the 2x_PSNR model, when using one of the models with a 4x resize factor my guess it that you at least need a 30xx card with 12GB or more VRAM.

Yes it would help if PRAGMA introduced the tiling script portion that some of the ESRGAN scripts and some GUI's use .

When using decent size margins, you don't see any seams or artifacts (but they are "microscopically" detected with amplified differences)