Avisynth AiUpscale v1.2.0

[Alexkral]

This is a script to test some Super-resolution convolutional neural networks implemented as hlsl pixel shaders, through Avisynth Shader.

Download

Github repository with explanation on its usage:

https://github.com/Alexkral/AviSynthAiUpscale

I became interested in this after reading some posts in which it was assumed that implementing the FSRCNNX shaders using AviSynth Shader was not possible, because they were written in GLSL. After all, GLSL and HLSL are not that different, and the mpv hook format doesn't change this. So I was able to do it without too much trouble, also with the other FSRCNN shaders by igv and haasn. That is what I thought to share at the time, but since igv provides the code not only to train the networks, but also to convert the weights into mpv hooks, I became interested in training different models. This also I think has worked very well (quite boring I must say, except for the part related to experimentation), and I could have shared the result but, how not, I became interested in adding other kinds of networks, which also allowed me to do other kinds of experiments. Finally, I realized that using a specific dataset for training allowed to obtain better results for specific content, which made me repeat the entire training process for the "Illustration" models.

So it's been a while since I started with this, every single part has needed to be redone multiple times, lots of training and retraining, having to deal with things that I didn't even know existed, such as the chroma positioning when resizing or some AviSynth Shader specific problems, so I can't be happier to have finished it at last.

Special thanks to MysteryX, igv and Shiandow for writing such amazing code without which this couldn't be possible.

Here are some results on performance tested with AVSMeter for a YV12 clip with default settings (chroma upscaled with Lanczos). GPU is a Nvidia GTX 1080:

1080p -> 2160p

AiUpscale Fast =	 38.47 fps
AiUpscale Medium =	 25.30 fps
AiUpscale HQ =		 17.16 fps

[real.finder]

thanks!

I did try it and it work with server that don't has gpu, is this ok? but anyway it a good thing

edit: FFT3DGPU also work! seems directx things can work with cpu only

[goorawin]

Your AiUpscale looks very interesting, so I'm interested to test it against other upscaling options.
You mention that it is able to adjust the target width and height.
The 4x Upscaling looks good.
If that is the case could you please provide a sample script that upscales from 720x576SD to 1920x1080HD and HD to UHD.

[Alexkral]

Originally Posted by goorawin

If that is the case could you please provide a sample script that upscales from 720x576SD to 1920x1080HD and HD to UHD.

AiUpscale (clip, Luma = "VDSR", Width = 1920, Height = 1080)

This will scale the 720x576 clip to 1920x1080. Since the height scaling ratio is 1.875, only the VDSR model can be used. When the ratio is at least 2 (and less than or equal to 4) for both width and height, the HQ and HQ Sharp models can also be used.

For HD to UHD it is the same.

[poisondeathray]

Thanks for this. I'm having problems getting it working


Shader: Failed to open pixel shader Shaders\Photo\Medium\2x\feature_1.cso
AiUpscale.avsi, line 393
line 217
line 218
line 239


I think it's some path related issue. Win8

AiUpscale.avsi and Shaders folder are in the avisynth+ plugins directory. AviSynthShader works ok for me separately .

(The .avsi says "AviSinth+ plugins folder", but I'm assuming it's obviously a typo, the github page says "AviSynth+ plugins directory", but maybe you can edit it for the next release)

Does mpv need to be installed ? I installed it anyways but still same error; I normally use a "portable" version, but I might uninstall if you technically don't need it. I have mpv.conf in AppData\Roaming\mpv and edited some entries .

profile = gpu-hq
glsl-shader = "<PATH TO HOOK>"

Where the actual path of mpv.hooks is located; and I tried different things such as specifying the directory above (container mpv.hooks folder), and "/" vs. "" slashes , I tried escaping colon, tried ' instead of " , I tried specifying specific .glsl files in the mpv.hooks directory

[Alexkral]

That's weird, the path seems correct.

AiUpscale.avsi and the Shaders folder should be in the same folder as AviSynth Shader, on my PC that is the AviSynth+\plugins64+ folder.

I have only tested it on Win7 and Win10 x64, but I don't see any reason why it won't work with x86.

Other than that you could try opening VirtualDub2 with administrator privileges to see if that makes any difference, and then open the script.

mpv is only needed to use the hooks, if you extract them in the same folder as mpv.conf, you would have to edit it for example like this:

profile = gpu-hq
glsl-shader = "~~/AiUpscale_Fast_2x_Photo.glsl"

[real.finder]

I think the script need to use ScriptDir() http://avisynth.nl/index.php/Interna...ions#ScriptDir

[poisondeathray]

Originally Posted by Alexkral

I have only tested it on Win7 and Win10 x64, but I don't see any reason why it won't work with x86.

Sorry it is x64

My plugins are in a custom directory location, not default (registry entries are ok , everything else works, other scripts/plugins). Could that be related?

Other than that you could try opening VirtualDub2 with administrator privileges to see if that makes any difference, and then open the script.

no difference

mpv is only needed to use the hooks, if you extract them in the same folder as mpv.conf, you would have to edit it for example like this:

profile = gpu-hq
glsl-shader = "~~/AiUpscale_Fast_2x_Photo.glsl"

I tried moving them and this configuration too, same error

[Alexkral]

Just to be clear, the mpv hooks are not required at all for AviSynth AiUpscale to work, and the path in mpv.conf is also not used. They are included only to be used with mpv.

Please change line 109 in AiUpscale.avsi to this and report if this fixes the problem:

Global Path = ".\Shaders\"

[poisondeathray]

Originally Posted by Alexkral

Just to be clear, the mpv hooks are not required at all for AviSynth AiUpscale to work, and the path in mpv.conf is also not used. They are included only to be used with mpv.

Got it, thanks.

Please change line 109 in AiUpscale.avsi to this and report if this fixes the problem:

Global Path = ".\Shaders\"

No , similar error

Shader: Failed to open pixel shader .\Shaders\Photo\Medium\2x\feature_1.cso
(rest of message is the same)

[Alexkral]

Sorry but I can't figure out how to fix this since AviSynth doesn't provide anything to find out the path to the plugins directory. It should work if you specify the full path to the Shaders folder (ended with a backslash and in quotes) on line 109, but obviously that's not going to help others with the same problem.

[poisondeathray]

Originally Posted by Alexkral

Sorry but I can't figure out how to fix this since AviSynth doesn't provide anything to find out the path to the plugins directory. It should work if you specify the full path to the Shaders folder (ended with a backslash and in quotes) on line 109, but obviously that's not going to help others with the same problem.

Yes, this works for me. Was it just me having the problem ?

Global Path = "<full path...\Shaders\>"


I don't understand why either, because I'm pretty sure other working scripts use that variable method too.

Thanks again

[real.finder]

Originally Posted by Alexkral

Sorry but I can't figure out how to fix this since AviSynth doesn't provide anything to find out the path to the plugins directory. It should work if you specify the full path to the Shaders folder (ended with a backslash and in quotes) on line 109, but obviously that's not going to help others with the same problem.

as I said here https://forum.doom9.org/showthread.p...11#post1919111 anyway see my Pull request

[Alexkral]

Originally Posted by real.finder

as I said here https://forum.doom9.org/showthread.p...11#post1919111 anyway see my Pull request

Thanks but this doesn't work since ScriptDir() returns the path of the loaded script, not the .avsi

[real.finder]

Originally Posted by Alexkral

Thanks but this doesn't work since ScriptDir() returns the path of the loaded script, not the .avsi

you sure? it's work fine here from the avsi

[real.finder]

poisondeathray, can you try it? https://github.com/realfinder/AviSyn...AiUpscale.avsi

[poisondeathray]

Originally Posted by real.finder

poisondeathray, can you try it? https://github.com/realfinder/AviSyn...AiUpscale.avsi

Different error message. The current directory location of the .avs script is location is appended (but Shaders directory actually isn't there)

Shader: Failed to open pixel shader F:\test\Shaders\Photo\Medium\2x\feature_1.cso

If I copy/paste Shaders folder to that F:\test folder, it works

[real.finder]

Originally Posted by poisondeathray

Different error message. The current directory location of the .avs script is location is appended (but Shaders directory actually isn't there)

Shader: Failed to open pixel shader F:\test\Shaders\Photo\Medium\2x\feature_1.cso

If I copy/paste Shaders folder to that F:\test folder, it works

but it work fine for me, maybe because I made it load from subfolder by another avsi

anyway

make an avsi named AviSynthPluginsDir.avsi and put this in it

global AviSynthPluginsDir = VersionNumber() < 2.60 ? RT_GetWorkingDir() : ScriptDir()

and put it in your plugins folder

and replace ScriptDir() with AviSynthPluginsDir in AiUpscale.avsi

[poisondeathray]

r.f - it works.

I deleted the copied shaders folder, find/replaced all 4 entries in AiUpscale.avsi as per instructions

Thanks to you both

[Alexkral]

Some tests here:

https://forum.doom9.org/showthread.p...65#post1919165