Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 20th July 2020, 13:39   #1  |  Link
Alexkral
Registered User
 
Join Date: Oct 2018
Posts: 214
Avisynth AiUpscale v1.0.1

This is a script to test some Super-resolution convolutional neural networks implemented as hlsl pixel shaders, through Avisynth Shader.

Download

Github repository with explanation on its usage:

https://github.com/Alexkral/AviSynthAiUpscale

I became interested in this after reading some posts in which it was assumed that implementing the FSRCNNX shaders using AviSynth Shader was not possible, because they were written in GLSL. After all, GLSL and HLSL are not that different, and the mpv hook format doesn't change this. So I was able to do it without too much trouble, also with the other FSRCNN shaders by igv and haasn. That is what I thought to share at the time, but since igv provides the code not only to train the networks, but also to convert the weights into mpv hooks, I became interested in training different models. This also I think has worked very well (quite boring I must say, except for the part related to experimentation), and I could have shared the result but, how not, I became interested in adding other kinds of networks, which also allowed me to do other kinds of experiments. Finally, I realized that using a specific dataset for training allowed to obtain better results for specific content, which made me repeat the entire training process for the "Illustration" models.

So it's been a while since I started with this, every single part has needed to be redone multiple times, lots of training and retraining, having to deal with things that I didn't even know existed, such as the chroma positioning when resizing or some AviSynth Shader specific problems, so I can't be happier to have finished it at last.

Special thanks to MysteryX, igv and Shiandow for writing such amazing code without which this couldn't be possible.

Here are some results on performance tested with AVSMeter for a YV12 clip with default settings (chroma upscaled with Lanczos). GPU is a Nvidia GTX 1080:

Code:
1080p -> 2160p

AiUpscale Fast =	30.060 fps
AiUpscale Medium =	15.720 fps
AiUpscale VDSR =	 7.066 fps
AiUpscale HQ =		 7.174 fps

540p -> 1080p

AiUpscale Fast =	107.10 fps
AiUpscale Medium =	 57.57 fps
AiUpscale VDSR =	 26.25 fps
AiUpscale HQ =		 26.89 fps

Last edited by Alexkral; 25th July 2020 at 21:44. Reason: update
Alexkral is offline   Reply With Quote
Old 20th July 2020, 15:12   #2  |  Link
real.finder
Registered User
 
Join Date: Jan 2012
Location: Mesopotamia
Posts: 1,890
thanks!

I did try it and it work with server that don't has gpu, is this ok? but anyway it a good thing

edit: FFT3DGPU also work! seems directx things can work with cpu only
__________________
See My Avisynth Stuff

Last edited by real.finder; 20th July 2020 at 15:17.
real.finder is offline   Reply With Quote
Old 22nd July 2020, 02:02   #3  |  Link
goorawin
Registered User
 
Join Date: Feb 2012
Posts: 72
Your AiUpscale looks very interesting, so I'm interested to test it against other upscaling options.
You mention that it is able to adjust the target width and height.
The 4x Upscaling looks good.
If that is the case could you please provide a sample script that upscales from 720x576SD to 1920x1080HD and HD to UHD.
goorawin is offline   Reply With Quote
Old 22nd July 2020, 02:44   #4  |  Link
Alexkral
Registered User
 
Join Date: Oct 2018
Posts: 214
Quote:
Originally Posted by goorawin View Post
If that is the case could you please provide a sample script that upscales from 720x576SD to 1920x1080HD and HD to UHD.
Code:
AiUpscale (clip, Luma = "VDSR", Width = 1920, Height = 1080)
This will scale the 720x576 clip to 1920x1080. Since the height scaling ratio is 1.875, only the VDSR model can be used. When the ratio is at least 2 (and less than or equal to 4) for both width and height, the HQ and HQ Sharp models can also be used.

For HD to UHD it is the same.
Alexkral is offline   Reply With Quote
Old 22nd July 2020, 05:07   #5  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 4,372
Thanks for this. I'm having problems getting it working


Shader: Failed to open pixel shader Shaders\Photo\Medium\2x\feature_1.cso
AiUpscale.avsi, line 393
line 217
line 218
line 239


I think it's some path related issue. Win8

AiUpscale.avsi and Shaders folder are in the avisynth+ plugins directory. AviSynthShader works ok for me separately .

(The .avsi says "AviSinth+ plugins folder", but I'm assuming it's obviously a typo, the github page says "AviSynth+ plugins directory", but maybe you can edit it for the next release)

Does mpv need to be installed ? I installed it anyways but still same error; I normally use a "portable" version, but I might uninstall if you technically don't need it. I have mpv.conf in AppData\Roaming\mpv and edited some entries .

profile = gpu-hq
glsl-shader = "<PATH TO HOOK>"

Where the actual path of mpv.hooks is located; and I tried different things such as specifying the directory above (container mpv.hooks folder), and "/" vs. "" slashes , I tried escaping colon, tried ' instead of " , I tried specifying specific .glsl files in the mpv.hooks directory

Last edited by poisondeathray; 22nd July 2020 at 05:14.
poisondeathray is offline   Reply With Quote
Old 22nd July 2020, 06:19   #6  |  Link
Alexkral
Registered User
 
Join Date: Oct 2018
Posts: 214
That's weird, the path seems correct.

AiUpscale.avsi and the Shaders folder should be in the same folder as AviSynth Shader, on my PC that is the AviSynth+\plugins64+ folder.

I have only tested it on Win7 and Win10 x64, but I don't see any reason why it won't work with x86.

Other than that you could try opening VirtualDub2 with administrator privileges to see if that makes any difference, and then open the script.

mpv is only needed to use the hooks, if you extract them in the same folder as mpv.conf, you would have to edit it for example like this:

profile = gpu-hq
glsl-shader = "~~/AiUpscale_Fast_2x_Photo.glsl"

Last edited by Alexkral; 22nd July 2020 at 06:25.
Alexkral is offline   Reply With Quote
Old 22nd July 2020, 17:20   #7  |  Link
real.finder
Registered User
 
Join Date: Jan 2012
Location: Mesopotamia
Posts: 1,890
I think the script need to use ScriptDir() http://avisynth.nl/index.php/Interna...ions#ScriptDir
__________________
See My Avisynth Stuff
real.finder is offline   Reply With Quote
Old 22nd July 2020, 19:33   #8  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 4,372
Quote:
Originally Posted by Alexkral View Post

I have only tested it on Win7 and Win10 x64, but I don't see any reason why it won't work with x86.
Sorry it is x64

My plugins are in a custom directory location, not default (registry entries are ok , everything else works, other scripts/plugins). Could that be related?


Quote:
Other than that you could try opening VirtualDub2 with administrator privileges to see if that makes any difference, and then open the script.
no difference

Quote:
mpv is only needed to use the hooks, if you extract them in the same folder as mpv.conf, you would have to edit it for example like this:

profile = gpu-hq
glsl-shader = "~~/AiUpscale_Fast_2x_Photo.glsl"

I tried moving them and this configuration too, same error
poisondeathray is offline   Reply With Quote
Old 22nd July 2020, 21:16   #9  |  Link
Alexkral
Registered User
 
Join Date: Oct 2018
Posts: 214
Just to be clear, the mpv hooks are not required at all for AviSynth AiUpscale to work, and the path in mpv.conf is also not used. They are included only to be used with mpv.

Please change line 109 in AiUpscale.avsi to this and report if this fixes the problem:

Code:
Global Path = ".\Shaders\"
Alexkral is offline   Reply With Quote
Old 22nd July 2020, 23:31   #10  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 4,372
Quote:
Originally Posted by Alexkral View Post
Just to be clear, the mpv hooks are not required at all for AviSynth AiUpscale to work, and the path in mpv.conf is also not used. They are included only to be used with mpv.
Got it, thanks.


Quote:
Please change line 109 in AiUpscale.avsi to this and report if this fixes the problem:

Code:
Global Path = ".\Shaders\"
No , similar error

Shader: Failed to open pixel shader .\Shaders\Photo\Medium\2x\feature_1.cso
(rest of message is the same)
poisondeathray is offline   Reply With Quote
Old 23rd July 2020, 00:44   #11  |  Link
Alexkral
Registered User
 
Join Date: Oct 2018
Posts: 214
Sorry but I can't figure out how to fix this since AviSynth doesn't provide anything to find out the path to the plugins directory. It should work if you specify the full path to the Shaders folder (ended with a backslash and in quotes) on line 109, but obviously that's not going to help others with the same problem.
Alexkral is offline   Reply With Quote
Old 23rd July 2020, 00:52   #12  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 4,372
Quote:
Originally Posted by Alexkral View Post
Sorry but I can't figure out how to fix this since AviSynth doesn't provide anything to find out the path to the plugins directory. It should work if you specify the full path to the Shaders folder (ended with a backslash and in quotes) on line 109, but obviously that's not going to help others with the same problem.
Yes, this works for me. Was it just me having the problem ?

Global Path = "<full path...\Shaders\>"


I don't understand why either, because I'm pretty sure other working scripts use that variable method too.

Thanks again
poisondeathray is offline   Reply With Quote
Old 23rd July 2020, 03:36   #13  |  Link
real.finder
Registered User
 
Join Date: Jan 2012
Location: Mesopotamia
Posts: 1,890
Quote:
Originally Posted by Alexkral View Post
Sorry but I can't figure out how to fix this since AviSynth doesn't provide anything to find out the path to the plugins directory. It should work if you specify the full path to the Shaders folder (ended with a backslash and in quotes) on line 109, but obviously that's not going to help others with the same problem.
as I said here https://forum.doom9.org/showthread.p...11#post1919111 anyway see my Pull request
__________________
See My Avisynth Stuff
real.finder is offline   Reply With Quote
Old 23rd July 2020, 04:26   #14  |  Link
Alexkral
Registered User
 
Join Date: Oct 2018
Posts: 214
Quote:
Originally Posted by real.finder View Post
as I said here https://forum.doom9.org/showthread.p...11#post1919111 anyway see my Pull request
Thanks but this doesn't work since ScriptDir() returns the path of the loaded script, not the .avsi
Alexkral is offline   Reply With Quote
Old 23rd July 2020, 06:01   #15  |  Link
real.finder
Registered User
 
Join Date: Jan 2012
Location: Mesopotamia
Posts: 1,890
Quote:
Originally Posted by Alexkral View Post
Thanks but this doesn't work since ScriptDir() returns the path of the loaded script, not the .avsi
you sure? it's work fine here from the avsi
__________________
See My Avisynth Stuff
real.finder is offline   Reply With Quote
Old 23rd July 2020, 06:03   #16  |  Link
real.finder
Registered User
 
Join Date: Jan 2012
Location: Mesopotamia
Posts: 1,890
poisondeathray, can you try it? https://github.com/realfinder/AviSyn...AiUpscale.avsi
__________________
See My Avisynth Stuff
real.finder is offline   Reply With Quote
Old 23rd July 2020, 06:25   #17  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 4,372
Quote:
Originally Posted by real.finder View Post
Different error message. The current directory location of the .avs script is location is appended (but Shaders directory actually isn't there)

Shader: Failed to open pixel shader F:\test\Shaders\Photo\Medium\2x\feature_1.cso

If I copy/paste Shaders folder to that F:\test folder, it works
poisondeathray is offline   Reply With Quote
Old 23rd July 2020, 06:32   #18  |  Link
real.finder
Registered User
 
Join Date: Jan 2012
Location: Mesopotamia
Posts: 1,890
Quote:
Originally Posted by poisondeathray View Post
Different error message. The current directory location of the .avs script is location is appended (but Shaders directory actually isn't there)

Shader: Failed to open pixel shader F:\test\Shaders\Photo\Medium\2x\feature_1.cso

If I copy/paste Shaders folder to that F:\test folder, it works
but it work fine for me, maybe because I made it load from subfolder by another avsi

anyway

make an avsi named AviSynthPluginsDir.avsi and put this in it

Code:
global AviSynthPluginsDir = VersionNumber() < 2.60 ? RT_GetWorkingDir() : ScriptDir()
and put it in your plugins folder

and replace ScriptDir() with AviSynthPluginsDir in AiUpscale.avsi
__________________
See My Avisynth Stuff

Last edited by real.finder; 23rd July 2020 at 06:38.
real.finder is offline   Reply With Quote
Old 23rd July 2020, 06:38   #19  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 4,372
r.f - it works.

I deleted the copied shaders folder, find/replaced all 4 entries in AiUpscale.avsi as per instructions

Thanks to you both
poisondeathray is offline   Reply With Quote
Old 23rd July 2020, 14:48   #20  |  Link
Alexkral
Registered User
 
Join Date: Oct 2018
Posts: 214
Some tests here:

https://forum.doom9.org/showthread.p...65#post1919165
Alexkral is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 19:04.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.