Avisynth AiUpscale v1.2.0

[ReinerSchweinlin]

Thanx for this, it looks really interesting :)
Just set up a completeley fresh win10/64 (2004), installed avisynth+ and Avisynthshader, put all the files in ".....\avisynth+\plugins+", loaded up Virtualdub64..

First, I had to get rid of shader.dll, otherwise VD2 was complaining about "not loading 32bit plugin into 64bit avisynth"...

Now I get. "There is no function named 'convertToShader'", refering to 4 lines in aiupscaler.avsi..

Can you help me on this one please?

[Alexkral]

For VirtualDub2 x64 you need AviSynth Shader x64 and AiUpscale in the plugins64 folder, default is \AviSynth+\plugins64+

[gmail123]

any plan to add denoise filter?

[markiemarcus]

Thanx for this, it looks really interesting :)
Just set up a completeley fresh win10/64 (2004), installed avisynth+ and Avisynthshader, put all the files in ".....\avisynth+\plugins+", loaded up Virtualdub64..

First, I had to get rid of shader.dll, otherwise VD2 was complaining about "not loading 32bit plugin into 64bit avisynth"...

Now I get. "There is no function named 'convertToShader'", refering to 4 lines in aiupscaler.avsi..

Can you help me on this one please?

I had the same issue. Was solved by specifying the absolute directory of the Shaders folder in line 110 of AIUpscale. For example:

Global Path = "C:\Program Files (x86)\AviSynth+\plugins64+\Shaders"

There's no need to delete shader.dll.

[Alexkral]

any plan to add denoise filter?

There are already plenty of denoise filters in AviSynth that can be used in a better way than an embedded filter.

[ReinerSchweinlin]

For VirtualDub2 x64 you need AviSynth Shader x64 and AiUpscale in the plugins64 folder, default is \AviSynth+\plugins64+
Thanx. I copied the content of avisynthsader 1.6.6 zip into plugins64+, as well as the content of aiupscale. That´s what I did before what gave the error. I missed be missing something else.

[ReinerSchweinlin]

I had the same issue. Was solved by specifying the absolute directory of the Shaders folder in line 110 of AIUpscale. For example:

Global Path = "C:\Program Files (x86)\AviSynth+\plugins64+\Shaders"

There's no need to delete shader.dll.
Thank you. Changed the path, but still get:

"canot load 32.. in 64..: "c:/program..... /avisynth+/plugins64+/shader.dll".

Somehow it always tries to load shader.dll

If I remove shader.dll from the plugins+ folder, leaving everythnig else as it is (including shader-x64.dll), I get:

"avisynth open failure: Script Erro: There is no function named "converttoshader", followed by the mentioning of four lines of the aiupscale.avsi skript...

seems the error is the same as before

[poisondeathray]

If I remove shader.dll from the plugins+ folder, leaving everythnig else as it is (including shader-x64.dll), I get:

"avisynth open failure: Script Erro: There is no function named "converttoshader", followed by the mentioning of four lines of the aiupscale.avsi skript...

seems the error is the same as before

shader-x64.dll should be all you need to get rid of that error. It looks like it's not autoloading. Try moving it out of the autoload directory and explicitly loading with LoadPlugin()

If it still does not work, try cleaning out the plugins directory. Maybe some conflicting .dlls

[gmail123]

please add vdsl/hq/hq hard.glsl,thx.

[gmail123]

import to vs:thanks:

[SaurusX]

Fantastic work on this. The results speak for themselves. Definitely improved quality when compared to SuperResXBR.

[Alexkral]

Thanks! I have managed to improve the VDSR models considerably, I will make a new release in the next few days.

[ReinerSchweinlin]

Thanks! I have managed to improve the VDSR models considerably, I will make a new release in the next few days.

Nice :) Thanx for your efforts !

[Alexkral]

AviSynth AiUpscale v1.1.0 (https://github.com/Alexkral/AviSynthAiUpscale/releases/tag/v1.1.0)

Changes:
- Improved the VDSR models.
- Added new parameter to set the output bit depth.
- When the output is less than 16 bit, the conversion is now done on the GPU with ordered dithering.
- Alpha planes are now upscaled with Lanczos instead of being removed.

The VDSR models have been substantially improved for all scale factors after adjusting some training options. About the dithering, I honestly don't remember what I read that made me think that it wasn't good for this, but after doing some tests it is clear that this is not the case. So doing the conversion now on the GPU should mean a nice speed-up, more noticeable for the faster models.

Also, I have found that at least on my system there is a benefit when using four threads, so try using Prefetch(4) at the end of your scripts.

[gmail123]

vdsr model:
AiUpscale1.01--SSIM Mean Y:0.9707451 (15.338db)
--PSNR Mean Y:40.994 U:49.153 V:49.248 Avg:42.249

AiUpscale1.10-- SSIM Mean Y:0.9709488 (15.368db)
--PSNR Mean Y:40.976 U:49.103 V:49.290 Avg:42.233

[Alexkral]

Set5 - Photo

1.5x

Model PSNR SSIM
____________ ________________ _________________

'VDSR-1.1.0' 40.303107806444 0.977076850531857
'VDSR-1.0.1' 40.0231705258812 0.976284959358199

2x

Model PSNR SSIM
____________ ________________ _________________

'VDSR-1.1.0' 37.0541978031766 0.957678515410715
'VDSR-1.0.1' 36.6700956028294 0.955898000384579

2.5x

Model PSNR SSIM
____________ ________________ _________________

'VDSR-1.1.0' 34.7290845636078 0.936933199722221
'VDSR-1.0.1' 34.3833689195523 0.933502938036026

3x
Model PSNR SSIM
____________ ________________ _________________

'VDSR-1.1.0' 33.0708644076159 0.914888194061316
'VDSR-1.0.1' 32.7252438666527 0.910143847730622

3.5x

Model PSNR SSIM
____________ ________________ _________________

'VDSR-1.1.0' 31.7451881124588 0.891913536377046
'VDSR-1.0.1' 31.4385026528417 0.885549575366799

4x

Model PSNR SSIM
____________ ________________ _________________

'VDSR-1.1.0' 30.6333993713215 0.868059737258679
'VDSR-1.0.1' 30.3843028542906 0.861332075381644


Set5 - Illustration

1.5x

Model PSNR SSIM
____________ ________________ _________________

'VDSR-1.1.0' 40.3110321290525 0.976887220812664
'VDSR-1.0.1' 40.1070682034134 0.976346287794712

2x

Model PSNR SSIM
____________ ________________ _________________

'VDSR-1.1.0' 36.9362257004149 0.956827635790212
'VDSR-1.0.1' 36.631527053702 0.95544423942317

2.5x

Model PSNR SSIM
____________ ________________ _________________

'VDSR-1.1.0' 34.3816642584093 0.933170040816044
'VDSR-1.0.1' 34.1517733908771 0.931394422143814

3x

Model PSNR SSIM
____________ ________________ _________________

'VDSR-1.1.0' 32.8355848241999 0.911281952451763
'VDSR-1.0.1' 32.5152354802607 0.907160504583839

3.5x

Model PSNR SSIM
____________ ________________ _________________

'VDSR-1.1.0' 31.4385026528417 0.885549575366799
'VDSR-1.0.1' 31.1604700679502 0.881199445944243

4x

Model PSNR SSIM
____________ ________________ _________________

'VDSR-1.1.0' 30.3683034245151 0.861417745548344
'VDSR-1.0.1' 30.1770642431373 0.857356189888384

[Shinkiro]

ConvertBits(16)
AiUpscale(Factor=4, Luma="HQ", Chroma="Lanczos", Width = 1920, Height = 1080, Double=0, Mode="Illustration", CPlace="MPEG2", OutDepth=16)
http://i114.fastpic.ru/big/2021/0113/17/61c4341134e378fddf63a48029147117.png
does not accept yuv420p16 as input I cannot understand why

[Alexkral]

Neither do I, it works fine here. That should only show up if the input is YV411 or 32 bits.

[kopmjj]

Hi this looks like an interesting upscale feature
But I'm having trouble making this work, I put the avs and the other folder in plugins 64 but I dont know the basic script.

I'm trying to upscale DVD to 4k
Can someone show me a sample script to enter in my avspmod?
Just cant get my head round it. Thanks.

[Alexkral]

Which problem you have? You just need to use a factor equal to 4, and then use some interpolation method to get to 4K, something like this:

AiUpscale(Factor = 4)
LanczosResize(3840, 2160)

Note however that this will change the aspect ratio.

[kopmjj]

Which problem you have? You just need to use a factor equal to 4, and then use some interpolation method to get to 4K, something like this:

AiUpscale(Factor = 4)
LanczosResize(3840, 2160)

Note however that this will change the aspect ratio.

hi thanks im getting the following error image attached.

what am i missing?

[kopmjj]

ok looks like the photo upload doesnt work heres my script below, the error i get is there is no function named convertToshader
and the following error in aiupscale avsi in lines
341
217
228
239

DirectShowSource("C:\Users\king\Desktop\VID-20210125-WA0012.mp4")


Import("C:\Program Files (x86)\AviSynth+\plugins64+\AiUpscale.avsi")
LoadPlugin("C:\Program Files (x86)\AviSynth+\plugins64\AddGrainC.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth+\plugins64\mvtools2.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth+\plugins64\mt_masktools.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth+\plugins64\RgTools.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth+\plugins64\dfttest.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth+\plugins64\aWarpsharpMT.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth+\plugins64\nnedi3.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth+\plugins64\TDeint.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth+\plugins64\FFT3dGPU.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth+\plugins64\FFT3dGPU.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth+\plugins64\fft3dfilter.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth+\plugins64\gradfun2db.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth+\plugins64\flash3kyuu_deband.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth+\plugins64\Deblock-x64.dll")

Import("C:\Program Files (x86)\AviSynth+\plugins64\AiUpscale.avsi")

Converttorgb()

AiUpscale(Factor = 4)
LanczosResize(3840, 2160)

[poisondeathray]

the error i get is there is no function named convertToshader


You need AviSynthShader, the .dll and .avsi

It's listed under the requirements
https://github.com/Alexkral/AviSynthAiUpscale
https://github.com/mysteryx93/AviSynthShader/releases

[kopmjj]

You need AviSynthShader, the .dll and .avsi

It's listed under the requirements
https://github.com/Alexkral/AviSynthAiUpscale
https://github.com/mysteryx93/AviSynthShader/releases

thank you this worked.
now need to find out whats the best way to upscale dvd to 4k need to see what settings i need to enter.

is there anything else i need besides AiUpscale(Factor = 4) i will do the denoising and sharpening before ai upscale.

[Alexkral]

You will probably find the result too sharp if you use the HQ Sharp model after sharpening the image, you could also try it instead. There are some performance results in the first post of this thread, and everything else is explained in the Github readme.

[kopmjj]

You will probably find the result too sharp if you use the HQ Sharp model after sharpening the image, you could also try it instead. There are some performance results in the first post of this thread, and everything else is explained in the Github readme.

hi thanks for this, but no matter what i try my video is getting to sharp, doesnt look natural all i done was deinterlace the dvd, before using ai upscale, is there a setting to tone down the sharpness? tried fast setting but not much difference, also tried vdsr but no luck.

[Alexkral]

Try a factor = 3 or 2 and then some interpolation.

[ReinerSchweinlin]

hi thanks for this, but no matter what i try my video is getting to sharp, doesnt look natural all i done was deinterlace the dvd, before using ai upscale, is there a setting to tone down the sharpness? tried fast setting but not much difference, also tried vdsr but no luck.
You could try a slight blurring afterwards, as well as adding some random noise - it helps to get a more natural look sometimes.

[Alexkral]

AviSynth AiUpscale v1.2.0 (https://github.com/Alexkral/AviSynthAiUpscale/releases/tag/1.2.0)

Changes:
- Changed/Improved all models.
- Removed the VDSR models.
- Added Sharp versions of the Fast and Medium models.
- Removed the Double, Width and Height parameters.
- Added mpv user shaders of all models.
- Renamed the Illustration models to LineArt.
- Fixed: the mpv shaders didn't work properly with vs-placebo.

Now all models are based on FSRCNNX, with Global Residual Learning using Lanczos interpolation, and more layers and Local Residual Learning for the HQ models. I also trained some VDSR models with the processing done in the low-resolution space, obtaining the same results as the previous ones, but even those are outperformed by the new Medium models, so there's no reason to keep them. Also, the processing on the new HQ models is done entirely in the low-resolution space, so the arbitrary resize feature has been removed. I think it was handy, and I was using it myself for some things, but the traditional fractional upscaling method should give better results.

At this point, I think that considering the limitations of AviSynth Shader, there's not much room for improvement for the main application. The mpv shaders could still be greatly improved for use with some of the latest GPUs, but these are not yet available where I live, so that will have to wait.

[ReinerSchweinlin]

Thanx !

[real.finder]

at least for Anime I read that FSRCNNX is not good for things that less than 720 (will case oil pant effect), is it possible have another options like SSSR and Ravu? as they good for those cases

[Alexkral]

Despite being based on FSRCNNX, the latest models are actually very different in some important ways, and the LineArt models have been trained with an Anime dataset, which makes a big difference. It is true however that for more than 2x there are more artifacts, but not much more can be done with only 24 filters.

Anyway, to implement RAVU it is necessary to load a 32 bit texture, which I think is not possible with AviSynth. SSSR seems definitely possible, but it would not fit in this application, and also I'm not very interested.

[real.finder]

so that mean AviSynthShader need update to support float (32 bit)? maybe with opengl and Vulkan too

[Alexkral]

Avisynth Shader supports 16 bit half-float for internal processing, which should be enough, but it doesn't support 32 bit float input. The problem would also be to load a floating point image in AviSynth since ImageSource would not work. 16 bit integer is useless because there are negative values.

AviSynth Shader would surely also benefit from an update to DirectX 10 or higher, which would mean Shader Model 4 -> no limit on the number of constant registers per shader -> more filters per convolutional layer -> better models (and also slower :)).

[VincAlastor]

Really good script! Thanks!
I'm using it for anamorphic SD upscaling. At the end I'm resizing to 1280*720. I would like to use avisynth shader ResizeShader() with SSIM downscaling. But I'll getting a script error when using AiUpscale() and Avisynth shader in the same script. I know AiUpscale is using avisynth shader but is there nevertheless a way to load two instances avisynth shader?

[MysteryX]

Wow I hadn't seen this one before! Glad AvisynthShader has been of more modern use! I've been wondering for a while how to run more modern upscalers in Avisynth.

To clarify a few things. The output can be in any bitdepth you want. See ExecuteShader's OutputPrecision parameter. However, graphic cards are designed to push data to the graphic card with a large pipeline. The pipeline to get data back is a lot more limited and is a performance bottleneck. Returning in 16-bit instead of 8-bit can cut performance by half (might depend on graphic card too, haven't tested on my new system).

As for using DirectX 10, 11 or 12 instead of DirectX 9... it would be possible (madVR is doing it), but complicated. The approach (shortcut) I'm using in DirectX 9 doesn't work in later versions. It would require more complicated vertex calculation code or whatever that I simply don't understand to mimic the DX9 way.

I think that what's missing in DX10 and above is all the FVF stuff (https://docs.microsoft.com/en-us/windows/win32/direct3d9/d3dfvf) to convert this (https://github.com/mysteryx93/AviSynthShader/blob/master/Src/D3D9RenderImpl.cpp#L241). Searching FVF with DX10 or above returns nearly nothing. This thread (https://www.gamedev.net/forums/topic/705895-converting-from-directx-90c-to-directx-11/5422229/) mentions that "FVF Format's are pretty much abandoned" and "What this means in practice is getting rid of fixed pipeline code, moving everything over to shaders and using IDirect3DVertexDeclaration9 instead of FVF codes". I'll leave the interpretation up to you.

I'm not exactly sure why I hadn't supported 32-bit calculation. One problem I had is that supported video buffer formats vary greatly from one video card to the next -- hard to know which formats can be used reliably. Some formats would work on NVidia and not on AMD, and vice-versa. PlanarOut feature, in particular, doesn't work with all graphic cards.

btw software vertex processing is supported right here (https://github.com/mysteryx93/AviSynthShader/blob/master/Src/D3D9RenderImpl.cpp#L67). Glad it's working! (seeing a few surprised faces)

[Alexkral]

But I'll getting a script error when using AiUpscale() and Avisynth shader in the same script.

I'm not sure if that could be fixed, could you post your script and the errors you are getting?

Wow I hadn't seen this one before! Glad AvisynthShader has been of more modern use! I've been wondering for a while how to run more modern upscalers in Avisynth.

Hi MysteryX, I'm glad you like this!

I have to say that I have zero knowledge of DirectX programming, so I can't help with that. However when I was doing the whole training thing, I also trained a 2x HQ model with 64 filters (the published model has 24), which cannot be used with AviSynth Shader but can be used as an mpv shader. Using it in mpv with my GTX 1080 I was only able to get 12 fps for 4k upscaling, so I'm not sure it could be used with more modern GPUs. The results in terms of metrics were very similar to a full VDSR model, but the most important thing is that perceptually I could barely distinguish any difference between it, and the 24 filter model at a fairly short distance. So I think at least for 2x it would be overkill. For 3x and 4x however, these models usually produce quite noticeable artifacts with only 24 filters, so a 64 filter model would probably be more justified.

[VincAlastor]

I'm not sure if that could be fixed, could you post your script and the errors you are getting?


Sure, thank you very much to take a look to.

DGSource("", cl = 0, ct = 0, cr = 0, cb = 0, deinterlace=1, use_pf=true)
AiUpscale(Luma="HQ Sharp", Chroma="KrigBilateral")
ResizeShader(Width=1280, Height=720, Kernel="SSIM")

Script Error

Shader: Failed to open pixel shader Shaders\Lanczos3\Lanczos3.cso
(C:/Program Files (x86)/AviSynth+/plugins64+/AiUpscale.avsi, line 362)
(C:/Program Files (x86)/AviSynth+/plugins64+/AiUpscale.avsi, line 366)
(C:/Program Files (x86)/AviSynth+/plugins64+/AiUpscale.avsi, line 146)
(C:/Program Files (x86)/AviSynth+/plugins64+/AiUpscale.avsi, line 147)
(C:\Temp\script.avs, line 6)

[Alexkral]

It works fine here. Try changing line 96 in AiUpscale.avsi to reflect the full path to your Shaders folder, e.g:

Global Path = "C:\Program Files (x86)\AviSynth+\plugins64+\Shaders\"

[MysteryX]

Is there a problem running 64 filters in a row? The only restriction I'm seeing is to not have more than 10 different source clips. Unless you need more than 10 intermediary clips, like needing to merge the result of 64 different shaders or something.

Btw you'd greatly benefit from running SIMD downscaling in the same chain before returning it from the GPU; otherwise you pull everything and push everything back, killing performance for nothing.

[VincAlastor]

It works fine here. Try changing line 96 in AiUpscale.avsi to reflect the full path to your Shaders folder, e.g:

Global Path = "C:\Program Files (x86)\AviSynth+\plugins64+\Shaders\"

thats the solution! thank you very much :)

[VincAlastor]

Is there a problem running 64 filters in a row? The only restriction I'm seeing is to not have more than 10 different source clips. Unless you need more than 10 intermediary clips, like needing to merge the result of 64 different shaders or something.

Btw you'd greatly benefit from running SIMD downscaling in the same chain before returning it from the GPU; otherwise you pull everything and push everything back, killing performance for nothing.

sure, you're totally right. Thank you.

[MysteryX]

GPU processing isn't a big deal; you can process as much as you want once the video data is there. The bottleneck is data transfer between the CPU and GPU.

Which is where it's beneficial to downscale in 16-bit and then return back from the GPU in 8-bit.

[LigH]

I tested this upscaler too, on possibly not so easy material: Interlaced, compressed with obvious artifacts in motion even after applying QTGMC. So far, I don't see much difference to the usual approach with NNEDI3_rpow2. I guess it depends much more on the cleaning before upscaling. There seem to be stronger AI models, but only implemented in VapourSynth yet.

[Alexkral]

I tested this upscaler too, on possibly not so easy material: Interlaced, compressed with obvious artifacts in motion even after applying QTGMC. So far, I don't see much difference to the usual approach with NNEDI3_rpow2. I guess it depends much more on the cleaning before upscaling. There seem to be stronger AI models, but only implemented in VapourSynth yet.

That's normal, the quality of the upscaler is of less importance as the quality of the content worsens. For this type of content you will probably get better results with the FSRCNNX (https://github.com/igv/FSRCNN-TensorFlow/releases/tag/1.1) shaders, or even the FSRCNNX-Enhance (https://github.com/HelpSeeker/FSRCNN-TensorFlow/releases/tag/1.1_distort) shaders. Both have been trained with different levels of increased distortion, and therefore should be able to reduce compression artifacts. You can try both in VS with vs-placebo.

[spam_sakura]

Hi, need some help.

My script:
Import("W:\TEST StaxRip-x64-2.0.8.0-stable\Apps\Plugins\AiUpscale.avsi")
Import("W:\TEST StaxRip-x64-2.0.8.0-stable\Apps\Plugins\Shaders\AviSynthShader-1.6.6\Shader.avsi")
LoadPlugin("W:\TEST StaxRip-x64-2.0.8.0-stable\Apps\Plugins\Shaders\AviSynthShader-1.6.6\Shader-x64.dll")
DirectShowSource("V:\1.mp4").AiUpscale
spline64resize(2880,2160)

Error
System exception - Access Violation
(W:\TEST StaxRip-x64-2.0.8.0-stable\Apps\Plugins\AiUpscale.avsi, line 281)
(W:\TEST StaxRip-x64-2.0.8.0-stable\Apps\Plugins\AiUpscale.avsi, line 285)
(W:\TEST StaxRip-x64-2.0.8.0-stable\Apps\Plugins\AiUpscale.avsi, line 142)
(W:\TEST StaxRip-x64-2.0.8.0-stable\Apps\Plugins\AiUpscale.avsi, line 147)
(E:\!DownloaD\4k\TEST_4K_PC_4x3.avs, line 4)
(E:\!DownloaD\4k\TEST_4K_PC_4x3_temp\TEST_4K_PC_4x3_source.avs, line 1)

[Alexkral]

It works fine for me with the latest versions, I don't know where the problem may be.

[LigH]

I would not trust DirectShowSource to be reliable. For MP4 clips, I would prefer L-SMASH Works with LSMASHVideoSource (or alternatively LwLibavVideoSource if the source container is a little incomplete).

[goorawin]

You need to include brackets after AiUpscale, like so
DirectShowSource("V:\1.mp4").AiUpscale()

[Floatingshed]

I'm having trouble getting this to load. I think the problem is with Shader-x64.dll. It and all the other files are in the correct plugins directory.
I am getting the No function named "ConvertToShader" message. If I move Shader-x64.dll out of the plugins folder and load it manually I get:
"Cannot load.... platform returned code 126.... specified module could not be found.
So, it looks like I'm missing a runtime, can anyone please let me know which one? Thanks.