AviSynthShader + SuperRes

[Alexkral]

Sorry, I think I'm complicating things. What I would like to know is if, as it seems, the use of the Width and Height parameters with any Output is enough to make sure that the size of each texture is that intended.

[MysteryX]

if I remember, the shader takes parameters to know the input size, and the parameters added to the command are the output size. I could be wrong, but I think it's that.

[Alexkral]

Okay, it seems to work fine so thanks.

[amayra]

any plan for VapourSynth support ?

[Alexkral]

@Amayra, just out of curiosity, what would you use this for in VapourSynth? I mean, something that is not possible with AviSynth?

[MysteryX]

I would like to add VapourSynth support. I think AviSynth/VapourSynth cross-platoform compatibility should be the standard in 2020 (although this particular project can't work outside Windows because of DirectX9)

Yet I still haven't looked into learning VapourSynth at all.

It would be useful if someone would write documentation with standard guidelines to write plugins for both VapourSynth and Avisynth, and also for making them cross-platform. Otherwise each dev needs to figure it out on their own.

[Myrsloik]

I would like to add VapourSynth support. I think AviSynth/VapourSynth cross-platoform compatibility should be the standard in 2020 (although this particular project can't work outside Windows because of DirectX9)

Yet I still haven't looked into learning VapourSynth at all.

It would be useful if someone would write documentation with standard guidelines to write plugins for both VapourSynth and Avisynth, and also for making them cross-platform. Otherwise each dev needs to figure it out on their own.

Cross-platform development is so far outside the scope of everything ending in synth it's not even funny. You need a fairly thick programming book to cover that subject in general.

Here's the uncomfortable truth:
1. Did you write GPU code? Probably not portable
2. Did you write code with unaligned memory access? Probably not portable
3. Did you access the filesystem? Probably not portable
4. Did you partially access a wider numerical type in memory? Definitely not portable!
5. Are you super great at build systems? No? No portability for you!

If you write sane code that only performs some CPU calculations and nothing else it'll probably be portable without much effort but that's about it.

[MysteryX]

These would be good guidelines for those starting to write new plugins. Things to know before writing the first line of code.

What's easier to do is port to VapourSynth and support both. If the actual work is done by a well-encapsulated class, then it should only require 2 separate code files to interop with each *synth?

[Myrsloik]

These would be good guidelines for those starting to write new plugins. Things to know before writing the first line of code.

What's easier to do is port to VapourSynth and support both. If the actual work is done by a well-encapsulated class, then it should only require 2 separate code files to interop with each *synth?

Yes, supporting both is very easy if you know what a function looks like.

[real.finder]

I can reproduce this; easier to see with colorbars or similar patterns

fulls=true seems to "fix" it (or match the others)


ConvertBits(16, fulls=true)
SuperResXBR(MatrixIn="Rec601")
ConvertBits(8, fulls=true, dither=-1)


Many thanks for this! It does indeed work. I guess the big question is why? I haven't observed this behaviour in KNLMeansCL for example. Just a bug?

I also wonder why it produces very slightly different results (detail-wise) to the LSB method. I was under the impression that despite the different approach to high bit depth processing, the results should be 100% identical.

it's was bug and fixed in https://forum.doom9.org/showthread.php?p=1916615#post1916615

[Alexkral]

Hi MysteryX, I've been working with AviSynth Shader for some time trying to use it to run some Super-resolution CNNs implemented as pixel shaders. You can see the result here (https://github.com/Alexkral/AviSynthAiUpscale). First thing I have to say is that it works very good and fast, so I wanted to thank you for your work, it's great to be able to use a tool like this. However I have found some problems and limitations that I wanted to inform you about, so that you can see what would be necessary to solve them if this is possible:

- This works by using each plane as the input/output for a filter, so with a maximum of 9 clips, the maximum number of filters is 36. To implement a network of 64 filters (most), 16 clips would therefore be necessary. For simple models like FSRCNN, this can be solved by spliting the layers between two or more shaders, but for more complex models this would mean a large increase in the number of shaders, and apart from what this would mean for performance, there is already a limit about the maximum number of shaders that can be used in the same pass (see below).
- I'm not sure what the problem is with Y16 as this seems to have been changing. Using it as input produces the error "CombinePlanes: source has no such plane U". Using it as output returns a YUVA444P16 clip. Maybe this could help to increase performance.
- Trying to return a Y8 clip now with Precision = 0 produces the error "CombinePlanes: source bit depth is different from 16".
- Shader Model 3.0 has a limit of 224 Constant registers. This means that, for example, the maximum number of clips that can be used in a shader for a Conv3x3 layer is equal to 6. Unfortunately this can only be solved by using Shader Model 4.0 or spliting the layers as above.
- There is a limit on the maximum number of shaders that can be used in a single pass, after which an access violation error occurs. I have noticed that this limit varies slightly depending on Precision and PlanarOut. This is what I've found in case it could help you:

With Precision = 1 and PlanarOut = false, the limit is 79.
With Precision = 1 and PlanarOut = true, the limit is 76.
With Precision = 2 and PlanarOut = false, the limit is 80.
With Precision = 2 and PlanarOut = true, the limit is 77.

And I think that would be it. For the kind of models I am using, troubleshooting the Y8 and Y16 issues could help to increase performance. I don't think that solving the other issues related to increasing the number of filters or layers could be especially beneficial, because the models would be much slower and the improvement in results would be only marginal. To have the possibility of implementing other more complex networks, a solution would be necessary though.

[amayra]

@Amayra, just out of curiosity, what would you use this for in VapourSynth? I mean, something that is not possible with AviSynth?
will like my script to be cross-platform and all work in one VS script to run it in mpv

I find VapourSynth faster/stable then AviSynth when it comes to frame interpolation

[ENunn]

Don't know if anyone here still uses this, but do I have to use a combination of SuperRes and another upscaler to upscale say a 720p source to 4k? If not, how do I make it so that it doesn't scale 2x.
The results are pretty good tbh :P

[SaurusX]

You can just chain the upscalers together. Call one after the other with whatever sharpening you want between them. Then downscale afterwards to a final 4K res.

[tyee]

Trying to get this to work in AVS but having problems. I've tried to fix the syntax as best I can but after loading the avs file into Vdub I get this error -

https://i.imgur.com/Mp2lKHT.png

I see the suggested script -

ConvertToShader(1)
Input
Shader("Diff1.cso", Output=2)
Shader("Diff2.cso", Output=3)
Shader("Merge.cso", Clip1=2, Clip2=3, Output=1)
ShaderExecute(last, Input, Clip1Precision=1, Precision=3, OutputPrecision=1)
ConvertFromShader(1)

I'm not sure what "Input" means and whether "1" means precision=1?

Here is my script - I tried to copy another script in this thread.

mov = FFVideoSource("J:\video.mp4")
ConvertToShader(mov)
Shader("G:\Adaptive-sharpen - Pass one.hlsl", Output=2)
Shader("G:\Adaptive-sharpen - Pass two.hlsl", Output=1)
ExecuteShader(last, mov, Clip1Precision=1, Precision=3, OutputPrecision=1)
ConvertFromShader()

Can you see what's wrong?

[Alexkral]

Try this:

mov = FFVideoSource("J:\video.mp4")
Input = ConvertToShader(mov, 1)
Shader(Input, "G:\Adaptive-sharpen - Pass one.hlsl", Output = 2)
Shader("G:\Adaptive-sharpen - Pass two.hlsl", Clip1 = 2)
ExecuteShader(last, Input, Clip1Precision = 1, Precision = 3, OutputPrecision = 1)
ConvertFromShader()

Default precision in ConvertToShader is 2, so you have to specify it. Default Output on the last shader line is always 1.

[tyee]

OK, thanks. Tried it and got an error about opening the shader. I removed the "G:\" and copied the .hlsl files to the plugin folder and now it loads fine but the image in Vdub is not the correct aspect ratio (width is 1/2 what it should be) and it's all covered in blue. I think there was something in the instructions about floating point?

[Alexkral]

Yeah, I guess the shaders are these (https://github.com/bacondither/Adaptive-sharpen/tree/master/shaders). You have to set also precision in ConvertFromShader(1)

[tyee]

Thanks, I did have the latest but I forgot that (1) on the last line. Works fine now!

[butterw2]

I've tried out Avisynth shader on a simple mp-hc video player shader. Any further advice welcome.

Mpc-hc shaders only run in rgb so it is necessary to convert to RGB32 at the input and the output.
It also seems necessary to modify the shader so that it takes bgra as input and output to get correct colors.

sampler s0: register(s0);

float4 main(float2 tex: TEXCOORD0): COLOR {
float4 c0 = tex2D(s0, tex).bgra;
...
c0+= float4(1, 0, 0, 0); //saturate red channel
...
return c0.bgra;
}


Perf overhead: tested on old intel integrated graphics i3-4150,
realtime preview at 720p60 and 1080p25 seem near achievable.
adding Prefetch(2) at the end of the script improves perf slightly but does increase cpu usage.

# AviSynth Shader v1.6.6 with Avisynth+ v3.6.1 x64 on Win10
https://github.com/mysteryx93/AviSynthShader/releases


- The default precision for ConvertToShader and ConvertFromShader is not 1, so you have to specify it.
- The input has to be RGB because otherwise the shader receives YUV channels.
- The first shader receives the channels as BGRA. The last shader has to return them as BGRA for RGB32 by changing the order again, or as GBRA for YV12 or YV24.
- The Alpha channel can be used to pass data between the shaders, but obviously it is lost at the end.

There is no need to compile the shaders.

input = FFMS2("video.mkv", colorspace = "RGB32")

c = ConvertToShader(input, Precision = 1)
Shader(c, "Anime4K_ComputeLum.hlsl", output=2)
Shader(last, "Anime4K_Push.hlsl", Clip1 = 2, output = 3)
Shader(last, "Anime4K_ComputeGradient.hlsl", Clip1 = 3, output = 4)
Shader(last, "Anime4K_PushGrad.hlsl", Clip1 = 4, output = 1)
ExecuteShader(last, c, Precision = 3, OutputPrecision = 1)
ConvertFromShader(last, Precision = 1, Format = "RGB32")


EDIT: My understanding of how avisynth shader works is as follows:
- The input video file (ex: yuv420) is loaded into avisynth (cpu).
- avisynth shader can transfer this to a pixel shader chain (dx9 .hlsl, gpu).
- mpc-hc video player shaders run in rgba. Thus the input has to be converted to RGB first (!!! avisynth shader input/output is in .bgra format meaning the shader code will have to be modified if it performs specific operations on color component).
- it would be possible to write hlsl shaders that run in yuv and you can pass parameters to the shaders via registers.
- the output gets transfered back to avisynth (cpu). The transfers between cpu/gpu have significant overhead, especially at higher resolution.
Further processing can be done in avisynth at input or output.

[Dogway]

Is it possible t use NNEDI3 with this? I tried with the hlsl from this git (https://github.com/zachsaw/MPDN_Extensions/tree/master/Extensions/RenderScripts/NNEDI3)but fails to open the file. Or maybe any other alternative for GPU based nnedi3.

[Kogarou]

Is it possible t use NNEDI3 with this? I tried with the hlsl from this git (https://github.com/zachsaw/MPDN_Extensions/tree/master/Extensions/RenderScripts/NNEDI3)but fails to open the file. Or maybe any other alternative for GPU based nnedi3.

Seconding this, would be great to have a way to run NNEDI3 (and even other things from MPDN) on a GPU in AVS+ :cool:

[MysteryX]

Is it possible t use NNEDI3 with this? I tried with the hlsl from this git (https://github.com/zachsaw/MPDN_Extensions/tree/master/Extensions/RenderScripts/NNEDI3)but fails to open the file. Or maybe any other alternative for GPU based nnedi3.
Not sure what would be easier; getting NNEDI3 to work in HLSL, or porting NNEDI3CL back from VapourSynth? The latter would probably perform better.

One challenge is that no scripts nowadays are written in DirectX9 HLSL format.

[kedautinh12]

Maybe ask asd-g for backport nnedi3cl from Vapoursynth :D

[lewyturn]

please help

Cannot load file 'C:/Avisynth+/plugins64+/Shader-x64.dll'. platform returned code 126

[kedautinh12]

What your avisynth version and what version of avisynthshader you using??

[lewyturn]

AviSynthShader-1.6.6
AviSynth+ 3.7.0

[kedautinh12]

You can check with avsmeter to show clearly errors
https://forum.doom9.org/showthread.php?t=174797

[StainlessS]

platform returned code 126
Some kind of missing requirement.

[lewyturn]

I tested it with "AVSMeter" and still got the same prompt. No other prompts.

[StainlessS]

dependency walker:- https://www.dependencywalker.com/
shows required dll's @ top left in image at link.
Microsoft includes it with various SDKs,
also used to be included in Windows setup disks [maybe still is].

[Blankmedia]

I tested it with "AVSMeter" and still got the same prompt. No other prompts.


For me it was DirectX 9

https://www.microsoft.com/en-US/download/details.aspx?id=35

dependency walker:- https://www.dependencywalker.com/
shows required dll's @ top left in image at link.
Microsoft includes it with various SDKs,
also used to be included in Windows setup disks [maybe still is].


Thad did the trick. Neat piece of software.