AviSynthShader + SuperRes

[MysteryX]

Do you have plans on porting the plugin to VapourSynth?

Eventually but it's far in my todo list

[Dogway]

pity, looks like the best tool aren't never in the same boat, wanted to try BM3D with SSIM resizer. FRC would also be a nice addition.

[ChaosKing]

use avsproxy in VS for FRC.

[poisondeathray]

If you have a working avisynth install, you can use core.avisource.AVISource too for FRC (or any avisynth script)

[Dogway]

Thanks ChaosKing, not sure if in this case it defeats my purpose for using more robust memory and MT handling if in the end I use avisynth. I will try SSIM and FRC with avsproxy.

[ChaosKing]

I like avsproxy bcs its like you're using a plugin. Input clip -> process -> output. And in VS I can also test for frame accurate seeking. Idk how if this is possible in avs.

[Dogway]

I don't know if it's possible to port the Jinc Resizer? the Jinc plugin that exists in avisynth doesn't support downscaling. Jinc (EWA filtering) is a very good kernel for aliasing/moiree, my aim is to downscale rendered cloth patterns using EWA filtering (not usually available in rendering or compositing software).

[Alexkral]

Hi, I'm having some problems with this script:

input = FFmpegSource2("video.mkv")

ScriptClip(input, """
Mean = string(AverageLuma(input) / 255.0) + "f"
c1 = ConvertToShader(input)
c2 = Shader(c1, "shader.hlsl", Param2 = Mean)
ExecuteShader(c2, c1)
ConvertFromShader(last, format = "YV12")
""")

This loads in VirtualDub, but in MPC-HC I got:

Shader: Failed to open pixel shader shader.hlsl
ExecuteShader: Initialize failed

Is there any other way to pass a runtime function to a shader?

Edit: Nevermind, copying the shader to the plugins folder solved the problem. Another thing, is there any reason for changing the order of the channels? I mean, I can handle it, but it doesn't seem like the way it should work.

[ChaosKing]

I found this https://github.com/bloc97/Anime4K/blob/master/Preprint.md#1080p-to-4k by accident and wanted to try this HLSL filter with your shader plugin. But I'm a bit stuck, I can't load the hlsl files. I think I can only use "compiled?" hlsl -> cso files? If yes, how can I "compile" them?

https://github.com/bloc97/Anime4K/releases

This is what I tried:
ConvertToShader(1)
Input = ffms2("test.mkv")
Shader("d:\Anime4K_Push.hlsl", Output=2)

ExecuteShader(Input, Clip1Precision=1, Precision=3, OutputPrecision=1)
ConvertFromShader(1)
Error: ExecuteShader: Source must be a command chain

p.s. Please change ShaderExecute to ExecuteShader in your github example. It took me some time to realize that the command is just wrong :)

EDIT
ok I think i need to run multiple shaders https://github.com/bloc97/Anime4K/blob/master/HLSL_Instructions.md

Input = ConvertToShader(ffms2("d:\test.mkv"))

Shader("d:\Anime4K_ComputeLum.hlsl", output=2)
Shader("d:\Anime4K_Push.hlsl", Clip1=2, output=3)
Shader("d:\Anime4K_ComputeGradient.hlsl", Clip1=3, output=4)
Shader("d:\Anime4K_PushGrad_Weak.hlsl", Clip1=4, output=1)

ExecuteShader(last, Input)
ConvertFromShader(last)
This loads now, but the output looks a bit broken.

[StainlessS]

I dont know nuttin bout this, but apparently the compiler (Effect-Compiler Tool - FXC.EXE) is included in DirectX SDK, and VS 2012:- https://docs.microsoft.com/en-us/windows/win32/direct3dhlsl/dx-graphics-hlsl-part1
Click on the fxc.exe link on linked page:- https://docs.microsoft.com/en-gb/windows/win32/direct3dtools/fxc
Offline Compiling:- https://docs.microsoft.com/en-gb/windows/win32/direct3dtools/dx-graphics-tools-fxc-using

Or Google stuff:- https://www.google.com/search?source=hp&ei=vylZXe8-p-OSBZT4plg&q=compile+hlsl&oq=compile+hlsl&gs_l=psy-ab.3..0j0i30l3j0i5i30l3j0i8i30l3.697.14203..14547...0.0..1.232.1427.9j4j1......0....1..gws-wiz.....0..0i131j0i10j0i22i30.F1Cav0KTJjU&ved=0ahUKEwivmvupnIzkAhWnsaQKHRS8CQsQ4dUDCAc&uact=5

[ChaosKing]

thx, found it later too.

Also found this https://artoriuz.github.io/mpv_upscaling.html
FSRCNNX is an upscaler which seems to be similar to NGU (madvr)

It can be downloaded here in glsl format: https://github.com/igv/FSRCNN-TensorFlow/releases

Maybe Avisynth shader could support glsl in (a not so distant) future :D


EDIT: Ahh finally found the diffenrence between hlsl and glsl (directx vs opengl) https://anteru.net/blog/2016/mapping-between-HLSL-and-GLSL/
and the future seems to be SPIR-V (vulkan)

[Alexkral]

This loads now, but the output looks a bit broken.

I managed to make this work with some problems, this is what I have found so far:

- The default precision for ConvertToShader and ConvertFromShader is not 1, so you have to specify it.
- The input has to be RGB because otherwise the shader receives YUV channels.
- The first shader receives the channels as BGRA. The last shader has to return them as BGRA for RGB32 by changing the order again, or as GBRA for YV12 or YV24.
- The Alpha channel can be used to pass data between the shaders, but obviously it is lost at the end.

There is no need to compile the shaders. The naive luminance aproximation in the ComputeLum shader doesn't make any sense, doing it the right way compiles to less instructions.

Also, these shaders run on screen space so you have to resize before (but if you don't, the effect is quite fun with low-res videos :)).

[ChaosKing]

I managed to make this work with some problems, this is what I have found so far:

- The default precision for ConvertToShader and ConvertFromShader is not 1, so you have to specify it.
- The input has to be RGB because otherwise the shader receives YUV channels.
- The first shader receives the channels as BGRA. The last shader has to return them as BGRA for RGB32 by changing the order again, or as GBRA for YV12 or YV24.
- The Alpha channel can be used to pass data between the shaders, but obviously it is lost at the end.

There is no need to compile the shaders. The naive luminance aproximation in the ComputeLum shader doesn't make any sense, doing it the right way compiles to less instructions.

Also, these shaders run on screen space so you have to resize before (but if you don't, the effect is quite fun with low-res videos :)).

Ahh ok thx.
But the only thing I had to change was to add Precision=1 in ConvertToShader / ConvertFromShader. Basically it is exactly like the example on github now :p

Input is yv12 and it seems to work.

EDIT:
But the sharpening effect is a bit too extreme and produces sometimes aliasing. NGU looks much better in the comparison.
The good thing is, it's "fast".

[Alexkral]

It works, and the result may even be identical, but it is not correct at all. Use RGB for input and output and change the "float lum ..." line in Anime4K_ComputeLum.hlsl by this:

float lum = (c0[2] + c0[2] + c0[1] + c0[1] + c0[1] + c0[0]) / 6;

Again, it probably won't make any difference.

[Alexkral]

Would it be possible to increase the number of clips to 16?

[poisondeathray]

I can't get the Anime4k hlsl to work . "Invalid arguments to function 'Shader'" . I tried different combinations of CK's post 559, and added Precision=1 for ConvertToShader, ConvertFromShader

Can you post a full script? or whatever the "proper" way to use this ?

Thanks

[Alexkral]

input = FFMS2("video.mkv", colorspace = "RGB32")

c = ConvertToShader(input, Precision = 1)
Shader(c, "Anime4K_ComputeLum.hlsl", output=2)
Shader(last, "Anime4K_Push.hlsl", Clip1 = 2, output = 3)
Shader(last, "Anime4K_ComputeGradient.hlsl", Clip1 = 3, output = 4)
Shader(last, "Anime4K_PushGrad.hlsl", Clip1 = 4, output = 1)
ExecuteShader(last, c, Precision = 3, OutputPrecision = 1)
ConvertFromShader(last, Precision = 1, Format = "RGB32")

[poisondeathray]

Thanks Alexkral, it's working now

[Alexkral]

@MysteryX

Since I have read that you have said that the bottleneck is memory transfers, I would like to know if the intermediate results are also being transferred. I ask it because using a chain of shaders the GPU load is below 15%, so I wonder if maybe grouping some shaders into a heavier one could be useful to increase performance.

[Dogway]

I'm unable to resize single planes:

ConvertToY()
ResizeShader(3840,2160,"bicubic",0,0.4)

CombinePlanes: source bit depth is different from 16
(C:/Program Files (x86)/AviSynth+/plugins64+/Shader.avsi, line 357)
(New File (1), line 23)

[StainlessS]

MysteryX, (Not on-line since 27 Sept 2019),

Can you update the dead PostImage.org images in first post please.
[ Easy to do, just edit the post, and change all PostImage.org to PostImage.cc ]

EDIT: Actually, quite a lot of dead images in later posts too.
EDIT: Same in FramerateConverter thread and its predecessor thread SVP-like frame interpolation?.

[tebasuna51]

Can you update the dead PostImage.org images in first post please.
[ Easy to do, just edit the post, and change all PostImage.org to PostImage.cc ]

Work in first post, but not in the second.

[StainlessS]

Thanks for trying Teb.
I dont remember ever seeing the "Image Not Found or Removed" thing (unless it was deleted deliberately by user).
When it first happened (maybe 12 to 18 months ago), I spend several days (3 or 4 maybe) altering several hundred url's,
and I also altered some only a few days ago to restore some broken images (must have been broken for more than 12 months, they had not been deleted).

Thanx again :)

[fozter]

I'm kind of a noob when it comes to video post processing, so please bear with me.

I've got this 1080p/HDR10/HEVC-track that I'm trying to upscale(SuperResXBR) to 2160p while still maintaining the HDR10-part.
Running this line: SuperResXBR(passes=5, factor=2)
But when reading the github I interpret it as colormatrix bt.2020 isn't supported and that by default it will convert it to bt.709.
Is this correct? And if, can someone please point me to another plugin that can achieve this? NNEDI3 maybe?

[Alexkral]

You can use the HDRTools plugin to convert to BT.709 and then use SuperResXBR with FormatOut = "YUV420P10". For NNEDI3 you can do the same and use NNedi3 resize 16:

https://forum.doom9.org/showthread.php?t=175488
http://avisynth.nl/index.php/Nnedi3_resize16

Edit: It seems that Nnedi3 resize 16 is no longer available but I think if you use ConvertYuv=false AviSynth Shader will cast YUV as RGB so there's no colorspace conversion and MatrixIn/MatrixOut are not used.

[fozter]

Okay, will try that. Thanks!

[markiemarcus]

I'm observing a very slight colour shift towards green (and I'd swear a slight drop in brightness or contrast) when using high bit depth processing in Avisynth+. This is with a Rec601 source and an AMD RX 480. Doesn't occur with standard bit depths. Any suggestions? It isn't as severe as a 601/709 shift, but it is there.

SuperResXBR(MatrixIn="Rec601")
^This works fine

ConvertBits(16)
SuperResXBR(MatrixIn="Rec601")
ConvertBits(8, dither=-1)
^This experiences the colour shift.

Dither_convert_8_to_16()
SuperResXBR(MatrixIn="Rec601",lsb_in=true,lsb_out=true)
DitherPost (mode=-1)
^No colour shift. Otherwise visually different results from the above.

Perhaps it's something I'm doing wrong? It is very minor and could just be the nature of the beast.

[Alexkral]

As you say maybe it's just because of the upscaling, I'm also experiencing color shifts with other algos so it seems quite possible to me. Anyway you could try with ColorYuv = false.

[amayra]

i wish if this work in vapoursynth natively

[poisondeathray]

I'm observing a very slight colour shift towards green (and I'd swear a slight drop in brightness or contrast) when using high bit depth processing in Avisynth+. This is with a Rec601 source and an AMD RX 480. Doesn't occur with standard bit depths. Any suggestions? It isn't as severe as a 601/709 shift, but it is there.

SuperResXBR(MatrixIn="Rec601")
^This works fine

ConvertBits(16)
SuperResXBR(MatrixIn="Rec601")
ConvertBits(8, dither=-1)
^This experiences the colour shift.

Dither_convert_8_to_16()
SuperResXBR(MatrixIn="Rec601",lsb_in=true,lsb_out=true)
DitherPost (mode=-1)
^No colour shift. Otherwise visually different results from the above.

Perhaps it's something I'm doing wrong? It is very minor and could just be the nature of the beast.


I can reproduce this; easier to see with colorbars or similar patterns

fulls=true seems to "fix" it (or match the others)


ConvertBits(16, fulls=true)
SuperResXBR(MatrixIn="Rec601")
ConvertBits(8, fulls=true, dither=-1)

[markiemarcus]

I can reproduce this; easier to see with colorbars or similar patterns

fulls=true seems to "fix" it (or match the others)


ConvertBits(16, fulls=true)
SuperResXBR(MatrixIn="Rec601")
ConvertBits(8, fulls=true, dither=-1)


Many thanks for this! It does indeed work. I guess the big question is why? I haven't observed this behaviour in KNLMeansCL for example. Just a bug?

I also wonder why it produces very slightly different results (detail-wise) to the LSB method. I was under the impression that despite the different approach to high bit depth processing, the results should be 100% identical.

[ChaosKing]

thx, found it later too.

Also found this https://artoriuz.github.io/mpv_upscaling.html
FSRCNNX is an upscaler which seems to be similar to NGU (madvr)

It can be downloaded here in glsl format: https://github.com/igv/FSRCNN-TensorFlow/releases

Maybe Avisynth shader could support glsl in (a not so distant) future :D

If someone needs it. https://github.com/Lypheo/vs-placebo supports glsl shader files now, it's a VapourSynth plugin.

[markiemarcus]

It seems that there is an incompatibility between AvisynthShader and AVS+ 3.6. I'm getting:

"System exception Access Violation". Shader.avsi, line 218.

Can anybody else confirm?

[goorawin]

Yes it has the same error for me

[markiemarcus]

Damn. TMM2 won't load either. I guess I'll be sticking with 3.5.1 for the foreseeable future!

[Reel.Deel]

These issues have been solved in Avisynth+ v3.6.1. No official release yet, for now download test 8 here: https://forum.doom9.org/showthread.php?t=181351
As for TMM2, download update here: https://github.com/Asd-g/TMM2/releases

[markiemarcus]

Thanks so much for the reply!

TMM2 seems to work fine with test 8. Unfortunately, still no dice with AvisynthShader. I'm getting exactly the same system exception.

Line 218 in Shader.avsi reads:

ExecuteShader(last, Input, Precision=3, Clip1Precision=PrecisionIn, OutputPrecision=PrecisionOut, PlanarOut=PlanarOut, Engines=Engines, Resource=true)
convert ? ConvertFromShader(PrecisionOut, format=sourceFormat, lsb=lsb_out) : last

[MysteryX]

By popular demand, v1.6.6 is released! (https://github.com/mysteryx93/AviSynthShader/releases/tag/v1.6.6)
- Updated headers to support Avisynth+ 3.6

I didn't do much testing so let me know if you encounter any issues.

[StainlessS]

Cheers MX :)

[DJATOM]

https://github.com/mysteryx93/AviSynthShader/commit/2481cd0db898151ec49a7455fc3262b99a521006#diff-af6f79275b08d74be68ade0a84bb01acR156
I think you slipped b = in your edits, didn't tested though

[gispos]

By popular demand, v1.6.6 is released! (https://github.com/mysteryx93/AviSynthShader/releases/tag/v1.6.6)
- Updated headers to support Avisynth+ 3.6

I didn't do much testing so let me know if you encounter any issues.

Thanks

[markiemarcus]

By popular demand, v1.6.6 is released! (https://github.com/mysteryx93/AviSynthShader/releases/tag/v1.6.6)
- Updated headers to support Avisynth+ 3.6

I didn't do much testing so let me know if you encounter any issues.

Legend, thank you!

Edit: Seems to work fine here.

[StvG]

By popular demand, v1.6.6 is released! (https://github.com/mysteryx93/AviSynthShader/releases/tag/v1.6.6)
- Updated headers to support Avisynth+ 3.6

I didn't do much testing so let me know if you encounter any issues.
It crashes on non Avisynth+ 3.6 environment.
There is no env->Allocate/env->GetEnvProperty for anything than V8 interface.

[MysteryX]

It crashes on non Avisynth+ 3.6 environment.
There is no env->Allocate/env->GetEnvProperty for anything than V8 interface.
dang.... how am I supposed to deal with that!?

Perhaps supporting older versions of AVS+ isn't important; if we consider it to have a more unstable API so far. Those using AVS+ can get the latest version of it. No reason to not upgrade.

[real.finder]

dang.... how am I supposed to deal with that!?

see how asd and pinterf did it

[StvG]

dang.... how am I supposed to deal with that!?

Perhaps supporting older versions of AVS+ isn't important; if we consider it to have a more unstable API so far. Those using AVS+ can get the latest version of it. No reason to not upgrade.
Check this post https://forum.doom9.org/showthread.php?p=1915174#post1915174

[MysteryX]

Check this post https://forum.doom9.org/showthread.php?p=1915174#post1915174
Simplest: for previous AVS versions, do the same I was doing for AVS 2.6

[Alexkral]

Hi, I'm not sure what's happening here:

clip = BlankClip(width = 100, height = 100)
W = clip.Width
H = clip.Height
ConvertToShader(clip, Precision = 1)
for (i = 1, 2) {
W = W * 2
H = H * 2
Shader("shader.hlsl", output = 2)
Shader("shader.hlsl", clip1 = 2, Width = W, Height = H)
}
ExecuteShader(last, clip, Clip1Precision = 1, Precision = 3, OutputPrecision = 1)
ConvertFromShader(Precision = 1, Format = "RGB32")

The result is a 400 x 400 clip as it should be, but if I look at the c0 register of the first shader on the second pass, the width and height are 100 again after they have been doubled on the first pass. If I add "Width = W / 2, Height = H / 2" to the first shader then everything is correct, again suggesting that by not doing so the texture is being downscaled. :confused:

[MysteryX]

Hi, I'm not sure what's happening here:

clip = BlankClip(width = 100, height = 100)
W = clip.Width
H = clip.Height
ConvertToShader(clip, Precision = 1)
for (i = 1, 2) {
W = W * 2
H = H * 2
Shader("shader.hlsl", output = 2)
Shader("shader.hlsl", clip1 = 2, Width = W, Height = H)
}
ExecuteShader(last, clip, Clip1Precision = 1, Precision = 3, OutputPrecision = 1)
ConvertFromShader(Precision = 1, Format = "RGB32")

The result is a 400 x 400 clip as it should be, but if I look at the c0 register of the first shader on the second pass, the width and height are 100 again after they have been doubled on the first pass. If I add "Width = W / 2, Height = H / 2" to the first shader then everything is correct, again suggesting that by not doing so the texture is being downscaled. :confused:

If I remember correctly... each output buffer (Clip1 - Clip9) are initialized at the start and cannot be changed. You're trying to fill Cilp1 with 2 different frames of different sizes. You'll need to use Output = 3 to create a separate buffer.

Output=1 needs to be the final output, so you may have to start with outputs 2, 3 and 4 for processing.

[Alexkral]

Thanks, I tested it and you're right.

EDIT: Well, it works, but I'm not sure if the explanation is correct. This is what I've done:

clip = BlankClip(width = 100, height = 100)
W = clip.Width
H = clip.Height
ConvertToShader(clip, Precision = 1)
for (i = 1, 2) {
W = W * 2
H = H * 2
Shader("shader.hlsl", output = i+1)
Shader("shader.hlsl", clip1 = i+1, Width = W, Height = H)
}
ExecuteShader(last, clip, Clip1Precision = 1, Precision = 3, OutputPrecision = 1)
ConvertFromShader(Precision = 1, Format = "RGB32")

Now output is 3 on the second pass, and has the correct size, so it's the output which can't change its size, and clip1 is receiving outputs of different sizes. Also I've seen that in SuperXbrMulti you solve this by sending the size to the shader as a parameter, so I'm not sure the best way of doing it:

1 - Setting the size: Shader("shader.hlsl", output = 2, Width = x, Height = y)
2 - Using a parameter: Shader("shader.hlsl", Param2 = size, output = 2)

or doing the above.