AviSynthShader + SuperRes

[Gser]

I am now trying to figure out how I could convert some 4:2:2 to YV12 with this, using SSim as a chroma downscaler. Anybody have any ideas if this is possible?

[MysteryX]

Originally Posted by Gser

I am now trying to figure out how I could convert some 4:2:2 to YV12 with this, using SSim as a chroma downscaler. Anybody have any ideas if this is possible?

AviSynthShader doesn't support working on single planes (I was about to add support for that but NVidia graphic cards don't support output on single plane textures).

It works only with full RGB data. Optionally, you could interpret the RGB fields as being YUV and processing them accordingly -- which works for some shaders and causes distortions for other shaders; and then keeping only the chroma channel.

However, considering the textures in and out of the shader are 4:4:4, it won't be ideal for your situation.

[Mystery Keeper]

Originally Posted by MysteryX

AviSynthShader doesn't support working on single planes (I was about to add support for that but NVidia graphic cards don't support output on single plane textures).

It works only with full RGB data. Optionally, you could interpret the RGB fields as being YUV and processing them accordingly -- which works for some shaders and causes distortions for other shaders; and then keeping only the chroma channel.

However, considering the textures in and out of the shader are 4:4:4, it won't be ideal for your situation.

What a pile of bullshit. All cards can work with a single component textures.

[dipje]

I'm trying this for the first time and I'm getting a "System Exception - Illegal Instruction" when I'm just trying a ConvertToShader(). The supplied SuperRes (and other) functions in Shader.avsi also give this on the ConvertToShader lines.

Windows 10 x64, Avisynth+ r2294 MT x64., AviSynthShader 1.5.6

Edit after trying some more:

Got it working on my x86 Avisynth. I actually don't know which one it is exactly, it seems to be an 'official' 2.6 (2.6.0.6?) build with no MT.
With that one I got the 'ProcessImage' error. Adding 'PlanarIn = false, PlanarOut = false' made it work.

NNedi3 + superres. NNedi3 is the software version, so my gpu usage (for superres and downscaling afterwards) is only about 13% so it must be possible to optimize a bit more, let's try to find some shaders that work with this.
Awesome to play with this, now only if it would work on (my) x64 .

[sacd]

Not getting this to run using this script (x64)

LoadPlugin("C:\apps\AvisynthSwitcher\AVSPlugins\x64\Shader-x64")
LoadPlugin("C:\apps\AvisynthSwitcher\AVSPlugins\x64\nnedi3")
colorbars(width = 720, height = 576, pixel_type = "yv12").killaudio().assumefps("pal_double") 
SuperRes(2, .43, 0, """nnedi3_rpow2(2, nns=4, cshift="Spline16Resize")""")

in current Avisynth+, avsmeter says:

AVSMeter 2.4.4 (x64) - Copyright (c) 2012-2016, Groucho2004
AviSynth+ 0.1 (r2294, MT, x86_64) (0.1.0.0)

ExecuteShader: ProcessFrame failed.
(C:/apps/AvisynthSwitcher/AVSPlugins/AutoLoad64/Shader.avsi, line 144)
(C:\Projekte\Test\Test_SuperRes.avs, line 4)

[dipje]

@sacd One post above yours is a possible solution. Searching for 'processframe' in this thread will yield more information.

@others: can it be that avisynthshader is compiled with AVX2 or something ? Or any other reason why I get 'illegal instruction' only with the x64 version ?

[sacd]

Oh boy, I really did not search before posting, what a no no, sorry that, geez.

@dipje Thanks, got it running now by adding those two

[sacd]

Originally Posted by dipje

@others: can it be that avisynthshader is compiled with AVX2 or something ? Or any other reason why I get 'illegal instruction' only with the x64 version ?

I can confirm this to a degree, as on my old i7 980 I'm having illegal instruction too, while on i5 6600 it runs fine.

[dipje]

well that's a bummer. Would mean my i7-860 is out of luck and any AMD users are out of luck?

[MysteryX]

For CPU instructions, as per doc:

ConvertToShader, Opt: Optimization path. 0 for only C++, 1 for SSE2, 2 for AVX(only used with Precision=3), -1 to auto-detect. Default=-1

The AVX version is compiled separately, and in the latest version, I think I wasn't even compiling the AVX version because that code path wasn't even used by default. For those having troubles, does your CPU support SSE2? Does it work with Opt=0?

For those having trouble with x64, does it work with x86? I would need more precise details. Please make some tests with both x86 and x64 and tell me exactly which versions work and don't work, as well as the exact script you're using. From there, I can debug with these Avisynth versions.

Originally Posted by Mystery Keeper

What a pile of bullshit. All cards can work with a single component textures.

All cards (or most) can take single component textures as input, but NVIDIA cards can't use single component textures as the render target. Thus, single-component input is fine, but single-component output breaks on NVIDIA cards.

[MysteryX]

I just saw there is some SSSE3 code as well. This might be the issue.

[dipje]

I'll play around with the opt parameter . My script was a simple ffms2() with a converttoshader() with yv12 source IIRC.

Avisynth+ build at that time, the same revision of avisynth+ for x86 and x64.
The x64 gave 'illegal instruction' and the x86 worked fine.
CPU with sse4.2 level (so no AVX, but it does have Sse3).

Like I said, I will redo some testing (recently reinstalled os so everything is pretty clean ) to give you exact build numbers and versions and such.

[MysteryX]

I have to admit I did very little testing on x64 (and the memory usage is WAY higher which I found no way to fix -- at least on my graphic card, perhaps on other systems the driver handles memory differently -- can you test this?). If you tell me with more precision what works and what doesn't, that will help me correct the issue.

[dipje]

Well can't test memory usage for x64 since I didn't manage to get it working at all yet.

So, I didn't had any Avisynth at all installed on this system.
I installed Avisynth+R2294-MT, both x86 and x64. (You have to install it by first using the installer from R2172, then overwriting DLLs from R2294-MT..which is what I did).

Then copied 1.5.6 AvisynthShader package x86 and x64 to the corresponding plugin folders.
I used FFMS2 2.23.1 as source plugin, so my Avisynth script looks like this:

ffms2(source = "P9170146.MOV")
ResizeShader(Width = 1280, Height = 720, Kernel = "SSim", PlanarIn = True, PlanarOut = False)

The source file is a H264 1920x1080 p30 file, YV12 8bpp, progressive, no alpha. Nothing weird with it as far as I know.

I open the AVS file in x64 Virtualdub FilterMod (r38544) and I get 'Illegal Instruction':


I open the AVS file in x86 Virtualdub Filter Mod (r38543) and it works just fine.

The error is on line 339 in the Shader.avsi file, which is the 'ConvertToShader' call. I tried adding 'opt = 0', 'opt = 1', 'opt = 2', 'opt = 3' or 'opt = -1' to the parameters, but they all gave the same error.

I then tried setting PlanarIn to True in the 'ResizeShader' call. I get the same error message, but now on line 341 of Shader.avsi, on the Shader("LinearToYuv") call pretty much directly under it. Once again, the x86 version works fine.

I tried setting PlanarOut = True but that doesn't work at all, and gives the error 'ExecuteShader: ProcessFrame failed' which is to be expected on my nvidia card if I see the whole discussion about planar in/out.

If I convert the video to RGB first (ConvertToRGB24 / ConvertToRGB32. Is that still the way to do things with the new avs+ color format support? My Avisynth-syntax is a bit rusty) I can't get past the ConvertToShader call at all (I keep getting the 'System Error - Illegal instruction' at line 339 of Shader.avsi). No matter if I set PlanarIn to true or to false.

So, haven't got a single frame out of X64 yet. Can't say anything about performance or memory usage. But since all of my scripting pipeline is in x64 it's kinda of a bummer that I can't use the stuff in here.

Completely different question: Is there a shader variant of NNEDI3 that can be used in this easily / efficiently? I could only find the nnedi opencl port or the software versions.

[MysteryX]

There is a NNEDI3 version here but I hadn't been successful at getting it to work. Have to confirm... what HLSL version is it using?
https://github.com/zachsaw/MPDN_Exte...Scripts/NNEDI3

[MysteryX]

x64 works here so I can't test it, furthermore it seems to be coming from the code from Chikuzen and he hasn't posted on this forum for 6 months now. It takes someone with knowledge of assembly to look into it; I can't.

As for the conversion bug with ResizeShader with RGB clip, here's the fix.
Shader.avsi, line 341, replace
ConvertYuv ? Shader((PlanarIn ? "YV" : "Yuv") + "ToLinear" + MatrixIn + ".cso") : last
with
ConvertYuv ? Shader((PlanarIn ? "YV" : "Yuv") + "ToLinear" + MatrixIn + ".cso") : Shader("GammaToLinear.cso")

It performs the resize in Linear Light but wasn't converting from Gamma to Linear before resizing RGB clips.

[MysteryX]

I just saw there were changes from Chikuzen that I hadn't downloaded locally and thus weren't in the latest builds. I doubt it will fix the issue, but you can try. I just made a new compilation of the x64 library with his commit. Let me know whether that helps.
https://mega.nz/#!fdATyQRT!8Q_dYE1BH...Zxk-S5lgm2eUm0

[MysteryX]

AviSynthShader v1.6 is ready!

What's new:
- Added support for all new Avisynth+ clip formats
- Added support for native high-bit-depth clip formats
- ConvertToShader and ConvertFromShader now use native Avisynth+ functions for conversion, which results in better performance in several cases
- AviSynth 2.6 is still supported. ConvertToShader and ConvertFromShader have 2 code paths. Set Opt=2 to use the legacy code path. Opt=-1 will use Avisynth+ functions.
- Updated SSIM downscaler: more advanced XYZ processing

Known issues:
- RGB32 input with RGB64 output, or RGB64 input with RGB32, results in a color shift.
- PlanarIn works only with YUV input, and isn't supported on NVIDIA graphic cards. Leave it to false.
- 32-bit float formats are not currently supported

You might want to test the new vs old code paths with ConvertToShader and ConvertFromShader to detect any differences or subtle color shifts. To do so, edit shader.avsi and add Opt=2 to ConvertToShader and ConvertFromShader to use the legacy code path. This is a way of testing Avisynth+'s conversion functions to detect any rounding errors. I tested the old functions extensively, but haven't done this testing on the new functions.

The new code path shouldn't crash on Avisynth+ x64 anymore!

[amayra]

thank you for make AviSynthShader great again i will test 1.6 and comeback if i find issues

[MysteryX]

AviSynthShader v1.6.1 is ready!

What's new:
- Fixed color shift with RGB32 input and RGB48 output
- Compiled with Visual Studio 2017