AviSynthShader + SuperRes

[gmail123]

avs 2.6 is ok

[gmail123]

but avs + is wrong:

avs+,in megui, script:
AddAutoloadDir("D:\video\MeGUI\tools\avisynth_plugin")
LoadPlugin("D:\video\MeGUI\tools\lsmash\LSMASHSource.dll")
LWLibavVideoSource("C:\Users\pc\Desktop\am.mkv")
ConvertBits(16)
SuperRes(3, .43, 0, """nnedi3_rpow2(2, nns=4, cshift="Spline16Resize")""")
ConvertBits(8, dither=0)
but
https://i.imgbox.com/Cd4F7oz6.png

[gmail123]

in this script:
AddAutoloadDir("D:\video\MeGUI\tools\avisynth_plugin")
LoadPlugin("D:\video\MeGUI\tools\lsmash\LSMASHSource.dll")
LWLibavVideoSource("C:\Users\pc\Desktop\12.mkv")
#LanczosResize(704,380) # Lanczos (Sharp)
ConvertToShader(2)
last
Shader("D:\video\MeGUI\tools\avisynth_plugin\nnedi3.hlsl" )
Shader("D:\video\MeGUI\tools\avisynth_plugin\SuperRes.hlsl")
ExecuteShader(last,last,Clip1Precision=1,Precision=3,OutputPrecision=2)
ConvertFromShader(2)

ConvertBits(16)
SuperRes(3, .43, 0, """nnedi3_rpow2(2, nns=4, cshift="Spline16Resize")""")
ConvertBits(8, dither=0)

https://imgbox.com/m1amfWuz

.hlsl is ok
http://imgbox.com/jfBfgJWk

[MysteryX]

but avs + is wrong:
Oh right there was this bug when handling HBD with the Upscale command.

SuperResXbr is probably working fine for you.


With NNEDI3.hlsl, is it using PS_3_0 format? I haven't yet found a NNEDI3 HLSL script written in 3_0 format, all I've seen were in 4_0 format which doesn't work here.

Try compiling it manually with fxc as I mentioned a few posts back.

[gmail123]

in this script:
Dither_convert_8_to_16()
SuperRes(3, .43, 0, """nnedi3_rpow2(2, nns=4, cshift="Spline16Resize")""")
ResizeShader(720,400)
DitherPost()
output:720*200
bug?

[MysteryX]

You're using Stack16 format as input, so you need to set lsb_in=true and lsb_out=true. lsb_upscale=lsb_in by default.

Also, you're using nnedi3_rpow2 on a Stack16 clip which isn't valid.

[gmail123]

because of :
https://thumbs.imgbox.com/93/72/fMDWgfWQ_t.png (http://imgbox.com/fMDWgfWQ)

[gmail123]

how to use "nnedi3_rpow2" on a Stack16 clip?
use "lsb_in=true and lsb_out=true" all wrong.

now ,I have no idea...

[MysteryX]

AFAIK nnedi3_rpow2 doesn't support Stack16, but you can use ConvertToStacked and ConvertFromStacked (http://avisynth.nl/index.php/ConvertStacked) to convert between native 16-bit and Stack16

csp invalid? Make sure you have the latest version of DitherTools. csp parameter is in the docs. (http://avisynth.nl/index.php/Dither_tools#Dither_resize16.2C_Dither_resize16nr)

The but with AvisynthShader is in properly detecting 16-bit clips in the Upscale command.

Perhaps this will work, haven't tested

ConvertBits(16)
SuperRes(3, .43, 0, """nnedi3_rpow2(2, nns=4, cshift="Spline16Resize").ConvertToStacked()""", lsb_upscale=true)
ConvertBits(8)

[gmail123]

AFAIK nnedi3_rpow2 doesn't support Stack16, but you can use ConvertToStacked and ConvertFromStacked (http://avisynth.nl/index.php/ConvertStacked) to convert between native 16-bit and Stack16

csp invalid? Make sure you have the latest version of DitherTools. csp parameter is in the docs. (http://avisynth.nl/index.php/Dither_tools#Dither_resize16.2C_Dither_resize16nr)

The but with AvisynthShader is in properly detecting 16-bit clips in the Upscale command.

Perhaps this will work, haven't tested

ConvertBits(16)
SuperRes(3, .43, 0, """nnedi3_rpow2(2, nns=4, cshift="Spline16Resize").ConvertToStacked()""", lsb_upscale=true)
ConvertBits(8)


wrong...
dither ver 1.27.2

I am crazy...

[MysteryX]

wrong...
dither ver 1.27.2

I am crazy...
hum... I have 1.27.1

Weird. Perhaps someone else knows about that

[MysteryX]

Version 1.6.4 is ready! (https://github.com/mysteryx93/AviSynthShader/releases/tag/v1.6.4)

What's new:
- Fixed SuperRes when using high-bit-depth upscale

[SaurusX]

When using SuperRes is it always necessary to use ConvertoRGB24() when encoding from a DVD source? Is there any way to keep the video in YV12 and use this upscaling? There are slight color shifts as a consequence of converting color space. I'm using a GTX750 Ti.

[MysteryX]

It needs to be converted to RGB32 internally. However, if you pass in YUV data, it will do the YUV to RGB conversion on the GPU in 16-bit and convert back to YUV. Alternatively, you can convert to RGB32 in 16-bit and do several operations and then convert back to YUV yourself in 16-bit. You shouldn't be seeing color shifts unless you convert colors in 8-bit.

[SaurusX]

It needs to be converted to RGB32 internally. However, if you pass in YUV data, it will do the YUV to RGB conversion on the GPU in 16-bit and convert back to YUV. Alternatively, you can convert to RGB32 in 16-bit and do several operations and then convert back to YUV yourself in 16-bit. You shouldn't be seeing color shifts unless you convert colors in 8-bit.

I am converting in 8-bit, but how do I NOT do that? I'm just using straight ConverttoRGB24(matrix="Rec709") and ConverttoYV12(matrix="Rec709") later. I was lacking the matrix specification which was causing even worse color shifts before. Now it's mitigated, but there's still something happening.

[MysteryX]

If you're using Avisynth+, you can use ConvertBits(16) and ConvertBits(8) to work in high-bit-depth.

Also is there a reason why you wouldn't just let the RGB conversation happen on the GPU? If there's a color shift, it might be due to a wrong color matrix. When doing GPU conversion, you can use MatrixIn and MatrixOut which are Rec709 by default. Perhaps your clip has Rec601. Double-check what color matrix is your source material.

[SaurusX]

If you're using Avisynth+, you can use ConvertBits(16) and ConvertBits(8) to work in high-bit-depth.

Also is there a reason why you wouldn't just let the RGB conversation happen on the GPU? If there's a color shift, it might be due to a wrong color matrix. When doing GPU conversion, you can use MatrixIn and MatrixOut which are Rec709 by default. Perhaps your clip has Rec601. Double-check what color matrix is your source material.

I'm using AviSynth through megui. Without an explicit call of ConverttoRGB24(matrix="Rec709") I get the following error message:

https://i.imgsafe.org/5d/5db08ce153.png

So I'm not quite sure what you mean by letting the conversion happen on the GPU. Is this problem an Nvidia thing? I've already had to make two small alterations to shader.avsi where I've set PlanarOut=False.

Thanks for your help.

[SaurusX]

OK, reading up on the Convert filters apparently I need to be using "PC.709" instead of "Rec.709" as I only want it to keep it's current color space range and not try to stretch it again. I'll try that and see if the color changing disappears.

EDIT: I figured it out. I have to use the following in order for the colors not to shift.

ConverttoRGB24(matrix="Rec601")
SuperResXBR(3, 1, 0.15, XbrStr=2.7, XbrSharp=1.3, fWidth=960, fHeight=720, fKernel="Bicubic", fB=0, fC=.75)
ConvertToYV12(matrix="Rec709")

This is from a DVD source, BTW. I don't know why the internal conversion process isn't working, but at least the problem is solved.

[MysteryX]

It's possible that there's still a bug with Avisynth 2.6 if that's what you're using.

Here it's working but you're losing data on both Convert functions resulting in loss of details accuracy.

[SaurusX]

I am using AVISynth 2.6 with megui. For me, the ConvertoX calls seem to be unavoidable for now. I may upgrade to AVISynth+ in the future to get rid of this problem.

How much detail is lost in these conversions at any rate? For animation it seems to be completely unnoticeable.

[MysteryX]

Just rounding errors. Alternatively, you can also use DitherTools to do 16-bit conversion with Stack16 format.

[SaurusX]

SOLVED. When using AVISynth+ instead of AVISynth2.6 there is no error on loading SuperRes or SuperResXBR. Using the flag matrixin="Rec601" for this DVD source results in perfect colors that match the source and no clipping at the extremes. Thanks again for the assist.

[Gser]

I can't get 10-bit input to work correctly even with AVS+.

LWLibavVideoSource("file.ts", format="YUV420P10")

ResizeShader(width=1920, height=1080, kernel="SSim")
8-bit works fine though. All I get for 10-bit is this:
https://imgur.com/a/v82jg
It's not even the right resolution, its 960x1080.

ps. have you thought about adding support for bt.2020?

[sneaker_ger]

L-Smash doesn't support the native high bitdepth formats of AviSynth+. Either use ffms2 or try something like:
LWLibavVideoSource("file.ts", format="YUV420P16", stacked=true)
ConvertFromStacked()
ResizeShader(width=1920, height=1080, kernel="SSim")

[Gser]

L-Smash doesn't support the native high bitdepth formats of AviSynth+. Either use ffms2 or try something like:
LWLibavVideoSource("file.ts", format="YUV420P16", stacked=true)
ConvertFromStacked()
ResizeShader(width=1920, height=1080, kernel="SSim")
L-Smash works just fine with avs+ formats http://avisynth.nl/index.php/LSMASHSource/LWLibavVideoSource
But it seems that I will have to use stacked as the only resizers that work directly with 10-bit are the internal ones.

[sneaker_ger]

L-Smash works just fine with avs+ formats http://avisynth.nl/index.php/LSMASHSource/LWLibavVideoSource
I don't read anything about AVS+ formats in there.

But it seems that I will have to use stacked as the only resizers that work directly with 10-bit are the internal ones.
I don't really know but to me this implies the AVS+ formats should be supported:
If you're using Avisynth+, you can use ConvertBits(16) and ConvertBits(8) to work in high-bit-depth.


Did you try my suggestions (unsuccessfully)?

[Gser]

I don't read anything about AVS+ formats in there.


I don't really know but to me this implies the AVS+ formats should be supported:



Did you try my suggestions (unsuccessfully)?
I can't demonstrate it any better that this picture.

https://imgur.com/a/60gYY

Yes I understood also that avs+ formats were supported but at least 10-bit didn't work.

Now I am using this code
LWLibavVideoSource("file.ts", stacked=true, format="YUV420P16" )
Dither_convert_yuv_to_rgb (matrix="2020", output="rgb48y", lsb_in=true)
Dither_y_gamma_to_linear (tv_range_in=true, tv_range_out=true, curve="2020", sigmoid=true)
Dither_resize16 (1920, 1080, kernel="bicubic", a1=0, a2=0.75)
Dither_y_linear_to_gamma (tv_range_in=true, tv_range_out=true, curve="2020", sigmoid=true)
r = SelectEvery (3, 0)
g = SelectEvery (3, 1)
b = SelectEvery (3, 2)
Dither_convert_rgb_to_yuv (r, g, b, matrix="2020", lsb=true)
Dither_quantize (10, mode=7, reducerange=true)
Dither_out ()
Prefetch(3)I'm doing the rgb conversion with dither tools as avisynthshader doesn't support bt.2020. I was hoping to process the rgb video then in avisynthshader but it processes stacked formats only in YV12 and YV24.

[MysteryX]

The image above isn't Stack16 format otherwise it would appear in 2 vertical sections with the bottom half showing random junk.

Note that if you pass 10-bit, it needs to be converted to 16-bit to pass into the shader. Use ConvertBits(16) before it and it should work fine.

[Gser]

The image above isn't Stack16 format otherwise it would appear in 2 vertical sections with the bottom half showing random junk.

Note that if you pass 10-bit, it needs to be converted to 16-bit to pass into the shader. Use ConvertBits(16) before it and it should work fine.

Yes it wasn't supposed to be stacked, I was trying to directly convert planar 10-bit. And also convertbits didn't work, just gives an error about reading video. Only converting it to stacked works. There might be a problem in convertbits because just using it without resizing already gives me an error. Quite bizarre.

[MysteryX]

Just to be clear. The GPU works only in 8-bit, 16-bit, or 32-bit.

If you pass other formats like 10-bit, it should automatically handle the conversions but perhaps I haven't tested that much.

[MysteryX]

Madshi replaced NNEDI3 with NGU in madVR. I haven't tried it yet but it appears to be better as he also removed NNEDI3.

Anyone knows of available HLSL implementation of NGU?

[cork_OS]

Madshi replaced NNEDI3 with NGU in madVR. I haven't tried it yet but it appears to be better as he also removed NNEDI3.

Anyone knows of available HLSL implementation of NGU?
Note that there are two main NGU variants, NGU AA (madshi's rewrite of NNEDI3, somewhat improved) and NGU Sharp/Standard/Soft (madshi's own edge-directed upscaler like waifu2x, FSRCNN etc.).
I assume that NGU sources will never be shared (however madshi plans to release avisynth plugin with madVR algorithms someday). For now you can try FSRCNN from mpv, I assume.

[MysteryX]

Oh madshi wrote these himself? Yeah that might make it harder to implement it elsewhere as he tends to be protective with his hard work.

[MysteryX]

OK I talked with Madshi. He's planning to release a madVR toolkit for Avisynth+ and VapourSynth, so you'll be able to use NGU + SuperRes without using AvisynthShader. When he gets the time to finish it.

[edcrfv94]

OK I talked with Madshi. He's planning to release a madVR toolkit for Avisynth+ and VapourSynth, so you'll be able to use NGU + SuperRes without using AvisynthShader. When he gets the time to finish it.

I do a fast search for super-resolution FSRCNN look like much better than nnedi3, But AviSynthShader not compatible MPV glsl Shaders.

NGU Sharp compares to mpv's latest FSRCNN(X):
https://forum.doom9.org/showthread.php?p=1825432#post1825432

[MysteryX]

NGU looks considerably better than FSRCNN(X) that causes much more edge/color distortion. It's also a lot faster. Thanks for the benchmarks and comparison.

Now NGU+SuperRes should also look great

[cork_OS]

Now NGU+SuperRes should also look great
madshi said several times that SuperRes doesn't seem to be beneficial for NGU (https://forum.doom9.org/showpost.php?p=1786250&postcount=40259) and even can harm it.

[MysteryX]

Have they also done comparison of SuperRes with NNEDI3 or SuperXbr, with NGU? SuperRes+Xbr was doing a great job already but had some distortion in some cases with geometric forms. I'd be curious to see NGU side-by-side.

[edcrfv94]

madshi said several times that SuperRes doesn't seem to be beneficial for NGU (https://forum.doom9.org/showpost.php?p=1786250&postcount=40259) and even can harm it.

EDSR look super!!!
https://github.com/LimBee/NTIRE2017

[MysteryX]

AviSynthShader v1.6.5 is ready! (https://github.com/mysteryx93/AviSynthShader/releases/tag/v1.6.5)

What's new:
- Fixed crash on NVidia cards when PlanarOut=true

Btw I'm hoping someone will port EDSR or other such algorithms to Avisynth... but I can only hope.

[SaurusX]

AviSynthShader v1.6.5 is ready! (https://github.com/mysteryx93/AviSynthShader/releases/tag/v1.6.5)

What's new:
- Fixed crash on NVidia cards when PlanarOut=true

Thanks for the fix! Does this provide any speed boost for nvidia cards?

[MysteryX]

Thanks for the fix! Does this provide any speed boost for nvidia cards?
Not unless you were running it with PlanarOut=false. If you did specify that, this version DOES support PlanarOut on NVidia cards. It supports L8 format in sysmem Textures, but not in PlainSurface! So I just switched it to use sysmem Textures for output as they both do the same thing.

[SaurusX]

@MysteryX Any plans to port over other HLSL shaders to AVISynth? There are many out there that would be very valuable when it comes to encoding. I keep hoping.

[MysteryX]

@MysteryX Any plans to port over other HLSL shaders to AVISynth? There are many out there that would be very valuable when it comes to encoding. I keep hoping.

You can always list some that would be useful to port, if they are written in a compatible format. Then someone might find the time to do it. Most useful scripts already have Avisynth implementations, and this has the disadvantage of requiring RBG conversion.

[SaurusX]

You can always list some that would be useful to port, if they are written in a compatible format. Then someone might find the time to do it. Most useful scripts already have Avisynth implementations, and this has the disadvantage of requiring RBG conversion.

In that case let me say that it would be great to have SMAA ported over. It would be a great addition to the anti-aliasing arsenal.

https://github.com/iryoku/smaa

https://kentaroid.com/wp-content/uploads/2014/02/383b9583e09f61e6f16d1ca3c45790dd.png

[Rumbah]

The thing with SMAA is that it's not just a simple post filter but it uses things like a depth buffer. So it might not be usable on a simple picture/video at all.

[wonkey_monkey]

The thing with SMAA is that it's not just a simple post filter but it uses things like a depth buffer. So it might not be usable on a simple picture/video at all.

It can use a depth buffer for edge detection but it doesn't have to.

I'm not sure what that image is showing though. The bit in green on the left seems to already be anti-aliased after some fashion, I can't believe that the red is a simple application of SMAA to the bit in green. That looks way too good.

[MysteryX]

Anti-aliasing is used for rendering 3D content into the screen.

If you're only using it as a post-processing for 2D content, then it's simply yet another desharpening or blurring algorithm.

[wonkey_monkey]

Anti-aliasing is used for rendering 3D content into the screen.

No it isn't. Aliasing isn't a 3D-only problem.

[Dogway]

Do you have plans on porting the plugin to VapourSynth?