Port of NNEDI under new v2.6 AVS API - Page 28

LouieChuckyMerry · 26th May 2019, 17:52

Thank you so much for the information, real.finder

.

I finally had some time to test things. I've updated all necessary plugins-filters and am using the latest AviSynth+, but I'm struggling to solve the riddle. Here's my original script:

Code:

DGIndexNV INFORMATION HERE
### Deinterlace ###
TFM(Mode=7,UBSCO=False)
### Color Conversion ###
ColorMatrix(Mode="Rec.601->Rec.709")
### Adjust Color ###
MergeChroma(aWarpSharp2(Depth=16))
### Crop ###
Crop(8,0,-8,0)
### Fix Frame Borders ###
BorderControl(YTS=1,YTSF=1,YBS=1,YBSF=1)
### Gibbs Noise Block ###
Edge=MT_Edge("prewitt",ThY1=20,ThY2=40).RemoveGrain(17)
Mask=MT_Logic(Edge.MT_Expand().MT_Expand().MT_Expand().MT_Expand(),Edge.MT_Inflate().MT_Inpand(),"xor").Blur(1.0)
MT_Merge(Minblur(),Mask,Luma=True)
### Overall Temporal Denoise ###
SMDegrain(TR=2,ThSAD=200,ContraSharp=True,RefineMotion=True,Plane=0,PreFilter=2,Chroma=False,LSB=True,LSB_Out=True)
### Resize ###
NNEDI3_Resize16(Target_Width=960,Target_Height=720,Mixed=True,ThR=1.0,Elast=1.5,NNS=4,Qual=2,EType=0,PScrn=4,Threads=0,Kernel_D="Spline",Kernel_U="Spline",Taps=6,F_D=1.0,F_U=2.0,Sharp=0,LSB_In=True,LSB=True)
aWarpSharp4xx(Depth=5,LSB=True)
### Darken-Thin Lines ###
F=DitherPost(Mode=-1)
S=F.FastLineDarkenMod4(Strength=24,Prot=6).aWarpSharp2(Blur=4,Type=1,Depth=8,Chroma=2)
D=MT_MakeDiff(S,F).Dither_Convert_8_To_16()
Dither_Add16(Last,D,Dif=True,U=2,V=2)
### Deband ###
F3KDB(Y=100,Cb=100,Cr=100,GrainY=0,GrainC=0,Input_Mode=1,Input_Depth=16,Output_Mode=1,Output_Depth=16)
## Trim()
# SelectRangeEvery(1000,66)
ConvertFromStacked

To test I started with a default SMDegrain call:

Code:

DGIndexNV INFORMATION HERE
SMDegrain(n16=True,n16_Out=True)

which output proper, if interlaced, video. Then I inserted the 8-bit lines:

Code:

DGIndexNV INFORMATION HERE
### Deinterlace ###
TFM(Mode=7,UBSCO=False)
### Color Conversion ###
ColorMatrix(Mode="Rec.601->Rec.709")
### Adjust Color ###
MergeChroma(aWarpSharp2(Depth=16))
### Crop ###
Crop(8,0,-8,0)
### Fix Frame Borders ###
BorderControl(YTS=1,YTSF=1,YBS=1,YBSF=1)
### Gibbs Noise Block ###
Edge=MT_Edge("prewitt",ThY1=20,ThY2=40).RemoveGrain(17)
Mask=MT_Logic(Edge.MT_Expand().MT_Expand().MT_Expand().MT_Expand(),Edge.MT_Inflate().MT_Inpand(),"xor").Blur(1.0)
MT_Merge(Minblur(),Mask,Luma=True)
### Overall Temporal Denoise ###
SMDegrain(n16=True,n16_Out=True)

which outputs proper deinterlaced video. The trouble begins when I try to resize:

Code:

DGIndexNV INFORMATION HERE
### Deinterlace ###
TFM(Mode=7,UBSCO=False)
### Color Conversion ###
ColorMatrix(Mode="Rec.601->Rec.709")
### Adjust Color ###
MergeChroma(aWarpSharp2(Depth=16))
### Crop ###
Crop(8,0,-8,0)
### Fix Frame Borders ###
BorderControl(YTS=1,YTSF=1,YBS=1,YBSF=1)
### Gibbs Noise Block ###
Edge=MT_Edge("prewitt",ThY1=20,ThY2=40).RemoveGrain(17)
Mask=MT_Logic(Edge.MT_Expand().MT_Expand().MT_Expand().MT_Expand(),Edge.MT_Inflate().MT_Inpand(),"xor").Blur(1.0)
MT_Merge(Minblur(),Mask,Luma=True)
### Overall Temporal Denoise ###
SMDegrain(n16=True,n16_Out=True)
### Resize ###
NNEDI3_Resize16(Target_Width=960,Target_Height=720,Mixed=True,ThR=1.0,Elast=1.5,NNS=4,Qual=2,EType=0,PScrn=4,Threads=0,Kernel_D="Spline",Kernel_U="Spline",Taps=6,F_D=1.0,F_U=2.0,Sharp=0,LSB_In=True,LSB=True)
aWarpSharp4xx(Depth=5,LSB=True)

I receive the error message "needi3_resize16: only accept Y8, YV12, YV16, YV24 input". If I add "ConvertToYV16()" after the SMDegrain call, then I still receive the same error message. I've searched a lot and tried many things--to many to actually remember them all--but I can't find a solution. Thanks for any help.

videoh · 26th May 2019, 18:45

Temporally replace the last two lines with Info() to see what color space the SMDegrain call is delivering.

LouieChuckyMerry · 27th May 2019, 02:57

Quote:

Originally Posted by videoh

Temporally replace the last two lines with Info() to see what color space the SMDegrain call is delivering.

That's so cool. As I was trying to find a solution I, literally, wondered how to know what colorspace SMDegrain was outputing. And the answer is "ColorSpace: YUV420P16, BitsPerComponent: 16". A screenshot.

Groucho2004 · 27th May 2019, 03:18

Quote:

Originally Posted by LouieChuckyMerry

As I was trying to find a solution I, literally, wondered how to know what colorspace SMDegrain was outputing.

Just for reference:
"AVSMeter script -i" will also provide that info.

Code:

[Clip info]
Number of frames:             10000000
Length (hh:mm:ss.ms):    111:06:40.000
Frame width:                        32
Frame height:                       16
Framerate:                      25.000 (25/1)
Colorspace:                  YUV420P16

LouieChuckyMerry · 27th May 2019, 16:15

Quote:

Originally Posted by Groucho2004

Just for reference:
"AVSMeter script -i" will also provide that info.

Code:

[Clip info]
Number of frames:             10000000
Length (hh:mm:ss.ms):    111:06:40.000
Frame width:                        32
Frame height:                       16
Framerate:                      25.000 (25/1)
Colorspace:                  YUV420P16

Thanks for that, Groucho2004; I actually forgot that I'd added an AVSMeter shortcut to my "Send to" menu. Sadly, my understanding of the Windows cmd line is even less than my understanding of AviSynth. Much much much less. So I can right-click on a script and send it to AVSMeter, but it's going to want to run the whole script. Rather than take the time to make a special script, how would you process "AVSMeter script -i"?

real.finder · 27th May 2019, 16:24

Quote:

Originally Posted by LouieChuckyMerry

Code:

DGIndexNV INFORMATION HERE
### Deinterlace ###
TFM(Mode=7,UBSCO=False)
### Color Conversion ###
ColorMatrix(Mode="Rec.601->Rec.709")
### Adjust Color ###
MergeChroma(aWarpSharp2(Depth=16))
### Crop ###
Crop(8,0,-8,0)
### Fix Frame Borders ###
BorderControl(YTS=1,YTSF=1,YBS=1,YBSF=1)
### Gibbs Noise Block ###
Edge=MT_Edge("prewitt",ThY1=20,ThY2=40).RemoveGrain(17)
Mask=MT_Logic(Edge.MT_Expand().MT_Expand().MT_Expand().MT_Expand(),Edge.MT_Inflate().MT_Inpand(),"xor").Blur(1.0)
MT_Merge(Minblur(),Mask,Luma=True)
### Overall Temporal Denoise ###
SMDegrain(n16=True,n16_Out=True)
### Resize ###
NNEDI3_Resize16(Target_Width=960,Target_Height=720,Mixed=True,ThR=1.0,Elast=1.5,NNS=4,Qual=2,EType=0,PScrn=4,Threads=0,Kernel_D="Spline",Kernel_U="Spline",Taps=6,F_D=1.0,F_U=2.0,Sharp=0,LSB_In=True,LSB=True)
aWarpSharp4xx(Depth=5,LSB=True)

I receive the error message "needi3_resize16: only accept Y8, YV12, YV16, YV24 input". If I add "ConvertToYV16()" after the SMDegrain call, then I still receive the same error message. I've searched a lot and tried many things--too many to actually remember them all--but I can't find a solution. Thanks for any help.

P.S.--Sorry about the width of the post; this happens sometimes and I'm not sure why

.

you must not use any lsb filter with HBD, look for alternative filters (or updated ver.) that work with HBD, if you didn't find then you can use this http://avisynth.nl/index.php/ConvertStacked

LouieChuckyMerry · 27th May 2019, 16:41

Quote:

Originally Posted by real.finder

you must not use any lsb filter with HBD, look for alternative filters (or updated ver.) that work with HBD, if you didn't find then you can use this http://avisynth.nl/index.php/ConvertStacked

Which filter is LSB? I thought that the latest SMDegrain with "n16=True,n16_Out=True" was OK:

Quote:

Originally Posted by real.finder

so the last one is using n16=True,n16_Out=True instead of lsb and you will get native 16 HBD clip from your 8bit source with more speed than lsb without hacks

real.finder · 27th May 2019, 17:59

Quote:

Originally Posted by LouieChuckyMerry

Which filter is LSB? I thought that the latest SMDegrain with "n16=True,n16_Out=True" was OK:

the problem in the next filters, NNEDI3_Resize16 and aWarpSharp4xx (but this one work with both since I did update it, did also some update to NNEDI3_Resize16 but not for HBD since I chose edi_rpow2)

LouieChuckyMerry · 27th May 2019, 18:19

Ahhh, I didn't realize NNEDI3_Resize16 was such, thanks for the information (and for updating aWarpSharp4xx). Do you have a recommendation for a resizer that works in native HBD? And, would this then be the correct way to work with SMDegrain-NNEDI3_Resize16:

Code:

### Overall Temporal Denoise ###
SMDegrain(TR=2,ThSAD=200,ContraSharp=True,RefineMotion=True,Plane=0,PreFilter=2,Chroma=False,LSB=True,LSB_Out=True)
### Resize ###
NNEDI3_Resize16(Target_Width=960,Target_Height=720,Mixed=True,ThR=1.0,Elast=1.5,NNS=4,Qual=2,EType=0,PScrn=4,Threads=0,Kernel_D="Spline",Kernel_U="Spline",Taps=6,F_D=1.0,F_U=2.0,Sharp=0,LSB_In=True,LSB=True)
aWarpSharp4xx(Depth=5,LSB=True)
### Darken-Thin Lines ###
FastLineDarkenMod4(Strength=24,Prot=6) # Is this working in 16-bits? I couldn't find any info on LSB for FastLineDarkenMod4.
aWarpSharp4xx(Blur=4,Type=1,Depth=8,Chroma=2)
### Deband ###
F3KDB(Y=100,Cb=100,Cr=100,GrainY=0,GrainC=0,Input_Mode=1,Input_Depth=16,Output_Mode=1,Output_Depth=16)
Trim(17666,17777)
# SelectRangeEvery(1000,66)
ConvertFromStacked

Thanks again for your time

.

Groucho2004 · 27th May 2019, 18:48

Quote:

Originally Posted by LouieChuckyMerry

Rather than take the time to make a special script, how would you process "AVSMeter script -i"?

Add "-i" to your "Target" in the shortcut properties and change the AVSMeter.ini setting "PauseBeforeExit=0" to "PauseBeforeExit=1".

real.finder · 27th May 2019, 22:26

Quote:

Originally Posted by LouieChuckyMerry

Ahhh, I didn't realize NNEDI3_Resize16 was such, thanks for the information (and for updating aWarpSharp4xx). Do you have a recommendation for a resizer that works in native HBD?
.

edi_rpow2 is like NNEDI3_Resize16 but it work with HBD

Quote:

Originally Posted by LouieChuckyMerry

And, would this then be the correct way to work with SMDegrain-NNEDI3_Resize16:

Code:

### Overall Temporal Denoise ###
SMDegrain(TR=2,ThSAD=200,ContraSharp=True,RefineMotion=True,Plane=0,PreFilter=2,Chroma=False,LSB=True,LSB_Out=True)
### Resize ###
NNEDI3_Resize16(Target_Width=960,Target_Height=720,Mixed=True,ThR=1.0,Elast=1.5,NNS=4,Qual=2,EType=0,PScrn=4,Threads=0,Kernel_D="Spline",Kernel_U="Spline",Taps=6,F_D=1.0,F_U=2.0,Sharp=0,LSB_In=True,LSB=True)
aWarpSharp4xx(Depth=5,LSB=True)
### Darken-Thin Lines ###
FastLineDarkenMod4(Strength=24,Prot=6) # Is this working in 16-bits? I couldn't find any info on LSB for FastLineDarkenMod4.
aWarpSharp4xx(Blur=4,Type=1,Depth=8,Chroma=2)
### Deband ###
F3KDB(Y=100,Cb=100,Cr=100,GrainY=0,GrainC=0,Input_Mode=1,Input_Depth=16,Output_Mode=1,Output_Depth=16)
Trim(17666,17777)
# SelectRangeEvery(1000,66)
ConvertFromStacked

Thanks again for your time

.

FastLineDarkenMod4 work with HBD, for F3KDB see this

LouieChuckyMerry · 28th May 2019, 05:28

Quote:

Originally Posted by Groucho2004

Add "-i" to your "Target" in the shortcut properties and change the AVSMeter.ini setting "PauseBeforeExit=0" to "PauseBeforeExit=1".

Thank you. I've always had "PauseBeforeExit=1" set--I need time to read

--but never would've conceived of adding "-i" to the shortcut properties "Target", which works perfectly.

Quote:

Originally Posted by real.finder

edi_rpow2 is like NNEDI3_Resize16 but it work with HBD

Ooh wee! Getting EDI_RPow2 to work took some effort, and thanks for the challenge. I received a couple error messages about "what 'color_gray' means"--I've attached snips in order of appearance--and after hashing out (#) the offensive lines in ResizeX.avsi and EDI_RPow2.avsi I was eventually, I think, because "ConvertFromStacked" and my x264 custom cmd line "--demuxer raw --input-depth 16 --sar 1:1" were no longer necessary, able to send native 16-bit video to 10-bit x264 with this script:

Code:

### Deinterlace ###
TFM(Mode=7,UBSCO=False)
### Color Conversion ###
ColorMatrix(Mode="Rec.601->Rec.709")
### Adjust Color ###
MergeChroma(aWarpSharp2(Depth=16))
### Crop ###
Crop(8,0,-8,0)
### Fix Frame Borders ###
BorderControl(YTS=1,YTSF=1,YBS=1,YBSF=1)
### Gibbs Noise Block ###
Edge=MT_Edge("prewitt",ThY1=20,ThY2=40).RemoveGrain(17)
Mask=MT_Logic(Edge.MT_Expand().MT_Expand().MT_Expand().MT_Expand(),Edge.MT_Inflate().MT_Inpand(),"xor").Blur(1.0)
MT_Merge(Minblur(),Mask,Luma=True)
### Overall Temporal Denoise ###
# SMDegrain(n16=True,n16_Out=True)
SMDegrain(n16=True,n16_Out=True)
### Resize ###
EDI_RPow2(CShift="Spline64Resize",FWidth=960,FHeight=720)
aWarpSharp4xx(Depth=5)
### Darken-Thin Lines ###
FastLineDarkenMod4(Strength=24,Prot=6)
aWarpSharp4xx(Blur=4,Type=1,Depth=8,Chroma=2)
### Deband ###
F3KDB(Y=100,Cb=100,Cr=100,GrainY=0,GrainC=0,Input_Mode=2,Input_Depth=16,Output_Mode=2,Output_Depth=16)

Now I need to fine-tune my native HBD SMDegrain call....

Quote:

Originally Posted by real.finder

FastLineDarkenMod4 work with HBD, for F3KDB see this

Thanks for the reminder. Changing the input and output modes from 1 to 2 made it work correctly.

StainlessS · 28th May 2019, 12:50

Color_Gray ($808080) comes from the avisynth default supplied colors_rgb.avsi file which should be in plugins.
(I usually remove from there unless I need it).

Colors_rgb.avsi Demo:- https://forum.doom9.org/showthread.p...75#post1741775

LouieChuckyMerry · 28th May 2019, 14:41

I'd ignorantly removed the Colors_RGB.avsi from my Plugins folder years ago. After digging up a copy and running a quick test, all is well with unedited versions of ResizeX and Edi_RPow2. Thanks StainlessS

real.finder · 28th May 2019, 14:45

Quote:

Originally Posted by LouieChuckyMerry

Thanks for the reminder. Changing the input and output modes from 1 to 2 made it work correctly.

you need also ConvertToDoubleWidth() ConvertFromDoubleWidth() to made it work correctly

edit: I did some changes to the wiki http://avisynth.nl/index.php/ConvertStacked

StainlessS · 28th May 2019, 17:42

LCM, Older version of Colors_rgb.avsi had two entries for Color_PaleGoldenrod, in graphic above, is in 11th page after Gray set, in third and forth rows.
I discovered it after doing the avs to create animated GIF, and whilst watching it.
If you are using old version script, then maybe yours has the double entry, no real problem if it does.

LouieChuckyMerry · 28th May 2019, 19:24

Quote:

Originally Posted by real.finder

you need also ConvertToDoubleWidth() ConvertFromDoubleWidth() to made it work correctly

edit: I did some changes to the wiki http://avisynth.nl/index.php/ConvertStacked

Ahhh, thanks for that

.

I've had some time to tweak my SMDegrain settings with "n16=True,n16_Out=True" and have gotten everything to work but "ContraSharp=True". This runs fine:

Code:

SMDegrain(TR=2,ThSAD=200,ContraSharp=False,RefineMotion=True,Plane=0,PreFilter=2,Chroma=False,n16=True,n16_Out=True)

but this:

Code:

SMDegrain(TR=2,ThSAD=200,ContraSharp=True,RefineMotion=True,Plane=0,PreFilter=2,Chroma=False,n16=True,n16_Out=True)

gives the error message:

Quote:

Both clips should have the same colorspace!
([File Path]...SMDegrainMod_rf_3.1.2.101s.avsi, Line 591)
([File Path]...SMDegrainMod_rf_3.1.2.101s.avsi, Line 545)
([File Path]...TrimTest.avs Line 18)

Line 591:

Quote:

ssDD = ssD.repair(HD?ssD.repair(allD,1,planar=planar):allD,HD?12:1,planar=planar) # Limit the difference to the max of what the denoising removed locally.

Line 545:

Quote:

lsb_out ? StackVertical (last, BlankClip (last, pixel_type=type, color_yuv=0,length=1)) : last""")}

Quote:

Originally Posted by StainlessS

LCM, Older version of Colors_rgb.avsi had two entries for Color_PaleGoldenrod, in graphic above, is in 11th page after Gray set, in third and forth rows.
I discovered it after doing the avs to create animated GIF, and whilst watching it.
If you are using old version script, then maybe yours has the double entry, no real problem if it does.

As far as I can tell it's from 8 July 2005 and it does, in fact, have two "global color_palegoldenrod = $EEE8AA" entries, lines 110 and 111. Well, had...

real.finder · 28th May 2019, 21:09

Quote:

Originally Posted by LouieChuckyMerry

Ahhh, thanks for that

.

I've had some time to tweak my SMDegrain settings with "n16=True,n16_Out=True" and have gotten everything to work but "ContraSharp=True". This runs fine:

Code:

SMDegrain(TR=2,ThSAD=200,ContraSharp=False,RefineMotion=True,Plane=0,PreFilter=2,Chroma=False,n16=True,n16_Out=True)

but this:

Code:

SMDegrain(TR=2,ThSAD=200,ContraSharp=True,RefineMotion=True,Plane=0,PreFilter=2,Chroma=False,n16=True,n16_Out=True)

gives the error message:

Line 591:

Line 545:

try this https://pastebin.com/1HUc33kX

not 100% sure about it, will wait for the answer of this

LouieChuckyMerry · 28th May 2019, 23:27

It works fine,

.

StvG · 29th May 2019, 05:55

Quote:

Originally Posted by real.finder

you need also ConvertToDoubleWidth() ConvertFromDoubleWidth() to made it work correctly

edit: I did some changes to the wiki http://avisynth.nl/index.php/ConvertStacked

Both ConvertToStacked() and ConvertToDoubleWidth() need 16-bit input video. So:

Code:

## 10bit clip here
ConvertToDoubleWidth()
F3KDB(Input_Mode=2,Input_Depth=10,Output_Mode=2,Output_Depth=10)
ConvertFromDoubleWidth(bits=10)

should be

Code:

## 10bit clip here
ConvertBits16()
ConvertToDoubleWidth()
F3KDB(Input_Mode=2,Input_Depth=16,Output_Mode=2,Output_Depth=10)
ConvertFromDoubleWidth(bits=10)

28th May 2019, 12:50	#553 \| Link
StainlessS HeartlessS Usurer Join Date: Dec 2009 Location: Over the rainbow Posts: 10,980	Color_Gray ($808080) comes from the avisynth default supplied colors_rgb.avsi file which should be in plugins. (I usually remove from there unless I need it). Colors_rgb.avsi Demo:- https://forum.doom9.org/showthread.p...75#post1741775 __________________ I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

28th May 2019, 17:42	#556 \| Link
StainlessS HeartlessS Usurer Join Date: Dec 2009 Location: Over the rainbow Posts: 10,980	LCM, Older version of Colors_rgb.avsi had two entries for Color_PaleGoldenrod, in graphic above, is in 11th page after Gray set, in third and forth rows. I discovered it after doing the avs to create animated GIF, and whilst watching it. If you are using old version script, then maybe yours has the double entry, no real problem if it does. __________________ I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 28th May 2019 at 18:22.

28th May 2019, 23:27	#559 \| Link
LouieChuckyMerry Registered User Join Date: Feb 2014 Posts: 355	It works fine, . Last edited by LouieChuckyMerry; 29th May 2019 at 02:11. Reason: Syntax

26th May 2019, 18:45	#542 \| Link
videoh Useful n00b Join Date: Jul 2014 Posts: 1,667	Temporally replace the last two lines with Info() to see what color space the SMDegrain call is delivering.

28th May 2019, 14:41	#554 \| Link
LouieChuckyMerry Registered User Join Date: Feb 2014 Posts: 355	I'd ignorantly removed the Colors_RGB.avsi from my Plugins folder years ago. After digging up a copy and running a quick test, all is well with unedited versions of ResizeX and Edi_RPow2. Thanks StainlessS