Port of NNEDI under new v2.6 AVS API

[TheFluff]

I think it is still related to Intel compiler.

What do you base that on? Most of the NNEDI3 performance-critical code is in hand-written assembler in this version. In the Vapoursynth version (which uses mostly compiler intrinsics in C++ rather than pure asm) you could argue that the compiler could have a big impact, but not in this version. In fact it's been repeatedly pointed out in this very thread that having a special DLL with compiler optimizations for AVX2 enabled is completely pointless - it has essentially no performance impact at all, since there's nothing performance critical that the compiler can actually optimize.

[Aktan]

What do you base that on? Most of the NNEDI3 performance-critical code is in hand-written assembler in this version. In the Vapoursynth version (which uses mostly compiler intrinsics in C++ rather than pure asm) you could argue that the compiler could have a big impact, but not in this version. In fact it's been repeatedly pointed out in this very thread that having a special DLL with compiler optimizations for AVX2 enabled is completely pointless - it has essentially no performance impact at all, since there's nothing performance critical that the compiler can actually optimize.

I will admit it was just a bad guess as I'm not that familiar with compilers. I guess I won't be able to compile it then since there was hand written ASM. I based it on the fact that in the past Intel did selectively optimize to only Intel chips in their compiler.

Edit: Just curious, if I use the Release_Intel_W7_Core2_AVX DLL or even the Release_Intel_W7_Core2_SSE4.2 DLL and set opt to 5, would AVX2 still be used? I tried that and I still get the same slow speed. I was expecting it to throw an error like invalid opt mode, but I was just guessing.

[TheFluff]

What happens if you use threads=1?

BTW, if you're willing to try Vapoursynth, you can try znedi3 (https://github.com/sekrit-twc/znedi3), an attempt at making NNEDI3 on the CPU competitive with the OpenCL version running on the GPU. IIRC it was something like 50-100% faster than the original VS NNEDI3 on 8-bit input, but I don't know if anyone ever tested it on Ryzen.

[Aktan]

What happens if you use threads=1?

BTW, if you're willing to try Vapoursynth, you can try znedi3 (https://github.com/sekrit-twc/znedi3), an attempt at making NNEDI3 on the CPU competitive with the OpenCL version running on the GPU. IIRC it was something like 50-100% faster than the original VS NNEDI3 on 8-bit input.

I'll try both in a bit.

Edit: threads=1:

opt FPS
4 ~4
5 0.11 (~1 frame every 9 seconds)

[pinterf]

It's not Ryzen. This one:
nnedi3_rpow2(rfactor=4, cshift="lanczosresize", fwidth=2880, fheight=1920, ep0=2, nsize=0, nns=4, qual=2, pscrn=4, opt=5)
is giving me 0.03 fps for the first 3-4 frames with opt=0 or opt=5
and >3 fps with opt=4

x64 version freshly recompiled with VS2017 15.5
Intel i7-7700 (AVX2), Win10 x64, Avisynth+ 2591 (dev)

[Aktan]

That's interesting, I guess now to figure out which of the param is actually causing it.

[jpsdr]

Edit: Just curious, if I use the Release_Intel_W7_Core2_AVX DLL or even the Release_Intel_W7_Core2_SSE4.2 DLL and set opt to 5, would AVX2 still be used?
No differences in the code, just compiler options, all versions will behave the same.

[jpsdr]

Odd... I don't remember so much differences when testing... But on my tests, i've never changed ep0 and always tested pscrn with 2 or 0.
Btw, with 0, very small frames (480p) to have some frames without having to wait hours... ;)
Don't have time for now, but will redo tests later.

[pinterf]

Missing else

https://github.com/jpsdr/NNEDI3/blob/master/nnedi3/nnedi3.cpp#L1725

https://github.com/jpsdr/NNEDI3/blob/master/nnedi3/nnedi3.cpp#L1734

fps is OK again for AVX2

(And a request: the AVX2 asm files were missing from the sln file had to add them manually)

[jpsdr]

I was about to said that i'll check if i didn't messed-up something, but you beats me... :p
:thanks:
About ASM files :
Yes it's normal they are missing, because i want to keep the project on github on VS2010 (if you want to use more than VS2010, you can just upgrade the project), but VS2010 will not be able to to compile the AVX2 asm files, so it will result on a incorrect project if they were included. This is why the AVX2 asm are on others files.

[Aktan]

Awesome catch! Yep, changing prescreen to original (1) where the bug doesn't exist gave me results comparable to AVX.

[jpsdr]

New version, see first post, and i've also added on it a part about the multi-threading.

[jpsdr]

There is issue with the Intel versions.

I'll update the release files on github, removing the Intel versions, and keeping only VS version, and adding an VS AVX2 version. Wait at least 24h to check/re-download the files.

[jpsdr]

Trashed Intel version, file updated, redownload it.

[jpsdr]

New version, see first post, updated also the Multi-treading text part.

[mp3dom]

It seems there's a bug in YUY2 colorspace that create a green vertical bar on the right side of the image under some circumstances (both x86 and x64):


colorbars(width=1416,height=1080,pixel_type="yuy2")
nnedi3(0)

[Taurus]

It seems there's a bug in YUY2 colorspace that create a green vertical bar on the right side of the image under some circumstances (both x86 and x64):


colorbars(width=1416,height=1080,pixel_type="yuy2")
nnedi3(0)

But only if the width is at some "odd" resolutions...not devidable with mod16.
Older versions show the same behaviour.
Just a short test..

[jpsdr]

Thanks for reporting, i've found what was wrong.
I want to finish something else i'm working on before making new releases of some filters. So, several days before i'll make new builds, but it's fixed on Github.

[mp3dom]

So, several days before i'll make new builds, but it's fixed on Github.
Thanks

[jpsdr]

New version, see first post.

[LouieChuckyMerry]

Hello, and thank you jpsdr for your hard work :) . I'm preparing to upscale my PAL Futurama DVD's to 960x720 and found this:

### Resize ###
NNEDI3_RPow2(RFactor=2,CShift="Spline64Resize",FWidth=960,FHeight=720)
aWarpSharp2(Depth=5)
Sharpen(0.2)

to do a very nice job. I'm having some trouble, though, fitting this into my 16-bit pipeline and was hoping you could help. Here's the part of the script that's causing trouble:

### Overall Temporal Denoise ###
SMDegrain(TR=2,ThSAD=200,ContraSharp=True,RefineMotion=True,Plane=0,PreFilter=2,Chroma=False,LSB=True,LSB_Out=True)### Resize ###
NNEDI3_RPow2(RFactor=2,CShift="Spline64Resize",FWidth=960,FHeight=720)
aWarpSharp2(Depth=5)
Sharpen(0.2)

This outputs video with a resolution of 960x360 and for the life of me I can't figure out why. SMDegrain is sending 16-bits to NNEDI3, which I know from reading the thread should be OK. Please, do you have any ideas? Thanks for your time.

Edit: I just noticed that this thread hasn't seen action in almost a year; however, if anyone can answer my question I'd be very appreciative.

[gonca]

Your source filter might be stacking in order to give you a 16 bit pipeline.
At conclusion of workflow when the "8 over 8" is undone you get half height resolution
Try a quick sample with resize of 960 x 1440 or get a better source filter if possible

[LouieChuckyMerry]

Thanks for your reply, gonca. My source filter is DGIndexNV for the .m2v video. I know that SMDegrain stacks the video then sends stacked 16-bit to NNEDI3, but it seems NNEDI3 isn't happy about this. I'm using the latest AviSynth+. Let me try a quick resize... That actually worked, but is it the correct way or is there some AviSynth+ syntax I'm missing?

Edit: I indexed with LSMASH and it's the same story, 960x360 output. Hmmm...

[gonca]

DGIndexNV is fine and not the problem
Can you give DGDenoise a try instead of using SMDegrain
or use
ConvertFromStacked(clip clip, int bits )
http://avisynth.nl/index.php/ConvertStacked
after SMDegrain

[LouieChuckyMerry]

DGIndexNV is fine and not the problem
Can you give DGDenoise a try instead of using SMDegrain

I searched about for DGDenoise but couldn't find anything, yet it seems familiar. Could you please give me a hint.


http://avisynth.nl/index.php/ConvertStacked
after SMDegrain

I actually tried this some days ago but received the error message "ConvertFromStacked: Input stacked clip must be YV12, YV16, YV24 or YV8". At the time I wondered how this could be, given that the final line of my functional script is "ConvertFromStacked", but then I went about my business. Here's my script starting from the SMDegrain call (where the high bit depth begins); the earlier calls are deinterlacing, color conversion, Gibbs noise block, etc, all done in 8-bit:

### Overall Temporal Denoise ###
SMDegrain(TR=2,ThSAD=200,ContraSharp=True,RefineMotion=True,Plane=0,PreFilter=2,Chroma=False,LSB=True,LSB_Out=True)
### Resize ###
NNEDI3_RPow2(RFactor=2,CShift="Spline64Resize",FWidth=960,FHeight=720)
aWarpSharp2(Depth=5)
Sharpen(0.2)
### Darken-Thin Lines ###
F=DitherPost(Mode=-1)
S=F.FastLineDarkenMod(Strength=24,Prot=6).aWarpSharp2(Blur=4,Type=1,Depth=8,Chroma=2)
D=MT_MakeDiff(S,F).Dither_Convert_8_To_16()
Dither_Add16(Last,D,Dif=True,U=2,V=2)
### Deband ###
F3KDB(Y=100,Cb=100,Cr=100,GrainY=0,GrainC=0,Input_Mode=1,Input_Depth=16,Output_Mode=1,Output_Depth=16)
ConvertFromStacked

If I reverse the Overall Temporal Denoise and Resize lines, then the script runs fine and outputs 960x720 video; however, everything I've ever read regarding video encoding states that denoising should come before resizing (which makes perfect sense). I know the issue begins with the resizing because removing everything but "ConvertFromStacked" after the Resize section still results in the above error message.

I guess now I'm wondering why SMDegrain outputs seemingly different 16-bit than F3KDB, since the F3KDB output has no issue with the "ConvertFromStacked" call. Anyway, thanks again for your help.

[gonca]

As for DGDenoise, look in the DGDecodeNV manual

Re
ConvertFromStacked(16)
Try it after SMDegrain and before NNEDI3

[jpsdr]

What do you mean "stack" ? I support 16 bits, but only "true" 16 bits, so only on avs+ for more than 8 bits data. I don't support "hacked" format.

[gonca]

What do you mean "stack" ? I support 16 bits, but only "true" 16 bits, so only on avs+ for more than 8 bits data. I don't support "hacked" format.
If this is meant for me, I don't know what source filter is being used or how the 8 bit DVD got to be 16 bit
And there is no mention of NNEDI3 not handling 16 bits

[FranceBB]

@LouieChuckyMerry... You are feeding NNEDI3_RPow2 with 16bit stacked data while it's expecting 8bit.
Please note that the Jean-Philippe mod of NNEDI3 works in planar only, so it expects 8bit/10bit/12bit/14bit/16bit planar, not stacked nor interleaved, however SMDegrain with lsb=true is working in 16bit stacked.

This is what you gotta do:


#Indexing your 8bit source using your preferred indexer
FFMpegSource2("video.ts")

#Bring everything up to 16bit stacked
Dither_Convert_8_To_16()

#Denoise with 16bit stacked precision
SMDegrain(TR=2,ThSAD=200,ContraSharp=True,RefineMotion=True,Plane=0,PreFilter=2,Chroma=False,LSB=True,LSB_Out=True)

#Upscale with 16bit stacked precision
nnedi3_resize16(target_width=960, target_height=720, mixed=true, thr=1.0, elast=1.5, nns=4, qual=2, etype=0, pscrn=4, threads=0, tv_range=true, kernel_d="Spline", kernel_u="Spline", taps=6, f_d=1.0, f_u=2.0, sharp=0, lsb_in=true, lsb=true)

#Sharpening
#please note that you can't use awarpsharp2 as you did
#because awarpsharp is expecting 8bit planar
#while you are feeding it with 16bit stacked
#so you need to use aWarpSharp4xx
aWarpSharp4xx(depth=5, lsb=true)

#Do not use Sharpen(0.2)
#not only you can't use it with 16bit stacked
#but it's also a very basic and simple sharpening
#which does not work well;
#if you really have to use something else

#Line Darkening
#You did everything correctly this time
#as you brought everything back to 8bit
#you filtered with FastLineDarkenMod and awarpsharp2
#two filters that work in 8bit planar
#then you brought the result back to 16bit stacked
#and you applied the differences to the original 16bit stacked
F=DitherPost(Mode=-1)
S=F.FastLineDarkenMod(Strength=24,Prot=6).aWarpSharp2(Blur=4,Type=1,Depth=8,Chroma=2)
D=MT_MakeDiff(S,F).Dither_Convert_8_To_16()
Dither_Add16(Last,D,Dif=True,U=2,V=2)

#Debanding with 16bit stacked precision
#You did everything correctly this time as well
#as you specifically told f3kdb that it's getting 16bit stacked
#as input and that it should work internally at 16bit
#and output 16bit stacked
F3KDB(Y=100,Cb=100,Cr=100,GrainY=0,GrainC=0,Input_Mode=1,Input_Depth=16,Output_Mode=1,Output_Depth=16)


I hope it helps.

[LouieChuckyMerry]

As for DGDenoise, look in the DGDecodeNV manual

Re
ConvertFromStacked(16)
Try it after SMDegrain and before NNEDI3

gonca: Again, thank you for your help :) . I tried the above but to no avail, although now we know why:

What do you mean "stack" ? I support 16 bits, but only "true" 16 bits, so only on avs+ for more than 8 bits data. I don't support "hacked" format.

jpsdr: Thank you for solving the mystery (and for your port of NNEDI3 :) ).

FranceBB: Thank you very much for your educational explanations; they're really helpful to an AviSynth hobbyist such as myself :) .

1) After perusing this entire thread I eventually came to understand that jpsdr's port of NNEDI3 doesn't work with stacked 16-bit. What I wonder is, would this:

### Overall Temporal Denoise ###
SMDegrain(TR=2,ThSAD=200,ContraSharp=True,RefineMotion=True,Plane=0,PreFilter=2,Chroma=False,LSB=True,LSB_Out=False)
### Resize ###
NNEDI3_RPow2(RFactor=2,CShift="Spline64Resize",FWidth=960,FHeight=1440)
aWarpSharp2(Depth=5)
Sharpen(0.2)
### Darken-Thin Lines ###
Dither_Convert_8_To_16()
F=DitherPost(Mode=-1)
S=F.FastLineDarkenMod(Strength=24,Prot=6).aWarpSharp2(Blur=4,Type=1,Depth=8,Chroma=2)
D=MT_MakeDiff(S,F).Dither_Convert_8_To_16()
Dither_Add16(Last,D,Dif=True,U=2,V=2)

be the "correct" way to use NNEDI3_RPow2 in my original script? Not that I would use this, though, because...

2) I encountered your NNEDI3_Resize16 line in this thread and was able to make it work after some effort. Thank you. I reckon it's better to upscale in 16-bits, no?

3) I did some searching, with two different engines, for aWarpSharp4 and found, literally, nothing. Do you know where I might find a copy?

4) Please, any suggestions to replace Sharpen(0.2)?

5) Thanks for your kind help :) .

[FranceBB]

1) Yes, that would be the correct way as you are feeding it with 8bit planar data.

2) Yep, definitely better to have more precision whenever you can (at the expense of speed).

3) Sure, there you go: https://forum.doom9.org/showthread.php?p=1715287#post1715287

4) If you are working in 8bit planar: LSFMod. If you are working in 16bit interleaved, either MCLS_16 or BicubicSharp. For 16bit stacked, I'm using RKS but it's closed source and I can't share it as it has been developed years ago by the engineer who taught me encoding.

Thank you very much for your educational explanations; they're really helpful to an AviSynth hobbyist such as myself

The whole 16bit stacked, interleaved, planar is kinda tricky for me as well sometimes, so I'm not surprised that it's confusing for people who don't use Avisynth everyday for work and for people who didn't really went through the "evolution" of Avisynth through the years 'till the "normal" high bit depth arrived.
Honestly, if planar was introduced years ago in "vanilla" Avisynth, we wouldn't have had this "mess" of filters supporting stacked, some others supporting interleaved, some others supporting both (like f3kdb) but not planar > 8bit, some others supporting planar only 8bit, some others supporting planar high bit depth (8bit/10bit/12bit/14bit/16bit/32bit) but not stacked nor interleaved ecc.

[LouieChuckyMerry]

1) Yes, that would be the correct way as you are feeding it with 8bit planar data.

Thanks for scratching a brain itch; I've been wondering about this for a couple weeks.


2) Yep, definitely better to have more precision whenever you can (at the expense of speed).

For Futurama speed is no issue ;) .


3) Sure, there you go: https://forum.doom9.org/showthread.php?p=1715287#post1715287

:thanks: It's been tested and added to the script.


4) If you are working in 8bit planar: LSFMod. If you are working in 16bit interleaved, either MCLS_16 or BicubicSharp. For 16bit stacked, I'm using RKS but it's closed source and I can't share it as it has been developed years ago by the engineer who taught me encoding.

Any suggestions for 16-bit stacked that's not closed source, or is there no longer a reason to add extra sharpening after aWarpSharp4xx given the new, improved script? I stole, er, borrowed the original NNEDI3_RPow2-aWarpSharp2-Sharpen upscale block and figured that the "Sharpen(0.2)" call was like a final touch-up to the upscaling.


The whole 16bit stacked, interleaved, planar is kinda tricky for me as well sometimes, so I'm not surprised that it's confusing for people who don't use Avisynth everyday for work and for people who didn't really went through the "evolution" of Avisynth through the years 'till the "normal" high bit depth arrived.
Honestly, if planar was introduced years ago in "vanilla" Avisynth, we wouldn't have had this "mess" of filters supporting stacked, some others supporting interleaved, some others supporting both (like f3kdb) but not planar > 8bit, some others supporting planar only 8bit, some others supporting planar high bit depth (8bit/10bit/12bit/14bit/16bit/32bit) but not stacked nor interleaved ecc.

It does get quite confusing, with almost too may possibilities to understand. I love videos, and the ability to "improve" them--at least to my eyes--makes me happy. Kind people such as yourself are especially appreciated. Danke schön!

[FranceBB]

is there no longer a reason to add extra sharpening after aWarpSharp4xx given the new, improved script? I stole, er, borrowed the original NNEDI3_RPow2-aWarpSharp2-Sharpen upscale block and figured that the "Sharpen(0.2)" call was like a final touch-up to the upscaling.


I don't think it's needed. Spline is a very sharp resizing kernel due to the nature of its implementation and if you need it to be sharper, try raising the "taps" number in NNEDI 'till you'll find the "sweet spot". As you probably know, internally, Spline is limited to Spline16Resize, Spline36Resize and Spline64Resize, however in NNEDI you can choose the number of taps yourself, so you can go above and get Spline144Resize using 12 sampling points and so on. Please note though that a very high number of taps won't be as beneficial as you might expect, which is why people are generally using Spline36 or Spline64... except Katie... She loves Spline144Resize.

[videoh]

If you are using Avisynth+ and DGDecNV, then you have DGSharpen(), which is a very fast CUDA implementation with functionality like LSFmod. It works in 8 or 16-bit depth.

You really should get away from all the high-bit-depth hacks, IMHO, and go for native support.

[LouieChuckyMerry]

I don't think it's needed. Spline is a very sharp resizing kernel due to the nature of its implementation and if you need it to be sharper, try raising the "taps" number in NNEDI 'till you'll find the "sweet spot". As you probably know, internally, Spline is limited to Spline16Resize, Spline36Resize and Spline64Resize, however in NNEDI you can choose the number of taps yourself, so you can go above and get Spline144Resize using 12 sampling points and so on. Please note though that a very high number of taps won't be as beneficial as you might expect, which is why people are generally using Spline36 or Spline64... except Katie... She loves Spline144Resize.

Thanks for the very informative answer--I didn't know that about Spline--and thank you so much for all your instructive help :) .


If you are using Avisynth+ and DGDecNV, then you have DGSharpen(), which is a very fast CUDA implementation with functionality like LSFmod. It works in 8 or 16-bit depth.

You really should get away from all the high-bit-depth hacks, IMHO, and go for native support.

Thanks for letting me know about DGSharpen; I'll definitely check it out. As for the hacks I know that you're right, but when I began trying to learn AviSynth some years ago I tested a dozen different denoisers and found SMDegrain to be by far the best for my needs and, as far as I know, there isn't yet a version available with native 16-bit support. Please correct me if I'm wrong, though. Also, could you point me to a site (or thread, or any place) where filters-plugins with native support are gathered? Thank you.

[videoh]

Thanks for letting me know about DGSharpen; I'll definitely check it out. As for the hacks I know that you're right, but when I began trying to learn AviSynth some years ago I tested a dozen different denoisers and found SMDegrain to be by far the best for my needs and, as far as I know, there isn't yet a version available with native 16-bit support. Please correct me if I'm wrong, though. Also, could you point me to a site (or thread, or any place) where filters-plugins with native support are gathered? Thank you. Hard to say you are wrong about what you find best, although I am a little confused why you are comparing a denoiser to a sharpener. I'm not aware of any gathering like you mentioned.

[real.finder]

like videoh said, go for native HBD

SMDegrain already work with native HBD in many settings included the Default setting for now, other settings waiting for updates for dll's to support HBD, LSFmod work with HBD too

and aside form that, aWarpSharp4 link is not last update, the last one in my signature

[LouieChuckyMerry]

Hard to say you are wrong about what you find best, although I am a little confused why you are comparing a denoiser to a sharpener. I'm not aware of any gathering like you mentioned.

Sorry for the confusion. I wasn't comparing a denoiser to a sharpener, just lamenting the fact that SMDegrain doesn't natively support high bit depth (although real.finder's below post gives me great hope). I'll certainly add DGSharpen to my toolbag, though, and in native high bit depth (which I should find time to understand better). Thanks again :) .


like videoh said, go for native HBD

SMDegrain already work with native HBD in many settings included the Default setting for now, other settings waiting for updates for dll's to support HBD, LSFmod work with HBD too

and aside form that, aWarpSharp4 link is not last update, the last one in my signature

Thank you for the information :) . Would my usual SMDegrain call:

SMDegrain(TR=3,ThSAD=XXX,RefineMotion=True,Plane=0,Chroma=False,LSB=True,LSB_Out=True)

work with native HDB? I'll update my aWarpSharp4 and check out LSFMod. How does LSFMod compare to FastLineDarkenMod?

[videoh]

Thanks for the clarification, LCM. It's all clear to me now. Good luck for your projects! You're in good hands with real.finder.

[real.finder]

Thank you for the information :) . Would my usual SMDegrain call:

SMDegrain(TR=3,ThSAD=XXX,RefineMotion=True,Plane=0,Chroma=False,LSB=True,LSB_Out=True)

work with native HDB?

you need to remove ,LSB=True,LSB_Out=True first since they are for lsb hack

there are many ways, the standard way is just use the SMDegrain with any avs+ HBD


ConvertBits(16) #if your source not HBD already
SMDegrain(tr=4,thSAD=600,contrasharp=false,refinemotion=false,truemotion=true,plane=4)
ConvertBits(8,dither=1) #if you want to get back to 8bit


but this way not as fast as lsb one, why? pinterf and I already talk about this many times in both MVTools-pfmod (https://forum.doom9.org/showthread.php?t=173356) and real.finder's Avisynth Stuff (https://forum.doom9.org/showthread.php?t=174121) replies

so back then pinterf came with another method (https://forum.doom9.org/showthread.php?p=1826390#post1826390)

with SMDegrain it will be like this

#8bit source
c=last

SMDegrain(globals=3,tr=4,PreFilter=4)
global fv1 = fv1.MScalevect(bits=16)
global fv2 = fv2.MScalevect(bits=16)
global fv3 = fv3.MScalevect(bits=16)
global fv4 = fv4.MScalevect(bits=16)
global bv1 = bv1.MScalevect(bits=16)
global bv2 = bv2.MScalevect(bits=16)
global bv3 = bv3.MScalevect(bits=16)
global bv4 = bv4.MScalevect(bits=16)

c
ConvertBits(16)
SMDegrain(globals=1,tr=4,thSAD=600,contrasharp=false,refinemotion=false,truemotion=true,plane=4)
ConvertBits(8,dither=1)


which is faster but still not fast as lsb one, and there are another implementation (https://forum.doom9.org/showthread.php?p=1827085#post1827085) for it

then for more speed pinterf allow MDegrainX passing 8/16/32bit clip and super with 8bit vectors https://forum.doom9.org/showthread.php?p=1834939#post1834939

simply just use SMDegrain with HBD but with 8bit prefilter, but it's still not fast enough https://forum.doom9.org/showthread.php?p=1835018#post1835018

so the last one is using n16=True,n16_Out=True instead of lsb and you will get native 16 HBD clip from your 8bit source with more speed than lsb without hacks

but keep in mind that even if n16 is lsb alternative, it not 100% same (https://forum.doom9.org/showthread.php?p=1837161#post1837161)

[LouieChuckyMerry]

Thank you so much for the information, real.finder :) .

I finally had some time to test things. I've updated all necessary plugins-filters and am using the latest AviSynth+, but I'm struggling to solve the riddle. Here's my original script:

DGIndexNV INFORMATION HERE
### Deinterlace ###
TFM(Mode=7,UBSCO=False)
### Color Conversion ###
ColorMatrix(Mode="Rec.601->Rec.709")
### Adjust Color ###
MergeChroma(aWarpSharp2(Depth=16))
### Crop ###
Crop(8,0,-8,0)
### Fix Frame Borders ###
BorderControl(YTS=1,YTSF=1,YBS=1,YBSF=1)
### Gibbs Noise Block ###
Edge=MT_Edge("prewitt",ThY1=20,ThY2=40).RemoveGrain(17)
Mask=MT_Logic(Edge.MT_Expand().MT_Expand().MT_Expand().MT_Expand(),Edge.MT_Inflate().MT_Inpand(),"xor").Blur(1.0)
MT_Merge(Minblur(),Mask,Luma=True)
### Overall Temporal Denoise ###
SMDegrain(TR=2,ThSAD=200,ContraSharp=True,RefineMotion=True,Plane=0,PreFilter=2,Chroma=False,LSB=True,LSB_Out=True)
### Resize ###
NNEDI3_Resize16(Target_Width=960,Target_Height=720,Mixed=True,ThR=1.0,Elast=1.5,NNS=4,Qual=2,EType=0,PScrn=4,Threads=0,Kernel_D="Spline",Kernel_U="Spline",Taps=6,F_D=1.0,F_U=2.0,Sharp=0,LSB_In=True,LSB=True)
aWarpSharp4xx(Depth=5,LSB=True)
### Darken-Thin Lines ###
F=DitherPost(Mode=-1)
S=F.FastLineDarkenMod4(Strength=24,Prot=6).aWarpSharp2(Blur=4,Type=1,Depth=8,Chroma=2)
D=MT_MakeDiff(S,F).Dither_Convert_8_To_16()
Dither_Add16(Last,D,Dif=True,U=2,V=2)
### Deband ###
F3KDB(Y=100,Cb=100,Cr=100,GrainY=0,GrainC=0,Input_Mode=1,Input_Depth=16,Output_Mode=1,Output_Depth=16)
## Trim()
# SelectRangeEvery(1000,66)
ConvertFromStacked

To test I started with a default SMDegrain call:

DGIndexNV INFORMATION HERE
SMDegrain(n16=True,n16_Out=True)

which output proper, if interlaced, video. Then I inserted the 8-bit lines:

DGIndexNV INFORMATION HERE
### Deinterlace ###
TFM(Mode=7,UBSCO=False)
### Color Conversion ###
ColorMatrix(Mode="Rec.601->Rec.709")
### Adjust Color ###
MergeChroma(aWarpSharp2(Depth=16))
### Crop ###
Crop(8,0,-8,0)
### Fix Frame Borders ###
BorderControl(YTS=1,YTSF=1,YBS=1,YBSF=1)
### Gibbs Noise Block ###
Edge=MT_Edge("prewitt",ThY1=20,ThY2=40).RemoveGrain(17)
Mask=MT_Logic(Edge.MT_Expand().MT_Expand().MT_Expand().MT_Expand(),Edge.MT_Inflate().MT_Inpand(),"xor").Blur(1.0)
MT_Merge(Minblur(),Mask,Luma=True)
### Overall Temporal Denoise ###
SMDegrain(n16=True,n16_Out=True)

which outputs proper deinterlaced video. The trouble begins when I try to resize:

DGIndexNV INFORMATION HERE
### Deinterlace ###
TFM(Mode=7,UBSCO=False)
### Color Conversion ###
ColorMatrix(Mode="Rec.601->Rec.709")
### Adjust Color ###
MergeChroma(aWarpSharp2(Depth=16))
### Crop ###
Crop(8,0,-8,0)
### Fix Frame Borders ###
BorderControl(YTS=1,YTSF=1,YBS=1,YBSF=1)
### Gibbs Noise Block ###
Edge=MT_Edge("prewitt",ThY1=20,ThY2=40).RemoveGrain(17)
Mask=MT_Logic(Edge.MT_Expand().MT_Expand().MT_Expand().MT_Expand(),Edge.MT_Inflate().MT_Inpand(),"xor").Blur(1.0)
MT_Merge(Minblur(),Mask,Luma=True)
### Overall Temporal Denoise ###
SMDegrain(n16=True,n16_Out=True)
### Resize ###
NNEDI3_Resize16(Target_Width=960,Target_Height=720,Mixed=True,ThR=1.0,Elast=1.5,NNS=4,Qual=2,EType=0,PScrn=4,Threads=0,Kernel_D="Spline",Kernel_U="Spline",Taps=6,F_D=1.0,F_U=2.0,Sharp=0,LSB_In=True,LSB=True)
aWarpSharp4xx(Depth=5,LSB=True)

I receive the error message "needi3_resize16: only accept Y8, YV12, YV16, YV24 input". If I add "ConvertToYV16()" after the SMDegrain call, then I still receive the same error message. I've searched a lot and tried many things--to many to actually remember them all--but I can't find a solution. Thanks for any help.

[videoh]

Temporally replace the last two lines with Info() to see what color space the SMDegrain call is delivering.

[LouieChuckyMerry]

Temporally replace the last two lines with Info() to see what color space the SMDegrain call is delivering.

That's so cool. As I was trying to find a solution I, literally, wondered how to know what colorspace SMDegrain was outputing. And the answer is "ColorSpace: YUV420P16, BitsPerComponent: 16". A screenshot (http://www.mediafire.com/file/aydabb5yf8fqu8a/S1.E1-[PAL]-TrimTest[SMD%28n16%29][Info].7z).

[Groucho2004]

As I was trying to find a solution I, literally, wondered how to know what colorspace SMDegrain was outputing.Just for reference:
"AVSMeter script -i" will also provide that info.
[Clip info]
Number of frames: 10000000
Length (hh:mm:ss.ms): 111:06:40.000
Frame width: 32
Frame height: 16
Framerate: 25.000 (25/1)
Colorspace: YUV420P16

[LouieChuckyMerry]

Just for reference:
"AVSMeter script -i" will also provide that info.
[Clip info]
Number of frames: 10000000
Length (hh:mm:ss.ms): 111:06:40.000
Frame width: 32
Frame height: 16
Framerate: 25.000 (25/1)
Colorspace: YUV420P16


Thanks for that, Groucho2004; I actually forgot that I'd added an AVSMeter shortcut to my "Send to" menu. Sadly, my understanding of the Windows cmd line is even less than my understanding of AviSynth. Much much much less. So I can right-click on a script and send it to AVSMeter, but it's going to want to run the whole script. Rather than take the time to make a special script, how would you process "AVSMeter script -i"?

[real.finder]

DGIndexNV INFORMATION HERE
### Deinterlace ###
TFM(Mode=7,UBSCO=False)
### Color Conversion ###
ColorMatrix(Mode="Rec.601->Rec.709")
### Adjust Color ###
MergeChroma(aWarpSharp2(Depth=16))
### Crop ###
Crop(8,0,-8,0)
### Fix Frame Borders ###
BorderControl(YTS=1,YTSF=1,YBS=1,YBSF=1)
### Gibbs Noise Block ###
Edge=MT_Edge("prewitt",ThY1=20,ThY2=40).RemoveGrain(17)
Mask=MT_Logic(Edge.MT_Expand().MT_Expand().MT_Expand().MT_Expand(),Edge.MT_Inflate().MT_Inpand(),"xor").Blur(1.0)
MT_Merge(Minblur(),Mask,Luma=True)
### Overall Temporal Denoise ###
SMDegrain(n16=True,n16_Out=True)
### Resize ###
NNEDI3_Resize16(Target_Width=960,Target_Height=720,Mixed=True,ThR=1.0,Elast=1.5,NNS=4,Qual=2,EType=0,PScrn=4,Threads=0,Kernel_D="Spline",Kernel_U="Spline",Taps=6,F_D=1.0,F_U=2.0,Sharp=0,LSB_In=True,LSB=True)
aWarpSharp4xx(Depth=5,LSB=True)

I receive the error message "needi3_resize16: only accept Y8, YV12, YV16, YV24 input". If I add "ConvertToYV16()" after the SMDegrain call, then I still receive the same error message. I've searched a lot and tried many things--too many to actually remember them all--but I can't find a solution. Thanks for any help.

P.S.--Sorry about the width of the post; this happens sometimes and I'm not sure why :( .

you must not use any lsb filter with HBD, look for alternative filters (or updated ver.) that work with HBD, if you didn't find then you can use this http://avisynth.nl/index.php/ConvertStacked

[LouieChuckyMerry]

you must not use any lsb filter with HBD, look for alternative filters (or updated ver.) that work with HBD, if you didn't find then you can use this http://avisynth.nl/index.php/ConvertStacked

Which filter is LSB? I thought that the latest SMDegrain with "n16=True,n16_Out=True" was OK:

so the last one is using n16=True,n16_Out=True instead of lsb and you will get native 16 HBD clip from your 8bit source with more speed than lsb without hacks

[real.finder]

Which filter is LSB? I thought that the latest SMDegrain with "n16=True,n16_Out=True" was OK:

the problem in the next filters, NNEDI3_Resize16 and aWarpSharp4xx (but this one work with both since I did update it, did also some update to NNEDI3_Resize16 but not for HBD since I chose edi_rpow2)

[LouieChuckyMerry]

Ahhh, I didn't realize NNEDI3_Resize16 was such, thanks for the information (and for updating aWarpSharp4xx). Do you have a recommendation for a resizer that works in native HBD? And, would this then be the correct way to work with SMDegrain-NNEDI3_Resize16:

### Overall Temporal Denoise ###
SMDegrain(TR=2,ThSAD=200,ContraSharp=True,RefineMotion=True,Plane=0,PreFilter=2,Chroma=False,LSB=True,LSB_Out=True)
### Resize ###
NNEDI3_Resize16(Target_Width=960,Target_Height=720,Mixed=True,ThR=1.0,Elast=1.5,NNS=4,Qual=2,EType=0,PScrn=4,Threads=0,Kernel_D="Spline",Kernel_U="Spline",Taps=6,F_D=1.0,F_U=2.0,Sharp=0,LSB_In=True,LSB=True)
aWarpSharp4xx(Depth=5,LSB=True)
### Darken-Thin Lines ###
FastLineDarkenMod4(Strength=24,Prot=6) # Is this working in 16-bits? I couldn't find any info on LSB for FastLineDarkenMod4.
aWarpSharp4xx(Blur=4,Type=1,Depth=8,Chroma=2)
### Deband ###
F3KDB(Y=100,Cb=100,Cr=100,GrainY=0,GrainC=0,Input_Mode=1,Input_Depth=16,Output_Mode=1,Output_Depth=16)
Trim(17666,17777)
# SelectRangeEvery(1000,66)
ConvertFromStacked

Thanks again for your time :) .

[Groucho2004]

Rather than take the time to make a special script, how would you process "AVSMeter script -i"?Add "-i" to your "Target" in the shortcut properties and change the AVSMeter.ini setting "PauseBeforeExit=0" to "PauseBeforeExit=1".