Port of NNEDI under new v2.6 AVS API

[real.finder]

Ahhh, I didn't realize NNEDI3_Resize16 was such, thanks for the information (and for updating aWarpSharp4xx). Do you have a recommendation for a resizer that works in native HBD?
.

edi_rpow2 is like NNEDI3_Resize16 but it work with HBD


And, would this then be the correct way to work with SMDegrain-NNEDI3_Resize16:

### Overall Temporal Denoise ###
SMDegrain(TR=2,ThSAD=200,ContraSharp=True,RefineMotion=True,Plane=0,PreFilter=2,Chroma=False,LSB=True,LSB_Out=True)
### Resize ###
NNEDI3_Resize16(Target_Width=960,Target_Height=720,Mixed=True,ThR=1.0,Elast=1.5,NNS=4,Qual=2,EType=0,PScrn=4,Threads=0,Kernel_D="Spline",Kernel_U="Spline",Taps=6,F_D=1.0,F_U=2.0,Sharp=0,LSB_In=True,LSB=True)
aWarpSharp4xx(Depth=5,LSB=True)
### Darken-Thin Lines ###
FastLineDarkenMod4(Strength=24,Prot=6) # Is this working in 16-bits? I couldn't find any info on LSB for FastLineDarkenMod4.
aWarpSharp4xx(Blur=4,Type=1,Depth=8,Chroma=2)
### Deband ###
F3KDB(Y=100,Cb=100,Cr=100,GrainY=0,GrainC=0,Input_Mode=1,Input_Depth=16,Output_Mode=1,Output_Depth=16)
Trim(17666,17777)
# SelectRangeEvery(1000,66)
ConvertFromStacked

Thanks again for your time :) .

FastLineDarkenMod4 work with HBD, for F3KDB see this (https://forum.doom9.org/showthread.php?p=1864929&highlight=F3KDB#post1864929)

[LouieChuckyMerry]

Add "-i" to your "Target" in the shortcut properties and change the AVSMeter.ini setting "PauseBeforeExit=0" to "PauseBeforeExit=1".

Thank you. I've always had "PauseBeforeExit=1" set--I need time to read ;) --but never would've conceived of adding "-i" to the shortcut properties "Target", which works perfectly.


edi_rpow2 is like NNEDI3_Resize16 but it work with HBD

Ooh wee! Getting EDI_RPow2 to work took some effort, and thanks for the challenge. I received a couple error messages about "what 'color_gray' means"--I've attached snips in order of appearance (http://www.mediafire.com/file/lo391zm39jh8o9o/EDI_RPow2ErrorMessages.7z/file)--and after hashing out (#) the offensive lines in ResizeX.avsi and EDI_RPow2.avsi I was eventually, I think, because "ConvertFromStacked" and my x264 custom cmd line "--demuxer raw --input-depth 16 --sar 1:1" were no longer necessary, able to send native 16-bit video to 10-bit x264 with this script:

### Deinterlace ###
TFM(Mode=7,UBSCO=False)
### Color Conversion ###
ColorMatrix(Mode="Rec.601->Rec.709")
### Adjust Color ###
MergeChroma(aWarpSharp2(Depth=16))
### Crop ###
Crop(8,0,-8,0)
### Fix Frame Borders ###
BorderControl(YTS=1,YTSF=1,YBS=1,YBSF=1)
### Gibbs Noise Block ###
Edge=MT_Edge("prewitt",ThY1=20,ThY2=40).RemoveGrain(17)
Mask=MT_Logic(Edge.MT_Expand().MT_Expand().MT_Expand().MT_Expand(),Edge.MT_Inflate().MT_Inpand(),"xor").Blur(1.0)
MT_Merge(Minblur(),Mask,Luma=True)
### Overall Temporal Denoise ###
# SMDegrain(n16=True,n16_Out=True)
SMDegrain(n16=True,n16_Out=True)
### Resize ###
EDI_RPow2(CShift="Spline64Resize",FWidth=960,FHeight=720)
aWarpSharp4xx(Depth=5)
### Darken-Thin Lines ###
FastLineDarkenMod4(Strength=24,Prot=6)
aWarpSharp4xx(Blur=4,Type=1,Depth=8,Chroma=2)
### Deband ###
F3KDB(Y=100,Cb=100,Cr=100,GrainY=0,GrainC=0,Input_Mode=2,Input_Depth=16,Output_Mode=2,Output_Depth=16)

Now I need to fine-tune my native HBD SMDegrain call....


FastLineDarkenMod4 work with HBD, for F3KDB see this (https://forum.doom9.org/showthread.php?p=1864929&highlight=F3KDB#post1864929)

Thanks for the reminder. Changing the input and output modes from 1 to 2 made it work correctly.

[StainlessS]

Color_Gray ($808080) comes from the avisynth default supplied colors_rgb.avsi file which should be in plugins.
(I usually remove from there unless I need it).

Colors_rgb.avsi Demo:- https://forum.doom9.org/showthread.php?p=1741775#post1741775
https://dl.dropboxusercontent.com/s/akrph1tq1tszmsi/hexyuv.gif

[LouieChuckyMerry]

I'd ignorantly removed the Colors_RGB.avsi from my Plugins folder years ago. After digging up a copy and running a quick test, all is well with unedited versions of ResizeX and Edi_RPow2. Thanks StainlessS :thanks:

[real.finder]

Thanks for the reminder. Changing the input and output modes from 1 to 2 made it work correctly.

you need also ConvertToDoubleWidth() ConvertFromDoubleWidth() to made it work correctly

edit: I did some changes to the wiki http://avisynth.nl/index.php/ConvertStacked

[StainlessS]

LCM, Older version of Colors_rgb.avsi had two entries for Color_PaleGoldenrod, in graphic above, is in 11th page after Gray set, in third and forth rows.
I discovered it after doing the avs to create animated GIF, and whilst watching it.
If you are using old version script, then maybe yours has the double entry, no real problem if it does.

[LouieChuckyMerry]

you need also ConvertToDoubleWidth() ConvertFromDoubleWidth() to made it work correctly

edit: I did some changes to the wiki http://avisynth.nl/index.php/ConvertStacked

Ahhh, thanks for that :) .


I've had some time to tweak my SMDegrain settings with "n16=True,n16_Out=True" and have gotten everything to work but "ContraSharp=True". This runs fine:

SMDegrain(TR=2,ThSAD=200,ContraSharp=False,RefineMotion=True,Plane=0,PreFilter=2,Chroma=False,n16=True,n16_Out=True)

but this:

SMDegrain(TR=2,ThSAD=200,ContraSharp=True,RefineMotion=True,Plane=0,PreFilter=2,Chroma=False,n16=True,n16_Out=True)

gives the error message:

Both clips should have the same colorspace!
([File Path]...SMDegrainMod_rf_3.1.2.101s.avsi, Line 591)
([File Path]...SMDegrainMod_rf_3.1.2.101s.avsi, Line 545)
([File Path]...TrimTest.avs Line 18)


Line 591: ssDD = ssD.repair(HD?ssD.repair(allD,1,planar=planar):allD,HD?12:1,planar=planar) # Limit the difference to the max of what the denoising removed locally.

Line 545: lsb_out ? StackVertical (last, BlankClip (last, pixel_type=type, color_yuv=0,length=1)) : last""")}


LCM, Older version of Colors_rgb.avsi had two entries for Color_PaleGoldenrod, in graphic above, is in 11th page after Gray set, in third and forth rows.
I discovered it after doing the avs to create animated GIF, and whilst watching it.
If you are using old version script, then maybe yours has the double entry, no real problem if it does.

As far as I can tell it's from 8 July 2005 and it does, in fact, have two "global color_palegoldenrod = $EEE8AA" entries, lines 110 and 111. Well, had...

[real.finder]

Ahhh, thanks for that :) .


I've had some time to tweak my SMDegrain settings with "n16=True,n16_Out=True" and have gotten everything to work but "ContraSharp=True". This runs fine:

SMDegrain(TR=2,ThSAD=200,ContraSharp=False,RefineMotion=True,Plane=0,PreFilter=2,Chroma=False,n16=True,n16_Out=True)

but this:

SMDegrain(TR=2,ThSAD=200,ContraSharp=True,RefineMotion=True,Plane=0,PreFilter=2,Chroma=False,n16=True,n16_Out=True)

gives the error message:



Line 591:

Line 545:





try this https://pastebin.com/1HUc33kX

not 100% sure about it, will wait for the answer of this (https://forum.doom9.org/showthread.php?p=1875572#post1875572)

[LouieChuckyMerry]

It works fine, :thanks: .

[StvG]

you need also ConvertToDoubleWidth() ConvertFromDoubleWidth() to made it work correctly

edit: I did some changes to the wiki http://avisynth.nl/index.php/ConvertStacked
Both ConvertToStacked() and ConvertToDoubleWidth() need 16-bit input video. So:## 10bit clip here
ConvertToDoubleWidth()
F3KDB(Input_Mode=2,Input_Depth=10,Output_Mode=2,Output_Depth=10)
ConvertFromDoubleWidth(bits=10)should be ## 10bit clip here
ConvertBits16()
ConvertToDoubleWidth()
F3KDB(Input_Mode=2,Input_Depth=16,Output_Mode=2,Output_Depth=10)
ConvertFromDoubleWidth(bits=10)

[LouieChuckyMerry]

StvG: If input is 16-bit, then would

ConvertToDoubleWidth()
F3KDB(Input_Mode=2,Output_Mode=2)
ConvertFromDoubleWidth()

be correct?

[LouieChuckyMerry]

real.finder: I was wanting to compare my 16-bit native HBD script with it's 16-bit LSB counterpart, but encountered a problem.

Using the latest SMDegrain (it also happens with the "fixed" version from your last post, so I don't think that's the problem), latest NNEDI_Resize16, and latest EDI_RPow2, this scipt:

SMDegrain(LSB=True,LSB_Out=True)
NNEDI3_Resize16(Target_Width=960,Target_Height=720,LSB_In=True,LSB=True)
ConvertFromStacked

outputs proper 960x720 video. However, this script

SMDegrain(LSB=True,LSB_Out=True)
EDI_RPow2(CShift="Spline",FWidth=960,FHeight=720,LSB=True,LSB_Out=True)
ConvertFromStacked

outputs 960x720 video that's still stacked, with the top half stretched out proper video and the bottom half green-acid-flashback.

I'm probably doing something wrong, but I read the EDI_RPow2 .avsi and I think the LSB calls are correct.

[real.finder]

real.finder: I was wanting to compare my 16-bit native HBD script with it's 16-bit LSB counterpart, but encountered a problem.

Using the latest SMDegrain (it also happens with the "fixed" version from your last post, so I don't think that's the problem), latest NNEDI_Resize16, and latest EDI_RPow2, this scipt:

SMDegrain(LSB=True,LSB_Out=True)
NNEDI3_Resize16(Target_Width=960,Target_Height=720,LSB_In=True,LSB=True)
ConvertFromStacked

outputs proper 960x720 video. However, this script

SMDegrain(LSB=True,LSB_Out=True)
EDI_RPow2(CShift="Spline",FWidth=960,FHeight=720,LSB=True,LSB_Out=True)
ConvertFromStacked

outputs 960x720 video that's still stacked, with the top half stretched out proper video and the bottom half green-acid-flashback.

I'm probably doing something wrong, but I read the EDI_RPow2 .avsi and I think the LSB calls are correct.

EDI_RPow2 don't has lsb_in that why, and that why using lsb hack is so annoying and you should avoid it, that aside from most lsb_in things are not real 16bit

[StvG]

StvG: If input is 16-bit, then would

ConvertToDoubleWidth()
F3KDB(Input_Mode=2,Output_Mode=2)
ConvertFromDoubleWidth()

be correct?

Yes, If you want 16-bit output.


real.finder, in wiki input_depth=10 should be input_depth=16 or it can be omitted (input_depth=16 is default).

[real.finder]

real.finder, in wiki input_depth=10 should be input_depth=16 or it can be omitted (input_depth=16 is default).

I fix it with faster way

[StvG]

Thanks.

[LouieChuckyMerry]

EDI_RPow2 don't has lsb_in that why, and that why using lsb hack is so annoying and you should avoid it, that aside from most lsb_in things are not real 16bit

OK, thanks, I must've misunderstood the .avsi.


Yes, If you want 16-bit output.

I do want 16-bit output (I should've indicated that when I asked my question), thank you very much for your answer :) .

[jpsdr]

Going back to original topic after some long off-topic... (from what i've seen, it's been a lot of post not related anymore to the thread topic... :D).
New version, see first post.

[LouieChuckyMerry]

Going back to original topic after some long off-topic... (from what i've seen, it's been a lot of post not related anymore to the thread topic... :D).
New version, see first post.

Sorry about that, jpsdr :o . Thanks for the update :thanks:

[StainlessS]

Sorry about that, jpsdr :o . Thanks for the update :thanks:

What he said +1,
Thanks also LCM, I missed the update post.

[real.finder]

Going back to original topic after some long off-topic... (from what i've seen, it's been a lot of post not related anymore to the thread topic... :D).
New version, see first post.

:thanks:

and I suggest Moderators move those posts to my Thread http://forum.doom9.org/showthread.php?t=174121

[Taurus]

Let me chime in...
Thanks for the update, jpsdr!
:thanks:

[Forteen88]

Thanks!
Change list:
30/05/2018 v0.9.4.52

Shouldn't it be the year 2019?!

[jpsdr]

Yes, i've noticed it for the readme, forget the first page here.

[jpsdr]

New version, see first post.

[StainlessS]

OOOooooh lovely, thanks again.

[jpsdr]

New version, see first post.

[ChaosKing]

Hmm the link works on the first page, but there is no binary linked in the latest release on github?
https://github.com/jpsdr/NNEDI3/releases

[jpsdr]

It seems that something gone wrong with github when i've uploaded the binary, but i didn't notice. It should be fine now.

[mp3dom]

Am I the only one to have problems with the latest version (.54)? Especially with qtgmc, I get lots of black frames at the end of a video (that doesn't have black frames) that I'm going to deinterlace, and lots of ghosting. Keeping all the other plugins the same (mvtools, masktools) and reverting back to version .53 seems to fix everything.

Edit: Ok, I'm not the only one... It's exactly this:
https://forum.doom9.org/showthread.php?p=1910416#post1910416

[jpsdr]

New version, see first post, should be fixed.

[jpsdr]

New version, see first post.

[jpsdr]

New version, see first post.

[jpsdr]

New version, see first post.

[StainlessS]

Cheers jpsdr :)

[kedautinh12]

I wonder why do you add avx and avx2 in one w7 ver and use parameter opt=0 (auto detect) with all your plugins??

[jpsdr]

It's the instruction set the compiler is configured with, but build the same code. Will not make a specific code for each specific instruction set of the compiler. The same code is just build with different cpu instruction set.

[jpsdr]

New version, see first post.

[FranceBB]

Thanks! I'll let you know how it behaves on the 56c/112th Xeon! :D

[servese43]

Hi everyone,
How difficult would it be to port NNEDI3 to linux for use with Avisynth+? (already know there's a vapoursynth port but I'd like to use avisynth+ for its multithreading and audio support.)

Edit: Never mind. Just around the time I posted this it was announced that audio support has been added to vapoursynth anyway.

[jpsdr]

Finaly sooner than expected, new version, see first post.

[tormento]

Finaly sooner than expected, new version, see first post.
I am getting interested into neural resizing.

What about incorporating OpenCL code (https://forum.doom9.org/showthread.php?t=169766)?

AFAIK madVR too uses a OpenCL version of it.

Perhaps even VS version (https://github.com/HomeOfVapourSynthEvolution/VapourSynth-NNEDI3CL) could be of some help.

[jpsdr]

Sorry, not interested in learning and doing OpenCL.

[StainlessS]

Is there any known workaround for this error,
"nnedi3: Error with the Threadpool while getting UserID!"
nnedi3_rpow2 is being called quite a few times (previous successful was 127th call EDIT: Actually 128th call, index was 0 relative).

EDIT: Used in my DBSC script [just added nnedi3 to it], is called once for each scene cut, so can be called many times.
EDIT: v0.9.4.60. Same problem in v0.9.4.56

Actual code used, (_W_ and _H_ will be altered to actual size)

Resizer=Default(Resizer,"""nnedi3_rpow2(rfactor=2,cshift="Spline36Resize",fwidth=_W_,fheight=_H_)""") # Resizer to use


The total count for clip I'm procesing is 873 scene cuts. (about 55 mins clip)

I guess its not a big deal if cannot be fixed, can go back to previous resizer, ie

Resizer=Default(Resizer,"Spline36Resize(_W_,_H_)") # Resizer to use


EDIT: 873 scene cuts.

00009214 1.42599630 DBSC_DynaSplice: 0) Start=0 End=284 Length=285 Crop( 11, 2,696,572) WWR=-13 HHR=-2
00009215 1.50341260 DBSC_DynaSplice: 1) Start=285 End=301 Length=17 Crop( 11, 2,696,572) WWR=-13 HHR=-2
00009216 1.50465596 DBSC_DynaSplice: 2) Start=302 End=313 Length=12 Crop( 11, 2,696,572) WWR=-13 HHR=-2

# ...

00010084 42.17188644 DBSC_DynaSplice: 870) Start=86687 End=86823 Length=137 Crop( 11, 2,696,570) WWR=-13 HHR=-4
00010085 42.22408295 DBSC_DynaSplice: 871) Start=86824 End=86951 Length=128 Crop( 16, 4,684,568) WWR=-20 HHR=-4
00010086 42.27487183 DBSC_DynaSplice: 872) Start=86952 End=87350 Length=399 Crop( 11, 2,696,572) WWR=-13 HHR=-2

[jpsdr]

This message is at the constructor of nnedi3, it faills probably because you reached the limit of 2000 users...
Quick fix, you can rebuild increasing MAX_USERS in ThreadPoolDef.h.
I'll put in my todo list to change some static size arrays with the <vector> class in my threadpool... ;)
It will remove several possible limits, but it will take time, so, not for soon.

[StainlessS]

No sweat right now, I've gone back to Spline36Resize.
I'll keep my eyes out for updates and re-try again later.

Thanks.

[jpsdr]

I'll probably during my holydays next week take the time to make new build increasing the number of max users.
Do you know at how many scene cuts it fails ? Is it 128 to 873 ? (Meaning you need at lest 8 times more users possibles ?)

[StainlessS]

Fails at cut 128, 873 is from when I went back to ordinary Spline36Resize().
Last frame for that clip (bout 55 mins) was about 87,000, so a movie could easily have many more cuts than that [eg Cleopatra, Lizzy Taylor, Dickie Burton].

Enjoy your holiday, no urgency here :)

[jpsdr]

Are you using the jpsdr.dll package with all the filters, or are you using just nnedi3.dll ?

[StainlessS]

or are you using just nnedi3.dll
Just nnedi3.dll,
(I think I may have full package somewhere, but not installed, in a MASSIVE folder of things I gotta do/install/test/plusmore, it goes back years and years :( ).