Port of NNEDI under new v2.6 AVS API

[luquinhas0021]

Do you think to make some improvement on algorithm itself? I know that nnedi3 is a neural network trained for minimize absolute and squared error, however can you explain detailed the nnedi3's work?

[jpsdr]

To make the port, i didn't have to know how the core is working. What you're asking for as nothing to do with what i've done.
So i can't explain because i don't know how the core is working, and i don't intend to make improvement on the algorithm itself.

[GMJCZP]

I tested the latest version with a brief coding (two times for verify) of a short video of 1 min and resizing with nnedi3:

nnedi3 0.9.4.0: 131s
nnedi3 0.9.4.11 (XP version only): 122s

Good job jpsdr!

[luquinhas0021]

jpsdr, is kinda strange you modify a thing you don`t know entirely, not? If you don`t what system color nnedi3 was made for, if it was made for only one color system, how do you did it compatible with YUV 411 and others? Unless you interpolate for YUV 444 and then convert for RGB 888.
GMJCZP, what is your machine? what it hardware?

[jpsdr]

jpsdr, is kinda strange you modify a thing you don`t know entirely, not?

Not if what you're modifying it's not in the part you don't know.

You don't have to know all the theory of how a car engine is working if you want to paint your car of another color, change the seats or the tyres.
Here it's somehow the same thing.

[GMJCZP]

gmjczp, what is your machine? What it hardware?

C2d e4400

[real.finder]

hi, about fturn support, it not work with all color formats, so it willl show error with rgb and yuy2 in nnedi3_rpow2

I suggest make fturn support be in color formats that fturn support it only, or use try as Nnedi3_resize16 did

[Desbreko]

YUY2 works fine because it gets split into separate Y8 clips before the turning happens.

For RGB, you can get around FTurn's limitations by doing this:
# RGB source
Interleave(ShowRed(pixel_type="Y8"),ShowGreen(pixel_type="Y8"),ShowBlue(pixel_type="Y8"))
nnedi3_rpow2(rfactor=2)
MergeRGB(SelectEvery(3,0),SelectEvery(3,1),SelectEvery(3,2))
It would be nice to have nnedi3_rpow2 do it internally, though.

[real.finder]

YUY2 works fine because it gets split into separate Y8 clips before the turning happens.

For RGB, you can get around FTurn's limitations by doing this:
# RGB source
Interleave(ShowRed(pixel_type="Y8"),ShowGreen(pixel_type="Y8"),ShowBlue(pixel_type="Y8"))
nnedi3_rpow2(rfactor=2)
MergeRGB(SelectEvery(3,0),SelectEvery(3,1),SelectEvery(3,2))
It would be nice to have nnedi3_rpow2 do it internally, though.

I just note that the new one work with yuy2, unlike the old one that show error message

but the output is yv16 not yuy2, and this has an effect on AnimeIVTC and SMDegrain and maybe others

[real.finder]

New version, i've tried to fix the chroma shift according the informations on this thread :
http://forum.doom9.org/showthread.php?t=170029

Test and report any issue, any feedback is welcomed.

it's still there, you can test with this http://forum.doom9.org/showpost.php?p=1500031&postcount=377

and this can help too http://forum.doom9.org/showpost.php?p=1506532&postcount=388

[jpsdr]

Ok, i'll try to work further again latter...
I've re-organised the code, it should be easier to read and understand, so, for this specific issue, anyone who want to take a look is and comment is welcomed.
The "difficult" part is that allmost all post give information of offset applied on the resize filter. This offset is applied on the picture before resizing.
nnedi3 is first doing the whole x2 resize(s) parts, and eventualy after a standard resize filter is applied.
So, compensation has to be done first for the x2 parts which were made without the correction, and also eventualy for the final resize (if any) made by standard resizer.

[jpsdr]

hi, about fturn support, it not work with all color formats, so it willl show error with rgb and yuy2 in nnedi3_rpow2

I suggest make fturn support be in color formats that fturn support it only, or use try as Nnedi3_resize16 did
... Are you talking to me ?
I've tested all the color formats, and don't remember any error...

[real.finder]

... Are you talking to me ?
I've tested all the color formats, and don't remember any error...

you have fturn.dll in your autoload folder?

Desbreko confirm that too, and fturn itself didn't support all color formats, now your Nnedi3 + fturn work with all color formats except rgb

[jpsdr]

Ok, no, i don't, i thought it was something internal to avs. Thanks for the information, i'll fix this for next release and don't use it if color is RGB.
Where this dll can be found ? Code source ?

[real.finder]

Ok, no, i don't, i thought it was something internal to avs. Thanks for the information, i'll fix this for next release and don't use it if color is RGB.
Where this dll can be found ? Code source ?

here http://forum.doom9.org/showthread.php?t=168315

but it better to do turn in y8 in all color formats, your Nnedi3 do this now as Desbreko say, but not in rgb, in rgb you can use the code that Desbreko put it here http://forum.doom9.org/showpost.php?p=1733098&postcount=109

and for yuy2 now the output is yv16, and this because YToUV() I think, so it must be convert to yuy2 before output if the source is yuy2

:thanks:

[jpsdr]

I've allready try to split RGB using something like this :

YToUV(a.ShowBlue("Y8"), a.ShowRed("Y8"), a.ShowGreen("Y8"))
Split planes and process
MergeRGB(V,U,Y)

It's a hell lot slower !!

About YUY2, you're right, it's a miss from my part !

[Desbreko]

The Y8 workaround for RGB is slower if you don't have FTurn, but it's faster if you do. Optimally, nnedi3_rpow2 would use the workaround only when FTurn is available, but it's not that big a deal since you can always do it externally.

ColorBarsHD()
KillAudio()
Trim(0,499)
ConvertToRGB24()
Interleave(ShowRed("Y8"),ShowGreen("Y8"),ShowBlue("Y8"))
nnedi3_rpow2(rfactor=2)
MergeRGB(SelectEvery(3,0),SelectEvery(3,1),SelectEvery(3,2),"RGB24")

AVSMeter 2.1.0 (x86)
AviSynth 2.60 (ICL10) (2.6.0.6)
Number of frames: 500
Length (hh:mm:ss.ms): 00:00:16.683
Frame width: 2576
Frame height: 1440
Framerate: 29.970 (30000/1001)
Colorspace: RGB24

No FTurn, RGB24
Frames processed: 500 (0 - 499)
FPS (min | max | average): 8.490 | 10.58 | 9.897
Memory usage (phys | virt): 85 | 81 MB
Thread count: 21
CPU usage (average): 39%
Time (elapsed): 00:00:50.520

No FTurn, Y8 workaround
Frames processed: 500 (0 - 499)
FPS (min | max | average): 7.329 | 8.073 | 7.852
Memory usage (phys | virt): 533 | 534 MB
Thread count: 21
CPU usage (average): 25%
Time (elapsed): 00:01:03.677

With FTurn, Y8 workaround
Frames processed: 500 (0 - 499)
FPS (min | max | average): 11.79 | 14.02 | 13.25
Memory usage (phys | virt): 534 | 534 MB
Thread count: 21
CPU usage (average): 47%
Time (elapsed): 00:00:37.732

Regarding the chroma shift, I think you'd just need to add the appropriate src_left offset to compensate for it when correcting the center shift of the chroma planes. That's what Resize8 (https://www.nmm-hd.org/newbbs/viewtopic.php?t=1323) does to avoid chroma shift from Avs's resizers, so looking at it will probably be helpful.

[jpsdr]

New version.

As said, the chroma shift is not "so easy", because the shift resizer parametters are applied on the picture before doing the resize, and formulas in Resize8 or the thread i've linked previously are all for this case.
But, NNEDI is first doing one (or several) x2 steps using its algorithm. Then, after, we are trying to compensate the chroma shift. Formulas are not the same.

I've made tests, and doing :

video=AviSource("Test.avi").ConvertToY8()
Spline36Resize(video,100,Height(video),src_left=100)

is exactly the same as doing :

video=AviSource("Test.avi").ConvertToY8()
a=Spline36Resize(video,Width(video),Height(video),src_left=100)
Spline36Resize(a,100,Height(video))


But we are trying to reproduce :

video=AviSource("Test.avi").ConvertToY8()
Spline36Resize(video,100,Height(video),src_left=100)

when we are at the step :

video=AviSource("Test.avi").ConvertToY8()
Spline36Resize(video,100,Height(video))

so, after the resize.
It seems that the closest is :

video=AviSource("Test.avi").ConvertToY8()
a=Spline36Resize(video,100,Height(video))
Spline36Resize(a,100,Height(video),src_left=100.0*float(100.0/Width(video)))

But it's too late, it's impossible to get the exact same result than

video=AviSource("Test.avi").ConvertToY8()
Spline36Resize(video,100,Height(video),src_left=100)


According all these tests and the formulas in the thread, i've deduced the formula i've tried in the pluggin.

[jpsdr]

I have implemented internaly the RGB convertion in case FTurn is present, and indeed there is a speedup.
Nice tip, thanks.
Now, just to find out the chroma shift and NNEDI will be perfect ! :cool:

[real.finder]

New version.

As said, the chroma shift is not "so easy", because the shift resizer parametters are applied on the picture before doing the resize, and formulas in Resize8 or the thread i've linked previously are all for this case.
But, NNEDI is first doing one (or several) x2 steps using its algorithm. Then, after, we are trying to compensate the chroma shift. Formulas are not the same.

I've made tests, and doing :

video=AviSource("Test.avi").ConvertToY8()
Spline36Resize(video,100,Height(video),src_left=100)

is exactly the same as doing :

video=AviSource("Test.avi").ConvertToY8()
a=Spline36Resize(video,Width(video),Height(video),src_left=100)
Spline36Resize(a,100,Height(video))


But we are trying to reproduce :

video=AviSource("Test.avi").ConvertToY8()
Spline36Resize(video,100,Height(video),src_left=100)

when we are at the step :

video=AviSource("Test.avi").ConvertToY8()
Spline36Resize(video,100,Height(video))

so, after the resize.
It seems that the closest is :

video=AviSource("Test.avi").ConvertToY8()
a=Spline36Resize(video,100,Height(video))
Spline36Resize(a,100,Height(video),src_left=100.0*float(100.0/Width(video)))

But it's too late, it's impossible to get the exact same result than

video=AviSource("Test.avi").ConvertToY8()
Spline36Resize(video,100,Height(video),src_left=100)


According all these tests and the formulas in the thread, i've deduced the formula i've tried in the pluggin.

thank you for new version

about the shifting

it seems ok with this



r = blankclip(width=64,height=64,pixel_type="YV12",color_yuv=$FF0000)
g = blankclip(width=64,height=64,pixel_type="YV12",color_yuv=$00FF00)

stackhorizontal(r,g)
stackvertical(last,last.fliphorizontal())
o=last

rfactor=2
nWidth=200
nHeight=200

ncWidth=nWidth/2
ncHeight=nHeight/2

fturnright().nnedi3(1, dh=true,nns=2,nsize=3).fturnleft().nnedi3(1, dh=true,nns=2,nsize=3) #rfactor=2
YToUV(utoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),vtoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),ConvertToY8().Spline36Resize(nWidth,nHeight,-0.25*rfactor,-0.25*rfactor))


full test



r = blankclip(width=64,height=64,pixel_type="YV12",color_yuv=$FF0000)
g = blankclip(width=64,height=64,pixel_type="YV12",color_yuv=$00FF00)

stackhorizontal(r,g)
stackvertical(last,last.fliphorizontal())
o=last

rfactor=2
nWidth=200
nHeight=200

ncWidth=nWidth/2
ncHeight=nHeight/2

fturnright().nnedi3(1, dh=true,nns=2,nsize=3).fturnleft().nnedi3(1, dh=true,nns=2,nsize=3) #rfactor=2
YToUV(utoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),vtoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),ConvertToY8().Spline36Resize(nWidth,nHeight,-0.25*rfactor,-0.25*rfactor))
fturnright().nnedi3(1, dh=true,nns=2,nsize=3).fturnleft().nnedi3(1, dh=true,nns=2,nsize=3) #rfactor=2
YToUV(utoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),vtoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),ConvertToY8().Spline36Resize(nWidth,nHeight,-0.25*rfactor,-0.25*rfactor))
fturnright().nnedi3(1, dh=true,nns=2,nsize=3).fturnleft().nnedi3(1, dh=true,nns=2,nsize=3) #rfactor=2
YToUV(utoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),vtoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),ConvertToY8().Spline36Resize(nWidth,nHeight,-0.25*rfactor,-0.25*rfactor))
fturnright().nnedi3(1, dh=true,nns=2,nsize=3).fturnleft().nnedi3(1, dh=true,nns=2,nsize=3) #rfactor=2
YToUV(utoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),vtoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),ConvertToY8().Spline36Resize(nWidth,nHeight,-0.25*rfactor,-0.25*rfactor))
fturnright().nnedi3(1, dh=true,nns=2,nsize=3).fturnleft().nnedi3(1, dh=true,nns=2,nsize=3) #rfactor=2
YToUV(utoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),vtoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),ConvertToY8().Spline36Resize(nWidth,nHeight,-0.25*rfactor,-0.25*rfactor))
fturnright().nnedi3(1, dh=true,nns=2,nsize=3).fturnleft().nnedi3(1, dh=true,nns=2,nsize=3) #rfactor=2
YToUV(utoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),vtoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),ConvertToY8().Spline36Resize(nWidth,nHeight,-0.25*rfactor,-0.25*rfactor))
fturnright().nnedi3(1, dh=true,nns=2,nsize=3).fturnleft().nnedi3(1, dh=true,nns=2,nsize=3) #rfactor=2
YToUV(utoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),vtoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),ConvertToY8().Spline36Resize(nWidth,nHeight,-0.25*rfactor,-0.25*rfactor))
fturnright().nnedi3(1, dh=true,nns=2,nsize=3).fturnleft().nnedi3(1, dh=true,nns=2,nsize=3) #rfactor=2
YToUV(utoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),vtoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),ConvertToY8().Spline36Resize(nWidth,nHeight,-0.25*rfactor,-0.25*rfactor))
fturnright().nnedi3(1, dh=true,nns=2,nsize=3).fturnleft().nnedi3(1, dh=true,nns=2,nsize=3) #rfactor=2
YToUV(utoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),vtoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),ConvertToY8().Spline36Resize(nWidth,nHeight,-0.25*rfactor,-0.25*rfactor))
fturnright().nnedi3(1, dh=true,nns=2,nsize=3).fturnleft().nnedi3(1, dh=true,nns=2,nsize=3) #rfactor=2
YToUV(utoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),vtoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),ConvertToY8().Spline36Resize(nWidth,nHeight,-0.25*rfactor,-0.25*rfactor))
fturnright().nnedi3(1, dh=true,nns=2,nsize=3).fturnleft().nnedi3(1, dh=true,nns=2,nsize=3) #rfactor=2
YToUV(utoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),vtoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),ConvertToY8().Spline36Resize(nWidth,nHeight,-0.25*rfactor,-0.25*rfactor))
fturnright().nnedi3(1, dh=true,nns=2,nsize=3).fturnleft().nnedi3(1, dh=true,nns=2,nsize=3) #rfactor=2
YToUV(utoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),vtoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),ConvertToY8().Spline36Resize(nWidth,nHeight,-0.25*rfactor,-0.25*rfactor))
fturnright().nnedi3(1, dh=true,nns=2,nsize=3).fturnleft().nnedi3(1, dh=true,nns=2,nsize=3) #rfactor=2
YToUV(utoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),vtoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),ConvertToY8().Spline36Resize(nWidth,nHeight,-0.25*rfactor,-0.25*rfactor))

Import(AviSynthPluginsDir + "New Folder (2)/nnedi3_resize16_v3.3.avsi")
interleave(o.nnedi3_resize16(nWidth,nHeight),last)

return(last)


but not sure about rfactor

and it seems that yv411 in nnedi3 is broken, it will show white horizontal lines, and crash after some time

[jpsdr]

Thanks to everyone, i've now my weapon :

function ResizeUnderTest1(clip clp, int SizeX, int SizeY){
return nnedi3_rpow2(clp,2,cshift="spline36resize",fwidth=SizeX,fheight=SizeY)
}


function ResizeUnderTest2(clip clp, int SizeX, int SizeY){
video=nnedi3_rpow2(clp,2,cshift="spline36resize")
return Spline16Resize(video, SizeX, SizeY)
}

function ResizeUnderTest3(clip clp, int SizeX, int SizeY){
video=nnedi3_rpow2(clp,2,cshift="spline36resize")
Y = ConvertToY8(video).Spline36Resize(SizeX, SizeY)
U = UToY8(video).Spline36Resize(Width(Y)/2, Height(Y)/2, src_left=0.25*(1-Float(Width(video))/Width(Y)))
V = VToY8(video).Spline36Resize(Width(Y)/2, Height(Y)/2, src_left=0.25*(1-Float(Width(video))/Width(Y)))
return YToUV(U, V, Y)
}

function ResizeUnderTest4(clip clp, int SizeX, int SizeY){
video=nnedi3_rpow2(clp,2)
return Spline36Resize(video, SizeX, SizeY)
}

function ResizeUnderTest5(clip clp, int SizeX, int SizeY){
video=nnedi3_rpow2(clp,2)
Y = ConvertToY8(video).Spline36Resize(SizeX, SizeY)
U = UToY8(video).Spline36Resize(Width(Y)/2, Height(Y)/2, src_left=0.25*(1-Float(Width(video))/Width(Y)))
V = VToY8(video).Spline36Resize(Width(Y)/2, Height(Y)/2, src_left=0.25*(1-Float(Width(video))/Width(Y)))
return YToUV(U, V, Y)
}

function ResizeUnderTest6(clip clp, int SizeX, int SizeY){
Y = ConvertToY8(clp).Spline36Resize(2*SizeX,2*SizeY)
U = UToY8(clp).Spline36Resize(Width(Y)/2, Height(Y)/2, src_left=0.25*(1-Float(Width(clp))/Width(Y)))
V = VToY8(clp).Spline36Resize(Width(Y)/2, Height(Y)/2, src_left=0.25*(1-Float(Width(clp))/Width(Y)))
video=YToUV(U, V, Y)
Y = ConvertToY8(video).Spline36Resize(SizeX, SizeY)
U = UToY8(video).Spline36Resize(Width(Y)/2, Height(Y)/2, src_left=0.25*(1-Float(Width(video))/Width(Y)))
V = VToY8(video).Spline36Resize(Width(Y)/2, Height(Y)/2, src_left=0.25*(1-Float(Width(video))/Width(Y)))
return YToUV(U, V, Y)
}

function ResizeUnderTest7(clip clp, int SizeX, int SizeY){
Spline36Resize(clip,2*SizeX,2*SizeY)
return Spline36Resize(last, SizeX, SizeY)
}

function ResizeTest(clip clp, int SizeX, int SizeY, int p, int n){
Assert (n>=0)
return (n==1)
\ ? (p==1) ? ResizeUnderTest1(clp,SizeX,SizeY)
\ : (p==2) ? ResizeUnderTest2(clp,SizeX,SizeY)
\ : (p==3) ? ResizeUnderTest3(clp,SizeX,SizeY)
\ : (p==4) ? ResizeUnderTest4(clp,SizeX,SizeY)
\ : (p==5) ? ResizeUnderTest5(clp,SizeX,SizeY)
\ : (p==6) ? ResizeUnderTest6(clp,SizeX,SizeY)
\ : (p==7) ? ResizeUnderTest7(clp,SizeX,SizeY) : ResizeUnderTest1(clp,SizeX,SizeY)
\ : (p==1) ? ResizeUnderTest1(ResizeTest(clp,SizeX,SizeY,p,n-1),SizeX,SizeY)
\ : (p==2) ? ResizeUnderTest2(ResizeTest(clp,SizeX,SizeY,p,n-1),SizeX,SizeY)
\ : (p==3) ? ResizeUnderTest3(ResizeTest(clp,SizeX,SizeY,p,n-1),SizeX,SizeY)
\ : (p==4) ? ResizeUnderTest4(ResizeTest(clp,SizeX,SizeY,p,n-1),SizeX,SizeY)
\ : (p==5) ? ResizeUnderTest5(ResizeTest(clp,SizeX,SizeY,p,n-1),SizeX,SizeY)
\ : (p==6) ? ResizeUnderTest6(ResizeTest(clp,SizeX,SizeY,p,n-1),SizeX,SizeY)
\ : (p==7) ? ResizeUnderTest7(ResizeTest(clp,SizeX,SizeY,p,n-1),SizeX,SizeY) : ResizeUnderTest1(ResizeTest(clp,SizeX,SizeY,p,n-1),SizeX,SizeY)
}

r = blankclip(width=128,height=128,pixel_type="YV12",color_yuv=$FF0000)
g = blankclip(width=128,height=128,pixel_type="YV12",color_yuv=$00FF00)

stackhorizontal(r,g)
stackvertical(last,last.fliphorizontal())
o=last

a=ResizeTest(o,256,256,1,32)
b=ResizeTest(o,256,256,2,32)
c=ResizeTest(o,256,256,3,32)
ResizeTest(o,256,256,4,32)

interleave(o,a,b,c,last)

return(last)


Test in progress...:D

[Reel.Deel]

@jpsdr

Thanks for all your efforts in trying to fix this long standing issue with nnedi3_rpow2. One suggestion, can you add a chroma placement parameter to choose "MPEG1" or "MPEG2"? I assume right now it is (or will be) hardcoded for MPEG2 but I sometimes use it on Jpegs with MPEG1 chroma placement.

I always like to be safe so I would do something like this in your script above:

r = blankclip(width=128,height=128,pixel_type="YV24",color_yuv=$FF0000).ConvertToYV12(ChromaOutPlacement="MPEG2")
g = blankclip(width=128,height=128,pixel_type="YV24",color_yuv=$00FF00).ConvertToYV12(ChromaOutPlacement="MPEG2")

[Desbreko]

As said, the chroma shift is not "so easy", because the shift resizer parametters are applied on the picture before doing the resize, and formulas in Resize8 or the thread i've linked previously are all for this case.
But, NNEDI is first doing one (or several) x2 steps using its algorithm. Then, after, we are trying to compensate the chroma shift. Formulas are not the same.

Seems to work fine for me, at least on YV12.

ColorBars(pixel_type="YV12")

# The rfactor and output width and height values we want to use for testing
rfac = 2
ow = 800
oh = 600

w = Width()
h = Height()

# For the ratio of the output and input chroma widths used in the chroma shift correction formula, the input width is the
# width after all doubling by nnedi3 is done, so we have to multiply the original clip's chroma width by the rfactor value.
w_c = w/2
ow_c = ow/2
dblw_c = w_c*rfac
c_ratio = Float(ow_c)/Float(dblw_c)

# Center shift correction src_left and src_top values for YV12
hshift_y = Float(-rfac)/2.0+0.5
vshift_y = -0.5
hshift_c = hshift_y/2.0
vshift_c = vshift_y/2.0

# Add the chroma shift correction to the center shift correction to get the final src_left value
hshift_c_fix = 0.25-0.25/c_ratio
vshift_c_yv12 = -0.25
hshift_c_final = hshift_c+hshift_c_fix
vshift_c_final = vshift_c+vshift_c_yv12


# Dither_resize16 doesn't cause chroma shift, so we can check our results against it.
r16 = Dither_convert_8_to_16().Dither_resize16(ow,oh).DitherPost(mode=-1)

# No center shift correction
rpow2 = nnedi3_rpow2(rfac)

# Internal center shift correction that causes chroma shift
rpow2cs = nnedi3_rpow2(rfac, cshift="Spline36Resize", fwidth=ow, fheight=oh)

# External center shift correction with Resize8 to avoid chroma shift, but this causes the chroma to be resized twice
rpow2r8 = rpow2.Resize8(ow,oh,hshift_y,vshift_y, kernel="Spline36", kernel_c="Spline36", noring=false, noring_c=false)

# No center shift correction
y1 = ConvertToY8().nnedi3double(rfac)
u1 = UToY8().nnedi3double(rfac).Spline36Resize(w/2*rfac,h/2*rfac,0,vshift_c_yv12)
v1 = VToY8().nnedi3double(rfac).Spline36Resize(w/2*rfac,h/2*rfac,0,vshift_c_yv12)
dbl1 = YToUV(u1,v1,y1)

# Center shift correction without chroma shift and only one chroma resize
y2 = ConvertToY8().nnedi3double(rfac).Spline36Resize(ow,oh,hshift_y,vshift_y)
u2 = UToY8().nnedi3double(rfac).Spline36Resize(ow/2,oh/2,hshift_c_final,vshift_c_final)
v2 = VToY8().nnedi3double(rfac).Spline36Resize(ow/2,oh/2,hshift_c_final,vshift_c_final)
dbl2 = YToUV(u2,v2,y2)

Interleave(r16,dbl2)#.UToY8()


function nnedi3double(clip input, int rfactor) {
return rfactor == 2 ? input.nnedi3(1,dh=true,nsize=0,nns=3,U=false,V=false).FTurnRight().nnedi3(1,dh=true,nsize=0,nns=3,U=false,V=false).FTurnLeft()
\ : rfactor == 4 ? input.nnedi3(1,dh=true,nsize=0,nns=3,U=false,V=false).FTurnRight().nnedi3(1,dh=true,nsize=0,nns=3,U=false,V=false).FTurnLeft()
\ .nnedi3(0,dh=true,nsize=0,nns=3,U=false,V=false).FTurnRight().nnedi3(1,dh=true,nsize=0,nns=3,U=false,V=false).FTurnLeft()
\ : Assert(false, "nnedi3double: Too lazy to support more than rfactor 2 and 4 for this test")
}

[real.finder]

@jpsdr

Thanks for all your efforts in trying to fix this long standing issue with nnedi3_rpow2. One suggestion, can you add a chroma placement parameter to choose "MPEG1" or "MPEG2"? I assume right now it is (or will be) hardcoded for MPEG2 but I sometimes use it on Jpegs with MPEG1 chroma placement.

I always like to be safe so I would do something like this in your script above:

r = blankclip(width=128,height=128,pixel_type="YV24",color_yuv=$FF0000).ConvertToYV12(ChromaOutPlacement="MPEG2")
g = blankclip(width=128,height=128,pixel_type="YV24",color_yuv=$00FF00).ConvertToYV12(ChromaOutPlacement="MPEG2")


r = blankclip(width=64,height=64,pixel_type="rgb24",color=$FF0000)
g = blankclip(width=64,height=64,pixel_type="rgb24",color=$00FF00)

stackhorizontal(r,g)
stackvertical(last,last.fliphorizontal())
ConvertToYV12(ChromaOutPlacement="MPEG2") #or ConvertToYV12(ChromaOutPlacement="MPEG1") for mpeg1 (jpeg)
o=last

rfactor=2
nWidth=200
nHeight=200

ncWidth=nWidth/2
ncHeight=nHeight/2

fturnright().nnedi3(1, dh=true,nns=2,nsize=3).fturnleft().nnedi3(1, dh=true,nns=2,nsize=3) #rfactor=2
YToUV(utoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),vtoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),ConvertToY8().Spline36Resize(nWidth,nHeight,-0.25*rfactor,-0.25*rfactor))
fturnright().nnedi3(1, dh=true,nns=2,nsize=3).fturnleft().nnedi3(1, dh=true,nns=2,nsize=3) #rfactor=2
YToUV(utoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),vtoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),ConvertToY8().Spline36Resize(nWidth,nHeight,-0.25*rfactor,-0.25*rfactor))
fturnright().nnedi3(1, dh=true,nns=2,nsize=3).fturnleft().nnedi3(1, dh=true,nns=2,nsize=3) #rfactor=2
YToUV(utoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),vtoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),ConvertToY8().Spline36Resize(nWidth,nHeight,-0.25*rfactor,-0.25*rfactor))
fturnright().nnedi3(1, dh=true,nns=2,nsize=3).fturnleft().nnedi3(1, dh=true,nns=2,nsize=3) #rfactor=2
YToUV(utoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),vtoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),ConvertToY8().Spline36Resize(nWidth,nHeight,-0.25*rfactor,-0.25*rfactor))
fturnright().nnedi3(1, dh=true,nns=2,nsize=3).fturnleft().nnedi3(1, dh=true,nns=2,nsize=3) #rfactor=2
YToUV(utoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),vtoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),ConvertToY8().Spline36Resize(nWidth,nHeight,-0.25*rfactor,-0.25*rfactor))
fturnright().nnedi3(1, dh=true,nns=2,nsize=3).fturnleft().nnedi3(1, dh=true,nns=2,nsize=3) #rfactor=2
YToUV(utoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),vtoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),ConvertToY8().Spline36Resize(nWidth,nHeight,-0.25*rfactor,-0.25*rfactor))
fturnright().nnedi3(1, dh=true,nns=2,nsize=3).fturnleft().nnedi3(1, dh=true,nns=2,nsize=3) #rfactor=2
YToUV(utoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),vtoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),ConvertToY8().Spline36Resize(nWidth,nHeight,-0.25*rfactor,-0.25*rfactor))
fturnright().nnedi3(1, dh=true,nns=2,nsize=3).fturnleft().nnedi3(1, dh=true,nns=2,nsize=3) #rfactor=2
YToUV(utoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),vtoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),ConvertToY8().Spline36Resize(nWidth,nHeight,-0.25*rfactor,-0.25*rfactor))
fturnright().nnedi3(1, dh=true,nns=2,nsize=3).fturnleft().nnedi3(1, dh=true,nns=2,nsize=3) #rfactor=2
YToUV(utoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),vtoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),ConvertToY8().Spline36Resize(nWidth,nHeight,-0.25*rfactor,-0.25*rfactor))
fturnright().nnedi3(1, dh=true,nns=2,nsize=3).fturnleft().nnedi3(1, dh=true,nns=2,nsize=3) #rfactor=2
YToUV(utoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),vtoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),ConvertToY8().Spline36Resize(nWidth,nHeight,-0.25*rfactor,-0.25*rfactor))
fturnright().nnedi3(1, dh=true,nns=2,nsize=3).fturnleft().nnedi3(1, dh=true,nns=2,nsize=3) #rfactor=2
YToUV(utoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),vtoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),ConvertToY8().Spline36Resize(nWidth,nHeight,-0.25*rfactor,-0.25*rfactor))
fturnright().nnedi3(1, dh=true,nns=2,nsize=3).fturnleft().nnedi3(1, dh=true,nns=2,nsize=3) #rfactor=2
YToUV(utoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),vtoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),ConvertToY8().Spline36Resize(nWidth,nHeight,-0.25*rfactor,-0.25*rfactor))
fturnright().nnedi3(1, dh=true,nns=2,nsize=3).fturnleft().nnedi3(1, dh=true,nns=2,nsize=3) #rfactor=2
YToUV(utoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),vtoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),ConvertToY8().Spline36Resize(nWidth,nHeight,-0.25*rfactor,-0.25*rfactor))

Import(AviSynthPluginsDir + "New Folder (2)/nnedi3_resize16_v3.3.avsi")
interleave(o.nnedi3_resize16(nWidth,nHeight,cplace="MPEG2"),last)

return(last)


seems ok

edit: but here not


ImageSource("E:\PM5544_with_non-PAL_signals.png") #http://forum.doom9.org/showthread.php?p=1706504#post1706504

ConvertToYV12(ChromaOutPlacement="MPEG2") #or ConvertToYV12(ChromaOutPlacement="MPEG1") for mpeg1 (jpeg)
o=last

rfactor=2
nWidth=Width*32
nHeight=576

ncWidth=nWidth/2
ncHeight=nHeight/2

fturnright().nnedi3(1, dh=true,nns=2,nsize=3).fturnleft().nnedi3(1, dh=true,nns=2,nsize=3) #rfactor=2
YToUV(utoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),vtoy8().Spline36Resize(ncWidth,ncHeight,-0.25*rfactor,-0.25*rfactor),ConvertToY8().Spline36Resize(nWidth,nHeight,-0.25*rfactor,-0.25*rfactor))

Import(AviSynthPluginsDir + "New Folder (2)/nnedi3_resize16_v3.3.avsi")
interleave(o.nnedi3_resize16(nWidth,nHeight,cplace="MPEG2"),last)

ConvertToRGB(ChromaInPlacement="MPEG2")
Spline36Resize(Width/32, height)
Crop(240,460,-240,-30)
PointResize(Width*2, Height*2) #for better visibility

[jpsdr]

I'm still struggling, it may take a lot longer than i thought...

For adding a parameter, i'll see. If i do, it will be on the end of list, to not broke existing scripts.

Find out the issue of YV411, fix in next release.

[Reel.Deel]

For adding a parameter, i'll see. If i do, it will be on the end of list, to not broke existing scripts.


That would be the correct thing to do.

[jpsdr]

New version.

Chroma shift issue is now totaly gone. You can torture it as you want, it will not fail ! :cool:
(I hope... :p)

Otherwise, i've out of curiosity made some benchmark.

Without FTurn
YV12 : 35.946
YV411 : 37.768
YYU2 : 44.071
YV16 : 43.417
YV24 : 59.304
RGB24 : 01:55.424

With FTurn
YV12 : 36.013
YV411 : 37.797
YUY2 : 43.781
YV16 : 43.379
YV24 : 59.102
RGB24 : 01:37.585

Well... Except RGB24, there is not realy big difference.
This make me wondering if the trick for RGB24 will also be faster even without FTurn... I'll check this later, but not today.

Oh... And it seems that there is no need for an MPEG1/MPEG2 parameter, not being able to see any difference.

[real.finder]

New version.

Chroma shift issue is now totaly gone. You can torture it as you want, it will not fail ! :cool:
(I hope... :p)

Otherwise, i've out of curiosity made some benchmark.

Without FTurn
YV12 : 35.946
YV411 : 37.768
YYU2 : 44.071
YV16 : 43.417
YV24 : 59.304
RGB24 : 01:55.424

With FTurn
YV12 : 36.013
YV411 : 37.797
YUY2 : 43.781
YV16 : 43.379
YV24 : 59.102
RGB24 : 01:37.585

Well... Except RGB24, there is not realy big difference.
This make me wondering if the trick for RGB24 will also be faster even without FTurn... I'll check this later, but not today.

Oh... And it seems that there is no need for an MPEG1/MPEG2 parameter, not being able to see any difference.

thanks for the new ver

but seems you didn't take the yv411 spec into account


ImageSource("E:\PM5544_with_non-PAL_signals.png")

ConvertToYV411()
#~ ConvertToYV12(ChromaOutPlacement="MPEG2") #for mpeg2 (dvd bd)
#~ ConvertToYV12(ChromaOutPlacement="MPEG1") #for mpeg1 (jpeg)

o=last

nWidth=Width*32
nHeight=576

nnedi3_rpow2(2,cshift="spline36resize",fwidth=nWidth,fheight=nHeight)

fixresize=o.Dither_convert_8_to_16().Dither_Resize16(nWidth, nHeight, kernel="Spline36").Ditherpost(mode=-1)

interleave(fixresize,last)

ConvertToRGB()
#~ ConvertToRGB(ChromaInPlacement="MPEG2") #if you test yv12 only

Spline36Resize(Width/32, height)
Crop(240,460,-240,-30)
PointResize(Width*2, Height*2) #for better visibility


almost same thing for yv12 in mpeg2, more details http://www.mir.com/DMG/chroma.html

[jpsdr]

I allready know the chroma link, very interesting.
But actualy, it passes the test : http://forum.doom9.org/showpost.php?p=1500031&postcount=377
If you change of any little value the actual chroma shift, the test fails.
So, i don't understand what's still wrong, or finaly the test you've provided link is not good ? :confused:

[real.finder]

I allready know the chroma link, very interesting.
But actualy, it passes the test : http://forum.doom9.org/showpost.php?p=1500031&postcount=377
If you change of any little value the actual chroma shift, the test fails.
So, i don't understand what's still wrong, or finaly the test you've provided link is not good ? :confused:

this http://forum.doom9.org/showpost.php?p=1500031&postcount=377 was for the shift issue in nnedi3_rpow2

this http://forum.doom9.org/showthread.php?p=1733493#post1733493 is caused by the avs resizer for treats everything as mpeg1

[Groucho2004]

Otherwise, i've out of curiosity made some benchmark.

Without FTurn
YV12 : 35.946
YV411 : 37.768
YYU2 : 44.071
YV16 : 43.417
YV24 : 59.304
RGB24 : 01:55.424

With FTurn
YV12 : 36.013
YV411 : 37.797
YUY2 : 43.781
YV16 : 43.379
YV24 : 59.102
RGB24 : 01:37.585

Well... Except RGB24, there is not realy big difference.
This make me wondering if the trick for RGB24 will also be faster even without FTurn... I'll check this later, but not today.
Which version of Avisynth did you use for the benchmark?

[jpsdr]

@Grouncho
avs+ r1576 x64

@real.finder
If i implement indeed the chroma correction shift of the resize explained here (http://forum.doom9.org/showthread.php?t=170029), the test will fail for output size different than rf*original size. Is it intended to ?
By the way, from what i've understood on the thread, 4:1:1 don't need the correction, so my actual code should be correct... (unless i've misunderstood something).

[Groucho2004]

@Grouncho
avs+ r1576 x64
You're aware that AVS+ has FTurn already built in, right? Therefore the performance with or without the FTurn plugin will be more or less identical.

[jpsdr]

Euh... No... But in that case, why when i remove the Fturn dll, i should have the same behavior, because in both cases the function should be detected, and clearly on RGB24, it can be seen it's not... So...?

[Groucho2004]

Euh... No... But in that case, why when i remove the Fturn dll, i should have the same behavior, because in both cases the function should be detected, and clearly on RGB24, it can be seen it's not... So...?
Don't know. You'll have to compare the code of TP7's Fturn and AVS+ internal implementation.

[real.finder]

@Grouncho
avs+ r1576 x64

@real.finder
If i implement indeed the chroma correction shift of the resize explained here (http://forum.doom9.org/showthread.php?t=170029), the test will fail for output size different than rf*original size. Is it intended to ?
By the way, from what i've understood on the thread, 4:1:1 don't need the correction, so my actual code should be correct... (unless i've misunderstood something).

see this http://forum.doom9.org/showthread.php?p=1706504#post1706504

and about 4:1:1, if the yv12 need src_left=-0.5 the yv411 will need src_left=-1.5

see here http://avisynth.nl/index.php/Known_Issues#Resizers:_Chroma_Positioning_Bugs and http://forum.doom9.org/showthread.php?p=1506374#post1506374

[real.finder]

Euh... No... But in that case, why when i remove the Fturn dll, i should have the same behavior, because in both cases the function should be detected, and clearly on RGB24, it can be seen it's not... So...?

the avs+ have Fturn as normal turn in name

that difference in speed can happen even in the same script and conditions

[jpsdr]

@real.finder
Thanks for these informations, i'll check them later, not this WE. I've the feeling that i'll lose my hairs again.

@Grouncho
Ah... You mean that the internal turn use the code of fturn, not that fturn exist internally... Ok.
Well... It means i'll have to test with avs 2.60...

Edit
... In that case, it will be difficult to create a code specific when fturn is avaible. I'll end up bold...

[Reel.Deel]

Euh... No... But in that case, why when i remove the Fturn dll, i should have the same behavior, because in both cases the function should be detected, and clearly on RGB24, it can be seen it's not... So...?

It makes sense that RGB24 is slower when FTurn is not available, according to this post by tp7 it sounds like there no optimized turning function for RGB24.


7) Code from FTurn (http://forum.doom9.org/showthread.php?t=168315) is now integrated into the core (with some additional optimizations and new RGB32 routines).


So when FTurn is not available your using the regular TurnRight/Left but when FTurn is available, RGB24 gets split into separate channels (Y8 in this case) and processed with the optimized turning functions.


Oh... And it seems that there is no need for an MPEG1/MPEG2 parameter, not being able to see any difference.

Hmm, that's odd. If everything's correct there should a small difference between the two. I'll do some testing and report back. What method did you use to test this?

[Groucho2004]

It makes sense that RGB24 is slower when FTurn is not available, according to this post by tp7 it sounds like there no optimized turning function for RGB24.

So when FTurn is not available your using the regular TurnRight/Left but when FTurn is available, RGB24 gets split into separate channels (Y8 in this case) and processed with the optimized turning functions.
Thanks for digging these posts up. :)

[jpsdr]

But, if i'm under avs+, fturn is avaible without knowing...

My script test :

function ResizeUnderTest1(clip clp, int SizeX, int SizeY){
return nnedi3_rpow2(clp,2,cshift="spline36resize",fwidth=SizeX,fheight=SizeY)
}


function ResizeUnderTest2(clip clp, int SizeX, int SizeY){
video=nnedi3_rpow2(clp,2,cshift="spline36resize")
return Spline36Resize(video, SizeX, SizeY)
}

function ResizeUnderTest3_(clip clp, int SizeX, int SizeY){
video=nnedi3_rpow2(clp,4,cshift="spline36resize")
Y = ConvertToY8(video).Spline36Resize(SizeX, SizeY)
U = UToY8(video).Spline36Resize(Width(Y)/4, Height(Y), src_left=0.25*(1-Float(Width(video))/Width(Y)))
V = VToY8(video).Spline36Resize(Width(Y)/4, Height(Y), src_left=0.25*(1-Float(Width(video))/Width(Y)))
return YToUV(U, V, Y)
}

function ResizeUnderTest3(clip clp, int SizeX, int SizeY){
video=nnedi3_rpow2(clp,4,cshift="spline36resize")
Y = ConvertToY8(video).Spline36Resize(SizeX, SizeY)
U = UToY8(video).Spline36Resize(Width(Y)/2, Height(Y)/2, src_left=0.5*(1-Float(Width(video))/Width(Y)))
V = VToY8(video).Spline36Resize(Width(Y)/2, Height(Y)/2, src_left=0.5*(1-Float(Width(video))/Width(Y)))
return YToUV(U, V, Y)
}

function ResizeUnderTest4_(clip clp, int SizeX, int SizeY){
video=nnedi3_rpow2(clp,2)
return Spline36Resize(video, SizeX, SizeY)
}

function ResizeUnderTest4(clip clp, int SizeX, int SizeY){
return nnedi3_rpow2(clp,2,cshift="spline36resize")
}

function ResizeUnderTest5(clip clp, int SizeX, int SizeY){
return nnedi3_rpow2(clp,2)
}


#YV12
function ResizeUnderTest6_(clip clp, int SizeX, int SizeY){
rf=16
video=nnedi3_rpow2(clp,rf)

Y_hshift = -0.5
Y_vshift = -0.5
C_hshift = Y_hshift
C_vshift = Y_vshift

# YV12 seulement
C_vshift = C_vshift-0.5
C_vshift = C_vshift/2.0
C_hshift = C_hshift/2.0

#Correct ressampling chroma shift
#First correct the first past increasing of rf
C_hshift = C_hshift-0.25*(rf-1)
#Correct for the final resolution
#C_hshift = C_hshift+0.25*(1.0-(Float(Width(video))/Float(SizeX)))

#Y = ConvertToY8(video).Spline36Resize(Width(video), Height(video), src_left=Y_hshift, src_top=Y_vshift)
Y = ConvertToY8(video).Spline36Resize(SizeX,SizeY, src_left=Y_hshift, src_top=Y_vshift)
U = UToY8(video).Spline36Resize(Width(Y)/2, Height(Y)/2, src_left=C_hshift, src_top=C_vshift)
V = VToY8(video).Spline36Resize(Width(Y)/2, Height(Y)/2, src_left=C_hshift, src_top=C_vshift)
return YToUV(U, V, Y)
}


#YV16
function ResizeUnderTest6_(clip clp, int SizeX, int SizeY){
rf=16
video=nnedi3_rpow2(clp,rf)

Y_hshift = -0.5
Y_vshift = -0.5
C_hshift = Y_hshift
C_vshift = Y_vshift

C_hshift = C_hshift/2.0

#Correct ressampling chroma shift
#First correct the first past increasing of rf
C_hshift = C_hshift-0.25*(rf-1)
#Correct for the final resolution
#C_hshift = C_hshift+0.25*(1.0-(Float(Width(video))/Float(SizeX)))

#Y = ConvertToY8(video).Spline36Resize(Width(video), Height(video), src_left=Y_hshift, src_top=Y_vshift)
Y = ConvertToY8(video).Spline36Resize(SizeX,SizeY, src_left=Y_hshift, src_top=Y_vshift)
U = UToY8(video).Spline36Resize(Width(Y)/2, Height(Y), src_left=C_hshift, src_top=C_vshift)
V = VToY8(video).Spline36Resize(Width(Y)/2, Height(Y), src_left=C_hshift, src_top=C_vshift)
return YToUV(U, V, Y)
}


#YV24
function ResizeUnderTest6_(clip clp, int SizeX, int SizeY){
rf=2
video=nnedi3_rpow2(clp,rf)

Y_hshift = -0.5
Y_vshift = -0.5
C_hshift = Y_hshift
C_vshift = Y_vshift

C_hshift = C_hshift

#Correct ressampling chroma shift
#First correct the first past increasing of rf
#C_hshift = C_hshift-0.25*(rf-1)
#Correct for the final resolution
#C_hshift = C_hshift+0.25*(1.0-(Float(Width(video))/Float(SizeX)))

#Y = ConvertToY8(video).Spline36Resize(Width(video), Height(video), src_left=Y_hshift, src_top=Y_vshift)
Y = ConvertToY8(video).Spline36Resize(SizeX,SizeY, src_left=Y_hshift, src_top=Y_vshift)
U = UToY8(video).Spline36Resize(Width(Y), Height(Y), src_left=C_hshift, src_top=C_vshift)
V = VToY8(video).Spline36Resize(Width(Y), Height(Y), src_left=C_hshift, src_top=C_vshift)
return YToUV(U, V, Y)
}

#YV411
function ResizeUnderTest6(clip clp, int SizeX, int SizeY){
rf=16
video=nnedi3_rpow2(clp,rf)

Y_hshift = -0.5
Y_vshift = -0.5
C_hshift = Y_hshift
C_vshift = Y_vshift

#C_hshift = C_hshift/4.0
C_hshift = 0

#Correct ressampling chroma shift
#First correct the first past increasing of rf
C_hshift = C_hshift-0.5*(rf-1)
#Correct for the final resolution
#C_hshift = C_hshift+0.25*(1.0-(Float(Width(video))/Float(SizeX)))

#Y = ConvertToY8(video).Spline36Resize(Width(video), Height(video), src_left=Y_hshift, src_top=Y_vshift)
Y = ConvertToY8(video).Spline36Resize(SizeX,SizeY, src_left=Y_hshift, src_top=Y_vshift)
U = UToY8(video).Spline36Resize(Width(Y)/4, Height(Y), src_left=C_hshift, src_top=C_vshift)
V = VToY8(video).Spline36Resize(Width(Y)/4, Height(Y), src_left=C_hshift, src_top=C_vshift)
return YToUV(U, V, Y)
}


function ResizeUnderTest5_(clip clp, int SizeX, int SizeY){
video=nnedi3_rpow2(clp,2)
Y = ConvertToY8(video).Spline36Resize(SizeX, SizeY)
U = UToY8(video).Spline36Resize(Width(Y)/2, Height(Y)/2, src_left=0.25*(1-Float(Width(video))/Width(Y)))
V = VToY8(video).Spline36Resize(Width(Y)/2, Height(Y)/2, src_left=0.25*(1-Float(Width(video))/Width(Y)))
return YToUV(U, V, Y)
}

function ResizeUnderTest6_(clip clp, int SizeX, int SizeY){
Y = ConvertToY8(clp).Spline36Resize(2*SizeX,2*SizeY)
U = UToY8(clp).Spline36Resize(Width(Y)/2, Height(Y)/2, src_left=0.25*(1-Float(Width(clp))/Width(Y)))
V = VToY8(clp).Spline36Resize(Width(Y)/2, Height(Y)/2, src_left=0.25*(1-Float(Width(clp))/Width(Y)))
video=YToUV(U, V, Y)
Y = ConvertToY8(video).Spline36Resize(SizeX, SizeY)
U = UToY8(video).Spline36Resize(Width(Y)/2, Height(Y)/2, src_left=0.25*(1-Float(Width(video))/Width(Y)))
V = VToY8(video).Spline36Resize(Width(Y)/2, Height(Y)/2, src_left=0.25*(1-Float(Width(video))/Width(Y)))
return YToUV(U, V, Y)
}

function ResizeUnderTest7(clip clp, int SizeX, int SizeY){
Spline36Resize(clip,2*SizeX,2*SizeY)
return Spline36Resize(last, SizeX, SizeY)
}


function ResizeUnderTest(clip clp, int SizeX, int SizeY){

#return nnedi3_rpow2(clp,2,cshift="spline36resize",fwidth=SizeX,fheight=SizeY)

video=nnedi3_rpow2(clp,2,cshift="spline36resize")
#video=nnedi3_rpow2(clp,2)
#Y = ConvertToY8(video).spline36resize(SizeX, SizeY)
#U = UToY8(video).spline36resize(Width(Y)/2, Height(Y)/2, src_left=0.25*(1-Float(Width(video))/Width(Y)))
#V = VToY8(video).spline36resize(Width(Y)/2, Height(Y)/2, src_left=0.25*(1-Float(Width(video))/Width(Y)))
#return YToUV(U, V, Y)
return spline36resize(video, SizeX, SizeY)

#Y = ConvertToY8(clp).spline36resize(SizeX, SizeY)
#U = UToY8(clp).spline36resize(Width(Y)/2, Height(Y)/2, src_left=0.25*(1-Float(Width(clp))/Width(Y)))
#V = VToY8(clp).spline36resize(Width(Y)/2, Height(Y)/2, src_left=0.25*(1-Float(Width(clp))/Width(Y)))
#return YToUV(U, V, Y)

#return spline36resize(clp, SizeX, SizeY)
}


function ResizeTest(clip clp, int SizeX, int SizeY, int p, int n){
Assert (n>=0)
return (n==1)
\ ? (p==1) ? ResizeUnderTest1(clp,SizeX,SizeY)
\ : (p==2) ? ResizeUnderTest2(clp,SizeX,SizeY)
\ : (p==3) ? ResizeUnderTest3(clp,SizeX,SizeY)
\ : (p==4) ? ResizeUnderTest4(clp,SizeX,SizeY)
\ : (p==5) ? ResizeUnderTest5(clp,SizeX,SizeY)
\ : (p==6) ? ResizeUnderTest6(clp,SizeX,SizeY)
\ : (p==7) ? ResizeUnderTest7(clp,SizeX,SizeY) : ResizeUnderTest1(clp,SizeX,SizeY)
\ : (p==1) ? ResizeUnderTest1(ResizeTest(clp,SizeX,SizeY,p,n-1),SizeX,SizeY)
\ : (p==2) ? ResizeUnderTest2(ResizeTest(clp,SizeX,SizeY,p,n-1),SizeX,SizeY)
\ : (p==3) ? ResizeUnderTest3(ResizeTest(clp,SizeX,SizeY,p,n-1),SizeX,SizeY)
\ : (p==4) ? ResizeUnderTest4(ResizeTest(clp,SizeX,SizeY,p,n-1),SizeX,SizeY)
\ : (p==5) ? ResizeUnderTest5(ResizeTest(clp,SizeX,SizeY,p,n-1),SizeX,SizeY)
\ : (p==6) ? ResizeUnderTest6(ResizeTest(clp,SizeX,SizeY,p,n-1),SizeX,SizeY)
\ : (p==7) ? ResizeUnderTest7(ResizeTest(clp,SizeX,SizeY,p,n-1),SizeX,SizeY) : ResizeUnderTest1(ResizeTest(clp,SizeX,SizeY,p,n-1),SizeX,SizeY)
}

o_Size=152
#r1 = blankclip(width=o_Size,height=o_Size,pixel_type="YV24",color_yuv=$FF0000).ConvertToYV12(ChromaOutPlacement="MPEG2")
#g1 = blankclip(width=o_Size,height=o_Size,pixel_type="YV24",color_yuv=$00FF00).ConvertToYV12(ChromaOutPlacement="MPEG2")

#r = blankclip(width=o_Size,height=o_Size,pixel_type="YV24",color_yuv=$FF0000).ConvertToYV12(ChromaOutPlacement="MPEG1")
#g = blankclip(width=o_Size,height=o_Size,pixel_type="YV24",color_yuv=$00FF00).ConvertToYV12(ChromaOutPlacement="MPEG1")

r = blankclip(width=o_Size,height=o_Size,pixel_type="YV411",color_yuv=$FF0000)
g = blankclip(width=o_Size,height=o_Size,pixel_type="YV411",color_yuv=$00FF00)

#r = blankclip(width=4,height=4,pixel_type="YV411",color_yuv=$FF0000)
#g = blankclip(width=4,height=4,pixel_type="YV411",color_yuv=$00FF00)

#stackhorizontal(r1,g1)
#stackvertical(last,last.fliphorizontal())
#o1=last

stackhorizontal(r,g)
stackvertical(last,last.fliphorizontal())
o=last

#a=ResizeTest(o,512,512,1,8)
#b=ResizeTest(o,512,512,2,8)
ResizeTest(o,500,500,6,8)
#b=ResizeTest(o,128,128,3,8)

#interleave(a,b,c)

#Spline36Resize(512,512)

ConvertToYV24()


I adjusted it/changed it to test several cases :
- Using at start a 4x4, with succesives calls (with rf=2 or 8) without changing the output (=> each call increase size).
- Using a "normal" video size with output resize, testing a lot (32) of succesive calls with rf=2, or less succesive (around 8) calls with rf=16.

But for this, i've used a special build, where i've removed the code of when there is no cshift srting, in YV12 some vertical shift is nevertheless made with Spline36Resize, allowing me to have a "pure" output without cshfit. On the other hand, as it's only vertical, it shouldn't affect the horizontal issues, so you can still made tests on YV12, but in that case remove the [C_vshift = C_vshift-0.5] in the code specific to the YV12 part.

[luquinhas0021]

jpsdr, what about you make a port of eedi3 for Avisynth 2.6? The port that already exist sucks: it doesn' add support to bunch of colorspaces that you've added to nnedi3.

[jpsdr]

I don't use eedi3, so sorry, i will not work on this.

[jpsdr]

After a little thought, my mistake. Implementing the chroma resize shift will not make the test fail, because after the 1rst call of resize, input/ouput will have the same size => no shift add to chroma.
In that case, and only for 4:2:x (and not 4:1:1 according the thread) formula should be :
src_left=0.25*(1.0-Input/Ouput)
but... only if shift was applyed before the rf resize.

X0,Y0 : Input size, X1,Y1 : Output size, c=0.25*(1.0-X0/X1)
Resize(X1,Y1,src_left=c) is the same than :
Resize(X0,Y0,src_left=c)
Resize(X1,Y1)
or :
Resize(X1,Y1)
Resize(X1,Y1,src_left=c*X1/X0)

So 2 solutions :
call a resize without changing the resolution with only a shift before applying the rf change,
or after calcultate all the chroma compensation for the rf, which i think is better and avoid an extra resize call.
First : Compensate for the rf increase not done :
add : 0.25*(1-In/(rf*in))*((rf*In)/In)=0.25*(rf-1).
Now, we have again a good 4:2:0 chroma placement. This one will be resized again, but this time, the shift will be applied properly (before the next resize). We are now changing size from In*rf to out.
We have to add again : 0.25*(1-(rf*In)/out)
So, finaly it will result on : 0.25*rf*(1-In/Out)
Test will not fail, because after the 1rst call, In=Out.
I'll check this tomorrow.

[cretindesalpes]

what about you make a port of eedi3 for Avisynth 2.6?
I started working on an eedi3 (and dfttest as well because they share the same kind of planar input) port for Avs(+) 32/64 bits and Vapoursynth, including all the new colorspaces, but since there is already a decent VS port for these plug-ins I somewhat gave up.

[jpsdr]

Some benchs under avs 2.60

RGB24
Original method : 47.87
New method without FTurn : 43.16
New method with Fturn : 35.88
=> Don't bother anymore, always use new method.

YV12
Without FTurn : 17.44
With Fturn : 14.19

YV24
Without FTurn : 26.95
With Fturn : 19.66

Ok, this time, it's significant.

[jpsdr]

New version. Add chroma resize shif correction and mpeg2 parameter.

[kuchikirukia]

New version. Add chroma resize shif correction and mpeg2 parameter.

No.
No.
Don't tell me it was wrong.
This DVD.
Interlaced.
Telecined.
Progressive.
Film
Video
So much trimming.
So much splicing.
And... I just finished.

Or did this only affect resize and not QTGMC?


How does this interact with Santiag (http://forum.doom9.org/showthread.php?t=153835#post1393006)? Santiag does a Spline36resize with a chroma shift to fix... something, but with the chroma shift bug in Spline36resize and now no bug in nnedi3, I have no idea if it's still fixing what it's supposed to fix or if it's now introducing an error.

But thank you! I use nnedi3 in so much.

[Reel.Deel]

The chroma shift only affects nnedi3_rpow2() since it uses AviSynth's internal resizers. So this does not effect nnedi3() nor QTGMC().

On the bright side the chroma shift is for the most part hardly noticeable, so you should be fine. I guess that's one of the reasons this issue has been around for some time now...

[kuchikirukia]

The chroma shift only affects nnedi3_rpow2() since it uses AviSynth's internal resizers. So this does not effect nnedi3() nor QTGMC().

Phew, good to know.
Though does that mean when they finally get around to fixing the internal resizers that nnedi3 is going to be wrong again?