nnedi3 shift

[bxyhxyh]

I'm upscaling a dvd to 720p, I needed yv24 colorspace for further part of my script.

I'm using this script

#Source
ColorMatrix("Rec.601->Rec.709",clamp=0)
y=ConvertToY8().nnedi3_rpow2(2).Dither_convert_8_to_16().dither_resize16(1280,720,kernel="bilinear",invks=true).Ditherpost()
u=UToY8().nnedi3_rpow2(4).Dither_convert_8_to_16().dither_resize16(1280,720,kernel="bilinear",invks=true).Ditherpost()
v=VToY8().nnedi3_rpow2(4).Dither_convert_8_to_16().dither_resize16(1280,720,kernel="bilinear",invks=true).Ditherpost()
YToUV(u,v,y)


How much my shifts should be on this case?

[raffriff42]

Add the center shift argument to nnedi3_rpow2, and no chroma shift correction will be needed: nnedi3_rpow2(2, cshift="Spline16Resize")

EDIT Note,
- I tested your script sans dither calls, for simplicity
- I tested at 4x enlargement to exaggerate the artifacts. The output looks great; this is a good idea!
- nnedi3_rpow2 does not accept Y8, only YV12, YUY2 and RGB24 - at least for me. ## source = YV12
y=nnedi3_rpow2(4, cshift="Spline16Resize")
u=UToY8.ConvertToYV12
\ .nnedi3_rpow2(8, cshift="Spline16Resize")
v=VToY8.ConvertToYV12
\ .nnedi3_rpow2(8, cshift="Spline16Resize")
YToUV(u, v, y)
## output=YV24 EDIT 2 completely wrong, please ignore

[Reel.Deel]

- nnedi3_rpow2 does not accept Y8, only YV12, YUY2 and
The updated nnedi3 (http://forum.doom9.org/showthread.php?t=170083)accepts the additional planar colorspaces. Also, VToY() is probably better than VToY8.ConvertToYV12.

[bxyhxyh]

Yes I'm using updated nnedi3 and UToY8() and VToY8() for speed and low memory usage.

[colours]

I'm using this script

#Source
ColorMatrix("Rec.601->Rec.709",clamp=0)
y=ConvertToY8().nnedi3_rpow2(2).Dither_convert_8_to_16().dither_resize16(1280,720,kernel="bilinear",invks=true).Ditherpost()
u=UToY8().nnedi3_rpow2(4).Dither_convert_8_to_16().dither_resize16(1280,720,kernel="bilinear",invks=true).Ditherpost()
v=VToY8().nnedi3_rpow2(4).Dither_convert_8_to_16().dither_resize16(1280,720,kernel="bilinear",invks=true).Ditherpost()
YToUV(u,v,y)


Why are you using an invks resize after nnedi3_rpow2? If you want sharpening, you'd be better served with nonlinear sharpening filters instead of resorting to Dither_resize16's invks.

Add the center shift argument to nnedi3_rpow2, and no chroma shift correction will be needed: nnedi3_rpow2(2, cshift="Spline16Resize")

Assuming that the chroma channels are copied to luma before calling nnedi3_rpow2 with UToY/VToY, this is only correct for MPEG-1/JPEG chroma siting, not for MPEG-2 chroma siting.

Also, nnedi3_resize16 is a thing (https://www.nmm-hd.org/newbbs/viewtopic.php?f=7&t=1117) and it probably already handles converting from 4:2:0 to 4:4:4 using nnedi3 for upscaling correctly.

[bxyhxyh]

I've changed my script to this

y=converttoy8().nnedi3_rpow2(2,cshift="spline16resize").dither_convert_8_to_16().dither_resize16(1280,720,kernel="bilinear",invks=true).ditherpost()
u=utoy8().nnedi3_rpow2(4,cshift="spline16resize").dither_convert_8_to_16().dither_resize16(1280,720,0.25,kernel="bilinear",invks=true).ditherpost()
v=vtoy8().nnedi3_rpow2(4,cshift="spline16resize").dither_convert_8_to_16().dither_resize16(1280,720,0.25,kernel="bilinear",invks=true).ditherpost()
YToUV(u,v,y)


Now, Is 0.25 still needed here? Or is nnedi3 shifts as well?

[colours]

The shift of 0.25 px is correct only for the original resolution; in this case, nnedi3_rpow2 results in chroma with four times the width of the original, so the shift should also be scaled to 1 px.

[Reel.Deel]

I forgot that nnedi3_resize16 does colorspace conversion. For learning purposes I compared nnedi3_resize16(1280,720, output="YV24") against a manual conversion. The following seems to produce the closest output between the 2.

Y = ConvertToY8().nnedi3_rpow2(2).Dither_convert_8_to_16().Dither_resize16(1280,720, src_left=-0.5, src_top=-0.5).Ditherpost(mode=6)
U = UToY8().nnedi3_rpow2(4).Dither_convert_8_to_16().Dither_resize16(1280,720, src_left= 0.5, src_top=-0.5).Ditherpost(mode=6)
V = VToY8().nnedi3_rpow2(4).Dither_convert_8_to_16().Dither_resize16(1280,720, src_left= 0.5, src_top=-0.5).Ditherpost(mode=6)
YToUV(U, V, Y)

Any thoughts?

[bxyhxyh]

Any thoughts?

That's what I'm asking for.
For nnedi3_rpow2(2), -0.5 works.

But is it same -0.5 for nnedi3_rpow2(4)?

[DarkSpace]

That's what I'm asking for.
For nnedi3_rpow2(2), -0.5 works.

But is it same -0.5 for nnedi3_rpow2(4)?

It's either 0.5 or -1.5 for rpow(4), depending on whether rpow() switches the field it interpolates from or not.