Edge-directed interpolation, green stripes, and vertical resolutions

[Katie Boundary]

mpeg2source("blahblah.d2v").crop(8,0,-8,-2)

...looks fine.

mpeg2source("blahblah.d2v").crop(8,0,-8,-0).yadifmod2()

or

mpeg2source("blahblah.d2v").crop(8,0,-8,-0).nnedi3()

...also looks fine.

BUT...

mpeg2source("blahblah.d2v").crop(8,0,-8,-2).yadifmod2()

or

mpeg2source("blahblah.d2v").crop(8,0,-8,-2).nnedi3()

...produces a thick green line at the bottom of the screen!

Crop(8,0,-8,-4) gets rid of the green line
Crop(8,0,-8,-6) makes it appear again.

Yadif uses edge-directed interpolation, just like NNEDI, so... does EDI require the vertical resolution to be evenly divisible by 4?

[poisondeathray]

I don't get the green line for nnedi3, but yadifmod2 was affected using colorbars

This is ok for me , avs+ x64 r3682, NNEDI3_v0_9_4_60

colorbars(720,480,"YV12").crop(8,0,-8,-2).nnedi3()


Setting yadifmod2 opt option, or SetMaxCPU had no effect (green line still there on some frames, so probably not related to CPU optimizations)

What version of avisynth and nnedi3 are you using ?

It might have something to do with mod-4 vertical crop restrictions on interlaced video in the case of mpeg2source and DVD
http://avisynth.nl/index.php/Crop#Crop_restrictions

[StainlessS]

It might have something to do with mod-4 vertical crop restrictions on interlaced video in the case of mpeg2source and DVD
http://avisynth.nl/index.php/Crop#Crop_restrictions
+1 [probably part of the reason (deinterlacer really should throw error if non vertical mod 4 YV12)]
vertical crop after deinterlace, or maybe try with convert to YUY2, Mpeg2source(UpConv=1) [back to YV12 after deint].

int upConv = 0

Upsample from 4:2:0 to YUY2 (4:2:2) or RGB24.

0 : Do not upsample
1 : Upsample to YUY2 (ignored if input is already 4:2:2)
2 : Upsample to RGB24


NOTE, UpConv not implemented in current official Mpeg2Source 64 bit. [EDIT: think several additional options not implemented in current x64]

NOT sure if both YDifMod2 and nnedi3 support YUY2. [both old-ish plugs, so probably do]

[Katie Boundary]

So I should deinterlace first, then convert to RGB, then crop the bottom 2 lines?

deinterlacer really should throw error if non vertical mod 4 YV12

From the sounds of it, AVIsynth itself should throw such an error.

I don't get the green line for nnedi3

Even on my end, nnedi3 produced a weaker/subtler green bar than Yadif, so I think you're on to something.

[Reel.Deel]

You can also use padding (https://forum.doom9.org/showpost.php?p=1596804&postcount=5)before nnedi3:

colorbars(720,480,"YV12").crop(8,0,-8,-2).padding(0,0,0,6).nnedi3().crop(0,0,0,-6)


NOTE, UpConv not implemented in current official Mpeg2Source 64 bit. [EDIT: think several additional options not implemented in current x64]


The latest Mpeg2Source (x64 only) silently got rid of a lot of parameters after switching to the MPEG2DecPlus codebase (see here (https://github.com/Asd-g/MPEG2DecPlus/issues/9#issuecomment-1120229431)). Asd's D2VSource retained some of those options, see wiki page (http://avisynth.nl/index.php/MPEG2DecPlus) for details.

[StainlessS]

Thank RD.

EDIT:
+1 [probably part of the reason (deinterlacer really should throw error if non vertical mod 4 YV12)]
When non vert mod 4, probably accesses out of bitmap for U & V on bottom + 1 choma row, I think U=0 and V=0 would produce Green.
(may not be green if memory content after valid bottom chroma row, something other than zero's).


From the sounds of it, AVIsynth itself should throw such an error.
Avisynth has no idea that you are gonna try deinterlace it, nor any idea what a plugin is intended to do.
All Avisynth builtin filters that operate on Interlaced material will/should throw an errror if vertical non mod 4.
[eg, BlankClip(Width=320,height=240-2,pixel_Type="YV12").ConvertToYV24(Interlaced=True) # Should produce error, untested.]

[Reel.Deel]

EDIT:

When non vert mod 4, probably accesses out of bitmap for U & V on bottom + 1 choma row, I think U=0 and V=0 would produce Green.
(may not be green if memory content after valid bottom chroma row, something other than zero's).


Exactly what was described in the "Green line at the bottom" issue on GitHub: (https://github.com/sekrit-twc/znedi3/issues/11)
If the chroma res is odd, you are effectively trying to read or write outside of the frame when you interpolate the "other" line.

I thought it was widely known not to crop interlaced footage to less than mod4. I understand cropping out large black bars as it may provide some speedup but in this case cropping just a few pixels and making the source non mod4 is not needed. Either deinterlace then crop or crop, pad, deinterlace, crop afterwards, I prefer the first option most of the time.

[StainlessS]

Thanks again RD, I had not seen that issue on Github, should throw an error to help user not make a boo-boo.
I thought it was widely known not to crop interlaced footage to less than mod4.
Yes, you CAN crop at top mod 4, and crop to result height mod 4, OK. eg BlankClip(Width=320,height=240,pixel_Type="YV12").Crop(0,4,-0,-4)
But of course source height before cropping should be mod 4 else there was already an existing error in processing the interlaced source.
Cropping eg 2 off bottom and trying to deinterlace can/will produce rubbish on bottom chroma row {assuming mod 4 error not thrown by deinterlace plugin},
cropping eg 2 off top prior to deinterlace will produce rubbish/BadResult for full height of clip when deinterlacing {assuming mod 4 error not thrown by deinterlace plugin}.
Above mod 4 applies only for YV12/YUV420 interlaced, other colorspace must be mod 2.

[Reel.Deel]

Thanks again RD, I had not seen that issue on Github, should throw an error to help user not make a boo-boo.

Probably, idk about nnedi3 though since it can also be used to upscale and progressive YV12 mod2 is perfectly valid in that case.

I think jpsdr was able to do something about it: https://github.com/jpsdr/NNEDI3/issues/9

Don't know what was done...

Changes: Duplicate destination result line is better (https://github.com/jpsdr/NNEDI3/commit/fc18e768427d9f946e97967b7c88d9a7452c8844) and Fix issue in 4:2:0 when dh=false and height not mod 4 (https://github.com/jpsdr/NNEDI3/commit/4771fbcdb5a182c9bc8d4c1beb8055fca376f4db).

[StainlessS]

Changes: Duplicate destination result line is better and Fix issue in 4:2:0 when dh=false and height not mod 4.
Yep good fix.

Yes, you CAN crop at top mod 4, and crop to result height mod 4, OK. eg BlankClip(Width=320,height=240,pixel_Type="YV12").Crop(0,4,-0,-4)
But of course source height before cropping should be mod 4 else there was already an existing error in processing the interlaced source.
Cropping eg 2 off bottom and trying to deinterlace can/will produce rubbish on bottom chroma row {assuming mod 4 error not thrown by deinterlace plugin},
cropping eg 2 off top prior to deinterlace will produce rubbish/BadResult for full height of clip when deinterlacing {assuming mod 4 error not thrown by deinterlace plugin}.
Above mod 4 applies only for YV12/YUV420 interlaced, other colorspace must be mod 2.
But would not help in above bolded case. [Mangled chroma result, full height of clip]

EDIT: I think Avisynth Standard v2.60, would be less likely to produce rubbish/green bottom chroma row, as cropping may be a matter of adjusting pointers and/or internal height variable
of clip [where the clip is not actually cropped internally, only makes it seem so, ie bottom chroma row + 1 is still in original pre-cropped area of bitmap],
whereas AVS+, now forces physical cropping with result copied into new clip
[Top LHS of bitmap channels must start on an exact multiple of maybe 32 bytes, for fast SIMD assembler access, or something like that].

[Katie Boundary]

Avisynth has no idea that you are gonna try deinterlace it, nor any idea what a plugin is intended to do.

Doesn't matter. It should just treat YV12 content with a non-mod4 height as illegal. Is there any particular edge case where operating in YV12 with a mod2 height would be necessary or ideal just to avoid a colorspace conversion that's probably going to happen sooner or later anyway?

Exactly what was described in the "Green line at the bottom" issue on GitHub: (https://github.com/sekrit-twc/znedi3/issues/11)

Not everybody cares about Github.

I thought it was widely known not to crop interlaced footage to less than mod4.

It seems to depend on the colorspace. At a minimum, I don't think it was mentioned in READFAG. (shrug)


EDIT: by the way, there's a little trick to remember if you want to use the resizers to crop!

converttorgb().resize(src_height=-2) works fine for garbage at the bottom of the screen, but
converttorgb().resize(src_top=2) will perform a subpixel shift!

converttorgb().resize(src_top=2,src_height=-0) is the correct way to crop stuff from the top of the screen

I almost screwed up 70 episodes of Ducktales this way.

[Reel.Deel]

Not everybody cares about Github.


Googling 'deinterlace "mod4" avisynth site:forum.doom9.org' brings up a handful of results:

When dealing with interlaced YV12 video, the bare minimum is to crop in mod4 steps from the top border. (Here: either 100 pixels, or 104 pixels. 102 is not divisible by 4.) When cropping is not mod4, you effectively get a phase-swap of chroma channels in interlaced sections.

BTW, your crop is "illegal" , vertical interlaced YV12 crop must be mod4
http://avisynth.nl/index.php/Crop

Also, if you crop before deinterlacing, vertical cropping must be on mod4 boundaries.

You can crop horizontally and vertically before QTGMC with these restrictions (same for interlaced or progressive)
YV12: horizontal & vertical mod4. YUY2: horizontal mod4, vertical mod2
Cropping will speed things up of course, but if you crop a significant amount of real image for some reason then you lose some motion analysis possibilities and reduce quality at edges.

[StainlessS]

The -Vit- thing above, "YV12: horizontal & vertical mod4. YUY2: horizontal mod4, vertical mod2"
YV12/YUY2 horizontal only needs mod 2 whether interlaced or not,
in the past, some encoders may have required multiples of 4 (or maybe even 8 or 16) both H and V for YUV, not so now for eg x264,
also, methinks old VirtualDubMod required mod 4 for H and/or V YUV,
but those are restrictions to software outside of Avisynth.

EDIT: x264 might internally encode eg 1920x1080 as 1920x1088, with 6 [EDIT: 8] dummy vertical pixels encoded, but ignored on decode.
Not sure if it vertical pads to multiples of 8 or 16, [EDIT: 16] horizontal might similarly be padded, dont know. (DGIndexNV shows internal and external coded size in the log window)

Doesn't matter. It should just treat YV12 content with a non-mod4 height as illegal.
But it's not illegal, some input src might be progressive mod 2 only.

Is there any particular edge case where operating in YV12 with a mod2 height would be necessary or ideal just to avoid a colorspace conversion that's probably going to happen sooner or later anyway?
YV12 must be mod4 height if interlaced and mod2 if not, non YV12 interlaced must be mod2 in all other colorspaces, YUV or RGB.

just to avoid a colorspace conversion that's probably going to happen sooner or later anyway?
what, you wanna encode in RGB or something ??? [most of us try to avoid RGB where possible]

EDIT:
YV12 interlaced, vertically, the even numbered chroma samples are for even field chroma, odd chroma samples for odd fields, so mod 4 is always necessary, ie 2 vertical chroma samples and 4 vertical luma samples, always sampled together.
Y0 ----C0
Y1 ----C1
Y2 ----C0
Y3 ----C1
Above, both C0's are the same chroma sample, same for both C1's. They cannot be split apart, always multiple of 4 when interlaced.

[Katie Boundary]

YV12 must be mod4 height if interlaced and mod2 if not

Progressive YV12 is a thing?

I don't just mean progressive-looking, I mean YV12 can "know" that the images are progressive and store the information appropriately?

If so, can AVIsynth discriminate between progressive YV12 and interlaced YV12, and block all attempts to convert video to interlaced non-mod4-height YV12?

[StainlessS]

Progressive YV12 is a thing?
Of course it is, most BluRay will be progressive YV12/YUV420 [dont think I've got any BD thats not progressive [but I assume Interlaced BD is possible, somebody will likely say if so].

I don't just mean progressive-looking ...
Even if some flag denoting interlaced/progressive, you can probably take it with a pinch of salt.
DVD source quite often flags progressive as Interlaced, gods might know why, but I dont.
Usually recommended to 'use your eyes' and ignore any interlaced flag.

If so, can AVIsynth discriminate between progressive YV12 and interlaced YV12
Plenty of Comb detectors attempt that.

block all attempts to ...
Nope, you gotta be able to fix clips where flags tell lies.

Life's a bitch and then you die. :(

[Katie Boundary]

Okay, no, you clearly didn't get what I was saying. I'm talking about a form of YV12 that stores chroma in true 2x2-pixel blocks, not interlaced pairs of 2x1-pixel blocks. If that doesn't exist, then there's no reason for heightmod2 YV12 to be legal.

[StainlessS]

I only showed vertical representation of YV12 interlaced, for each chroma sample there are always two horizontal luma samples for each chroma [did not think that needed saying].
There is only one form of YV12, the only difference is how it is interpreted and processed by software, so crop requirement for YV12 is
mod2 both horizontally and vertically, but if cropping vertically top mod2, or cropping height mod2, then interlaced content would later be incorrectly interpreted.
[EDIT: Above, where I say mod2, I mean mod 2 only, but not mod 4]

Also for MPeg2 chroma positioning, there is some kind of averaging between chroma samples when resizing (ie Progressive) [which would make more sense if 2x2 chroma only, ie 4x4 luma samples],
but apart from resizers, think most software totally ignores that (interpreting as MPEG1 positioning). [EDIT: resizers may have additional modulo requirements, over and above mod2, W and H]

If you want to enforce w/h both mod 4, then nothing stopping you from doing that, but physical crop limit on YV12 is 2x2 luma samples, the 2x4 limit is only where software
it interpreting as Interlaced (but any software implementation may have its own additional w/h modulo requirements, but that has little to do with YV12 format requirements).

It is an imperfect world, but on the bright side, it'll soon be over. :)

[Katie Boundary]

So, in other words, there's no reason for heightmod2 YV12 to be legal.

[StainlessS]

You can treat it just exactly as you like, unlikely to bother anybody else.

[Katie Boundary]

It'll bother anyone who comes across the green stripe bug and doesn't think to ask about it here.

[Katie Boundary]

The AVIsynth wiki says YV12 can be interlaced or progressive: http://avisynth.nl/index.php/FAQ_YV12#Are_there_any_disadvantages_of_processing_in_YV12.3F

[Reel.Deel]

The AVIsynth wiki says YV12 can be interlaced or progressive: http://avisynth.nl/index.php/FAQ_YV12#Are_there_any_disadvantages_of_processing_in_YV12.3F

Because it can be, just like any other colorspace.