interlaced vs progressive YV12

[plugh]

Been reading, and I've got myself confused / am missing something...

mpeg2 has progressive flag

flag says whether chroma is associated with luma lines
progressive, luma lines 0/1, 2/3, 4/5, 6/7...
or
interlaced, luma lines 0/2, 1/3, 4/6, 5/7, ...


What does mpeg2source deliver to avisynth?

At what stage (if any) does "interlaced YV12" get converted to "progressive YV12"?

Does Telecide do this?

Do seperatefields / weave recompute chroma?

How do functions like resizers, which compute luma/chroma values, know which chroma are associated with which luma?

:confused:

[Guest]

What does mpeg2source deliver to avisynth? If the encoding is 4:2:0, it delivers YV12.

At what stage (if any) does "interlaced YV12" get converted to "progressive YV12"? It happens when you apply a chroma upsampling to YUY2 or RGB. You can ask mpeg2source() to upsample via a parameter. Or you can add a ConvertToYUY2() call in your script. Or you can let your editing/encoding application do it.

Does Telecide do this? No, it operates directly in the color space of the source stream.

Do seperatefields / weave recompute chroma? No.

How do functions like resizers, which compute luma/chroma values, know which chroma are associated with which luma? You're going to resize interlaced video directly? :confused:

[Wilbert]

To clear up your remaining confusion:

At what stage (if any) does "interlaced YV12" get converted to "progressive YV12"?
None. mpeg2source delivers the material as it is (unless you upsample it like neuron2 says; the upsampling is done respecting the interlaced flag).

Of course, if you deinterlace "interlaced YV12" you will get progressive YV12.

Does Telecide do this?
from the manual: Telecide() recovers progressive frames (by finding and aligning matching fields). Do not use Telecide() on streams that do not contain telecined progressive frames, such as pure interlaced video.

So Telecide() should only be used for phase shifted material.

How do functions like resizers, which compute luma/chroma values, know which chroma are associated with which luma?
If you have to feed them progressive or bobbed material (unless the resizer has an interlaced parameter).

[plugh]

All the answers supplied so far are as I expected.
Which still leaves me confused. Please bear with me...

(Setting aside telecide/decimate for the moment)
I can have an encode of progressive material with the progressive flag set to "interlaced", encoded in an interlaced fashion. That is perfectly legal, correct?

ie, they started with an array of YUV 4:4:4 data for one progressive picture, down sampled it to 4:2:2 (YUY2) then create

luma line 1, luma line 3, and
chroma line A (.75 chroma line 1, .25 chroma line 3)

luma line 2, luma line 4, and
chroma line B (.25 chroma line 2, .75 chroma line 4)

Now eventually mpeg2source decodes this and delivers to avisynth

luma line 1
luma line 2
luma line 3
luma line 4
...
and
chroma A
chroma B
...

--------------as opposed to--------------

they started with an array of YUV 4:4:4 data for one picture, down sampled it to 4:2:2 then create

luma line 1, luma line 2, and
chroma line A (.5 chroma line 1, .5 chroma line 2)

luma line 3, luma line 4, and
chroma line B (.5 chroma line 3, .5 chroma line 4)

Now mpeg2source decodes this and delivers to avisynth

luma line 1
luma line 2
luma line 3
luma line 4
...
and
chroma A
chroma B
...

===========================

Now I'm aware that mpeg2source has an optional "hint" that will indicate whether this was progressive or interlaced. (For example, colormatrix plugin will read this hint so that *IT* knows whether chroma A is associated with luma 1&2 or luma 1&3, as well as the proportions.)

I also note that the avisynth docs say
http://avisynth.org/mediawiki/Advanced_topics
"AviSynth has no interlaced flag which can be used for interlaced video. There is a field-based flag, but contrary to what you might expect, this flag is not related to interlaced video. In fact, all video (progressive or interlaced) is frame-based, ..."

So how do other filters, say for example BilinearResize (which supports "YV12"), know *which* YV12 they are dealing with?
How do they know whether Chroma A is associated with luma 1&2 or luma 1&3?

See also http://avisynth.org/mediawiki/Sampling#YV12_color_format

[plugh]

Continuing, and reintroducing telecide

So assuming I have progressive material encoded as interlaced, with "YV12 interlaced" data being handed to telecide

It goes off and does field matching - compares luma 1 to luma 2, luma 3 to luma 4, ... and (rightly) decides they constitute a single progressive picture.

Does telecide then recompute the chroma lines, so that chroma A is now associated with luma 1&2, chroma B with luma 3&4 ... instead of 1&3, 2&4, ... , so that I now have "YV12 progressive" data (that can be resized etc)?

As I said at the beginning, I'm missing something, I just don't know what... :confused:

[Wilbert]

I can have an encode of progressive material with the progressive flag set to "interlaced", encoded in an interlaced fashion. That is perfectly legal, correct?
Yes, that's correct.

Now I'm aware that mpeg2source has an optional "hint" that will indicate whether this was progressive or interlaced. (For example, colormatrix plugin will read this hint so that *IT* knows whether chroma A is associated with luma 1&2 or luma 1&3, as well as the proportions.)
If you mean the interlaced=true/false option, then yes. The hint option in colormatrix is something else.

So how do other filters, say for example BilinearResize (which supports "YV12"), know *which* YV12 they are dealing with?
They always assume it's progressive by default.

Continuing, and reintroducing telecide

So assuming I have progressive material encoded as interlaced, with "YV12 interlaced" data being handed to telecide
In that case, Telecide doesn't do anything to the stream, because the field matching is done by looking at the luma.

Btw, note that the chroma is not phase shifted here, it's just smoothed out a bit too much.

[plugh]

Hmmm. perhaps I'm NOT missing something
They always assume it's progressive by default.

Btw, note that the chroma is not phase shifted here, it's just smoothed out a bit too much.

So the frames still have "YV12 interlaced" format data after the IVTC. hmmm.

(I'm writing as I think...)

Resizers assume an evenly spaced grid for the input pixels.
If they are assuming "YV12 progressive" data, they assume the chroma is located equidistant from the four associated lumas. i.e.(upper/lower case shows association)
L L L L
C C
L L L L

l l l l
c c
l l l lbut in reality we have
L L L L
C C
l l l l

L L L L
c c
l l l l
Seems like that would have to throw the calculations off.
I need to think about that some more...

Regarding the mpeg2source hints (info=3)
Changes.txt says
DGMPGDecVersion 1.2.0 Functional Changes
[...]
8. Revised "info" parameter as follows:

0: Do not generate debug information
1: Overlay debug information on the video
2: Output debug information via OutputDebugString() (Use DebugView utility)
3: Output hints in the video (as defined in utilities.cpp/utilities.h)
Current hints are progressive/interlaced and colorimetry.

Is there any way to latch onto this flag in a script and use it in a conditionalfilter / scriptclip / ... ?

Thanks!

[plugh]

Seems like that would have to throw the calculations off.
I need to think about that some more...
More 'hmm'ing

Even if you don't resize - consider this scenario

You telecide/decimate, yeilding progressive frames but "YV12 interlaced" data.

Feed that YV12 data to XVID as progressive frames.

Eventually gets decoded by a player that reasonably assumes it is getting "YV12 progressive" data. The player takes the erroneously associated luma/chroma and (using inverse BT.601 eqns) converts to RGB for display.

Hmmm... That's the "Chroma Upsampling Error"!
http://avisynth.org/mediawiki/Advanced_topics#The_Chroma_Upsampling_Error

[Guest]

That's right. And that's why it's better to upsample in a domain where you have control of which method is used.

[scharfis_brain]

if you are afraid, which kind of sampling to use, you may want oto go the automated approach:

http://forum.doom9.org/showthread.php?p=618364#post618364

[plugh]

So, if the Progressive/Interlaced flag is "interlaced" for anything in the source (e.g., if, while playing the source in dgindex the display shows "Frame Type" = Interlaced for ANY frames) then sticking with YV12 and deinterlacing and/or IVTCing the data is insufficient. (note: do not confuse Frame Type with Frame Struct)

You will have progressive frames, but the actual YV12 data won't be in "YV12 progressive" format for any of the Frame_Type = Interlaced frames. This will lead to Chroma Upsampling Error for those frames. That is, anything that relies on 'knowing' which luma and chroma values are associated will likely be wrong on those frames.

As things stand, if the above situation exists with the source material, it appears the only option available to deal with it is to set mpeg2source option upconv=1 and icc (auto). This will cause mpeg2source to upsample & output YUY2, reacting to the Frame Type flag as it goes along so that it outputs correctly (?) associated luma / chroma values for each frame (no matter it's Frame Type).

Correct?

[plugh]

if you are afraid, which kind of sampling to use, you may want oto go the automated approach:

http://forum.doom9.org/showthread.php?p=618364#post618364

Looks interesting. I'll have to read up on the functions used to determine what it is doing, but I'm not sure it addresses what I'm talking about above. Now if it somehow sensed the progressive/interlaced hint coming out of mpeg2source...

[scharfis_brain]

my functions do this:

- bob-deinterlace the chroma to 50fps / 59.94fps
- convert it to the wished colorspace
- reinterlace it
- merge the upsampled chroma back to the luma

This means:
For static areas you'll get progressive sampling.
For moving areas you'll get interlaced sampling with ELA-interpolation (lesser chroma jaggies).
For full progressive frames you'll get full progressive sampling if matchfields is enabled.

[Guest]

You might also look at my native Avisynth filter, which was inspired by scharfis_brain's work.

http://neuron2.net/autoyuy2/autoyuy2.html

Scharfi discovered that you can get by without the flag by using other image heuristics to determine which upsampling to use.

[plugh]

Wow, that's impressive. Cudo's to both of you.

Any chance there is something around that (adaptively) resamples "YV12 interlaced" to "YV12 progressive"? Converting to YUY2 isn't the goal, it just seems to be the only way available to 'undo' "YV12 interlaced" and thus avoid CUE...

[scharfis_brain]

you cannot resample YVinterlaced to YV12progressive.

both are the same, in fact.

the artifacts you see aren't caused by YV12 itself. Nor they are due to interlaced or progressive downsampling.
The artifacts you see are caused by the routine that finally displays the image on the screen.
A screen NEVER can display YV12 natively. You always need to upsample to YUV 4:4:4 / RGB.
If this upsampling is done correctly (ie. adaptive) using my proposed method, you won't notice these artifacts.

[plugh]

Unfortunately I don't have control over the output device.

But perhaps I'm asking the wrong question. Bear with me and please excuse if I don't use exactly the correct terminology.

What started me on this path was thinking about resizers, and the calculation of luma and associated chroma values. (Well actually, it started a bit earlier with some observations of banding and what I took to be block noise wrt colormatrix, but I'll defer that one for now.)

So the question: In the specific case of
-- progressive frames (ie source was FILM - no motion between fields)
-- "YV12 interlaced" chroma (ie source was encoded with Frame Type Interlaced)
-- I want to apply a 3:2 vertical resolution reduction (ie 1080 to 720)
-- and noting that the playback device assumes that progressive frames are "YV12 progressive" sampled

is there an optimal way for the resizer to calculate the new luma / chroma values to minimize artifacts like CUE?

ie
mpeg2source("1080i.d2v")
telecide(guide=1,post=0)
decimate(cycle=5)
magicresizer(1280,720)

ie
source is encoded Frame Type Interlaced
it is IVTC'ed to recover the progressive film frames
the resulting YV12 data is then passed to the resizer


The resizer has to recalculate all those pixel values. I see here (http://avisynth.org/mediawiki/Sampling#Upsampling) that differant equations are used to interpolate the missing chroma values for YUY2 upsampling depending upon whether the data is "YV12 interlaced" or "YV12 progressive".

Therefore I ponder whether differant equations could / should be used by the _resizer_ to interpolate the chroma values at the new loci, as a function of the input 'YV12 type'.

In particular, I wonder whether this could be done in such a fashion as to make the resulting data (when upsampled for display by a device that assumes it was given "YV12 progressive" data) yeild less pronounced Chroma Upsampling Errors.

And I also ask myself whether any advantage might be taken of a specific 3:2 vertical downsize ratio in this process.

Comments? Am I completely off-base?

[scharfis_brain]

for this case, you can completely ignore that interlaced/progressive YV12 stuff.

just do your IVTC and scale down the video.

In other words: there is no thing like interlaced or progressive YV12. Both are the same!


If you got interlacing in the image before resizing, you'll need interlaced treatment for luma as well as for chroma.
If you don't see any interlacing in the image before resizing, just do your resize and treat it progressively.

YV12 is just a lower resoluted chroma plane. In general it needs exactly the same treatment like the fully resoluted luma.

You are thinking too much.

[plugh]

for this case, you can completely ignore that interlaced/progressive YV12 stuff.

just do your IVTC and scale down the video.
That's what I have been doing, but
In other words: there is no thing like interlaced or progressive YV12. Both are the same!
Uh, so the avisynth docs are incorrect about the difference in luma / chroma associations? :eek:

You are thinking too much.
I have been told that before ;) but that's how one learns (and I greatly appreciate the 'experts' taking the time to explain things).

If nothing else, this thread has been useful in helping me to refine my understanding and clarify my problem statement. I need to (gasp) think about all this and determine where the error in my thinking is, or develop a proof of concept.

Again, thanks to all...

[scharfis_brain]

if you are looking at the YV12 stored image itself there is no difference between laced and progressive.

It is the way from and towards YV12 that has to be treated differently when converting to and from YV12.

But since you do NOT intend to convert from or to YV12, there is no need to break up your mind.

[Wilbert]

You telecide/decimate, yeilding progressive frames but "YV12 interlaced" data.

Feed that YV12 data to XVID as progressive frames.

Eventually gets decoded by a player that reasonably assumes it is getting "YV12 progressive" data. The player takes the erroneously associated luma/chroma and (using inverse BT.601 eqns) converts to RGB for display.

Hmmm... That's the "Chroma Upsampling Error"!
http://avisynth.org/mediawiki/Advanc...sampling_Error
At the risk of confusing you more :)

The main point is that when converting progressive YUY2 to YV12 with the interlaced flag on, you won't get interlaced content. Sure you get interlaced YV12, but it's not the case that the chroma fields originate from different time instants (which is what interlaced content is).

The CUE arises (becomes very visible) when you upsample interlaced content (YUY2) to YV12 with the progressive flag on.

Regarding the mpeg2source hints (info=3)
Changes.txt says
You are correct, i forgot about that.

And finally, it's critical to understand what scharfis_brain is saying in his post above mine. If you don't understand that, you should ask questions about it first.

[plugh]

FWIW...

Made some inquiries, and am told my output device is doing this in the progressive case
YUY2 line 1 chroma of YV12_lines 1
YUY2 line 2 chroma of YV12_lines (0.5)*1 + (0.5)*3
YUY2 line 3 chroma of YV12_lines 3
YUY2 line 4 chroma of YV12_lines (0.5)*3 + (0.5)*5

[scharfis_brain]

What kind of output device are you referring to?

[plugh]

xbox running xbmc - relevant topic:

http://www.xboxmediacenter.com/forum/showthread.php?t=27499
http://xboxmediacenter.com/wiki/index.php?title=Rendering

They use GPU code to do some of these manipulations.