YV12toYUY2

[Guest]

Where is the documentation for Avisynth 2.5, and specifically for YV12toYUY2()? I need to know if it is properly handling interlaced versus progressive source material.

[trbarry]

Donald -

I think that function is actually shipped with MPEG2DEC3 and described in that readme. But I don't know the answer.

But if you find how how to properly inquire/set whether a frame is framed based or field based I'd like to know.

- Tom

[Guest]

Thanks, Tom. I had a look at the source and it appears to have been coded to properly handle things.

But what about Avisynth 2.0x internally? What does it use to convert YV12 to YUY2?

I'm still working through the issues raised by the interlaced/progressive source dilemma (regarding YV12 chroma interpolation required for deinterlacing) and I don't have an answer for your question at this time. :(

Coding goes fast, when you know what to code. :) Previously, Decomb was isolated from these problems by the "hidden" YV12->YUY2 conversion.

[sh0dan]

@neuron2:
You can use ConvertToYUY2() to convert YV12 to YUY2, but it does NOT handle interlaced material properly - the main problem being that I cannot reliably know if we have a fieldbased or frame-based clip (yet).


But what about Avisynth 2.0x internally? What does it use to convert YV12 to YUY2?

There are no routines for converting YV12 to YUY2 in the 2.0 version.

[bb]

How are you going to determine if the video is frame- or field-based? I would prefer an AviSynth command (like "Interlaced" or "Progressive") over an automatic detection, because the latter will never work 100% reliably.

bb

[hakko504]

@bb

What's wrong with AssumeFieldBased() and AssumeFrameBased() ? (except maybe the names)

[bb]

Umm, yeah, right. I've overlooked these two (in conjunction with ComplementParity). So where's sh0dan's problem:
the main problem being that I cannot reliably know if we have a fieldbased or frame-based clip (yet).
Maybe the above commands should be mandatory for a clip, so that all filters know if they have to deal with interlaced or progressive material.

bb

[sh0dan]

Using an MPEG source, it should be quite reliable, however the internal structure isn't very clear on this.

Each video clip has a property set, if they are fieldbased or not. However most external filter don't care much about it, and most internal filters doesn't process field-based material properly. We really should have this cleaned up.

The problem about parity (tff/bff) is also problematic, since I can't see any logic in how GetParity is used. Therefore, I have added this as properties to the videoinfo now, since it seems a bit more logical there. I'm considering completely removing GetParity, since it just seems strange - if dividee drops by I would very much hear his opinion on this.

[trbarry]

Each video clip has a property set, if they are fieldbased or not. However most external filter don't care much about it, and most internal filters doesn't process field-based material properly. We really should have this cleaned up.

Even in 2.0 and earlier releases with YUY2 there are probably many filters that would not do really well if given interlaced material. They would blend data from two different points in time and maybe cause ghosting.

But, like with YUY2, we can get around this by doing IVTC or deinterlace first, at the top of the script.

For YV12 for instance, TomsMoComp already converts the clip to progressive frame based, and it looks like Decomb also will soon. The remaining problem is how to communicate this fact to Avisynth and other filters. I took a brief look at AssumeFrameBased() and IsFrameBased() code when I did TomsMoComp and I added coded that would change the property of the output clip. But then I commented it out because I didn't have any confidence that it worked right. ;)

So for now it is best to add AssumeFrameBased after TomsMoComp or other "progressive makers".

Be aware an MPEG2 clip may actually have a mixture of field based and frame based frames in it. But luckily it does not seem to be a problem for a deinterlacer or IVTC program to assume it is field based and to operate without access to those MPEG2 flags (they would be nice).

And happily once the IVTC or deinterlace process is done then a YV12->YUY2 function can assume frame based when doing a conversion.

So the only case a user will have to specify field or frame based is probably for passing through interlaced clips that are to remain interlaced. Otherwise you will eventually specify it by putting some sort of IVTC/Deinterlace filter at the top, like we do now. Most filters (like with YUY2) will continue to assume they have frame based progressive video.

But this raises another question. Assume Avisynth did no conversions and passed YV12 all the way through to Xvid or another codec or program. Does Avisynth communicate whether it the output is field or frame based. And does Xvid somehow treat with this information. Or does Xvid also just assume frame based unless you turn on interlace in the Xvid properties? (which would work)

- Tom

[Guest]

> There are no routines for converting YV12 to YUY2 in the 2.0 version.

I was under the impression that the native color space on the DVD was YV12. If that is the case, and given that Decomb receives YUY2 in Avisynth 2.0x, where does the conversion occur, and how is that conversion avoided when using Avisynth 2.5?

Both progressive and field-based (interlaced) frames can be combed. Progressive frames can be combed due to pulldown, phase shift, etc. Field-based frames will be combed inherently. The process for uncombing them while avoiding the chroma bug requires to know whether the source was progressive or field-based.

Decomb must deal with captures as well as rips. Therefore, it has access only to the raw video data. Of course when doing IVTC (with Telecide), Decomb can assume progressive source, and when doing deinterlacing (with FieldDeinterlace) it can assume field-based source. But what do we do with the hybrid content? Which chroma mode should be used by Telecide's postprocessing for deinterlacing frames that come through the field matching still combed? Tentatively, I plan to treat them as field-based to avoid having to add a different mode to FieldDeinterlace.

But even that may not be a good idea, because some people use deinterlacers instead of IVTC on progressive source. Must FieldDeinterlace be able to operate in both chroma modes?

Tom, do you support both modes in TomsMoComp, and if so, how do you select the mode to use?

[scmccarthy]

@neuron2

where does the conversion occur, and how is that conversion avoided when using Avisynth 2.5?

The conversion occurs in mpeg2dec2.dll. Mpeg2dec3.dll is the first to pass the YV12 data unaltered. AviSynth 2.0 cannot read the YV12 format, however. So Mpeg2dec3 and AviSynth 2.5 work together to give us access to YV12.

Stephen

[trbarry]

Let me come at it from a different angle.

Each byte in either YUY2 or YV12 represents a measurement at a point in space and time.

For either a field based or frame based clip we know exactly which points in space are represented, without having to know whether it was frame based or field based. They are the same either way. But there is some abiguity about which points in time are referred to.

YV12 only seems more confusing because the conversion to YUY2 has more complicated color conversion forumulas in the event of field based clips which are presumed to have the time ambiguity. And that is only to avoid blending fields from different times.

But when doing IVTC/Deint we don't want to convert chroma anyway. We just want to make sure we choose the values from the proper points in time for the weave field so they match the sample points in time of the other field.

But for our purposes once we have ensured that we have done our best to make both fields in a frame represent samples from the same point in time then we can call it a progressive frame. From that point on all subsequent filters can also treat it as a progressive frame, including converting the color space using the more precise progressive method if needed.

So yes, I treat everything as field based, but I do not do the sloppy field-based color conversions at all.

But note I consider "field based" just to meant that the time of the sample points is ambiguous for the reasons Donald stated above. But that does NOT mean anybody will have to ever do a field based color conversion on it because we will fix that ambiguity.

That's what deinterlacers do. So do Decomb'ers. ;)

Likewise if I'm given a progressive frame to deinterlace I'll assume that the user wanted me to check that the fields really match. Often they don't so I'll still line up the fields or pixels to try to reduce combing.

So I assume field based in and make frame based out. And the trick for YV12 is I deinterlace each chroma plane separately, without any color conversion. Seems to work, and probably would for Decomb also.

- Tom

[scmccarthy]

@tom

Maybe the easiest way to describe the difference between converting field ro frame based frames from YV12 to YUY2 is to explain a workaround to fix the problem. I'm not saying we don't need a better way than this workaround, only that it explains what we are dealing with and why it is different.

YUY2 has chroma samples cosited with every other luma sample on each line, starting with the first one. We all know that YV12 has half the chroma samples that YUY2 has, but instead of cositing the chroma samples with the luma samples every fourth sample or with every other luma sample on every other line, YV12 has chroma samples sited between every two lines in every other column. In order to convert from YV12 to YUY2, any interpolation filter must shift each chroma sample up a down a half lime. If you do not interplate, you just copy the chroma sample to the line above and below itself. This only works for frame based, however.

The workaround for field based is to separate and stack the fields first. Then convert as frame based. unstack and reweave. The way to do it internally (ie, the proper way to do the field based conversion) is to treat all the even lines and all the odd lines on the frame as if they were consecutive lines, thus processing each field separetely without actually separating them.

@shodan

Let me see if I understand you correctly.

Lets way I write a version of ConvertYV12ToYUY2 that works correctly for field based frames.

Now, when I put the IsFieldBased() condition in the code, I can't rely on a truthful answer;AviSynth might not know whether the clip is fieldbased or not. Right?

Why don't I go ahead and warn people that you will have to insert an AssumeFieldBased() or AssumeFrameBased() line before calling a version of ConvertYV12ToYUY2() that handles both cases?

I realize you cannot inplement that solution internally, but will it be alright to just warn people an external filter like this will not work reliably yet unless they are careful to follow directions?

[Guest]

Thank you, Gentlemen, for your assistance.

There *is* a real problem, I believe. Deinterlacing all planes of a combed progressive source frame (two different pictures weaved) as if the frame was interlaced will end up imputing wrong colors to every other line. This can be easily shown. Doing it to a field-based frame will be fine.

Nevertheless, I now have decided on a strategy for dealing with it and coding begins tonight. :sly:

[scmccarthy]

O.K. neuron2 effectively rounded out this thread, so this will be my last post to this thread.

To convert YV12 to YUY2 for field based frames :

Pair the first line of luma with the first line of chroma.
Pair the second line to luma with the second line of chroma.
Pair the third line of luma with the first line of chroma.
Pair the fourth line of luma with the second line of choma.

Use the filter that shifts the chroma up a half line for the first two and the filter that shifts the chroma down a half line for the last two.

That's how I do it. Does anybody have an objection to that?

[trbarry]

based, however.

The workaround for field based is to separate and stack the fields first. Then convert as frame based. unstack and reweave. The way to do it internally (ie, the proper way to do the field based conversion) is to treat all the even lines and all the odd lines on the frame as if they were consecutive lines, thus processing each field separetely without actually separating them.

Sam -

No, I don't think I agree. It is too easy to look at the recommendations in the books for color conversion code and figure that is the best you can do. But in Avisynth we have the luxory of deinterlace/decomb first and I believe that is better.

Basically, if you have to interpolate a value between 2 points it is almost always better to use the closest 2 points for a more accurate answer.

- Tom

[scmccarthy]

Tom -

You have given me something to think about. Which is better? To go by the book and reverse the process of subsampling or to look at the closest lines to each chroma as a better approximation to the original picture?

At high resolutions it will be difficult to tell the difference.

The idea of reversing subsampling reminds me of your description of the unfilter. To reverse the softening that interlaced material is often subjected to or soften to reverse the effect of edge enhancement. Going by the book has to work at least reasonably well.

Stephen