Avisynth video properties (was "Help me solve this YV12 mystery")

[neuron2]

Either I'm doing something real stupid or it appears that we still do not have a reliable YV12 decoder for Avisynth 2.5. First let me show you the problem I am experiencing. The attached JPEG shows on the bottom correct decoding, and on the top messed up decoding. Zoom in to see correct interlacing on the bottom and messed up interlacing on the top.

My tools are DivX5, VirtualDubMod and the filters given in the script below. If you want to experiment, get the small VOB from http://neuron2.net/dgraft/misc/sample.vob

Here is the test script I have used. Any explanation or fix would be greatly appreciated, otherwise I'll have to resume my modification of dividee's mpeg2dec to output YV12.

-------
### Dividee's ported to 2.5 YUY2 WORKS!
#loadplugin("d:\avisynthplugins\mpeg2dec.dll")
#mpeg2source("sample.d2v")

### MarcFD's decoders YV12 BOTH FAIL!
#loadplugin("d:\avisynthplugins\mpeg2dec3_094.dll") # ported to 2.5
loadplugin("d:\avisynthplugins\mpeg2dec3.dll")
mpeg2source(d2v="sample.d2v")
#YV12ToYUY2() # BUT WORKS WITH THIS

### Nic's MPEGDecoder YV12 FAIL!
#loadplugin("d:\avisynthplugins\mpegdecoder.dll")
#mpegsource("sample.vob")

### It works when displaying the VOB in DVD2AVI
### and when opening the VOB directly in VirtualDubMod

## It seems we do not have a reliable YV12 decoder for Avisynth 2.5!
-------

[neuron2]

Oh, both DivX and XviD do this.

Since it works correctly when using YV12ToYUY2(), I am wondering if the codecs suffer from the chroma upsampling bug.

One candy cigar to whoever figures it out. :-)

[onesoul]

Have you tried the FixBrokenChromaUpsampling() command? Dunno if this will help.

[angelyote]

FixBrokenChromaUpsampling() requires YUY2 input. Since the problem doesn't appear after converting to YUY2 it won't work.

In case anyone's interested converting back to YV12 after going to YUY2 gives the same bad output.

Dave

[Kurosu]

Just a guess (haven't tested yet)...

Maybe the UV planes are stored as interlaced (see avisynth.org about YV12 interlacing or what was discussed in Turn thread to see what I mean) and both YV12 decoders assume to always have data 'progressively' stored. Or the data is badly reported or...

I'm far from understanding the field manipulation filters in avisynth but that may give an evidence to what is really happening...

[EDIT]
I think it's almost a problem of how the chroma interlacing is reported:
Please try those both scripts using YV12toYUY2() from MarFD's mpeg2dec3.dll

mpeg2source("sample.d2v")
YV12toYUY2(interlaced=false)

and

mpeg2source("sample.d2v")
YV12toYUY2(interlaced=true)


Replacing mpeg2source("sample.d2v") by mpegsource("sample.vob",-2) of course still works

[EDIT2]
Therefore, it's either the stream's chroma storage that is considered and reported to XviD/DivX5 filter as not interlaced, or their colorspace converting functions that always assume it's progressive.

[EDIT3]
Using ATI's YV12 codec, the result are exactly the same, so such a general error lies at the avisynth level (or the mpeg decoders badly report to avisynth that the chroma is stored interlaced).

[onesoul]

Originally posted by angelyote
FixBrokenChromaUpsampling() requires YUY2 input. Since the problem doesn't appear after converting to YUY2 it won't work.
Didn't know it required yuy2 input. Sorry...

[fisix]

at least when i used it last there was a check box in divx5.2or3 to assume progressive frames or interlaced frames when encoding, it was right near the place where you can have the encoder deinterlace your stream (doom9 says their deinterlacer isn't very good). this check box seems to be greyed out if it detects a progressive input. load the yv12 avs into virtualdub and see if the check box is greyed out etc etc, it might be a quick test to see if the frames are detected correctly.

shrug. sorry i cant be of any real help, no time to do the yv12 tests

[sh0dan]

The problem you are seeing is broken chroma-upsampling, but not necessarily in AviSynth. The problem is: When-ever an YV12-> RGB/YUY2 conversion has to be made, the conversion routine needs to know if the material is interlaced.

YV12ToYUY2() knows if each frame is interlaced because it can get the information from mpeg2dec. But when you get your information from an AVI source, there is no way for AviSynth to know if the source is interlaced or not - so it assumes the frame is not interlaced, and therefore upsamples chroma is if it is a progressive image.

The problem when displaying things in Vdub is, that DivX or XviD does not have access to the frame/fieldbased information, so they also assume frame-based, when decoding YV12->RGB/YUY2. YV12 fast recompress however will not give the problem if the recieving codec knows that the material it recieves is field-based. The only way to correct this for display is to do the conversion inside AviSynth for previews.

IIRC the converttoyuy2 reacts correctly if you put in an Assumefieldbased() before the conversion.

For the conversion to be accurate at all times with an AVI source, there needs to be a per-frame interlace-detection - probably mostly like the one in Telecide.
Converttoyuy2 should probably also react to the parity bit, but that's not easy to do right now, since the field->frame->field conversions are done at filter creation time (by automatically inserting separatefields/weave).

[Kurosu]

@Sh0dan

You're almost right... But AssumeFieldBased() does this for the whole sequence. On the sample.vob provided by neuron2, you clearly see a difference between:
- AssumeFieldBased().ConvertToYUY2()
- YV12toYUY2(interlaced=true)
- YV12toYUY2()

I think that at least at the avisynth filter level, it's an important matter: what members of vi / PClip / ... have to be modifed so that other filters have the right information at the frame level (I fear that the VideoInfo can't be changed on the fly).

Anyway, such a sequence is just what I needed to test the Turn() in interlaced YV12... (which make me see that a SeparateFields can't be used before Turn, as the field has been rotated after Turn).

[neuron2]

Yes, I had concluded that the chroma upsampling is involved. The scary thing is that all my software players (BSPlayer, FMV, etc.) show the same artifacts when fed the AVS.

Is there not a way to tell DivX that the source is interlaced? Seems like a serious omission if there isn't.

We need to really think things through very carefully for Avisynth 3.0 in regards to proper per-stream and per-frame flags and their handling.

Thank you for your responses, gentlemen.

[trbarry]

What happens if you just deinterlace it and they use an AssumeFrameBased() statement?

- Tom

[neuron2]

Originally posted by trbarry
What happens if you just deinterlace it and then use an AssumeFrameBased() statement? If I use Decomb, I don't need the AssumeFrameBased(), because it sets that correctly. Things look OK then. What concerns me now is that all the players seem to suffer from the CUE.

[sh0dan]

Is there not a way to tell DivX that the source is interlaced? Seems like a serious omission if there isn't.

Since VFW was not designed to handle interlaced material from design, there is no way that we can communicate that a source is interlaced. So DivX (as the converter) cannot know that the YV12 material it's decoding is interlaced.

The players basicply acts the same way = assuming framebased.

Both DivX and Xvid assumes that what they are decoding is frame-based, and therefore upsamples chroma as such.
It's another issue when they are encoding, since they both have interlaced mode, where they are given the information that they will recieve interlaced YV12 -> no problem. Decoding MPEG4 they of course know how it was encoded (interlace mode is stored in the MPEG).

[neuron2]

I meant telling DivX through its user interface that the source is interlaced.

[sh0dan]

That isn't very easy.

When VdubMod opens the file it does it through the VFW interface. So in essense it opens it as an AVI.
When it does so, and recieves YV12 data it requests the currently installed VFW-codecs to decode the YV12 to RGB. This is where XviD or DivX kicks in. They decode the image for vdub, so vdub can display it in it's windows.

So the DivX conversion part is out of AviSynth control - it is handled AFTER the data is delivered from AviSynth. Thus we have no way to tell it about interlacing, etc.

When "Fast Recompress" is enabled, vdub ACCEPTS YV12, and does NOT request any filters to decode it -> speedup, and YV12 data is passed directly through the VFW from AviSynth->codec.

[neuron2]

Yeah. And anyway it just seems to be a display/preview issue.

[trbarry]

Yeah. And anyway it just seems to be a display/preview issue.

That does seem to be the bottom line, and then only for interlaced content.

We had discussions of the chroma conversion problem when starting YV12 support and it was known that you couldn't reliably tell on a frame by frame basis in Avisynth (or after) whether a frame was interlaced.

But when we deinterlace or IVTC it then it becomes properly progressive and frame based. And if we don't convert anything but use fast recompress and tell Divx or Xvid to use interlace then that apparently works okay also.

And I basically deinterlace everything anyway so I guess it hasn't bothered me much. ;)

- Tom

[kilg0r3]

i don't know much about the problem. but, this is not only a preview problem please look at my posts here (http://forum.doom9.org/showthread.php?s=&postid=259519#post259519)

i get these steps / lines whenever ther are pure red or blue components in a picture. please take this problem seriously. its been three months now that i have been thinking it's just my eyes, it's just my eyes, it's just my eyes.

the last two sources (2001, spiderman) were both progressive, so it does not only pertain to interlaced content.

[Prosper]

I dunno what kind of video card you are using, but some of the beta versions of nvidia's detonator drivers do not dispay yv12 overlays correctly. Try using ffdshow alpha to render the video, and in the configuration, turn off yv12 output so it forces the overlay to YUY2 mode: that threw me for quite a loop for a while.

[kilg0r3]

overlay is not the problem. the screen shots are taken without overlay.

[neuron2]

Originally posted by kilg0r3
i don't know much about the problem. but, this is not only a preview problem please look at my posts here (http://forum.doom9.org/showthread.php?s=&postid=259519#post259519)I only really see the issue on your pic of the guy in the ski mask. But I don't follow your reasoning of why it is not only a display/preview issue. If the upsampling is being done always progressive and the source was YV12 sampled as if interlaced, you'll have the problem. Seems to me the issue is the mismatch between the choice made during YV12 sampling and that done during upsampling. Just because the frame appears not to be combed you can't be sure it wasn't sampled using an interlaced chroma algorithm.

i get these steps / lines whenever ther are pure red or blue components in a picture. please take this problem seriously. its been three months now that i have been thinking it's just my eyes, it's just my eyes, it's just my eyes.

the last two sources (2001, spiderman) were both progressive, so it does not only pertain to interlaced content. See above. Can you make available a few frames of the source which is like the ski mask guy (not the other stuff which is not as clearly the same issue)? Thank you.

[kilg0r3]

hi neuron2.

i just thought when i get in the encoded file it can't be a preview thing only. my bad.

another problem is that i am not so familiar with the technical details of the, e.g., don't even know what upsampling means :blush:
kind of material do you need to identify the problem? screen shots, avis or vobs (how can i cut a vob :blush even more:?

btw. have you read my last post in the above mentioned thread? The strange thing is that the preview image of dvd2avi, when captured, does not seem to produce these artifacts.

[IanB]

The confusion evident in this thread looks like another example of the misuse of Framebased vs FieldBased in the YV12 codebase. See my ravings (http://forum.doom9.org/showthread.php?s=&threadid=45550) about this problem.

Perhaps there is a need for new filters YV12InterlacedToYV12Progressive() and
YV12ProgressiveToYV12Interlaced() that remap the chroma subsampling in the
YV12 colour space.

IanB

[neuron2]

Originally posted by kilg0r3
another problem is that i am not so familiar with the technical details of the, e.g., don't even know what upsampling means :blush:YV12 has chroma samples only every other line. YUY2 and RGB have them every line. So, when the YV12 gets converted to RGB for display/preview, the extra lines of chroma must be created. That is called upsampling. There are two ways to do the upsampling, depending on whether the source material was sampled as interlaced, or sampled as progressive. That is not the same as saying whether it is interlaced or not, though it should be. You can sample wrongly or you can upsample wrongly. A mismatch between the two is what causes the artifacts. It's like this:

sample interlaced + upsample interlaced -> OK
sample interlaced + upsample progressive -> BAD
sample progressive + upsample progressive -> OK
sample progressive + upsample interlaced -> BAD

What kind of material do you need to identify the problem? screen shots, avis or vobs (how can i cut a vob :blush even more:?A VOB fragment you can make by using a VOB splitter tool that you can find on the doom9 downloads page!
btw. have you read my last post in the above mentioned thread? The strange thing is that the preview image of dvd2avi, when captured, does not seem to produce these artifacts. That's because it performs the upsampling correctly! And so does VirtualDubMod if you drop the VOB on directly.

[neuron2]

Originally posted by IanB
The confusion evident in this thread looks like another example of the misuse of Framebased vs FieldBased in the YV12 codebase. See my ravings (http://forum.doom9.org/showthread.php?s=&threadid=45550) about this problem.

Perhaps there is a need for new filters YV12InterlacedToYV12Progressive() and
YV12ProgressiveToYV12Interlaced() that remap the chroma subsampling in the
YV12 colour space.

IanB I'll review your ravings later. :)

But for now, your proposal for these remapping tools is an excellent one. I plan to do it. Thank you for thinking of it.

EDIT: It could be done with a general color space conversion suite, e.g., YV12InterlacedToYV12Progressive() would be:

YV12ToYUY2(interlaced=true)
YUY2ToYV12(interlaced=false)

We have the YV12ToYUY2() filters in mpeg2dec3, but the inverses are not there. I haven't looked at the internal ConvertXXX() filters to see if they can do the job. They don't have an interlaced parameter but maybe they observe the frame/field-based status???

[kilg0r3]

@neuron
a 'small' vob fragment is here (http://kilg0r3.cjb.net/downloads/yyy.vob). it is 10 mb. hope you've got a broadband connection. my isdn line moaned for half an hour.

[sh0dan]

I would like to have the time for a proper direction on this, but unfortunately too many external factors are pressing me now for having the time to properly deal with this.

- Handling of interlaced material in 2.5.0 is not very good (at best) - and AviSynth 2.5.0 is not very reliable on this area. :(

- IMO AviSynth should not have so many problems dealing with interlaced material, so this must be fixed!

- Your solutions are good enough for standard operation, but should not be necessary. They are patches over bigger problems, that should be solved gracefully by AviSynth.

- Some problems cannot be solved - as "wrong" YV12 chroma upsampling by external filters for display - those are however not a big problem as it will not have any influence on the final result.

- We (I for instance) should take the time, and get all conversion filters handle interlaced material as it should be handled.

[trbarry]

- Handling of interlaced material in 2.5.0 is not very good (at best) - and AviSynth 2.5.0 is not very reliable on this area.

Sh0dan -

Where do the problems lie? Is it just converting interlaced material YV12 <-> YUY2?

Does ConvertToYUY2 now do a properly different conversion depending upon the IsFrameBased flag?

If not can Marc FD's convert to YUY2 (whatever it's called) be manually specified instead to do it properly?

Are there any known problems in handling progressive (or deinterlaced) material still?

- Tom

[sh0dan]

Framebased processing is ok!

But as I work with interlaced video a lot of problems arise. Mostly because of the deeply mysterious GetParity.
I looked through field.h, and saw that it is mostly used for showing TFF or BFF.

Problems I encountered:

ConvertToYUY2 has problems with some pitches - making it impossible to use after "separatefields". This is probably a bug that never surfaced in XviD development.

Framebased/Fieldbased settings are switched (or at least illogical) in Separatefields / Weave. It seems it would have been so for a long time - but since it is affection other parts of AviSynth this is now getting problematic.

We need color conversion routines that are capable of converting interlaced YV12->YUY2 (and everything else) without breaking upsampling - MPEG2DEC seems like a logical place to start.

It needs testing, testing, testing.. Mostly on my part - that's the time-hog.

Currently AviSynth is in a non-compileable state in CVS - until that get's fixed, testing will not be that easy.

[kilg0r3]

the thing i don't understand is why i get the same artefacts with the old mpeg2dec and avisynth 2 and with nic's mpegdecoder.

@neuron2
have you downloaded the vob? i'd like to delete it from the server.
how could i use decomb to correct the problem?

[neuron2]

Originally posted by kilg0r3
the thing i don't understand is why i get the same artefacts with the old mpeg2dec and avisynth 2 and with nic's mpegdecoder.

@neuron2
have you downloaded the vob? i'd like to delete it from the server.
how could i use decomb to correct the problem? I have it so go ahead and delete it. I am still investigating so please standby. Got distracted by MarcFD's new codec. :)

[neuron2]

Originally posted by sh0dan
- IMO AviSynth should not have so many problems dealing with interlaced material, so this must be fixed!I've made a radical proposal at the avisynth.org forum. Let's follow it up there.

[sh0dan]

I agree very much with you on this issue actually.

Filters acting differently if the video is set to be fieldbased or framebased is not a working solution in real-life, so I actually agree that this paramter should be removed from the core.

Parity should probably remain, since it is needed to pass pairty when doing trims & splices of material that doesn't have the same parity (Parity is a per frame bool that shows if a frame is top-field or bottom field when separated, or TFF or BFF when interlaced video).

So I would definately go for removing the frame/fieldbased property.

[neuron2]

@sh0dan

I see you prefer the followup here. :)

I've had a look at the source for Splice in edit.cpp and although there is a special method Splice::GetParity(), I don't see it being called anywhere. Ditto for Trim.

Please explain in detail in functional terms, why we need parity (other than possibly as a configurable default for functions that require an order parameter).

[trbarry]

I think I never properly understood how Avisynth was using FieldBased, etc.

However most (not all) clips are one or the other all the way through and also maintain the same parity. That does allow a codec to return that info and have it be automatically used for things like ConvertToYUY2, deinterlace, etc.

I did put in code in a recent version of TomsMoComp to use the parity setting though I never recommended anyone using it since it didn't seem very reliable.

Not sure what the answer is. I think I sort of like having this info associated with a clip if it could be made consistant and more or less reliable.

- Tom

PS-pls don't send me off to Avisynth.org again. I just got back from the round trip. ;)

edit: Parity is per-frame? I didn't know that either.

[scmccarthy]

@neuron2&sh0dan

GetFrame isn't always called by the plugin either is it? I think the five virtual functions are also callbacks. So providing a version of the callback functions changes the default behavior of the functions when AviSynth calls it.

Stephen

[sh0dan]

Originally posted by neuron2
[BI've had a look at the source for Splice in edit.cpp and although there is a special method Splice::GetParity(), I don't see it being called anywhere. Ditto for Trim.

Please explain in detail in functional terms, why we need parity (other than possibly as a configurable default for functions that require an order parameter). [/B]

Splice contains this code: bool Splice::GetParity(int n)
{
if (n < video_switchover_point)
return child->GetParity(n);
else
return child2->GetParity(n - video_switchover_point);
}

GetParity is used for maintaining TFF and BFF in advanced edits.

It works like this:

Every filter contains a GetParity call, that will return true or false based on the frame number requested.

If a frame contains interlaced material (with both fields in the same frame), GetParity(int f) will return true if the top field is the first first (TFF). It will return false if the clip is BFF.

When a video is separated, GetParity is used to determine which of two frames is the first. This is used to produce the same (BFF or TFF) parity when a frame is weaved from to separated fields.

The reason GetParity is a good thing to maintain is that it will ensure correct parity if multiple separated sequences are spliced, trimmed or similar. In mpeg2dec parity is derived from the MPEG2 headers.

IsBFF() and isTFF() is merely a supplement to the parity code (as I didn't know what it did).

(this is what I've been able to gather from looking through the current code).

The changes I propose will not break the current API, since it will not change any calls, but it is merely a matter of ignoring the function of SetFieldBased.

[IanB]

Okay we are back here again :sly: (other forum) (http://www.avisynth.org/forum/viewtopic.php?p=94)

My original problem is that in Avisynth 2.5 there are inconvieniences to using the
SeparateFields()/Weave() functionality in my standard Avisynth 2.0 scripts. i.e.
forwards compatibility is broken in the new version.

I did some research, read some code, compared the CVS history and reached the
conclusion my problem was caused by the usurping of the IsFieldBased() internal
method to provide IsInterlaced() functionality in the new YV12 code base.

I have offered some suggestions about how to unbreak the original SeparateFields()
Weave() functionality and provide the obviously needed IsInterlaced() resource
the YV12 codebase needs. I offer any help I can.

@neuron2 (Donald Graft)
...Remove all notion of parity!...
Unfortunately this would destroy any hope for forwards compatibility and make
me quite sad :(

My current plea is :-

Resolve that IsFieldBased() has nothing to do with Interlaced/Progressive:devil:

Leave the current SeparateFields()/Weave()/AssumeFrameBased()/AssumeFieldBased()
parity stuff alone. i.e. as per 2.0

Use some new way of determining Interlaced/Progressive that does not interfere with
any current functionality.

I await the next 2.5beta with baited breath.

IanB

[sh0dan]

The point is not to remove any functionality - just making is consistent.

As it were:

If framebased material (images with both fields in it) is served to separatefields, it separates it. It also marks the video as fieldbased.

If material marked fieldbased is handed to separatefields(), nothing is done.

If material that is has fieldbased property is handed to weave, it is put together again. If it doesn't, material is weaved with every second line from each frame.

I propose:

Separatefield always split material - no matter what it's marked. Weave always weaves frames together, halfens the framerate and doubles the size. We create switches for weave to make it throw away every second line. This way we don't have to deal with the dreaded properties.

[IanB]

Don't forget about Bob(), etc, if you really must do this.

IanB

[Bidoche]

@sh0dan

I am not sure whether changing behavior is a good idea, after all it's this kind of adaptive polymorphim which makes avisynth power.


Anyway, here are my two cents about all this FB stuff :

Maybe all the mess comes from a confusion between clip and frames properties.
FieldBased/Framebased is obviously a clip property.
But TFF/BFF are not, it's fields who should be marked as top or bottom.
Same for Interlaced/Progressive this should be a frame property.

So if we move the props at their good place, everything should be alright.

[trbarry]

I propose:

Separatefield always split material - no matter what it's marked. Weave always weaves frames together, halfens the framerate and doubles the size. We create switches for weave to make it throw away every second line. This way we don't have to deal with the dreaded properties.

What would be the property that determines which sort of color conversion would be done? My deinterlacers assume 2 fields in a frame, regardless of flags, and output 1 field in a frame, currently not setting any flags.

But I was previously suggesting setting AssumeFrameBased after the deinterlace step, which was probably backwards.

So let me get this straight. If I have a progressive clip it should be called FieldBased in order to have a correct color conversion done. Right?

I'll have to think about this one for a bit. I'm not sure a progressive clip and a field based clip created by SeparateFields should both get the same color conversion. Maybe we also need a progressive flag? Dunno.

Are any of these attributes passed along to Xvid or Vdub?

- Tom

[neuron2]

Tom,

Progressive/interlaced is a per-frame property. It is a serious mistake to make it per-clip. Since we don't currently have per-frame properties, that is the source of your dilemma.

I'm supporting Bidoche's position now. :) My first intuition was to get rid of properties that aren't per-clip (because they caused these kinds of unnecessary dilemmas), but Bidoche is right that it's better to keep them but put them where they belong.

In this new scheme, to answer your question: A frame would have a per-frame property marking it as progressive or interlaced. For YV12, you could use that to do correct chroma sampling. It may be that the progressive/interlaced status is not known and is so marked. In that case, you'd fall back to a filter parameter. Also, a progressive frame can be separated, so field-based/frame-based would not be relevant to the color work. Note that after separation, an interlaced frame becomes two progressive frames, and the clip becomes frame-based. Anyway, field-based/frame-based is per-clip, so you wouldn't want to use that anyway.

After deinterlacing *a frame*, you would mark *the frame* as progressive.

[trbarry]

I'm supporting Bidoche's position now.

So am I, mostly, I think. ;)

But his last post crossed with my last one above. Anyway, is there currently a progressive flag passed along with each frame? MPEG2DEC has all this information but is it being passed along like (apparently) TFF & BFF?

And I am still not convinced that doing a SeparateFields creates a frame that needs either progressive or interlaced color conversion. Maybe I just didn't have enough coffee yet but I think that creates sort of a third possibility, an interlaced field with half the pitch. The chroma samples still don't represent points half way between the 2 lines.

One thing we definititely need is an agreed and rigorous definition of where these flags are and what the darn things mean. ;)

(but still keeping compatibility even with the counterintuitive ones)

- Tom

edit: I'm willing to change one or both of the MPEG2DEC's if we need more info from them.

[neuron2]

Originally posted by trbarry
Anyway, is there currently a progressive flag passed along with each frame? MPEG2DEC has all this information but is it being passed along like (apparently) TFF & BFF? No. At least not in the D2V files from DVD2AVI 1.76. I've not looked at later versions. I think all available per-clip and per-frame flags should be put in the D2V file and then mpeg2dec* should put them into the corresponding (new) Avisynth properties.

And I am still not convinced that doing a SeparateFields creates a frame that needs either progressive or interlaced color conversion. Maybe I just didn't have enough coffee yet but I think that creates sort of a third possibility, an interlaced field with half the pitch. The chroma samples still don't represent points half way between the 2 lines. It depends on the algorithm used for the sampling. For example, if you have a 50-50 average, then separating makes a progressive-sampled with 50-50 average. If 75-25, you get a progressive with 75-25, which is not fully correct, but that results from the separation. The invariant is that after separation, each chroma sample applies to two adjacent lines, which is not the case for interlaced. The problem with separating YV12 comes from the nature of YV12, not from the property scheme.

While on the subject... Not everything uses the same sampling scheme and so that too can cause incorrect results. For example, if I take a 50-50 sampled frame and upconvert it using 75-25.

One thing we definititely need is an agreed and rigorous definition of where these flags are and what the darn things mean. ;) For sure. We should base it on the MPEG2 syntax, because that has all the needed properties in the right places. I will shortly post a proposed mapping from MPEG2 syntax to Avisynth properties. An advantage of basing it on MPEG2 syntax is that it is already very well known and understood.

[neuron2]

Now I'm convinced that even field-based/frame-based should be per-frame. Here's why: MPEG2 has picture_structure, which codes a picture as either a field or a frame. In our world, our atoms are frames and not pictures, so we need to denote whether a frame contains a single picture or two field pictures. So, we need a per-frame property.

To further underline this, one can join a non-separated clip to a separated one as long as the sizes, etc., match. Also, a separated clip can have SelectEven() run on it. And a separated clip can be separated again. What is it then!? These show how problematic a per-clip property is, and how the dilemmas dissolve when the property is per-frame.

Field-based/frame-based is per-frame!

EDIT: It seems that field-based/frame based collapses into progressive/interlaced. Since our atom is always a frame, there is no need for a flag that is equivalent to picture_structure. Refer to the following post.

[neuron2]

Tentative proposed properties:

Per-Clip
--------
size
aspect ratio
type (RGB32, YUY2, etc.)
frame rate
user-defined

Per-Frame
---------
progressive/interlaced
tff/bff
rff
user-defined

A properly written mpeg2dec can fully populate these. The information can be very useful to many filters.

A tool can be written that will scan an arbitrary AVI and populate the properties, as well as save the results to a file that filters could use. For example, combing detection as in FD could be used to determine progressive/interlaced. It wouldn't be perfect but could be useful.

But the first step is to agree on the properties and their meanings. Then we can talk about what Avisynth and filters do with them.

@IanB

Will this scheme resolve the difficulties you described? If not, why not?

[sh0dan]

Well, all I can say is, that I see no way of frame-based properties being integrated inti 2.5 - 3.0 is another matter.
Of course we should not change anything in 2.5 that will be changed in 3.0.

But as I see it we have two choices:

1) Make filters behave the same way - always.
2) Have intelligent filters that acts based on user-given properties.

The line is a fine one to draw. 1) Gives consistency, but will most likely add more properties to each filter. 2) May help filters produce more correct results without user intervention, but it also adds the risk of giving unexpected results.

From a programmer point of view, 2) is the most elegant - it will make AviSynth behave more intelligent.
From a user point of view 1) is actually what I see as the best way.

In my experience the best things about working with AviSynth is always knowing what the result will be, and being able to combine filters in strange ways. I find it very annoying, when weave() doesn't perform as I expected and I have to look through documentation to see why it isn't performing right.

IMO ConvertToYUY2(interlaced=true) is more logical than AssumeFieldBased().ConvertToYUY2(). It took me some time getting used to this, but now it actually makes more sense to me. Furthermore we don't risk filters not properly supporting these featues, ie. another deinterlacer than decomb may not set the frame/fieldbased property as it should. Should FieldDeinterlace skip, if the clip was set to framebased?

We might be creating more complexity, both code-wise and user-wise, without getting the payoff that should come from it. The Assume-things haven't been much more that annoyment for me user-wise - and seeing how little they actually matter I would be happy to get rid of them.

(of course I support custom perframe/perclip flags for filters, but that's entirely another matter).

[neuron2]

@sh0dan

Yes, what I am saying is for 3.0. Are your remarks directed at 2.5 or 3.0?

I'm not so bothered about 2.5 but I'm curious what you think about the proposal for 3.0 properties.

[sh0dan]

2.5 (and subsequent 3.0 - as I wrote - I don't think we should change stuff in 2.5 just to be changed back in 3.0).

- So the question remains: Should the properties (Framebased/Fieldbased) be retained or removed from 2.5?

Either way I'll code the changes. But it would be a nice thing for me to be able to remove them - IMO making usability of AviSynth better.

Another thought - the properties could become parameters for the source filter - that would be a compromise I'd be happy to make.

[trbarry]

Donald -

You once posted a link for recommended MPEG2 DVD mastering or some such using all these flags. Do you still have that?

- Tom

[scmccarthy]

mpeg2dec could easily be set to save more info from the mpeg-2 header. Would that help, or is the header info unreliable?

Stephen

[neuron2]

@trbarry

This one?

http://www.dumbterm.net/graphics/dvd/ (scroll down)

@sh0dan

For 2.5, I agree that field-based/frame-based should be removed because it is not a valid per-clip property and it causes more problems than it solves. Filters that need an interlaced flag would take it as a parameter, e.g., ConvertToYV12(interlaced=true) would use interlaced chroma down sampling.

Let's see what others think about it.

[neuron2]

Originally posted by scmccarthy
mpeg2dec could easily be set to save more info from the mpeg-2 header. Would that help, or is the header info unreliable?
Did you miss this bit above?

"A properly written mpeg2dec can fully populate these. The information can be very useful to many filters."

and this bit:

"I'm willing to change one or both of the MPEG2DEC's if we need more info from them." [Tom]

[scmccarthy]

In case you guys are wondering what I was talking about, here's a quote from 'Writing Avisynth plugins':External plugins do not link with avisynth.dll, so they can't directly access functions that are defined in the main Avisynth source code. Therefore, every important function in avisynth.h is either defined inline or declared as virtual. The virtual functions act as callbacks for external DLLs.
Stephen

And the following quote:An Avisynth filter is simply a C++ class implementing the IClip interface. IClip has four pure virtual methods: GetVideoInfo, GetFrame, GetParity, and GetAudio. And now there's a fifth virtual method called SetCacheHints(). A filter whether external or internal does not call any of these directly. They only optionally redefine them. So GetParity is not like GetPitch(), it does not have to be called from within GetFrame().

[Bidoche]

@sh0dan

For per-frame properties who can actually change during a clip, an adaptive behavior is necessary in order to get them right.
Of course we can still provide fallback parameters to force algorithm process.
But I think keeping the smart behavior is good

[sh0dan]

@Bidoche: Could you elaborate?

[trbarry]

Thinking about it a bit more it seems that if you have YV12 and don't change that somewhere in your script then it will eventually be passed to code like Xvid that will assume it is progressive, unless instructed otherwise. We might as well do the same thing and for 2.5 just document that progressive color conversion will be assumed.

An explicit override could still be specified on a ConvertToYUY2 statement if needed.

But there is one more place it should be specified. It seems silly after all the work you've done here but I think I'd like to combine MPEG2DEC2 and MPEG2DEC3 such that the ability to return YUY2 is still optional. The reason is of course that any YUY2 conversion done there still has access to all the MPEG2 flags and, AFAIK, can do it correctly for the folks that still want YUY2 or RGB for some reason.

Note that I'm not backing off my new found love for YV12. I think that is still much better for MPEG2 sourced material. But that assumes you are going to be deinterlacing it somehow in a proper fashion before trying to convert it to something else.

- Tom

[Bidoche]

@sh0dan

Imagine you have a clip with progessive/interlaced frames mixed.
If an operation should be performed differently on each type, an adpative behavior is really useful.
Choosing the algorithm will always get half of your clip wrong.

[ErMaC]

Just thought I'd pipe in here, since I noticed Donald listed Framerate as a per-clip property.

Must AVISynth still be restricted by pseudo-AVI compatability, or could we actually create variable framerate AVS files in the future? That would make dealing with hybrid material so much easier (i.e. Donald's ST:TNG Hybrid Challenge).
Just a quick thought/suggestion, but I know it would then break compatability with the AVI/VFW interface, but DShow can handle multiple framerates if I recall.
And for the standard VFW interface you could always fall back on drop-frame-insertion to fix playback. I.e. if a clip has 24fps and 30fps parts, report it back as 60fps and insert a dropped frame inbetween each 30fps frame and alternating 1 and 2 dropped frames between the 24fps footage (I actually have AVI files that do this).

[scmccarthy]

@ermac

Can any DirectShow player play such an avi file?

Stephen

[ErMaC]

So far I have had no trouble playing it in anything. The trouble is encoding it :P

I first ran into these kinds of files when I was doing Fansub encoding about a year and a half ago. Nowadays, to look cool, many shows have opening sequences which are in 29.97fps progressive, while the rest of the show is in 23.976fps telecined to 29.97. What raw encoders used to do was just decimate the opening sequence and keep it at 24 like the rest of the episode when they IVTC'ed their TV captures. But then someone came out with a tool (I don't have it, I just know it exists) that allowed the insertion of Drop frames to sync them up at 59.94fps by inserting 1 drop frame every other frame in the 30fps portion and alternating 1 and 2 frames in the 23.976fps portion.
The problem was when I would open them up in VirtualDub, and would try to recompress them, it would recompress all the frames (all 59.94 per second of them) and thus the resulting file would be unplayable.
I used AVISynth's SelectEvery to get around this problem, and fed the sections and encoded them separately, but I instead just tacked on the opening and ending sequences from previous episodes so I didn't have to decimate them myself, which is what I would've had to do anyways since I didn't have the tools to re-add the drop frames.

This was back in the days when raw encoders were using MP42, MP43, or DIV3 (occaisionally). Their DShow playback filters dealt with the files fine - you would go to Media Player 6.4's Statistics and you would see Frame Rate: 59.94 and Actual Rate: 29.97 or 23.976 depending on the portion. I thought it was pretty ingenious - it's just a pain to deal with.

Really the drop-frame insertion would only be useful for DirectShow playback, doing that with the VFW interface would be ugly unless VDubMod could be told "If there's a drop frame reported by the AVS file, insert a drop frame in the file being encoded" which I suppose wouldn't be too hard to do, we'd just have to ask them to implement said feature.

Now I will completely acknowledge here that we are totally bastardizing the AVI standard and making it do things it really really was never meant to do - but you can say the same thing about Bi-directional Predicate Frames, too :)

I can post a short sample of one of the Raw files that does this if you guys want to take a look at it.

[sh0dan]

Can't really see any good way of putting in 100% variable bitrate. It is not a part of VFW-API - it has one framerate per AVI file. I don't see using "output_fps = fps1*fps2" with an extremely large amount of duplicate frames as a solution.
Could we move this OT discussion another place?

Back on track:

I have created an ISSE/MMX interlaced YV12 -> YUY2 converter from scratch, and put it in CVS. I hope to be able to do an interlaced YUY2 -> YV12 soon also.

It is currently implemented as ConvertToYUY2(interlaced=true). This may be changed!

My current priority list on needed conversions are:

Interlaced YUY2 -> YV12.
Cleanup of convert.cpp I just realized what a mess it is!
Rewrite of non-interlaced YV12 <-> YUY2.
Interlaced YV12 <-> RGB. (This will probably be implemented using a combination of existing filters - this is NOT easy code!)
Hybrid Frametype detector. As proposed a routine that is capable of detecting interlacing in hybrid material (telecined material for instance).

[Stabmaster-Arson]

I pray to our Lord and savior, Jesus, that variable framerate does not become popular. Variable framerate is BAD mmkay.

Its an encoders/converters worst nightmare.

[sh0dan]

ok - these two points are done:

Rewrite of non-interlaced YV12 <-> YUY2.
Cleanup of convert.cpp I just realized what a mess it is!


Or at least - the progressive YV12 -> YUY2 rewrite is finished and the cleanup has begun.

I realized that the XviD routines doesn't interpolate chroma when doing YV12 -> YUY2 - this is also done now. The speed should be about the same as the XviD routines (a little give and take). Also the speed for non-mod8 resolutions should be up.

I'll release a binary ASAP.

[trbarry]

I can't guarantee the YV12->YUY2 interlaced conversion functions in MPEG2DEC2 are correct but I believe they are. But if so then comparing the results of MPEG2DEC2 vs (non-PP) MPEG2DEC3 converted to YUY2 should now give identical results for pure interlaced source.

But I haven't performed this test yet. ;)

- Tom

[IanB]

Just two quotes from the many to stress this point.....
Originally posted by neuron2
Now I'm convinced that even field-based/frame-based should be per-frame....Yes, if we continue down the current evil path of hijacking the IsFieldBased() property
to distinguish Progressive/Interlaced status.

As originally implemented it is most definitly a clip attribute, it simply meant the clip had
been SeparateFields()'ed.

Originally posted by sh0dan
IMO ConvertToYUY2(interlaced=true) is more logical than
AssumeFieldBased().ConvertToYUY2(). It took me some time
getting used to this, but now it actually makes more sense to me....The first is more logical because the second should never have seen the light of day.
AssumeFieldBased() should NEVER have been hijacked to mean Interlaced, it originally
meant the clip has been SeparateFields()'ed and it should have stayed that way.

:mad: Let me try and get this point across 1 more time. :mad:

IsFieldBased() as originally defined simply meant the clip had been thru SeparateFields()
It has been HIJACKED to try and mean the clip is Interlaced and this great cock-up has
caused all the problems and confussion in this current thread and others.

This is a programming attribute collision!! Where one piece of data means two
different and distinct things. It is a software design BUG!

Really simply solution
Stop abusing IsFieldBased() to determine Interlaced
Revert all the code abusing IsFieldBased() back to the original working state.
Leave SeperateFields()/Weave() alone until 3.0
Decide if Progressive/Interlaced will be a command argument, a frame attribute
or a clip attribute.
Implement it that way everywhere! Don't even think about using IsFieldBased().

If the chosen implementation is deamed to involve IsFieldBased() then reference
it IN ONLY ONE PLACE, in the IsInterlaced() implementation. Don't mix up
these two concepts again.

IanB

P.S. If anything the frames of clip.AssumeFieldBased() are certainly NOT interlaced,
they are the individual fields of a possibly interlaced, possible progressive clip and
individual fields can never be interlaced (a 2nd time)!

[sh0dan]

@IanB: We DO know. And we are currently INVESTIGATING what method is worth pursuing - <g> relax, man!

IMO the original idea of adding i VideoInfo property called "Fieldbased", and using it for very filter specific purposes isn't a very clever design, but when there are only one developer, these things do easily happend.

Using an "is_field_separated" property would be much more nice, if it is a good idea. IMO the original property is NOT a good idea to preserve, since it creates an unpredictable behaviour.

IMO SeparateFields/Weave should always behave the same, no matter the Video!

You could have been there shouting at me, when I asked if anybody knew how the Fieldbased/Parity stuff worked MONTHS ago! You are NOT being very constructive! :mad:

[sh0dan]

IMO SeparateFields/Weave should always behave the same, no matter the Video!


Let me elaborate:

I'm my opinion the primary power of AviSynth is total user control - you have complete control over every aspect of your video. When AviSynth makes decisions for me, I get annoyed, because I always want a filter to produce the same results.

So when AviSynth makes decisions on how to treat a video signal based on previous filters this is kinda annoying to me. This is why I want to get rid of this situation. The point of the new conversions, and the suggestions to Separatefields/Weave is to make them behave the same no matter what material they are fed with.

[Si]

I'm my opinion the primary power of AviSynth is total user control - you have complete control over every aspect of your video. When AviSynth makes decisions for me, I get annoyed, because I always want a filter to produce the same results.

Can I add in my two pennyworth.

The syntax ConvertToYUY2(interlaced=true) is excellent and the way to go in my opinion.

Therefore SeparateFields and Weave should have optional parameters to force their behaviour instead of relying on the parity flag.

regards
Simon
PS I don't think IanB is being aggressive just assertive :)

[IanB]

Originally posted by sh0dan
@IanB: We DO know. And we are currently INVESTIGATING what method is worth
pursuing - <g> relax, man!Most sorry for Yelling I've been dancing around this issue for many days in many
forums and I haven't been getting acknowledgment I was on the right track or
argument that I was wrong and completely off tangent. ;)

IMO the original idea of adding i VideoInfo property called "Fieldbased", and using it
for very filter specific purposes isn't a very clever design, but when there are only
one developer, these things do easily happend.

Using an "is_field_separated" property would be much more nice, if it is a good idea.
IMO the original property is NOT a good idea to preserve, since it creates an
unpredictable behaviour. I totally agree the Naming used in this part (and others) is evil :devil: and "Parity" as a
name, what has that got to do with field order.

IMO SeparateFields/Weave should always behave the same, no matter the Video!
SeparateFields() no argument here.

Weave() and Bob() have a little problem in needing Field Order information i.e. "Parity".
This gets very complicated when Trim(), DeleteFrame() or DuplicateFrame() get involved.

Think how you would handle deleting a single field in a clip, now the clip has 2 top or
bottom fields adjacent. The current very obtuse code does handle this gracefully.

Hence my suggestion 2. leave field.cpp alone until 3.0

You could have been there shouting at me, when I asked if anybody knew how
the Fieldbased/Parity stuff worked MONTHS ago! You are NOT being very
constructive! :mad: I only joined this forum in January, and I did make an attempt at providing a leg up on
how Parity() abuses OO method overlay in "Dilema - Crop restriction - Fieldbased clips (http://forum.doom9.org/showthread.php?s=&postid=259748#post259748)"

Anyway how can I help? Revue code, suggest algorithms, Consolidate my current
ravings into one post, just ask.

IanB

[neuron2]

Originally posted by IanB
I've been dancing around this issue for many days in many
forums and I haven't been getting acknowledgment I was on the right track or
argument that I was wrong and completely off tangent. ;) How can you say that? You sparked a big discussion that is actively going on right now. But your solution doesn't prevail because you use the biggest and boldest font. One solution presented is to ditch the whole idea of field-based/frame-based. If that were done it could not be confused with interlaced/progressive. Why don't you comment on that?

Weave() and Bob() have a little problem in needing Field Order information i.e. "Parity".
This gets very complicated when Trim(), DeleteFrame() or DuplicateFrame() get involved.I don't think so. You'd know the tff/bff status of each frame, so where is the problem?

Think how you would handle deleting a single field in a clip, now the clip has 2 top or
bottom fields adjacent. The current very obtuse code does handle this gracefully. I'm thinking, what is the problem? Deleting a field is a reasonable thing to do in some circumstances, and can already be done with Avisynth. Anyway, our concern is not with parity for 2.5, but whether we really need the field-based/frame-based property.

EDIT: I see now there is ambiguity about what sh0dan is proposing regarding parity (tff/bff handling). Does he propose to remove its influence from Weave() and others, or does he just propose to change its syntax? shodan, can you please elaborate? On the field-based/frame-based property, no one has yet argued to keep it.

Anyway how can I help? We're seriously considering ditching the field-based/frame-based property for the reasons given in this thread. Please comment on that.

[sh0dan]

I'll do a suggestion for an implementation, so we can have mode code and less talk. :)

[scmccarthy]

I had to preceed Bob() with SwapFields().

My comment is that taking a hard look at how mpeg2dec works might be the only way to fix these problems. AviSynth and mpeg2dec have to come to an agreement.

mpeg2dec must work the same way as DV captured material. You see? We want Avisynth to work the same betweem DVDs and Digital Video.

Stephen

[trbarry]

But MPEG2DECx also gets to look at the progressive frame flag (whatever it's called) which is not currently being passed along.

On the good side, I've even got P4 optimized SSE2 code for the MPEG2DEC color conversions, though it is not included yet.

- Tom

[scmccarthy]

Does it make sense to suggest that the GOP in mpeg2dec needs to be stuffed with more info? Or am I getting the terminology wrong?

Stephen

[trbarry]

"GOP" to me is a Group Of Pictures, a I-frame and it's following P&B frames. I don't think any control block is passed to Avisynth for each of these so I'm not sure what you are asking.

- Tom

[IanB]

So as to keep all the relevant discussion in one place the following
is a transcript from the Developer Forum (http://www.avisynth.org/forum/viewtopic.php?p=94).

By: IanB Posted: Fri, 21 Feb 2003 3:37 pm
Post subject: Frame/Field Based vs Interlaced/Progressive

Dear Richard Berg, TRBarry, Hakko504, Neuron2, Sh0dan, et el

I have been trying to gently provoke some sane discussion about the
contorted use of the IsFieldBased() flag with respect to the new YV12
implementation in avisynth 2.5 on the Doom9/Avisynth forum.

Being somewhat unsuccessful I thought I might try a more direct
approach to some of the key players here in this more quiet forum.

My research into the original Avisynth source reveals that FrameBased
and FieldBased concept derive from SeparateFields() and Weave()
[field.cpp,h] implying the following definitions:-

FrameBased :-
Video samples are whole frames, either progressive or interlaced.

FieldBased :-
Video samples are individual fields. Alternate samples are top, bottom
field. Usually the result of a SeparateFields() operation.

The new YV12 codebase has subsumed these definition and is using
FieldBased to imply Interlaced! Which is at odds with the original
coding intent and is causing subtle changes in the beviour of
unrelated verbs like Crop().

What is apparent is this needs to be clearly and concisely defined
and rigidly adheared to in the core Avisynth code.

I was proposing a new verb AssumeProgressive() to match with the
current AssumeTFF() and AssumeBFF() both which imply a
FrameBased interlaced source or indicate the parity with FieldBased
(i.e field separated) samples.

For background the following thread is a current example of the
difficulty http://forum.doom9.org/showthread.php?s=&postid=263776

And the following post my original discussion of the problem
http://forum.doom9.org/showthread.php?s=&threadid=45550#post259273

Regards
IanB


By: Guest Posted: Fri, 21 Feb 2003 6:50 pm

I wrote a "note to self" when I looked at GetParity.

http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/avisynth2/avisynth/field.h?rev=1.5&only_with_tag=MAIN&sortby=date&content-type=text/vnd.viewcvs-markup

IsFieldBased is unfortunately used in the complete opposite way as it
would be expected.

AssumeProgressive() actually makes nice enough sense.

But the field-based code needs reworking.


By: sh0dan Posted: Fri, 21 Feb 2003 6:58 pm

Oh "Guest appear cortesy of sh0dan Records"

Quote:

> AssumeFrameBased throws away the existing information and assumes
> that the clip is frame-based, with the bottom (even) field dominant
> in each frame. (This happens to be what the source filters guess.)
> If you want the top field dominant, use ComplementParity afterwards.
>
> AssumeFieldBased throws away the existing information and assumes
> that the clip is field-based, with the even-numbered fields being
> bottom fields and the odd-numbered fields being top fields. If you
> want it the other way around, use ComplementParity afterwards.

This is a stretch in my opinion.

Framebased - makes AviSynth think that both fields are present in the
same frame - completely backwards to me.

AssumeFieldBased makes AviSynth assume that fields have been separated.

The above distinguishment doesn't make any sense to me. For now, I have
flipped (most) functionality around, so that:

Fieldbased: Video is asummed to be interlaced.
Framebased: Video is assumed to be progressive.

We could add a separate property for video that has been split, but
IMO it would be much better if the SeparateFields and Weave/DoubleWeave
didn't act on this property, but always performed the same action on
the video.

Does this make any sense?
_________________
Regards sh0dan // Klaus Post.


By: IanB Posted: Fri, 21 Feb 2003 9:47 pm

Sh0dan wrote:

> AssumeFrameBased() throws away the existing information and assumes
> that the clip is frame-based, with the bottom (even) field dominant
> in each frame. (This happens to be what the source filters guess.)
> If you want the top field dominant, use ComplementParity afterwards.
>
> AssumeFieldBased() throws away the existing information and assumes
> that the clip is field-based, with the even-numbered fields being
> bottom fields and the odd-numbered fields being top fields. If you
> want it the other way around, use ComplementParity afterwards.

Apart for the effects of the new AssumeTFF() and AssumeBFF() verbs.

> This is a stretch in my opinion.
>
> Framebased - makes AviSynth think that both fields are present in
> the same frame - completely backwards to me.
>
> AssumeFieldBased makes AviSynth assume that fields have been separated.

This is a very eloquent and succinct explanation of how we both appear
to understand how the original Avisynth works.

> The above distinguishment doesn't make any sense to me. For now,
> I have flipped (most) functionality around, so that:

> Fieldbased: Video is asummed to be interlaced.
> Framebased: Video is assumed to be progressive.

Although I agree with your sentiment, I have to say it is a little bit
unfortunate that you are redefining script behaviour with these changes.
Old Avisynth users know and expect the current behaviour and redefining
it just ends up confusing everybody and losing any hope of backwards
script compatibility.

> We could add a separate property for video that has been split, but IMO
> it would be much better if the SeparateFields and Weave/DoubleWeave
> didn't act on this property, but always performed the same action on
> the video.
>
> Does this make any sense?
> _________________
> Regards sh0dan // Klaus Post.

I would urge you in the most strongest terms to leave the current
definitions of AssumeFrameBased() and AssumeFieldBased() and their
interaction with SeparatedFields() and Weave() alone.

For better or worse that is the way they work and the way peoples
existing scripts expect them to work.

In order to move forward I would propose that the old verbs
AssumeFrameBased() and AssumeFieldBased() simply be declared
depreciated and brand new script verbs be declared that accuratly
describe the state of play. i.e

Verbs
=====
AssumeCompleteFrames() depreciates AssumeFrameBased()
AssumeSeparatedFields() depreciates AssumeFieldBased()

AssumeUnknownPI() declares Progressive/Interlaced unknown state
AssumeTFF() declares Interlaced source, top field dominant
AssumeBFF() declares Interlaced source, bottom field dominant
AssumeProgressive() declares Progressive source

Properties
==========
IsCompleteFrames() depreciates IsFrameBased()
IsSeparatedFields() depreciates IsFieldBased()

IsUnknownPI() Test for Progressive/Interlaced state unknown
IsInterlaced() Test for Interlaced source
IsTFF() depreciates/extends Getparity()
IsBFF() depreciates/extends Getparity()
IsProgressive() Test for Progressive source

For safety clips with undefined Progressive/Interlaced state should
have all the even height protection of interlaced clips and doing
things that need the P/I status like YV12 -> YUY2 or RGB conversion
throw a script exception.

For completness internal methods like isFieldbased() should only
be used in field.cpp/h and be replaced with new isProgressive()
and isInterlaced() methods in the new YV12 code that need this
information.

A search of a good Thesaurus may yield better terms than I have
used but I hope the idea and sentiment is clear.

IanB


By: neuron2 Posted: Sat, 22 Feb 2003 3:26 am

What a contorted mess we are creating. And why!? I am going to
propose a simple solution.

Remove all notion of parity! For filters that want to
disassemble/reassemble frames, just give them a parameter to
specify the order of splitting/reassembling. Leave it totally
up to the user to determine whether his actions make sense for
the source material and task he is trying to accomplish.

For example, instead of SeparateFields(), which makes an unnecessary
assumption that the source is composed of fields, just have:

Separate(first=)

You could use it on a progressive clip (indeed on any clip) if
you like. Why not? For example this would be a reduce by two
vertically for a progressive frame:

Separate(first=even).SelectEven()

It sounds radical but why do we really need this whole idea of
parity? It doesn't work correctly anyway for all the known reasons.
And there seems no reason for having the frame-based/field-based
distinction either (at the scripting level, per-frame flags such
as progressive/interlaced carried in the clip itself is another
matter entirely). Think of how this would simplify the bizarre
code we have today and make everything consistent for all kinds
of source material. We are creating all these issues for ourselves
for nothing, IMHO.

IanB proposed these properties:

IsUnknownPI() Test for Progressive/Interlaced state unknown
IsInterlaced() Test for Interlaced source
IsTFF() depreciates/extends Getparity()
IsBFF() depreciates/extends Getparity()
IsProgressive() Test for Progressive source

Every one of them can change during a single clip! That is the
bottom line. All of this per-clip flag business is useless.

I humbly wait to be refuted. I am willing to table a complete
and consistent scheme if people are willing to consider it.
_________________
Donald Graft
Filters for Avisynth and VirtualDub
http://shelob.mordor.net/dgraft/


By: sh0dan Posted: Sat, 22 Feb 2003 6:19 am

Followup:

http://forum.doom9.org/showthread.php?s=&postid=266410#post266410
_________________
Regards sh0dan // Klaus Post.

[trbarry]

Well, we've gone round and round and mostly proved that most of us (including me) had it backwards on the first cut. But I still lean towards keeping it compatible with the way it used to be and just adding some extra flags later.

I'm not sure. Did anything get decided?

- Tom

[IanB]

Okay, I have had yet another read thru the source. The following is
a list by functional entity that currently uses the .IsFieldBased()
method and what I have found about it with respect to nuking the Field
Separated Attribute.

SeparateFields()
In SeparateFields() the use of IsFieldBased() is a protectionist
restriction to prevent the lose of top field/bottom field state
information from an already field separated clip by the user.

--- It can probably be derestricted right now without problem.

Weave()
Weave() is implemented internally as a DoubleWeave().SelectEven()
hence the apparent transparency (see below) on IsFrameBased() clips.

--- Refer to DoubleWeave()

DoubleWeave()
In DoubleWeave() the use of IsFieldBased() causes 2 different and
distinct behaviours. (This is what sh0dan so rightly loaths)

For IsFieldBased() clips it produces a unique weaving of each
field with both adjacent fields. It uses the internal primitive
DoubleWeaveFields() i.e.

input 0b 1t 2b 3t 4b 5t
output 0b1t 2b1t 2b3t 4b3t 4b5t 4b5t

Note :- the last 2 frames are 4b5t (Boundary condition in GetFrame())

For IsFrameBased() clips it does an implicit SeparateFields() and
then produces a unique weaving of each field with both adjacent
fields only with much greater efficiency. It uses the internal
primitive DoubleWeaveFrames() i.e.

input 0b1t 2b3t 4b5t
output 0b1t 2b1t 2b3t 4b3t 4b5t 4b5t

Having the following piece of script do a 1 field phase shift
with high internal efficient I find a very great boon.

. DoubleWeave().SelectOdd()

--- Removing the Field Separated property will break legacy script
support for DoubleWeave().

Bob()
In Bob() the use of IsFieldBased() causes 2 different and distinct
behaviours. (This is what sh0dan so rightly loaths)

For IsFrameBased() clips it adds a SeparateFields() to the front of
the GetFrame() chain and proceeds as per the IsFieldBased() case.

For IsFieldBased() clips it produces a double rate, IsFrameBased(),
Vertically resized clip with the top fields moved up 0.5 lines and
the bottom fields moved down 0.5 lines.

--- Removing the Field Separated property will break legacy script
support for Bob().


GetParity()

This basic GetParity() method chain holds the Top field/Bottom field
information for a clip.

inline bool __stdcall GetParity(int n)
{ return parity ^ (vi.IsFieldBased() && (n & 1)); }

(parity is a private bool)

Each child that does something to effect that information overlays
this method with a new piece of code that possible chains to it's
parents original method. i.e.

For ComplementParity() the following code is overlayed

inline bool __stdcall GetParity(int n)
{ return !child->GetParity(n); }

After unwinding the murkey mire of this "parity" code I have to
respect the ease with which it tracks top field/bottom field state
thru an AviSynth session.

---Removing the Field Separated property will require the design
of a new top field/bottom field state tracking algorithm.


ConvertFPS()

The algorithm used in ConvertFPS() doesn't appear quite right
when used with field separated content. It needs some help from
Bob(). It hacks with the GetParity() method when IsFieldBased()

---It currently does not make correct use of the information


ShowFrameNumber()

For IsFieldBased() clips the odd numbers are offset to the left
of the display area, this is a useful diagnostic feature.

---Cosmetic effect only


The rest (new YV12 code)

This code is basicly treating IsFieldBased() to mean IsInterlaced()
and enforcing 2**n height restrictions.

---Should be using Progressive/Interlaced determination


YV12 progressive/interlaced chroma sampling

Progressive YV12 is easy to understand each 2x2 Luminance group
has single corresponding chroma samples. i.e.

Progressive Frame
=================
Y_Y.Y_Y Line 1
C__.C__ Lines (0.5)*1 & (0.5)*2
Y_Y.Y_Y Line 2
___.___
Y_Y.Y_Y Line 3
C__.C__ Lines (0.5)*3 & (0.5)*4
Y_Y.Y_Y Line 4

Interlaced YV12 has a bit of a twist, the samples of each field
go together and just to add some spice there is an alternating
1 line vertical spacial displacement of the chroma data.

Interlaced Frame
================
Y_Y.Y_Y Line 1t
C__.C__ Lines (0.75)*1t + (0.25)*3t
Y_Y.Y_Y Line 2b
___.___
Y_Y.Y_Y Line 3t
C__.C__ Lines (0.25)*2b + (0.75)*4b
Y_Y.Y_Y Line 4b

In one of my earlier posts I suggested Field Separated video
samples could be considered as Progressive, this is very clearly
wrong. Field Seperated YV12 Interlaced samples are a completely
different animal to Vanilla Progressive samples and (forced) Field
Separated YV12 Progressive are just a really viscious little animal.

Top Seperated Field (Note (0.5) line up weighting)
===============
Y_Y.Y_Y Line 1t
C__.C__ Lines (0.75)*1t + (0.25)*3t
Y_Y.Y_Y Line 3t
___.___
Y_Y.Y_Y Line 5t
C__.C__ Lines (0.75)*5t + (0.25)*7t
Y_Y.Y_Y Line 7t

Bottom Seperated Field (Note (0.5) line down weighting)
=================
Y_Y.Y_Y Line 2b
C__.C__ Lines (0.25)*2b + (0.75)*4b
Y_Y.Y_Y Line 4b
___.___
Y_Y.Y_Y Line 6b
C__.C__ Lines (0.25)*6b + (0.75)*8b
Y_Y.Y_Y Line 8b


Top Progressive Separated Frame (Note the chroma samples are (0.5) from another frame)
=========================
Y_Y.Y_Y Line 1
C__.C__ Lines (0.5)*1 & (0.5)*2
Y_Y.Y_Y Line 3
___.___
Y_Y.Y_Y Line 5
C__.C__ Lines (0.5)*5 & (0.5)*6
Y_Y.Y_Y Line 7

Bottom Progressive Separated Frame
===========================
Y_Y.Y_Y Line 2
C__.C__ Lines (0.5)*3 & (0.5)*4
Y_Y.Y_Y Line 4
___.___
Y_Y.Y_Y Line 6
C__.C__ Lines (0.5)*7 & (0.5)*8
Y_Y.Y_Y Line 8


---Truely accurate YV12 rendering is going to need to be aware
of Field Separated clips

I'll continue this discussion further in the next post.
(I need a break)

IanB

Fix Dyslexia with Top/Bottom in ascii art for siwalters
Fix Horiziontal chroma siteing in ascii art for siwalters

[IanB]

SummarisingSeparateFields()
--- It can probably be derestricted right now without problem.

Weave()
--- Refer to DoubleWeave()

DoubleWeave()
--- Removing the Field Separated property will break legacy script
support for DoubleWeave().

Bob()
--- Removing the Field Separated property will break legacy script
support for Bob().

GetParity()
---Removing the Field Separated property will require the design
of a new top field/bottom field state tracking algorithm.

ConvertFPS()
---It currently does not make correct use of the information

ShowFrameNumber()
---Cosmetic effect only

The rest (new YV12 code)
---Should be using Progressive/Interlaced determination

YV12 progressive/interlaced chroma sampling
---Truely accurate YV12 rendering is going to need to be aware
of Field Separated clips

Ironicly the only stuff that would depend on a Fields are Separated
flag is the new YV12 conversion code. Everything else is either
misusing the current flag, cosmetic or giving rise to variant
behavior.

I have a particularly strong bias towards maintaining legacy support
across adjacent releases of software. It help tremendously when doing
Quality Assurance. Test #1 is run the final QA suite of the last
release against the current product. Test #2 is run the new QA suite
against the product. The new QA suite of course has all the tests
for depreciated features scheduled to be expunged in the next release
removed and tests for all the new features added.

I strongly agree with sh0dan's sentement about variant behavior in
software generally, and to that end we should aim to clean up those
elements of the language in a controlled and scheduled way.

To that aim I would suggest that :-
Weave()
DoubleWeave()
Bob()
All be scheduled to be depreciated in version 3.(?)

The following invariant versions of the functions be added :-
WeaveFields()
DoubleWeaveFields()
DoubleWeaveFrames()
BobFields()
These new functions always do the same task irrespective of
any flags or states. Let the user beware.

And our friend :-
SeparateFields()
Can be quietly modified to always separate fields irrespective
of any flags or states.


As for the Fields have been Separated flag it is fairly
ingrained in core of how AviSynth works. Removing it is certainly
possible but I am not sure anything that plugs the gaps would be
much less offensive or really worth the effort of replacing it.

Getting things like Progressive/Interlaced determination and per
Frame attributes working should be a much higher priority than
depreciating working functionality.

IanB

[Si]

Ian - I think your post finally gets across to me what interlaced YV12 is about :)

But PLEASE edit your post and put the top separated fields above the bottom ones to make things easy for us spatial challenged readers. :)
Ta very much for swapping them :D

regards
Simon

[Bidoche]

Here is what I have in mind for 3.0 (as of the discussion for now) :

Clips are either fields clip or frames clips, so this flag remains in VideoInfo but not top/bottom

if the clip is made of fields, frames are marked with a field property which has a top/bottom flag.

if it's a frame clip, frames have a frame property with 3 states :
progressive, interlaced and unknown.
Somehow when a frame is queried for its state, unknown state should be resolved in one of the others.


For methods :

in IClip, GetParity is deprecated (if clip is field one it can be mapped to GetFrame(n).IsTop() ...)

VideoFrame gets new methods (non exhaustive list) :

bool IsField();
bool IsFrame();

bool IsTop() throw(IllegalStateException);
bool IsBottom() throw(IllegalStateException);

void SetAsField(bool top)

bool IsInterlaced() throw(IllegalStateException);
bool IsProgressive() throw(IllegalStateException);

void SetAsFrame(bool interlaced);

[IanB]

Originally posted by Bidoche
Here is what I have in mind for 3.0 (as of the discussion for now) :
.....
if it's a frame clip, frames have a frame property with 3 states :
progressive, interlaced and unknown.
You might try toying with a 4 state model :- progressive, interlaced_TFF, interlaced_BFF
and unknown. If Neuron2 gets his wish for per frame properties this will extend better and
in the meantime encourage people to code as if the property can vary.Somehow when a frame is queried for its state, unknown state should be resolved in one of the others.
.....And set coding standards that unknown should have the worst case restrictions applied.

IanB

[Bidoche]

You might try toying with a 4 state model :- progressive, interlaced_TFF, interlaced_BFF I don't get the point of interlaced_TFF and interlaced_BFF.
What is it for ?
It's separatefields who should say if it's want TFF or BFF, no ?

If Neuron2 gets his wish for per frame properties this will extend better and With my current properties design, everybody can create and add new ones (but not mess with others, you need the property key for that)

And set coding standards that unknown should have the worst case restrictions appliedI am thinking of it as an internal transitory value, VideoFrame will never report this state.
If this can be done, others won't have to worry about, only VideoFrame security check would.

[IanB]

Originally posted by Bidoche
I don't get the point of interlaced_TFF and interlaced_BFF.
What is it for ?
It's separatefields who should say if it's want TFF or BFF, no ?

I was thinking maybe some intelligent source filters (mpeg?) or perhaps some fancy
autoanalysis filter might like to be able to throw field order info into the mix.

I guess it's a data representation problem.

1. Unknown
2. Progressive
3. Interlaced
3.1. Top Field First
3.2. Bottom Field First

or

1. Unknown
2. Progressive
3. Interlaced - Top Field First
4. Interlaced - Bottom Field First

IanB

[Bidoche]

Yes, but I stil don't get the use we can get from saving the original field order.

[Si]

I don't get the point of interlaced_TFF and interlaced_BFF.

For completeness.

However, I personally would vote to just have a separate TFF (true/false) flag because ...

I deal mostly with captured TV material that is inherently interlaced as a medium.

I then try to make it as progressive as possible (as most of the actual footage is film progressive) and encode it as progressive MPEG-2 as SVCD.

However, I still need to correctly set the TFF flag (to true in my case) in the MPEG stream otherwise my standalone DVD player will not output the occasional frames that contain true interlaced fields in the correct order.

regards
Simon

[sh0dan]

Brand new code for interlaced and progressive YUY2 -> YV12 is in CVS. It currently requires ISSE, but I'll do an MMX version soon - hopefully before the next CVS binary is relased.

(It's both better and faster than the current XviD code).

[scmccarthy]

Field Seperated YV12 Interlaced samples are a completely Either I have no idea what you mean or you were right the first time.

I have carefully traced the way chroma is stored in an interlaces frame. It is exactly how it would be stored it you interlaced two progressive frames. It is purely the result of each field being sampled separately. That is all it is and it is much easier to understand if seen that way.

Stephen

[trbarry]

Yes, but I stil don't get the use we can get from saving the original field order.

Maybe I'm getting lost in Avisynth terminology again. But if 2 fields are stored in a frame then you need to know which one is first (in time) in order to deinterlace them, or even to display them on an interlaced TV.

- Tom

[Bidoche]

Hum... interlaced display, didn't think of that.

Ok then interlaced_TFF and interlaced_BFF and the according VideoFrame methods.

[Si]

@scmccarthy
I have carefully traced the way chroma is stored in an interlaces frame. It is exactly how it would be stored it you interlaced two progressive frames. It is purely the result of each field being sampled separately. That is all it is and it is much easier to understand if seen that way.

Have you got a reference I could look at.

I'm beginning to wonder if saying something is YV12 is sufficient definition.

Is YV12 not a storage concept with the ability to store different types of 4:2:0 video. From my searches it would seem that MPEG-2 4:2:0 is not chroma sampled the same as PAL DV 4:2:0 for instance.

(or am I just coming to the party too late and clueless as usual?)

regards
Simon

[IanB]

Originally posted by scmccarthy
Either I have no idea what you mean or you were right the first time.

I have carefully traced the way chroma is stored in an interlaces frame. It is exactly how it would be stored it you interlaced two progressive frames. It is purely the result of each field being sampled separately. That is all it is and it is much easier to understand if seen that way.

Stephen

Yes it is stored the same way, but the spacial position of the chroma is offset 0.5 line up and down in alternate fields, and this is very important when converting to another color space or just even processing the the image information.

Top Seperated Field
> ===============
> Y_Y.Y_Y Line 1t
> C__.C__ Lines (0.75)*1t + (0.25)*3t <-- Note the weighting, 0.5 line up
> Y_Y.Y_Y Line 3t
>
> Bottom Seperated Field
> =================
> Y_Y.Y_Y Line 2b
> C__.C__ Lines (0.25)*2b + (0.75)*4b <-- Note the weightings, 0.5 line down
> Y_Y.Y_Y Line 4b

And with the Force separated Progressive the sample is (0.5) from the other field.

A really good article "The Chroma Upsampling Error and The 4:2:0 Interlaced Chroma Problem by Don Munsil and Stacey Spears (http://www.hometheaterhifi.com/volume_8_2/dvd-benchmark-special-report-chroma-bug-4-2001.html)" gets right into 4:4:4, 4:2:2 & 4:2:0 chroma sampling for both progressive and interlaced formats. The graphic are also a lot better than my crappy ascii art.

IanB

:Edit^3: Simon, beat me to the punch by 2 minutes and yes DV spacial chroma sampling is
subtly different Cr 0.5 pixel down, Cb 0.5 pixel up for PAL DV and NTSC DV starts life as 4:1:1. **********! :rolleyes:

For good images of the issue see Adam Wilt site - DV Pix (http://www.adamwilt.com/pix-sampling.html)

:Edit^3: Smells like we need a general purpose YV12 chroma realignment filter.

:Edit^4: Fix ascii art horizontal positioning

[sh0dan]

@IanB: Nice spotted! Thanks for the info.

I corrected the interlaced YV12 -> YUY2 algorithm to have properly 25/75 weighing, and it actually helps some extreme cases. (I use the "Twin Peaks" logo - saturated green outline, with red filling). I'll be put in CVS as soon as I have done the MMX version.

An easy test is:

weave()
converttoyuy2(interlaced=true)
separatefields()

[IanB]

Shodan,

I noticed in Convert.cpp Line 379-394 a latent bug. When the clip is FrameBased vi.height can attempt to be odd (Nice crash!) :eek:


/**********************************
******* Convert to YV12 ******
*********************************/


ConvertToYV12::ConvertToYV12(PClip _child, bool _interlaced, IScriptEnvironment* env)
: GenericVideoFilter(_child),
interlaced(_interlaced)
{
if (vi.width & 1)
env->ThrowError("ConvertToYV12: image width must be multiple of 2");
if (interlaced && (vi.height & 3))
env->ThrowError("ConvertToYV12: Interlaced image height must be multiple of 4");
if ((!vi.IsFieldBased()) && (vi.height & 1))
env->ThrowError("ConvertToYV12: image height must be multiple of 2");
isYUY2=isRGB32=isRGB24=false;

Should be something like this....

if (interlaced && (vi.height & 3))
env->ThrowError("ConvertToYV12: Interlaced image height must be multiple of 4");
if (vi.height & 1)
env->ThrowError("ConvertToYV12: image height must be multiple of 2");

IanB

[sh0dan]

@IanB: Yes - all fieldbased/framebased stuff will have to be changed (based on this thread). There is also some checking in avisynth.cpp and other places (crop/resize) that needs to be changed. But I would like to do them all at once - but soon, I promise!

[Bidoche]

I am currently thinking at integrating constraints check into the properties system. (bascically each property will have to check the integrity of the whole set and will throw errors when needed).

More news later.

[trbarry]

For 3.0 I'd like to propose one more flag attribute for each frame. This would say the frame contains unknown garbage.

There is a version of my MPEG2DEC_ER.dll (shipped with MPEG2DEC2) that has a tiny bit of error recovery for handling questionable clips. It sets a trap before trying to decode each frame and recovers, returning garbage if an exception occurs. This is less useful (but can occur) for DVD rips but is a lot more probable for digital TV captures which can have garbage sections whenever there are reception breakups. So in problem cases this is sometimes the only way to continue.

Currently this version is only for YUY2/2.0 Avisynth but if I brought it over to 2.5 it might be nice to flag those bad frames and have Avisynth filters optionally deal with it. All current filters could ignore that flag, so it would just be an option. But in a special filter it would be possible to turn all garbage frames black or (better) replace all garbage frames with the last known good frame.

- Tom

[Si]

@IanB
Your diagrams all seem to show your chroma sampling horizontally between two Y samples but all the stuff I can find on MPEG-2 4:2:0 (cause that's what the main use of the YV12 is for it seems) shows the samples as being horizontally co-sited with the 1st Y sample.



regards
Simon

[IanB]

Originally posted by trbarry
For 3.0 I'd like to propose one more flag attribute for each frame. This would say the frame contains unknown garbage.Sort of like the infamous .AVI Drop Frame attribute we currently provide. NOT!

And while we are into frame based attributes, it might be nice to pass Keyframe knowledge thru the pipe to the output .AVI synthisizer (pseudo-codec) so that keyframe seeking in applications like VirtualDub, et el is actually useful and fast.

A simple flag probably would not cut it when filters with temporal dispersion are used. i.e. TemporalSoften(radius=1) seeking to a keyframe thru this filter accesses the keyframe (K), the frame after the keyframe (K+1) and the frame before the keyframe (K-1) opps maximum really long seek delay. But some fancy OO crawl-back (or is that claw-back :p) method counting max cache hint width could certainly work extremely well, i.e seeking to K+1 and accessing K, K+1, K+2 would be really fast.But in a special filter it would be possible to turn all garbage frames black or (better) replace all garbage frames with the last known good frame.

- Tom Or a Temporal Interpolated frame from the last known good frame and the next known good frame :D

IanB

[IanB]

Originally posted by siwalters
@IanB
Your diagrams all seem to show your chroma sampling horizontally between two Y samples but all the stuff I can find on MPEG-2 4:2:0 (cause that's what the main use of the YV12 is for it seems) shows the samples as being horizontally co-sited with the 1st Y sample.

regards
Simon Damn! So I have (to wrapped up in the vertical problems), I'll fix the posts.

The art I grabbed is actually correct for MPEG-1 (which knows squat about interlaced) and I just embelished it without thinking. All this YV12 chroma subsampling and siteing is making my head spin, and sh0dan write MMX and SSE2 assembler to implement it as well. I don't know how he does it and stays sane.:cool:

Simon, I dare you to ask for a PAL DV chroma resiteing filter:D

Thanks

IanB

[Si]

I dare you to ask for a PAL DV chroma resiteing filter

I'll just write a plugin myself ;)

The only thing is, do any of the source filters actually provide YV12 from a DV source or do thay all already upsample to YUY2 (and do they get it right) ?

regards
Simon

[sh0dan]

The converters stick to the MPEG2 scheme - the only thing changed is the way chroma is weighed when interpolating interlaced YV12 -> YUY2. The horizontal alignment of the chroma isn't touched, so it realigns perfectly according to the diagrams on the page you linked, and how your ASCII art now shows. I guess I must have missed your MPEG1 suggestion :rolleyes:

(edit: spelling)

[sh0dan]

Originally posted by siwalters
The only thing is, do any of the source filters actually provide YV12 from a DV source or do thay all already upsample to YUY2 (and do they get it right) ?


All DV filters I heard of converts to YUY2 - some with broken chroma upsampling, but even though they should be able to deliver an YV12 signal on PAL sources they don't. Guess they "go safe".

[trbarry]

I sometimes capture Buffy from UPN which has softly upconverted it to 1080i. Since there is not that much detail anyway I've been doing:

SeparateFields()
SelectOdd()

and then resizing the result. And even with Avisynth 2.5 this seems to work okay, fast, and I don't really notice any problems.

But from rereading all the above posts (including my own) it occurs to me that I am probably introducing a fractional pixel vertical chroma delay here because of the interlaced YV12 format.

With all the changes here is there currently a fast simple way to do this properly? In a couple hours I could probably create a fast mmx YV12TopFields() filter to use for now. Should I do this or isn't it needed?

- Tom

edit: I never thought or cared about it much before. Is the first field in Avisynth 0 or 1. That is, does SelectOdd take the first field?

[Si]

@trbarry
from avisynth.org
Since frames are numbered starting from zero, SelectEven actually selects the first, third, fifth,... frames by human counting conventions

So I think that makes SelectOdd do the 2nd,4th,6th etc.

I think we need a syntax change to SelectEveryOtherFrameStartingFromThe2ndOne ;)

regards
Simon

[IanB]

@sh0danthe only thing changed is the way chroma is weighed when interpolating interlaced YV12 -> YUY2.Shouldn't the weighting be applied in the YUY2 -> YV12 direction?:confused:

I did stumble across some bi-cubic interpolation algae for YV12 -> YUY2 in MSDN somewhere, it looked expensive but might be worth it in a ...yv12To...(HiQmode=true). It was somewhat of the form :-

uv(x,2y)=-k1*UV(x,y-2)+k2*UV(x,y-1)+k3*UV(x,y)-k4*UV(x,y+1)
uv(x,2y+1)=-k5*UV(x,y-1)+k6*UV(x,y)+k7*UV(x,y+1)-k8*UV(x,y+2)

Perhaps poaching the code from BicubicResize or LanczosResize could be the go.;)

And how are you doing the 75/25 top field - 25/75 bottom field flip,:confused:I guess it must be something to do with the
for(..;..;i++) { switch (i) { case 2:_srcY-=src_pitch;
but I can't quite find it or have I jumped the gun again [convert_yv12.cpp Rev 1.5 Fri Feb 28 10:09:33 2003 UTC]

@trbarry

I also can never remember:confused:so I chuck in a ShowFrameNumber() whenever I need to know

SeparateFields()
ShowFrameNumber()
SelectOdd()

Returns Frames 1,3,5,7,...:cool:

IanB

[trbarry]

Since frames are numbered starting from zero, SelectEven actually selects the first, third, fifth,... frames by human counting conventions

Right, I knew that. ;) (well, I should have RTFD)

Anyway, I just went ahead and made the darn filter. See:

www.trbarry.com/Readme_YV12InterlacedReduceBy2.txt and
www.trbarry.com/YV12InterlacedReduceBy2.zip (source and dll)

This can go in the filter pack if anyone thinks it would help for the moment.

It does a horizontal and vertical "reduce by 2" by taking only the top fields, doing the needed chroma adjustment, and also horizontally reducing by 2 with no filtering. There are no parms.

The same dll also contains a YV12InterlacedSelectTopFields() function. This does what the name suggests, also doing the chroma adjustment but leaving the width alone. This makes it sort of a fast (but hopefully proper) version of SeperateFields().SelectEven().

Since the source is there, and very small, I guess it can show how I think the chroma adjustment should be, though I was just going by the pictures in Video Essentials.

- Tom

[sh0dan]

And how are you doing the 75/25 top field - 25/75 bottom field flip,I guess it must be something to do with the
for(..;..;i++) { switch (i) { case 2:_srcY-=src_pitch;
but I can't quite find it or have I jumped the gun again [convert_yv12.cpp Rev 1.5 Fri Feb 28 10:09:33 2003 UTC]


It is not implemented in C - only assembler.

The switch stuff is for the top and bottom lines (8 lines total in interlaced mode, 4 in progressive). These lines are not interpolated, so chroma is simply copied here.

The interpolation is done by "pavgb mmx,mmy// interpolate chroma U ". It actually translates to:

chroma = (chroma1 + chroma2 + 1)/2
chroma = (chroma + choma1 +1 )/2

This is when chroma1 must be represented 75%. The reason we use two averages and not accumulate and divide by 4 is performancewise, since this operation can be done without unpacking bytes to words (which the slower MMX-routine has to).

I personally think bicubic chroma scaling is highly overkill. But if someone implements it I will of course put it in. We already have much more precise conversion than most filters (XviD does no chroma interpolation at all), so if there are no obvoius bug, I will not use too much more time on this (of course we need an MMX version of the YUY2->YV12).

[neuron2]

Just adding this thread link:

http://forum.doom9.org/showthread.php?s=&postid=283094#post283094

...because it demonstrates that cropping for YV12 must observe the interlaced/progressive chroma sampling frame flag to perform proper enforcement of crop values.

[Bidoche]

I finally got those video properties done in the 3.0 code.
Here are what I made :

Clips are now marked (through VideoInfo) as either frames or fields clips.
Filters are expected to enforce this distinction the way you can imagine. (essentially no mixing field/frame)

Frames have self knowledge of ColorSpace, width, height and type.
Type can be FIELD_TOP, FIELD_BOTTOM, PROGRESSIVE, INTERLACED_TTF, INTERLACED_BFF, UNKNOWN.
The first two are for use in fields clips, the four others in frames clips.

When creating a videoframe from a videoinfo, if the clip is a field one, frame type is defaulted to top. If it is a frame clip and colospace/dimension allow interlacing frame type becomes UNKNOWN, and otherwise PROGRESSIVE.
UNKOWN type is not supposed to be kept, filters are expected to change it into one of the 3 others.


As for frame properties, after some brainstorming, thinking about really complex things, finally remenbering the KISS rule...
I came to this proposal :

Distinguish 3 types of properties/flags :
- state properties : their value depends only of the videoframe, and will always be the same whenever, wherever it is needed. Then it makes sense to allow their attachment to const VideoFrame as they a re not a modification but just an additional info.
These properties are discarded when the frame is accessed for modification.
An example of these could be decomb frame comb status.

- forward porperties : properties we want passed through the filter chain for some reason. It suppose a method to make new videoframe inherit them correctly, and more importantly a (polymophic) method to deal with collisions (what to do when more than one ancestors with the property). They can be added normally to a PVideoFrame, but change a CPVideoFrame to a clone with the propery added (trick to keep state properties).
Of course they are not discarded by frame modifications.
The only example I could come up with is to time filters chain :
a filter mark frames with a property with the processing time as value and an addition collision method, and an other make the difference between the processing time and the value passed through.

- other properties : those who are none of the above, are discarded by modifications and are attached like forward ones.