Avisynth video properties (was "Help me solve this YV12 mystery")

[trbarry]

Donald -

You once posted a link for recommended MPEG2 DVD mastering or some such using all these flags. Do you still have that?

- Tom

[scmccarthy]

mpeg2dec could easily be set to save more info from the mpeg-2 header. Would that help, or is the header info unreliable?

Stephen

[Guest]

@trbarry

This one?

http://www.dumbterm.net/graphics/dvd/ (scroll down)

@sh0dan

For 2.5, I agree that field-based/frame-based should be removed because it is not a valid per-clip property and it causes more problems than it solves. Filters that need an interlaced flag would take it as a parameter, e.g., ConvertToYV12(interlaced=true) would use interlaced chroma down sampling.

Let's see what others think about it.

[Guest]

Originally posted by scmccarthy
mpeg2dec could easily be set to save more info from the mpeg-2 header. Would that help, or is the header info unreliable?
Did you miss this bit above?

"A properly written mpeg2dec can fully populate these. The information can be very useful to many filters."

and this bit:

"I'm willing to change one or both of the MPEG2DEC's if we need more info from them." [Tom]

[scmccarthy]

In case you guys are wondering what I was talking about, here's a quote from 'Writing Avisynth plugins':External plugins do not link with avisynth.dll, so they can't directly access functions that are defined in the main Avisynth source code. Therefore, every important function in avisynth.h is either defined inline or declared as virtual. The virtual functions act as callbacks for external DLLs.
Stephen

And the following quote:An Avisynth filter is simply a C++ class implementing the IClip interface. IClip has four pure virtual methods: GetVideoInfo, GetFrame, GetParity, and GetAudio. And now there's a fifth virtual method called SetCacheHints(). A filter whether external or internal does not call any of these directly. They only optionally redefine them. So GetParity is not like GetPitch(), it does not have to be called from within GetFrame().

[Bidoche]

@sh0dan

For per-frame properties who can actually change during a clip, an adaptive behavior is necessary in order to get them right.
Of course we can still provide fallback parameters to force algorithm process.
But I think keeping the smart behavior is good

[sh0dan]

@Bidoche: Could you elaborate?

[trbarry]

Thinking about it a bit more it seems that if you have YV12 and don't change that somewhere in your script then it will eventually be passed to code like Xvid that will assume it is progressive, unless instructed otherwise. We might as well do the same thing and for 2.5 just document that progressive color conversion will be assumed.

An explicit override could still be specified on a ConvertToYUY2 statement if needed.

But there is one more place it should be specified. It seems silly after all the work you've done here but I think I'd like to combine MPEG2DEC2 and MPEG2DEC3 such that the ability to return YUY2 is still optional. The reason is of course that any YUY2 conversion done there still has access to all the MPEG2 flags and, AFAIK, can do it correctly for the folks that still want YUY2 or RGB for some reason.

Note that I'm not backing off my new found love for YV12. I think that is still much better for MPEG2 sourced material. But that assumes you are going to be deinterlacing it somehow in a proper fashion before trying to convert it to something else.

- Tom

[Bidoche]

@sh0dan

Imagine you have a clip with progessive/interlaced frames mixed.
If an operation should be performed differently on each type, an adpative behavior is really useful.
Choosing the algorithm will always get half of your clip wrong.

[ErMaC]

Just thought I'd pipe in here, since I noticed Donald listed Framerate as a per-clip property.

Must AVISynth still be restricted by pseudo-AVI compatability, or could we actually create variable framerate AVS files in the future? That would make dealing with hybrid material so much easier (i.e. Donald's ST:TNG Hybrid Challenge).
Just a quick thought/suggestion, but I know it would then break compatability with the AVI/VFW interface, but DShow can handle multiple framerates if I recall.
And for the standard VFW interface you could always fall back on drop-frame-insertion to fix playback. I.e. if a clip has 24fps and 30fps parts, report it back as 60fps and insert a dropped frame inbetween each 30fps frame and alternating 1 and 2 dropped frames between the 24fps footage (I actually have AVI files that do this).

[scmccarthy]

@ermac

Can any DirectShow player play such an avi file?

Stephen

[ErMaC]

So far I have had no trouble playing it in anything. The trouble is encoding it :P

I first ran into these kinds of files when I was doing Fansub encoding about a year and a half ago. Nowadays, to look cool, many shows have opening sequences which are in 29.97fps progressive, while the rest of the show is in 23.976fps telecined to 29.97. What raw encoders used to do was just decimate the opening sequence and keep it at 24 like the rest of the episode when they IVTC'ed their TV captures. But then someone came out with a tool (I don't have it, I just know it exists) that allowed the insertion of Drop frames to sync them up at 59.94fps by inserting 1 drop frame every other frame in the 30fps portion and alternating 1 and 2 frames in the 23.976fps portion.
The problem was when I would open them up in VirtualDub, and would try to recompress them, it would recompress all the frames (all 59.94 per second of them) and thus the resulting file would be unplayable.
I used AVISynth's SelectEvery to get around this problem, and fed the sections and encoded them separately, but I instead just tacked on the opening and ending sequences from previous episodes so I didn't have to decimate them myself, which is what I would've had to do anyways since I didn't have the tools to re-add the drop frames.

This was back in the days when raw encoders were using MP42, MP43, or DIV3 (occaisionally). Their DShow playback filters dealt with the files fine - you would go to Media Player 6.4's Statistics and you would see Frame Rate: 59.94 and Actual Rate: 29.97 or 23.976 depending on the portion. I thought it was pretty ingenious - it's just a pain to deal with.

Really the drop-frame insertion would only be useful for DirectShow playback, doing that with the VFW interface would be ugly unless VDubMod could be told "If there's a drop frame reported by the AVS file, insert a drop frame in the file being encoded" which I suppose wouldn't be too hard to do, we'd just have to ask them to implement said feature.

Now I will completely acknowledge here that we are totally bastardizing the AVI standard and making it do things it really really was never meant to do - but you can say the same thing about Bi-directional Predicate Frames, too :)

I can post a short sample of one of the Raw files that does this if you guys want to take a look at it.

[sh0dan]

Can't really see any good way of putting in 100% variable bitrate. It is not a part of VFW-API - it has one framerate per AVI file. I don't see using "output_fps = fps1*fps2" with an extremely large amount of duplicate frames as a solution.
Could we move this OT discussion another place?

Back on track:

I have created an ISSE/MMX interlaced YV12 -> YUY2 converter from scratch, and put it in CVS. I hope to be able to do an interlaced YUY2 -> YV12 soon also.

It is currently implemented as ConvertToYUY2(interlaced=true). This may be changed!

My current priority list on needed conversions are:

Interlaced YUY2 -> YV12.
Cleanup of convert.cpp I just realized what a mess it is!
Rewrite of non-interlaced YV12 <-> YUY2.
Interlaced YV12 <-> RGB. (This will probably be implemented using a combination of existing filters - this is NOT easy code!)
Hybrid Frametype detector. As proposed a routine that is capable of detecting interlacing in hybrid material (telecined material for instance).

[Stabmaster-Arson]

I pray to our Lord and savior, Jesus, that variable framerate does not become popular. Variable framerate is BAD mmkay.

Its an encoders/converters worst nightmare.

[sh0dan]

ok - these two points are done:

Rewrite of non-interlaced YV12 <-> YUY2.
Cleanup of convert.cpp I just realized what a mess it is!


Or at least - the progressive YV12 -> YUY2 rewrite is finished and the cleanup has begun.

I realized that the XviD routines doesn't interpolate chroma when doing YV12 -> YUY2 - this is also done now. The speed should be about the same as the XviD routines (a little give and take). Also the speed for non-mod8 resolutions should be up.

I'll release a binary ASAP.

[trbarry]

I can't guarantee the YV12->YUY2 interlaced conversion functions in MPEG2DEC2 are correct but I believe they are. But if so then comparing the results of MPEG2DEC2 vs (non-PP) MPEG2DEC3 converted to YUY2 should now give identical results for pure interlaced source.

But I haven't performed this test yet. ;)

- Tom

[IanB]

Just two quotes from the many to stress this point.....
Originally posted by neuron2
Now I'm convinced that even field-based/frame-based should be per-frame....Yes, if we continue down the current evil path of hijacking the IsFieldBased() property
to distinguish Progressive/Interlaced status.

As originally implemented it is most definitly a clip attribute, it simply meant the clip had
been SeparateFields()'ed.

Originally posted by sh0dan
IMO ConvertToYUY2(interlaced=true) is more logical than
AssumeFieldBased().ConvertToYUY2(). It took me some time
getting used to this, but now it actually makes more sense to me....The first is more logical because the second should never have seen the light of day.
AssumeFieldBased() should NEVER have been hijacked to mean Interlaced, it originally
meant the clip has been SeparateFields()'ed and it should have stayed that way.

:mad: Let me try and get this point across 1 more time. :mad:

IsFieldBased() as originally defined simply meant the clip had been thru SeparateFields()
It has been HIJACKED to try and mean the clip is Interlaced and this great cock-up has
caused all the problems and confussion in this current thread and others.

This is a programming attribute collision!! Where one piece of data means two
different and distinct things. It is a software design BUG!

Really simply solution
Stop abusing IsFieldBased() to determine Interlaced
Revert all the code abusing IsFieldBased() back to the original working state.
Leave SeperateFields()/Weave() alone until 3.0
Decide if Progressive/Interlaced will be a command argument, a frame attribute
or a clip attribute.
Implement it that way everywhere! Don't even think about using IsFieldBased().

If the chosen implementation is deamed to involve IsFieldBased() then reference
it IN ONLY ONE PLACE, in the IsInterlaced() implementation. Don't mix up
these two concepts again.

IanB

P.S. If anything the frames of clip.AssumeFieldBased() are certainly NOT interlaced,
they are the individual fields of a possibly interlaced, possible progressive clip and
individual fields can never be interlaced (a 2nd time)!

[sh0dan]

@IanB: We DO know. And we are currently INVESTIGATING what method is worth pursuing - <g> relax, man!

IMO the original idea of adding i VideoInfo property called "Fieldbased", and using it for very filter specific purposes isn't a very clever design, but when there are only one developer, these things do easily happend.

Using an "is_field_separated" property would be much more nice, if it is a good idea. IMO the original property is NOT a good idea to preserve, since it creates an unpredictable behaviour.

IMO SeparateFields/Weave should always behave the same, no matter the Video!

You could have been there shouting at me, when I asked if anybody knew how the Fieldbased/Parity stuff worked MONTHS ago! You are NOT being very constructive! :mad:

[sh0dan]

IMO SeparateFields/Weave should always behave the same, no matter the Video!


Let me elaborate:

I'm my opinion the primary power of AviSynth is total user control - you have complete control over every aspect of your video. When AviSynth makes decisions for me, I get annoyed, because I always want a filter to produce the same results.

So when AviSynth makes decisions on how to treat a video signal based on previous filters this is kinda annoying to me. This is why I want to get rid of this situation. The point of the new conversions, and the suggestions to Separatefields/Weave is to make them behave the same no matter what material they are fed with.

[Si]

I'm my opinion the primary power of AviSynth is total user control - you have complete control over every aspect of your video. When AviSynth makes decisions for me, I get annoyed, because I always want a filter to produce the same results.

Can I add in my two pennyworth.

The syntax ConvertToYUY2(interlaced=true) is excellent and the way to go in my opinion.

Therefore SeparateFields and Weave should have optional parameters to force their behaviour instead of relying on the parity flag.

regards
Simon
PS I don't think IanB is being aggressive just assertive :)

[IanB]

Originally posted by sh0dan
@IanB: We DO know. And we are currently INVESTIGATING what method is worth
pursuing - <g> relax, man!Most sorry for Yelling I've been dancing around this issue for many days in many
forums and I haven't been getting acknowledgment I was on the right track or
argument that I was wrong and completely off tangent. ;)

IMO the original idea of adding i VideoInfo property called "Fieldbased", and using it
for very filter specific purposes isn't a very clever design, but when there are only
one developer, these things do easily happend.

Using an "is_field_separated" property would be much more nice, if it is a good idea.
IMO the original property is NOT a good idea to preserve, since it creates an
unpredictable behaviour. I totally agree the Naming used in this part (and others) is evil :devil: and "Parity" as a
name, what has that got to do with field order.

IMO SeparateFields/Weave should always behave the same, no matter the Video!
SeparateFields() no argument here.

Weave() and Bob() have a little problem in needing Field Order information i.e. "Parity".
This gets very complicated when Trim(), DeleteFrame() or DuplicateFrame() get involved.

Think how you would handle deleting a single field in a clip, now the clip has 2 top or
bottom fields adjacent. The current very obtuse code does handle this gracefully.

Hence my suggestion 2. leave field.cpp alone until 3.0

You could have been there shouting at me, when I asked if anybody knew how
the Fieldbased/Parity stuff worked MONTHS ago! You are NOT being very
constructive! :mad: I only joined this forum in January, and I did make an attempt at providing a leg up on
how Parity() abuses OO method overlay in "Dilema - Crop restriction - Fieldbased clips (http://forum.doom9.org/showthread.php?s=&postid=259748#post259748)"

Anyway how can I help? Revue code, suggest algorithms, Consolidate my current
ravings into one post, just ask.

IanB

[Guest]

Originally posted by IanB
I've been dancing around this issue for many days in many
forums and I haven't been getting acknowledgment I was on the right track or
argument that I was wrong and completely off tangent. ;) How can you say that? You sparked a big discussion that is actively going on right now. But your solution doesn't prevail because you use the biggest and boldest font. One solution presented is to ditch the whole idea of field-based/frame-based. If that were done it could not be confused with interlaced/progressive. Why don't you comment on that?

Weave() and Bob() have a little problem in needing Field Order information i.e. "Parity".
This gets very complicated when Trim(), DeleteFrame() or DuplicateFrame() get involved.I don't think so. You'd know the tff/bff status of each frame, so where is the problem?

Think how you would handle deleting a single field in a clip, now the clip has 2 top or
bottom fields adjacent. The current very obtuse code does handle this gracefully. I'm thinking, what is the problem? Deleting a field is a reasonable thing to do in some circumstances, and can already be done with Avisynth. Anyway, our concern is not with parity for 2.5, but whether we really need the field-based/frame-based property.

EDIT: I see now there is ambiguity about what sh0dan is proposing regarding parity (tff/bff handling). Does he propose to remove its influence from Weave() and others, or does he just propose to change its syntax? shodan, can you please elaborate? On the field-based/frame-based property, no one has yet argued to keep it.

Anyway how can I help? We're seriously considering ditching the field-based/frame-based property for the reasons given in this thread. Please comment on that.

[sh0dan]

I'll do a suggestion for an implementation, so we can have mode code and less talk. :)

[scmccarthy]

I had to preceed Bob() with SwapFields().

My comment is that taking a hard look at how mpeg2dec works might be the only way to fix these problems. AviSynth and mpeg2dec have to come to an agreement.

mpeg2dec must work the same way as DV captured material. You see? We want Avisynth to work the same betweem DVDs and Digital Video.

Stephen

[trbarry]

But MPEG2DECx also gets to look at the progressive frame flag (whatever it's called) which is not currently being passed along.

On the good side, I've even got P4 optimized SSE2 code for the MPEG2DEC color conversions, though it is not included yet.

- Tom

[scmccarthy]

Does it make sense to suggest that the GOP in mpeg2dec needs to be stuffed with more info? Or am I getting the terminology wrong?

Stephen

[trbarry]

"GOP" to me is a Group Of Pictures, a I-frame and it's following P&B frames. I don't think any control block is passed to Avisynth for each of these so I'm not sure what you are asking.

- Tom

[IanB]

So as to keep all the relevant discussion in one place the following
is a transcript from the Developer Forum (http://www.avisynth.org/forum/viewtopic.php?p=94).

By: IanB Posted: Fri, 21 Feb 2003 3:37 pm
Post subject: Frame/Field Based vs Interlaced/Progressive

Dear Richard Berg, TRBarry, Hakko504, Neuron2, Sh0dan, et el

I have been trying to gently provoke some sane discussion about the
contorted use of the IsFieldBased() flag with respect to the new YV12
implementation in avisynth 2.5 on the Doom9/Avisynth forum.

Being somewhat unsuccessful I thought I might try a more direct
approach to some of the key players here in this more quiet forum.

My research into the original Avisynth source reveals that FrameBased
and FieldBased concept derive from SeparateFields() and Weave()
[field.cpp,h] implying the following definitions:-

FrameBased :-
Video samples are whole frames, either progressive or interlaced.

FieldBased :-
Video samples are individual fields. Alternate samples are top, bottom
field. Usually the result of a SeparateFields() operation.

The new YV12 codebase has subsumed these definition and is using
FieldBased to imply Interlaced! Which is at odds with the original
coding intent and is causing subtle changes in the beviour of
unrelated verbs like Crop().

What is apparent is this needs to be clearly and concisely defined
and rigidly adheared to in the core Avisynth code.

I was proposing a new verb AssumeProgressive() to match with the
current AssumeTFF() and AssumeBFF() both which imply a
FrameBased interlaced source or indicate the parity with FieldBased
(i.e field separated) samples.

For background the following thread is a current example of the
difficulty http://forum.doom9.org/showthread.php?s=&postid=263776

And the following post my original discussion of the problem
http://forum.doom9.org/showthread.php?s=&threadid=45550#post259273

Regards
IanB


By: Guest Posted: Fri, 21 Feb 2003 6:50 pm

I wrote a "note to self" when I looked at GetParity.

http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/avisynth2/avisynth/field.h?rev=1.5&only_with_tag=MAIN&sortby=date&content-type=text/vnd.viewcvs-markup

IsFieldBased is unfortunately used in the complete opposite way as it
would be expected.

AssumeProgressive() actually makes nice enough sense.

But the field-based code needs reworking.


By: sh0dan Posted: Fri, 21 Feb 2003 6:58 pm

Oh "Guest appear cortesy of sh0dan Records"

Quote:

> AssumeFrameBased throws away the existing information and assumes
> that the clip is frame-based, with the bottom (even) field dominant
> in each frame. (This happens to be what the source filters guess.)
> If you want the top field dominant, use ComplementParity afterwards.
>
> AssumeFieldBased throws away the existing information and assumes
> that the clip is field-based, with the even-numbered fields being
> bottom fields and the odd-numbered fields being top fields. If you
> want it the other way around, use ComplementParity afterwards.

This is a stretch in my opinion.

Framebased - makes AviSynth think that both fields are present in the
same frame - completely backwards to me.

AssumeFieldBased makes AviSynth assume that fields have been separated.

The above distinguishment doesn't make any sense to me. For now, I have
flipped (most) functionality around, so that:

Fieldbased: Video is asummed to be interlaced.
Framebased: Video is assumed to be progressive.

We could add a separate property for video that has been split, but
IMO it would be much better if the SeparateFields and Weave/DoubleWeave
didn't act on this property, but always performed the same action on
the video.

Does this make any sense?
_________________
Regards sh0dan // Klaus Post.


By: IanB Posted: Fri, 21 Feb 2003 9:47 pm

Sh0dan wrote:

> AssumeFrameBased() throws away the existing information and assumes
> that the clip is frame-based, with the bottom (even) field dominant
> in each frame. (This happens to be what the source filters guess.)
> If you want the top field dominant, use ComplementParity afterwards.
>
> AssumeFieldBased() throws away the existing information and assumes
> that the clip is field-based, with the even-numbered fields being
> bottom fields and the odd-numbered fields being top fields. If you
> want it the other way around, use ComplementParity afterwards.

Apart for the effects of the new AssumeTFF() and AssumeBFF() verbs.

> This is a stretch in my opinion.
>
> Framebased - makes AviSynth think that both fields are present in
> the same frame - completely backwards to me.
>
> AssumeFieldBased makes AviSynth assume that fields have been separated.

This is a very eloquent and succinct explanation of how we both appear
to understand how the original Avisynth works.

> The above distinguishment doesn't make any sense to me. For now,
> I have flipped (most) functionality around, so that:

> Fieldbased: Video is asummed to be interlaced.
> Framebased: Video is assumed to be progressive.

Although I agree with your sentiment, I have to say it is a little bit
unfortunate that you are redefining script behaviour with these changes.
Old Avisynth users know and expect the current behaviour and redefining
it just ends up confusing everybody and losing any hope of backwards
script compatibility.

> We could add a separate property for video that has been split, but IMO
> it would be much better if the SeparateFields and Weave/DoubleWeave
> didn't act on this property, but always performed the same action on
> the video.
>
> Does this make any sense?
> _________________
> Regards sh0dan // Klaus Post.

I would urge you in the most strongest terms to leave the current
definitions of AssumeFrameBased() and AssumeFieldBased() and their
interaction with SeparatedFields() and Weave() alone.

For better or worse that is the way they work and the way peoples
existing scripts expect them to work.

In order to move forward I would propose that the old verbs
AssumeFrameBased() and AssumeFieldBased() simply be declared
depreciated and brand new script verbs be declared that accuratly
describe the state of play. i.e

Verbs
=====
AssumeCompleteFrames() depreciates AssumeFrameBased()
AssumeSeparatedFields() depreciates AssumeFieldBased()

AssumeUnknownPI() declares Progressive/Interlaced unknown state
AssumeTFF() declares Interlaced source, top field dominant
AssumeBFF() declares Interlaced source, bottom field dominant
AssumeProgressive() declares Progressive source

Properties
==========
IsCompleteFrames() depreciates IsFrameBased()
IsSeparatedFields() depreciates IsFieldBased()

IsUnknownPI() Test for Progressive/Interlaced state unknown
IsInterlaced() Test for Interlaced source
IsTFF() depreciates/extends Getparity()
IsBFF() depreciates/extends Getparity()
IsProgressive() Test for Progressive source

For safety clips with undefined Progressive/Interlaced state should
have all the even height protection of interlaced clips and doing
things that need the P/I status like YV12 -> YUY2 or RGB conversion
throw a script exception.

For completness internal methods like isFieldbased() should only
be used in field.cpp/h and be replaced with new isProgressive()
and isInterlaced() methods in the new YV12 code that need this
information.

A search of a good Thesaurus may yield better terms than I have
used but I hope the idea and sentiment is clear.

IanB


By: neuron2 Posted: Sat, 22 Feb 2003 3:26 am

What a contorted mess we are creating. And why!? I am going to
propose a simple solution.

Remove all notion of parity! For filters that want to
disassemble/reassemble frames, just give them a parameter to
specify the order of splitting/reassembling. Leave it totally
up to the user to determine whether his actions make sense for
the source material and task he is trying to accomplish.

For example, instead of SeparateFields(), which makes an unnecessary
assumption that the source is composed of fields, just have:

Separate(first=)

You could use it on a progressive clip (indeed on any clip) if
you like. Why not? For example this would be a reduce by two
vertically for a progressive frame:

Separate(first=even).SelectEven()

It sounds radical but why do we really need this whole idea of
parity? It doesn't work correctly anyway for all the known reasons.
And there seems no reason for having the frame-based/field-based
distinction either (at the scripting level, per-frame flags such
as progressive/interlaced carried in the clip itself is another
matter entirely). Think of how this would simplify the bizarre
code we have today and make everything consistent for all kinds
of source material. We are creating all these issues for ourselves
for nothing, IMHO.

IanB proposed these properties:

IsUnknownPI() Test for Progressive/Interlaced state unknown
IsInterlaced() Test for Interlaced source
IsTFF() depreciates/extends Getparity()
IsBFF() depreciates/extends Getparity()
IsProgressive() Test for Progressive source

Every one of them can change during a single clip! That is the
bottom line. All of this per-clip flag business is useless.

I humbly wait to be refuted. I am willing to table a complete
and consistent scheme if people are willing to consider it.
_________________
Donald Graft
Filters for Avisynth and VirtualDub
http://shelob.mordor.net/dgraft/


By: sh0dan Posted: Sat, 22 Feb 2003 6:19 am

Followup:

http://forum.doom9.org/showthread.php?s=&postid=266410#post266410
_________________
Regards sh0dan // Klaus Post.

[trbarry]

Well, we've gone round and round and mostly proved that most of us (including me) had it backwards on the first cut. But I still lean towards keeping it compatible with the way it used to be and just adding some extra flags later.

I'm not sure. Did anything get decided?

- Tom

[IanB]

Okay, I have had yet another read thru the source. The following is
a list by functional entity that currently uses the .IsFieldBased()
method and what I have found about it with respect to nuking the Field
Separated Attribute.

SeparateFields()
In SeparateFields() the use of IsFieldBased() is a protectionist
restriction to prevent the lose of top field/bottom field state
information from an already field separated clip by the user.

--- It can probably be derestricted right now without problem.

Weave()
Weave() is implemented internally as a DoubleWeave().SelectEven()
hence the apparent transparency (see below) on IsFrameBased() clips.

--- Refer to DoubleWeave()

DoubleWeave()
In DoubleWeave() the use of IsFieldBased() causes 2 different and
distinct behaviours. (This is what sh0dan so rightly loaths)

For IsFieldBased() clips it produces a unique weaving of each
field with both adjacent fields. It uses the internal primitive
DoubleWeaveFields() i.e.

input 0b 1t 2b 3t 4b 5t
output 0b1t 2b1t 2b3t 4b3t 4b5t 4b5t

Note :- the last 2 frames are 4b5t (Boundary condition in GetFrame())

For IsFrameBased() clips it does an implicit SeparateFields() and
then produces a unique weaving of each field with both adjacent
fields only with much greater efficiency. It uses the internal
primitive DoubleWeaveFrames() i.e.

input 0b1t 2b3t 4b5t
output 0b1t 2b1t 2b3t 4b3t 4b5t 4b5t

Having the following piece of script do a 1 field phase shift
with high internal efficient I find a very great boon.

. DoubleWeave().SelectOdd()

--- Removing the Field Separated property will break legacy script
support for DoubleWeave().

Bob()
In Bob() the use of IsFieldBased() causes 2 different and distinct
behaviours. (This is what sh0dan so rightly loaths)

For IsFrameBased() clips it adds a SeparateFields() to the front of
the GetFrame() chain and proceeds as per the IsFieldBased() case.

For IsFieldBased() clips it produces a double rate, IsFrameBased(),
Vertically resized clip with the top fields moved up 0.5 lines and
the bottom fields moved down 0.5 lines.

--- Removing the Field Separated property will break legacy script
support for Bob().


GetParity()

This basic GetParity() method chain holds the Top field/Bottom field
information for a clip.

inline bool __stdcall GetParity(int n)
{ return parity ^ (vi.IsFieldBased() && (n & 1)); }

(parity is a private bool)

Each child that does something to effect that information overlays
this method with a new piece of code that possible chains to it's
parents original method. i.e.

For ComplementParity() the following code is overlayed

inline bool __stdcall GetParity(int n)
{ return !child->GetParity(n); }

After unwinding the murkey mire of this "parity" code I have to
respect the ease with which it tracks top field/bottom field state
thru an AviSynth session.

---Removing the Field Separated property will require the design
of a new top field/bottom field state tracking algorithm.


ConvertFPS()

The algorithm used in ConvertFPS() doesn't appear quite right
when used with field separated content. It needs some help from
Bob(). It hacks with the GetParity() method when IsFieldBased()

---It currently does not make correct use of the information


ShowFrameNumber()

For IsFieldBased() clips the odd numbers are offset to the left
of the display area, this is a useful diagnostic feature.

---Cosmetic effect only


The rest (new YV12 code)

This code is basicly treating IsFieldBased() to mean IsInterlaced()
and enforcing 2**n height restrictions.

---Should be using Progressive/Interlaced determination


YV12 progressive/interlaced chroma sampling

Progressive YV12 is easy to understand each 2x2 Luminance group
has single corresponding chroma samples. i.e.

Progressive Frame
=================
Y_Y.Y_Y Line 1
C__.C__ Lines (0.5)*1 & (0.5)*2
Y_Y.Y_Y Line 2
___.___
Y_Y.Y_Y Line 3
C__.C__ Lines (0.5)*3 & (0.5)*4
Y_Y.Y_Y Line 4

Interlaced YV12 has a bit of a twist, the samples of each field
go together and just to add some spice there is an alternating
1 line vertical spacial displacement of the chroma data.

Interlaced Frame
================
Y_Y.Y_Y Line 1t
C__.C__ Lines (0.75)*1t + (0.25)*3t
Y_Y.Y_Y Line 2b
___.___
Y_Y.Y_Y Line 3t
C__.C__ Lines (0.25)*2b + (0.75)*4b
Y_Y.Y_Y Line 4b

In one of my earlier posts I suggested Field Separated video
samples could be considered as Progressive, this is very clearly
wrong. Field Seperated YV12 Interlaced samples are a completely
different animal to Vanilla Progressive samples and (forced) Field
Separated YV12 Progressive are just a really viscious little animal.

Top Seperated Field (Note (0.5) line up weighting)
===============
Y_Y.Y_Y Line 1t
C__.C__ Lines (0.75)*1t + (0.25)*3t
Y_Y.Y_Y Line 3t
___.___
Y_Y.Y_Y Line 5t
C__.C__ Lines (0.75)*5t + (0.25)*7t
Y_Y.Y_Y Line 7t

Bottom Seperated Field (Note (0.5) line down weighting)
=================
Y_Y.Y_Y Line 2b
C__.C__ Lines (0.25)*2b + (0.75)*4b
Y_Y.Y_Y Line 4b
___.___
Y_Y.Y_Y Line 6b
C__.C__ Lines (0.25)*6b + (0.75)*8b
Y_Y.Y_Y Line 8b


Top Progressive Separated Frame (Note the chroma samples are (0.5) from another frame)
=========================
Y_Y.Y_Y Line 1
C__.C__ Lines (0.5)*1 & (0.5)*2
Y_Y.Y_Y Line 3
___.___
Y_Y.Y_Y Line 5
C__.C__ Lines (0.5)*5 & (0.5)*6
Y_Y.Y_Y Line 7

Bottom Progressive Separated Frame
===========================
Y_Y.Y_Y Line 2
C__.C__ Lines (0.5)*3 & (0.5)*4
Y_Y.Y_Y Line 4
___.___
Y_Y.Y_Y Line 6
C__.C__ Lines (0.5)*7 & (0.5)*8
Y_Y.Y_Y Line 8


---Truely accurate YV12 rendering is going to need to be aware
of Field Separated clips

I'll continue this discussion further in the next post.
(I need a break)

IanB

Fix Dyslexia with Top/Bottom in ascii art for siwalters
Fix Horiziontal chroma siteing in ascii art for siwalters

[IanB]

SummarisingSeparateFields()
--- It can probably be derestricted right now without problem.

Weave()
--- Refer to DoubleWeave()

DoubleWeave()
--- Removing the Field Separated property will break legacy script
support for DoubleWeave().

Bob()
--- Removing the Field Separated property will break legacy script
support for Bob().

GetParity()
---Removing the Field Separated property will require the design
of a new top field/bottom field state tracking algorithm.

ConvertFPS()
---It currently does not make correct use of the information

ShowFrameNumber()
---Cosmetic effect only

The rest (new YV12 code)
---Should be using Progressive/Interlaced determination

YV12 progressive/interlaced chroma sampling
---Truely accurate YV12 rendering is going to need to be aware
of Field Separated clips

Ironicly the only stuff that would depend on a Fields are Separated
flag is the new YV12 conversion code. Everything else is either
misusing the current flag, cosmetic or giving rise to variant
behavior.

I have a particularly strong bias towards maintaining legacy support
across adjacent releases of software. It help tremendously when doing
Quality Assurance. Test #1 is run the final QA suite of the last
release against the current product. Test #2 is run the new QA suite
against the product. The new QA suite of course has all the tests
for depreciated features scheduled to be expunged in the next release
removed and tests for all the new features added.

I strongly agree with sh0dan's sentement about variant behavior in
software generally, and to that end we should aim to clean up those
elements of the language in a controlled and scheduled way.

To that aim I would suggest that :-
Weave()
DoubleWeave()
Bob()
All be scheduled to be depreciated in version 3.(?)

The following invariant versions of the functions be added :-
WeaveFields()
DoubleWeaveFields()
DoubleWeaveFrames()
BobFields()
These new functions always do the same task irrespective of
any flags or states. Let the user beware.

And our friend :-
SeparateFields()
Can be quietly modified to always separate fields irrespective
of any flags or states.


As for the Fields have been Separated flag it is fairly
ingrained in core of how AviSynth works. Removing it is certainly
possible but I am not sure anything that plugs the gaps would be
much less offensive or really worth the effort of replacing it.

Getting things like Progressive/Interlaced determination and per
Frame attributes working should be a much higher priority than
depreciating working functionality.

IanB

[Si]

Ian - I think your post finally gets across to me what interlaced YV12 is about :)

But PLEASE edit your post and put the top separated fields above the bottom ones to make things easy for us spatial challenged readers. :)
Ta very much for swapping them :D

regards
Simon

[Bidoche]

Here is what I have in mind for 3.0 (as of the discussion for now) :

Clips are either fields clip or frames clips, so this flag remains in VideoInfo but not top/bottom

if the clip is made of fields, frames are marked with a field property which has a top/bottom flag.

if it's a frame clip, frames have a frame property with 3 states :
progressive, interlaced and unknown.
Somehow when a frame is queried for its state, unknown state should be resolved in one of the others.


For methods :

in IClip, GetParity is deprecated (if clip is field one it can be mapped to GetFrame(n).IsTop() ...)

VideoFrame gets new methods (non exhaustive list) :

bool IsField();
bool IsFrame();

bool IsTop() throw(IllegalStateException);
bool IsBottom() throw(IllegalStateException);

void SetAsField(bool top)

bool IsInterlaced() throw(IllegalStateException);
bool IsProgressive() throw(IllegalStateException);

void SetAsFrame(bool interlaced);

[IanB]

Originally posted by Bidoche
Here is what I have in mind for 3.0 (as of the discussion for now) :
.....
if it's a frame clip, frames have a frame property with 3 states :
progressive, interlaced and unknown.
You might try toying with a 4 state model :- progressive, interlaced_TFF, interlaced_BFF
and unknown. If Neuron2 gets his wish for per frame properties this will extend better and
in the meantime encourage people to code as if the property can vary.Somehow when a frame is queried for its state, unknown state should be resolved in one of the others.
.....And set coding standards that unknown should have the worst case restrictions applied.

IanB

[Bidoche]

You might try toying with a 4 state model :- progressive, interlaced_TFF, interlaced_BFF I don't get the point of interlaced_TFF and interlaced_BFF.
What is it for ?
It's separatefields who should say if it's want TFF or BFF, no ?

If Neuron2 gets his wish for per frame properties this will extend better and With my current properties design, everybody can create and add new ones (but not mess with others, you need the property key for that)

And set coding standards that unknown should have the worst case restrictions appliedI am thinking of it as an internal transitory value, VideoFrame will never report this state.
If this can be done, others won't have to worry about, only VideoFrame security check would.

[IanB]

Originally posted by Bidoche
I don't get the point of interlaced_TFF and interlaced_BFF.
What is it for ?
It's separatefields who should say if it's want TFF or BFF, no ?

I was thinking maybe some intelligent source filters (mpeg?) or perhaps some fancy
autoanalysis filter might like to be able to throw field order info into the mix.

I guess it's a data representation problem.

1. Unknown
2. Progressive
3. Interlaced
3.1. Top Field First
3.2. Bottom Field First

or

1. Unknown
2. Progressive
3. Interlaced - Top Field First
4. Interlaced - Bottom Field First

IanB

[Bidoche]

Yes, but I stil don't get the use we can get from saving the original field order.

[Si]

I don't get the point of interlaced_TFF and interlaced_BFF.

For completeness.

However, I personally would vote to just have a separate TFF (true/false) flag because ...

I deal mostly with captured TV material that is inherently interlaced as a medium.

I then try to make it as progressive as possible (as most of the actual footage is film progressive) and encode it as progressive MPEG-2 as SVCD.

However, I still need to correctly set the TFF flag (to true in my case) in the MPEG stream otherwise my standalone DVD player will not output the occasional frames that contain true interlaced fields in the correct order.

regards
Simon

[sh0dan]

Brand new code for interlaced and progressive YUY2 -> YV12 is in CVS. It currently requires ISSE, but I'll do an MMX version soon - hopefully before the next CVS binary is relased.

(It's both better and faster than the current XviD code).

[scmccarthy]

Field Seperated YV12 Interlaced samples are a completely Either I have no idea what you mean or you were right the first time.

I have carefully traced the way chroma is stored in an interlaces frame. It is exactly how it would be stored it you interlaced two progressive frames. It is purely the result of each field being sampled separately. That is all it is and it is much easier to understand if seen that way.

Stephen

[trbarry]

Yes, but I stil don't get the use we can get from saving the original field order.

Maybe I'm getting lost in Avisynth terminology again. But if 2 fields are stored in a frame then you need to know which one is first (in time) in order to deinterlace them, or even to display them on an interlaced TV.

- Tom

[Bidoche]

Hum... interlaced display, didn't think of that.

Ok then interlaced_TFF and interlaced_BFF and the according VideoFrame methods.

[Si]

@scmccarthy
I have carefully traced the way chroma is stored in an interlaces frame. It is exactly how it would be stored it you interlaced two progressive frames. It is purely the result of each field being sampled separately. That is all it is and it is much easier to understand if seen that way.

Have you got a reference I could look at.

I'm beginning to wonder if saying something is YV12 is sufficient definition.

Is YV12 not a storage concept with the ability to store different types of 4:2:0 video. From my searches it would seem that MPEG-2 4:2:0 is not chroma sampled the same as PAL DV 4:2:0 for instance.

(or am I just coming to the party too late and clueless as usual?)

regards
Simon

[IanB]

Originally posted by scmccarthy
Either I have no idea what you mean or you were right the first time.

I have carefully traced the way chroma is stored in an interlaces frame. It is exactly how it would be stored it you interlaced two progressive frames. It is purely the result of each field being sampled separately. That is all it is and it is much easier to understand if seen that way.

Stephen

Yes it is stored the same way, but the spacial position of the chroma is offset 0.5 line up and down in alternate fields, and this is very important when converting to another color space or just even processing the the image information.

Top Seperated Field
> ===============
> Y_Y.Y_Y Line 1t
> C__.C__ Lines (0.75)*1t + (0.25)*3t <-- Note the weighting, 0.5 line up
> Y_Y.Y_Y Line 3t
>
> Bottom Seperated Field
> =================
> Y_Y.Y_Y Line 2b
> C__.C__ Lines (0.25)*2b + (0.75)*4b <-- Note the weightings, 0.5 line down
> Y_Y.Y_Y Line 4b

And with the Force separated Progressive the sample is (0.5) from the other field.

A really good article "The Chroma Upsampling Error and The 4:2:0 Interlaced Chroma Problem by Don Munsil and Stacey Spears (http://www.hometheaterhifi.com/volume_8_2/dvd-benchmark-special-report-chroma-bug-4-2001.html)" gets right into 4:4:4, 4:2:2 & 4:2:0 chroma sampling for both progressive and interlaced formats. The graphic are also a lot better than my crappy ascii art.

IanB

:Edit^3: Simon, beat me to the punch by 2 minutes and yes DV spacial chroma sampling is
subtly different Cr 0.5 pixel down, Cb 0.5 pixel up for PAL DV and NTSC DV starts life as 4:1:1. **********! :rolleyes:

For good images of the issue see Adam Wilt site - DV Pix (http://www.adamwilt.com/pix-sampling.html)

:Edit^3: Smells like we need a general purpose YV12 chroma realignment filter.

:Edit^4: Fix ascii art horizontal positioning

[sh0dan]

@IanB: Nice spotted! Thanks for the info.

I corrected the interlaced YV12 -> YUY2 algorithm to have properly 25/75 weighing, and it actually helps some extreme cases. (I use the "Twin Peaks" logo - saturated green outline, with red filling). I'll be put in CVS as soon as I have done the MMX version.

An easy test is:

weave()
converttoyuy2(interlaced=true)
separatefields()

[IanB]

Shodan,

I noticed in Convert.cpp Line 379-394 a latent bug. When the clip is FrameBased vi.height can attempt to be odd (Nice crash!) :eek:


/**********************************
******* Convert to YV12 ******
*********************************/


ConvertToYV12::ConvertToYV12(PClip _child, bool _interlaced, IScriptEnvironment* env)
: GenericVideoFilter(_child),
interlaced(_interlaced)
{
if (vi.width & 1)
env->ThrowError("ConvertToYV12: image width must be multiple of 2");
if (interlaced && (vi.height & 3))
env->ThrowError("ConvertToYV12: Interlaced image height must be multiple of 4");
if ((!vi.IsFieldBased()) && (vi.height & 1))
env->ThrowError("ConvertToYV12: image height must be multiple of 2");
isYUY2=isRGB32=isRGB24=false;

Should be something like this....

if (interlaced && (vi.height & 3))
env->ThrowError("ConvertToYV12: Interlaced image height must be multiple of 4");
if (vi.height & 1)
env->ThrowError("ConvertToYV12: image height must be multiple of 2");

IanB

[sh0dan]

@IanB: Yes - all fieldbased/framebased stuff will have to be changed (based on this thread). There is also some checking in avisynth.cpp and other places (crop/resize) that needs to be changed. But I would like to do them all at once - but soon, I promise!

[Bidoche]

I am currently thinking at integrating constraints check into the properties system. (bascically each property will have to check the integrity of the whole set and will throw errors when needed).

More news later.

[trbarry]

For 3.0 I'd like to propose one more flag attribute for each frame. This would say the frame contains unknown garbage.

There is a version of my MPEG2DEC_ER.dll (shipped with MPEG2DEC2) that has a tiny bit of error recovery for handling questionable clips. It sets a trap before trying to decode each frame and recovers, returning garbage if an exception occurs. This is less useful (but can occur) for DVD rips but is a lot more probable for digital TV captures which can have garbage sections whenever there are reception breakups. So in problem cases this is sometimes the only way to continue.

Currently this version is only for YUY2/2.0 Avisynth but if I brought it over to 2.5 it might be nice to flag those bad frames and have Avisynth filters optionally deal with it. All current filters could ignore that flag, so it would just be an option. But in a special filter it would be possible to turn all garbage frames black or (better) replace all garbage frames with the last known good frame.

- Tom

[Si]

@IanB
Your diagrams all seem to show your chroma sampling horizontally between two Y samples but all the stuff I can find on MPEG-2 4:2:0 (cause that's what the main use of the YV12 is for it seems) shows the samples as being horizontally co-sited with the 1st Y sample.



regards
Simon