Looking for a special-purpose field-matcher

[Katie Boundary]

Is there a field-matching filter that can be set to alternate the field that it's trying to find a match for, so that it tries to find matches for either (a) all the top fields of the even-numbered frames and the bottom fields of the odd-numbered frames, or (b) the top fields of the odd-numbered frames and the bottom fields of the even-numbered frames?

It could be an excellent follow-up filter to get progressive frames back after certain bob-deinterlacers.

[hello_hello]

Aren't de-interlacers supposed to output progressive video? What's there to field match?

I don't see the point, but I think this would do what you want, if I understood the question correctly.

A=SelectEven().TFM(field=0)
B=SelectOdd().TFM(field=1)
Interleave(A,B)

[Katie Boundary]

Alright, the good news is that I forgot to log in before checking this thread, and thus accidentally saw Hello's post. You interpreted my question correctly and gave a sufficiently correct response, so I'm taking you off ignore.

Thew bad news is that, even following a "bad" bob, the original fields apparently prefer to be matched with newly created bob-deinterlaced counterparts, rather than with their original counterparts, so this neat trick isn't quite as useful as I had hoped. It might be useful following Separatefields and Doubleweave. I'll have to do some tests.

EDIT: yes, that works for preserving the progressive frames when converting hybrid content to 60 fps. Replacing the remaining interlaced frames with YADIFed clips in Premiere should give a 100% perfect result.

EDIT 2: I'm getting frame-accuracy problems around scene changes. Specifically, some orphaned fields around scene changes are being decimated and replaced with nearby progressive frames. WTF?

[hello_hello]

Alright, the good news is that I forgot to log in before checking this thread, and thus accidentally saw Hello's post. You interpreted my question correctly and gave a sufficiently correct response, so I'm taking you off ignore.

The good news is you already forgot to pretend (https://forum.doom9.org/showthread.php?p=1785397#post1785397) to be ignoring me, so it makes the old "forgot to log in" excuse a little more amusing.

The bad news is you still don't seem to understand I've always interpreted your questions correctly and given you the correct response, even while you were asking the wrong questions, refusing to listen to anyone and arguing about it like a nutter.

Thew bad news is that, even following a "bad" bob, the original fields apparently prefer to be matched with newly created bob-deinterlaced counterparts, rather than with their original counterparts, so this neat trick isn't quite as useful as I had hoped. It might be useful following Separatefields and Doubleweave. I'll have to do some tests.

Well you'd only "bob" interlaced video, you can't field match interlaced video, and after bob deinterlacing you have progressive video which doesn't need field matching anyway....
What's a "bad bob"?

EDIT: yes, that works for preserving the progressive frames when converting hybrid content to 60 fps. Replacing the remaining interlaced frames with YADIFed clips in Premiere should give a 100% perfect result.

I'll take your word for it.

EDIT 2: I'm getting frame-accuracy problems around scene changes. Specifically, some orphaned fields around scene changes are being decimated and replaced with nearby progressive frames. WTF?

Let me gaze into my crystal ball.....
I can only imagine what you're doing now and it's futile to suggest you upload a sample.

[ndjamena]

Shouldn't selecteven and selectodd come AFTER TFM?... the way you're doing it means it only gets to choose from fields that were at least two fields away before being bobbed.

Not that any of this makes sense in the first place...

[hello_hello]

Shouldn't selecteven and selectodd come AFTER TFM?... the way you're doing it means it only gets to choose from fields that were at least two fields away before being bobbed.

Yes I imagine it should. Apparently I didn't give it enough thought at the time.

[Katie Boundary]

The good news is you already forgot to pretend (https://forum.doom9.org/showthread.php?p=1785397#post1785397) to be ignoring me

No, I was ignoring you back then too. I was just sort of guessing at what stupid nonsense you had posted.

The bad news is you still don't seem to understand I've always interpreted your questions correctly and given you the correct response, even while you were asking the wrong questions, refusing to listen to anyone and arguing about it like a nutter.

It's cute that you still think so, despite all evidence to the contrary.

Well you'd only "bob" interlaced video

Or in my case, hybrid content.

What's a "bad bob"?

A haircut from the '70s, or a low-quality bob-deinterlace.

I can only imagine what you're doing now and it's futile to suggest you upload a sample.

https://scontent.fsnc1-1.fna.fbcdn.net/v/t1.0-9/15940467_1792438221020632_5027785062503523592_n.jpg?oh=e1c9d8b2308c376729e609796dde8082&oe=590FB7F9

Shouldn't selecteven and selectodd come AFTER TFM?... the way you're doing it means it only gets to choose from fields that were at least two fields away before being bobbed.

Not that any of this makes sense in the first place...

I'll try that.

EDIT: it made no difference.

[ndjamena]

There's no reason it would. The whole point of a bob deinterlacer, assuming it leaves the original fields lossless, is to invent fields to match the original fields.

To TFM the invented fields will be a BETTER match for the original fields than the original partners.

That's why this doesn't make sense. And to be honest, if you know you've got progressive frames passed through a bob deinterlacer you'd be better off using separatefields.selectevery(4,0,3).weave

If you're talking about recovering some of the original progressive frames from video that was originally VFR (hybrid) that would involve comparing the top fields with the surrounding bottom fields (excluding it's current partner) to see if there's any combing present and replacing the current partner with the match if there isn't.

The problem is you used the word "bob" specifically. The whole point of a bob deinterlacer is that it discards half the fields and resizes what's left to create a full frame. The original fields no longer exist once you've done that, QTGMC would require SourceMatch and Lossless enabled to produce output that might be used with the filter you're describing.

If you just meant double rate deinterlacers that would make more sense. However, once it's been encoded using a lossy codec the fields will have been altered to better match the invented fields than its old partner and you'd need to decomb the output to some extent anyway.

[hello_hello]

No, I was ignoring you back then too. I was just sort of guessing at what stupid nonsense you had posted.

It's cute that you still think so, despite all evidence to the contrary.

Doesn't maintaining the fantasy world you live in require an enormous amount of energy?

I like the pretty pictures. It's a wonder the weren't covered in large text proclaiming a straw man argument or some such nonsense. You've had the definition of a sample explained to you, been given instructions on how to create one, and given a list of sites where they can be uploaded, yet you still can't do it.
https://forum.doom9.org/showthread.php?p=1776197#post1776197
https://forum.doom9.org/showthread.php?p=1771342#post1771342

No wonder there's very few posters left who are willing to waste their time on you.

[Katie Boundary]

There's no reason it would. The whole point of a bob deinterlacer, assuming it leaves the original fields lossless, is to invent fields to match the original fields.

To TFM the invented fields will be a BETTER match for the original fields than the original partners.

That's why this doesn't make sense.

Well, yes, I knew that would be a possibility before I posted. I just needed to be sure.

If you're talking about recovering some of the original progressive frames from video that was originally VFR (hybrid) that would involve comparing the top fields with the surrounding bottom fields (excluding it's current partner) to see if there's any combing present and replacing the current partner with the match if there isn't.

I also need all of the bottom fields, untouched and in their correct locations.

The whole point of a bob deinterlacer is that it discards half the fields and resizes what's left to create a full frame. The original fields no longer exist once you've done that

That it not correct... at least not always.

If you just meant double rate deinterlacers that would make more sense. However, once it's been encoded using a lossy codec the fields will have been altered to better match the invented fields than its old partner and you'd need to decomb the output to some extent anyway.

This is a part of a larger project. For this part, the top priority is keeping the original fields untouched and in whatever places they'd be in if I had just run the whole thing through ibob(), bob(0,1), or YADIF(mode=3). SECOND priority is finding correct field matches for the fields that actually have matches. Whether the orphaned fields get matched incorrectly, or bob-deinterlaced, or whatever at this point doesn't really matter as long as they stay where they're supposed to be. I hope that clarifies things.

EDIT: I tried simplifying the approach by cutting out all forms of bobbing, weaving, duplicating, and decimating, and ran straight with this:

assumetff()

A=TFM(field=1,mode=0,pp=0)
B=TFM(field=0,mode=0,pp=0)
Interleave(A,B)

Unfortunately, it offered no improvement.

[Katie Boundary]

off-topic garbage and pointless attempts at insults

Welcome back to my ignore list.

[hello_hello]

Welcome back to my ignore list.

Excellent. I imagine sticking your fingers in your ears again will cut down on the amount of nonsense arguing you do, like claiming a request for a proper sample is off topic garbage.

[ndjamena]

http://forum.videohelp.com/threads/370888-Suggestions-For-Optimizing-SD-Animation-Encoding?p=2402091&viewfull=1#post2402091

Look at the "Function FieldMatch(Clip C)" part of the script. That's the kind of thing you want. You just need to check the neighbouring opposing fields first and keep the current one if they don't work.

[hello_hello]

http://forum.videohelp.com/threads/370888-Suggestions-For-Optimizing-SD-Animation-Encoding?p=2402091&viewfull=1#post2402091

Look at the "Function FieldMatch(Clip C)" part of the script. That's the kind of thing you want. You just need to check the neighbouring opposing fields first and keep the current one if they don't work.

The whole approach seems rather "Katie" to me, as proven by the fact she's already run back to her ignore list.

The problem, I discovered while messing around, is if you take a progressive clip, apply pulldown for 29.970fps, then separate fields and double weave, the following script doesn't correctly match all the fields and relies on SelectEven() to remove the unmatched ones.

A=TFM(field=0,mode=0,pp=0).selecteven()
B=TFM(field=1,mode=0,pp=0).selecteven()
Interleave(A,B)

However after deliberately creating a clip with an orphaned field to see what Katie was doing wrong there, it became obvious that after the pulldown pattern changes, SelectEven() will no longer remove the unmatched frames.

The best method seems to be to do what everyone else in the world does, and let TFM do it's thing. Then the output is fine.
ie SeparateFields().DoubleWeave().TFM(mode=1,pp=1)

Either way though, TFM leaves any orphan fields untouched if you tell it to. Probably either the wrong field order for each instance of TFM was used (it should have been the other way around) or the unmatched frame was removed by SelectEven(), so SelectOdd() should be used instead.

Orphaned field on scene change after SeparateFields().DoubleWeave(), No TFM
https://s27.postimg.org/ud7uaxc9r/image.jpg (https://postimg.org/image/ud7uaxc9r/)

SeparateFields().DoubleWeave().AssumeBFF()
A=TFM(field=0,mode=1,pp=0).selecteven()
B=TFM(field=1,mode=1,pp=0).selecteven()
Interleave(A,B)
https://s27.postimg.org/byxb6xzz3/image.jpg (https://postimg.org/image/byxb6xzz3/)

SeparateFields().DoubleWeave().AssumeBFF()
A=TFM(field=0,mode=1,pp=3).selecteven()
B=TFM(field=1,mode=1,pp=3).selecteven()
Interleave(A,B)
https://s27.postimg.org/edp6rdg7z/image.jpg (https://postimg.org/image/edp6rdg7z/)

[Katie Boundary]

Well, I found part of the problem. There are field order transitions that aren't being reported by DGIndex.

http://forum.videohelp.com/threads/370888-Suggestions-For-Optimizing-SD-Animation-Encoding?p=2402091&viewfull=1#post2402091

Look at the "Function FieldMatch(Clip C)" part of the script. That's the kind of thing you want. You just need to check the neighbouring opposing fields first and keep the current one if they don't work.

I don' understand 90% of that :confused:

[johnmeyer]

I have never once understood anything Katie has posted. The topic always seems manufactured, and not quite real-world. This one does too.

However, having made my snarky comment, I will offer one suggestion that I have found useful when TFM isn't able to correctly sort out strange field order problems: play around with the TFM "order" variable. In theory, it shouldn't do anything that you couldn't do simply by making sure the AssumeFPS() statement matches the actual field order. But, in practice, I have found that changing it can have some very pleasant results, where minor weirdness goes away.

It is a simple think to try: just create a test case and use Order=-1 (the default) to establish your baseline, and then compare that to Order=0 and Order=1.

[Katie Boundary]

Wait, would field order transitions in the original stream even matter after pulldown is applied?

I have never once understood anything Katie has posted. The topic always seems manufactured, and not quite real-world. This one does too.

It's not surprising. I tend to involve myself in a lot of really weird projects.

However, having made my snarky comment, I will offer one suggestion that I have found useful when TFM isn't able to correctly sort out strange field order problems: play around with the TFM "order" variable.

Tried that yesterday.


EDIT: Correct me if I'm wrong, but shouldn't the following script force all of the original bottom fields to be included in the final stream regardless of any other circumstances?


mpeg2source("212.d2v")
Tfm(field=0,mode=0,pp=0)

That's literally what I've reduced one of my test scripts down to at this point and it's STILL decimating some bottom fields. Could there be a bug in TFM?

[hello_hello]

That's literally what I've reduced one of my test scripts down to at this point and it's STILL decimating some bottom fields. Could there be a bug in TFM?

I consulted the tarot cards and they seem to indicate it's operator error.

Given you're incapable of uploading a sample, why do you even bother? You're asking "how long is this piece of string" while refusing to show the string to anyone.

[Katie Boundary]

This is weird. Even though the following script failed yesterday...

mpeg2source("212.d2v")

separatefields()
doubleweave()

A=Tfm(field=1,pp=0).selecteven()
B=Tfm(field=0,pp=0).selecteven()
Interleave(A,B)

I just tried this one and it gives the exact results that I'm looking for:

mpeg2source("212.d2v")

A=separatefields().doubleweave().Tfm(field=1,pp=0).selecteven()
B=separatefields().doubleweave().Tfm(field=0,pp=0).selecteven()
Interleave(A,B)

So... don't know what caused it, don't care. I'm now chugging through Premiere in full seek-and-replace mode. Our work here is done.

[johnmeyer]

Can someone close this thread? I totally agree with hello and others that the refusal to upload a sample makes this a completely pointless exercise, a total waste of time, and sorry misuse of precious bandwidth and bits.

Can the OP provide any reason whatsoever why he (or she) cannot upload a sample??? If no answer is provided, close the thread.

[lansing]

I have a feeling that Katie is treating this forum as her personal blog, in which she kept on talking to herself and nobody knows what she's talking about.

[ndjamena]

I prefer my filters to be simple, fast, and dumb (separatefields, uncomb, ibob, simpleresize, et cetera). This has caused a lot of problems for me.

She likes filters she can relate to.

[Katie Boundary]

Let the record show that, after I said "our work here is done", no less than 3 other users, who had the chance to let this thread quietly die, decided to keep spamming...

Can the OP provide any reason whatsoever why he (or she) cannot upload a sample???

What would be the point, exactly? There's nothing unique about the content that I'm working with. The script devised here can be tested with literally any content that has a few orphaned fields.

[ndjamena]

mpeg2source("212.d2v")

A=separatefields().doubleweave().Tfm(field=1,pp=0).selecteven()
B=separatefields().doubleweave().Tfm(field=0,pp=0).selecteven()
Interleave(A,B)

If you actually bothered reading the documentation, you'd realise separatefields.doubleweave.selecteven just returns the frames you started with.

I thought the point of this was to get the original frames back from double-rate deinterlacers. TFM alone couldn't do it but somehow adding an entire frame constructed purely of interpolated fields between the fields you want to bring together and then passing it through TFM does.

It doesn't make any sense, nor does that fact that you're getting double rate footage from mpeg2source in the first place.

[ndjamena]

OK, Look

OP:
assumetff()

A=TFM(field=1,mode=0,pp=0)
B=TFM(field=0,mode=0,pp=0)
Interleave(A,B)

Endless Greetings:
A=TFM(field=0,mode=0,pp=0).selecteven()
B=TFM(field=1,mode=0,pp=0).selecteven()
Interleave(A,B)


SeparateFields().DoubleWeave().AssumeBFF()
A=TFM(field=0,mode=1,pp=0).selecteven()
B=TFM(field=1,mode=1,pp=0).selecteven()
Interleave(A,B)


Should I understand the transition here? She wasn't using selecteven with TFM, so that entire argument is debunked. How the hell can you compare tfm.selecteven with doubleweave.tfm.selecteven?

Since we're starting with MPEG2Source, plus reading a few hints in the thread, I'm assuming we've abandoned the whole reconstruction from a bobbed source and have reverted to starting from the original source, which is what it should have been in the first place.

given that:

assumetff
interleave(tfm(field=1, slow=2, PP=0, micmatching=0),tfm(field=0, slow=2, PP=0, micmatching=0))

Should be enough to imitate double-rate deinterlacing perfectly, while keeping all the fields in the same place.

Or, as I prefer to use:

assumetff
a1 = qtgmc(preset="placebo", sourcematch=3, lossless=2)
interleave(tfm(field=1, slow=2, clip2=a1.selecteven,PP=5, micmatching=0),tfm(field=0, slow=2,clip2=a1.selectodd,PP=5, micmatching=0))

Am I missing something or is this entire thread dumb?

[hello_hello]

ndjamena,
Were you referring to me with your last post?

To be honest I'm not sure what Katie was trying to achieve. She originally said the idea was to "get the original progressive frames back after certain bob de-interlacers" and that apparently was after a "bad bob" and when I asked what the meant she replied with one of her stupids.

Post #3 she mentioned converting hybrid content to 60fps and that's what I thought she was trying to achieve from there....
Film with pulldown for 29.970 -> SeperateFields().DoubleWeave().TFM() for 59.940fps progressive.

If you look at the pic in post #7 you'll see she was using the following script at that stage. The trouble is for "film" to 59.940fps it seems to fall apart if the pulldown pattern changes, and I think that's because it simply doesn't work as I initially thought it would (even when mode=1). Now I think about it, my original idea was wrong and I don't think it does "field match all the top fields of the even-numbered frames and the bottom fields of the odd-numbered frames", as Katie originally wanted.

SeparateFields().DoubleWeave().AssumeBFF()
A=TFM(field=0,mode=0,pp=0).selecteven()
B=TFM(field=1,mode=0,pp=0).selecteven()
Interleave(A,B)

Now I feel silly. This would probably do it though, wouldn't it?

A=TFM(field=0).selecteven()
B=Trim(1,0).TFM(field=1).selecteven()
Interleave(A,B)

Anyway, she only reduced the script to the following for testing later:

assumetff()
A=TFM(field=1,mode=0,pp=0)
B=TFM(field=0,mode=0,pp=0)
Interleave(A,B)

I think the problem is she decides she knows better than automated tools and comes up with ideas like "field matching all the top fields of the even-numbered frames and the bottom fields of the odd-numbered frames", won't explain what she's trying to achieve properly, then runs into problems and complains about the bottom fields not always being used for matching when that's exactly what she said she didn't want..... or something..... but then she decided this works:

A=separatefields().doubleweave().Tfm(field=1,pp=0).selecteven()
B=separatefields().doubleweave().Tfm(field=0,pp=0).selecteven()
Interleave(A,B)

And from there she announced she's chugging through Premiere in full seek-and-replace mode, which apparently involves "replacing the remaining interlaced frames with YADIFed clips in Premiere" which probably means after days and days and days of arguing about separating and resizing fields as the best method of de-interlacing, she's finally discovered Yadif(mode=1) looks better. Mind you why that requires Premier when you can split a script into sections with Trim() I don't know.

Of course we'll never know for sure because she refuses to upload samples, but that was my take on it, and yes, the whole thread was completely pointless.

[ndjamena]

micmatching=0

Apparently TFM will replace unmatched fields unless micmatching is turned off.

Damn.

What the hell does micmatching mean anyway? This is all just gibberish if they don't actually explain anything.

There's a warning in the manual that they turned it on by default in an update, and that you should turn it off if you don't want it. Yet somehow they fail to explain what it actually does.

[TheFluff]

As per the TFM manual:
micmatching -

When micmatching is greater than 0, tfm will take into account the mic values of matches
when deciding what match to use as the final match. micmatching has 5 possible settings:

0 - disabled
1 - mics are only used to determine if of the 5 possible matches there is one match
that has a significantly lower mic value than all of the other four matches.
With this option enabled it is possible for tfm to use any match (p/c/n/b/u)
regardless of the matching mode!
2 - use mics to aid the decision within the current matching mode (only matches that
could be used within the current matching mode are considered)
3 - do both option 1 and option 2
4 - do a post field matching decision check only (this operation is included in
micmatching modes 1, 2, and 3, and is not subject to the mmsco parameter)

Debug output will indicate when micmatching has led to a change in the match decision.
Enabling micmatching will make processing slower due to needing to calculate mic values
for matches that they normally wouldn't be calculated for.

The "mmsco" parameter can be used to restrict the use of micmatching to scenechanges only.
To do this, set mmsco to true... then, only frames on either side of a scenechange
will have the possibility of being effected by micmatching.

**** As of v0.9.10.0 of TFM, micmatching defaults to 1. That means any one of the five
**** matches could be used regardless of the matching mode! If you don't want this
**** behavior then set micmatching to 0!

**** As of v0.9.12.6 of TFM, all micmatching modes will run a check after each
**** field matching decision to check for cases where the mic values make it
**** obvious that the wrong match was chosen. This part acts independently of
**** the mmsco parameter.

While this does not actually explain what a "mic" is (nor is it explained anywhere else in the manual), it may be inferred from the context that the mics are some kind of measure of the combing that is not the same as the regular old "cthresh" (which was inherited straight from Decomb). If you read the entire manual though, things become a bit clearer:

MI -

The # of combed pixels inside any of the blocky by blockx size blocks on the frame
for the frame to be detected as combed. While cthresh controls how "visible" the
combing must be, this setting controls "how much" combing there must be in any localized
area (a window defined by the blockx and blocky settings) on the frame. Min setting = 0,
max setting = blocky x blockx (at which point no frames will ever be detected as combed).
This is clearly related. As it turns out, to get an actual explanation, you have to go to the VIVTC (which is essentially a reverse engineered TIVTC for VS) manual, where mics are explained in passing:
VFMMics

Array of five integers.

It will contain the mic values for the five possible matches (p/c/n/b/u). Some of them may be unset (-1), depending on micout and micmatch.

These numbers represent the highest concentration of combed pixels found in any block in the frame.

tl;dr: the mics and the micmatching are a mechanic that are supposed to prevent TFM from matching wrong in static scenes where choosing the wrong match only leads to a tiny little area of the frame being combed. The traditional test case for this is talking heads scenes in anime, where Decomb used to match the mouths (which was the only thing moving) wrong.

I agree by the way that the TFM documentation is fairly awful and it was not my intention with this post to smugly tell you "if you actually RTFM...", but rather just demonstrate how annoying this information was to find. I myself should have remembered it since I used to know TFM upside down and backwards by heart back in the day (excessive exposure to anime encoding and YATTA may lead to brain damage) but I didn't, so I had to look it up just for old times' sake.

[ndjamena]

Yes, but that still doesn't really explain why using this:

interleave(tfm(field=1, slow=2, PP=0),tfm(field=0, slow=2, PP=0))

ie, without disabling micmatching, will actually cause orphaned fields to be completely removed from the output video, even though logically every field should be there somewhere.

Maybe I just don't understand what TFM actually does, or maybe it was just coincidence that disabling mismatching kept the orphan fields.

I assumed setting the "field" parameter meant all the fields in whatever field it is set to would be kept, and TFM would run off and find matches for them from the opposing field. The fact that one particular orphan field simply vanished from the output video whenever micmatching is enabled kind of puts a lie to that assumption. It means TFM isn't as straight forward as I assumed it was and that makes it's output difficult to predict, which makes me question whether I ever want to use it again.

[TheFluff]

The answer to that actually is in the manual and it's not even all that weird or hard to find. If you understand what all the five field matches do, it should be obvious, but to spell it out: the 4th and 5th way matches (u and b) do not actually use the field being matched from, they use the opposing parity instead. See the section on overrides in the TFM manual if you want a visualization of what's going on with those. As also mentioned in the manual, micmatching being enabled also enables all five matches, regardless of what the mode parameter is set to. By default, though, this behavior is only used around scenechanges (mmsco and ubsco are both true by default). TFM assumes as a default - rather reasonably, I'd say - that you'd rather have a duplicate frame than a combed one around scenechanges, so orphaned fields vanishing is the entire point. VIVTC rectifies this admittedly somewhat unexpected behavior and follows what matches are allowed by the mode parameter strictly, if that makes you feel any better.

Of course the output of anything is unpredictable if you don't understand what it actually does, but I don't find TFM to be particularly difficult in this regard, at least not by Avisynth standards. The documentation is rather confusing in places (like the basic parameter "mode" having its behavior completely changed by advanced parameters that are enabled by default), but the field matching logic is fairly straightforward and it does have the debug option that will tell you exactly what it is doing and why. You also have the option of writing override files if you're feeling particularly masochistic.

[hello_hello]

Looks like TheFluff beat me to it, but from the help file in the overrides section:

These three matches, p, c, and n are the most common or the usual matches. In the mode parameter description these are the three matches that make up the 2-way and 3-way matching. The other matches: u and b are slightly different in that they don't use the field that is being matched from.

However the default for mmsco is true, which means even though micmatching is enabled by default, it's limited to only being enabled around scene changes.

But then it goes on to explain that from version 0.9.12.6 of TFM, all micmatching modes will run a check after each field matching decision to check for cases where the mic values make it obvious that the wrong match was chosen. This part acts independently of the mmsco parameter.

I ran some tests and checked it. You definitely have to set micmatching=0 if you don't want it to be used, even at scene changes, despite the fact mmsco is true by default. When it's blatantly obvious a field has no match, micmatching will still be used unless you set micmatching=0.
Which all makes sense. Why output a horrible combed or blended frame if it can be avoided?

The metrics files I created show it does work as expected, once you know what to expect. I created the following script from a 23.976fps progressive source.

Trim(1032,1999)
Separatefields()
SelectEvery(8, 0,1, 2,3,2, 5,4, 7,6,7)
Trim(0,516)+Trim(641,0)
Weave()
TFM(pp=1, field=1, Output="D:\TFM.txt", micout=2)

The upshot is, frame 258 of the output video contains a field so orphaned, it's the only field for an entire scene. No match is possible. It's the top field.

Here's what the metrics file looks like when micmatching=1 (the highlights).

#TFM v1.0.4 by tritical
field = top
crc32 = ec531e59
#TFM v1.0.4 by tritical
field = top
crc32 = ec531e59
0 c - [5] (5 5 145 5 133)
1 c - [2] (133 2 2 145 178)
2 n - [5] (178 178 5 2 241)
3 n - [1] (241 204 1 5 204)
4 c - [1] (204 1 220 1 227)

254 c - [2] (256 2 256 2 256)
255 c - [1] (256 1 256 256 256)
256 c - [2] (256 2 2 256 256)
257 n - [1] (256 256 1 2 256)
258 b - [1] (256 256 256 1 256)
259 c - [8] (256 8 8 256 150)
260 n - [6] (150 150 6 8 210)
261 n - [8] (210 132 8 6 132)
#
# OVR HELP INFORMATION:
#
# [COMBED FRAMES]
#
# [Individual Frames]
# FORMAT: frame_number (mic_value)
#
# none detected
#
# [Grouped Ranges Allowing Small Breaks]
# FORMAT: frame_start, frame_end (percentage combed)
#
# none detected
#
#
# [POSSIBLE MISSED COMBED FRAMES]
#
# FORMAT: frame_number (mic_value)
#
# none detected
#
#
# [u, b, AND AGAINST ORDER (p) MATCHES]
#
# FORMAT: frame_number match or range_start,range_end match
#
# 258 b

When micmatching=0

#TFM v1.0.4 by tritical
field = top
crc32 = ec531e59
0 c - [5] (5 5 145 5 133)
1 c - [2] (133 2 2 145 178)
2 n - [5] (178 178 5 2 241)
3 n - [1] (241 204 1 5 204)
4 c - [1] (204 1 220 1 227)

254 c - [2] (256 2 256 2 256)
255 c - [1] (256 1 256 256 256)
256 c - [2] (256 2 2 256 256)
257 n - [1] (256 256 1 2 256)
258 n + [256] (256 256 256 1 256)
259 c - [8] (256 8 8 256 150)
260 n - [6] (150 150 6 8 210)
261 n - [8] (210 132 8 6 132)
#
# OVR HELP INFORMATION:
#
# [COMBED FRAMES]
#
# [Individual Frames]
# FORMAT: frame_number (mic_value)
#
# 258 (256)
#
# [Grouped Ranges Allowing Small Breaks]
# FORMAT: frame_start, frame_end (percentage combed)
#
#
#
# [POSSIBLE MISSED COMBED FRAMES]
#
# FORMAT: frame_number (mic_value)
#
# none detected
#
#
# [u, b, AND AGAINST ORDER (p) MATCHES]
#
# FORMAT: frame_number match or range_start,range_end match
#
# none detected

Now I'm not sure why I was seeing a different result yesterday. Maybe there wasn't enough combing for micmatching to be used. Maybe the way I tested was flawed.

I assume the numbers contained in [ ] show the mic value for the used field match, and lower is better. The numbers to the right apparently represent matches in this order: p/c/n/b/u

p = match to previous field
c = match to current field
n = match to next field
b = match to previous field (matches from opposite parity field)
u = match to next field (matches from opposite parity field)

So the way I understand it now, is if the metrics file shows "field = bottom" (and it'll show one or the other, even if the field option isn't set or the parity changes), matches of "b" or "u" really mean TFM has switched to "field = top" for that particular frame.

[ndjamena]

So that means, the answer to the thread is:

assumetff
interleave(tfm(field=1, slow=2, PP=0, micmatching=0),tfm(field=0, slow=2, PP=0, micmatching=0))

More or less.

I'd feel like a dumb-arse, except no one else brought it up and apparently the closest thing to an explanation is in the overrides section, which is a feature I've never felt the need to use.

[hello_hello]

I'd feel like a dumb-arse, except no one else brought it up and apparently the closest thing to an explanation is in the overrides section, which is a feature I've never felt the need to use.

I had no idea it worked like that either till I started experimenting today, but even if Katie never learns anything from her threads, at least I do now and then. ;)

[Katie Boundary]

If you actually bothered reading the documentation, you'd realise separatefields.doubleweave.selecteven just returns the frames you started with.

I did read the documentation and I did realize that, though I didn't chain them together in that order so I'm not sure how that's relevant.

I thought the point of this was to get the original frames back from double-rate deinterlacers.

That was one possible application. I lost interest in that route when I confirmed that it wasn't a viable route to my ultimate goal.

TFM alone couldn't do it but somehow adding an entire frame constructed purely of interpolated fields between the fields you want to bring together and then passing it through TFM does.

An entire frame constructed purely of interpolated fields? As if I had chained two bob-deinterlacers together? Where are you getting THAT from?

What's weird that separatefields().doubleweave() prevented TFM from incorrectly decimating certain orphaned fields.

It doesn't make any sense, nor does that fact that you're getting double rate footage from mpeg2source in the first place.

I'm not.

Should I understand the transition here? She wasn't using selecteven with TFM, so that entire argument is debunked.

Yes, I was. I started using it as soon as H_H suggested it.

Since we're starting with MPEG2Source, plus reading a few hints in the thread, I'm assuming we've abandoned the whole reconstruction from a bobbed source and have reverted to starting from the original source, which is what it should have been in the first place.

I explicitly said so in post #3

Am I missing something

Obviously.

micmatching=0

Apparently TFM will replace unmatched fields unless micmatching is turned off.

Damn.

YOU SOLVED THE MYSTERY!

THANK YOU:thanks::thanks::thanks::thanks::D:D

(I still have no idea how you managed that when so many of your assumptions about the flow of conversation were wrong, but still, you did it)

[ndjamena]

I had no idea it worked like that either till I started experimenting today, but even if Katie never learns anything from her threads, at least I do now and then. ;)

That's how I've always rationalised marcorocchini being on the Videohelp forum. If he ever gets irritating I just remember how FFProbe works, which is a program I'd have never touched if he knew how to do anything himself.

[ndjamena]

I explicitly said so in post #3


Try to imagine reading it through a veil of pain, and you'll realise it says nothing explicitly at all.

[Katie Boundary]

By the way, in case anyone is curious, I've discovered an added benefit of using manual frame replacement instead of relying 100% on algorithms. Some fields are what I call "motion orphans" - they're what you normally think of when you think of orphaned fields. Others are merely "chroma orphans". They're what happens when the last field of one shot and the first field of another shot get encoded as a single interlaced frame. Some orphaned fields are both. If a field is a chroma orphan but not a motion orphan, I can cut it out and replace it with a neighbor that represents the same image but with correct chroma; the motion orphans, however, I prefer to simply bob, regardless of whether or not they are also chroma orphans.

So, unless someone here has a filter capable of automatically doing THAT, I suggest that we all go home and STFU.

Try to imagine reading it through a veil of pain, and you'll realise it says nothing explicitly at all.

It's pretty explicit about how it didn't have the intended effect on bobbed footage, so I was going to try it on doubleweaved footage next. I don't see how any amount of pain can cause you to mis-read something THAT badly.

[hello_hello]

That's how I've always rationalised marcorocchini being on the Videohelp forum. If he ever gets irritating I just remember how FFProbe works, which is a program I'd have never touched if he knew how to do anything himself.

And to think only yesterday anyone continuing to post in this thread after Katie had officially declared it ended was labelled a spammer. Maybe there's a lesson in listening to other people there for Katie, given a spammer solved her problem. ;)

So that means, the answer to the thread is:
assumetff
interleave(tfm(field=1, slow=2, PP=0, micmatching=0),tfm(field=0, slow=2, PP=0, micmatching=0))

Technically, to answer the original question, I still think SelectOdd and SelectEven would have to be added.

Or this?
A=TFM(field=0, PP=0, micmatching=0).selecteven()
B=Trim(1,0).TFM(field=1, PP=0, micmatching=0).selecteven()
Interleave(A,B)

Would that match all the top fields of the even-numbered frames and the bottom fields of the odd-numbered frames?

Still I'd be keen to know when micmatching=0 is a good idea. What does keeping a completely orphaned field achieve when you're not encoding through the looking glass?

I still have no idea how you managed that when so many of your assumptions about the flow of conversation were wrong, but still, you did it

TFM is decimating orphaned fields. It's a simple fact that's easily checked. Why would understanding a workflow from the Twilight Zone be required for that?

By the way, in case anyone is curious, I've discovered an added benefit of using manual frame replacement instead of relying 100% on algorithms. Some fields are what I call "motion orphans" - they're what you normally think of when you think of orphaned fields. Others are merely "chroma orphans". They're what happens when the last field of one shot and the first field of another shot get encoded as a single interlaced frame. Some orphaned fields are both. If a field is a chroma orphan but not a motion orphan, I can cut it out and replace it with a neighbor that represents the same image but with correct chroma; the motion orphans, however, I prefer to simply bob, regardless of whether or not they are also chroma orphans.

So, unless someone here has a filter capable of automatically doing THAT, I suggest that we all go home and STFU.

So after all the effort to help her, despite how hard she makes it to do so, she finishes with that to make sure we don't forget she's a total prat.

Although could someone explain this motion vs chroma orphaned field thing to me? I don't get it. Is she referring to some sort of chroma bleeding from one field to the next (how does that happen anyway)?
Maybe someone else could explain to Katie how to upload a sample so we can see what she's talking about.
And a sample of the finished product too. I'd be rather keen to see the end result after all the fussing over separating fields, weaving, decimating, bobbing orphaned fields and replacing orphaned chroma fields. Maybe the rest of use could learn something.

[TheFluff]

If doing mixed mode IVTC by hand sounds attractive to you there are tools to help you with that, you know. YATTA is the obvious one. It kinda requires understanding what you're doing, though.