Compensating for retrograde field behavior

[Katie Boundary]

There are two kinds of pathologically f'ed-up content. The first kind is made by people who don't have any idea what they're doing. The second kind is made by people who know exactly what they're doing, and who've chosen to use their powers for evil. This thread is about the second type.

I'm once again dabbling in possible ways to deal with retrograde field behavior that don't require cutting the frame rate below 30 FPS or knowing every single place in the video stream where this problem occurs. My test case for this kind of content will be the Gargoyles opening because there can be no correct field order due to the way it was edited. I'm on to an approach that looks promising:

mpeg2source("102h.d2v")

A=Separatefields().weave()
B=separatefields().doubleweave().selectodd()
C=Tfm(field=1,mode=0,pp=3,slow=2,cthresh=2,clip2=A,micmatching=0)
D=Tfm(field=0,mode=0,pp=3,slow=2,cthresh=2,clip2=B,micmatching=0)
Interleave(C,D)

{Post-process as needed}


Here are the problems I'm noticing so far:

1) I'd like a better "no-op" than Separatefields().weave(), or Blur(0), or resizing to the current size, or crop(0,0,-0,-0), etc.; alternatively, I could rearrange the script to not need a no-op, but I can't think of a way to do that elegantly.

2) TFM will pair orphaned fields around scene changes with the nearest field from their own scene, but then replace these matches with their Doubleweave equivalents, often resulting in two fields from different clips getting woven together. I think micmatching can be used to fix this but I tend to avoid micmatching as if it's Kanye West with ebola.

3) Whenever motion occurs during a fade-in or fade-out, TFM will again make sensible matches (where two fields display the same image but at different brightness levels) and then replace them with their Doubleweave equivalents. This is not something that I think micmatching can fix, at least not without screwing up the fluidity of the motion.

4) Anything that's natively >30hz but without field order issues, like the live-action/CGI composite shots in Babylon 5, will get butchered.

Thoughts?

[wonkey_monkey]

1) I'd like a better "no-op" than Separatefields().weave(), or Blur(0), or resizing to the current size, or crop(0,0,-0,-0), etc.

What's wrong with "doing nothing"?

A=last

[Katie Boundary]

A=last

I didn't know this was a thing. Thanks! :D

Now, for problems 2-4...

[`Orum]

I was writing an IVTC & decimation filter a while ago to solve this exact problem (which stalled, and I eventually pulled just the decimation part out for a different filter), and what I learned from that is there is no perfect solution depending on how poorly your footage was TCed.

At scene changes you may have up to two orphaned fields (or theoretically worse as there's no limit to how bad TC conversion can be), and to try and deal with this you have a few options (assuming you are leaving audio and duration untouched):

Timecode fudging to try and cover up things around the orphaned fields, resulting in either duplicated frames or stretched durations
Full deinterlacing to generate the missing field(s), anything from a basic duplication of a field to some nnedi-level interpolation
Give up and look for a progressive source of the same material

As in some cases the last wasn't an option and the second makes me recoil in horror, I was trying to do the first only, and mostly succeeded. I'd give you the filter but right now it doesn't handle chroma deinterlacing as I never figured out (https://forum.doom9.org/showthread.php?t=173185) what standard (de jure or de facto) should be used to deinterlace it, or how to tell AviSynth that the chroma planes were progressive vs. interlaced (I suspect you just have to SetFieldBased(false) and lay the chroma planes out in memory in a progressive manner).

That said if you're interested I can go into more detail on the matching process I was using to detect orphaned fields as well as the theory behind it, but it's not going to be a short read. Once you know if a field is orphaned and the pairs for all the other fields, the rest is pretty easy.

[Katie Boundary]

I was writing an IVTC & decimation filter a while ago to solve this exact problem (which stalled, and I eventually pulled just the decimation part out for a different filter), and what I learned from that is there is no perfect solution depending on how poorly your footage was TCed.

...

Once you know if a field is orphaned and the pairs for all the other fields, the rest is pretty easy.

ROFL. You think I'm dealing with ordinary telecine. You poor, sweet, innocent child :rolleyes:

I'm talking about the kind of content in which almost every field is an orphaned field, and the fields themselves are presented in a nonsensical order, so even running the content through a bob-deinterlacer results in jerky back-and-forth motion or strobing effects. Let me see if I can find one of our previous discussions about this kind of content.

EDIT: here: https://forum.doom9.org/showthread.php?t=174198&page=2

[Frank62]

An example clip would be very interesting. Did you upload something somewhere?

[Katie Boundary]

Yes, there are several sample clips posted in the thread that I linked to in my previous post.

EDIT: never mind, it looks like those files have mysteriously evaporated. I'll see what I can do about providing new ones.

[wonkey_monkey]

The polite thing to do would be to link directly to the clip(s) instead of expecting someone to play "find the link" on an old thread. They weren't even on thread page you linked to.

Furthermore, both links (that I could find) are no longer active.

[TheFluff]

hello, is this where I remind you that this is literally the problem yatta was created to solve, op?

[Katie Boundary]

hello, is this where I remind you that this is literally the problem yatta was created to solve, op?

Does YATTA require knowledge of the exact places in the video stream where this problem occurs?

[TheFluff]

Well, kinda. The problem YATTA was created to solve was that automatic IVTC simply isn't clever enough for sufficiently weird hybrid content. It's essentially a GUI for editing TIVTC override files with a bunch of additional bells and whistles - you let TIVTC try to sort things out as well as it can and then you muck around and fix things manually where needed. You can use various metrics to try to detect where manual intervention is needed, but the traditional way is to just look at the video.

Basic features include overriding field matches on a frame-by-frame basis, flagging individual frames or entire sections for postprocessing (with whatever bobber you like), applying match patterns to sections of video, applying filter chains to sections of video, replacing frames with adjacent ones, toggling decimation on/off for sections of video and automatically creating timecode files for creating VFR, etc etc.

It's a pain in the ass to fix things manually but if the source is sufficiently broken it's the only way to get halfway decent results.

[Katie Boundary]

Thanks for the suggestion but you appear to be in the wrong thread:

I'm once again dabbling in possible ways to deal with retrograde field behavior that don't require... knowing every single place in the video stream where this problem occurs

[TheFluff]

I was trying to imply that experience has taught me (well, us - YATTA was already old by the time I got into fixing weird hybrid content) that there is no good way of doing it given that restriction, and that's the reason YATTA exists. Either nuke it with a good bobber like nnedi or fix it manually, and if you chose to nuke it you'll still need to set that up so it doesn't nuke the whole video. IVTC filters are by their nature reliant on detecting patterns to do the right thing, and if those patterns are sufficiently broken there's simply no automated way I know of that can reliably determine the least bad course of action. If you tune it to handle one thing well it'll break or something else. AnimeIVTC was an attempt to do this kind of thing automatically, but it's a huge kludge which doesn't even really work for the tricky content anyway. If you want to examine prior research on the subject you can go check it out though and look at the heuristics it attempts to use.

[`Orum]

ROFL. You think I'm dealing with ordinary telecine. You poor, sweet, innocent child :rolleyes:
I'm not that young, and I have no idea what "ordinary" telecine is, as there's really no way to classify what's what considering how many different correct and broken implementations there are. There's really only "will it work with filter 'x' well or not."

I'm talking about the kind of content in which almost every field is an orphaned field
Stop right there. If this is the case, there's almost no point in even attempting IVTC (assuming one can even consider it IVTC at this point if they are truly orphans and not just way out of order), and you should just get a better source if possible, or cut your losses and use a deinterlacer instead.

[johnmeyer]

'Orum, as you just found out, trying to help Katie is a great demonstration of the old adage, "no good deed goes unpunished." You sincerely try to help, and she gives you a condescending reply.

As to what her video actually looks like, since she once again refuses to upload a sample, we can't really know. However, if "almost every field is an orphaned field," as she says, then perhaps she has telecined material where the fields have been reversed. Field reversal is really common, when converting from HD to SD (or vice versa) because field dominance is typically TFF for HD and BFF for SD. I still see field reversal all the time on the evening news. It is really easy to spot, once you know how it looks.

IMDB claims "Babylon 5" was shot on 35mm film and then converted to video, so it is likely telecined. Depending on what road the video took from broadcast to Katie's computer, it is possible the fields got reversed. If so, "un-reversing" fields is pretty straightforward. Having said that, depending on when this happened relative to any telecine or any attempt at IVTC, things could get pretty screwed up.

In the end, the best course of action is to simply get a better source. I just checked, and four of the five seasons are available for rent, on DVD, at Netflix. If you want to stream it, Amazon has it available for $30/season. That is rather expensive. However, the Netflix DVD option is really cheap: $7.99/month for one DVD out at a time. Their turnaround is really fast, so you could get easily get all four DVDs in less than a month.

[StainlessS]

There are 5 seasons of Babylon 5.
PAL versions are really quite messed up, I've encoded mine twice and need to have yet another go to do em properly/better.
Seaon 1 (I think) has some episodes which get the Aspect ratio messed up.
Some parts are also mixed interlaced and progressive switching back and forth constantly, really nasty.
I hope the NTSC versions are better.

[johnmeyer]

There are 5 seasons of Babylon 5.
PAL versions are really quite messed up, I've encoded mine twice and need to have yet another go to do em properly/better.
Seaon 1 (I think) has some episodes which get the Aspect ratio messed up.
Some parts are also mixed interlaced and progressive switching back and forth constantly, really nasty.
I hope the NTSC versions are better.I remember seeing this in the Chuck Norris "Walker, Texas Ranger" TV series. It was shot on film, but during some of the transition shots you could see it suddenly change to normal 29.97 NTSC video cadence. If you tried to IVTC the video, it would obviously look really bad at those transitions.

However, I have never seen retrograde (back and forth) problems. I cannot imagine that those are actually on the DVD or original broadcast, although as I said above, I do see those problems all the time on new footage.

[`Orum]

However, if "almost every field is an orphaned field," as she says, then perhaps she has telecined material where the fields have been reversed.
Ah, I guess I've always considered orphaned fields as those with no matching field whatsoever, e.g. those usually found at a cut between two scenes.

If it is simply incorrect field order, wouldn't the parity (http://avisynth.nl/index.php/Parity) functions (AssumeTFF/BFF()) suffice to correct the issue?

[Katie Boundary]

I have no idea what "ordinary" telecine is

Telecine is when 24 fps content is slowed down to 23.976 fps, and then either every fourth frame or every fourth field is doubled to bring the frame rate up to 29.97 fps

If this is the case, there's almost no point in even attempting IVTC

Yes, I can see how the phrase "ways to deal with retrograde field behavior that don't require cutting the frame rate below 30 FPS" would make you think that I was trying to achieve a 24 fps frame rate. 30 is less than 24, right?

you should just get a better source if possible

The source isn't the problem. We're talking about content that was designed and edited pathologically from the very beginning, and then transferred with perfect accuracy.

'Orum, as you just found out, trying to help Katie is a great demonstration of the old adage, "no good deed goes unpunished."

http://images5.fanpop.com/image/photos/31900000/begone-rick-the-heroes-of-olympus-31954768-265-190.jpg

since she once again refuses to upload a sample

Wrong. I uploaded several samples when I first noticed this problem. You even saw them at the time and falsely accused me of editing them somehow to make them that way.

IMDB claims "Babylon 5" was shot on 35mm film and then converted to video, so it is likely telecined. Depending on what road the video took from broadcast to Katie's computer...

Wait, what? Why the hell are you talking about Babylon 5 in a thread about retrograde field behavior? B5 has that stupid thing where they separated the fields, rescaled them incorrectly, and then re-wove them, resulting in scanline-like artifacts. We already had two threads about that, with the mystery solved and the solutions found.

In the end, the best course of action is to simply get a better source.

Okay, I'll be sure to do that as soon as Gargoyles, Andromeda, or Robocop: Prime Directives get their visual effects completely redone for a re-release.

However, I have never seen retrograde (back and forth) problems.

Yes, you have. You accused me of adding it to the m2v files.

If it is simply incorrect field order, wouldn't the parity (http://avisynth.nl/index.php/Parity) functions (AssumeTFF/BFF()) suffice to correct the issue?

No. I guess I'll have to dig out my Gargoyles DVDs again. Hold on.

EDIT: here, hav sum Gargoyles! https://www.sendspace.com/file/rhkoto

MOAR EDITZ: here, hav sum Robocop! https://www.sendspace.com/file/99v0xp

[hello_hello]

As to what her video actually looks like, since she once again refuses to upload a sample, we can't really know. However, if "almost every field is an orphaned field," as she says, then perhaps she has telecined material where the fields have been reversed. Field reversal is really common, when converting from HD to SD (or vice versa) because field dominance is typically TFF for HD and BFF for SD. I still see field reversal all the time on the evening news. It is really easy to spot, once you know how it looks.

If I remember correctly from the original samples, there's CGI overlaid on film and for whatever reason it ended up odd in places. As though the CGI was overlaid on progressive video with the fields out of alignment and then it was telecined (I can't remember if it's flagged or encoded telecined) but when applying IVTC the CGI parts of the image would sometimes bounce back and forward a little. I assume TFM would have to (effectively) match the underlying progressive fields and de-interlace any combing in the overlaid CGI, or it'd match up the overlaid CGI fields and de-interlace any combing in the underlying progressive video. Alternatively, is it possible the film was hard telecined before progressive CGI was overlaid at 30fps?
I don't know anything about DVD production, but I don't think it has anything to do with orphaned fields as such. Then again, if the fields of the underlying video and the overlaid CGI don't match, you probably could say every field is an orphan.

At least Katie threads invariably make you smile.
4) Anything that's natively >30hz but without field order issues, like the live-action/CGI composite shots in Babylon 5, will get butchered.
Wait, what? Why the hell are you talking about Babylon 5 in a thread about retrograde field behavior?

BTW, the 1990's called to request you return the last surviving "troll" picture.

Telecine is when 24 fps content is slowed down to 23.976 fps, and then either every fourth frame or every fourth field is doubled to bring the frame rate up to 29.97 fps

Only fields are repeated, not whole frames.

[johnmeyer]

Only fields are repeated, not whole frames.Yes, this is a very common misconception. It is so common that I've actually seen video where people who don't know any better have done this (repeated frames instead of fields). While you get some judder artifacts when you do telecine correctly, if you instead repeat frames, that judder becomes massively worse because the intentional temporal glitches are done half as often, and across the entire frame.

Repeating fields is actually extraordinarily clever and, in the old CRT-based, continuous scanning system, looked amazingly good. 99.99% of the population never thought twice about how film looked when shown on television.

[hello_hello]

EDIT: here, hav sum Gargoyles! https://www.sendspace.com/file/rhkoto

MOAR EDITZ: here, hav sum Robocop! https://www.sendspace.com/file/99v0xp

Maybe I'm missing something, but I don't see a reason for anything but standard IVTC of the Gargoyles sample. You just need to set a low de-interlacing threshold for TFM so it catches what looks like combing during fade-outs/fade-ins.
I'm not sure it is combing as such though. Anyone know what causes that? Is it a chroma subsampling issue when blending interlaced video?
There's lots of dot crawl, which seems to be baked in, and a fair amount of rainbowing, but that's another story.

I could be wrong, but the Lions Gate promo at the beginning of RoboCop looks like interlaced content resized, similar to the Babylon 5 CGI. If everything following it is progressive, this would be an easy way to fix it.

TFM().TDecimate()
Trim(205, 0)

[manono]

Telecine is when 24 fps content is slowed down to 23.976 fps, and then either every fourth frame or every fourth field is doubled to bring the frame rate up to 29.97 fps

Only fields are repeated, not whole frames.
Well, I have seen retail DVDs where every fourth frame is repeated (12344). But that's not telecine. True telecine is, as you mentioned, when fields are repeated. Does it depend on the definition of "telecine"? Is it telecine when frames are repeated? Is it telecine when a film source is field-blended to 29.97fps? Is it telecine when a 25fps PAL source is field-blended to 29.97fps? I say no.

Is it telecine when a 20fps silent film is hard or soft telecined to 29.97fps? Is it telecine when a 25fps PAL source is hard or soft telecined to 29.97fps? I say yes. The Wikipedia (https://en.wikipedia.org/wiki/Telecine) seems to agree.