Deinterlace portions of hybrid video?

[tppytel]

I frequently reencode special features from blurays for my Plex server that are hybrids of telecined film content and video material. In some cases, the only video material appears progressive in VirtualDub - just still pictures with zooms or pans. I know how to reencode those as variable frame rate MKV's using TDecimate to generate a timecodes file. But many of these features mix telecined original film content with true interlaced NTSC video (interviews and such). Is there some way to send just the interlaced portions to QTGMC to deinterlace in order to get a hybrid 23.967/29.97 (or 23.967/59.94) video?

In the past, I've handled these sources by just throwing everything into QTGMC to get 59.97 output, and that's not bad given that it's not critical viewing material. But my two main Plex clients (where I'd generally be watching things like bluray extras) have no problem playing VFR MKV's. So it would be nice for both picture quality and encoding efficiency if I could generate proper hybrid progressive videos from these sources. I suspect avisynth has some mechanism for this kind of branched processing, but I'm not sure what keywords to search.

[hello_hello]

The only problem is I don't think there's a way to automatically de-interlace the video sections to 59.940fps. You're stuck with 29.97fps, but from your description it probably doesn't matter.

Avisource("video.avi")
A=last
B=A.QTGMC().SelectEven()
TFM(clip2=B, Output="D:\TfmMetrics.txt")
TDecimate(hybrid=2, mode=4, Output="D:\TDecMetrics.txt")

That'd be something like what you'd use for the first pass. For testing, and maybe even the first pass, I'd try replacing the QTGMC line with Yadif (cause it's faster) and when in non-VFR mode or after the first pass if you make it black and white it helps to see what's being "de-interlaced".

B=A.Yadif().Greyscale()

This is what the TFM help file says about using external de-interlacing:

Clip2 - .
When using PP > 1 this parameter may be used to specify a clip to take deinterlaced frames from instead of using one of TFM's inbuilt deinterlacing methods. If PP < 5 then frames will be taken as is from clip2, if PP >= 5 (i.e. using motion adaptation) then TFM will build the mask as usual and only pixels in moving areas will be taken from the frames in clip2. Don't specify anything for clip2 to use TFM's internal deinterlacing.

As far as I know the usual TFM options for adjusting combing detection still work when "deinterlacing" from an external clip.



Or you could do something like this if you want 59.94fps for the video sections. The speed of the film sections will be wrong, but that doesn't matter when creating the script. It keeps Avisynth happy. I'd temporarily make the video parts B/W so they're easily identified later, and probably replace QTGMC with Yadif(mode=1) until I was finished. I'm pulling frame numbers out of a hat.

Trim(0,1039).QTGMC().AssumeFPS(60000,1001).GreyScale()\
++Trim(1040,1094).TFM().TDecimate().AssumeFPS(60000,1001)\
++Trim(1095,2478).QTGMC().AssumeFPS(60000,1001).GreyScale()\
++Trim(2479,2958).TFM().TDecimate().AssumeFPS(60000,1001)\
++Trim(2959,3958).QTGMC().AssumeFPS(60000,1001).GreyScale()

And so on. You'd probably need to keep the film sections as multiples of 5 frames so as not to mess with the audio sync after decimating. Once you're done, open the script again and add a second lot of trims, matching only the B/W video frames. Skip the film sections. For the above, you'd end up with something like this (the frame numbers won't be correct as I'm making them up again).

Trim(0, 2078)\
++Trim(2166, 4932)\
++Trim(5506, 7280)

Make the video sections colour again, and before deleting the second lot of trims from the script, use them as the basis for a timecodes file.

# timecode format v1
Assume 23.976024
0, 2078, 59.940060
2166, 4932, 59.940060
5106, 7280, 59.940060

I did something similar recently, although not quite the same, to create some VFR encodes, and the above works like a treat in my head, but I can vouch for the fact it can get tedious. For obtaining the frame numbers I used MeGUI's preview, although I assume AvsPmod displays them too, or add Info() to the top of the script. You don't want to refresh any preview until after you've initially split it into sections though, or add the deinterlacing and decimation after you've split it into sections, otherwise the frame numbers will change.

[tppytel]

Thanks - that's a helpful start. I'll give the first method a try next time I bump into this kind of source, which will probably be soon. The second method is way too much work for this kind of source. Bluray bonus features aren't worth that much work.

[tppytel]

Originally Posted by hello_hello

Avisource("video.avi")
A=last
B=A.Yadif().Greyscale()
TFM(clip2=B, Output="D:\TfmMetrics.txt")
TDecimate(hybrid=2, mode=4, Output="D:\TDecMetrics.txt")

Gave this a try with a hybrid source just now. Yadif doesn't seem to be detecting the interlaced portions consistently - I see the greyscale flicking on and off sporadically during the video segments. But that may just be Yadif's problem. I'll try QTGMC and see if it works any better.

Regardless of Yadif/QTGMC, could someone help me with the more basic avisynth syntax/grammar at work here? I understand the purpose of the call to Greyscale() is to show what Yadif is deinterlacing, but I don't understand why the script works that way. Isn't the dot operator just feeding the output of one function into another? Why doesn't the above script just greyscale everything? I haven't worked with variables in my own scripts so far - I've just used QTGMC(args), Resize(args), etc. on separate lines.

[hello_hello]

Yes, adding GreyScale makes the entire de-interlaced (B) version of the clip B/W whether Yadif catches all the combing or not. It's showing you what TFM is doing, not what Yadif is de-interlacing.
B=A.Yadif().GreyScale()

I think Yadif only de-interlaces the areas of the picture where it detects combing but it doesn't matter. If TFM is configured to replace the entire frame when it detects combing (PP > 1), the entire frame should be black and white regardless of what Yadif is doing. If PP >=5, TFM will only replace the parts of the frame where it detects combing so those areas will be B/W, and you'll no doubt see lots of B/W "flickering". Once again it doesn't matter what Yadif is doing. As long as the combed sections of the picture are B/W they'll be de-interlaced properly when you eventually replace Yadif with QTGMC. If not all the combed areas are B/W then you need to adjust TFM's combing detection, but naturally there'll be B/W areas of the video that don't look combed because Yadif has de-interlaced them. I just use GreyScale now and then as a way to see what will be de-interlaced.... eventually.

Have a read of the TFM help file, specifically the "cthresh" "MI" and "mthresh" options, as they adjust how enthusiastically TFM detects combing... if that's the problem.

Thinking about it (I haven't used TIVTC a great deal for VFR encoding) you can try specifying Display=true for TFM instead of adding GreyScale, or in addition to GreyScale.

Display -
If set to true, information will be drawn on the video frames themselves in the upper left hand corner. This info is not as detailed as the debug output and only includes the current frame number, the values of mode, field, order, and MIC the match that was used for the current frame, and whether or not the output frame was combed if PP > 0. This option can be useful when using overrides and you need to see what match is currently being used. If you need more detailed information, such as the match statistic numbers or to see if specific matches were detected as combed then you'll need to use debug output.

As of v0.9.1 display will also indicate if the final output frame is combed and whether or not it was deinterlaced if PP > 0. If PP = 1 and display is enabled, then TFM will draw a white box around the area of the frame that triggered its detection as combed when a frame is found to be combed through its internal detection (this wont be done if a frame is forced to be combed through overrides).
true = enables
false = disables
Default: false (bool)

[tppytel]

OK, I get it now. Thank you. TFM acts a selector between the two complete streams. So if a frame remains in color, that doesn't necessarily mean it's film rate - it could just be an NTSC frame that has so little motion that combing is undetected and the B stream is never switched in. And if I understand correctly, that in itself isn't a big problem - TDecimate will still see a sequence of 30fps frames without a duplicate telecined frame to remove. So whether TFM grabs the frame from the A or the B stream it will still be marked as a 30fps frame for the VFR encode. Does that sound accurate?

How valuable is it to get consistent TFM matching within a scene? My intuition is that constant switching between the A and B streams within a scene could lead to artifacts, but that setting the thresholds super-low is going to needlessly deinterlace a bunch of frames that wouldn't display noticeable combing in motion and maybe introduce its own artifacts. I tried the clip again with tfm(cthresh=7,MI=60) and that seemed to work fairly consistently. Color would flash in maybe once or twice in an NTSC segment, presumably because of minimal motion in that frame. But TFM mostly decided on either A or B and stuck with it through any particular scene.

[hello_hello]

Originally Posted by tppytel

Does that sound accurate?

It's sound accurate. Although if there's a bunch of duplicate frames in a video sequence and TDecimate does treat them as film, in that respect it doesn't matter because there's no motion, so you'd have either four or five frames over the same period of time but there's no motion for it to adversely effect.
As I said earlier, I haven't worked with TIVTC a huge amount this way (I live in PAL-Land), but there's a few settings that effect TDecimate's decisions when deciding if a sequence of frames is video or film. vidThresh, vidDetect and conCycleTP, although the defaults work pretty well.
Reading the help file, it appears by default it'll effectively assume everything's film and all frames in a sequence have to differ by a certain amount for the sequence to be considered video (vidThresh) and TFMs matching also has to report it as video (vidDetect), otherwise it's assumed to be film. In VFR mode, by default there only needs to be one sequences of video frames in a row for it to be treated as video, but that's also dependant on the vidDetect setting..
Try Display=true for TDecimate. I think it's fairly informative.

Originally Posted by tppytel

How valuable is it to get consistent TFM matching within a scene? My intuition is that constant switching between the A and B streams within a scene could lead to artifacts, but that setting the thresholds super-low is going to needlessly deinterlace a bunch of frames that wouldn't display noticeable combing in motion and maybe introduce its own artifacts. I tried the clip again with tfm(cthresh=7,MI=60) and that seemed to work fairly consistently. Color would flash in maybe once or twice in an NTSC segment, presumably because of minimal motion in that frame. But TFM mostly decided on either A or B and stuck with it through any particular scene.

Once again, not speaking from a massive amount of experience with NTSC.....
If you were going to get TFM to take whole frames from the B clip for deinterlacing, when using QTGMC, which does denoise a little and stabilise the picture as part of what it does, I think ideally you'd want to de-interlace all the frames, otherwise you might end up with a bunch of de-interlaced frames followed by some non-de-interlace frames and possibly be able to see the "quality" change.
I guess using the motion adaptive method you avoid that more but there's also the possibility TFM won't detect all the combing.

There's an option to create an overrides file to tell TFM which ranges of frames to assume are combed and which are not, but it'd be much like adding Trims to a script in that you have to manually find the frame numbers for creating the over-rides file.

I've only used TFM for VFR encoding a few times and the last few hybrid sources I've encoded must have been something I cared about enough to do it the hard way with Trims so I could make the video sections 59.940fps, and TIVTC is designed for NTSC whereas for PAL it's more temping to de-interlace the whole thing to 50fps, as QTGMC can stabilise the film sections anyway and you just end up with each film frame repeated.

Maybe someone else with more NTSC experience will come along, but that's some thoughts on it, at least.