TDeint and TIVTC

[flossy_cake]

imo the combing detection is the real weak point with TFM and TDeint. I've tried DG's isCombed (qCombed) as well and that is not reliable enough either. It needs per-scene threshold tweaks to work reliably. Even on one scene I raised the threshold to 50 and it was still declaring the whole frame as combed. I need a better metric, and it has to be fast for real time use.

Or maybe I need to come up with a whole new strategy like a new type of mask with some hysteresis characteristic. Or a mod of IsCombedTIVTC which assigns each 16x16 block a boolean isCombed and only deinterlaces blocks with true.

[flossy_cake]

I'm finding TDecimate's mode2 seems to be doing a better job at managing bad edits and orphaned fields in problematic 3:2 content. In mode2 we don't specify cycle & cycleR but instead just give it a frame rate value like 23.976 and it uses an alternate algorithm to decide which frames to remove. Whereas with mode0 & mode1 I was finding there would be some scenes where there would be 2 candidate frames for decimation within a cycle of 5 frames, and it would pick the wrong one. Not necessarily by virtue of bad logical reasoning, as in one case it picked the one with the lower framediff value, but it just so happened the frame with the slightly lower framediff value was the real frame and the other was the duplicate. It's a tricky situation.

[tormento]

And what about duration? Does all that lead to audio desync? I could be really hard to fix it per scene based.

[flossy_cake]

And what about duration? Does all that lead to audio desync?

I don't see any reason why it would, since 60 frames are going in & 60 frames are coming out. Nothing is getting re-timed, it's just doing field matching and deint on the 60 frame input.

For some reason TFM needs micmatching=4/5 to work when the input framerate is duplicated from 30 to 60, and I'm not sure why. I don't even know what micmatching is & couldn't find any documentation explaining it. It looks like something to do with the field matching process -- if anyone knows the details please enlighten me.

[flossy_cake]

I think that is why its best to use

d2v = "..."
Mpeg2Source(d2v)
TFM(d2v=d2v,...) # maybe PP = 0 for no post processing


It seems the clip already contains the necessary flags to handle it with ScriptClip.

Test clip (laundry sequence, soft telecine with bad edits on scene change): https://drive.google.com/file/d/1dwa7gB3HHuSPXroA2ifm4LtLX0fSs2bM/view?usp=share_link


file = "soft telecine.mkv"

LWlibavVideoSource(file, repeat=false)

ScriptClip(last,
\ """

fieldbased = propGetAny("_FieldBased")
parity = GetParity(current_frame)

if (!IsInt(fieldbased)){fieldbased="undefined"}
else if (fieldbased==0){fieldbased = "progressive"}
else if (fieldbased==1){fieldbased = "bottom field first"}
else if (fieldbased==2){fieldbased = "top field first"}

if (!IsBool(parity)){parity="undefined"}
else if (parity==true){parity = "top field first"}
else if (parity==false){parity = "bottom field first"}

SubTitle("_FieldBased: " + fieldbased + "\n" + "GetParity: " + parity, font="courier new", size=24, lsp=10)

\ """)

LanczosResize(720, 540)


Notably the _FieldBased frame property stays "progressive" the whole time, except for those combed frames on scene change.

It seems LWLibavVideoSource is the only source filter which actually sets the _FieldBased and parity values for each frame.

But it should be a trivial matter of returning a deinterlaced clip when _FieldBased != progressive.

Hard telecined is still problematic but I'm already getting ideas :devil:
edit: it looks like hard telecined doesn't contain any information to detect orphaned fields -- both _FieldBased and Parity remain locked at the same value for all frames.

[flossy_cake]

Here's an updated version which handles the combing detection issue with a latching system where if a scene contains 4 or more combed frames, the remainder of the scene is treated as combed until the next scene at which point the latch is reset.



# Convert 30i to 60p
# 3:2 sections IVTC'd to 24p and 3:2'd
# 2:2 sections weaved to 30p and 2:2'd
# 1:1 sections deinterlaced to 1:1


global isCombed = false
global combedCount = 0
global yDiff = 0
global sceneThresh = 35
global lastScene = 0
global lastSceneYDiff = 0
global lastFrameNum = 0
global latchDeint = false


file = "https://drive.google.com/file/d/1KdE0np8016zr7ovRassdbH7OtbKU7e-a/view?usp=sharing"
global orig30 = LWLibavVideoSource(source=file, stream_index=-1, repeat=true, cache=false)
global bwdif60 = bwdif(orig30, field=3, thr=2)
global tfm24as60 = TFM(orig60, mode=5, PP=0, cthresh=9, MI=80, scthresh=100,
\ slow=2, ubsco=false, mmsco=false, micmatching=4, display=false)

Select(tfm24as60)
LanczosResize(720,540)


function Select(clip c)
{
c = ScriptClip(c,
\ """

global yDiff = YDifferenceToNext(tfm24as60, -2)
global isCombed = IsCombedTIVTC(tfm24as60, cthresh=9, MI=64)

if (yDiff > sceneThresh || current_frame > lastFrameNum+2 || current_frame < lastFrameNum-2 ){
global latchDeint = false
global lastScene = current_frame
global lastSceneYDiff = yDiff
global combedCount = 0
}

if (isCombed){
global combedCount = combedCount + 1
}

if (!latchDeint && isCombed && combedCount >= 3){
global latchDeint = true
}

global lastFrameNum = current_frame

debug = "frame: " + String(current_frame)
\ + "\n" + "yDiff: " + String(yDiff)
\ + "\n" + "isCombed: " + String(isCombed) + "\n" + "combedCount: " + String(combedCount)
\ + "\n" + "latchDeint: " + String(latchDeint) + "\n" + "lastScene: " + String(lastScene)
\ + " (" + String(lastSceneYDiff) + ")"

if (latchDeint || isCombed) {
bwdif60.SubTitle("BWDIF \n" + debug, font="courier new", size=24, lsp=10)
}
else {
tfm24as60.SubTitle("TFM \n" + debug, font="courier new", size=24, lsp=10)
}


\ """ , after_frame=true)

return c
}



edit: I'll be reworking this considerably over the next few days. I'll try to make something more generic as well so it should work for any 25i or 30i file, hopefully it can function as a "one size fits all" general purpose deinterlacer.

[SaurusX]

I've mentioned it before, but a feature of TFM I'd love to see is a "pattern bias" feature. It would attempt to keep to a defined telecine pattern as it compares the combing metrics for each field combination. The pattern would be adhered to if the matches pass the deinterlacing test even if another field combo has a slightly better metric. I'm hoping this would make field matching for shaky animation telecine transfers better and not have those random chunky frames where the two matched fields are basically identical.

[flossy_cake]

and not have those random chunky frames where the two matched fields are basically identical.

That sounds like an issue I'm having when using bwdif on 3:2 sections. For some reason it generates a weird looking aliased frame every now and then, as if it had just copied field 1 into field 2 or something. Here are 2 consecutive frames coming out of bwdif:

https://i.ibb.co/pnRHYjy/2-bad.png
https://i.ibb.co/68MdYtP/1-good.png

The second one is how an interpolated field should look. The first one is too aliased for some reason. Now here are the same 2 frames coming out of TDeint:

https://i.ibb.co/3Fxdqk9/3-good.png
https://i.ibb.co/YD2ZBPx/4-good.png

Now that looks better, just interpolation as I would expect to see. So perhaps bwdif is only good for 1:1 content? When I think about it, 3:2 is kind of a strange beast to deinterlace what with the pattern of 2 combed frames followed by 3 progressive frames. I'm sure that is somehow making things confusing for the deinterlacer.

edit: seems to be an issue with LWLibavVideoSource, as the issue goes away when using FFmpegsource2 with the same clip. Hard telecined clips works fine with both though.

edit2: false alarm. The issue appears to be with the clipping software that I used to generate the short clips from the full 22 minute episode (XMediaRecode, which uses ffmpeg). It must have mucked up the repeat field flags during the clipping process, because everything is working fine with the full 22 minute clip and both LWLibav & FFMpegSource2, except for the latter with rffmode=1, which causes the issue to occur again. Also the full episode shows more consistent frame rates and timecodes, in relation to:


LWLibavVideoSource("D:\soft telecine.lwi", repeat=true)
29.97fps, 317 frames, 10.577 sec

LWLibavVideoSource("D:\soft telecine.lwi")
29.97fps, 256 frames, 8.541 sec

FFVideoSource("E:\soft telecine.mkv", cachefile="D:\soft telecine.ffindex", threads=1, RFFMode=1)
29.97fps, 318 frames, 10.610 sec

FFVideoSource("E:\soft telecine.mkv", cachefile="D:\soft telecine.ffindex", threads=1)
24.3789fps, 256 frames, 10.5 sec

The frame rate for the second ffms2 example is usually the indicator there's repeat flags being ignored, because there's hard telecined or video sections too, making the average frame rate non standard. It'll also mess with the audio sync. I don't know why lsmash didn't output the same frame rate.

Sorry about that, I guess we can't use short clips to debug this kind of issue, which is really annoying.

edit3: forgot to mention, must use the latest version of BWDIF which contains some fixes, and use it like this:

propDelete("_FieldBased") # latest version bwdif requires this
BWDIF(field=-2) #-2 = auto detect field order, which is necessary to avoid above aliasing problem

[flossy_cake]

I've mentioned it before, but a feature of TFM I'd love to see is a "pattern bias" feature. It would attempt to keep to a defined telecine pattern as it compares the combing metrics for each field combination.

Are you sure it's not TDecimate causing that? TDecimate has [mode2, rate, maxndl] params which should do what you are after. With some of my videos I need to use this configuration to get a consistent decimation pattern.

[SaurusX]

Are you sure it's not TDecimate causing that? TDecimate has [mode2, rate, maxndl] params which should do what you are after. With some of my videos I need to use this configuration to get a consistent decimation pattern.

Pretty sure. TFM is the one combining the fields into a full frame and it's the one giving a string of frames for TDecimate to choose from. That combined frame is the problem since two essentially identical stacked fields will of course have a lower combing metric.

[hello_hello]

After having another play, I'm still at a loss as to why field matchers/deinterlacers think the progressive frames need deinterlacing and why they make such a mess of it when they do, but for re-encoding.....

What seems to work for the sample at least, is to use much less enthusiastic field matching than TFM's defaults and to back off the combing detection quite a lot. The easiest way to do the former is to use TDeint instead of TFM, and they both have the MI argument.

This fails to find a match for the orphan fields around the scene changes so it de-interlaces them instead, which IMHO is better than duplicate frames.

TDeint(MI=200, TryWeave=true)

Trouble is, there's somehow aliasing built in anyway, and even denoising seems to make it worse, and anti-aliasing tends to blur so I'd sharpen to counter-act it, and of course I'd also be keen not to sharpen noise much....

Both screenshots are the script output upscaled to 1080p by MPC-HC.

FFVideoSource("E:\soft telecine.mkv", FPSNum=24000, FPSDen=1001, threads=1)
CropResize(0,0, 9,0,-7,0, InDAR=15.0/11.0, ResizeWO=true)

https://i.ibb.co/N9Hhd4S/Cropping-and-resizing-only.png

FFVideoSource("E:\soft telecine.mkv", FPSNum=24000, FPSDen=1001, threads=1)
TDeint(MI=200, TryWeave=true)
MCDAA3()
MDegrainNL(Tr=3, thSAD=300)
LSFMod()
CropResize(0,0, 9,0,-7,0, InDAR=15.0/11.0, ResizeWO=true)

https://i.ibb.co/f2f6SMZ/Filtered-cropped-and-resized.png

soft telecine encode.mkv (https://files.videohelp.com/u/210984/soft%20telecine%20encode.mkv) (5.1MB)

[flossy_cake]

After having another play, I'm still at a loss as to why field matchers/deinterlacers think the progressive frames need deinterlacing

Actually Asd-g is changing all that and making it so that for yadif & bwdif, if frame flag _FieldBased=0 (progressive) then it won't deinterlace and throws an error. I'm trying to convince them here (https://github.com/Asd-g/AviSynth-BWDIF/issues/4#issuecomment-1326042092) to just passthrough the frame instead of throwing error.

This fails to find a match for the orphan fields around the scene changes so it de-interlaces them instead, which IMHO is better than duplicate frames.

Yep, and that is how my Sony DVD player handles it when playing the disc as well, which I find ideal. LWlibavVideoSource actually flags those frames as _FieldBased=2 so it's trivial to do a ScriptClip to switch to bwdif/tdeint etc. when _FieldBased=2.

Also I edited one of my previous comments to mention that the short clips I've been cutting appear to have mucked up metadata in the process of clipping them from the full episode, so I wouldn't necessarily trust the result on those short clips. For instance bwdif was producing some unexpected aliasing but TDeint was not, and I never got to the bottom of it apart from finding that both performed as expected on the full episode but not the short clip.

Anyway, I can see the first screenshot of yours is full resolution progressive frame, and the second one appears blurred on the white rims of the blue cups. I'm not sure if it's weaving or not (I'm guessing it should since you set the MI so high?) but it still looks rather clean and nice -- looks like you are doing antialiasing, denoise & sharpen. For sharpening I'm using MadVR's options but I'm definitely keen on using an antialiasing filter for some of my content so I'll check out MCDAA3 ... just hope it's fast enough for real time.

[hello_hello]

Actually Asd-g is changing all that and making it so that for yadif & bwdif, if frame flag _FieldBased=0 (progressive) then it won't deinterlace and throws an error. I'm trying to convince them here (https://github.com/Asd-g/AviSynth-BWDIF/issues/4#issuecomment-1326042092) to just passthrough the frame instead of throwing error.

I suspect, given they're fairly new, frame properties will cause a lot of pain for a while.

A friend PM'd me last week wanting to know why some of his scripts were being encoded as interlaced without him changing the encoder command line. I assumed it was operator error until I remembered he likes to pipe scripts to ffmpeg and it supports frame properties now. It turned out the source filter was setting _FieldBased=1 & he was mostly deinterlacing with bwdif and it set _FieldBased=0, but when he used anything else to de-interlace.....

I'm not an expert on this, but is there any way a source filter can know if a video is really progressive unless it's something like an NTSC DVD with repeat flags for applying telecine? If there's no repeat flags I assume it could be interlaced, progressive or hard telecined, and if it's combinations of each then it's probably all encoded as interlaced anyway.

Yep, and that is how my Sony DVD player handles it when playing the disc as well, which I find ideal. LWlibavVideoSource actually flags those frames as _FieldBased=2 so it's trivial to do a ScriptClip to switch to bwdif/tdeint etc. when _FieldBased=2.

In your case there's orphaned fields so you want to treat them as interlaced, but is _FieldBased=2 just a safe assumption that could be incorrect?
As far as I know a progressive scan player would output either 23.976 or 59.97 etc, but without repeat flags it has to physically check for repeated fields and combing as plugins do and choose to de-interlace or apply IVTC on the fly.

Also I edited one of my previous comments to mention that the short clips I've been cutting appear to have mucked up metadata in the process of clipping them from the full episode, so I wouldn't necessarily trust the result on those short clips. For instance bwdif was producing some unexpected aliasing but TDeint was not, and I never got to the bottom of it apart from finding that both performed as expected on the full episode but not the short clip.

I did see a mention of bwdif crapping out at the beginning of the sample, and I did notice the first three frames had the same timecodes, Maybe that had something to do with it, but I think the reason I used FPSNum=24000, FPSDen=1001 instead of repeat=false is the former dropped two of them so there wasn't a pause for a second at the beginning.

If you use rffmode=1, the output would be 60i and all fields would need to be set to interlaced, even for the soft telecined sections. It's then up to the de-interlacer/field matcher to look at them and decide if they're video or film and handle them accordingly.
If rffmode=0, in a perfect world the film sections would be output as progressive and could be marked as progressive, but any sections without repeat flags would still need to be checked. If there's hard telecine I don't think bwdif is the right tool for the job, but I can't imagine why it should throw an error if _FieldBased=0.

Well here's a fun fact for you. I'm going to bang my head against the desk very hard shorty to punish myself, because I think your sample is actually soft telecine from start to finish. If you use rffmode=0 with ffms2 there's no orphaned fields on the scene changes. My guess is it's outputting 24.379fps because without FPSNum=24000, FPSDen=1001 it operates in variable frame rate mode and outputs the average frame rate, and those three frames with the same timecodes at the beginning have fudged the average.
It looks like it's rffmode=1 shuffling fields around scene changes, maybe because of the first three frames, but &%$@ me! How did I not notice that until now??!?

In my defence I did remux the MKV as a TS file to index it with DGIndex, which probably created more problems, and I think LSmash was decoding it differently again, but still.... &%$@ me!

Anyway, I can see the first screenshot of yours is full resolution progressive frame, and the second one appears blurred on the white rims of the blue cups. I'm not sure if it's weaving or not (I'm guessing it should since you set the MI so high?) but it still looks rather clean and nice -- looks like you are doing antialiasing, denoise & sharpen. For sharpening I'm using MadVR's options but I'm definitely keen on using an antialiasing filter for some of my content so I'll check out MCDAA3 ... just hope it's fast enough for real time.

The blurring of the cups would have been MCDAA3 cleaning up the all the wobble (alaising) on the cup rims, but it probably doesn't really exist. I think that's one of real.finder's functions.

[flossy_cake]

In your case there's orphaned fields so you want to treat them as interlaced, but is _FieldBased=2 just a safe assumption that could be incorrect?

I suppose it depends how reliable LWLibavVideoSource is at setting _FieldBased based on whatever metadata/flags are inside the MPEG2 stream. To be safe I'll probably do a check for _FieldBased and an additional IsCombedTIVTC check, although the latter should be sufficient most of the time.



Well here's a fun fact for you. I'm going to bang my head against the desk very hard shorty to punish myself, because I think your sample is actually soft telecine from start to finish. If you use rffmode=0 with ffms2 there's no orphaned fields on the scene changes. My guess is it's outputting 24.379fps because without FPSNum=24000, FPSDen=1001 it operates in variable frame rate mode and outputs the average frame rate, and those three frames with the same timecodes at the beginning have fudged the average. It looks like it's rffmode=1 shuffling fields around scene changes, maybe because of the first three frames, but &%$@ me! How did I not notice that until now??!?

Well, on my system with ffmpegsource2(rffmode=0) I'm still getting the orphaned field on scene change

https://i.lensdump.com/i/RRX1XC.png

Adding (fpsnum=24000, fpsden=1001) has no effect, except playback stutters for a few seconds every now and again, and the short clip isn't long enough to show it. I'm kind of at the point of giving up trying to understand what's going on under the hood with soft telecine. Although I came across this (good luck understanding it)


Some quotes from ISO 13818-2
note there are TWO different "progressive" flags

top_field_first -- The meaning of this element depends upon picture_structure, progressive_sequence and repeat_first_field.
If progressive_sequence is equal to ‘0’, this flag indicates what field of a reconstructed frame is output first by the decoding process:
In a field picture top_field_first shall have the value ‘0’, and the only field output by the decoding process is the decoded field picture.
In a frame picture top_field_first being set to ‘1’ indicates that the top field of the reconstructed frame is the first field output by the decoding process. top_field_first being set to ‘0’ indicates that the bottom field of the reconstructed frame is the first field output by decoding process

If progressive_sequence is equal to 1, this flag, combined with repeat_first_field, indicates how many times (one, two or three) the reconstructed frame is output by the decoding process.
If repeat_first_field is set to 0, top_field_first shall be set to ‘0’. In this case the output of the decoding process corresponding to this reconstructed frame consists of one progressive frame.
If top_field_first is set to 0 and repeat_first_field is set to ‘1’, the output of the decoding process corresponding to this reconstructed frame consists of two identical progressive frames.
If top_field_first is set to 1 and repeat_first_field is set to ‘1’, the output of the decoding process corresponding to this reconstructed frame consists of three identical progressive frames.

repeat_first_field -- This flag is applicable only in a frame picture, in a field picture it shall be set to zero and does not affect the decoding process.
If progressive_sequence is equal to 0 and progressive_frame is equal to 0, repeat_first_field shall be zero, and the output of the decoding process corresponding to this reconstructed frame consists of two fields.
If progressive_sequence is equal to 0 and progressive_frame is equal to 1:
If this flag is set to 0, the output of the decoding process corresponding to this reconstructed frame consists of two fields. The first field (top or bottom field as identified by top_field_first) is followed by the other field.
If it is set to 1, the output of the decoding process corresponding to this reconstructed frame consists of three fields. The first field (top or bottom field as identified by top_field_first) is followed by the other field, then the first field is repeated.
If progressive_sequence is equal to 1:
If this flag is set to 0, the output of the decoding process corresponding to this reconstructed frame consists of one frame.
If it is set to 1, the output of the decoding process corresponding to this reconstructed frame consists of two or three frames, depending on the value of top_field_first.

https://forum.videohelp.com/threads/163723-program-sequence-flag-and-program-frames-flag-always-same#post806241



As far as I know a progressive scan player would output either 23.976 or 59.97 etc, but without repeat flags it has to physically check for repeated fields and combing as plugins do and choose to de-interlace or apply IVTC on the fly.

I suspect my Sony DVD player is not analysing pixels for combing because the earlier seasons of that show are hard telecined and the Sony does mocomp deint for those (anything that moves drops to 240p). Therefore, I'm inferring that if it was obeying the repeat flags for the soft telecined seasons, then internally it would have 60i and do mocomp deint as well, but instead it outputs all frames progressively at 1080p60 and deints only those frames with orphan fields (I've stepped through frame-by-frame to observe those frames drop to 240p).

[hello_hello]

You're correct, there's still orphan fields. I was tired and too quick to get excited.

I've been seeing this on scene changes with rffmode=1, and when it went away with rffmode=0, I incorrectly assumed there'd no longer be orphans.

https://i.ibb.co/Lxw5MQ9/Clipboard01.jpg

Converting the frame rate with fpsnum=24000, fpsden=1001 doesn't seem to do much, but the frame count drops from 256 to 252. Naturally the frame rate changes from 24.379 to 23.976. Hopefully the glitches are only little pauses in decoding and it'll playback smoothly once it's encoded.... ummmmm sorry..... I'm used to that being the end goal... but if it's pure soft telecine rffmode=0 should give you 23.976fps and there'd be no need for ffms2 to do any frame rate conversion.

I suspect my Sony DVD player is not analysing pixels for combing because the earlier seasons of that show are hard telecined and the Sony does mocomp deint for those (anything that moves drops to 240p).

Thinking about it, maybe players do just interpolate "interlaced" fields into full frames, whether it's a hard telecined field or a video field. If not it'd have to field match first, and if it did that before weaving fields together it'd probably have to check the weaved frames for combing. I'm just guessing though as I have no idea what they do.

What about decoding this way?
Rather than obey the repeat flags, FFMS2 should decode the soft telecine sections as film while duplicating the progressive frames for 29.97fps.

Hopefully TryWeave will handle hard telecine, with TDecimate() removing the duplicates in the film sections, and if there's any video sections hiding amongst the film (opening or closing credits maybe), it'll be converted to 23.976fps using frame blending.

FFVideoSource("D:\soft telecine.mkv", threads=1, rffmode=0, fpsnum=30000, fpsden=1001)
A = last
TDeint(MI=200, TryWeave=true, clip2=A)
TDecimate(Hybrid=1)

[flossy_cake]

It occurred to me that with 3:2 stuff containing orphaned fields on scene changes, it is logically/mathematically impossible to pull out a perfectly clean 24p. Because the source doesn't actually contain 24p, it contains 24p plus the occasional extra field here and there. This offsets the timing slightly such that over time those extra fields add up and would cause the video to go out of sync. So the only way to TFM such content is to use a final output mode of 60p -- only then we have the necessary time slice granularity to represent all images in the source without video going out of sync. Another option is to do frame blending of the orphaned field and its neighbouring 24p frame. This might be my preferred method as it avoids 3:2 judder of 60p, but I couldn't get it to work as it's implemented by TDecimate which comes after TFM, and TFM is making the decisions about orphaned field handling before TDecimate gets a chance at it.

[poisondeathray]

It occurred to me that with 3:2 stuff containing orphaned fields on scene changes, it is logically/mathematically impossible to pull out a perfectly clean 24p. Because the source doesn't actually contain 24p, it contains 24p plus the occasional extra field here and there. This offsets the timing slightly such that over time those extra fields add up and would cause the video to go out of sync. So the only way to TFM such content is to use a final output mode of 60p -- only then we have the necessary time slice granularity to represent all images in the source without video going out of sync. Another option is to do frame blending of the orphaned field and its neighbouring 24p frame. This might be my preferred method as it avoids 3:2 judder of 60p, but I couldn't get it to work as it's implemented by TDecimate which comes after TFM, and TFM is making the decisions about orphaned field handling before TDecimate gets a chance at it.


It should be in sync - and I've never seen any those types of film (or film equivalent) based NTSC DVD's that are not because of those types of cadence interruption edits after IVTC

The "granularity" of 59.94p is potentially "finer" than 23.976p, but the running time is the same (same length as audio), and so the sync is pretty much the same (maybe a few ms difference, less than a frame duration, so not out of sync to human eyes/ears) . That interpolated 59.94p from film content actually has 3:2 duplicates, so the "granularity" isn't really better in that case (in terms of motion , or unique frames)

When you perform "normal" IVTC, the frame count @23.976p is the same whether or not you have a a) duplicate at that spot, b) or a blend, or c) a deinterlaced frame, so the timing is unchanged, sync is unchanged

The deinterlaced frame (a frame from an interpolated single field) is the best option in most cases (if a good algorithm is used) because is is an actual unique frame. It best represents what actually should have been there. Blends look bad, and duplicates aren't ideal either. Some cases where deinterlace might not be as good is if the remaining orphaned field is of noticable lower quality (e.g. at the end of a GOP, sometimes the bitrate allocation for the single field is lower than normal, it can be full of encoding artifacts - if you can't selectively clean them up, then a duplicate might look better)

[flossy_cake]

Sorry I should have been clearer - a sync issue would only arise if we tried to represent every unique image from the 3:2 source in the 24p output without any duplicates, skipped frames or blended frames. The point I'm trying to make is that the source isn't really genuine 24p, it's in a way "hard coded" to 60hz if you want to see every unique image in the source as it was intended to be seen. eg. if the first frame of a new scene is combed, and only contains image data from that scene, then field 1 is an orphan field and can't be matched to anything by TFM so it ends up getting removed in the subsequent TDecimate. So the first image from that new scene is actually lost, whereas if the final output is 60p, that orphaned field could have the opportunity to get deinterlaced and shown in the output.

[flossy_cake]

I'm kind of at the point of giving up trying to understand what's going on under the hood with soft telecine.

MPC Video Renderer's ctrl+J debug screen seems to show the frame rate changes in real time. MadVR doesn't show this and lead me into all kinds of wrong assumptions. With this clip (https://rationalqm.us/misc/lainvob.vob) for instance, MPC Video Renderer shows the frame rate actually changes between 29.97 and 23.976 dynamically depending on the scene. Sometimes it even changes between 29.97 and 29.97i as if to imply there are some frames flagged as interlaced. On top of this, the field order also changes dynamically from scene to scene, sometimes rapidly back and forth each frame! I wish I could just force it all to 29.97i TFF so I have a known quantity which I could then use TFM & TDecimate on like with normal hard telecined stuff. I've got a feeling the flags in the clip were also damaged by the software which clipped it (probably ffmpeg) as even using Bob() on it was resulting in wrong field order artefacts regardless of what field order I set. Or maybe that's an issue with the source filter, I don't really know.

[flossy_cake]

2:2-centric content with the occasional 1:1 scene, eg. The Office, Garth Marenghi's Darkplace

Had a look at these discs on the Sony DVD player and to my surprise it's quite bad at handling it -- the 1:1 sections are combed until there is a very large amount of motion, at which point the DVD player realises it's 1:1 and switches to mocomp deint. Then on switchback to 2:2 there is something different about the image quality, can't quite understand it. Some kind of really mild combing artefact that is almost imperceptible. It sort of looks progressive but then sort of doesn't? Can't describe it, but it's not the same clarity as the 2:2 section before the 1:1 section.

Anyway, I thought this was remarkable as the same DVD player has no trouble identifying those orphan fields from NTSC 3:2 soft telecine. I guess for NTSC content it's looking at the repeat field flags, whereas with PAL content there generally doesn't seem to be any such flags so it must be relying on trying to detect combing in the image itself (much like IsCombedTIVTC but way worse accuracy... or thresholds have deliberately been set high to maximise progressiveness on 2:2).

[Genji]

In TDecimate, if specify very large numbers for cycleR and cycle (ex. cycleR=7779,cycle=38895), cannot see the result on the screen.
(To begin with, setting display=true in this case will force AvsPmod to close.)

Is it possible to split the result display on the screen only, while leaving the cycleR and cycle specifications as they are?
For example, even if cycleR=7779,cycle=38895 is specified, if the screen display can show 250 frames at a time, etc., can check and make progress.

[kedautinh12]

You can create issue here
https://github.com/pinterf/TIVTC/issues

[pinterf]

Released TIVTC v1.0.28 (20231210) & TDeInt v1.9 (20231210)
https://github.com/pinterf/TIVTC/releases/tag/v1.0.28

For more details read the relevant documentations included in the packages.

TIVTC:
## TIVTC

**v1.0.28 (20231210)**
- Request #43: (https://github.com/pinterf/TIVTC/issues/43)
TDecimate debug parameters displayDecimation, displayOpt.
Able to show << or >> instead of ** on debug display by the distance from the last decimated frame compared to displayOpt.
- Fix minor display glitch (regression since v1.0.19pack): display=true would duplicate the
most bottom frame info text to the top position of the next column.

**v1.0.27 (20230511)**

- Fix #40: TDecimate: frame properties were not inherited at specific modes (20230511)
- Implement #39: CFrameDiff, FrameDiff: add offset parameter (int, default 1), to compare frames more than 1 distance away.
offset value must be >=1, already existing prevf parameter is used to set the direction.

**v1.0.27test (20220915)**
(bugfixes, frame property support backport from VapourSynth version -20220915)
- Fix: TDecimate mode 0,1 crash in 10+bits in blend (dubhater)
- Fixes in Mode 0,1 when clip2 is different format (dubhater)
- Fix: slow C code was used in calcMetricCycle.blurframe (dubhater)
- Fix: V14(?) regression TDecimate fullInfo was always false (dubhater), ("Don't know what it affected")
- MacOS build fixes (akarin)
- mingw build fixes
- Source code: refactorings, backported from VapourSynth port (dubhater, https://github.com/dubhater/vapoursynth-tivtc),
- Source code: backport other simplification changes.
- Frame properties (priority over burned-into-image hints). Similarly to bitmapped hints, they are used when parameter "hint" is true.
Frame properties set by TFM: TFMMatch, _Combed, TFMD2VFilm, TFMField, TFMMics, TFMPP
Frame properties read by TFM and TDecimate: TFMMatch, TFMD2VFilm, TFMField
Frame properties read by TFMPP: TFMField, _Combed
Frame properties read by TDecimate creation: TFMPP (but if Avisynth variable "TFMPPValue" exists then the variable has priority over it)
Frame properties never read: TFMMics
Traditionally, TFM is using a 32 bit "hint" encoded in the lsb bits of the first 32 pixels of a frame.
When Avisynth interface V8 is available, then frame properties are used instead of the image destructing bit hints.
- Bitmapped hint replacement frame property names are: TFMMatch, _Combed, TFMD2VFilm, TFMField.
Mapping:
Bits 0-2: (ISP, ISC, ISN, ISB, ISU, ISDB, ISDT) are filled upon match (TFMMatch) and combed (_Combed) and field (TFMField)
ISP: match == 0
ISC: match == 1 && combed < 2
ISN: match == 2
ISB: match == 3
ISU: match == 4
ISDB: match == 1 && combed > 1 && field == 0
ISDT: match == 1 && combed > 1 && field == 1
Bit 3: set to 1 if field == 1 (TFMField 0 or 1)
Bit 4: set to 1 if combed > 1 (_Combed)
Bit 5: set to 1 if d2vfilm is true (TFMD2VFilm)
- Other frame properties:
TFMMics: array[6] of integer
TFMPP: integer
Frame properties set by TDecimate:
TDecimateCycleStart (not used)
TDecimateCycleMaxBlockDiff uint64_t[] (not used)
TDecimateOriginalFrame (not used)
_DurationNum, _DurationDen (not used yet, for variable frame rate (?), backport from VapourSynth port is in progress)

TDeint

## TDeint
**v1.9 (20231209) - pinterf**

- Fix C reference code of YUY2+luma-only: check_combing_c_Metric1 metrics
Probably no one had problems with it on Intel, since it's implemented in SSE2 as well.

[hello_hello]

Released TIVTC v1.0.28 (20231210) & TDeInt v1.9 (20231210)
https://github.com/pinterf/TIVTC/releases/tag/v1.0.28

Thanks!

[FranceBB]

Thank you as always, Ferenc!
I actually followed the whole discussion on GitHub that led to this build (until you closed the issue) as I was getting the notifications via email (although I didn't reply) and it's highly appreciated. ;)
I just swapped it on prem in prod the other day on Windows Server 2019, while on my PC I can say that it works just fine on XP.
It will also be included in the upcoming FFAStrans 1.4.0 due to be released on Christmas Day. :)

[pinterf]

Ikotas has published a nice detailed guide for IVTC from 29.97fps to 25fps utilizing TDecimate.
Activation emails are kinda slowish here at doom9, so I was asked to help in publishing.

Here it is (in Japanese)
https://ikotas.github.io/25fpsIVTCGuide/

And in English - easy-click-version - for those without auto-translate:
https://ikotas-github-io.translate.goog/25fpsIVTCGuide/?_x_tr_sl=auto&_x_tr_tl=en&_x_tr_pto=wapp

Thanks to Ikotas for taking the time to prepare this description.

[Micheal813]

I have some mpg video that is mostly hard telecined to play at 29.97 fps. There are some small sections that are just progressive 29.97 fps. I index it with DGIndex and it shows 100% video.

If I use this script and go frame by frame in VDub, some sections show pattern AABBCCDD (Progressive?) and some show AABBBCCDDD(Hard Telecine?).

mpeg2source("path\to\video.mpg")
AssumeTFF()
SeparateFields()


I'm using this script for the final product:

mpeg2source("path\to\video.d2v")
TFM(d2v="path\to\video.d2v")
TDecimate()

Is this correct? It seems to work and convert everything to 23.976, but just wanted to check.

Thanks

[FranceBB]

I have some mpg video that is mostly hard telecined to play at 29.97 fps. There are some small sections that are just progressive 29.97 fps.

You have two paths in front of you:

Option A) You deinterlace everything, so that the telecined parts will be duplicated to 29.970p but you'll save the progressive ones

Option B) You IVTC everything to 23.976p so that the telecined parts will be 23.976 and the progressive parts will be decimated to 23.976p


I'm using this script for the final product:

mpeg2source("path\to\video.d2v")
TFM(d2v="path\to\video.d2v")
TDecimate()

Is this correct? It seems to work and convert everything to 23.976, but just wanted to check.

Yes, that's option B, it will IVTC the telecined parts and decimate the progressive ones so that you're gonna get 23.976p in the end.

To be absolutely fair, I don't know what I would personally do. I had to make this decision long time ago with animations which were hard telecined in the normal parts ('cause animating stuff costs money) with 3 "good" and 2 repeated,
3 good and 2 repeated, 3 good and 2 repeated etc and 29.970p parts whenever a panning was occurring ('cause panning is done by a computer, so it didn't cost any money). I actually decided to go with option A and preserve the 29.970p parts by duplicating the 23.976p ones. This was back when I was working at Viewster after moving away from Crunchyroll, so around 2015.

[Micheal813]

You have two paths in front of you:

Option A) You deinterlace everything, so that the telecined parts will be duplicated to 29.970p but you'll save the progressive ones

Option B) You IVTC everything to 23.976p so that the telecined parts will be 23.976 and the progressive parts will be decimated to 23.976p



Yes, that's option B, it will IVTC the telecined parts and decimate the progressive ones so that you're gonna get 23.976p in the end.

To be absolutely fair, I don't know what I would personally do. I had to make this decision long time ago with animations which were hard telecined in the normal parts ('cause animating stuff costs money) with 3 "good" and 2 repeated,
3 good and 2 repeated, 3 good and 2 repeated etc and 29.970p parts whenever a panning was occurring ('cause panning is done by a computer, so it didn't cost any money). I actually decided to go with option A and preserve the 29.970p parts by duplicating the 23.976p ones. This was back when I was working at Viewster after moving away from Crunchyroll, so around 2015.

Thank you for the help. What would the script for option A look like?

[FranceBB]

What would the script for option A look like?

A simple tdeint should suffice:

MPEG2Source("video.d2v")

tdeint(mode=2, order=-1, field=-1, mthreshL=6, mthreshC=6, map=0, type=2, debug=false, mtnmode=1, sharp=true, cthresh=6, blockx=16, blocky=16, chroma=true, MI=64, tryWeave=true, link=1, denoise=true, slow=2)

[kedautinh12]

TIVTC v1.0.29 (20240302) (https://github.com/pinterf/TIVTC/releases)

[FranceBB]

Yep, TDecimate() is now populating frame properties correctly.
Another tiny step forward.
At this point it's been a while since frame properties were introduced alongside the old clip properties but we're still finding those things and filters/functions not handling them correctly.
I feel like the migration to frame properties in Avisynth will probably end up the same as the migration from 16bit stacked / interleaved to 16bit planar (i.e slowly lots of functions/plugins will use them, but there will forever be exceptions that will have to be handled manually with propSet()... :( )

[pinterf]

TIVTC 1.0.29 (https://github.com/pinterf/TIVTC/releases/tag/v1.0.29) released with flossy83's additions.

(20240302)
- TDecimate: allow noblend=true when hybrid=1, noblend default value is false when hybrid=1 to keep
backward compatibility. (flossy83)
- TDecimate to fill new frame properties. (flossy83)
TDecimateCycleMetrics, TDecimateCycleMetricsPrev, TDecimateCycleMetricsNext (float arrays),
TDecimateCycleFrameNums, TDecimateCycleFrameNumsPrev, TDecimateCycleFrameNumsNext (int arrays),
TDecimateCycleBlendStatus
See #48: https://github.com/pinterf/TIVTC/issues/48
- Fix #46: TDecimate(hybrid=3) blending with wrong frame. (flossy83)
https://github.com/pinterf/TIVTC/issues/46

(from TDecimate - READ ME.txt):

Frame properties set by TDecimate (since TDecimate v1.0.11, pack v1.0.29):

TDecimateCycleMetrics (float array): normalised metrics for all frames in current cycle
TDecimateCycleMetricsPrev and TDecimateCycleMetricsNext: same as TDecimateCycleMetrics but for previous/next cycle

TDecimateCycleFrameNums (int array): frame numbers corresponding to TDecimateCycleMetrics
TDecimateCycleFrameNumsPrev and TDecimateCycleFrameNumsNext: same as TDecimateCycleFrameNums but for previous/next cycle

TDecimateCycleBlendStatus (integer): indicates if frames in the current cycle are blended:
-20=undefined,
0=not blended,
1=blended,
2=not blended due to scenechange,
3=not blended due to 2 dup cycle workaround

Example usage:

ScriptClip("PrintProps(last, current_frame)", after_frame=true, local=false)
function PrintProps(clip c, int current_frame){
metrics = propGetAsArray(c, "TDecimateCycleMetrics")
fnums = propGetAsArray(c, "TDecimateCycleFrameNums")
blendStatus = propGetAny(c, "TDecimateCycleBlendStatus")
s = ""
for (i=0, ArraySize(metrics)-1) {
s = s + "Frame " + String(fnums[i]) + ": " + String(metrics[i], "%0.2f") + "\n"
}
s = s + "Blend status: " + String(blendStatus) + "\n"
c.Text(s, lsp=0, align=7, size=20, bold=true)
}



and change in noblend:

noblend -

With hybrid = 0 or 1 there is a special case when tdecimate will use blend decimation, and
that is when it detects two duplicates in a cycle, cycleR=1, and neither duplicate is next
to a scenechange. In such cases tdecimate will drop one of the duplicates and replace the
other with a blend of its neighbors. The "noblend" parameter is used to disable this behaviour.

From TDecimate v1.0.11 (pack *v1.0.29) noblend has effect for hybrid = 1, for backward
compatibility its default value is different for hybrid = 0 and 1.

true - no blending in the two duplicate case when hybrid = 0 or 1
false - use blending in the two duplicate case

Default: true (bool) when hybrid = 0, false (bool) when hybrid = 1

[Matching_Mole]

I'm currently using TFM/TDecimate with Avisynth Filter (and Avisynth+) to perform on the fly IVTC with some of my japan bluray and DVD. So I'm trying to optimise my script as the rendering time is critical in this case.

Where I'm quite lost is regarding SetMTMode. It seems that TFM, to perform the field matching, can't use it or in serialised mode which is not the better mode. Is it true or does SetMTMode or Prefetch could be useful to optimise my rendering time?

So far, the betters options to optimise TFM seems to be slow and PP (the encoding of my bluray seems pretty clean and sometime no PP seems required). AssumeTTF (or BFF but I didn't see any occurence for the moment) help also. Does any other options I should considerate to optimise the rendering time?

Thanks in advance!

[Boulder]

IIRC TDecimate is the one that doesn't like multithreading and automatically uses the serial mode, so maybe you could try placing Prefetch right after TFM for best results.

[Matching_Mole]

Thanks, I will try.