TDecimate removes wrong frame

[sonic41592]

Sample! (https://mega.nz/file/0cRC0J4A#_eHSJ1BAGjQx6CWuHo9IpNPmcyb72quv6AKqPgooZH8)

I've been using TIVTC on my old DVDs, and I noticed it makes some scenes judder. In the sample, the scene is animated on twos, so each frame should have a duplicate. After tfm, it correctly outputs the frames with a duplicate pattern 2 3 2 3, which TDecimate(mode=1) should decimate to 2 2 2 2. However, it's not consistent in its output, sometimes giving patterns of 2 3 1 2, or something similar, making the video jerky and juddery. I tried telecide decimate, which gives the correct 2 2 2 2 pattern, but I prefer using TIVTC if possible. Has anyone else experienced this? The avisynth file is a simple:
mpeg2souce("video.d2v")
tfm(d2v="video.d2v")
tdecimate(mode=1)

or:
mpeg2souce("video.d2v")
telecide
decimate(mode=2)

[real.finder]

maybe it's better if you tried clean Rainbow and Dot Crawl first to make both tfm and tdecimate work better

maybe something like this

tcomb
tfm
tdecimate(mode=1)

[sonic41592]

Unfortunately that doesn't fix the issue, even with checkmate, bifrost, and tcomb as preprocessing, it still isn't giving the evenly distributed frames like telecide decimate does.

[johnmeyer]

Since it is a non-standard pattern, you need to explicitly set CycleR and Cycle. The defaults will not work.

Your sample has five fields for each image, but you want the output to have four fields (i.e., two repeated frames for each image). Thus Cycle should be set to 5 and CycleR should be set to 1.

tfm()
tdecimate(cycle=5, cycleR=1)

[sonic41592]

That didn't work either. Is it a non-standard pattern? It's just hard telecined animation drawn on twos, I thought that was pretty common, and what tdecimate(mode=1) was specifically made for. And it's not drawn on twos the entire time, just slow moving scenes.

[sonic41592]

Is it ok to mix tfm with decimate from Decomb.dll? Tfm is the better field matcher, but decimate works better than tdecimate in my case. It seems to work well on the small sample, could it cause something weird in other places?

tfm(d2v="video.d2v")
decimate(mode=2)

[hello_hello]

TDecimate(Mode=1, Cycle=5, CycleR=1) uses the default settings, aside from Mode=1.
It's exactly the same as TDecimate(Mode=1).

I haven't looked at the sample yet, but TDecimate does it's thing over each cycle of frames (5 by default), starting with the first frame.
If the pattern is odd, you can try increasing the frame cycle, and the number of frames it should remove in each cycle accordingly.

TDecimate(Mode=1, Cycle=10, CycleR=2)

Edit: It didn't help. See a couple of posts down.

Is it ok to mix tfm with decimate from Decomb.dll? Tfm is the better field matcher, but decimate works better than tdecimate in my case. It seems to work well on the small sample, could it cause something weird in other places?

I wouldn't think so.
Edit: I was answering the "could it cause something weird in other places" question.

[videoh]

There's no problem combining TFM with Decimate.

[hello_hello]

I'm not sure what TDecimate is seeing that I'm not, but this seems to help.

TFM()
TDecimate(Mode=1, Chroma=false, Denoise=true)

[sonic41592]

That does help, maybe it's some residual rainbowing or something. I think I'll stick with decimate(mode=2) though since it works by default. I wonder if the hints left by tfm are compatible with decimate.

[hello_hello]

videoh would know for sure, as Decomb is his thing, but the TDeint help file (same author as TIVTC) says it can read hints from TFM() and Telecide(), so they're at least somewhat compatible.

The difference between TDecimate and Decimate may simply be the default sensitivity.
Decimate's threshold2 seems to be the equivalent of dupThresh for TDecimate. Their defaults are 3.0 and 1.1 respectively, taking the mode being used for each into account.

I had a look at your sample with this:
TFM()
TDecimate(Mode=1, Display=true)

Before decimation, frames 53, 54 and 55 are duplicates, so either 54 or 55 should go.

Without denoising and disabling chroma, TDecimate shows thresholds of 18.01, 1.69 and 3.62 respectively. For TDecimate to decide 54 and 55 are duplicates, it seems dupThresh needs to be at least 3.63, but the upshot is, it appears to pick the correct frames for removal from your sample with this (I'd probably bump dupThresh up to 4).

TFM()
TDecimate(Mode=1, dupThresh=4)

I imagine Decimate(Mode=2, threshold2=1.1) would cause Decimate to remove the wrong frames too, but I haven't checked.

[sonic41592]

Excellent explanation! So it is because of how ridiculously noisy the source is. I wonder if putting Tdecimate after TemporalDegrain2 would fix everything, even though it would slow it down a bit

[hello_hello]

By definition, noise changes from frame to frame, so if duplicate frames are in fact duplicates, including the noise, it might effect the noise removal.
I tend to decimate first, but you can only try it the other way around, although unless you were planning on denoising you could just enable TDecimate's denoise argument. I don't think it effects the output video just the decision on duplicates, so the output after decimation won't be denoised. Don't quote me on that though, but logically that's how it'd work.