TDeint and TIVTC

[8day]

Originally Posted by hello_hello

I'm not sure I understand the logic behind specifying a couple of consecutive frames (or even 4 consecutive frames) as being video

It's not about small segments per se. It's that it behaves like it adjusts boundaries of overriden segments to fit mod5 blocks/detected telecine GOPs, and small segments seemed like the most obvious choice to test this. I think it will behave same way with segments of length 102, 103 and 104 frames.

Originally Posted by hello_hello

So the question would be.... did TDecimate remove any of the frames you specified as video?

Yes, it did. For 10,11 #11 was removed/not used, for 20,22 #21 and for 30,33 it was #33. And that's the problem: it ignores overrides. What should've happened is if previous and/or next telecine GOP got split, incomplete part must've been shown at 29.97 fps, and the defined video segment itself should've been shown at 29.97 fps. If anything, such segments should've been larger, not smaller, and definitely not ignored completely.

[hello_hello]

Originally Posted by 8day

Yes, it did. For 10,11 #11 was removed/not used, for 20,22 #21 and for 30,33 it was #33. And that's the problem: it ignores overrides. What should've happened is if previous and/or next telecine GOP got split, incomplete part must've been shown at 29.97 fps, and the defined video segment itself should've been shown at 29.97 fps. If anything, such segments should've been larger, not smaller, and definitely not ignored completely.

The frames you specify in the overrides file do have to correspond to what TDecimate considers to be a whole cycle, which seems to always begin at frame 0.

I created a metrics file where frame 12 was obviously a duplicate. Specifying "10,14 v" caused TDecimate to keep frame 12 and set a frame rate of 29.970 for those frames in the timecodes file. Specifying "9,13 v" however had no effect. Those frames were still considered to be film and frame 12 was dropped.

According to my metrics file, frames 12, 17 and 22 were duplicates. When I specified "9,23 v" for the overrides file, frames 12 & 17 were kept but not frame 22.

I'm not sure it's a problem though, as it seems it's easy enough to work around once you know you need to specify ranges of 5. To ensure all frames in the range 17 to 22 are treated as video instead of film, you'd need to specify the range as "15,24 v". Admittedly if that's how it works, and so far it seems to be, it would've been better to make that clear in the help file.

The same applies to not putting a space after the comma like so:
42, 59 v
The help file doesn't warn not to and there's no error if you do. I couldn't work out why my overrides file was being ignored completely until the penny dropped and I realised I'd added a space.

[8day]

hello_hello, this thought never crossed my mind, but you seem to be right, it kind of makes sense. Now I hope I'll be able to write necessary scripts in a day or two...

[8day]

hello_hello, I think I've done it. It took longer than expected, but should work.

All the files, including links to custom plugins, can be found here: https://github.com/8day/avs-tivtcex All the stuff you need to know is described in the readme.md on that page.

[hello_hello]

I haven't checked it out properly yet but it looks interesting. I definitely will.

Out of curiosity, what do you use for this part of the phase 2 script?

# favourite filter chain for analysis of hybrid video.

Cheers.

[8day]

Anything that makes combing more visible and choices more easy will do. ATM I can't share my filter chain for various reasons (I'll share it later, when I'll finish another project, which hopefully won't take more than a month: it's mostly ready, but I want to write replacement for TDecimate), but considering what's expected from TIVTCEx, Histogram("luma") should be more than enough.

In case you didn't know, TFM generates misc stats at the end of its metrics file on combed frames, ranges, and supposedly missed combed frames. All you have to do is inspect them and create overrides file for TFM and/or TIVTCEx (I suspect that's what YATTA is built around).

[Gser]

Originally Posted by 8day

Anything that makes combing more visible and choices more easy will do. ATM I can't share my filter chain for various reasons (I'll share it later, when I'll finish another project, which hopefully won't take more than a month: it's mostly ready, but I want to write replacement for TDecimate), but considering what's expected from TIVTCEx, Histogram("luma") should be more than enough.

In case you didn't know, TFM generates misc stats at the end of its metrics file on combed frames, ranges, and supposedly missed combed frames. All you have to do is inspect them and create overrides file for TFM and/or TIVTCEx (I suspect that's what YATTA is built around).

Any chance of getting x64 versions of the new filters?

[8day]

I doubt it. My understanding of C++/AviSynth is relatively primitive. The only reason there's this "hack" is because TIVTC compiled straight out of the box. But I'll try to compile pinterf's mod once again and maybe I'll get lucky this time... Otherwise you'll have to ask pinterf to copy this hack and recompile DLL with it.

[pinterf]

Originally Posted by 8day

I doubt it. My understanding of C++/AviSynth is relatively primitive. The only reason there's this "hack" is because TIVTC compiled straight out of the box. But I'll try to compile pinterf's mod once again and maybe I'll get lucky this time... Otherwise you'll have to ask pinterf to copy this hack and recompile DLL with it.

No problem adding that hack, which source files should I check for modifications? I have filecompared TDecimate and found one place, but it would be easier to know that w/o comparing all files. I can see only root and initial commits in your git repo.
EDIT: found them and integrated. Who wants to test it?

EDIT: New build: TIVTC 1.0.10

**v1.0.10 (20180119)**
- integrate new orgOut parameter for TDecimate (by 8day) (see TDecimate - READ ME.txt)

[8day]

Just realized why Gser was talking about multiple DLLs... Here's EDL compiled using latest AviSynth+ headers in VS2017: https://github.com/8day/avs-edl/releases/tag/0.1. But please note that I haven't tested it since I don't have AviSynth+ on my system.

[Gser]

Okay I finally managed to write down the frames like so but I didn't quite understand the formatting of the extended overrides file. How would I format this information?

0 265 v
266 421 vc
422 470 v
471 480 e
481 2097 v
2098 2102 e
2103 2686 v
2687 2691 e
2692 3053 v
3054 3110 ii
3111 3902 v
3903 3963 ii
3964 5969 v
5970 5992 e
5993 10290 v
10291 10417 ii
10418 50437 v
50438 50460 e
50461 86904 v
86905 87064 ii
87065 90390 v

[DJATOM]

It seems I found a bug. 1.0.8 work as expected while 1.0.9 and 1.1.10 randomly produces timecodes without 30fps sections. It might be due to workaround, I didn't tried to research there. Just downgraded it step by step and 1.0.8 is safe.
How to reproduce:
1) run vfr 1st pass (I tried with Pokemon pearl and diamond 2007, that's definitely vfr content)
2) refresh video in avspmod few times (more often it contains less frames and shows less duration)

[pinterf]

New build, fix for the issue mentioned by DJATOM
It was fixed a bit earlier but only published now.
https://github.com/pinterf/TIVTC/releases/tag/v1.0.11

**v1.0.11 (20180323)**
- Revert to pre-1.0.9 usehint detection: conflicted with mode 5 (sometimes bad clip length was reported)
  (reason of the workaround (crash at mysterious circumstances) was eliminated: mmx state was not cleared in mvtools2) 
- Fix: bad check for emptiness of orgOut parameter

**v1.0.10 (20180119)**
- integrate new orgOut parameter for TDecimate (by 8day) (see TDecimate - READ ME.txt)

[GMJCZP]

Thanks.

[hello_hello]

Thanks for the update.

Out of curiosity, did you have a chance to look at this bug pinterf?
https://forum.doom9.org/showthread.p...05#post1812705

[FranceBB]

Thanks. I appreciate! I use TIVTC almost on a daily basis!

[Katie Boundary]

TFM's biggest weakness is that it's terrible at telling the difference between interlacing and any other kind of fine horizontal edge. Given these two parts of the same image...





There is no combination of settings (as far as I can tell) that will recognize the bottom part, but not the top, as interlaced; however, there is a huge range of settings where the opposite will happen, and the top part will get "deinterlaced" but the actual interlacing gets through. Does anyone know of a solution or workaround for this?

[hello_hello]

Originally Posted by Katie Boundary

There is no combination of settings (as far as I can tell) that will recognize the bottom part, but not the top, as interlaced; however, there is a huge range of settings where the opposite will happen, and the top part will get "deinterlaced" but the actual interlacing gets through. Does anyone know of a solution or workaround for this?

What settings have you tried?
It's pretty hard to do anything but wildly guess without a sample, but you can always use an external deinterlacer instead, assuming you know of one that's better at "telling the difference between interlacing and any other kind of fine horizontal edge".

If you specify a clip for the clip2 argument, and PP<5, then TFM will take the entire frame from the specified clip whenever it detects combing.
The frame from the specified clip could be fully de-interlaced or partially de-interlaced or not de-interlaced at all, based on the decisions of the de-interlacer de-interlacing it.

When messing around with TFM's settings I've found doing this sort of thing useful at times:

A = Last
B = A.TFM(pp=1).GreyScale()
C = A.TFM(pp=6, Clip2=B)
Return C

Nothing gets de-interlaced, but the second instance of TFM builds the usual mask for the sections where it detects combing and takes them from clip2, the upshot being they'll be black and white, making it easier to see what it's doing.
Or try PP=1 with Display=true and TFM will draw a white box around the area that triggered the frame's detection as combed.

[StainlessS]

Originally Posted by hello_hello

A = Last
B = A.TFM(pp=1).GreyScale()
C = A.TFM(pp=1, Clip2=B, PP=6)
Return C

HH, above cant be right, would throw an error as pp aready given.
I've CODE blocked your snippet for you in quote, shame I cant do that in your post, perhaps I should be a moderator

[hello_hello]

Thanks. I probably copied and pasted one of the lines, but failed to edit it properly. I'll fix the post.