How to read a d2v file

[Ceppo]

I generated a d2v file with DGIndex, it gives me these kind of lines:

d00 5 0 2048 0 1 1 f3 f0 d1 f2 f3 e0 f1 f2 e3 f0 f1 e2
900 5 0 59392 0 1 1 73 70 d1 f2 f3 e0 f1 f2 e3 f0 f1 e2
900 5 0 110592 0 1 1 73 70 d1 f2 f3 e0 f1 f2 e3 f0 f1 e2
900 5 0 161792 0 1 1 73 70 d1 f2 f3 e0 f1 f2 e3 f0 f1 e2
900 5 0 212992 0 1 1 73 70 d1 f2 f2 a2 b2 f3 e0 f0 f1 e2
900 5 0 507904 0 1 1 73 70 d1 f2 f3 e0 f1 f2 e3 f0 f1 e2
900 5 0 899072 0 1 1 73 70 d1 f2 f3 e0 f1 f2 e3 f0 f1 e2
900 5 0 1294336 0 1 1 73 70 d1 f2 f3 e0 f1 f2 e3 f0 f1 e2
900 5 0 1687552 0 1 1 73 70 d1 f2 f3 e0 f1 f2 e3 f0 f1 e2
900 5 0 2082816 0 1 1 73 70 d1 f2 f3 e0 f1 f2 e3 f0 f1 e2
900 5 0 2461696 0 1 1 73 70 d1 f2 f3 e0 f1 f2 e3 f0 f1 e2
900 5 0 2842624 0 1 1 73 70 d1 f2 f3 e0 f1 f2 e3 f0 f1 e2
900 5 0 3221504 0 1 1 73 70 d1 f2 f3 e0 f1 f2 e3 f0 f1 e2
900 5 0 3596288 0 1 1 73 70 d1 f2 f3 e0 f1 f2 e3 f0 f1 e2
900 5 0 3971072 0 1 1 73 70 d1 f2 f3 e0 f1 f2 e3 f0 f1 e2
900 5 0 4345856 0 1 1 73 70 d1 f2 f3 e0 f1 f2 e3 f0 f1 e2
900 5 0 4710400 0 1 1 73 70 d1 f2 f3 e0 f1 f2 e3 f0 f1 e2
900 5 0 5068800 0 1 1 73 70 d1 f2 f3 e0 f1 f2 e3 f0 f1 e2
900 5 0 5427200 0 1 1 73 70 d1 f2 f3 e0 f1 f2 e3 f0 f1 e2
900 5 0 5795840 0 1 1 73 70 d1 f2 f3 e0 f1 f2 e3 f0 f1 e2
900 5 0 6182912 0 1 1 73 70 d1 f2 f3 e0 f1 f2 e3 f0 f1 e2
900 5 0 6541312 0 1 1 73 70 d1 f2 f3 e0 f1 f2 e3 f0 f1 e2
900 5 0 6883328 0 1 1 73 70 d1 f2 f3 e0 f1 f2 e3 f0 f1 e2
900 5 0 7245824 0 1 1 73 70 d1 f2 f3 e0 f1 f2 e3 f0 f1 e2
900 5 0 7604224 0 1 1 73 70 d1 f2 f3 e0 f1 f2 e3 f0 f1 e2
900 5 0 7960576 0 1 1 73 70 d1 f2 f3 e0 f1 f2 e3 f0 f1 e2
900 5 0 8325120 0 1 1 73 70 d1 f2 f3 e0 f1 f2 e3 f0 f1 e2
900 5 0 8695808 0 1 1 73 70 d1 f2 f3 e0 f1 f2 e3 f0 f1 e2
900 5 0 9048064 0 1 1 73 70 d1 f2 f3 e0 f1 f2 e3 f0 f1 e2
900 5 0 9408512 0 1 1 73 70 d1 f2 f3 e0 f1 f2 e3 f0 f1 e2
900 5 0 9766912 0 1 1 73 70 d1 f2 f3 e0 f1 f2 e3 f0 f1 e2
900 5 0 10125312 0 1 1 73 70 d1 f2 f3 e0 f1 f2 e3 f0 f1 e2
900 5 0 10483712 0 1 1 73 70 d1 f2 f3 e0 f1 f2 e3 f0 f1 e2
900 5 0 10844160 0 1 1 73 70 d1 f2 f3 e0 f1 f2 e3 f0 f1 e2
900 5 0 11194368 0 1 1 72 32 92 b2 b2 e3 f0 f1 e2 f3 f0 e1
900 5 0 11571200 0 1 1 72 73 d0 f1 f2 e3 f0 f1 e2 f3 f0 e1
900 5 0 11980800 0 1 1 72 73 d0 f1 f2 e3 f0 f1 e2 f3 f0 e1
900 5 0 12359680 0 1 1 72 73 d0 f1 f2 e3 f0 f1 e2 f3 f0 e1
900 5 0 12736512 0 1 1 72 73 d0 f1 f2 e3 f0 f1 e2 f3 f0 e1
900 5 0 13096960 0 1 1 72 73 d0 f1 f2 e3 f0 f1 e2 f3 f0 e1
900 5 0 13457408 0 1 1 72 73 d0 f1 f2 e3 f0 f1 e2 f3 f0 e1
900 5 0 13817856 0 1 1 72 73 d0 f1 f2 e3 f0 f1 e2 f3 f0 e1
900 5 0 14178304 0 1 1 72 73 d0 f1 f2 e3 f0 f1 e2 f3 f0 e1
900 5 0 14532608 0 1 1 72 73 d0 f1 f2 e3 f0 f1 e2 f3 f0 e1
900 5 0 14893056 0 1 1 72 73 d0 f1 f2 e3 f0 f1 e2 f3 f0 e1
900 5 0 15278080 0 1 1 72 73 d0 f1 f2 e3 f0 f1 e2 f3 f0 e1
900 5 0 15648768 0 1 1 72 73 d0 f0 b0 a0 f1 f2 e3 f0 f1 e2
900 5 0 16080896 0 1 1 73 70 d1 f2 f3 e0 f1 f2 e3 f0 f1 e2
900 5 0 16437248 0 1 1 73 70 d1 f2 f3 e0 f1 f2 e3 f0 f1 e2
900 5 0 16793600 0 1 1 73 70 d1 f2 f3 e0 f1 f2 e3 f0 f0 a0
900 5 0 17268736 0 1 1 30 30 90 b0 b0 a0 b0 b0 a0 b0 b0 a0
900 5 0 17686528 0 1 1 30 30 90 b0 b0 a0 b0 b0 a0 b0 b0 a0
900 5 0 18096128 0 1 1 30 30 90 b0 b0 e1 f2 f3 e0 f1 f2 e3
900 5 0 18436096 0 1 1 70 71 d2 f3 f0 e1 f2 f3 e0 f1 f2 e3
900 5 0 18782208 0 1 1 70 71 d2 f3 f0 e1 f2 f3 e0 f1 f2 e3
900 5 0 19126272 0 1 1 70 71 d2 f3 f0 e1 f2 f3 e0 f1 f2 e3
900 5 0 19480576 0 1 1 70 71 d2 f3 f0 e1 f2 f3 e0 f1 f2 e3
900 5 0 19832832 0 1 1 70 71 d2 f3 f0 e1 f2 f3 e0 f1 f2 e3
900 5 0 20185088 0 1 1 70 71 d2 f3 f0 e1 f2 f3 e0 f1 f2 e3
900 5 0 20609024 0 1 1 70 71 d2 f3 f0 e1 f2 f3 e0 f1 f2 e3
900 5 0 21088256 0 1 1 70 71 d2 f2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 21594112 0 1 1 32 73 d0 f1 f2 e3 f0 f1 e2 f3 f0 e1
900 5 0 21958656 0 1 1 72 73 d0 f1 f2 e3 f0 f1 e2 f3 f0 e1
900 5 0 22321152 0 1 1 72 73 d0 f1 f2 e3 f0 f1 e2 f3 f0 e1
900 5 0 22689792 0 1 1 72 73 d0 f1 f2 e3 f0 f1 e2 f3 f0 e1
900 5 0 23111680 0 1 1 72 73 d0 f1 f2 e3 f0 f1 e2 f3 f0 e1
900 5 0 23558144 0 1 1 72 73 d0 f1 f2 e3 f0 f1 e2 f3 f0 e1
900 5 0 23980032 0 1 1 72 73 d0 f1 f2 e3 f0 f0 a0 b0 b0 a0
900 5 0 24334336 0 1 1 71 72 d3 f0 f1 e2 f3 f0 e1 f2 f3 e0
900 5 0 24676352 0 1 1 71 72 d3 f0 f1 e2 f3 f0 e1 f2 f3 e0
900 5 0 25026560 0 1 1 71 72 d3 f0 f1 e2 f3 f0 e1 f2 f3 e0
900 5 0 25382912 0 1 1 71 72 d3 f0 f1 e2 f3 f0 e1 f2 f3 e0
900 5 0 25751552 0 1 1 71 72 d3 f0 f1 e2 f3 f0 e1 f2 f3 e0
900 5 0 26116096 0 1 1 71 72 d3 f0 f1 e2 f3 f0 e1 f2 f3 e0
900 5 0 26503168 0 1 1 71 72 d3 f0 f1 e2 f3 f0 e1 f2 f3 e0
900 5 0 26877952 0 1 1 71 72 d3 f0 f1 e2 f3 f0 e1 f2 f3 e0
900 5 0 27242496 0 1 1 71 72 d3 f0 f1 e2 f3 f0 e1 f2 f3 e0
900 5 0 27600896 0 1 1 71 72 d3 f0 f1 e2 f2 b2 a2 b2 b2 e3
900 5 0 27963392 0 1 1 70 71 d2 f3 f0 e1 f2 f3 e0 f1 f2 e2
900 5 0 28358656 0 1 1 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 28731392 0 1 1 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 29198336 0 1 1 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 29644800 0 1 1 32 32 92 b2 b2 a2 b2 f3 e0 f1 f2 e3
900 5 0 30003200 0 1 1 70 71 d2 f3 f0 e1 f2 f3 e0 f1 f2 e3
900 5 0 30416896 0 1 1 70 71 d2 f3 f0 e1 f2 f3 e0 f1 f2 e3
900 5 0 30783488 0 1 1 70 70 90 b0 b0 a0 b0 b0 a0 b0 b0 a0
900 5 0 31232000 0 1 1 30 30 90 b0 f1 e2 f3 f0 e1 f2 f2 a2
900 5 0 31617024 0 1 1 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 32008192 0 1 1 32 32 92 b2 b2 a2 f3 f0 e1 f2 f3 e0
900 5 0 32462848 0 1 1 71 72 d3 f0 f0 a0 b0 b0 a0 f1 f2 e3
900 5 0 32804864 0 1 1 70 71 d2 f3 f0 e1 f2 f3 e0 f0 b0 a0
900 5 0 33267712 0 1 1 30 30 90 b0 b0 a0 b0 b0 a0 b0 b0 a0
900 5 0 33726464 0 1 1 30 30 90 b0 f1 e2 f2 b2 a2 b2 f3 e0
900 5 0 34072576 0 1 1 71 72 d3 f0 f0 a0 b0 b0 a0 f1 f2 e2
900 5 0 34463744 0 1 1 32 32 d3 f0 f1 e2 f3 f0 e1 f2 f2 a2
900 5 0 34811904 0 1 1 73 70 d1 f2 f3 e0 f1 f2 e3 f0 f1 e2
900 5 0 35186688 0 1 1 73 70 d1 f2 f3 e0 f1 f2 e3 f0 f1 e2
900 5 0 35543040 0 1 1 73 70 d1 f2 f3 e0 f1 f2 e3 f0 f1 e2
900 5 0 35891200 0 1 1 73 70 d1 f2 f2 a2 b2 b2 a2 f3 f0 e0
900 5 0 36290560 0 1 1 30 30 90 b0 f1 e2 f2 b2 a2 b2 b2 e3
900 5 0 36663296 0 1 1 70 70 90 b0 b0 a0 f1 f2 e2 b2 b2 a2
900 5 0 37036032 0 1 1 32 73 d0 f0 b0 a0 b0 b0 e1 f2 f2 a2
900 5 0 37449728 0 1 1 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 37773312 0 1 1 32 32 92 b2 b2 a2 f3 f0 e1 f2 f2 a2
900 5 0 38170624 0 1 1 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 38580224 0 1 1 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 39020544 0 1 1 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 39411712 0 1 1 32 32 92 b2 b2 e3 f0 f0 a0 f1 f2 e3
900 5 0 39880704 0 1 1 70 71 d2 f3 f0 e1 f2 f3 e0 f1 f2 e3
900 5 0 40230912 0 1 1 70 71 d2 f3 f0 e1 f2 f3 e0 f1 f2 e3
900 5 0 40611840 0 1 1 70 71 d2 f3 f0 e1 f2 f3 e0 f0 b0 a0
900 5 0 41146368 0 1 1 30 30 d1 f2 f2 a2 b2 b2 a2 b2 b2 a2
900 5 0 41603072 0 1 1 32 32 d3 f0 f1 e2 f2 b2 a2 b2 b2 e3
900 5 0 42067968 0 1 1 70 70 90 b0 b0 a0 f1 f2 e2 b2 b2 a2
900 5 0 42491904 0 1 1 32 73 d0 f0 b0 a0 b0 b0 a0 b0 b0 a0
900 5 0 42897408 0 1 1 30 30 90 b0 b0 a0 b0 b0 a0 b0 b0 e1
900 5 0 43327488 0 1 1 72 72 92 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 43767808 0 1 1 32 32 92 b2 b2 e3 f0 f1 e2 f3 f0 e1
900 5 0 44075008 0 1 1 72 73 d0 f1 f2 e3 f0 f1 e2 f3 f0 e1
900 5 0 44400640 0 1 1 72 73 d0 f1 f2 e3 f0 f1 e2 f3 f0 e1
900 5 0 44711936 0 1 1 72 73 d0 f1 f2 e3 f0 f1 e2 f3 f0 e1
900 5 0 45021184 0 1 1 72 73 d0 f1 f2 e3 f0 f1 e2 f3 f0 e1
900 5 0 45328384 0 1 1 72 73 d0 f1 f2 e3 f0 f1 e2 f3 f0 e1
900 5 0 45645824 0 1 1 72 73 d0 f1 f2 e3 f0 f1 e2 f2 b2 a2
900 5 0 45983744 0 1 1 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 46325760 0 1 1 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 46665728 0 1 1 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 46991360 0 1 1 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 47312896 0 1 1 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 47665152 0 1 1 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 47988736 0 1 1 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 48343040 0 1 1 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 48697344 0 1 1 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 49010688 0 1 1 32 32 92 b2 b2 a2 b2 b2 e3 f0 f0 e1
900 5 0 49321984 0 1 1 72 72 d3 f0 f0 a0 b0 b0 a0 b0 b0 a0
900 5 0 49643520 0 1 1 30 30 90 b0 b0 a0 f1 f2 e2 b2 b2 a2
900 5 0 49954816 0 1 1 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 50268160 0 1 1 32 32 92 b2 f3 e0 f1 f2 e3 f0 f1 e2
900 5 0 50618368 0 1 1 73 70 d1 f2 f3 e0 f1 f2 e2 b2 f3 e0
900 5 0 50968576 0 1 1 71 72 d2 b2 b2 a2 f3 f0 e1 f2 f3 e0
900 5 0 51320832 0 1 1 71 72 d3 f0 f1 e2 f3 f0 e1 f2 f3 e0
900 5 0 51677184 0 1 1 70 30 90 b0 b0 a0 b0 b0 a0 b0 b0 a0
900 5 0 51963904 0 1 1 30 30 d1 f2 f3 e0 f1 f2 e3 f0 f0 a0
900 5 0 52295680 0 1 1 71 72 d3 f0 f0 e1 f2 f2 a2 f3 f0 e0
900 5 0 52621312 0 1 1 30 30 90 b0 f1 e2 f2 b2 a2 b2 b2 a2
900 5 0 52899840 0 1 1 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 53219328 0 1 1 32 32 92 b2 b2 a2 b2 b2 a2 f3 f0 e1
900 5 0 53610496 0 1 1 72 73 d0 f1 f2 e3 f0 f0 e1 f2 f2 e3
900 5 0 54071296 0 1 1 70 70 d1 f2 f3 e0 f1 f2 e2 b2 b2 a2

How do I read them? The objective is field matching.

[Selur]

http://rationalqm.us/dgmpgdec/DGIndexManual.html#AppendixA should help to understand the format.

[Ceppo]

Based on that, how do you read:

900 5 0 59392 0 1 1 73 70 d1 f2 f3 e0 f1 f2 e3 f0 f1 e2

I don't get it, maybe I'm doing too many things at the same time, the only thing I understand is that each line is a GOP, and I might be wrong. :p What are those letters, hexdecimal, like should I read a as 10? Or do I have to do hex->binary?

EDIT:
I decoded to binary this line: f3 f0 d1 f2 f3 e0 f1 f2 e3 f0 f1 e2
and I got this:
11110011
11110000
11010001
11110010
11110011
11100000
11110001
11110010
11100011
11110000
11110001
11100010
It says that they are all progressive since bit 6 is the second bit on the left, isn't it? Or it is the other way?

[Selur]

I don't think so reading:
Notes on Progressive_Frame Flag: This specifies that the picture is part of a frame (either a frame picture making up the entire frame, or a field picture making up one of the fields of the frame) whose fields are sampled at one moment. A RFF flag may be present however, such that the displayed frame after pulldown is interlaced.
but videoh should be able to clear this up. :)

Cu Selur

[Ceppo]

It's a XOR. Either I'm stupid and I can't send him a PM, or He got banned or something and I can't send him a PM.

[kedautinh12]

Just tag him, @videoh :D

[Reel.Deel]

He got banned or something

I believe that's the case, there are 3 recent strikes shown on his user page.

Just tag him, @videoh :D

I don't think that does anything...

[real.finder]

he got a forum http://rationalqm.us/board/viewforum.php?f=7

[StainlessS]

I believe that's the case, there are 3 recent strikes shown on his user page.
Yes, 30 day suspension. [3 strikes {within a 30 day period} := 30 day D9 vacation]
From PM (via a 'go between'), he said something like "It will be a cold day in hell before I post on Doom9 again".
So, post on his forum will be best bet.

@ Moderator, VideoH strike history should NOT be visable to all on Public Profile.

[Ceppo]

I made the post, but it says it needs the approval of a moderator. So, I guess I have to wait :p

EDIT:
If someone is interested http://rationalqm.us/board/viewtopic.php?f=7&t=1157&p=15628#p15628

[manono]

I have no idea what the letters mean, and don't much care. But the numbers hold valuable information. 0 1 2 3 0 1 2 3 means soft pulldown. Encoded as progressive 23.967 with 3:2 (or 2:3) pulldown added. When you see 2 2 2 2 or 0 0 0 0 it drops to video. If it's a mix, then you should IVTC it as a percentage of the DVD is video. You can see the video percentage by reading at the bottom of the D2V file.

If it's 100% film, then you can use Forced Film. If it's mostly film, then you "Honor Pulldown Flags" and IVTC. I have no use for VFR. If you have questions, post samples cut using DGIndex.

[Ceppo]

I tried, but I'm getting no results. My source is BFF and I still had no time to figure out the TFF/RFF combinations, that's why it doesn't work. Still, here is a sample of mixed 24fps soft telecined and 60i sections.
https://www.mediafire.com/file/k9o6gu5rqf27xqd/VTS_01_2.demuxed.m2v/file

I tried making a python script to convert all TFF/RFF combination (assuming TFF) relative to the pattern and skipping where I'm not getting it, assuming it is a VFR section. If the pattern is found it is converted into a CTelecine.txt file and the grammar is as follow:
(frame_number) (isNmatch ? 1 : 0)

Here the code:

f = open("VTS_01_2.d2v", "r")
w = open("file.txt","w")
array = ['0','1','2','3','4','5','6','7','8','9','a','b','c','d','e','f']
lineSrc = f.readlines()

hex = []
count = 0
for line in lineSrc:
if line[0] in array and line[1] in array and line[2] in array:
line = line[-36:-1].replace(' ','')
for x, y in zip(line[0::2],line[1::2]):
hex.append(x+y)

temp = []
for x in range(0,len(hex)-1,4):
a = bin(int(hex[min(len(hex)-1,x+0)],base=16))[2:].zfill(8)
a = str(a)[-3:-1]
b = bin(int(hex[min(len(hex)-1,x+1)],base=16))[2:].zfill(8)
b = str(b)[-3:-1]
c = bin(int(hex[min(len(hex)-1,x+2)],base=16))[2:].zfill(8)
c = str(c)[-3:-1]
d = bin(int(hex[min(len(hex)-1,x+3)],base=16))[2:].zfill(8)
d = str(d)[-3:-1]
y = a + b + c + d
if y == '00011011':
w.write(str(count+0)+' 1\n')
w.write(str(count+1)+' 1\n')
w.write(str(count+2)+' 0\n')
w.write(str(count+3)+' 0\n')
w.write(str(count+4)+' 0\n')
count += 5
elif y == '11000110':
w.write(str(count+0)+' 0\n')
w.write(str(count+1)+' 1\n')
w.write(str(count+2)+' 1\n')
w.write(str(count+3)+' 0\n')
w.write(str(count+4)+' 0\n')
count += 5
elif y == '10110001':
w.write(str(count+0)+' 0\n')
w.write(str(count+1)+' 0\n')
w.write(str(count+2)+' 1\n')
w.write(str(count+3)+' 1\n')
w.write(str(count+4)+' 0\n')
count += 5
elif y == '01101100':
w.write(str(count+0)+' 0\n')
w.write(str(count+1)+' 0\n')
w.write(str(count+2)+' 0\n')
w.write(str(count+3)+' 1\n')
w.write(str(count+4)+' 1\n')
count += 5
elif y == '01101100':
w.write(str(count+0)+' 1\n')
w.write(str(count+1)+' 0\n')
w.write(str(count+2)+' 0\n')
w.write(str(count+3)+' 0\n')
w.write(str(count+4)+' 1\n')
count += 5
else:
w.write(str(count+0)+' 0\n')
w.write(str(count+1)+' 0\n')
w.write(str(count+2)+' 0\n')
w.write(str(count+3)+' 0\n')
count += 4

w.close()


The problem is that if I use "ignore" instead of "honor" in the field operation, the soft telecine is not executed and I get the 24 fps section perfectly. Yet, if I do that, all 60i and post telecine cuts are left @ 29.970 giving me a mixed frame count from which I can't generate a VFR timecode for those sections. Sadly, there is no way to dynamically create a d2v file for each gop to choose which option to use based on the film percentage so that I can create a VFR file from it, and merging all the d2v file in avisynth later with a script.

EDIT:
I'm dropping the idea since my head hurts and this doesn't look easy to nail down.

[real.finder]

"ignore" should not be used anyway

anyway, you don't need to read d2v file to know if there are soft pulldown if you read _EncodedFrameBottom and _EncodedFrameTop properties numbers, if they show same number then it not film, if they not show same number then it film with soft pulldown and I think their numbers can be use to undo the pulldown (edit: if we got TFF and the top number is less than the Bottom then the top field have to be matched with the bottom field from previous frame, but if top number is more than the Bottom then it matched from next frame, for BFF things will be same but it will be the bottom field will match with the top field from ether either next or previous frame)

but there are some problem, for some reason (maybe bug) in dgindex or d2vSource _FieldBased always be 1 for your sample, edit: I think it d2vSource bug https://github.com/Asd-g/MPEG2DecPlus/issues/8

LWLibavVideoSource seems work fine with _FieldBased but it got no _EncodedFrameBottom and _EncodedFrameTop! edit: https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/issues/17

[Ceppo]

but there are some problem, for some reason (maybe bug) in dgindex or d2vSource _FieldBased always be 1 for your sample, edit: I think it d2vSource bug

I was about to drop the encoding world because of a bug!?

BTW, Thanks.

EDIT:
D2VSource("VTS_01_1.d2v")
subtitle(string(propGetInt("_EncodedFrameTop")))
gives me 0 for all frames, same with Bottom, I'm doing it right?

[real.finder]

gives me 0 for all frames, same with Bottom, I'm doing it right?

that normal in the script without runtime (scriptclip)

mean you must use it as

scriptclip("""subtitle(string(propGetInt("_EncodedFrameTop")))""")

but if you need only to show them as text on the video then just use propShow() :)

[Ceppo]

I was losing my mind over something already made simple :\
Following the idea "if BFF and Bottom > Top then N match", I have trouble with post telecine cuts, so I need SC detection to fix that, I guess. Also, 60i content has different Top/Bottom for each frame, so I need to check the next frame as well. And whatever I find tomorrow in this cursed source. :)

[Ceppo]

Basic Script:

D2VSource("VTS_01_1.d2v")
N = DoubleWeave().SelectOdd()

ScriptClip("""
ATOP = PropGetInt("_EncodedFrameTop")
ABOT = PropGetInt("_EncodedFrameBottom")
ATOP == ABOT ? last : N
""")

Seems to work, lacks SC detection for Post telecine cuts, and 60i check.
In the end, I still need to check 5 frames at a time for the 60i check and for the decimate.

[real.finder]

I have trouble with post telecine cuts, so I need SC detection to fix that, I guess.

that in 24 fps (film) section? it should not be like this, the 24 fps (film) section is progressive already (matched already and don't need any postprocess too), for 30 fps progressive it same or even easier since it not need undo the soft pulldown, others like hard pulldown and 60i has nothing to do with these d2v things :)

[real.finder]

In the end, I still need to check 5 frames at a time for the 60i check and for the decimate.

just add special property like how tivtc in vs do. btw, even in avs tivtc (and others) use some hack called hints which is in "64 bytes of the frame upper left corner" to help in these cases

[Reel.Deel]

@Ceppo

Not sure if this would be any help but perhaps NTSCanalyze (https://forum.doom9.org/showthread.php?p=1251823#post1251823) can be of some use? Overall it does an OK job, but it's prone to some errors on some frames. There was a discussion on how to improve it here: https://forum.doom9.org/showthread.php?t=173905

[Ceppo]

Not sure if this would be any help but perhaps NTSCanalyze can be of some use? Overall it does an OK job, but it's prone to some errors on some frames. There was a discussion on how to improve it here: https://forum.doom9.org/showthread.php?t=173905

Thanks for the info, but the point is to use the frame properties from the d2v to see if we can improve CTools :)


just add special property like how tivtc in vs do. btw, even in avs tivtc (and others) use some hack called hints which is in "64 bytes of the frame upper left corner" to help in these cases

Indeed, but I still need to call 5 frames to do the check, yet they will be saved in a class variable and called only once :)

that in 24 fps (film) section? it should not be like this, the 24 fps (film) section is progressive already (matched already and don't need any postprocess too), for 30 fps progressive it same or even easier since it not need undo the soft pulldown, others like hard pulldown and 60i has nothing to do with these d2v things

If there is a post telecine cut you get a cycle that is not film anymore :)

However, I wake up to smoke, and I did some other test while smoking. And we have some problems: some progressive frames have TOP/BOTTOM different number, and some interlaced frames have a TOP/BOTTOM same number. It doesn't happen often and after some frame it fixes itsel. I don't know if it is a bug or a Post Telecine Cut madness that confuses the flags in the source. To fix that I had to do the following:

D2VSource("VTS_01_1.d2v")
N = DoubleWeave().SelectOdd()
V1 = vinverse(thr=3)
V2 = N.vinverse(thr=3)
last
ScriptClip("""
ATOP = PropGetInt("_EncodedFrameTop")
ABOT = PropGetInt("_EncodedFrameBottom")
VL = LumaDifference(V1)
VN = LumaDifference(N,V2)
ATOP == ABOT && VL <= VN ? last : VL > VN ? N : last
""")

But it become pointless to use frame property at this point :( If we don't fix those strange flags the d2v flags are not reliable.

[real.finder]

some progressive frames have TOP/BOTTOM different number

well I think that because there are a duplicate frames, aside from they are progressive no mater what and they just soft pulldown anyway but you see them as interlaced after dgindex and D2VSource made them like this, so we need to undo dgindex and D2VSource work using frame properties (_EncodedFrameTop and _EncodedFrameBottom) so we restore the original encoded film (24 fps) progressive frames

anyway we should wait for _FieldBased fix in D2VSource

some interlaced frames have a TOP/BOTTOM same number

This is very normal, it's the case for full hard pulldown or 60i clips

[real.finder]

so we restore the original encoded film (24 fps) progressive frames

in short, we see if there are number difference between _EncodedFrameTop and _EncodedFrameBottom, if we find some we will match their numbers to get the original frame (which is progressive and not need any other work!)

edit: btw, I think there are no true progressive 30fps in dvd, so if _EncodedFrameTop and _EncodedFrameBottom is same and the _FieldBased == 0 (after it fixed) it mean the frame in 24 fps (aka film) progressive section

[real.finder]

If there is a post telecine cut you get a cycle that is not film anymore

ok, I see what you mean here, with d2vSource("VTS_01_2.demuxed.d2v",info=1) I can see there are cases with _EncodedFrameTop and _EncodedFrameBottom not same but "Progr Frame" is false in scene change, in these cases CTools should work as there are no frame properties (as it do now) :)

[Ceppo]

Let's wait for the fix then :p

EDIT: seems to work with this script

D2VSource("VTS_01_1.d2v")
N = DoubleWeave().SelectOdd()
VS = vinverse(thr=3)
VN = N.vinverse(thr=3)
Q = CPostProcessing()
last
ScriptClip("""
ATOP = PropGetInt("_EncodedFrameTop")
ABOT = PropGetInt("_EncodedFrameBottom")
FIELD = PropGetInt("_FieldBased")
VSD = LumaDifference(last,VS)
VND = LumaDifference(N,VN)
ATOP == ABOT && FIELD == 0 && VND >= VSD ? last : \
ATOP != ABOT && FIELD == 0 && VND < VSD ? N : \
ATOP == ABOT && FIELD == 2 && VND < VSD ? N : \
ATOP == ABOT && FIELD == 2 && VND >= VSD ? last : Q
""")
PropShow()

However FIELD == 2 has false positive and false negative, when ATOP != ABOT, so with the comb check of CPostProcessing we can avoid that, but I think we need more prop to avoid using vinverse/CpostProcessing to gain speed. (vinverse is used for post telecine cuts).

@real.finder
Please test the script on a source to see what I mean and if you spot something off :cool:

EDIT:
Forgot to say that we still have strange things going on and we need vinverse for the final check. This should be fixed, otherwise, there is little advantage from doing this.

[real.finder]

with Asd-g fix


D2VSource("VTS_01_1.d2v")
N = DoubleWeave()
Q = CTelecine.CPostProcessing()
ScriptClip("""
ATOP = PropGetInt("_EncodedFrameTop")
ABOT = PropGetInt("_EncodedFrameBottom")
FIELD = PropGetInt("_FieldBased")
ATOP < ABOT && FIELD == 0 ? N.SelectEven() : \
ATOP > ABOT && FIELD == 0 ? N.SelectOdd() : FIELD == 0 ? last : Q.Subtitle("Not Soft",400)
""")
PropShow()

[manono]

I figured you were writing about a mix of hard and soft telecine. That, of course, is fixed with a simple IVTC. But you have interlaced credits. To which I'd say, "So what? Who cares about credits?"

The easiest way to handle something like this is to bob the whole thing to 59.94fps. That's how a television will play it anyway. As I mentioned, I don't do VFR, so I'd get the credits to 23.976fps as well, although that will leave behind some slight jerkiness. I used this script:

A=MPEG2Source("Movie.d2v")
B=A.TFM(d2v="Movie.d2v").TDecimate(Mode=1)
B=B.Trim(0,4281)#after IVTC
C=A.Trim(5352,9210)#while still at 29.97fps
C=C.QTGMC#or your favorite bobber
C=C.SRestore(Frate=23.976)
B+C

https://www.sendspace.com/file/j8uyny

[Ceppo]

@real.finder
With your script, I get a false N match ad wrong post telecine cuts. Still needs vinverse field matching. Also I get same ATOP/ABOT frame but combed that should be a n match. You can find everything here:
https://www.mediafire.com/file/5v5wopo7w28bwwm/INTRO.demuxed.m2v/file
The metrics are not reliable, so we need to double check with a field matching pass. Both can do it wrong, if I get a XOR who should win? That's the problem.

@manono
We are just making tests to write a plugin that does the field matching and other things based on the d2v flags. We don't care about that source, is just a very hard one to nail down.

[real.finder]

vinverse or any others will kill the topic since we need to do "undo soft pulldown" only which mean it should be faster and no PostProcessing

and yes my script is bad, seems it can't be done in script level (at least with simple code) also DoubleWeave not suitable here (it even don't change _EncodedFrameBottom and _EncodedFrameTop since it not know them!)

it need to be done in plugin level but instead of doing the maths with the pixels it should be done with the _EncodedFrameBottom and _EncodedFrameTop to match them, of course only when _FieldBased = 0!

[Ceppo]

Done it:
https://www.mediafire.com/file/p9wgb3jcrtziwwb/CTelecineProps.dll/file

#include "CTelecineProp.h"

CTelecineProp::CTelecineProp(PClip _child, IScriptEnvironment* env) : GenericVideoFilter(_child)
{
N = env->Invoke("DoubleWeave", AVSValue({ child }, 1)).AsClip();
}

PVideoFrame __stdcall CTelecineProp::GetFrame(int n, IScriptEnvironment* env)
{
int err;
CTelecinePropsFrame src;
src.Frame = child->GetFrame(n, env);
src.FrameProps = env->getFramePropsRO(src.Frame);
src.FieldBased = env->propGetInt(src.FrameProps, "_FieldBased", 0, &err);
src.EncodedFrameTop = env->propGetInt(src.FrameProps, "_EncodedFrameTop", 0, &err);
src.EncodedFrameBottom = env->propGetInt(src.FrameProps, "_EncodedFrameBottom", 0, &err);

if (!src.FieldBased && src.EncodedFrameTop != src.EncodedFrameBottom)
{
PVideoFrame nxt = N->GetFrame(n * 2 + 1, env);
return nxt;
}
else
{
return src.Frame;
}
}

What next?
Since we care about "soft Pulldown", I can check each cycle for a nnccc,cnncc,ccnnc,cccnn, ncccn pattern and use the standard field matching method for the not "soft" cycles, what do you think?

[real.finder]

Since we care about "soft Pulldown", I can check each cycle for a nnccc,cnncc,ccnnc,cccnn, ncccn pattern and use the standard field matching method for the not "soft" cycles, what do you think?

but in the case of your sample the not soft most of times are 60i or ifade, but seems fine since we can use CPostProcessing :)

btw, CTelecineProp seems to forget to edit _EncodedFrameBottom and _EncodedFrameTop same as DoubleWeave!

[Ceppo]

You mean, to change the prop if is a n match? I will later, let's nail the matching first.

[real.finder]

You mean, to change the prop if is a n match? I will later, let's nail the matching first.

yes, also how about the p match?

[Ceppo]

That's needed only if bob=true, you just change DoubleWeave GetFrame number and you have a P match.

[Ceppo]

https://www.mediafire.com/file/oa2rajsb0y0q9pl/CTelecineProps.dll/file

#include "CTelecineProp.h"

CTelecineProp::CTelecineProp(PClip _child, IScriptEnvironment* env) : GenericVideoFilter(_child)
{
N = env->Invoke("DoubleWeave", AVSValue({ child }, 1)).AsClip();
}

PVideoFrame __stdcall CTelecineProp::GetFrame(int n, IScriptEnvironment* env)
{
int err;
int m = n / 5 * 5;
for (int i = 0; i < 5; ++i)
{
if (Cycle[i].FrameNumber != m + i)
{
Cycle[i].Frame = child->GetFrame(m + i, env);
Cycle[i].FrameNumber = m + i;
Cycle[i].FrameProps = env->getFramePropsRW(Cycle[i].Frame);
Cycle[i].FieldBased = env->propGetInt(Cycle[i].FrameProps, "_FieldBased", 0, &err);
Cycle[i].EncodedFrameTop = env->propGetInt(Cycle[i].FrameProps, "_EncodedFrameTop", 0, &err);
Cycle[i].EncodedFrameBottom = env->propGetInt(Cycle[i].FrameProps, "_EncodedFrameBottom", 0, &err);
}
}

// NNCCC,CNNCC,CCNNC,CCCNN, NCCCN HUNTING
bool A = !Cycle[0].FieldBased && Cycle[0].EncodedFrameTop != Cycle[0].EncodedFrameBottom;
bool B = !Cycle[1].FieldBased && Cycle[1].EncodedFrameTop != Cycle[1].EncodedFrameBottom;
bool C = !Cycle[2].FieldBased && Cycle[2].EncodedFrameTop != Cycle[2].EncodedFrameBottom;
bool D = !Cycle[3].FieldBased && Cycle[3].EncodedFrameTop != Cycle[3].EncodedFrameBottom;
bool E = !Cycle[4].FieldBased && Cycle[4].EncodedFrameTop != Cycle[4].EncodedFrameBottom;

if (A && B && !C && !D && !E)
{
Cycle[0].isN = true;
Cycle[1].isN = true;
Cycle[2].isN = false;
Cycle[3].isN = false;
Cycle[4].isN = false;
}
else if (!A && B && C && !D && !E)
{
Cycle[0].isN = false;
Cycle[1].isN = true;
Cycle[2].isN = true;
Cycle[3].isN = false;
Cycle[4].isN = false;
}
else if (!A && !B && C && D && !E)
{
Cycle[0].isN = false;
Cycle[1].isN = false;
Cycle[2].isN = true;
Cycle[3].isN = true;
Cycle[4].isN = false;
}
else if (!A && !B && !C && D && E)
{
Cycle[0].isN = false;
Cycle[1].isN = false;
Cycle[2].isN = false;
Cycle[3].isN = true;
Cycle[4].isN = true;
}
else if (A && !B && !C && !D && E)
{
Cycle[0].isN = true;
Cycle[1].isN = false;
Cycle[2].isN = false;
Cycle[3].isN = false;
Cycle[4].isN = true;
}
else
{
//STANDARD FIELD MATCHING.
}

if (Cycle[n % 5].isN)
{
PVideoFrame nxt = N->GetFrame(n * 2 + 1, env);
return nxt;
}
else
{
return Cycle[n % 5].Frame;
}
}

By taking 5 frames every time we can flag the frame to decimate and generate a VFR file with CTelecine. So yes, it can be done in 1-pass.

[real.finder]

By taking 5 frames every time we can flag the frame to decimate and generate a VFR file with CTelecine. So yes, it can be done in 1-pass.

that nice, also did the last update do deal with hard Pulldown? I note they still combed

[real.finder]

btw, I note that INTRO.demuxed is kinda damaged or incorrectly produced! it showed as BFF and has frames that hard Pulldown flagged as soft Pulldown! even with d2v directly tfm(d2v="INTRO.demuxed.d2v",pp=0) it still wrong

[Ceppo]

This seems to be the second pokemon movie, they took the first movie and re-edited it so I guess it is damaged due to this.

I added the field matching for the not soft cycles.

I guess we will just CPostProcessing absurd cases :(

I have minor tuning for CQTGMC when I'm done with this I will update it.

[Ceppo]

https://www.mediafire.com/file/yapvlj69igs5skg/CToolsTest.dll/file

I encoded the whole pokemon movie as follow:

CTelecineProps()
CDecimateProps()

To generate a VFR file and decimate in VFR in 1-pass the frames must be called from the first frame to the last only once and linearly. You need also to trim at the end since in 1-pass I can't set the frame count since I don't know yet what it is.

The movie had many section flagged as "soft" while they where not. Post telecine cuts where a problem also. I didn't see any shuttering from wrong decimate.

Now we have a problem, if we CPostProcessing the props of the frame got wiped out, how do we fix that?

[Ceppo]

https://www.mediafire.com/file/sj3qtcdl5uimv41/CToolsTest.dll/file
BOB version, will deinterlaced 60i content with bob edeint clip.

I have 3 problems:
1) How to deal with post telecine cuts interlaced frames.
2) How to deal with incorrectly flagged as soft cycles.
3) I don't know how to copy the props to the edeint clip.

Here the code.

#pragma once
#include <cmath>
#include <fstream>
#include <iostream>
#include <string>
#include <windows.h>
#include "avisynth.h"
using namespace std;

struct CTelecinePropsFrame
{
PVideoFrame Frame;
PVideoFrame FrameMatch;
PVideoFrame FrameMatchPrev;
PVideoFrame FrameVinverse;
PVideoFrame FrameMatchVinverse;
PVideoFrame BOB_ODD;
PVideoFrame BOB_EVN;

int FrameNumber = -1;
bool isN = false;
bool isP = false;
bool is60i = false;
bool isDecimateN = false;
bool isDecimateP = false;

bool isSC = false;
PVideoFrame FrameSC;
const unsigned char* FrameSCP = nullptr;

AVSMap* FrameProps = nullptr;
AVSMap* FramePropsMatch = nullptr;
AVSMap* FramePropsMatchPrev = nullptr;
int64_t FieldBased = -1;
int64_t EncodedFrameTop = -1;
int64_t EncodedFrameBottom = -1;
};

class CTelecineProps : public GenericVideoFilter
{
PClip N;
PClip CV;
PClip NV;
PClip SCClip;
PClip edeint;
CTelecinePropsFrame Cycle[5];

int offset;
ofstream file;

public:
CTelecineProps(PClip _child, PClip _edeint, IScriptEnvironment* env);
PVideoFrame __stdcall GetFrame(int f, IScriptEnvironment* env);
};

bool CTelecineProps_isN(PVideoFrame& c, PVideoFrame& cv, PVideoFrame& n, PVideoFrame& nv, int nt, int ntN, bool sse, bool isY8);



#include "CTelecineProps.h"

CTelecineProps::CTelecineProps(PClip _child, PClip _edeint, IScriptEnvironment* env) : GenericVideoFilter(_child), edeint(_edeint)
{
offset = 0;
file.open("VFR.txt", ios::trunc);
file << "# timecode format v1\n";
file << "Assume 23.976000\n";
N = env->Invoke("DoubleWeave", AVSValue({ child }, 1)).AsClip();
CV = env->Invoke("Vinverse", AVSValue({ child }, 1)).AsClip();
NV = env->Invoke("Vinverse", AVSValue({ N }, 1)).AsClip();

int blksize = 64;
int thSCD1 = 400;
int thSCD2 = 130;
AVSValue sup_args[1] = { child };
PClip Super = env->Invoke("MSuper", AVSValue(sup_args, 1)).AsClip();
AVSValue vec_args[5] = { Super, blksize, blksize / 2, false, 1 };
const char* vec_names[5] = { NULL, "blksize", "overlap", "isb", "delta" };
PClip forwvec = env->Invoke("MAnalyse", AVSValue(vec_args, 5), vec_names).AsClip();
AVSValue sClip_args[4] = { child, forwvec, thSCD1, thSCD2 };
const char* sClip_names[4] = { NULL, NULL, "thSCD1", "thSCD2" };
SCClip = env->Invoke("MSCDetection", AVSValue(sClip_args, 4), sClip_names).AsClip();

if (!edeint)
{
try
{
edeint = env->Invoke("QTGMC", AVSValue({ child }, 1)).AsClip();
}
catch (IScriptEnvironment::NotFound)
{
env->ThrowError("CTelecineProps: QTGMC(.avsi) function not found!");
}
}

vi.num_frames *= 2;
vi.SetFPS(vi.fps_numerator * 2, vi.fps_denominator);
}

PVideoFrame __stdcall CTelecineProps::GetFrame(int f, IScriptEnvironment* env)
{
int n = f / 2;
int m = n / 5 * 5;
int o = Cycle[0].FrameNumber;

for (int i = 0; i < 5; ++i)
{
if (Cycle[i].FrameNumber != m + i)
{
Cycle[i].Frame = child->GetFrame(min(m + i, vi.num_frames), env);
Cycle[i].FrameSC = SCClip->GetFrame(min(m + i, vi.num_frames), env);
Cycle[i].FrameSCP = Cycle[i].FrameSC->GetReadPtr(PLANAR_Y);
Cycle[i].isSC = min(1, Cycle[i].FrameSCP[0]);
Cycle[i].FrameVinverse = CV->GetFrame(min(m + i, vi.num_frames), env);
Cycle[i].FrameMatch = N->GetFrame(min(m + i, vi.num_frames) * 2 + 1, env);
Cycle[i].FrameMatchVinverse = NV->GetFrame(min(m + i, vi.num_frames) * 2 + 1, env);
Cycle[i].FrameMatchPrev = i == 0 ? N->GetFrame(max(min(m + i, vi.num_frames) * 2 - 1, 0), env) : Cycle[i - 1].FrameMatch;
Cycle[i].FrameNumber = m + i;
Cycle[i].FrameProps = env->getFramePropsRW(Cycle[i].Frame);
Cycle[i].FramePropsMatch = env->getFramePropsRW(Cycle[i].FrameMatch);
Cycle[i].FramePropsMatchPrev = env->getFramePropsRW(Cycle[i].FrameMatchPrev);
Cycle[i].FieldBased = env->propGetInt(Cycle[i].FrameProps, "_FieldBased", 0, nullptr);
Cycle[i].EncodedFrameTop = env->propGetInt(Cycle[i].FrameProps, "_EncodedFrameTop", 0, nullptr);
Cycle[i].EncodedFrameBottom = env->propGetInt(Cycle[i].FrameProps, "_EncodedFrameBottom", 0, nullptr);
}
}

if (o != m)
{
bool SCA = !Cycle[0].isSC;
bool SCB = !Cycle[0].isSC;
bool SCC = !Cycle[0].isSC;
bool SCD = !Cycle[0].isSC;
bool SCE = !Cycle[0].isSC;

bool A = !Cycle[0].FieldBased && Cycle[0].EncodedFrameTop != Cycle[0].EncodedFrameBottom;
bool B = !Cycle[1].FieldBased && Cycle[1].EncodedFrameTop != Cycle[1].EncodedFrameBottom;
bool C = !Cycle[2].FieldBased && Cycle[2].EncodedFrameTop != Cycle[2].EncodedFrameBottom;
bool D = !Cycle[3].FieldBased && Cycle[3].EncodedFrameTop != Cycle[3].EncodedFrameBottom;
bool E = !Cycle[4].FieldBased && Cycle[4].EncodedFrameTop != Cycle[4].EncodedFrameBottom;

bool F = Cycle[0].FieldBased && Cycle[0].EncodedFrameTop != Cycle[0].EncodedFrameBottom;
bool G = Cycle[1].FieldBased && Cycle[1].EncodedFrameTop != Cycle[1].EncodedFrameBottom;
bool H = Cycle[2].FieldBased && Cycle[2].EncodedFrameTop != Cycle[2].EncodedFrameBottom;
bool I = Cycle[3].FieldBased && Cycle[3].EncodedFrameTop != Cycle[3].EncodedFrameBottom;
bool L = Cycle[4].FieldBased && Cycle[4].EncodedFrameTop != Cycle[4].EncodedFrameBottom;

bool M = !Cycle[0].FieldBased && Cycle[0].EncodedFrameTop == Cycle[0].EncodedFrameBottom;
bool N = !Cycle[1].FieldBased && Cycle[1].EncodedFrameTop == Cycle[1].EncodedFrameBottom;
bool O = !Cycle[2].FieldBased && Cycle[2].EncodedFrameTop == Cycle[2].EncodedFrameBottom;
bool P = !Cycle[3].FieldBased && Cycle[3].EncodedFrameTop == Cycle[3].EncodedFrameBottom;
bool Q = !Cycle[4].FieldBased && Cycle[4].EncodedFrameTop == Cycle[4].EncodedFrameBottom;

Cycle[0].isN = false;
Cycle[1].isN = false;
Cycle[2].isN = false;
Cycle[3].isN = false;
Cycle[4].isN = false;
Cycle[0].isP = false;
Cycle[1].isP = false;
Cycle[2].isP = false;
Cycle[3].isP = false;
Cycle[4].isP = false;
Cycle[0].is60i = false;
Cycle[1].is60i = false;
Cycle[2].is60i = false;
Cycle[3].is60i = false;
Cycle[4].is60i = false;
Cycle[0].isDecimateN = false;
Cycle[1].isDecimateN = false;
Cycle[2].isDecimateN = false;
Cycle[3].isDecimateN = false;
Cycle[4].isDecimateN = false;
Cycle[0].isDecimateP = true;
Cycle[1].isDecimateP = true;
Cycle[2].isDecimateP = true;
Cycle[3].isDecimateP = true;
Cycle[4].isDecimateP = true;

if (A && B && !C && !D && !E && SCA && SCB && SCC && SCD && SCE)
{
Cycle[0].isN = true;
Cycle[1].isN = true;
Cycle[0].isP = true;
Cycle[1].isP = true;
Cycle[1].isDecimateN = true;
offset += 1;
}
else if (!A && B && C && !D && !E && SCA && SCB && SCC && SCD && SCE)
{
Cycle[1].isN = true;
Cycle[2].isN = true;
Cycle[1].isP = true;
Cycle[2].isP = true;
Cycle[2].isDecimateN = true;
offset += 1;
}
else if (!A && !B && C && D && !E && SCA && SCB && SCC && SCD && SCE)
{
Cycle[2].isN = true;
Cycle[3].isN = true;
Cycle[2].isP = true;
Cycle[3].isP = true;
Cycle[3].isDecimateN = true;
offset += 1;
}
else if (!A && !B && !C && D && E && SCA && SCB && SCC && SCD && SCE)
{
Cycle[3].isN = true;
Cycle[4].isN = true;
Cycle[3].isP = true;
Cycle[4].isP = true;
Cycle[4].isDecimateN = true;
offset += 1;
}
else if (A && !B && !C && !D && E && SCA && SCB && SCC && SCD && SCE)
{
Cycle[0].isN = true;
Cycle[4].isN = true;
Cycle[0].isP = true;
Cycle[4].isP = true;
Cycle[0].isDecimateN = true;
offset += 1;
}
else if (M && N && O && P && Q)
{
file << Cycle[0].FrameNumber + offset << "," << Cycle[0].FrameNumber + offset + 4 << "," << "29.970000\n";
}
else if (F && G && H && I && L)
{
Cycle[0].is60i = true;
Cycle[1].is60i = true;
Cycle[2].is60i = true;
Cycle[3].is60i = true;
Cycle[4].is60i = true;
Cycle[0].isDecimateP = false;
Cycle[1].isDecimateP = false;
Cycle[2].isDecimateP = false;
Cycle[3].isDecimateP = false;
Cycle[4].isDecimateP = false;
file << Cycle[0].FrameNumber + offset << "," << Cycle[0].FrameNumber + offset + 9 << "," << "59.940000\n";
for (int i = 0; i < 5; ++i)
{
Cycle[i].BOB_EVN = edeint->GetFrame(min((m + i) * 2, vi.num_frames), env);
Cycle[i].BOB_ODD = edeint->GetFrame(min((m + i) * 2 + 1, vi.num_frames), env);
}
}
else
{
Cycle[0].isN = Cycle[0].isP = CTelecineProps_isN(Cycle[0].Frame, Cycle[0].FrameVinverse, Cycle[0].FrameMatch, Cycle[0].FrameMatchVinverse, 0, 0, true, vi.IsY8());
Cycle[1].isN = Cycle[1].isP = CTelecineProps_isN(Cycle[1].Frame, Cycle[1].FrameVinverse, Cycle[1].FrameMatch, Cycle[1].FrameMatchVinverse, 0, 0, true, vi.IsY8());
Cycle[2].isN = Cycle[2].isP = CTelecineProps_isN(Cycle[2].Frame, Cycle[2].FrameVinverse, Cycle[2].FrameMatch, Cycle[2].FrameMatchVinverse, 0, 0, true, vi.IsY8());
Cycle[3].isN = Cycle[3].isP = CTelecineProps_isN(Cycle[3].Frame, Cycle[3].FrameVinverse, Cycle[3].FrameMatch, Cycle[3].FrameMatchVinverse, 0, 0, true, vi.IsY8());
Cycle[4].isN = Cycle[4].isP = CTelecineProps_isN(Cycle[4].Frame, Cycle[4].FrameVinverse, Cycle[4].FrameMatch, Cycle[4].FrameMatchVinverse, 0, 0, true, vi.IsY8());

if (Cycle[0].isN && Cycle[4].isN)
{
Cycle[0].isDecimateN = true;
offset += 1;
}
else if (Cycle[3].isN && Cycle[4].isN)
{
Cycle[4].isDecimateN = true;
offset += 1;
}
else if (Cycle[2].isN && Cycle[3].isN)
{
Cycle[3].isDecimateN = true;
offset += 1;
}
else if (Cycle[1].isN && Cycle[2].isN)
{
Cycle[2].isDecimateN = true;
offset += 1;
}
else if (Cycle[0].isN && Cycle[1].isN)
{
Cycle[1].isDecimateN = true;
offset += 1;
}
else if (Cycle[0].isN && !Cycle[1].isN && !Cycle[2].isN && !Cycle[3].isN && !Cycle[4].isN)
{
Cycle[0].isDecimateN = true;
offset += 1;
}
else if (!Cycle[0].isN && Cycle[1].isN && !Cycle[2].isN && !Cycle[3].isN && !Cycle[4].isN)
{
Cycle[1].isDecimateN = true;
offset += 1;
}
else if (!Cycle[0].isN && !Cycle[1].isN && Cycle[2].isN && !Cycle[3].isN && !Cycle[4].isN)
{
Cycle[2].isDecimateN = true;
offset += 1;
}
else if (!Cycle[0].isN && !Cycle[1].isN && !Cycle[2].isN && Cycle[3].isN && !Cycle[4].isN)
{
Cycle[3].isDecimateN = true;
offset += 1;
}
else if (!Cycle[0].isN && !Cycle[1].isN && !Cycle[2].isN && !Cycle[3].isN && Cycle[4].isN)
{
Cycle[4].isDecimateN = true;
offset += 1;
}
else
{
Cycle[0].is60i = true;
Cycle[1].is60i = true;
Cycle[2].is60i = true;
Cycle[3].is60i = true;
Cycle[4].is60i = true;
Cycle[0].isDecimateP = false;
Cycle[1].isDecimateP = false;
Cycle[2].isDecimateP = false;
Cycle[3].isDecimateP = false;
Cycle[4].isDecimateP = false;
file << Cycle[0].FrameNumber + offset << "," << Cycle[0].FrameNumber + offset + 9 << "," << "59.940000\n";
for (int i = 0; i < 5; ++i)
{
Cycle[i].BOB_EVN = edeint->GetFrame(min((m + i) * 2, vi.num_frames), env);
Cycle[i].BOB_ODD = edeint->GetFrame(min((m + i) * 2 + 1, vi.num_frames), env);
}
}
}
}

f == vi.num_frames - 1 ? file.close() : NULL;

if (f & 1 && Cycle[n % 5].is60i)
{
return Cycle[n % 5].BOB_ODD;
}
else if (f & 1)
{
!Cycle[n % 5].isN ? env->propSetInt(Cycle[n % 5].FrameProps, "_FieldMatch", 0, 0) : NULL;
!Cycle[n % 5].isN ? env->propSetInt(Cycle[n % 5].FrameProps, "_SceneChange", Cycle[n % 5].isSC ? 1 : 0, 0) : NULL;
!Cycle[n % 5].isN ? env->propSetInt(Cycle[n % 5].FrameProps, "_FrameToDecimate", 0, 0) : NULL;
Cycle[n % 5].isN ? env->propSetInt(Cycle[n % 5].FramePropsMatch, "_FieldMatch", 1, 0) : NULL;
Cycle[n % 5].isN ? env->propSetInt(Cycle[n % 5].FramePropsMatch, "_SceneChange", Cycle[n % 5].isSC ? 1 : 0, 0) : NULL;
Cycle[n % 5].isN ? env->propSetInt(Cycle[n % 5].FramePropsMatch, "_FrameToDecimate", Cycle[n % 5].isDecimateN ? 1 : 0, 0) : NULL;

bool order = child->GetParity(n);
Cycle[n % 5].isN && order ? env->propSetInt(Cycle[n % 5].FramePropsMatch, "_EncodedFrameTop", Cycle[n % 5].EncodedFrameTop + 1, 0) : NULL;
Cycle[n % 5].isN && !order ? env->propSetInt(Cycle[n % 5].FramePropsMatch, "_EncodedFrameBottom", Cycle[n % 5].EncodedFrameBottom + 1, 0) : NULL;

return Cycle[n % 5].isN ? Cycle[n % 5].FrameMatch : Cycle[n % 5].Frame;
}
else if (Cycle[n % 5].is60i)
{
return Cycle[n % 5].BOB_EVN;
}
else
{
!Cycle[n % 5].isP ? env->propSetInt(Cycle[n % 5].FrameProps, "_FieldMatch", 0, 0) : NULL;
!Cycle[n % 5].isP ? env->propSetInt(Cycle[n % 5].FrameProps, "_SceneChange", Cycle[n % 5].isSC ? 1 : 0, 0) : NULL;
!Cycle[n % 5].isP ? env->propSetInt(Cycle[n % 5].FrameProps, "_FrameToDecimate", Cycle[n % 5].isDecimateP ? 1 : 0, 0) : NULL;
Cycle[n % 5].isP ? env->propSetInt(Cycle[n % 5].FramePropsMatchPrev, "_FieldMatch", 2, 0) : NULL;
Cycle[n % 5].isP ? env->propSetInt(Cycle[n % 5].FramePropsMatchPrev, "_SceneChange", 0, 0) : NULL;
Cycle[n % 5].isP ? env->propSetInt(Cycle[n % 5].FramePropsMatchPrev, "_FrameToDecimate", Cycle[n % 5].isDecimateP ? 1 : 0, 0) : NULL;

bool order = child->GetParity(n);
Cycle[n % 5].isP && order ? env->propSetInt(Cycle[n % 5].FramePropsMatchPrev, "_EncodedFrameBottom", Cycle[n % 5].EncodedFrameBottom - 1, 0) : NULL;
Cycle[n % 5].isP && !order ? env->propSetInt(Cycle[n % 5].FramePropsMatchPrev, "_EncodedFrameTop", Cycle[n % 5].EncodedFrameTop - 1, 0) : NULL;

return Cycle[n % 5].isP ? Cycle[n % 5].FrameMatchPrev : Cycle[n % 5].Frame;
}
}

bool CTelecineProps_isN(PVideoFrame& c, PVideoFrame& cv, PVideoFrame& n, PVideoFrame& nv, int nt, int ntN, bool sse, bool isY8)
{
const unsigned char* cp;
const unsigned char* cvp;
const unsigned char* np;
const unsigned char* nvp;

int p;
int x, y;
int temp[2];
int height;
int row_size;
int c_pitch, cv_pitch;
int n_pitch, nv_pitch;
int planes[] = { PLANAR_Y, PLANAR_U, PLANAR_V };
int64_t sum[3][2] = { {0,0}, {0,0}, {0,0} };

for (p = 0; p < 3; p++)
{
if (p > 0 && isY8)
{
break;
}

cp = c->GetReadPtr(planes[p]);
cvp = cv->GetReadPtr(planes[p]);
np = n->GetReadPtr(planes[p]);
nvp = nv->GetReadPtr(planes[p]);

c_pitch = c->GetPitch(planes[p]);
cv_pitch = cv->GetPitch(planes[p]);
n_pitch = n->GetPitch(planes[p]);
nv_pitch = nv->GetPitch(planes[p]);

height = c->GetHeight(planes[p]);
row_size = c->GetRowSize(planes[p]);

for (y = 0; y < height; y += 1)
{
for (x = 0; x < row_size; x += 1)
{
temp[0] = abs(cp[x] - cvp[x]);
temp[1] = abs(np[x] - nvp[x]);
temp[0] = temp[0] < nt ? 0 : temp[0];
temp[1] = temp[1] < ntN ? 0 : temp[1];
sum[p][0] += sse ? temp[0] * temp[0] : temp[0];
sum[p][1] += sse ? temp[1] * temp[1] : temp[1];
}
cp += c_pitch;
cvp += cv_pitch;
np += c_pitch;
nvp += cv_pitch;
}
}
return isY8 ? sum[0][1] < sum[0][0]
: sum[0][1] + sum[1][1] + sum[2][1] < sum[0][0] + sum[1][0] + sum[2][0];
}


EDIT:
For 1) I will calculate P match on the last frame before a SC, if P win, I will replace the C/N match with a P match.
For 3) I will just add the Decimate property to the edeint clip for the moment.
For 2) Plz help :(

[StainlessS]

3) I don't know how to copy the props to the edeint clip.
Neither do I, but below might point in right direction.

All intenal Prop related functions, extracted from v3.7.2 [there may also be alternative prototypes, but we can only extract the first one]

propClearAll "c"
propCopy "cc[merge]b[props]s+"
propDelete "cs+"
propGetAll "c[offset]i"
propGetAny "cs[index]i[offset]i"
propGetAsArray "cs[offset]i"
propGetClip "cs[index]i[offset]i"
propGetDataSize "cs[index]i[offset]i"
propGetFloat "cs[index]i[offset]i"
propGetInt "cs[index]i[offset]i"
propGetKeyByIndex "c[index]i[offset]i"
propGetString "cs[index]i[offset]i"
propGetType "cs[offset]i"
propNumElements "cs[offset]i"
propNumKeys "c[offset]i"
propSet "csc[mode]i"
propSetArray "csn"
propSetClip "csn[mode]i"
propSetFloat "csn[mode]i"
propSetInt "csn[mode]i"
propSetString "csn[mode]i"
propShow "c[size]i[showtype]b"


http://avisynth.nl/index.php/Internal_functions

propCopy

propCopy(clip, clip [,bool "merge"]) AVS+(v3.7.1)
Copies the frame properties of the second clip to the first.

Parameter 'merge' (default false):

when false: exact copy (original target properties will be lost)
when true: keeps original properties, appends all parameters from source but overwrite if a parameter with the same name already exists.


EDIT: from here[describing the 1st code block prototype stuff]:- https://forum.doom9.org/showthread.php?p=1953625#post1953625

RT_PluginParam(String FunctionName)
FunctionName, String, name of Avisynth built-in or Plugin filter or function name.
Returns "", if FunctionName Parameter string not found.
Returns String, the Avisynth CPP Interface style argument specifier string used by Avisynth to determine argument types and optional names.
Optional arguments have square brackets surrounding their name as in [name] and are followed by a type specifier character that gives
the type. Unnamed arguments are not optional. eg "cc[arg1]b[arg2]i" would be two compulsory unnamed clip args, followed by optional
'arg1' of type bool and optional 'arg2' of type int.
# Argument type specifier strings.
c - Video Clip
i - Integer number
f - Float number
s - String
b - boolean
. - Any type (dot)
# Array Specifiers
i* - Integer Array, zero or more
i+ - Integer Array, one or more
.* - Any type Array, zero or more
.+ - Any type Array, one or more
# Etc
To show params:-
colorbars().Killaudio()
S=RT_PluginParam("RT_YStats")
S=RT_StrReplace(S,"[",Chr(10)+"[")
RT_Subtitle(s)

Above description does not include any of the new builtin Array or function object name, or whatever other new stuff is implemented in arg specifiers.

[real.finder]

For 2) just add an option to do additional field matching after the soft matching and maybe with another additional check to do Post Processing later if the source is that bad! since I didn't see that bad source like the sample you got :) I think it the same for "For 1)" and isn't safe to always do P match for the soft cases?

For 3) just do the CTelecine frame properties and ignore the edeint one since I think there will be nothing important in it

also why made new function for the soft case?! why not just update CTelecine and the others to support it, with an option to ignore _FieldBased, _EncodedFrameBottom and _EncodedFrameTop in CTelecine and another option to use the text log file for Decimate and 60i instead of frame properties to back to ctools 1.2.2 behavior

[Ceppo]

@StainlessS
Thanks, I will read it tomorrow, already got my beer :D

@real.finder
I think P is not safer since if you got a post telecine cut and the first frame of the scene is a n match, you get the same problem, so nothing changes.

If I do field matching check, it will undo the soft ones, so how should I do it? if they have 1 n match in common, I keep the soft one? or what?

I made a new function to experiment with it, if I modded CTelecine it would become a bug nightmare :(

[real.finder]

I think P is not safer since if you got a post telecine cut and the first frame of the scene is a n match, you get the same problem, so nothing changes.

If I do field matching check, it will undo the soft ones, so how should I do it? if they have 1 n match in common, I keep the soft one? or what?
how tfm deal with it? I note without pp=0 it work, maybe it do check if it comb after the soft ones and do field matching if it comb

btw, kinda off-topic but there are cFrameDiff in tivtc, can this be ported to ctools with update as cLessComb that have two clips inputs and chose the less comb one as the output? this can be used with soft ones vs field matching :) also in another advanced use case like qtgmc vs cqtgmc or qtgmc vs yadifmod2 or whatever

[Ceppo]

I will try the last idea, if this doesn't work, it's a happy drop, since comb detection is not reliable in my opinion.

About the filter, I think that the most deinterlaced will look less combed so you are getting the worst output if you do so. And in QTGMC VS CQTGMC, QTGMC will win cuz CQTGMC has sharper edges.

[real.finder]

seems this is how tfm work

flags -

Controls how much of the info from the d2v file is used when the "d2v" parameter is set.
Possible options:

0 - Check the d2v file for illegal transitions and set the order parameter if it
is not already manually set. Also, pass on rff flag duplicate info to
tdecimate.
1 - Same as 0, plus use the trf flags for field matching in film sections (sections
where the trf flags follow the 012301... pattern)
2 - Same as 1, but use the trf flags for field matching in all areas (doesn't have
to be in 0123 pattern) (very much not recommended!)
3 - Same as 0, but don't pass on any info to tdecimate (i.e. only set order and check
for illegal transitions)
4 - Same as 1, but d2v matches are checked for being combed. If a d2v match is detected
as combed then tfm uses its own matching routine for that frame.
5 - Same as 4, but d2v matches are only checked for being combed around scenechanges.

VERY IMPORTANT (MUST READ): For options 0, 1, 2, 4, and 5 to work correctly, tfm/tdecimate
must either immediately follow the mpeg2source() command or
any filters inbetween them must not modify the order or number
of fields in the stream in any way. Or use a trimIn file to
tell TFM what frames have been discarded.

Default: 4 (int)


d2vsource maybe need an update? specifically adding rff flag to frame properties

[Ceppo]

Yes, it's worth a shot. I think we need both the TFF and RFF flags, can you make the request? I hate GitHub lol

[real.finder]

Yes, it's worth a shot. I think we need both the TFF and RFF flags, can you make the request? I hate GitHub lol

done https://github.com/Asd-g/MPEG2DecPlus/issues/9

[Ceppo]

TFF flag?

Also try https://www.mediafire.com/file/blc719mfvmjaj0b/CToolsTest.dll/file


CD2VPropFix(write=true)


CD2VPropFix(inputTxt=true)


Anything strange is flagged as uuuuu and we will use manual fieldmatching. Did write/input way in the case I needed to do some huge temporal check, but it's too random to do it :\

[real.finder]

TFF flag?

isn't that already in _FieldBased=2 ? RFF seems mean "Repeat First Field" https://forum.doom9.org/showthread.php?p=154482#post154482