An idea for an IVTC filter

[Ceppo]

I have an idea for an IVTC filter and since is pretty simple I was wondering why nobody ever did that thus I'm asking what are the downsides to this approach.

Basically:
You take all the c and n match metrics for each frame then you divide the video into a cycle of 5 frames. For each cycle, you try the average metric for each pattern (ccnnc and so on), the pattern with the lowest metric is used for the field matching and for the decimate.

I tried taking the data with avisynth and then process them with a python script then I tried it on a constant pattern DVD and it guesses the pattern right. Also I tried it on a DVD with post telecide cuts and 60i animation and I got the ccccc match right.

So I was wandering why is it a bad idea?

Edit: I mean that the pattern for each cycle is chosen this way, not the pattern for the whole clip, just to be sure I was clear.

[johnmeyer]

Have you used:

TFM()
TDecimate()

?

Dozens of customization settings. Has worked every time for me.

You may be reinventing the wheel.

[Ceppo]

Have you used:

TFM()
TDecimate()

?

Dozens of customization settings. Has worked every time for me.

You may be reinventing the wheel.

Yes, I used it. It is true that most times it works well but if it was perfect it would not explain why people would use yatta to do ivtc semi-manually. So I decided to look into it and I got this result.

[StainlessS]

Provide sample and demo script where TMF sometimes gets it wrong and you get it right.
[~30 secs with plenty of movement]

[Ceppo]

I had to rewrite the whole thing from 0 since I can't find my script. Now I couldn't get my hands on a file with post telecide cuts, 60i animations and vfr, I only have constant pattern sources so as long someone is not willing to help me giving me one I can't work on the script for those type of sources.

Here the source and the output:
http://www.mediafire.com/file/hptu71s63jfq5po/Source+video+and+My+script+output.zip/file

When you get the data from avisynth be sure that all frames are called.

First script:

WriteFile("c.txt","String( CFieldDiff(0,sse=true) )")

Second script:


SeparateFields()
interleave(selecteven().trim(1,0),selectodd()).Weave()
WriteFile("n.txt","String( CFieldDiff(0,sse=true) )")

Now run my script:

with open('c.txt','r') as a:
c = a.readlines()
c = [float(data.replace('\n','')) for data in c]

with open('n.txt','r') as b:
n = b.readlines()
n = [float(data.replace('\n','')) for data in n]

p_back = "ccnnc"
for i in range(0,len(c)-2,5):

ccccc = c[i]+c[i+1]+c[i+2]+c[i+3]+c[i+4]
ncccc = n[i]+c[i+1]+c[i+2]+c[i+3]+c[i+4]
cnccc = c[i]+n[i+1]+c[i+2]+c[i+3]+c[i+4]
ccncc = c[i]+c[i+1]+n[i+2]+c[i+3]+c[i+4]
cccnc = c[i]+c[i+1]+c[i+2]+n[i+3]+c[i+4]
ccccn = c[i]+c[i+1]+c[i+2]+c[i+3]+c[i+4]
nnccc = n[i]+n[i+1]+c[i+2]+c[i+3]+c[i+4]
cnncc = c[i]+n[i+1]+n[i+2]+c[i+3]+c[i+4]
ccnnc = c[i]+c[i+1]+n[i+2]+n[i+3]+c[i+4]
cccnn = c[i]+c[i+1]+c[i+2]+n[i+3]+n[i+4]
ncccn = n[i]+c[i+1]+c[i+2]+c[i+3]+n[i+4]

val = min(ccccc,nnccc,cnncc,ccnnc,cccnn,ncccn)

if val == ccccc:
p = "ccccc"
elif val == ncccc:
p = "ncccc"
elif val == cnccc:
p = "cnccc"
elif val == ccncc:
p = "ccncc"
elif val == cccnc:
p = "cccnc"
elif val == ccccn:
p = "ccccn"
elif val == nnccc:
p = "nnccc"
elif val == cnncc:
p = "cnncc"
elif val == ccnnc:
p = "ccnnc"
elif val == cccnn:
p = "cccnn"
elif val == ncccn:
p = "ncccn"

if p == "ncccc":
if p_back[0] == "n":
p = p_back
elif p == "cnccc":
if p_back[1] == "n":
p = p_back
elif p == "ccncc":
if p_back[2] == "n":
p = p_back
elif p == "cccnc":
if p_back[3] == "n":
p = p_back
elif p == "ccccn":
if p_back[4] == "n":
p = p_back
if p_back != p:
if eval(p_back)*1.1 < eval(p):
p = p_back

p_back = p

f1 = open("p.txt","a")


f1.write(str(i)+","+str(i+4)+" "+p+"\n")

f1.close()

with open('p.txt','r') as e:
f = e.readlines()

dec = [data.replace('\n','') for data in f]


for i in range(0,len(dec)):
pattern = dec[i][len(dec[i])-5:len(dec[i])]

if pattern == "ccccc":
f2 = open("d.txt","a")
f2.write(str(i*5)+"vfr\n")
elif pattern == "nnccc":
f2 = open("d.txt","a")
f2.write(str(i*5+1)+" -\n")
elif pattern == "cnncc":
f2 = open("d.txt","a")
f2.write(str(i*5+2)+" -\n")
elif pattern == "ccnnc":
f2 = open("d.txt","a")
f2.write(str(i*5+3)+" -\n")
elif pattern == "cccnn":
f2 = open("d.txt","a")
f2.write(str(i*5+4)+" -\n")
else:
f2 = open("d.txt","a")
f2.write(str(i*5)+"vfr\n")

f2.close()

len(c)-2 is there because the clip's last frame must have a last digit of 4 or 9 so that there is a full cycle, so atm we just skip the last 2 frames.

There is only 1 paramater here

if p_back != p:
if eval(p_back)*1.1 < eval(p):
p = p_back

"1.1" This is because sometimes due to still pictures or noise a differente pattern with a lower matric is chosen even if it is not the original one, in the clip happens only once and you can even delete it if you like since it's just to retain the ongoing pattern but nothing bad will happen if you delete it.

Notice how my script guesses the constant pattern right:

0,4 ccnnc
5,9 ccnnc
10,14 ccnnc
15,19 ccnnc
20,24 ccnnc
25,29 ccnnc
30,34 ccnnc
35,39 ccnnc
40,44 ccnnc
45,49 ccnnc
50,54 ccnnc
55,59 ccnnc
60,64 ccnnc
65,69 ccnnc
70,74 ccnnc
75,79 ccnnc
80,84 ccnnc
85,89 ccnnc
90,94 ccnnc
95,99 ccnnc
100,104 ccnnc
105,109 ccnnc
110,114 ccnnc
115,119 ccnnc
120,124 ccnnc
125,129 ccnnc
130,134 ccnnc
135,139 ccnnc
140,144 ccnnc
145,149 ccnnc
150,154 ccnnc
155,159 ccnnc
160,164 ccnnc
165,169 ccnnc
170,174 ccnnc
175,179 ccnnc
180,184 ccnnc
185,189 ccnnc
190,194 ccnnc
195,199 ccnnc
200,204 ccnnc
205,209 ccnnc
210,214 ccnnc
215,219 ccnnc
220,224 ccnnc
225,229 ccnnc
230,234 ccnnc
235,239 ccnnc
240,244 ccnnc
245,249 ccnnc
250,254 ccnnc
255,259 ccnnc
260,264 ccnnc
265,269 ccnnc
270,274 ccnnc
275,279 ccnnc
280,284 ccnnc
285,289 ccnnc
290,294 ccnnc
295,299 ccnnc
300,304 ccnnc
305,309 ccnnc
310,314 ccnnc
315,319 ccnnc
320,324 ccnnc
325,329 ccnnc
330,334 ccnnc
335,339 ccnnc
340,344 ccnnc
345,349 ccnnc
350,354 ccnnc
355,359 ccnnc
360,364 ccnnc
365,369 ccnnc
370,374 ccnnc
375,379 ccnnc
380,384 ccnnc
385,389 ccnnc
390,394 ccnnc
395,399 ccnnc
400,404 ccnnc
405,409 ccnnc
410,414 ccnnc
415,419 ccnnc
420,424 ccnnc
425,429 ccnnc
430,434 ccnnc
435,439 ccnnc
440,444 ccnnc
445,449 ccnnc
450,454 ccnnc
455,459 ccnnc
460,464 ccnnc
465,469 ccnnc
470,474 ccnnc
475,479 ccnnc
480,484 ccnnc
485,489 ccnnc
490,494 ccnnc
495,499 ccnnc
500,504 ccnnc
505,509 ccnnc
510,514 ccnnc
515,519 ccnnc
520,524 ccnnc
525,529 ccnnc
530,534 ccnnc
535,539 ccnnc
540,544 ccnnc
545,549 ccnnc
550,554 ccnnc
555,559 ccnnc
560,564 ccnnc
565,569 ccnnc
570,574 ccnnc
575,579 ccnnc
580,584 ccnnc
585,589 ccnnc
590,594 ccnnc
595,599 ccnnc
600,604 ccnnc
605,609 ccnnc
610,614 ccnnc
615,619 ccnnc
620,624 ccnnc
625,629 ccnnc
630,634 ccnnc
635,639 ccnnc
640,644 ccnnc
645,649 ccnnc
650,654 ccnnc
655,659 ccnnc
660,664 ccnnc
665,669 ccnnc
670,674 ccnnc
675,679 ccnnc
680,684 ccnnc
685,689 ccnnc
690,694 ccnnc
695,699 ccnnc
700,704 ccnnc
705,709 ccnnc
710,714 ccnnc
715,719 ccnnc
720,724 ccnnc
725,729 ccnnc
730,734 ccnnc
735,739 ccnnc
740,744 ccnnc
745,749 ccnnc
750,754 ccnnc
755,759 ccnnc
760,764 ccnnc
765,769 ccnnc
770,774 ccnnc
775,779 ccnnc
780,784 ccnnc
785,789 ccnnc
790,794 ccnnc
795,799 ccnnc
800,804 ccnnc
805,809 ccnnc
810,814 ccnnc
815,819 ccnnc
820,824 ccnnc
825,829 ccnnc
830,834 ccnnc
835,839 ccnnc
840,844 ccnnc
845,849 ccnnc
850,854 ccnnc
855,859 ccnnc
860,864 ccnnc
865,869 ccnnc
870,874 ccnnc
875,879 ccnnc
880,884 ccnnc
885,889 ccnnc
890,894 ccnnc
895,899 ccnnc
900,904 ccnnc
905,909 ccnnc
910,914 ccnnc
915,919 ccnnc
920,924 ccnnc
925,929 ccnnc
930,934 ccnnc
935,939 ccnnc
940,944 ccnnc
945,949 ccnnc

Getting the right pattern means also getting the right decimate.

Last script:

TFM(field=1,pp=0,ovr="p.txt")
Tdecimate(ovr="d.txt")