blended frames idea (could be added to decomb?)

[APF_Gandalf]

First of all, I'd want to say that I'm not sure it'll work because my idea is only based on about 50 video files made with the same capture card (not by me, and i don't know what card was used). Maybe stuff captured with other cards won't show the same blend pattern. If it's the case, please just ignore this post, I try to share my ideas, but if I'm totally mistaken, there is no need to.

As you have already noticed, blend isn't regular, if I consider my video as multiple packs on 5 frames, sometimes, i got a blend on the first, sometimes on the second frame...
but when I watched carefully at the video, I noticed that every blended frame is a blend of the previous and the following frame:

let's say my frames are A,B,C,D..., I always got something like

A B B+C C D E F F+G G H

This pattern was repeating for a while with alway the third frame that was a blend of the second and the fourth ones until I got a scene change that seems to "reset" the pattern to something else until the next scene change

A B C C+D D E F G G+H H
or
A A+B B C D E E+F F G H

for example.

The problem is that I'm not a programmer, but I guess that, if I can compare each frame with a blend of the previous and the next frame, I can easily detect blended frames and delete them or (better IMO) replace the blended frame with a duplicate of the previous or following frame to keep the original frame rate so you can still use a decimate if you need/want.

This idea needs to be checked on other video files with blended frames, but I hope someone will be interested to code it if it can work.

[trbarry]

I wonder how it would work if you first did Telecide and then used an algorithm that deleted the one frame out of each 5 that had the lowest total sum of differences between that frame and both the prev and next frames? It would seem that any frame that was a blend would tend to be more similar to both its neighbors. And in the case of still scenes it wouldn't matter anyway. ???

- Tom

[sh0dan]

Unless I'm mistaking this is how it's done right now in Decimate! (neuron?)

[Xesdeeni]

I still haven't found the quote I referenced in http://forum.doom9.org/showthread.php?s=&threadid=47862#post274495. But it seems a complete reversal might be possible.

Xesdeeni

[trbarry]

Suppose we have the sequence of frames A B C where B is a blend of A plus C. I guess the absolute difference A-B is almost equal to the difference B-C. So, if we get the smallest value of the difference of these differences (delta of differences?) we will get the blended frame. Doesn't make sence?

Bach -

I'm not sure that would get it. If I understand it correctly without having read the Decomb code, you would be finding 2 frames in a row that each have a similar amount of difference from the previous frame. But that doesn't guarantee they both have a small difference, which is what you would maybe want. I think the sum would probably be better.

for (int d = 0 ; d < cycle ; d++)
{
count[d] = count[d] + count[d+1];
}

But I still haven't completely convinced myself it would still not cause problems in the case where the frames were not blended. I think that would work also, but not sure. Maybe I'll have to write a quickie to find out.

- Tom

[APF_Gandalf]

@Bach
I tried your "hack" but without success, decimate keeps deleting a frame that isn't the blended one. Maybe I did something wrong, but the DLL compiled without any error.
in fact, those caps are anime so there are many duplicates and decimate deletes a duplicate rather than a blend because (IMHO) if you compare frame B with A and with frame C, the difference will exist unless you'll have duplicates and you can't get lower difference than with duplicates so the blends won't be deleted.
the way to avoid this is comparing frame B with a blend of frames A+C.

@all
I've thinked about it for quite a long time and wrote a little improvement of my idea and tried to give a more detailed overview of it:

it's a pseudo avisynth scripting code

avisource(clip)
stream1=selectEven()

# this will give me frames A,C,E,G,I... from my A,B,C,D,E,F,G... clip

stream2=selectOdd()

# this will give me frames B,D,F,H,J... from my clip

stream1b=Trim(stream1,1,0)
stream2b=Trim(stream2,1,0)

# this will remove the first frame from the stream
# so i'll be able to blend A with C and C with E easily
# using MergeLuma(streamX,streamXb,0.5)and MergeChroma(streamX,streamXb,0.5)

stream01=MergeLuma(stream1,stream1b,0.5).MergeChroma(stream1,stream1b,0.5)

stream02=MergeLuma(stream2,stream2b,0.5).MergeChroma(stream2,stream2b,0.5)

# effectively blend my frames together

Compare(stream01,stream2)
Compare(stream02,stream1)

# this is where I got stuck because I don't know how to
# get the values from Compare() in order to use them
# for the end of the script

## IF MAD or PSNR (I don't know which one will be the best) for a
## frame is less or equal than a value defined after testing, then
## replace the frame with a duplicate of the frame that have the
## least difference with it.
## By replacing it with a duplicate, you can avoid deleting
## a complete range of frames in still scenes and even if
## there is 4 duplicates and a blended frames in a 5 frames row,
## you can be pretty sure that the duplicate will be replaced
## correctly and you keep the possibility tu use the decimate from decomb


so this is how I'll make it, but again, I'm not a coder, maybe the way I think is completely wrong so I'd want some comments on the process I detailed, is it feasible? do you think it'll work? is there someone interested in using such a plugin for AVS? and is there someone to code it?

Thanks in advance for your answers.

[Xenoproctologist]

Thoughts on blended frame detection:

Wouldn't a fourier transform of the frames show the blended frames as having less high-frequency components?

How about a simple frame comparison, since each blended frame has at least two recovery formulas?

Example Telecine Pattern:
NTSC: [A] [B] [C] [D] [E]
FILM(top): [1] [2] [2] [3] [4]
FILM(bottom): [1] [2] [3] [4] [4]


[1] = [A]

[2] = [B]
= 2*[C]-2*[D]+[E]

[3] = 2*[C]-[B]
= 2*[D]-[E]

[4] = [E]
= 2*[D]-2*[C]+[B]