actionman133
5th February 2006, 08:16
Hey all,
I've been working around ideas for another IVTC plugin, because I feel that the others availabe of AVISynth, while innovative and quite effective in their own right, it still hasn't been completely 'mastered' (if you can use such a word). I think Decomb has gone has far as you can go if you want to keep material at 30p, but 24p (which is what I like better, and am trying to improve), it's not quite as effective. Decomb and SmartDecimate are close to perfection, like 90-95% there, but I think there's still a bit more room to go...
One thing which I think we can do to improve the routines is to actually have the plugin identify film and video. Currently - to my knowledge anyway - the filters perform field matching then searches for interlacing. This potentially leaves low motion video interlaced and can deinterlace parts of progressive frames. If it can identify the difference between film and video, it could provide a more precise output. Another issue that can sometimes throw off the results is the never-quite-perfect field matching. Decomb is quite good, using half-pixel shifts to align the fields, but because they resample the frame, it's never 100% accurate.
These two issues can be worked around with this idea that I have developed a little, but am at a loss to be able to apply in a filter. The concept is that it tries to find the duplicated field in film frames using DoubleWeave ().
DoubleWeave ()
Subtract (Last, Trim (1, 0))
ScriptClip ("Subtitle (String (YPlaneMinMaxDifference ()))")
a = SelectEvery (5, 0)
b = SelectEvery (5, 1)
c = SelectEvery (5, 2)
d = SelectEvery (5, 3)
e = SelectEvery (5, 4)
ShowFiveVersions (a, d, b, e, c)
The result is this image:
http://actionman133.isa-geek.net:8080/avs/test.png
The duplicate field is identified by the lowest difference to the next frame (bottom left in the image). Of course, it has to be significantly lower than all the other frames to distinguish it as a duplicate. I think it should run two test to see and must pass at least one. First, average the values of the four largest frames. If the lowest value is either below a percentage of that average, or if it is less than the average by a defined integer, then it is considered film. Example: if the four frames with the greatest difference average to 75 (low motion scene), and the dupe frame's value is 35, then setting the percentage value to 55% (meaning the dupe must be LESS THAN 55% to qualify as a dupe), it will result as film. In a high motion scene, the average of the 4 frames will probably be 255 and the dupe can sometimes be as high as 150 (due to compression noise). This would fail the percentage test, but because it is 100 less than the average it's pretty much certain that it's film. If the 5 frames passes either of those tests, it is declared as film.
From here, since you know where the duplicated fields are, you can blend them together with Merge () to help improve the image a little. And the other progressive frame is either 2 frames before the duplicate or 3 frames after, depending on the position of the duplicate.
Obviously, in the above image, the high motion source makes it an easy match, but even low-motion material produces distinct results with this test filter. If it were with video material (either 60i or 30p), it would not conform to this pattern and be identified as non-film frames. If it identifies one frame with a significantly lower value than the other 4, it can be certain it is film, weave the frames together, blend the dupes and skip deinterlacing altogether.
One potential problem is scene changes. It is possible that in the 5 frame test that no duplicates may be found, but it is still film material. This method would return one set of video frames, with film frames on either side. If it finds a single non-film set, and both the preceeding and proceeding frames are film frames, then it can safely assume that it is a scene change. It can find the progressive frames, but will have to search for interlacing because of possible bad edits. I haven't come up with a good algorithm for working this out yet, but I'm looking into it.
For those who made it this far, does anyone think this idea is worth pursuing for an IVTC filter? I can do some of this in AVISynth, but the only languages I know are Delphi and VisualBasic (don't laugh... ;-) and I've never learnt how to deal with media in either.
PS. By the way, my comments about other IVTC filters is not meant to sound derogatory to either the filters or their respective writers/authors. They are excellent and respect them very much (particularly SmartDecimate and Decomb), and they work 98% of the time. But that little 2% can cause me a lot of anguish. I'm hoping this proposed method might help reduce that percantage a little.
I've been working around ideas for another IVTC plugin, because I feel that the others availabe of AVISynth, while innovative and quite effective in their own right, it still hasn't been completely 'mastered' (if you can use such a word). I think Decomb has gone has far as you can go if you want to keep material at 30p, but 24p (which is what I like better, and am trying to improve), it's not quite as effective. Decomb and SmartDecimate are close to perfection, like 90-95% there, but I think there's still a bit more room to go...
One thing which I think we can do to improve the routines is to actually have the plugin identify film and video. Currently - to my knowledge anyway - the filters perform field matching then searches for interlacing. This potentially leaves low motion video interlaced and can deinterlace parts of progressive frames. If it can identify the difference between film and video, it could provide a more precise output. Another issue that can sometimes throw off the results is the never-quite-perfect field matching. Decomb is quite good, using half-pixel shifts to align the fields, but because they resample the frame, it's never 100% accurate.
These two issues can be worked around with this idea that I have developed a little, but am at a loss to be able to apply in a filter. The concept is that it tries to find the duplicated field in film frames using DoubleWeave ().
DoubleWeave ()
Subtract (Last, Trim (1, 0))
ScriptClip ("Subtitle (String (YPlaneMinMaxDifference ()))")
a = SelectEvery (5, 0)
b = SelectEvery (5, 1)
c = SelectEvery (5, 2)
d = SelectEvery (5, 3)
e = SelectEvery (5, 4)
ShowFiveVersions (a, d, b, e, c)
The result is this image:
http://actionman133.isa-geek.net:8080/avs/test.png
The duplicate field is identified by the lowest difference to the next frame (bottom left in the image). Of course, it has to be significantly lower than all the other frames to distinguish it as a duplicate. I think it should run two test to see and must pass at least one. First, average the values of the four largest frames. If the lowest value is either below a percentage of that average, or if it is less than the average by a defined integer, then it is considered film. Example: if the four frames with the greatest difference average to 75 (low motion scene), and the dupe frame's value is 35, then setting the percentage value to 55% (meaning the dupe must be LESS THAN 55% to qualify as a dupe), it will result as film. In a high motion scene, the average of the 4 frames will probably be 255 and the dupe can sometimes be as high as 150 (due to compression noise). This would fail the percentage test, but because it is 100 less than the average it's pretty much certain that it's film. If the 5 frames passes either of those tests, it is declared as film.
From here, since you know where the duplicated fields are, you can blend them together with Merge () to help improve the image a little. And the other progressive frame is either 2 frames before the duplicate or 3 frames after, depending on the position of the duplicate.
Obviously, in the above image, the high motion source makes it an easy match, but even low-motion material produces distinct results with this test filter. If it were with video material (either 60i or 30p), it would not conform to this pattern and be identified as non-film frames. If it identifies one frame with a significantly lower value than the other 4, it can be certain it is film, weave the frames together, blend the dupes and skip deinterlacing altogether.
One potential problem is scene changes. It is possible that in the 5 frame test that no duplicates may be found, but it is still film material. This method would return one set of video frames, with film frames on either side. If it finds a single non-film set, and both the preceeding and proceeding frames are film frames, then it can safely assume that it is a scene change. It can find the progressive frames, but will have to search for interlacing because of possible bad edits. I haven't come up with a good algorithm for working this out yet, but I'm looking into it.
For those who made it this far, does anyone think this idea is worth pursuing for an IVTC filter? I can do some of this in AVISynth, but the only languages I know are Delphi and VisualBasic (don't laugh... ;-) and I've never learnt how to deal with media in either.
PS. By the way, my comments about other IVTC filters is not meant to sound derogatory to either the filters or their respective writers/authors. They are excellent and respect them very much (particularly SmartDecimate and Decomb), and they work 98% of the time. But that little 2% can cause me a lot of anguish. I'm hoping this proposed method might help reduce that percantage a little.