Choosing the Better Frame from 59.94fps Material

[Pookie]

I'm Converting 59.94 720p to 23.976 or 29.97 480

Here's an example of the same frame from even and odd. Sometimes, the noise is on even, sometimes it is on odd, so a SelectEven() or SelectOdd() won't fix this. Is there a function that can "choose" the better frame ? TIA

http://www.uploadhouse.com/images/471688415even.png (http://www.uploadhouse.com/)


http://www.uploadhouse.com/images/690543152odd.png (http://www.uploadhouse.com/)

[mg262]

Is it possible that the best frames tend to be I-frames, P-frames are less good and B-frames (if you're using them) are the worst? If you have a moment, would you check what the types of the two frames you post are? Also itmight be useful to know how the source material is encoded...

Leaving that aside, in this particular case, the blocking is so heavy that it should be very easy to detect... for example, assuming that we have 8 x 8 blocks starting at the top left corner pixel (0,0), you will find that the following quantity:

Absolute value of ((0,7) - (0,8)) +
Absolute value of ((1,7) - (1,8)) +
Absolute value of ((2,7) - (2,8)) +
...
+

Absolute value of ((0,15) - (0,16)) +
Absolute value of ((1,15) - (1,16)) +
Absolute value of ((2,15) - (2,16)) +
...

Etc is much higher on the blocky framethan on the other. Having written that, I realise it looks a bit opaque... all I'm suggesting is to look for the sudden jumps in colour that happen at the edges of blocks, either horizontally or vertically. The vertical version could probably be rigged up quite quickly by abusing separatefields, although I'm going to leave it to the resident script-masters to try this or something similar...

[krieger2005]

This could not be a Frame-type difference.. because Pookie said, that this is the same frame.. simply the odd and even part.

But i think this is a joke in general. I can't imagine that one part of the frame looks sooo good and the other so overcopressed.

But if you ask me REALLY for a plugin, which can choose the better frame, i can't help you. I does not know one which can this.

[mg262]

59.94 720p... I thought this meant that it was encoded with every frame occurring twice, and that the above screenshots were (in the stream) distinct frames which happened to encode the same picture. But I could well be wrong...

There are such quality metrics as PSNR floating around... I've never looked at them much because I have never seen evidence that they reflect genuine visual quality (perhaps this is close minded of me)... but for the generalisation of a case like this, they probably would function better than always selecting the even frame (even field?)

[manono]

But I could well be wrong...

No, you're correct. If 59.94fps, each frame is repeated, and if film, some will be there 3 times. I have no idea, though, how to make sure the best ones are chosen. TIVTC, perhaps.

Edit later: I shouldn't have said to use a full IVTC, since you have progressive frames already. You just need to decimate, and hope that the good ones are selected to be kept, something like:

TDecimate(Mode=0,CycleR=6,Cycle=10)

if it's 23.976fps film. Now to have a look at the sample...

[Pookie]

Here's a small post of around 450 total frames. One thing I just noticed, the blocking seems to happen on the first frame of a scene change. In this case, open the file in VirtualDubMod or VirtualDubMpeg2. Look at frames 30, 124, 441.

http://rapidshare.de/files/4134883/example.m2v.html (9mb)

I think the only way is going to be to scrub through these files with SelectEven and SelectOdd and see which has less noise. And yes, perhaps a decimate declared in caps :) will get rid of some of them.

[MOmonster]

@Pookie
For such problems I created a function, called DupHq. I think with the right settings, it should work for you. You can begin with this:
a = last.bilinearresize(360,240)

DupHq(analyse=a, savev=0.8, maxv=7.0, chroma=false)

Hope this helps you. ;)

[kassandro]

Here's a small post of around 450 total frames. One thing I just noticed, the blocking seems to happen on the first frame of a scene change. In this case, open the file in VirtualDubMod or VirtualDubMpeg2. Look at frames 30, 124, 441.

http://rapidshare.de/files/4134883/example.m2v.html (9mb)

I think the only way is going to be to scrub through these files with SelectEven and SelectOdd and see which has less noise. And yes, perhaps a decimate declared in caps :) will get rid of some of them.
I made similar observations with animes, however with ringing instead of blockiness. The first frame of a duplicate series was usually the worst frame. I frames were probably the only exception. Blockiness is much easier to catch than ringing. Checking horizontal block lines, as pointed out by mg262, can be done with very high speed - i would estimate 4000 fps on my PC. On the other hand, if frames are so extremely different as in your above pics, it may be difficult to identify them as duplicates.

[Pookie]

mg262, krieger2005, manono, kassandro, MOmonster - thanks. The recommendations are very much appreciated. I'll try Tdecimate.

DupHQ did a nice job. I used DupHq(analyse=a, savev=0.4, maxv=6.0, chroma=false) .

Check it out:
http://www.uploadhouse.com/images/215765508-before.png (http://www.uploadhouse.com/)

http://www.uploadhouse.com/images/738162556-after.png (http://www.uploadhouse.com/)

[MOmonster]

@kassandro
The princip of my DupHq function is just more simple (works just with the difference of the frames), but is also able to detect frames with this strong difference as duplicate without destroying motion in other low motion scenes. Also ringing or blocks makes no difference for this method (I also tried a spatial method, but normally these methods tends to use just the blurred frames, so this is just effective), If you want to know more about just ask ;)

[kassandro]

@kassandro
The princip of my DupHq function is just more simple (works just with the difference of the frames), but is also able to detect frames with this strong difference as duplicate without destroying motion in other low motion scenes. Also ringing or blocks makes no difference for this method (I also tried a spatial method, but normally these methods tends to use just the blurred frames, so this is just effective), If you want to know more about just ask ;)
I just looked at your script. By using a downsized clip for comaprison, you check for duplicates - a fine idea. However, how do you evaluate the quality of a frame?

[MOmonster]

@kassandro
By using a downsized clip for comaprison, you check for duplicates - a fine idea. However, how do you evaluate the quality of a frame?
No, sorry, it isn´t that easy. The difference is still bigger than in low motion scenes. More important for this point is the variable thereshold between savev and maxv and the motionbreak parameter. In generell we have such big artefacts only in scenes that are hard to encode, so in scenes with big differences and if we speak about static scenes like in anime especially in scenechanges. So detect such frames it use the difference to the not similar frame. That bigger this difference is (that harder the scenechange or that harder the motionbreak is), that higher the thereshold is choosen (up to maxv). The motionbreak parameter controls how strong the break has to be to that the thereshold higher (how fast the thereshold is choosen higher).
The resizer is just there to tweak this process, because in this case the artefacts are very strong. If the blurring is too strong the method to get a value for the quality won´t work.
Ok at this point where we know what frames can be duplicated, the quality comparison is very easy. Because we work on static scenes we just have to use the frame with the smallest sum of the difference to the previous and the difference to the next frame, because that smaller the difference between two static frames is that better is the quality (that stronger are the artefact of the compression that bigger is the change to the next frame, because the encoder want to produce the output without artefacts).
That this happens also before scenechanges and not only after you can see on the screenshots in the thread, but in general next frames seems to be better with really similar sums, for that point we have the forward parameter, that says how much smaller the sum of the previous frame has to be to be choosen and not the next frame. ;)

If we have a scenechange from a high motion scene to a low motion scene or the other direction my function creates a duplicate in the low motion scene, but because of the strong motionbreak this isn´t so important in our eyes.

Edit: For not so hard artefacts it isn´t recommed to use the analyse clip (just don´t set it).

[kassandro]

No, sorry, it isn´t that easy. The difference is still bigger than in low motion scenes. More important for this point is the variable thereshold between savev and maxv and the motionbreak parameter. In generell we have such big artefacts only in scenes that are hard to encode, so in scenes with big differences and if we speak aboat static scenes like in anime especially in scenechanges. So detect such frames it use the difference to the not similar frame. That bigger this difference is (that harde the scenechange or that harder the motionbreak is), that higher the thereshold is choosen (up to maxv). The motionbreak parameter controls how strong the break has to be to that the thereshold higher (how fast the thereshold is choosen higher).
The resizer is just there to tweak this process, because in this case the artefacts are very strong. If the blurring is too strong the method to get a value for the quality won´t work.

There is one weakness, which you can't resolve with a script based filter: if there is only motion in a small box, then your algo will consider the frames as duplicates. This happens often in animes, but certainly not in Pookie's clip.


Ok at this point where we know what frames can be duplicated, the quality comparison is very easy. Because we work on static scenes we just have to use the frame with the smallest sum of the difference to the previous and the difference to the next frame, because that smaller the difference between two static frames is that better is the quality (that stranger are the artefact of the compression that bigger is the change to the next frame, because the encoder want to produce the output without artefacts).
That this happens also before scenechanges and not only after you can see on the screenshots in the thread, but in general next frames seems to be better with really similar sums, for that poit we have the forward parameter, that says how much smaller the sum of the previous frame has to be to be choosen and not the next frame. ;)

Very nice idea. Seems to be better the averaging of duplicates, at least for Pookies clip (the HDTV hardware encoders don't seem to be properly tuned yet). This idea combined with blockwise comparison should be put into a plugin. ScriptClip and FrameEvaluate are simple awful filters, because the use the Avisynth script parser, which is normally only used during initialisation.

[Didée]

There is one weakness, which you can't resolve with a script based filter: if there is only motion in a small box, then your algo will consider the frames as duplicates.
Impossible it is not. Okay, a plain "windowed" (rectangular) comparison is impossible, or close to that, but ...
by the method I posted here (http://forum.gleitz.info/showthread.php?p=175507&highlight=k%F6nigreich#post175507) on Gleitz' forums, something similar can be achieved: building an average over the whole frame, while including only values bigger than <threshold> into the averaging. I've the method acually running, optionally, in the upcoming R24.

[MOmonster]

@kassandro
Yes, you are right, but if the filter doesn´t detect a motionbreak, he can do the comparison similar to dup. I mean using smaller windows (32 * 32) for motiondetection, but this isn´t that easy to realize in a avisynth script. :sly:
@Didee
Yes, really nice idea, but ones more that complex. I don´t think kassandro can realize your method in a filter. It´s really simpler to make the motion detection just smaller windowed, if the whole difference is really small. That way we can also catch mouth movements and so on without the need of different masks and so on. ;)

Edit: If we want to solve this problem script based, I can try the smaller windowed motiondetection or would use mvtools, but a stronger cleaner can also just have the right effect because it kills the difference because noise and the only left are motiondifferences. Here is just a smaller savev value necessary.
But till now I don´t plan to realize something like this. Also if I set the savev parameter to 0, my function would still catch the bad Frames of motionbreakes. And because the quality differences in real static scenes are really less and the range of my function will be so or so strong limited, I prefer to use it that way and then after it dup or staticfreeze (not tested till now).