Doom9's Forum - View Single Post

frustum · 1st November 2010, 22:54

I sent a pm to the author, but his profile says he joined doom9 in order to post his filter, then last posted a few months later, in the fall of 2007, so I suspect he won't notice it.

I've had a look at the code and it is something I could pick up and generalize, and perhaps improve it in other ways. It would be a good way for me to get my feet wet. However, the whole reason I started using avisynth was in order to transfer and restore about 35 8mm films to DVD in time for Christmas, as a present to my siblings. I may not have time to work on the plugin until that task is done.

However, I will mention a few weaknesses (other than blindly assuming YV12 format pixels) I see in the code, in order of difficulty.

(1) The average luma is computed by looking +/- N frames from the current frame (settable as a parameter). It starts its average from either frame -N or the most recent scene change before the current frame. It ends its average either at frame +N or at the frame before the last scene change before frame +N. This is all perfectly good unless there is more than one scene change detected after the current frame. This is cured by adding one "break;" after noticing a scene change after the current position, and is a very minor point.

(2) Once the rolling time averaged min/max luma for the current frame is computed, the luma distribution is shifted to map to 16 .. 235. This is desirable for some purposes, other times mapping to 0..255 is desirable. Perhaps this should be a parameter.

(3) The scene detection logic is very simple minded, although it probably works most of the time. The "min" and "max" luma for the frame are computed by ignoring roughly the outer 1% of the brightest/darkest pixels, so outliers don't have undue influence. If either the min or max for frame N has changed by more than a threshold value as compared to the previous frame, it is considered to be a new frame. This is probably pretty easy to produce false positives as well as false negatives.

Just from thought experiments, no actual data gathering, I think this can be improved. Using changes in the min/max is OK, but the threshold needs to be higher to avoid false positives, at the expense of false negatives. We can add other tests to help fix this.

Each frame in the averaging window has already exhaustively (and expensively) measured the luma histogram, but only the min/max are cached for the frame. It would be better save the whole histogram as well (only 256 ints). In addition to detecting whether the min/max changed significantly, with the histogram we could add tests for whether the mean (or median) has changed, and whether the autocorrelation between the histograms of successive frames has changed much. Further, rather than just considering whether frames n-1 and n correlate, look at the correlation for each successive pair. If there is a sudden change in the correlation it would be less prone to false positives. That is, even if the correlation changes a lot between frames n-1 and n, it is meaningful only if the correlation has been somewhat stable either before or after frame n. This will avoid a situation I've seen where in the middle of an ordinary scene the luma flashes rapidly up and down a few times.

1st November 2010, 22:54	#44 \| Link
frustum Registered User Join Date: Sep 2010 Location: Austin, TX Posts: 40	autolevels() improvements I sent a pm to the author, but his profile says he joined doom9 in order to post his filter, then last posted a few months later, in the fall of 2007, so I suspect he won't notice it. I've had a look at the code and it is something I could pick up and generalize, and perhaps improve it in other ways. It would be a good way for me to get my feet wet. However, the whole reason I started using avisynth was in order to transfer and restore about 35 8mm films to DVD in time for Christmas, as a present to my siblings. I may not have time to work on the plugin until that task is done. However, I will mention a few weaknesses (other than blindly assuming YV12 format pixels) I see in the code, in order of difficulty. (1) The average luma is computed by looking +/- N frames from the current frame (settable as a parameter). It starts its average from either frame -N or the most recent scene change before the current frame. It ends its average either at frame +N or at the frame before the last scene change before frame +N. This is all perfectly good unless there is more than one scene change detected after the current frame. This is cured by adding one "break;" after noticing a scene change after the current position, and is a very minor point. (2) Once the rolling time averaged min/max luma for the current frame is computed, the luma distribution is shifted to map to 16 .. 235. This is desirable for some purposes, other times mapping to 0..255 is desirable. Perhaps this should be a parameter. (3) The scene detection logic is very simple minded, although it probably works most of the time. The "min" and "max" luma for the frame are computed by ignoring roughly the outer 1% of the brightest/darkest pixels, so outliers don't have undue influence. If either the min or max for frame N has changed by more than a threshold value as compared to the previous frame, it is considered to be a new frame. This is probably pretty easy to produce false positives as well as false negatives. Just from thought experiments, no actual data gathering, I think this can be improved. Using changes in the min/max is OK, but the threshold needs to be higher to avoid false positives, at the expense of false negatives. We can add other tests to help fix this. Each frame in the averaging window has already exhaustively (and expensively) measured the luma histogram, but only the min/max are cached for the frame. It would be better save the whole histogram as well (only 256 ints). In addition to detecting whether the min/max changed significantly, with the histogram we could add tests for whether the mean (or median) has changed, and whether the autocorrelation between the histograms of successive frames has changed much. Further, rather than just considering whether frames n-1 and n correlate, look at the correlation for each successive pair. If there is a sudden change in the correlation it would be less prone to false positives. That is, even if the correlation changes a lot between frames n-1 and n, it is meaningful only if the correlation has been somewhat stable either before or after frame n. This will avoid a situation I've seen where in the middle of an ordinary scene the luma flashes rapidly up and down a few times.