amDCT "adaptive multifunction DCT", deblock, smooth, sharpen, range expansion filter.

[jconklin]

amDCT() is an adaptive video filter providing deblocking, sharpening, local range expansion, smoothing, and bright noise removal in a single filter that can be used on video of any level of quality. These four operations work synergistically to drastically reduce block artifacts while maintaining detail and increasing local contrast.

A key advantage of amDCT--and one way in which it is a major advance over previous approaches--is that it determines how much smoothing, local range expansion and sharpening should be done at three different levels: the frame as a whole, block-by-block, and pixel-by-pixel.

Adaptivity was built into the core algorithm in order to provide a filter that could automatically adjust its strength to varying conditions in badly-encoded material so that good frames or good parts of bad frames would not be oversmoothed and bad frames or bad parts of good frames would be smoothed and deblocked more.


amDCT is built around a processing core modifed from my SmoothD2 filter that runs each block of a frame thru a DCT quant dequant inverse DCT process. Each frame is run through this process up to 64 times. Each time through the loop the frame is shifted up down left or right, and the resultant frame is then shifted back so that it is aligned with the original image. The values of each frame generated by this loop are added to each other and then averaged to produce a new frame.


The basic problem I found when using SmoothD2 which led me to start work on what became amDCT is that some frames needed more smoothing than others did.

So I decided I would find a way for the program to measure the amount of smoothing a frame needed and adjust the strength of smoothing accordingly.
After all, how hard could that be?

The gods laugh; the wife cries.

Well it turns out doing a first swipe at it is fairly easy. Just take an existing deblocker and measure the amount of change it makes in the frame and use that as a measure for the amount of smoothing you need.

The first rush of success. I've been through this before and I know I am hooked.

The problem is that you then start seeing that there are cases that the simple method of determining the global amount of smoothing doesn't work all of the time.
For example a blocky but almost black frame with a small area of light with a lot of detail. The deblocker tells you the frame is very blocky so you should strongly smooth the frame.
Unfortunately if you smooth it at a high level the area of detail gets smoothed out.

When you start getting a number of special cases like this you figure that a measure of the internal detail of each block might be useful.
It is useful. I am not an expert in this, so think of a hamster running in a wheel as I try all kinds of different measures and try to figure out which ones work best.

During that exercise I realize that I might have a measure of the amount of smoothing that I want for each block of the frame. So I figure out how to rebuild the DCT processing loop so that the amount of smoothing can be controlled on a block by block basis.
And it works. It took awhile to get it to work.

Hamster running really fast, wheel falls out of bearings, wheel and hamster fly out of cage and down the hall.

Acidentally discover that if you screw up the quant dequant code you can make the DCT processing loop do local range expansion instead of smoothing.

A NEW TOY!!! Spend a bunch of time messing around with the quant dequant code and figure out how to make it sharpen the image.
Two new problems. Range expansion intoduces halos on very strong black white borders. Sharppening introduces artifacts.
Sharpening artifacts can be reduced by presmoothing with a hand crafted matrix using shift=2.

Notice the most visible noise is in the brightest parts of the image and that the most oversmoothed parts are in dark areas.
Add controls that allow the user to specify at what brightness and at what strength dark areas should start being protected from being oversmoothed.
Add controls that allow the user to specify at what brightness and at what strength bright areas should start being more smoothed.
To add the extra smoothing at bright areas required building a very smoothed frame, shift=2, quant=6, qtype=2, which it turns out also makes an excellent smoothed frame for mvtools2() to use.
The very smoothed frame is also used to build a boundary mask to reduce bright halos that would be introduced by range expansion.

Hamster finds note taped on monitor

todo list not getting done
todo list will be glued to monitor.

Note understood.

amDCT version 1.0 is out the door.
Let me know what works, what doesn't, and what can be improved. My own improved list will easily take a year.
I think I just heard laughter.

Download code and manual https://sites.google.com/site/amdctfilter/
or just the manual https://docs.google.com/viewer?a=v&p...JiYWViNjEyMmIz