Block statistic script and block quality detection and masks

[redfordxx]

Hi, I am working on a method how to detect quality of the video source encoding. That would allow us to adaptively postprocess the video based on some mask created. High quality blocks should not be changed, low quality block probably have blocking and ringing and should be corrected. Noise is something else, in fact it means high quality.

Shortly, my goal is to create mask which tells where should be applied deblocking and deringing

This assesment I want to do on 8x8 blocks without looking behind the border, so blocky neighborhood is not an indicator of quality, because I dont look at it...at least at first phase.

My input data will be:
- what detail is in 8x8 block (of course)
- frequency values in 8x8 block, that's new (I modified DCTFilter for this purpose)

Feel free to throw any Ideas, except that "it is not possible unless you have access to quantizers in the source"

Starting to work on this, I made some script functions which I thought might be interesting to publish. That I am going to do in next post.

[redfordxx]

Here are scripts which might be useful:
BlockOp - does various statistics on small block of video like 8x8 or so, basicly does similar things like mt_luts but block-based, mostly faster and without clipping
Convolve - does convolution on 8bit video or stacked 16bit video (it is based on mt_convolution.
Dither_makediff16 - like mt_makediff but on 16bit with optional multiplication

In the attachment are those scripts. Scripts are very long. so it has to be attachment...

Depending on what you use, you might need:
Average, AverageM (http://forum.doom9.org/showthread.php?p=1536107#post1536107)
DCTFilter (http://forum.doom9.org/showthread.php?p=1537175#post1537175)(my version)
Dither (http://forum.doom9.org/showthread.php?p=1386559#post1386559)
mt_Masktools

[redfordxx]

Usage:function BlockOp(clip o, string mode, int "bx", int "by", int "sx", int "sy", int "size", int "y", int "u", int "v", int "zero", float "limit", float "fac_startzero", float "fac_endzero",int "info", string "Override")
Performs mode operation on input clip, similar to modes in luts. The operation is performed within specified block and not over the borders. Produces interleaved video with all the resulting clips
modespecifiec what and how we want and can be following, coma separated:
- min,rng,range,max,sad,avg,std,dif,maxabs...all these operations on given block can be performed and returned in one clip.
- dct,dctback...means that we first carry on dct decimation using DCTFilter and then mode operations above is performed
- dctback, orig...specifies that among others, also original and dct decimated clips are returned
- luts,mask,convo,16bit...how the calculation should be performed
- - - 16bit... works only with std but is most accurate
- - - luts... performs the operation using mt_luts and I believe is slowest in all cases and not very flexible. SAD has probably wrong results. Changing limit parameter decreases accuracy and introduces clipping)
- - - mask,convo...alternative method, using various workarounds mostly significantly faster than luts. Not very exact with std, unless very small values specified (therefore use 16bit mode).
bx,by,sx,sy...dimension and shift of block (defaults 8,8,0,0)
size...though the process the clip is downsized so that one block is represented by one pixel and then enlarged to original dimension. setting size=0 prevents this enlargement so we receive significantly smaller videos, which can be sometime useful
zero...where is logical zero value of the clip (default 0), sometimes 128 can be useful
limit...maximum absolute value of the clip (default 255 if zero is 0, 128 if zero is 128) used to enforce higher contrast. Eg. if zero=0 and limit=100, then values are multiplied by 255/100
fac_startzero,fac_endzero...begining and end of sequence of 0 for DCTFilter. Example fac_startzero=4,fac_endzero=1 results in calling DCTFilter(0,1,1,1,1,0,0,0,0)
info...bitwise 0-31, try and see, eg 2 shows the performed script
Override...commad(s) to be evalulated afrer all settings. eg Override=""" stdMulti=" 2 * " """ ensures multiplication of the std clip *2 during lut operation (stdMulti is variable which is added to some mt_lut operation somewehre)

function Convolve(clip c, val hor, val ver, float "tot", bool "lsb", int "hctr", int "vctr", int "y", int "u", int "v", bool "info")
Does convolution like mt_convolution, but on either 8bit or 16bit clips. This option is specified by lsb parameter. The description is in the file, but main parameters are:
hor, ver specifies the kernel. If it is string, it expects kernel like mt_convolution. If it is number, it is number of ones in the horizontal or vertical direction.
hctr, vctr only if hor, ver is number, specifies center of the kernel
lsb (false)...if true, stacked 16bit clip is expected


Function Dither_makediff16 (clip src1,clip src2,int "y", int "u", int "v", float "multi")It is like mt_makediff from masktools, but works on 16bit clips.
multi additional parameter to specify multiplication of the value around 32768 (to enhance contrast)

[redfordxx]

BlockOp("range") #delivers clip containing range of values in every 8x8 block
BlockOp("dct,orig", info=-1) #delivers DCT filtered clip and original interleaved with all possible onscreen info
BlockOp("min,max,dctback",bx=16,by=16,sx=4,sy=2,fac_endzero=0,fac_startzero=2) #deliver max and min performed on shifted clip but first filtered with DCTFilter(1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0)
BlockOp("avg,std,dif,16bit",bx=16,by=16, zero=128, limit=32) #Delivers 16bit calculated standard deviation on each 16x16 block enhance 4x
BlockOp("dif,std,convo,avg,16bit",bx=16,by=16, zero=0, limit=32, info=2) #Delivers Interleave(avg,std,dif) of which std and dif is enhanced 8x and std only is 16bit calculated
BlockOp("std,luts",bx=16,by=16, zero=0, info=2+8+16) #standard deviation using mt_luts (slow), showing the name of the clip and the code on doublesized clip
convolve(3, 2) #Performs mt_convolution("1 1 1", "0 1 1")
convolve(3, 2, hctr=3, vctr=-2, tot=-1.4) #Performs mt_convolution("1 1 1.0 0 0", "0 0 0 0 0 0 0 1 1.0", total=8.4)
convolve(3, 2, hctr=3, vctr=-2, lsb=true, tot=-1.4, info=true) #Performs Dither_resize16 in two passes with "1 1 1.0 0 0", "0 0 0 0 0 0 0 1 1.0", total is ignored, shows info
convolve("2 1 1", 2, tot=0, lsb=false, hctr=5, vctr=5) #performs mt_convolution("2 1 1", "1 1 0 0 0 0 0 0 0")

[redfordxx]

So that would be all for now...
BTW, if someone sees a speedup of BlockOp somewhere, you are welcome to tell me. You don't need to go through code, just run it with info=8 (+16 for resizing, if the video is too small)