BlockOverlap deblocking filter for MVTools etc

[Fizick]

New BlockOverlap deblocking filter for MVtools.
The idea of shifted procesing is not new and was discussed for Dust plugin here at forum.

Introduction

Great motion compensation plugin MVTools by Manao often produces local blocking artifactes on compensated clip.
BlockOverlap plugin deblock it by using two input compensated clips with shifted block positions.
This plugin make blocks windowed overlapping (uniform or kernel non-uniform blending).
It uses high weight of central (kernel) parts of blocks of two frames, and lower weight of edges parts of blocks to form more smooth output frame.
The blending also greatly decreases the noise on compensated frame.
The deblocked and denoised output can improve the temporal motion compensated filtering of original clip.
Of course, the processing speed is halved due to two motion estimation and compensation.
This plugin may be also useful for other filters with block artifactes (Dust, FFT3Dfilter ?).

Function parameters

BlockOverlap(clip, clip "shifted", int "xblocksize", int "yblocksize", float "kernel")

first parameter - first input clip
shifted - second clip, its blocks must be shifted by half of block diagonal (no default)
xblocksize - horizontal block size (integer, default=8)
yblocksize - vertical block size (integer, default=8).
kernel - blending window form (float, from 0.0 (uniform) to 1.0 (cosine kernel), default =0.5).

Features and limitations

Works only in YV12 color format.
Directly works with progressive clips only. For interlaced sources, you must use appropriate methods (SeparateFieds, SelectEven,SelectOdd, Bob, etc).
Block sizes must be mod 2.
Tested with Avisynth 2.5.6.
Not assembler optimized, but quite fast (itself).
Blending mode with kernel=0 is the same as Avisynth command Overlay with opacity=0.5. In this mode the filter can not remove all block artifactes, but it halve them, try use some additional deblock filter.
Mode kernel=1.0 effectively smoothes blocks, but can produce some dot (circle) artefactes instead.

Sample script for overlapped motion compensation

loadplugin("BlockOverlap.dll")
loadplugin("MVTools.dll")

function MVOverlap(clip source, int "blksize", float "kernel", bool "isb", int "lambda", bool "chroma", int "delta", int "thSCD1")
{
# Overlapped block motion compensation by shifted clips processing and windowed blending
# Fizick, 2005
# Uses BlockOverlap plugin by Fizick
# Uses MVTools plugin by Manao
# source clip must have YV12 format,
# all other parameters are optional
blksize=default(blksize,8)
kernel=default(kernel,0.5)
isb = default(isb,false) # backward compensation?
lambda=default(lambda,2000) # vector smooth
chroma=default(chroma,false) # use chroma
delta=default(delta,1) # frame step
thSCD1=default(thSCD1,300) # scenechange threshold
add = (blksize>=8) ? blksize : 8
vec = source.MVAnalyse(blksize=blksize, isb = isb, lambda = lambda, chroma=chroma, delta=delta)
comp = source.MVCompensate(vec,thSCD1=thscd1)
shifted = source.addborders(blksize/2,blksize/2,add-blksize/2,add-blksize/2) # diagonal shift by half of block
vecshifted = shifted.MVAnalyse(blksize=blksize, isb = isb, lambda = lambda, chroma=chroma, delta=delta)
compshifted = shifted.MVCompensate(vecshifted, thSCD1=thscd1)
compshiftedback = compshifted.crop(blksize/2,blksize/2,blksize/2-add,blksize/2-add) # remove shifting borders
ov = BlockOverlap(comp,compshiftedback,xblksize=blksize, yblksize=blksize, kernel=kernel)
return ov
}

avisource("video.avi")
src=ConvertToYV12(interlaced=false) # progressive clip example
blksize=8
MVOverlap(src, blksize, 0.5, false, 1000, true, 1, 500) # forward overlapped motion compensation



Sample script for overlapped motion compensated strong denoising

loadplugin("BlockOverlap.dll")
loadplugin("MVTools.dll")

# use the same MVOverlap function as above
# ... put it here

loadplugin("degraimedian.dll") # if you use it
loadplugin("fft3dfilter.dll") # if you use it

AviSource("video.avi")
src=ConvertToYV12(interlaced=false) # progressive clip example
fo = MVOverlap(src, blksize=8, kernel=0.5, isb=false) # forward overlapped motion compensation
bo = MVOverlap(src, blksize=8, kernel=0.5, isb=true) # backward overlapped motion compensation
Interleave(fo,src,bo) # 3-length interleaved clip, every source has 2 motion compensated neigbours
DegrainMedian(mode=0) # use any temporal 3-frames denoising filter (FluxSmoothT, Despot, etc)
fft3dfilter(sigma=2.0) # or more filters
SelectEvery(3,1) # get filtered result


Try!

[mg262]

I haven't tried this out because I have no suitable content at hand (and real life has become too busy to let me capture some) -- but on previous experience, MVTools gave results that outperformed anything else, but with substantial blocking in places... so I just wanted to say that this is very much appreciated -- and I look forward to trying it out! (It's particularly nice to see something for those of us who are unbothered about speed!)

[cobo]

Is there a way to use MVdegrain1 instead of DegrainMedian in the second sample script for overlapped motion compensated strong denoising? I'd like to try that, but I can't figure out how to script it.

[Didée]

When you're going to use MDegrain1/2/3 for denoising, then you don't need the BlockOverlap function at all. For those internal functions of MVTools, you can just use the overlap parameter in MAnalyse:

super = source.MSuper()
vec_fw = super.MAnalyse(isb=false, blksize=8, overlap=4)
vec_bw = super.MAnalyse(isb=true, blksize=8, overlap=4)
source.MDegrain1(super,vec_fw,vec_bw)

And similar for MDegrain2/MDegrain3, like shown in the examples section of MVTools' documentation.

No need for BlockOverlap here. It's only for "external" denoising filters that are not provided by MVTools.dll.

[cobo]

I guess I didn't understand it too well. I assumed that the script was for temporal denoising between the current frame and two oppositely half-block-shifted versions of the current frame.

Anyways, I tried Fitzick's script with Average instead, which is the closest thing to MVdegrain I could think of. I also tried with Clense and MedianblurT. MedianblurT seems to soften the edges of the blocks best on my source, which shows chroma blocking on low detail areas such as light grey walls and in areas with high chroma detail such as a red velvet dress, but it produces some aliasing on fast moving near horizontal sloped edges.