Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 23rd May 2006, 20:08   #1  |  Link
redfordxx
Registered User
 
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 791
SmoothDeblock (under development)

Hi
hope you are not bored with my deblocking attempts... Here is my latest trial:

Remember SmoothD? I tried to figure out how it works and made similar script (that§s why I borrowed the name) but adaptive... and slow

I made shifted DCTFilter applications to remove high frequencies. This caused removing the blocking on original block borders (shifted into middles of the blocks) but introduced new blocking on the new borders. So, when I shift the clips back, I have (for example) four clips with blocking on diferent places. I merge the clips so that I take only the inner part of the block (without blocking edge).
For this I use weighting mask made by function WeightMatrixClip which is clip made of all the same 8x8 blocks with values 255 in the middle and going down to borders. Additional functionWeightMatrixSum delivers the average value of the mask clip which is later used to normalize the weighted average.
This all delivers function SmoothDeblockLayer and it is possible to choose the density, quant, weightmask.
density = number of shifts on each axis per 8pixels - possible values are 2 or 4. (so 4 means 16 shifts per 8x8 block), 4 produces sharper image and less artifacts on hi quants but is more than 4times slower than 2, which means only 4 shifts...)
quant = 0,...,7 number of coeficients removed by DCTfilter. 0 makes no change to the picture.
weightmask = WeightMatrixClip mask together with WeightMatrixSum controls how the shifted clips are merged.
So, SmoothDeblockLayer returns deblocked clip on certain quant.

I can have 7 different deblocked clips according to quant. I want to merge some of them depending on detail presence.

Then, my assumption (not 100% correct) was, blocking is where is hi quantization, so, let's detect the quantization. I detect the quantization with DCTFilter again on certain quant again. That's function QuantDiff. It produces the mask of "what could be removed with lowpass on certain quant".

From this mask, the function AdaptionMask produces mask for merging the deblocked "layers". It is just some values increasing, blockaveraging and blurring depending on given parameters. This is part wherer I am not sure. Now it is block oriented but maybe the mask should more precisely follow the detail of the clip.

Finally the SmoothDeblock function puts everything together. Now it is fixed for 3 deblocking layer, later is should be parametrizable.
It is pretty slow now and it will be probably always. I think still there is significant speedup possible when reorganizing the script, but I wanna keep the script structured like this until it is finished, to keep it a little readable.
Code:
find it in posts 2, 16 and 17
Important: source has to be perfecly deinterlaced, otherwise the deblocker does nothing

Last edited by redfordxx; 25th November 2007 at 21:48.
redfordxx is offline   Reply With Quote
Old 23rd May 2006, 20:14   #2  |  Link
redfordxx
Registered User
 
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 791
You need
Arrays and Loops.avsi
Patterns.avsi
Code:
function Matrix (clip clp, int "uv")
{
uv     = default(uv,3)       

unit=blankclip(clp,width=4,height=4,pixel_type="YV12")
row=stackhorizontal( 	unit.mt_lut(y=0,u=2,v=2),
	\		unit.mt_lut(y=-1,u=2,v=2),
	\		unit.mt_lut(y=-2,u=2,v=2),
	\		unit.mt_lut(y=-3,u=2,v=2),
	\		unit.mt_lut(y=-4,u=2,v=2),
	\		unit.mt_lut(y=-5,u=2,v=2),
	\		unit.mt_lut(y=-6,u=2,v=2),
	\		unit.mt_lut(y=-7,u=2,v=2))
square=stackvertical( 	row.mt_lut("x 0 +",y=3,u=2,v=2),
	\		row.mt_lut("x 8 +",y=3,u=2,v=2),
	\		row.mt_lut("x 16 +",y=3,u=2,v=2),
	\		row.mt_lut("x 24 +",y=3,u=2,v=2),
	\		row.mt_lut("x 32 +",y=3,u=2,v=2),
	\		row.mt_lut("x 40 +",y=3,u=2,v=2),
	\		row.mt_lut("x 48 +",y=3,u=2,v=2),
	\		row.mt_lut("x 56 +",y=3,u=2,v=2))
block=square#.mt_lut("x 4 *",y=3,u=2,v=2)
block4=stackvertical(stackhorizontal(block,block),stackhorizontal(block,block)).pointresize(16,16)
block4 = (uv!=3) 	? block4.mt_lut(y=3,u=uv,v=uv)
	\		: YtoUV(block4.crop(0,0,8,8),block4.crop(0,0,8,8),block4)
return(block4.trim(1,1))
}




function PtrnFormat(clip clp, clip form)
{
fh=form.height
fw=form.width
clp=StackHorizontal(clp,clp,clp,clp,clp,clp,clp,clp,clp,clp,clp)
clp=StackVertical(clp,clp,clp,clp,clp,clp,clp,clp,clp)
clp=(fw > 160) ? StackHorizontal(clp,clp,clp,clp) : clp
clp=(fh > 128) ? StackVertical(clp,clp,clp,clp) : clp
clp=(fw > 640) ? StackHorizontal(clp,clp,clp) : clp
clp=(fh > 512) ? StackVertical(clp,clp,clp) : clp
clp=clp.crop(0,0,fw,fh)

return(clp.loop(framecount(form)))
}




function PtrnVertical(clip clp, int "uv", int "v0", int "v1", int "v2", int "v3", int "v4", int "v5", int "v6", int "v7")
{
v1     = default(v1,255-v0)       
v2     = default(v2,v1)       
v3     = default(v3,v2)       
v4     = default(v4,v3)       
v5     = default(v5,v2)       
v6     = default(v6,v1)       
v7     = default(v7,v0)       

mtrx=Matrix(clp,uv)
ptrn=mt_lutxy(mtrx,mtrx,"x 8 % 0 == "+String(v0)+" "+String(v1)+" ?",y=3,u=uv,v=uv)
ptrn=mt_lutxy(mtrx,ptrn,"x 8 % 2 == "+String(v2)+" y ?",y=3,u=uv,v=uv)
ptrn=mt_lutxy(mtrx,ptrn,"x 8 % 3 == "+String(v3)+" y ?",y=3,u=uv,v=uv)
ptrn=mt_lutxy(mtrx,ptrn,"x 8 % 4 == "+String(v4)+" y ?",y=3,u=uv,v=uv)
ptrn=mt_lutxy(mtrx,ptrn,"x 8 % 5 == "+String(v5)+" y ?",y=3,u=uv,v=uv)
ptrn=mt_lutxy(mtrx,ptrn,"x 8 % 6 == "+String(v6)+" y ?",y=3,u=uv,v=uv)
ptrn=mt_lutxy(mtrx,ptrn,"x 8 % 7 == "+String(v7)+" y ?",y=3,u=uv,v=uv)

return(ptrn.PtrnFormat(clp))
}




function PtrnHorizontal(clip clp, int "uv", int "v0", int "v1", int "v2", int "v3", int "v4", int "v5", int "v6", int "v7")
{
v1     = default(v1,255-v0)       
v2     = default(v2,v1)       
v3     = default(v3,v2)       
v4     = default(v4,v3)       
v5     = default(v5,v2)       
v6     = default(v6,v1)       
v7     = default(v7,v0)       

mtrx=Matrix(clp,uv)
ptrn=mt_lutxy(mtrx,mtrx,"x 4 - 8 / 8 % 0 == "+String(v0)+" "+String(v1)+" ?",y=3,u=uv,v=uv)
ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 8 % 2 == "+String(v2)+" y ?",y=3,u=uv,v=uv)
ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 8 % 3 == "+String(v3)+" y ?",y=3,u=uv,v=uv)
ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 8 % 4 == "+String(v4)+" y ?",y=3,u=uv,v=uv)
ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 8 % 5 == "+String(v5)+" y ?",y=3,u=uv,v=uv)
ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 8 % 6 == "+String(v6)+" y ?",y=3,u=uv,v=uv)
ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 8 % 7 == "+String(v7)+" y ?",y=3,u=uv,v=uv)

return(ptrn.PtrnFormat(clp))
}



function PtrnBlock(clip clp, int "uv", int "v0", int "v1", int "v2", int "v3")
{
v1     = default(v1,255-v0)       
v2     = default(v2,v1)       
v3     = default(v3,v2)       

mtrx=Matrix(clp,uv)
ptrn=mt_lutxy(mtrx,mtrx,"x 4 - 8 / 8 % 1 >= x 4 - 8 / 8 % 6 <= & x 8 % 1 >= x 8 % 6 <= & & "+String(v1)+" "+String(v0)+" ?",y=3,u=uv,v=uv)
ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 8 % 2 >= x 4 - 8 / 8 % 5 <= & x 8 % 2 >= x 8 % 5 <= & & "+String(v2)+" y ?",y=3,u=uv,v=uv)
ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 8 % 3 >= x 4 - 8 / 8 % 4 <= & x 8 % 3 >= x 8 % 4 <= & & "+String(v3)+" y ?",y=3,u=uv,v=uv)

return(ptrn.PtrnFormat(clp))
}




function PtrnChess(clip clp, int "uv", int "v0", int "v1", int "v2", int "v3")
{
v1     = default(v1,255-v0)       
v2     = default(v2,v0)       
v3     = default(v3,v1)       

mtrx=Matrix(clp,uv)
ptrn=mt_lutxy(mtrx,mtrx,"x 4 - 8 / 2 % 0 == x 2 % 0 == & "+String(v0)+" "+String(v1)+" ?",y=3,u=uv,v=uv)
ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 2 % 1 == x 2 % 1 == & "+String(v2)+" y ?",y=3,u=uv,v=uv)
ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 2 % 1 == x 2 % 0 == & "+String(v3)+" y ?",y=3,u=uv,v=uv)

return(ptrn.PtrnFormat(clp))
}




function PtrnSawH(clip clp, int "uv", string "shift", int "v0", int "v1", int "v2", int "v3", int "v4", int "v5", int "v6", int "v7")
{
v1     = default(v1,255-v0)       
v2     = default(v2,v1)       
v3     = default(v3,v2)       
v4     = default(v4,v3)       
v5     = default(v5,255-v4)       
v6     = default(v6,v5)       
v7     = default(v7,v6)       

mtrx=Matrix(clp,uv)
ptrn=mt_lutxy(mtrx,mtrx,"x 4 - 8 / 8 % 0 == x 2 % "+shift+" 2 %  == & x 4 + 8 / 8 % 0 == x 1 + 2 % "+shift+" 2 % == & | "+String(v0)+" "+String(v1)+" ?",y=3,u=uv,v=uv)
ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 8 % 2 == x 2 % "+shift+" 2 %  == & x 4 + 8 / 8 % 2 == x 1 + 2 % "+shift+" 2 % == & | "+String(v2)+" y ?",y=3,u=uv,v=uv)
ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 8 % 3 == x 2 % "+shift+" 2 %  == & x 4 + 8 / 8 % 3 == x 1 + 2 % "+shift+" 2 % == & | "+String(v3)+" y ?",y=3,u=uv,v=uv)
ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 8 % 4 == x 2 % "+shift+" 2 %  == & x 4 + 8 / 8 % 4 == x 1 + 2 % "+shift+" 2 % == & | "+String(v4)+" y ?",y=3,u=uv,v=uv)
ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 8 % 5 == x 2 % "+shift+" 2 %  == & x 4 + 8 / 8 % 5 == x 1 + 2 % "+shift+" 2 % == & | "+String(v5)+" y ?",y=3,u=uv,v=uv)
ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 8 % 6 == x 2 % "+shift+" 2 %  == & x 4 + 8 / 8 % 6 == x 1 + 2 % "+shift+" 2 % == & | "+String(v6)+" y ?",y=3,u=uv,v=uv)
ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 8 % 7 == x 2 % "+shift+" 2 %  == & x 4 + 8 / 8 % 7 == x 1 + 2 % "+shift+" 2 % == & | "+String(v7)+" y ?",y=3,u=uv,v=uv)

return(ptrn.PtrnFormat(clp))
}



function PtrnSawV(clip clp, int "uv", string "shift", int "v0", int "v1", int "v2", int "v3", int "v4", int "v5", int "v6", int "v7")
{
v1     = default(v1,255-v0)       
v2     = default(v2,v1)       
v3     = default(v3,v2)       
v4     = default(v4,v3)       
v5     = default(v5,255-v4)       
v6     = default(v6,v5)       
v7     = default(v7,v6)       

mtrx=Matrix(clp,uv)
ptrn=mt_lutxy(mtrx,mtrx,"x 4 - 8 / 2 % "+shift+" 2 % == x 8 % 0 == & x 4 + 8 / 2 % "+shift+" 2 % == x 1 + 8 % 0 == & | "+String(v0)+" "+String(v1)+" ?",y=3,u=uv,v=uv)
ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 2 % "+shift+" 2 % == x 8 % 2 == & x 4 + 8 / 2 % "+shift+" 2 % == x 1 + 8 % 2 == & | "+String(v2)+" y ?",y=3,u=uv,v=uv)
ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 2 % "+shift+" 2 % == x 8 % 3 == & x 4 + 8 / 2 % "+shift+" 2 % == x 1 + 8 % 3 == & | "+String(v3)+" y ?",y=3,u=uv,v=uv)
ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 2 % "+shift+" 2 % == x 8 % 4 == & x 4 + 8 / 2 % "+shift+" 2 % == x 1 + 8 % 4 == & | "+String(v4)+" y ?",y=3,u=uv,v=uv)
ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 2 % "+shift+" 2 % == x 8 % 5 == & x 4 + 8 / 2 % "+shift+" 2 % == x 1 + 8 % 5 == & | "+String(v5)+" y ?",y=3,u=uv,v=uv)
ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 2 % "+shift+" 2 % == x 8 % 6 == & x 4 + 8 / 2 % "+shift+" 2 % == x 1 + 8 % 6 == & | "+String(v6)+" y ?",y=3,u=uv,v=uv)
ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 2 % "+shift+" 2 % == x 8 % 7 == & x 4 + 8 / 2 % "+shift+" 2 % == x 1 + 8 % 7 == & | "+String(v7)+" y ?",y=3,u=uv,v=uv)

return(ptrn.PtrnFormat(clp))
}
and
Code:
function AverageBlock8(clip orig)
{
o1=orig.mt_convolution("0 0 0 0 0 0 0 1 1 1 1 1 1 1 1","0 0 0 0 0 0 0 1 1  
1 1 1 1 1 1",y=3,u=3,v=3)
o1=o1.pointresize(o1.width/8,o1.height/8)
o1=o1.pointresize(o1.width*8,o1.height*8)
return(o1)
}


function VerticalLinearBlock(clip orig)
{
p1=PtrnHorizontal(orig,3,255,219,182,146,109,73,36,0)
top1=orig.PointResize(orig.width,orig.height/8)
top2=top1.PointResize(orig.width,orig.height)
shift1=orig.mt_convolution("1","0 0 0 0 0 0 0 0 0 0 0 0 0 0 1",y=3,u=3,v=3)
bottom1=shift1.PointResize(orig.width,orig.height/8)
bottom2=bottom1.PointResize(orig.width,orig.height)
mt_merge(bottom2,top2,p1,y=3,u=3,v=3)
}


function HorizontalLinearBlock(clip orig)
{
p1=PtrnVertical(orig,3,255,219,182,146,109,73,36,0)
top1=orig.PointResize(orig.width/8,orig.height)
top2=top1.PointResize(orig.width,orig.height)
shift1=orig.mt_convolution("0 0 0 0 0 0 0 0 0 0 0 0 0 0 1","1",y=3,u=3,v=3)
bottom1=shift1.PointResize(orig.width/8,orig.height)
bottom2=bottom1.PointResize(orig.width,orig.height)
mt_merge(bottom2,top2,p1,y=3,u=3,v=3)
}
+ DCTFilter
+ Average
which are
http://mywebpages.comcast.net/trbarry/DctFilter.zip
http://people.pwf.cam.ac.uk/mg262/po...ge_21Oct05.dll
from:
http://mywebpages.comcast.net/trbarry/downloads.htm
http://forum.doom9.org/showthread.ph...hlight=average

Last edited by redfordxx; 5th June 2006 at 20:05.
redfordxx is offline   Reply With Quote
Old 23rd May 2006, 20:28   #3  |  Link
redfordxx
Registered User
 
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 791
Usage examples

#deblocks on defaults, ...
source.SmoothDeblock()

#deblocks on density 4, ...
source.SmoothDeblock(density=4,wieghtMX="16center255border0smooth_1",gain=10,pregain=10, pregainoffset=0)

#deblocks on quant 5 density 2, ...
source.SmoothDeblockLayer(5,2,orig.AddBorders(16,16,0,0).WeightMatrixClip("04center255border0smooth_1"),WeightMatrixSum("04center255border0smooth_1"))

#shows which detail would be destroyed by DCTFilter on quant 4
qd4=source.QuantDiff(4)

# makes block oriented mask based on clip from previous function
qd4.AdaptionMask(10,0,10,"1","1 1 1 1 1")

# the higher following values the mor detail and blocking is kept
pregainoffset= default(pregainoffset,-1)
pregain= default(pregain,10)
gain= default(gain,5)
redfordxx is offline   Reply With Quote
Old 25th May 2006, 01:31   #4  |  Link
krieger2005
Registered User
 
krieger2005's Avatar
 
Join Date: Oct 2003
Location: Germany
Posts: 377
+ DCTFilter
+ Average
Line 101: o1=o1.pointresize(o1.width/8,o1.height/8) -> o1=o1.pointresize(m8(o1.width/8),m8(o1.height/8))

I will look on this filter. Thanks
krieger2005 is offline   Reply With Quote
Old 26th May 2006, 12:04   #5  |  Link
redfordxx
Registered User
 
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 791
Quote:
Originally Posted by krieger2005
...-> o1=o1.pointresize(m8(o1.width/8),m8(o1.height/8))
What do you mean by that?
redfordxx is offline   Reply With Quote
Old 26th May 2006, 13:12   #6  |  Link
krieger2005
Registered User
 
krieger2005's Avatar
 
Join Date: Oct 2003
Location: Germany
Posts: 377
I Got an error, because the Resize was not a 'mod 4' resize. So i just want to say, that you must change this to be sure to resize to a 'mod 8'-resolution (YV12-Limits).
krieger2005 is offline   Reply With Quote
Old 26th May 2006, 13:34   #7  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,413
That will mess it up though, it makes some blocks now 1x2 instead of 1x1. (PointResize is picky that way.) You should pad borders to make it mod32 (so the output is mod4, doesn't need to be mod8). Then crop after resizing back up.
__________________
There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order.
foxyshadis is offline   Reply With Quote
Old 26th May 2006, 17:48   #8  |  Link
redfordxx
Registered User
 
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 791
more about WeightMatrixClip()

This function produces the weighting mask for the merging. Parameter clp is there to have same fps,size...

The masks can be named... my convention is that matrix with name 04* needs 4 shifted layers to average and obtain correct image, 16* needs 16 shifted layers to average and obtain correct image. One of advantages of the 16* matrix (for density=4) is that you can pick any numbers there and as long as it is symetric, it will work.

Example --- creation of matrix 04center255border0smooth_1:
For creating the matrix I use WeightMatrixClip function with heavy support of Patterns.avsi
Code:
function WeightMatrixClip{
#getting 8x8 matrix with pixel values 0...63 formated to size of original clip
mx1=clp.Matrix(3).PtrnFormat(clp)
#creating base matrix with color 0 at the border and 255 in the middle etc. (3 is for chroma processing as in mt)
mx=clp.PtrnBlock(3,0,50,200,255)

#each block "is made of 4 squares inside each other". Let's number them 0...3. (3 is in the middle)  
#middle1 (correction of the side middle pixels on square no. 1 and setting on color 55. on mx1 the pixels are numbered 11,12,25,30...)
mx = mt_lutxy(mx,mx1,"y 12 == y 11 == y 25 == y 30 == y 33 == y 38 == y 51 == y 52 == | | | | | | | 55" x ?",y=3,u=3,v=3)
#diag1 (setting the on diagonal pixels on square no. 1 on color 25. on mx1 the pixels are numbered 9,14,49,54)
mx = mt_lutxy(mx,mx1,"y 9 == y 14 == y 54 == y 49 == | | | 25 x ?",y=3,u=3,v=3)
#diag2  (setting the on diagonal pixels on square no. 2 on color 130. on mx1 the pixels are numbered 18,21,42,45)
mx = mt_lutxy(mx,mx1,"y 18 == y 21 == y 42 == y 45 == | | | 130 x ?",y=3,u=3,v=3)
mx
}
The result matrix looks like this:
Code:
0	0	0	0	0	0	0	0
0	25	50	55	55	50	25	0
0	50	130	200	200	130	50	0
0	55	200	255	255	200	55	0
0	55	200	255	255	200	55	0
0	50	130	200	200	130	50	0
0	25	50	55	55	50	25	0
0	0	0	0	0	0	0	0
When you add this matrix fourtimes shifted with vectors <0;0>,<0;4>,<4;0>,<4;4>, you get clip with value 255 everywhere. Therefore, this matrix is sufficient for density=2 which means 2x2 shifts per 8x8 block.
It is necessary to write down the average or nominal value of the matrix. This time it is 255.0/4 (because we added 4 shifted and the result was 255) Later in the script it is used for normalizing the merged clips.The value should be stored in WeightMatrixSum function.

My observation is that when the matrix is steep (hi values in the middle and low at hte border) it produces sharper image but also some artefacts in hi detail areas on hi quants (can look like dots). Flat matrices do not make dots, but make edges .

Last edited by redfordxx; 27th May 2006 at 03:59.
redfordxx is offline   Reply With Quote
Old 27th May 2006, 03:54   #9  |  Link
redfordxx
Registered User
 
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 791
I think I am going to replace the first SmoothDeblockLayer(quant=6 or so...) with quant=3? on half resolution.
Can anybody recommend me some nice downsize and upsize to achieve nice, smooth image after upsizing.
redfordxx is offline   Reply With Quote
Old 27th May 2006, 05:57   #10  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,413
GaussResize with a low p will probably work, or Bicubic playing with the b and c parameters. (I don't know their behavior well enough to guess what would make for a very blurry resize.)
__________________
There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order.
foxyshadis is offline   Reply With Quote
Old 28th May 2006, 13:40   #11  |  Link
redfordxx
Registered User
 
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 791
New version

- it's mod 16 (but prefers 32)
- a little speedup
- added SmoothDeblock2 which has different layers masking, although, I am not sure which can be better...

Last edited by redfordxx; 28th May 2006 at 13:49.
redfordxx is offline   Reply With Quote
Old 28th May 2006, 13:47   #12  |  Link
redfordxx
Registered User
 
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 791
I must say I like the results of deblocking of the SmoothDeblockLayer function provided it is on correct quant. It restores the detail nicely on places I never thought it is possible.

ITOH I am not satisfied with the layers merging mask. There are always problems al least in the areas, where blocking neighbours detail.

Does anybody have an idea, how to implement the adaptive switching among the layers? --- How to create the masks?
redfordxx is offline   Reply With Quote
Old 30th May 2006, 00:49   #13  |  Link
redfordxx
Registered User
 
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 791
I made some samples of deblocking.
I found six different areas throughout the clip, deblocked, cropped and stacked into one picture.

There are to compare:
oroginal

Deblock default

Deblock stronger

DGDecode deblocker default

DGDecode deblocker stronger

SmoothDeblock2(4)

IMO:
SmoothDeblock2 - the deblocking looks very nice and it's most succesful in deblocking. Detail retention is quite good, as long as the detail contrast is over certain level --- otherwise it just disappears (see the lines in the sixth box or drops or thin hair on third box). I hope some tweaking will help. Also I think the total image look is the most similar to original (when fast switching the picture with original, the blocking just disappears and there is no color shift)
Deblock defaults - good detail retention but weak
Deblock stronger - almost as strong as SmoothDeblock and retains more fine detail (means a detail with low contrast), but again (I repeat myself) the hor/vert grid makes the setting higher than default unusable. If there is good detail retention, grid destroys it. You can see it on diagonal lines...
DGDecode (not BlindPP but directly the adaptive one) - a little more succesful in deblocking, than Deblock. The strong setting is already smearing too much.
Now I see, that the adaptivity of DGDecode deblocker suffers the similar problem I complained few posts ago. When meeting hi quant and lo quant blocks, the blocking remains.
(btw, what happened to the fourth box in the strong version? it is quite different from orig)

Conclusion, SmoothDeblock needs
* more detail retention,
* better adaptivity allocation
* probably some protection of "what happens when there is a real edge directly on blockborder"
Then I think it can be OK;-) I don't give up yet...

[EDIT] Newer, more succesful samples later in this thread

Last edited by redfordxx; 5th June 2006 at 16:43.
redfordxx is offline   Reply With Quote
Old 30th May 2006, 01:01   #14  |  Link
krieger2005
Registered User
 
krieger2005's Avatar
 
Join Date: Oct 2003
Location: Germany
Posts: 377
Thanks redfordxx. I use your function for an anime. I get the best results with it. I also this "SmoothDeblock2" give the best result (i use it for my encode ).
I used your function in combination with one of mine, which i made on the fly. It recover some Detail. Still i encode an anime. But maybe it is usefull... so here is it:
Code:
CLmedian(last,SmoothDeblock2(),2,1,1)

function ClMedian(clip c, clip n, int rad, int th1, int th2){
s=gaussianblur(c,rad,rad)
str1="x y - abs "+string(th1)+" > 255 0 ?"
m=YV12LutXY(c,s,str1,str1,str1,Y=3,U=3,V=3)

str2="x y - abs "+string(th2)+" > 255 0 ?"
r=YV12LutXY(c,n,str2,str2,str2,Y=3,V=3,U=3)
r=MaskedMerge(c,n,r,Y=3,V=3,U=3)

MaskedMerge(c,r,m,Y=3,U=3,V=3)
}
Try to play with the red number.

Thanks again for your energy.
krieger2005 is offline   Reply With Quote
Old 2nd June 2006, 08:11   #15  |  Link
708145
Professional Lemming
 
708145's Avatar
 
Join Date: Dec 2003
Location: Stuttgart, Germany
Posts: 359
Quote:
Originally Posted by redfordxx
Remember SmoothD? I tried to figure out how it works and made similar script (that§s why I borrowed the name) but adaptive... and slow

[...] shifted DCTFilter [...] weighting mask [...]
I just had a very quick look at your description and code. The basic idea is OK but I think you shouldn't use DCTfilter.
I experimented with DCTfilter as well a while ago and came to the conclusion that it overfilters crisp details.
The alternative I suggest is a filter (DQI.dll) which sits on my HD for almost a year in a preprealpha state

A realtime version is possible but my first (all C) implementation is quite slow (and still buggy as hell)

But I like your script and I guess it's just a simple substitute of DCTfilter to DQI.

Quote:
Originally Posted by redfordxx
Then, my assumption (not 100% correct) was, blocking is where is hi quantization, so, let's detect the quantization. I detect the quantization with DCTFilter again on certain quant again. That's function QuantDiff. It produces the mask of "what could be removed with lowpass on certain quant".
I have to think more about that idea. At first it sounds great but at second thought I have some doubts.
I fear it will smooth noisy flat regions too much... any test results regarding this?

To sum it up, my plan for a realtime version of this is:
o filter strength detection
o just use 4 shifts
o use DQI with MMX/SSE2
o use a weighting mask

And here is the acronym
CAPS - Content adaptive detail preserving smoother

bis besser,
T0B1A5
__________________
projects page: ELDER, SmoothD, etc.

Last edited by 708145; 2nd June 2006 at 08:15.
708145 is offline   Reply With Quote
Old 2nd June 2006, 09:05   #16  |  Link
redfordxx
Registered User
 
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 791
SmoothDeblock script part one

Code:
###############################################
# SmoothDeblock 0.3 by Redford
#
# 
# issues and todo's:
# find best averaging weight matrices for combining DCT shifts (=minimize artefacts at high quants and increase sharpness)
# find best smoothing of the adaptation mask clip or better, adaptationmask shouldnot be block oriented, but only detail oriented
# 
# tuning and speed up
#
###############################################

function ShiftDCTBack(clip orig,int density, clip wmc, float wms, string dct_type, int i, int j, int DCTorig, int "quant")
{
l=orig.crop(0,0,Int(8/density)*i+16,-0).FlipHorizontal()
r=orig.crop(orig.Width-Int(8/density)*i-16,0,-0,-0).FlipHorizontal()
cMain=(i==0) ? orig : StackHorizontal(l,orig,r)
t=cMain.crop(0,0,0,Int(8/density)*j+16).FlipVertical()
b=cMain.crop(0,orig.height-Int(8/density)*j-16,0,-0).FlipVertical()
cMain=(j==0) ? cMain : StackVertical(t,cMain,b)
cMain=(i+j==0) ? orig : MergeChroma(cMain,cMain.AddBorders(Int(8/density)*i,Int(8/density)*j,0,0).Crop(0,0,cMain.Width,cMain.Height))
cMain=(i==0) ? cMain : cMain.crop(16,0,orig.Width+16,0)
cMain=(j==0) ? cMain : cMain.crop(0,16,0,orig.Height+16)
cMain=((i+j==0)&&(DCTorig==0)) ? orig : (dct_type=="DCTFilterD") ? cMain.DCTFilterD(quant) : cMain.DCTFilter(DctParams7,DctParams6,DctParams5,DctParams4,DctParams3,DctParams2,DctParams1,DctParams0)
cFlot=mt_lutxy(cMain,wmc,"x y * 255 %",y=3,u=3,v=3)
cMain=mt_lutxy(cMain,wmc,"x y * x y * 255 % - 255 /",y=3,u=3,v=3)
cFlot=(i+j==0) ? cFlot : MergeChroma(cFlot,cFlot.AddBorders(0,0,Int(8/density)*i,Int(8/density)*j).Crop(Int(8/density)*i,Int(8/density)*j,cFlot.Width,cFlot.Height))
cMain=(i+j==0) ? cMain : MergeChroma(cMain,cMain.AddBorders(0,0,Int(8/density)*i,Int(8/density)*j).Crop(Int(8/density)*i,Int(8/density)*j,cMain.Width,cMain.Height))
shiftClip.Set(i*density+j+Int(Pow(density,2)),(i+j==0) ? cFlot : cFlot.crop(Int(8/density)*i,Int(8/density)*j,orig.Width(),orig.height))
shiftClip.Set(i*density+j,                    (i+j==0) ? cMain : cMain.crop(Int(8/density)*i,Int(8/density)*j,orig.Width(),orig.height))
global avgI = avgI+", shiftClip"+String(i*density+j)+", 255.0/"+String(wms)
global avgF = (i+j==0) ? avgF : avgF+", shiftClip"+String(i*density+j+Int(Pow(density,2)))+", "+String(1/Pow(density,2))
}

function SmoothDeblockLayer(clip orig,float quant,int density, clip wmc, float wms, string "dct_type",int "DCTorig")
{
global gDCTtype=default(dct_type,"DCTFilter")    #fixed now
global gDensity=density  #number of DCT per 8 pix 
global gOrig=orig
global rep=wms*Pow(density,2)
global q=Int(quant)
global matrix1=wmc
global avgI="clipAverageFloat, "+String(1/wms)
global avgF="shiftClip"+String(Int(Pow(density,2)))+", "+String(1/Pow(density,2))
global DCTO=default(DCTorig,0)

Dim("DctParams",8,0)
Dim("shiftClip",Int(Pow(density,2))*2,Blackness())
For2("i=7","i>="+String(quant),"i=i-1",
\"""
    DctParams.Set(i,1)
""")
For2("i=0", "i<gDensity", "i=i+1",
\"""
    For2("j=0", "j<gDensity", "j=j+1",
    \"
ShiftDCTBack(gOrig,gDensity,matrix1,rep,gDCTtype,i,j,DCTO,q)    
    ")
""")
Eval("clipAverageFloat=Average("+avgF+")")
Eval("Average("+avgI+")")
#Subtitle(avgI)
}




function QuantDiff(clip orig,float "quant",string "dct_type")
{
dct_type = default(dct_type,"DCTFilter")    #fixed now
quant    =default(quant,4)

Dim("DctParams",8,0)
For2("i=7","i>="+String(quant),"i=i-1",
\"""
    DctParams.Set(i,1)
""")
#a=AverageBlock8(orig)
a=orig.DCTFilter(1,0,0,0,0,0,0,0)
n=mt_makediff(orig,a,y=3,u=3,v=3)
d=n.DCTFilter(1,1-DctParams6,1-DctParams5,1-DctParams4,1-DctParams3,1-DctParams2,1-DctParams1,1-DctParams0)
d
}




function AdaptionMask(clip orig,int "pregain",int "pregainoffset",int "gain",string "adaption_blursmall",string "adaption_blurbig")
{
pregainoffset   =default(pregainoffset,-2)
pregain    =default(pregain,100)
gain    =default(gain,100)
adaption_blursmall  =default(adaption_blursmall,"1")
#adaption_blurbig    =default(adaption_blurbig,"1 1 2 2 2 1 1")
adaption_blurbig    =default(adaption_blurbig,"1")

o1=orig.mt_lut("x 128 - abs x 128 == 0 "+string(pregainoffset)+" ? + "+string(pregain)+" *",y=3,u=3,v=3)
o1=o1.mt_convolution("0 0 0 0 0 0 0 1 1 1 1 1 1 1 1","0 0 0 0 0 0 0 1 1 1 1 1 1 1 1",y=3,u=3,v=3).AddBorders(0,0,(32 - (orig.width % 32)) % 32,(32 - (orig.height % 32)) % 32)
o1=o1.pointresize(o1.width/8,o1.height/8)
o1=(adaption_blursmall=="1") ? o1 : \
   (adaption_blursmall=="expand") ? o1.mt_expand(y=3,u=3,v=3) : \
   (adaption_blursmall=="inpand") ? o1.mt_inpand(y=3,u=3,v=3) : \
   (adaption_blursmall=="inflate") ? o1.mt_inflate(y=3,u=3,v=3) : \
   (adaption_blursmall=="deflate") ? o1.mt_deflate(y=3,u=3,v=3) : \
  o1.mt_convolution(adaption_blursmall,adaption_blursmall,y=3,u=3,v=3)
o1=o1.pointresize(o1.width*8,o1.height*8).Crop(0,0,orig.width,orig.height)
o1=(adaption_blurbig=="1") ? o1 : o1.mt_convolution(adaption_blurbig,adaption_blurbig,y=3,u=3,v=3)
o1=o1.mt_lut("x "+string(gain)+" *",y=3,u=3,v=3)
return(o1)
}



function AdaptionMaskDiff(clip orig, clip deb,float "quant", int "pregain", int "pregainoffset", int "gain", int "thresh", string "adaption_blur", int "borders", int "expands" ,string "dct_type")
{
pregainoffset   =default(pregainoffset,-3)
pregain    =default(pregain,100)
borders   =default(borders,0)
expands    =default(expands,1)
gain    =default(gain,8)
thresh    =default(thresh,2)
adaption_blur    =default(adaption_blur,"1 2 1")
dct_type =default(dct_type,"DCTFilter")    #fixed now
quant    =default(quant,4)

Dim("DctParams",8,0)
For2("i=7","i>="+String(quant),"i=i-1",
\"""
    DctParams.Set(i,1)
""")
#a=AverageBlock8(orig)
a=orig.DCTFilter(1,0,0,0,0,0,0,0)
n=mt_makediff(orig,a,y=3,u=3,v=3)
d=n.DCTFilter(1,1-DctParams6,1-DctParams5,1-DctParams4,1-DctParams3,1-DctParams2,1-DctParams1,1-DctParams0)

o1=d.mt_lut("x 128 - abs x 128 == 0 "+string(pregainoffset)+" ? + "+string(pregain)+" *",y=3,u=3,v=3)
o1=o1.DCTFilter(1,0,0,0,0,0,0,0)
o1=o1.mt_lut("x "+string(pregain/5)+" *",y=3,u=3,v=3)

diff=mt_makediff(deb,orig,y=3,u=3,v=3)
diff2=mt_lutxy(o1,diff,"y 128 - x * 255 / abs "+string(thresh)+" -",y=3,u=3,v=3) 
p1=diff2.PtrnBlock(3,borders,255,255,255)
diff3=mt_lutxy(diff2,p1,"x y * 255 / "+string(gain)+" *",y=3,u=3,v=3)
diff4=diff3.mt_expand(y=3,u=3,v=3)
(adaption_blur=="1") ? diff4 : diff4.mt_convolution(adaption_blur,adaption_blur,y=3,u=3,v=3)
mt_lut("x "+string(gain)+" *",y=3,u=3,v=3)
(expands>0) ? mt_expand(y=3,u=3,v=3) : last
(expands>1) ? mt_expand(y=3,u=3,v=3) : last
(expands>2) ? mt_expand(y=3,u=3,v=3) : last
(expands>3) ? mt_expand(y=3,u=3,v=3) : last
(expands>4) ? mt_expand(y=3,u=3,v=3) : last
}





function WeightMatrixClip(clip orig, string "name",int "level")
{
level=default(level,0)
clp=orig.Crop(0,0,16,16).Trim(1,1)
name=default(name,"04center255x2")

mx1 =  (name=="04all78")||(name=="04center255x2")||(name=="16center255") ? clp : clp.Matrix(3).PtrnFormat(clp)
mx  =  (name=="04center255border0smooth_1") ? clp.PtrnBlock(3,0,50,200,255) : \
   (name=="16center255border0smooth_1") ? clp.PtrnBlock(3,0,70,185,255) : \
   (name=="16center255border0smooth_2") ? clp.PtrnBlock(3,100,150,220,255) : clp
#middle1
mx  =   (name=="04all78")||(name=="04center255x2")||(name=="16center255") ? mx : mt_lutxy(mx,mx1,"y 12 == y 11 == y 25 == y 30 == y 33 == y 38 == y 51 == y 52 == | | | | | | | " + ( \
(name=="04center255border0smooth_1") ? "55"  : \
(name=="16center255border0smooth_1") ? "115" : \
(name=="16center255border0smooth_2") ? "180" : \
  "x") + " x ?",y=3,u=3,v=3)
#diag0
#diag1
mx  =   (name=="04all78")||(name=="04center255x2")||(name=="16center255") ? mx : mt_lutxy(mx,mx1,"y 9 == y 14 == y 54 == y 49 == | | | " + ( \
(name=="04center255border0smooth_1") ? "25" : \
(name=="16center255border0smooth_1") ? "20" : \
(name=="16center255border0smooth_2") ? "130" : \
  "x") + " x ?",y=3,u=3,v=3)
#diag2  
mx  =   (name=="04all78")||(name=="04center255x2")||(name=="16center255") ? mx : mt_lutxy(mx,mx1,"y 18 == y 21 == y 42 == y 45 == | | | " + ( \
(name=="04center255border0smooth_1") ? "130" : \
(name=="16center255border0smooth_1") ? "140" : \
(name=="16center255border0smooth_2") ? "200" : \
  "x") + " x ?",y=3,u=3,v=3)

(name=="16center255")   ? (clp.PtrnBlock(3,0,0,0,255)) : \
(name=="04center255x2") ? (clp.PtrnBlock(3,0,0,255,255)): \
(name=="04all78") ? (clp.PtrnBlock(3,150,150,212,212)): \
   mx
#mt_lut(String(level)+" 255 "+String(level)+" - 255 / x * +",y=3,u=3,v=3)
PtrnFormat(orig)
}

function WeightMatrixSum(string "name",int "level")
{
level=default(level,0)
l=(name=="16center255")    ? (255.0/16) : \
(name=="04center255x2")  ? (255.0/4): \
(name=="04center255border0smooth_1") ? (255.0/4): \
(name=="16center255border0smooth_1") ? (1155.0/16): \
(name=="16center255border0smooth_2") ? (2385.0/16): \
(name=="04all78")  ? (662.0/4): \
     (255.0/4)
#return((255.0-level)/255.0*l+level)
l
}



function BorderMask(clip orig, int "bth1", int "bth2", int "bshare", int "eth1", int "eth2", int "eshare", float "buvmod", float "euvmod")
{
bth1=default(bth1,0)
bth2=default(bth2,2)
bshare=default(bshare,200)
eth1=default(eth1,30)
eth2=default(eth2,50)
eshare=default(eshare,0)  #0=real edge protection off
buvmod=default(buvmod,2.0)
euvmod=default(euvmod,2.4)

ph=orig.PtrnHorizontal(3,0,1,2,3,4,5,6,7)
pv=orig.PtrnVertical(3,0,1,2,3,4,5,6,7) 
h1a=orig.mt_lutxy(ph,"y 0 == y 7 == | x 128 ?",y=3,u=3,v=3)
v1a=orig.mt_lutxy(pv,"y 0 == y 7 == | x 128 ?",y=3,u=3,v=3)
h1b=orig.mt_lutxy(ph,"y 1 == y 6 == | x 128 ?",y=3,u=3,v=3)
v1b=orig.mt_lutxy(pv,"y 1 == y 6 == | x 128 ?",y=3,u=3,v=3)
h2a=h1a.mt_convolution(horizontal="1",vertical="1 -1 1",total=1,y=3,u=3,v=3)
v2a=v1a.mt_convolution(horizontal="1 -1 1",vertical="1",total=1,y=3,u=3,v=3)
h2b=h1b.mt_convolution(horizontal="1",vertical="1 -1 1 -1 1",total=1,y=3,u=3,v=3)
v2b=v1b.mt_convolution(horizontal="1 -1 1 -1 1",vertical="1",total=1,y=3,u=3,v=3)
v2=mt_lutxy(v2a,v2b,"x 128 - y 128 - 3 / - abs x 128 - abs > x x y 128 - 3 / - ?",y=3,u=3,v=3)
h2=mt_lutxy(h2a,h2b,"x 128 - y 128 - 3 / - abs x 128 - abs > x x y 128 - 3 / - ?",y=3,u=3,v=3)
v3b=v2.mt_lutxy(pv,"y 0 == y 7 == | x 128 ? 128 - abs 5 * 1.1 * ",y=3,u=3,v=3).mt_convolution(horizontal="1",vertical="1 1 1",y=3,u=3,v=3)
h3b=h2.mt_lutxy(ph,"y 0 == y 7 == | x 128 ? 128 - abs 5 * 1.1 * ",y=3,u=3,v=3).mt_convolution(horizontal="1 1 1",vertical="1",y=3,u=3,v=3)
v3e0=v2a.mt_convolution(horizontal="1",vertical="1 1 1 1 1 1 1 1 1",y=3,u=2,v=2).mt_convolution(horizontal="1",vertical="1 1 1 1 1 1 1",y=2,u=3,v=3)
h3e0=h2a.mt_convolution(horizontal="1 1 1 1 1 1 1 1 1",vertical="1",y=3,u=2,v=2).mt_convolution(horizontal="1 1 1 1 1 1 1",vertical="1",y=2,u=3,v=3)
v3e=(eshare==0) ? orig : v3e0.mt_lutxy(pv,yexpr="y 0 == y 7 == | x 128 ? 128 - abs "+String(eth1)+" - "+String(eth2)+" "+String(eth1)+" - / 255 *", \
					 uexpr="y 0 == y 7 == | x 128 ? 128 - abs "+String(eth1/euvmod)+" - "+String(eth2/euvmod)+" "+String(eth1/euvmod)+" - / 255 *", \
					 vexpr="y 0 == y 7 == | x 128 ? 128 - abs "+String(eth1/euvmod)+" - "+String(eth2/euvmod)+" "+String(eth1/euvmod)+" - / 255 *",y=3,u=3,v=3)
h3e=(eshare==0) ? orig : h3e0.mt_lutxy(ph,yexpr="y 0 == y 7 == | x 128 ? 128 - abs "+String(eth1)+" - "+String(eth2)+" "+String(eth1)+" - / 255 *", \
					 uexpr="y 0 == y 7 == | x 128 ? 128 - abs "+String(eth1/euvmod)+" - "+String(eth2/euvmod)+" "+String(eth1/euvmod)+" - / 255 *", \
					 vexpr="y 0 == y 7 == | x 128 ? 128 - abs "+String(eth1/euvmod)+" - "+String(eth2/euvmod)+" "+String(eth1/euvmod)+" - / 255 *",y=3,u=3,v=3)
v4e=(eshare==0) ? orig : v3e.HorizontalLinearBlock
h4e=(eshare==0) ? orig : h3e.VerticalLinearBlock
e1=(eshare==0) ? orig : mt_lutxy(v4e,h4e,"x y > x y ?",y=3,u=3,v=3)
e2=(eshare==0) ? orig : e1.mt_lut("255 x - "+String(eshare)+" * 255 / "+String(eshare)+" - 255 +",y=3,u=3,v=3)
v4b=(eshare==0) ? v3b : v3b.mt_lutxy(v3e,"x y 1 + /",y=3,u=3,v=3)
h4b=(eshare==0) ? h3b : h3b.mt_lutxy(h3e,"x y 1 + /",y=3,u=3,v=3)
v5b=v4b.HorizontalLinearBlock
h5b=h4b.VerticalLinearBlock
b1=mt_lutxy(v5b,h5b,yexpr="x y > x y ? 5 / "+String(bth1)+" - "+String(bth2)+" "+String(bth1)+" - / 255 *", \
			        uexpr="x y > x y ? 5 / "+String(bth1/buvmod)+" - "+String(bth2/buvmod)+" "+String(bth1/buvmod)+" - / 255 *", \
			        vexpr="x y > x y ? 5 / "+String(bth1/buvmod)+" - "+String(bth2/buvmod)+" "+String(bth1/buvmod)+" - / 255 *",y=3,u=3,v=3)
b2=b1.mt_convolution(horizontal="1 1 1 1 1",vertical="1 1 1 1 1",y=3,u=3,v=3)
b3=b2.mt_lut("x "+String(bshare)+" * 255 / "+String(bshare)+" - 255 +",y=3,u=3,v=3)
(eshare==0) ? b3 : mt_lutxy(e2,b3,"x y < x y ?",y=3,u=3,v=3)
}

Last edited by redfordxx; 3rd June 2006 at 11:55.
redfordxx is offline   Reply With Quote
Old 2nd June 2006, 09:07   #17  |  Link
redfordxx
Registered User
 
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 791
SmooothDeblock part two

Code:
function SmoothDeblock1(clip orig, int "passes", int "density", int "inclOrig", string "wieghtMX",string "dct_type",int "pregain",int "pregainoffset",int "gain",string "adaption_blursmall",string "adaption_blurbig")
{
passes=3#fixed for now
inclOrig=default(inclOrig,1)
density=default(density,2)
pregainoffset=default(pregainoffset,0)
pregain=default(pregain,10)
gain=default(gain,10)
b1=default(adaption_blursmall,"1")
b2=default(adaption_blurbig,"1 1 2 2 2 1 1")
wieghtMX=default(wieghtMX,"04center255border0smooth_1")
dct_type=default(dct_type,"DCTFilter")

mx2=orig.AddBorders(16,16,0,0).WeightMatrixClip(wieghtMX)
mxs=WeightMatrixSum(wieghtMX)





sdl8=orig.LanczosResize(orig.width/2,orig.height/2)
sdl8=((sdl8.Width % 32)==0) ? sdl8 : StackHorizontal(sdl8,sdl8.crop(sdl8.Width-32+(sdl8.Width % 32),0,-0,-0).FlipHorizontal())
sdl8=((sdl8.height % 32)==0) ? sdl8 : StackVertical(sdl8,sdl8.crop(0,sdl8.height-32+(sdl8.height % 32),-0,-0).FlipVertical())
sdl8=sdl8.SmoothDeblockLayer(5,density,mx2,mxs,"DCTFilter",DCTorig=1)
sdl8=sdl8.GaussResize(sdl8.width*2,sdl8.height*2,p=20).crop(0,0,orig.width,orig.height)
#sdl8=sdl8.mt_convolution("1 2 1","1 2 1",y=3,u=3,v=3)
sdl5=orig.SmoothDeblockLayer(5,density,mx2,mxs,"DCTFilter")
sdl3=orig.SmoothDeblockLayer(3,density,mx2,mxs,"DCTFilter")
qd8=orig.QuantDiff(6)
qd5=orig.QuantDiff(5)
qdo=orig.QuantDiff(inclOrig)
am8=qd8.AdaptionMask(pregain,pregainoffset,gain,b1,b2)
am5=qd5.AdaptionMask(pregain,pregainoffset,gain,b1,b2)
amo=qdo.AdaptionMask(pregain,pregainoffset,gain,b1,b2)

deblocked=mt_merge(sdl8,sdl5,am8,y=3,u=3,v=3) 
deblocked=mt_merge(deblocked,sdl3,am5,y=3,u=3,v=3) 
deblocked=(inclOrig==0) ? deblocked : mt_merge(deblocked,orig,amo,y=3,u=3,v=3) 
return(deblocked)
}



function SmoothDeblock2(clip orig, int "passes", int "density", int "inclOrig", string "wieghtMX",string "dct_type",int "pregain",int "pregainoffset",int "gain", int "thresh", string "adaption_blur")
{
passes=3#fixed for now
inclOrig=default(inclOrig,2)
density=default(density,2)
#pregainoffset=default(pregainoffset,-2)
#pregain=default(pregain,100)
#gain=default(gain,30)
#thresh=default(thresh,1)
#b2=default(adaption_blurbig,"1 1 1")
wieghtMX=default(wieghtMX,(density==4) ? "16center255border0smooth_1" : "04center255border0smooth_1")
dct_type=default(dct_type,"DCTFilter")

mx2=orig.AddBorders(16,16,0,0).WeightMatrixClip(wieghtMX)
mxs=WeightMatrixSum(wieghtMX)

sdl8=orig.LanczosResize(orig.width/2,orig.height/2)
sdl8=((sdl8.Width % 32)==0) ? sdl8 : StackHorizontal(sdl8,sdl8.crop(sdl8.Width-32+(sdl8.Width % 32),0,-0,-0).FlipHorizontal())
sdl8=((sdl8.height % 32)==0) ? sdl8 : StackVertical(sdl8,sdl8.crop(0,sdl8.height-32+(sdl8.height % 32),-0,-0).FlipVertical())
sdl8=sdl8.SmoothDeblockLayer(5,density,mx2,mxs,"DCTFilter",DCTorig=1)
sdl8=sdl8.GaussResize(sdl8.width*2,sdl8.height*2,p=20).crop(0,0,orig.width,orig.height)
#sdl8=sdl8.mt_convolution("1 2 1","1 2 1",y=3,u=3,v=3)
sdl5=orig.SmoothDeblockLayer(5,density,mx2,mxs,"DCTFilter")
sdl3=orig.SmoothDeblockLayer(3,density,mx2,mxs,"DCTFilter")
am8=orig.AdaptionMaskDiff(sdl8,6,pregain,pregainoffset,gain,thresh,adaption_blur,expands=5)
am5=orig.AdaptionMaskDiff(sdl5,5,pregain,pregainoffset,gain,thresh,adaption_blur,expands=2)
amo=orig.AdaptionMaskDiff(sdl3,inclOrig,pregain,pregainoffset,gain,thresh,adaption_blur,expands=1)

deblocked=sdl8
deblocked=mt_merge(deblocked,sdl5,am8,y=3,u=3,v=3) 
deblocked=mt_merge(deblocked,sdl3,am5,y=3,u=3,v=3) 
deblocked=(inclOrig==0) ? deblocked : mt_merge(deblocked,orig,amo,y=3,u=3,v=3) 
return(deblocked)
}



function SmoothDeblock3(clip orig, int "passes", int "density", int "inclOrig", string "wieghtMX",string "dct_type",int "pregain",int "pregainoffset",int "gain", int "thresh", string "adaption_blur", int "bth1", int "bth2", int "bshare", int "eth1", int "eth2", int "eshare", float "buvmod", float "euvmod")
{
passes=3#fixed for now
inclOrig=default(inclOrig,2)
density=default(density,2)
wieghtMX=default(wieghtMX,(density==4) ? "16center255border0smooth_1" : "04center255border0smooth_1")
dct_type=default(dct_type,"DCTFilter")

mx2=orig.AddBorders(16,16,0,0).WeightMatrixClip(wieghtMX)
mxs=WeightMatrixSum(wieghtMX)
bdm=orig.BorderMask(bth1, bth2, bshare, eth1, eth2, eshare)

sdl8=orig.LanczosResize(orig.width/2,orig.height/2)
sdl8=((sdl8.Width % 32)==0) ? sdl8 : StackHorizontal(sdl8,sdl8.crop(sdl8.Width-32+(sdl8.Width % 32),0,-0,-0).FlipHorizontal())
sdl8=((sdl8.height % 32)==0) ? sdl8 : StackVertical(sdl8,sdl8.crop(0,sdl8.height-32+(sdl8.height % 32),-0,-0).FlipVertical())
sdl8=sdl8.SmoothDeblockLayer(5,density,mx2,mxs,"DCTFilter",DCTorig=1)
sdl8=sdl8.GaussResize(sdl8.width*2,sdl8.height*2,p=20).crop(0,0,orig.width,orig.height)
#sdl8=sdl8.mt_convolution("1 2 1","1 2 1",y=3,u=3,v=3)
sdl5=orig.SmoothDeblockLayer(5,density,mx2,mxs,"DCTFilter")
sdl3=orig.SmoothDeblockLayer(3,density,mx2,mxs,"DCTFilter")
am8=orig.AdaptionMaskDiff(sdl8,6,pregain,pregainoffset,gain,thresh,adaption_blur,expands=5).mt_lutxy(bdm,"255 255 x - y * 255 / -",y=3,u=3,v=3)#   255-((255-x)*y)/255 = x+255-y-x*(255-y)/255
am5=orig.AdaptionMaskDiff(sdl5,5,pregain,pregainoffset,gain,thresh,adaption_blur,expands=2).mt_lutxy(bdm,"255 255 x - y * 255 / -",y=3,u=3,v=3)
amo=orig.AdaptionMaskDiff(sdl3,inclOrig,pregain,pregainoffset,gain,thresh,adaption_blur,expands=1).mt_lutxy(bdm,"255 255 x - y * 255 / -",y=3,u=3,v=3)

deblocked=sdl8
deblocked=mt_merge(deblocked,sdl5,am8,y=3,u=3,v=3) 
deblocked=mt_merge(deblocked,sdl3,am5,y=3,u=3,v=3) 
deblocked=(inclOrig==0) ? deblocked : mt_merge(deblocked,orig,amo,y=3,u=3,v=3) 
return(deblocked)
}



function SmoothDeblock4(clip orig, int "passes", int "density", int "inclOrig", string "wieghtMX",string "dct_type",int "pregain",int "pregainoffset",int "gain",string "adaption_blursmall",string "adaption_blurbig", int "bth1", int "bth2", int "bshare", int "eth1", int "eth2", int "eshare", float "buvmod", float "euvmod")
{
passes=3#fixed for now
inclOrig=default(inclOrig,1)
density=default(density,2)
pregainoffset=default(pregainoffset,0)
pregain=default(pregain,10)
gain=default(gain,10)
b1=default(adaption_blursmall,"1")
b2=default(adaption_blurbig,"1 1 2 2 2 1 1")
wieghtMX=default(wieghtMX,"04center255border0smooth_1")
dct_type=default(dct_type,"DCTFilter")

mx2=orig.AddBorders(16,16,0,0).WeightMatrixClip(wieghtMX)
mxs=WeightMatrixSum(wieghtMX)
bdm=orig.BorderMask(bth1, bth2, bshare, eth1, eth2, eshare)



sdl8=orig.LanczosResize(orig.width/2,orig.height/2)
sdl8=((sdl8.Width % 32)==0) ? sdl8 : StackHorizontal(sdl8,sdl8.crop(sdl8.Width-32+(sdl8.Width % 32),0,-0,-0).FlipHorizontal())
sdl8=((sdl8.height % 32)==0) ? sdl8 : StackVertical(sdl8,sdl8.crop(0,sdl8.height-32+(sdl8.height % 32),-0,-0).FlipVertical())
sdl8=sdl8.SmoothDeblockLayer(5,density,mx2,mxs,"DCTFilter",DCTorig=1)
sdl8=sdl8.GaussResize(sdl8.width*2,sdl8.height*2,p=20).crop(0,0,orig.width,orig.height)
#sdl8=sdl8.mt_convolution("1 2 1","1 2 1",y=3,u=3,v=3)
sdl5=orig.SmoothDeblockLayer(5,density,mx2,mxs,"DCTFilter")
sdl3=orig.SmoothDeblockLayer(3,density,mx2,mxs,"DCTFilter")
qd8=orig.QuantDiff(6)
qd5=orig.QuantDiff(5)
qdo=orig.QuantDiff(inclOrig)
am8=qd8.AdaptionMask(pregain,pregainoffset,gain,b1,b2).mt_lutxy(bdm,"255 255 x - y * 255 / -",y=3,u=3,v=3)
am5=qd5.AdaptionMask(pregain,pregainoffset,gain,b1,b2).mt_lutxy(bdm,"255 255 x - y * 255 / -",y=3,u=3,v=3)
amo=qdo.AdaptionMask(pregain,pregainoffset,gain,b1,b2).mt_lutxy(bdm,"255 255 x - y * 255 / -",y=3,u=3,v=3)

deblocked=mt_merge(sdl8,sdl5,am8,y=3,u=3,v=3) 
deblocked=mt_merge(deblocked,sdl3,am5,y=3,u=3,v=3) 
deblocked=(inclOrig==0) ? deblocked : mt_merge(deblocked,orig,amo,y=3,u=3,v=3) 
return(deblocked)
}
SmoothDeblock3 added - there is detail preservation and real border-edge preservation routine added.
How it works: it looks how strong are the edges of the block. If they are small, the deblocking is reduced (not necessary) if they are big it is also reduced (coz the edge should stay there).

Thresholds for blocking:
bth1=0 bth2=3 means that the deblocking will be eliminated, when the border difference is 0 and deblocking will be applied without change when the block border difference is > 3
bshare=overall impact of the correction 0=none 255=full. The (bth1;bth2) interval mapped onto mask scaled (0;255) is squeezed to (255-bshare;255) For example bshare=150 mean that when there is 0 block edge detected, the deblocking is reduced only to ((255-150)/255*100)%.
bshare=0 --> 100% deblocking

Thresholds for real edges are similar:
When the block edge difference is < eth1, no change.
When the block edge difference is > eth2, full deblocking reduction.
eshare=overall correction impact modifier.

The b-part is imo quite acceptable, but the e-part is prete bad. There are many edges missed and some real blocking is misjudged as edge.
And I think the real edges destruction is the worst part of this script.


IMO SmoothDeblock3 at default settings performs best.

Last edited by redfordxx; 3rd June 2006 at 12:02.
redfordxx is offline   Reply With Quote
Old 2nd June 2006, 10:31   #18  |  Link
redfordxx
Registered User
 
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 791
Quote:
Originally Posted by 708145
I suggest is a filter (DQI.dll)
And what is the difference?
Quote:
I have to think more about that idea. At first it sounds great but at second thought I have some doubts.
I fear it will smooth noisy flat regions too much... any test results regarding this?
Well the question is whether smoothing noisy areas is bad or good;-) But definitely, fine detail was problem in SmoothDeblock2. SmoothDeblock3 is little better... But it does not remove the cause of the problem it only rescues something of what was damaged
Quote:
To sum it up, my plan for a realtime version of this is:
I always thought that coding this script would dramatically increase speed, because is the script many operations are useless just because I have to apply the operation on whole clip. But the idea still needs to be improved.
Quote:
o just use 4 shifts
benefits of 16:
little sharper and less artifact (artifacts appear when are highly quantized hi freq areas... compare SmoothDeblockLayer(quant=6,density=2 or 4))
you can choose any weighting mask you want
Quote:
And here is the acronym
CAPS - Content adaptive detail preserving smoother
Hmmm shouldn't be the word deblock in the name? (reason - search)



I have one more approach to try to implement: First adapting and then merging shifts.
For instance split the frame to areas 32x32 and apply appropriate quant only to that area. It would be milions of crops in script...;-)
For this is needed very good and smooth adaptation mask prepared.

Last edited by redfordxx; 2nd June 2006 at 10:34.
redfordxx is offline   Reply With Quote
Old 2nd June 2006, 14:23   #19  |  Link
708145
Professional Lemming
 
708145's Avatar
 
Join Date: Dec 2003
Location: Stuttgart, Germany
Posts: 359
DQI is a simple filter that simulates real quantization similar to jpeg compression and decompression.

Quote:
Originally Posted by redfordxx
Well the question is whether smoothing noisy areas is bad or good;-) But definitely, fine detail was problem in SmoothDeblock2. SmoothDeblock3 is little better... But it does not remove the cause of the problem it only rescues something of what was damaged

benefits of 16:
little sharper and less artifact (artifacts appear when are highly quantized hi freq areas... compare SmoothDeblockLayer(quant=6,density=2 or 4))
So you are curing the symptoms
Maybe improving the previous step is better.

Quote:
Originally Posted by redfordxx
I have one more approach to try to implement: First adapting and then merging shifts.
For instance split the frame to areas 32x32 and apply appropriate quant only to that area. It would be milions of crops in script...;-)
For this is needed very good and smooth adaptation mask prepared.
Yeah! As a script it's ugly. As a filter even 8x8 is easy and fast.

bis besser,
T0B1A5
__________________
projects page: ELDER, SmoothD, etc.
708145 is offline   Reply With Quote
Old 2nd June 2006, 17:19   #20  |  Link
redfordxx
Registered User
 
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 791
Quote:
Originally Posted by 708145
So you are curing the symptoms
Maybe improving the previous step is better.
100% agree... but for now... it was just faster.

But the "real edge protection"...I have no idea at the moment, how to improve it


Quote:
Yeah! As a script it's ugly. As a filter even 8x8 is easy and fast.
Agree again...
... but I wanted first to test it a little before "convincing someone else that it is worth coding".

OTOH, if this would be coded not as one filter but as a set of filters, the partial solution can be made.

Let's say the final solution would be:
1 - prepare adapt masks
2 - shift clips
3 - adaptively apply DCT on shifts
4 - shift back
5 - mask merge

Then for step 3 could be 1 filter (name DCTFilterMask) with two input clips:
- original
- 8xreduced resolution mask
DCTFilter(c,i0,i1,i2,i3,i4,i5,i6,i7) filter normally applies i0-i7 parameters
DCTFilterMask(c,m) would read the parameters for each block in the pixel of clip m. I wonder, whether the DCTFilter can be modified that way easily
or your filter:
DQIFilterMask(c,m,pathtoquantmatrixtxtfile) - m would contain quantizers. I know you mentioned somewhere you use 6of9 as default, but from many sources it is possibler to read the true quant matrix, so why not to use it.

steps 5 can be maybe easily solved by modifying the Average filter... look here

with these two filters the script would be much faster and there is more time for experimenting with the adapt mask creation
and later, after it is tested, the adapt matrix creation filter will be made

Last edited by redfordxx; 2nd June 2006 at 17:22.
redfordxx is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 05:22.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.