Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
23rd May 2006, 20:08 | #1 | Link |
Registered User
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 863
|
SmoothDeblock (under development)
Hi
hope you are not bored with my deblocking attempts... Here is my latest trial: Remember SmoothD? I tried to figure out how it works and made similar script (that§s why I borrowed the name) but adaptive... and slow I made shifted DCTFilter applications to remove high frequencies. This caused removing the blocking on original block borders (shifted into middles of the blocks) but introduced new blocking on the new borders. So, when I shift the clips back, I have (for example) four clips with blocking on diferent places. I merge the clips so that I take only the inner part of the block (without blocking edge). For this I use weighting mask made by function WeightMatrixClip which is clip made of all the same 8x8 blocks with values 255 in the middle and going down to borders. Additional functionWeightMatrixSum delivers the average value of the mask clip which is later used to normalize the weighted average. This all delivers function SmoothDeblockLayer and it is possible to choose the density, quant, weightmask. density = number of shifts on each axis per 8pixels - possible values are 2 or 4. (so 4 means 16 shifts per 8x8 block), 4 produces sharper image and less artifacts on hi quants but is more than 4times slower than 2, which means only 4 shifts...) quant = 0,...,7 number of coeficients removed by DCTfilter. 0 makes no change to the picture. weightmask = WeightMatrixClip mask together with WeightMatrixSum controls how the shifted clips are merged. So, SmoothDeblockLayer returns deblocked clip on certain quant. I can have 7 different deblocked clips according to quant. I want to merge some of them depending on detail presence. Then, my assumption (not 100% correct) was, blocking is where is hi quantization, so, let's detect the quantization. I detect the quantization with DCTFilter again on certain quant again. That's function QuantDiff. It produces the mask of "what could be removed with lowpass on certain quant". From this mask, the function AdaptionMask produces mask for merging the deblocked "layers". It is just some values increasing, blockaveraging and blurring depending on given parameters. This is part wherer I am not sure. Now it is block oriented but maybe the mask should more precisely follow the detail of the clip. Finally the SmoothDeblock function puts everything together. Now it is fixed for 3 deblocking layer, later is should be parametrizable. It is pretty slow now and it will be probably always. I think still there is significant speedup possible when reorganizing the script, but I wanna keep the script structured like this until it is finished, to keep it a little readable. Code:
find it in posts 2, 16 and 17 Last edited by redfordxx; 25th November 2007 at 22:48. |
23rd May 2006, 20:14 | #2 | Link |
Registered User
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 863
|
You need
Arrays and Loops.avsi Patterns.avsi Code:
function Matrix (clip clp, int "uv") { uv = default(uv,3) unit=blankclip(clp,width=4,height=4,pixel_type="YV12") row=stackhorizontal( unit.mt_lut(y=0,u=2,v=2), \ unit.mt_lut(y=-1,u=2,v=2), \ unit.mt_lut(y=-2,u=2,v=2), \ unit.mt_lut(y=-3,u=2,v=2), \ unit.mt_lut(y=-4,u=2,v=2), \ unit.mt_lut(y=-5,u=2,v=2), \ unit.mt_lut(y=-6,u=2,v=2), \ unit.mt_lut(y=-7,u=2,v=2)) square=stackvertical( row.mt_lut("x 0 +",y=3,u=2,v=2), \ row.mt_lut("x 8 +",y=3,u=2,v=2), \ row.mt_lut("x 16 +",y=3,u=2,v=2), \ row.mt_lut("x 24 +",y=3,u=2,v=2), \ row.mt_lut("x 32 +",y=3,u=2,v=2), \ row.mt_lut("x 40 +",y=3,u=2,v=2), \ row.mt_lut("x 48 +",y=3,u=2,v=2), \ row.mt_lut("x 56 +",y=3,u=2,v=2)) block=square#.mt_lut("x 4 *",y=3,u=2,v=2) block4=stackvertical(stackhorizontal(block,block),stackhorizontal(block,block)).pointresize(16,16) block4 = (uv!=3) ? block4.mt_lut(y=3,u=uv,v=uv) \ : YtoUV(block4.crop(0,0,8,8),block4.crop(0,0,8,8),block4) return(block4.trim(1,1)) } function PtrnFormat(clip clp, clip form) { fh=form.height fw=form.width clp=StackHorizontal(clp,clp,clp,clp,clp,clp,clp,clp,clp,clp,clp) clp=StackVertical(clp,clp,clp,clp,clp,clp,clp,clp,clp) clp=(fw > 160) ? StackHorizontal(clp,clp,clp,clp) : clp clp=(fh > 128) ? StackVertical(clp,clp,clp,clp) : clp clp=(fw > 640) ? StackHorizontal(clp,clp,clp) : clp clp=(fh > 512) ? StackVertical(clp,clp,clp) : clp clp=clp.crop(0,0,fw,fh) return(clp.loop(framecount(form))) } function PtrnVertical(clip clp, int "uv", int "v0", int "v1", int "v2", int "v3", int "v4", int "v5", int "v6", int "v7") { v1 = default(v1,255-v0) v2 = default(v2,v1) v3 = default(v3,v2) v4 = default(v4,v3) v5 = default(v5,v2) v6 = default(v6,v1) v7 = default(v7,v0) mtrx=Matrix(clp,uv) ptrn=mt_lutxy(mtrx,mtrx,"x 8 % 0 == "+String(v0)+" "+String(v1)+" ?",y=3,u=uv,v=uv) ptrn=mt_lutxy(mtrx,ptrn,"x 8 % 2 == "+String(v2)+" y ?",y=3,u=uv,v=uv) ptrn=mt_lutxy(mtrx,ptrn,"x 8 % 3 == "+String(v3)+" y ?",y=3,u=uv,v=uv) ptrn=mt_lutxy(mtrx,ptrn,"x 8 % 4 == "+String(v4)+" y ?",y=3,u=uv,v=uv) ptrn=mt_lutxy(mtrx,ptrn,"x 8 % 5 == "+String(v5)+" y ?",y=3,u=uv,v=uv) ptrn=mt_lutxy(mtrx,ptrn,"x 8 % 6 == "+String(v6)+" y ?",y=3,u=uv,v=uv) ptrn=mt_lutxy(mtrx,ptrn,"x 8 % 7 == "+String(v7)+" y ?",y=3,u=uv,v=uv) return(ptrn.PtrnFormat(clp)) } function PtrnHorizontal(clip clp, int "uv", int "v0", int "v1", int "v2", int "v3", int "v4", int "v5", int "v6", int "v7") { v1 = default(v1,255-v0) v2 = default(v2,v1) v3 = default(v3,v2) v4 = default(v4,v3) v5 = default(v5,v2) v6 = default(v6,v1) v7 = default(v7,v0) mtrx=Matrix(clp,uv) ptrn=mt_lutxy(mtrx,mtrx,"x 4 - 8 / 8 % 0 == "+String(v0)+" "+String(v1)+" ?",y=3,u=uv,v=uv) ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 8 % 2 == "+String(v2)+" y ?",y=3,u=uv,v=uv) ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 8 % 3 == "+String(v3)+" y ?",y=3,u=uv,v=uv) ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 8 % 4 == "+String(v4)+" y ?",y=3,u=uv,v=uv) ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 8 % 5 == "+String(v5)+" y ?",y=3,u=uv,v=uv) ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 8 % 6 == "+String(v6)+" y ?",y=3,u=uv,v=uv) ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 8 % 7 == "+String(v7)+" y ?",y=3,u=uv,v=uv) return(ptrn.PtrnFormat(clp)) } function PtrnBlock(clip clp, int "uv", int "v0", int "v1", int "v2", int "v3") { v1 = default(v1,255-v0) v2 = default(v2,v1) v3 = default(v3,v2) mtrx=Matrix(clp,uv) ptrn=mt_lutxy(mtrx,mtrx,"x 4 - 8 / 8 % 1 >= x 4 - 8 / 8 % 6 <= & x 8 % 1 >= x 8 % 6 <= & & "+String(v1)+" "+String(v0)+" ?",y=3,u=uv,v=uv) ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 8 % 2 >= x 4 - 8 / 8 % 5 <= & x 8 % 2 >= x 8 % 5 <= & & "+String(v2)+" y ?",y=3,u=uv,v=uv) ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 8 % 3 >= x 4 - 8 / 8 % 4 <= & x 8 % 3 >= x 8 % 4 <= & & "+String(v3)+" y ?",y=3,u=uv,v=uv) return(ptrn.PtrnFormat(clp)) } function PtrnChess(clip clp, int "uv", int "v0", int "v1", int "v2", int "v3") { v1 = default(v1,255-v0) v2 = default(v2,v0) v3 = default(v3,v1) mtrx=Matrix(clp,uv) ptrn=mt_lutxy(mtrx,mtrx,"x 4 - 8 / 2 % 0 == x 2 % 0 == & "+String(v0)+" "+String(v1)+" ?",y=3,u=uv,v=uv) ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 2 % 1 == x 2 % 1 == & "+String(v2)+" y ?",y=3,u=uv,v=uv) ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 2 % 1 == x 2 % 0 == & "+String(v3)+" y ?",y=3,u=uv,v=uv) return(ptrn.PtrnFormat(clp)) } function PtrnSawH(clip clp, int "uv", string "shift", int "v0", int "v1", int "v2", int "v3", int "v4", int "v5", int "v6", int "v7") { v1 = default(v1,255-v0) v2 = default(v2,v1) v3 = default(v3,v2) v4 = default(v4,v3) v5 = default(v5,255-v4) v6 = default(v6,v5) v7 = default(v7,v6) mtrx=Matrix(clp,uv) ptrn=mt_lutxy(mtrx,mtrx,"x 4 - 8 / 8 % 0 == x 2 % "+shift+" 2 % == & x 4 + 8 / 8 % 0 == x 1 + 2 % "+shift+" 2 % == & | "+String(v0)+" "+String(v1)+" ?",y=3,u=uv,v=uv) ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 8 % 2 == x 2 % "+shift+" 2 % == & x 4 + 8 / 8 % 2 == x 1 + 2 % "+shift+" 2 % == & | "+String(v2)+" y ?",y=3,u=uv,v=uv) ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 8 % 3 == x 2 % "+shift+" 2 % == & x 4 + 8 / 8 % 3 == x 1 + 2 % "+shift+" 2 % == & | "+String(v3)+" y ?",y=3,u=uv,v=uv) ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 8 % 4 == x 2 % "+shift+" 2 % == & x 4 + 8 / 8 % 4 == x 1 + 2 % "+shift+" 2 % == & | "+String(v4)+" y ?",y=3,u=uv,v=uv) ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 8 % 5 == x 2 % "+shift+" 2 % == & x 4 + 8 / 8 % 5 == x 1 + 2 % "+shift+" 2 % == & | "+String(v5)+" y ?",y=3,u=uv,v=uv) ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 8 % 6 == x 2 % "+shift+" 2 % == & x 4 + 8 / 8 % 6 == x 1 + 2 % "+shift+" 2 % == & | "+String(v6)+" y ?",y=3,u=uv,v=uv) ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 8 % 7 == x 2 % "+shift+" 2 % == & x 4 + 8 / 8 % 7 == x 1 + 2 % "+shift+" 2 % == & | "+String(v7)+" y ?",y=3,u=uv,v=uv) return(ptrn.PtrnFormat(clp)) } function PtrnSawV(clip clp, int "uv", string "shift", int "v0", int "v1", int "v2", int "v3", int "v4", int "v5", int "v6", int "v7") { v1 = default(v1,255-v0) v2 = default(v2,v1) v3 = default(v3,v2) v4 = default(v4,v3) v5 = default(v5,255-v4) v6 = default(v6,v5) v7 = default(v7,v6) mtrx=Matrix(clp,uv) ptrn=mt_lutxy(mtrx,mtrx,"x 4 - 8 / 2 % "+shift+" 2 % == x 8 % 0 == & x 4 + 8 / 2 % "+shift+" 2 % == x 1 + 8 % 0 == & | "+String(v0)+" "+String(v1)+" ?",y=3,u=uv,v=uv) ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 2 % "+shift+" 2 % == x 8 % 2 == & x 4 + 8 / 2 % "+shift+" 2 % == x 1 + 8 % 2 == & | "+String(v2)+" y ?",y=3,u=uv,v=uv) ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 2 % "+shift+" 2 % == x 8 % 3 == & x 4 + 8 / 2 % "+shift+" 2 % == x 1 + 8 % 3 == & | "+String(v3)+" y ?",y=3,u=uv,v=uv) ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 2 % "+shift+" 2 % == x 8 % 4 == & x 4 + 8 / 2 % "+shift+" 2 % == x 1 + 8 % 4 == & | "+String(v4)+" y ?",y=3,u=uv,v=uv) ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 2 % "+shift+" 2 % == x 8 % 5 == & x 4 + 8 / 2 % "+shift+" 2 % == x 1 + 8 % 5 == & | "+String(v5)+" y ?",y=3,u=uv,v=uv) ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 2 % "+shift+" 2 % == x 8 % 6 == & x 4 + 8 / 2 % "+shift+" 2 % == x 1 + 8 % 6 == & | "+String(v6)+" y ?",y=3,u=uv,v=uv) ptrn=mt_lutxy(mtrx,ptrn,"x 4 - 8 / 2 % "+shift+" 2 % == x 8 % 7 == & x 4 + 8 / 2 % "+shift+" 2 % == x 1 + 8 % 7 == & | "+String(v7)+" y ?",y=3,u=uv,v=uv) return(ptrn.PtrnFormat(clp)) } Code:
function AverageBlock8(clip orig) { o1=orig.mt_convolution("0 0 0 0 0 0 0 1 1 1 1 1 1 1 1","0 0 0 0 0 0 0 1 1 1 1 1 1 1 1",y=3,u=3,v=3) o1=o1.pointresize(o1.width/8,o1.height/8) o1=o1.pointresize(o1.width*8,o1.height*8) return(o1) } function VerticalLinearBlock(clip orig) { p1=PtrnHorizontal(orig,3,255,219,182,146,109,73,36,0) top1=orig.PointResize(orig.width,orig.height/8) top2=top1.PointResize(orig.width,orig.height) shift1=orig.mt_convolution("1","0 0 0 0 0 0 0 0 0 0 0 0 0 0 1",y=3,u=3,v=3) bottom1=shift1.PointResize(orig.width,orig.height/8) bottom2=bottom1.PointResize(orig.width,orig.height) mt_merge(bottom2,top2,p1,y=3,u=3,v=3) } function HorizontalLinearBlock(clip orig) { p1=PtrnVertical(orig,3,255,219,182,146,109,73,36,0) top1=orig.PointResize(orig.width/8,orig.height) top2=top1.PointResize(orig.width,orig.height) shift1=orig.mt_convolution("0 0 0 0 0 0 0 0 0 0 0 0 0 0 1","1",y=3,u=3,v=3) bottom1=shift1.PointResize(orig.width/8,orig.height) bottom2=bottom1.PointResize(orig.width,orig.height) mt_merge(bottom2,top2,p1,y=3,u=3,v=3) } + Average which are http://mywebpages.comcast.net/trbarry/DctFilter.zip http://people.pwf.cam.ac.uk/mg262/po...ge_21Oct05.dll from: http://mywebpages.comcast.net/trbarry/downloads.htm http://forum.doom9.org/showthread.ph...hlight=average Last edited by redfordxx; 5th June 2006 at 20:05. |
23rd May 2006, 20:28 | #3 | Link |
Registered User
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 863
|
Usage examples
#deblocks on defaults, ...
source.SmoothDeblock() #deblocks on density 4, ... source.SmoothDeblock(density=4,wieghtMX="16center255border0smooth_1",gain=10,pregain=10, pregainoffset=0) #deblocks on quant 5 density 2, ... source.SmoothDeblockLayer(5,2,orig.AddBorders(16,16,0,0).WeightMatrixClip("04center255border0smooth_1"),WeightMatrixSum("04center255border0smooth_1")) #shows which detail would be destroyed by DCTFilter on quant 4 qd4=source.QuantDiff(4) # makes block oriented mask based on clip from previous function qd4.AdaptionMask(10,0,10,"1","1 1 1 1 1") # the higher following values the mor detail and blocking is kept pregainoffset= default(pregainoffset,-1) pregain= default(pregain,10) gain= default(gain,5) |
26th May 2006, 13:34 | #7 | Link |
ангел смерти
Join Date: Nov 2004
Location: Lost
Posts: 9,556
|
That will mess it up though, it makes some blocks now 1x2 instead of 1x1. (PointResize is picky that way.) You should pad borders to make it mod32 (so the output is mod4, doesn't need to be mod8). Then crop after resizing back up.
|
26th May 2006, 17:48 | #8 | Link |
Registered User
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 863
|
more about WeightMatrixClip()
This function produces the weighting mask for the merging. Parameter clp is there to have same fps,size...
The masks can be named... my convention is that matrix with name 04* needs 4 shifted layers to average and obtain correct image, 16* needs 16 shifted layers to average and obtain correct image. One of advantages of the 16* matrix (for density=4) is that you can pick any numbers there and as long as it is symetric, it will work. Example --- creation of matrix 04center255border0smooth_1: For creating the matrix I use WeightMatrixClip function with heavy support of Patterns.avsi Code:
function WeightMatrixClip{ #getting 8x8 matrix with pixel values 0...63 formated to size of original clip mx1=clp.Matrix(3).PtrnFormat(clp) #creating base matrix with color 0 at the border and 255 in the middle etc. (3 is for chroma processing as in mt) mx=clp.PtrnBlock(3,0,50,200,255) #each block "is made of 4 squares inside each other". Let's number them 0...3. (3 is in the middle) #middle1 (correction of the side middle pixels on square no. 1 and setting on color 55. on mx1 the pixels are numbered 11,12,25,30...) mx = mt_lutxy(mx,mx1,"y 12 == y 11 == y 25 == y 30 == y 33 == y 38 == y 51 == y 52 == | | | | | | | 55" x ?",y=3,u=3,v=3) #diag1 (setting the on diagonal pixels on square no. 1 on color 25. on mx1 the pixels are numbered 9,14,49,54) mx = mt_lutxy(mx,mx1,"y 9 == y 14 == y 54 == y 49 == | | | 25 x ?",y=3,u=3,v=3) #diag2 (setting the on diagonal pixels on square no. 2 on color 130. on mx1 the pixels are numbered 18,21,42,45) mx = mt_lutxy(mx,mx1,"y 18 == y 21 == y 42 == y 45 == | | | 130 x ?",y=3,u=3,v=3) mx } Code:
0 0 0 0 0 0 0 0 0 25 50 55 55 50 25 0 0 50 130 200 200 130 50 0 0 55 200 255 255 200 55 0 0 55 200 255 255 200 55 0 0 50 130 200 200 130 50 0 0 25 50 55 55 50 25 0 0 0 0 0 0 0 0 0 It is necessary to write down the average or nominal value of the matrix. This time it is 255.0/4 (because we added 4 shifted and the result was 255) Later in the script it is used for normalizing the merged clips.The value should be stored in WeightMatrixSum function. My observation is that when the matrix is steep (hi values in the middle and low at hte border) it produces sharper image but also some artefacts in hi detail areas on hi quants (can look like dots). Flat matrices do not make dots, but make edges . Last edited by redfordxx; 27th May 2006 at 03:59. |
27th May 2006, 03:54 | #9 | Link |
Registered User
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 863
|
I think I am going to replace the first SmoothDeblockLayer(quant=6 or so...) with quant=3? on half resolution.
Can anybody recommend me some nice downsize and upsize to achieve nice, smooth image after upsizing. |
28th May 2006, 13:40 | #11 | Link |
Registered User
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 863
|
New version
- it's mod 16 (but prefers 32)
- a little speedup - added SmoothDeblock2 which has different layers masking, although, I am not sure which can be better... Last edited by redfordxx; 28th May 2006 at 13:49. |
28th May 2006, 13:47 | #12 | Link |
Registered User
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 863
|
I must say I like the results of deblocking of the SmoothDeblockLayer function provided it is on correct quant. It restores the detail nicely on places I never thought it is possible.
ITOH I am not satisfied with the layers merging mask. There are always problems al least in the areas, where blocking neighbours detail. Does anybody have an idea, how to implement the adaptive switching among the layers? --- How to create the masks? |
30th May 2006, 00:49 | #13 | Link |
Registered User
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 863
|
I made some samples of deblocking.
I found six different areas throughout the clip, deblocked, cropped and stacked into one picture. There are to compare: oroginal Deblock default Deblock stronger DGDecode deblocker default DGDecode deblocker stronger SmoothDeblock2(4) IMO: SmoothDeblock2 - the deblocking looks very nice and it's most succesful in deblocking. Detail retention is quite good, as long as the detail contrast is over certain level --- otherwise it just disappears (see the lines in the sixth box or drops or thin hair on third box). I hope some tweaking will help. Also I think the total image look is the most similar to original (when fast switching the picture with original, the blocking just disappears and there is no color shift) Deblock defaults - good detail retention but weak Deblock stronger - almost as strong as SmoothDeblock and retains more fine detail (means a detail with low contrast), but again (I repeat myself) the hor/vert grid makes the setting higher than default unusable. If there is good detail retention, grid destroys it. You can see it on diagonal lines... DGDecode (not BlindPP but directly the adaptive one) - a little more succesful in deblocking, than Deblock. The strong setting is already smearing too much. Now I see, that the adaptivity of DGDecode deblocker suffers the similar problem I complained few posts ago. When meeting hi quant and lo quant blocks, the blocking remains. (btw, what happened to the fourth box in the strong version? it is quite different from orig) Conclusion, SmoothDeblock needs * more detail retention, * better adaptivity allocation * probably some protection of "what happens when there is a real edge directly on blockborder" Then I think it can be OK;-) I don't give up yet... [EDIT] Newer, more succesful samples later in this thread Last edited by redfordxx; 5th June 2006 at 16:43. |
30th May 2006, 01:01 | #14 | Link |
Registered User
Join Date: Oct 2003
Location: Germany
Posts: 377
|
Thanks redfordxx. I use your function for an anime. I get the best results with it. I also this "SmoothDeblock2" give the best result (i use it for my encode ).
I used your function in combination with one of mine, which i made on the fly. It recover some Detail. Still i encode an anime. But maybe it is usefull... so here is it: Code:
CLmedian(last,SmoothDeblock2(),2,1,1) function ClMedian(clip c, clip n, int rad, int th1, int th2){ s=gaussianblur(c,rad,rad) str1="x y - abs "+string(th1)+" > 255 0 ?" m=YV12LutXY(c,s,str1,str1,str1,Y=3,U=3,V=3) str2="x y - abs "+string(th2)+" > 255 0 ?" r=YV12LutXY(c,n,str2,str2,str2,Y=3,V=3,U=3) r=MaskedMerge(c,n,r,Y=3,V=3,U=3) MaskedMerge(c,r,m,Y=3,U=3,V=3) } Thanks again for your energy. |
2nd June 2006, 08:11 | #15 | Link | ||
Professional Lemming
Join Date: Dec 2003
Location: Stuttgart, Germany
Posts: 359
|
Quote:
I experimented with DCTfilter as well a while ago and came to the conclusion that it overfilters crisp details. The alternative I suggest is a filter (DQI.dll) which sits on my HD for almost a year in a preprealpha state A realtime version is possible but my first (all C) implementation is quite slow (and still buggy as hell) But I like your script and I guess it's just a simple substitute of DCTfilter to DQI. Quote:
I fear it will smooth noisy flat regions too much... any test results regarding this? To sum it up, my plan for a realtime version of this is: o filter strength detection o just use 4 shifts o use DQI with MMX/SSE2 o use a weighting mask And here is the acronym CAPS - Content adaptive detail preserving smoother bis besser, T0B1A5
__________________
projects page: ELDER, SmoothD, etc. Last edited by 708145; 2nd June 2006 at 08:15. |
||
2nd June 2006, 09:05 | #16 | Link |
Registered User
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 863
|
SmoothDeblock script part one
Code:
############################################### # SmoothDeblock 0.3 by Redford # # # issues and todo's: # find best averaging weight matrices for combining DCT shifts (=minimize artefacts at high quants and increase sharpness) # find best smoothing of the adaptation mask clip or better, adaptationmask shouldnot be block oriented, but only detail oriented # # tuning and speed up # ############################################### function ShiftDCTBack(clip orig,int density, clip wmc, float wms, string dct_type, int i, int j, int DCTorig, int "quant") { l=orig.crop(0,0,Int(8/density)*i+16,-0).FlipHorizontal() r=orig.crop(orig.Width-Int(8/density)*i-16,0,-0,-0).FlipHorizontal() cMain=(i==0) ? orig : StackHorizontal(l,orig,r) t=cMain.crop(0,0,0,Int(8/density)*j+16).FlipVertical() b=cMain.crop(0,orig.height-Int(8/density)*j-16,0,-0).FlipVertical() cMain=(j==0) ? cMain : StackVertical(t,cMain,b) cMain=(i+j==0) ? orig : MergeChroma(cMain,cMain.AddBorders(Int(8/density)*i,Int(8/density)*j,0,0).Crop(0,0,cMain.Width,cMain.Height)) cMain=(i==0) ? cMain : cMain.crop(16,0,orig.Width+16,0) cMain=(j==0) ? cMain : cMain.crop(0,16,0,orig.Height+16) cMain=((i+j==0)&&(DCTorig==0)) ? orig : (dct_type=="DCTFilterD") ? cMain.DCTFilterD(quant) : cMain.DCTFilter(DctParams7,DctParams6,DctParams5,DctParams4,DctParams3,DctParams2,DctParams1,DctParams0) cFlot=mt_lutxy(cMain,wmc,"x y * 255 %",y=3,u=3,v=3) cMain=mt_lutxy(cMain,wmc,"x y * x y * 255 % - 255 /",y=3,u=3,v=3) cFlot=(i+j==0) ? cFlot : MergeChroma(cFlot,cFlot.AddBorders(0,0,Int(8/density)*i,Int(8/density)*j).Crop(Int(8/density)*i,Int(8/density)*j,cFlot.Width,cFlot.Height)) cMain=(i+j==0) ? cMain : MergeChroma(cMain,cMain.AddBorders(0,0,Int(8/density)*i,Int(8/density)*j).Crop(Int(8/density)*i,Int(8/density)*j,cMain.Width,cMain.Height)) shiftClip.Set(i*density+j+Int(Pow(density,2)),(i+j==0) ? cFlot : cFlot.crop(Int(8/density)*i,Int(8/density)*j,orig.Width(),orig.height)) shiftClip.Set(i*density+j, (i+j==0) ? cMain : cMain.crop(Int(8/density)*i,Int(8/density)*j,orig.Width(),orig.height)) global avgI = avgI+", shiftClip"+String(i*density+j)+", 255.0/"+String(wms) global avgF = (i+j==0) ? avgF : avgF+", shiftClip"+String(i*density+j+Int(Pow(density,2)))+", "+String(1/Pow(density,2)) } function SmoothDeblockLayer(clip orig,float quant,int density, clip wmc, float wms, string "dct_type",int "DCTorig") { global gDCTtype=default(dct_type,"DCTFilter") #fixed now global gDensity=density #number of DCT per 8 pix global gOrig=orig global rep=wms*Pow(density,2) global q=Int(quant) global matrix1=wmc global avgI="clipAverageFloat, "+String(1/wms) global avgF="shiftClip"+String(Int(Pow(density,2)))+", "+String(1/Pow(density,2)) global DCTO=default(DCTorig,0) Dim("DctParams",8,0) Dim("shiftClip",Int(Pow(density,2))*2,Blackness()) For2("i=7","i>="+String(quant),"i=i-1", \""" DctParams.Set(i,1) """) For2("i=0", "i<gDensity", "i=i+1", \""" For2("j=0", "j<gDensity", "j=j+1", \" ShiftDCTBack(gOrig,gDensity,matrix1,rep,gDCTtype,i,j,DCTO,q) ") """) Eval("clipAverageFloat=Average("+avgF+")") Eval("Average("+avgI+")") #Subtitle(avgI) } function QuantDiff(clip orig,float "quant",string "dct_type") { dct_type = default(dct_type,"DCTFilter") #fixed now quant =default(quant,4) Dim("DctParams",8,0) For2("i=7","i>="+String(quant),"i=i-1", \""" DctParams.Set(i,1) """) #a=AverageBlock8(orig) a=orig.DCTFilter(1,0,0,0,0,0,0,0) n=mt_makediff(orig,a,y=3,u=3,v=3) d=n.DCTFilter(1,1-DctParams6,1-DctParams5,1-DctParams4,1-DctParams3,1-DctParams2,1-DctParams1,1-DctParams0) d } function AdaptionMask(clip orig,int "pregain",int "pregainoffset",int "gain",string "adaption_blursmall",string "adaption_blurbig") { pregainoffset =default(pregainoffset,-2) pregain =default(pregain,100) gain =default(gain,100) adaption_blursmall =default(adaption_blursmall,"1") #adaption_blurbig =default(adaption_blurbig,"1 1 2 2 2 1 1") adaption_blurbig =default(adaption_blurbig,"1") o1=orig.mt_lut("x 128 - abs x 128 == 0 "+string(pregainoffset)+" ? + "+string(pregain)+" *",y=3,u=3,v=3) o1=o1.mt_convolution("0 0 0 0 0 0 0 1 1 1 1 1 1 1 1","0 0 0 0 0 0 0 1 1 1 1 1 1 1 1",y=3,u=3,v=3).AddBorders(0,0,(32 - (orig.width % 32)) % 32,(32 - (orig.height % 32)) % 32) o1=o1.pointresize(o1.width/8,o1.height/8) o1=(adaption_blursmall=="1") ? o1 : \ (adaption_blursmall=="expand") ? o1.mt_expand(y=3,u=3,v=3) : \ (adaption_blursmall=="inpand") ? o1.mt_inpand(y=3,u=3,v=3) : \ (adaption_blursmall=="inflate") ? o1.mt_inflate(y=3,u=3,v=3) : \ (adaption_blursmall=="deflate") ? o1.mt_deflate(y=3,u=3,v=3) : \ o1.mt_convolution(adaption_blursmall,adaption_blursmall,y=3,u=3,v=3) o1=o1.pointresize(o1.width*8,o1.height*8).Crop(0,0,orig.width,orig.height) o1=(adaption_blurbig=="1") ? o1 : o1.mt_convolution(adaption_blurbig,adaption_blurbig,y=3,u=3,v=3) o1=o1.mt_lut("x "+string(gain)+" *",y=3,u=3,v=3) return(o1) } function AdaptionMaskDiff(clip orig, clip deb,float "quant", int "pregain", int "pregainoffset", int "gain", int "thresh", string "adaption_blur", int "borders", int "expands" ,string "dct_type") { pregainoffset =default(pregainoffset,-3) pregain =default(pregain,100) borders =default(borders,0) expands =default(expands,1) gain =default(gain,8) thresh =default(thresh,2) adaption_blur =default(adaption_blur,"1 2 1") dct_type =default(dct_type,"DCTFilter") #fixed now quant =default(quant,4) Dim("DctParams",8,0) For2("i=7","i>="+String(quant),"i=i-1", \""" DctParams.Set(i,1) """) #a=AverageBlock8(orig) a=orig.DCTFilter(1,0,0,0,0,0,0,0) n=mt_makediff(orig,a,y=3,u=3,v=3) d=n.DCTFilter(1,1-DctParams6,1-DctParams5,1-DctParams4,1-DctParams3,1-DctParams2,1-DctParams1,1-DctParams0) o1=d.mt_lut("x 128 - abs x 128 == 0 "+string(pregainoffset)+" ? + "+string(pregain)+" *",y=3,u=3,v=3) o1=o1.DCTFilter(1,0,0,0,0,0,0,0) o1=o1.mt_lut("x "+string(pregain/5)+" *",y=3,u=3,v=3) diff=mt_makediff(deb,orig,y=3,u=3,v=3) diff2=mt_lutxy(o1,diff,"y 128 - x * 255 / abs "+string(thresh)+" -",y=3,u=3,v=3) p1=diff2.PtrnBlock(3,borders,255,255,255) diff3=mt_lutxy(diff2,p1,"x y * 255 / "+string(gain)+" *",y=3,u=3,v=3) diff4=diff3.mt_expand(y=3,u=3,v=3) (adaption_blur=="1") ? diff4 : diff4.mt_convolution(adaption_blur,adaption_blur,y=3,u=3,v=3) mt_lut("x "+string(gain)+" *",y=3,u=3,v=3) (expands>0) ? mt_expand(y=3,u=3,v=3) : last (expands>1) ? mt_expand(y=3,u=3,v=3) : last (expands>2) ? mt_expand(y=3,u=3,v=3) : last (expands>3) ? mt_expand(y=3,u=3,v=3) : last (expands>4) ? mt_expand(y=3,u=3,v=3) : last } function WeightMatrixClip(clip orig, string "name",int "level") { level=default(level,0) clp=orig.Crop(0,0,16,16).Trim(1,1) name=default(name,"04center255x2") mx1 = (name=="04all78")||(name=="04center255x2")||(name=="16center255") ? clp : clp.Matrix(3).PtrnFormat(clp) mx = (name=="04center255border0smooth_1") ? clp.PtrnBlock(3,0,50,200,255) : \ (name=="16center255border0smooth_1") ? clp.PtrnBlock(3,0,70,185,255) : \ (name=="16center255border0smooth_2") ? clp.PtrnBlock(3,100,150,220,255) : clp #middle1 mx = (name=="04all78")||(name=="04center255x2")||(name=="16center255") ? mx : mt_lutxy(mx,mx1,"y 12 == y 11 == y 25 == y 30 == y 33 == y 38 == y 51 == y 52 == | | | | | | | " + ( \ (name=="04center255border0smooth_1") ? "55" : \ (name=="16center255border0smooth_1") ? "115" : \ (name=="16center255border0smooth_2") ? "180" : \ "x") + " x ?",y=3,u=3,v=3) #diag0 #diag1 mx = (name=="04all78")||(name=="04center255x2")||(name=="16center255") ? mx : mt_lutxy(mx,mx1,"y 9 == y 14 == y 54 == y 49 == | | | " + ( \ (name=="04center255border0smooth_1") ? "25" : \ (name=="16center255border0smooth_1") ? "20" : \ (name=="16center255border0smooth_2") ? "130" : \ "x") + " x ?",y=3,u=3,v=3) #diag2 mx = (name=="04all78")||(name=="04center255x2")||(name=="16center255") ? mx : mt_lutxy(mx,mx1,"y 18 == y 21 == y 42 == y 45 == | | | " + ( \ (name=="04center255border0smooth_1") ? "130" : \ (name=="16center255border0smooth_1") ? "140" : \ (name=="16center255border0smooth_2") ? "200" : \ "x") + " x ?",y=3,u=3,v=3) (name=="16center255") ? (clp.PtrnBlock(3,0,0,0,255)) : \ (name=="04center255x2") ? (clp.PtrnBlock(3,0,0,255,255)): \ (name=="04all78") ? (clp.PtrnBlock(3,150,150,212,212)): \ mx #mt_lut(String(level)+" 255 "+String(level)+" - 255 / x * +",y=3,u=3,v=3) PtrnFormat(orig) } function WeightMatrixSum(string "name",int "level") { level=default(level,0) l=(name=="16center255") ? (255.0/16) : \ (name=="04center255x2") ? (255.0/4): \ (name=="04center255border0smooth_1") ? (255.0/4): \ (name=="16center255border0smooth_1") ? (1155.0/16): \ (name=="16center255border0smooth_2") ? (2385.0/16): \ (name=="04all78") ? (662.0/4): \ (255.0/4) #return((255.0-level)/255.0*l+level) l } function BorderMask(clip orig, int "bth1", int "bth2", int "bshare", int "eth1", int "eth2", int "eshare", float "buvmod", float "euvmod") { bth1=default(bth1,0) bth2=default(bth2,2) bshare=default(bshare,200) eth1=default(eth1,30) eth2=default(eth2,50) eshare=default(eshare,0) #0=real edge protection off buvmod=default(buvmod,2.0) euvmod=default(euvmod,2.4) ph=orig.PtrnHorizontal(3,0,1,2,3,4,5,6,7) pv=orig.PtrnVertical(3,0,1,2,3,4,5,6,7) h1a=orig.mt_lutxy(ph,"y 0 == y 7 == | x 128 ?",y=3,u=3,v=3) v1a=orig.mt_lutxy(pv,"y 0 == y 7 == | x 128 ?",y=3,u=3,v=3) h1b=orig.mt_lutxy(ph,"y 1 == y 6 == | x 128 ?",y=3,u=3,v=3) v1b=orig.mt_lutxy(pv,"y 1 == y 6 == | x 128 ?",y=3,u=3,v=3) h2a=h1a.mt_convolution(horizontal="1",vertical="1 -1 1",total=1,y=3,u=3,v=3) v2a=v1a.mt_convolution(horizontal="1 -1 1",vertical="1",total=1,y=3,u=3,v=3) h2b=h1b.mt_convolution(horizontal="1",vertical="1 -1 1 -1 1",total=1,y=3,u=3,v=3) v2b=v1b.mt_convolution(horizontal="1 -1 1 -1 1",vertical="1",total=1,y=3,u=3,v=3) v2=mt_lutxy(v2a,v2b,"x 128 - y 128 - 3 / - abs x 128 - abs > x x y 128 - 3 / - ?",y=3,u=3,v=3) h2=mt_lutxy(h2a,h2b,"x 128 - y 128 - 3 / - abs x 128 - abs > x x y 128 - 3 / - ?",y=3,u=3,v=3) v3b=v2.mt_lutxy(pv,"y 0 == y 7 == | x 128 ? 128 - abs 5 * 1.1 * ",y=3,u=3,v=3).mt_convolution(horizontal="1",vertical="1 1 1",y=3,u=3,v=3) h3b=h2.mt_lutxy(ph,"y 0 == y 7 == | x 128 ? 128 - abs 5 * 1.1 * ",y=3,u=3,v=3).mt_convolution(horizontal="1 1 1",vertical="1",y=3,u=3,v=3) v3e0=v2a.mt_convolution(horizontal="1",vertical="1 1 1 1 1 1 1 1 1",y=3,u=2,v=2).mt_convolution(horizontal="1",vertical="1 1 1 1 1 1 1",y=2,u=3,v=3) h3e0=h2a.mt_convolution(horizontal="1 1 1 1 1 1 1 1 1",vertical="1",y=3,u=2,v=2).mt_convolution(horizontal="1 1 1 1 1 1 1",vertical="1",y=2,u=3,v=3) v3e=(eshare==0) ? orig : v3e0.mt_lutxy(pv,yexpr="y 0 == y 7 == | x 128 ? 128 - abs "+String(eth1)+" - "+String(eth2)+" "+String(eth1)+" - / 255 *", \ uexpr="y 0 == y 7 == | x 128 ? 128 - abs "+String(eth1/euvmod)+" - "+String(eth2/euvmod)+" "+String(eth1/euvmod)+" - / 255 *", \ vexpr="y 0 == y 7 == | x 128 ? 128 - abs "+String(eth1/euvmod)+" - "+String(eth2/euvmod)+" "+String(eth1/euvmod)+" - / 255 *",y=3,u=3,v=3) h3e=(eshare==0) ? orig : h3e0.mt_lutxy(ph,yexpr="y 0 == y 7 == | x 128 ? 128 - abs "+String(eth1)+" - "+String(eth2)+" "+String(eth1)+" - / 255 *", \ uexpr="y 0 == y 7 == | x 128 ? 128 - abs "+String(eth1/euvmod)+" - "+String(eth2/euvmod)+" "+String(eth1/euvmod)+" - / 255 *", \ vexpr="y 0 == y 7 == | x 128 ? 128 - abs "+String(eth1/euvmod)+" - "+String(eth2/euvmod)+" "+String(eth1/euvmod)+" - / 255 *",y=3,u=3,v=3) v4e=(eshare==0) ? orig : v3e.HorizontalLinearBlock h4e=(eshare==0) ? orig : h3e.VerticalLinearBlock e1=(eshare==0) ? orig : mt_lutxy(v4e,h4e,"x y > x y ?",y=3,u=3,v=3) e2=(eshare==0) ? orig : e1.mt_lut("255 x - "+String(eshare)+" * 255 / "+String(eshare)+" - 255 +",y=3,u=3,v=3) v4b=(eshare==0) ? v3b : v3b.mt_lutxy(v3e,"x y 1 + /",y=3,u=3,v=3) h4b=(eshare==0) ? h3b : h3b.mt_lutxy(h3e,"x y 1 + /",y=3,u=3,v=3) v5b=v4b.HorizontalLinearBlock h5b=h4b.VerticalLinearBlock b1=mt_lutxy(v5b,h5b,yexpr="x y > x y ? 5 / "+String(bth1)+" - "+String(bth2)+" "+String(bth1)+" - / 255 *", \ uexpr="x y > x y ? 5 / "+String(bth1/buvmod)+" - "+String(bth2/buvmod)+" "+String(bth1/buvmod)+" - / 255 *", \ vexpr="x y > x y ? 5 / "+String(bth1/buvmod)+" - "+String(bth2/buvmod)+" "+String(bth1/buvmod)+" - / 255 *",y=3,u=3,v=3) b2=b1.mt_convolution(horizontal="1 1 1 1 1",vertical="1 1 1 1 1",y=3,u=3,v=3) b3=b2.mt_lut("x "+String(bshare)+" * 255 / "+String(bshare)+" - 255 +",y=3,u=3,v=3) (eshare==0) ? b3 : mt_lutxy(e2,b3,"x y < x y ?",y=3,u=3,v=3) } Last edited by redfordxx; 3rd June 2006 at 11:55. |
2nd June 2006, 09:07 | #17 | Link |
Registered User
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 863
|
SmooothDeblock part two
Code:
function SmoothDeblock1(clip orig, int "passes", int "density", int "inclOrig", string "wieghtMX",string "dct_type",int "pregain",int "pregainoffset",int "gain",string "adaption_blursmall",string "adaption_blurbig") { passes=3#fixed for now inclOrig=default(inclOrig,1) density=default(density,2) pregainoffset=default(pregainoffset,0) pregain=default(pregain,10) gain=default(gain,10) b1=default(adaption_blursmall,"1") b2=default(adaption_blurbig,"1 1 2 2 2 1 1") wieghtMX=default(wieghtMX,"04center255border0smooth_1") dct_type=default(dct_type,"DCTFilter") mx2=orig.AddBorders(16,16,0,0).WeightMatrixClip(wieghtMX) mxs=WeightMatrixSum(wieghtMX) sdl8=orig.LanczosResize(orig.width/2,orig.height/2) sdl8=((sdl8.Width % 32)==0) ? sdl8 : StackHorizontal(sdl8,sdl8.crop(sdl8.Width-32+(sdl8.Width % 32),0,-0,-0).FlipHorizontal()) sdl8=((sdl8.height % 32)==0) ? sdl8 : StackVertical(sdl8,sdl8.crop(0,sdl8.height-32+(sdl8.height % 32),-0,-0).FlipVertical()) sdl8=sdl8.SmoothDeblockLayer(5,density,mx2,mxs,"DCTFilter",DCTorig=1) sdl8=sdl8.GaussResize(sdl8.width*2,sdl8.height*2,p=20).crop(0,0,orig.width,orig.height) #sdl8=sdl8.mt_convolution("1 2 1","1 2 1",y=3,u=3,v=3) sdl5=orig.SmoothDeblockLayer(5,density,mx2,mxs,"DCTFilter") sdl3=orig.SmoothDeblockLayer(3,density,mx2,mxs,"DCTFilter") qd8=orig.QuantDiff(6) qd5=orig.QuantDiff(5) qdo=orig.QuantDiff(inclOrig) am8=qd8.AdaptionMask(pregain,pregainoffset,gain,b1,b2) am5=qd5.AdaptionMask(pregain,pregainoffset,gain,b1,b2) amo=qdo.AdaptionMask(pregain,pregainoffset,gain,b1,b2) deblocked=mt_merge(sdl8,sdl5,am8,y=3,u=3,v=3) deblocked=mt_merge(deblocked,sdl3,am5,y=3,u=3,v=3) deblocked=(inclOrig==0) ? deblocked : mt_merge(deblocked,orig,amo,y=3,u=3,v=3) return(deblocked) } function SmoothDeblock2(clip orig, int "passes", int "density", int "inclOrig", string "wieghtMX",string "dct_type",int "pregain",int "pregainoffset",int "gain", int "thresh", string "adaption_blur") { passes=3#fixed for now inclOrig=default(inclOrig,2) density=default(density,2) #pregainoffset=default(pregainoffset,-2) #pregain=default(pregain,100) #gain=default(gain,30) #thresh=default(thresh,1) #b2=default(adaption_blurbig,"1 1 1") wieghtMX=default(wieghtMX,(density==4) ? "16center255border0smooth_1" : "04center255border0smooth_1") dct_type=default(dct_type,"DCTFilter") mx2=orig.AddBorders(16,16,0,0).WeightMatrixClip(wieghtMX) mxs=WeightMatrixSum(wieghtMX) sdl8=orig.LanczosResize(orig.width/2,orig.height/2) sdl8=((sdl8.Width % 32)==0) ? sdl8 : StackHorizontal(sdl8,sdl8.crop(sdl8.Width-32+(sdl8.Width % 32),0,-0,-0).FlipHorizontal()) sdl8=((sdl8.height % 32)==0) ? sdl8 : StackVertical(sdl8,sdl8.crop(0,sdl8.height-32+(sdl8.height % 32),-0,-0).FlipVertical()) sdl8=sdl8.SmoothDeblockLayer(5,density,mx2,mxs,"DCTFilter",DCTorig=1) sdl8=sdl8.GaussResize(sdl8.width*2,sdl8.height*2,p=20).crop(0,0,orig.width,orig.height) #sdl8=sdl8.mt_convolution("1 2 1","1 2 1",y=3,u=3,v=3) sdl5=orig.SmoothDeblockLayer(5,density,mx2,mxs,"DCTFilter") sdl3=orig.SmoothDeblockLayer(3,density,mx2,mxs,"DCTFilter") am8=orig.AdaptionMaskDiff(sdl8,6,pregain,pregainoffset,gain,thresh,adaption_blur,expands=5) am5=orig.AdaptionMaskDiff(sdl5,5,pregain,pregainoffset,gain,thresh,adaption_blur,expands=2) amo=orig.AdaptionMaskDiff(sdl3,inclOrig,pregain,pregainoffset,gain,thresh,adaption_blur,expands=1) deblocked=sdl8 deblocked=mt_merge(deblocked,sdl5,am8,y=3,u=3,v=3) deblocked=mt_merge(deblocked,sdl3,am5,y=3,u=3,v=3) deblocked=(inclOrig==0) ? deblocked : mt_merge(deblocked,orig,amo,y=3,u=3,v=3) return(deblocked) } function SmoothDeblock3(clip orig, int "passes", int "density", int "inclOrig", string "wieghtMX",string "dct_type",int "pregain",int "pregainoffset",int "gain", int "thresh", string "adaption_blur", int "bth1", int "bth2", int "bshare", int "eth1", int "eth2", int "eshare", float "buvmod", float "euvmod") { passes=3#fixed for now inclOrig=default(inclOrig,2) density=default(density,2) wieghtMX=default(wieghtMX,(density==4) ? "16center255border0smooth_1" : "04center255border0smooth_1") dct_type=default(dct_type,"DCTFilter") mx2=orig.AddBorders(16,16,0,0).WeightMatrixClip(wieghtMX) mxs=WeightMatrixSum(wieghtMX) bdm=orig.BorderMask(bth1, bth2, bshare, eth1, eth2, eshare) sdl8=orig.LanczosResize(orig.width/2,orig.height/2) sdl8=((sdl8.Width % 32)==0) ? sdl8 : StackHorizontal(sdl8,sdl8.crop(sdl8.Width-32+(sdl8.Width % 32),0,-0,-0).FlipHorizontal()) sdl8=((sdl8.height % 32)==0) ? sdl8 : StackVertical(sdl8,sdl8.crop(0,sdl8.height-32+(sdl8.height % 32),-0,-0).FlipVertical()) sdl8=sdl8.SmoothDeblockLayer(5,density,mx2,mxs,"DCTFilter",DCTorig=1) sdl8=sdl8.GaussResize(sdl8.width*2,sdl8.height*2,p=20).crop(0,0,orig.width,orig.height) #sdl8=sdl8.mt_convolution("1 2 1","1 2 1",y=3,u=3,v=3) sdl5=orig.SmoothDeblockLayer(5,density,mx2,mxs,"DCTFilter") sdl3=orig.SmoothDeblockLayer(3,density,mx2,mxs,"DCTFilter") am8=orig.AdaptionMaskDiff(sdl8,6,pregain,pregainoffset,gain,thresh,adaption_blur,expands=5).mt_lutxy(bdm,"255 255 x - y * 255 / -",y=3,u=3,v=3)# 255-((255-x)*y)/255 = x+255-y-x*(255-y)/255 am5=orig.AdaptionMaskDiff(sdl5,5,pregain,pregainoffset,gain,thresh,adaption_blur,expands=2).mt_lutxy(bdm,"255 255 x - y * 255 / -",y=3,u=3,v=3) amo=orig.AdaptionMaskDiff(sdl3,inclOrig,pregain,pregainoffset,gain,thresh,adaption_blur,expands=1).mt_lutxy(bdm,"255 255 x - y * 255 / -",y=3,u=3,v=3) deblocked=sdl8 deblocked=mt_merge(deblocked,sdl5,am8,y=3,u=3,v=3) deblocked=mt_merge(deblocked,sdl3,am5,y=3,u=3,v=3) deblocked=(inclOrig==0) ? deblocked : mt_merge(deblocked,orig,amo,y=3,u=3,v=3) return(deblocked) } function SmoothDeblock4(clip orig, int "passes", int "density", int "inclOrig", string "wieghtMX",string "dct_type",int "pregain",int "pregainoffset",int "gain",string "adaption_blursmall",string "adaption_blurbig", int "bth1", int "bth2", int "bshare", int "eth1", int "eth2", int "eshare", float "buvmod", float "euvmod") { passes=3#fixed for now inclOrig=default(inclOrig,1) density=default(density,2) pregainoffset=default(pregainoffset,0) pregain=default(pregain,10) gain=default(gain,10) b1=default(adaption_blursmall,"1") b2=default(adaption_blurbig,"1 1 2 2 2 1 1") wieghtMX=default(wieghtMX,"04center255border0smooth_1") dct_type=default(dct_type,"DCTFilter") mx2=orig.AddBorders(16,16,0,0).WeightMatrixClip(wieghtMX) mxs=WeightMatrixSum(wieghtMX) bdm=orig.BorderMask(bth1, bth2, bshare, eth1, eth2, eshare) sdl8=orig.LanczosResize(orig.width/2,orig.height/2) sdl8=((sdl8.Width % 32)==0) ? sdl8 : StackHorizontal(sdl8,sdl8.crop(sdl8.Width-32+(sdl8.Width % 32),0,-0,-0).FlipHorizontal()) sdl8=((sdl8.height % 32)==0) ? sdl8 : StackVertical(sdl8,sdl8.crop(0,sdl8.height-32+(sdl8.height % 32),-0,-0).FlipVertical()) sdl8=sdl8.SmoothDeblockLayer(5,density,mx2,mxs,"DCTFilter",DCTorig=1) sdl8=sdl8.GaussResize(sdl8.width*2,sdl8.height*2,p=20).crop(0,0,orig.width,orig.height) #sdl8=sdl8.mt_convolution("1 2 1","1 2 1",y=3,u=3,v=3) sdl5=orig.SmoothDeblockLayer(5,density,mx2,mxs,"DCTFilter") sdl3=orig.SmoothDeblockLayer(3,density,mx2,mxs,"DCTFilter") qd8=orig.QuantDiff(6) qd5=orig.QuantDiff(5) qdo=orig.QuantDiff(inclOrig) am8=qd8.AdaptionMask(pregain,pregainoffset,gain,b1,b2).mt_lutxy(bdm,"255 255 x - y * 255 / -",y=3,u=3,v=3) am5=qd5.AdaptionMask(pregain,pregainoffset,gain,b1,b2).mt_lutxy(bdm,"255 255 x - y * 255 / -",y=3,u=3,v=3) amo=qdo.AdaptionMask(pregain,pregainoffset,gain,b1,b2).mt_lutxy(bdm,"255 255 x - y * 255 / -",y=3,u=3,v=3) deblocked=mt_merge(sdl8,sdl5,am8,y=3,u=3,v=3) deblocked=mt_merge(deblocked,sdl3,am5,y=3,u=3,v=3) deblocked=(inclOrig==0) ? deblocked : mt_merge(deblocked,orig,amo,y=3,u=3,v=3) return(deblocked) } How it works: it looks how strong are the edges of the block. If they are small, the deblocking is reduced (not necessary) if they are big it is also reduced (coz the edge should stay there). Thresholds for blocking: bth1=0 bth2=3 means that the deblocking will be eliminated, when the border difference is 0 and deblocking will be applied without change when the block border difference is > 3 bshare=overall impact of the correction 0=none 255=full. The (bth1;bth2) interval mapped onto mask scaled (0;255) is squeezed to (255-bshare;255) For example bshare=150 mean that when there is 0 block edge detected, the deblocking is reduced only to ((255-150)/255*100)%. bshare=0 --> 100% deblocking Thresholds for real edges are similar: When the block edge difference is < eth1, no change. When the block edge difference is > eth2, full deblocking reduction. eshare=overall correction impact modifier. The b-part is imo quite acceptable, but the e-part is prete bad. There are many edges missed and some real blocking is misjudged as edge. And I think the real edges destruction is the worst part of this script. IMO SmoothDeblock3 at default settings performs best. Last edited by redfordxx; 3rd June 2006 at 12:02. |
2nd June 2006, 10:31 | #18 | Link | |||||
Registered User
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 863
|
Quote:
Quote:
Quote:
Quote:
little sharper and less artifact (artifacts appear when are highly quantized hi freq areas... compare SmoothDeblockLayer(quant=6,density=2 or 4)) you can choose any weighting mask you want Quote:
I have one more approach to try to implement: First adapting and then merging shifts. For instance split the frame to areas 32x32 and apply appropriate quant only to that area. It would be milions of crops in script...;-) For this is needed very good and smooth adaptation mask prepared. Last edited by redfordxx; 2nd June 2006 at 10:34. |
|||||
2nd June 2006, 14:23 | #19 | Link | ||
Professional Lemming
Join Date: Dec 2003
Location: Stuttgart, Germany
Posts: 359
|
DQI is a simple filter that simulates real quantization similar to jpeg compression and decompression.
Quote:
Maybe improving the previous step is better. Quote:
bis besser, T0B1A5
__________________
projects page: ELDER, SmoothD, etc. |
||
2nd June 2006, 17:19 | #20 | Link | ||
Registered User
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 863
|
Quote:
But the "real edge protection"...I have no idea at the moment, how to improve it Quote:
... but I wanted first to test it a little before "convincing someone else that it is worth coding". OTOH, if this would be coded not as one filter but as a set of filters, the partial solution can be made. Let's say the final solution would be: 1 - prepare adapt masks 2 - shift clips 3 - adaptively apply DCT on shifts 4 - shift back 5 - mask merge Then for step 3 could be 1 filter (name DCTFilterMask) with two input clips: - original - 8xreduced resolution mask DCTFilter(c,i0,i1,i2,i3,i4,i5,i6,i7) filter normally applies i0-i7 parameters DCTFilterMask(c,m) would read the parameters for each block in the pixel of clip m. I wonder, whether the DCTFilter can be modified that way easily or your filter: DQIFilterMask(c,m,pathtoquantmatrixtxtfile) - m would contain quantizers. I know you mentioned somewhere you use 6of9 as default, but from many sources it is possibler to read the true quant matrix, so why not to use it. steps 5 can be maybe easily solved by modifying the Average filter... look here with these two filters the script would be much faster and there is more time for experimenting with the adapt mask creation and later, after it is tested, the adapt matrix creation filter will be made Last edited by redfordxx; 2nd June 2006 at 17:22. |
||
Thread Tools | Search this Thread |
Display Modes | |
|
|