All existing deblocking algorithms implementations

[redfordxx]

Hi
I am starting to be interested in AVS filter development.
I want to create new deblocking filter which uses DCT (if I succeed with both the algorithm and AVS development).

There are many deblocking filters existing so I don't know, whether it exists already or something similar exists. I don't want to do something existing.

Can you help me BRIEFLY describe all of them?

I can remember following filters:
BlindPP("xxxx")(DCDecode):
Deblock(DGDecode,MVTools): (Uses full frequencies of Deblock's changes on block borders, but DCT-lowpassed changes on block interiours.)
DeblockM: (spatial domain - finding discontinuities on block borders and repairing border values)
MPEG2Source("xxxx"):
MSUDeblock: (replaced by the smart one)
MSUSmartDeblocking:
SmoothD: (quantizing DCT on shifted clip, then IDCT and averaging)
other...Scripts:BlockTerminator:
Deblock_QED: (combination of Deblock and quantized DCT IDCT on block borders)
DeBlock_QED_MT2: (MaskTool2 version of previous)
FunkyDeblock:
MaxDeblock: (buggy and obsolete)
MSUDeCor: (uses MSUSmartDeblocking, only corrects some errors of the MSU filter)
ReDeblock: (buggy and obsolete)
SmoothDeblock: (quantizing DCT on shifted clip, then IDCT and averaging, detail content adaptive)
other...

(maybe this thread could be useful also as reference for choosing deblocking method)

MDeblock thread (http://videoprocessing.11.forumer.com/viewtopic.php?p=406)
MSUSmartDeblocking thread (http://forum.doom9.org/showthread.php?t=103873)

MaxDeblock thread (http://forum.doom9.org/showthread.php?t=111065)
MSUDeCor avs script (http://forum.doom9.org/showthread.php?p=913497#post913497)
ReDeblock thread (http://forum.doom9.org/showthread.php?t=110936)
SmoothDeblock thread (http://forum.doom9.org/showthread.php?t=111526)

[Manao]

DGDecode Deblock:
MVTools Deblock: It's the same. It's the algorithm of AVC's intra deblocking, without some of its subtleties.

[redfordxx]

algorithm of AVC's intra deblocking
Is it possible to describe it in "one sentence"
And maybe some link to reader-friendly description?

[Didée]

Nothing to add at the moment, except for a small correction on above list: Deblock_QED and Deblock_QED_MT2 are the same, just using v1.5/v2.0 MaskTools respectively.
Could be written as "Deblock_QED/_MT2"

Don't know if it's of much interest here, but the description is a little off ... probably better:
"A postprocessed Deblock(): Uses full frequencies of Deblock's changes on block borders, but DCT-lowpassed changes on block interiours."

Can't get it shorter, sorry. :)

[redfordxx]

"A postprocessed Deblock(): Uses full frequencies of Deblock's changes on block borders, but DCT-lowpassed changes on block interiours."

That's what I was afraid of.
I understand all the words but cannot figure out the meaning. This looks like quotation of something. Can you link it?

[Didée]

No, not a quotation. Spontaneous wording from the top of my head.
The method is (tried to be) described in this post (http://videoprocessing.11.forumer.com/viewtopic.php?p=406#406) of the MDeblock thread. Idea is just that blocking will show high frequencies at block borders, so high frequencies at block borders are reduced (If deblock() did so). In block interiours, the chances of differentiating 1st/2nd generation blocking artefacts from actual detail are poor anyway, and danger is high to remove detail. So, for block interiours get only a different "shading" ... try to imagine the "rubber" model at the end of that post.

Let it be good with that ... this thread is about new ideas, isn't it. ;)

[redfordxx]

Idea is just that blocking will show high frequencies at block borders, so high frequencies at block borders are reducedWow, so it is the same basic idea as SmoothD and SmoothDeblock...

[redfordxx]

New ideas you said... ok:
My planned approach:
Assumptions:
1) there was no blocking before encoding
2) aim only on blocking on 8x8borders

Implications
3) For each 8x8 block
OriginalDCT[i,j]=CurrentDCT[i,j]+Error[i,j]
4) If we know the quantizer and the qmatrix for each block we know the MaxError[i,j], where
-MaxError[i,j]<=Error[i,j]<=MaxError[i,j]
(I thought I know it already but that was wrong so I have to study up)
5) After IDCT with guessed original coefficients we want obtain continuous function on block borders, continuous in derivations, something like that...
...which leads to biiiiiig set of equations
6) We may want to minimize sum of squared weighted differences in spatial domain over whole frame
SUM[(OldPixelValue-NewPixelValue)/Weight]^2
where Weight is somehow growing with quantizer of the block
... another set of equations

And I still think there is some paper on the net where is everything written and all the formulas calculated...i cannot find it anymore.


There are many issues:
*We may not know matrix and quantizers, so we have to guess
*The current DCT coefficients could be already little bit different e.g. due to deinterlacing
*What happens on P and B frames?
*It's been long time since I was on the college and my math is worse (in other words... I wonder whether I solve these equations)
...

But I am a little optimistic...
...coz I installed yesterday VS6 C++ and made successfully my first build in C ever. I changed DCTFilter, so that it writes down DCT coefficients of each block to text file and it works...

[foxyshadis]

If you're going to update DCTFilter, could you add this to the constructor? :p


if(!(vi.width % 8 == 0 && vi.height % 8 == 0)) {
env->ThrowError("DCTFilter: Mod8 video required.");
}

Because right now it just crashes if you give it non-mod8. (Hmm, I wonder if an env->Assert() would make sense for the 2.6 generation.)

[MfA]

4) If we know the quantizer and the qmatrix for each block we know the MaxError[i,j], where
Just to point out the obvious ... you don't know this in avisynth unfortunately.
And I still think there is some paper on the net where is everything written and all the formulas calculated...i cannot find it anymore.
http://www.iti.gr/db.php/en/publications/gatrian.html

Do note that it is designed for images, not video ... for images you can use the laplacian distribution assumption to correct the DCT coefficients during dequantization ... and if you made an estimate for the quantizers you could do it in avisynth. For video the encoded DCT coefficients are for the residual after motion compensation though, and those are the coefficients you have to correct using the assumption of a laplacian distribution before adding them to the DCT transform of the motion compensated frame for the DCT domain filtering. Unfortunately again something you can not do in avisynth.

To sum up, you need access to the per block quantizers, the decoded DCT coefficients and the motion compensated frame. You'd have to do this in xvid, mplayer or ffdshow.

[redfordxx]

If you're going to update DCTFilter, could you add this to the constructor? :p


if(!(vi.width % 8 == 0 && vi.height % 8 == 0)) {
env->ThrowError("DCTFilter: Mod8 video required.");
}

Because right now it just crashes if you give it non-mod8. (Hmm, I wonder if an env->Assert() would make sense for the 2.6 generation.)

I am not the right person to do it, coz my knowledge is to low... I had lot of warnings and errors, probably outdated includes, YUY2 loop part did not compile, I had to comment some things and the dll was three times bigger than the original.

[MfA]

BTW, even extracting the the weights for the DCT domain filter from his paper is a PITA.

You might consider instead implementing his more traditional spatial filter from his paper "An Efficient Algorithm for the Enhancement of JPEG Coded Images".

[redfordxx]

PITA
Well, I don't know this abbreviation, but I guess it does not mean something like "easy".;-)

I wonder about the quantization process. I thought it is something like
CurrentDCT[i,j]=Round(OriginalDCT[i,j]/QuantKoef[i,j]/Quantizer)*QuantKoef[i,j]*Quantizer

where QuantKoef[i,j]=number in q matrix which is 8 and more i guess
Quantizer=1,2,3...
How is it possible that I see some CurrentDCT coefficient as small as 1,-1,2,-2,...

What am I missing?

[MfA]

http://www.urbandictionary.com/define.php?term=pita

Did you see my previous post about the algorithm you chose being plain impossible to implement in avisynth BTW?

MPEG-4 uses a dead zone quantizer, you can find the details in the video verification model which you can download here :
http://www.chiariglione.org/mpeg/working_documents.htm

As for why there are a lot of small coefficients, that is the point of the DCT :)

[redfordxx]

Did you see my previous post...

Yes.. I see and do appreciate all your posts:)

As for why there are a lot of small coefficients, that is the point of the DCT :)

I know, but I expected they would be multiplications of quantization weight. Which numbers as small as 1, 2... are not.

[MfA]

Well if you are working with video then the actual frames which are output are no longer purely the product of the iDCT, they are the sum of the motion compensated frame and the decoded residual ... so if you then apply the DCT to the result as DCTFilter does you can not expect to see the original quantization intervals in the coefficients anymore.

[redfordxx]

Well if you are working with video then the actual frames which are output are no longer purely the product of the iDCT, they are the sum of the motion compensated frame and the decoded residual
But that was on I-frame, so there should not be any mocomp

[redfordxx]

Concerning the quantization estimation, i thought maybe it could be done somehow via comparing DCT coefficients and finding their common dividors of quantization coefficients or something like that?
The matrix can be learned from DGDecode or other tool. If not maybe via scannig lot of blocks and statistically analysing the coefficients' common divisors again.
But that's only my spontaneous idea.

[MfA]

It could be simply DCT mismatch ... or your decoder could be doing some post-processing. I don't know.

I completely forgot about the quantization matrix ... yet another thing you don't have access to in avisynth. The matrix from d2v isn't the quantization matrix, it's the color conversion matrix.

Even if you made a guess about the quantization steps it doesn't matter on p/b frames, because you have no idea what part of the signal is the residual you have no idea about the error distribution. Apart from not knowing the exact residual most coefficients will have been coded with a 0 residual, which thanks to the dead zone have a higher quantization bound ... and you won't have a clue which coefficients that are.

Rather than spending a lot of effort trying to do something fundamentally impossible I think your time would be better spend doing things in the right place.

Or find a way to pass the information you need from the decoder to avisynth.

[redfordxx]

The matrix from d2v isn't the quantization matrix, it's the color conversion matrix.
This is from DGIndex;)
Intra Luma and Chroma Matrix:
8 16 43 70 106 115 133 178
16 16 70 88 115 133 178 205
43 70 106 115 133 178 178 214
70 70 106 115 133 178 205 232
70 106 115 133 160 187 232 255
106 115 133 160 187 232 255 255
106 115 133 178 214 255 255 255
115 133 187 214 255 255 255 255

NonIntra Luma and Chroma Matrix:
16 25 34 43 52 61 70 79
25 34 43 52 61 70 79 88
34 43 52 61 70 79 88 97
43 52 61 70 79 88 106 115
52 61 70 79 97 106 115 124
61 70 79 88 106 115 124 142
70 79 88 106 115 124 142 151
79 88 97 115 124 142 151 169

Rather than spending a lot of effort trying to do something fundamentally impossible I think your time would be better spend doing things in the right place.

Or find a way to pass the information you need from the decoder to avisynth.
I am not trying to do something impossible, I am trying to understand things before I continue with anything.

[redfordxx]

Question:
I looked at this (http://www.iti.gr/db.php/en/publications/details/445.html).

I guess there is a part chich can be used without knowing quantiyers at all.

However I wonder that the formula (40) for calculating new coefficients does not incorporate the old coefficients values. That's strange. Do I miss something?

[MfA]

Well it isn't really strange considering he tried to minimize a blockiness measure. The minimum of the measure is only dependent of the neighboring values.

The original value is only used for a "Do No Harm" policy by making sure the result is within the quantization interval.

[Fizick]

http://library.graphicon.ru/pubbin/view_prop.pl?prop_id=240

[redfordxx]

Guys, as lot of you have much more knowledge about this DCT stuff than me, I wouldn't mind, if you spend a minute reading these last SmoothDeblock posts (http://forum.doom9.org/showthread.php?p=918601#post918601) and maybe dropping an idea.