SmoothDeblock (under development)

[redfordxx]

Hi everyone (this was originally intended to Didée, but has full PM-Box so it's here...otoh, anyone can help),

I have still some energy to continue on SmoothDeblock, but now I have problem to solve something. You wrote something about your mathematical background, so if you want to bother a little, pls read further.

My idea is that zeroing hi freq DCT coefficients is not the best way to smooth edges. I have to change the remaining coeffs so that the SSD (comparing original and new texture in spatial domain) is minimal. Which is not the same I believe. Morover, I wanna add weights which will be related to distance of the pixel from the edge and the complexity of block the pixel is member of.

So, what I have:
Definitions
boundaries parameters (defined):
N=8...block size(I know)
C,D...(0,N-1)...where begin zeroed hi frequencies (I know)
C<N or D<N
spatial domain indexes:
i,j...(0,N-1)
frequency domain indexes:
u,v...(0,N-1)
a...(0,C-1)
b...(0,D-1)
pixel intensities, coefficients and weight:
f_ij...original pixel value (I know)
g_ij...new pixel value (I search for)
G_uv...new frequency value (I search for)
w_ij...weight of the SSD (I know)
define abbreviation (so that formulas are shorter):
Z(u,i)=lambda(u)*cos((2*i+1)*(pi*u)/(2*N))
so we have DCT and IDCT formulas like this:
G_uv=2/N*SUM[i=0..N-1 ; j=0..N-1 ; g_ij*Z(u,i)*Z(v,j)]
g_ij=2/N*SUM[u=0..N-1 ; v=0..N-1 ; G_uv*Z(u,i)*Z(v,j)]

Nothing new so far, now set of equations:
(1) if ((u>=C) || (v>=D))
G_uv=0
(2) I want minimize
SUM[i=0..N-1 ; j=0..N-1 ; w_ij*(f_ij-g_ij)^2] = SUM[i=0..N-1 ; j=0..N-1 ; w_ij*(f_ij-2/N*SUM[u=0..N-1 ; v=0..N-1 ; G_uv*Z(u,i)*Z(v,j)])^2]
so derivatives according to G_ab gives C*D<64-7 equations (one equation for each (a,b) combination):
SUM[i=0..N-1 ; j=0..N-1 ;-2/N*Z(a,i)*Z(b,j)*2*w_ij*(f_ij-2/N*SUM[u=0..N-1 ; v=0..N-1 ; G_uv*Z(u,i)*Z(v,j)])]=0
and further from (1):
SUM[i=0..N-1 ; j=0..N-1 ;-2/N*Z(a,i)*Z(b,j)*2*w_ij*(f_ij-2/N*SUM[u=0..C-1 ; v=0..D-1 ; G_uv*Z(u,i)*Z(v,j)])]=0
2/N*SUM[i=0..N-1 ; j=0..N-1 ;w_ij*f_ij*Z(a,i)*Z(b,j)] = 2/N*2/N*SUM[i=0..N-1 ; j=0..N-1 ; SUM[u=0..C-1 ; v=0..D-1 ; w_ij*G_uv*Z(u,i)*Z(v,j)]*Z(a,i)*Z(b,j)]
here is possible to make DCT of weighted original if it helps:
WF_ab = 2/N*2/N*SUM[i=0..N-1 ; j=0..N-1 ; SUM[u=0..C-1 ; v=0..D-1 ; w_ij*G_uv*Z(u,i)*Z(v,j)]*Z(a,i)*Z(b,j)]Here is where I stayed for few hours figuring out that as I don't have 64 equations like this I cannot make I/DCT on right side.
I don't have enough skill to continue and find G_ab. Anyone?

[redfordxx]

Moreover, it would be nice to be able to limit solution values interval to
-Limit_ab<G_ab<Limit_ab
but I guess it's too much and not that important.

Nevertheless, the progress in coding is there and according the first test promising at least significant improvement in speed. Won't be lightning fast, but usable.

[708145]

Hi everyone (this was originally intended to Didée, but has full PM-Box so it's here...otoh, anyone can help),

I have still some energy to continue on SmoothDeblock, but now I have problem to solve something. You wrote something about your mathematical background, so if you want to bother a little, pls read further.

[... some math ...]

I don't have enough skill to continue and find G_ab. Anyone?

Hi redfordxx,

I want to motivate you to continue with this. I think there is a lot to do in the deblocking/smoothing area... especially on the codec side i.e. a codec that does not block even if color is flat.

I'm currently involved with 2 other projects (ELDER and n0153) but maybe I look at that problem above over christmas... if I'm not just sitting on a couch holding my belly :)

A good thing is that I don't have to get back to SmoothD as your filters are already better, right? OTOH I still plan to do a realtime SmoothD as playback filter... if time would only allow so much freetime coding.

Anyway I wish you a merry xmas and bis besser,
T0B1A5

[redfordxx]

Thanx Tobias,
that was nice;)

so also merry xmas and 'till better;)
R.

[dukey]

Personally ..
I dont notice blocking so much in areas of high spatial resolution. In these instances when areas of high spatial resolution are deblocked, it can lead to a really ugly blurry mess.

[redfordxx]

Personally ..
I dont notice blocking so much in areas of high spatial resolution. In these instances when areas of high spatial resolution are deblocked, it can lead to a really ugly blurry mess.

I don't see your point:

That should be reaction on something here posted or just a general statement?

[dukey]

Well if i was to write a deblocker
I would probably ONLY deblock the areas with very low spatial resolution. Like with xvid most of the picture can look fine but areas which are out of focus like the background of a scene with say a colour graduation. This will often lead to blocking and mach banding type artifects. This will make your eye see things which shouldn't really be there. You won't notice these mach banding type effects so much in areas of high spatial resolution anyway .. and deblocking areas with high spatial resolution can just murder the detail.

You could probably pick out areas of low spatial resolution simply by seeing how close in colour the neighbouring blocks are.

I've never seen a deblocker do what I have mentioned above, but I think it might be worth it. I think the 'problem' a lot of deblockers is not that they don't work, but they often deblock parts of the image (killing the details) which often don't need it. Or you dont see many artifacts in these places.

just my thoughts

[redfordxx]

mach banding type artifects.What do you mean by that?


You could probably pick out areas of low spatial resolution simply by seeing how close in colour the neighbouring blocks are.IMO that's not that easy:
you can have highly quantized (blocky) blocks with big block-to-block color difference. (I think it happens often on keyframes with not enough bitrate)
you can have very similar colors with high detail (yellow sand e.g.)

I think the 'problem' a lot of deblockers is not that they don't work, but they often deblock parts of the image (killing the details) which often don't need it.I mostly agree with you but I would say that it is not because they deblock 'all'. They often try to decide where to deblock and where not, but they cannot distinguish it well.

What I want here, is to be able decide the level of detail according to quantization coefficients.

Or you dont see many artifacts in these places.There is another point in deblocking than preventing you seeing the artifacts: making easier source for next compression. Because recompression (which is what are all the Avisynth deblockers meant for) often involves cropping or resizing and that brings the blocking edges in the middle of the new blocks which will eat lot of bitrate which will cause additional blocking somewhere else.

[dukey]

your eyes naturally enhance edges
mach banding is this sort of. Happens in areas of different brightnesses.
Imagine a colour graduation and reducing the colour depth to 16bit. It'll look horrid and your eyes will enhance the edges where the brightness changes making it look even worse than it actually is !

The point im trying to make it. In areas of high spatial resolution (detail) if there is blocking, you probably wont notice it so much as it is surrounded by a lot of other detail. But in areas where there is very low detail, you will create obvious artifacts which will stand out.

Take a look at this pic .. just something i snapped from an xvid encode. Was encoded at pretty low bitrate.

http://img153.imageshack.us/img153/6649/deletens3.png

Look at the sky .. your eyes are enhancing the edges of the blocks, even though the colours maybe pretty damn close together.. By deblocking only those areas of with very low detail, you might be able to get good results without destroying the picture and making it look like a blurry mess.

I am no expert in deblocking, these are just my observations as a n00b :)

[redfordxx]

seems to have found time again to work on it:

I have been thinking about the deblocking strength algo... it decides on every place in the frame the strength of deblocking. But, if the frame is encoded on one quantizer it is not necessary. The whole frame should be deblocked with one strength. It can be sognificant speedup.

However, not all codecs use constant quant all over the frame.
My current feeling is that MPEG4 uses constant quant per frame and MPEG1 and MPEG2 do not.
1. Can anyone confirm or correct this?
2. Is it possible to quess somehow the logic of MPEG2 or MPEG1 encoder to to better decide deblocking strength?
I mean:
- a block having high frequencies means low quantization --- thats easy
- a block having high frequencies means low detail in original before compression or high quantization during the compression? How to decide this? Could in the decision help knowing the level of detail in neighbouring blocks, average detail of the frame, min detail of the frame, max detail, color of the block...whatever?

I had similar question here (http://forum.doom9.org/showthread.php?t=131929)...

Tnx 4 ur ideas
R*

[MfA]

High frequencies can mean an edge, in which case you want a low quantizer ... or it can mean noise or texture, in which case you want a high quantizer. Also expecting the encoders to make sane choices all the time is probably way too optimistic.

I still think your efforts would be better spend in a place where the data you want is actually available ... try taking a look at DMPGdec, writing a drop in replacement for it's post processing is relatively easy, and the postprocess function already gets an array with the per block quantizer values.

[redfordxx]

High frequencies can mean an edge, in which case you want a low quantizer ... or it can mean noise or texture, in which case you want a high quantizer
Well, I don't know what do you mean by texture here, but speaking about noise...I am not interested in denoising at this moment, so if I am not mistaken: presence of noise=>high freq=>was low quant=>no blocking...





writing a drop in replacement for it's post processing is relatively easy, and the postprocess function already gets an array with the per block quantizer values.
Confused...my english is probably not good enough;-)

[MfA]

Take DGMPGdec source code, replace postprocess with your own deblocking code, compile, run.

[redfordxx]

Aaa, that's what writing a drop means... Well, how about only adding an option to the MPEG2Source to produce the mask of quantizers? I could do then whatever i want with that later...

Then I really like to have confirmed from someone, whether there are other codecs with variable quant within one frame than those which can be processed by DGMPGdec...

[foxyshadis]

Sure, practically any codec can use adaptive quantization, and they can be stored in most containers. Not all implementations do, but it's very common to find MPEG (1-4) and VC-1 with quants varying per-block.

You have to take into account the quant matrix, if you really want to figure out what quant was there before, because a 'high detail' matrix can cause a lot of blocking while also keeping a lot of noise/texture. (Texture means a pattern, though it doesn't have to be regular; here it means all of the useful non-edge information.) High quants with these matrices cause a lot of ringing/mosquito noise, instead of just removing high frequencies.

[redfordxx]

Sure, practically any codec can use adaptive quantization, and they can be stored in most containers. Not all implementations do, but it's very common to find MPEG (1-4) and VC-1 with quants varying per-block.
interesting...I tested few codecs with show quants option in FFDShow and X264, XVID, MP42 had constant and MPG1, MPG2 had varying...


You have to take into account the quant matrix, if you really want to figure out what quant was there before, because a 'high detail' matrix can cause a lot of blocking while also keeping a lot of noise/texture. (Texture means a pattern, though it doesn't have to be regular; here it means all of the useful non-edge information.) High quants with these matrices cause a lot of ringing/mosquito noise, instead of just removing high frequencies.OK... the point is, when I know quant and matrix for block, i can calculate error or compression information loss, right?
error_uv=quant*m_uv/2
error_uv=max error for frequency uv
m_uv=value in quant matrix
...at least for non AVC keyframes, right?

[foxyshadis]

interesting...I tested few codecs with show quants option in FFDShow and X264, XVID, MP42 had constant and MPG1, MPG2 had varying...

If you never use the capabilities, then AQ won't be present, obviously. You can make your own by enabling that option in xvid, x264 (patched), mainconcept h.264, Nero, or various ffdshow codecs (in the 'Masking' option page).

[redfordxx]

Hi,
after reading some MPEG-2 docs and thinking over I got following (pls comment whether I am mistaken if you like):
- maybe I know what is pattern already. extreme example: when all freqs are veeeery strongly quantized except the last one, then on decoded frame the last frequency can dominate and we see it as 2D wave…thats the pattern…? This example does not happen because the coefs are increasing to higher freqs…but is that the principle.
- the amount of detail in decoded block does not say anything about the possible error=>blocking magnitude
- the amount of detail in decoded block does say about the possible damage when zeroing hi-freq coefs with DCTFilter on shifted video
- if I learn the quantizer from DGDecode and read the Qmatrix, I can calculate various values and averages of max or expected error…this could help for deblocking adaptability
- this calculation is same way possible for I frames or P frames (except the [0,0] coef?), I don't know how about B frames
- surprise: it is not that a frame DCT is either frame based or field based. My feeling is it can differ every macroblock. Some blocks are encoded progressive, some interlaced within one frame…
R*

[redfordxx]

Hi, again begging for opinions:
I played with excel and simulated the codec process using MPEG quant matrix I obtained from DGDecode and random 8x8 sample blocks with following results (maybe I am discovering discovered thing but thats the risk:):
I defined TME(Q)= (theoretical maximal error for quantizer Q) = (value of pixel[0,0] if all values of freq coefs before inverse quantization equal 0.5)
example:
TME(1)=7
TME(2)=12
TME(3)=19
TME(4)=24
TME(5)=30

Maybe my calculation is not 100% MPEG compliant, but roughly so...
Now all calculated values can be formed as multiplication of this TME for desired quantizer.
Result looks like:
EA(Vert)=expected absolut value of average of absolute values of pixel errors on vertical edges
E(Vert)=expected absolut value of average of pixel errors on vertical edges
MA(Vert)=maximum absolut value of average of absolute values of pixel errors on vertical edges
M(Vert)=maximum absolut value of average of pixel errors on vertical edges
....and
E(Vert)=1.6%
E(Hor)=1.7%
M(Vert)=6.2%
M(Hor)=6.4%

EA(Vert)=7%
EA(Hor)=6.7%
MA(Vert)=14.1%
MA(Hor)=13.3%

M(Max8x8)=47%
E(Max8x8)=28%
and so on

so then, when processing the video:...macroblock with quant 10 comes, I know:
TME(10)=60
E(Vert)=1.6%*60=0.96
E(Hor)=1.7%*60=1.02
E(Hor)=6.4%*60=3.84
..and even per pixel if needed
E(pixel[0,0])=6.9%*60=4.14
M(pixel[0,0])=24.61%*60=14.766
E(pixel[1,0])=8.65%*60=5.19
M(pixel[1,0])=29,28%*60=17.568

The problem of the theory can be here

E(Hor)*TME(40)=1.7%*242=4.114
M(Hor)*TME(40)=6.4%*242=15.488
M(Max8x8)*TME(40)=47%*242=113.74

...estimated maximal possible error is 113. Is it really possible in real world or in such cases encoder chooses lower quant?

Conclusion: I think, this can be together with with edge detection solid base for adaptive deblocking in my filter. However
1) I modelled the samples simply =INT(RAND()*256), which is not reality...in reality the samples would be correlated and the error will be smaler. How to model more real-like situation, any ideas?
2) I hope there are no errors like the 113 example or higher. I guess the encoder chooses lower quant but how to estimate this. Is there any SNR or block complexity/variance rule which can help?

Thanks for ideas.

R*

[redfordxx]

Step one finished: I have the quantizer mask from MPEG file from here (http://forum.doom9.org/showthread.php?t=132326);-)

Now...can anyone guess the distribution of the error values in non-intra block? Or at least some average...or something. What I think is, that the values are small compared to intra values but can be negative... but any better estimation?

[redfordxx]

Can you tell me opinion how to make weighted smoothing correctly?

If we have weight clip w and values clip x then (symbolically):

s="1 2 3 2 1" #weights of moving average
y=w*x
z=y.convolution(s)
ww=w.convolution(s)
output=z/ww
this is clear.

But how about smoothing using DCTFilter?

y=w*x
z=y.DCTFilter(1,1,1,1,0,0,0,0) #for example
#now which formula for ww is correct?
ww=w
ww=w.DCTFilter(1,1,1,1,0,0,0,0)
ww=w.DCTFilter(1,0,0,0,0,0,0,0) #probably not
output=z/ww


Thank you.

[redfordxx]

@Tobias (or whoever else can answer)

Once you mentioned, that for smoothing in my filter, quantization with the original matrix which was used to encode would be better than using DCTFilter.
I am thinking about implementation. I plan to read DGDecode quants, and DGIndex quantmatrixlog, so it is possible. But can you please explain the reasons, so I know whether it is worth the effort?

If I finish, expect improved MMX SmoothDeblock, so no more speed issues… so I wanna do it well.

[708145]

Hi redfordxx,

good work. Keep up the speed :D

Once you mentioned, that for smoothing in my filter, quantization with the original matrix which was used to encode would be better than using DCTFilter.
I am thinking about implementation. I plan to read DGDecode quants, and DGIndex quantmatrixlog, so it is possible. But can you please explain the reasons, so I know whether it is worth the effort?

If you the original matrix and original quant values you keep the same detail that was retained during encoding. If you use a "smoother" matrix it would blur too much.

Could you post performance figures for your MMX code once you get them? It would be great to use that for playback on HD resolutions :)

bis besser,
708145

[redfordxx]

If you the original matrix and original quant values you keep the same detail that was retained during encoding.
Even if used shifted?

There will be the option to smooth with larger matrices...I think 12x12 or 16x16 might be interesting.
How to apply the rule you mentioned on that? What should look a matrix bigger than 8x8 like to smooth similarily like a the known 8x8 matrix?
Seems to me that the down right corner could look similar...but I dont know

Could you post performance figures for your MMX code once you get them?Well there is still a way to go...
It would be great to use that for playback on HD resolutions :)The thing is, I intend to make this filter two pass:
* first pass, which would be very fast, will be done on a clip with showQ=true and saved (look at the link few posts above)
* second, the real deblocking will happen

[Soulhunter]

The thing is, I intend to make this filter two pass:
* first pass, which would be very fast, will be done on a clip with showQ=true and saved (look at the link few posts above)
* second, the real deblocking will happen

Well, for realtime processing the 2 steps could be interleaved, no?

Bwt, good to see some movement in here... :]


Thx n' Bye

[redfordxx]

If you the original matrix and original quant values you keep the same detail that was retained during encoding. If you use a "smoother" matrix it would blur too much.I still don't see it:
Whatever frequencieds the matrix and quantizer quantize, it always results in high frequencies in shifted block. And these high frequencies should be partially/fully removed. Or it is different?

[708145]

Even if used shifted?
There will be the option to smooth with larger matrices...I think 12x12 or 16x16 might be interesting.
How to apply the rule you mentioned on that? What should look a matrix bigger than 8x8 like to smooth similarily like a the known 8x8 matrix?
Seems to me that the down right corner could look similar...but I dont know

Well this is a tough question as I have not tried that. But guessing from the fact that larger blocks tend to result in more ringing, but the ringing averages out with the shifting approach I have truely no clue :p

I still don't see it:
Whatever frequencieds the matrix and quantizer quantize, it always results in high frequencies in shifted block. And these high frequencies should be partially/fully removed. Or it is different?

Those high frequencies that define sharp edges have to be kept. And those frequences will be treated in the same way as by the original matrix, so the look and feel stays the same.

bis besser,
708145

[foxyshadis]

If you remove all or most high frequencies, you may as well just use Deblock() or fft3dfilter(). In fact I can't really find any way around the inevitable detail-or-noise aspect of ringing, except in special conditions like anime.

[MfA]

Ringing is a harder problem, but deblocking doesn't need to do damage to detail (although the approach chosen here isn't good in that respect).

[redfordxx]

Those high frequencies that define sharp edges have to be kept. And those frequences will be treated in the same way as by the original matrix, so the look and feel stays the same.I think we are talking about different thingsOriginal: Highly quantized - blocking:
o |
o | |
o | o o o o |
o | o o o o|
o | |
o | |
o | |
o| |
|o |
| o |
| o |
| o |
| o |o o o o
| o | o o o o
| o |
| o |
high frquencies here ^
Removed hi freq on shifted block:
|
|
|
o o o o |
o |
o o |
o|
|
|
|o
| o o
| o
| o o o o
|
|
|
| shifted block |

[MfA]

High frequencies aren't "here" ... at least not with DCT, which has a very coarse localization in space (undecimated wavelets are better in that respect). The shifted block will have energy in high frequencies if there is blocking, also if there is texture, also if there are high contrast edges other than from blocking.

If you can't distinguish, you're gonna kill it all ... even if you can distinguish, you can't selectively kill it with this approach.

PS. blocking isn't really something restricted to high frequencies with DCT, because it can not capture a smooth gradient sparsely ... the energy gets spread out over the entire frequency range. Killing the high frequency content will reduce the blocking, it turns the hard edges into softer edges, but it won't give you a smooth gradient.

[redfordxx]

High frequencies aren't "here" ... at least not with DCT, which has a very coarse localization in space (undecimated wavelets are better in that respect).Hmh...don't understand...


The shifted block will have energy in high frequencies if there is blocking, also if there is texture, also if there are high contrast edges other than from blocking.

If you can't distinguish, you're gonna kill it all ... even if you can distinguish, you can't selectively kill it with this approach.

PS. blocking isn't really something restricted to high frequencies with DCT, because it can not capture a smooth gradient sparsely ... the energy gets spread out over the entire frequency range.Of, course...this was only simplified example for the discussion about the Tobias'es statement "Those high frequencies that define sharp edges have to be kept"
Killing the high frequency content will reduce the blocking, it turns the hard edges into softer edges, but it won't give you a smooth gradient.Well, nothing is perfect, unless some great ideas appear here;-)
Ringing is a harder problem, but deblocking doesn't need to do damage to detailHm, but all deblockers I know so far either do not deblock enough or damage detail:-(

[Dark Shikari]

A thought--why not remove all high frequencies from the image, then deblock the low frequencies, then re-add all the high frequencies?

This of course would require that the blocks line up with your frequency-removal.

[redfordxx]

A thought--why not remove all high frequencies from the image, then deblock the low frequencies, then re-add all the high frequencies?

This of course would require that the blocks line up with your frequency-removal.This is also one option I have in mind. However, removing high frequencies from DCT would not be OK, at first sight.
It would be necessary to ensure that the borders stay where they are (solving 28 equations with 28 variables --- maybe less when decided not all pixels should be exactly same) plus maybe some more equations to ensure the smoothed block is similar to the original using SSD. I plan to include it as one of the features of my new DCTFilter...if I find some good algo.

The good thing is, this could be separate plug-in and it could be used as preprocessor for any existing deblocker. Didee does something similar in Deblock_QED

[MfA]

Hmh...don't understand...
Frequencies in the DCT transform don't represent localized features.
Hm, but all deblockers I know so far either do not deblock enough or damage detail:-(
Most deblockers apart from SPP (but including this one) are effectively selective blur filters. If you want to improve on what's out there then selective blurring is not the way.

You always run the risk of confusing blocking with image edges at the block edge, but there is no reason detail inside blocks should be damaged by deblocking.

SPP would be a good starting point, one quick and easy way to try to improve on it would be to get it to respect quantization bounds (although doing it at the end probably wouldn't produce good results).

Dark Shakiri's, that is only going to help on blocks which are relatively smooth to start with ... the high frequency quantization artifacts also cause blocking.

[redfordxx]

Frequencies in the DCT transform don't represent localized features.Meaning...presence of hi freq does not imply amount of detail?

Most deblockers apart from SPP (but including this one) are effectively selective blur filters. If you want to improve on what's out there then selective blurring is not the way.Well, you seem knowing more of the stuff than me...so what would be the way?


SPP would be a good starting point.I tried to google it little but didn't find the algo. The one thing I found is that it is similar to SmoothD...well SmoothD having similar principles like Smoothdeblock, does what I understand under blurring. And I don't think it is selective...

Damn, I posted this already yesterday but didn't work...so this is fast repro, hope it makes sense... I am in Brussels now, wow guys the keyboards here are really funny;-)

[MfA]

Meaning...presence of hi freq does not imply amount of detail?
Well, no. It can be texture, it can be original edges, it can be blocking (assuming a shifted transform).

But this is making too much of a deal about a very simple remark ... you said there were high frequencies "here" at the blocking boundary and I was simply saying you couldn't know they came from there (or anywhere specific in the block).
I tried to google it little but didn't find the algo. The one thing I found is that it is similar to SmoothD...well SmoothD having similar principles like Smoothdeblock, does what I understand under blurring. And I don't think it is selective...I was talking about Mplayer's SPP (http://svn.mplayerhq.hu/mplayer/trunk/libmpcodecs/vf_fspp.c?revision=25249&view=markup).

MSE/PSNR wise it can still be improved (and made even slower) by having it respect quantization boundaries.

[redfordxx]

Well, no. It can be texture, it can be original edges, it can be blocking (assuming a shifted transform).So, it can be said that if the shifted block has higher freqs than original block then it is blocking...maybe?

[708145]

So, it can be said that if the shifted block has higher freqs than original block then it is blocking...maybe?

Only with some probability.
Because it could still be that a real steep gradient is located at the block border. Then you have to make sure this doesn't get deblocked.

[redfordxx]

I was talking about Mplayer's SPP (http://svn.mplayerhq.hu/mplayer/trunk/libmpcodecs/vf_fspp.c?revision=25249&view=markup).Well, I don't think I am strong enough to read it right now...is maybe some algo/paper describing it?

[redfordxx]

I see that you have many ideas, but writing me only what is not possible...so let's get some work done, guys...

I sumarize what info we have available:
1 - I can read quantizer values from DGDecode for each macroblock
2 - I'd like to implement reading quant matrix from the file which is produced by DGIndex (if someone recommends me where can I use some opensource code, I'd appreciate)
3 - If I know 1 and 2, I can estimate for each macroblock whether it is I-block progressive, I-block interlaced or something else (just by trying to apply the matrix and quantizer I have
4 - I believe I can estimate the mean and max error of each pixel, whole block or block border as a function of quantizer and variance of the pixels for given matrix (but this is still to be explored)
5 - I can probably measure the magnitude of edges on block borders. I am still not sure which method is most suitable...I mean is not confused by edges going across or diagonally across the borders (what do you think?)
6 - I can calculate something like second derivation inside the block to detect the detail
7 - In case of blocking, there is probably some relationship among edges on various borders of the same block or edges connected to each other. Let's use it... although now I don't know how;-)

There more types of solutions I thought of:
- first maybe as Dark Shikari mentioned, remove some detail
then
1) perform smoothing by weighted quantization (weights=expected errors, strenght=derived from the quants form the dgdecode showQ information, quantization can be substituted by just removing high frequencies, this can maybe be limited by max expected error)
2) find completely new freq coeficients by using SSD method while having some high frequencies zeroed;-)...well, system of linear equations...lot of them

[OK, third time writing this message and trying to post it (I am still in Brussels' hotel and the internet works weird here)]

[Leak]

Well, I don't think I am strong enough to read it right now...is maybe some algo/paper describing it?
A bit of Google-fu after noticing the URL in the comment at the top of the code isn't valid anymore... (http://citeseer.ist.psu.edu/118727.html) (click the "PDF" link at the top right of the page)

np: Yello - She's Got A Gun (Live At The Palladium NY Sep. 1985) (Claro Que Si)s

[MfA]

1) perform smoothing by weighted quantization
The high frequency quantization = smoothing meme is rather dangerous ... it's only true when the coefficient is actually zero'd, otherwise it simply introduces quantization noise. When you are not coding you usually only want the first effect and not the latter, so you are better off just thresholding (ie. making the coefficient zero if it's below a threshold, leaving it alone otherwise). SPP being the exception ...

Anyway, I stumbled across a rather original use of 16x16 DCTs for deblocking. It provides a way to determine which high frequencies are probably original image content and which to remove for deblocking.

http://stevehoelzer.dyndns.org/projects/capidd/

[redfordxx]

@MfA:
I looked only shortly on the link..will study up. Isn't is something similar to this (http://forum.doom9.org/showpost.php?p=913708&postcount=10)?
Again...with the problems on non I-frames?
Well anyway..will read it...

[MfA]

No not really similar, in that paper (http://www.iti.gr/db.php/en/publications/details/445.html) they minimized a blockiness measure for each DCT coefficient separately to get weights for predicting the "least blocky coefficient" which they then clipped to the quantization bounds (without knowing the quantization bounds it just doesn't work).

Capidd uses a shifted DCT across the blocking boundaries (just like you) but it has a novel way to try to determine which of the high frequencies in the block are likely causing the blocking. It will probably work decently even without knowing exactly how the frame was quantized. It's pretty close to what you wanted to do.

[redfordxx]

Hmm, after reading it...I like it. I will play with it a little;-)
What I see already now:
-it will desire perfectly deinterlaced source. Or some other trick can help. But as combing changes per each macroblock, the postprocessing cannot be easily applied for each whole field separately. Anyway...I was thinking about decomb algo with similar type of prediction some time ago.
-on Figure 8(b), when there is strong blocking, the rest of blocking (although smoothed) is visible. I experienced similar problem in SmoothDeblock...maybe the weighting/averaging could be improved...
-maybe also the prediction could be improved using the quantizer and QM info... (not just 0/1 weighting)...but this is just thinking too much in advance

[Max73]

Hello redfordxx,

I just have been playing around with your script ("SmoothDeblock3.avsi"), and I find it AWESOME!, no comparison with MSU smart deblocker, or even with bilateral filtering,
neither of the 2 manage to remove strong blocking artifacts near true edges, which still appear jagged.
With your approach instead such artifacts totally disappear,
while most of the true detail is preserved!

I have also played with temporalDegrain, which can also attenuate artifacts strongly, but that one is slower and kills detail to a large extent. Even for heavily compressed material, where you expect little detail to be irrelevant, i still prefer the results with SmoothDeblock3.

About the parameters (which go to the function BorderMask), this are the most important conclusions i could draw:

#bth2 : Suggest 96: the higher the more detail preserved; 100) seems to preserve quite some detail yet removes blocks
#bshare : no effect
#eth1,eth2 : no effect
#eshare : keep 0 or get no deblocking at all


Just AMAZING! Great work :-)!:thanks:

[koopa]

Really? With this enthusiastic resurection, I tried Smoothdeblock3() only to get a:
"there is no function named "matrix" on line 164 and 349"
error.

Perhaps I have unsuccessfully made a Smoothdeblock.avsi by splicing together the quotes in posts 16/17?

[Didée]

Have a closer look at post#2. ;)

[koopa]

Have a closer look at post#2. ;)

It's alright, I still have my dignity *smacks head*