Silly ideas for Motion Masking

[OUTPinged_]

Yes, it's me again.

1. Use SAD to detect "macroblocks that should be quantized more". The larger SAD is, the heavyer quantization.

2. Use per-macroblock size calculations for each frame. Calculate a difference for smallest and largest MB and quantize accodringly less and more. Or use some thresholded value.

[sysKin]

Originally posted by OUTPinged_
[B]Yes, it's me again.

1. Use SAD to detect "macroblocks that should be quantized more". The larger SAD is, the heavyer quantization.
Can you explain why? The worse quality is (of a given macroblock), the smaller precision you want to use to fix it.... I don't get it ;)

And why you call it 'motion masking' ?

Radek

[OUTPinged_]

Why 'motion masking'? because -h calls it like that.

He's "da man" you know...

As for SAD:

motion masking is supposed to increase quantization value on per-macroblock basis for 'high motion' macroblocks. Now look at SAD values:

1. Zero motion macroblocks: assuming noise level is small, we will have a _very_ close match produced by ME engine so SAD will be very low, too.

2. High motion movements that are being predicted very well are usually pannings, which shouldnt be supposed to be affected by motion masking, too.

3. Now take non-linear motion, explosions and extra-noisy scenes. SAD will probably be larger than in cases 1&2.

These are ideas only. As I seem to not be able to compile xvid with libraries i have on my pc, I can test them.

If somebody could build a per-macroblock SAD value's table for a clip that contains both high and low motion sequences, it would show us if that idea is any good.

[OUTPinged_]

err, it seems i didnt answer a question :-)


Here it is. Motion masking isnt supposed to make ugly macroblocks look worse. Instead it is supposed to make low motion areas in large frames to be quantized less than they will be if canonic CC will be applied.


ie, we have large frame that has left half a still pic and right half a high-motion. CC will use high quantization value for it and still part of image will look crappy. Motion masking will preserve picture quality for still part.

[sysKin]

Originally posted by OUTPinged_
Motion masking isnt supposed to make ugly macroblocks look worse. Instead it is supposed to make low motion areas in large frames to be quantized less than they will be if canonic CC will be applied.

ie, we have large frame that has left half a still pic and right half a high-motion. CC will use high quantization value for it and still part of image will look crappy. Motion masking will preserve picture quality for still part.
OK, I understand what you mean now ;). Hovever, note: if you'll spend more bits on low motion areas to make them even better, you have no other choice but to spend less bits on high-motion areas, which will produce ugly macroblocks there.
Also, using SAD for detecting nonlinear motion won't work. It will detect texture changes instead - human faces, edges between two moving objects (like a person walking on a sharp background), trees waving on the wind... every time a picture change is not really a movemment of 16x16 block. And all edge macroblocks too, because they often have a bad compensation.
If you want to detect motion which is not panning, it would be better to check how different the motion vector is compared to it's prediction.

[OUTPinged_]

Well the motion masking is quite old idea. -h suggested it in reply to first post i wrote in xvid section (a flame one).

Difference between motion masked frame and regular should be compensated by overflow control, not frame' quantizer increase.

Also, using SAD for detecting nonlinear motion won't work. It will detect texture changes instead - human faces, edges between two moving objects (like a person walking on a sharp background), trees waving on the wind...

...and that is exactly the place where it is supposed to increase compression.

This isnt HVS we are talking about. The choice should be more simple here, just "is there motion" or not.

Hmm, i read all thread again and it looked like i suggested to use SAD instead MV values. My fault. If that ever will be implemented, SAD only should serve as a hint. It can tell us if long MV points to a "predictable" macroblock. In that case it should be less compressed.

"MV only" method is limited. It works bad with near-zero motion vectors, very long vectors and random texture swtiches (your example). I hope "MV+SAD" way will be more effecient.

[vinouz]

OK, I understand what you mean now . Hovever, note: if you'll spend more bits on low motion areas to make them even better, you have no other choice but to spend less bits on high-motion areas, which will produce ugly macroblocks there.
Looking at some recent WMV9 encodes, it seems that's exactly what they do. A hand took a paper on the bed (memento), taking some 4 or 5 frames for the whole movement, and using half of the picture. During encoding I noticed the hand was completely macroblocked and thought I would see it. But when playing that part, even being concentrated and aware of the fact, I just didn't notice them (Well, ok, if you're very careful, and waiting for it, else it's too short for you notice anything).
So I guess crappy quality doesn't means anything on an object you see still less than a 20th second (assuming still means not moving more than 20% of the object's size in a given direction). Here then hand wasn't even "still" from one frame to another, so although it appeared 4 20th of a second I couldn't notice it being macroblocked.

but a little artifact adount the shoulder of that still woman in the background titillates me as "this little button inside the mouth that would certainly disappear if you stopped touching it" (sorry for my poor translation of the french translatoin of fight club) ;)

Vincent

[OUTPinged_]

That's very interesting.

Are you sure these are quantization artifacts and not traces of preprocessing?


Good example was in Lain DVDs. I dont have them yet but one scene had exactly half screen of still picture and half screen of train passing very close. Was a real b%tch to encode, quantizer4 frames were too big and quant7 was showing ugly blocking in still part (nandub encode).

[MfA]

Until XviD has some way to take into account dquant bit costs adaptive quantization of any sort is of severely limited use IMO. Filtering of the coefficients of the motion compensated MB might be usefull though (see for instance this thesis (http://www-isl.kaist.ac.kr/Papers/PhDTheses/phd_sdkim.zip) and this paper (http://ivpl.ece.nwu.edu/Publications/Conferences/2001/ICIP01b.pdf)).

One problem with using SAD is when a disoccluded MB is inter-coded (for instance because it was only partly occluded). You do not want to smear the error this causes across multiple frames, if it is going to be entirely corrected anyway it is much cheaper to do it in a single frame.

[vinouz]

@OUTPinged :
Yes, these are real blocking artifacts. The contour of the hand are very blocky too, and even strong filtering couldn't give you that. (at least IMO, since I squashed this encode anyways ;P)

@MfA : In think the problem doesn't raises that much for partially occluded blocks, as the block changes juste after (partial occlusion only one frame). So we treat them as moving object (and if they're only little occluded, the SAD won't trigger any threshold anyways).

And as the costs of dquant, I didn't read about how dquant in a frame are coded, but assuming you choose 2 different quants (one for still blocks, one for moving ones) for a given frame, only one bit per macroblock would be needed, which costs, without counting any compression on the quant selection bits (certainly easy to implement, moving objects represent contiguous MB), and on 8x8 MB, 1/64th bit/pixel/frame. That shouldn't be that much of a bottleneck. But maybe i misunderstood the real problem (going to read these papers, but gotta go work now ! :)).

vinouz

[sysKin]

Originally posted by vinouz
And as the costs of dquant, I didn't read about how dquant in a frame are coded, but assuming you choose 2 different quants (one for still blocks, one for moving ones) for a given frame, only one bit per macroblock would be needed, which costs, without counting any compression on the quant selection bits (certainly easy to implement, moving objects represent contiguous MB), and on 8x8 MB, 1/64th bit/pixel/frame. That shouldn't be that much of a bottleneck. But maybe i misunderstood the real problem (going to read these papers, but gotta go work now ! :)).
There is one bit saying if quant changed (always present) and if it says that quant changed, it's followed by two bits saying how mych it changed (-2, -1, +1, +2). Coding is from left to right, from top to bottom - so if you change quant for one macroblock, you need four bits to do that. Also remember that macroblock is 16x16 pixels.

Radek

[MfA]

At low bitrates you are encoding at well below .1 bits per pel ...

Treating them as high motion blocks is exactly the problem vinouz, it is inefficient to first code the error with a high quant and then having to come back the next frame to clean up the mess because it simply takes more bits that way. Deocclusion will give high SADs no matter what, it is hard to distuingish it from other forms of distortion which occur across a motion path over a greater time span (not impossible, but hard ... see "Look-Ahead Coding considering Rate/Distortion-Optimization") and I think it is relatively common too.

[vinouz]

@Sysquant, @MfA : so, let's consider an occluding moving object. At most, per MB line, you change twice of quantizer (let's say 4 times, if we really want to de/increase the quantizer more, in two steps), which takes 12 bits every 16 lines. that makes, on a 512 pixel wide picture, 12/8192~=.0015bit/pixel/frame/occluding_object max. And this is if the object takes the whole height. (or let's say 26 bytes on a whole ???*272 image for example.)
And if there are more objects, you don't often have many objects taking the whole height and moving fast at the same time (the only clear example I could see here would be a cam travelling by the side of a jail cell, looking at the end of the cell, with bars scrolling horizontally in front of the picture). But, let's approve that's not common.
Probably scanning the line to see when we'll lower the quant back, and deciding whether we gain or not from it.
I think we must be gaining in many situations.

But yes, for the contour, if it costs less to quantize low and not have to pay it back twice later (which I fully understand, given the artifacts it would leave on the unoccluded part after), let's use that decision we took to rise described above, meaning we're in the case we'll raise for at least a certain number of MB (knowing that number). We're at the left of the contour. Let's consider raising quant the MB later ! ... And lower it one before (of course taking this fact into account in the decision ;op ).
The points raising a problem are :
-the top and bottom contour : Imagine a 2-bitmap of the state of each MB (0 = no probable occlusion, 1 = contour, 2 = occlusion). Beginning all at 0. In case of a possible occlusion, let's see at the upper MB. If it was 0 we might be on a top contour, put 1 for us. If it was 1 let's put 2, we can proceed. If it was 2, Let's look down. If its SAD/MV estimation gives it for an occlusion we're still on the occlusion. Put 2, proceed. Else, we're on a contour. Put 1, don't.
Well, that's a rough algorithm, and many people already discussed contour algorithm. We can even use a 1-bitmap of the candidates, and use vectorisation algorithms (try on Flash, their algorithm is quite cool), but then wen enter in the dark realm of objects recognition
:scared:
- small objects : the decision taken earlier should avoid this kind of "wasting my time and bits" process on too small objects.

And thanks for the dquant decision description :)

Vinouz