Upcoming feature: "Reduced Resolution"

[crOOk]

Hi everyone!
It has come to my knowledge that the upcoming dev-api-4 binaries will include a feature called "Reduced Resolution".
I've also heard that it is supposed to change the internal resolution of a frame/scene.
To me this sounds a lot like blurring the clip before compression. Preprocessing is something of which the XviD developers always wanted to stay far away from.
So what's the actual difference between blurring a clip (frame/scene) and reducing the internal resolution (mathematically)? And, more importantly, what is the difference to DivX5's postprocessing approach besides having the ability to adapt to the content of a frame or scene?
I was just wondering if anyone else thought about this because to me this feature sounds a lot like cheating (I hope the devs don't feel too offended by that word).

Oh, one more thing (my usual prodedure): Please don't tell me that I can easily obtain the info somewhere else. I've not come to the Doom9-forum without a reason... ;) hey, actually i sould make this my sig!

[Sharktooth]

It sounds to me like a spatial compression (as seen in VP6 codec).

[mf]

RRV is undecodable by 90% of MPEG4 decoders, and works only with 2 profiles; ARTS and Unrestricted. Since unrestricted doesn't keep itself to any standards I wouldn't use it. Not to forget that it's pretty useless. All you're doing is creating files that won't play anywhere.

[crOOk]

originally posted by mfRRV is undecodable by 90% of MPEG4 decoders, and works only with 2 profiles; ARTS and Unrestricted. Since unrestricted doesn't keep itself to any standards I wouldn't use it. Not to forget that it's pretty useless. All you're doing is creating files that won't play anywhere.

Are you in the wrong thread or am I stupid?

[slavickas]

Originally posted by crOOk
Are you in the wrong thread or am I stupid?

better u haven't asked it :D

[crOOk]

Well, shame on me!

@mf
Thank you for the information!

@slavickas
Calling me stupid? ;)

I got confused because I didn't know the abbreviations mf used (got that now). I also was in a hurry.
AND, I also expected an answer to my question which mf didn't give me. Still, thank you for the explanation! PLUS I'm German! Well, enough with the excuses, maybe slavickas is right and I'm just plain stupid.
So the questions remain:
How does this compare to preprocessing done by DivX5? Are there any reasons not to call it preprocessing? I thought the XviD devs never wanted that.

[bilu]

Originally posted by mf
RRV is undecodable by 90% of MPEG4 decoders, and works only with 2 profiles; ARTS and Unrestricted. Since unrestricted doesn't keep itself to any standards I wouldn't use it. Not to forget that it's pretty useless. All you're doing is creating files that won't play anywhere.

Not useless. You can generate MPEG-4 3GPP streams for GPRS/UMTS.

Read this thread

http://forum.doom9.org/showthread.php?s=&threadid=53278


Bilu

[sysKin]

Originally posted by crOOk
Are there any reasons not to call it preprocessing?Sure - it's not preprocessing.
Reduced Resolution is in fact reduced resolution. Macroblocks become 32x32 pixels big, and all DCT data - once decoded - is scaled up to be 16x16 (dct itself is still 8x8). Motion is also severly restricted, with vector components restricted to either zero or odd values (odd values point to halfpixel positions). Also, an in-loop deblocker kicks in.
Everything described in the standard.

As mf said, RRV is part of realtime profile. It was designed to keep the picture "visible" if bandwidth can suddenly become very limited. Reduced resolution VOPs can be turned on and off at any time, but XviD does not offer this - at least not by VfW.

I don't see any good reason to use RRV, because if decision is not dynamic, you'd be better off if you just encoded with halved resolution.
Also, xvid is the only decoder I know which can play such files.

Radek

[bilu]

@SysKin

Isn't RRV an obligatory part of the 3GPP specifications?

Bilu

[TheXung]

Reduced Resolution is different from postprocessing. They're not even that similar. It plays on an idea that rippers have worked with for a long time: the video can look better if you make the resolution smaller versus increasing the quantizer. So it is another avenue for degrading the quality of the clip, no longer it is just a choice of choosing a higher quant. However, mpeg4 profiles restrict the only reduced resolution to half the height and half the width. With no other degree in between, it only becomes useful for when a clip starts using quants greater than say 10. I guess you can try this concept for yourself; encode a clip at quant 12 and one of half the height and half the width at quant 6 (or whatever quant it takes to get the same filesize) and play in fullscreen and see which one looks better.

Reduced resolution is a very interesting concept. As evidenced by our ripping habits, we choose to use a smaller resolution over getting macroblocks in demanding clips. However, like syskin already mentioned, it should be done dynamically, since this only seems useful for when there is really bad macroblocking. This is also one of the features of WM9 apparently.

If you think this is somehow "cheating" or against the philosophy of the codec, ask yourself, why shouldn't this avenue of video degradation be taken versus the quantizer? Dynamically changing either one outside the codec isn't really feasible.


Again, we are discussing something that will probably not see the light of day or ever get used. So don't get too excited about it.

[bilu]

It is used, just not in XVID. :sly:

[Blight]

It's a feature that should be useful for low-bitrate encodes, maybe give RealVideo a run for it's money.

Might also be useful on high-speed high-quant scenes where the picture is quite blurred by motion already and macroblocks would easily be noticed (reduce quant, work on a smaller image). Slightly more blurred for fast motion scenes is better than macroblocks.

A feature such as "reduce resolution if quant=[n]" might be useful.

[crOOk]

Wow, great response. THX to you all!

originally posted by sysKin
Reduced Resolution is in fact reduced resolution. Macroblocks become 32x32 pixels big, and all DCT data - once decoded - is scaled up to be 16x16 (dct itself is still 8x8). Motion is also severly restricted, with vector components restricted to either zero or odd values (odd values point to halfpixel positions). Also, an in-loop deblocker kicks in.
Brilliant. This is the exact explanation I was waiting for. The cheating part I was referring to is about why reducing the resolution is (mathematically) not the same as blurring.

having the ability to adapt to the content of a frame or scene
What I was trying to express in very poor English here is actually dynamic decision of RRV usage. While I was reading through this thread I found out that this dynamic decision does not seem to be included in the specs which imo makes it useless (as most of you have already told me). Blight's approach would be a nice one I think.

originally posted by TheXung
Dynamically changing either one outside the codec isn't really feasible.
I'm pretty confident that 'variable resolution' will be made possible one day. Maybe someone who wants to support my laziness could tell me if there are specs of any codecs/containers which include 'VR'. Frameserving would be a problem though. But I think I remember reading something like this in another thread about AviSynth. After all, think about it: For how long are we using VBR (I sure don't know the answer:))?

But still, let's get back to the 'cheating' (excuse me) part:
To me it still sounds like some sort of preprocessing thing. Would the picture actually look any different from one which I downscaled with avs just to upscale it again? If it smells like it, tastes like it and looks like it in the end, then what's the difference to pp? I know, technically it might not be pp, but if the result really looks the exact same way (if it does...?) isn't this like going the long way round, isn't this what you call cheating? Not that this feature will decide over the future of video compression, but I think (IF the result is the same as blurring/resizing) the devs should do some more philosophy before implementing RRV into XviD.

I hope I haven't offended any devs... needless to say I have all the confidence in your work that you could get.

[PowerMacG4]

Originally posted by crOOk

But still, let's get back to the 'cheating' (excuse me) part:
To me it still sounds like some sort of preprocessing thing. Would the picture actually look any different from one which I downscaled with avs just to upscale it again? If it smells like it, tastes like it and looks like it in the end, then what's the difference to pp? I know, technically it might not be pp, but if the result really looks the exact same way (if it does...?) isn't this like going the long way round, isn't this what you call cheating? Not that this feature will decide over the future of video compression, but I think (IF the result is the same as blurring/resizing) the devs should do some more philosophy before implementing RRV into XviD.

I hope I haven't offended any devs... needless to say I have all the confidence in your work that you could get.

Reduced resolution doesn't refer to downscaling and then upscaling again (blurring). What the encoder does is basically do this:

You give it a 640x480 frame
It encodes as 320x240
During playback it scales back up to 640x480

That is not 100% accurate, since there is a lot of other stuff going on, but basically... that is what is happening as far as I understand.

[TheXung]

Originally posted by crOOk
I'm pretty confident that 'variable resolution' will be made possible one day. Maybe someone who wants to support my laziness could tell me if there are specs of any codecs/containers which include 'VR'. Frameserving would be a problem though. But I think I remember reading something like this in another thread about AviSynth. After all, think about it: For how long are we using VBR (I sure don't know the answer:))?

But still, let's get back to the 'cheating' (excuse me) part:
To me it still sounds like some sort of preprocessing thing. Would the picture actually look any different from one which I downscaled with avs just to upscale it again? If it smells like it, tastes like it and looks like it in the end, then what's the difference to pp? I know, technically it might not be pp, but if the result really looks the exact same way (if it does...?) isn't this like going the long way round, isn't this what you call cheating? Not that this feature will decide over the future of video compression, but I think (IF the result is the same as blurring/resizing) the devs should do some more philosophy before implementing RRV into XviD.

Well first of all most people don't consider resizing their video to fullscreen postprocessing.

Second, you can't do variable resolution without hacking the heck out of the codec interface and codec. You can't just feed a codec variable frame resolution because it wouldn't know how exactly to compensate the motion vectors, short of upsizing it back to the original resolution inside the codec. And what makes you think external filtering can do a better job than something inside the codec? And about your point of VBR, that is done inside the codec. We don't do variable video filtering all in an avs script and then feed it into a codec only quanting at 2 do we? Basically, video degradation can be done internally or externally. Where the divide should be is that anything that needs to be done dynamically to account for different content should be done inside the codec, anything else outside. That's the way it has always been, and that's why people are saying RRV is only useful if it can be done dynamically.


You're also forgeting the argument that the idea of reduced resolution can be taken below the frame level. In theory, nothing prevents a codec from having some blocks reduced resolution and some normal, something you can't just easily do in an avs script. This would be very useful for scenes where not everything is macroblocked.


Again, this isn't something that will get used in XviD because it won't be playable in hardly anything.

[mf]

Actually, this was once an idea for kludge. RR as a macroblock mode, in a similar way as MipSmooth. In this case only the luma block would be downsampled, making the macroblock consist of 2 8x8 chroma blocks and 1 8x8 luma block (as opposed to 2 8x8 chroma blocks and 4 8x8 luma blocks). I'm still interested in seeing it work. SYSKIN!!!! :D

[sysKin]

Originally posted by PowerMacG4
You give it a 640x480 frame
It encodes as 320x240
During playback it scales back up to 640x480

That is not 100% accurate, since there is a lot of other stuff going on, but basically... that is what is happening as far as I understand. Well.... allow me to explain more accurately.
Imagine that a scene has been encoded with full resolution (which is the 'normal mode') so far. At one point, for whatever reason, next frame is encoded with reduced resolution. What happens is:

- Motion vectors can only use zero or odd components. If there was no motion in a macroblock, a zero vector will just copy all pixels (not reduced pixels, real pixels) from previous p-frame. No blurring, no details lost. If motion was vertical or horizontal, one of the components will be 0 and again no blurring happens in this direction.

For the components which are not zero, only odd values are allowed. Odd values are normaly used to obtain "pixels in between" - halfpixel positions. They are indeed blurred compared to the source (OT: and this is the reason why halfpixel encoding is more blurry than qpel: qpel uses different process to obtain halfpixel positions, which is similar but slower and doesn't blur). The resolution is not limited at this point, but the nature of motion compensation might blur the picture.

The second step of compression is DCT coding of what motion didn't compensate. *This* is where the resolution is really reduced. Normally, every block (8x8) of the residue pixels (the difference between the picture we want and the picture we have so far after motion compensation) is dct-ed, quantized and coded. With RR, the block is 16x16 however - so it's scaled down to be 8x8 first. As you can see, details of this residue are lost - codec can't encode *new* details, only the details left from motion compensation stay.

To sum up: the resolution of the picture is constant all the time, but encoder has its own ability draw details of this picture limited to 1/4 of normal resoluton.

I hope this explaination helped ;)

Radek

[crOOk]

@sysKin
Wow, thank you for the detailed explanation. I didn't know much about motion vectors and how they're internally processed (zero and odd components, again a great explanation), so it took me a while to get through your post, but I think I'm a lot smarter now. It's actually a little more complicated than I thought it was.

[trbarry]

I've been curious about ideas like this for awhile now though never quite enough to go research it. And I didn't even know MPEG-4 had any such thing.

But I like the idea of driving it off the quants as an additional means of rate control. This would be especially true on b-frames where the adjusted quant is higher and thus using the adjusted b-frame quant could bias the method towards using RRV on more b-frames when needed.

- Tom

[sysKin]

(Advanced) Realtime profile and (Advanced) Simple profile don't mix, so if you have RRV, you can't have bframes, qpel, gmc... :(

[Selur]

Reduced resolution VOPs can be turned on and off at any time, but XviD does not offer this - at least not by VfW.

Is this still valid?
(asking since the post is a bit older)

Cu Selur

[mf]

Originally posted by Selur
Is this still valid?
(asking since the post is a bit older)

Cu Selur
Yep, afaik the code hasn't been touched since api3.

[Selur]

Yep, afaik the code hasn't been touched since api3.
Ehmm, did I miss something? Reduced Resolution was available in dev3 ?
(why was there never a option to (de-)activate it in dev3 builds?)

Cu Selur

[communist]

Originally posted by sysKin
Also, xvid is the only decoder I know which can play such files.

Radek
How is a file encoded with RR enabled supposed to look? I though it would look blurred / low res. I did a test encode with it and the result came out blocky / 'bump-mapped'. I'm curious to know whether XviD decodes it properly or not. I do know that it is not useful for DVD-backups etc. but if its supposed to be any useful it must be decoded 'properly'?

I hope I dont get a warning again coz there seems to be no thread containing info about RR :|

[MfA]

It is designed for very low bitrate applications, and even then it is mostly meant to make up for unexpectedly complex frames. Basically making it work right would be a lot of work, for very little application. You wouldnt even be sure beforehand if it will be usefull at all in the end, I have the feeling that anyone encoding today will use bitrates where RRVs will simply not be used by a smart codec even if it supported them.

If you want to use xvid for telephony you might need RRV, otherwise you almost certainly dont.

[sysKin]

There is a big chance it's even broken completely. I haven't tested it for ages... And if it is, I have no idea if it's decoder or encoder that has it broken (or both).

I really shiver when reading anything that would delay 1.0 final release. I want to make new bugs SOON, not fix some old ones. It's no fun.

:D

[Selur]

hmm,..

Reduced resolution VOPs can be turned on and off at any time, but XviD does not offer this - at least not by VfW.

is it planed to have this fixed for final, or will the final only be able to enable or disable rr ?