Why PSNR drops with xvid_encraw?

[Strongling]

Hi,

When I compiled and ran xvid_encraw on many standard test sequences I noticed a strange trend in PSNR statistics. Atleast it is strange to me. There is always a drop in PSNR (in P frames) as the sequence moves along.

The command line I use to encode is for example like this:

xvid_encraw.exe -w 352 -h 288 -max_key_interval 1000 -i mobile_352x288_300.yuv -o mobile_352x288_300.m4v


There are no B frames and the rate controller is pretty much off i.e. all quants are 2.

Now with this setup, I would expect the distortion to stay constant (as long as the nature of the scene remains similar). But I see that there is a steady drop in PSNR. There might have been some room for debate on this conclusion, however, I checked with another encoder and I see no such drop with that so I am sure there is something happening with xvid/encraw (maybe I didn't compile right or something).

For example, I will post the results of two sequences: mobile (352x288, 300 frames) and bus (352x288, 150 frames).

For the mobile sequence, the PSNR for the first P frame is 42.2905474654 for the Y channel but by the 300th frame (there are no I frames in between) it drops to 38.6751965529! I consider that a serious drop in just about 10 secs without an I frame (assuming 30 fps).

For the bus sequence, the PSNR for the first P frame is 42.4682142656 for the Y channel and by the 150th frame it dropped down to 40.5693291811. Again, I think a serious drop for just 5 secs of video.

I am posting the screen shots of the PSNR graphs as well as all the PSNR figures as text files so you can make the graphs yourself.

I see the same trend no matter which sequence I encode, there is always a steady drop if I don't put I frames in between. Whenever there is an I frame, it raises back the PSNR (according to the quantizer value) and then there is a steady drop again from that point onwards.

My questions are: is this intended behaviour for xvid? If it is intended then what feature causes this? Maybe this has to do with encraw and not xvid? Can anyone else please verify using the same sequences, maybe I made a mistake or something?

Any help will be appreciated.



Regards,

Strongling

[Brother John]

Can't help you with your PSNR issue, but I noticed another thing (might be related):

There are no B frames

You sure? Have you checked? Your commandline should work with Encraw's default setting of max 2 consecutive b-frames. Use "-max_bframes 0" to disable them.

[foxyshadis]

There is only 1 I-frame per encode, right? Normally PSNR tends to drop off within a GOP and pick back up at the next I-frame, that's simply because q2 is not lossless and the many quantization and roundoff errors add up to a steady quality reduction. Try it with a longer sequence, it'll get painfully obvious after a few thousand frames. They also add up much faster if you decode with the wrong idct, like libmpeg2, but if you use xvid's that isn't the case.

Based on the tiny zig-zag pattern in your graphs, I'd say you might be using 1 b-frame. The drop-off would probably be worse if every frame was a reference. (But that might just be a weird artifact of some other default setting.) I didn't approve them because they don't really add anything to your description of the problem.

[Manao]

foxyshadis : what graphs ? the attachments aren't yet approved.

[foxyshadis]

Well, if you really want to see them, there they are. Pretty huge though.

[Strongling]

Hi all,

Thanks for all the replies. The reason I am doing this is because I am a huge XviD fan and would hate for XviD to have something unexplained in it that causes the PSNR drop.

Now...

@Brother John

You sure? Have you checked? Your commandline should work with Encraw's default setting of max 2 consecutive b-frames. Use "-max_bframes 0" to disable them.

Yes I am 100% sure. After making the encode, I am using an MPEG analyzer tool that tells me a whole bunch of things like per frame encode type and PSNR etc. I double checked, there are no B frames. I would encurage you (and other people) to do your own tests. That will help verify whether there is even an issue or not.

@foxyshadis

There is only 1 I-frame per encode, right?

Yes there is only one I frame (the first one) indeed and the purpose was to check if the encoder holds well through the sequence. It is mostly a test scenario, otherwise for practical purposes, I doubt people would have such a huge gap between I frames.


Normally PSNR tends to drop off within a GOP and pick back up at the next I-frame, that's simply because q2 is not lossless

Sorry I can't agree with that. Again, I have done some tests (with another encoder) that show otherwise. Please look at some new results I am attaching. The PSNR should pretty much hold a steady value even without I frames. The attached results are from the mobile sequence encoded without B and with just one (first) I frame. For a fixed quantizer of 2 my results show a steady PSNR of around 42 and for a quantizer of 5 the value is around 35. Not to get too attached to these figures (I think they will change for different sequences) but the important point here is that there is no drop. Just small fluctuations which I would consider normal.

and roundoff errors add up to a steady quality reduction

Yes, that is one of my concerns, could someone please verify whether this is the case with XviD? I am not familiar with the XviD source.

Try it with a longer sequence, it'll get painfully obvious after a few thousand frames.

Nope! The longest sequence I have is the HD 1920x1080 "Tractor" sequence with 690 frames. I see no drop with just 1 I frame.


Based on the tiny zig-zag pattern in your graphs, I'd say you must be using 1 b-frame. The drop-off would probably be worse if every frame was a reference. (But that might just be a weird artifact of some other default setting.)

Again, I would just like to say that the issue I am mentioning has nothing to do with B frames. There were no B frames in any of the sequences I have tested.


Thanks again everyone for the replies, they were helpful and made me do more experiments. I await further response.


Regards,

Strongling

[henryho_hk]

Is PSNR a per-frame number or an cumulative number?

[Strongling]

Is PSNR a per-frame number or an cumulative number?


Its a per frame number.

[Didée]

Sorry for the lame question, but: how exactly did you achieve the PSNR numbers?
(Poking for iDCT mismatch ...)

[Manao]

Didee's guess is a good one. You must use the PSNR figures from encraw, anything else is potentially biased by DCT mismatches.

[Strongling]

Hi,

Sorry for the lame question, but: how exactly did you achieve the PSNR numbers?


I am using an MPEG analyzer tool called MTS4EA, a really friendly name indeed :) by Tektronics. It used to be called Vprove by Vqual but then I guess they got bought/merged into Tektronics.

This thing costs like $10,000 for a license per user so I guess they know what they are talking about! I have this thing available at work. Ofcourse I can't buy it myself.

Anyways, the way it works is you give it the source YUV file and the encoded MPEG4 file, it will calculate the PSNR from those two.

[Strongling]

Hi again,

Poking for iDCT mismatch ...


If you look at the second results I posted of the mobile sequence, I am using another encoder which uses the famous "Chem-Wang Fast IDCT" function. I guess that's pretty much standard and XviD uses that too, right?

So the point is that there is no drop with that encoder for the same sequence at the same quantizers.

I thought it might be the IDCT too at the beginning but it appears that it is not the case.

What else could it be?

[Strongling]

Hello,

Did some more testing. Interesting finding so I thought I'd share it.

Attaching the encoding stats as reported by xvid_encraw. It seems that encraw does not report this drop in PSNR! It reports around 42 PSNR (for mobile at qp=2) throughout the sequence.

I have some new questions about this:

1. Are there many ways to measure PSNR distortion? I thought there would be only one way to do it.

2. Even if there are different ways, does the xvid encoder (or the rate controller inside it, for example) look at these to make any decisions? I mean is the PSNR taken into account when deciding anything inside the encoder?

[Manao]

There are different ways to compute PSNR ( or rather, PSNR can be defined in a lot of ways ), but all the definition of PSNRs behaves identically on the frame basis.

Your problem is DCT mismatches : the iDCT used during the PSNR measurements must be the same as the one used during the encoding. In your case, it seems your measurements software uses Chem-Wang Fast IDCT, while XviD doesn't.

[Strongling]

Thanks Manao!

You hit the nail right on the head. Turns out that XviD wasen't using the Chen-Wang IDCT but an optimized MMX version.

I did some more experiments, I set it to the simple (plain C based, non hardware optimized) version and complied XviD again. The PSNR drop goes away. Look at the new attached graph.


However, now the question is, why is the MMX implementation not equivalent to the non-MMx one? Shouldn't they be the same thing? Is the MMX version sacraficing precision (or something else) for speed?


Thanks again for the help.

[Manao]

Actually, Didee was right on spot.

Back to the different iDCTs in XviD. There are 4 different optimized iDCT in XviD ( simple_mmx, mmx, sse2, 3dnow ), all with different output.

The MPEG4 standard doesn't define a single iDCT, it defines a mathematical operation and an allowed margin error. All these iDCTs respect the margin error, so all are valid ones. So they are all as precise, even if all give different results.

[Strongling]

Yes, you are right, Didee deserves the credit as he was the one who mentioned it first.

And thanks for the IDCT clarification. I knew that a little bit of tolerence is allowed in the output but I always thought that all the integer implementations were exact according to the Chen-Wang algorithm.

Thanks again everyone, I learned something valuable here.

[squid_80]

Back to the different iDCTs in XviD. There are 4 different optimized iDCT in XviD ( simple_mmx, mmx, sse2, 3dnow ), all with different output.
Hold up, mmx + sse2 + 3dnow should all be the same. The only one that's different is simple and it's not used unless you change the code and compile yourself. Remember when some people made simple the default in their builds and all the fuss it caused?

[Manao]

My bad, I actually didn't check if the output was identical or not.

However they still don't need to be identical ( though it's better, from a programming point of view, if they are ).