When to use MPEG vs. H.263 quantizer?

[grennis]

I see blanket statements everywhere that the H.263 is the preferred quantizer for xvid.

However I am encoding camcorder footage for archival purposes at high bitrates, and I have seen it mentioned only in a very old thread (2002) that the MPEG quantizer will yield better detail (no blurring or softening) and no need to worry about blocking (since I am at high bitrate).

Is this true and if so why does everyone use H.263 without question?

Thanks

[Sulik]

Most people use MPEG-4 for low bitrates. At high bitrates, there is little difference between MPEG-2 and MPEG-4.

[olnima]

Sulik, You misunderstood the question.
Normally h263 is better for lower bitrates and/or if You like it not too crisp. mpeg might be better for higher bitrates (let's say >1000) but some people say that the picture is too "hard".
Let your eyes decide. As a side note h263 is a bit faster in encoding as mpeg-quant.

Olnima

[grennis]

Sulik I don't want to use MP4/H264, because I am using high bitrates here, and so was wondering about the 2 different xvid quantizers.

I guess there is no absolute rule, it's just whatever looks better?

[Sulik]

I should have been more specific. H.263 quantization will usually always perform better than MPEG quantization in a MPEG-4 codec, especially at low bitrates.

MPEG quantization will quantize high frequencies more than low frequencies, so it will tend to produce less blocking in flat areas, but more ringing.

In the case of high bitrates and/or high texture complexity (grain, sharp edges, busy texture, ie more high frequencies), the MPEG quantization will start looking better than H.263 (especially if tweaked for a specific content).

What I meant to say in my previous post is that at high bitrates, the compression efficiency of MPEG-4 with MPEG quantization is very similar to MPEG-2.

[grennis]

OK thanks for the info.

[Soulhunter]

- h.263 -> good for anime n cartoon stuff...

- MPEG / CQMs -> good for film n high bitrates!


Bye

[fight2win]

thanks for the infos....

[henryho_hk]

Home cam recordings can be very hard to compress (gain noises, unstable shots, etc.). >3000kbps is usual for many common family scenes if you want archival quality. You may want to use software video filtering to pre-processing the video first (I would introduce you to the AVISynth forum experts). BTW, I presume that your cam produces interlaced footage and so you should either encode in interlaced mode (a checkbox inside XviD), or deinterlace your video and then encode progressively.

Back to H.263, it’s more softening; hence, it tends not to produce as much artifacts when the user forgets to deinterlace the video, “opts” not to pre-process (noise reduction, etc.), picks a low bitrate and so on. It is faster as well. Hence, it’s a good starting point for XviD encoding.

MPEG preserves more details and produces more artifacts when the bitrate is insufficient. The problem is that, without a proper compatibility test (such as that described in the sticky XviD preset thread), it is difficult to systematically deduct a “sufficient” bitrate and we can only judge by our experience. At the right settings, it surely shines over H.263 (except for CG anime, etc).

Actually, XviD supports custom quant. matrix and there are a bunch of good matrix available. You can visit the stickly XviD Preset thread to name a few.

[Ranguvar]

H.263 -- Best for <1000 bitrate, almost all anime/cartoons. It preserves less detail, which means less noise and less artifacts at low bitrate, but less detail. Slightly faster.

MPEG -- Best for >1000 bitrate of DVDs, film, "real" stuff. It preserves more detail. But that also means you may get more grain/noise, and artifacting if you have too low a bitrate.


best way is just to try and see :)

[blizard]

Home cam recordings can be very hard to compress (gain noises, unstable shots, etc.). >3000kbps is usual for many common family scenes if you want archival quality. You may want to use software video filtering to pre-processing the video first (I would introduce you to the AVISynth forum experts). BTW, I presume that your cam produces interlaced footage and so you should either encode in interlaced mode (a checkbox inside XviD), or deinterlace your video and then encode progressively.

Back to H.263, it’s more softening; hence, it tends not to produce as much artifacts when the user forgets to deinterlace the video, “opts” not to pre-process (noise reduction, etc.), picks a low bitrate and so on. It is faster as well. Hence, it’s a good starting point for XviD encoding.

MPEG preserves more details and produces more artifacts when the bitrate is insufficient. The problem is that, without a proper compatibility test (such as that described in the sticky XviD preset thread), it is difficult to systematically deduct a “sufficient” bitrate and we can only judge by our experience. At the right settings, it surely shines over H.263 (except for CG anime, etc).

Actually, XviD supports custom quant. matrix and there are a bunch of good matrix available. You can visit the stickly XviD Preset thread to name a few.

Would H.264 (MPEG-4 AVC) need less pre-processing (clean up noise, grain and other image artefacts that might cause problem), then Xvid (MPEG-4 ASP) before encoding? Could it be that MPEG quantizer isn't mention since 2002 (see grennis first post) as most encoding at high bit rate would now be done in MPEG-4 AVC (H.264) instead where there are other option to control encoder?

Most encoding at high bit rate for Xvid can now be done with lower bit rate in H.264 and still keep a image that is very near in visual quality. All this depends on what kind of display that are in use (CRT/LCD) and which kind of hardware limits that are in question if I have understood this correctly.

[Soulhunter]

Would H.264 (MPEG-4 AVC) need less pre-processing (clean up noise, grain and other image artefacts that might cause problem), then Xvid (MPEG-4 ASP) before encoding?

Yes, to some extend... Often you can get away with some leftover noise because h.264 will smooth it away! The question is: Is it a good thing? Imo a codec should try to reproduce the source as exact as possible... Because, not only the left over noise will get smoothed away, but also other fine texture/details with it! Source alteration like this should only happen at pre-processing level, not while encoding... But thats just my opinion! ;]



Most encoding at high bit rate for Xvid can now be done with lower bit rate in H.264 and still keep a image that is very near in visual quality. All this depends on what kind of display that are in use (CRT/LCD) and which kind of hardware limits that are in question if I have understood this correctly.

Yes, the only "very near in visual quality" at high bitrates and the lower FPS rate [en/de-coding] for x264 are probably the main reasons some ppl still use Xvid [me included]. Another reason could be that x264 is a bit harder to setup correctly, no!?


Bye

[Sharktooth]

never used deadzones in x264? h.264 (depending on the implementation) can retain as much fine details as ASP codecs. you should simply not use RDO (possibly a low subme for x264, for example 1 or 2 and appropriate deadzones values) or, if the codec has it, enable FGM and PSY enhancements.
h.264 codecs are harder to setup and that's cause they have much more features than old-gen codecs.

[Dark Shikari]

you should simply not use RDO (possibly a low subme for x264, for example 1 or 2 and appropriate deadzones values)That's a really great way to not only waste loads of bits, but to ruin the quality. I've found if only one hpel iteration is done with subme, there are certain cases in which the true motion vector is missed by half a pixel or so, and as a result, motion no longer looks smooth due to the difficulty of using the residual to compensate for that bad motion vector.

I have never understood your claim of "low submes being good for visual quality."

[Soulhunter]

Conversation above [edit: and below] shows what I mean "with harder to setup correctly"

;]

[Sharktooth]

@Dark Shikari: try it by yourself. use a dirty source with fine grain or just white noise, set deadzones to a superlow value (1 or 2 for intra and about 10 for inter) and do 2 CRF 18 encodes. one at subme 7, the other at subme 1. look at the results.

Returning back to the thread topic, h.263 quantization is not that bad, but it's simply not as good as a "dedicated" custom matrix quantization. it's a sort of general purpouse quantization that is almost good for every source material.
Theoretically you can create a specially tuned custom matrix for each movie and obtain better results than h.263... but that's not an easy task.

[Dark Shikari]

@Dark Shikari: try it by yourself. use a dirty source with fine grain or just white noise, set deadzones to a superlow value (1 or 2 for intra and about 10 for inter) and do 2 CRF 18 encodes. one at subme 7, the other at subme 1. look at the results.Since subme1 is going to be vastly larger in that case, what kind of comparison is that? You can't compare a 10 megabit encode and a 5 megabit one and say "HEY LOOK THE 10 MEGABIT ONE IS BETTER."

[Sharktooth]

than use a lower CRF value for the subme 7 encode until you reach more or less the same filesize...
i can assure you the subme 1 encode, even with a higher CRF, will have the "noise" while the subme 7 encode wont...

[Dark Shikari]

than use a lower CRF value for the subme 7 encode until you reach more or less the same filesize...
i can assure you the subme 1 encode, even with a higher CRF, will have the "noise" while the subme 7 encode wont...Yes, it'll have different noise than the original, however, because the "noise" is solely in the form of wasted residual bits that have very little correlation to the original noise.

[Sharktooth]

we're going off topic. however try it with HD and noisy sources...

[jethro]

I would like to add here that Dark Shikari's very own Adaptive Quantization (latest one) greatly improves grain and noise preservation for x264 and makes output video look natural and more xvid-like.
IMHO (I'm on CRT) it is infintely better to use this AQ for natural look and fidelty than to use CQMs or to lower deadzones/subme/trellis settings (especially when it leads to worse x264 efficiency).

[Sharktooth]

AQ lowers x264 efficiency too...

[Dark Shikari]

AQ lowers x264 efficiency too...Not necessarily... AQ compensates for an inherent lack of precision in the H.264 quantizer at low DCT coefficient values.

[jethro]

AQ lowers x264 efficiency too...

PSNR and SSIM numbers are a bit lower when using AQ but it doesn't matter because for the same size the video looks more detailed. If I understand things correctly AQ only shifts quantizers in macroblocks and you can still use full x264 efficiency tools to estimate and encode them.

[Dark Shikari]

PSNR and SSIM numbers are a bit lower when using AQ but it doesn't matter because for the same size the video looks more detailed. If I understand things correctly AQ only shifts quantizers in macroblocks and you can still use full x264 efficiency tools to estimate and encode them.AQ can actually be used to raise SSIM, PSNR, or even both, if applied in the correct way. Its usually not used to raise PSNR though, but my AQ tends to raise SSIM considerably.

Proof of this is in what spent a few minutes coding today--a bruteforce unreferenced-B-frame-only AQ to test a theory of mine--it raised PSNR by up to 0.15db at the same bitrate, along with a small SSIM boost.

[Lenny_Nero]

H.263 -- Best for <1000 bitrate,
MPEG -- Best for >1000 bitrate,
best way is just to try and see :)
This was the way I have settled on Xvid's MPEG setting for 99% of my DVB-TV encodes that I plan to keep, so encode with a >1000 (1000~2000) bit rate and aim for a Bits/(Pixel*Frame) of 0.180~0.350 via StaxRip.

I spent over a week re-encoding over and over and watching via my SAP's. I am by no means an expert, and only just getting into the custom quant. matrix thing (tips hat to Sharktooth) which I hope will open me up to more quality and a lot more learning :)

[Sharktooth]

b/(p*f) is a useless metric. dont trust it coz it doesnt take into consideration the source material compressibility.
compressibility wildly changes from one souce to another... a proper compression test or a trained eye are much better than b/(p*f)
also there are "low bitrate" matrices (jawor's 1CD, EQM V3LR and others) that provide much better results than h.263 at lower bitrates.
however if you aim at SAPs compatibility, then h.263 is your choice, unless your SAP supports custom matrices.

[olnima]

...and watching via my SAP's...

...be carefull if You want to use SAPs for playback. Not every SAP is able to decode every custom-matrix. In this case I would stay at h263/mpeg. I prefer h263 it is not so crisp and looks a bit "nicer" especially on flatscreens.

Olnima

[ToS_Maverick]

about the subme1 issue:
like Dark Shikari mentioned, i also found out, that noise is getting MORE noisy than it actually is. the background starts flickering where it isn't in the original

[Sharktooth]

try higer subme levels (max 5).
the same applies to xvid. VHQ=4 flattens some fine details. lower it to 2 or less if you feel the codec is "dropping" some fine details, noise, grain...