divx 5.1.1 vs xvid 1.0rc3 (ni hao)

[abubin]

I haven't been encoding for some time now.

Just started again yeasterday. Both the codecs have changed a lot since 6 month back.

I am seeing majority going for xvid. So, I tested xvid first. I tried encoding 2 music video, one is mpg and another xvid avi. Both are in very high quality. Result is ok considering I almost reduce the size to half of the original.

Then I try to encode the xvid avi with divx 5.1.1 using virtualdub. For both cases I tried 1 pass and 2 passes encoding.

Here is my summary :

DIVX 5.1.1 gets better quality with 2-pass with performance/quality set to slowest. No doubt, this will make the encoding speed to a crawl but the quality is good.

For the same quality I am getting with the divx 5.1.1 setting i mentioned above, I can't get that quality on xvid without sacrificing filesize.

So, I just wonder, why more people are going for xvid? Maybe I did something wrong with my xvid encoding? Any secrets to xvid?

[SeeMoreDigital]

Hi abubin,

Welcome to the forum.

Unfortunately your simple comparison may be flawed.

You've not provided any information about bitrates used. Or the codec settings ie B-frames, GMC, Qpel etc......

Cheers

[jggimi]

See Doom9's codec comparisons (http://www.doom9.org/codec-comparisons.htm) for some good examples of how to compare codecs.

[piscator]

Ah well, this whole DivX/XViD discussion depends on religion (opensource is better than commercial?) and personal preference.

Originally posted by jggimi
See Doom9's codec comparisons (http://www.doom9.org/codec-comparisons.htm) for some good examples of how to compare codecs.

This codec comparison is quite nice, but keep in mind that a few snapshots are compared and the conclusion is based on that (what else can you do to compare quality?). But it could well be that if you take other snapshots, the conclusion could be a bit different. Video quality is more than comparing still images :D.

Anyway, given enough bitrate I don't think DivX or XVid makes so much a difference, both are superb!

Personally, I prefer DivX better because it's more stable/reliable and features aren't so often broken -- just check XViD forum on bugs ;). But that's of course because it's still in beta.

Also for me, it looks like the slowest encoding option in DivX gives slightly better results than XViD (I use 4-pass, first 3-passes standard, last pass slowest). But I don't care about encoding speed. If you do, XViD could be better. Anyway, as I said personal preference :D

greetz,
Piscator

[jggimi]

...a few snapshots are compared and the conclusion is based on that...Actually, the snapshots are there for your reference to the scenes and individual frame comparisons only.

Doom9's evaluations were of video. Complete information was included so that you could reproduce the videos, including scripts. From the latest version:...All files were reviewed using Media Player Classic...

[piscator]

Originally posted by jggimi
Actually, the snapshots are there for your reference to the scenes and individual frame comparisons only.

Ah, well fair enough :) But my point is that one should take care of comparing individual frames. The overall bitrate distribution could favor a single frame for one codec and another single frame for another codec.

If I look at the settings used in the test setup, I wouldn't use slow psy for DivX (for me, gives a bit of blurry results).

And for XViD I wouldn't use QPel (makes playback on standalone impossible) and I wouldn't use a custom matrix (can also give problems on a standalone (if I'm correct) and more importantly I don't have the patience to tamper with that!)

Given that DivX looks slightly better for me (though XViD seems to work better on anime). Anyway, XViD is much faster, so if you're impatient...

greetz,
Piscator

[abubin]

Like i said, divx definitely have the edge on quality if you are patient enough to wait for the encoding which takes horrendous amount of time. But for a quick encode with decent quality, xvid is still the best.

I don't think i need to post any comparison results as I am no noob to encoding with both codecs. I have played with both codecs for quite some time before I stopped for half year.

My question is still unanswered. That is, how to make xvid quality as good as divx (with slowest setting). But the catch here is to make both around same file size. That means the bitrate have to be around the same. I mean, no point to have xvid quality as good as divx but size is double or bigger. I have tried changing options in xvid including - GMC, quartel pixel and so on. Most of the values I leave at default which was recommended. Even GMC in xvid cause green motion artifacts. So, I have to turn off GMC for xvid.

Oh yes, i am using the 2pass encoding for comparison of both codecs.

[Sharktooth]

Originally posted by abubin
Like i said, divx definitely have the edge on quality if you are patient enough to wait for the encoding which takes horrendous amount of time. But for a quick encode with decent quality, xvid is still the best.

I don't think i need to post any comparison results as I am no noob to encoding with both codecs. I have played with both codecs for quite some time before I stopped for half year.

My question is still unanswered. That is, how to make xvid quality as good as divx (with slowest setting). But the catch here is to make both around same file size. That means the bitrate have to be around the same. I mean, no point to have xvid quality as good as divx but size is double or bigger. I have tried changing options in xvid including - GMC, quartel pixel and so on. Most of the values I leave at default which was recommended. Even GMC in xvid cause green motion artifacts. So, I have to turn off GMC for xvid.

Oh yes, i am using the 2pass encoding for comparison of both codecs.
From my point of view, i prefer xvid in most cases. It preserves more details than divx and with the right custom matrix xvid can make miracles. Fast first pass and Turbo mode also make it faster than divx.
BTW my usual encodes are made with divx coz i havent so much time to experiment with xvid settings for every movie. So when i need to encode and have no time to spend in testing (90% of the times) i go for divx coz im sure i will obtain a good result.
Coming back to your question, xvid doesnt need particular settings to stay on par with divx. As a general suggestion just use the highest motion search precision and the highest VHQ mode and for the sake of details use the MPEG matrix (if you're aiming for a good bitrate). All the other settings can be left to the default value to have a good encode (at least as good as divx if not better). If you get undersize, oversize or artifacts well... its time to play with advanced settings and other stuff.

[Manao]

alubin : artifacts with GMC in XviD means you are doing something wrong, because it works. Won't you be trying to read the XviD file with DivX decoder ?

Now, for the xviD's settings, it all depends of the bitrate at which your trying to do your comparison, and of the compressibility of the clip you want to encode.

As a rule of thumbs, for a music clip, with a resolution of 640*480, it's rather hard to encode, so with XviD, I would use b-frames ( default setting ), adaptive quant, vhq 4, treillis, and HVS_Good matrix for a bitrate around 1000 Kbit/sec, or HVS_Best you feel the bitrate is high enough. Don't use Qpel, since I think you may not like the effect.

[jk888]

If you want to learn how to encoded using Xvid you should really read up everything on the Xvid forum. And read this guide here as well:
http://www.doom9.org/xvid-vdub-final.htm

It's a bit outdated so I would not trust it entirely. For example the packed bitstream feature still isn't safe to use. Note that if you leave the settings at default then it will look like crap and most likely not work with ffdshow. The experts in the Xvid forum will give you full details on what settings are safe to use.

Here's the settings I use for Xvid:

HVS-Best matricies
QPel
GMC
BVOP - 2
Q ratio - 1.5
Q offset - 1
Closed GOV
Motion Search Precision - 6
VHQ - 4
chroma motion
Treillis
Encode to target file size (much easier than playing with bitrate numbers)

I've made the switch from DivX to Xvid about 4 months ago and I'm never going back. The video is just sooo sharp even in the high motion scenes, and encoding is sooo fast it's a breeze. The movies I released after I made the switch were so good I got fan letters coming to my email.

So was it easy for me to make the switch? nope it was hard cause I had to learn new stuff like Avisynth and virtualdub (previously I used XMpeg). But if you read Doom9's codec comparison (has the AVS script) and read his guides on Avisynth and Virtualdub then you will learn it eventually. If you get stuck then ask for help in the Xivd forum cause they are really quick at getting newbies on the right track.

[21_already]

Hey, have been reading this and it seemed quite interesting.

I was doing some testing (playing around with matricies on xvid RC3 actually) on a high resolution clip from the matrix reloaded (heavy action), when i decided to extend my tests to include divx 5.11 and ended up viewing them on MPC:

Aside from the fact that i'm starting to find the quality to me is VERY similar (i think images are a little more distinct on XVID) i'm starting to be amazed at what Divx can do with its relatively low profile level.

On XVID I had double b-frames, trellis, Adaptive quantisation, GMC(3 warp points right? how do i change this?), VHQ4 precision 6 and 2 passes (obviously), um and oh yeah, tried not one or two but 4 different matricies

On Divx I had single b-frames, GMC (1 warp point) Slow Psych Slow motion Search and did 3 passes.

No Quarter Pel (for very good reasons)
bitrate for both was 1900kbps - pretty small distribution of file sizes resulted

Well i wanted to investigate if it was true that Divx did not do adaptive quantisation even though i read somewhere that it did (conflicting reports) and i eventually found on the ffdshow filter they had the rather interesting quantisation and motion vector display capability.

Results were very interesting:

1.Divx does not in any way have adaptive quantisation
2.Divx displayed relatively few vectors

Now assuming they have some kind of advanced Psych model as they say, how could it possibly make use of it without varying quantisation across the image (not much leeway with temporal quantisation variance)
And why was it producing such a reasonable image if it seemed to be picking up less spatial redundancy than Xvid. And why oh why would it be picking up so many less vectors if it's damn motion search takes soo much longer???!!!! And then it struck me.

Divx Psych works to reduce motion complexity where it can rather than texture complexity (which it is not enabled to do tho it is part of MPEG-4 spec - i think).
There were clearly less vectors in dark areas where Xvid had picked up hundreds.
Which makes perfect sense:
quantising a dark area harder (a basic psychovisual principle i think-right?) will save you a couple of bits perhaps, but coding a null macro block vector takes just a single bit versus (i think) quite a few to code all 4 vectors within it. The saving could be huge!
This is more practical too: hardware implementations would be simpler, and maybe another reason why divx filter is a little speedier than xvid filter.

Has this all been said before?
Anyway, my conclusion:
I reckon using psych models works more effectively to reduce motion complexity than texture complexity as divx networks have learned. Adaptive Quantisation may be a proverbial red herring (or maybe not, different opinions and whatnot).

[21_already]

oh yeah, chroma motion on Xvid turned on and Slower Chroma Psych on Divx turned on

[jimmy basushi]

try xvid again but turn off the settings that are for low bitrates. change your b frames to 1, dont use gmc, its slow and gives not much. turn off trellis. turn off adaptive quantisation also. if your going to use a different matrix, try the favourite high bitrate matrices of the xvid world: andreas 78er, hvs-best, sixofnine hvs, and the standard mpeg. some people recommend chroma optimiser for xvid also, im not sure how much it gives/takes (anyone take that one?) but i use it myself.

and when your doing these tests, you did do the first pass again with the new matrix didnt you? not just 1 first pass, then 4 with different matrix...

[piscator]

Originally posted by 21_already
Divx Psych works to reduce motion complexity where it can rather than texture complexity (which it is not enabled to do tho it is part of MPEG-4 spec - i think).


Actually DivX PVE works on texture complexity. Look at the DivX 5.1 Guide (http://www.divx.com/support/guides/DivXGuide51.pdf) (pp. 56-60).

greetz,
Piscator

[21_already]

->jimmy basushi
Why not use all those features on Xvid? I wasn't doing PSNR tests, just my own mere human perception so i don't mind using a psych method like adaptive quantisation on Xvid. If it makes adaptive decisions about b-frames then why not use two? If the situation is too difficult to code then it will use 1 or none when required right?
I don't mind using GMC, was not too bothered for my tests about encoding or decoding speed (they maintained the frame rate at the original either way). Trellis i got no clue. just heard that it creates a little quality loss at a significant bitrate reduction, so when working to a fixed bitrate it should be beneficial right?
Didn't originally think it was a high bitrate cause of the resolution, but i'll try your suggestions and see what happens :-D
Oh yeah, definitely did the first pass fresh each time ;)


->piscator
riiiiight. Ok, so if i understand this correctly... no wait, i don't.
Does it mean it's like a special type of pre processing filter when performing DCT, modifying the transition into the frequency plane, so that noisy bits we don't percieve well get 'more compressible' allowing more bits to be allocated to the parts we do percieve strongly? I thought the only way to vary allocated texture data (forgeting mv data for the moment) was through modifying the quantisation level which is only allowed to take 32 interger values and is like a coefficient to control the strength of the quantisation matrix.
I guess i was wrong.
Am i right that i'm wrong?
Yes i guess it makes sense right?
that's the point of the quantisation matricies isn't it? to throw out information at selected frequencies, so it's inherantly variable depending on what was there. Does this mean that divx psych is using an ADAPTIVE matrix???!!! Is that possible?? Having the matrix is required for decoding as well as encoding isn't it so this wouldn't work would it? I'm so confused now. Somebody help me.
Anyway, thank you for the corrections

Excuse me for any noobishness :p

[Manao]

No, DivX5 PVE doesn't work by using adaptive matrices, but adaptive quantization. In MPEG4, the quantizer used can vary inside a frame ( if Q is the mean quantizer of the frame, you can use the quantizers Q-2, Q-1, Q, Q+1 and Q+2 iirc ).

So, the PVE works by overquantization of the high frequency areas, in order to used lower quantization for areas where artifacts would be much more visible.

But DivX5 doesn't allow ( yet ? ) to change the quantization matrix, nor to used modulated quantization.

[Marcel]

Originally posted by Manao
No, DivX5 PVE doesn't work by using adaptive matrices, but adaptive But DivX5 doesn't allow ( yet ? ) to change the quantization matrix, nor to used modulated quantization.
"yet" is correct; quoting the Heise Newsticker (http://www.heise.de/newsticker/meldung/45860), DivX 5.2 will allow them:
"So dürfen die User selbst Hand an die Quantisierungsmatrizen legen und auch aufeinanderfolgende B-Frames soll DivX 5.2 unterstützen. Ebenso wie XviD soll DivX 5.2 die Zahl der aufeinanderfolgenden B-Frames adaptiv anpassen."
("The users may modify the quantizer matrices on their own, and more than one consecutive b-frame will also be supported by DivX 5.2. As well as Xvid does, DivX 5.2 will adjust the number of consecutive b-frames adaptive." [DivX 5.2 is expected to be out in june 2004])

[piscator]

Originally posted by Manao
No, DivX5 PVE doesn't work by using adaptive matrices, but adaptive quantization. In MPEG4, the quantizer used can vary inside a frame ( if Q is the mean quantizer of the frame, you can use the quantizers Q-2, Q-1, Q, Q+1 and Q+2 iirc ).

So, the PVE works by overquantization of the high frequency areas, in order to used lower quantization for areas where artifacts would be much more visible.

I was under the impression that 'something' more happens than only the adaptive quantization. When using 1-pass Q=2 encoding, the quantizer should be at its max (so psy isn't able to quantify better, right?). Looking at this discussion (http://forum.doom9.org/showthread.php?s=&threadid=71731) it shows that using psy=fast and psy=slow have a huge impact on increased file size (which can't be because of quantification). So I'm still wondering what that 'something' is that psy does.

greetz,
Piscator

[Manao]

Quant 1 is still possible, which may explain the increase in size. I never encoded with DivX5, so I can't help more than that, but from what I read from DigitAl56k, Quant1 can be used with PVE enabled.

[piscator]

Originally posted by Manao
Quant 1 is still possible, which may explain the increase in size. I never encoded with DivX5, so I can't help more than that, but from what I read from DigitAl56k, Quant1 can be used with PVE enabled.

I figured that, but upto 60% increase in filesize for psy=slow seems a bit much to just attribute to some Quant1.

[Manao]

A frame at quant1 is significantly bigger ( more than 60% anyway ) than a frame at quant2.

Remember what happened with early XviD 1.0 beta, when quant1 were used in underflow situation. It resulted with an overflow, because the rate-control algorithm wasn't able to compensate the size taken by a frame at quant1.

[piscator]

Ok, thanks! I wasn't aware that it would make such a big difference.

[jimmy basushi]

21_already - ur not newbish, i just personally thought that if your using that high bitrate, why use bitrate saving measures that just reduce sharpness to save a few bits here and there? i could be totally wrong tho, i was just going by what i read around.

[21_already]

No, DivX5 PVE doesn't work by using adaptive matrices, but adaptive quantization. In MPEG4, the quantizer used can vary inside a frame ( if Q is the mean quantizer of the frame, you can use the quantizers Q-2, Q-1, Q, Q+1 and Q+2 iirc ).

Wait a sec, this is exactly what i wasn't so sure about. On the feedback window, if you ask it to display quantisation figures in the image on every macroblock (i assume it's variable at the macroblock level), they're all exactly the same whether you use psych or not. I was suspicious that maybe this was just a problem with the feedback window displaying the wrong thing which is why i also tried displaying the quantisers as numbers overlayed on the image with ffdshow to the same result: All the quantisers were the same on every macro block (ffdshow definitely works at this since it correctly displayed variying values on xvid's i made).

having mulled over what i took away from the divx guide (thank u for pointing it out) was that divx was converting the blocks into the frequency domain, adjusting the values inside so that they corresponded to the psychovisual investigation (making things more or less compressible where they should be), and then using the standard quantisation matrix, thereby allowing efectively adaptive quantisation without having to change the matrix (which would break spec - right?) and wihtout having to vary the overall block quantiser. Then again i could be wrong again. Let me just make sure, for every block it's like this:

quantisation ~= Macroblock Quantiser X [Quantisation Matrix]

Where only macroblock quantiser is allowed to vary for mp4 files

thanks for telling me i'm not a newb, but i can't help thinking i am one :p - i guess this applies to many things in life
oh yeah, if i work it out correctly i was using 0.15 bits per pixel which i gather is low or something (i hate to work those out). Is it? i use stuff with this range of compression a lot and it seems pretty good 2 me

[piscator]

Originally posted by 21_already
if i work it out correctly i was using 0.15 bits per pixel which i gather is low or something (i hate to work those out). Is it? i use stuff with this range of compression a lot and it seems pretty good 2 me

A common misconception. I seem to have answered variations on this question a few times lately (e.g. see thread (http://forum.doom9.org/showthread.php?s=&threadid=72818) )

If you use GKnot and its compressability check, it shows two "bits/(pixels* Frame)" values:
1) A Current value depenpending on your project settings.
2) A Max value calculated from the source.
Comp. percentage is current/max * 100%.

So if your source has 0.15 (max) and you encode on 0.15 (current), you will get a fabubulous result (100% Comp. Check).

So note that this value heavily depends on the source (so that's why you want to do a comp. check!)

greetz,
Piscator

[21_already]

I calculated it by hand:

1945600 bits/sec was target (1900 kbps)
81066.666 bits per frame (24fps)
540000 pixels (1000 x 540)
540000 / 81066.66 = 0.1501
(This is probably the wrong way to do it huh)

and so i end up with this number (done with virtualdubmod - the encodes, not the calculation)

I don't like working with bits per pixel though, i prefer using the absolute compression ratio, i.e.

a roughly 300MB uncompressed source file reduced to 2MB gives about 150:1 ratio

I know this depends on the resolution range your working in and heavily also on the colour planes you're using (RGB24 to YUY i think, though it came from a quicktime source) not to mention the contents of the video, so maybe it's not such a sensible measure, but in the end it is all about how much data it would take versus how much data you can get something decent looking squeezed into, right?

Well anyway, it seemed like a great big whoping data saving at the time :D

[Edit] Come to think of it i remeber why this is an unfair test, when storing in YUY you're throwing out source data, which is as good as throwing out resolution like those phoney MPEG1 claims go, but seen as quantisation does the same thing, and throwing crominance out is not nearly as perceptible as throwing resolution out, i say what the heck

[Manao]

In MPEG4, the quantizer used can vary inside a frame ( if Q is the mean quantizer of the frame, you can use the quantizers Q-2, Q-1, Q, Q+1 and Q+2 iirc ).I said something wrong here. The correct statement is : if a macroblock is encoded at quant Q, the next one can be encode at quant Q+2, Q+1, Q, Q-1, Q-2. So we could have four macroblocks in a row, one at quant 8, the second at 6, the third at 4 and the last at 2.

Thanks a lot sysKin for pointing that out.

[lordadmira]

Originally posted by 21_already
1945600 bits/sec was target (1900 kbps)
81066.666 bits per frame (24fps)
540000 pixels (1000 x 540)
540000 / 81066.66 = 0.1501
(This is probably the wrong way to do it huh)


I have a spreadsheet for doing those calculations, where I put in width, height, framerate, bitdepth, and length and it spits out the pixel/bit ratio. You should calculate this as
(width * height * #ofFrames) / (bytes in video stream * 8)
Of course the bit/pixel ratio is just the reciprocal of this. I use pixel/bit for my comparisons instead of bit/pixel cuz I get a nice small integral number instead of a << 1 fraction. Like 11.2 is easier to deal with than 0.089. To get the uncompressed size you multiply the # of pixels with the bitdepth. It's amazing when ur compressing something by 99.5% and it still looks awesome. :)

These kinds of comparisons are only valid for the same type of (if not the same) source material and the same general type of encoding method.

[lordadmira]

Originally posted by 21_already
when storing in YUY you're throwing out source data, which is as good as throwing out resolution like those phoney MPEG1 claims go,

IIRC YUY (4:4:4) does not throw out source data (chrominace subsampling). Other YUV formats do do chrominace subsampling like YV12 and YV16. Their subsampling is like 4:2:0 and 4:2:2.

[Wolfman]

Then I try to encode the xvid avi with divx 5.1.1 using virtualdub Well you cant add in quality that isnt present in the source.. eg your futher divx encode of an xvid avi cant have more quality than the source??:confused: