XviD Options Speed Test

[TorgoGuy]

Sorry for the long post...

In working on my concise XviD (http://wellness.uwsp.edu/staff/ken/xvid/faq/) FAQ (see below), I figured it would be useful to quantify how using various XviD features affect encoding speed. I've done this initial test to start to figure this out. I've also included a few interesting (I hope) observations based on this data. There may not be anything new here, but I haven't found a recent thread with this information.

The Test:
I did a two pass encode with the options at the defaults and recorded encoding times for first and second passes. This became my baseline. The build was Koepi's XviD RC2. Actually not quite all settings were at defaults--all encodes in this test were done with the XviD status window disabled.

I then repeated the process turning features on and off individually and comparing the speed to the baseline.

The source material is the entire LOTR Fellowship (extended) movie, with a simple crop(16,64,688,352) and a trim to remove the fan club credits. No resizing was done. XviD was asked to hit a 1,222,000KB target size.

Results:
Below you will find the results. The first column tells you which non-default options were selected for a particular encode. The next three columns give the encoding time for the 1st pass, 2nd pass, and the total time. The times are given as a percentage compared to with default settings. For example, a value of 100% means it encoded at the same speed as with default options, a value of 200% would mean it took twice as long, and 50% would be half the time.

1st 2nd
Settings pass pass total
-------------------------------------
Defaults 100% 100% 100%
VHQ 4 100% 156% 138%
MPEG Matrix 105% 104% 104%
Adaptive Quant. 101% 102% 102%
Qpel 100% 169% 147%
GMC 100% 170% 148%
No B-VOPs 78% 78% 78%
Chroma Optim. 100% 101% 101%
MSP 0 56% 26% 35%
No Chroma Mot. 100% 78% 84%
Trellis 100% 104% 102%
VHQ 0 100% 89% 92%
Turbo 100% 76% 83%
Qpel w/ Turbo 100% 108% 105%
GMC & VHQ 4 100% 208% 173%
Cartoon 102% 100% 101%
"Fast" 59% 45% 50%
"Slow" 110% 381% 294%

"Fast" was defaults except: no B-VOPs, MSP 0, VHQ 0, No Chroma Motion, and Turbo (even though turbo should have no effect).

"Slow" was defaults except: MPEG quant, Adaptive quant, Qpel, GMC, Chroma Optimizer, VHQ 4, and Trellis. MSP is already 6 by default and chroma motion is already also "on" by default.

Observations:

Many options do not affect encoding time much. (Quant matrix, Adaptive Quant, Chroma Optimizer, Trellis, Qpel if turbo is enabled, and cartoon mode)

You can save time by reusing pass files: Looking at the .pass files, it looks like the following settings are ignored for the first pass when using fast first pass: VHQ, Qpel, GMC, Chroma Motion, Trellis, and Turbo. The good news about this is that if you keep your .pass file you don't have to re-do your first pass even if you want to modify any those settings on your second pass. For this test I did NOT reuse the pass file and redid the first pass every time.

XviD has great rate control: XviD's rate control performed very well except when MSP was set to 0, (see below). All encodes met the requested KB with less than 0.05% deviation except: 1) Both encodes with MSP 0 were 36% over requested size and 2) All 3 encodes that included Qpel were about 0.1% under size

Surprising fastest encode: The fastest encoding time was acheived by using the defaults with MSP set to 0. I thought I could improve on that speed (but not quality of course) by also disabling B-Frames, using VHQ 0, and no chroma motion. I even enabled Turbo (although without Qpel and B-Frames it shouldn't matter). That is the so-called "Fast" profile above. It ended up being slower. I have no idea why and will repeat those tests just to make sure the time measurements are correct. (I am confident that they are).

Quality Assessment? I kept all of the .pass files and AVI files. If there is interest I will post my subjective impression on the quality using the various options. Also if there is interest I can run some mathematical measurements (PSNR, SSIM, etc).

Notes:

I haven't done any testing (yet) with fast first pass disabled. (I forgot to).

My concise XviD FAQ (http://wellness.uwsp.edu/staff/ken/xvid/faq/) isn't really fit for consumption yet, but it is starting to take form. I know it is wordy in many spots (kinda reminds me of this post!) and I will fix it before I consider it "fit for consumption." Please don't email me with mistakes in it (yet) although feel free to let me know what you think of the general ideas in it. I do have Crusty's blessing to make this derivative of his FAQ.

I will repeat this with RC3 and 1.0 when it is available. It takes quite some time to run all of these encodes with such a long movie and I had my RC2 test in progress when RC3 came out. I wanted to do the whole movie to have a more "realistic" measurement rather than just a scene.

Remember, LOTR 1 is not necessarily representative of what will happen with other sources, but these numbers provide a general guide. I plan to do more testing.

Yes, I know this is not a cartoon and I used cartoon mode for one of the tests.

If you have a set of options you'd like see compared, let me know and I'll do it.

"What do you think sirs?"

[Bear]

Thanks a lot. What a good work! May be we can start speed test on different machines. For example, using the sampe clip, have same setting and using the same software. So people can test the clip on their machine.

[communist]

Nice - we could do something like mentioned in this thread http://forum.doom9.org/showthread.php?s=&threadid=71236 but all things would need to be finalized and set so all use the same source etc...
Originally posted by TorgoGuy
Surprising fastest encode: The fastest encoding time was acheived by using the defaults with MSP set to 0. I thought I could improve on that speed (but not quality of course) by also disabling B-Frames, using VHQ 0, and no chroma motion. I even enabled Turbo (although without Qpel and B-Frames it shouldn't matter). That is the so-called "Fast" profile above. It ended up being slower. I have no idea why and will repeat those tests just to make sure the time measurements are correct. (I am confident that they are).
[/B]
That is not very surprising ;)
If you had turned on your status window you would have seen that XviD encodes all frames as I-frames. Since no motion search is done there will be no predicted frames - therefore only I-frames. I made that observation when I tried to tune XviD to make real-time caps. It makes sense if you think about it :)

[TorgoGuy]

That is not very surprising. If you had turned on your status window you would have seen that XviD encodes all frames as I-frames. Since no motion search is done there will be no predicted frames - therefore only I-frames. I made that observation when I tried to tune XviD to make real-time caps. It makes sense if you think about it Yes, looking at the pass file, you are right--all frames are i-frames for both the "fast" profile and the "msp 0" profile. I did not realize that MSP0 turns off motion estimation completely. Thanks for the tip.

However, I would say that the results are still surpising. Turning OFF b-frames and chroma motion should have zero effect since xvid wouldn't be tracking motion. Likewise, turning on turbo should have no effect since we're not using b-frames or qpel (which would require motion estimation themselves of course).

Given the new information that MSP 0 disables ME, the surprising part is that these settings had any effect at all. They decreased the encoding speed (by a significant margin--2nd pass is over 40% slower with "fast" settings vs the "msp 0" settings)!

This could be a development oversight (bug?) with XviD doing some work it shouldn't be doing.

[TorgoGuy]

Nice - we could do something like mentioned in this thread http://forum.doom9.org/showthread.php?s=&threadid=71236 but all things would need to be finalized and set so all use the same source etc...Sounds like a big project to create and execute well.

A automated bootable linux encoding cd with included reference public domain video might be the best bet to minimize software configuration variability.

You might be able to get good data without going this far though.

[crusty]

Looks like a nice test TorgoGuy!

I can confirm those readings using Qpel and GMC, both slow down my encodings in a roughly similar way, (16-18 fps with GMC as opposed to 25 fps without; same goes for Qpel).

PSNR doesn't really say that much as it's not the same way as how our eyes work. Best would be if you could just say if there were clips that really looked bad or looked worse or better than others; i.e. a visual comparative review. It's still subjective, but it's less abstract than PSNR.

Also, the 'default except for MSP 0' looks incredibly fast...how is the quality? If it's not too bad, it could be used on low end machines.

It's also interesting to see that VHQ=4 is faster than GMC or Qpel. I already knew that but this just confirms my opinion.

Perhaps you should rename 'Toon' to 'Cartoon'. I had to look three times before recognizing it as cartoon.

Your Concise Xvid FAQ is looking good, but perhaps you should size up the pictures a bit. What tool are you using for the screencaps?

[TorgoGuy]

Originally posted by crusty
Looks like a nice test TorgoGuy!
Thanks. I'm working on a more comprehensive test using RC3 right now. Knowing the fast speed that the devs work at they'll probably have RC4 or 1.0 ready before I get it done. (I still can't believe that people complain that the devs are slow. For such a project I'm amazed at how quickly things get done.)

Originally posted by crusty
PSNR doesn't really say that much as it's not the same way as how our eyes work. Best would be if you could just say if there were clips that really looked bad or looked worse or better than others

Also, the 'default except for MSP 0' looks incredibly fast...how is the quality? If it's not too bad, it could be used on low end machines.
Yes, I agree that PSNR isn't a necessarly a measure of good subjective quality. SSIM is better, I've been told, but haven't tried it out myself. But in my message I offered to give my subjective quality assessment, but (understandably) no one seems to be intersted in it. :-)

As far as MSP 0, the quality is very bad. All frames are I-Frames so it is using double digit quants most of the time to try and hit the target bitrate (which it overshot by 36% anyway). Blocky, blocky, blocky.

Avoid MSP 0 unless you've got some very, very, very slow machine that you need to do realtime encoding on. You'll be disappointed with the results. If someone needs a speed boost they should try No Chroma motion, No B Frames, VHQ 0, or Turbo before they resort to MSP 0. I can give you more details if you (or your FAQ) are interested.

Originally posted by crusty
It's also interesting to see that VHQ=4 is faster than GMC or Qpel. I already knew that but this just confirms my opinion.
Yeah, and the gains by GMC seem to be small. Still, if you have plenty of time for encoding and a good enough decoding machine, I would recommend GMC.

Originally posted by crusty
Perhaps you should rename 'Toon' to 'Cartoon'. I had to look three times before recognizing it as cartoon.
Thanks. Done.

Originally posted by crusty
Your Concise Xvid FAQ is looking good, but perhaps you should size up the pictures a bit. What tool are you using for the screencaps?
Thanks for the feedback. Yeah, I shrunk the screencaps a bit in hopes of saving a bit of space, but yeah, you're probably right, I may have gone too small. To do the captures, I'm simply using the Alt-PrintScreen trick and pasting into a graphics program (even Paint will do).

If you'd like I can send you a copy of the resized and non-resized screen captures that I did of XviD. It might save you some time if you haven't already done the screencaps yourself.

[crusty]

Never mind, I already figured how to do it with irfanview.
Very handy, as it lets you capture only the active window.

[communist]

Originally posted by crusty
Never mind, I already figured how to do it with irfanview.
Very handy, as it lets you capture only the active window.
PrintScreen = full screen capture
Alt + PrintScreen = capture active window only
No need for IrfanView ;)
Though I wouldn use MSPaint either coz saving to gif and jpg is very primitive and color reduction sucks (for gifs) :p
You can use Gimp for that - open source :D
http://www.gimp.org/download.html

I still remember using it the first time on a linux machine - man that program kicked ass at that time (it still does) :)

[acrespo]

You spoke about MSP 0 but and what about MSP 1 to 6? I always use MSP 6 but If I can reduce to MSP 1 without quality impact I will do that. Anybody know the quality/speed differences of these settings?

[TorgoGuy]

Originally posted by acrespo
You spoke about MSP 0 but and what about MSP 1 to 6? I always use MSP 6 but If I can reduce to MSP 1 without quality impact I will do that. Anybody know the quality/speed differences of these settings?
I am working on a follow-up for that initial test. It is going to be more comprehensive and will include all MSP levels. It will also include my subjective assessment of quality along with SSIM and PSNR for those interested.

I am going to skip RC3 and am waiting for whatever release comes out next. I am planning on doing three video sources (regular video, traditional animation, computer animation). Right now the sources will be LOTR 1, Lion King, and Shrek.

I would prefer if I could find some public domain video so that I could post clips for others to view, but I haven't come up with any good picture quality sources that are of similar character to a movie. (Is there some NASA video around or something? NASA also does animation.) Ideas would be appreciated...

[neelX]

Originally posted by TorgoGuy
I would prefer if I could find some public domain video so that I could post clips for others to view, but I haven't come up with any good picture quality sources that are of similar character to a movie. (Is there some NASA video around or something? NASA also does animation.) Ideas would be appreciated...What about Fanimatrix (http://www.fanimatrix.net)? But it's quite short and noisy:(

nX

[bond]

great comparison, thanks a lot :)

[SeeMoreDigital]

A very well structured and detailed post TorgoGuy.

I reckon the information is well worth including in XviD's FAQ's as it should be of use to all.

I think also makes a goog 'example template' should any other forum members decide to generate 'detailed' speed tests with other codecs....

What do you say 'mods'?

Cheers

[neelX]

One suggestion:

It would be useful to see the 2nd pass/1st pass ratios.

I appreciate your work,
nX

[TorgoGuy]

Originally posted by neelX
It would be useful to see the 2nd pass/1st pass ratios.
Good point. I will include that in the next test.

I don't know if any XviD devs are looking at this thread, but if you are, I'm still trying to figure out why defaults+MSP0 is (significantly) faster than defaults+MSP0+VHQ0+No_B-Frames+NoChromaMotion+Turbo. If not devs comment on this, I'm going to try and figure out exactly which part of this combination is causing the slowdown.

krv.

[CruNcher]

TorgoGuy nice tests :) i know how hard those are you dont have to tell hmm i think PSNRLab could help ya i developed it especialy for the purpose of various settings testing (to find bugs) but unfortunately it's @ the moment a little outdated (no turbo test, no carton mode) useing other defaults and some older pre RC stuff that isn't needed anymore im gonna update it soon :) but if you need it urgently you could adjust it to your needs.

[mikeX]

TorgoGuy,
first off, congrats on a great test :)

I think you should also give a brief hardware description of your machine (at least processor type)
You may think this is irrelevant since your test is about relevant and not absolute speed, but my small compiler benchmark here (http://forum.doom9.org/showthread.php?threadid=71803&perpage=20&pagenumber=10) gave some weird results which indicate differences in relevant speeds as well

i would also suggest (as i did to crusty here (http://forum.doom9.org/showthread.php?postid=459234#post459234)) using PNG for your screenshots, my screenshots were about 8KB each at full resolution [i used The Gimp] (although i find the little screenshots in your FAQ really 'cute')

[neelX]

Originally posted by mikeX
I think you should also give a brief hardware description of your machine (at least processor type)
You may think this is irrelevant since your test is about relevant and not absolute speed, but my small compiler benchmark here (http://forum.doom9.org/showthread.php?threadid=71803&perpage=20&pagenumber=10) gave some weird results which indicate differences in relevant speeds as wellDidn't you mean relative??;)

nX

[TorgoGuy]

Originally posted by mikeX
I think you should also give a brief hardware description of your machine (at least processor type)
You may think this is irrelevant since your test is about relevant and not absolute speed, but my small compiler benchmark here (http://forum.doom9.org/showthread.php?threadid=71803&perpage=20&pagenumber=10) gave some weird results which indicate differences in relevant speeds as well
My updated test is going to be run on a few different machines (700mhz celeron, 933mhz pentium iii, 2.4ghz pentium 4, and my dual processor setup--sorry I don't currently have an AMD machine) to try and see if the relative differences for different options differs depending on the model of processor. For the record, the above tests were done on a Dual Xeon 3.06GHz machine. Decoding was done with one processor, encoding with the other in order to try and focus on the slowest part (encoding).

Originally posted by mikeX
i would also suggest (as i did to crusty here (http://forum.doom9.org/showthread.php?postid=459234#post459234)) using PNG for your screenshots, my screenshots were about 8KB each at full resolution [i used The Gimp] (although i find the little screenshots in your FAQ really 'cute')
Thanks for the suggestion and feedback--I do appreciate it--but my reason for reducing the resolution of the screenshots wasn't to reduce filesize (it might have even increased it). I did it to save some screen space, but I might go back to the unresized ones.

As far as GIF vs. PNG, my full resolution GIF versions on average are already smaller in filesize than your PNG versions. I have PNG versions that are smaller still (about 4KB each), again at full resolution. The secret? Disabling the gradiant in the titlebar and disabling font smoothing before doing the screen capture.

I will probably use your suggestion though and switch to my 4KB PNG versions. I haven't added to the concise FAQ in a week or so.

[mikeX]

@neelX
yes i meant relative (:o in some cases you can have relevant=relative in greek so i got confused :o)

@TorgoGuy
oh, i saw you mention 'space' but 'screen space' didn't cross my mind :o
Current size fits really nice to the whole faq layout :) (well maybe just a tiny bit bigger to make out some more detail)

[TorgoGuy]

Originally posted by CruNcher
TorgoGuy nice tests :) i know how hard those are you dont have to tell hmm i think PSNRLab could help ya i developed it especialy for the purpose of various settings testing (to find bugs) but unfortunately it's @ the moment a little outdated (no turbo test, no carton mode) useing other defaults and some older pre RC stuff that isn't needed anymore im gonna update it soon :) but if you need it urgently you could adjust it to your needs.
Cruncher,
I'm not familiar with your program, but will take a look at it. It sounds very interesting.

I currently do all of these tests with plain 'ole batch files and a mini-program I wrote to change XviD options via a command line. I'll see if I can add it to my testing toolkit. I'll check into it and get back to you.

Thanks!