XviD presets - preliminary thoughts and suggestions

[henryho_hk]

B-frame sensitivity and chroma optimizer can be specified only using the alternative -zones switch.

Thank you very much. I have updated the script accordingly. Does the syntax support a negative B-frame sensitivity value? The value does not seem to affect my test clip outputs.

[Teegedeck]

The script is updated.

BTW, I was using Didees-SixOfNine-HVS.cqm but not Didees-SixOfNine.cqm. Do the outcomes differ much?Yes, they do. The HVS-version produces an output that's less sharp and aims at slightly lower filesizes.

[henryho_hk]

Yes, they do. The HVS-version produces an output that's less sharp and aims at slightly lower filesizes.

I have reverted to the non-HVS version now.

BTW, could you please help to verify the quant. min/max settings? For i-min=2, i-max=5 and b-ratio=1.62, does it mean that p-min=i-min, p-max=i-max, b-min=i-min and b-max=roundup(i-max*1.62)=roundup(8.1)=9 ?

[tehpwner2]

Ok so where are the following settings in the xvid program??
(1st pass @q=3, curve-compr. H10, L3, quantizer-restrictions min. 2, max. 6)
I've just recently installed Xvid1.1.0 but I'm not sure what you're talking about here.

[squid_80]

Does the syntax support a negative B-frame sensitivity value? The value does not seem to affect my test clip outputs.
I'm 90% percent sure it does. I think you'd need to use a large -max_bframes setting to test properly.

[firered]

[QUOTE=Teegedeck
low&fast medium&fast strong&fast
low&norm medium&norm strong&norm
low&slow medium&slow strong&slow



[

Speed
As for the 'speed' axis, I'd suggest

line 1 could imply ME 5, no VHQ, no chroma ME
line 2 could imply ME6, VHQ=1, VHQ for b-frames, chroma ME
line 3 could imply ME6, VHQ=4, VHQ for b-frames, chroma ME
'near lossless' or 'for editing' could imply ME6, VHQ=4, VHQ for b-frames, b-frames 1/1.0/1, chroma ME and GMC + Sharktooth's UHR matrix and would be meant specifically for near-lossless encoding.
'anime' could trigger using the H.263 matrix, cartoon mode(!), a mild b-frame setting like ratio 1.0, offset=1, ME6, VHQ=1, VHQ for b-frames, chroma ME.



those presets you mentioned you use them as well for 2nd pass?

[Teegedeck]

You really shouldn't quote such a lengthy post just to ask one simple question.
The answer is: Yes, of course. Those are settings for the 2nd pass. I'm gonna change that post in order to make clear what the 1-pass settings should look like.

[BigDid]

+1

Please shorten your post (by editing it)
Even with page scrolling, I had to scroll down 5 times. For a lazy guy like me this is awful :D (this is a joke as the smiley indicates it)
Thank you.

Did

[firered]

You really shouldn't quote such a lengthy post just to ask one simple question.
The answer is: Yes, of course. Those are settings for the 2nd pass. I'm gonna change that post in order to make clear what the 1-pass settings should look like.

Thanks for showing those presets it help me out alot, the only thing that is a little confusing is that are the presets for both 2 pass or just for the 2nd pass

[shon3i]

I tested some presents for DVD backup and all work great, I think what about overflow treatment what to set, i in most cases use 0/4/9 or 0/5/5. Teegedeck can you put overflow treatment in presents. Thanks

[Teegedeck]

Alas, I have little experience with overflow settings, I always keep the defaults because I don't care much about some MB over- or undersizing. :-\

Something completely different:
I have come up with a first, still very untested way of determining the right preset to choose via compressibility-testing: It's at the bottom of the up-to-date preset-post (http://forum.doom9.org/showthread.php?p=796900#post796900). Help testing if you feel like it.

[shon3i]

Alas, I have little experience with overflow settings, I always keep the defaults because I don't care much about some MB over- or undersizing. :-\
I don't know where is that treat about overflow treatment on this forum but 0/5/5 have big and better quantizers distribution but not over/undersize instead 5/5/5. sysKin somewhere thold that but i can remeber. Anyway thanks

Something completely different:
I have come up with a first, still very untested way of determining the right preset to choose via compressibility-testing: It's at the bottom of the up-to-date preset-post. Help testing if you feel like it.Very useful. Can you put this on the first page or put to sticky.

[shpitz]

Very useful. Can you put this on the first page or put to sticky.

i second that !

i missed it myself and that is a critical piece of information right there ;-)

thanks Teege for all your hard work, i really appreciate it. it is all starting to come together for me. thanks !

[Teegedeck]

Well, this is really just testing some stuff out but good if it helps you. :)

I hope that someday we will have these or similar presets actually accessible in the GUI.

Things that still need extensive testing now are: Determining how to change the DVD-R preset (the current one oversharpens which is OK for clean sources but I'm gonna at least add another DVD-R preset with SixOfNine, optimized for sources with film-grain). Checking out whether the 'fast' presets should perhaps also get different settings for coping better with grain. Overflow treatment.

Once this has more or less been tackled I think I'm gonna wrap that post up and open a new thread.

[shpitz]

hehe.

i think you should also give some recommendations for DVB capture of some sort since this is, i think, becoming a major source, not only DVDs and VHSs.

since DVB is an mpeg2 that is highly compressed i wonder if there is something that can be done to ease the pain of watching the SD capture after seeing HD :D

[BigDid]

Hi Teegedeck,

There might be a typo in the edited clean-up:
The seven basic presets in detail again:
* normal extreme compression: VHQ=1, AQ, EQM v3 ULR, 1st pass @q=4, curve-compr. H30, L15, quantizer-restrictions min. 3, max. 5
It seems the VHQ for B-frames disappeared :confused:

Did

[shpitz]

ok Teege, i've done the following compression test:
a football match from DVB capture, 544x480 30fps.

i've done a compression test shooting for 2 cds (1400mb).
settings for the test were set just as you described, 5% extended method.

the football match is 1hr 42mins and the compression test using vdubmod took just over 6mins.

the compression test yielded 45.85% so i chose 'HQ, 1 - 2 CD-Rs: at ~15-25% of original filesize, apt for sports'.

i split the encode into 2 cds, cd1 came out around 50mb undersized while cd2 came out 105mb undersized (i'm excluding audio).

i've used the following simple script:

mpeg2source("F:\football1.d2v",iPP=true,cpu=0,moderate_h=25,moderate_v=30)
colormatrix()
LeakKernelDeint(order=1, sharp=true, forceCPU=5)
Crop(8,6,-14,-14)
bicubicResize(640,480,0,0.75)


how can i get closer to my target size? the resulting bitrate was around 1600, i would much rather have it fill the whole 700mb per cd and get much higher bitrate.

thanks

[Teegedeck]

Hi Teegedeck,

There might be a typo in the edited clean-up:

It seems the VHQ for B-frames disappeared :confused:

DidThanks for keeping a watchful eye! :) Though actually I dropped it from the 'fast' and 'normal' presets. Note to myself: add 'testing whether this really is a good idea' to my to-do list.

@shpitz: Well, the idea is that whenever you get undersizing you choose a higher preset. Thanks for the help in testing. BTW, didn't a 45% result point to using the 2-3CD preset? Of course a prediction based one comp-tests isn't an exact science. I'm gonna lower the threshold for the 2-3CD recommendation to 44% then.

[firered]

yea the presets are working for me just have tweak them a little change a value to get what im looking for but hey thanks for doing this

[neutrogenik]

I have a small question : how do you do "no b-frames" ?

[NeD tHe OnE]

I have a big problem with presets given in this thread ... "
slow extreme compression: VHQ=4, VHQ for b-frames, AQ, EQM v3 ULR, 1st pass @q=4,
curve-compr. H30, L15, quantizer-restrictions min. 4, max. 7
"

I get way undersized files .. But after making some changes like quantizer-restrictions min. 1, max. 31 .. I get perfect Video size with good quality...!

Can anyone explain?

[Teegedeck]

YES:

Firstly, you got the quant-restrictions wrong; for 'extreme' it is 'min. 3, max. 5', now.
If that doesn't solve it there is these general rules on how to use the presets:


If you get undersizing use a higher preset
If you get oversizing use a lower preset
If you get oversizing at the lowest preset, decrease your resolution

Starting out with full resolution (anamorphic, no resizing) would be a good idea.

:)

Thanks for reporting back anyway!

And BTW, quant=1 is just there to waste bits, using it doesn't make sense.

[shpitz]

Teege,

my problem is how do i approach a situation like this?

i mean, i got it undersized, how can i pre-determine what the bitrate should be in order to fill 700mb with video? is there a formula or a calculation i can make to find out?

for example, i got around 1600kbps average and the file turned out to be around 50mb short of my target. can i estimate what the bitrate should be in order to achieve the desired filesize?

also, is there a way to know that it will be undersized BEFORE i do a full encode of the source? waiting 4hrs just in order to see if i'm right on target is a total waste of time... or is it?

thanks

EDIT: also Teege, how do i set "curve-compr. H30, L15" ? not setting that could've resulted in undersized encode?

[Teegedeck]

Ahem (http://forum.doom9.org/showthread.php?p=808661#post808661).

I have a small question : how do you do "no b-frames" ?max. consec. b-frames=0

for example, i got around 1600kbps average and the file turned out to be around 50mb short of my target. can i estimate what the bitrate should be in order to achieve the desired filesize?If your first attempt is undersized just use the next-higher preset. Each higher preset can produce results that are a few hundred MBs bigger than the next-lower preset. So the second guess should always be right and it should never be necessary to encode the same video three times.


You did a compressibility check to determin which preset to use and it came out just on the border between two presets. So your first attempt could go wrong. I'm sorry i tightened those presets a little but I felt it necessary. The undersizing is a pointer that you don't get the best quality you could get with that preset.

[NeD tHe OnE]

Teege,

I am trying 2 rip 704*272 res, length 96 mins VOB file to Xvid same res (if its possible or will go for resizing ) 96 kbps vorbis [for 1CD]... and if i want to go for 2 CD then i'll choose higher preset with 6ch vorbis 300 kbps audio

Now my question is which preset should for either these and what should be the resolution?

Thank you

[shpitz]

thanks for the information, i will experiment more with it.

EDIT: also Teege, how do i set "curve-compr. H30, L15" ? not setting that could've resulted in undersized encode?


can you answer that question as well?
thanks

[shon3i]

Starting out with full resolution (anamorphic, no resizing) would be a good idea.Did you mean always to try to encode @ 720x576 only crop for example 720x400.

[Teegedeck]

Now my question is which preset should for either these and what should be the resolution?That differs from movie to movie, so there is no general answer. The naming of the presets ("1-2CD", "2-3CD"...) is just a general pointer and assumes using full resolution - more like 704x432 than like 704x272. Here's some hints:
If you get undersizing use a higher preset
If you get oversizing use a lower preset
If you get oversizing at the lowest preset, decrease your resolution

Starting out with full resolution (anamorphic, no resizing) would be a good idea. (http://forum.doom9.org/showthread.php?p=809083#post809083)I have come up with a first, still very untested way of determining the right preset to choose via compressibility-testing: It's at the bottom of the up-to-date preset-post (http://forum.doom9.org/showthread.php?p=796900#post796900). Help testing if you feel like it.I hope that helps.
also Teege, how do i set "curve-compr. H30, L15" ? not setting that could've resulted in undersized encode?Nope, all you should do is choose a higher preset. Sorry, that means doing both passes again. :( You set the curve-compression in the two-pass settings; accessible if you click on the 'more'-button next to the second pass filesize.

BTW, since when am I 'Teege'? If you have to use a short form of my nick, 'Tee' is traditional. :)

Did you mean always to try to encode @ 720x576 only crop for example 720x400.Right on! :) (Though most of the time it's something like '704x432'.) Crop away the black bars completely and keep in mind that the resulting resolution must be a multiple of 16 (even though this means cropping into the picture) in order to avoid problems with some hardware. Use XviD's aspect ratio feature and the movie will automatically resize to 16:9 on playback (i.e. to 1024x432 or something).

It's darn easy, you only ever use one of two AR settings. Either 16:9 or 4:3. I'm reffering to Pixel Aspect Ratio settings: For anamorphic movies use '16:9 PAL' or '16:9 NTSC' accordingly; for TV content use '4:3 PAL/NTSC'.
For correct playback, enable ffdshow's 'allow output format changes during playback' and 'use
overlay mixer' options in its 'output' tab.

[NeD tHe OnE]

YES:

Firstly, you got the quant-restrictions wrong; for 'extreme' it is 'min. 3, max. 5',

So , where will i find "correct" preset settings ...

[NeD tHe OnE]

And as u said "Starting out with full resolution (anamorphic, no resizing) would be a good idea." then Bits/(pixel*frame) would be less ... and by doing that the output might be of low quality

[Teegedeck]

So , where will i find "correct" preset settings ...In front of your nose! Sorry but if you want answers you have to read them. How many times do I have to link to this post (http://forum.doom9.org/showthread.php?p=796900#post796900) till you realize it?
And as u said "Starting out with full resolution (anamorphic, no resizing) would be a good idea." then Bits/(pixel*frame) would be less ... and by doing that the output might be of low qualityNo, it won't. If you mess up your encode will be oversized.

One more time:
If you get undersizing use a higher preset
If you get oversizing use a lower preset
If you get oversizing at the lowest preset, decrease your resolution

which you might also translate as:

If your bitrate is too high for the chosen quality you get undersizing
If your bitrate is too low for your chosen quality you get oversizing
If your chosen resolution is too high you'll get oversizing even at the lowest quality preset

[NeD tHe OnE]

What shd b d optimum Bits/(pixel*frame) value 4 1cd and 2 cd?

[NeD tHe OnE]

Thank You SIR

[Teegedeck]

None! Forget that bits and pixel thing. This is a completely different system. It forces you to encode at good quality, the only thing you have to do is to find out which preset fits and I described in detail how that works (either percentage of original VOB or compr. check). Edit: OK, OK, you probably could base your decision on Bits/(pixel*frame) values but please don't, it's imprecise. Rather use a compressibility check.

Edit: Where I do get your meaning, I hope is, that if you already have to use the 'extreme' preset you could think about lowering resolution even before your get actual oversizing. But in your case I'd well think about using full resolution.

Edit(2): No need for 'Sir-ing' me; but I really don't like to repeat the same things over and over. The top line of the very first post of this thread links to the valid presets and it says "Read an up-to-date version of the presets here" in shiny, blue letters. Did you mistake that for a heading instead of a link? I'll try to make that more clear. Also I've linked to them several times in the last few posts of mine. Sorry if I came across as rude.

[shon3i]

Bits/(pixel*frame) i think is only good for DivX3 SBC.

[Teegedeck]

The bits'n'pixel thing is meant for guessing a fitting resolution before/without doing a compr. check. But the guesses it allows are so unprecise that I refuse to give recommendations based upon it. Even looking at the size of the original VOBs gives you a better idea on which preset to use. It's also not really needful because you're not supposed to adjust your resolution unless the 'extreme' preset gives you oversizing.

[NeD tHe OnE]

So while we are encoding the resolution should be like this '720*420' The resulting video will be stretched bu height and while playin we can use 16:9 AR for correct AR ....

Is this good?

[Teegedeck]

Yes, actually it is what DVD-players (and DVD playback apps like WinDVD) do.

Edit: Note that the lowest preset is called '1-2CD preset'. It rarely works storing a movie in full resolution onto 1 CD with XviD at good quality! I encourage people to go for 2 or 3 CD-Rs. Compressibility checks are incredibly helpful.

[shpitz]

BTW, since when am I 'Teege'? If you have to use a short form of my nick, 'Tee' is traditional. :)


LOL, note taken. i will call you Mr. T from now on hehe

[NeD tHe OnE]

Yes, actually it is what DVD-players (and DVD playback apps like WinDVD) do.

Edit: Note that the lowest preset is called '1-2CD preset'. It rarely works storing a movie in full resolution onto 1 CD with XviD at good quality! I encourage people to go for 2 or 3 CD-Rs. Compressibility checks are incredibly helpful.

So what should we do? If we want a 90 mins movie backup only for 1 CD

[Teegedeck]

Well; do a comp. check. If it comes out lower than 30% reduce the resolution. Reduce the resolution till the comp. check comes out 30% or higher. That should about equal an average quant=4.

(Remember that the % recommendations are yet based on only few tests yet; but 30% seems a good value from all we know till now. If you want it more precise do a size prediction with Enc instead of a comp. check. Set XviD to the 'extreme' preset in Enc and set it to one-pass at constant quant=4. If that comes out to a bigger filesize than you aimed for, then you definitely have to reduce resolution.)

Edit: The 'extreme' presets allow for max. quant=5 in order to benefit from curve-compression but quant=5 should never be constant. So try to avoid getting near oversizing at the extreme preset.

[shon3i]

Teegedeck i must to ask why in this two present won't use QPEL, QPEL helps in low bitrate, i ask this becouse i wanna put DVD's at full resoulution on averge duration 1:30 hours @ 700mb.

normal extreme compression: VHQ=1, AQ, EQM v3 ULR, 1st pass @q=4, curve-compr. H30, L15, quantizer-restrictions min. 3, max. 5
slow extreme compression: VHQ=4, VHQ for b-frames, AQ, EQM v3 ULR, 1st pass @q=4, curve-compr. H30, L15, quantizer-restrictions min. 3, max. 5

[shpitz]

If you get undersizing use a higher preset
If you get oversizing use a lower preset
If you get oversizing at the lowest preset, decrease your resolution



ok, i've done a 2nd encode, this time i went to a higher preset, from originally using "slow extreme compression: HQ, 1 - 2 CD-Rs" which yielded 650mb file instead of 700mb, to the "slow strong compression: HQ, 2 - 3 CD-Rs" which yielded 850mb instead of 700mb.

1. how can i get it close to my target size if it is so unpredictable?

2. when i encode, in xvid settings for 2nd-pass, should i calculate bitrate using the calc button or should i just set a target size in the editbox to 700mb?

in the 2nd encode that yielded 850mb i used the calc function to calculate the bitrate, could that be the thing that threw the outcome out the window? was i supposed to only tell xvid 2nd pass that i want a filesize of 700mb?

thanks

[Teegedeck]

ok, i've done a 2nd encode, this time i went to a higher preset, from originally using "slow extreme compression: HQ, 1 - 2 CD-Rs" which yielded 650mb file instead of 700mb, to the "slow strong compression: HQ, 2 - 3 CD-Rs" which yielded 850mb instead of 700mb.To exclude the possibility that this is merely a miscalculation: You actually wanted the video to be 700 MBs big? No audio? So you did enter a destination filesize of 716800 KB? XviD doesn't care how you arrive at the size that you enter in the dialogue box, it just aims at reaching that figure. I would use GKnot to calculate the video size.

The target filesize in the main windows is really the only figure that counts. If XviD doesn't reach it, there's something wrong with the presets. And you can make sure by doing a filesize prediction in Enc at constant quantizer; the highest allowed quantizer (from the quant. restrictions) of a preset should always produce a smaller filesize than the lowest allowed quantizer of the next-lower preset.

Teegedeck i must to ask why in this two present won't use QPEL, QPEL helps in low bitrate, i ask this becouse i wanna put DVD's at full resoulution on averge duration 1:30 hours @ 700mb.Actually it is highly disputed at which levels of compression Qpel helps and it is also a matter of taste. The only thing everyone agreed on (up till now) was that it certainly isn't useful at very strong compression. High quantization --> motion vectors get shorter --> QPel which makes long motion vectors more precise doesn't help (also QPel itself uses additional bits, thus triggering slightly higher quantization).

Putting a DVD onto 1 CD-R at full resolution might or might not work with these quality presets. Use Enc for filesize-prediction, configure it to the extreme preset but do a constant-quantizer one-pass at quantizer=4. If the predicted filesize is too high you should not even try it and rather turn to AVC.

[Teegedeck]

Some stuff from my list.
Things that still need extensive testing now are: Determining how to change the DVD-R preset (the current one oversharpens which is OK for clean sources but I'm gonna at least add another DVD-R preset with SixOfNine, optimized for sources with film-grain).
Done. Many, many thanks to Didée for his tremendous technical insights! Without you I'd never even have tried that.
Checking out whether the 'fast' presets should perhaps also get different settings for coping better with grain.Nope, they're actually better the way they are now.
Overflow treatment.Pending.

Note to myself: add 'testing whether this really is a good idea' to my to-do list.Nope, certainly not. The speed gain is not worth the quality-loss. B-VHQ back in for all presets!

Working on wrapping up that post now...

[shon3i]

Actually it is highly disputed at which levels of compression Qpel helps and it is also a matter of taste. The only thing everyone agreed on (up till now) was that it certainly isn't useful at very strong compression. High quantization --> motion vectors get shorter --> QPel which makes long motion vectors more precise doesn't help (also QPel itself uses additional bits, thus triggering slightly higher quantization).

Putting a DVD onto 1 CD-R at full resolution might or might not work with these quality presets. Use Enc for filesize-prediction, configure it to the extreme preset but do a constant-quantizer one-pass at quantizer=4. If the predicted filesize is too high you should not even try it and rather turn to AVC.This is very interesting answer, i must ask how you backup DVD's to XviD, did you put it to 1,2,3 CD,DVD. I am nearly switched to x264 but his is so slow, i get with enc about 35% at compresivity test for most movies in 2 hours range in time, XviD for me have good picture at low rates but sometimes is whrose, i can't wait until XviD AVC come into public.

[Teegedeck]

35% sounds OK to me.

I put 2 movies onto 1 DVD-R, occasionally only one movie per DVD-R. My idea of a backup is that it should be undistinguishable from the original. So I'm quite comfortable with XviD, not much need for using AVC yet.

[henryho_hk]

The speed gain is not worth the quality-loss. B-VHQ back in for all presets!

"slow strong" is looking too similar to "normal strong".... both are at VHQ=1. Should "slow strong" use VHQ=4 too?

Can I use VHQ=1 in the first pass for all slow presets?

[trying to update my script file too....]

[Teegedeck]

Oh, I'm sorry - it seems I only ever said 'In the first pass I'd like to see ME=5 instead of =4, meaning the rest should be set up as in XviD's default first-pass. But that doesn't suffice, so as we're at it now:

First pass settings:

Same CQM as in second pass; same b-frame max. nr., ratio, offset as in second pass; AQ if used in second pass; Trellis; ME precision=5, (except for the 'DVD-R perfect' preset, where it should be '4'); VHQ=1; Turbo. A zone with constant quantizer as specified in the preset-settings.
And that's it; no QPel, no GMC, no VHQ for-b-frames, no chroma ME.

Thanks for noticing!

As for 'slow strong' --> 'normal strong'. [Well, yes, the only difference between the 'normal' and the 'slow' speed is the VHQ level. But this difference is still good for 50%-100% speed improvement for the fast setting. The idea is that the fast settings should produce almost the same quality as the HQ settings but simply at a much better speed.]
Sorry, I read your post before I had my coffee! Indeed, you have spotted a typo. Thanks a lot! :)
I'm a sloppy typer; I need your sharp eyes!

(The new DVD-R preset doesn't fit into these categories, it's dedicated to sacrifice efficiency for the sake of accurate reproduction of the source - which incidentally means it's quite fast.)

[shon3i]

I put 2 movies onto 1 DVD-R,That is about 3cd per movie, that is almost near the lossless encodig.

occasionally only one movie per DVD-R.Why should copy simply this DVD to another whitout recompression

So I'm quite comfortable with XviD, not much need for using AVC yet.
XviD is great solution becouse have near quality to the AVC, but XviD devs now stoping updating him, XviD now need small optimizations like better AQ, Skal now work on XviD AVC and i think is right time to jump to AVC when XviD AVC come up. I now working some movies with XviD and some with x264.