XviD presets - preliminary thoughts and suggestions

[Teegedeck]

Currently thinking whether for the sake of coherence the 'fast' 3-CD preset should be made a bit faster by deactivating VHQ for b-frames and whether there should be an 'HQ' 2-CD preset that uses EQM HR instead of LR (would using VHQ=4 instead of VHQ=1 plus the use of B-VHQ make up for the growth in filesize so that it would be worthwhile?). On the other hand, it already is a _lot_ of presets IMHO and one doesn't really need such high-quality settings in the middle-of-the-road area (again IMHO). Ahhhrg... I see some more dozen test encodings coming up...

I'd also like to add a max. quantizer restriction to the 1-CD presets - it's called a 'quality' preset after all, not a 'rotten' preset. ;) If it doesn't fit on 1 CD with the '1 CD preset' then you'll have to reduce the resolution, simple 's that...

Edit: And the quantizer restrictions seem to need some tightening in general.

Edit: I'm pretty sure now that after coming to terms with the things above, I'm gonna post a new thread and ask our forum members to help testing some basic but quite specific points.

[Teegedeck]

Currently thinking whether for the sake of coherence the 'fast' 3-CD preset should be made a bit faster by deactivating VHQ for b-frames and whether there should be an 'HQ' 2-CD preset that uses EQM HR instead of LR (would using VHQ=4 instead of VHQ=1 plus the use of B-VHQ make up for the growth in filesize so that it would be worthwhile?). At least the latter one can be answered with a 'yes'. EQM v3HR with 'HQ' settings produces almost exactly the same filesizes at same quantizer as v3LR with 'fast' settings. So I think there will be two presets for 'strong compression' after all.

On the other hand the 'insane' preset with EQM v3UHR is not worthwhile as it is now. I've tested on a noisy source and while it expectedly occupies roughly the same compressibility range as the medium compression presets with SixOfNine, there's no notable improvement in perceived quality. It seems more sensible to go from SixOfNine straight to using EHR without b-frames for the 'insane' preset. EHR at quant=4 (without b-frames ) yielded about the same filesize as the medium presets (with SixOfNine) at quant=2 and preserved noticeably more detail of that wretched source. Of course we're talking about compressiblity ranges that no-one really uses, here - i.e. 60% of the original DVD's size and more.

Speeding up the 'fast medium' preset seems OK, but I'm still a bit hesitant.

[Sharktooth]

UHR was tuned down to rise the compression.
The original UHR (rev 1) had 8s in place of 10s and was showing more detailed noise than 6of9 at high bitrates.
It was tested by someone but cant find that thread right now.

[Teegedeck]

As my first post doesn't reflect the changes made anymore (I kinda made a mess when I accidentally 'lost' the 'fast' setting at strong compression and replaced it with the 'medium' one on the way; and also I changed the 'normal' setting in medium compression...), here's a summary of where these presets stand now:

These presets are meant for two-pass. Of course you can also use the presets for constant-quantizer encoding but it makes little sense. The quantizer restrictions won't work then, so that quality cannot be guaranteed.

We got four categories of speed in the presets and four categories of quality/compression - though not all of them are paired. That is because we don't need 4x4=16 presets; differences would be too small and it would be annoying like heck.

compression categories: extreme, strong, medium, low
speed categories: fast, normal, slow, slower

You realize that I now decided to drop the 'fast' category entirely.

The meaning of the speed categories is:

fast: VHQ=1 [not used]
normal: VHQ=1, VHQ for b-frames
slow: VHQ=4, VHQ for b-frames
slower: VHQ=4, no b-frames, GMC


The meaning of compression categories now is:
extreme: EQM v3ULR, AQ
strong: EQM v3LR or -HR, depending on speed-preset
medium: SixOfNine, QPel
low: EQM v3EHR, QPel


actual pairings: extreme(normal, slow), strong (normal, slow), medium (normal, slow), low (slower)

So we have seven basic presets.

In addition to this we have two presets for special purposes that stand apart:

a fast, 'near lossless' preset for editing, 1-pass only: ME6, VHQ=1, no b-frames, chroma ME, EQM EHR, constant quantizer = 2
an 'anime' preset: cartoon mode(!), b-frames ratio 1.0, offset=1, ME6, VHQ=1, VHQ for b-frames, chroma ME, H.263, 1st pass @q=2, quantizer-restrictions min. 2 max. 6
The seven basic presets in detail again:

normal extreme compression: VHQ=1, VHQ for b-frames, AQ, EQM v3 ULR, 1st pass @q=4, curve-compr. H30, L15, quantizer-restrictions min. 3, max. 5
slow extreme compression: VHQ=4, VHQ for b-frames, AQ, EQM v3 ULR, 1st pass @q=4, curve-compr. H30, L15, quantizer-restrictions min. 3, max. 5
normal strong compression: VHQ=1, VHQ for b-frames, EQM v3 LR, 1st pass @q=3, curve-compr. H20, L7, quantizer-restrictions min. 3, max. 4
slow strong compression: VHQ=4, VHQ for b-frames, EQM v3 HR, QPel, 1st pass @q=3, curve-compr. H20, L7, quantizer-restrictions min. 3, max. 4
fast medium compression: VHQ=1, VHQ for b-frames, SixOfNine, Qpel, 1st pass @q=3 curve-compr. H15, L5, quantizer-restrictions min. 2, max. 5
slow medium compression: VHQ=4, VHQ for b-frames, SixOfNine, Qpel, 1st pass @q=3, curve-compr. H10, L3, quantizer-restrictions min. 2, max. 5
DVD-R perfect:ME precision=4, VHQ=3, VHQ for b-frames, b-frames 1/1.00/1.00, b-frame sensitivity=-3, SixOfNine, QPel, 1st pass @q=2, no curve-compression, quantizer-restrictions min., 2 max. 3
slower low compression: VHQ=4, VHQ for b-frames, b-frames 1/1.00/1.00, b-frame sensitivity=-3, GMC, EQM v3UHRv1 QPel, 1st pass @q=4, quantizer-restrictions min., 2 max. 4

Settings that are always active if not specified otherwise: ME precision=6, chroma ME, Trellis, chroma opt., b-frames: max. 2 consecutive, ratio 1.62, offset 0, ME precision = 5 for first pass.
Note: 1st pass @ quant=xy means that you have to set that quantizer as a zone in the first pass and have to change it back to weight=1 for the second pass.

First pass settings:

Same CQM as in second pass; same b-frame max. nr., ratio, offset as in second pass; AQ if used in second pass; Trellis; ME precision=5, (except for the 'DVD-R perfect' preset, where it should be '4'); VHQ=1; Turbo. A zone with constant quantizer as specified in the preset-settings.
And that's it; no QPel, no GMC, no VHQ for-b-frames, no chroma ME.



These are how the settings could actually be named in the config:

normal extreme compression: fast, 1 - 2 CD-Rs
slow extreme compression: HQ, 1 - 2 CD-Rs
normal strong compression: fast, 2 - 3 CD-Rs
slow strong compression: HQ, 2 - 3 CD-Rs
fast medium compression: fast, 3 CD-Rs
slow medium compression: HQ, 3 CD-Rs
slower low compression: insane, 3 CD-Rs - 1 DVD-R
anime
for editing only

And these would be tooltips:
[tooltip for the title:]100 minutes at full resolution (if you get undersizing choose a higher preset, if you get oversizing choose a lower preset)

fast, 1 - 2 CD-Rs: at ~15-25% of original filesize
HQ, 1 - 2 CD-Rs: at ~15-25% of original filesize, apt for sports
fast, 2 - 3 CD-Rs: apt for standard DVD backup at ~25-35% of original filesize
HQ, 2 - 3 CD-Rs: apt for standard DVD backup at ~25-35% of original filesize
fast, 3 CD-Rs: apt for transparent DVD backup at ~30-60% of original filesize, also for noisy sources at >40%
HQ, 3 CD-Rs: apt for transparent DVD backup at ~30-60% of original filesize
DVD-R perfect: apt for transparent DVD-to-DVD-R backup at >90% in Enc comp. check, preserves film-grain MPEG-2-style
DVD-R sharpen: apt for DVD-to-DVD-R backup (clean sources) at >50% of original filesize, also for noisy sources at >60
anime: ?
for editing only: fast, near lossless encoding, huge filesizes, NOT for archiving purposes


Hope I didn't mix anything up. Edit: Already found and corrected some mistakes.

Edit: How to choose the right preset
You can use the percentage of the original VOB-file that you aim for as a pointer. But that only works if you encode at full-resolution and the VOB doesn't contain audio. It's rather a rule of thumb.
You can also do a quick compressibility check with Enc (http://forum.doom9.org/showthread.php?s=&threadid=50714) which should take only 10 minutes or something. This could be a bit more precise. (BTW, it's a good idea to try and keep the full resolution of the source if possible. No sense in encoding 320x at the DVD-R preset...) Just start up Enc, select your .avs, choose 'compressibility check' and enter the filesize you're aiming for in kilobytes. Then you have to configure XviD. Set it to single-pass, constant quantizer=3. Load the SixOfNine (non-VHS!) CQM and set b-frames to max=2, ratio=1.62, offset=0, ME-precision to 4, VHQ=off, Trellis=on, Turbo=on and no other switches at all. (That means, it'll go damn fast...) Then run Enc. The outcome will be a percentage value. And here are pointers as to how to interpret the result:
<30%: bad luck, decrease resolution
>30%: you can use the 1-2 CD preset
>44%: you can use the 2-3 CD preset
>60%: you can use the 3CD - DVD-R preset
>90%: you can use the DVD-R preset
Attention: still not thoroughly tested; if you get over- or undersizing please report!

Edit: After further testing back to UHRv1 for DVD-R preset; also corrected max. quant. for that mode.
Edit: After obviously using the presets at for wrong target-filesizes is not unusual, here's very 'restrictive restrictions'... Try it.
Edit: The presets are hardly overlapping anymore, now. And the extreme preset can not compress that extremely, anymore, really. But at least its results cannot look that bad anymore, either.
Edit: QPel for HQ, 2 - 3 CD-Rs

[shpitz]

that is an excellent summary, thanks !

[Teegedeck]

UHR was tuned down to rise the compression.
The original UHR (rev 1) had 8s in place of 10s and was showing more detailed noise than 6of9 at high bitrates.
It was tested by someone but cant find that thread right now.But it does seem OK to use EQM v3EHR instead, keeping in mind that this is meant to represent better quality than the 3-CD preset? Kind of a 'hyper-ultra-super-extra-insane' quality that some people with big disks keep wishing for. EHR @4 seems to make for a smooth transgression from SixOfNine @2. I have only tested this on one source (Closer) as of yet, so I'm not entirely sure.

The 'old' UHR version vs. SixOfNine as contenders for the 'HQ' 2-CD preset would probably be not an easy comparison and I'm a bit weary of testing already ATM.

[BigDid]

... The only ones that REALLY create problems are Qpel, Packed bitstream and GMC.
In terms of settings, things could be more compllicated... i.e. SAP have problems with higher resolutions (over 800pixels)
Even over 640 width for some

and very high bitrate (over 5-6Mbps)
Even 4mbp/s for some

Anyway this could be a way to go. The SAP users are becoming a very big legion...:D +1

I also have read Teegedeck true reflexion about hardware profile but... it is just my reflexions aloud; Could it be possible -in term of information- or hints to display a "SAP friendly" notice somewhere like:

* normal extreme compression: fast, 1 - 2 CD-Rs
Most "SAP friendly" (custom matrix may not be playable)
* slow extreme compression: HQ, 1 - 2 CD-Rs
Most "SAP friendly" (custom matrix may not be playable)
* normal strong compression: fast, 2 - 3 CD-Rs
Some "SAP friendly" (1/custom matrix may not be playable, 2/Over 2cd may be too much bitrate)
* slow strong compression: HQ, 2 - 3 CD-Rs
Some "SAP friendly" (1/custom matrix may not be playable, 2/Over 2cd may be too much bitrate)
* fast medium compression: fast, 3 CD-Rs
Few "SAP friendly" (1/custom matrix may not be playable, 2/Over 2cd may be too much bitrate, 3/ Qpel seldom playable)
* slow medium compression: HQ, 3 CD-Rs
Few "SAP friendly" (1/custom matrix may not be playable, 2/Over 2cd may be too much bitrate, 3/ Qpel seldom playable)
* slower low compression: insane, 3 CD-Rs - 1 DVD-R
Not "SAP friendly" (GMC not playable)
* anime
Most "SAP friendly" (Over 2cd may be too much bitrate)
* for editing only
Few "SAP friendly" (1/custom matrix may not be playable, 2/Over 2cd may be too much bitrate)

All this if I understand correctly; if not other propositions are welcome.

Did

[Teegedeck]

It's simply that 'the better the quality, the more probable it is that your hardware player won't play it'! Do you really need another tooltip in order to remember that? ;)

But you can help me telling me one bit: What I thought is that hardware profiles should just deactivate features that don't comply with them. Do they do that? I haven't ever tried it as I don't own a standalone and don't intend buying one.

[yaz]

@teegedeck
1st of all, big thx for all of your efforts. i'm sure it is the way of bringing down this continuous 'how-to-set' problem.

however, having read your proposal some questions occured to me.

what's the reason of not using b-frames while using gmc at 'slower lo-comp' (very last settings) ? using b-frames will (would) increase compression and decrease speed (esp.w/vhq). so, i can't get it.

as regards gmc itself. imho, it is (can be) useful for anime but 'real movies' benefit insignificantly from it. while it drops down the encoding speed significantly and causes problems for lots of hw players. (note, in most instances, the term 'capable of playing xvid gmc' in feature lists means only that the player wont stop/jerk/hook on s-frames, nothing more)

what is the logic behind the curve-compression settings ? would u point me/us some info about it ? say, i've been never able to figure out what's the difference between H25/L15 and H10/L0. by watching it i can't see the difference. (which accords to my simple math sense) :confused:

further, 'cd-scaled compression' is a bit misleading for me. it prompts sg depending on the aim of the user not on the characteristics of the video stream. some movies may be compressed well to 1cd others may result in horrible quality at the same size. imho, using the percent (compression) scale exclusively would be more adequate here.

as i see, compression is mainly governed by cqms. wout doubting any of your selections what if cqm is not applicable (for any reason) ? remember, lots of ppl encodes for hw players, where cqm may be problematic.

stone-carving me6 prompts that (almost) noise-free sources are considered only. for noisy sources me6 is suboptimal, imho.

it's not obvious how to perform the 1st pass. would it be the same settings or sg different ? if the latter, what exactly ? (if not, why not ? ;) )

pls, take it all as my loud murmur :D (any smell of offense or negativism was unintentional).

thx, again
y

[Teegedeck]

First of all, thanks for the encouragement! :)
what's the reason of not using b-frames while using gmc at 'slower lo-comp' (very last settings) ? using b-frames will (would) increase compression and decrease speed (esp.w/vhq). so, i can't get it.Well-spotted. The overall concept of these presets is that when you choose higher presets quality rises and speed drops (except for anime & editing presets) but this is of course not always true. While b-frames slow down encoding, they don't improve quality at bitrates/filesizes where you don't really need b-frames anymore. Like, when you transcode a DVD to a DVD-R you don't need b-frames. The 'insane' AKA DVD-R preset is just what it claims to be: insanely slow, insanely high-filesize and insanely high quality. No nonsense and relentless, uncompromising quality...

as regards gmc itself. imho, it is (can be) useful for anime but 'real movies' benefit insignificantly from it. while it drops down the encoding speed significantly[...]As I wrote above, the idea is, quality goes up, speed drops. The highest preset will logically be the one that uses the slowest and highest-quality settings.
Are you sure GMC improves anime signifcantly?
[...] and causes problems for lots of hw players. (note, in most instances, the term 'capable of playing xvid gmc' in feature lists means only that the player wont stop/jerk/hook on s-frames, nothing more)You shouldn't use that preset then. Or manually deactivate GMC after selecting it. But really, if you have selected the correct hardware profile shouldn't GMC be deactivated at any rate? (And BTW, wouldn't any standalone choke on bitates of 3-5 Mbps with peaks of, dunno, 9 Mbps?)

To repeat it once again: These presets are quality-presets, not hardware-presets. Either the hardware-profiles work correctly and then you needn't worry about incompatiblilities because GMC is deactivated automatically or if it doesn't someone should fix hardware-profiles.

These presets are meant to deliver the best quality at full anamorphic resolution for the respective filesize. The filesizes to be covered range from 1 CD to 1 DVD-R. Hardware players are bound to be unable to play back files encoded at best quality settings. In this respect the very concept of quality presets conflicts with standalone compatibility. The solution to this conflict is working hardware profiles. Not the watering-down of quality presets.

If hardware-profiles don't work you'll have to work out up to which preset your standalone still plays back the results. If you have found that point, that is the maximum quality your standalone delivers and henceforth you can forget about the higher presets.

what is the logic behind the curve-compression settings ? would u point me/us some info about it ? say, i've been never able to figure out what's the difference between H25/L15 and H10/L0. by watching it i can't see the difference. (which accords to my simple math sense) :confused:I am not an expert in curve-compression usage myself but some curve-compression is almost always better than none because if you use none, if your 2nd-pass average quantizer is 3.5, the decision which frame is compressed at 3 and which one is compressed at 4 will be rather arbitrary. With CC, the smaller frame should be compressed at 3 and the bigger frame should be compressed at 4. That's the economically sane and psychovisually safe decision. It means 'bitrate' is shuffled from one big frame, where you shouldn't realize much, unto several smaller frames where you would possibly have realized a higher quantizer. The higher the flucuations in a second pass are (bigger quantizers, higher-coefficient matrices), the more important CC should grow. So while I cannot predict whether a user would use a preset at it's upper limit (high quantizers or not) I do know that the lower presets use high-coefficienty CQMs. So that's why these presets also use higher CC-values. One can also guess that the 'fast' presets will trigger higher quantizers in 2nd pass than their 'HQ' counterparts and thus attain slightly higher CC-values to them, too. The visible difference should not be much but, as said, it is better to have some CC than none.

further, 'cd-scaled compression' is a bit misleading for me. it prompts sg depending on the aim of the user not on the characteristics of the video stream. some movies may be compressed well to 1cd others may result in horrible quality at the same size. imho, using the percent (compression) scale exclusively would be more adequate here. You're absolutely right there and I don't like that naming-convention. But a user-reaction (asking whether the medium-compression HQ preset with SixOfNine was good for 1-CD encodes) prompted me to think that there are worse things than vastly inaccurate names - that is to say, giving the user a hint for correct usage that he doesn't understand and thus ignores. So I'm gonna go for the terribly inaccurate indication that at least implies clearly that higher quality will need appropriately higher filesizes anyway and I'd leave the percentages and the 'for noisy sources'-hints to the (hypothetical) tooltips. At least it is a working general pointer: If you want a 1-CD encode of a terribly noisy source the 1-2 CD-preset will be better for that than the 2-CD preset anyway. And if you use the wrong preset you get either undersizing or oversizing but NEVER bad quality... :p

Let's face it; while it's fine for us to check the filesize for some VOBs - which we made sure carry just video, not audio - the average user would possibly prefer an ogg/lamedropxp-like usability: Drag'n drop a video file onto the XviD-proggie which is preconfigured to deliver 1-CD encodes and there it goes. (Such functionality is of course more of AutoGK's or MeGUI's field, but those could use XviD presets, too, like lamedrop uses lame-presets, and it really isn't important where presets are stored.) We have to kinda of bear that in mind. And I think presets as the ones I suggest would offer better usability to those people and also to expert users who could load a preset and then just change a few settings, like usage of GMC, instead of reconfiguring everything again and again manually for different projects.

as i see, compression is mainly governed by cqms. wout doubting any of your selections what if cqm is not applicable (for any reason) ? remember, lots of ppl encodes for hw players, where cqm may be problematic.Read above, I don't concern myself with hardware-incompatibilities. If your player can't play maximum quality, it simply can't. Without CQMs there's really not so much visible difference and presets would be hardly worthwhile

stone-carving me6 prompts that (almost) noise-free sources are considered only. for noisy sources me6 is suboptimal, imho.That is a point I'd really like to delve into. There are voices that recommend ME=4 but I'd like to test that personally in 2-pass thoroughly before I'd accept the enormous waste in filesize/bitrate that this would ensue. If it proves to be worthwhile, ME=4 will have its place in the 'fast' presets that are recommended (in the hypothetical tooltips) for noisy sources (fast 2-CD preset, fast 3-CD preset).

it's not obvious how to perform the 1st pass. would it be the same settings or sg different ? if the latter, what exactly ? Presets would only make sense if the same CQM is used in both passes . And the first pass would be performed at the recommended quantizer, of course. Other than that, fast first pass switches off a lot of things as it is now, too. But I would like fast first pass to use ME=5 instead of 3 because it is much more accurate in filesize-prediction and not so much slower.
pls, take it all as my loud murmur :D (any smell of offense or negativism was unintentional).Thanks again for your interesting comments/questions and the encouragement! You're welcome.

Edited: Lots already. Have you ever tried to edit posts on an ancient computer running Win98? Don't do it, it's hell. I don't know why but some of my edits are gone. And gone again. And again. I have no idea why. I have written the whole darn post at least three times over now! Edit: And now I see why - apparently instead of editing it posted the post anew two times...

[shpitz]

lol, that's why when i have something long to write, i write it in notepad 1st, then after i'm done i paste into the reply box.

[Teegedeck]

Come to think of it, I used to do that, too, when I still had Win98 on my own PC. :) Nowadays I tend to forget such security measures, I just keep Firefox open for days and weeks and haven't lost anything for as long as I can remember.

[BigDid]

... What I thought is that hardware profiles should just deactivate features that don't comply with them. Do they do that? ...
Well it was not the case end of 2005, I remember big trouble for AGK user's when len0x had updated Xvid and the theatre profile kept packed bitstreams instead of removing it, I think it is now corrected but this is one recent example.

Mtk users are lucky because Celtic-Druid is, quite regularly (2 or 3 times a year IMO), updating his MTK and MTK 6000 profiles but it is programming instead of just creating a preset and re-using it.

However, I appreciate the efforts you are making to promote the ease of use for the presets. I read a lot of people have enthusiasm for this project as I do. I believe the actual questions or demands are just reflecting this enthusiasm.

When I will get out of my survivor situation (got a major overvoltage, actually running with an old small Power supply and an old graphic card) I will also try using/testing these presets to be able to speak of concrete experience instead of great notions or ideas (I'm targetting myself only :) )

Continue the good work.

Did

[elenhil]

Sorry for not being able make it out from all you've written here myself: do quantizer-restrictions min./max. apply to I- & P-frames or to I- & P- & B-frames?

[Teegedeck]

Well, yes; but b-frame max. quantizers are actually ('given value' x ratio) + offset . It's not really a very meaningful restriction but more of a safeguard; except if you'd set it lower than the max. p-frame quant...

[elenhil]

What about "anime" preset - can it be optimized for various speed/quality settings or there is really no point in applying different matrices, quantizer-restrictions etc. to such a content?

[Teegedeck]

Most anime-centred people here swear by H.263. Of course in the higher-bitrate areas one could use sharper matrices. Didee once was so nice to send me a draft of one. But you need a higher bitrate with such matrices in order to avoid ringing. My impression was that anime fans don't go for high-bitrate encodes. If there's demand and someone to test all that, it could be done. But I don't feel like it, I'm still not finished comparing fast modes. Perhaps we should have a dedicated film-grain-preserving mode, that's what I'm concerned with ATM.

[elenhil]

In fact, I was wondering if there could be lower bitrate settings for anime (well, not anime proper, but for cartoons in general). Something closer to "extreme compression" profile.

Anyway, here's another question: what settings from these profiles should be used in compressibility check?

[micmac]

Hi guys,

I'd like to have an IPod Video profile as well :)

Why? With xvid 1.0.3 I can just choose Simple Profile L3 and the codec still uses my bitrate and resolution. Say 1000kbit/s and 480x368. With xvid 1.1 it doesn't use my bitrate anymore, it uses ~340 no matter what.

Now I'd have to use AS5 or something and this can be a pain because I don't really know what options the IPod accepts :(

So yes, I really'd like an IPod Video profile.

Thank you!


mic

[Teegedeck]

Please read the whole thread. This has been talked to death already.

[elenhil]

It is not addressed to me, is it? My question was what settings from these profiles should be used in a compressibility check? Found no discussion about it in this thread.

As I understand it, owing to quantizer-restrictions these presets limit the resulting filesize themselves (giving under-/oversized results when the content exceeds the limits of a particular matrix' applicability). But it is a rather long way to find that you should have used another preset. Compressibility check can help, or so it seems, but what settings should one use for it? Those for the 1st pass? How exactly do its results (% value) correspond to the possible under-/oversizing?

[Teegedeck]

Ah, sorry, this was adressed to micmac. (micmac, I'm sorry if hardware-profiles don't work anymore - someone should really fix it then. How about posting to the XviD mailing-list?)

elenhil, use the first-pass-settings given for the preset, yes. As I never do compressibility-checks but only fixed-quantizer-based filesize predictions, I cannot give you precise recommendations as to a percentage that you should aim for. Well, ideally it should be 100%. :) The easiest way is a fixed-quantizer prediction with Enc at the one-but-highest quantizer. If such a filesize predictions exceeds what you are aiming for, then use the next-lower preset. The other way round, if a prediction at the minimum quantizer gives less filesize than you planned for, use the next-higher preset. Certainly one could work out a compressibility-check which would vaguely point to the correct preset to use in one go - and really, I have already thought of that - but, well, I haven't started on finding out how to set this up.

At the moment, just have a look at the recommendations as to 'percentages of original (VOB) filesizes' that I've given. These are rough, yet should be basically reliable. Take a look at the VOBs you want to compress (VOBs that don't contain audio, of course) and calculate at how many percent of their combined size you're aiming.

Like, the 2-3 CD preset being "apt for standard DVD backup at ~25-35% of original filesize". At full resolution (no resizing, just cropping) is always implied

[henryho_hk]

is it "1st pass" or "one pass" ?

[Teegedeck]

OK, sorry that I didn't spell that out: If you want to use these presets, you cannot do a first pass at the usual XviD defaults. Most importantly, you have to use the CQM according to the preset, you should use AQ if AQ is mentioned for that preset and you have to set a zone with a constant quantizer that I mention as '...first pass @ quant=....'. Apart from that I'd say use no XviD feature but Trellis and VHQ=1; set ME precision to 5, using Turbo seems a good idea.

[henryho_hk]

Trellis and VHQ=1; set ME precision to 5, using Turbo seems a good idea.

Actually, I am writing a VBS script to automate all these with the help of the wonderful avs2avi program. Also, the script will insert keyframes at chapter point (by zones) for subsquent OGM & MKV muxing.

Hence, I would like to know the proper way to instruct XviD to use q=3 for all zones for the 1st pass. Is it sufficient to specify the min/max I-quant values? Or must I specify the quant mode for all the zones in the 1st pass and use weight mode in the 2nd pass?

EDIT ---> I just discovered the wonderful xvid_encraw utility! All can be done on a single command line now! No need for any VBScript! For example:



for %%i in (*.avs) do (
start "Pass 1" /b /wait /low xvid_encraw.exe -i %%~ni.avs -type 2 -avi %%~ni_p1.avi -max_bframes 2 -bquant_ratio 162 -bquant_offset 0 -bitrate 1350 -vhqmode 1 -bvhq -qpel -qmatrix Didees-SixOfNine-HVS.cqm -nopacked -imin 2 -imax 5 -bmin 2 -bmax 8 -pmin 2 -pmax 5 -progress 300 -pass1 %%~ni.stats -turbo -quality 5 -zq 0 3
start "Pass 2" /b /wait /low xvid_encraw.exe -i %%~ni.avs -type 2 -avi %%~ni_p2.avi -max_bframes 2 -bquant_ratio 162 -bquant_offset 0 -bitrate 1350 -vhqmode 1 -bvhq -qpel -qmatrix Didees-SixOfNine-HVS.cqm -nopacked -imin 2 -imax 5 -bmin 2 -bmax 8 -pmin 2 -pmax 5 -progress 300 -pass2 %%~ni.stats -quality 6 -chigh 10 -clow 3 -zw 0 1
)
pause

[Teegedeck]

The min. and max. quantizers (=quantizer-restrictions) work only for the second pass, for the first pass you need to specify the constant quantizer by using a constant quantizer for zones. And in the second pass, it's 'weight=1.00' in the zones, yes. :)

[henryho_hk]

for the first pass you need to specify the constant quantizer by using a constant quantizer for zones.

I found XviD (1.2 packaged w/ AutoGK 2.27) drops to Q=2~3 after encoding for 30 minutes in the 1st pass even when I put it in constant q=3 mode in all zones (about 37 zones). Any hint?

[Teegedeck]

Nope, I've never encoded with that many zones (mostly only two) and I've never used AutoGK. I really don't know whether AutoGK allows you complete control over your 1st-pass settings.

[henryho_hk]

No no no... I used the XviD version bundled w/ AutoGK 2.27.... I did not encode through AutoGK. I used avs2avi.

EDIT -> I have switched to xvid_encraw.exe

[Teegedeck]

Oh, sorry, you mentioned that above. Perhaps a slip in the commandline? I'm not 'fluent in XviD CLI' myself...

[henryho_hk]

Oh, sorry, you mentioned that above. Perhaps a slip in the commandline? I'm not 'fluent in XviD CLI' myself...

There is no such problem after I switched to xvid_encraw (which does support 37 zones :cool:).

In addition, all these presets use 2-pass encoding. What bitrate values can I use for extreme, strong, medium, etc. compression?

[Teegedeck]

Bitrates are a sham, don't trust them buggers! :)

Please inform yourself about the relation between content-type, resolution, fps and filesizes.

[henryho_hk]

content-type, resolution, fps and filesizes.

As such, how many minutes for a 2 CDR backup of a typical movie is considered an "extreme" compression?

[Teegedeck]

Ho-humm - how about actually reading the first post and its upated version before asking such questions? It's all there you just need to read thoroughly.
http://forum.doom9.org/showthread.php?t=107897
http://forum.doom9.org/showthread.php?p=796900#post796900

Edit: OK, the 'minutes'-pointer was missing from the 'updated post'. Still, it's only a sham. All settings are for a purely virtual, non-existing average movie of 100 minutes. A 70-minute-movie may come out larger at the same quantizer than a 120-minute-movie, these are all guestimates. The presets should be quite tolerant. For example, the 3CD preset is allowed to use quantizers from 3-5; this probably means a filesize varying between about (again, for a virtual, non-existing average movie) 2.5 GB and 1 GB. If you hit the sweet spot of usage that it's really meant for it's quant 3 or 4, meaning perhaps 2.5 to 1.5 GB. If you could turn that around, that may well mean you can use this preset to put an average 70-minute movie on 3 CD-Rs as well as an average 120-minute-movie and it would certainly never look bad.

The presets are designed to let you choose a setting by using your intuition. It's doesn't need exact calculations and lets you get away with this huge inaccuracy. You still get well-tuned settings. All this is only possible because XviD doesn't really need to be meticuously tuned for each different movie. All that should be improved over the standard preset courtesy of sysKin is a few things to optimize quality and a safeguard that prevents utter crap.

[henryho_hk]

I swear that the minute figures were not there :( :( :( :(

[Teegedeck]

Well, the figures in the second post weren't there because, as I said, I had forgotten them. :)

[henryho_hk]

I have drafted a batch file implementing the presets. Please enjoy.

Unfortunately, i dunno how to obtain the movie length in commandline. Hence, I am still using bitate as a parameter. The bitrate is for reference only. The file will still be undersize or oversize due to those tight quantizer restrictions.

====> Updated 20060531: corrected the -bmax parameters
====> Updated 20060327: now you can specify the target size in M or in K.
====> Updated 20060406: updated settings & dvd_r preset
====> Updated 20060409: updated settings; presets added: edit_1pass, anime_2pass, dvdr_2pass and (my own) dirty_1pass; all seem to be working
====> Updated 20060409b: fixed a bug in the -zones parameters
====> Updated 20060410: slower_low renamed to dvdr_sharpen; some settings updated
====> Updated 20061004: obsolete old code deleted

Usage:


Save the BAT as, e.g., xvid1xenc.bat
Put it in the same directory as, xvid_encraw.exe, Didees-SixOfNine.cqm, eqm_v3hr_rev1.xcm, eqm_v3lr_rev1.xcm and eqm_v3ulr_rev3.xcm
Command-line is "xvid1xenc.bat <method> <bitrate>", "xvid1xenc.bat <method> <size>K" or "xvid1xenc.bat <method> <size>M"
For <method>, normal_extreme, normal_strong, slow_strong and slow_medium are available. Default is slow_strong.
For <bitrate>, use the defaults or specify one yourself.
To encode, put the avs file (say, input.avs) in the same directory; if you want to have keyframes in chapter points, put a plain text file (say, input_zones.txt) listing the frame numbers on its own lines. After that, run xvid1xenc.bat and wait.
When it finishes, your CPU load goes down to zero. :D

[boombastic]

How can i use this batch file?Should i put it in the same folder of xvid encrew?How can i tell the source directory?Sorry if i missed something...

[Teegedeck]

Why don't you put it into your encraw directory and try it out, I'm curious myself! :)

BTW, @henryho_hk::thanks:

[squid_80]

Nice batch file. :) But what exactly do you mean by this:
Unfortunately, i dunno how to obtain the movie name in commandline. Hence, I am still using bitate as a parameter.?

[JoeBG]

Very nice skript :)

Maybe two remarks:
-nopacket is no longer needed
-vhqmode 1 is not neede, 1 is default

One question:
-quality 5 for the first pass, why?

[squid_80]

-nopacket is no longer needed
Not true, packed mode is on by default with avi output.
Unfortunately, i dunno how to obtain the movie length in commandline.Ok now I get it. There's a program somewhere around called avsutil which takes an avisynth script or avi file as a parameter and returns the number of frames in %errorlevel%. Might be useful, if you know the framerate of the material being processed.

(Sorry for getting slightly off-topic.)

[Teegedeck]

Sorry, henryho_hk, just changed those settings... :D

For the DVD-R preset the orginal idea of using EQMv3 UHRv1 with b-frames 1/1.00/1.00 and sensitivity=-3 is more realistic for not-so-clean sources after all. Also, a max. quant. of 5.

[henryho_hk]

For the DVD-R preset the orginal idea of using EQMv3 UHRv1 with b-frames 1/1.00/1.00 and sensitivity=-3 is more realistic for not-so-clean sources after all. Also, a max. quant. of 5.

Why do you use UHRv1 but not UHRv2?

And since I can't find b-frame sensitivity option in xvid_encraw, your "slower_low" preset is not implemented. :P

BTW, squid_80, I have 4 questions on xvid_encraw.exe

Is there a "discard 1st pass" option?
How can I enable the chroma optimizer option for zones?
Is there a b-frame sensitivity option for zones?
Should I use some -qtype value for custom matrices?

[squid_80]

BTW, squid_80, I have 4 questions on xvid_encraw.exe

Is there a "discard 1st pass" option?
How can I enable the chroma optimizer option for zones?
Is there a b-frame sensitivity option for zones?
Should I use some -qtype value for custom matrices?

To discard 1st pass, simply don't specify an output file (drop the -avi switch).
B-frame sensitivity and chroma optimizer can be specified only using the alternative -zones switch. For the format see this post: http://forum.doom9.org/showthread.php?p=702759#post702759
It should be possible to use multiple -zones switches instead of separating them with a /.
Older versions required -qtype 1 in addition to -qmatrix <matrix> in order to work correctly. This has been fixed since then. But it would probably be better to be safe and add it to the command line anyway.

[Teegedeck]

Why do you use UHRv1 but not UHRv2? Because v1 yields results that are hardly different from SixOfNine encodes. v1 produces larger files but there is a visible difference. So it fits for a DVD-R preset. EHR produces files too large for even the DVD-R preset.

[henryho_hk]

B-frame sensitivity and chroma optimizer can be specified only using the alternative -zones switch.

It is possible to omit the end frame of zones in this switch? Getting the end frame is a bit too complicated for this simple batch, especially since I can't find anywhere to download "avsutil.exe" now. Also, XviD 1.1's configure panel/registry does not require specifying the end frames.

[squid_80]

avsutil. (http://forum.doom9.org/showthread.php?p=798368#post798368) I haven't tested this build though, I compiled it myself using the source code given in the same thread.
Sorry I forgot I dropped the end frame parameter for zones, it is meant to be left out.
A zone will run until the start of the next zone or the end of the movie, same as in the vfw config.

[Teegedeck]

Very nice skript :)
One question:
-quality 5 for the first pass, why?Sorry, forgot about that question. ATM ME precision 4 is XviD's default for 1st pass AFAIK which is not much different from ME precision 3. This is superbly fast but delivers very inaccurate results in terms of filesize-prediction if you use VHQ etc. in 2nd pass. An unprecise prediction results in quite unnecessary quantizer-fluctuations in 2nd pass. The general recommendation is to use a full-quality first pass if one aims for highest quality. But that is of course much slower than it needs to be. Especially if your full-quality first pass includes VHQ=4. I think ME precision 5 is a good compromise, much more accurate and still much faster than a full-quality first pass.

BTW, I just might radically revamp that DVD-R preset and this might include using ME precision=4... In such a case the first pass should also be done at 4, of course.

[henryho_hk]

The script is updated.

BTW, I was using Didees-SixOfNine-HVS.cqm but not Didees-SixOfNine.cqm. Do the outcomes differ much?