"Safe" settings for XviD

[BluDChyLD]

I've been using DivX for years now, but since XviD has become final I'm giving it a try :)

After surfing the forum, reading Doom9's guide and some FAQs I've conluded that these are the safest settings to use. I use the phrase "safest" as I do not want to risk artifacts in my encodes! Can a more experianced XvID user please verify that these settings are ok and reliable to use, thanks!

Chroma Optimizer in Zone Options
Croma Motion
Turbo Mode ;)
Tellis (at low bitrates)

Contrary to Doom9's guide, I have read that unless you using a third party decoder (DivX, ffdshow) you should turn packed bitstream off. Is this true, as it is XviDs default setting to have it on?

[Teegedeck]

Safe? I don't know what can be called 'safe' with a video codec - it won't explode, you know? :)

More seriously, some weeks ago I was quite suprised that even my oldest XviD encodes played back fine. These are so old that I had set their fourccs to 'divx' in order to let the DivX4 dshow filter decode them - because back then XviD didn't have a decoder of its own nor did ffdshow support an XviD-fourcc.

The worst thing you could ever do with XviD was to use a resolution that wasn't a multiple of 16. And even for that there are workarounds. The only thing you could ever do to seriously compromise MPEG4-compliance was to use 'modulated quantization'. But it's no problem to play back such files with XviD or ffdshow.

Packed bitstream is something you should use if you want your video to be played back by a standalone or want to edit it heavily. Turbo mode is faster (only IF you use quarterpel and/or b-frames) but may be a slight bit less hq; nothing good comes without a price. Trellis; chroma opt.; chroma motion - yes, it's safe. I'd even encourage you to use adaptive quantization and quarterpel and the hvs-best-picture custom matrix and b-frames and gmc (not if you use one of those ever-so-restrictive standalones for playback).

The bottomline of my sermon is, be a little daring and enjoy all these advanced-simple-profile XviD features! The worst thing you could get is sub-optimal quality; but quite the opposite you will get.

Edited: 'turbo' speeds up quarterpel, not adaptive quantization!!

[bond]

Originally posted by Teegedeck
Packed bitstream is something you should use if you want your video to be played back by a standaloneit heavily depends on the standalone, some player b0rk on packed bitstreams
packed bitstream, gmc, qpel and custom quants are the options which should be very carefully used when aiming at playing in standalones
some players support them, some not

Turbo mode is faster (only IF you use adaptive quantization and/or b-frames) but may be a slight bit less hqafaik it also speeds up when using qpel (which is a must imho), but yes it will hurt quality a little bit

[SeeMoreDigital]

Originally posted by bond
it heavily depends on the standalone, some player b0rk on packed bit-streams Yes true. There is still much to be resolved about this issue.

Some stand-alone player users who generate packed bitstream encodes with more than 1 b-frame are ok, some are not. Some users who generate non packed bit-stream encodes with more than 1 b-frame are ok, some are not

Some say it's XviD's fault, some say it's the players firmware... either way there is a cock up, which is slow at being resolved.


Cheers

[BluDChyLD]

Thanks for the help! As I encode overnight, speed isn't much of an issue so I'll keep the turbo ;-) mode off.

I've been playing around with adaptive quanitisation and it does a good job on reducing file size, I assume it is similar to DivX's Psychovisual Enhancement? I looked at Crusty's FAQ and it advises against AQ as it has not been extensively tested - but he is refering to the betas and RCs not the final. Is it more reliable now?

[Teegedeck]

Yes, it is. I haven't seen a single artifact caused by AQ since before the RCs, and that's been a long time now. Use it, it gives an overall plus in quality as it lowers the mean frame quantizers being used (NOT for use on anime, as said before).

Adaptive quantization works by using slightly stronger compression for macroblocks in areas where the human eyse won't notice it. Judgement is made upon relative luminance (frame-based) and it works amazingly well. Actually quantizers of individual macroblocks only go up by one most of the time, and it isn't used on all frames, either. Still, the effect is remarkable.

[bond]

Originally posted by Teegedeck
I haven't seen a single artifact caused by AQ since before the RCs, and that's been a long time nowgreat to hear that AQ is absolutely safe to use now, i myself wasnt sure as i didnt really test it for some time now

btw last time i tested psych in divx5, it seemed to me to be more like a smoothing filter, not really usable imo

[Teegedeck]

:eek: Gahhh, just realized that terrible mix-up in my post, that you quoted there, bond. I meant to say 'turbo' speeds up quarterpel, NOT adaptive quantization!! Where was my mind? Sorry, I just shouldn't post at 3 AM.

[BluDChyLD]

thanks for verify that for me Teegedeck, much appreciated :) What your opinion on using Trellis, my tests havent shown any improvments but I'm guessing it will be good for low bitrates?

[Teegedeck]

I blindly trust sysKin here. Its initial versions were better for low bitrates IMHO; but since Trellis has been improved so much since then I now use it for my high-bitrate stuff, too. I'd be interested in picture-by-picture comparisons though if anyone wants to do them.

[BluDChyLD]

Hmmm... Don't have time at the moment but I'll see what I can do on the weekend! What kind of source material/shots do you recommend using for testing it?

[Teegedeck]

Clean (good DVD for example), well-lit shot (good contrasts) with the camera focussing on an actor's face. I guess our eyes most easily detect differences when looking at skin-texture on faces - no wonder...

[SeeMoreDigital]

Originally posted by Teegedeck
Clean (good DVD for example), well-lit shot (good contrasts) with the camera focussing on an actor's face. I guess our eyes most easily detect differences when looking at skin-texture on faces - no wonder... I think we could do with a good short source reference vob file!

I'll have to have a look...


Cheers

[BluDChyLD]

Ok, I'll have a look aswell :) Got some nice clips from American Beauty to use - but there aren't many close ups, but lots of complex motion and colour. I'll test them when I have time and let you know.

[jon.schaffer]

Originally posted by Teegedeck
... I haven't seen a single artifact caused by AQ since before the RCs, and that's been a long time now. Use it, it gives an overall plus in quality as it lowers the mean frame quantizers being used (NOT for use on anime, as said before).

... it works amazingly well. Actually quantizers of individual macroblocks only go up by one most of the time, and it isn't used on all frames, either. Still, the effect is remarkable.


I agree concerning the overall advantage when using AQ. Most of the time the quality loss is invisible. But, I encountered many movies with only 2 or 3 scenes in which AQ led to blocking. These scenes obviously present some frames with already high quantizers (high motion), and the AQ increased even more these quantizers => blocking (which is usually very rare with XviD).

I anyway use AQ because it makes more good than bad. But this makes me answer:
could the zone options include the AQ setting in future releases? Or is it impossible to imagine this, due to the way AQ works? ... that way, we would be able to disable AQ in some particular scenes? (it seems to me that the resulting loss of bytes would be negligible face to the entire movie duration)

I'd maybe better post in the suggestions post...?:confused:


Jon

[Teegedeck]

Can you do a comparison sometime and post screenshots of some frames from the same video with and without AQ?

[jon.schaffer]

@Teegedeck

I guess the test request is for me?...
Well, yes, there's no problem. I'll make it ASAP.
I'll be able to post next week (university closed tomorrow - public holyday).

A junior member question (I've never posted with joint pictures):
I guess I must find a host for the pictures? (if yes: could you recommend me any site?)
And, in the end: is low-compressed-JPEG good or PNG absolutely needed?
Thanks...


Jon

[Teegedeck]

There is for example imageshack.us or photobucket.com. Lowly compressed jpegs are fine. You could/should make it a new comparison-thread and perhaps leave a redirect here. Thanks for making the effort; I myself have little time. Why do I post - I have NO time!! :cool:

[Koepi]

Originally posted by jon.schaffer
[...]And, in the end: is low-compressed-JPEG good or PNG absolutely needed?[...]

yes, absolutely! imagine the blocks introduced by mid- or high-range jpeg compression. How would you separate them from the blocks you want to show us?

Regards
Koepi

[Teegedeck]

Hm, in the 'Matrix HQ encode' thread and others, JPEG seemed fine (at 90% quality still much smaller than PNG and no compression artefacts). If it's gonna be PNG, perhaps a zoomed detail of the frames would do in order to keep size down.

BTW, for comparison I'd suggest encoding a reasonably small portion of video in two-pass to identical filesizes (would be most fair).

[Soulhunter]

Originally posted by Teegedeck
I'd be interested in picture-by-picture comparisons though if anyone wants to do them. Ok, but after my 2nd CQM comparision... ;)

Is next weekend a adequate date ???


Bye

[SeeMoreDigital]

I think I might found a good source!

It's actually one of the original 'This is DVD' trailers. It weighs in at around 45MB (including AC3 audio).

Does anybody have any spare web-space. I don't mind hosting small files 5MB max on my home server but this whopper will slow my ISP connection to zero when people come to downloading it!


Cheers

[jon.schaffer]

About Adaptative Quantization...

Originally posted by Teegedeck
You could/should make it a new comparison-thread and perhaps leave a redirect here.

OK, I opened a new thread:
http://forum.doom9.org/showthread.php?s=&threadid=76741


Jon

[Alxemi]

Does anybody have any spare web-space.
I can host the files for you :). send me the files to alxemi in hal9000 dot eui dot upm dot es

[crusty]

Nice info about AQ, I'll alter it if my moron webadmin allows me to upload a newer version of the FAQ.....

AFAIK, all options are now safe to use, at least as long as you decode with Xvid.

It's just when decoding with other stuff that you have to start worrying about compatibility.

DivX decodes only one consecutive B-frame, and doesn't handle XviD's GMC, which is more advanced.

Standalone decoders don't work with many advanced features of both DivX and XviD, and they usually don't eat Ogm's and MKv's, only avi.
Use only mp3 or ac3 sound with those files, not wma or Ogg Vorbis.

As a rule of thumb, XviD options that only affect the encoding proces without altering fundamentals of the video stream work fine.

Examples of fundamental changes:
-Qpel adds another bit for every moving block, so it alters the video stream in a fundamental way.
-B-frames are different types of frames, that can't be handled like I- or P-frames without additional firmware.

Examples of non-fundamental changes:
-Adaptive Quantization alters quantizers in certain blocks according to a psychovisual model, nothing fundamental is altred in the video stream
-VHQ does some smart brute force optimization on how to find motion, but the end effect is completely transparent to any encoder wheter you used it or not.

[BluDChyLD]

I switched to the divx decoder a few weeks ago because it's a lot better than XviD or fddshow at hiding the artifacts in dark areas on my tv-out.

DivX's decoder only decodes 1 consecutive b-frame? That would explain the problem i've been having with jerky playback on some footage! Saying that though, the problem isn't that bad - I only noticed it on 1 out of the 10 xvid films i've watch. I was wondering what was up with it... Cheers for the info!

[lamer_de]

DivX decodes only one consecutive B-frame,[...]
I just decoded a file with 8 consecutive b-frames (at least that were my settings (8/1.0/1.0, sensitivity 1000), I counted at least 6 consecutive bframes in the "show me the internals" window) with the DivX 5.1.1 decoder without problems, so I guess at least the decoder can handle more than 1 bframe.

Anyody knows about standalone compatibility with such huge amounts of bframes? Or is it safe to assume that if a standalone decodes xvid with 2bframes it'll be able to decode any amount?

CU,
lamer_de

[Alxemi]

take a look here (http://forum.doom9.org/showthread.php?s=&threadid=71554)

[lamer_de]

Thanks. I should've used the search button :o

CU,
lamer_de

[Jawor]

The DivX 5.1.1 Decoder plays XviD content with more than 1 B-frame fine if it doesn't contain packed bitstream. XviD's Qpel is also decoded fine, only XviD's GMC causes artifacts (I was curious, so I just tested some clips encoded with 1.0.1).

[Alxemi]

Then, packed bitstream is completely useless now, isn´t?
AFAIK, the only decoder that needs Packed Bitstream is the sigma hardware chip.

[lordadmira]

You need packed bitstream for Virtual Dub. It also slightly speeds up decoding.

[Mug Funky]

i think 8 b-frames should probably be for testing purposes only.

if you watch the bitrate graph in virtualdub while encoding, you see that the further apart the p frames are, the less efficient the intermediate b-frames will be. 2 or 3 is usually best unless you've got an almost entirely static clip (in which case you'll get codec saturation no matter what settings you use).

it would be cool to have b-vop sensitivity expanded to allow max and average b-frames, so in regular video it would be 2 b-vops, but in a near-static scene it could be up to max b-vops (maybe 8?). probably not worth it, as the stream will mostly be skipped blocks no matter what kind of frame it uses.

[BluDChyLD]

So, if most of the hardware players can handle xvid files without packed bitstream what purpose does it serve now? I haven't had any problems using vdubmod with my non-packed bitstream xvids, except for the "b-frame decoder lag" message. But I've have encoded and split files with no problems.

[lordadmira]

Originally posted by Mug Funky
you see that the further apart the p frames are, the less efficient the intermediate b-frames will be. 2 or 3 is usually best unless you've got an U can't make that kind of categorical statement about B frame usage. It would be true if there were fixed numbers of B frames but Xvid inserts them dynamically. I've tested it at 28/1/0 sens 80 and Xvid did not choke! It put in sensible numbers of B frames and the frame sizes confirmed the efficiency of the frame decision. Those are my default B frame settings for all encodes now. If u are using B frame quant offsets then obviously u don't want more than 3 or 4. I haven't tested these settings on any standalone boxes.


LA

[Teegedeck]

Lordadmira, shouldn't you mention that you encode anime? I believe this is a pretty special case, and most other users here encode natural video for which I would rather not see your settings advertised as a standard.

[malkion]

Originally posted by Teegedeck
Lordadmira, shouldn't you mention that you encode anime? I believe this is a pretty special case, and most other users here encode natural video for which I would rather not see your settings advertised as a standard.

I'll second that!!

The image of such a video for natural film introduces a fragileness I've never seen before. The images seems almost transparent that it might shatter, were it not just an image. I believe for a quality xvid encode, b-vops @ 2 should be the highest, 3 for intensive usage, just as recommended.

edited

Cuz I tested it and I just didn't like it, and I dont think with 2800+ views, settings at 28/1.00/0.00 should even be mentioned as standard usage, as Meg Funky agrees that just 8 b-frames should be for testing purposes only. :rolleyes:

[Teegedeck]

Wouldn't a simple 'I've tested it and didn't like the result' have been enough?

[malkion]

Originally posted by Teegedeck
Wouldn't a simple 'I've tested it and didn't like the result' have been enough?

You're absolutely correct Teegedeck... I had only honest intentions with my original analogy, however.

2 b-frames at default b-vop settings will normally generate slightly over 3 b-frames out of 5 frames, correct? 3 b-f's recommended for intensive usage. I can certainly see some people always going for more with say 4-5 b-frames and tweak sensitivity to their liking, however, 28 bframes will only produce slightly over 4 b-frames out of 5. It's definitely an abnormally high setting. Overkill.

[lordadmira]

I also encode live material. I've encoded live TV material and music videos at 8/1/0 sens 80 and the results have be just fine. I don't know why this idea persists that the Bmax setting means that there *are* going to be that many. People need to realize that the codec does not ignorantly put in B frames willy nilly. Setting Bmax at 1000 will not result in more B frames than there *should* be. Setting Bmax high just gives the codec the *flexibility* to put B frames where it sees fit.

[Teegedeck]

So let me tell you what makes me hesitate to let your b-frame settings go uncommented. As you correctly mention, setting a higher maximum for b-frames doesn't mean XviD will always use that number of b-frames. In fact XviD will rarely use more than 3 IF you leave b-vop sense alone. Using a higher maximum number without raising the b-vop sensibility won't give you much advantage in compressibility but it slightly increases the possibility of XviD erring (that happens, you just tend to overrate XviD's capabilities) and actually inserting a too long row of b-frames where it hurts. Slight risk, no real harm done.

IF on the other hand you also raise the sensibility for b-vops you actually should get more b-frames. No matter whether you get them in a row or in places where you wouldn't get them at all with a lower b-frame-sense, you'll get more b-frames. I bet you that IF you'd left your other b-frame settings (ratio, offset) at defaults, this would look worse.

But you've also set your quantizer ratio to 1.0 and your offset to 0. That way there'll hardly be a significant difference in size between b-frames and p-frames. Encode the same file with and without b-frames to verify, just eyeballing the framesizes doesn't tell much. Also b-frames look a little worse than p-frames because there's no VHQ for them. And not only that. Let me quote sysKin from here (http://forum.doom9.org/showthread.php?postid=501026&#post501026):
B-frames at the same quant are not what bitstream expects, so the compression is much worse. In the end you don't gain anything with your b-frame settings but probably loose something.

Also keep in mind that the more b-frames in a row occur the harder it is for the decoder.

Again, with your settings there`s no real harm done. But why risk loosing something if there's no chance of gaining anything?

[lordadmira]

I don't have any problem with asterisks put next to settings I recommend, or anybody's, so long as they're correct and not biased. There are caveats to the codec no matter what setting scheme u use.

I read Syskin's post from that other thread and it doesn't really make any sense. :P What does the "bitstream expect"? I understand that the B frame routines aren't as sophisticated as those for P frames but I think their benefits outweigh any drawbacks. Ur statement "In the end you don't gain anything with your b-frame settings but probably loose something" is incorrect. The real power of the B frame is not the quant offset ability but additional motion compensation. Unrestricted B frame settings allow the codec to wring every last drip and drop of motion compensation out of an encode. That saves vast numbers of bits and thereby allows the residual BVOP to have a *lower* quantizer than it otherwise would have. I've verified that the B frames are 25-60% the size of the proximal P frames. (of course every once in a while this fails) This can be tested by doing encodes at x/1/0, where x has different interesting values, with Bsens at some high value. The other settings can be anything. I can almost garuntee that the higher Bmax goes the better the bang for the bit. So what u gain with these settings is great bit savings due to vastly increased motion compensation which will be redistributed back to all frames and allow for lower quantizers. Ergo higher quality.

Once we get VHQ for BVOPs and the algorithms get more spit and polish I hope all this hand wringing will be overwith. I believe there's still more potential to be had from B frames. So to sum up, the future is in massive motion compensation motion compensation motion compensation.

LA

Edit: Anime represents the best case scenario for massive B frame usage.

[unmei]

i also use B-frame settings similar to lordadmira for anime and when the source is rather bad (sadly too common with older anime), it allows for compression that otherwise were only possible by totally ruining the picture with filters or resize to a smaller resolution. I also applied these settings a little less extreme to a 'real life' movie once - and yes the results are by far not as positive as for anime :)

[Teegedeck]

Originally posted by lordadmira
...Ur statement "In the end you don't gain anything with your b-frame settings but probably loose something" is incorrect.
That's not what I've written. I said "why risk loosing something" which is quite different from "probably"

Originally posted by lordadmira
This can be tested by doing encodes at x/1/0, where x has different interesting values, with Bsens at some high value.
No, you can't, unless 'interesting values' also meant '0'.
If the proposition is 'using b-frames at ratio 1, offset 0 gives better compression/lower filesize than NOT using b-frames' then your proof must be a constant quantizer encode with b-frames x(bigger or equal 0)/1/0 against a constant quantizer encode without b-frames. A comparison of resulting filesizes will verify whether b-frames at your settings yield any better compressibility.
If the proposition is 'using b-frames at ratio 1, offset 0 gives better quality than the standard settings' then your proof must be a two-pass encode with b-frames x(bigger or equal 0)/1/0 against a two-pass encode with b-frames 2/1.5/1. A visual comparison will (subjectively) verify whether b-frames at your settings yield better quality.

I'm actually looking forward to your results and I'm especially curious to see whether your settings help with bad-quality sources. If you've already tested this ('interesting value=0') post your results here. Please share your experiences a little more detailed. This could mean a big contribution to our community's seach for good XviD settings and I hope you make the effort.

Edited: Candidates for quality comparison

Greets,

Tee

[lordadmira]

Originally posted by Teegedeck
That's not what I've written. I said "why risk loosing something" which is quite different from "probably" Look again, u did write that. :D

I'll look into doing that controlled test, but don't expect me to be swift about it. ;) I'm somewhat astonished that u wrote what u did about B frames.

[Aktan]

The only reason I don't increase the max b-frame to something greater than 1 is because of b-frame errors said in this:

http://forum.doom9.org/showthread.php?s=&threadid=76404

I have tested a max b-frame of 132 before and it seems the higher xvid decides to put the b-frames consectivatly, the more errors I see. I can make a sample clip to show you what I mean if you like.

PS. This is all anime source I've tested it on.

[lordadmira]

I'm still not sure what the actual error was that people were complaining about in that thread. A solid color block by itself is not necessarily an error. There was the post about a block impinging on a border, that could be an error in the ME. Are u seeing any other errors?

[Aktan]

No the block imping on a border is a main one but much worse than that sample, if I remember right. The clip where I saw it really bad and had, I think, 63 consect B-frames in a row before a P-frame in which another 50 or so B-frames followed. This was during a black background with just some white text at one corner (forgot which). There would we a very noticable artifact block of misplaced text parts that only happens during a b-frame and does not disappear really until the next p-frame.

[malkion]

People need to realize that the codec does not ignorantly put in B frames willy nilly. I read Syskin's post from that other thread and it doesn't really make any sense. :P I'm still not sure what the actual error was that people were complaining about in that thread lordadmira, you're stabbing in the dark here. I believe you're the person who is willy nillying setting b-frame values to use in your personal encodes. I believe you're the person who can't make sense out of other's people's posts, which are easily understandable. I believe you're the person going on and on about something that needs to be laid to rest. XviD makes do with the settings you provide, arguing about safe b-frames settings beyond the recommended 2 or 3 is way off topic.

[lordadmira]

Originally posted by malkion
arguing about safe b-frames settings beyond the recommended 2 or 3 is way off topic. It's evolution baby!