New Xvid FAQ...feedback requested.

[crusty]

I'll look into it Soulhunter, I just can't get around to everything in time. Patience, my man, patience! :)

EDIT:
Oh yeah right, forgot to mention:
The links to Doom9's guides were broken apparently because of an alteration at the server side that suddenly makes linking to it case sensitive. Fixed and checked.

[Soulhunter]

Originally posted by crusty
I'll look into it Soulhunter, I just can't get around to everything in time. Patience, my man, patience! :) No problem... Take as much time you need !!!


Bye

[virus]

Originally posted by crusty
@virus:
Where did you find the word 'Cosinus' I can't find it.

In your thread "Understanding a Quantization Matrix", first post ;)
(just in case you want to add this explanation or a link)
In the FAQ it's reported correctly (cosine).
BTW: the problem with the links to doom9's guides was Opera's fault, they don't work even if corrected. They work in IE6.

[Selur]

@crusty:

'I-frame closer than ... frames' sets the minimum number of non-keyframes that have to exist between two consecutive Keyframes.
Note that crossfades (the fluid transformation of one scene into the next) are particularly sensitive to this setting; They will flash or look like crap if you set this too high. Never set it higher than 10. Set this to 1 to disable forced I-frame spacing.


Are you sure about this? Your explaination makes it sound like it's a 'minimum i-frame intervall', but as far as I understood it, it just sais how many non-iframes have to be between two I-frames, so that the second one will not be '... reduced by %'

Cu Selur

[crusty]

@virus:
I had the same problem in mozila firebird, it's one of those 'weird' issues.
I'll change the Cosinus thingy in that post.

@Selur:
That's the problem...all the info I could find didn't make that clear either. I'll dig into it a bit deeper some time soon, but for now it's better than nothing.

@Soulhunter:

How about this:
<START>
Q: Why do red areas in my clip look blocked or pixelated?

A:
This is an issue with some decoding applications, it has nothing to do with encoding. The colour space used by XviD is YV12 and is a very lossy colour space. When decoding the decoder has to convert this to a full quality picture that the video card can send to either a monitor or a TV.
The lossy Xvid file has much less colour information than uncompressed video, so the decoder has to reconstruct the lost information somehow if it wants to put out a proper result. Since there is way less colour information in the Xvid than there is brightness information, the colour information the decoder has to work with is far less precise.
The underlying process is called upsampling and the colour space gets converted from YV12 to RGB32, which isn't (very) lossy. Then the RGB32 signal gets thrown at you via your monitor or TV screen.

The problem occurs when the upsampling isn't done properly; Part of proper upsampling is taking that sparse colour information and making a 'best guess' about what the values inbetween are supposed to be, creating a fine colour gradient across the picture.
A process like this is usually called interpolation and if the decoder doesn't do that properly the decoded output will not have a proper colour gradient but will look a bit blocky; After all the 'in between pixels' will not have an 'in between value' but the closest orginal value, which is far courser, and creates very visible artifacts.
For some currently unknown reason the artifacts tend to be more visible in the red parts of the spectrum. Current best guess is that it is an interaction between YV12 colour space-weirdness and the colour-sensitivity of the human eye, unforeseen by the creators of MPEG.

Now the bad news is that this is an inherent issue with the colour-space used by MPEG, not just with MPEG-4 but also with MPEG-1 and MPEG-2 (VCD and DVD). As long as the output was just on TV noboby seemed to care, but modern monitors and HDTV's are far more precise and it is now an obvious weakness in many MPEG applications.

The good news is that it's just a small issue with decoding and that it is very fixable. Also, if you turn on the chroma motion option of XviD the problem will be smaller and less obvious than usual, because it will make colour information more precise and colour gradients less abrupt.

On windows these are the known fixes:

-First method is to use ffdshow with "ConvertToRGB32()" command in its AviSynth tab (use an ffdshow version that has it).
This will make use of the upsampling features of Avisynth which work properly.

-Second method is to buy another video Card. Not recommended really...
Especially Geforce video Cards and older cards seem to suffer from improper upsampling methods. Trying another driver might help, but feedback on this has been too few and far between to really make any assumptions.

-A third method is to try to use the "VMR 9 Renderless" mode in MPC.
This will make MPC use another type of playback, possibly using proper upsampling.

-A fourth method (untested at this moment) that could hypothetically work is to first open another media file and play this simultaneously with the file you really want to play.
(the idea behind this is that only one media playback can use 'hardware overlay' which makes use of the hardware acceleration features of your video card. The next file opened cannot use these features and would be played using software overlay, which might use other (and proper) upsampling.)
Needless to say this takes quite some extra CPU and it denies you the use of all the nifty hardware features your video card has. But, it might work.
<END>

Some feedback on this would be nice.... :D

[virus]

some more improvements suggested:

from A3:

1) "ISO (International Standards Organisation)"

ISO = International Organization for Standardization
(from their homepage www.iso.ch)

2) "MPEG-4 became a proper MPEG standard in 1998, and became an ISO standard almost immediately".

On their webpage ISO labels MPEG-4 as being released as standard in 2001, not in 1998.
quote from www.iso.ch:
'ISO/IEC 14496-1:2001 Information technology -- Coding of audio-visual objects -- Part 1: Systems'
'ISO/IEC 14496-2:2001 Information technology -- Coding of audio-visual objects -- Part 2: Visual'

3) "XviD uses the improved MPEG-4 version 2 or ISO standard #14496-2"

Better "XviD follows MPEG-4 Part 2, also known as MPEG-4 Visual or ISO standard #14496-2"
(ISO standards are usually structured in "parts" which work together, eventually adding/extending functionalities)

4) you can also add this:
"the latest addition to the MPEG-4 family of standards for video coding is MPEG-4 Part 10 (14496-10) or MPEG-4 AVC (Advanced Video Coding) also known as H.264. XviD does not support H.264 though."


from A4:

"Typically these are called Keyframes (also called I-frames or INTRA-frames), P-frames (for Predicted frames) and B-frames (for BI-directional frames)"

better "P-frames (for Predicted frames, also called inter-frames)"
this should be useful since the Edit Matrix dialog use "intra/inter matrix"

hope this helps :)
virus

[Leak]

Originally posted by crusty
After all the 'in between pixels' will not have an 'in between value' but the closest orginal value, which is far courser, and creates very visible artifacts.


I think you really want to say "coarser" here. :)

Also, if you turn on the chroma motion option of XviD the problem will be smaller and less obvious than usual, because it will make colour information more precise and colour gradients less abrupt.

And of course you'll want to use "Chroma Optimizer" here... ;)

(By the way - are you sure you want to use underlining in HTML? It's easy to mistake underlined text for a hyperlink; also, it's only part of the HTML 4.01 Loose DTD and deprecated (http://www.w3.org/TR/html4/present/graphics.html#edef-U) in all other versions of HTML 4.01 and XHTML 1.0...)

np: Kilogram - A While Ago & Recently (Erlend Oye - Unrest)

[sh03z]

This FAQ is fantastic...
I read it through just once, and I went from crappy rips to perfect ones...
Once again, this FAQ r0x0rz!!!

[crusty]

@virus:
1)
Well I still use the old name..... I don't like the modern practice of institutions altering the name just to be more stylish...especially when the old name makes much more sense. :)

2)
Roger, altered it.

3)
Altered it

4)
I took over part of it. Check it out.

A4:
Roger, will alter

@Leak:
Coarser...got it.

optimizer..got it

@the underlining being deprecated:
Well it's been part of HTML for as long as I can remember. Just because some people would like everybody to use style sheets (which are much more complex) doesn't mean we have to. As far as I can tell, it works in all current browsers, and it is to be supported for backwards compatibility in the future as well.

"User agents should continue to support deprecated elements for reasons of backward compatibility."
I guess you can call me backwards... :D

@sh03z:
Well that's the kind of feedback I LOVE to hear...great that people are actually getting help from my FAQ!! ;)

[crusty]

Update: altered the 'Cartoon Mode' explanation. Gonna do Qpel next...

[Soulhunter]

Originally posted by crusty

This is an issue with some decoding applications, it has nothing to do with encoding. The color space used by XviD is YV12 and is a very lossy color space. When decoding the decoder has to convert this to a full quality picture that the video card can send to either a monitor or a TV.
The lossy XviD file has much less color information than uncompressed video, so the decoder has to reconstruct the lost information somehow if it wants to put out a proper result. Since there is way less color information in the XviD than there is brightness information, the color information the decoder has to work with is far less precise.
The underlying process is called upsampling and the color space gets converted from YV12 to RGB32, which isn't (very) lossy. Then the RGB32 signal gets thrown at you via your monitor or TV screen.

The problem occurs when the upsampling isn't done properly; Part of proper upsampling is taking that sparse color information and making a 'best guess' about what the values in between are supposed to be, creating a fine color gradient across the picture.
A process like this is usually called interpolation and if the decoder doesn't do that properly the decoded output will not have a proper color gradient but will look a bit blocky; After all the 'in between pixels' will not have an 'in between value' but the closest original value, which is far courser, and creates very visible artifacts.
For some currently unknown reason the artifacts tend to be more visible in the red parts of the spectrum. Current best guess is that it is an interaction between YV12 color space-weirdness and the color-sensitivity of the human eye, unforeseen by the creators of MPEG.
Looks really nice to me... :)

But some changes could be still there !!!


I am a bad typer, so I just try to give some infos... ;)

- This YV12 saves the color (chroma) information of a picture in a 1/4 resolution of its original...

- While playback, the chroma gets resized back to the original framesize...

- While "wrong" playback, the chroma gets only resized with lousy nearest neighbor method...

- While "right" playback, the chroma gets bilinear resized...
Originally posted by crusty
The good news is that it's just a small issue with decoding and that it is very fixable. Also, if you turn on the chroma motion option of XviD the problem will be smaller and less obvious than usual, because it will make color information more precise and color gradients less abrupt.
- No, chroma motion will not help with this problem... :(

- Its really only related to the YV12 colorspace !!!

- No possibility to fix this before playback...


Bye

[crusty]

@Soulhunter:

I don't get it...did you just completely missed the point? :confused:
- This YV12 saves the color (chroma) information of a picture in a 1/4 resolution of its original...

- While playback, the chroma gets resized back to the original framesize...
Which is exactly what I wrote....
- While "wrong" playback, the chroma gets only resized with lousy nearest neighbor method...

- While "right" playback, the chroma gets bilinear resized...
...which is a difficult way of saying something that I put in a much simpler way.
- No, chroma motion will not help with this problem...
Well, theoretically, it should mitigate the effect a bit, not solve it.
That's what I said.
- Its really only related to the YV12 colorspace !!!
That's what I wrote. :rolleyes:
- No possibility to fix this before playback...
Which is exactly what I wrote.

Really, this is too silly... do you have anything real to comment? :confused:

[atreya2011]

- detect_static_motion is a motion estimation flag. The threshold below which a macroblock is concidered static is increased so that very fine detail is lost. Since A LOT of 'very fine detail' is actually noise (especially with cartoons) it really helps saving many bits which would otherwise be used to code noise on a static picture.
- vop_cartoon is about quantization - when a block is motion-compensated well enough (with total error below the limit) it's just not coded at all. XviD doesn't drop any data in normal mode (limit = 1), but drops quite a lot in cartoon mode. Again, this usually means that noise is ignored. It might also remove some small details, but small details shouldn't really happen in "proper" cartoons.


A little help here. I don't fully understand that explanation. First
"whats a motion estimation flag" and second "whats vop_cartoon" is it another flag?

:thanks:

[crusty]

Updated it...try it now and see if makes more sense to you.
(it has too to be a proper part of the FAQ)
:)

I also updated the Qpel answer. Please tell me if you understand it.

[mikeX]

wow, i see many new additions, changelog coming?
the cartoon mode stuff sound really interesting

some small corrections:
- find/replace color with colour or vice versa (to be consistent)

Appendix:
A. useful Links
A. Useful Links

I am a bad typer, so I just try to give some infos...

- This YV12 saves the color (chroma) information of a picture in a 1/4 resolution of its original...

- While playback, the chroma gets resized back to the original framesize...

- While "wrong" playback, the chroma gets only resized with lousy nearest neighbor method...

- While "right" playback, the chroma gets bilinear resized...
i think soulhunter just meant that you could add some more details next to your explanation (like just how lossy is YV12 --> 1/4, and what is actually the bad/good upsampling method --> nearest/bilinear)

also (never noticed it myself but i've seen it mentioned) shouldn't there be a similar effect on highly saturated blue areas as well? (maybe add it in the Q?)

keep it up :)

edit: ---
C3c:
The more precise
The more precisely

[crusty]

Will update soon.

'- find/replace color with colour or vice versa (to be consistent)'
Apparently only 'color' is proper, according to spellcheck.com

'i think soulhunter just meant that you could add some more details next to your explanation'
I'll take another look at it and see if I can improve on it.

'also (never noticed it myself but i've seen it mentioned) shouldn't there be a similar effect on highly saturated blue areas as well? (maybe add it in the Q?)'

Well, the thing is, the human eye isn't very sensitive for blue in general, but it is very sensitive to blocks in blue areas. This has been known for quite a while. So the designers of MPEG-4 quickly noticed, that while you can usually assign few bits to blue areas because our eyes are less sensitve to those, assigning them too few bits creates artifacts very quickly.
So they adjusted the blue system for this.
Why the red-block thing slipped them by I don't know...it's quite obvious from some other sources that there are several issues involving MPEG compression schemes and the colour red, not just MPEG-4.
Maybe one of them was colorblind. :)


btw: Were the Qpel and cartoon mode explanations understandable?

[mikeX]

just read the whole qpel explanation, i found it amazing.
i haven't read through the entire qpel thread yet, but one wonders, is there really more to know? :)

about cartoon mode:

both designed to help with cartoons:
maybe " both designed to help with animations:"

i find it enlightening, even though there is some 'techie-talk' in it i don't think it's more advanced than in other parts of the faq where options are explained in detail. And even if one doesn't get it, there is still the 'conclusion' "So, while the first technique helps with removing movements that are so tiny that they can be considered not-to-be-part-of-the-source, the second helps compressibility of the cartoon by removing texture detail that's considered too-small-to-be-part-of-the-source." which should suffice for most people, right?

you could elaborate on why does this happen specifically with animations etc, but that doesn't have to do with xvid really...

ps: colour --> British, color --> American, none of the two will be considered wrong really... (although 'colour' should be considered proper imo (since that's the original), evil conspiracy of spellcheck (hmmmmm))

Maybe one of them was colorblind.
:D or it could be an evil conspiracy of MPEG against red (hmmmmm)

[atreya2011]

@crusty
Well The first part of the explanation on cartoon mode makes lot of sense. In the second, "vop_cartoon is about quantization", is it some sort "quant multiplier"?

As for the explanation on Qpel, most of the explanation was a wee.. bit complicated for my simple brain. Although I will read it 5 or 6 times to see if I can comprehend it because knowledge is power:D

[Soulhunter]

Originally posted by crusty
Really, this is too silly... do you have anything real to comment? :confused: Dont wanted to make you abused... :(

Just thought adding this stuff would make the problem a bit more understandable !!!


And for the chroma motion...

Really, Ive tested this !!!

You will not see reduced pixelisation, even at 4x zoom... :o


Bye

[crusty]

OK, no problem dude.. :D

Perhaps the chroma motion thingy won't work or perhaps it's source dependent. I'll give it another thought..

No updates today really...does anybody have any other additional questions that I could add?

[crusty]

Updates:

Added C17: Recommended newbie settings
(I used safe options only, If something isn't safe please tell me)

Updated D12: Removed the Chroma optimezer/motion bit.

Glossary:
Added YV12, RGB32, upsampling, VMR-7/9 and Video Renderer and FourCC.

Some minor fixes here and there, nothing fancy.


EDIT:
Update:
Also added:
"D0: I get all sorts of garbage during play!"
What happens when DivX decoder overrides XviD by the 'support generic mpeg-4' setting.

[Soulhunter]

C. Glossary

MPEG: Motion Picture Expert Group. A conglomerate of companies and institutions that sit around a table once in a while making up new standards. Creators of the MPEG-1 (VCD) layer-II and layer-III (mp3), MPEG-2 (DVD), MPEG-4 (SVCD, DivX, XviD, etc)) and other standards, they are a very important factor in the media industry.Dont like nitpicking, but...

- AFAIK SVCD is not MPEG4, its MPEG2 !!!

- MPEG-1 layer-xx, should be MPEG-1 audio-layer-xx !!!

- MPEG-1 audio-layer-II is mp2, and not mp3 !!!


Bye

[atreya2011]

Well, I didnt want to post the same post again, but I think my previous request was not looked upon. :(

Originally posted by mf
*VERY* simple explanation: the image is converted to detail signals (DCT), and then those signals are divided (coefficient cutting) to make it smaller. The more signal you cut off, the more detail you lose and more artifacts (blocks, ringing aka mosquitoes) you get. The quantizer is the detail removal factor (DivX3/Nandub thus calls quantizers DRF). The higher the quant, the more detail gets cut off, the lower quality.


Just a sugesstion.

Maybe you could add this in A4. MPEG-4 Basics section

Q. What's a quantizer?

[crusty]

'- AFAIK SVCD is not MPEG4, its MPEG2 !!!'
You're absolutely right and I was already aware of the error, I just hadn't changed it yet. Thanks for reminding me.

'- MPEG-1 layer-xx, should be MPEG-1 audio-layer-xx !!!'
I'll doublecheck and alter it.

'- MPEG-1 audio-layer-II is mp2, and not mp3 !!!'
It wasn't meant to say that layer II was mp3 also, I'll alter it.

@atreya2011:
I kinda forgot, sorry for that.
I'll see where I can slip it in.

Thanks for the comments,
Crusty

EDIT:
Ok, added those alterations to the FAQ.
Check it now.

[BoNz1]

-Turbo ;-): This setting skips some search techniques when using Qpel or B-frames during the first pass to speed it up a bit. Without those options on it has no effect at all. The impact on quality is negligible.

I am sorry I have not had the chance to read all the FAQ but I believe this is wrong, turbo doesn't give any speed up in the first pass I don't think unless if perhaps you have fast first pass turned off. I believe although correct me if I am wrong that it speeds up the second pass it makes no difference whatsoever if it is activated or not in the first pass IIRC.

[crusty]

OK, I will look into it.

EDIT:

Well I looked into 20 threads, including the RC1 and RC2 threads and couldn't find anymore else to say about Turbo, except that I should remove the bit about 'first pass'. ....Done.

[Selur]

All "turbo" optins are related to b-frames and qpel. It won't make Simple Profile faster.
It won't make any differnce in first pass either.
by syskin here (http://forum.doom9.org/showthread.php?s=&threadid=67495)

[nine]

Just a quick note on section A2...

"...but also a life video-feed..."

should read:

"...but also a live video-feed..." :)

[yaz]

@crusty
many thx for your continuous efforts on polishing it. i got some small quests (not having read the very last version)

C17 (noob sect.)
-Profile: Use AS@Level5, nothing else.why? afaik, the whole profile business has no effect at the moment. just as u mentioned earlier.
Don't mess with Interlaced encoding, greyscale, ... , GMC, Reduced Resolution ... These settings are simply not for newbies;it prompts (me) that there's sg risky about using that. These all are 1click options, so no further tweak's needed. why do u think them so risky? if sy reads your guide ... can be no problem with them.
... you need to understand them to use them properly! yeah, that's the point! but it's true for all the options. that's why we all read your faq :-))

thx
y

[atreya2011]

Updated PDF...

Crusty's Unofficial XviD FAQ updated as of 21-02-2004 (http://www.geocities.com/atreya2011/index.htm)


Have Fun... :D

One more thing, if you want to print it out in an A4 paper, its 40 pages long :) (8 pages + Last week's PDF), in the near future, it can be called as the Unofficial XviD Manual :D

[crusty]

Selur:
by syskin here
I read it already...didn't say anything extra really. I think current explanation is good.

@nine:
Thanks..altered.

@yaz:
why? afaik, the whole profile business has no effect at the moment. just as u mentioned earlier.
That's not entirely true. True, Xvid won't keep within maximum bitrates specifief for that particular profile abd level settings.
But that's only half the story.
The other half is, that altering this setting will alter the possible options, including really experimental ones or non-mpeg-4 compliant ones, like RRV and above-DVD resolutions.
So not sticking to AS@Level5 should be considered not safe for newbies. AS@L5 is what I consider safe for newbies.
:D
it prompts (me) that there's sg risky about using that. These all are 1click options, so no further tweak's needed. why do u think them so risky? if sy reads your guide ... can be no problem with them.
Offcourse, if people read the FAQ through they won't be newbies for long anymore...but we live in an imperfect world, so we have to assume the worst. About those options:

-Interlaced encoding:
Really not a newbie thingy, since the vast majority of source is either progressive or (possibly badly) telecined or originally non-interlaced content. Telling a newbie it's safe to use simply isn't true, as proper IVTC and deinterlacing, especially of poor interlaced conversions of originally progressive content, is a whole ballpark by itself.

-grayscale:
Might be used if the original was grayscale, but there's:
a) usually no need to use it with proper B/W content, as it doesn't really add much to compressibility AFAIK .
b) some B/W movies actually have a bit yellowish tint due to aging of the film, and some people might like to keep this tint.
(btw: I think I'll add this extra bit ofinfo to the FAQ)

-Debug settings:
Newbies have got nothing to do with these. Only the FourCC thing could be interesting to them, but even then they have to know the consequences really. Consider a newbie encoding with GMC and a fourCC of DX50...doesn't really sound good now does it? ;)

-Curve Compression & overflow treatment:
You need a very good mental image of the whole effect of these settings to alter them in a positive way.

-GMC:
Has many incompatibilities with non-xvid decoders, and a proper understanding of what one wants to do with it is required for use.

-RRV:
Highly experimental, not completely implemented yet, highly incompatible, no beneficial effect, not in the AS profile...need I go on? :D

-Aspect Ratio:
Difficult stuff even to me. Proper use requires:
-understanding of how the original source was supposed to look like, and on what media it was supposed to look like that (PAL, NTSC, PC, etc.)
-understanding of the different aspect ratio standards
-No errors made in the conversion proces
-Understanding of the type of media it's going to be played on (standalones and their support for aspect ratios, Xvid or ffdshow or DivX decoder, container format, etc. etc. etc.)

For all these reasons I think I'm quite safe and correct when I say in the FAQ: " you need to understand them to use them properly."
As that's really the truth.
For instance the proper handling of Aspect Ratio alone would require a separate document, probably as lenghty as this FAQ.

Not all options are created equal...C17 is there just to point newbies to the safest of them, and like I already stated in the FAQ, it's probably the most often asked question, so that alone warrants at least something of a usable answer in my point of view.

atreya2011:
Thanks for the conversion again. I mirrored it here:
http://www.vslcatena.nl/~ronald/docs/Xvid-FAQ-Crusty-21-02-2004.pdf

EDIT: I will make some more updates later today

[mikeX]

@yaz

I totally agree with crusty, the 'newbie settings' question is completely useless to someone who has already read and understood the FAQ, but it's not there for someone like that, it's for someone clueless and maybe uninterested in learning all those things it takes to understand certain options/settings. As such, i find it a useful and meaningful addition to the FAQ.

@crusty

B6:
All the speed-hungry parts of XviD are written in assembler.
shouldn't that be assembly??

edit: and maybe also 'cpu-hungry' instead of 'speed-hungry'??

[sysKin]

Hi,
I finally found some time to read the FAQ one more time :)

I have some suggestions. Unfortunately not all are very specific, but I'm not a good writer and I'm not trying to rewrite them fully...

Generally, I would like to say that some answers are quite messy and don't focus on answering a question. Instead, they're trying to feed reader with specific and (often) useless in formation that isn't really on topic.

A1 is a perfect example of this. Write that XviD is a multi-platform video encoding and decoding library. Don't introduce virtualdub here. You can write that there are both codecs (like vfw) and decoders (like Nic's directshow-based) which use it, or are based on it. I wouldn't mention Nic's decoder because it's quite old. I wouldn't mention windows media players, they are not xvid (read the question). You can write that XviD aims for medium-to-high bitrates and medium-to-high-resolutions, although works with the other end like mobile phones. You can write that development focuses on encoder.

A2 is even more messy. Don't mention disk drive as part of the stream, it just confuses everyone (would you mention it in notepad.exe FAQ?). Xvid is a library, not a program, so it's logical that some application takes video data (from avs if you want to), uses xvid to encode it and writes the result to a file. Why making it so long and complex? It will be pretty obvious later, when you write about setting up stuff.

A3 Ah I see you've already fixed the "second version of mpeg-4". Nice. I would rewrite the thing about h264 - xvid supports mpeg-4 part 2 and only 2 (which means not 10).

The thing about software and hardware implementations is off topic. Flashing firmware - even more.

A4. Hm... I could rewrite that I think. I'm not sure it belongs to "introduction" part, could be "advanced"... but ok, doesn't hurt.

A7 Who needs to know mpeg-4 to use xvid? Sorry, noone ;) You also assume that user is ripping DVDs in win32 here... XviD is so much more. I would change the question and the answer, too. The most popoular question definitely is about *de*coding.

A8/A10/A11 are sort of in random order and without explaination... first there are binaries (where?), then someone doesn't know how to compile (why, if there are binaries. where's the sourcecode anyway. why developers don't compile for me?).
A11 only works if you've read A10 to the end, so maybe it should be sticked to A10 or named "what are the risks of using bleeding edge binaries?" and combined with A12.

A12 you never mentioned about tested and untested binaries before. Where are the tests made exactly? How can I read about these tests?

A15 is A12.

A13 and A14 and A16 can never be recent in current form... Basically one can either use XviD 1.0 (very soon I hope). There should be many binaries available once it's out. Second, one can use most recent 1.1 build (again: soon) from Koepi, or anyone else who starts building them. He should go to Doom9 and read about known problems and features. Third, someonce can use automatic instabuilds from 1.1 tree and hope for the best. These are the three options.

Dev-api-3 are not recommended anymore. Old 0.9.2 is completely not recommended.

Generally I would make "getting xvid" a separate chapter - third after "introduction" and "playback of xvid files". BTW as for decoding, I'll add more answers about most common problems, solutions and tricks. And a summary of all known mpeg-4 decoders and how they perform.

B4 is A13 and A14 and A16 again.

[** I will edit the post with the rest - I have to go now and don't want to loose what I've written **]

[yaz]

@crusty (mikex)
ok, i got the point. nothing to disagree but ... why don't u cut it short by 'use defaults unless u don't know what, why & how to change'. imho, default setting of the last releases is quite robust & safe.

sg more. i agree with syskin. it seems that the most crucial quest at the moment is playback. if u take a look at the xvid forum(s) the most quests relate to that. in this meaning, the playback part (B1) is bit fishy. starting from there, nobody will ever be able to make good playback. syskin promised to make it clear (& imho he's the only one who can make it), but until u should recommend only the dsf shipped with koepi's rc2 pack.
don't link koepi's site here as there is a standalone playback filter too. afaik, that's not recommended anymore.
don't link ffdshow as its state is messy & the versions downloadable from sf.net are out-of-date & inappropriate for decoding devapi4 encodes. more up to date versions can be downloaded from athos and kurosu, but these are 'unofficial' (& afaik, milan asked them to stop releasing such versions) instead, the ffdshow development thread may be linked (if sy is interested in, at all)
syskin's dsfs work fine (for me!) but the link u dropped is dead (for me!) as it's mispelled (all in small letters, pls) anyway, the zip coming down is broken (for me!) would sy drop me a working pack, pls !

u don't distinguish enough the devapi3 & the devapi4 branches. it is another point where the most 'common users' get confused. afais, both versions are in heavy use at the moment (despite, devels don't recommend devapi3 anymore) the download part (B4) reflects this confusion. imho, there's nothing to do with nic's & umaniac's site at the moment as they don't seem to deal with devapi4 releases. (it's a pity but that's what we have:-()
anyway, to those who wants to use the devapi3 version i would recommend umaniac's last insta build. that's the latest & the most debugged & updated version (to the date of its release).
& here comes ffdshow again. it is the best for decoding devapi3 stuffs but it may fake completely with devapi4 versions. however, it is good for postprocessing(!) both.

about instability. i've tested (almost) every built of gamrdev & i didn't find 'broken' or instable any of them. rc & insta builts are not different in this meaning. all need testing & all can be 'broken' (whatever it means:-).

still love your faq, anyway :-) (despite it may seem different:-)

the bests
y

[virus]

Originally posted by sysKin

A3 Ah I see you've already fixed the "second version of mpeg-4". Nice. I would rewrite the thing about h264 - xvid supports mpeg-4 part 2 and only 2 (which means not 10).
sorry if I enter the discussion, but this is mainly because I suggested this change myself, so I think I have to state something about that... my suggestion, which crusty reported literally, was:
"XviD does not support H.264". It is written very clearly in the FAQ (and in this thread too), so I'm just wondering what you're talking about... ;)

cheers
virus

[sysKin]

Originally posted by virus
It is written very clearly in the FAQ (and in this thread too), so I'm just wondering what you're talking about... ;)Ok I might have gone too far... It's just that when I read it first time, I had the feeling "where did h264 come from?"... It's kinda like saying "note: very important: don't forget: it's not a hamster" :D
Imho all answers should be as clear as possible, if it's mpeg-4 part 2 than that's what it is, no need for extra warnings... There could be an extra question about h264 (in fact it is a frequently asked question) - just not in "introduction" chapter.

I might be too picky I admit, I would just like this FAQ to be as good as possible :) and first answers are quite messy in my opinion. It's much better later.

Radek

[mikeX]

@ yaz

i can decode rc2 encoded material with the latest official alpha build of ffdshow. (given no packed bitstream)
quality can be inferior but i can make use of a wide variety of postprocessing and filtering, whereas the xvid decoder maxes out my cpu (with resolutions >= 5..x4.. & Q-Pel) if i try any postprocessing.
without Q-Pel i can safely use Y deblocking and internal conversion to RGB32

[virus]

Originally posted by sysKin
Ok I might have gone too far... It's just that when I read it first time, I had the feeling "where did h264 come from?"... It's kinda like saying "note: very important: don't forget: it's not a hamster" :D

:)
ok no problem...
BTW when I suggested this change (along with the part/version thingy) I was tempted to write it as:
"XviD does not support H.264. Please do not ask for versions that do."
Don't you think it may be a useful addition? ;)

cheers :)
virus

[yaz]

@mikex
of course, u can decode rc2 with (any release of) ffdshow. i only stated that it's inappropriate. (just as u wrote:-)
i've just made an encode with qpel/qmc/2b-frames & i tried to playback by decoded with xvid rc2 dsf (no pp inside) & postprocd with kurosu's ffdshow (rgb conversion, full postproc, sharpen with asharp hq, noise on y&u). no jerk anywhere on my athlon 2.2+. anyway, i never use ffdshow this way, i just performed it for u :-)

the bests
y

[TorgoGuy]

Hi. I'm a long time lurker...

Crusty has done a great job with his FAQ, but I was wondering if anyone would be interested in a more concise version of his FAQ? I think it would be less confusing to many newbies if you left out many unnecessary details, but still left in enough to be informative.

In that spirit, I've created a rough draft of "Part A" to generate some discussion. If you like it, I'll rewrite what I've already done (to make it clearer) and then continue on to parts B through E.

Here is the link:
http://143.236.28.105/staff/ken/xvid/faq/

I've resisted the (strong) temptation to reorganize the FAQ. This allows the reader of the concise FAQ to consult the exact same question number in Crusty's FAQ to get the full details.


EDIT: Looking up at Syskin's post, it looks like we're thinking along the same lines. I didn't mean to be redundant.

[crusty]

Well that's a lot of discussion suddenly !!

@syskin:

That's a lot of comments...Generally, I would like to say that some answers are quite messy and don't focus on answering a question.
I agree that some of them may be a bit messy, I'm still in the process of writing it. :)

Write that XviD is a multi-platform video encoding and decoding library.
That's a good one...I will make some way to include/replace it.
A1 certainly requires rewriting, it's just that I've been busy with all sorts of things, and I'm currently really only reacting to comments than actively making changes.
I will see what I can do.The thing about software and hardware implementations is off topic. Flashing firmware - even more.
I think I could add another question about XviD on standalones that includes this info.

@A7:
Well, certainly, most newbies do tend to start with DVD-ripping, and most Linux users tend to have a higher level of computer-knowledge.
So if the FAQ weighs heavily towards win32 and DVD-stuff, it's not without reason.
The most popoular question definitely is about *de*coding.
I could split it up of course, making the first question 'What should I know to play (decode) XviD?' and the second one this one. That would first give an easy answer to those who just want to view XviD files, tending to that group, and then to those who want to use XviD to decode. It's not a bad idea I think.

For those who want to use XviD to encode stuff, I consider it important that they should know something about the underlying processes.
And the most important of those are MPEG-4 and avi A/V-processing, (because no matter what we advanced guys do, most people still use avis).
The DVD FAQ also includes many information relating or comparing DVD's to other formats, which are very helpful to many.
(btw, I'm gonna include a Laserdisc FAQ as well later on)

A8 to A16 are to be replaced with a standard 'use XviD 1.0 because it's the best and finest' -answer once that comes out, I've mentioned this before. Consider them obsolete and removed once XviD goes 1.0....that's a promise :D
B4 is A13 and A14 and A16 again.
Well like I said, A13, A14 and A16 will be removed soon anyway.
Also of course B4 will be completely rewritten once XviD goes 1.0.
As of know, I think it's only proper to give people more than one option...I like options.

@yaz:
why don't u cut it short by 'use defaults unless u don't know what, why & how to change'. imho, default setting of the last releases is quite robust & safe.
Well I will alter it to include that at front. But still then there's nothing wrong with both question and answer.
Don't forget, it's still the #1 question after people where told to 'stick with the defaults' so it remains a valid question.

B1: You're right, I will alter it to include only the RC2-pack.
Also if sysKin can tell me where to find his latest ones I will add those as well.(he told me before but I can't remember exactly)
And you're right, the link is dead. I'll alter it to just the directory, that still works.
@B4:
I will make it clear that only Koepi's sysKins and GameR's binaries are current.
rc & insta builts are not different in this meaning.
Not so, Koepi tests his builds before he releases them AFAIK. Insta-builds are completely untested and just represent the latest CVS (which can be broken by just a single typo).
Of course, Koepi's builds can break stuff just as well..just less obvious stuff. :)
It's kinda like saying "note: very important: don't forget: it's not a hamster"
Where!? WHERE?!
<Crusty running around with large club bashing Hamsters all over the place>
There could be an extra question about h264 (in fact it is a frequently asked question) - just not in "introduction" chapter.
You're right. Rereading it right now it does feel a bit unessential (but still noteworthy) information.
"XviD does not support H.264. Please do not ask for versions that do."
Hmm...sounds nice. :D
u can decode rc2 with (any release of) ffdshow. i only stated that it's inappropriate.
I think that's already properly answered in the FAQ with:

"Although FFDShow has more options, it is not based on the XviD project and may contain incompatibilities from time to time. It is advised to try both and see what works best for you."

@TorgoGuy:
Making a short version of the FAQ is a good idea.
Yours and mine represent two different philosophies on FAQ's;
One with short answers and one with very elaborate answers.
There's nothing wrong with either way, it's just a point of view.
Personally, I've seen way too many FAQ's (especially by ISP's, Manufacturers and software companies) that are so short and uninformative I'd like to shoot the morons that wrote it. :D

Basically, my opinion is that a FAQ can never be too elaborate; as with everything in life, one question answered almost invariably leads to another question asked.
It's just a matter of author's decision when the FAQ is so big that making it any bigger would impair proper updates.I didn't mean to be redundant.
Redundancy is good.
Better two FAQ's than none as far as I'm concerned.

EDIT:
Updates:
Altered : B1, B4 and some bits of C17..check it out.
Will make more updates later.

[Soulhunter]

MPEG: Motion Picture Expert Group. A conglomerate of companies and institutions that sit around a table once in a while making up new standards. Creators of the MPEG-1 (VCD), MPEG Audio Layer-II (mp2), MPEG Audio Layer-III (mp3), MPEG-2 (DVD), MPEG-4 (3ivX, DivX, XviD, etc)) and other standards, they are a very important factor in the media industry.
Re-add SVCD, but now to MPEG-2 !!!

Add AC3 to audio...

Add AAC to MPEG-4 audio part...
MPC: Short for Media Player Classic. It's a Media Player with an interface very similar to the Windows Media Player 6.4, which comes with Windows 98 and 2000. But it's much better and there are a gazillion differences under the hood. It's considered better than any media player Microsoft has. MUCH better
For just basic's good old MPlayer2...

For TV-Out users, ZoomPlayer is also nice...

Just mention to be fair !!!
B1. These filters do not decode audio! only video! XviD does not decode or deal with audio in anyway
Maybe mention Alex's AC3 filter here...


Bye

[Bogalvator]

Super dooper looking FAQ; lots of useful info in it - thanks for taking the time out to make it.

Typo spotted btw: C11.Zones - Chroma optimizer "you might want to turn it off when encoding in greyscale."

I did not know that XviD used YV12 all the time. I thought it just supported it (as well as YUY2 etc) and it just used whatever colour space that was passed to it via VirtualDub etc. With this in mind, is it optimal to pass a YV12 clip to the encoder?

[crusty]

@Soulhunter:

'Re-add SVCD, but now to MPEG-2 !!!'
OK, did it. Removing of SVCD was somewhere else btw.

'Add AC3 to audio...'
'Add AAC to MPEG-4 audio part...'
I'm planning to give those two separate entries to the glossary.
There's no need to put every A/V implementation in this entry, the entries are supposed to be examples, not a complete list of MPEG's achievements. :)

'For just basic's good old MPlayer2...'
I'll add it to a separate WMP entry.

'For TV-Out users, ZoomPlayer is also nice...
Just mention to be fair !!!'
True, Zoomplayer is also very nice, I use two players: MPC and Zoomplayer.
I found that Zoomplayer can handle video files with single bit errors in them without hanging the video, which MPC cannot (yet). OTOH MPC plays some files better than Zoomplayer, so they are complementary.

How does Zoomplayer help users with TV-out btw? I never used that feature.

'Maybe mention Alex's AC3 filter here...'
Well I have no clue who Alex is but I guess a mentioning/link of D5: "I want to playback this movie I d/l'ed but there is no sound!"
can't hurt...(in D5 I mention audio filters).

@Bogalvator:
Typo...got it. Thanx.

YV12 is the standard 4:2:0 colour space used for all MPEG-2 and MPEG-4 implementations (not so sure about MPEG-1 really). So whenever an MPEG-4 codec encodes something it does it in YV12, can't really be anything else. That doesn't mean it won't accept anything else as it's input, it's just that there will be less color conversions during the encoding process. This speeds up the process.

Additions to glossary:
3ivX, ABR, CBR, DivX, Ogg Vorbis, Ogm, and VBR...please check them for errors.

[sysKin]

Okay let me start the "decoding" section. I'm posting it here so that we can discuss it before it makes to FAQ ;) (or, maybe, you won't like it at all :D )

Q: I want to play xvid file. What should I install in windows?

A: XviD is an mpeg-4 compiliant encoder, so in theory you need any mpeg-4 decoder for playback. The following mpeg-4 decoders are available for windows:

- XviD's decoder. Althought seems like an obvious choice for XviD files, it's not necessarly your first choice. It supports all XviD's formats and decodes them well. However, it needs a fast CPU for playback - it takes more processing power than most other decoders. It has two postprocessing options [make it a link to postprocessing below] - deblocker and film noise. Some people say it's the best-looking deblocker of all - but it needs HUGE processing power to run. At your choice, you can make xvid's decoder also support divx4, divx5 and generic mp4v files.

- ffdshow. A free GPL-ed decoder for many video formats. Very versitile and useful. It supports all XviD files with one exception - playback might not be fluent with [link?]packed bitstream and more than one b-frame. It's very fast and thus recommended on older systems. It has numerous postprocessing options, can filter or resize the picture, display subtitles and more.
Note: in its configuration, you can change iDCT routine used. It's recommended that you set it to "XviD" for both XviD and DivX5 playback, it prevents [link to a Q below] "floating" artifacts. You can always use its configuration to enable or disable any format support.

- DivX5. Fast decoder with good postprocessing options. Note that you only need the "basic" version for decoding, so don't install spyware or spend money if you don't encode with it. When installed, it decodes XviD files by default and you can disable it from its configuration - look for "generic mpeg-4 formats". It doesn't support XviD's GMC or interlacing, and current version (5.1.1) sometimes doesn't decode b-frames correctly (with any content, not only xvid) [is it still true?]. It also has shuttering problems with more than one b-frame, although unchecking "smooth playback" seems to fix that. [I'll download this 5.1.1 and check compatibility, especially with qpel]

- 3vix decoder. [I'll make some tests with its speed, compared to the rest]. When installed, it decodes XviD files by default, but you can disable it in its configuration. It does not decode XviD's GMC [I'll test b-frames, interlacing, and everyhting else] It has standard postprocessing options. [Anything more?]

- NeroDigital decoder. [Where can I get it?] Decodes XviD when installed, and you can't disable it [am I right?]. It doesn't decode many xvid files correctly [bframes? gmc? qpel? intrlacing?... I'll do my best to test as much as I can][does it have postprocessing?][maybe it's at least fast?]

[are there any other mpeg-4 decoderson windoze?]


Q: Help, I have problems with sound!
A: XviD handles video only. It does not care or interfere with audio decoding. Most likely you need appropriate audio decoder, but you won't find help here.

Q: Help! playback is wrong/I see artifacts/green/ugly/shutters!
A: First of all, you have to find out which decoder is being used. As you can see [link to the first question]here, all mpeg-4 decoders hijack XviD files by default, and only ffdshow will ask you if you really want to do that. Most players will let you see and configure decoding filters by right clicking on video and selecting "filters" (like [link]mpc) or "filter properties" like [link]zoomplayer. WMP9 will let you see the filters used (in video's properties) but will not let you configure them.

If you're not using XviD, you might try switching to XviD first - go to the options of filter used and disable XviD support in that filter.

Q: Can I make XviD's decoder use less CPU?
A: Disable postprocessing first, of course. Another thing you can do is: Go to decoder's configuration and force output colourspace to "YV12". It's the fastest colourspace that can be used, but unfortunately it's not supported by some hardware well, so it's not used by default. It should lower cpu usage by about 5% - if that's not enough, you have no choice but using different decoder.

Q: I'm using XviD decoder and video is still wrong.
A: Unfortunately not all xvid alpha builds were stable. In particular, they never had proper interlacing support for GMC and b-frames. If someone used such settings, he created buggy streams. I don't think you can do something about it.
There was another bug in alpha builds - resolutions not divisable by 16 were also not supported correctly. Newest XviD decoder, as well as ffdshow, have an automatic workaround for this problem and should decode such buggy files corretly.

Q: I'm using XviD decoder and video is still wrong, part 2.
A: Try using different decoder. XviD's one might have some problems on some systems... Decoding was never in the focus of xvid's development.

Q: I see "floating" walls on my video.
Q: What's iDCT and what does it have to do with "floating walls"
Q: When I play video, picture gets worse and worse / colours slowly get wrong, and suddenly everything becomes good again and the process starts over.

A: [iDCT mismatch, let me think how to explain this...]

Q: What's a postprocessing? Deblocker? Deringer? Film effect?
A: [I ran out keys on my keyboard... can someone fill this? LOL]

Q: What's "B-frame decoder lag" message?
A: [still out keys on my keyboard, I'll add answer later]

[more questions? ask them now and they shall be added LOL]

Phew,
Radek

[Stux]

-Current 3ivX implementation normally uses no sub-pixel precision. As an 'advanced' option, you can tell it to use half-pixel resolution. So much for 3ivX being 'advanced'...


Hey! I object to this.

BY DEFAULT 3ivx uses Half Pixel precision, as an advanced option a user can *choose* to disable Half Pixel precision. A user might want to do this because he would rather encode faster (for instance live capturing).

[omion]

I've been lurking around here for the past year, and now there's finally something I can write about!

I section A4, you say:
The result is that, while a Luminance value is stored for every pixel, Chroma is only stored for every two pixels.
Sould be:
The result is that, while a Luminance value is stored for every pixel, Chroma is only stored for every four pixels.

<edit:>
Also, there are references in sections B2, B3, and E3 of a 'Cpu', and sections B2, B3, and E4 of 'cpu'. Should all be capitalized to 'CPU'.

Keep up the awesome work!

[sysKin]

Originally posted by omion
Sould be:
The result is that, while a Luminance value is stored for every pixel, Chroma is only stored for every four pixels.Indeed, it's every two pixels horizontally and every two pixels vertically, so it's total of one chroma pixel for every four luma pixels.

@omion: now *that* is a good first post :) welcome to the forum :)

Radek

[Wilbert]

some useless comments (I still have to read your faq :)

You're right. Rereading it right now it does feel a bit unessential (but still noteworthy) information.
You can put stuff like this in the appendix.

The result is that, while a Luminance value is stored for every pixel, Chroma is only stored for every two pixels.
Like others noted, this is for YUY2/YUYV.

[crusty]

Lots of comments...good!

@sysKin:
@Q: I want to play xvid file. What should I install in windows?

Sounds like a very good addition.

Also, all the other parts sound good as well.

In fact i'm thinking about altering the FAQ like this:

A: Introduction
B: Decoding (playing) XviD questions (including many of the troubleshooting items out of current section D)
(also, a question on standalones would fit in here)
C: How to encode (parts from original B section + additions about avisynth, Vdub etc.
D: Options, Schmoptions (relatively unaltered)
E: Feedback and questions (this part is very much done I think)
Appendix

@Stux:
Sorry, I was not aware of that.
On 3ivX website it says:
Half Pixel Motion
If checked the codec will perform half pixel precision motion search. Half Pixel motion dramatically increases quality and codec efficiency in almost all cases. Disabling Half Pixel Motion can speed up an encode.

Which made me believe Half Pixel is not the default. Still, an option allowing to use full-pixel resolution does seem a bit outdated now..
Anyway, I altered it.

@omion:
Ah, I knew that...just not at that moment. :D
Will update. I think I meant to say that there are two chroma values for every four pixels, which equates to one for every two pixels.
Of course those two values are not identical, so it's not really correct.
I altered it to:
"The result is that, while a Luminance value is stored for every pixel, two (different) Chroma values are stored for every four pixels."

I'll do a 'replace all' on CPU.

@Wilbert:
Yeah I know, I've been searching (and finding) some nice color space informational websites. I'll add them later to the FAQ.

sysKin:
[more questions? ask them now and they shall be added LOL]
I somehow expected you to say:
'ask them now or be forever silent!'
:D :D :D

EDIT:
Updated:
A7 and B1 to give more support to people who just want to play XviD files.