Can Xvid beat Mpeg-2 at DVD res?

[lordadmira]

Well I think we can all acknowledge that Xvid and Mpeg-4 in general rules the lower bitrate realms. But what about the full resolution full quality world of DVD material? Meaning resolutions equal or higher than 640x480 and quality indistinguishable from the original source. I'm pondering this question now as I'm working on more and more DVD ripped (music) videos that I'm trying to get to look exactly like the source in terms of subjective quality and, to as much as possible, mathematical similitude. So far the answer and conclusion I am coming to is a resounding no. On these videos I can see blockiness, detail loss, and other artifacts even at quant 2 with all settings on the maximum possible quality. Amazingly, one video was even on track to be bigger than the source file! Even with substantial filtering. That was with the MPEG matrix I think; after that I went back to H.263.

So I guess I'm soliciting comments on the theoretical breaking point of the whole Xvid paradigm in HD situations. I'm working on this one video now that just doesn't want to compress. First off the source is noisy, I've mostly compensated for that with cleaning filters, but the killer is that *everything* is in constant motion! The camera is constantly panning, zooming, and rotating; and, the backgrounds all have their own motion. Whether it's an animated greenscreen backdrop or some fluttering banner or hay. Moving hay has got to be the most difficult thing in the world to compress!

Questions, comments, reguritations? What kind of HD experiences have u had? Any opinions on the future?

LA

[kadajawi]

I'm working on music videos too, but captured from TV, so I always have to use filters and I don't really care that much about blocks since file size has to be ok too. I had some killer clips too... usually live stuff from rock/metal concerts is terrible, constantly changing (the whole scene since the lights go on and off, different colours etc.) or even stroboscope (?)... Rob Zombie clips are a mess too, very fast.

I'm doing 2 passes, deciding the target size by "feeling" (after looking at the first pass result, got to keep the first pass of course to do that). I'm taking out quite a bit (maybe 20-40%), sometimes it even "improves quality" by taking out some noise.

But you're trying to keep the original quality (and I don't think DVDs are of perfect quality, IMHO sometimes you can see blocks etc.), yes, I think that is possible, just check the HDTV thread, I don't know if the sample is still up, but the quality was just perfect. Then there is the Reloaded thread, where they tried to make a DVD sized encode of Matrix Reloaded. It should be on this page.
In any case, I would try out 2 passes, you'll just have to experiment how much you can reduce the file size.

From what I read you only used the MPEG and H.263 matrix, you should try out ones like six-of-nine etc. and experiment a bit with the quality improving/space decreasing settings.

[KpeX]

Remember that XviD is a lossy encoder, meaning it will never be the same quality as the source. Obviously a source MPEG2 will beat a recompressed encode with any lossy encoder. Given the same source, an MPEG-4 encoder will beat MPEG-2 at any bitrate almost every time.

[kadajawi]

Originally posted by KpeX
Given the same source, an MPEG-4 encoder will beat MPEG-4 at any bitrate almost every time.
Uh, you mean beat MPEG-2, huh?

If you do some proper filtering quality can be IMHO improved over the original. But thats probably a matter of taste.

[KpeX]

Originally posted by kadajawi
Uh, you mean beat MPEG-2, huh? I didn't type that. I don't know what you're talking about ;)Originally posted by kadajawi
If you do some proper filtering quality can be IMHO improved over the original. But thats probably a matter of taste. From a technical theoretical standpoint, no, it cannot be improved. If the source is flawed, then it can be smoothed out, but a lossy encoder can never achieve the same level of quality.

[kadajawi]

damn, I already typed subjective quality, but then I thought it was unnecessary.
Yes, technical, the quality would be decreased, since you would alter the source. But then if it looks better I have no problem with that.

[lordadmira]

Maybe I should add another stipulation to help clarify this. The source is an Mpeg-2 DVD file. We want to produce an equal quality output at the same res with significant size savings. The viewer should not be able to tell which is the Xvid file and which is the source. Mathematical theoretical superiority is demonstrated by indistiguishable quality at a lower bitrate. But the fact that a source happens to be a DVD file isn't exactly germaine. From a mathematical point of view all sources are equivalent.

[KpeX]

Yes, but the point remains that you are doing an additional lossy re-encoding on an already lossy source. In such a scenario it will be very difficult to reduce the size greatly and stil retain near-indistinguishable quality, especially for music video-type sources.

[Blue_MiSfit]

I would agree that MPEG4 can equal or beat MPEG2 at equal bitrates if they are coming from the same source. In my own experience with encoding DV from a camcorder, XviD always looked just as good as MPEG2 at 6000kbit (both the xvid, mpeg2, and dv looked virtually indistinguishable at this bitrate however).

To me it is really more a matter of being perceptibly lossless allthewhile achieving highly reduced filesizes. This is where MPEG4 excels.

[bond]

as kpex already pointed out theoretically/technically mpeg-4 is offering more advanced compression technologies than mpeg-2, therefore it could be drawn the conclusion that a mpeg-4 encoder will beat a mpeg-2 encoder if compared at the same bitrate

practically everything is relative:
if you compare a totally untuned, crap mpeg-4 encoder with a highly tuned mpeg-2 encoder at the same bitrate it can surely also happen that that the mpeg-2 encoder beats the mpeg-4 one

[Sharktooth]

MPEG-2 can easily beat MPEG-4.
Compressing data in a lossy way is a science. The more you try to compress the more are the chances you loose quality and details.
Im talking about maximum best achievable quality and not bitrate...

[mf]

With the right matrix and B-Frame settings, MPEG4 can beat MPEG2 hands down. It has unlimited GOP size, and thus is able to benefit greatly from static scenes. Not to mention all the Advanced Simple quality refinement features. Custom matrices compensate for MPEG4's lack of DC precision. Just search the XviD forums for "near lossless" - I think there's a thread that discusses studio quality MPEG4 encoding.

[Sharktooth]

I dont think any MPEG-4 codec at its maximum achievable quality (even at quant 1) and with the best possible quant. matrix can beat a good MPEG-2 codec at its best.
Its simply a matter of compression. It's like audio codecs... mp4 cant beat mp3 at its maximum quality (regardless of the bitrate) because mp3 "removes" less data...
A special pourpose MPEG-4 codec or Studio Profiles and other features aimed for near lossless quality in current codecs maybe...

[Teegedeck]

Sharktooth, with all due respect, this is... Well; not true.

MP3 is nowhere 'less lossy' with audio data than more modern codecs, that's simply a legend. In fact it's more crude and buggy in its concept, concerning filterbanks etc. A lot of work by a lot of motivated, brillant people went into tuning LAME in order to overcome problems that might have been caused by that.

We could discuss this in the audio forum or at hydrogenaudio. Let me just say that MP3 is by design and in practice worse than AAC ('mp4') at high bitrates, too. LAME aps might be able to stand up against AAC at a setting that produces a comparable bitrate but we are talking transparency here. That means, both codecs are undistinguishable from the original, so there is no better or worse anymore. And AAC seems to reach that transparency at quite a lower bitrate than MP3.

See this blindtest (http://www.rjamorim.com/test/128extension/results.html) for the 128 kbps range, overall results at the bottom. Note that '5' would equal transparency. AAC (iTunes) reaches 4.42, MPC reaches 4.51 and MP3 (LAME) reaches 3.66. So whereas MPC and AAC already are almost transparent (on these samples) at 128 kbps, LAME should need around 200 kbps (depending on sample) for that.

And if you do blindtests with MPEG-4 (XviD) against MPEG-2 (CCE) from an uncompressed source I bet my pants that you'll similarly reach transparency with MPEG-4 at a much lower bitrate (about half, I guess). Similar to MP3, MPEG-2 is nowhere 'more benign' to video data. It simply is a less sophisticated way of compression.

[Sharktooth]

That's a good point if we're discussing about bitrate.
I stated "regardless of the bitrate".
I mean a complex compression algorithm is always reducing quality somewhere to achieve better compressibility and less data to store.
A simplier (and tuned) one just behaves in the opposite way. It still reduces data and quality but it does it in a less aggressive way keeping more informations about the image (or audio).
Someone may say those data are the less relevant for human eyes (or hears) but still the most advanced and complex codecs try to "remove" unuseful data to optimize for occupied space.
Those data are reconstructed (at least) at playback time or lost.

[Teegedeck]

Yes, and I was talking about transparency. All that lossy codecs aim for is reaching a quality for the compressed result that the human eye cannot distinguish from the uncompressed original. If it reaches that aim it is successful, and I for one am satisfied.

The feeling that although you can't name (=see) it something is lost, is very common and buggs many quality fanatics, like the people at hydrogenaudio. The only way out is to use lossless codecs, not to use older codecs.

But for a reliefing story: An acquaintance of mine works for a TV station. Many years ago, he was a Laserdisc fanatic, paid immense summs for the things; players and discs were abolutely rare here and had to be imported and bought at steep prices. The state of video compression was MPEG-1 at that time. He and a Laserdisc shop-owner simply laughed at me when I prophecised that compactdisc-sized compressed video would have a big breakthrough, one day. They went at lengths to explain to me why this never could be done with so many lines of video data per picture and how they would never buy such a thing where so much must be lost. Of course my aquaintance bought one of the first DVD-players and didn't feel hard about lossy compression anymore, simply because in spite of being lossy DVDs still yield much better quality than LDs.

So if you feel uneasy with lossy (lossier) compression, use huffyuv or MJPEG, otherwise trust your eyes. Of course this doesn't go for recompressing MPEG-2 - here you should stick with the original or compressed-domain transcoders if bitrate doesn't matter.

[Tommy Carrot]

Originally posted by Sharktooth
That's a good point if we're discussing about bitrate.
I stated "regardless of the bitrate".
I mean a complex compression algorithm is always reducing quality somewhere to achieve better compressibility and less data to store.
A simplier (and tuned) one just behaves in the opposite way. It still reduces data and quality but it does it in a less aggressive way keeping more informations about the image (or audio).
Someone may say those data are the less relevant for human eyes (or hears) but still the most advanced and complex codecs try to "remove" unuseful data to optimize for occupied space.
Those data are reconstructed (at least) at playback time or lost.

Hmm, just a (perhaps bad) example: H.264 has much more complex algorithm than mpeg4 ASP, still in my experience it can reach the perceived transparency at similar bitrate range as mpeg4 ASP quant 3, where the latter is nowhere near to indistinguishable(?) from the original source. I guess the same improvement is true in the relation of mpeg4 and mpeg2.

[Teegedeck]

I think so. It is a purely theoretical question whether an older codec with restricted options will potentially have problems with less samples (because what ain't there cannot break, or because it's been tuned for a lot of known problem samples) or whether a newer, more complex codec will have problems with less samples (because it has overcome the design-flaws of older codecs or at least doesn't suffer from the same restrictions - i.e. GOP structure etc.). Fact is, restrictions in design cannot be overcome. In practice, however it comes down to the implementation.

If a codec cannot deliver a 'transparent' encoding at all, although it claims it can, then it simply fails to do what it has been designed to do and hence can be called 'buggy'. Whether an MPEG-4 codec is as good as an MPEG-2 codec, regardless of bitrate, depends on the quality of the work its developers put into it, not on the design of the specs.

It gets interesting when people like a compressed sample more than the original. :) This often happens in MP3 listening-tests when it isn't known which sample is the original. How do we judge a codec when it clearly alters the original's look but people rate it better? To me such a codec fails because this isn't what a codec should do IMHO.

[KpeX]

Originally posted by Sharktooth
I dont think any MPEG-4 codec at its maximum achievable quality (even at quant 1) and with the best possible quant. matrix can beat a good MPEG-2 codec at its best.
Its simply a matter of compression. It's like audio codecs... mp4 cant beat mp3 at its maximum quality (regardless of the bitrate) because mp3 "removes" less data...
A special pourpose MPEG-4 codec or Studio Profiles and other features aimed for near lossless quality in current codecs maybe... As Teegedeck already mentioned, a more advanced compression algorithm does not remove more information regardless of the situation; rather, it improves the quality at any given bitrate. I think the fact that the vast majority of video sources available today are MPEG-2 is leading to this false impression that MPEG-2 is higher quality.

[Soulhunter]

Im still under impression that MPEG2 copes much better with noisy stuff... :o


Bye

[lordadmira]

Yes, that's one of my big gripes. Xvid just dies on noisy material. Movies and DVD's that have "grain" don't get any benefit from Xvid's "superior" algorithms. But mathematically "noise" is no different from any other similarly complex data. Noise is a human subjective term. Legitimate image data can be just as complex. So what can we do about image data that is pushing Xvid to the breaking point?

[lordadmira]

Originally posted by KpeX
As Teegedeck already mentioned, a more advanced compression algorithm does not remove more information regardless of the situation; rather, it improves the quality at any given bitrate. I think the fact that the vast majority of video sources available today are MPEG-2 is leading to this false impression that MPEG-2 is higher quality. But you can't deny the fact that the maximum achievable quality contest is being won by Mpeg-2. And under high stress conditions Mpeg-2 can keep going where Xvid fails.

[RadicalEd]

Originally posted by lordadmira
But mathematically "noise" is no different from any other similarly complex data. Noise is a human subjective term. Legitimate image data can be just as complex.

Not really, you'll be hard pressed to find natural images that have completely random pixel data...

Also, I'm not really sure where the compression algorithms of mpeg-4 differ that significantly from mpeg-2 or 1. The only quality limiter is the quantization method, really, and AFAIK mpeg 4's is no more subject to loss than any other.

[lordadmira]

Dirty objects exhibit this noisy structure. Especially wood and rock. And also many special effects.

Ok ur making a distinction between theory and implementation. So which is it that's failing, and where.

[RadicalEd]

Even those are patterned, and the DCT is well enough fit to cope with this. So codec-wise, there is a difference between a natural image and random data. As for theory vs implementation, well, theory leaves little question. Implementation is responsible for doing the best at concretizing that. So I'd blame implementation http://www.animemusicvideos.org/phpBB/images/smiles/icon_neutral.gif The only cause for data loss at the maximum possible quality is rounding errors, and most speed optimizations usually come from compromising accuracy. An mpeg-2 encoder that uses a 32 bit floating point accuracy DCT will certainly retain the slightest amount more than an mpeg-4 encoder using a fixed-point integer approximation.

[Teegedeck]

1)
Originally posted by lordadmira
But you can't deny the fact that the maximum achievable quality contest is being won by Mpeg-2.

Proofs for that statement, please! At least I haven't read such a test...

We don't let such global statements go without proof, anymore.

2)
And under high stress conditions Mpeg-2 can keep going where Xvid fails.

What do you mean by 'fails'? And on which sources have you tested that? I sure hope it wasn't already compressed data. Did you test at the same bitrate?

As for noise being a problem: I'd expect XviD to perform as well as MPEG-2 at least at the same bitrate. Whereas there probably isn't much bitrate to save with XviD in such a case. There are two explanations for such behavior (which I haven't encountered, yet). Firstly, what is detected as motion by the codec? IMHO as a layman, MPEG-4 (XviD) improves over MPEG-2 by more effciently coding the differences between frames. But if those frames differ radically due to random noise (motion), there's little to be optimized, in other words mechanisms designed to encode real motion vectors more space-efficiently might perhaps become useless when trying to encode random motion vectors. Whereas MPEG-2 just throws in an i-frame every half a second or something and doesn't even attempt coding longer motion-vectors. The correct solution is to filter out random motion before encoding. Possibly VHQ and high motion-estimation efforts become counter-productive without such filtering. This would have to be tested. Secondly, most matrices quantize grain away to a good extend. Use for example Didée's semi-insane matrix to prevent that. That means solving a problem by throwing a lot of bits at the problem sample. So I can imagine that the statement might perhaps be valid that 'the efficiency gain of MPEG-4 in its XviD-implementation over MPEG-2 doesn't hold true for very noisy material' but I cannot believe that 'MPEG-2 is better than MPEG-4 at noisy sources at the same bitrate'. But I stand to be corrected by more knowledgeable people.

3)
Originally posted by lordadmira
Dirty objects exhibit this noisy structure. Especially wood and rock. And also many special effects.
Firstly, structure is not the same as motion. A piece of wood isn't noise by nature, it is because of the quality of the equipment used in filming it / in mastering. Secondly, the motion vectors of special effects, like explosions, are not random.

[mf]

You could call falling sand complex enough to be considered random.

[Teegedeck]

Well, it does obey to the law of gravity by going down... ;) (Even sand in zero-gravity would not change directions randomly.) Also the question is how much detail becomes quantized away so that what were individual, moving particles are treated as structure, actually.

Anyway, maybe if one wants to counter a noisy problem-sample at the cost of loosing MPEG-4's compressiblity gain, one should also imitate MPEG-2's behaviour in doing away with long GOPs and throwing in I-frames at every 16th frame. I should think it would look very similar to MPEG-2 then. :D

Edit: I've added a sentence to my reply above for the sake of completeness.

[hellfred]

@ lordadmira and kadajawi and all the others
As both of you, and maybe others, too, have been fighting to compress noisy material, i will ask you for your advice:
I want to compress material (animes and real movies) captured with my capture card that are very noisy. They are next to equally noisy no matter if i capture in RGB, YUY2 or YV12 colourspace, no matter if i use PicMotion MJPEG codec, HuffYUV (original or ffdshow for YV12 material) or if i leave it uncompressed.
So what cleaner can you suggest?
Some examples can be found here:

One (http://www.stud.tu-muenchen.de/~stefan.guertler/one.png)
Two (http://www.stud.tu-muenchen.de/~stefan.guertler/two.png)
Three (http://www.stud.tu-muenchen.de/~stefan.guertler/three.png)
Four (http://www.stud.tu-muenchen.de/~stefan.guertler/four.png)
Five (http://www.stud.tu-muenchen.de/~stefan.guertler/five.png)
Six (http://www.stud.tu-muenchen.de/~stefan.guertler/six.png)
Seven (http://www.stud.tu-muenchen.de/~stefan.guertler/seven.png)

One tip i got but have not tryed yet is to reduce the saturation of the capture card before record.
Any other suggestions?
The capture card has a Philips chip on it.
I am using VirtualVCR for capturing and the machine is quite powerfull . P4 @3.0g6 GHz and next to no dropped frames, so i may be able to use a cleaning filter at recording time.

Hellfred

[ominte]

Originally posted by Teegedeck
Proofs for that statement, please! At least I haven't read such a test...

We don't let such global statements go without proof, anymore.
If you would like proof just take a look at the following thread titled
XviD can give Studio Quality (http://forum.doom9.org/showthread.php?threadid=53632&highlight=studio+quality). In the thread two members of the forum state that when it comes to uncompromising quality MPEG 2 is superior to MPEG 4.

Originally posted by temporance
Studio quality is normally taken to mean a high bitrate (>34Mbit/s) and the use of 4:2:2 colorspace. xvid always encodes in 4:2:0, so you lose half the chrominance vertical resolution and, by convention, xvid is not capbable of studio quality.
Oringinally posted by Syskin
Mpeg-2 studio profiles support different colourspaces (YUY2) and have slightly different quantization (allows 12-bit DC precision, normal profiles can't go higher than 8 bit).
From the above comments you can see the reason MPEG 2 is ultimately more powerful than MPEG 4 is not because of compression algorithms but due to colorspaces.

[gotaserena]

In addition to the points above I'd like to add that MPEG-2 is still better when it comes to encoding interlaced material. In my captures I usually get a better result with MPEG-2 at DVD bitrates (4-6 Mbps), and XviD usually asks for 7-8 Mbps to encode a source at full D1 (PAL) at a reasonable quantizer. That's at least my experience.

But I take well lordadmira's point above. Even without the caveat of (de)interlacing, MPEG-4 in general doesn't cope too well with noisy sources, or even scenes with lots of gradations of grey. Foggy scenes are usually labelled "difficult to encode" by people here. I guess that the colourspace is to blame, with MPEG-2's YUY2 being superior to the half-chroma resolution MPEG-4's YV12 (is this standard?). But I'll retract this statement and wait till somebody with more knowledge bumps into the discussion.

[Omni]

xvid normally beats mpeg2 at the same bitrate but at lower bitrates dark scenes are horrible. the only way to get those scenes stabel is to reduce or completly drop b-frames and use a very high bitrate matrix and then you'll loose your bitrate advantage over mpeg2. mpeg2 is a bit older and mpeg4 is till now not out of it's developmentphase so maybe there is some space for improvement. (would it be possible to implement another colorspace in xvid?) i'm sure they'll find a solution for dark scenes@lower bitrates.
i made some tests with computer rendered scenes (some cgi movies) and used uncompressed avi which was later converted to mpeg2 and mpeg4 (xvid)
i used one of my own matrices and i have to say that xvid beats mpeg2 (bitrate was for both about 3mbit/s, mpeg2 was encoded with cce, xvid was an early build of 2003 since i've done this test last year) in the overall quality. no big blocks at critical colors (full red green or blue) dark scenes looked a bit noiser but overall looked good. mpeg2 has the typical edge artifacts an blocks. the only thing i really like on mpeg2: it looks more dynamik even if it has blocks.
(this is only thought as my personal opinion just read it, i didn't want to disturb this discussion ^^ but i think mpeg2 is good, mpeg4 better if you look at the concept mpeg4 just isn't tuned correctly)

[Teegedeck]

Friends, friends; let's not get carried away. When I say 'proof' I mean the only proof there is for human-perception related quality and that is: a blind-test.

You can try to summ up features missing from MPEG-2 against features missing from MPEG-4 but you don't prove anything with that, ominte.

How much visible impact does MPEG-2's support for 12-bit DC precision have when compared to MPEG-4's quarterpel and use of custom quantization matrices? I would demand at least one well-performed blind-test for a statement as ominte's to let it pass.

How does MPEG-4 compare against MPEG-2 in a noisy scene when you reduce XviD's GOP-length or fiddle with VHQ-settings? These things still need to be tested out. How well does MPEG-4 (Xvid) handle dark scenes compared to MPEG-2 once sysKin's XviD HVS-plugins are ready? This cannot be tested, yet.

So, please, understand that we don't have any solid proof for claims that there exists any case where MPEG-2 as a compressions scheme could be better than MPEG-4! Partially this is due to development of this MPEG-4 codec (XviD) still in progress (as for dark scenes); partially this is because I would not let noisy scenes go as an example before you've tried to make MPEG-4 behave like MPEG-2 by reducing GOP length and search intensity - but most importantly because there hasn't been a blind-test with a sufficiently broad selection of samples, encoded at well-discussed and approved settings and conducted with enough participants to be statistically valid. Perhaps it would also be better to talk about implementations only, like XviD vs. CCE.

Therefore: In your statements use words like 'to my eyes' and 'with these two/three/whatever samples' as for example gotaserena did. And if you yourself have done a blind-test and thus can seriously claim to make a valid statement, always give us your settings for both encodings, starting with the bitrate and the source you've used and continuing with your scripts and XviD-settings.

If you don't take at least my former advice ('to my eyes' etc.) seriously this thread is in danger of becoming a senseless codec-bashing (both directions) and will be closed. It's getting dangerously near a 'what's best' discussion.

[gotaserena]

Since you are the moderator, there's very little one can do then, isn't there? I mean if we take for granted that

"MPEG-4 is superior to MPEG-2 no matter what,"

there is indeed very little point in discussing.

But as far as I could gather the argument was going along the lines of what the encoders can do as of today with the tools available. Heck, one can even make MPEG-1 look nice (or "transparent") with enough fiddling. But that is not the point, or isn't as I saw this thread. The point is that it seems that the MPEG-4 encoders aren't doing a suitable job in some specific scenes. The question posed is whether this is the result of the MPEG-4 encoders being very young and not really optimised for a broad type of sources or is it indeed some technical limitation of the MPEG-4 format.

I, for my part, like to believe the former. In that case, I guess that one should try and work these impairments out. As for us laymen in programming, the only thing we can do is to point the direction out. I don't think it was in anyone's mind to say that "A" is a superior codec than "B", just that the encoders of "A" are doing a better job in some situations than the encoders of "B".

[Teegedeck]

Don't get me wrong. If you've read the thread from the beginning, you know that IMHO both MPEG schemes should be capable of reaching transparency (i.e. a state where it should be undistinguishable from the uncompressed original) at some bitrate. MPEG-4 is designed to reach transparency at a lower bitrate. At a level of transparency there should be no difference between MPEG-2 and MPEG-4 simply because the definition of transparency doesn't allow for such a difference.

The problem is whether, and if yes where, we can conclude from shortcomings of the implementation to shortcomings of the scheme. I'm not so sure here, so I'd like to ask everyone to be careful and only talk about XviD and, for example, CCE instead of MPEG-2 and MPEG-4.

Edit: So the original question 'Can XviD beat MPEG-2 at DVD res?' could be answered: 'If XviD cannot look as good as MPEG-2 at DVD res then there's still something to be improved, isn't there?'

Right? :D

[Wilbert]

Since you are the moderator, there's very little one can do then, isn't there? I mean if we take for granted that

"MPEG-4 is superior to MPEG-2 no matter what,"
You can start by making a representative test. Capture a music clip (or sports) from tv (huffyuv). Encode it to XviD and mpeg2 (svcd for example) with comparable bitrate (deinterlace it, resize it, but no additional filtering and no post/pre processing). Post a bunch of screenshots for comparison.

AFAIC, a blind test is not necessary. We all have good eyes, when telling each other what to look for.

Be aware that XviD and DivX5 are less good than DivX3 SBC/FFVFW wrt motion estimation when there's noise present:

http://www.geocities.com/wilbertdijkhof/analog_comp/comparison.htm

[Nic]

@gotaserena:
Good post. And I myself do not know the answer. Its a question that has been brought up many times. In theory MPEG-4 should always win at high bitrates/res, as it is just more complicated and more efficient than MPEG-2. The fact it doesn't is either down to the tuning of the encoder or MPEG-4 itself. My guess is its probably a little of both...but it is only a guess.
Using a "high quality" matrix for encoding should really help determine MPEG-4 as the winner at high-bitrate/high-res tests with MPEG-2

With regards to colorspace. Yes, AFAIK (from looking at the spec) MPEG-4 only supports 4:2:0, where as MPEG-2 does support 4:2:2 and even 4:4:4, but seeing as probably none of us use such profiles. The argument isn't everso valid here.

With regard to this thread, please keep it friendly and try not to voice too strongly an opinion if you cannot back it up. If this thread starts to turn sour, it will be closed.

-Nic

[kadajawi]

Originally posted by hellfred
@ lordadmira and kadajawi and all the others
As both of you, and maybe others, too, have been fighting to compress noisy material, i will ask you for your advice:
I want to compress material (animes and real movies) captured with my capture card that are very noisy. They are next to equally noisy no matter if i capture in RGB, YUY2 or YV12 colourspace, no matter if i use PicMotion MJPEG codec, HuffYUV (original or ffdshow for YV12 material) or if i leave it uncompressed.
So what cleaner can you suggest?
Some examples can be found here:

Hellfred
I'm using WinDVR 3 to capture. First thing is the file size is pretty ok (MPEG 2 at a high bitrate (7000+), but if you don't look too hard it looks like the original source). Then second it seems to filter a bit, or it has a better reception quality than VirtualDub etc.

ConvertToYV12
UnDot()
VagueDenoiser(threshold=0.4,method=1,nsteps=6,chroma= true)
Convolution3d (preset="movieLQ")
This is the script I'm using, mainly varying on the treshold or switching from movieLQ to movieHQ. Depending on the source. I don't know if this is the best set of filters (I don't really believe that), but it seems to work fine for me. Last "filtering" is being done by XviD, I'm doing 2 pass and selecting the second pass target file size so small that some of the noise gets thrown out while keeping detail etc. I'm using hvs good btw.

Would be nice to get some suggestions on doing better.

[SeeMoreDigital]

I would have thought we could use some 1920x1080 'true 16:9 frame' HiDef Mpeg2 captures as a reference source. Or even (dare I say it) HiDef WMV9!

As long as the source is HiDef and containes more pixels than the re-encoded Mpeg2 and Mpeg4 frame (ie: anamorphic 720x480), the results should still be valid.

What say you guys?

[hellfred]

Originally posted by kadajawi
I'm using WinDVR 3 to capture. First thing is the file size is pretty ok (MPEG 2 at a high bitrate (7000+), but if you don't look too hard it looks like the original source). Then second it seems to filter a bit, or it has a better reception quality than VirtualDub etc.

ConvertToYV12
UnDot()
VagueDenoiser(threshold=0.4,method=1,nsteps=6,chroma= true)
Convolution3d (preset="movieLQ")
This is the script I'm using, mainly varying on the treshold or switching from movieLQ to movieHQ. Depending on the source. I don't know if this is the best set of filters (I don't really believe that), but it seems to work fine for me. Last "filtering" is being done by XviD, I'm doing 2 pass and selecting the second pass target file size so small that some of the noise gets thrown out while keeping detail etc. I'm using hvs good btw.

Would be nice to get some suggestions on doing better.

Three filters in a row, doesn't that have a hard impact on encoder time needed? And when and how are you doing deinterlacing?
Lately i stumbled over doom9's postprocessing using avisynth page (http://www.doom9.org/index.html?/capture/postprocessing_avisynth.html). I am now trying the filters suggested for AviSynth v2.5 in the section Smoothing:
# AviSynth v2.5x plugins:
# Convolution3d(preset="vhsBQ")
# FluxSmooth(7, 7)
# STMedianFilter(8, 15, 4, 7)
# deen("c3d", 0, 10, 12, 3)
I will have a look at your VagueDenoiser-Filter. But i am afraid that up to now i am not able to advice others on filtering.

Hellfred

[lordadmira]

Yes! Yes! Now we're actually getting somewhere. I'll post a more detailed reply later when I have time, but I think I've narrowed it to three specific components.

[Stux]

Originally posted by Nic
@gotaserena:
Good post. And I myself do not know the answer. Its a question that has been brought up many times. In theory MPEG-4 should always win at high bitrates/res, as it is just more complicated and more efficient than MPEG-2. The fact it doesn't is either down to the tuning of the encoder or MPEG-4 itself. My guess is its probably a little of both...but it is only a guess.
Using a "high quality" matrix for encoding should really help determine MPEG-4 as the winner at high-bitrate/high-res tests with MPEG-2

With regards to colorspace. Yes, AFAIK (from looking at the spec) MPEG-4 only supports 4:2:0, where as MPEG-2 does support 4:2:2 and even 4:4:4, but seeing as probably none of us use such profiles. The argument isn't everso valid here.

With regard to this thread, please keep it friendly and try not to voice too strongly an opinion if you cannot back it up. If this thread starts to turn sour, it will be closed.

-Nic

MPEG-4 supports YUV422 and 12bpp colour

Its just I don't know of any codecs which actually implement that ;)

MPEG-4 uses a 12bit DC coefficient.

Actually I think the latest revisions also have support for 444

[Stux]

Originally posted by ominte
If you would like proof just take a look at the following thread titled
XviD can give Studio Quality (http://forum.doom9.org/showthread.php?threadid=53632&highlight=studio+quality). In the thread two members of the forum state that when it comes to uncompromising quality MPEG 2 is superior to MPEG 4.

Do not confuse one implementation with the full capabilities of the standard.

I have studied the said standards quite deeply, and I can think of no reason why MPEG-2 would be 'superior' to MPEG-4, other than the fact that it's implemented in a helluva lot of hardware.

[gotaserena]

Great Stux, this is refreshing. It would make sense for the MPEG-4 to somehow "contain" MPEG-2, but it is good to hear this from someone involved in the process.

So it is really a matter of XviD not being particularly optimised for the task. Thankfully the codec is very flexible so one can fiddle a lot with the options given so far. The question is whether there is some not yet implemented point of the standard which can help in these situations or if this is just a matter of finding the exact settings.

By the way, I wanted to ask one question, if I may: is XviD purpose to be the most complete video encoding MPEG-4 implementation, or is it going to be tuned to the job it is being used for so far, namely DVDs backups?

I'm still puzzled as to why this question is evoking so many calls of "let's keep this civil, please". Maybe it's my job but I have not seen so far anything that would raise the temperature of anyone's blood :D

[Teegedeck]

Heh, it's more a consideration because we've seen heated MPEG2-against-MPEG4-discussions before...

And I haven't found anything in your posts to be criticized, quite the opposite. :)

BTW, my bet would be that the XviD devels would answer your question with "XviD will be optimized to do anything the people participating in the project want and can optimize it to do". ;)

And thanks, Stux, for your insider-comments.

[Nic]

Thanks loads Stux...obviously the small part of the MPEG-4 spec I have, isn't enough. :)

Cheers,
-Nic

@gotaserena:
As a moderator, you learn to preempt problems, which helps keep things under control. Thanks why myself and Tee gently keep things on track. Just in case :)

OT:
@Stux: I want to purchase the full document, mainly so I can be a bit more afformative about such issues. Is 14496-2 the only document I really need?
I'm assuming: http://www.iso.ch/iso/en/CatalogueDetailPage.CatalogueDetail?CSNUMBER=36928&ICS1=35&ICS2=40&ICS3=
Is the best place to buy it from....

[Stux]

@Stux: I want to purchase the full document, mainly so I can be a bit more afformative about such issues. Is 14496-2 the only document I really need?
I'm assuming: http://www.iso.ch/iso/en/CatalogueD...5&ICS2=40&ICS3=
Is the best place to buy it from....

14496-2 is the visual spec I believe, that should cover all this type of stuff, it won't necessarily cover visual in MP4, aac, MP4, BIFS etc

The difference is I'm usually working from the latest specs where the ink is barely dry, and sometimes even the draft-specs.

But I don't actually know about best places to buy and if that's all you'll need for instance. I'm not our "specmaster" (like a webmaster... but for specs ;) )

[Nic]

Thanks Stux, maybe ill wait to see if I can get a more recent spec from somewhere else then. Ill have to find me a specmaster ;)

Cheers,
-Nic

[mf]

Maybe we'll have to call in Mfa a.k.a. "The Paper" :D :D.

[Soulhunter]

Originally posted by Soulhunter
Im still under impression that MPEG2 copes much better with noisy stuff... :oWow, after 1 day net abstinence its quite interesting to see what such a drop could effect... :eek:


Bye