Interlaced Encoding,Cartoon Mode,Capping Quants,Lots of questions,lots of answers!

[gpower2]

When DivX Networks released DivX 5.03, I was so disappointed by the results, that I turned to XviD. Since then, I use XviD and it seems to me that it's the best Mpeg-4 codec that exists out there! I like to consider myself an advanced user, however, there are a lot of issues that I would like to discuss in this forum that might sound stupid and newbie-like. If they are indeed, then just say so and I will shut up! I just want to know...

1st:
Interlaced encoding.
It's an option that exists in XviD and DivX but I never really understood what it does exactly. Ok, when I have an interlaced source e.g. TV captures, VideoCamera captures or even some DVD source, I check this box and I don't use any deinterlacers. What's the difference if I didn't check it? Personally I never saw any difference. And why is this recommended instead of using a deinterlacer? Does it affect quality? I know that interlaced sources are considered better than progressive ones but I never SAW any difference. And when I have an interlaced compressed video, when I play it, I still see the lines between the fields, so I need a deinterlacer anyway! So my question is, when do we use it and what for.

2nd:
Deinterlacing NTSC.
Well, this is a question that I should probably not ask it here, but lets try! Since I live in Europe, most DVDs I compress are PAL-Progressive. Occasionally I find some other Region DVDs, US mostly, and some Asian ones. Most of them are NTSC interlaced or film content. I usually use TomsMoComp() for PAL-Interlaced content, but for NTSC that doesn't work at all! So I use DeComb from Donald Graft and I try fieldDeinterlace() and Telecide() but I don't get better results. Can you help me with this? Any suggestions for other Deinterlacers or maybe settings?

3rd:
Cartoon Mode.
It's an option for anime. Ok. What does it do? I never saw anything about this, only "use it when you have anime content". There are also a lot of people who doesn't propose the use of it even with anime content! Why is that?

4th:
Capping Quantizers.
It's an issue I've discussed a year later here, but I never got some objective answers. When I started encoding with DivX 4, I had read an article in a local PC magazine which consulted readers to cap the quants. Since then,I cap them! For quality compressions I may use 2min-4max for I&P frames, but normally I use 2min-6max (I leave B-frames intact 2min-31max). Is this good, bad or just unnecessary? When I tried not to cap them I got some results that were not visually acceptable for me. Since I go for 2-passes and 2CD encodes, I never saw size prediction failure or other weird effects. Your opinion?

5th:
Meaning of quantizers.
I know that quantizers define the level of compression of a frame, I know that Q2 is better than Q8 or Q31 and I know that Q1 should not be used, but I don't know how this is done. Meaning, does it affect the bitrate that is going to be used for the specified frame, does it affect the final filesize, does it affect the bitrate curve?

6th:
Bitrate Curve for 2-pass encodes.
In 2-pass encodes, during the first pass, the codec creates a stats file which contains the statistics of the video file to be compressed and determines the allocation of the bitrate according to bitrate curve. Am I correct? I hope yes, or else I can't understand what I read! Well, my question is, should the bitrate curve be linear as it is by default or would it be better to change the settings of Curve Compression? I am not asking to tell me which is best, I just want to discuss this, because I've never read anything about it.

7th:
Pause Feature?
This is perhaps the most stupid question ever, but I've never seen anyone asking or talking about it: Why can't we pause the process of encoding and resume it at a later stage? Is it doable but really complex? Or it just can't be done? Does it have to do with the Codec architecture? I might sound silly, but I'm dying to have an answer!

@TeegeDeck
I read a lot of your posts and you say that you like to encode @2000kbps bitrate with GMC and Adaptive quantization(apart from the other settings). Do you find it necessary to use these options? Aren't they supposed to lower quality in order to save bits for encoding? I think that at these high bitrates you don't need these extra bits. Can you convince me for the opposite?

That's all folks! I hope you find my thread interesting! :)

[scharfis_brain]

1) mainly:

progressive encoding = progressive YV12

interlaced encoding = interlaced YV12

encoding interlaced as progressive will mess up all the colors in moving areas!
also the interlaced structur may be harmed.
that's why there is an interlaced option.

2) depends on source.
pure NTSC-video : tomsmocomp
pure NTSC-film :telecide(...).decimate(5)

everything other needs custom treatment. (stepping through the video fieldwise, counting unique fields, looking for blends and more...)
so if you feel unable to do this yourself an unprocessed sample is needed.

[Teegedeck]

Originally posted by gpower2
@TeegeDeck
I read a lot of your posts and you say that you like to encode @2000kbps bitrate with GMC and Adaptive quantization(apart from the other settings). Do you find it necessary to use these options? Aren't they supposed to lower quality in order to save bits for encoding? I think that at these high bitrates you don't need these extra bits. Can you convince me for the opposite?
Well, I certainly hope I can.:) You could only think these bitrates are high if you are used to downsizing your resolution. I am not. I don't resize. And when you keep in mind that I re-encode a 5 Mbit MPEG-2 file into a 2 Mbit MPEG-4 file (all bitrates just estimates) at no visible loss in quality you'll understand that I really need all ASP features.

Let me give an example: If you want to encode a movie with the SixOfNine matrix at a maximum quantizer of 4, you'll have to use AQ. If you don't want quant=5 your only other option would be a lower resolution - bah!

I find all this discussion about advanced features 'degrading picture quality' or 'not being necessary' counterproductive. Lots of MPEG-4-users still don't seem to think they play in the same league as CCE-users, don't seem to dare to aim for backups that are better and smaller than what others do with CCE or InstantCopy. Others seem to be content with overfiltered 640x encodes on a CD. It is just nonsense that Advanced Simple Profile features should be there for 1 CD encodes and not for 'transparent', i.e. undistinguishable backups.

And BTW, GMC was NEVER NEVER EVER reported to lower quality in any way, but absolutely the opposite! Its only drawback is that it takes more processing power.

Edit: other selective answers:

Originally posted by gpower2
3rd:
Cartoon Mode.
It's an option for anime. Ok. What does it do? I never saw anything about this, only "use it when you have anime content". There are also a lot of people who doesn't propose the use of it even with anime content! Why is that?
Cartoon mode adjusts the ME sensitivity to cartoon-style motion and thus prevents bits getting wasted on noise. Reports that it 'smoothers' the outcome and would thus not be apt for anime but only for cartoon are untrue. And considering that anime coders usually like their stuff smoothered to a pseudo-CG look & feel, I find these claims all the more curious. Do not blindly follow any advice that says a feature is no good! Try it for yourself first or you may too easily discard something that was meant to help your encoding! (You can test this now: I say, adaptive quantization causes blocking in anime because the psychovisuals that it is based on don't apply for such artificial material. But try it for yourself before you trust me.)

Originally posted by gpower2
4th:
Capping Quantizers.
It's an issue I've discussed a year later here, but I never got some objective answers. When I started encoding with DivX 4, I had read an article in a local PC magazine which consulted readers to cap the quants. Since then,I cap them! For quality compressions I may use 2min-4max for I&P frames, but normally I use 2min-6max (I leave B-frames intact 2min-31max). Is this good, bad or just unnecessary? When I tried not to cap them I got some results that were not visually acceptable for me. Since I go for 2-passes and 2CD encodes, I never saw size prediction failure or other weird effects. Your opinion?
Capping quantizers means you don't trust the 2-pass algo. A good algo should distribute quantizers in second pass only in a narrow range (like a minimum of quant=3, a maximum of quant=5) IF you use a reasonable bitrate. If you don't capping quants prevents overquantization but at the same time throws the 2-pass algo totally of balance and will result in a) unpredictable filesize and b) wasting huge amounts of bits because of erratic over/underflows. Rather use reasonable bitrates, well-scaling custom matrices (like SixOfNine(-HVS)) or go for constant-quantizer one-pass encoding if you want to use strong compression but can afford to over- or undershoot your filesize-aim by some dozen MB. Personally I think that the quantizer distribution that SixOfNine-HVS produces even at high compression ratios is a fine example of how it should be. (Note that I don't use SixOfNine-HVS but SixOfNine for my high-bitrate encodes.)
Hope this helps a bit.

Tee

[Manao]

For the interlacing support : if your video is meant to be watched on a progressive screen ( PC monitor for example ): deinterlace before encoding it ( without interlaced in XviD then ). If it's meants to be watched on an interlaced display ( TV ), do not deinterlace it, and encode it with interlacing in XviD ( because it will be more efficient than without interlacing ).

[pogo stick]

I think that interlaced encoding is underestimated. It will look nice at both progressive and interlaced displays. Post-deinterlacing can be a good thing to do. Especially with minolta’s "xvid.ax" filter hack.
Also I think that interlaced encoding has very good potential in working with hybrid NTSC (interlaced/film) video. Imagine how auto-BOB/IVTC in decoder would be helpful in such cases!

[Manao]

pogo stick : I can assure you that if you intend to deinterlace your video before watching it, it is better to deinterlace it before encoding . Deinterlacing will be more efficient, because deinterlacer won't have to cope with encoding artifacts, and encoding size will be smaller.

[pogo stick]

Yes, you are right. But not in case with bob deinterlacing. In that case there can be any encoding artifacts. It shows fields separately and resized. Maybe it’s not the best method (I am not deinterlacing guru :)), but it looks good for me and definitely will not cause deinterlacing artifacts. And it’s easier to do - no deinterlacing.
And you are right about final file size too.
So, it’s like: should I encode anamorphic or should I resize?
Questionable, but definitely worth trying. :)

[minolta]

When encoding telecined or badly-blended clips, one should choose progressive.

With truly-interlaced or hybrid, preserving interlace should be strongly considered. De-interlace forever destroys the original (quality freaks beware). The overhead of interlace is usually under 15-20% (depends upon source). And, future xvid versions (v1.1) will de-interlace on-the-fly during playback with VMR (using 'bob' or better).

-Minolta

Also, let's stop calling interlace an "artifact" (I know this is very common, just bugs me). I'd hardly call two fields separated by 1/50th or 1/60th seconds “artifacts”. Consider that interlace has been used for the past 50 years or better. When users read "artifact", they think of negative anomalies that should be fixed. In fact, most users that happily de-interlace don't even know its true meaning...they excuse the ugly, jagged, horizontal lines as simply tv remnants that must be cleaned. The mpeg2 users have been correct (_always_ keep interlace when possible), but we “new-age” mpeg4 folks have been "destroying" perfectly valid 50i/60i videos for the sake of convenience. If bit-rate allows, interlace should be preserved...

[Bogalvator]

I agree 100% percent with Minolta - if you have a truly interlaced source you should encode as interlaced. Consider some high action, fast moving material like sports footage. Deinterlacing makes it too jerky, it should played back at it's original rate.

My main reason for requesting Milan put DGBob into FFDShow, was for the above - now one can get high quality playback at full frame rate :)

[Clock]

Okay, since this is a thread with questions, which seem to haven't been answered at other places, here's mine, concerning quants and CQM.

I have to admit, I'm a user which aims for best quality/size with premium attention to size. This means, that I even encode whole movies down to 300MB if I have the impression, that compared to a 600MB encode, it doesn't have "half quality". Sure that's personal taste, just wanted to note anyhow...

My problem starts with choosing a CQM. Usually, I tried some of the rather heavy compressing matrizes (AVAMAT, 1CD, ...) on one 1st pass and compared the results. As you can guess, I'm no quality-freak. Sure I also aim for the best one, but within constraints.
So I mostly compare the filesizes of the first pass, and if I don't see big visual differences, I'll go with the best compressing matrix.

But more and more, reading posts, I get the impression that this is not the right way. That some matrizes are better at higher quants than those, that beat them at q2.

I admit, that I don't know jack about CQMs. I read about frequenzies and the likes, but I'd like to know a way to choose a CQM after the first pass, not the second (and it would be great, if there were only few to choose).

Clock

[gpower2]

@Teegedeck:

"Global Motion Compensation can be useful to save bits on panning scenes..."
"Adaptive quantization is the formerly called lumi-masking, which tries to give less bitrate to scenes that the human eye can't see very well..."

These are taken from the Xvid-Vdub-final guide@Doom9. Less bits doesn't always mean less quality, I know that, but in some cases it does. Personally I find that AQ lowers quality at bitrates above 1Mbit, as for GMC, I find that it makes the decoder's job more difficult at pan scenes, since I find the movement not so smooth. I also can't see the difference in quality between VHQ4 and VHQ1. Is it visually distinguishable? As for the resolution I use, I crop and then resize at resolutions that are approximately 240000-300000 pixels (640x480=307200). I believe it is a nice resolution and the new Lanzcos4resize resizer of Avisynth2.55 is beautiful! Last thing to ask you, do you think it is best to use a CQM or does MPEG still does the trick? Thanks for your answers!

@scharfis_brain

Hmm, I get the picture here, but when you say "custom treatment" what do you mean? The use of other deinterlacers? The use of other parameters? And how do you decide what to use, by trying and testing? If you have a link to a guide or some site with info about this I would really like to have it and save you the writing! Thanx!


@minolta
"And, future xvid versions (v1.1) will de-interlace on-the-fly during playback with VMR (using 'bob' or better)."

You mean that the codec itself will include a deinterlacer? That's quite cool! I have to ask you, which player do you suggest to watch interlaced videos? I personally prefer first bsplayer and second media player classic by Gabest. Do you know how I can enable deinterlacing in these players? Thank U!


@clock
Well, I haven't read any threads about CQMs at such low bitrates, or should I say Ultra Low? 300MBs? I guess you use 500kbps max? You should probably compress the audio too! Maybe Mp3@32000Hz, 96kbps, stereo? Anyway, about the CQMs, best thing to do is try and see which suits you best, but first read the Soulhunter's thread about CQM testing, and you should learn some basic, which ones are for high bitrates and which ones are for low. The rest is up to your taste!


@everyone
Still noone has replied to my last 3 questions, I would be really gratefull if someone did, because I am most curious about their answers! Thank U all!

[Manao]

5th : before encoding a frame, the encoder knows which size it wants to reach. If it's a second pass, he knows the size the frame had taken when it was encoded at quant 2, and inside the encoder is a model which allows to deduce from framesize at quant 2 what will be the framesize at quant n. Hence, it can choose the quantizer which should give the closest framesize. Appart from that explanation, I don't quite see what you're asking for in this question

7th : pause feature : it would need modifications from both codec and encoding tool ( vdub for example ) : the codec would need to save its current state ( two reference frames, and all its internal data ), the encoding app would need know at which frame the encoding stopped, and be able to give the next one to the codec on resume. It could be done, of course, but not easily, and there are other priorities.

For the player question : both can use ffdshow to playback files, and deinterlacing can be done through ffdshow.

For the rest : you can see a difference between lanczos3 and 4 ? And not between VHQ4 and VHQ1 ? Strange, because the difference between VHQ4 and 1 is higher than the difference between Lanczos3 and 4 ( which I can't spot ). Use the CQM which you like the most visually speaking ( I prefer HVS-Best for example, but I have tried only a few of them )

[Teegedeck]

Originally posted by gpower2
@Teegedeck:
Less bits doesn't always mean less quality, I know that, but in some cases it does. Personally I find that AQ lowers quality at bitrates above 1Mbit, as for GMC, I find that it makes the decoder's job more difficult at pan scenes, since I find the movement not so smooth. I also can't see the difference in quality between VHQ4 and VHQ1. Is it visually distinguishable?

If you don't see the difference between VHQ=1 and VHQ=4 that's fine! You don't need to spend that much time on VHQ=4 then. Unless you are using an inferior display right now and plan to upgrade some day.

As for AQ lowering quality at bitrates above 1 Mbit: How have you tested that? Did you go for a full 2-pass encoding?

Like many people before, your seem to have the misconception that saving bits is a bad thing. No, it isn't. It's the thing that video codecs are all about - compression. Loss of quality on the other hand stems from the users decision to use too strong compression/quantization or from his/her efforts to mask its effects during some stage of the encoding process (avisynth-filtering, in-loop filtering (H.264/AVC) or postprocessing).

I for my part find that the higher base-quantizers in encodings without AQ make the video visibly and annoyingly worse when compared to an encoding of the same filesze done with AQ. I don't think you will be able to correctly spot the macroblocks that AQ compresses more strongly than others in a frame (it is all the less probable if you don't see a difference between the VHQ modes) but I think you, too, will be able to distinguish a quant=4-frame from a quant=5-frame.

Originally posted by gpower2
Last thing to ask you, do you think it is best to use a CQM or does MPEG still does the trick? Thanks for your answers!
MPEG standard seems not very detailed to me when compared to SixOfNine. But I'm talking unresized anamorphic encodes with little birate-restraints, of course.

[Didée]

Is it so that XviD uses AQ only for raising block-quantizers on the very dark/bright frame areas? Since the specs allow a quantizer change of +/- 2, it could do it otherwise as well.
(But then, regarding the fact that in XviD it's working only on those areas where stronger compression should not be noticeable, it makes sense again to only use higher block quants, and not lower ones - okay.)

And: (I assume that) in a standard 1st-pass AQ gets disabled, which would hand over the managment of AQ's 2nd-pass compression gain to overflow control only ... hmmh.

Any objections from the "XviD advanced feature expert", Tee?


On another sidenote ... dunno if it's related to AQ or not:

What the f*** is that "accurate deblocking" thingy that's often talked about? Also, when activating the displaying of it in ffdshow's OSD, it shows 2-digit numbers like "40", "44", "60", etc -- but what to tell from those numbers?

I'm feeling a 'lil dumb today ...

[Teegedeck]

*Sigh* Okay, I went overboard with my comment (being called expert by Didée feels like being called 'a selfless person' by Gandhi) about any ASP feature being good for quality. Make that well-implemented ones.
I attempt to get users to at least try using ASP features on anything else but 1-CD encodings... MPEG-4 ASP seems to have that bad reputation that it isn't good for anything but low-quality compression, while MPEG-2 is for DVDs and MPEG-4 AVC is for HDTV.

Anyway, I would have liked it, too, if XviD had redistributed data within the same frame instead of the way it is done now - though I cannot really tell how much different the outcome in two-pass would be - or whether it would have been worse. Seems to work nice the way it does now.

And no, AQ should also used in first pass, it doesn't get disabled automatically .

[Ark]

I think that MPEG4 ASP is so used because is the only that can put a movie in a simple single CD-ROM at a "watchable" (and sometimes rally good) quality, and this is for me a good reputation.

But i also think that at high bitrates it can beat the "almigthy" MPEG2, and with the raise of DVD-burners market, many people will do backups at the same (or better, with the help of Avisynth and some filters...) quality of the original!

When standalone players will support MPEG4 on DVD (IF they'll support that...)that will be the best quality out of a DVD...think of a dual-layer DVD with some HDTV-XviD content!

So for me XviD (as all others MPEG 4 ASP) ia an all-around codec that wil be used for years to come...

Regarding AQ, in my tests it has been only a benefit to the encoding, and some blocks on a dark wall is way less visible that on a face, no? :)

[Ark]

OOps, double post :) ...sorry..

[Didée]

Come off it, Tee! Mentioning me in one breath together with Ghandi ... that's forsooth a little too much of a good thing :o

[gpower2]

@Manao
What you said about quantizers is something that I've never read in such a way so you can say that it was the answer I was looking for,thanks! For the pause feature, I now feel less stupid, because I thought that it could not be done since noone has ever mentioned it, but I always believed it could be! As for the player, I dislike ffdshow, I prefer native codecs for decoding, so I was looking for another way, maybe a dshow filter or something like that. And for the resizer, I usually like to enlarge the resolution so the differences there tend to be more noticeable between resizers, or so I see! Maybe I underestimated VHQ4, I will do some more testing!

@Teegedeck
I have tested VHQ4 @full 2-pass and I admit I didn't see any noticeable visual difference between the samples except of course the encoding time! Maybe the source I used wasn't appropriate, what can I say? I'll give it another try soon! As for AQ, I've also tested it and @high bitrates I could see the blocks at dark surfaces such as walls etc. Maybe it's because I used to increase brightness at overlay controls and the blocks became visible. Now I increase the gamma in the .avs so I don't change the brightness setting. I'll test it again and I'll let you know!
For the record, I don't like using heavy filtering in avisynth when encoding from nice looking sources like DVDs. I crop, resize and subtitle the source and then straight to XviD! Maybe some gamma correction and that's it! So I use MPEG Matrix because it's said that it keeps more details and it does less smoothing than H.263. SixOfNine is an MPEG or an H.263 matrix?


Thanks for giving me ideas for more testing, and opening my orizons!

[Teegedeck]

Originally posted by gpower2
Maybe it's because I used to increase brightness at overlay controls and the blocks became visible.
Definitely. :)
Originally posted by gpower2
SixOfNine is an MPEG or an H.263 matrix?
All custom matrices are MPEG matrices.

@Didée: come on, I didn't have much time to think of a comparison! :cool:

Greets,
Tee

[Manao]

I dislike ffdshow, I prefer native codecs for decodingHowever, the good thing about MPEG-4 compliant codecs is that they are compliant, hence can be decoded with the same quality ( * ) by different decoders. FFDShow, in this regard, gives the user the same quality, with better speed, and more postprocessing options. For me, it is a must.
I subtitle the sourceYou haven't any other choice ? Because overlaying subtitles before encoding hurts the quality of the encoding.

( * ) : if we except the idct, which can gives slightly different results with different decoders. However, ffdshow implements xvid idct.

[Leak]

Originally posted by Manao
However, the good thing about MPEG-4 compliant codecs is that they are compliant, hence can be decoded with the same quality ( * ) by different decoders. FFDShow, in this regard, gives the user the same quality, with better speed, and more postprocessing options. For me, it is a must.

Also, by selecting "XviD 1" instead of "libavcodec" in ffdshow's codecs page you can have it use the XviD core libraries installed on your system for decoding and still do post-processing through ffdshow...

np: Luke Vibert & BJ Cole - This Stuff Is Fresh (Stop The Panic)

[gpower2]

Well, after all those answers, I decided to check some things by testing. I rented a PAL progressive region-2 DVD from my DVDclub with Peter Pan (all the other were either rented or I had rented them before!) and I put in VdubMod Job Control, 6 different codec settings for full 2-pass encoding! I think it is a nice test since it's on "real life" conditions, I encode the whole movie, not just a part of it, and I use the original AC3 for sound, so I can test the real speed too (with no sound file the speed is quite faster)! I hope that tomorrow the test will be finished, it takes 3-6hours just for the 2-nd pass depending on the settings so...

In my next post I'll try to have a link with some samples from the test clips so that you can say your opinion on the results. I'll also refer to the settings I used and all the progs and their versions too. I think it's a good test and I believe we should see interesting results!

Till tomorrow then...Adios!

[crusty]

"1st:
Interlaced encoding."

A simple tip: Never keep originally un-interlaced footage interlaced. Many NTSC-DVD's have films that were shot in 24fps film, but were later converted to interlaced for TV. Those were never meant to be interlaced.
Also, when playing back Mpeg-4 files via a modern videobeamer, (de-)interlacing artifacts will show up much sooner. Older beamers were much more TV-style and you could get away with some artifacts, but any beamer with VGA or better resolution is unforgiving.

Most important tip: Xvid's interlaced encoding can never handle hybrid or bad telecined material as well as a third-party filter like Decomb.
This often leaves you no other option but to handle it first with Decomb and then encode it non-interlaced. It allows you total control AND better quality.

"3rd:
Cartoon Mode.
It's an option for anime. "

Uhum. Cartoon mode is really meant for cartoons aka Disney, Tom & Jerry, Duckman, South Park, The Simpsons etc. Although people get decent results with anime as well, anime can be very different from cartoons. Anime is well-known for mixing in SGI-graphics and other special effects in the footage that don't respond well to cartoon-style filtering. Use with care on anime.


"5th:
Meaning of quantizers.
I know that quantizers define the level of compression of a frame, I know that Q2 is better than Q8 or Q31 and I know that Q1 should not be used, but I don't know how this is done. Meaning, does it affect the bitrate that is going to be used for the specified frame, does it affect the final filesize, does it affect the bitrate curve?"


From the FAQ:
After the image is converted to detail signals by DCT those signals are divided (coefficient cutting) to make it smaller. The more signal you cut off, the more detail you lose and more artifacts (blocks, ringing aka mosquito noise) you get. The quantizer is the detail removal factor (DivX3/Nandub thus calls quantizers DRF). The higher the quant, the more detail gets cut off, and the lower video quality will be. So, as an example, a Quantizer of 2 means you keep almost all detail, whereas a Quantizer of 31 throws almost everything away.

"does it affect the bitrate that is going to be used for the specified frame?" = yes, higher quantizers will throw more information away, making the frame smaller.
"does it affect the final filesize?"= Yes, same thing, higher quantizers = smaller filesize and crappier end result.
"does it affect the bitrate curve?"=Yes, because the bitrate curve is simply a curve of all framesizes of all the individual frames put together, like a group of small and tall people forming a curved line . And they are all affected by a change in max or min quantizer (the frames, not the people).

You can read a bit more about curve compression in my FAQ. Use the link in my sig to get there.


"7th:
Pause Feature?"
Would be wonderful...but we haven't got it. So perhaps if you look somebody really nice into his eyes... :D

[Teegedeck]

Originally posted by crusty
Anime is well-known for mixing in SGI-graphics and other special effects in the footage that don't respond well to cartoon-style filtering. Use with care on anime.
Ummm; cartoon mode doesn't do any filtering. Let me quote your FAQ:


-'Cartoon mode' activates two different techniques, both designed to help with cartoons:
- detect_static_motion is a motion estimation flag and it works as a limit. If the motion found by the motion search process is beneath this limit the macroblock is considered static, and no motion information is encoded. When Cartoon mode is enabled this limit below which a macroblock is concidered static is increased so that (more) small movements are lost. Since A LOT of these 'small movements' is actually noise (especially with cartoons) it really helps saving many bits which would otherwise be used to code noise on a static picture.
- vop_cartoon is about quantization - when a block is motion-compensated well enough (with total error below the limit) it's just not coded at all. XviD doesn't drop any data in normal mode (limit = 1), but drops quite a lot in cartoon mode. Again, this usually means that noise is ignored. It might also remove some small details, but small details shouldn't really happen in "proper" cartoons.
So, while the first technique helps with removing movements that are so tiny that they can be considered not-to-be-part-of-the-source, the second helps compressibility of the cartoon by removing texture detail that's considered too-small-to-be-part-of-the-source.
But I agree that anime shouldn't be noise-filtered. ;)
We are talking about very, very slight motion which is ignored by the encoder in cartoon mode - motion which in anime would probably be caused by noise; noise which most people will filter out BTW). You can even use cartoon mode on real-life material and will hardly see a difference (as far as I remember limit=1, as one of cartoon mode's two components, had been hardcoded in Koepi's builds for quite some time before people like me complained about a loss of tiniest details - also film-grain), so I dare say you can use it with anime and save some bits alright.

And thanks for giving us the new address of your FAQ, I updated the sticky!

[crusty]

I was using 'filtering' in a more general sense of the word. ;)

[gpower2]

Sorry it took me so long, I finally managed to finalize the test and get some results! Prepare for boring info :

Avisynth Script Used:
LoadPlugin("F:\Gp2\Progs\Dvd\Ripping\AviSynth\plugins\DGDecode.dll")
MPEG2Source("F:\Dvd Rip\Peter Pan\Peter Pan.d2v",idct=5)
crop(0,74,-0,-74)
Levels(0, 1.3, 255,0, 255)
Lanczos4Resize(848,352)
vobsub("F:\Dvd Rip\Peter Pan\vts_17_0.idx")

Progs Used:
SmartRipper 2.41
DGIndex 1.0.12
VsRip
VobSub 2.23
AviSynth 2.55
VirtualDubMod 1.5.10.1 (build 2439)
XviD 1.0.2 Koepi's build
BsPlayer 1.02 (build 812)

PC used:
P4 @ 3GHz, 800MHz FSB, HyperThreading
Asus C800
1024MB DDR400 @ Dual Channel
Maxtor 80GB PATA 7200rpm
Windows XP Pro SP1 English

XviD Common Settings:
Everything Default Except:
Profile@Level: Unrestricted
Q Type:MPEG
QPel :On
BVops: On
Packed Bitstream: On
Closed GOV: On
Encoding Type: 2-pass
Target Bitrate: 2000kbps
Zone Options
Weight 1.00
Begin With Keyframe: On
Chroma Optimizer Enabled: On
Advanced Options
Motion Search Precision: 6-Ultra High
Use Chroma Motion: On
Maximum I-frame Interval: 250
Quantizer Restrictions
2,6,2,6,2,31

The 6 different settings are obvious from the filenames of the samples and are:
Final_VHQ1_NoTrellis_GMC_NoAQ.avi
Final_VHQ1_NoTrellis_NoGMC_AQ.avi
Final_VHQ1_NoTrellis_NoGMC_NoAQ.avi
Final_VHQ1_NoTrellis_NoGMC_NoAQ_DifBitCurve.avi
Final_VHQ1_Trellis_NoGMC_NoAQ.avi
Final_VHQ4_NoTrellis_NoGMC_NoAQ.avi

For DifBitCurve I used the following settings @ the 2-pass Second Pass tab:
15,2,15,0,15,5,0,0

Time Elapsed:
All first passes lasted approx. 1:09:00 -> 39.40fps

2nd passes:
Final_VHQ1_NoTrellis_GMC_NoAQ.avi : 4:29:00 -> 10.11fps
Final_VHQ1_NoTrellis_NoGMC_AQ.avi : 3:22:00 -> 13.46fps
Final_VHQ1_NoTrellis_NoGMC_NoAQ.avi : 3:17:00 -> 13.80fps
Final_VHQ1_NoTrellis_NoGMC_NoAQ_DifBitCurve.avi : 3:16:00 -> 13.87fps
Final_VHQ1_Trellis_NoGMC_NoAQ.avi : 3:26:00 -> 13.20fps
Final_VHQ4_NoTrellis_NoGMC_NoAQ.avi : 6:10:00 -> 07.35fps

File Sizes:
All first pass files were the same
2nd-pass files:
Final_VHQ1_NoTrellis_GMC_NoAQ.avi : 1,953,550 KB
Final_VHQ1_NoTrellis_NoGMC_AQ.avi : 1,953,552 KB
Final_VHQ1_NoTrellis_NoGMC_NoAQ.avi : 1,953,552 KB
Final_VHQ1_NoTrellis_NoGMC_NoAQ_DifBitCurve.avi : 1,953,550 KB
Final_VHQ1_Trellis_NoGMC_NoAQ.avi : 1,953,550 KB
Final_VHQ4_NoTrellis_NoGMC_NoAQ.avi : 1,953,538 KB

Original Files:
vts_17_1.vob : 724,598 KB
vts_17_2.vob : 737,592 KB
vts_17_3.vob : 737,446 KB
vts_17_4.vob : 719,078 KB
vts_17_5.vob : 156,056 KB
vts_17_([0x80]_Audio_English_AC3(6Ch)_48kHz___)_Delay_0ms.ac3 : 356,836 KB

Average MPEG II bitrate : 3860.31 kbps

Visual Results:
Unfortunately I couldn't find the webspace to upload snapshots of frames, I hope I will in a few days, so I'll only report what my eyes saw!

First, Teegedeck, you were right about AQ and VHQ4. You were wrong about GMC! GMC did a lot of smoothing and most of the details were just lost! Trellis did that too, but not so big! AQ managed to keep the balance between smoothing and keeping details! As for VHQ4, I saw some difference from VHQ1 but the extra time is too much, so if you go for really really good quality go for it, or else VHQ1 will do the job almost the same! Qpel is a must for me, so I didn't test withou it. As for the Different bitrate curve I tested, the results weren't so good, so I think I'll stick to the default settings!

Hope I'll manage to send you some sample frames quick!

[Teegedeck]

If that is true, the entire book on XviD has to be rewritten. Well, the book on GMC anyway. sysKiiiin! :D

Actually I think that there's a mistake lurking somewhere because your findings contradict my own. GMC hasn't ever lost me details.

And thanks for testing, BTW! Quite some effort.

[gpower2]

@Teegedeck
Well, I didn't expect these results I must say, I thought AQ would do more smoothing and GMC ...none! But as I wrote before, the differences were clear by the naked eye! And I saw it in a lot of frames, Intra-Inter and B-frames! Like I didn't expect to see differences between VHQ1-VHQ4. Yet I was glag to see that the settings I used for almost all encodes yield great results @high bitrates! Some may like the smoothing done by these features, because they "remove" the film noise which is common in most DVDs and a lot of people use filtering just to clear it. But I like to keep as much details from the source I can, even the film noise! And If you look at the final sizes, 2Kb for so smoothing? It is just not acceptable by me...

BTW I tested SixOfNine(Max=20) CQM @2000kbps with Kaena:The Prophecy, Along Came Polly and The Curse Of The Jade Scorpion and although I couldn't find great differences from MPEG, I liked the Q variation a lot! Maybe the fact that I cap the Qs is the reason I didn't see visual differences between MPEG & CQM.

If anyone has some webspace to borrow me to upload ~10MB of frame snapshots that would be great!

Also any critics about the test is welcome!

[Teegedeck]

Well, one thing is for sure, GMC gets better with higher VHQ. Perhaps you can compare VHQ=4 without GMC against VHQ=4 with GMC. As for webspace, there are several free webspace providers that specialize in images and have an easy-to-use interface. Sadly, I can remember not even one name at the moment. :p Just check the properties of others screenshots that were posted on this forum.

[gpower2]

Maybe I'll try this also (VHQ4-No GMC Vs VHQ4-GMC)! Another thing I forgot, for the specs of my PC, are the fps I have a good number? Should I worry/expect more fps?

I prefer to have a link to a zip/rar file containing the pictures because I have them in .tga and they are quite big!

[OCedHrt]

About the fps, not too sure since I think you're using full dvd res. I'm currently trying to find someway to re-encode some anime I have to fit the season on dvd. I see 175mb releases all the time so I'm trying to get them from 220->175 with minimal quality loss.

Current settings are: XViD 1.1b: GMC, AQ, Trellis, VHQ4, VHQ for Bf, quants 2-31, chroma opt, and cartoon mode(old anime, so should be fine?). The source is 640x480 WMV (>< smoothed like no other) and I'm getting about 16-18fps on pass 2 and about 35 fps on pass 1. Absolutely no filtering.

By the way, very nice thread :) Answeres some questions I had as well :D

Oh, and you can upload images on imageshack.us

[gpower2]

You wrote :

"Current settings are: XViD 1.1b: GMC, AQ, Trellis, VHQ4, VHQ for Bf, quants 2-31, chroma opt, and cartoon mode(old anime, so should be fine?). The source is 640x480 WMV (>< smoothed like no other) and I'm getting about 16-18fps on pass 2 and about 35 fps on pass 1. Absolutely no filtering."

I would use the following:
Qpel (a must use), no GMC, no AQ, VHQ4 and VHQ for BF, chroma optimizer, definetely NO cartoon mode (it is said that it is bugged for 1.1) and since you are using WMV as source, you should use MPEG to keep the few details. These for bitrates>1000kbps. For less, try GMC, trellis, maybe AQ(I dislike the results). For Speed try VHQ1, I am sure you will notice significant increase of fps!

Many thanks for your good words! Unfortunately it is quite an old thread and I have erased the sample images, maybe I will run a new test with the new 1.1 version!