Quantization > Trellis

[gurabli]

Hi!

I'am doing a lot of encoding, and always using xvid. I have read about quantization types but I don't really know which one to use:

My settings are:

H.263
Quartel P.
B-VOP: Max: 2, ratio: 1.50, offset:1.00
Packed bitstream
Closed GOV

Chroma Motion
6-Ultra high
4-Wide search
frame drop:0
i-frame: 300

Quantization:
1,31,1,31,1,31
Trellis OFF

Here is my question: Should I tunr Trellis ON for better results (the overall quality of the encoding is the most important factor for me).
And should I use different quant than above?


Zone Options:
Begin with keyframe
Chroma optimazier enabled
BVOP sens.: 0

Please help me with Trellis and quantization!

Thanx,
gurabli

[Magno]

If quality is the most important for you, then you minimum quantizer should be 1 for I, P and B frames, whereas the maximum could be whatever (the default 31 is fine); this way, the encoder has a wide range of quantizers to choose while encoding.

As for Trellis, I know almost everybody is going to kill me for what i am going to say but.... whenever i check Trellis quatization, the image quality decreases and i see a lot more of ringing. Maybe it really reduces filesize so those saved bits can be use in increase quality in the video where needed, but i can see any profits in using it.

[gurabli]

Thanx Magno!

Well, I have the same experinces with Trellis. They do decrease image quality and I did't notice any improovement in the overall quality. If I'm wrong, please correct me!

[Teegedeck]

OK, you are wrong, I correct you. ;)

Trellis always enhances picture quality - do constant-quantizer encodes with and without Trellis; make snaphots of the same frames for both; compare. I did.

[gurabli]

Thanx for the correction Teegedeck! So I should always enable Trellis for encoding?

What about my other settings? Are they OK for good quality?

[Teegedeck]

On first looks, most of it seems very OK, especially that you were wise enough to leave b-frame settings at their defaults and use a high VHQ-level. Packed bitstream is a bit problematic; disable it if you don't absolutely need to cut your file with VDub after encoding. Set a minimun quantizer of '2'; you might get undersized results after that, but if you do then you know that you can use a higher resolution. Please don't complain if you get undersizing, OK? Instead you could start thinking about using full anamorphic resolution in such a case, and maybe about using high-bitrate custom quant matrices (like SixOfNine) to push quality further a good notch.

Give adaptive quantization a try, you should be surprised that it delivers a visible improvement at anything except for saturated encodes.

Using XviD 1.1 with VHQ for b-vops would be a very good idea.

[gurabli]

Thanx for the tips!
Than I will use adaptative quant. What are they doing?

An setting quant min to 2? How will it improove quality?

What about the custom qant. matrices? I suppose they should be used when there is a high-bitrate encoding. But what is ment by high? And which resolution to use?

XviD 1.1 is still very unstable, isn't it? I'am afraid to use it, or?

Thanx for your help! I very preciate it!

[Poutnik]

Originally posted by gurabli XviD 1.1 is still very unstable, isn't it? I'am afraid to use it, or?[/B]
My Koepi's last Xvid1.1 always crashes when choosing calc button next to bitrate/size field/quant

[celtic_druid]

I would say that it is very stable. But then I don't use the calculator. Just tried it and it didn't crash here, that is with my latest compile.

[Sharktooth]

I can confirm CD's latest build is very stable (calc doesnt crash on my comp) and i regularly use it for my encodes.

[Poutnik]

Does it mean last by Koepi, or your own ?

[Sharktooth]

Sorry, CD = Celtic Druid.
It is newer (2004.12.20) than koepi's build.

[Poutnik]

So I vill try CDs one. Thanks. I have never used other but Koepi ones.

[Magno]

Originally posted by Teegedeck
OK, you are wrong, I correct you. ;)

Trellis always enhances picture quality - do constant-quantizer encodes with and without Trellis; make snaphots of the same frames for both; compare. I did.

I think this isn't a matter of being wrong or not, it is a matter of facts and preferences: i have tried Trellis lots and lots of times (more than 50 encodes) and I have always noticed a reduction in quality; moreover, it is well know that Trellis had a bug in later Xvid releases. I began to quit using that option since then and i noticed an improvement in quality. At my point of view, the results aren't good with Xvid 1.0.2 neither...
Maybe at a constant quantizer the results are better, but I think they aren't in two-pass encodes...

[gurabli]

Well, it is interesting.

In the new codec comparison - that was finished today - XviD developers suggested to use Trellis while encoding with XviD. It must be good if they suggested to turn it on. It is a very prestigious roundup, so they woudn't suggest it if it decreased quality?!?

[Poutnik]

Originally posted by Sharktooth
Sorry, CD = Celtic Druid.
It is newer (2004.12.20) than koepi's build.
CD build causes in my w98se box the same error as Koepis 1.1 one.
Strange. I will not use calc. I will use standalone one by start menu.
Originally posted by Magno
The results aren't good with Xvid 1.0.2 neither...... [/B]
About Koepis 1.0.3 build there is mentioned fixed trellis bug
for low Qs.

[Teegedeck]

Originally posted by Magno
I think this isn't a matter of being wrong or not, it is a matter of facts and preferences: i have tried Trellis lots and lots of times (more than 50 encodes) and I have always noticed a reduction in quality; moreover, it is well know that Trellis had a bug in later Xvid releases. I began to quit using that option since then and i noticed an improvement in quality. At my point of view, the results aren't good with Xvid 1.0.2 neither...
Maybe at a constant quantizer the results are better, but I think they aren't in two-pass encodes...
That bug you are talking of has been fixed a long time ago. It is not a good idea to stop using a feature for good just because it has been buggy once. Any- and everything has had a bug at some point in time.

There ain't a difference between Trellis in 2-pass and in constant-quantizer encodes, except that at constant quantizer you can make sure that the frames you compare are encoded at the same quantizer. At least I'm not aware of differences in filesize with and without Trellis.

Please bring forth proof or at least evidence when you make claims that contradict findings of most of XviD's users and its developers. It is easy to do: constant quantizer encodes (or two-pass encodes to same filesize), fed into Vdub through an Avisynth script that loads the script via 'directshowource'. Use ffdshow to decode the video and its OSD function to display frame numbers. Make snapshots with VDub.

[Magno]

Well, Teegedeck, don't get serious ;) I said it was my point of view... I haven't right now the tests i did with and without Trellis quant checked, nor the movies i encoded using it becasuse, obviously, i deleted them as i didn't like the results... I promise next time i encode a movie with Xvid, i will do some sample with both Trellis on and off and we will able to compare... maybe i am awfully wrong (i won't ever say i am not) :D

But i think this is as all in life: if i give you a prove with a frame of the last movie i used it in, where i saw clearly the effects (it was "The butterfly effect", i believe), maybe anybody could tell me that in some other DVDrip he got a perfect encoding with Trellis on. I could bet my left hand (i am right-handed :P) that few people could see clear benefits in image quality in ALL encodings with Trellis on.

BTW, talking about Trellis... Trellis coding is often used in data modulations (V.90 and later uses it) to reduce BER (Bit Error Rate); i read again my university notes in data communications and i can't see how Trellis could be useful for improving image quality (maybe it is a way to choose the proper quantizer for each macroblock?)... and i am very curious to know it :D Would you mind explaining it for me, please?

[chilledoutuk]

what always confuses me is that the default is for trellis to be off and also i noticed that autogk dosent use trellis.

[gurabli]

I have done some encoding with Adaptive Quant. ON, Trellis ON and the overall quality impressed me! It is really much better! Thanx guys!

One more question:

should I use GMC? Is it good to use or not and what is the benefit of it? (does it still has some bugs or safe to use?)

[Ark]

Originally posted by gurabli
I have done some encoding with Adaptive Quant. ON, Trellis ON and the overall quality impressed me! It is really much better! Thanx guys!

One more question:

should I use GMC? Is it good to use or not and what is the benefit of it? (does it still has some bugs or safe to use?)

GMC is safe to use, it's meant to increase compression efficiency in scenes like panning, when all parts of the screen goes to the same direction (very simplified explanation...the only one i know :D).

For it's quality, well many said that they even have to see it's effect, AFAIK it's not a quality-changing option, but a space-saving one (though on a target filesize encode you should have more bits to spend using it, so more quality...in theory).

[Manao]

Teegedeck, Magno : we had a discussion about trellis not a long time ago ( http://forum.doom9.org/showthread.php?s=&threadid=82357&highlight=trellis ) where the conclusion was that trellis was always ( at fixed quantizer ) reducing the filesize, while sometimes lowering, sometimes increasing the quality ( aka PSNR in that case, not subjective quality ). In a two passes encoding, quality will be higher with trellis ( because the filesize gain always compensates more than the possible loss of quality ).

Ark : I would not see GMC as a space saving option, nor as a quality enhancing one. But that's merely a semantic issue. GMC is always good, if you have some more encoding time to spare, and if you don't playback it on a setupbox.

[Ark]

Originally posted by Manao
Ark : I would not see GMC as a space saving option, nor as a quality enhancing one. But that's merely a semantic issue. GMC is always good, if you have some more encoding time to spare, and if you don't playback it on a setupbox.

I know that it creates a motion vector in scenes like panning, indicating the moving direction of the frame objects, thus reducing the info needed to store this information without it (=more vectors to store)..so it looked to me as a space-saving option...correct me if i'm wrong

[Manao]

It's more subtle than that : with xvid ( and nero, and hd4x perhaps ), it stores not one, but 3 vectors, which means it can code more complicated geometric transformations than translation : rotations and zooms for example. And such transfomations can't be achieved by just moving a block by a vector. So in such cases, GMC could raise quality ( in comparison to a non GMC one ).

[Ark]

Ok, thanks for the explanation!

[gurabli]

Yes, thanx for your help, folks!

Than I will use GMC as well! As quality is more important than speed for me!

[Leak]

Originally posted by gurabli
Yes, thanx for your help, folks!

Than I will use GMC as well! As quality is more important than speed for me!

Just keep in mind that none of the MPEG4-playing DVD players that are on the market today (and no, a modded Xbox doesn't count here) can't play files using XviD's 3 warppoint GMC correctly; you'll get stutters or broken frames whenever it is used.

np: RJD2 - One Day (Since We Last Spoke)