Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 ASP

Reply
 
Thread Tools Search this Thread Display Modes
Old 13th April 2012, 14:13   #1  |  Link
sirt
x264 fan
 
sirt's Avatar
 
Join Date: Feb 2011
Location: In the trap
Posts: 458
XviD Quantizers

Hi,

As you all know XviD is now obsolete compared to x264 but it doesn't prevent me from trying to understand how it works, so I have some questions :

Let's say I encode in two passes. My expectations : the slowest encoding as possible, the less blur as possible, no fluctuations around the bitrate setted : I need the encoder to respect it - at least - trough the second pass or both if possible. This is the parameters I think I have to focuse on :



Is that great for a video at 24 img/s ? In particulary, what will imply VHQ metric 1 and what will happen with 0 ? I've setted I-frame interval to 240, is it correct ?



Well I've setted all to 1 because I'm not sure how to achieve the goals mentionned below (especially the respec of the bitrate imposed), what are the consequences of setting everything to 1 ? My tests revealed the bitrate is too high compared to my settings, but I may be wrong ? I read somewhere I might use 1,8/1,16/1,24 to be optimal, what would you recommand ?



I have no idea how to set that window but I read somewhere I should only change the reduced % to 20, what I should I do to achieve my goals ?



I'm configurating those windows like that because I've read using MPEG matrix would be good to keep details (especially if the video doesn't have dark areas), but I'm not sure : what will be the difference with H263 exactly ? What if I would use a custom matrix such one of the matrix's made by *mp4 guy (http://forum.doom9.org/showthread.php?p=887125) ? Should I use Adaptize Quantization and how ? I don't tick anything excepted B-Vops regarding what I want because I've read it may be a bad thing, am I right ? Finally somebody told me raising the Quantizer Ratio (for example to 2) could improve the quality, should I do that ? I usually don't mess with Level and Aspect Ratio windows, is there something to change there ?
sirt is offline   Reply With Quote
Old 13th April 2012, 15:16   #2  |  Link
Guest
Guest
 
Join Date: Jan 2002
Posts: 21,924
Why are you obscuring filenames in your posts?
Guest is offline   Reply With Quote
Old 13th April 2012, 17:20   #3  |  Link
sirt
x264 fan
 
sirt's Avatar
 
Join Date: Feb 2011
Location: In the trap
Posts: 458
Quote:
Originally Posted by neuron2 View Post
Why are you obscuring filenames in your posts?
My name appears on the folder and I was lazy to rename it.
sirt is offline   Reply With Quote
Old 13th April 2012, 15:26   #4  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 12,800
Quote:
Originally Posted by sirt View Post
Let's say I encode in two passes. My expectations : no fluctuations around the bitrate setted
Why you don't want any bitrate fluctuations?

As a matter of fact, the "complexity" of almost any video varies over time. In order to get a consistent quality, the bitrate has to be adjusted with respect to the content of the video.

That's what the idea of 2-Pass mode is all about: Allow the bitrate to "fluctuate" (in other words: Move the bits to where they are actually needed!) and still be able to hit the target average(!) bitrate.

If you would be encoding with a 100% constant bitrate (i.e. no bitrate fluctuations), this would result in quality fluctuations, which certainly isn't desired

Quote:
Originally Posted by sirt View Post
I'm configurating those windows like that because I've read using MPEG matrix would be good to keep details (especially if the video doesn't have dark areas), but I'm not sure : what will be the difference with H263 exactly ? What if I would use a custom matrix such one of the matrix's made by *mp4 guy (http://forum.doom9.org/showthread.php?p=887125) ?
"MPEG" and "H.263" are two different types of quantization, supported by the MPEG-4 ASP format. As you may know, MPEG-4 ASP has evolved from H.263.

The "MPEG" type is known to retain more detail/sharpness, but also has a tendency to produce "blocking" artifacts. At the same time the "H.263" type is known to better avoid blocking and to produce a slightly smoother image.

Also note that only the "MPEG" type can use custom matrices. The "H.263" type can NOT do this. Also there is NO custom matrix for the "MPEG" type that would resemble the "H.263" type.

Quote:
Originally Posted by sirt View Post
Should I use Adaptize Quantization and how ?
You probably want to use Adaptive Quantization for the same reason you want to use it with x264
http://forum.doom9.org/showpost.php?...94&postcount=9

Xvid supports two AQ algorithms. The old "Lumi Masiking", which is not very good. And "Variance Masking", the AQ algorithm ported back from x264.
__________________
There was of course no way of knowing whether you were being watched at any given moment.
How often, or on what system, the Thought Police plugged in on any individual wire was guesswork.



Last edited by LoRd_MuldeR; 13th April 2012 at 15:30.
LoRd_MuldeR is offline   Reply With Quote
Old 13th April 2012, 17:34   #5  |  Link
sirt
x264 fan
 
sirt's Avatar
 
Join Date: Feb 2011
Location: In the trap
Posts: 458
Quote:
Originally Posted by LoRd_MuldeR View Post
Why you don't want any bitrate fluctuations?

As a matter of fact, the "complexity" of almost any video varies over time. In order to get a consistent quality, the bitrate has to be adjusted with respect to the content of the video.

That's what the idea of 2-Pass mode is all about: Allow the bitrate to "fluctuate" (in other words: Move the bits to where they are actually needed!) and still be able to hit the target average(!) bitrate.

If you would be encoding with a 100% constant bitrate (i.e. no bitrate fluctuations), this would result in quality fluctuations, which certainly isn't desired



"MPEG" and "H.263" are two different types of quantization, supported by the MPEG-4 ASP format. As you may know, MPEG-4 ASP has evolved from H.263.

The "MPEG" type is known to retain more detail/sharpness, but also has a tendency to produce "blocking" artifacts. At the same time the "H.263" type is known to better avoid blocking and to produce a slightly smoother image.

Also note that only the "MPEG" type can use custom matrices. The "H.263" type can NOT do this. Also there is NO custom matrix for the "MPEG" type that would resemble the "H.263" type.



You probably want to use Adaptive Quantization for the same reason you want to use it with x264
http://forum.doom9.org/showpost.php?...94&postcount=9

Xvid supports two AQ algorithms. The old "Lumi Masiking", which is not very good. And "Variance Masking", the AQ algorithm ported back from x264.
Yes another idea came to my mind...comparing an x264 encode with an xvid one ; I know x264 can win but I want to see how the XviD will proceed such-and-such video. I was still using not long time ago to save some of my favorite movies because it was faster and not killing my computer.

About the fluctuations : indeed I commited a mistake by expressing my goals like that. I just wanted to say I want the encoder not to overtake the bitrate limitation. With 1,1/1,1/1,1 quantization it didn't respect the final size intended whereas it worked almost correctly with 1,4/1,16/1,8 but I gave a try to a custom matrix called "For Home Enternainment" but it didn't reach my final size and the bitrate setted. That's why I was asking how to set up that Quantization window. I also tried 1,4/1,2/1,4 but again the final size intended was not respected. I simply don't know how to set that !

Then I will give a try to the Variance Masking. Futhermore this one seems to new (I just got latest XviD few days ago) because such adaptitve method was not avaible before. D
sirt is offline   Reply With Quote
Old 13th April 2012, 18:06   #6  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 12,800
Just a side-note: By forcing all the min/max quantizers to 1, you completely cripple all rate-control!

If you now use 1-Pass ABR mode or 2-Pass mode, you will effectively get 1-Pas Constant Quantizer mode with a quantizer of 1, which is not the intend of these modes. Needless to say that the file will also be LARGE

Also: If you want to compare your Xvid encodes to your x264 encodes, you must ensure that the file size (i.e. average bitrate) of the files you'll compare is identical. Otherwise the comparison is worthless!
__________________
There was of course no way of knowing whether you were being watched at any given moment.
How often, or on what system, the Thought Police plugged in on any individual wire was guesswork.



Last edited by LoRd_MuldeR; 13th April 2012 at 18:18.
LoRd_MuldeR is offline   Reply With Quote
Old 13th April 2012, 22:28   #7  |  Link
sirt
x264 fan
 
sirt's Avatar
 
Join Date: Feb 2011
Location: In the trap
Posts: 458
Quote:
Originally Posted by LoRd_MuldeR View Post
Just a side-note: By forcing all the min/max quantizers to 1, you completely cripple all rate-control!
Yes but I don't really know what quantizers will help to obtain the final size intended by the bitrate setted. What I'm sure is that letting 1,31/1,31/1,31 will result in the poorest quality possible ; at least, it's what I read on a web guide where they advise to use 1,4/1,16/1,8 but I ignore why.

Quote:
Also: If you want to compare your Xvid encodes to your x264 encodes, you must ensure that the file size (i.e. average bitrate) of the files you'll compare is identical. Otherwise the comparison is worthless!
I do but only for short clips or movies (I couldn't encode a movie with x264 even if I used medium preset) and honestly x264 always win the toss (in 2 pass, I haven't tried ABR) ; but I still use it for some movies or non HD TV Rips ; what I simply prefer is using LAGARITH and that's all (or MSU) ; I'm not sure if there is a "lossless" mode with xvid, but I guess not.
sirt is offline   Reply With Quote
Old 13th April 2012, 22:53   #8  |  Link
Asmodian
Registered User
 
Join Date: Feb 2002
Location: San Jose, California
Posts: 2,659
Quote:
Originally Posted by sirt View Post
Yes but I don't really know what quantizers will help to obtain the final size intended by the bitrate setted. What I'm sure is that letting 1,31/1,31/1,31 will result in the poorest quality possible ; at least, it's what I read on a web guide where they advise to use 1,4/1,16/1,8 but I ignore why.
The limits there don't really help with anything; they just limit what the rate control can use. Because quant 1 is much larger than quant 2 and not really any better looking I actually like using 2,31 as min,max. This gives Xvid the freedom to use whatever quant it thinks is best but if the file is looking undersized it won't go to the mostly pointless quant 1 just to hit the required bit rate.

IIRC the default was set to 1-31 because less experienced users would complain about undersized files using 2-31. Adding the quant 1 option doesn’t increase the visual quality of the encodes but it does make them larger, preventing angst about undersized files for users.

Remember those are just the min/max. Xvid doesn't have to use any particular quantizer unless it thinks it is the best option to hit a desired file size.

A caveat: I have not used Xvid in several years as x264 has superseded it.
Asmodian is offline   Reply With Quote
Old 13th April 2012, 23:15   #9  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 12,800
Quote:
Originally Posted by sirt View Post
Yes but I don't really know what quantizers will help to obtain the final size intended by the bitrate setted.
And you, as a user, don't have to know

In the bitrate-based encoding modes, like 1-Pass ABR or 2-Pass, the rate-control algorithm will pick the "right" qunatizers to hit the target average(!) bitrate that you have chosen.

By changing the "minimum/maximum quantizer" options you only restrict the interval of quantizers from which the rate-control algorithm can choose - which can easily lead to missing the desired average bitrate!

If, for example, you set both, the minimum and the maximum quantizer, to a value of 1, then you force the rate-control algorithm to always pick 1, regardless of the chosen bitrate

You would only limit the minimum (or maximum) quantizer, if you believe that the rate-control algorithm sometimes goes too high (or too low) for no good reason...
__________________
There was of course no way of knowing whether you were being watched at any given moment.
How often, or on what system, the Thought Police plugged in on any individual wire was guesswork.



Last edited by LoRd_MuldeR; 13th April 2012 at 23:24.
LoRd_MuldeR is offline   Reply With Quote
Old 14th April 2012, 08:27   #10  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 3,445
Quote:
Originally Posted by LoRd_MuldeR View Post
By changing the "minimum/maximum quantizer" options you only restrict the interval of quantizers from which the rate-control algorithm can choose - which can easily lead to missing the desired average bitrate!
I inquired about this in my own thread quite a while ago but didn't find anyone who could answer it at the time, so as it's relevant to this discussion....

I've checked how AutoGK sets up the Xvid encoder on quite a few occasions just to see what it's doing, and AutoGK seems to set the min and max quantizers fairly close together while changing them according to the result of the compression test.
For example if the compression test result is around 70% it'll use quantizers of 2 and 3, at 60% it'll use 2 and 4, at 40% it uses 2 and 5 etc.... in the case of AutoGK it doesn't seem to effect it's ability to achieve a particular file size.

I assume the reason for doing so would be to keep the quality of individual frames higher than Xvid otherwise would using the default quantizers of 1 and 31 (or 2 and 31). Would that be correct?
If I remember correctly limiting the maximum quantizer stops Xvid from being able to decrease the file size as much as it otherwise would, but the quality of individual frames of motion scenes tends to remain higher due to limiting the max quantizer (well the same applies to static scenes but I think I could only actually see a quality difference in frames with motion).

Anyway, any thoughts on the AutoGK way vs default quantizers?

One other question for which I never found an answer.....
My basic understanding is that Xvid's compression/quality.... say 70% for example.... is literally obtained by running a complete Q2 single pass encode and then using 70% of the final file size for a 2 pass encode (is that correct?). Or it's normally worked out by running a compression test on x% of the video and then doing the math, but given AutoGK reduces the maximum allowed quantizer, you'd have to calculate the quality differently.... I assume?
For instance when I ran a single pass Q2 encode on a small file (where AutoGK used min and max quantizers of 2) the final file size was 31.3MB. When I ran a 2 pass encode using a file size of 22MB (70%) AutoGK reported the quality as being only 65% (while using min and max quantizers of 2 and 4). Any idea how AutoGK does it's quality calculations?

Last edited by hello_hello; 14th April 2012 at 11:02.
hello_hello is offline   Reply With Quote
Old 15th April 2012, 09:14   #11  |  Link
manono
Moderator
 
Join Date: Oct 2001
Location: Hawaii
Posts: 7,088
Quote:
Originally Posted by hello_hello View Post
I assume the reason for doing so would be to keep the quality of individual frames higher than Xvid otherwise would using the default quantizers of 1 and 31 (or 2 and 31). Would that be correct?
Yes, that's correct. The idea is to make a reasonable attempt at a CQ encode, while at the same time doing it for a defined file size. For that reason the quants are heavily restricted.
manono is offline   Reply With Quote
Old 13th April 2012, 23:37   #12  |  Link
sirt
x264 fan
 
sirt's Avatar
 
Join Date: Feb 2011
Location: In the trap
Posts: 458
But I want to know ! It's possible I am not able to understand without any specifical knownedge ; so, according to you, there is no need to set that in so far as the quantizer will AUTOMATICALLY choose the adequate quantizers and lowering the values could restrict its work, am I right this time ? You will say I am asking too much but let's try again...how can you be sure the "right" quantizer will be chosen ? And what do you mean by "right" ? I am also confused with the I-Frames and B-Frames chosen during an encode...how is that possible the encoder will choose the right one as a reference ? From what I have read it is possible to set manually what will be a I-Frame or B-Frame trough an override text but it is probably dedicated to XviD or x264 gurus. Of course I won't assume to tell you if a good frame was used a reference by the encoder to reconstruct other frames around basing its work on that refrenced one but...what if this reference frame is one of the worse from the video such as a ringed or destroyed one ? Then I think the segment near that frame will be encoded with a bad quality and I couldn't control that.
sirt is offline   Reply With Quote
Old 13th April 2012, 23:40   #13  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 12,800
Quote:
Originally Posted by sirt View Post
But I want to know ! It's possible I am not able to understand without any specifical knownedge ; so, according to you, there is no need to set that in so far as the quantizer will AUTOMATICALLY choose the adequate quantizers and lowering the values could restrict its work, am I right this time ? You will say I am asking too much but let's try again...how can you be sure the "right" quantizer will be chosen ? And what do you mean by "right" ?
The "right" quantizers are that which will hit desired average(!) bitrate you have selected.

If you want to know which quantizers happened to be the "right" ones for a particular source and for a particular target bitrate, just take the final output file and throw it at some analyze tool which can show the qunatizers

Quote:
I am also confused with the I-Frames and B-Frames chosen during an encode...how is that possible the encoder will choose the right one as a reference ? From what I have read it is possible to set manually what will be a I-Frame or B-Frame trough an override text but it is probably dedicated to XviD or x264 gurus. Of course I won't assume to tell you if a good frame was used a reference by the encoder to reconstruct other frames around basing its work on that refrenced one but...what if this reference frame is one of the worse from the video such as a ringed or destroyed one ? Then I think the segment near that frame will be encoded with a bad quality and I couldn't control that.
As explained earlier, scenecut detection is used to detect scene boundaries. If a scenecut is detected, a key-frame (I-Frame in the case of Xvid, IDR-Frame in the case of x264) will be placed at this point.

Deciding whether a particular frame "in the middle of a scene" should be coded as a P-Frame or as a B-Frame is a complex task and requires pretty complex algorithms/heuristics.

If you want to know how x264's adaptive B-frame decision algorithm works (actually there are two), you will have to study the source code. Many more simple encoders use a fixed number of consecutive B-Frames...
__________________
There was of course no way of knowing whether you were being watched at any given moment.
How often, or on what system, the Thought Police plugged in on any individual wire was guesswork.



Last edited by LoRd_MuldeR; 14th April 2012 at 00:41.
LoRd_MuldeR is offline   Reply With Quote
Old 13th April 2012, 23:59   #14  |  Link
sirt
x264 fan
 
sirt's Avatar
 
Join Date: Feb 2011
Location: In the trap
Posts: 458
I think it will not be easy to study the C files ; I understand the language used (I study C/C++ on my own) but it would be a payoff I could understand like 5000 lines of code myself.

Then the question is perhaps more complex : what will be the most accurate bitrate to use (with XviD or x264 in 2 Pass mode ? Of course you could reply it would be around 800/900 for a movie and a bitrate calculator will help to find the proper value to reach 700 mb or 1400 mb (and same idea for HD Ready or HD Full Blurays) ; I know setting a high bitrate means the quality will - normally - be better, but in that case why not just setting a bitrate of 20000 for each encode and that's all in spite of the large file I might collect. More precisely, isn't each time a bitrate not to exceed (I only refer to high bitrates) ? I wonder for example if setting the bitrate too high (like 50 000 for example) won't give a bad output even though I've read in many places a high bitrate could never be bad.
sirt is offline   Reply With Quote
Old 14th April 2012, 00:21   #15  |  Link
Guest
Guest
 
Join Date: Jan 2002
Posts: 21,924
Quote:
Originally Posted by sirt View Post
why not just setting a bitrate of 20000 for each encode and that's all in spite of the large file I might collect
If you don't care about file size, then leave it uncompressed. Your questions are not making much sense to me.
Guest is offline   Reply With Quote
Old 14th April 2012, 00:27   #16  |  Link
sirt
x264 fan
 
sirt's Avatar
 
Join Date: Feb 2011
Location: In the trap
Posts: 458
Quote:
Originally Posted by neuron2 View Post
If you don't care about file size, then leave it uncompressed. Your questions are not making much sense to me.
neuron2, well I care about file size but I have not a precise idea of how to set a bitrate if you prefer. Of course there are "commercial" standards such as the one mentionned above (700 MB for a 1CD movie, eventually 1400 MB for a 2CD movie, 4 GB for a HD heavy Blu Ray and 8 GB for a HD Full Bluray) but my question was : is that accurate ? How would you know the "accurate" bitrate ? I could answer myself : it depends on the needs...of course it does but I wonder if setting a high bitrate could have negative effects that could be avoided by a lower one. That' why I ask for a "accurate" bitrate (depending on the source) not to be exceeded.
sirt is offline   Reply With Quote
Old 14th April 2012, 00:35   #17  |  Link
Guest
Guest
 
Join Date: Jan 2002
Posts: 21,924
It's all subjective. There is no "accurate bitrate" to find. You have two conflicting goals: minimizing file size and minimizing quality degradation. Only *you* can decide which point of the tradeoff satisfies you while adhering to the constraints of your application domain. The only negative for setting a high bitrate is that the file size grows, and if you are streaming it might exceed the channel capacity. You also have a "channel" from your hard disk and uncompressed HD (the maximum bitrate!) can exceed the hard disk maximum read rate.

Last edited by Guest; 14th April 2012 at 00:41.
Guest is offline   Reply With Quote
Old 14th April 2012, 01:36   #18  |  Link
sirt
x264 fan
 
sirt's Avatar
 
Join Date: Feb 2011
Location: In the trap
Posts: 458
Okay, let's do it simple...if you are encoding in a 2 pass mode how do you (personally) choose your bitrate regarding the input source, your expectations ect ?
sirt is offline   Reply With Quote
Old 14th April 2012, 01:50   #19  |  Link
Groucho2004
 
Groucho2004's Avatar
 
Join Date: Mar 2006
Posts: 3,446
Quote:
Originally Posted by sirt View Post
Okay, let's do it simple...if you are encoding in a 2 pass mode how do you (personally) choose your bitrate regarding the input source, your expectations ect ?
Is this question directed at neuron2 or anybody?
Groucho2004 is offline   Reply With Quote
Old 14th April 2012, 02:00   #20  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 12,800
Quote:
Originally Posted by sirt View Post
Okay, let's do it simple...if you are encoding in a 2 pass mode how do you (personally) choose your bitrate regarding the input source, your expectations ect ?
If you use 2-Pass mode, this usually indicates that you need to hit a specific average bitrate or a specific file size. It's both the same, because the file size is a direct consequence of the average bitrate.

So, if you need to hit a specific file size, then you know what that size that is. For example 4,7 GB to make the video fit on a DVD-R. Or maybe 25 GB to make it fit on a BD-R.

Now that you have made clear to yourself which file size you need to hit, calculating the corresponding average bitrate is simple math. However take into account the space for the audio track and the container overhead!

Also note: In case you do not need to hit a specific file size, there is not much reason to use 2-Pass mode. Instead you would simply use CRF mode with the highest possible CRF value that still satisfies you eyes.

(In case of Xvid there is no CRF mode, unfortunately, so you would have to use CQ mode instead - or switch to x264 ^^)
__________________
There was of course no way of knowing whether you were being watched at any given moment.
How often, or on what system, the Thought Police plugged in on any individual wire was guesswork.


LoRd_MuldeR is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 05:34.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2017, vBulletin Solutions Inc.