[Test] Little MPEG2 "Low Bitrate" comparison [Archive]

Sagittaire

13th May 2006, 23:30

I make little visual comparison between recent MPEG2 encoder for very low bitrate.

I use this Quick Time AVC 1080p source:
http://www.apple.com/trailers/wb/harry_potter/thegobletoffire/hd/

I use this avisynth script:
source=DirectShowSource("D:\Mes dossiers\High-Definition\harry_potter_goblet-tlr2_h1080p.mp4", fps=23.976)
source=Crop(source,4,4,-4,-4)
source=Lanczos4Resize(source,720,432)
source=AssumeFPS(source,25)
source=addborders(source,0,72,0,72)
source=limiter(source)
Return(source)

I use TMPGEnc CQ50 CQM "Notch" for find my initial bitrate : 1460 Kbps. This quality level is very "low bitrate" encoding. I make encoding with the other MPEG2 codec. I use the best possible quality profil for all MPEG2 encoder.

TMPGEnc 2.521.58.169 "KDVD profil" with "Notch Matrix"
http://jfl1974.free.fr/upload/TMPGEnc.m2v

Mencoder with "Sagittaire Matrix"
http://jfl1974.free.fr/upload/Mencoder.m2v

QuEnc 0.70 with "QLB matrix"
http://jfl1974.free.fr/upload/QuEnc.m2v

HC 0.18 with "Notch Matrix"
http://jfl1974.free.fr/upload/HCEnc.m2v

CCE 2.70.02.00 with "Notch Matrix"
http://jfl1974.free.fr/upload/CCE.m2v

danpos

13th May 2006, 23:41

@Sagittaire

Why not to include AQE (AutoQMatEnc) by SAPSTAR in yours tests?

Cheers,

Revgen

14th May 2006, 01:56

@Sagittaire

Did you use Quenc 70b for the testing?

The first Quenc .70 release had an issue with undersized bitrates that has since been fixed with .70b

IgorC

14th May 2006, 03:47

For my eyes:
hcenc - ringing on edges, artefacts, visual blocks etc. - 5 points
cce - smooth as a result of a low bitrate - 6 points
TMPGEnc - 6,5 points
quenc 0.7 - some artefacts but the details are ok. - 6 points
mencoder - excelent for SVCD bitrate - 7 points

Revgen

14th May 2006, 04:57

My order was

1) TMPEnc - Not as sharp as mencoder, but not as blocky through scene changes.

2) Mencoder- Looked good and sharper than TMPEnc except it was very blocky when scenes would change quickly. I would be more inclined to use it on a real movie.

3) HCEnc - Looked okay, but not as sharp as Mencoder and a slightly more blocky than TMPEnc. I typically use Avamat7 matrix with HC. I'll try encoding it myself and see what happens.

4) CCE - Too much ringing around characters and on characters faces.

5) Quenc - Too blocky. I wonder if .70b was used or not?

dragongodz

14th May 2006, 06:27

also note there is 3 different matrices being used which of course can impact on the end result.

I use the best possible quality profil for all MPEG2 encoder.
if people try for themselves can they please give a more detailed description of what settings they used ?

Nic

14th May 2006, 08:10

This whole thread is using the word "best" a little too much for my liking. Keep this thread constructive and friendly or it will be closed.

-Nic

xyloy

14th May 2006, 10:56

@Sagittaire:
A very MPEG2-experienced guy on another forum posted an unusual but very good way to encode with TMPGenc Plus by using an aviSynth script and VirtualDub's frame serving here (http://http://www.emule-france.com/forum/tutos-transformer-divx-dvd-super-video,t52515.htm)(careful other people it's in french! But the images speak for themselves, most of the time)

The tutorial(the two first posts on the link I gave you) is for converting an .avi file to optimized SV-CD, but I think the interest in thoses posts for your comparison is the use of an aviSynth script, and a DC componement precision of 10 bit(unlocked TMPGEnc Plus profile).

On another note, it might be useful to use an up-to-date version of TMPGEnc Plus in your comparison, in case there is some improvments/bugfixes on TMPGEnc's MPEG-2 encoder..

Hope this post will be useful to you. ;)

MOmonster

14th May 2006, 13:23

Sagittaire

14th May 2006, 15:21

if people try for themselves can they please give a more detailed description of what settings they used ?

TMPGEnc 2.521.58.169 "KDVD profil"
CQ50, "Notch Matrix", Highest quality mode

Mencoder
3 pass, "Sagittaire matrix", new bframe strategy, RDO max, trellis

QuEnc 0.70
2 pass, "QLB matrix", VBR, High Quality and "extreme", trellis

HC 0.18
2 pass, "Notch Matrix", best quality, auto GOP

CCE 2.70.02.00
4 pass, "Notch Matrix", adaptative CQM

also note there is 3 different matrices being used which of course can impact on the end result.

yes, it's true. Matrix like "Notch" seem particulary optimised for TMPGEnc and fight against blocking but produce too high ringing level. "Notch" seem not very good for libavcodec. I prefer and by far my "Sagittaire matrix" with my inter/intra setting. "QLB matrix" seem not very good for this example but my setting for Mencoder (Libavcodec) are really more powerfull than QuEnc (Libavcodec) ...

Sagittaire

14th May 2006, 15:23

In my opinion:

1) mencoder - stable quality, keep details

2) tmpgenc - also really stable, but less sharp and a bit more blocky

3) hcenc - clearly improved compared to older versions but still blocky with really low bitrates

4) cce - stable quality, ringing, blurred backgrounds

5) quenc - just too blocky on scenechanges, quality is really unstable

exactly like my eyes ... ;-)

Revgen

14th May 2006, 18:54

scharfis_brain

14th May 2006, 19:00

Sagittaire: you forgot ProCoder 2.

Revgen

14th May 2006, 19:13

@Sagittaire

How did you download the trailer?

danpos

14th May 2006, 20:00

@Sagittaire

How did you download the trailer?

I'd ask this too. I'd like to download it in order to try it out with AQE ... ;)

Revgen

14th May 2006, 20:06

Nevermind. I was able to dump it with VLC player.

IgorC

14th May 2006, 20:32

To download Harry Potter 4 Trailer 2 :
1. Click on 1080p icon
2. QT7 will open
3. go to Window -> Show Info ( Source : http:....... ) where you will get link

http://movies.apple.com/movies/wb/harry_potter_goblet/harry_potter_goblet-tlr2_h1080p.mov

MOmonster

14th May 2006, 22:51

I agree with you in general. But for this particular trailer, it really exposes mencoder's issues with handling scene changes. IMHO, real movies would put it ahead of TMPGEnc.
Both encoders did a damned nice job for this bitrate and source. And yes tmpgenc handles the scenechanges just a little bit better, but still the overall quality of mencoder, also for this small trailer, wins this comparison in my subjective perception.

danpos

14th May 2006, 23:14

dragongodz

14th May 2006, 23:55

"QLB matrix" seem not very good for this example
i designed it for lower bitrates, like sub 3000, not extremely low such as 1460. even with that though it should look better than not using a custom matrix at all. fairer to say may not be as good as some other custom matrices at such a VERY low bitrate with this source no ?

can you also confirm that you did redownload QuEnc 0.70, the b version, and used that ?

Revgen

15th May 2006, 00:07

dragongodz

15th May 2006, 01:28

I encoded with Quenc .70b using Saggitaire's settings and got the same result unfortunately.
its a pity he didnt give the "Sagittaire matrix" so that could have been tested with. also i should probably have suggested not using trellis for anything at the moment because of that weird lower quality result in the QuEnc thread.

On the other hand, trailers aren't exactly real movies.
true. they can be interesting at times though. actually i do not know what would have been wrong using the same clip he used in the last test. being on limited downloads and only 256kbps speed means i am not going to download that ridiculously large trailer only to resize it down to dvd resolution and test. :)

Sagittaire

15th May 2006, 10:47

I encoded with Quenc .70b using Saggitaire's settings and got the same result unfortunately.

On the other hand, trailers aren't exactly real movies. They change scenes very quickly and they don't run very long. IMHO, these tests should use a 10 minute real movie or tv show clip that isn't constantly changing. This would resemble a real movie more closely.

well don't change anything:
1) I use low quality reference with TMPGEnc for target bitrate definition. Bitrate here doesn't mean anything.
2) MPEG2 PAL DVD stream use very short GOP very smaller than change scene in these trailer.
3) Trailers are real part of real movie: you can find exactly the same frame for real movie or trailers.

its a pity he didnt give the "Sagittaire matrix" so that could have been tested with. also i should probably have suggested not using trellis for anything at the moment because of that weird lower quality result in the QuEnc thread.

well it's really easy to know this matrix with ffdshow for example. My matrix is very simple:

08,16,16,16,17,18,21,24,
16,16,16,16,17,19,22,25,
16,16,17,18,20,22,25,29,
16,16,18,21,24,27,31,36,
17,17,20,24,30,35,41,47,
18,19,22,27,35,44,54,65,
21,22,25,31,41,54,70,88,
24,25,29,36,47,65,88,115

16,16,16,16,17,17,18,19,
16,16,16,16,17,18,19,20,
16,16,16,17,18,19,20,22,
16,16,17,18,19,21,23,26,
17,17,18,19,21,24,27,31,
17,18,19,21,24,28,33,40,
18,19,20,23,27,33,42,51,
19,20,22,26,31,40,51,64

But problem here is not matrix. My libavcodec setting done very better result with QLB Matrix too.

dragongodz

15th May 2006, 13:06

MPEG2 PAL DVD stream use very short GOP very smaller than change scene in these trailer.
i think you missed what he meant. that is that a whole movie is made up of many scenes of many lengths. so bitrate can be dispersed much wider over such a long period. with a short clip, such as a trailer, you are having many scene changes which if it was cut from a full length movie would probably get more bitrate. so other calmer sections would get less etc.

But problem here is not matrix. My libavcodec setting done very better result with QLB Matrix too.
by having multiple matrices you are adding another non-consistant variable. to even things out the same matrix should be used if possible.
also will you stop with the better/best. Nic already gave a warning about it in this thread. its one thing to say which looks most pleasing to you but another thing to say flatly best or better. looking at the small number of results from this poll already shows a spread of what people prefer.
i downloaded the QuEnc, HC and Mencoder clips. QuEnc is a bit blocky in places but keeps very good detail. Mencoder is less blocky(there are still blocks easily visible) and smooths in some places to reduce blocks etc. HC ... wow how about that ringing, this was the most unpleasant to my eyes of these 3 clips.

ricardo.santos

15th May 2006, 13:22

Hi everyone!

Altough im no expert on this matters, i think using a bitrate of 1460 isnt really usefull as I THINK that specific bitrate value is really not used on a daily basis.

Its like having 3 cars and giving each driver 1/7 of a liter of gasoline and see which one makes more Kms out of it. Just my personal opinion.

Altough i didnt watch the trailers, on previous tests i done Quenc had the best result to my eyes.

dragongodz

15th May 2006, 15:16

i think using a bitrate of 1460 isnt really usefull as I THINK that specific bitrate value is really not used on a daily basis.
yes thats true. so really people should think of this more as an extreme or torture test and not what a full dvd backup would be.
that doesnt mean it cant be interesting at times. just that people should keep it in perspective. :)

Sagittaire

15th May 2006, 15:42

i think you missed what he meant. that is that a whole movie is made up of many scenes of many lengths. so bitrate can be dispersed much wider over such a long period. with a short clip, such as a trailer, you are having many scene changes which if it was cut from a full length movie would probably get more bitrate. so other calmer sections would get less etc.

well like I say :
- Scene change don't spend more size here simply because GOP in PAL use by default high Iframe frequencies. With PAL GOP scene change use always 15 frames period. High scene change in source doesn't change in practice anything with short GOP for bitrate (if GOP frequencies is higher than source scene change frequencies)
- Bitrate doesn't mean anything: I use CQ50 TMPGEnc (very low quality) for reference. This reference done my target bitrate : I can make that with complete movie too but it's useless.

by having multiple matrices you are adding another non-consistant variable. to even things out the same matrix should be used if possible.

Default MPEG Matrix perhaps ... ?
It's the default matrix for many MPEG2 encoder.

by having multiple matrices you are adding another non-consistant variable. to even things out the same matrix should be used if possible.
also will you stop with the better/best. Nic already gave a warning about it in this thread. its one thing to say which looks most pleasing to you but another thing to say flatly best or better. looking at the small number of results from this poll already shows a spread of what people prefer.
i downloaded the QuEnc, HC and Mencoder clips.

"better setting for libavcodec" here is simply documentation from libavcodec like for example new adaptative bframe: done very smaller size for same quantizer for exactly same quality for same encoder (libavcodec). I can say that libavcodec done better result if you use RDO too simply because it's always true.

Sagittaire

15th May 2006, 15:52

davidcw

15th May 2006, 22:05

I was pleasantly suprised at the extra sharpness I got with Sagittaire's matrix when used in HC 0.18 instead of the default MPEG matrix for Rebuilder DVD backups.
davidcw

Revgen

15th May 2006, 22:36

Perhaps an xvid .txt matrix of Sagittaire's CQM could be made for Quenc?

dragongodz

16th May 2006, 04:32

Default MPEG Matrix perhaps ... ?
thats up to you. the point is to to not make the matrix a factor for differences in the output becuase then it becomes encoder+matrix rather than just encoder.

"better setting for libavcodec" here is simply documentation from libavcodec like for example new adaptative bframe: done very smaller size for same quantizer for exactly same quality for same encoder (libavcodec). I can say that libavcodec done better result if you use RDO too simply because it's always true.
this will not ALWAYS hold true,especially when you are using peoples opinions and not metrics, no matter what any documentation says. the results of this poll should be showing you that. :)
it was just a suggestion to not use the words as absolutes but i am not going to get in to some big debate over it. that would be just taking this thread more off topic.

Bitrate doesn't mean anything here: with same average quant source like football/hockey/DV will use perhaps 4500 Kbps or more.
of course bitrate is relevant in the context of the source type and output target. you are encoding to dvd resolution so an average bitrate of 1460 is extremely low and one that would be used in such a case once in a blue moon in reality. so as i said this is more an extreme or torture test rather than what a normal dvd backup would be. there is nothing wrong with that.

Perhaps an xvid .txt matrix of Sagittaire's CQM could be made for Quenc?
you mean like this
http://d.turboupload.com/d/607966/Sagittaire-matrix.zip.html

this of course can be used with HC aswell. people do realise that both QuEnc and HC use Xvid type matrices files, such as in "Matrix" folder that comes with HC, and its only the extension thats different, doent they ?

Observer

16th May 2006, 05:24

@Sagittaire
Interesting results.

Where can I find the installer that contains all the files required to encode mpeg2 with Mencoder and libavcodec?

kolak

17th May 2006, 09:51

Out of topic:
Who encodes with Mpeg2 at 1.5 Mbit?
This is useless comparision for me:)
If you want 1.5 Mbit use AVC or any other Mpeg4 codec.
btw... don't forget that CCE will change your initial matrix if you don't set "Disable adaptive Q-Matrix switching" and use more than 2 passes

dragongodz

17th May 2006, 11:58

Who encodes with Mpeg2 at 1.5 Mbit?
this is already discussed in several posts. please read my last couple of posts and that may help you to look at it in a different way and serving a different purpose.
so while it is not a scenario that most people will really run in to that doesnt make it useless or to some of us interesting.

scharfis_brain

17th May 2006, 18:09

Once I needed bitrates of about 1.5 mbps (and below) for some interlaced stuff that I pressed on one SVCD...
(no, DVD was no option as well as VCD was't)

kolak

18th May 2006, 16:57

"so while it is not a scenario that most people will really run in to that doesnt make it useless or to some of us interesting."

TRUE:)

rahzel

19th May 2006, 00:18

3) HCEnc - Looked okay, but not as sharp as Mencoder and a slightly more blocky than TMPEnc. I typically use Avamat7 matrix with HC. I'll try encoding it myself and see what happens.
did you end up encoding it with Avamat7?

i find avamat7 (aka autoq2) is a very good matrix with HC (and CCE) for pretty much all projects. keeps details and removes a lot of blocking. im new to DVD-RB and HC, but im impressed enough with this combo (HC and AutoQ2) that im probably going to use this for all my projects that require mid-heavy compression.

Revgen

19th May 2006, 01:28

did you end up encoding it with Avamat7?

i find avamat7 (aka autoq2) is a very good matrix with HC (and CCE) for pretty much all projects. keeps details and removes a lot of blocking. im new to DVD-RB and HC, but im impressed enough with this combo (HC and AutoQ2) that im probably going to use this for all my projects that require mid-heavy compression.

I looked at the HCEnc encoding done by Sagittaire with the Notch matrix.

Sagittaire

20th May 2006, 14:11

poll result are strange : HC 0.18 produce best visual result for majority.

For my eyes: Mencoder or TMPGEnc produce definitively and objectively better visual result and by far for blocking, ringing and sharpening level for this example. Moreover I have 20 votes but not 20 download for each sample : user say "codec X is visualy the best" without real comparison here. IMO hank315 will say himself that HcEnc don't produce the best visualy quality here ... lol

for me it's just confirmartion:

Besides, subjective testing is also very difficult to do and is not always reliable.

GodofaGap

20th May 2006, 15:28

poll result are strange : HC 0.18 produce best visual result for majority.

I think its your interpretation of the poll results that is strange. :)

HCenc does not produce the best result for the majority. Right now it has 6 out of 19 votes, so that is about 30 percent. Mencoder and QuEnc score equally good. Furthermore, it's not a blinded test. Results from a non-blinded subjective test are dodgy at best. :rolleyes:

dragongodz

20th May 2006, 15:42

I think its your interpretation of the poll results that is strange.
what he said was technically correct. that more people prefered the result from HC to the other encoders. of course this is not a clear total majority of voters since its not over 50%.

for me it's just confirmartion:
well yes and no. yes subjective testing is hard to do and yes it isnt always reliable. however the reason it may not be reliable is because of factors such as sample size and if its not done blind etc.
if its not blind then there can be the placebo effect for some people. that is they will think something is better or worse based on their preconceptions and not on what they really see.
sample size is VERY important. 19 votes is absolutly nothing and in realty not give you any sort of reliable result. to get real reliability you would need more like thousands of votes. even in the hundreds would be becoming significant enough to warrant study on why votes went certain ways.

GodofaGap

20th May 2006, 16:56

what he said was technically correct.
I am sorry, I did not want to start a disucssion about the most common usage of the word "majority (http://dictionary.reference.com/search?q=majority)". I personally think the wording could have been chosen better.

But the essence of my post was that a contender with 6 out of 19 can hardly be called "best" when there are two other contenders with 5 out of 19 votes. The amount of votes and testing conditions do not warrant this.

I am sure you agree.

IgorC

20th May 2006, 17:14

There was a software for video blind test in HA forum.
Many people just vote for their favorite codec whithout even see others. For others speed has a higher prio ...etc. And of course devs will vote for their codec. :)
The Results of not blind test are far to be realistic.

Just name codec like A,B,C,D... and then say what each codec was. It would be some kind of blind test.

dragongodz

20th May 2006, 17:26

I am sorry, I did not want to start a disucssion about the most common usage of the word "majority". I personally think the wording could have been chosen better.
possibly but i think everyone would understand its meaning (http://www.askoxford.com/concise_oed/majority?view=uk) in this context. especially since they can see the numbers for themselves. ;)

But the essence of my post was that a contender with 6 out of 19 can hardly be called "best" when there are two other contenders with 5 out of 19 votes. The amount of votes and testing conditions do not warrant this.

I am sure you agree.
if you read my previous post you will see i do agree and also why.

foxyshadis

20th May 2006, 23:24

I guess people really like extreme ringing. Of course, it might be that everyone had post-processing on, and didn't think to turn it off to see how it'd look on a hardware player. PP can make an enormous difference.

Sagittaire

21st May 2006, 00:10

I am sorry, I did not want to start a disucssion about the most common usage of the word "majority". I personally think the wording could have been chosen better.

there are absolute majority (more than 50%) and relative majority (simply first). In this case it's relative majority ...

Mug Funky

21st May 2006, 14:38

a couple of suggestions for this kind of test (by no means stop running them - it'll keep the codec devs on their toes...):

- keep settings consistent between all encoders
- choose a bitrate and RC mode that is repeatable on all encoders, and pre-set by you rather than chosen based on what one of the contenders produced
- have a specific application in mind. not to be rude, but it doens't seem useful to me to test an MPEG-2 codec in full D1 size at such a low bitrate, as no sane person will ever use MPEG-2 for such things.* "DVD backup" is probably a more real world application, where you'll be looking at bitrates twice what you have or more. when you're running against the limits of MPEG-2 itself, one can say little about individual implementations of it - it's a matter of choosing blocks over rings, over spoiled motion, drifting colours, etc.

one thing that'd be an interesting test is comparing subjective results against objective results. an example would be using all the different encoders to do a CQ 10 encode and see which delivers the lowest bitrate, then see which looks the best and which one wins. this might give insights into what to use for low bitrate encodes or high bitrate encodes, etc. it would also say a fair bit about motion precision, and if speed were taken into account it could be a useful way to choose an encoder.

another possible test would be to flesh out the settings for one encoder only - try them all and see what happens to the output, choose a best one, etc (though a test like that could be done automatically based on metrics, then verified by human vision).

[edit]

* the lowest i've had to go is D1 cropped (704x576), 3700 CBR, and that was for some VOD service that required a max bandwidth of 4mbps including audio. surprisingly this looked really good using just a spruce encoder, which is normally crap at such a bitrate. i guess the CBR mode is more efficient than it's 2-pass mode in some weird way.