Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 AVC / H.264

Reply
 
Thread Tools Search this Thread Display Modes
Old 10th February 2005, 00:18   #1  |  Link
DmitriyV2
MSU G&M Lab
 
DmitriyV2's Avatar
 
Join Date: Oct 2004
Location: MSU G&M Lab
Posts: 318
MSU MPEG-4 AVC/ H.264 codec comparison test

MSU MPEG-4 AVC/ H.264 codec comparison test released!

Main features:
* 6 H.264 codecs was compared with last DivX.
* 3 codecs was recieved from codec developers directly for test.
(We would like to thank Moonlight Cordless LTD, Fraunhofer Institute for Integrated Circuits IIS and Ateme for kindly providing us their codecs for this test.)
* We measure the most used PSNR, Bitrate handling and visual comparison.

This comparison was first is series.

In next version (will be in 2 month):
* New codecs will be added
* We plan to add new measures
* 2 presets, recieved from codec developers will be measured:
"tuned" - maximum quality,
"fast" - maximum (optimum) speed
* Rules for uniformal comparison will be more formal.

http://compression.ru/video/codec_co...c_h264_en.html

Enjoy!
__________________
With regards
Dmitriy Vatolin
www.compression.ru/video/ (Last updates: MSU Video Quality Metric 2.5 - faster&more useful)
DmitriyV2 is offline   Reply With Quote
Old 10th February 2005, 00:59   #2  |  Link
akupenguin
x264 developer
 
akupenguin's Avatar
 
Join Date: Sep 2004
Posts: 2,393
Errors:
  • The graph labelled "On this picture Y-PSNR values are shown" is titled "V-PSNR". Which is it?
  • One of the entries in the graphs is labelled "AVC". All of the codecs (except DivX of course) are AVC; I assume you mean Mpegable?
  • You calculate PSNR as an arithmetic mean of per-frame PSNR. This is completely bogus. (Think: what does a single pure-black frame do to the average? Hint: infinity. And that's only an extreme case, anything involving an average of logarithms is bogus even if you don't actually have any perfect frames.) In a constant-quality encode it might be good enough, but this was CBR, so quality goes all over the place. The correct measure of global PSNR is log(mean(error)), not mean(log(error)).
... and there's no x264 . But I'm not particularly impressed by our CBR ratecontrol, so maybe that's a good thing.

Last edited by akupenguin; 10th February 2005 at 01:01.
akupenguin is offline   Reply With Quote
Old 10th February 2005, 16:07   #3  |  Link
SeeMoreDigital
Life looks better in UHD
 
SeeMoreDigital's Avatar
 
Join Date: Jun 2003
Location: Notts, UK
Posts: 11,321
I reckon this thread should be moved over to the "New A/V Formats - Codecs" section...


Cheers
__________________
| I've been testing hardware media playback devices and software A/V encoders and decoders since 2001 | My Network Layout & A/V Gear |
SeeMoreDigital is offline   Reply With Quote
Old 10th February 2005, 17:07   #4  |  Link
CruNcher
Registered User
 
CruNcher's Avatar
 
Join Date: Apr 2002
Location: Germany
Posts: 4,949
DmitriyV2 i hope in the second round you'll compare with XviD too
__________________
all my compares are riddles so please try to decipher them yourselves :)

It is about Time

Join the Revolution NOW before it is to Late !

http://forum.doom9.org/showthread.php?t=168004
CruNcher is offline   Reply With Quote
Old 10th February 2005, 21:14   #5  |  Link
lazyn00b
Registered User
 
Join Date: Feb 2003
Location: USA
Posts: 152
A minor quibble: "Ateme" should probably be changed to "Nero", unless you got this version of the codec directly from Ateme. My assumption (correct me if I'm wrong) is that Nero paid for this codec to be developed and for the right to distribute it under their name. As far as I know, there is no "Ateme" codec that can purchased or evaluated by the regular consumers reading your test.
lazyn00b is offline   Reply With Quote
Old 10th February 2005, 21:16   #6  |  Link
Sharktooth
Mr. Sandman
 
Sharktooth's Avatar
 
Join Date: Sep 2003
Location: Haddonfield, IL
Posts: 11,768
He tested the commandline encoder from Ateme and i suppose (but i cant really tell) he tested the high profile.
Sharktooth is offline   Reply With Quote
Old 10th February 2005, 21:18   #7  |  Link
Manao
Registered User
 
Join Date: Jan 2002
Location: France
Posts: 2,856
There are some strange things :
* foreman is 30 fps, not 15
* the black square helps codecs with a logo, and the bigger the logo, the bigger the help (!)
* there's no way for divx's psnr curve to be higher than ateme's, even at high bitrates, and certainly not on foreman ( and since you give per frame psnr for high bitrate on foreman, you can check that something is wrong ).
Manao is offline   Reply With Quote
Old 10th February 2005, 21:22   #8  |  Link
lazyn00b
Registered User
 
Join Date: Feb 2003
Location: USA
Posts: 152
Quote:
Originally posted by Sharktooth
He tested the commandline encoder from Ateme and i suppose (but i cant really tell) he tested the high profile.
Thanks, I stand corrected, then. Please ignore my post above.

lazyn00b
lazyn00b is offline   Reply With Quote
Old 10th February 2005, 22:00   #9  |  Link
Manao
Registered User
 
Join Date: Jan 2002
Location: France
Posts: 2,856
Quote:
i suppose (but i cant really tell) he tested the high profile.
Hard to tell : they don't give the settings for each codec, nor the encoding speed.
Manao is offline   Reply With Quote
Old 10th February 2005, 22:05   #10  |  Link
bobololo
Registered User
 
Join Date: May 2003
Posts: 328
Quote:
Originally posted by Manao
Hard to tell : they don't give the settings for each codec, nor the encoding speed.
They encoded using the default settings, so they only used main profile encoding.
bobololo is offline   Reply With Quote
Old 10th February 2005, 22:17   #11  |  Link
DmitriyV2
MSU G&M Lab
 
DmitriyV2's Avatar
 
Join Date: Oct 2004
Location: MSU G&M Lab
Posts: 318
Quote:
Originally posted by lazyn00b
A minor quibble: "Ateme" should probably be changed to "Nero", unless you got this version of the codec directly from Ateme. As far as I know, there is no "Ateme" codec that can purchased or evaluated by the regular consumers reading your test.
Sure! This codec does no available for customers now (unfortunatelly). We got codec directly from Ateme.
Please see screenshot at the beginning of comparison!
(We began to make screenshots after the same discuaaion with our previous test).
__________________
With regards
Dmitriy Vatolin
www.compression.ru/video/ (Last updates: MSU Video Quality Metric 2.5 - faster&more useful)
DmitriyV2 is offline   Reply With Quote
Old 10th February 2005, 22:18   #12  |  Link
DmitriyV2
MSU G&M Lab
 
DmitriyV2's Avatar
 
Join Date: Oct 2004
Location: MSU G&M Lab
Posts: 318
Quote:
Originally posted by bobololo
They encoded using the default settings, so they only used main profile encoding.
Sure! We plan to use 2 presets provided by codec developers from next comparison.
__________________
With regards
Dmitriy Vatolin
www.compression.ru/video/ (Last updates: MSU Video Quality Metric 2.5 - faster&more useful)
DmitriyV2 is offline   Reply With Quote
Old 10th February 2005, 22:28   #13  |  Link
DmitriyV2
MSU G&M Lab
 
DmitriyV2's Avatar
 
Join Date: Oct 2004
Location: MSU G&M Lab
Posts: 318
Quote:
Originally posted by akupenguin
... and there's no x264 . But I'm not particularly impressed by our CBR ratecontrol, so maybe that's a good thing. [/B]
Thanks for verification! We'l check it! Metric from next comparison will be published for free usage, hope you will test it as more, as possible!

x264 will be added to next comparison, so you have about 2 month to tune rate control. Hope you will improve it fundamentally!
__________________
With regards
Dmitriy Vatolin
www.compression.ru/video/ (Last updates: MSU Video Quality Metric 2.5 - faster&more useful)
DmitriyV2 is offline   Reply With Quote
Old 10th February 2005, 22:31   #14  |  Link
DmitriyV2
MSU G&M Lab
 
DmitriyV2's Avatar
 
Join Date: Oct 2004
Location: MSU G&M Lab
Posts: 318
Quote:
Originally posted by CruNcher
DmitriyV2 i hope in the second round you'll compare with XviD too
Not to this (H.264) comparison.

We plan MPEG-4 codecs comparison with all version of DivX (from early to modern) and with XviD of cource!
__________________
With regards
Dmitriy Vatolin
www.compression.ru/video/ (Last updates: MSU Video Quality Metric 2.5 - faster&more useful)
DmitriyV2 is offline   Reply With Quote
Old 10th February 2005, 22:45   #15  |  Link
DmitriyV2
MSU G&M Lab
 
DmitriyV2's Avatar
 
Join Date: Oct 2004
Location: MSU G&M Lab
Posts: 318
Quote:
Originally posted by Manao
There are some strange things :
* foreman is 30 fps, not 15
We have 15 fps foreman only. If you have really 30 fps (not 2 times faster moved) - please put it somewhere for me!
Quote:
* the black square helps codecs with a logo, and the bigger the logo, the bigger the help (!)
Yes! It's not justly, that bigger logo a little increase metric.
We console oneself only that bigger logo _essentially_ decrease number of codec users! So this damage, I think, is more serious.

And do not worry - our main course - work with codec developers directly, so we hope to clear away comparison from such codecs.

Quote:
* there's no way for divx's psnr curve to be higher than ateme's, even at high bitrates, and certainly not on foreman ( and since you give per frame psnr for high bitrate on foreman, you can check that something is wrong).
We will put measure program for free for
Really we found more such observations about codecs optimization.
__________________
With regards
Dmitriy Vatolin
www.compression.ru/video/ (Last updates: MSU Video Quality Metric 2.5 - faster&more useful)
DmitriyV2 is offline   Reply With Quote
Old 11th February 2005, 00:50   #16  |  Link
CruNcher
Registered User
 
CruNcher's Avatar
 
Join Date: Apr 2002
Location: Germany
Posts: 4,949
Maybe you missunderstood me i meant that
Main features:
* 6 H.264 codecs was compared with last DivX.
why did you compared vs DivX instead of XviD im not sure that this is really fair ?
__________________
all my compares are riddles so please try to decipher them yourselves :)

It is about Time

Join the Revolution NOW before it is to Late !

http://forum.doom9.org/showthread.php?t=168004
CruNcher is offline   Reply With Quote
Old 11th February 2005, 07:58   #17  |  Link
Sergey A. Sablin
Registered User
 
Join Date: Dec 2004
Location: Tomsk, Russia
Posts: 366
Quote:
Originally posted by DmitriyV2
We have 15 fps foreman only. If you have really 30 fps (not 2 times faster moved) - please put it somewhere for me!
Hi Dmitriy!
If you try to play back yours foreman with 30 fps you will see that in this case it is more closely to reality than playing it with 15 fps
Sergey A. Sablin is offline   Reply With Quote
Old 11th February 2005, 08:37   #18  |  Link
LordRPI
Registered User
 
Join Date: Sep 2004
Location: SoCal
Posts: 261
Quote:
Originally posted by CruNcher
Maybe you missunderstood me i meant that
Main features:
* 6 H.264 codecs was compared with last DivX.
why did you compared vs DivX instead of XviD im not sure that this is really fair ?
Not to mention the study used DivX 5.1.1, not DivX Fusion.
LordRPI is offline   Reply With Quote
Old 11th February 2005, 11:25   #19  |  Link
stephanV
gone
 
Join Date: Apr 2004
Posts: 1,709
DivX wasnt used as DivX as such, but as a MPEG4 ASP codec to compare with some H264 codecs, to see an improvement of H264 over MPEG4 ASP. DivX would suffice for that. It's not directly related to fairness.

If I interpret the conclusions of the test correctly, DivX ended up 2nd after Ateme so I doubt if using XviD would have really changed the results. Also, I hope that the possible difference between a H264 codec and a MPEG4 ASP codec is bigger than the difference between XviD and DivX right now, otherwise you could throw the whole standard in the trashcan right now IMO.
stephanV is offline   Reply With Quote
Old 13th February 2005, 11:10   #20  |  Link
peteag
Registered User
 
Join Date: Jul 2004
Posts: 170
They said, that H.264 codecs are comparable to a DivX 2.0-status (when looking at 5.2.1) and must be improved by the years, they're not exhausted. But how's the ASP-standard exhausted? Is it realy full-featured like the reference-code or are there even tons of features to be implemented?
peteag is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 14:41.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.