Fourth Annual H.264 video codecs comparison published!

[DmitriyV2]

Comparison was published now!

Comparison page is: http://www.compression.ru/video/codec_comparison/mpeg-4_avc_h264_2007_en.html
Direct link to PDF:
http://www.compression.ru/video/codec_comparison/pdf/msu_mpeg_4_avc_h264_codec_comparison_2007_eng.pdf

Enjoy! :)

D.
------------------------------------------------------------------
Please forward this information to codec developers!

===========================================================================
CALL FOR MPEG4-AVC/H.264 CODECS
Fourth Annual H.264 video codec comparison
For practical researchers and developers in the field of high-end video compression
===========================================================================

Scope of Test
-------------

* Encoding time, speed/quality analysis
* Objective quality measurements (PSNR, SSIM, Average Advantage, etc.)
* Analysis of averaged objective results
* Leaders in different areas
* Decoder tests (including cross-compatibility tests)
* Special analysis of codec parts

Important Dates
---------------

August, 26 – Deadline for receipt of a H.264 codec with required presets
September, 10 – Deadline for settling technical problems with codec’s functioning
November, 7 - Draft of the report that will be sent to all participants
November, 14 - Deadline for reception of comments on the draft
December, 15 - Comparison report release

Enhancements in comparison to Previous H.264/AVC Comparison
-----------------------------------------------------------

* Cross-compatibility test
* New sequences
* New type of special analysis for codecs
* Using natural sequences' special modifications
* Using synthetic sequences
* Separate analysis of codecs main subsystems

Developer Deliverables
----------------------

The following deliverables should be provided by each developer:
* Codec files (CLI executable file is preferable)
* Short description of codec parameters
* Codec's presets with mentioning what H.264/AVC profiles are used

The full text of Call for Codecs is available at
http://www.compression.ru/video/codec_comparison/call_for_codecs_07.html

Variants of Participation
-------------------------

There are two variants for companies to participate in our comparison:

1. Participation for free. All results of your codec will be published,
except special cases of measurements problems due to codec
instability.

2. Private participation. A special report will be prepared only for your
company. This report contains:
* Your codec results and all material from the free version
* Special additional analysis of your codec

If you are interested in the private participation, please contact us
for details.

Useful Links
------------

* Third Annual MSU MPEG-4 AVC/H.264 Video Codec Comparison
http://compression.ru/video/codec_comparison/mpeg-4_avc_h264_2006_en.html
* MSU Lossless Video Codecs Comparison'2007
http://compression.ru/video/codec_comparison/lossless_codecs_2007_en.html
* Subjective Comparison of Modern Video Codecs
http://compression.ru/video/codec_comparison/subjective_codecs_comparison_en.html

[Dark Shikari]

"Minimum 1 fps for "High Quality" preset"

What computer is used for the encoding? If its pretty fast, it sounds like I should add a patch to x264 to add a Hadamard-based rather than SAD-based motion search, since that's the only way you'll get close to 1 fps without doing something stupid like using 16 reference frames, which is pointless for quality anyways. Might as well get the absolute most out of the encoding possible.

[akupenguin]

Same CPU as last year: Athlon64 3000+ in 32bit mode.
x264 has no problem getting down to 1.5fps at 720p, but that does leave a little room for improvement.

[Dark Shikari]

Same CPU as last year: Athlon64 3000+ in 32bit mode.
x264 has no problem getting down to 1.5fps at 720p, but that does leave a little room for improvement.
Heh, that means one can add 50% more instructions without going over the limit. Not bad...

[DmitriyV2]

Heh, that means one can add 50% more instructions without going over the limit. Not bad...
Sure. We received presets for x264 now.

Also this year we change MPEG-4 reference codec from DivX to XviD. We have problems with official answer from DivX and presets for all other codecs was selected by developers, so we prefer to use codec, with presets provided by developers and developers comparison verification (we sent to developers raw measured data for sure). Xvid preset's tuning for comparison (time equalization) is in progress now.

Another interesting news - AMD H.264 will be a participant. Also we will add JM (by many requests from developers, that use JM as starting and reference point for commercial codecs).

[DmitriyV2]

Our fourth comparison is close to publication.

In this report we changed MPEG-4 ASP codec (used as "reference point for new H.264") from DivX to Xvid (due to lack of communication with DivX guys). And frankly speaking results of Xvid is very good, but also during our extensive tests we found some problems, that slightly decrease sometimes quality of Xvid (we will provide all info to developers). So next year results of Xvid can be better.

Other details coming soon.

[Dark Shikari]

Our fourth comparison is close to publication. Heh, so unfortunate that all the latest improvements to x264 came after the testing date.

So I guess we'll have to take these results with a grain of salt ;)

[Sagekilla]

Heh, so unfortunate that all the latest improvements to x264 came after the testing date.

So I guess we'll have to take these results with a grain of salt ;)

A shame indeed.. I'm still waiting for someone to compile a version of x264 with all these goodies soon :) Unless I somehow figure out how to compile x264 (Which looks like quite a bit far off from happening at this point)

[DmitriyV2]

Heh, so unfortunate that all the latest improvements to x264 came after the testing date.
Codec developers of all codecs regulary tell us similar things every years, that's why we organize comparison every year at the same time :).

Also it's very gaily to see such good progress every year.

[fields_g]

Well.... When we see x264 sweep it all :D (hopefully not too optimistic), we can just smile knowing the large improvements that have happened since ~r670. Just as long as the competitors haven't made similar advances. Don't worry... we'll always have next year! I'm looking forward to this report!

[patrick_]

I think these comparisons are interesting, but to me, it would be way more interesting to see a comparison using max compression settings.
That is x264 is better than MainConcept because it allows higher compression settings. Maybe I'm different from everybody else, but I don't care about the time it takes to compress a video, I only care about quality and filesize. BTW I also don't care about seek-times or CPU requirements.
Probable x264 is also better than VC-1 because of the flexible configuration. Maybe I haven't looked enough, but it seems that about no program allows a full configuration of codecs.
-----
I use StaxRip to convert with x264. But I wouldn't mind using anything else (I use Vegas to add effects).

[Sagekilla]

I think these comparisons are interesting, but to me, it would be way more interesting to see a comparison using max compression settings.
That is x264 is better than MainConcept because it allows higher compression settings. Maybe I'm different from everybody else, but I don't care about the time it takes to compress a video, I only care about quality and filesize. BTW I also don't care about seek-times or CPU requirements.
Probable x264 is also better than VC-1 because of the flexible configuration. Maybe I haven't looked enough, but it seems that about no program allows a full configuration of codecs.
-----
I use StaxRip to convert with x264. But I wouldn't mind using anything else (I use Vegas to add effects).

The thing with that is.. While yes, it does give an idea of how well two codecs compress using all their tricks, it doesn't represent what most people will actually do, since max compression can take maybe 20 to 30 hours instead of only 6, while only offering a 2 - 3% file size decrease. Those aren't exact numbers, I'm just saying that pumping compression to max really slows thing to a halt for a very minimal quality increase.

[patrick_]

Sure it's not the way most people compress at this time, but while processing power is increasing, people will use a more advanced configuration. So it will give an impression about the future of the codecs.

[DmitriyV2]

Probable x264 is also better than VC-1 because of the flexible configuration.
Initially Microsoft with VC-1 going to participate in this year comparison and we have long discussion. But next they changed one's mind. Maybe they done some internal tests.

[Sharktooth]

They dont like unbiased opinions...

1st rule of marketing: your product is always the best. dont let anyone prove it isnt.

[DmitriyV2]

They dont like unbiased opinions...
1st rule of marketing: your product is always the best. dont let anyone prove it isnt.
This is true. But I want draw attention only to one thing - some guys in MS try to "think different". This is not so easy, I suppose.

[bond]

you can see it that way: you dont need to waste time running the comparison also on vc-1 to realise that vc-1 is worse.
m$ told you that it is worse by not taking part in the comparison ;)

[Sulik]

It's almost December 15th... Any idea when we can expect the results of the 2007 H.264 codec comparison ?

It would be good to also include low-res baseline profile results in the next comparison (baseline is still quite useful for iPod and other portable devices)

[akupenguin]

Given that the drafts were mailed on Dec 5 and not the planned Nov 7, I don't have much hope for the final publication being on time.

[IgorC]

No even one word about participants until final publication. Is it any secret information? KGB style.
"The results will be lately". Too much vodka, guys.

[DmitriyV2]

It's almost December 15th... Any idea when we can expect the results of the 2007 H.264 codec comparison?
Now final fixes and remarks are included taking into consideration codec developers feedback. Sorry for delay!

It would be good to also include low-res baseline profile results in the next comparison (baseline is still quite useful for iPod and other portable devices)
Hm. Good idea!

[Nil Einne]

Microsoft don't want to participate with VC1 but how about Xiph.org with Theora? With it reaching v1.0 soon if it's not in this years hopefully it'll be in next years.

Edit: Although then again, after reading comments for people who seem to know what they're talking about it seems Theora can't even really compared to XviD yet so I guess there's no point

[DmitriyV2]

Microsoft don't want to participate with VC1 but how about Xiph.org with Theora? With it reaching v1.0 soon if it's not in this years hopefully it'll be in next years.

Edit: Although then again, after reading comments for people who seem to know what they're talking about it seems Theora can't even really compared to XviD yet so I guess there's no point
Ok! When they will be ready, we will be glad to include them. We invite second open source codec to comparison this year and there is no problem to add third. Of course this is not H.264 and not so resonable reference point, but we can measure it carefully in appendix, no problem.

[omf]

Dmitriy, when do you expect the results to be available?

[fatso485]

when do expect to see the results. is it the 15th yet.


also is there another recent codec comparison that i can look into

[CruNcher]

Ok! When they will be ready, we will be glad to include them. We invite second open source codec to comparison this year and there is no problem to add third. Of course this is not H.264 and not so resonable reference point, but we can measure it carefully in appendix, no problem.

Is this second Open Source Codec XviD 2.0 AVC :) ?

[pc2099]

Is the report out yet? Also given mainconcept was the winner in 2006 and is now owned by DivX...will DivX AVC be the leader in quality/speed in 2008?

Interesting times indeed

[Alex-Kid]

Date: 2 January, 2008 (btw, Happy New Year to everyone :))

Fourth AVC Video Codec Comparison still missing.

Does someone know something about Ateme codec for this comparison? I missed it last year comparison, because it was the winner in 2005.

Keeping in touch.

Saludos, by ALEX-KID

[foxyshadis]

Hey, come on, no need to pile onto the thread, one query is plenty.

[CruNcher]

Date: 2 January, 2008 (btw, Happy New Year to everyone :))

Fourth AVC Video Codec Comparison still missing.

Does someone know something about Ateme codec for this comparison? I missed it last year comparison, because it was the winner in 2005.

Keeping in touch.

Saludos, by ALEX-KID
A Happy New Year to you and everyone else too

I think Ateme is not going to take place in the Test again, as the results of it seem not to be important for them their Market @ the moment is another one (Broadcast), now with the buyout of Mainconcept to DivX Inc im not sure if that stays this way ;)
It does really nice especialy with many problems that other Codecs still suffer from very special HVS stuff that rarely occours in pre rendering, but more in Live Scenarios ofcourse it can also happen in non Live low bitrate conditions and they doing really some magical HVS tricks in such situations it's Amazing.

[DmitriyV2]

Dmitriy, when do you expect the results to be available?
Now preliminary final report is ready and we plan to publish it 14 January.

Sorry for delay - I did not want to announce exact date without exact information.

[omf]

Now preliminary final report is ready and we plan to publish it 14 January.

Sorry for delay - I did not want to announce exact date without exact information.

No problem, and thanks for the update.

[Sulik]

Dmitriy: Any updates ?

[DmitriyV2]

Specially for you :)

Please find results is here:
http://www.compression.ru/video/codec_comparison/mpeg-4_avc_h264_2007_en.html
Direct link to PDF:
http://www.compression.ru/video/codec_comparison/pdf/msu_mpeg_4_avc_h264_codec_comparison_2007_eng.pdf

Thanks for patience!

[Dark Shikari]

Quite an old version of x264 used there... :p

[Sulik]

Excellent. Too bad there isn't more RD curves or image comparisons this year (I guess we have to pay for it now - too high for my budget :)

If I had some suggestions for next year, it would be the following:

1. Use bitrates proportional to the resolutions (measured in bits per pixel). Realistically, regardless of the encoder used, if the PSNR is in the mid-to-low 30's, the quality is crap.

2. Include a separate section for Baseline profile (CAVLC, no B-frames), since this is a very commonly used scenario for H.264 encoding (iPod & other devices) -> this might be a bit more fair to encoders like AMD, since it's difficult to tell if the differences come mostly from Baseline vs High Profile or from encoder implementation (Or at least include a standard Baseline reference such as JM in constant quant mode).

3. (Related to #1) Go a bit higher on the bitrate scale, instead of only looking at the section of RD curves where the encoders start breaking down (though this might reduce SNR differences between encoders).

4. Include a wider range of content complexity, possibly using standard sequences (sequences such as Mobile & Calendar comes to mind)

5. I'd rather seem more RD graphs and screenshots for different sequences than basically the same data displayed in 4 different forms

[DeathTheSheep]

Only 9 revisions behind my own. :)

[CruNcher]

Reading the results i can be quiet sure now that my Visual Interpretation of Atemes Codec is not wrong and it seems still to be the most Advanced :)
Im not makeing this statement from a pure Scientific Point of View, more from constantly analysing the Look and Feel of X264, Mainconcept and Ateme over the years.
This results now confirm that X264 and Mainconcept are almost identical (i can confirm that is the case also Look and Feel wise) compared to Atemes Implementation wich imho is the best i saw so far for real footage. Also it's sad that XviD 2.0 AVC was not in the test @ all, im suprised that it's still not here yet :(
Who are those Artemis guys and what the Hell they are deoptimizing x264 for (looks like somekind of decoding complexity reduction optimization to me) that I frame "pulse" looks strange :D

[IgorC]

I'm totally agree about Ateme encoder.
x264 and MC are quite close but there is lack of psy optimizations. And Ateme has it.


DmitriyV2 Thanks for result.
For me AMD and Intel encoders were totally out of interest.
Maybe just pretest and then discard them.
Instead of it give a try to Apple and Ateme (or Nero). imo

[Dark Shikari]

I'm totally agree about Ateme encoder.
x264 and MC are quite close but there is lack of psy optimizations. And Ateme has it.Try the new AQ... I don't think a 22% bitrate-adjusted SSIM boost (with appropriately correlated increased visual quality) is something to scoff at ;)

[DeathTheSheep]

I had a feeling you'd eventually come in and say that, DS. I was biding my time with bated breath. :)

(Now if only the that 3% absolute quality boost in me-prepass can live again... A quality gain roughly equivalent to 1/3 of that derived from using CABAC is contained within it, after all.)

[CruNcher]

Jep Dark indeed your AQ boosts the Percepted Visual Quality tremendously for some hard scenes (no doub't in that) and i have to reevaluate with it now, but overal the look and feel based on the momentarily used deadzone system introduced by Alex stays the same (for most other scenes) it seems :)

[shon3i]

@CruNcher, IgorC, i am agree with you about ateme, realy can produce best quailty for given bitrate and comparing to MC/Elecard and x264 , always beat by hair. But sometimes from my testing Elecard's psy optimisations can done better job, than ateme. Complexity algo with negative values can produce magical to some footage. I think elecard converter studio 2.0 now have all options that one AVC encoder need for very quality encode and aslo fast, and most important to me is compactibility with hardware acceleration decoders to i can watch, only miss CQM :(

[Wilbert]

I'm a bit baffled by the conclusions. Statements like
The main reason of AMD encoder low quality is very high speed of the encoder.
are clearly false. The low quality is not the consequence of the 'very high speed' (more precisely the 'high speed preset' which was used in the encoding process). Both are the consequence of the fact that only features of the codec are used in the encoding process (or implemented from the standard ) which are computationally fast, but don't result in good quality.

I don't know some of the codecs, but reading your article it seems that all three can be the reason of the bad quality, depend on the codec under consideration.

Also, what do you mean with the following statement:
The XviD codec demonstrates difficulties with bitrate handling algorithms, so does the AMD encoder as well.


Last question. Is Artemis x264 also GPL (since you claim it is based on x264)?

[Ranguvar]

By the Xvid comment, I think he was referring to the 2-pass rate control in it.

[DmitriyV2]

I started to write FAQ for this comparison - see draft below:

----------------------------------------------------
Q. What was main goal of the comparison?

A. main idea of this annual comparison - official commercial companies
involving and so increasing number of participants. We are already
involved Intel, AMD, ATI, Apple (partially) and smaller Fraunhofer,
Elecard, Ateme, VSS, MainConcept, Sorenson, Artemis etc.

Q. Why official? You can get codec from a developer's site or (if
unavailable) from P2P networks and compare them successfully!

A. This is not true. For example, key feature of our comparisons is
comparison in several application areas (videoconferences, SD,
HDTV). Different areas require different codec settings. The
simplest strategy -- use of default settings -- definitely works,
but it is as much definitely nonoptimal (in terms of
quality-bitrate balance). So we received more optimal settings from
developers to test codec with higher accuracy. Also every year we
change sequences in an unpredictable way to prevent tuning to
specific sequences by developers.

Another problem is speed. Default settings of one codec can made it
10 time slower than another one and all your quality measurements
will not be so correct. Of course we can select suitable parameters
with close speed, but everybody can say "you select wrong
parameters for our codec, so you received wrong results", that's
why we do not use this approach in the annual H.264 comparison. Our
current solution is not sophisticated: We just asked companies for
optimal settings (as they think of them) for the predefined
application area and predefined time limits. The first step of
codec qualification to the comparison is time measurement: we
report time, measured on our system to developers with comments
like "you have 50% overtime, please make your preset faster", or
"your time reserve is still 40%, you can make your preset slower,
but with better quality/rate results".

Another advantage of official contacts: we typically receive the
newest internal versions, that are planned to be published not very
soon (it was up to half a year for one of our participants).


Q. Fine. But why you have so long timeout between starting till
publication?

Several reasons:

1. As we told previously - we have slow speed acceptance procedure,
that require correspondence with companies.

2. Next - if it's possible - we try to get companies chance to fix
clear bug. We receive fresh research version, so some bugs inside
is possible (especially this was actual for second annual
comparison). This can be very simple problem like "unlucky
build" with fast fixing. So if we have time reserve, we stop
measurement, report bug and get company chance to fix it. Of
course we wait very few time (2-3 days) with clear final
deadline, but this procedure also add additional delay.

3. Also we have measurements approval procedure. After we done all
measurements and after internal checking we provide measured
results of company's codec and test sequences to correspondent
companies, so companies are able to check our measurements with
there codec. We published our measurement tool (MSU Video Quality
Metric) and provide our measurement results in it's format,
so all results can be verified.

4. Next step is reports preparation (short, long versions and
versions for private comparison). Report size can be up to 200
pages, we prepare several reports, so this is also not so fast
step.

5. And final step is reports verification and reviewing. We provide
report draft to all participants for verification. They report
defect and commonly good suggestions. If we think, that some
suggestion is very important we can add appendix to comparison
with additional measurements or some clarifications. This also
require time. And final step is independent reviewing by very
good independent experts in H.264 codecs.

Q. Ok. Situation with time is clear. Is it possible for a company to
remove some results if it ask for this?

A. Now we avoid such situation by using a so-called private participation.

The motivation is pretty simple: at the start, without knowing its
real position against competitors, a developer did want to
participate in the public comparison. But after receiving results,
if this is not the first place or so, they did not want to see
these results published :). So, if we allow to remove, there will
be only one codec in the comparison. Maybe two. :) (somebody and
x264). So we restrict such removing now, but allow private
participation.

Some companies did not want to participate due to not so good
current results which they know beforehand about. In private
participation a company pay us and receive its personal report.
This company codec is not included in the public comparison at all.
So, if company pay us, we prevent any conflicts of interests. A
company can understand its place and does further works on codec
improvement, but do not get its current results published and
well-known (the annual comparisons are very popular - we have more
than 200000 total downloads now).

Q. And the main question: what codec is the best?

A. We really have maybe the maximum information about the best codecs in
the world and we really do not know. :) At first, please specify
usage area, speed and other requirements, etc, etc, etc... We can
answer what codec is better for some common usage samples.

Our another objective is to help codec developers to make there
codecs better and better and better. Commonly developers achieve
some really good results and believe that this is the absolute
limit, but we help them to see new limits.

---------------------------------------------------------

And again - any updates, fixes and suggestions are welcome!

Also I'll answer all your questions soon (thanks for good ideas!)

[CruNcher]

You should maybe better write.
"So, if companies pay us for the private participation, we avoid any conflicts of interests."
Else in context to the rest of the text it sounds a little corrupt, like hey if X264 devs would pay us they would stay ontop ;)
Im not sure how much effect this test actually had now for the Quality of X264 did you found any severe bugs in the progress and reported problems to the devs, for example the problems with that LOTR Sample seem to be a Quality problem?

[audyovydeo]

And again - any updates, fixes and suggestions are welcome!


I know I'l going to get tons of flak for this but so be it.
I think you got your report distribution model wrong.
My numbers are guestimates of course :

2008 price customers revenue
full_report 600$ 200 120000
half_report 0 200000 0

2009 price customers revenue
full_report 2$ 200000 400000

personally I'd spend 2$ and read the full report rather than have a free but wishy-washy one. This year's report was much anticipated but, if you noticed, it didn't spark a rain of forum messages. And I'm not sure you get thousands of companies buying it at full price.

Oh I have another question that I know you will not answer : did Apple decline to take the test this year, or was it excluded because of of its lack of High profile support ?

cheers
audyovydeo

[bob0r]

CruNcher why would x264 ever pay? LOOL
x264 is the only encoder widely used, and most "groups" use even such horrible settings, quality is something they dont care about anyway, sadly but true :/

[Dark Shikari]

CruNcher why would x264 ever pay? LOOL
x264 is the only encoder widely used, and most "groups" use even such horrible settings, quality is something they dont care about anyway, sadly but true :/Oh how true (http://i32.tinypic.com/f258k4.jpg).

[DmitriyV2]

You should maybe better write.
"So, if companies pay us for the private participation, we avoid any conflicts of interests."
Ok, you right. More correct - "if company pay us, we exclude this codec from publically available reports".

Im not sure how much effect this test actually had now for the Quality of X264 did you found any severe bugs in the progress and reported problems to the devs, for example the problems with that LOTR Sample seem to be a Quality problem?
If codec did not have first place - this is not bug in our terminology. But maybe this is an effect of sevelal small bugs, of course.

[DmitriyV2]

I know I'l going to get tons of flak for this but so be it. I think you got your report distribution model wrong.
2008 price customers revenue
full_report 600$ 200 120000
You are very optimistic. :) This is good strain, but...

By our experience only big companies interested full report now and even other codec developers (not participants) are satisfied with short version.

We are thinking how to make this report more useful for wide range of professionals, but looks like there exist not so big number of H.264 professionals in general.

Maybe report for H.264 funs is reasonable, but it's necessary to prepare it with another rules.

Oh I have another question that I know you will not answer : did Apple decline to take the test this year, or was it excluded because of of its lack of High profile support?
Looks like marketing guys in Apple was against partisication.

[DmitriyV2]

x264 is the only encoder widely used, and most "groups" use even such horrible settings, quality is something they dont care about anyway, sadly but true :/
That's right.

We are prepared freely available sponsored report with x264 settings efficiency comparison, that can be useful with settings selection (or to show different settings efficiency):
http://www.yuvsoft.com/pdf/x264_parameters_comparison.pdf

Maybe we will prepare something like "Case study" on x264 settings will published by MSU Lab later.

[CruNcher]

The problem is X264 is constantly changing (also all other codecs) and so such a Report if it comes out could be very outdated allready same for every comparison you do actually, in this field :). So maybe you could only do this LOTR sample test again and see if it still showing of this problems there :)
And could you guys please post all the H.264 tools used for the different Tests of all Encoders :)

[Dark Shikari]

A little comparison of my own, x264 vs Mainconcept, same bitrate, same source (parkrun.yuv):

x264 (http://www.mediafire.com/?4vsgypaltc9)

Mainconcept (http://www.mediafire.com/?1vf31cbm4t9)

;)

[Sagittaire]

Where is the source parkrun.yuv ... ?

[Dark Shikari]

Where is the source parkrun.yuv ... ?
ftp://ftp.ldv.e-technik.tu-muenchen.de/pub/test_sequences/

[Sagittaire]

Well in fact Elecard/mainconcept encoder is able to produce really better result than your encoding ...
http://jfl1974.free.fr/upload/Parkrun_Elecard.mp4

... and for me better than your x264 encoding for this source.

[shon3i]

x264 is the only encoder widely used, and most "groups" use even such horrible settings, quality is something they dont care about anyway, sadly but true :/Yes, but there is one reason for that. Nobody can find "retail" versions of mainconcept/elecard/ateme encoders :)

Most groups now use CCE in their MPEG2 projects.

... and for me better than your x264 encoding for this source.

For my eyes too.

[nm]

... and for me better than your x264 encoding for this source.
Better by SSIM or subjectively? I prefer the x264 encode visually. Branches and background behind the panning trees is preserved better while other parts of the frames are as good as in your Elecard encode.

[Dark Shikari]

Better by SSIM or subjectively? I prefer the x264 encode visually. Branches and background behind the panning trees is preserved better while other parts of the frames are as good as in your Elecard encode.Agreed... comparing frames, x264 looks much better overall than Elecard. Elecard loses a whole lot of detail that x264 keeps. I did a quick blind test on some random frames and x264 looked considerably better in all of them.

[Sagittaire]

Agreed... comparing frames, x264 looks much better overall than Elecard. Elecard loses a whole lot of detail that x264 keeps. I did a quick blind test on some random frames and x264 looked considerably better in all of them.

If you want HVS comparison then the most important part for this source is the "Runing Man" (Eyes look man and not branches). Elecard is by far better in this case. But if you want better branch and background it's simple to set AQ with Elecard (I use "complexity AQ": higher quant for complexe texture).

[Dark Shikari]

If you want HVS comparison then the most important part for this source is the "Runing Man" (Eyes look man and not branches). Elecard is by far better in this case. But if you want better branch and background it's simple to set AQ with Elecard (I use "complexity AQ": higher quant for complexe texture).This clip benefits greatly from complexity masking, so any other AQ approach would be atrocious.

[CruNcher]

Wait till someone posts parkrun_ateme ;)

[shon3i]

@Dark Shikari, can you post you x264 settings please?

@CruNcher, i will :)

[Dark Shikari]

@Dark Shikari, can you post you x264 settings please?

@CruNcher, i will :)

x264 - core 57 svn-721M - H.264/MPEG-4 AVC codec - Copyleft 2005 - http://www.vi
deolan.org/x264.html - options: cabac=1 ref=16 deblock=1:0:0 analyse=0x3:0x133 m
e=esa subme=7 brdo=1 mixed_ref=1 me_range=24 chroma_me=1 trellis=2 8x8dct=1 cqm=
0 deadzone=21,11 chroma_qp_offset=0 threads=3 nr=0 decimate=1 mbaff=0 bframes=16
b_pyramid=1 b_adapt=1 b_bias=0 direct=3 wpredb=1 bime=1 keyint=250 keyint_min=2
5 scenecut=40(pre) rc=crf crf=21.0 rceq='blurCplx^(1-qComp)' qcomp=0.60 qpmin=10
qpmax=51 qpstep=4 ip_ratio=1.20 pb_ratio=1.50 aq=1:1.0:15.0

Note the ESA range 24 is basically useless, I just used it because I had the time. You can go UMH with basically zero difference on this clip, which puts x264's speed at considerably faster than Mainconcept, at least on my computer.

[shon3i]

What is ratecontrol?

[Dark Shikari]

What is ratecontrol?Ratecontrol is the process of deciding what quantizer to use for which frames--usually with a specific goal, such as a particular bitrate, or a particular quality level.

[shon3i]

Did you joke with me? :)

I mean did you use CRF or multi pass encoding?

[Dark Shikari]

Did you joke with me? :)

I mean did you use CRF or multi pass encoding?Stupid question gets stupid answer ;)

rc=crf crf=21.0

[shon3i]

Well i really have no time to decrypt some x264 debug info or metadata. That's why i told you to post settings in command line like usual ;)

[Sagittaire]

Note the ESA range 24 is basically useless, I just used it because I had the time. You can go UMH with basically zero difference on this clip, which puts x264's speed at considerably faster than Mainconcept, at least on my computer.

test with c2d at 2.7 Ghz

For Mainconcept at "quality max"

MainConcept H.264/AVC encoder (build 2.1.6907 at 2006/10/09)
Copyright (c) 2006 MainConcept AG

THIS SOFTWARE IS FOR EVALUATION PURPOSES ONLY!

[time: 0:02:28] [left: 0:00:00] [speed: 3.4 fps] [queue: 0.000 ms]]

Summary information:
Number of coded frames 504
Total encoding time 148906 ms
Average time per frame 295.448 ms
Average speed achieved 3.4 fps
Average bitrate 11396.88 kbit/sec @ 50.00 Hz

Overall PSNR (Y) 30.244 dB
Overall PSNR (U) 38.107 dB
Overall PSNR (V) 39.952 dB
Overall PSNR (A) 31.720 dB


For x264 at "quality max"

D:\Mes dossiers\Codec\x264>x264.exe --fps 50 --threads 3 --thread-input --keyint 250 --min-keyint 1
--bframe 3 --bime --weightb --ref 5 --mixed-refs --direct auto --deblock -1:-1 --crf 28 --stats "x26
4_stat.log" --qcomp 0.75 --partitions "all" --8x8dct --me "umh" --subme 7 --trellis 2 --no-fast-pski
p --no-dct-decimate --sar 1:1 --progress -o 1080p_3.mp4 1280x720.yuv
x264 [info]: file name gives 1280x720
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX MMXEXT SSE SSE2 Cache64
mp4 [info]: initial delay 1000 (scale 50000)
x264 [info]: slice I:3 Avg QP:27.67 size: 55648 PSNR Mean Y:38.93 U:40.67 V:44.93 Avg:39.73 Gl
obal:38.16
x264 [info]: slice P:249 Avg QP:31.49 size: 47200 PSNR Mean Y:31.28 U:37.75 V:39.68 Avg:32.65 Gl
obal:32.53
x264 [info]: slice B:252 Avg QP:33.53 size: 8702 PSNR Mean Y:30.71 U:37.79 V:39.81 Avg:32.12 Gl
obal:31.84
x264 [info]: mb I I16..4: 7.2% 77.5% 15.3%
x264 [info]: mb P I16..4: 0.1% 0.5% 0.3% P16..4: 60.9% 17.2% 9.7% 0.9% 0.7% skip: 9.8%
x264 [info]: mb B I16..4: 0.0% 0.0% 0.0% B16..8: 7.8% 0.2% 1.6% direct:10.8% skip:79.5%
x264 [info]: 8x8 transform intra:67.9% inter:47.8%
x264 [info]: direct mvs spatial:99.2% temporal:0.8%
x264 [info]: ref P 82.4% 7.4% 5.7% 2.2% 2.2%
x264 [info]: ref B 87.8% 5.1% 4.2% 2.9%
x264 [info]: SSIM Mean Y:0.9122079
x264 [info]: PSNR Mean Y:31.038 U:37.792 V:39.779 Avg:32.424 Global:32.188 kb/s:11200.59

encoded 504 frames, 3.15 fps, 11198.83 kb/s


And my Elecard CLI encoder is very old build. At this time Elecard/Mainconcept core is by far faster.

[Sagekilla]

I'd hardly call that "quality max" for x264. You can bump up bframes to 16 and get some more quality out of it at little speed loss (should gain speed actually). Likewise, you can pump up reference frames to 16 at the cost of an extreme slowdown.

[Sagittaire]

I'd hardly call that "quality max" for x264. You can bump up bframes to 16 and get some more quality out of it at little speed loss (should gain speed actually). Likewise, you can pump up reference frames to 16 at the cost of an extreme slowdown.

Well bframe at 16 are completely useless setting simply because in practice x264 never use 16 bframe ... and if x264 use 16 frame it's for very particular case (consecutive black frame for example) and in this case the quality gain is very small. I use for x264 and Elecard comparable setting here: RDO at max, same ref number, same bframe number, wpred ... etc etc etc. For the same quality (aka PSNR/SSIM) Mainconcept is really faster than x264.

[Dark Shikari]

For the same quality (aka PSNR/SSIM) Mainconcept is really faster than x264.This is a joke, right?

In your comparison:

Mainconcept OPSNR: 30.244 dB

x264 OPSNR :32.188

[Sagittaire]

This is a joke, right?

In your comparison:

Mainconcept OPSNR: 30.244 dB

x264 OPSNR :32.188

... ?


Overall PSNR (Y) 30.244 dB
Overall PSNR (U) 38.107 dB
Overall PSNR (V) 39.952 dB
Overall PSNR (A) 31.720 dB


30.244 dB is overall Y. 31.720 dB is overall YUV. Without AQ Elecard produce 32.317 dB at quality max for the same speed.


Overall PSNR (Y) 30.878 dB
Overall PSNR (U) 38.195 dB
Overall PSNR (V) 40.003 dB
Overall PSNR (A) 32.317 dB

[Dark Shikari]

... ?


Overall PSNR (Y) 30.244 dB
Overall PSNR (U) 38.107 dB
Overall PSNR (V) 39.952 dB
Overall PSNR (A) 31.720 dB


30.244 dB is overall Y. 31.720 dB is overall YUV. Without AQ Elecard produce 32.317 dB at quality max for the same speed.


Overall PSNR (Y) 30.878 dB
Overall PSNR (U) 38.195 dB
Overall PSNR (V) 40.003 dB
Overall PSNR (A) 32.317 dB
Ah yes, you are correct. Still, that's a difference of 0.6db, which is gigantic (roughly equivalent to 12% bitrate). You can't say "they have roughly the same PSNR" if there's a gap that large.

If one says "I'm willing to lose 0.6db on the part of x264", one can easily make it 50% faster by changing the encoding settings.

[gwaitsi]

i am curious why nero recode was never included?

[Manao]

Dark Shikari : Without AQ Elecard produce 32.317 dB at quality max for the same speed.

[Sagittaire]

And here more complete result:

Nero/Ateme:
14081 Ko and 32.52 dB

Elecard/Mainconcept:
14087 Ko and 32.34 dB

x264 build 732:
13978 Ko and 32.26 dB

3 bframes, 5 ref, wpred, 8x8dct
mainconcept/elecard at insane mode
Nero/ateme at insane mode
x264 at insane mode (build 732)

x264.exe --fps 50 --threads 3 --thread-input --keyint 250 --min-keyint 1 --bframe 3 --b-rdo --bime --weightb --ref 5 --mixed-refs --direct auto --deblock -1:-1 --crf 27.9 --stats "x264_stat.log" --qcomp 0.75 --partitions "all" --8x8dct --me "tesa" --subme 7 --trellis 2 --no-fast-pskip --no-dct-decimate --sar 1:1 --progress -o 1280x720.mp4 1280x720.yuv

[CruNcher]

Saggittaire from wich eavc core are those results 1.6 ?

[MySchizoBuddy]

So next year we will see Divx back again , since they bought MainConcept and also Adobe Flash, since they are now licensing the MainConcept core codec :)

[Sagekilla]

Well bframe at 16 are completely useless setting simply because in practice x264 never use 16 bframe ... and if x264 use 16 frame it's for very particular case (consecutive black frame for example) and in this case the quality gain is very small. I use for x264 and Elecard comparable setting here: RDO at max, same ref number, same bframe number, wpred ... etc etc etc. For the same quality (aka PSNR/SSIM) Mainconcept is really faster than x264.

Yes, in practice x264 almost NEVER uses 16 b-frames. But simply not allowing the encoder the leverage to choose as much B-frames seems ridiculous -- If the encoder uses more B-frames, it usually never hurts quality (improves it), lowers bitrate and does increase encoding speed (very slightly) since B-frames are easier to encode. It might not be a massive increase, but I see no reason in not using something that has only benefits and no drawbacks.

[Dark Shikari]

Also, an interesting note: x264's B-frame decision is currently not very good, especially for multiple B-frames.

For example, you'll notice a drastic improvement in x264 OPSNR and SSIM (both with and without AQ) on parkrun if you use --no-b-adapt --bframes 3 --b-pyramid :cool:

[Sagekilla]

Another thing, why --no-dct-decimate? I've rarely seen that increase quality and in most cases it just blew up file size more than necessary.

@Dark Shikari: Any plans up for improving B-Frame decision yet? I'd love to take a look inside and try fiddling around to see if I can (probably not) do anything, but I suffer from epic failure at compiling.

[kosmonaut]

So next year we will see Divx back again , since they bought MainConcept and also Adobe Flash, since they are now licensing the MainConcept core codec :)

That would be a reasonable assumption. :cool:

[Dark Shikari]

@Dark Shikari: Any plans up for improving B-Frame decision yet? I'd love to take a look inside and try fiddling around to see if I can (probably not) do anything, but I suffer from epic failure at compiling.I talked with Pengvado about it last night.

There are three main problems:

1. Current B-frame decision beyond 1 B-frame is pretty bad and relies on an ugly guess of a heuristic.

2. It doesn't take into account B-pyramids when making the decision.

3. The lowres lookahead may or may not be good enough for the B-frame decision.

The solution pengvado proposed is a trellis-like algorithm for deciding future frametypes. Its actually not that hard to implement, and wouldn't be too much slower--while it would be drastically slower than the current method, lowres lookahead is orders of magnitude faster than actual encoding, since it doesn't actually do any encoding.

[Sagittaire]

Also, an interesting note: x264's B-frame decision is currently not very good, especially for multiple B-frames.

For example, you'll notice a drastic improvement in x264 OPSNR and SSIM (both with and without AQ) on parkrun if you use --no-b-adapt --bframes 3 --b-pyramid :cool:

Yes it's true. Make rdo for bframe decision is by far a better way for quality ... but not for speed. Libacodec can use rdo for bframe decision ...

[Dark Shikari]

Yes it's true. Make rdo for bframe decision is by far a better way for quality ... but not for speed. Libacodec can use rdo for bframe decision ...Don't need RDO, as I said above. x264's B-frame lookahead is extremely fast (orders of magnitude faster than regular RDRC) and can be used for vastly more intelligent B-frame decision without much speed loss.

Akupenguin has an idea for a Trellis-like algorithm for this that requires a mere 10 frame lookaheads per frame encoded.

[IgorC]

Not only x264 has problem with b-frames. For example MC jumps from 1 to 3 bframes. And almost never put 2 consecutive bframes. It's some kind of optimization indeed.

[skal]

Dmitriy: the document seems to lack details about the way you measure speed. MainConcept's encoder used to fork and/or use 2 threads at least, whether you ask it or not. What elapsed time do you measure exactly ? (real? user? sys?). And what machine were each tests conducted on, exactly?

[Nil Einne]

skal:

Same CPU as last year: Athlon64 3000+ in 32bit mode.
x264 has no problem getting down to 1.5fps at 720p, but that does leave a little room for improvement.

[Dark Shikari]

Not only x264 has problem with b-frames. For example MC jumps from 1 to 3 bframes. And almost never put 2 consecutive bframes. It's some kind of optimization indeed.That's because PbBbP is inherently quite efficient.

[IgorC]

Maybe cause of flexible and particular coding of each frame,good prediction and quantization algorithms.

[skal]

Nil Einne,
the document mentions 2 configs, but one don't know which was used for what exactly. And if MainConcept actually uses unrequested background thread, one need to know what is the timing reported, exactly.

[akupenguin]

And if MainConcept actually uses unrequested background thread, one need to know what is the timing reported, exactly.
It ran on a single core. What are you worried about, a little context switching overhead?

[skal]

It ran on a single core. What are you worried about, a little context switching overhead?

just worried that the time was actually measured with a wristwatch or a wallclock instead of just counting the number of ticks spent in the main thread. I've been fooled once like that, but maybe that's just me...

[skal]

Dmitriy: ping?