Fourth Annual H.264 video codecs comparison published!

[DmitriyV2]

You should maybe better write.
"So, if companies pay us for the private participation, we avoid any conflicts of interests."
Ok, you right. More correct - "if company pay us, we exclude this codec from publically available reports".

Im not sure how much effect this test actually had now for the Quality of X264 did you found any severe bugs in the progress and reported problems to the devs, for example the problems with that LOTR Sample seem to be a Quality problem?
If codec did not have first place - this is not bug in our terminology. But maybe this is an effect of sevelal small bugs, of course.

[DmitriyV2]

I know I'l going to get tons of flak for this but so be it. I think you got your report distribution model wrong.
2008 price customers revenue
full_report 600$ 200 120000
You are very optimistic. :) This is good strain, but...

By our experience only big companies interested full report now and even other codec developers (not participants) are satisfied with short version.

We are thinking how to make this report more useful for wide range of professionals, but looks like there exist not so big number of H.264 professionals in general.

Maybe report for H.264 funs is reasonable, but it's necessary to prepare it with another rules.

Oh I have another question that I know you will not answer : did Apple decline to take the test this year, or was it excluded because of of its lack of High profile support?
Looks like marketing guys in Apple was against partisication.

[DmitriyV2]

x264 is the only encoder widely used, and most "groups" use even such horrible settings, quality is something they dont care about anyway, sadly but true :/
That's right.

We are prepared freely available sponsored report with x264 settings efficiency comparison, that can be useful with settings selection (or to show different settings efficiency):
http://www.yuvsoft.com/pdf/x264_parameters_comparison.pdf

Maybe we will prepare something like "Case study" on x264 settings will published by MSU Lab later.

[CruNcher]

The problem is X264 is constantly changing (also all other codecs) and so such a Report if it comes out could be very outdated allready same for every comparison you do actually, in this field :). So maybe you could only do this LOTR sample test again and see if it still showing of this problems there :)
And could you guys please post all the H.264 tools used for the different Tests of all Encoders :)

[Dark Shikari]

A little comparison of my own, x264 vs Mainconcept, same bitrate, same source (parkrun.yuv):

x264 (http://www.mediafire.com/?4vsgypaltc9)

Mainconcept (http://www.mediafire.com/?1vf31cbm4t9)

;)

[Sagittaire]

Where is the source parkrun.yuv ... ?

[Dark Shikari]

Where is the source parkrun.yuv ... ?
ftp://ftp.ldv.e-technik.tu-muenchen.de/pub/test_sequences/

[Sagittaire]

Well in fact Elecard/mainconcept encoder is able to produce really better result than your encoding ...
http://jfl1974.free.fr/upload/Parkrun_Elecard.mp4

... and for me better than your x264 encoding for this source.

[shon3i]

x264 is the only encoder widely used, and most "groups" use even such horrible settings, quality is something they dont care about anyway, sadly but true :/Yes, but there is one reason for that. Nobody can find "retail" versions of mainconcept/elecard/ateme encoders :)

Most groups now use CCE in their MPEG2 projects.

... and for me better than your x264 encoding for this source.

For my eyes too.

[nm]

... and for me better than your x264 encoding for this source.
Better by SSIM or subjectively? I prefer the x264 encode visually. Branches and background behind the panning trees is preserved better while other parts of the frames are as good as in your Elecard encode.

[Dark Shikari]

Better by SSIM or subjectively? I prefer the x264 encode visually. Branches and background behind the panning trees is preserved better while other parts of the frames are as good as in your Elecard encode.Agreed... comparing frames, x264 looks much better overall than Elecard. Elecard loses a whole lot of detail that x264 keeps. I did a quick blind test on some random frames and x264 looked considerably better in all of them.

[Sagittaire]

Agreed... comparing frames, x264 looks much better overall than Elecard. Elecard loses a whole lot of detail that x264 keeps. I did a quick blind test on some random frames and x264 looked considerably better in all of them.

If you want HVS comparison then the most important part for this source is the "Runing Man" (Eyes look man and not branches). Elecard is by far better in this case. But if you want better branch and background it's simple to set AQ with Elecard (I use "complexity AQ": higher quant for complexe texture).

[Dark Shikari]

If you want HVS comparison then the most important part for this source is the "Runing Man" (Eyes look man and not branches). Elecard is by far better in this case. But if you want better branch and background it's simple to set AQ with Elecard (I use "complexity AQ": higher quant for complexe texture).This clip benefits greatly from complexity masking, so any other AQ approach would be atrocious.

[CruNcher]

Wait till someone posts parkrun_ateme ;)

[shon3i]

@Dark Shikari, can you post you x264 settings please?

@CruNcher, i will :)

[Dark Shikari]

@Dark Shikari, can you post you x264 settings please?

@CruNcher, i will :)

x264 - core 57 svn-721M - H.264/MPEG-4 AVC codec - Copyleft 2005 - http://www.vi
deolan.org/x264.html - options: cabac=1 ref=16 deblock=1:0:0 analyse=0x3:0x133 m
e=esa subme=7 brdo=1 mixed_ref=1 me_range=24 chroma_me=1 trellis=2 8x8dct=1 cqm=
0 deadzone=21,11 chroma_qp_offset=0 threads=3 nr=0 decimate=1 mbaff=0 bframes=16
b_pyramid=1 b_adapt=1 b_bias=0 direct=3 wpredb=1 bime=1 keyint=250 keyint_min=2
5 scenecut=40(pre) rc=crf crf=21.0 rceq='blurCplx^(1-qComp)' qcomp=0.60 qpmin=10
qpmax=51 qpstep=4 ip_ratio=1.20 pb_ratio=1.50 aq=1:1.0:15.0

Note the ESA range 24 is basically useless, I just used it because I had the time. You can go UMH with basically zero difference on this clip, which puts x264's speed at considerably faster than Mainconcept, at least on my computer.

[shon3i]

What is ratecontrol?

[Dark Shikari]

What is ratecontrol?Ratecontrol is the process of deciding what quantizer to use for which frames--usually with a specific goal, such as a particular bitrate, or a particular quality level.

[shon3i]

Did you joke with me? :)

I mean did you use CRF or multi pass encoding?

[Dark Shikari]

Did you joke with me? :)

I mean did you use CRF or multi pass encoding?Stupid question gets stupid answer ;)

rc=crf crf=21.0

[shon3i]

Well i really have no time to decrypt some x264 debug info or metadata. That's why i told you to post settings in command line like usual ;)

[Sagittaire]

Note the ESA range 24 is basically useless, I just used it because I had the time. You can go UMH with basically zero difference on this clip, which puts x264's speed at considerably faster than Mainconcept, at least on my computer.

test with c2d at 2.7 Ghz

For Mainconcept at "quality max"

MainConcept H.264/AVC encoder (build 2.1.6907 at 2006/10/09)
Copyright (c) 2006 MainConcept AG

THIS SOFTWARE IS FOR EVALUATION PURPOSES ONLY!

[time: 0:02:28] [left: 0:00:00] [speed: 3.4 fps] [queue: 0.000 ms]]

Summary information:
Number of coded frames 504
Total encoding time 148906 ms
Average time per frame 295.448 ms
Average speed achieved 3.4 fps
Average bitrate 11396.88 kbit/sec @ 50.00 Hz

Overall PSNR (Y) 30.244 dB
Overall PSNR (U) 38.107 dB
Overall PSNR (V) 39.952 dB
Overall PSNR (A) 31.720 dB


For x264 at "quality max"

D:\Mes dossiers\Codec\x264>x264.exe --fps 50 --threads 3 --thread-input --keyint 250 --min-keyint 1
--bframe 3 --bime --weightb --ref 5 --mixed-refs --direct auto --deblock -1:-1 --crf 28 --stats "x26
4_stat.log" --qcomp 0.75 --partitions "all" --8x8dct --me "umh" --subme 7 --trellis 2 --no-fast-pski
p --no-dct-decimate --sar 1:1 --progress -o 1080p_3.mp4 1280x720.yuv
x264 [info]: file name gives 1280x720
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX MMXEXT SSE SSE2 Cache64
mp4 [info]: initial delay 1000 (scale 50000)
x264 [info]: slice I:3 Avg QP:27.67 size: 55648 PSNR Mean Y:38.93 U:40.67 V:44.93 Avg:39.73 Gl
obal:38.16
x264 [info]: slice P:249 Avg QP:31.49 size: 47200 PSNR Mean Y:31.28 U:37.75 V:39.68 Avg:32.65 Gl
obal:32.53
x264 [info]: slice B:252 Avg QP:33.53 size: 8702 PSNR Mean Y:30.71 U:37.79 V:39.81 Avg:32.12 Gl
obal:31.84
x264 [info]: mb I I16..4: 7.2% 77.5% 15.3%
x264 [info]: mb P I16..4: 0.1% 0.5% 0.3% P16..4: 60.9% 17.2% 9.7% 0.9% 0.7% skip: 9.8%
x264 [info]: mb B I16..4: 0.0% 0.0% 0.0% B16..8: 7.8% 0.2% 1.6% direct:10.8% skip:79.5%
x264 [info]: 8x8 transform intra:67.9% inter:47.8%
x264 [info]: direct mvs spatial:99.2% temporal:0.8%
x264 [info]: ref P 82.4% 7.4% 5.7% 2.2% 2.2%
x264 [info]: ref B 87.8% 5.1% 4.2% 2.9%
x264 [info]: SSIM Mean Y:0.9122079
x264 [info]: PSNR Mean Y:31.038 U:37.792 V:39.779 Avg:32.424 Global:32.188 kb/s:11200.59

encoded 504 frames, 3.15 fps, 11198.83 kb/s


And my Elecard CLI encoder is very old build. At this time Elecard/Mainconcept core is by far faster.

[Sagekilla]

I'd hardly call that "quality max" for x264. You can bump up bframes to 16 and get some more quality out of it at little speed loss (should gain speed actually). Likewise, you can pump up reference frames to 16 at the cost of an extreme slowdown.

[Sagittaire]

I'd hardly call that "quality max" for x264. You can bump up bframes to 16 and get some more quality out of it at little speed loss (should gain speed actually). Likewise, you can pump up reference frames to 16 at the cost of an extreme slowdown.

Well bframe at 16 are completely useless setting simply because in practice x264 never use 16 bframe ... and if x264 use 16 frame it's for very particular case (consecutive black frame for example) and in this case the quality gain is very small. I use for x264 and Elecard comparable setting here: RDO at max, same ref number, same bframe number, wpred ... etc etc etc. For the same quality (aka PSNR/SSIM) Mainconcept is really faster than x264.

[Dark Shikari]

For the same quality (aka PSNR/SSIM) Mainconcept is really faster than x264.This is a joke, right?

In your comparison:

Mainconcept OPSNR: 30.244 dB

x264 OPSNR :32.188

[Sagittaire]

This is a joke, right?

In your comparison:

Mainconcept OPSNR: 30.244 dB

x264 OPSNR :32.188

... ?


Overall PSNR (Y) 30.244 dB
Overall PSNR (U) 38.107 dB
Overall PSNR (V) 39.952 dB
Overall PSNR (A) 31.720 dB


30.244 dB is overall Y. 31.720 dB is overall YUV. Without AQ Elecard produce 32.317 dB at quality max for the same speed.


Overall PSNR (Y) 30.878 dB
Overall PSNR (U) 38.195 dB
Overall PSNR (V) 40.003 dB
Overall PSNR (A) 32.317 dB

[Dark Shikari]

... ?


Overall PSNR (Y) 30.244 dB
Overall PSNR (U) 38.107 dB
Overall PSNR (V) 39.952 dB
Overall PSNR (A) 31.720 dB


30.244 dB is overall Y. 31.720 dB is overall YUV. Without AQ Elecard produce 32.317 dB at quality max for the same speed.


Overall PSNR (Y) 30.878 dB
Overall PSNR (U) 38.195 dB
Overall PSNR (V) 40.003 dB
Overall PSNR (A) 32.317 dB
Ah yes, you are correct. Still, that's a difference of 0.6db, which is gigantic (roughly equivalent to 12% bitrate). You can't say "they have roughly the same PSNR" if there's a gap that large.

If one says "I'm willing to lose 0.6db on the part of x264", one can easily make it 50% faster by changing the encoding settings.

[gwaitsi]

i am curious why nero recode was never included?

[Manao]

Dark Shikari : Without AQ Elecard produce 32.317 dB at quality max for the same speed.

[Sagittaire]

And here more complete result:

Nero/Ateme:
14081 Ko and 32.52 dB

Elecard/Mainconcept:
14087 Ko and 32.34 dB

x264 build 732:
13978 Ko and 32.26 dB

3 bframes, 5 ref, wpred, 8x8dct
mainconcept/elecard at insane mode
Nero/ateme at insane mode
x264 at insane mode (build 732)

x264.exe --fps 50 --threads 3 --thread-input --keyint 250 --min-keyint 1 --bframe 3 --b-rdo --bime --weightb --ref 5 --mixed-refs --direct auto --deblock -1:-1 --crf 27.9 --stats "x264_stat.log" --qcomp 0.75 --partitions "all" --8x8dct --me "tesa" --subme 7 --trellis 2 --no-fast-pskip --no-dct-decimate --sar 1:1 --progress -o 1280x720.mp4 1280x720.yuv

[CruNcher]

Saggittaire from wich eavc core are those results 1.6 ?

[MySchizoBuddy]

So next year we will see Divx back again , since they bought MainConcept and also Adobe Flash, since they are now licensing the MainConcept core codec :)

[Sagekilla]

Well bframe at 16 are completely useless setting simply because in practice x264 never use 16 bframe ... and if x264 use 16 frame it's for very particular case (consecutive black frame for example) and in this case the quality gain is very small. I use for x264 and Elecard comparable setting here: RDO at max, same ref number, same bframe number, wpred ... etc etc etc. For the same quality (aka PSNR/SSIM) Mainconcept is really faster than x264.

Yes, in practice x264 almost NEVER uses 16 b-frames. But simply not allowing the encoder the leverage to choose as much B-frames seems ridiculous -- If the encoder uses more B-frames, it usually never hurts quality (improves it), lowers bitrate and does increase encoding speed (very slightly) since B-frames are easier to encode. It might not be a massive increase, but I see no reason in not using something that has only benefits and no drawbacks.

[Dark Shikari]

Also, an interesting note: x264's B-frame decision is currently not very good, especially for multiple B-frames.

For example, you'll notice a drastic improvement in x264 OPSNR and SSIM (both with and without AQ) on parkrun if you use --no-b-adapt --bframes 3 --b-pyramid :cool:

[Sagekilla]

Another thing, why --no-dct-decimate? I've rarely seen that increase quality and in most cases it just blew up file size more than necessary.

@Dark Shikari: Any plans up for improving B-Frame decision yet? I'd love to take a look inside and try fiddling around to see if I can (probably not) do anything, but I suffer from epic failure at compiling.

[kosmonaut]

So next year we will see Divx back again , since they bought MainConcept and also Adobe Flash, since they are now licensing the MainConcept core codec :)

That would be a reasonable assumption. :cool:

[Dark Shikari]

@Dark Shikari: Any plans up for improving B-Frame decision yet? I'd love to take a look inside and try fiddling around to see if I can (probably not) do anything, but I suffer from epic failure at compiling.I talked with Pengvado about it last night.

There are three main problems:

1. Current B-frame decision beyond 1 B-frame is pretty bad and relies on an ugly guess of a heuristic.

2. It doesn't take into account B-pyramids when making the decision.

3. The lowres lookahead may or may not be good enough for the B-frame decision.

The solution pengvado proposed is a trellis-like algorithm for deciding future frametypes. Its actually not that hard to implement, and wouldn't be too much slower--while it would be drastically slower than the current method, lowres lookahead is orders of magnitude faster than actual encoding, since it doesn't actually do any encoding.

[Sagittaire]

Also, an interesting note: x264's B-frame decision is currently not very good, especially for multiple B-frames.

For example, you'll notice a drastic improvement in x264 OPSNR and SSIM (both with and without AQ) on parkrun if you use --no-b-adapt --bframes 3 --b-pyramid :cool:

Yes it's true. Make rdo for bframe decision is by far a better way for quality ... but not for speed. Libacodec can use rdo for bframe decision ...

[Dark Shikari]

Yes it's true. Make rdo for bframe decision is by far a better way for quality ... but not for speed. Libacodec can use rdo for bframe decision ...Don't need RDO, as I said above. x264's B-frame lookahead is extremely fast (orders of magnitude faster than regular RDRC) and can be used for vastly more intelligent B-frame decision without much speed loss.

Akupenguin has an idea for a Trellis-like algorithm for this that requires a mere 10 frame lookaheads per frame encoded.

[IgorC]

Not only x264 has problem with b-frames. For example MC jumps from 1 to 3 bframes. And almost never put 2 consecutive bframes. It's some kind of optimization indeed.

[skal]

Dmitriy: the document seems to lack details about the way you measure speed. MainConcept's encoder used to fork and/or use 2 threads at least, whether you ask it or not. What elapsed time do you measure exactly ? (real? user? sys?). And what machine were each tests conducted on, exactly?

[Nil Einne]

skal:

Same CPU as last year: Athlon64 3000+ in 32bit mode.
x264 has no problem getting down to 1.5fps at 720p, but that does leave a little room for improvement.

[Dark Shikari]

Not only x264 has problem with b-frames. For example MC jumps from 1 to 3 bframes. And almost never put 2 consecutive bframes. It's some kind of optimization indeed.That's because PbBbP is inherently quite efficient.

[IgorC]

Maybe cause of flexible and particular coding of each frame,good prediction and quantization algorithms.

[skal]

Nil Einne,
the document mentions 2 configs, but one don't know which was used for what exactly. And if MainConcept actually uses unrequested background thread, one need to know what is the timing reported, exactly.

[akupenguin]

And if MainConcept actually uses unrequested background thread, one need to know what is the timing reported, exactly.
It ran on a single core. What are you worried about, a little context switching overhead?

[skal]

It ran on a single core. What are you worried about, a little context switching overhead?

just worried that the time was actually measured with a wristwatch or a wallclock instead of just counting the number of ticks spent in the main thread. I've been fooled once like that, but maybe that's just me...

[skal]

Dmitriy: ping?