[Test Codec] H264, VP6, RV10, WMV9, DivX3, XviD, DivX5 .....

[Sagittaire]

Update with RealVideo 10 'Elysian'

[Big_Berny]

Hi Sagittaire,
your test is really great!
Two questions:
1. Do you used the newest VP6-version (6.2.0.10) in this test?
2. Can you post a speedtable of the different codecs? OR can you just tell me the speed which you get with VP6 (aprox.)?

Would be great
Big_Berny

[kilg0r3]

@ Sagitaire

I'd really like to know how xvid scores with the mpeg matrix and probabely even qpel and gmc.

[Sagittaire]

Best results with these seetings.

H263 or HVS better Quality
VHQ4
Ultra High
no Qpel and no GMC

Matrix MPEG isn't best matrix for XviD in this test ... ;-)

[h9903209]

Would you like to do a 1-pass comparison too? seems 2-pass result does not necessarily imply 1-pass the same result from my experience. Just wondering whether you also want to know the difference between 1-pass result of different codec and the difference between 1-pass and 2-pass of different codec... ^^

[Sagittaire]

Update with RealVideo 10 'Elysian'

- RV10 old Rate Control
- RV10 new Rate Control Inloop filtering Q12 & HFE=1

For me old RC is better for low bitrate and new RC is better for high bitrate ...

[kilg0r3]

@ Sagitaire

There is a way to turn of the inloop filtering of RV10 (encoder and decoder!) plus the high-frequency retention setting in the decoder.

BTW I am also a single pass user -> I'd be very interested in such a comparison.

Thank you very much for your efforts.

[eltoder]

Originally posted by Big_Berny
Hi Sagittaire,
your test is really great!
Two questions:
1. Do you used the newest VP6-version (6.2.0.10) in this test?
2. Can you post a speedtable of the different codecs? OR can you just tell me the speed which you get with VP6 (aprox.)?

Would be great
Big_Berny
I'm second for both :)
This test is really interesting. Thanks alot.

-Eugene

[Sagittaire]

VP6 6.2

Sharpeness is good for high bitrate but for not low/medium bitrate: more detail but more artefact too for low bitrate.

-> Best result with sharpeness = 0 for PSNR, OverallPSNR, SSIM in this test

[eltoder]

Originally posted by Sagittaire
VP6 6.2

Sharpeness is good for high bitrate but for not low/medium bitrate: more detail but more artefact too for low bitrate.

-> Best result with sharpeness = 0 for PSNR, OverallPSNR, SSIM in this test
But the first page still says it's using VP 6.1 ;)
And VP6.2 at sharpness = 0 produces somewhat softer than original VP6.1

-Eugene

[kilg0r3]

Some settings-nagging again. of course I assume you treid a lot of different settings. Yet is are my suggestion for Bframes anyway. BFrames 2/1.5/0.

Cheers

[webwonk]

Can One-Pass CBR testing be included? For those of us who do live video streaming this is the most crucial type of test paramter. Which of the popular codecs do it best in the 900kbs-2mbs range?:

VP 6.2.0.10
RV10 Elysian & HFE2
WMV9-Pro/VC9
h.264
XviD 1.0 rc4
DivX Pro 5.1.1

And what is the deal with post-processing? Does it help visual quality? Or does it tax the end-users' system too much (which might offset improvements)?

Any information and results would be most helpful. Thanks in advance.

Web

[karl_lillevold]

I just wanted to mention that I believe PSNR and PSNR-derivatives (SSIM, JND) will become less and less important for video as it is getting closer to transparency, just as has been the case for audio for many years already. Psycho-visual enhancements already exist that make the video appear more like the original, even though PSNR is decreased. How can this happen, you may ask.. The simplest example could be the video being shifted one pixel compared to the original. PSNR and all measurements that do some kind of comparison with the original, pixel for pixel, will be completely ruined, but visually, seeing the one pixel shift is almost impossible.

For instance, HFE2 should not be used for PSNR comparisons. Even the original HFE, like I think Sagittaire is using for his test, is not recommended. Here's a quote from the HFE2 post (http://forum.doom9.org/showthread.php?s=&threadid=75278). "Mathematical analysis between the source and the encoded video has shown that even though PSNR is reduced from HFE2, the visual resemblance is closer to the original."

Then, with regards to RV10 Elysian, in this post (http://forum.doom9.org/showthread.php?s=&threadid=71022&perpage=20&pagenumber=10) I explained how PSNR can be increased 1/2 dB just by setting patternAdaptivity=2, and rcHighBitrateReduce to 60. Still, my experience is that I prefer the default settings for visual quality, maybe even with
<rcPFrameRefQuant type="uint">6</rcPFrameRefQuant>
<rcBFrameRefQuant type="uint">9</rcBFrameRefQuant>
instead of the default 6 and 10. Others might prefer to max out PSNR though.

[CruNcher]

:)

[Sagittaire]

Update with Gladiator Test

[SeeMoreDigital]

Originally posted by Sagittaire
Update with Gladiator Test If your Gladiator source is from an HiDef 1080i 1.77:1 source, why not have a go at generating some 720x576 or even 1024x576 encodes?


Cheers

[Sagittaire]

Originally posted by SeeMoreDigital
If your Gladiator source is from an HiDef 1080i 1.77:1 source, why not have a go at generating some 720x576 or even 1024x576 encodes?


640*352 or 576*320 is the most common resolution used for 16/9 picture. perhaps later with another HDTV source ... Spiderman II HDTV trailer or Van Helsings II HDTV trailer for exemple ...

[SeeMoreDigital]

Originally posted by Sagittaire
640*352 or 576*320 is the most common resolution used for 16/9 picture. Personally I disagree,

Where ever possible it's advisable to keep as much vertical resolution as possible, otherwise why bother capturing and converting from an HiDef source in the first place?

Also remember that an 1080i source is interlaced not progressive like an DVD source.

All the 1080i source material I have looks great when converted to 1280x720. But if that's not possible then the only other option is to use 720x576 anamorphic.


Cheers

[karl_lillevold]

Google-translated conclusion:

"The RV10 "Elysian" with its new rate control and its threshold of activation of the inloop filtering shows from now on as powerful at all the bitrates. It is even with this source the most powerful codec with all bitrates. For the highest bitrates RV10 progressed the most. The excellent surprise comes from H264 ffvfw which is shown already powerful as of its first appearance in ffvfw. The VP6 if it disappoints a little with this source and this resolution remains an excellent choice. Even if the metric tests give a light advantage to the WMV9 compared to XviD 1.0, these two codecs are visually very close all the same. Thank you in Bugsan for its fantastic Java XviD Stats Viewer which allows a very fine analysis of the behavior of the codecs..."

[Sagittaire]

Update:
- Codec DivX 5.2
- Diaporama low and high motion

[Latexxx]

I'd like to see some results from the subjective part of Everwicked's Video Quality Studio (http://www.everwicked.com/vqstudio/).

[Sagittaire]

6 codecs x 4 bitrate = 24 tests and very slow test. Not possible make job with this log ...

I use AviSynth and virtualDub job for PSNR and SSIM test. It's impossible with VQM (LeFungus VQM.dll is not complete)

[Sagittaire]

Test in french (http://jfl1974.free.fr/HTM/32_Test_Codec_Video.htm) ... but Capture, Sample & Graphs are universal language

Update:

- H264 Nero Digital (MPEG4 AVC codec from Ateme)
- XviD 1.0.1 with H263 matrix (visually better with PP4)
- DivX 5.2.1 with best setting for Digital56K
- Bitrate equivalence (with metric)

[IgorC]

The samples of codecs has a quite significal different sizes. For example, VP6.2 500 kbit/s - 5.81 Mb and RV10 500 kbit/s - 5.41 Mb
So difference of size is 7,39%. But may be the differnce on lagrer files will be less.
Waiting for your next test with the upgrated RV10, WMV 9.1, XVID 1.1 (after release of course):)

[Razorblade2000]

How about adding SNOW to your comparison :D
It would be really cool to see how it performs :D

[guada]

It's astonishing Sagittaire,

All my encode visual that it is low or high bitrates gives winning the WMV9.
Besides, the middle bitrate to 1500 kbps that stays the properties of the Vp6.
But my general surprise comes of the codec H-264 SNOW (2pass), that at low bitrate (200kbps) is below the qualities of WMV9.

I would stain to give some pictures at next.

To soon ;)

[Sagittaire]

The samples of codecs has a quite significal different sizes. For example, VP6.2 500 kbit/s - 5.81 Mb and RV10 500 kbit/s - 5.41 Mb
So difference of size is 7,39%. But may be the differnce on lagrer files will be less.
Waiting for your next test with the upgrated RV10, WMV 9.1, XVID 1.1 (after release of course)


Sample extract from complete encoding but I update RV10 sample:

- H264 500 kbps : 5694 Ko
- VP6 500 kbps : 5952 Ko
- RV10 500 kbps : 5750 Ko
- WMV9 500 kbps : 5816 Ko
- XviD 500 kbps : 5824 Ko
- DivX5 500 kbps : 6038 Ko


Sample extract from complete encoding but I update RV10 sample:

- H264 1000 kbps : 6103 Ko
- VP6 1000 kbps : 5964 Ko
- RV10 1000 kbps : 6192 Ko
- WMV9 1000 kbps : 6198 Ko
- XviD 1000 kbps : 6208 Ko
- DivX5 1000 kbps : 6232 Ko

[708145]

But my general surprise comes of the codec H-264 SNOW (2pass), that at low bitrate (200kbps) is below the qualities of WMV9.

Apart from the fact that SNOW is not H.264 but wavelet based, it is still undergoing heavy development.
http://ffmpeg.sourceforge.net/index.php

But now back on topic:
I read all those discussions arguing about Sagittaires test methods, parameter selections, objective measurements, aso.
What I want to know is if you also object about his conclusions regarding codec efficiencies near the bottom of his page. (Like XVID being about 44% less efficient than h.264 @500kbit)

Do you think this is true WYOE (with your own eyes) as well?

bis besser,
Tobias

[guada]

A question: The numbers prove them really the reality ?
I have by habit to say, it is necessary to see to believe.
I don't hold to correct Sagittaire, I do note only that what I see.
But You, can you believe what you don't necessarily see.

It is necessary again that you see it; .:D


it was only a parenthesis.

[Sagittaire]

@ guada

In fact it's very easy: bitrate equivalence with metric graph (Average PSNR, Overall PSNR and SSIM). For example 43dB for H264 at 500 Kbps, 43dB with VP6 at 590 Kbps ...



@ 708145

Originally posted by 708145
Apart from the fact that SNOW is not H.264 but wavelet based, it is still undergoing heavy development.
http://ffmpeg.sourceforge.net/index.php

But now back on topic:
I read all those discussions arguing about Sagittaires test methods, parameter selections, objective measurements, aso.
What I want to know is if you also object about his conclusions regarding codec efficiencies near the bottom of his page. (Like XVID being about 44% less efficient than h.264 @500kbit)

Do you think this is true WYOE (with your own eyes) as well?

bis besser,
Tobias

Read transparent Teegedeck test (~q4 for XviD and 1000 Kbps in my test) in H264 Ateme thread ...

H264 Ateme for very low bitrate (~q8 for XviD and 500 Kbps in my test) is very very powerfull ...

[guada]

Thank you Saigittaire :)

Note: Sorry for this impertinent remark ;)

[708145]

Maybe I should rephrase my question:

Does the majority here believe that your (<- Sagittaire) bitrate equivalences are true for subjective evaluation as well?

(Ignoring the fact that those equivalences were determined using objective methods.)

bis besser,
T0B1A5

P.S.: This is NOT about my own subjective impressions. It's about the validity of objective tests.

[Sagittaire]

it's always the same question ... I don't know but:

On the complete Matrix 3 trailer this Ateme setting produced an output that was 37.05% smaller than XviD @q=4) - which most people will find transparent

[18%-27%] in my test with metric ...


Originally posted by Gej
We have tested several clips produced by the studios and mapped PSNR against bitrate for both codecs

DivX Developper use this technique for bitrate equivalence. One2Tech use certainely this technique too (VP4 25% better than VP3, VP5 15% better than VP4 ...)

[Sagittaire]

x264 seem very good too ...

http://jfl1974.free.fr/Video/x264vsVP6-1000.JPG

Average PSNR
H264 : 44.7337 dB
VP6 : 44.3975 dB
x264 : 44.2624 dB
RV10 : 44.0334 dB
WMV9 : 43.7891 dB
XviD : 43.6912 dB
DivX5 : 43.0575 dB

Overall PSNR
H264 : 44.1604 dB
VP6 : 43.9286 dB
x264 : 43.6708 dB
RV10 : 43.5277 dB
WMV9 : 43.3891 dB
XviD : 43.2531 dB
DivX5 : 42.4873 dB