MSU MPEG-2 codecs comparison prepararion

[DmitriyV2]

Dear all!

Now MSU Graphics&Media Lab Video group planing to do new codec comparison with MPEG-2. (Please see preveous comparisons MPEG-4, H.264, JPEG-2000 and etc on http://www.compression.ru/video/codec_comparison/index_en.html)

Basic idea of this new comparison is to test different MPEG-2 Decoders for decoding MPEG-2 Stream that contain errors (packet lost or bit errors). We want to encode video with some MPEG-2 Encoder. Then generate some errors in this stream and to decode with different decoders and to compare results using visual and objective metrics and to analyze the results.

What codec is more relevant to be reference encoder for stream generation fom your point of view? Any ideas are welcome!

Also please send your name, we will note you on "Thanks to" page.

Thank you in advance!

[Revgen]

When you say "stream generation" do you mean transport streams? DVD video streams?

DVD compatible encoders are:


HCEnc http://hcencoder.bitburners.com/

CCE http://www.cinemacraft.com/

Quenc http://nic.dnsalias.com/

AutoQmat http://www.autoqmatenc.com/

Procoder http://www.canopus.com


There may be others as well.

[DmitriyV2]

DVD compatible encoders are:
There is no problems with finding encoders. We collect links on http://www.compression-links.info/ for example. :)

Question was - what encoder is more proper for "reference" encoder to encode stream for other decoders?

Any ideas are welcome!

[dragongodz]

you want a reference encoder ?
http://www.mpeg.org/MPEG/MSSG/

seriously though. different encoders may produce or handle different footage better, for example QuEnc, HC and Procoder are mean to be better at low bitrate and interlaced encoding according to many i have seen than CCE but then you could compare the speed of CCE, especially against Procoder and Tmpgenc etc.

but really i dont think its really going to matter too much for comparing how decoders handle damaged streams.

[DmitriyV2]

you want a reference encoder ?
http://www.mpeg.org/MPEG/MSSG/
seriously though. different encoders may produce or handle different footage better, for example QuEnc, HC and Procoder are mean to be better at low bitrate and interlaced encoding according to many i have seen than CCE but then you could compare the speed of CCE, especially against Procoder and Tmpgenc etc.
Ok! Thanks! Maybe we will plan such tests.

but really i dont think its really going to matter too much for comparing how decoders handle damaged streams.
MPEG-2 is widely used not only for DVD, we will test broadcasting issues of decoders in next comparison of MPEG-2 decoders.

Any other ideas about encoders are welcome!

[Sagittaire]

TMPGEnc
FreeEnc
QEnc
NuEnc
Mencoder
FFDShow
HC017
AutoQMatEnc
Mainconcept
Canopus Procoder
Cinema Craft Encoder
Nero Recode
FFMPEG

be carefull : many Encoder like QEnc, FreeEnc, FFDShow, Mencoder ... use Libavcodec source code.

Mencoder is by far the best for Objective test and if you want produce compliant DVD stream with Mencoder you must use high tweak setting ...

[danpos]

TMPGEnc

Mencoder is by far the best for Objective test and if you want produce compliant DVD stream with Mencoder you must use high tweak setting ...



OT_ON

Hi there! Could you please speak out about these 'high tweak settings' ? I'm very interested to work with Mencoder in order to build MPEG2 videostreams full DVD compatible ... ;)

See ya,

OT_OFF

[DmitriyV2]

Hi there! Could you please speak out about these 'high tweak settings' ?
We also interesting of "high tweak settings". When we prepare test's - it's commonly very difficult to receive codec settings (even from codec developers), but when we finish tests - meny persons has suggestions about codec and settings selection. :) (This is people nature, I understand this.)

[bond]

hm i dont really understand your question about what codec to use? shouldnt it be possible to use any you want for your purpose?

[DmitriyV2]

hm i dont really understand your question about what codec to use?
We try to collect ideas how to select best and most correct encoder for comparison.
shouldnt it be possible to use any you want for your purpose?
Sure we do. :) But any suggestions from MPEG-2 experts are welcome.

-----------
What are the most popular MPEG-2 decoders? Any ideas are welcome!

[scharfis_brain]

Decoders:
AVS: mpeg2dec3 / dgdecode
DS: NVidia PureVideo, DSCALER, MPC's internal ...

[DmitriyV2]

Decoders:
AVS: mpeg2dec3 / dgdecode
DS: NVidia PureVideo, DSCALER, MPC's internal ...
Thanks! You think this decoders are most popular?

[Revgen]

The most popular mpeg2 decoder is whatever is bundled with WMP. Since most end-users use whatever is bundled.

However, I think the two best are Dscaler 5 (for film), Sonic Cinemaster (for anything else), and FFDshows decoder (also for anything else).

Dscaler 5 is great when it works, it can look bad when it doesn't since it is still being developed. Sonic Cinemaster and FFDshow are almost always stable.

[Mug Funky]

you could author a corrupt (but compliant) DVD and test it on various standalones. that'd be a good test IMHO, but quite hard to analyse. maybe limit it to ones with HDMI out, then capture that signal as 4:2:2 uncompressed or something (i believe you can get HDMI->SDI converter devices for not too much dosh). after all, these are the most used mpeg-2 decoders :)

as for encoders, that's a tough call. to the decoder, anything that produces compliant output is fine. the only difference is efficiency really. that could have an effect on error-concealment (the closer to real motion and the smarter the decoder, the better the errors will be concealed). conversely, bitrate and GOP size would have a big effect - more bitrate = more intra blocks = errors persist for less time. stuff like that IMHO has more bearing on the result than the actual encoder used.

[TEB]

Rhozet Carbon Coder PE (from the canopus guys.. procoder is a dead product..)

[DmitriyV2]

The most popular mpeg2 decoder is whatever is bundled with WMP. Since most end-users use whatever is bundled.
Hm... Reasonable! :)

However, I think the two best are Dscaler 5 (for film), Sonic Cinemaster (for anything else), and FFDshows decoder (also for anything else).

Ok!

Dscaler 5 is great when it works, it can look bad when it doesn't since it is still being developed. Sonic Cinemaster and FFDshow are almost always stable.
Good! Thank you for reasonable opinion!

[DmitriyV2]

you could author a corrupt (but compliant) DVD and test it on various standalones. that'd be a good test IMHO, but quite hard to analyse. maybe limit it to ones with HDMI out, then capture that signal as 4:2:2 uncompressed or something (i believe you can get HDMI->SDI converter devices for not too much dosh). after all, these are the most used mpeg-2 decoders :)

We thinking about analizing of several corrapted DVD's with special tools to analyze character of errors (probability, distribution, dispersion). And if we have time we do this - such test makes testing streams pretty real (captured and artificial later). Seems like we think in one directions, thank you!

as for encoders, that's a tough call. to the decoder, anything that produces compliant output is fine. the only difference is efficiency really. that could have an effect on error-concealment (the closer to real motion and the smarter the decoder, the better the errors will be concealed). conversely, bitrate and GOP size would have a big effect - more bitrate = more intra blocks = errors persist for less time. stuff like that IMHO has more bearing on the result than the actual encoder used.
Sure, but did not forgot - we are going to compare decoder. So bitrate and GOP size change situation in general, but the same way for all tested decoders. So difference will be the most interesting and indicative.

Smart comment! Thank you!

[DmitriyV2]

We begin to test codecs.

General methodology is following:


1. Video sequence is encoded with TMPGEnc with CBR 3Mb/s into Elementary Stream for MPEG-2
2. This sequence is remuxed into Transport Stream with Elecard Xmuxer Pro
3. Decode this sequence with decoder X1, decoder X2,…,decoder Xn
4. Loop (from 1, to 100)
a. Generate errors with BER (bit-error rate) = Y to remuxed encoded stream
b. Decode this stream with decoder X1, decoder X2,…,decoder Xn
c. Calculate Y-PSNR metric between decoded original stream (3) and decoded error stream (4b) for every decoder
i. Perframe metric
ii. Average metric for every error sequence
5. Average metric for 100 error sequences
6. Visualize metrics
7. Analyze results

First results looks like this chart:
http://download.graphicon.ru/video/mpeg2/Graph.png

PDF with methodology description and charts can be found here:
http://download.graphicon.ru/video/mpeg2/Methodology.pdf

Inspirit feedback is welcome! :)

[CruNcher]

you should take all "unique decoder cores" into the test and try to avoid SDK based dupes of them by some manufactures that do products like Authoring stuff Ulead, Cyberlink, Intervideo and Sonic for example all use SDKs from Video Compression Researchers like Ateme,Nero,Nvidia,Moonlight,Elecard/Mainconcept or Mpeg2dec and Libmpeg2

[faam]

Thanks! You think this decoders are most popular?
I am with scharfis_brain in his choices and I will add to
his directshow decoders to be as follow :
DScaller
Sonic Cinemaster (Cineplayer)
Gabest MPV (MPC internal decoder)
Nvidia Purevideo Decoder
Elecard
FFdshow
ATI MPEG2 Video Decoder
Those are the most popular
you can also include others as you are expert from previous comaprisons and general MPEG-2 decoders knowledge

Regards

[DmitriyV2]

Thanks! Will be included next time (I'l publish the same posting during comparison preparation)