[TEST] Visual, PSNR, SSIM, VQM tests with RV9, XviD ,DivX, WMV9 ... - Page 3

Teegedeck · 29th July 2003, 23:22

Where could we download that trailer you used as a source?

Sagittaire · 29th July 2003, 23:35

It's an trailer of Harry Potter's DVD ... 75 Mo

Teegedeck · 30th July 2003, 07:26

I just wondered because the highest resolution trailer I've found in the net has a 480x260-resolution.

Is it a trailer from the DVD?

Sagittaire · 30th July 2003, 08:20

it's a trailer from HP II DVD 720*576 high quality with no entrelacing ...

h9903209 · 30th July 2003, 10:57

Thx for your testing. Is that possible to include the speed of encoding? How long does each one take to finish the encode?

Sagittaire · 31st July 2003, 23:38

Update first post ...

XviD devapi4 test
PSNR, SSIM, VQM test and graph
DivX3 SBC ffvfw with Post-Precessing test

RV9 EHQ ... winner

Here my conclusions on these test:

The RV9 is the best codec has weak resolution and weak bitrate far in front of the others.

DivX Manihi and XviD Devapi4 are codecs which are almost equivalent.

WMV9 is the most powerful codec in high resolution and high bitrate.

DivX SBC is still a very good codec and it is still very powerful in encodings with a compressibility superior to 50%.

SeeMoreDigital · 31st July 2003, 23:52

Sagittaire

How about doing some more PSNR tests, using the same 'video only' input file, but this time generating the same output file size.

And all without cropping and resizing?

I think your results will be quite a bit different!

Cheers

CruNcher · 1st August 2003, 01:17

@Sagittaire

it's ridicilous, you still do this XviD Ultra High vs DivX Slowest compare thats flawed you need 3x more with Slowest Me in DivX then with Ultra High in XviD, wich is like Standard in DivX and you only win for this 3x slowness 0.1 dB, how you can say this makes DivX the most iso complaint Mpeg4 Codec, sorry can't follow you here. And use compareyv12 instead of converttoyuy2

Sagittaire · 1st August 2003, 01:33

It's a quality test ... not a speed test ...

But is true the Manihi is slowest than XviD in slow mode but the standard mode is very good too (download DivX5 for demo).

The best codec is not Manihi but RV9 EHQ ...

kastro68 · 1st August 2003, 01:46

@Sagittaire

Thanks for the tests. You have obviously spent a great deal of time and effort.

I've been using RV9 for low bitrate anime encoding [3.5hrs into 700megs] ever since I found out it was free. I'm just shocked that it gives higher psnr at higher bitrates too.

@animaniac

Quote:

I'd like to the same test with 1000 kbps and 1200 kbps bitrates...

You can use linear interpolation for these.[Assumes that the variance of the white noise is constant and normally distributed]

At 1000kbps RV9 and wmv9 should be on par. From the graphs, it looks like WMV9 would dominate at higher bitrates.

edit:typo

vinouz · 1st August 2003, 12:40

I may seem an idiot, but what means SSIM & VQM ? What are they based on ?

kilg0r3 · 1st August 2003, 13:06

1. @vinouz
you can google can you?
http://www.its.bldrdoc.gov/n3/video/Default.htm
http://www.cns.nyu.edu/~zwang/files/research/ssim/

2.
@sagitaire
could you add the fps or encoding time? And, it would be nie to see if qpel adds something to xvids performance.

3.
what method did the ct magazine use to meseure the codec quality?

Lefungus · 1st August 2003, 13:26

[Edited]: I need to think before posting

Sagittaire · 1st August 2003, 13:56

PSNR: high is better quality max = 105
SSIM: high is better quality max = 1
VQM: small is better quality max = 0

Qpel and GMC decrease PSNR and quality test ...

SeeMoreDigital · 1st August 2003, 13:59

Although this thread has been both informative and useful, and long may it continue to be. For me, it's confirmed that PSNR etc tests are not the diffinative way (or answer) to measuring how good an encode/codec is!

Reading the entire thread, reveals refreshing opinions regarding the codecs used and how we all have different methods and views about quality.

I'm happy that we are not all 'Mini Me' clones with regard to our opinions.

Sagittaire · 1st August 2003, 14:07

for the eyes tests ... ;-)

~~ Visual Test ~~

Video Encode 576*240 570 Kbps

http://jfl1974.free.fr/Video/XviD-570.rar
http://jfl1974.free.fr/Video/RV9-570.rar
http://jfl1974.free.fr/Video/WMV9-570.rar
http://jfl1974.free.fr/Video/Manihi-570.rar
http://jfl1974.free.fr/Video/DivX5-570.rar
http://jfl1974.free.fr/Video/DivX3-570.rar

Download and compare ...

SeeMoreDigital · 1st August 2003, 18:23

Thanks Sagittaire,

I downloaded and unpacked the following video only 'visual test' clips: -

RMV9-570kbps (8,939KB or 8.729MB)
Manihi-570kbps (9,008KB or 8.796MB)
XviD-570kbps (9,160KB or 8.945MB)
WMV9-570kbps (9'170KB or 8.955MB)

And this confirms exactly what I've been saying for many months now. Which is when you take the same 'video only' input file and encode at the same bitrate with different codecs, you obtain totally different file sizes!

OK the file size difference may be small (the clip is only 128 seconds long) but if you increase the bitrate setting of the other codecs to match the largest file size (which is always WMV9) then the 'visual' and 'PSNR etc' results will be different again.

This then means, that when you take this anomaly into consideration, the Real Media encode is even better!

Sagittaire · 1st August 2003, 18:38

a difference in size of 1% does not mean at all profit of 1% of PSNR for example ... make the test ...

and to visually make the difference between an encoding with 565 Kbps or 575 Kbps is impossible ...

The test are right and the small difference in size of encoding do not affect results ...

SeeMoreDigital · 1st August 2003, 19:05

Yes, like I said the file size differences are small.

However, if you encoded the entire Harry Potter 2 DVD (which is 9261 seconds long), you would have a 16.32MB difference between the RMV9 and WMV9 file sizes. Which is 2.6%!

And when encoding at low bitrates it's important to utilize every last MB!

RMV9 still comes out on top.... Karl will be pleased!

Cheers

Lefungus · 1st August 2003, 19:13

Quote:

Originally posted by SeeMoreDigital

However, if you encoded the entire Harry Potter 2 DVD (which is 9261 seconds long), you would have a 16.32MB difference between the RMV9 and WMV9 file sizes. Which is 2.6%!

Why ?
It may as well still have a 1% difference, like the trailers.

29th July 2003, 23:22	#41 \| Link
Teegedeck Moderator, Ex(viD)-Mascot Join Date: Oct 2001 Posts: 2,564	Where could we download that trailer you used as a source? __________________ It's a man's life in Doom9's 52nd MPEG division. "The cat sat on the mat." ATM I'm thoroughly enjoying the Banshee - a fantastic music player/ripper for Linux. Give it a whirl!

29th July 2003, 23:35	#42 \| Link
Sagittaire Testeur de codecs Join Date: May 2003 Location: France Posts: 2,484	It's an trailer of Harry Potter's DVD ... 75 Mo __________________ Le Sagittaire ... ;-) 1- Ateme AVC or x264 2- VP7 or RV10 only for anime 3- XviD, DivX or WMV9

30th July 2003, 07:26	#43 \| Link
Teegedeck Moderator, Ex(viD)-Mascot Join Date: Oct 2001 Posts: 2,564	I just wondered because the highest resolution trailer I've found in the net has a 480x260-resolution. Is it a trailer from the DVD? __________________ It's a man's life in Doom9's 52nd MPEG division. "The cat sat on the mat." ATM I'm thoroughly enjoying the Banshee - a fantastic music player/ripper for Linux. Give it a whirl!

30th July 2003, 08:20	#44 \| Link
Sagittaire Testeur de codecs Join Date: May 2003 Location: France Posts: 2,484	it's a trailer from HP II DVD 720*576 high quality with no entrelacing ... __________________ Le Sagittaire ... ;-) 1- Ateme AVC or x264 2- VP7 or RV10 only for anime 3- XviD, DivX or WMV9

31st July 2003, 23:38	#46 \| Link
Sagittaire Testeur de codecs Join Date: May 2003 Location: France Posts: 2,484	Update first post ... XviD devapi4 test PSNR, SSIM, VQM test and graph DivX3 SBC ffvfw with Post-Precessing test RV9 EHQ ... winner Here my conclusions on these test: The RV9 is the best codec has weak resolution and weak bitrate far in front of the others. DivX Manihi and XviD Devapi4 are codecs which are almost equivalent. WMV9 is the most powerful codec in high resolution and high bitrate. DivX SBC is still a very good codec and it is still very powerful in encodings with a compressibility superior to 50%. __________________ Le Sagittaire ... ;-) 1- Ateme AVC or x264 2- VP7 or RV10 only for anime 3- XviD, DivX or WMV9 Last edited by Sagittaire; 1st August 2003 at 07:57.

30th July 2003, 10:57	#45 \| Link
h9903209 Registered User Join Date: May 2003 Posts: 150	Thx for your testing. Is that possible to include the speed of encoding? How long does each one take to finish the encode?

31st July 2003, 23:52	#47 \| Link
SeeMoreDigital Life's clearer in 4K UHD Join Date: Jun 2003 Location: Notts, UK Posts: 12,227	Sagittaire How about doing some more PSNR tests, using the same 'video only' input file, but this time generating the same output file size. And all without cropping and resizing? I think your results will be quite a bit different! Cheers __________________ \| I've been testing hardware media playback devices and software A/V encoders and decoders since 2001 \| My Network Layout & A/V Gear \| Last edited by SeeMoreDigital; 1st August 2003 at 01:27.

1st August 2003, 01:17	#48 \| Link
CruNcher Registered User Join Date: Apr 2002 Location: Germany Posts: 4,926	@Sagittaire it's ridicilous, you still do this XviD Ultra High vs DivX Slowest compare thats flawed you need 3x more with Slowest Me in DivX then with Ultra High in XviD, wich is like Standard in DivX and you only win for this 3x slowness 0.1 dB, how you can say this makes DivX the most iso complaint Mpeg4 Codec, sorry can't follow you here. And use compareyv12 instead of converttoyuy2 __________________ all my compares are riddles so please try to decipher them yourselves :) It is about Time Join the Revolution NOW before it is to Late ! http://forum.doom9.org/showthread.php?t=168004 Last edited by CruNcher; 1st August 2003 at 01:25.

1st August 2003, 01:33	#49 \| Link
Sagittaire Testeur de codecs Join Date: May 2003 Location: France Posts: 2,484	It's a quality test ... not a speed test ... But is true the Manihi is slowest than XviD in slow mode but the standard mode is very good too (download DivX5 for demo). The best codec is not Manihi but RV9 EHQ ... __________________ Le Sagittaire ... ;-) 1- Ateme AVC or x264 2- VP7 or RV10 only for anime 3- XviD, DivX or WMV9

1st August 2003, 12:40	#51 \| Link
vinouz Grüt Power Maintainer Join Date: Mar 2002 Location: france Posts: 111	I may seem an idiot, but what means SSIM & VQM ? What are they based on ? __________________ A psychotic thinks that 2 + 2 = 5. A neurotic knows that 2 + 2 = 4... but it worries him!

1st August 2003, 13:06	#52 \| Link
kilg0r3 ! - User - ! Join Date: Nov 2001 Posts: 1,081	1. @vinouz you can google can you? http://www.its.bldrdoc.gov/n3/video/Default.htm http://www.cns.nyu.edu/~zwang/files/research/ssim/ 2. @sagitaire could you add the fps or encoding time? And, it would be nie to see if qpel adds something to xvids performance. 3. what method did the ct magazine use to meseure the codec quality? __________________ Keep your tone warm and your sigs decent! Last edited by kilg0r3; 1st August 2003 at 13:15.

1st August 2003, 13:26	#53 \| Link
Lefungus Registered User Join Date: Apr 2002 Location: Bretagne Posts: 192	[Edited]: I need to think before posting Last edited by Lefungus; 1st August 2003 at 13:40.

1st August 2003, 13:56	#54 \| Link
Sagittaire Testeur de codecs Join Date: May 2003 Location: France Posts: 2,484	PSNR: high is better quality max = 105 SSIM: high is better quality max = 1 VQM: small is better quality max = 0 Qpel and GMC decrease PSNR and quality test ... __________________ Le Sagittaire ... ;-) 1- Ateme AVC or x264 2- VP7 or RV10 only for anime 3- XviD, DivX or WMV9

1st August 2003, 13:59	#55 \| Link
SeeMoreDigital Life's clearer in 4K UHD Join Date: Jun 2003 Location: Notts, UK Posts: 12,227	Although this thread has been both informative and useful, and long may it continue to be. For me, it's confirmed that PSNR etc tests are not the diffinative way (or answer) to measuring how good an encode/codec is! Reading the entire thread, reveals refreshing opinions regarding the codecs used and how we all have different methods and views about quality. I'm happy that we are not all 'Mini Me' clones with regard to our opinions. __________________ \| I've been testing hardware media playback devices and software A/V encoders and decoders since 2001 \| My Network Layout & A/V Gear \| Last edited by SeeMoreDigital; 1st August 2003 at 17:49.

1st August 2003, 14:07	#56 \| Link
Sagittaire Testeur de codecs Join Date: May 2003 Location: France Posts: 2,484	for the eyes tests ... ;-) ~~ Visual Test ~~ Video Encode 576*240 570 Kbps http://jfl1974.free.fr/Video/XviD-570.rar http://jfl1974.free.fr/Video/RV9-570.rar http://jfl1974.free.fr/Video/WMV9-570.rar http://jfl1974.free.fr/Video/Manihi-570.rar http://jfl1974.free.fr/Video/DivX5-570.rar http://jfl1974.free.fr/Video/DivX3-570.rar Download and compare ... __________________ Le Sagittaire ... ;-) 1- Ateme AVC or x264 2- VP7 or RV10 only for anime 3- XviD, DivX or WMV9

1st August 2003, 18:23	#57 \| Link
SeeMoreDigital Life's clearer in 4K UHD Join Date: Jun 2003 Location: Notts, UK Posts: 12,227	Thanks Sagittaire, I downloaded and unpacked the following video only 'visual test' clips: - RMV9-570kbps (8,939KB or 8.729MB) Manihi-570kbps (9,008KB or 8.796MB) XviD-570kbps (9,160KB or 8.945MB) WMV9-570kbps (9'170KB or 8.955MB) And this confirms exactly what I've been saying for many months now. Which is when you take the same 'video only' input file and encode at the same bitrate with different codecs, you obtain totally different file sizes! OK the file size difference may be small (the clip is only 128 seconds long) but if you increase the bitrate setting of the other codecs to match the largest file size (which is always WMV9) then the 'visual' and 'PSNR etc' results will be different again. This then means, that when you take this anomaly into consideration, the Real Media encode is even better! __________________ \| I've been testing hardware media playback devices and software A/V encoders and decoders since 2001 \| My Network Layout & A/V Gear \| Last edited by SeeMoreDigital; 1st August 2003 at 18:25.

1st August 2003, 18:38	#58 \| Link
Sagittaire Testeur de codecs Join Date: May 2003 Location: France Posts: 2,484	a difference in size of 1% does not mean at all profit of 1% of PSNR for example ... make the test ... and to visually make the difference between an encoding with 565 Kbps or 575 Kbps is impossible ... The test are right and the small difference in size of encoding do not affect results ... __________________ Le Sagittaire ... ;-) 1- Ateme AVC or x264 2- VP7 or RV10 only for anime 3- XviD, DivX or WMV9

1st August 2003, 19:05	#59 \| Link
SeeMoreDigital Life's clearer in 4K UHD Join Date: Jun 2003 Location: Notts, UK Posts: 12,227	Yes, like I said the file size differences are small. However, if you encoded the entire Harry Potter 2 DVD (which is 9261 seconds long), you would have a 16.32MB difference between the RMV9 and WMV9 file sizes. Which is 2.6%! And when encoding at low bitrates it's important to utilize every last MB! RMV9 still comes out on top.... Karl will be pleased! Cheers __________________ \| I've been testing hardware media playback devices and software A/V encoders and decoders since 2001 \| My Network Layout & A/V Gear \| Last edited by SeeMoreDigital; 1st August 2003 at 19:08.