[Test Codec] H264, VP6, RV10, WMV9, DivX3, XviD, DivX5 .....

Sagittaire · 10th November 2003, 15:31

Matrix II 720*304

Source DVD PAL 720*576 MPEG2 MP@ML, chapter 20 to 26, ~35 000 frames

Test in french ... but Capture, Sample & Graphs are universal language

Codecs
- H264 Ateme
- XviD 1.0.1
- RV10 Elysean
- VP6 6.2
- DivX 5.2.1
- WMV9 VCM
- DivX 3 ffvfw

Tests
- 500-2000 Kbps and 25-100% compressibility (/quant2 XviD H263)
- SSIM, Average PSNR, Overall PSNR and Frames PSNR test
- Blind test
- Samples
- Diaporama

bilu · 10th November 2003, 15:35

What's the criteria difference between the two images?

One is called PSNR and the other PSNR Overall...

EDIT: does that trailer have high-motion parts?

Bilu

Sirber · 10th November 2003, 17:09

Nice comparison

DAvenger · 10th November 2003, 17:15

Ah, could you please explain the results to us uneducated plebs?

Thanks

TheXung · 10th November 2003, 18:19

Could you use more differing colors for xvid and vp6 or at least make the lines thicker than a pixel.

Assault · 10th November 2003, 21:33

@ Sagittaire

Did you use XviD with GMC and Trellis? And for your b-frame settings I would recommend 1/150/100. I think this should deliver a little better results but altogether it's a very nice comparison. Thanks

@ all

When you look at the two graphs it's very interesting to see that VP6 is a good codec but has a bad 2-pass algorithm. Its results are much better for average psnr than for overall psnr. I think that's because average psnr doesn't take into account how constant the quality is. So I suspect that VP6 has both frames with VERY good quality and frames with VERY bad quality. That should be the reason why the results for overall psnr arent really good. If On2 improves the 2-pass mode VP6 will be a very good codec in my opinion.

Assault

CruNcher · 11th November 2003, 00:54

@Sagittaire

please post more infos filesize and time needed to encode

SeeMoreDigital · 11th November 2003, 01:09

Quote:

Originally posted by Assault
When you look at the two graphs it's very interesting to see that VP6 is a good codec but has a bad 2-pass algorithm. Its results are much better for average psnr than for overall psnr. I think that's because average psnr doesn't take into account how constant the quality is. So I suspect that VP6 has both frames with VERY good quality and frames with VERY bad quality. That should be the reason why the results for overall psnr arent really good. If On2 improves the 2-pass mode VP6 will be a very good codec in my opinion.

Assault

It's interesting you should mention that!

I've been trying to generate 2pass VBR encodes with VP6 (using MPEGmediator) but on every occasion the encode looks either the same or only very slightly better than a 1pass CBR VP6 encode.

Personally, I've been thinking that I may be doing something wrong when generating 2pass encodes. But I don't know for sure!

I really would like to get to the bottom of this. Because if VP6 is able to look as good as it does in CBR mode, it should look astonishing in VBR mode. I just wish I could get it right!

Cheers

EDIT: Anyway, what's with the DivX 5.1.2 ident. I thought the latest is DivX 5.1.1 beta2.... Or am I wrong?

Assault · 11th November 2003, 11:56

@ SeeMoreDigital

I don't think that you're doing something wrong when creating 2pass encodes. I wasn't content with the results I got in my tests either. I also remember C0mPr355 saying that VP6 is their first codec with a pass mode. So perhaps that's the reason for the bad results.

Concerning DivX I think you're right. It's probably a typo.

Assault

SeeMoreDigital · 11th November 2003, 13:03

Thanks for that Assault.

I thought I was begining to flip out there for a second - twice!

Yesterday I did some more low bitrate CBR tests with VP6 using 720x576 image pixel frame sizes. It really is very good!

However, I personally would like to be able to enter kb's instead of KB's per second. And maybe the ability to store the stream in an MP4 container instead of (or as well as) on AVI.

I realize that this codec was originally concieved for use with cellular phone networks. So with that in mind I would have thought being able to shove the video stream in an MP4 container would have been more practical!

Or am I really flipping out now???

Cheers

Assault · 11th November 2003, 14:14

@ SeeMoreDigital

AFAIK VP6 is no mpeg4 codec. I don't think that it's possible to put non mpeg4 compliant codecs in an MP4 container. And even if it were possible it wouldn't be a good idea in my opinion because the resulting MP4 wouldn't be spec compliant.

Assault

CruNcher · 11th November 2003, 15:44

@ SeeMoreDigital
i think Christian is allready lurking arround and allready talks with the other Matroska guys howto support it in MKV aswell hehe im sure about that why ??? because i know him

Sagittaire · 11th November 2003, 17:36

Update: VP6 setting and file size

vinouz · 11th November 2003, 20:58

So please could someone explain (Sagittaire ?) what the difference between PSNR and PSNROverall is ? Do that mean that on PSNR you take the 1324th frame's PSNR ?

As I'm the third to ask and no one seemend to answer, I think it would be glad all the others (who surely understood, if they discussed the results without asking what they mean) or Sagittaire make a little step down the stairs to educate at least three ordinary folks.
If the results weren't that different between first and second graph I wouldn't have been much worried about my lack of comprehension...

Thanks in advance,
Vinouz.

C0mPr355 · 11th November 2003, 21:12

@ Sagi

I believe the best results for psnr you can get on vp6 should have the post processing level set to "Best PSNR." This can be done in Start>settings>control panel>vp6 decoder settings.

also could you retry your test with an auto keyframe of 360 rather than 120? I have stated that 360 is a much better number to use because of the way that vp6 generates keyframes.

Please only use Spatial and/or Temporal resampling when you are unable to hit the datarate on much lower bitrate clips. A datarate of 500-1000 really should not have a need to drop frames or dynamically downsize the encoding resolution to hit the datarate.

Assault · 11th November 2003, 21:51

@ vinouz

I'll try to explain the difference between Overall PSNR and Average PSNR as far a I understand it. The average PSNR (first graph) is calculated by the average PSNR values (as the name already says

)of the single frames. This means for the final result it doesn't matter how constant the quality of the video is. As an example you can take three frames. Codec A delivers a result of 10dB for the first frame 40dB for the second and 70dB for the third. Codec B delivers 39dB, 40dB and 41dB. The average PSNR for those two codecs will be exactly the same. Nevertheless in "real life" you would certainly prefer the quality of codec B because nobody wants a frame with 10dB in his encode.

Overall PSNR on the other hand takes into account how constant the quality of the video is. But please don't ask for the exact formula of Overall PSNR. It's quite complicated (You can search in the Avisynth forum when you are interested).
So IMHO the Overall PSNR is preferable because it's the aim of the 2pass mode to deliver as constant quality as possible.
I hope that helps.

Assault

Sagittaire · 11th November 2003, 22:34

Post-Process decrease PSNR and "Best PSNR" too ... for my trailer in 500~1000 Kbps interval

keyframe maxi 360 or 120 frames obtain the same PSNR because the encoder himself choose the keyframe position and frequency for bframe is generaly inferior at 360 frames ...

The VP6 is really bad with the PSNR Overall and really good with¨PSNR because it encode very well the not very complex frames which are in a majority and very badly the complex frames which are minority. Many blocking and ringing in the fast scenes which are cleaned with a very good Post-Process. The VP6 is thus not a good codec to make true VBR ...

vinouz · 12th November 2003, 00:43

@Assault : thanks for the explanation.

MfA · 12th November 2003, 02:34

"~~ PSNR, SSIM, VQM Test ~~"

?

C0mPr355 · 14th November 2003, 20:39

@ Sagi

can you please post the filters you used for Avisynth to resize the image as well as where the source is from (the dvd ?).

ty

10th November 2003, 15:31	#1 \| Link
Sagittaire Testeur de codecs Join Date: May 2003 Location: France Posts: 2,067	[Test Codec] H264, VP6, RV10, WMV9, DivX3, XviD, DivX5 ..... *Matrix II 720304** Source DVD PAL 720576 MPEG2 MP@ML, chapter 20 to 26, ~35 000 frames Test in french ... but Capture, Sample & Graphs are universal language Codecs* - H264 Ateme - XviD 1.0.1 - RV10 Elysean - VP6 6.2 - DivX 5.2.1 - WMV9 VCM - DivX 3 ffvfw Tests - 500-2000 Kbps and 25-100% compressibility (/quant2 XviD H263) - SSIM, Average PSNR, Overall PSNR and Frames PSNR test - Blind test - Samples - Diaporama __________________ Le Sagittaire ... ;-) 1- Ateme AVC or x264 2- VP7 or RV10 only for anime 3- XviD, DivX or WMV9 Last edited by Sagittaire; 11th October 2004 at 18:45.

10th November 2003, 15:35	#2 \| Link
bilu Registered User Join Date: Oct 2002 Location: Portugal Posts: 1,182	What's the criteria difference between the two images? One is called PSNR and the other PSNR Overall... EDIT: does that trailer have high-motion parts? Bilu Last edited by bilu; 10th November 2003 at 15:46.

10th November 2003, 17:09	#3 \| Link
Sirber Registered User Join Date: Oct 2002 Location: Canada Posts: 8,776	Nice comparison __________________ Detritus Software

10th November 2003, 17:15	#4 \| Link
DAvenger hacked Join Date: Nov 2001 Location: Bratislava, Slovak Republic Posts: 440	Ah, could you please explain the results to us uneducated plebs? Thanks __________________ Pokec o sexe, láske a vzťahoch : http://www.mojvztah.sk Spyware, malware a vírusy : http://www.antispyware.sk Hudba, texty, akordy, noty, albumy, mp3 a videoklipy : http://www.musicworld.sk

10th November 2003, 21:33	#6 \| Link
Assault Registered User Join Date: Nov 2002 Location: Germany Posts: 180	@ Sagittaire Did you use XviD with GMC and Trellis? And for your b-frame settings I would recommend 1/150/100. I think this should deliver a little better results but altogether it's a very nice comparison. Thanks @ all When you look at the two graphs it's very interesting to see that VP6 is a good codec but has a bad 2-pass algorithm. Its results are much better for average psnr than for overall psnr. I think that's because average psnr doesn't take into account how constant the quality is. So I suspect that VP6 has both frames with VERY good quality and frames with VERY bad quality. That should be the reason why the results for overall psnr arent really good. If On2 improves the 2-pass mode VP6 will be a very good codec in my opinion. Assault Last edited by Assault; 17th October 2004 at 21:03.

10th November 2003, 18:19	#5 \| Link
TheXung Registered User Join Date: Apr 2002 Posts: 201	Could you use more differing colors for xvid and vp6 or at least make the lines thicker than a pixel.

11th November 2003, 00:54	#7 \| Link
CruNcher Registered User Join Date: Apr 2002 Location: Germany Posts: 2,738	@Sagittaire please post more infos filesize and time needed to encode __________________ all my compares are riddles so please try to decipher them yourselves :) current Main XviD PSNRLab version 1.1 current XviD PSNRLab update 1.1.3 MT1389xx Player Infos XviD EDP Build CVS (CruNchers Visual System ©) V 1.4 new with predefined profiles ;) Code: Olivera: "We're expendable assets" "and we've just been expended" "It's not a trick it's a Sony Rootkit" pwnz :devil: :angry: :rolleyes:

11th November 2003, 11:56	#9 \| Link
Assault Registered User Join Date: Nov 2002 Location: Germany Posts: 180	@ SeeMoreDigital I don't think that you're doing something wrong when creating 2pass encodes. I wasn't content with the results I got in my tests either. I also remember C0mPr355 saying that VP6 is their first codec with a pass mode. So perhaps that's the reason for the bad results. Concerning DivX I think you're right. It's probably a typo. Assault

11th November 2003, 13:03	#10 \| Link
SeeMoreDigital Life looks better in HDTV Join Date: Jun 2003 Location: UK Posts: 10,026	Thanks for that Assault. I thought I was begining to flip out there for a second - twice! Yesterday I did some more low bitrate CBR tests with VP6 using 720x576 image pixel frame sizes. It really is very good! However, I personally would like to be able to enter kb's instead of KB's per second. And maybe the ability to store the stream in an MP4 container instead of (or as well as) on AVI. I realize that this codec was originally concieved for use with cellular phone networks. So with that in mind I would have thought being able to shove the video stream in an MP4 container would have been more practical! Or am I really flipping out now??? Cheers __________________ Still confused about anamorphic images? Then look here! \| Main Encoding Interests: Hardware device playback \| My Gear \|

11th November 2003, 14:14	#11 \| Link
Assault Registered User Join Date: Nov 2002 Location: Germany Posts: 180	@ SeeMoreDigital AFAIK VP6 is no mpeg4 codec. I don't think that it's possible to put non mpeg4 compliant codecs in an MP4 container. And even if it were possible it wouldn't be a good idea in my opinion because the resulting MP4 wouldn't be spec compliant. Assault

11th November 2003, 15:44	#12 \| Link
CruNcher Registered User Join Date: Apr 2002 Location: Germany Posts: 2,738	@ SeeMoreDigital i think Christian is allready lurking arround and allready talks with the other Matroska guys howto support it in MKV aswell hehe im sure about that why ??? because i know him __________________ all my compares are riddles so please try to decipher them yourselves :) current Main XviD PSNRLab version 1.1 current XviD PSNRLab update 1.1.3 MT1389xx Player Infos XviD EDP Build CVS (CruNchers Visual System ©) V 1.4 new with predefined profiles ;) Code: Olivera: "We're expendable assets" "and we've just been expended" "It's not a trick it's a Sony Rootkit" pwnz :devil: :angry: :rolleyes:

11th November 2003, 17:36	#13 \| Link
Sagittaire Testeur de codecs Join Date: May 2003 Location: France Posts: 2,067	Update: VP6 setting and file size __________________ Le Sagittaire ... ;-) 1- Ateme AVC or x264 2- VP7 or RV10 only for anime 3- XviD, DivX or WMV9

11th November 2003, 20:58	#14 \| Link
vinouz Grüt Power Maintainer Join Date: Mar 2002 Location: france Posts: 111	So please could someone explain (Sagittaire ?) what the difference between PSNR and PSNROverall is ? Do that mean that on PSNR you take the 1324th frame's PSNR ? As I'm the third to ask and no one seemend to answer, I think it would be glad all the others (who surely understood, if they discussed the results without asking what they mean) or Sagittaire make a little step down the stairs to educate at least three ordinary folks. If the results weren't that different between first and second graph I wouldn't have been much worried about my lack of comprehension... Thanks in advance, Vinouz. __________________ A psychotic thinks that 2 + 2 = 5. A neurotic knows that 2 + 2 = 4... but it worries him!

11th November 2003, 21:12	#15 \| Link
C0mPr355 On2 dungeon crawler Join Date: Oct 2001 Location: NY Posts: 78	@ Sagi I believe the best results for psnr you can get on vp6 should have the post processing level set to "Best PSNR." This can be done in Start>settings>control panel>vp6 decoder settings. also could you retry your test with an auto keyframe of 360 rather than 120? I have stated that 360 is a much better number to use because of the way that vp6 generates keyframes. Please only use Spatial and/or Temporal resampling when you are unable to hit the datarate on much lower bitrate clips. A datarate of 500-1000 really should not have a need to drop frames or dynamically downsize the encoding resolution to hit the datarate. __________________ I know nuhtzing!

11th November 2003, 21:51	#16 \| Link
Assault Registered User Join Date: Nov 2002 Location: Germany Posts: 180	@ vinouz I'll try to explain the difference between Overall PSNR and Average PSNR as far a I understand it. The average PSNR (first graph) is calculated by the average PSNR values (as the name already says )of the single frames. This means for the final result it doesn't matter how constant the quality of the video is. As an example you can take three frames. Codec A delivers a result of 10dB for the first frame 40dB for the second and 70dB for the third. Codec B delivers 39dB, 40dB and 41dB. The average PSNR for those two codecs will be exactly the same. Nevertheless in "real life" you would certainly prefer the quality of codec B because nobody wants a frame with 10dB in his encode. Overall PSNR on the other hand takes into account how constant the quality of the video is. But please don't ask for the exact formula of Overall PSNR. It's quite complicated (You can search in the Avisynth forum when you are interested). So IMHO the Overall PSNR is preferable because it's the aim of the 2pass mode to deliver as constant quality as possible. I hope that helps. Assault

11th November 2003, 22:34	#17 \| Link
Sagittaire Testeur de codecs Join Date: May 2003 Location: France Posts: 2,067	Post-Process decrease PSNR and "Best PSNR" too ... for my trailer in 500~1000 Kbps interval keyframe maxi 360 or 120 frames obtain the same PSNR because the encoder himself choose the keyframe position and frequency for bframe is generaly inferior at 360 frames ... The VP6 is really bad with the PSNR Overall and really good with¨PSNR because it encode very well the not very complex frames which are in a majority and very badly the complex frames which are minority. Many blocking and ringing in the fast scenes which are cleaned with a very good Post-Process. The VP6 is thus not a good codec to make true VBR ... __________________ Le Sagittaire ... ;-) 1- Ateme AVC or x264 2- VP7 or RV10 only for anime 3- XviD, DivX or WMV9 Last edited by Sagittaire; 11th November 2003 at 23:41.

12th November 2003, 00:43	#18 \| Link
vinouz Grüt Power Maintainer Join Date: Mar 2002 Location: france Posts: 111	@Assault : thanks for the explanation. __________________ A psychotic thinks that 2 + 2 = 5. A neurotic knows that 2 + 2 = 4... but it worries him!

12th November 2003, 02:34	#19 \| Link
MfA Registered User Join Date: Mar 2002 Posts: 1,045	"~~ PSNR, SSIM, VQM Test ~~" ?

14th November 2003, 20:39	#20 \| Link
C0mPr355 On2 dungeon crawler Join Date: Oct 2001 Location: NY Posts: 78	@ Sagi can you please post the filters you used for Avisynth to resize the image as well as where the source is from (the dvd ?). ty __________________ I know nuhtzing!

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode