XviD 1.0 vs all other codec ... ;-)

[Sagittaire]

Test in french ...
... but graphs are universal language


- XviD 1.0, RV10, VP6, H264, WMV9
- 500-2000 Kbps for very low and very high compressibility encoding
- SSIM and PSNR test
- Samples (30 sec)
- Captures

[virus]

hey, that's what I'd call a professional comparison!

What to say? I still believe that the MPEG quant. matrix for XviD would have performed better visually (maybe not for SSIM). Nevertheless, RV10 comes out on top in great style here. Time for a new Doom9 comparison?

thx for your work!
(BTW am I blind or you've totally ignored DivX? Not even mentioned in the "other codecs" section...)

cheers
virus

[Neo Neko]

I think a new comparrison is still a way off. With h264 codecs starting to pop up and become usefull, the release of Xvid 1.0, and recent Real Media developments a new comparrison is on the horizon. I think one more codec development will be required before a new test is started.

As to this test it was a bit to heavy on the artificial metrics and way to light on the visual tests. 2 shots from one or two films is hardly a decent crossection. And from the meger image selection I actually thought the h264, and VP6 shots on the 500kbps "image" looked better than RM10. [YMMV] But it still goes to show that artificial metrics are no substitute for the human eye.

[Teegedeck]

About SSIM see this post by Soulhunter. For SSIM h.263 quant looked better than an HQ custom-matrix at quant=2...
And we already knew that PSNR is pretty worthless in order to judge perceived quality. Better use your eyes, after the only (probably) valid testing-tool (JNDmetrix-IQ, c't used it) isn't downloadable for testing, anymore.

[Sagittaire]

make a test only with captures is difficult ...



Capture with Maxi difference peak PSNR for WMV9: advantage WMV9
Capture with Maxi difference peak PSNR for XviD: advantage XviD

make captures that supposes that Rate Control is identical for all codecs. the captures can also very different because of the I,P and Bframes ...

the best solution is to post small sample for visual test ... thus downlaod sample if you don't like metric tests.

[Teegedeck]

Samples would be great. No offense, but metric tests may be useful for analyzing isolated aspect of codec behaviour but not for an overall judgement of quality. Except - maybe - JNDmetrics. But even if JNDmetrics imitates human visual perception for its analysis, it only comes close, and it is our perception what this is all about.

PSNR likes a smooth picture, most viewers' eyes don't - so testing PSNR is not useful IMHO.

[Sagittaire]

SSIM is a very good quality metric ... Soulhunter test isn't a very good test ...

Originally posted by Sagittaire
Make test with animate it is not a good idea: visually you will have much difficulty to see the difference between matrix then how SSIM test could do it. 90 for SSIM is a very high value: it's impossible to make the difference visually between encoding with 90 and higher for SSIM ...

a small test with HPII trailer in XviD quant2 ...

andreas_78er matrix
35 586 Ko
SSIM=83.64

H263
28 516 Ko
SSIM=82.53

Make the test with a real movie ...

[Doom9]

Guess I have the read the changelogs again.. have there even been visual improvements since the build I've tested last time (when XviD is concerned).
I'm still waiting for a bunch of H.264 encoders.. there are a few in the pipeline that might just appear in an upcoming comparison but right now I think it's too early to get at it again. I even considered quality metrix but then when I got a bunch of codec settings optimized for quality metrics, that was the death of such an idea.

[SeeMoreDigital]

Personally I would rather trust my eyes than a bunch of graphs!

And as I've said many times before, why don't we make these codecs work a lot harder by generating some 'comp tests' using larger image pixel frame sizes!

For example, if you are encoding from an 2.35:1 NTSC source, lets have some 720x368 cropped 'anamorphicly framed' encodes or better still some 864x368 cropped 'true framed' encodes.

Similarly, if you are encoding from an 2.35:1 PAL source, lets have some 720x432 cropped 'anamorphicly framed' encodes or some 1024x432 cropped 'true framed' encodes.

Soulhunter has already shown (much better than I ever did) that 'astonishing' levels of detail are capable with XviD using such frame sizes. So lets see how well the other codecs compare.


Cheers

[SeeMoreDigital]

Originally posted by Doom9
Guess I have the read the changelogs again.. have there even been visual improvements since the build I've tested last time (when XviD is concerned).
I'm still waiting for a bunch of H.264 encoders.. there are a few in the pipeline that might just appear in an upcoming comparison but right now I think it's too early to get at it again. I even considered quality metrix but then when I got a bunch of codec settings optimized for quality metrics, that was the death of such an idea.

Hi Doom9,

Did you manage to take a look at the H.264 encode Tommy Carrot posted the other day http://forum.doom9.org/showthread.ph...454#post497454 The darn thing only ran at 253kbps!


Cheers

[Lefungus]

Objective quality metrics are just tools, and such, can be misused. I believe they're the most useful when you try to improve one codec performance and not when comparing various codecs.
About JNDMetrix-IQ (Sarnoff), this and that have shown ssim (basic version: ie what the current filter do) to be better on the VQEG set of tests, meaning it has better correlation to human visual system.
Now it would be a real treat if someone is up to the task of adding latest improvements in ssim, that is multi-scale ssim, and motion compensation.

[Doom9]

Did you manage to take a look at the H.264 encode Tommy Carrot posted the other day

So far I've yet to look at a single H.264 file. I have a couple of files around, codecs/tools to test and codecs I'm supposed to get for testing, but so far I haven't done anything. I'm seriously contemplating upping the ante though and use larger framesizes next time to make things a bit rougher on the codecs. If only codecs get better and the test remains the same, eventually there just won't be any differences that I can spot. As with PC hardware testing, where eventually newer generation games and benchmarking tools are used, I think 2004 will mark an overhaul of my testing methods. But all that is still a few months away.

[amango]

http://jfl1974.free.fr/Video/Sample-XviD-2000.avi
http://jfl1974.free.fr/Video/Sample-WMV9-2000.avi

If you compare those samples, you'll see that WMV9 does really have problems with the background. XVID just produces a fine und still picture with much less visible wandering pixels.

[SeeMoreDigital]

Thanks amango,

However, I thought Gladiator was a 2.35:1 movie. Why are your encodes 1.82:1 (1.85:1)?


Cheers

[Sagittaire]

@ amango

Use PP=1 for best quality with WMV9
http://jfl1974.free.fr/Download/PP-WMV9.reg
For high resolution/high bitrate use PP=0

Use PP=4 for best quality with VP6
http://jfl1974.free.fr/Download/PP-VP6.reg
For high resolution/high bitrate use PP=0

@ SMD

Source HDTV 1080i 16/9 ...

# AviSynth 2.55 #

LoadPlugin("C:\PROGRA~1\Autorv9\SOFTS\AVSFILE\MPEG2DEC3.DLL")
LoadPlugin("C:\PROGRA~1\Autorv9\SOFTS\AVSFILE\DECOMB.DLL")

Video=Mpeg2Source("D:\Ripp\Gladiator\azerty.d2v", idct=2, ipp=true )
Video=Telecide(Video,post=true,chroma=true,gthresh=50,guide=1)
Video=Decimate(Video,cycle=5)

Video=trim(Video,15244,28001)
Video=Crop(Video,0,0,-8,-0)
Video=LanczosResize(Video,640,352)
Video=AssumeFPS(video,25)
return video

[SeeMoreDigital]

Originally posted by Sagittaire
@ SMD

Source HDTV 1080i 16/9 ...

Is this source still available for download? If so I might generate and post some of my own test files.


Cheers

[virus]

about the "time for a new Doom9 comparison?" topic...

XviD: I don't have XviD beta3 clips anymore, so I cannot judge if the overall quality has been improved since then (my guess is: yes )
VP6: this is worth a comparison in its current state, that's for sure. The last comparison featured version 6.0.8.3 IIRC. Three different versions came out since then, with the latest 6.2.0.10 being a nice jump ahead in flexibility/quality.
RV10: can't say too much, but reading some posts by Karl about the improvements and some impressions by the userbase, I'd say it's ready for comparison.
H.264: well, x264 (ffdshow) is still unusable. We simply cannot ask "Doom9, please encode your movies 10 times till you find the right constant quantizer to match the size". Will be ready when 2-pass is implemented, since quality-wise it's already worth a look.
As for Ahead's upcoming implementation, I don't know. I don't think we are going to see a _really_ good H.264 implementation before late 2004, though. There are so many degrees of freedom in H.264, so tuning these codecs is going to be a nightmare

IMHO we should not forget that DivX 5.2 is coming (last time I took a look at their forums at divx.com they said they are in track with their time schedule --> June). Waiting for it is mandatory IMHO.

just my 2 eurocents
virus

[bond]

Sagittaire,
why did you use 3 b-frames, no qpel, h.263 and pp4 for xvid?

[Sagittaire]

Max consecutive Bframe 3

in fact the coder places only one or two bframe to the maximum and rarely three. I put 3 bframe to let the coder choose the best number of bframe. That doesn't modify the PSNR and the SSIM ...



Qpel
PSNR, SSIM and for me quality degradation for low bitrate

H263
best matrix for SSIM and PSNR

PP4
best quality for very low bitrate ... without XviD blocks everywhere at 500 Kbps and 25% for compressibility ... download sample.

[SeeMoreDigital]

Well, now that it's been brought to my attention that the source was HiDef (1080i 1.77:1). I really don't want to be negative but I personally can't see the point of generating an 640x352 encode in the first place.

Anyway, my reasons for thinking like this have already been posted here: - http://forum.doom9.org/showthread.ph...054#post499054


Cheers