Small XviD and VP6.2 test [due to VP62 quality fluctuations]

[pernas]

I've documented (hopefully) the VP62 quality fluctuations that Tommy was speaking about in Sagittaire's [New Test] XviD vs Other Codec ... ;-)

I'd like to add that i made this test after noting some strange things in my own (not very) scientific search for a good codec.

Details below:

The source was the first (DVD)chapter of Gladiator scaled to 672x288 @23.976fps

For XviD 1.0.1 the following settings were used;
Defaults + vhq4+ turbo + qpel + gmc + trellis + MPEG mtrx + 2pass 1100Kbit/s
Decoding done with xvid (PP: Y+UV deblocking)

For VP6.2 the following settings were used;
Default VP62HSP + 7 sharpness + 2pass best quality @1100Kbit/s
Decoding done with PP4

Results:

Reference;
http://valborg.hn.org/raw_200.png

XviD;
http://valborg.hn.org/xvid_200.png

VP62;
http://valborg.hn.org/vp62_200.png

Samples;
XviD (http://valborg.hn.org/xvid_500f_sample.avi)
VP62 (http://valborg.hn.org/vp62_500f_sample.avi)

According to above my personal choice at this bitrate would be XviD as i prefer (real) sharpness and some minor blocking over non-predictable lack of details and fuzziness.

But that's MHO :rolleyes:

Regards
Per

[Sagittaire]

Perhaps the problem comes from the Rate Control: Perhaps VP6 takes less bit for this part ... The RC for the VP6 and XviD is not identical and XviD could be better on short part -> ScreenShoot doesn't prove anything


Post small sample ... please

- We can see the local differences for each codec
- We can see the size for each sample and see the influance of RC
- We can see info (quant, size, bitrate, frame type ...) with XviD and ffdshow

[pernas]

They cause sudden sharpening in the video, which can be quite bothering.
Hopefully you meant sudden softening or did a misunderstand you ?

Small samples on their way :)

Regards
Per

[pernas]

XviD and VP62 samples

XviD (http://valborg.hn.org/xvid_500f_sample.avi)
VP62 (http://valborg.hn.org/vp62_500f_sample.avi)

Regards,
Per

[Sagittaire]

These spikes are not scene-changes, but frames with increased quality in pans and slow scenes. They cause sudden sharpening in the video, which can be quite bothering. And this is not a bug, the developers said it's all intentional to bring back the details.

Exactly the same observation for Bframe and XviD in IPBBP sequence with bframe 2/1.50/1.00 ... Pframe are very better than Bframe



Mean Max Max
Absolute Mean Pos. Neg.
Frame Dev. Dev. Dev. Dev. PSNR (dB)
-----------------------------------------------------

35107 0.6955 +0.0025 12 -13 46.5627 P
35108 0.7718 +0.0027 19 -20 45.1428 B
35109 0.7694 +0.0169 15 -22 45.4611 B
35110 0.6718 +0.0405 12 -14 46.7728 P
35111 0.7319 +0.0031 18 -26 45.5608 B
35112 0.7689 +0.0063 18 -28 45.4177 B
35113 0.6403 +0.0366 12 -18 47.0537 P
35114 0.6855 +0.0356 27 -21 45.9458 B
35115 0.6176 -0.0219 25 -27 46.4438 B
35116 0.6124 +0.0582 13 -15 47.2732 P
35117 0.6488 +0.0015 20 -20 46.7304 B
35118 0.6274 +0.0350 22 -21 46.7175 B
35119 0.5874 +0.0436 15 -15 47.7563 P

[Sagittaire]

Samples;
XviD
VP62

According to above my personal choice at this bitrate would be XviD as i prefer (real) sharpness and some minor blocking over non-predictable lack of details and fuzziness.


I am convinced... it's true for detail/sharpness

But it's a very high compressibility level (q2 and q3 for I,P compressibility > 75%) and all codec are very good for this compressibility ... if you want sharp DivX3 or RV10 could be better with compressibility > 75%

- High Compressibility 75%: Really difficult to compare and it's really subjective: RV10, VP6, XviD, WMV9 then DivX3

If your favorite codec for high compressibility is XviD ... why not ... it's a good choice and all the choise are good ... IMHO my fovorite is RV10.

[stephanV]

i must say that i both dont find them *really* good encodes...

i favor XviD a little bit though because with VP6 the guys beard and the area between the eyes constantly keeps 'moving'. its very annoying to watch, its like crazy ants or something ;) .

BTW - PSNR metrics dont mean anything when it comes to perceptual quality. here's a nice review (http://www.everwicked.com/content/QualityMetrics/index.php) on objective and subjective quality measurements.

[Sagittaire]

For me SSIM is better than Overall PSNR and very better than Average PSNR.
SSIM is an objective video quality metric (http://www.cns.nyu.edu/~zwang/) ...

SSIM is the metric which approaches the most my visual impressions

my "SSIM test"

http://jfl1974.free.fr/Test/Comparatif/SSIM.gif



and my "blind test"

- Low Compressibility 25%: VP6, RV10 or WMV9 are very better than XviD or DivX3.

- Medium Compressibility 50%: VP6 and RV10, then WMV9 and XviD but differences are small except for DivX3 ...

- Medium Compressibility 75%: Really difficult to compare and it's really subjective: RV10, VP6, XviD, WMV9 then DivX3

- High Compressibility 100%: Too much difficult to compare ...

-> make test with low and medium compressibility (1CD encoding) ... for high and very high compressibility it's really subjective

[stephanV]

last about this, before we go off-topic:

SSIM is still an image based comparison. Video is a series of images and therefor should not be judged with such a metric. In that link you gave it is explicitely used on JPEG compression, not on video.

[Sagittaire]

SSIM is still an image based comparison. Video is a series of images and therefor should not be judged with such a metric. In that link you gave it is explicitely used on JPEG compression, not on video.

http://www.cns.nyu.edu/~zwang/files/papers/vssim.pdf

[stephanV]

from the article (conclusions)

In order to improve the proposed algorithm, many other issues also need further investigations in the future. One important issue is related to motion. The current SSIM index is oriented for comparison of still image structures. Notice that there are several significant outliers in the scatter plots of the proposed algorithms (in the lower-left parts of Figs. 5(c) and 5(d), where the models give much lower scores than they should supply). In fact, most of these significant outliers corresponds to the video sequences with large global motions (such as SRC5, SRC9 and SRC19 in the VQEG Phase I test dataset).

thx for making my point :)

[Sagittaire]

SSIM isn't a perfect test ... but very better than PSNR for exemple.

If for first video sequence SSIM = 65 and for second video sequence SSIM = 70 then second sequence is always very better than first sequence

If for first video sequence SSIM = 80.5 and for second video sequence SSIM = 79.5 then a conclusion is difficult


PSNR isn't a perfet test ... but it's a very good test for codec improuvements and Rate Contol observation ...

for exemple: HP II 640*272 ~3000 frames

VHQ4, Ultra High, GMC, Qpel, AQ, Trelli, q2 for I,P et B

Bframe 0/1.00/0.00 et sensivity=0: 33 374 Ko
Bframe 1/1.00/0.00 et sensivity=0: 33 180 Ko
Bframe 2/1.00/0.00 et sensivity=0: 33 264 Ko
Bframe 3/1.00/0.00 et sensivity=0: 33 260 Ko
Bframe 1/1.00/0.00 et sensivity=100: 33 322 Ko

but 2 pass with Bframe 2/1.50/1.00 is very better for PSNR than 2 pass with Bframe 2/1.00/0.00 -> visual quality for 2/1.50/1.00 is better than 2/1.00/0.00


Between 25% and 50% compressibility XviD is really worse than VP6/RV10/WMV9. VP6/RV10/WMV9 could be very better for "low compressibility source" (~150 min 1CD or very high motion movie 1CD)

Between 50% and 100% for compressibility all codec are good and for very high compressibility DivX3 is a good choise. It's useless to want to compare the codec in these compressibility zones: it's too subjective (~100 min 1CD or medium motion movie 1CD)

[pernas]

Originally posted by stephanV
i must say that i both dont find them *really* good encodes...

i favor XviD a little bit though because with VP6 the guys beard and the area between the eyes constantly keeps 'moving'. its very annoying to watch, its like crazy ants or something ;) .

BTW - PSNR metrics dont mean anything when it comes to perceptual quality. here's a nice review (http://www.everwicked.com/content/QualityMetrics/index.php) on objective and subjective quality measurements.

Maybe some small amount of pre-filtering could have helped a bit ;)
I agree on the "crazy ants" - the (VP62)picture seems to be in constant motion most of the time - very annoying...

A note regarding PSNR, SSIM etc. - the input of an human eye is really hard to measure but Feng Xiao's Video Quality Metric (VQM) looks interesting though :rolleyes:

[Sagittaire]

To conclude download these sample

VP6 25% (http://jfl1974.free.fr/Video/Sample-VP6-500.avi)
RV10 25% (http://jfl1974.free.fr/Video/Sample-RV10-500.rmvb)
WMV9 25% (http://jfl1974.free.fr/Video/Sample-WMV9-500.avi)
XviD 25% (http://jfl1974.free.fr/Video/Sample-XviD-500.avi)
DivX3 25% (http://jfl1974.free.fr/Video/Sample-DivX3-500.avi)

VP6 50% (http://jfl1974.free.fr/Video/Sample-VP6-1000.avi)
RV10 50% (http://jfl1974.free.fr/Video/Sample-RV10-1000.rmvb)
WMV9 50% (http://jfl1974.free.fr/Video/Sample-WMV9-1000.avi)
XviD 50% (http://jfl1974.free.fr/Video/Sample-XviD-1000.avi)
DivX3 50% (http://jfl1974.free.fr/Video/Sample-DivX3-1000.avi)


Not need to make test SSIM, stop image or zoom screenshoot ... :devil:

[SeeMoreDigital]

Both encodes look very good when outputted to an 32" widescreen TV using RGB.

It would have been better if you had encoded something a little brighter and faster but they're OK.

Something is not quite right with the bottom matte though. It's showing a couple of 'rolled' pixel lines. You can't see it so much on an PC monitor but you can on an TV.


Cheers

[Mario Bros]

I must say that using postprocessing in the test is not very clever because even there are no artifacts or blocking PP turned on will blur the image. I suggest turning it OFF for comparison.

[Sagittaire]

VP6 25% (http://jfl1974.free.fr/Video/Sample-VP6-500.avi)
RV10 25% (http://jfl1974.free.fr/Video/Sample-RV10-500.rmvb)
WMV9 25% (http://jfl1974.free.fr/Video/Sample-WMV9-500.avi)
XviD 25% (http://jfl1974.free.fr/Video/Sample-XviD-500.avi)
DivX3 25% (http://jfl1974.free.fr/Video/Sample-DivX3-500.avi)

I must say that using postprocessing in the test is not very clever because even there are no artifacts or blocking PP turned on will blur the image. I suggest turning it OFF for comparison.


there are no artifacts with XviD PP4 or DivX3 PP4 ... ???
I suggest turning it OFF for comparison. ... ???

Finally the problem must come from my eyes ... lol.

[pernas]

Originally posted by Mario Bros
I must say that using postprocessing in the test is not very clever because even there are no artifacts or blocking PP turned on will blur the image. I suggest turning it OFF for comparison.

I simply used the settings most pleasing for my eyes while watching :)

Regards

[Sharktooth]

pernas could you post the source plz?

[Bulletproof]

VP6 can do quite well in some situations, I hope the developers haven't stopped tuning/updating it because it has really good potential. VP6 can beat XviD with PP off in some situations and with PP on it has a really big advantage, personally I think VP6 has one of the best PP engines cause it doesn't make the image blurry and pastey, and yes I think it's better than RV's. You can see in this thread http://forum.doom9.org/showthread.php?s=&threadid=77014 and where VP6 beats H.264 and XviD, also take a look at http://www.pbase.com/dfein/inbox and yes I did everything proper. VP6 can be sharper as well where as XviD has to use a custom matrix that increases filesize, VP6's sharpness settings do not increase filesize. XviD does beat VP6 however with 2-pass rate control and it's many features which VP6 does not have, these file saving/picture saving methods allow for the bitrate to be used more efficiently and allows more for more bits to be saved and thus more headroom if needed.

[pernas]

Originally posted by Sharktooth
pernas could you post the source plz?
The 500 frames uncompressed or the whole first chapter (huge but no problems) ?

Here it is: Raw_500f_sample (http://valborg.hn.org/raw_500f_sample.avi)

[TheUnforgiven]

completely agree with bulletproof. especially on PP, really superb nothing come close. but xvid is my choice still.

[stephanV]

@pernas

thx for sample :)

you might wanna ace, rar or zip it though. saves some bandwith ;)

[Sharktooth]

Originally posted by pernas
The 500 frames uncompressed or the whole first chapter (huge but no problems) ?

Here it is: Raw_500f_sample (http://valborg.hn.org/raw_500f_sample.avi)
139MBytes. Ok... not a problem.
i asked you the source coz i already encoded the gladiator with VP6 and the result was much better than your and the bitrate was also lower.

EDIT: Eh, i was right:) but i used TemporalSoften coz the source has a lot of mosquito noise.
AVS:

Clip1=directshowsource("raw_500f_sample.avi").ConvertToYv12
Clip1=TemporalSoften(Clip1,2,5,5)
Return Limiter(Clip1)

The temporal denoiser included in VP6 is not good for that source but once cleaned with the previous avisynth script and encoded with sharpness 4 it looks much better.
I'll try to post a couple of screenshots later.

EDIT2: I investigated about the quality "fluctuations" and i discovered something interesting. The more the Sharpness setting is set to high the more the "fluctuations" are visible. Setting sharpness to low values makes them disappear or hardly visible.
I have to admit i usually dont set sharpness higher than 2 or 3.
Also they appear on low-motion scenes only.

[Sharktooth]

Ok, i have something for you to play with.
Here's the original (http://ebola.gamersrevolt.it/avp_int_exclusive_m480.mov) AVP trailer and the 700kbit VP62 + 64kbit AAC encoded (http://ebola.gamersrevolt.it/avp_int_exclusive_m480-vp62-aac.avi) one.
If someone is able to produce a 700kbit xvid encode that looks better than the VP62 one i'll eat my shoes.

[pernas]

@Sharktooth

Thanks for your advice :)

I will do some tests with filtering and also experiment with lower sharpening
What about the sharpness in your encoding between the "suddening soft passages" compared to my unfiltered ?

- looking forward to some screenshots :)

Regards

[Sharktooth]

It looks less grainy with much less noise even with PP set to best quality (the last 2 PP settings in VP6 decoder add noise to mask blocks or something...).
If i can get rid of my PC black screenshots sindrome i will post some of them...

[pernas]

Originally posted by Sharktooth
Ok, i have something for you to play with.
Here's the original (http://ebola.gamersrevolt.it/avp_int_exclusive_m480.mov) AVP trailer and the 700kbit VP62 + 64kbit AAC encoded (http://ebola.gamersrevolt.it/avp_int_exclusive_m480-vp62-aac.avi) one.
If someone is able to produce a 700kbit xvid encode that looks better than the VP62 one i'll eat my shoes.
How about this: Avp_int_XviD @ 700kbit (http://valborg.hn.org/avp_int_exclusive_m480-xvid700_2.avi)
(My 2nd try, the first went directly to NIL) :(

Will you eat your shoes now ? :p

[SeeMoreDigital]

Originally posted by pernas
Will you eat your shoes now ? :p Looks nice. Even on a big screen. White to black / black to white transitions look good with very little pixelation.

With so many scene changes it's difficult to say which one looks better because each codec performs stronger in different areas.

I think Sharktooth's shoes are safe for now... but he better get his knife sharpened... for one of them - As according to Gspot you have approx another 20Kbps to play with!


Cheers

[pernas]

Great :p

[Sagittaire]

Here's the original AVP
It's a very bad source ... with blocking ... !!!

[Sharktooth]

Originally posted by Sagittaire
It's a very bad source ... with blocking ... !!!
Yep, and VP6 shows how good is its PP removing most of them.

[pernas]

Originally posted by Sagittaire
It's a very bad source ... with blocking ... !!!
I agree, what about your shoes Sharktooth? ;)

[Sharktooth]

My shoes are still there... the xvid encode is too blocky:)

[Tommy Carrot]

I don't know... The quality of the vp6 encode deteriorates quite badly at the very fast scenes. Xvid has more constant quality.

[Sharktooth]

vp6 encode = 1 pass and no filtering.
xvid encode = ?

[Tommy Carrot]

Originally posted by Sharktooth
vp6 encode = 1 pass and no filtering.
xvid encode = ?

So what, that's not an excuse, now you have to eat your shoes. :D

[Sharktooth]

No way man:) my encode still looks better... and now there's a new AvP trailer... http://www.apple.com/trailers/fox/avp/trailer/large.html
This time i will do a better 2 pass and filtered encode...:)

[Sagittaire]

No possible comparaison with blocky source ... but for low compressibility encoding VP6 are very better than XviD ... the true power of a codec it's the capacity to encode with low compressibility ... make the test with another source if you want ...

VP6 25% (http://jfl1974.free.fr/Video/Sample-VP6-500.avi)
RV10 25% (http://jfl1974.free.fr/Video/Sample-RV10-500.rmvb)
WMV9 25% (http://jfl1974.free.fr/Video/Sample-WMV9-500.avi)
XviD 25% (http://jfl1974.free.fr/Video/Sample-XviD-500.avi)
DivX3 25% (http://jfl1974.free.fr/Video/Sample-DivX3-500.avi)


For high compressibility level all codec are good and DivX3 too. IMHO RV10 is the best but it's subjective and if somebody say "me I prefer XviD" ... why not

[Sharktooth]

This is not a test about absolute quality.
I've just made an encode of a bad quality source to show VP6.2 can handle those kind of sources mantaining a respectable quality (some times it is able to compensate the original artifacts too...) even at low bitrates while MPEG4 codecs show their best only with better quality sources.
This indicates that every codec has its pro and cons and every source should be tested with different codecs to say "ok, this one is better than that one".

[pernas]

Originally posted by Sharktooth
vp6 encode = 1 pass and no filtering.
xvid encode = ?
xvid encode = 2 pass defaults + vhq4 + turbo + q2-31 + qpel + CM + AQ + trellis + HVS best mtrx + a bit filtered with Convolution3D(0,3,4,3,4,3,0) also..

[Sharktooth]

Uhm... i think QPel is not good at that bitrate. Try enabling GMC and unset QPel and Turbo.
Also HVS-Best is not the "best"... :)
6of9-HVS (aka 6of9-24) "could" be better.

[pernas]

Oki, here's my 3rd attempt with settings as adviced by Sharktooth;

xvid encode = 2 pass defaults + vhq4 + q2-31 + GMC + CM + AQ + trellis + 6of9-24 mtrx + a bit filtered with Convolution3D(0,3,4,3,4,3,0) also..

Clip here: 3rd XviD 700kbit attempt (http://valborg.hn.org/avp_int_exclusive_m480-xvid700_3.avi)

And some screenshots;

Original MOV "blocky" source;
http://valborg.hn.org/mov_26.png

Sharktooth's VP62 (ShoeKeeper?) with "PP3";
http://valborg.hn.org/vp62_26.png

My XviD 2nd attempt (Qpel & HVS-best mtx);
http://valborg.hn.org/xvid700_2_26.png

My XviD 3rd attempt (GMC & 6of9-24 mtx);
http://valborg.hn.org/xvid700_3_26.png

@Sharktooth: your advices gave less blocks :)
Who's the judge ? :p

Regards

[Sharktooth]

The VP62 shots has more fine details.
Both xvid shots seems to be a bit washed in some areas.
However i'm working on a new source with lots of details and stuff... Animatrix Ep.1.

[Sharktooth]

Ok, as i said in the previous post i have new clips for a comparison ready.
The only problem is i cant take screenshots due to the weirdness of the video card BIOS, so i need ~70Mbytes of web space to upload the .avi files... Anyone? :)
Here's some details:

Source: The Animatrix Euro/PAL version.
Encoded Frames: From 395 to 4139 (Ep.1 - Final Flight Of The Osiris - Virtual katana fight)
Resolution: 1024x432 (mod 8 rounded anamorphic encode)
Bitrate: 2000kbps for XVID, 1953Kbps for VP6.2 (remember the 1024 thing?)

Settings for XVID 1.0.1: 2Pass VBR, Profile=Unrestricted, MSP6, VHQ4, GMC, AQ, CM, Turbo, FFP, Trellis, B-VOPS=2/1.50/1.00, KF interval=250, Min Q=1, Max Q=31, 6of9-HVS matrix, 1 zone with CO enabled, PAR set to 16:9. The remaining options are set to xvid defaults.

Settings for VP6.2: 2Pass VBR, NR=0, SH=7, Auto KF, KF interval=250, Undershoot=90%, Variability=100, Min section=20, Max section=800, Min Q=1, Max Q=31.

Avisynth script:

# Usual BS and plugins... set idct accordingly to your CPU type. 2 uses MMX/SSE.
LoadPlugin("F:\SOFTS\AVS\PLUGINS\MPEG2DEC3DG.DLL")
LoadPlugin("F:\SOFTS\AVS\PLUGINS\UNDOT.DLL")
Clip1=Mpeg2Source("N:\ANIMATRIX\VIDEO_TS\animatrix.d2v", idct=2 )
#
# Anamorphic cropping rounded to mod 8 res...
Clip1=Crop(Clip1,0,72,720,432)
#
# Filtering after cropping and before resizing for the fastest processing
Clip1=UnDot(Clip1)
#
# Resizing to 1024x432 to get full anamorphic pleasure even in players that do not support the embedded PAR
Clip1=LanczosResize(Clip1,1024,432)
#
# Guess what?
Return Limiter(Clip1)


Final size:
XVID 36.642.816 bytes
VP6 37.265.408 bytes

So, where's the stuff?
On my PC... i still need a host (about 70Mbytes) for uploading the files. Any help will be really appreciated:)

[pernas]

One moment...:p

[Sharktooth]

While uploading the clips i wish to share some impressions about the comparison.
I've seen both clips again and again and my impressions are VP6.2 has something weird in bits distribuition.
For example: keyframes are not necessarily the most detailed frames (dont know about the quantization but when there's a scene change you see a keyframe but details start to grow in later frames... for example open the vp6 encode with virtualdub and look at the frames 516 thru 545. 516 is a keyframe, but details start to grow in subsequent frames until frame 545 that is much more detailed than 516)
This weird behavior makes a keyframe comparison between vp6 and xvid perfectly useless. Also because xvid inserts much more keyframes than VP6 in motion scenes (higher sensitivity?).
Looking at random frames of the clips you will notice how much artifacts are present and how much details are lost in the XVID encode (most are blocks, plus some ringing on bright areas - some banding is there even in the source...) but the VP6 encode has almost none of them and you can clarly see the details even where XVID shows blocks.
That leads me to think VP6 bits distribuition (even with variability=100 !!!) produce a much consistent encoding sacrificing keyframes in favour of a "add details later" policy that IMHO shows its strenght in motion scenes and low bitrate encodings.
Now i'm proceding with a second VP6 encode with different parameters to see if what i described is a "general rule" or it depends on other parameters.

EDIT: even with sharpness=9, MinQ=2, MaxQ=54 etc... its the same story. So i suppose it is intended to behave in that way.


OK, here we go! Here's the clips:

XVID > http://valborg.hn.org/sharktooth/xvid-2000kbps.avi (~35Mb)
VP62 (first encode - Sharpness 7) > http://valborg.hn.org/sharktooth/vp6-2000kbp_SH7.avi (~35Mb)
VP62 (second encode - Sharpness 9) > http://valborg.hn.org/sharktooth/vp6-2000kbp_SH9.avi (~35Mb)

Thanks goes to Pernas for hosting the files! :)

[Tommy Carrot]

Originally posted by Sharktooth
That leads me to think VP6 bits distribuition (even with variability=100 !!!) produce a much consistent encoding

Nah man, it's quite the opposite. This is exactly my main beef with VP6, the quality is inconsistent, the sharpness and detail level is changing frequently, and i must admit i find it pretty annoying.

[Sharktooth]

Download the clips... :)
However with the word "consistent" i mean an overall consistency.

[pernas]

1st look...

Looking at the clips - they are cool :) but it's late and i don't trust my eyes
when it's dark so at first i did my (stupid :rolleyes: ?) ".PNG size test"
to check details (without considering extra space needed for blocking/ringing)
and it looks (again) to me that VP62 smooths a little bit to much from time to time...
-at least compared to XviD...

Good night!