Devapi3_vs_Devapi4andKauehi

[CruNcher]

@ all
ok after a long time i made a new compare with some settings and also a little kauehi compare included but didn't tested it fully :)


Builds
---------
Devapi3 = Koepi 24/06/03
Devapi4 = CVS 13/07/03


4208 kb headed 2pass size 4208/1024= 4,109375 mb = 584 kbps

Matrix = H.263

Simple Profile
-----------------------------------------------------------------------------------------------------
devapi3-1pass = 15,9 MB (16.756.224 bytes) PSNR = 45.6864, AVG Quant = 2
devapi3-2pass = 4,10 MB (4.306.432 bytes) PSNR = 40.2379, AVG Quant = 6.35

devapi4-1pass = 15,9 MB (16.768.000 bytes) PSNR = 45.6979, AVG Quant = 2
devapi4-2pass = 4,10 MB (4.305.920 bytes) PSNR = 40.2464, AVG Quant = 6.36

Kauehi-Standard-2pass same speed = 4,04 MB (4.237.312 bytes) PSNR = 40.3550, AVG Quant = 6.81
Kauehi-Very Slow-2pass 2x Slower = 4,04 MB (4.237.312 bytes) PSNR = 40.6189, AVG Quant = 6.58
-----------------------------------------------------------------------------------------------------

Advanced Simple Profile
-----------------------------------------------------------------------------------------------------

devapi3-1pass-fullfeatures = 13,9 MB (14.577.664 bytes) PSNR = 44.8289 AVG Quant = 2
devapi3-2pass-fullfeatures = 4,10 MB (4.307.456 bytes) PSNR = 40.0167 AVG Quant = 7.18

devapi3-1pass-fullfeatures-nogmc = 12,7 MB (13.318.656 bytes) PSNR = 44.8396 AVG Quant = 2
devapi3-2pass-fullfeatures-nogmc = 4,10 MB (4.309.504 bytes) PSNR = 40.3732 AVG Quant = 6.70

devapi3-1pass-fullfeatures-novhq = 12,8 MB (13.475.328 bytes) PSNR = 44.7953, AVG Quant = 2
devapi3-2pass-fullfeatures-novhq = 4,10 MB (4.304.896 bytes) PSNR = 40.3055, AVG Quant = 6.72

devapi3-1pass-fullfeatures-noqpel = 13,6 MB (14.313.984 bytes) PSNR = 45.0342, AVG Quant = 2
devapi3-2pass-fullfeatures-noqpel = 4,10 MB (4.304.384 bytes) PSNR = 40.2320, AVG Quant = 7.14

devapi3-1pass-fullfeatures-noqpel-novhq = 12,7 MB (13.334.528 bytes) PSNR = 45.0284, AVG Quant = 2
devapi3-2pass-fullfeatures-noqpel-novhq = 4,10 MB (4.309.504 bytes) PSNR = 40.5375, AVG Quant = 6.65

devapi3-1pass-fullfeatures-noqpel-nogmc = 12,4 MB (13.089.792 bytes) PSNR = 45.0844, AVG Quant = 2
devapi3-2pass-fullfeatures-noqpel-nogmc = 4,10 MB (4.308.992 bytes) PSNR = 40.6092, AVG Quant = 6.64

devapi3-1pass-qpel = 16,3 MB (17.118.720 bytes) PSNR = 45.3596, AVG Quant = 2
devapi3-2pass-qpel = 4,10 MB (4.305.920 bytes) PSNR = 39.9093, AVG Quant = 6.38
-----------------------------------------------------------------------------------------------------

devapi4-1pass-fullfeatures = 12,9 MB (13.550.080 bytes) PSNR = 44.7996, AVG Quant = 2
devapi4-2pass-fullfeatures = 4,12 MB (4.323.840 bytes) PSNR = 39.7352, AVG Quant = 8.93

devapi4-1pass-fullfeatures-nogmc = 13,0 MB (13.633.536 bytes) PSNR = 44.7989, AVG Quant = 2
devapi4-2pass-fullfeatures-nogmc = 4,12 MB (4.323.328 bytes) PSNR = 39.7235, AVG Quant = 8.94

devapi4-1pass-fullfeatures-novhq = 13,2 MB (13.944.320 bytes) PSNR = 44.4736, AVG Quant = 2
devapi4-2pass-fullfeatures-novhq = 4,12 MB (4.324.864 bytes) PSNR = 39.2678, AVG Quant = 9.22

devapi4-1pass-fullfeatures-noqpel = 11,6 MB (12.200.448 bytes) PSNR = 44.9829, AVG Quant = 2
devapi4-2pass-fullfeatures-noqpel = 4,12 MB (4.321.280 bytes) PSNR = 40.8701, AVG Quant = 7.71

devapi4-1pass-fullfeatures-noqpel-novhq = 12,2 MB (12.866.048 bytes) PSNR = 44.7467, AVG Quant = 2
devapi4-2pass-fullfeatures-noqpel-novhq = 4,12 MB (4.322.304 bytes) PSNR = 40.4277, AVG Quant = 8.00

devapi4-1pass-fullfeatures-noqpel-nogmc = 11,6 MB (12.254.208 bytes) PSNR = 44.9900, AVG Quant = 2
devapi4-2pass-fullfeatures-noqpel-nogmc = 4,12 MB (4.321.280 bytes) PSNR = 40.8550, AVG Quant = 7.74

devapi4-1pass-qpel = 16,3 MB (17.125.888 bytes) PSNR = 45.3890, AVG Quant = 2
devapi4-2pass-qpel = 4,10 MB (4.305.920 bytes) PSNR = 39.9246, AVG Quant = 6.40
-----------------------------------------------------------------------------------------------------

Kauehi-Standard-2pass-psynew-bframes = 4,05 MB (4.247.552 bytes) PSNR = 40.0914, AVG Quant = 6.71
Kauehi-Standard-2pass-qpel-bframes = 4,04 MB (4.241.408 bytes) PSNR = 39.2992, AVG Quant = 7.85

Kauehi-Very Slow-2pass-psynew-bframes = 4,04 MB (4.241.408 bytes) PSNR = 40.3811, AVG Quant = 6.41


fullfeatures (devapi3) = b-frames 2/150/75/0 ,chroma me,trellis,qpel,vhq wide search,gmc 2 warppoint
fullfeatures (devapi4) = b-frames 2/1.50/1.00,chroma me,trellis,qpel,vhq wide search,gmc 3 warppoint


This information is supplied without liability.

[Sagittaire]

guten Tag ... ;-)

-Psy new lowers the PSNR ...
-For PSNR maxi, put bitrate modulation "slow motion" or 0.25 ...
-3 Pass is better ...

[Selur]

1st thx for the build
2nd could you also include Nics current build ? :)

Cu Selur

[CruNcher]

@ Selur
there is atm no reason for that both buildup on devapi3 cvs and both include the same features, the only difference atm is that koepis build includes the adujustable b-frame threshold and trellis quants in his vfw interface the rest is exactly the same.

@ Sagittaire
in my compares im not trying to reach the best PSNR i try to show the differences between the development in speed/quality and it wouldn't be fair to use hitech divx kauehi functions "slow ass hell" against xvids normal speed functions.

[Sagittaire]

@ CruNcher

-Psy new lowers the fps ... and lowers the PSNR ...
-Bitrate Modulation haven't any inflence of the fps but influnece bitrate repartion: the "slow motion" is better for the PSNR ...

if you choose the best for XviD, choose the best for Kauehi ... :devil:

The best for XviD PSNR (and quality too for me :cool: ) is encodage XviD ffvfw in 3 pass ... ;)

[Prettz]

What exactly is the purpose of this test supposed to be? What is it supposed to show?

[PowerMacG4]

Uhhh... for 'full features' you did both VHQ and GMC?
Isn't that evil/forbidden?

[superdump]

PowerMacG4: Currently, in dev-api-3 GMC is not working. However, CruNcher wanted to test all available features for the sake of a comparison. If you look at the results further you will notice a full features no GMC encode. :) GMC IS working in dev-api-4 (just not perfectly, yet... ;))

Prettz: Currently PSNR testing is our main means of testing the usefulness of a feature. It gives an idea of how large an improvement (or depreciation ;)) a particular feature is providing. PSNR tests are by no means perfect however and other methods which are much more related to how we actually see the image are being developed but most will probably be commercial, which is unfortunate for us.

[Selur]

"and other methods which are much more related to how we actually see the image are being developed but most will probably be commercial, which is unfortunate for us."
Just curious, which? And why aren't they used?
(hoping you still ment full automatic methods, not watching the movie and compare by watching :) )

Cu Selur

[superdump]

Selur: Have a look around on this site for one example....

link (http://www.cns.nyu.edu/~zwang/files/research/ssim/index.html)

[Selur]

thx, looks interesting,..

[Riley]

I know this is a total newbie question, but can someone give a quick explanation of what the PSNR scores mean and how to interpret them.

[superdump]

PSNR means "Peak Signal to Noise Ratio". It is a measure of how close an image (or audio signal or something of this sort) is to the original. The higher the value the better. It is measured in dB.

What I was saying a few posts ago is that our eyes don't rely quite so much on how close the image is to the original and our eyes "prefer" losses in quality in one area rather than losses in another area.

For example, you could attempt to compress a bitmap image by reducing the colour depth to say 4-bit (16 colours) but it wouldn't look very nice. This is in the spatial domain. However if you compress the same picture using a JPEG codec it will still retain much of its quality while the filesize will be reduced greatly. This is in the frequency domain.

[-=Stan=-]

After extensive PSNR tests i realised that the settings i have been using so far in my encodes are not optimal .

My old settings -
Motion Precision ==> 6
VHQ ==> 4
Quantization ==> H.263
Chroma Motion ==> On
Qpel ==> On
B-Frames ==> 2/150/75
Trellis R-D ==> On

Results -
Minimum Average Maximum
PSNR: 39.2256 43.0625 45.8738

______________________________________________________

Now the new settings are -

Motion Precision ==> 5
VHQ ==> 4
Quantization ==> H.263
Chroma Motion ==> On
Qpel ==> Off
B-frames ==> 3/150/100
Trellis R-D ==> Off

Results -
Minimum Average Maximum
PSNR: 39.8576 43.2930 46.3027

Moreover these settings were quite understandably much faster .
_____________________________________________________

Use of MPEG drastically increased PSNR however the increase in filesize was undesirable .

MPEG used instead of H.263 had result as -

Minimum Average Maximum
PSNR: 39.9616 43.4340 46.3077

___________________________________________________________________


All these tests were using Koepi's 24/06/2003 build ,ei, devapi3

[V-tec]

Originally posted by -=Stan=-

...Use of MPEG drastically increased PSNR however the increase in filesize was undesirable ...


I think the big difference on file-size is principally due to the fact you have used a lower motion search precision...


I add some results for some test I have made between different codecs:



The compression script:


directshowsource("d:\test uv2\a_foreign_affair.wmv",fps=25)
trim(0,3000)
crop(0,8,1280,704)
bicubicresize(640,352,0,0.6)

example of PSNR calculation script:


# --> Video Opening <--

source=directshowsource("d:\test uv2\a_foreign_affair.wmv",fps=25)
source=trim(source,0,3000)
source=crop(source,0,8,1280,704)
source=bicubicresize(source,640,352,0,0.6)
source=ConvertToYUY2(source)

video=AviSource("d:\test uv2\xvid_33h.avi")
video=trim(video,1,0)
video=ConvertToYUY2(video)

# --> PSNR analysis <--
Compare(video,source,"","x_33h.txt")


The clip at xvid q2 have a size of 35510 kb = 2420 kbps

for a more or less file-size:

_______________________________________________________________________________________
|Codec | Settings | Size | PSNR |
|--------------------------------------------------------------------------------------|
| Xvid | vh4.CromaMotion |q2 | 35510Ko | 45,6764 dB |
| Divx 505 pro | |q2 | 37948Ko | 45,5018 dB |
| Divx Kauehi | |q2 | 34846Ko | 45,4584 dB |
| VM9 | Max Quality Decoder Complex |2pass @ 2420 kbps | 36288Ko | 44,9720 dB |
| RV9 | |2pass @ 2420 kbps | 36239Ko | 45.9450 dB |
----------------------------------------------------------------------------------------

at 33%

_______________________________________________________________________________________
| Codec | Settings | Size | PSNR |
|--------------------------------------------------------------------------------------
| Xvid | vh4.CromaMotion ACC High |2pass @ 804 kbps | 11830Ko | 42,8682 dB |
| Divx 505 pro | Low motion 0.25 |2pass @ 804 kbps | 11850Ko | 42,5259 dB |
| Divx Kauehi | Low Motion 0.25 |2pass @ 804 kbps | 11882Ko | 42,6090 dB |
| VM9 | Max Quality Decoder Complex |2pass @ 804 kbps | 12136Ko | 42,0039 dB |
| RV9 | |2pass @ 804 kbps | 12441Ko | 43.2136 dB |
---------------------------------------------------------------------------------------


Here you can found more details

http://web.infinito.it/utenti/p/paolo.vtec/tests/foreigns.rar

[temporance]

Nice comparison, V-tec.

I'm wondering what the effect of decoder postprocessing is for xvid and DivX. Also are there any decoder settings in the other codecs that can be changed?

Kauehi has more configurable postprocessing - does this affect PSNR (good or bad)? What postprocessing did you use for your comparisons?

[V-tec]

I hope to have succeeded to disable post processing for all the codecs...


[OT]sorry for my maccaroni english... :D

[temporance]

Your English is great! :)

I tried recomputing your PSNRs as Overall PSNR, not Average PSNR. This gives a fairer result and takes rate control more into consideration. Using average PSNR puts too much emphasis on the best-looking frames.

I get:

| Xvid | vh4.CromaMotion ACC High |2pass @ 804 kbps | 11830Ko | 42.5251 dB |
| Divx 505 pro | Low motion 0.25 |2pass @ 804 kbps | 11850Ko | 42.1497 dB |
| Divx Kauehi | Low Motion 0.25 |2pass @ 804 kbps | 11882Ko | 42.3388 dB |
| VM9 | Max Quality Decoder Complex |2pass @ 804 kbps | 12136Ko | 41.3827 dB |
| RV9 | |2pass @ 804 kbps | 12441Ko | 42.6397 dB |
Calculated this way, RV9 has less of a lead and WMV9 is really more behind!

If you want to do this in your spreadsheets, for each frame you need to get a number proportional to the noise energy in the frame - as an Excel formula you need something like

=POWER(10, -A3/10)

where A3 is the frame PSNR.

Then, to get Overall PSNR, do:

=10*LOG(1/AVERAGE(N3:N3001))

where N3:N3001 is your column of noise energy numbers.

Edit: BTW, I meant to try the test with postprocessing turned on!! Perhaps this will make an even better PSNR??? :)

Edit2: What Kauehi settings did you use? Did you use the slow ME (like VHQ)? or psychovisual set at "new" (new PV may make worse PSNR?) Also did you use RV9-EHQ or regular RV9?

[V-tec]

I think pp can decrease PSNR beacue it introduces a image distortion to make them good-looking

[temporance]

Originally posted by V-tec
I think pp can decrease PSNR beacue it introduces a image distortion to make them good-looking Yes, but if the PP is very good, the distortions it introduces will make the image more like the original and the PSNR will increase!! It's just a theory :)

[karl_lillevold]

Originally posted by temporance
Edit2: What Kauehi settings did you use? Did you use the slow ME (like VHQ)? or psychovisual set at "new" (new PV may make worse PSNR?) Also did you use RV9-EHQ or regular RV9?
Interesting numbers. Please make sure to use RV9-EHQ (http://forum.doom9.org/showthread.php?s=&threadid=55193). If it is not specified, I tend to assume it was not used. It always helps greatly visually. PSNR improvement varies greatly. Usually around 0.6 dB to 1.3 dB for natural sequences, but I have seen up to 2.0 dB improvement for certain animation content.

Also, it may be very important to disable RV9's noisyEdgeFilter. If there is just one line of black (or another non-video color) along one edge of the video, PSNR compared to the original will be significantly reduced, unless the noisyEdgeFilter is disabled. See this thread (http://forum.doom9.org/showthread.php?s=&threadid=57687) for an explanation of the filter. Maybe I will make it not-default any longer, hoping users do a good job cropping :)

How to disable noisyEdgeFilter. Add this line in the <codecProperties> section of the audience:

<codecProperties type="bag">
...
<noisyEdgeFilter type="bool">false</noisyEdgeFilter>
...
</codecProperties>

[CruNcher]

@ stan

here you can see how Qpel reacts to PSNR tests

The Qpel PSNR Mystery (http://www.mufflastig.com/CruNcher/XviD/QpelMystery.txt)

For Kludge it's planed to introduce Qpel triggering per frame basis this isn't currently allowed in the Mpeg4 Specs only for the whole time of the encoding.

[karl_lillevold]

I just found that the latest intermediate Milestone release of Helix DNA Procucer, Milestone 5, happens to accidentally disable EHQ, no matter what the encoding tool sets it to :angry: Please see this thread (http://forum.doom9.org/showthread.php?s=&threadid=58196). This also affects AutoRV9 1.3b3, which includes M5. It will be fixed shortly though, just something to be aware of, since I just happened to mention EHQ, I would not want it to be "enabled" and result in no improvement.