Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > New and alternative video codecs

Reply
 
Thread Tools Search this Thread Display Modes
Old 1st March 2005, 12:25   #1  |  Link
Sagittaire
Testeur de codecs
 
Sagittaire's Avatar
 
Join Date: May 2003
Location: France
Posts: 2,489
[20-06-2005] Metric Benchmark Challenge ...

~ Movie Metric Benchmark Challenge ~

You think that your codec is the best : prove that ... !!?


A - Introduction

The purpose of this challenge is to determine which is the best codec for the metrics and only for the metrics : this test will not speak about subjective visual quality. If you want subjective visual comparison download sample and compare yourself ...



B - Rules

Rule 1 : Source

You must use this source for the test : Harry Potter II trailer, 720*576 MPEG2 MP@ML 4.5 Mbps
HPII Part1 winrar, HPII Part2 winrar, HPII Part3 winrar, HPII Part4 winrar & HPII Part5 winrar


Rule 2 : Pre-process and encoding

You must use DGindex with 32-bit SSE/MMX iDCT algorithm for .d2v project and DGDecode.dll for MPEG2 decoding in avisynth scipt. You must use VirtualDub or VirtualDubMod in "fast compress" mod for VFW codec. You must use YV12 space color for complete process with all codec. You must use this avisynth script for make encoding:
Code:
source=Mpeg2Source("C:\...\HPII.d2v",idct=2)
source=Trim(source,70,3145)
source=Crop(source,0,76,-0,-76)
source=LanczosResize(source,720,304)
Return(source)



Rule 3 : Bitrate

You must use 2 bitrates for encoding. The first use XviD in quant8 for reference, it's the "Streaming Quality". The second use XviD in quant4 for reference, it's the "1CDR Quality". You can use container of your choice for all Video Elementary Stream.

Streaming: 450 Kbps for Video Elementary Stream and container or 6750 Ko with +/- 0.30 % for tolerance. The size must be in [6730 Ko - 6770 Ko] interval.

1CDR: 900 Kbps for Video Elementary Stream and container or 13500 Ko with +/- 0.30 % for tolerance. The size must be in [13460 Ko - 13540 Ko] interval.


Rule 4 : Metric Tests

SSIM (YV12): IMO the best metric. it uses some HVS properties but not all ...
PDF description for VSSIM

Overall PSNR (YV12): Very good metric test, particullary for Rate Control Quality test ...
MSE = Sum t = 1 -> NFrame Sum x = 1 -> width Sum y = 1 -> height (F'(x,y,t)-F(x,y,t))² / (NFrame*width*height)
Overall PSNR = 10 * Log ( 255² / MSE)


Average PSNR (YV12):The worse metric test but Frame PSNR graph is very good to compare RC strategy ...
MSE = Sum x = 1 -> width Sum y = 1 -> height (F'(x,y,t)-F(x,y,t))² / (width*height)
Average PSNR = Sum t = 1 -> NFrame 10 * Log ( 255² / MSE) / Nframe


Here the test archives if you want all original Avisynth script for metric test.


SSIM vs PSNR

"For a given reference video and a given compressed video, it is meant to compute a quality metric, based on perceived visual distortion. Unlike the well-known PSNR measure, it's not purely mathematical, and should correlate much better with human vision."

"A higher MSE (and so lower PSNR) should mean that the compressed clip is a worse image but MSE and PSNR are flawed in this respect as numerous tests have shown. However with SSIM, according to tests carried out on the VQEG dataset, a higher Q (SSIM value) has a much better relation to the visual quality of the compressed clip. Despite this, bear in mind the SSIM metric still isn't perfect."

Here last AviSynth plugin for VSSIM in YV12


Average PSNR vs Overall PSNR

It's easy to increase artificially Average PSNR with Rate Control tweak. In certain case CBR (0% for variability) will be better than real VBR (100% for variability or constant quant). It's generaly not possible with Overall PSNR. Overall PSNR is very good to test Rate Control quality. If difference between Average PSNR and Overall PSNR is very high, "local quality difference" (difference between Pframe or Bframe for exemple) or "scene quality difference" (difference between low motion and high motion for exemple) will be too high and Rate Control for this codec not very good.

Here last AviSynth plugin for Overall PSNR in YV12


You must mux or remux x264, VP6, VP7, WMV9, DivX6 and XviD in avi container (with VDM for example) to solve problem with color space output, synchronisation or acquisition for metric test. You must use AviSource in avisynth for VP6, VP7 and WMV9. You must use DirectShowSource in avisynth for MPEG4 ASP, MPEG4 AVC and RV10. You must change trim for frame synchronization or for clip length if you have some probleme with SSIM plugin. You must use these AviSynth type script for metric test:
Code:
# --> Source Opening <--
source=Mpeg2Source("G:\...\HPII.d2v",idct=2)
source=Trim(source,70,3145)
source=Crop(source,0,76,-0,-76)
source=LanczosResize(source,720,304)

# --> Video Opening <--
video=DirectShowSource("G:\...\DivX6-450.avi",fps=25)

# --> PSNR analysis <--
compareYV12(video,source,"YUV","PSNR-DivX6-450.log")


Code:
# --> Source Opening <--
source=Mpeg2Source("G:\...\HPII.d2v",idct=2)
source=Trim(source,70,3144)
source=Crop(source,0,76,-0,-76)
source=LanczosResize(source,720,304)

# --> Video Opening <--
video=DirectShowSource("G:\...\RV10-450.mkv",fps=25)

# --> SSIM analysis <--
return SSIM(source,video,"results.csv","SSIM-RV10-450.txt",lumimask=2)



C - Results

At this time here my result ... but done better if you can ...
Code:
|--------------|---------|---------|---------|---------|---------|---------|
| Codec        |  PProc  | Bitrate |   Size  |  OPSNR  |  SSIM 0 |  SSIM 2 |
|--------------|---------|---------|---------|---------|---------|---------|
| DivX6 ASP    |   PP4   |   446   |   6757  |  39.85  |  70.17  |  65.02  |
| XviD  ASP    |   PP4   |   446   |   6756  |  39.66  |  69.55  |  64.33  |
| LAVC  ASP    |   PP4   |   447   |   6775  |  39.71  |  69.51  |  63.82  |
| Nero AVC HP  |   PP0   |   446   |   6747  |  41.10  |  76.30  |  72.10  |
| x264 AVC HP  |   PP0   |   446   |   6747  |  40.85  |  74.82  |  70.40  |
| Nero AVC MP  |   PP0   |   446   |   6747  |  40.72  |  74.65  |  70.25  |
| x264 AVC MP  |   PP0   |   446   |   6750  |  40.64  |  73.83  |  69.23  |
| VP7          |   PP2   |   446   |   6760  |  40.84  |  74.42  |  70.05  |
| VP6          |   PP4   |   446   |   6769  |  40.28  |  71.86  |  66.89  |
| WMV9         |   PP1   |   446   |   6746  |  39.51  |  69.58  |  64.07  |
| RV10         |   HF2   |   446   |   6743  |  39.84  |  69.86  |  64.60  |
| DivX3        |   PP4   |   447   |   6771  |  38.70  |  65.11  |  58.56  |
|--------------|---------|---------|---------|---------|---------|---------|
| DivX6 ASP    |   PP4   |   896   |  13493  |  42.78  |  81.14  |  77.99  |
| XviD  ASP    |   PP4   |   896   |  13489  |  42.66  |  80.99  |  77.96  |
| LAVC  ASP    |   PP4   |   897   |  13528  |  42.77  |  80.95  |  77.77  |
| Nero AVC HP  |   PP0   |   896   |  13490  |  43.70  |  84.39  |  81.54  |
| x264 AVC HP  |   PP0   |   896   |  13491  |  43.46  |  83.27  |  80.50  |
| Nero AVC MP  |   PP0   |   896   |  13491  |  43.43  |  83.45  |  80.63  |
| x264 AVC MP  |   PP0   |   896   |  13498  |  43.26  |  82.66  |  79.75  |
| VP7          |   PP2   |   897   |  13534  |  43.34  |  82.75  |  80.02  |
| VP6          |   PP4   |   896   |  13481  |  42.99  |  81.39  |  78.41  |
| WMV9         |   PP1   |   896   |  13510  |  42.52  |  79.88  |  76.73  |
| RV10         |   HF2   |   896   |  13493  |  42.77  |  80.82  |  77.72  |
| DivX3        |   PP4   |   896   |  13508  |  41.84  |  78.27  |  74.40  |
|--------------|---------|---------|---------|---------|---------|---------|
SSIM 0: Lumimask Off
SSIM 1: Lumimask On (Original Lumimask)
SSIM 2: Lumimask On (One2Tech Patch)

MPEG4 ASP : DivX6
Coder: DivX6 Codec
Setting: 2 pass, Unconstrained, Insame Quality, Multi-Bframe, H263 optmized, BM = 0.1
decoder: DivX decoder with PP4
files: DivX 450 Kbps & DivX 900 Kbps


MPEG4 ASP : XviD
Coder: Koepi XviD 1.1.0 beta2
Setting: 2 pass, Unconstrained, Motion Ultra-High, VHQ4, BVHQ, Chroma Motion, Trellis, Adaptative Quantisation, GMC, BFrame 2/1.50/1.00
decoder: XviD decoder with PP4
files: XviD 450 Kbps & XviD 900 Kbps


MPEG4 ASP : Libavcodec
Coder: MEncoder dev-CVS-050426-18:49-3.4.2
Setting: 3 pass, adaptative BFrame 2/1.50/1.00, Chroma Motion, Trellis, RDO, SAD, 6 diamond size
decoder: ffdshow decoder with PP4
files: Libavcodec ASP 450 Kbps & Libavcodec ASP 900 Kbps


DivX 3.11 : Libavcodec
Coder: MEncoder dev-CVS-050426-18:49-3.4.2
Setting: 3 pass, Chroma Motion, RDO, SAD, 6 diamond size
decoder: ffdshow decoder with PP4
files: DivX3 450 Kbps & DivX3 900 Kbps


MPEG4 AVC : Ateme AVC Main Profil
Coder: Ateme encavc 1.2.0.17
Setting: 2 pass, Quality Full, 8 Ref, 8 BRef, 2 Bframes, CABAC, BPred, MPart, WPred, Inloop strength 0, Chroma, psy 3
decoder: Ateme AVC decoder
files: Nero AVC Main Profil 450 Kbps & Nero AVC Main Profil 900 Kbps


MPEG4 AVC : Ateme AVC High Profil
Coder: Ateme encavc 1.2.0.14
Setting: 2 pass, Quality Full, 16 Ref, 16 BRef, 2 Bframes, CABAC, BPred, MPart, WPred, Inloop strength 0, Chroma, psy 3, transform 8x8
decoder: Ateme AVC decoder
files: Nero AVC High Profil 450 Kbps & Nero AVC High Profil 900 Kbps


MPEG4 AVC : x264 Main Profil
Coder: x264 VFW rev 264
Setting: 3 pass, variabilty 75%, Quality Max, 16 Ref frames, 2 Bframes, CABAC, WPred, Inloop deblock strength -1, RDO
decoder: ffdshow H264 decoder
files: x264 Main Profil 450 Kbps & x264 Main Profil 900 Kbps


MPEG4 AVC : x264 High Profil
Coder: x264 VFW rev 264
Setting: 3 pass, variabilty 75%, Quality Max, 16 Ref frames, 2 Bframes, CABAC, WPred, Inloop deblock strength -1 , 8x8dct, RDO
decoder: ffdshow H264 decoder
files: x264 High Profil 450 Kbps & x264 High Profil 900 Kbps


One2Tech : VP7
Coder: VP7 VFW Codec 7.0.0.7
Setting: 2 pass, best quality, VBR, Noisered 0, Sharpeness 0, Default Rate Control setting
decoder: VP7 decoder PP2
files: VP7 450 Kbps & VP7 900 Kbps


One2Tech : VP6
Coder: VP6 VFW Codec 6.4.1.0
Setting: 2 pass, best quality, VBR, Noisered=0, Sharpeness=0, MinQ 6/12, MaxQ 56, Vbr bias 70, Vbr min 30, Vbr max 200, Undershoot 95%
decoder: VP6 decoder PP4
files: VP6 450 Kbps & VP6 900 Kbps


Microsoft : WMV9
Coder: WMV9 VCM
Setting: 2 pass VBR, best quality, decoder complex
decoder: WMV3 DMO decoder PP1
files: WMV9 450 Kbps & WMV9 900 Kbps


RealNetwork : RV10
Coder: erv4 10.0.0.2
Setting: 2 pass, EHQ 100, old Rate Control, chroma
decoder: RV10 decoder HFE2
files: RV10 450 Kbps & RV10 900 Kbps


MPEG2 MP@ML : MEncoder dev-CVS-050426-18:49-3.4.2
find size for metric equivalence with best codec for quality "1CDR"
"Quality" must be at ~ 43 dB for OPSNR or at ~ 80 for SSIM
files: MPEG2 1250 Kbps



D - Conclusion

At this time for this trailer

MPEG4 AVC codecs: NDAVC vs x264
Ateme H264 and x264 are comparable codec for OPSNR but not for SSIM (visually NDAVC is IMO better than x264 too). x264 is the best Open Source Codec and XviD can't fight with the last version.

MPEG4 ASP codecs: XviD vs DivX vs LAVC
DivX6 is the best MPEG4 ASP codec for OPSNR and SSIM but XviD, DivX6 and Libavcodec ASP are very close codec.

Proprietary "free" codecs: RV10 vs VP6 vs WMV9
VP6 is always better than RV10 or WMV9 for OPSNR or SSIM. VP6 are better than all MPEG4 ASP codec for OPSNR and SSIM. RV10 is an excellent codec for high quant encoding (low quality or "low bitrate").

Higest performance codecs: VP7 vs NDAVC vs x264
VP7, Ateme AVC or x264 are really better than the others codecs. VP7 seem better than Ateme H264 Main Profil for very high quant (low quality or "low bitrate") and Ateme H264 Main Profil seem better than VP7 for lower quant (high quality or "high bitrate"). x264 High Profil is the best codec actually available. Ateme H264 is by far the best codec in this test ...

MPEG2 vs all
The best codecs (H264 or VP7) are 50% better than MPEG2 MP@ML but MPEG2 MP@ML with good encoder is always a good codec. IMO Mencoder is by far the best MPEG2 encoder for low bitrate ...



Annexe - Update

14.07.05
- Ateme codec Full quality update

19.06.05
- Ateme codec Full quality
- VP7 7.0.0.7 better RC

17.06.05
- x264 rev 263 Main Profil
- x264 rev 263 High Profil
- DivX6
- VP7 7.0.0.7
- MPEG2 MP@ML with Mencoder

08.06.05
- x264 rev 253 Main Profil
- x264 rev 253 High Profil

24.05.05
- SSIM plugin
- Nero Digital High Profil for demo only

22.05.05
- x264 rev 232
- VP7 7.0.0.6
- Libavcodec MPEG4 ASP (MEncoder dev-CVS-050426-18:49-3.4.2)
- Libavcodec DivX 3.11 (MEncoder dev-CVS-050426-18:49-3.4.2)
- XviD 1.1.0 Beta2

03.04.05
- x264 rev 184
- VP7 7.0.0.4
__________________
Le Sagittaire ... ;-)

1- Ateme AVC or x264
2- VP7 or RV10 only for anime
3- XviD, DivX or WMV9

Last edited by Sagittaire; 15th July 2005 at 12:58. Reason: Update x264 MP & HP, VP7, Ateme AVC MP & HP, DivX6, MPEG2 ...
Sagittaire is offline   Reply With Quote
Old 1st March 2005, 16:03   #2  |  Link
Manao
Registered User
 
Join Date: Jan 2002
Location: France
Posts: 2,855
You encoded XviD at fixed quantizer. Encoding it two-passes will give you a higher psnr. Furthermore, you stated 450 kbps / 6882 KB, which isn't coherent ( 1024 / 1000 issue ). You're better off targetting 458kbps ( the difference been greater than the error margin, those 8 kbps are important ).

I made the test with XviD @ '450' kbps ( actually, in order to hit the wanted bitrate, i had to type 480 in the bitrate box ). I obtained a psnr of 40.49 dB overall ( 41.69 average ).

Edit : btw, the loss implied by doing a constant quantizer encoding is 0.8 dB, which is _huge_. I hope nobody still believe that constant quantizer yields better quality that a two passes encoding.

Last edited by Manao; 1st March 2005 at 16:29.
Manao is offline   Reply With Quote
Old 1st March 2005, 16:30   #3  |  Link
Sagittaire
Testeur de codecs
 
Sagittaire's Avatar
 
Join Date: May 2003
Location: France
Posts: 2,489
Quote:
Originally posted by Manao
You encoded XviD at fixed quantizer. Encoding it two-passes will give you a higher psnr.
For me priority are in order:
- SSIM
- Overall PSNR
- Average PSNR


Quote:
Originally posted by Manao
Furthermore, you stated 450 kbps / 6882 KB, which isn't coherent ( 1024 / 1000 issue ). You're better off targetting 458kbps ( the difference been greater than the error margin, those 8 kbps are important ).
Yes ... it's true ... 458 Kbps +/-1% and 919 Kbps +/-1% exactly (I edit)


Quote:
Originally posted by Manao
I made the test with XviD @ '450' kbps ( actually, in order to hit the wanted bitrate, i had to type 480 in the bitrate box ). I obtained a psnr of 40.49 dB overall ( 41.69 average ).
Yes I use one pass q4 (q4 for I and P, q6 for B) ... but your encoding seem very good (perhabs not for SSIM) ... you can post this file or your setting ...
__________________
Le Sagittaire ... ;-)

1- Ateme AVC or x264
2- VP7 or RV10 only for anime
3- XviD, DivX or WMV9
Sagittaire is offline   Reply With Quote
Old 1st March 2005, 16:42   #4  |  Link
Manao
Registered User
 
Join Date: Jan 2002
Location: France
Posts: 2,855
SSIM : 63.72
Settings : XviD defaults + bvhq + vhq4 + GMC

Btw, you praise psnr overall over psnr average when it comes to rate control comparison, however ssim is computed as average psnr is, so ssim should only be used to compare frames together, not to compare rate control algorithms.

Edit : forgot : playback with ffdshow Nic's PP4

Last edited by Manao; 1st March 2005 at 17:27.
Manao is offline   Reply With Quote
Old 1st March 2005, 19:38   #5  |  Link
springl
Registered User
 
Join Date: Nov 2002
Posts: 32
My results with Xvid:

Streaming.avi
size: 6894 KB
SSIM_PP4: 64.86

1CD.avi
size: 13806 KB
SSIM_PP4: 78.27

Settings :
XviD defaults + bvhq + vhq4 + GMC + MIN/MAX I/P/B frame 2/31 + some tuning in
overflow treatment and curve compression.

playback with ( Xvid decoder + PP4 ) or ( ffdshow + Nic's PP4 ) will produce same results.

Greetings
Springl
springl is offline   Reply With Quote
Old 1st March 2005, 20:38   #6  |  Link
Manao
Registered User
 
Join Date: Jan 2002
Location: France
Posts: 2,855
Sagittaire :

CruNcher and I doublechecked your psnr results for your XviD @ 450 kbps. We obtain an overall of 40.40 and a SSIM of 63.94. How did you manage to get that wrong ? What can we assume for the other figures for the other codecs ?
Manao is offline   Reply With Quote
Old 1st March 2005, 22:24   #7  |  Link
bobololo
Registered User
 
Join Date: May 2003
Posts: 328
@Sagittaire: Just a stupid question, why don't you use compareYV12() instead of compare() ? Don't all the codecs here work natively in YUV 4:2:0 colorspace ?
bobololo is offline   Reply With Quote
Old 1st March 2005, 23:18   #8  |  Link
LordRPI
Registered User
 
Join Date: Sep 2004
Location: Silly Valley
Posts: 261
Sagittaire,

Would it be possible to list the codec settings you used for all comparisons?
LordRPI is offline   Reply With Quote
Old 1st March 2005, 23:58   #9  |  Link
Sagittaire
Testeur de codecs
 
Sagittaire's Avatar
 
Join Date: May 2003
Location: France
Posts: 2,489
@Bobololo
compare is only YUY2 or RGB compatible in avisynth ... i don't understand why ... perhabs function begin in avisynth YUY2 2.0x version but avisynth 2.5x is YV12 now ...

You can confirm my result with your AVC codec and with my test protocol ... perhabs you can make better encoding ... ???


@ Manao
Problem seem to be ffdshow for MPEG4 ASP class codec ... I will use XviD decoder and DivX decoder now ... update in progress


@LordRPI
update in progress ...


@springl
please if possible post your encoding or your setting with complete reg file for exemple:

Code:
Windows Registry Editor Version 5.00

[HKEY_CURRENT_USER\Software\GNU\XviD]
"mode"=dword:00000002
"bitrate"=dword:000001ca
"desired_size"=dword:00001ac2
"use_2pass_bitrate"=dword:00000000
"desired_quant"=dword:00000190
"quant_type"=dword:00000000
"lum_masking"=dword:00000000
"interlacing"=dword:00000000
"tff"=dword:00000000
"qpel"=dword:00000000
"gmc"=dword:00000000
"use_bvop"=dword:00000001
"max_bframes"=dword:00000002
"bquant_ratio"=dword:00000096
"bquant_offset"=dword:00000064
"packed"=dword:00000001
"ar_mode"=dword:00000000
"aspect_ratio"=dword:00000000
"par_x"=dword:00000001
"par_y"=dword:00000001
"ar_x"=dword:00000004
"ar_y"=dword:00000003
"num_zones"=dword:00000001
"rc_reaction_delay_factor"=dword:00000010
"rc_averaging_period"=dword:00000064
"rc_buffer"=dword:00000064
"discard1pass"=dword:00000001
"full1pass"=dword:00000000
"keyframe_boost"=dword:0000000a
"kfreduction"=dword:00000014
"kfthreshold"=dword:00000001
"curve_compression_high"=dword:00000000
"curve_compression_low"=dword:00000000
"overflow_control_strength"=dword:00000005
"twopass_max_overflow_improvement"=dword:00000005
"twopass_max_overflow_degradation"=dword:00000005
"container_type"=dword:00000001
"target_size"=dword:000a2800
"subtitle_size"=dword:00000000
"hours"=dword:00000001
"minutes"=dword:0000001e
"seconds"=dword:00000000
"fps"=dword:00000002
"audio_mode"=dword:00000000
"audio_type"=dword:00000000
"audio_rate"=dword:00000080
"audio_size"=dword:00000000
"motion_search"=dword:00000006
"vhq_mode"=dword:00000004
"vhq_bframe"=dword:00000001
"chromame"=dword:00000001
"turbo"=dword:00000000
"max_key_interval"=dword:0000012c
"frame_drop_ratio"=dword:00000000
"min_iquant"=dword:00000001
"max_iquant"=dword:0000001f
"min_pquant"=dword:00000001
"max_pquant"=dword:0000001f
"min_bquant"=dword:00000001
"max_bquant"=dword:0000001f
"trellis_quant"=dword:00000001
"fourcc_used"=dword:00000000
"debug"=dword:00000000
"vop_debug"=dword:00000000
"display_status"=dword:00000001
"Brightness"=dword:00000000
"Deblock_Y"=dword:00000001
"Deblock_UV"=dword:00000001
"Dering_Y"=dword:00000000
"Dering_UV"=dword:00000000
"FilmEffect"=dword:00000000
"profile"="(unrestricted)"
"stats"="\\video.pass"
"qmatrix_intra"=hex:08,11,12,13,15,17,19,1b,11,12,13,15,17,19,1b,1c,14,15,16,\
  17,18,1a,1c,1e,15,16,17,18,1a,1c,1e,20,16,17,18,1a,1c,1e,20,23,17,18,1a,1c,\
  1e,20,23,26,19,1a,1c,1e,20,23,26,29,1b,1c,1e,20,23,26,29,2d
"qmatrix_inter"=hex:10,11,12,13,14,15,16,17,11,12,13,14,15,16,17,18,12,13,14,\
  15,16,17,18,19,13,14,15,16,17,18,1a,1b,14,15,16,17,19,1a,1b,1c,15,16,17,18,\
  1a,1b,1c,1e,16,17,18,1a,1b,1c,1e,1f,17,18,19,1b,1c,1e,1f,21
"zone0_frame"=dword:00000000
"zone0_mode"=dword:00000000
"zone0_weight"=dword:00000064
"zone0_quant"=dword:00000190
"zone0_type"=dword:00000000
"zone0_greyscale"=dword:00000000
"zone0_chroma_opt"=dword:00000000
"zone0_bvop_threshold"=dword:00000000
"zone0_cartoon_mode"=dword:00000000
"ForceColorspace"=dword:00000001
"FlipVideo"=dword:00000000
"Supported_4CC"=dword:00000000
"Videoinfo_Compat"=dword:00000000
__________________
Le Sagittaire ... ;-)

1- Ateme AVC or x264
2- VP7 or RV10 only for anime
3- XviD, DivX or WMV9
Sagittaire is offline   Reply With Quote
Old 2nd March 2005, 00:21   #10  |  Link
DigitAl56K
Registered User
 
Join Date: Nov 2002
Location: San Diego, CA
Posts: 936
Which settings were used for DivX? I might be missing something obvious here, but I don't see them. Did you constrain all codecs via VBV, or was this test a mixed bag? Was there any emphasis on comparible testing (e.g. time to encode, CE compatibility, etc.) or was every codec simply allowed to run wild with the features it used?

What were the settings used for WM9? I'm surprised at this result.

Also, when you use a decoder you really ought to use the decoder for the codec you are testing, i.e. use the DivX decoder for DivX, the XVID decoder for XVID, etc. Each codec may implement different technologies in their decoder and these are no less a part of the format than any encode-side features. For DivX we recommend full deblock, no dering, no frame delay, no film effect. We notice you also used DirectShowSource. When using this input filter please ensure that the DivX Decoder is configured so that it does not overlay the DivX Video logo during decoding.

Last edited by DigitAl56K; 2nd March 2005 at 00:26.
DigitAl56K is offline   Reply With Quote
Old 2nd March 2005, 00:25   #11  |  Link
sparky
DivX Team
 
Join Date: Oct 2001
Location: San Diego, CA
Posts: 24
Quote:
Streaming: 458 Kbps or 6885 Ko
Just to clarify. Do you mean 458 kbps = 458000 bps = (458000/8)*(3076/25) bytes = 7044040 bytes = 6879 Kb, and 919 kbps = 919000 bps = 14134220 bytes = 13803 Kb?

Also, you have Trim(70,3144) in SSIM script, and Trim(70,3145) in all other scripts. It causes an error ( SSIM won't work with different-length clips ). You want to change 3144->3145.

Last edited by sparky; 2nd March 2005 at 00:35.
sparky is offline   Reply With Quote
Old 2nd March 2005, 00:42   #12  |  Link
Sagittaire
Testeur de codecs
 
Sagittaire's Avatar
 
Join Date: May 2003
Location: France
Posts: 2,489
@ DigitAl56K
problem with ffdshow for MPEG4 ASP class codec (i use DivX dec and XviD dec now ...) and I update bench for MPEG4 ASP codec ...

For WMV9 I use best quality with complex dec in 2 two pass VBR. I use PP1 DMO for Post-process ...

For DivX I use unconstrained, insame quality, H263, multibframe, GMC, Bitrate Modultion = 0.25. You can propose your encoding with setting used if you want ...

Code:
Windows Registry Editor Version 5.00

[HKEY_CURRENT_USER\Software\DivXNetworks\DivX4Windows]
"Auto PP Level"=dword:00000000
"Auto PP"=dword:00000000
"Disable PP"=dword:00000000
"Dering PP Level"=dword:00000000
"Deblock PP Level"=dword:0000000f
"Warmth Strength"=dword:00000002
"Warmth Filter"=dword:00000000
"Postprocessing"=dword:00000028
"Brightness"=dword:00000000
"Contrast"=dword:00000000
"Saturation"=dword:00000000
"AspectRatio_X"=dword:00000000
"AspectRatio_Y"=dword:00000000
"Smooth Playback"=dword:00000000
"Extended Yuv Mode"=dword:00000001
"Double Buffering"=dword:00000000
"Extended Overlay Mode"=dword:00000000
"Disable Logo"=dword:00000001
"ATI Disable"=dword:00000000
"Support Generic Mpeg-4"=dword:00000001
"Bitrate"=dword:000be6e0
"VBV Channel Bitrate"=dword:006a1120
"VBV Buffer Size"=dword:00300000
"VBV Occupancy"=dword:00240000
"Complexity Modulation"="0.000000"
"Max Keyframe Interval"=dword:0000012c
"Quality"=dword:00000005
"VBR Mode"=dword:00000000
"VBR Quality v5"=dword:00000028
"Frame Dropping"=dword:00000000
"Deinterlace"=dword:00000000
"CPU Limit"=dword:00000064
"Use CPU Limit"=dword:00000000
"Data Partitioning"=dword:00000000
"Bidirectional Encoding"=dword:00000001
"Scene change threshold"=dword:00000032
"do_add"=dword:00000001
"do_mc"=dword:00000001
"do_mc_b"=dword:00000001
"show_gmc"=dword:00000000
"no_vbr_rc"=dword:00000000
"use_mvfile"=dword:00000000
"do_writeyuv"=dword:00000000
"do_writeyuvmul"=dword:00000000
"Psychovisual Enhancement"=dword:00000000
"Psychovisual Strength - Frame Level"="0.200000"
"Psychovisual Strength - Macroblock level"="0.200000"
"Write MP4 file"=dword:00000000
"Use GMC"=dword:00000000
"Quarter-pel"=dword:00000000
"Crop"=dword:00000000
"Resize"=dword:00000000
"Crop left"=dword:00000000
"Crop right"=dword:00000000
"Crop top"=dword:00000000
"Crop bottom"=dword:00000000
"Resize width"=dword:00000280
"Resize height"=dword:000001e0
"Resize mode"=dword:00000000
"Bicubic spline B"="0.000000"
"Bicubic spline C"="0.500000"
"Temporal enable"=dword:00000000
"Spatial passes"=dword:00000000
"Temporal level"="0.000000"
"Spatial level"="0.000000"
"Interlace mode"=dword:00000000
"Use dialogs"=dword:00000000
"Disable feedback"=dword:00000001
"Write log file"=dword:00000000
"5.0 Psychovisual Mode"=dword:00000000
"5.0 Preprocessing Mode"=dword:ffffffff
"5.0 Resize Mode"=dword:00000001
"Write nth pass logfile"=dword:00000000
"Disable Profile Page"=dword:00000000
"Active Profile"=dword:00000002
"Nth Pass debug file name"="c:\\newrc.txt"
"Log File Name"="c:\\divx.log"
"MP4 File Name"="c:\\test.divx"
"YUV Dir Name"="c:\\yuv"
"MV File Name"="c:\\mvinfo.bin"
"Top field first"=dword:00000001
"MPEG Quant"=dword:00000000
"Rotate Artwork"=dword:00000001
"Check for Updates"=dword:00000001
"Disable Notifications"=dword:00000000
"Intra quant matrix"=hex:00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,\
  00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,\
  00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00
"Inter quant matrix"=hex:00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,\
  00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,\
  00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00,00
"Feedback refresh"=dword:00000001
"Force Color Mode"=dword:00000000


command line:
-bvnn 915000 -vbv 27336400,100663296,75497472 -pq 8610 -g -complexity_modulation=0.25
__________________
Le Sagittaire ... ;-)

1- Ateme AVC or x264
2- VP7 or RV10 only for anime
3- XviD, DivX or WMV9

Last edited by Sagittaire; 2nd March 2005 at 01:15.
Sagittaire is offline   Reply With Quote
Old 2nd March 2005, 01:04   #13  |  Link
Sagittaire
Testeur de codecs
 
Sagittaire's Avatar
 
Join Date: May 2003
Location: France
Posts: 2,489
Quote:
Originally posted by sparky
Just to clarify. Do you mean 458 kbps = 458000 bps = (458000/8)*(3076/25) bytes = 7044040 bytes = 6879 Kb, and 919 kbps = 919000 bps = 14134220 bytes = 13803 Kb?

Also, you have Trim(70,3144) in SSIM script, and Trim(70,3145) in all other scripts. It causes an error ( SSIM won't work with different-length clips ). You want to change 3144->3145.
yes for bitrate ...

not explain for variable length clip ... clip for SSIM function must have the same length and not for compare function. Length problem is perhabs bframe problem for MPEG4 ASP, for RV10 i don't know ... but it isn't big problem with trim modification ...
__________________
Le Sagittaire ... ;-)

1- Ateme AVC or x264
2- VP7 or RV10 only for anime
3- XviD, DivX or WMV9
Sagittaire is offline   Reply With Quote
Old 2nd March 2005, 01:20   #14  |  Link
bobololo
Registered User
 
Join Date: May 2003
Posts: 328
Quote:
Originally posted by Sagittaire
@Bobololo
compare is only YUY2 or RGB compatible in avisynth ... i don't understand why ... perhabs function begin in avisynth YUY2 2.0x version but avisynth 2.5x is YV12 now ...

You can confirm my result with your AVC codec and with my test protocol ... perhabs you can make better encoding ... ???
For native YV12 PSNR computation, try compareYV12.

Regarding your result with our codec, let me know your settings first. Also, there is an undocumented quality level "full" which is insanely slow but that can surely achieve slightly higher figures
bobololo is offline   Reply With Quote
Old 2nd March 2005, 01:38   #15  |  Link
Sagittaire
Testeur de codecs
 
Sagittaire's Avatar
 
Join Date: May 2003
Location: France
Posts: 2,489
Quote:
Originally posted by bobololo
For native YV12 PSNR computation, try compareYV12.

Regarding your result with our codec, let me know your settings first. Also, there is an undocumented quality level "full" which is insanely slow but that can surely achieve slightly higher figures
yes, I know but just Average PSNR with this plugin ...
YUY2 test order is not different to YV12 ... there are good scaling with result between YUY2 and YV12 test ... perhabs Average PSNR in YV12 and Overall PSNR in YUY2 will be good solution ...


for your AVC codec I use "my best setting" ... lol
encavc.exe -i azerty.avs -o clip.mp4 -qual extra -rcmode 1st -log 1pass.log -br 450000 -deblock 0 -ref 5 -setef wpred -cartoon -psy 1 -priority idle
encavc.exe -i azerty.avs -o clip.mp4 -qual extra -rcmode 2nd -log 1pass.log -br 450000 -deblock 0 -ref 5 -setef wpred -cartoon -psy 1 -priority idle


Quote:
Also, there is an undocumented quality level "full" which is insanely slow but that can surely achieve slightly higher figures
[méthode couet=On]
i want this command line ... i want this command line ... i want this command line ... lol
[méthode couet=Off]
__________________
Le Sagittaire ... ;-)

1- Ateme AVC or x264
2- VP7 or RV10 only for anime
3- XviD, DivX or WMV9

Last edited by Sagittaire; 2nd March 2005 at 01:47.
Sagittaire is offline   Reply With Quote
Old 2nd March 2005, 01:59   #16  |  Link
Sagittaire
Testeur de codecs
 
Sagittaire's Avatar
 
Join Date: May 2003
Location: France
Posts: 2,489
XviD update:
new sample, new bench, new graph

@Bobololo

-qual full ... ah yes ... 0.16 fps ... lol

[French=On]
je comprend vite mais faut quand même m'expliquer longtemps ... lol
[French=Off]
__________________
Le Sagittaire ... ;-)

1- Ateme AVC or x264
2- VP7 or RV10 only for anime
3- XviD, DivX or WMV9

Last edited by Sagittaire; 2nd March 2005 at 02:03.
Sagittaire is offline   Reply With Quote
Old 2nd March 2005, 02:01   #17  |  Link
bobololo
Registered User
 
Join Date: May 2003
Posts: 328
Quote:
Originally posted by Sagittaire
yes, I know but just Average PSNR with this plugin ...
YUY2 test order is not different to YV12 ... there are good scaling with result between YUY2 and YV12 test ... perhabs Average PSNR in YV12 and Overall PSNR in YUY2 will be good solution ...
Ah you're right, I didn't notice that compareYV12 doesn't report overall PSNR. That's really weird it isn't included. It should be easy to add it. I'll try to make tomorrow.

Quote:
for your AVC codec I use "my best setting" ... lol
encavc.exe -i azerty.avs -o clip.mp4 -qual extra -rcmode 1st -log 1pass.log -br 450000 -deblock 0 -ref 5 -setef wpred -cartoon -psy 1 -priority idle
encavc.exe -i azerty.avs -o clip.mp4 -qual extra -rcmode 2nd -log 1pass.log -br 450000 -deblock 0 -ref 5 -setef wpred -cartoon -psy 1 -priority idle
I believe the target bitrate was 458000 bits/s, am I wrong ?

Quote:
[méthode couet=On]
i want this command line ... i want this command line ... i want this command line ... lol
[méthode couet=Off]
I explicitly gave it
bobololo is offline   Reply With Quote
Old 2nd March 2005, 02:15   #18  |  Link
Sagittaire
Testeur de codecs
 
Sagittaire's Avatar
 
Join Date: May 2003
Location: France
Posts: 2,489
Quote:
[i]Originally posted by bobololo

I believe the target bitrate was 458000 bits/s, am I wrong ?

I explicitly gave it
6811 Ko for Nero AVC and 6852 Ko for VP7 ... it's less than 1% ... difficult to make better ...
__________________
Le Sagittaire ... ;-)

1- Ateme AVC or x264
2- VP7 or RV10 only for anime
3- XviD, DivX or WMV9
Sagittaire is offline   Reply With Quote
Old 2nd March 2005, 02:25   #19  |  Link
CruNcher
Registered User
 
CruNcher's Avatar
 
Join Date: Apr 2002
Location: Germany
Posts: 4,949
Sagittaire please keep the values correct the XviD-450.avi is not 450 kbps as stated in the table it's 457.0 kbps
__________________
all my compares are riddles so please try to decipher them yourselves :)

It is about Time

Join the Revolution NOW before it is to Late !

http://forum.doom9.org/showthread.php?t=168004
CruNcher is offline   Reply With Quote
Old 2nd March 2005, 02:45   #20  |  Link
Sagittaire
Testeur de codecs
 
Sagittaire's Avatar
 
Join Date: May 2003
Location: France
Posts: 2,489
All my files are upload and hyperlink in post ...

And 450 must be 450 Kbit/s too (like for VP6 or WMV9) ... in fact the most important for size/bitrate is to respect [6810 Ko - 6960 Ko] interval and [13660 Ko - 13940 Ko] interval. With same codec/setting difference between 2% for size is very small for metric (perhabs 0.1 dB for PSNR and 0.1 for SSIM)
__________________
Le Sagittaire ... ;-)

1- Ateme AVC or x264
2- VP7 or RV10 only for anime
3- XviD, DivX or WMV9
Sagittaire is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 00:14.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, vBulletin Solutions Inc.