Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
|
|
Thread Tools | Search this Thread | Display Modes |
19th August 2006, 13:44 | #1 | Link |
Testeur de codecs
Join Date: May 2003
Location: France
Posts: 2,484
|
[HD-DVD Challenge] MPEG2, VC-1 and H264 with real uncompressed source movie
~ HD-DVD Benchmark Challenge ~
You think that your standard is the best : prove that ... !!? A - Introduction The purpose of this challenge is to determine which is the best codec format encoding. HD-DVD & BD can use MPEG2, H264 and VC-1 for video codec. B - Rules Rule 1 : Source Uncompressed source is available Here for reproduce the test. Open source Elephant Dream movie, video 1920*1080 PNG lossless, audio 5.1 Flac lossless, 15 691 frames Rule 2 : Pre-process and encoding You must use this avisynth script for make encoding: Code:
Source=ImageSource("C:\LossLess\images\%05d.png", start=1, end=15691, fps=23.976) Source=ConvertToYV12(source, matrix="Rec709") return source Other pre-process is not autorized. It's possible to propose example with high quality pre-process (objective like high color conversion or subjective like sharp process ... ect) but just for demonstration and not for this challenge. Video Stream must be like HD-DVD compliant stream. You can use all the compliant audio codec and all the compliant bitrate for audio. This test is just video challenge and not audio challenge. All the video stream must be compliant with these setting: MPEG2 Encoding Profil & Level: MP@HL except specific restrictions Max GOP lenght: 14 frames Maximum bitrate: 20.0 Mbps, 24.0 Mbps and 28.0 Mbps Buffer size: 9781 Kbits for principal HD video stream Horizontal Vector Range: +/- 1024 pixels Vertical Vector Range: +/- 128 pixels Other Restrictions Setting: max adaptative GOP at 14, max adaptative bframe at 2 VC-1 Encoding Profil & Level: AP@L3 except specific restrictions Max GOP lenght: 14 frames Maximum bitrate: 20.0 Mbps, 24.0 Mbps and 28.0 Mbps Buffer size: 14745 Kbits for principal HD video stream Horizontal Vector Range: +/- 1024 pixels Vertical Vector Range: +/- 256 pixels Other Restrictions Setting: max adaptative GOP at 14, max adaptative bframe at 2 H264 Encoding Profil & Level: HP@L4.1 except specific restrictions Max GOP lenght: 14 frames Maximum bitrate: 20.0 Mbps, 24.0 Mbps and 28.0 Mbps Buffer size: 14745 Kbits for principal HD video stream Horizontal Vector Range: +/- 1024 pixels Vertical Vector Range: +/- 512 pixels Other Restrictions Setting: max adaptative GOP at 14, max adaptative bframe at 2, Max reference at 4, Max breference at 3, no film grain modeling Rule 3 : Bitrate and Max Bitrate You must use these bitrate/size for encoding: HD-DVD with "super bitrate" video stream and simple HDDVD authoring: 18 Mbps (Max at 28.0 Mbps) for video stream with +/- 0.5 % for bitrate tolerance HD-DVD with "medium bitrate" video stream and standard HDDVD authoring: 12 Mbps (Max at 24.0 Mbps) for video stream with +/- 0.5 % for bitrate tolerance HD-DVD with "low bitrate" video stream and standard HDDVD authoring: 6 Mbps (Max at 20.0 Mbps) for video stream with +/- 0.5 % for bitrate tolerance NB: we can use HD-DVD structure too on simple DVD DL 12 cm at 8.5 GB Rule 4 : Visual Tests Here you can find encoding in ifo & evo structure for subjective test ... try to find yourself the best quality: HD-DVD & BD MPEG2 at 18 Mbps : File: not available HD-DVD & BD MPEG2 at 12 Mbps : File: not available HD-DVD & BD MPEG2 at 6 Mbps : File: not available HD-DVD & BD VC-1 at 18 Mbps : File: not available HD-DVD & BD VC-1 at 12 Mbps : File: not available HD-DVD & BD VC-1 at 6 Mbps : File: not available HD-DVD & BD H264 at 18 Mbps : File: not available HD-DVD & BD H264 at 12 Mbps : File: not available HD-DVD & BD H264 at 6 Mbps : File: not available Rule 5 : Metrics Tests SSIM (YV12): IMO the best metric. it uses some HVS properties but not all ... PDF description for VSSIM Overall PSNR (YV12): Very good metric test, particullary for Rate Control Quality test ... MSE = Sum t = 1 -> NFrame Sum x = 1 -> width Sum y = 1 -> height (F'(x,y,t)-F(x,y,t))² / (NFrame*width*height) Overall PSNR = 10 * Log ( 255² / MSE) Average PSNR (YV12):The worse metric test but Frame PSNR graph is very good to compare RC strategy ... MSE = Sum x = 1 -> width Sum y = 1 -> height (F'(x,y,t)-F(x,y,t))² / (width*height) Average PSNR = Sum t = 1 -> NFrame 10 * Log ( 255² / MSE) / Nframe SSIM vs PSNR "For a given reference video and a given compressed video, it is meant to compute a quality metric, based on perceived visual distortion. Unlike the well-known PSNR measure, it's not purely mathematical, and should correlate much better with human vision." "A higher MSE (and so lower PSNR) should mean that the compressed clip is a worse image but MSE and PSNR are flawed in this respect as numerous tests have shown. However with SSIM, according to tests carried out on the VQEG dataset, a higher Q (SSIM value) has a much better relation to the visual quality of the compressed clip. Despite this, bear in mind the SSIM metric still isn't perfect." Here last AviSynth plugin for VSSIM in YV12 Average PSNR vs Overall PSNR It's easy to increase artificially Average PSNR with Rate Control tweak. In certain case CBR (0% for variability) will be better than real VBR (100% for variability or constant quant). It's generaly not possible with Overall PSNR. Overall PSNR is very good to test Rate Control quality. If difference between Average PSNR and Overall PSNR is very high, "local quality difference" (difference between Pframe or Bframe for exemple) or "scene quality difference" (difference between low motion and high motion for exemple) will be too high and Rate Control for this codec not very good. Here last AviSynth plugin for Overall PSNR in YV12 Open your avs metric script with VirtualDub. Select file > preview input for run the test. Select file > close video file for obtain final result in log metric files. You must use these AviSynth type script for metric test: Code:
# --> Source Opening <-- Source=ImageSource("C:\LossLess\images\%05d.png", start=1, end=15691, fps=24) Source=ConvertToYV12(source, matrix="Rec709") # --> Video Opening <-- video=Mpeg2Source("D:\...\MPEG2_24Mbps.d2v",idct=2) # --> PSNR analysis <-- compareYV12(video,source,"YUV","OPSNR_MPEG2_24Mbps.log") Code:
# --> Source Opening <-- Source=ImageSource("C:\LossLess\images\%05d.png", start=1, end=15691, fps=24) Source=ConvertToYV12(source, matrix="Rec709") # --> Video Opening <-- video=Mpeg2Source("D:\...\MPEG2_24Mbps.d2v",idct=2) # --> SSIM analysis <-- return SSIM(source,video,"results.csv","SSIM_MPEG2_24Mbps.txt",lumimask=2) C - Results At this time here the best result ... but done better if you can ... Code:
|--------------|---------|---------|---------|---------|---------| | Codec | PProc | Bitrate | Size | OPSNR | SSIM 2 | |--------------|---------|---------|---------|---------|---------| | MPEG2 | PP4 | 0000 | 0000 | 00.00 | 00.00 | | VC-1 | PP1 | 0000 | 0000 | 00.00 | 00.00 | | H264 | PP0 | 0000 | 0000 | 00.00 | 00.00 | |--------------|---------|---------|---------|---------|---------| | MPEG2 | PP4 | 00000 | 0000 | 00.00 | 00.00 | | VC-1 | PP1 | 00000 | 0000 | 00.00 | 00.00 | | H264 | PP0 | 00000 | 0000 | 00.00 | 00.00 | |--------------|---------|---------|---------|---------|---------| | MPEG2 | PP4 | 00000 | 0000 | 00.00 | 00.00 | | VC-1 | PP1 | 00000 | 0000 | 00.00 | 00.00 | | H264 | PP0 | 00000 | 0000 | 00.00 | 00.00 | |--------------|---------|---------|---------|---------|---------| SSIM 0: Lumimask Off SSIM 1: Lumimask On (Original Lumimask) SSIM 2: Lumimask On (One2Tech Patch) D - Conclusion At this time for this movie THE BEST IS ?????????? IF YOU DON'T AGREE WITH THAT, TRY WITH YOUR ENCODER ... !!! Annexe - Update 01.07.2007 - NTSC setting for encoding replace PAL setting 26.10.2006 - New rules for more real HDDVD encoding (new bitrate, new vbv) 03.10.2006 - buffer for H264 is confidential but use 14400 Kbits is certainely compliant with HDDVD 02.10.2006 - official buffer for VC1 is 14400 Kbits or 480 ms for better HD-DVD compliancy 26.09.2006 - MPEG2 and H264 files with audio are available 19.09.2006 - MPEG2 metric result 18.09.2006 - H264 metric result will coming ... if you want ... !!! All the developpers are wellcome ... !!?
__________________
Le Sagittaire ... ;-) 1- Ateme AVC or x264 2- VP7 or RV10 only for anime 3- XviD, DivX or WMV9 Last edited by Sagittaire; 30th July 2007 at 11:10. |
19th August 2006, 13:58 | #2 | Link |
brainless
Join Date: Mar 2003
Location: Germany
Posts: 3,653
|
Code:
Source=ImageSource("C:\LossLess\images\%05d.png", start=1, end=15691, fps=24) Source=ConvertToYV12(source, matrix=whatever) return source MPEG2 encoders for example commonly only accept YUY2 or RGB24 input. If YV12 is given to the encoder it either throws an error or converts it to YUY2 using DShow (or something) and then converts it to YV12 again. This results in blurring the chroma in vertical. The only MPEG2 Encoders I know of that accept YV12 without conversion are HCEnc and QuEnc. CCE and ProCoder will only work with YUY2. As well as TMPGenc does. So I suggest to use Code:
Source=ImageSource("C:\LossLess\images\%05d.png", start=1, end=15691, fps=24) Source=ConvertToYUY2(source, matrix=whatever) return source
__________________
Don't forget the 'c'! Don't PM me for technical support, please. Last edited by scharfis_brain; 3rd September 2006 at 11:50. |
19th August 2006, 14:36 | #3 | Link | |
Testeur de codecs
Join Date: May 2003
Location: France
Posts: 2,484
|
Quote:
I will make HD-DVD MPEG2 encoding with Mencoder ... ;-)
__________________
Le Sagittaire ... ;-) 1- Ateme AVC or x264 2- VP7 or RV10 only for anime 3- XviD, DivX or WMV9 Last edited by Sagittaire; 19th August 2006 at 14:50. |
|
19th August 2006, 14:54 | #5 | Link | |
Registered User
Join Date: Mar 2006
Location: Grand StrateGuerre
Posts: 362
|
Quote:
I think the size resolution is locked. Because MPEG2 and VC1 at 6Mbps with 1920x1080 resolution, it will be very hard. And there is not a limit for encoding pass. (NTT Docomo have N-pass H264 encoder => 2.8Mbps for video at 1920x1080 24fps, thirteen pass and one pass isn't in realtime...) Golgot13 |
|
19th August 2006, 15:20 | #6 | Link | |||
Testeur de codecs
Join Date: May 2003
Location: France
Posts: 2,484
|
Quote:
Quote:
Quote:
__________________
Le Sagittaire ... ;-) 1- Ateme AVC or x264 2- VP7 or RV10 only for anime 3- XviD, DivX or WMV9 Last edited by Sagittaire; 19th August 2006 at 15:28. |
|||
19th August 2006, 17:38 | #7 | Link |
brainless
Join Date: Mar 2003
Location: Germany
Posts: 3,653
|
@Sagittaire: I think you're aiming for high quality?
So why do you forbit YUY2 when needed for good quality? Some encoders even won't accept YV12 and will refuse to encode the video. So you'll knock out valuable encoders in this Benchmark Challange! I think to stick with YUY2 and correct colorimetry will be enough. Converting from RGB->YUY2->YV12 is a transparent process. Converting from RGB->YV12->YUY2->YV12 isn't.
__________________
Don't forget the 'c'! Don't PM me for technical support, please. |
19th August 2006, 17:53 | #8 | Link |
Life's clearer in 4K UHD
Join Date: Jun 2003
Location: Notts, UK
Posts: 12,219
|
I must admit I would be interested to know how well MPEG-4 AVC's "lossless" encoding option would cope with these sources.... If anybody is up for it?
By-the-way, does VC-1 have a "lossless" encoding option? Cheers
__________________
| I've been testing hardware media playback devices and software A/V encoders and decoders since 2001 | My Network Layout & A/V Gear |
|
19th August 2006, 19:08 | #9 | Link | |
Testeur de codecs
Join Date: May 2003
Location: France
Posts: 2,484
|
Quote:
__________________
Le Sagittaire ... ;-) 1- Ateme AVC or x264 2- VP7 or RV10 only for anime 3- XviD, DivX or WMV9 Last edited by Sagittaire; 19th August 2006 at 19:11. |
|
19th August 2006, 19:43 | #10 | Link |
Registered User
Join Date: Jan 2004
Posts: 849
|
I get a message saying that torrent is invalid, is anyone else having trouble getting it to work?
__________________
Geforce GTX 260 Windows 7, 64bit, Core i7 MPC-HC, Foobar2000 Last edited by lexor; 19th August 2006 at 19:47. |
19th August 2006, 21:25 | #11 | Link |
Registered User
Join Date: Jan 2006
Posts: 57
|
@Sagittaire
a) What about the HD-DVD restictions like max (peak) video bitrate (29Mbps) and others (GOP size, reference B frames.. etc) we read about (I realy dont know if those "others" exist because of the HD-DVD format or the decoding chip.) b) Isn't this test a good opportunity to to compare HD-DVD vs BD ??. Like in the following case: Todays bests -> HD-DVD DL(30GB, 29Mbps peak video bitrate) VS BD SL (25GB 40Mbps peak video bitrate) This will need one more category for BD (20Mbps average video bitrate with peaks up to 40Mbps). Last edited by Dethis; 19th August 2006 at 21:31. |
19th August 2006, 21:41 | #12 | Link | ||
Testeur de codecs
Join Date: May 2003
Location: France
Posts: 2,484
|
Quote:
adaptative GOP 15 with 2 adaptative bframe Buffer size at 448 Ko, max bitrate at 30 000 Kbps Max motion vector at +/- 128 pixel Quote:
__________________
Le Sagittaire ... ;-) 1- Ateme AVC or x264 2- VP7 or RV10 only for anime 3- XviD, DivX or WMV9 |
||
20th August 2006, 03:26 | #15 | Link |
Registered User
Join Date: Apr 2002
Location: Germany
Posts: 4,926
|
x264 cant be used here as it isn't Broadcast stable and exactly that is needed here for STB/SAP Encoding, what this test is about
so we only have 2 options Ateme (wich most of the users here only have the very outdated Beta) or Elecard/Mainconcepts Encoder wich would obey to the Hardware limits needed for this test. Im sure that Atemes outdated Beta can't cope anymore with Elecard/Mainconcepts Encoder and also it's rather useless to test such stuff as HD-DVD H.264 encoding won't be available still for some time now for most average users their only option @ the moment is Mpeg-2 HD-DVD, and no i have no idea how much longer the industry is holding H.264 HD-DVD encoding back from the average joe user, my guess is end of this year begining next year.
__________________
all my compares are riddles so please try to decipher them yourselves :) It is about Time Join the Revolution NOW before it is to Late ! http://forum.doom9.org/showthread.php?t=168004 Last edited by CruNcher; 20th August 2006 at 03:40. |
20th August 2006, 03:32 | #16 | Link | |
x264 developer
Join Date: Sep 2004
Posts: 2,392
|
Quote:
|
|
20th August 2006, 08:51 | #18 | Link | |
Testeur de codecs
Join Date: May 2003
Location: France
Posts: 2,484
|
Quote:
__________________
Le Sagittaire ... ;-) 1- Ateme AVC or x264 2- VP7 or RV10 only for anime 3- XviD, DivX or WMV9 |
|
20th August 2006, 09:11 | #19 | Link | |
Registered User
Join Date: Feb 2006
Posts: 823
|
Ok I'll repeat my question.
Quote:
|
|
20th August 2006, 09:40 | #20 | Link | |
Testeur de codecs
Join Date: May 2003
Location: France
Posts: 2,484
|
Quote:
__________________
Le Sagittaire ... ;-) 1- Ateme AVC or x264 2- VP7 or RV10 only for anime 3- XviD, DivX or WMV9 |
|
Thread Tools | Search this Thread |
Display Modes | |
|
|