Define Testing Of Hardware Players

[cdrinfo]

Hi people,

We have some upcoming reviews of DivX enabled desktop DVD players. We want to improve our testing so this thread has the purpose to give us some hints about it. The source was EU Gladiator Disc 1 (PAL), ripped 5mins and compressed with GKnot 0.34.5. What we have so far created are:

- DivX 5.2.1 (using standard profiles)
Divx521_HomeTheater_576x240x700Kbit_AC3
Divx521_HomeTheater_702x288x1.5Mbit_128MP3
Divx521_HomeTheater_702x288x1Mbit_128MP3
Divx521_HomeTheater_702x288x2Mbit_128MP3
Divx521_HomeTheater_702x288x4Mbit_128MP3
Divx521_HD_702x288x6Mbit_128MP3
Divx521_HD_702x288x8Mbit_128MP3
DivX521_HD_1280x720x?Mbit_MeetTheFockers (D/l from Divx Website)

- DivX 5.2.1 (non-standard - Single Consecutive)
Divx521_GMC_702x288x1Mbit_128MP3
Divx521_QP_702x288x1Mbit_128MP3
Divx521_QP_GMC_702x288x1Mbit_128MP3

- XVid 1.1.beta1
XVid1.1_704x288x1Mbit_128MP3
XVid1.1_BVOP_704x288x1Mbit_128MP3
XVid1.1_BVOP_PS_704x288x1Mbit_128MP3
XVid1.1_BVOP_PS_QP_704x288x1Mbit_128MP3
XVid1.1_BVOP_PS_QP_GMC_704x288x1Mbit_128MP3
XVid1.1_BVOP_QP_GMC_704x288x1Mbit_128MP3

- WMV9 (D/l from Microsoft)
WMVHD_The_Magic_of_Flight_720P
WMVHD_The_Magic_of_Flight_1080P

- Nero Digital (Mp4) (D/l from Nero website)
ND_Mobile_176x144_400Kbps_Stereo
ND_Portable_352x288_2Mbps_Stereo
ND_Standard_720x576_4Mbps_Stereo
ND_Cinema_1280x720_9.8Mbps_Stereo
ND_HDTV_1920x1072_9.8Mbs_Stereo

I imagine there are more test files that can be created, either from us from other members, your help is well appreciated for deeper testing and better reviews

Some test files that will added soon

- AVI
Uncompressed AVI around 4GB

- Mpeg1
VCD
XVCD

- Mpeg2
SVCD
DVD
Mpeg2-TS (downloaded several clips from Internet)

[Doom9]

XVid1.1_BVOP_PS_704x288x1Mbit_128MP3

what is PS? And why only use 1 MBit for all XviD clips whereas DivX is tested in different bitrates?

What about audio, CBR, VBR? Why not try two audio tracks, and mix audio tracks (1x VBR, 1x AC3).

When it comes to NeroDigital, I'm sure many will be interested if players can handle HE AAC, and multichannel HE AAC.

You might want to try different resolutions as well to see if they are properly scaled.

Then of course, subs, be it srt or VobSub.

Last but not least, I started to devise a player test in 2003: http://forum.doom9.org/showthread.php?s=&threadid=62239

[SeeMoreDigital]

Hi cdrinfo,

I doubt many people would want to encode DivX (or any other Mpeg4 codec) at 702x288 at any bit-rate.

Depending on whether you are generating cropped and re-sized "square pixelled" encodes to correlate with the aspect ratio of the source/movie then, 702x288 would equate to an AR of 2.43:!

Maybe, the following would be more appropriate: -

• 688x288 = 2.39:1 (near as damn to 2.40:1)
• 672x288 = 2.33:1 (near as damn to 2.35:1)
• 528x288 = 1.83:1 (near as damn to 1.85:1)
• 512x288 = 1.77:1 (aka 16:9)
• 448x288 = 1.55:1 (aka 14:9)
• 384x288 = 1.33:1 (aka 4:3)

Or even: -

• 720x304 = 2.37:1 (between 2.35:1 and 2.40:1)
• 720x384 = 1.87:1 (near as damn to 1.85:1)
• 720x400 = 1.80:1 (near as damn to 1.77:1 aka 16:9)
• 720x464 = 1.55:1 (aka 14:9)
• 720x544 = 1.32:1 (near as damn to 1.33:1 aka 4:3)

I guess "PS" must refer to "Packed Bit-stream"


Cheers

[spectra]

Originally posted by cdrinfo
[B]Hi people,

We have some upcoming reviews of DivX enabled desktop DVD players. We want to improve our testing so this thread has the purpose to give us some hints about it.

perhaps you should take a peek to the divxtest.com crew. they already have a test-cd and result browser ( http://divxtest.com/resultats.php3 ) out. They're just planning a new version 2 of their cd ( http://forum.surdvd.com/viewtopic.php?t=34506 ).

[bond]

to get an idea about testing various mpeg-4 encoders and settings have a look at what i did with my MP4 testing samples discussed here

i dont think it makes much sense to test the private profiles defined in divx5/nerodigital as they will of course be supported in the certified players (i am sure they are tested by the manufacturers themselves)...
what is interesting is testing the real, full capabilities of the players: things that are covered by the official mpeg-4 advanced simple profile (and useable in the encoders) but go beyond the private profiles, eg multiple b-frames, qpel, gmc (1 warppoing and 3 warppoints), mpeg quants, all sorts of custom quants, adaptive quantisation

also what is interesting would be testing not only divx5, xvid and nd, but also other mpeg-4 codecs, like 3ivx, ffmpeg, quicktime aso... which of course also should be playable on mpeg-4 players as their streams are also mpeg-4 compliant of course

[cdrinfo]

First i have to thank you all for your comments and suggestions. As you understand its quite hard, not even impossible to cover all needs for testing, and at the same time, keep the testing at a normal time frame for our reviewers, but we will try for the best.

My initial idea, after reading this forum's hardware player section was to improve our hardware tests with Mpeg4 enabled players and see if some neat features are not supported or how well are supported. For example a common problem is the support for European Eastern languages (like Greek), for file names and subtitles...Some answers to the questions:

@Doom9: PS, sorry for the mistype, SeeMoreDigital was correct, is Packet Bit-Stream, already renamed the file name :-). As for an AVI file with 2 audio formats (Mp3/AC3), I will add it to the do to list. As for the 1Mbit for all XVid files, i had read that some player cannot play above 4,6Mbit encoding, so i thought to make from 1~8Mbit Divx test files (testing Encoding bitrate), since most Mpeg4 players supposed to be DivX compatible. Making from 1~8Mbit Xvid test files, i don't feel its necessery, at least for testing the bitrate decoding strength of a Mpeg4 player.

@SeeMoreDigital: Sorry for the mistype, the video size for all encodings was 704x288 (aspect ratio 2.444). I will make again some files for checking the aspect ratio, according to your suggestions (why GKnot won't let me enter manually your suggested video file sizes?).

@spectra: Thanks for the tip, even its in french i will try to see what they have come up...

@bond: I have already downloaded your test disc for testing .mp4 players, so we will use it for the upcoming test reports. As for testing (like 3ivx, ffmpeg, quicktime aso...), i think we can find some Quicktime mpeg4 files, as for using 3ivx, do you find its neseccery, since most people use either DivX or XVid?

[SeeMoreDigital]

Originally posted by cdrinfo
...Sorry for the mistype, the video size for all encodings was 704x288 (aspect ratio 2.444). I will make again some files for checking the aspect ratio, according to your suggestions (why GKnot won't let me enter manually your suggested video file sizes?).

I don't use Gordian Knot myself... but I think I remember some people saying they have to disable GK's "ITU" setting.

As far as "high-bitrates" are concerned, I wonder how many people actually generate Mpeg4 encodes above 6000Kbps. And if they do, I can't imagine they would be 2pass VBR encodes!

Which begs another question, if some players are b0rking at high bit-rates. Do they b0rk at the same bit-rate for both CBR and VBR streams?

You might also want to consider including "dedicated Mpeg4 PAR/DAR signalling tests" for both the .AVI and .MP4 containers!


Cheers

[spectra]

Originally posted by cdrinfo

@spectra: Thanks for the tip, even its in french i will try to see what they have come up...

most of it is selfexplaining (at least with babelfish) and if you ask in english, i'm sure they will answer

[bond]

Originally posted by cdrinfo
@bond: I have already downloaded your test disc for testing .mp4 players, so we will use it for the upcoming test reports.

you can extract the files under "video" to .avi with mp4ui. that way you will get .avi files with "DIVX" fourcc, every player should support, which test specifically features, like qpel, custom quants aso...
and the streams are encoded with different mpeg-4 encoders too

As for testing (like 3ivx, ffmpeg, quicktime aso...), i think we can find some Quicktime mpeg4 files

no player will support .mov files
you will have to encode in quicktime yourself with apples mpeg-4 codec to .mp4 and than extract to .avi with mp4ui (or another mp4-enabled tool, like mp4box)

as for using 3ivx, do you find its neseccery, since most people use either DivX or XVid?

well altough the players are always advertised as "divx" players, they are in fact mpeg-4 players, so testing different mpeg-4 codecs (at least with one sample, eg following simple profile) should be done imho

and again i for myself dont see the sense in testing divx5 streams encoded following the divx profiles on players which are certified for that profile...
i mean of course the player will support it, people will not learn anything from a test that shows that the players support what the certification claims...
and i dont know any reports that there are problems with players not handling their certified profiles

i would expect from a test to learn the following things:
- does qpel work
- does 3 warppoint gmc work
- if not, does 1 warppoint gmc work
- do custom quants work
- does adaptive quant work
- do multiple b-frames work (packed or normally stored)
- what fourccs does the player handle

if those questions are answered it doesnt matter what codec you use to create a fitting stream, cause all are mpeg-4 compliant (of course you have to change the fourcc to something the player understands, eg DIVX)

[cdrinfo]

Whoa, that was pretty fast

@SeeMoreDigital: So your proposal is not to test high bitrates, or test them only at CBR, not VBR? Where i can find the dedicated Mpeg4 PAR/DAR signalling tests?

@spectra: Thanks, i have started downloading the CD already!

@bond: I see, so we can use your files to test the nasty points where most players will fail...I saw you have two versions of the same file, the included AVI is the same at both versions?

I also found some test files, which have Mpeg2@40Mbit encoded, probably not many players will handle them

[SeeMoreDigital]

Originally posted by cdrinfo
...So your proposal is not to test high bitrates, or test them only at CBR, not VBR? Where i can find the dedicated Mpeg4 PAR/DAR signalling tests?

Well there's got to be a limit at which Mpeg4 "tops-out" quality wise... Meaning, using higher bit-rates would offer no appreciable difference to the quality of the image!

As far as I know most Mpeg2 encoders top-out somewhere in the 9000+Kbps range. So given that Mpeg4 is supposed to be far a more efficient codec, one would assume its top-out bit-rate would be far less.

I started a thread about this very subject some time ago but it generated little enthusiasm!

When it comes to Mpeg4 encodes with PAR/DAR signalling, I guess I can generate some examples for you.


Cheers

[cdrinfo]

Divx 5.2.1 at non-certified mode, allows up to 16Mbit encoding...I have created some samples from 10-12-14-16Mbit just for a quick testing.

Edit: It seems that all files have the same file size, meaning DivX doesn't support above 8Mbit/sec encoding, i am wondering why they leave user the option to move slider up to 16Mbit..

>I guess I can generate some examples for you.

Great, your help is well appreciated

PS. Is there any util that can report possible QP/GMC encoding modes?

[SeeMoreDigital]

Originally posted by cdrinfo
Divx 5.2.1 at non-certified mode, allows up to 16Mbit encoding...I have created some samples from 10-12-14-16Mbit just for a quick testing.

Blimey this is high!

I guess DivX are expecting people will use such bit-rates when generating high-def/resolution encodes!

But as far as I'm aware, none of DivX's HD samples run higher than 4Mbps and none have been encoded above 1280x720 pixels... And I suppose this is because, none of the current high-def/resolution players are specified at being able to decode Mpeg4 resolutions higher than this!

That said, some users have reported that they are able to spin Mpeg4 at 1920x1088... so who knows!


Cheers

[cdrinfo]

>Blimey this is high!

Well doesn't work, as you thought correctly before, i have edited my initial post