"Realistically Insane" H.264 samples?

[benwaggoner]

So, we're making good progress on the H.264 support for Silverlight we announced a ways back:

http://on10.net/blogs/benwagg/H264-and-AAC-support-coming-in-Silverlight/

While we've got a pretty big library of content for testing, I wanted to see if anyone had any particularly challenging or complex files that might be worth us testing.

In particular, I don't have a lot of content with very complex multiple reference frame and pyramid B-frame patterns; our encodes allow for those things, but the sources we're using aren't triggering anything all that unusual.

Is there any content out there anyone can suggest we take a look at?

[Atak_Snajpera]

Are you looking for 1080p@59.94fps 50MBps samples :) ?

[Dark Shikari]

There are a bunch of samples exhibiting various features and trickiness over at samples.mplayerhq.hu (http://samples.mplayerhq.hu/V-codecs/h264/). There's also all of the official compliance testing streams. Here's a list of some I have (I don't know if I'm allowed to upload them or not).

BA1_FT_C.264
BA1_Sony_D.264
BA2_Sony_F.264
BA3_SVA_C.264
BAMQ1_JVC_C.264
BAMQ2_JVC_C.264
BANM_MW_D.264
BASQP1_Sony_C.264
BA_MW_D.264
CABA1_SVA_B.264
CABA1_Sony_D.264
CABA2_SVA_B.264
CABA2_Sony_E.264
CABA3_SVA_B.264
CABA3_Sony_C.264
CABA3_TOSHIBA_E.264
CABACI3_Sony_B.264
CABAST3_Sony_E.264
CABASTBR3_Sony_B.264
CABREF3_Sand_D.264
CACQP3_Sony_D.264
CAFI1_SVA_C.264
CAMA1_Sony_C.264
CAMA1_TOSHIBA_B.264
CAMA3_Sand_E.264
CAMACI3_Sony_C.264
CAMANL1_TOSHIBA_B.264
CAMANL2_TOSHIBA_B.264
CAMANL3_Sand_E.264
CAMASL3_Sony_B.264
CAMP_MOT_MBAFF_L30.264
CAMP_MOT_MBAFF_L31.264
CANL1_SVA_B.264
CANL1_Sony_E.264
CANL1_TOSHIBA_G.264
CANL2_SVA_B.264
CANL2_Sony_E.264
CANL3_SVA_B.264
CANL3_Sony_C.264
CANL4_SVA_B.264
CANLMA2_Sony_C.264
CANLMA3_Sony_C.264
CAPA1_TOSHIBA_B.264
CAPAMA3_Sand_F.264
CAPCM1_Sand_E.264
CAPCMNL1_Sand_E.264
CAPM3_Sony_D.264
CAQP1_Sony_B.264
CAWP1_TOSHIBA_E.264
CAWP5_TOSHIBA_E.264
CI1_FT_B.264
CI_MW_D.264
CVBS3_Sony_C.264
CVCANLMA2_Sony_C.264
CVFC1_Sony_C.264
CVFI1_SVA_C.264
CVFI1_Sony_D.264
CVFI2_SVA_C.264
CVFI2_Sony_H.264
CVMA1_Sony_D.264
CVMA1_TOSHIBA_B.264
CVMANL1_TOSHIBA_B.264
CVMANL2_TOSHIBA_B.264
CVMAPAQP3_Sony_E.264
CVMAQP2_Sony_G.264
CVMAQP3_Sony_D.264
CVMP_MOT_FLD_L30_B.264
CVMP_MOT_FRM_L31_B.264
CVNLFI1_Sony_C.264
CVNLFI2_Sony_H.264
CVPA1_TOSHIBA_B.264
CVPCMNL1_SVA_C.264
CVPCMNL2_SVA_C.264
CVSE2_Sony_B.264
CVSE3_Sony_H.264
CVSEFDFT3_Sony_E.264
CVWP1_TOSHIBA_E.264
CVWP2_TOSHIBA_E.264
CVWP3_TOSHIBA_E.264
CVWP5_TOSHIBA_E.264
FI1_Sony_E.264
FREXT01_JVC_D.264
FREXT02_JVC_C.264
FRExt1_Panasonic.264
FRExt3_Panasonic.264
FRExt4_Panasonic.264
Freh12_B.264
Freh1_B.264
Freh2_B.264
Freh7_B.264
HCAFF1_HHI.264
HCAFR1_HHI.264
HCAFR2_HHI.264
HCAFR3_HHI.264
HCAFR4_HHI.264
HCAMFF1_HHI.264
HCBP1_HHI_A.264
HCBP2_HHI_A.264
HCHP1_HHI_B.264
HCHP2_HHI_A.264
HCHP3_HHI_A.264
HCMP1_HHI_A.264
HPCADQ_BRCM_B.264
HPCAFLNL_BRCM_C.264
HPCAFL_BRCM_C.264
HPCALQ_BRCM_B.264
HPCAMAPALQ_BRCM_B.264
HPCANL_BRCM_C.264
HPCA_BRCM_C.264
HPCVFLNL_BRCM_A.264
HPCVFL_BRCM_A.264
HPCVNL_BRCM_A.264
HPCV_BRCM_A.264
HVLCFI0_Sony_B.264
HVLCMFF0_Sony_B.264
HVLCPFF0_Sony_B.264
LS_SVA_D.264
MIDR_MW_D.264
MPS_MW_A.264
MR1_BT_A.264
MR1_MW_A.264
MR2_MW_A.264
MR2_TANDBERG_E.264
MR3_TANDBERG_B.264
MR6_BT_B.264
MR7_BT_B.264
MR8_BT_B.264
MR9_BT_B.264
Makefile
Makefile.vs
NL1_Sony_D.264
NL2_Sony_H.264
NL3_SVA_E.264
NLMQ1_JVC_C.264
NLMQ2_JVC_C.264
NRF_MW_E.264
SL1_SVA_B.264
SVA_BA1_B.264
SVA_BA2_D.264
SVA_Base_B.264
SVA_CL1_E.264
SVA_FM1_E.264
SVA_NL1_B.264
SVA_NL2_E.264
Sharp_MP_Field_1_B.264
Sharp_MP_Field_2_B.264
Sharp_MP_Field_3_B.264
Sharp_MP_PAFF_1r2.264
Sharp_MP_PAFF_2.264
cama1_vtc_c.264
cama2_vtc_b.264
cama3_vtc_b.264
camp_mot_fld0_full.264
camp_mot_frm0_full.264
camp_mot_mbaff0_full.264
camp_mot_picaff0_full.264
cvmp_mot_fld0_full_B.264
cvmp_mot_frm0_full_B.264
cvmp_mot_mbaff0_full_B.264
cvmp_mot_picaff0_full_B.264
freh10.264
freh11.264
freh3.264
freh4.264
freh5.264
freh6.264
freh8.264
freh9.264
mbaff_weighted.264
src19td.264

One thing I might request is lossless support: while you might claim it isn't very useful, it requires almost no code. I imagine it might actually be useful for things like screen captures and such where transform-based coding would be a rather bad idea.

Some particular features I know that exist (or may be useful in the future for actual, real encoding) but may break some decoders are:

Multiple uses of the same frame in a reference list, usually with different weights (a certain software decoder breaks with more than 2 uses of a frame in the same list).
Multiple uses of the same frame in two different reference lists (useful for 8thpel motion compensation).
Streams consisting of nothing but B-frames (except for the I-frame at the start). This is possible with creative frame reordering and can be trivially implemented in x264, though I don't have the patch on me.
Multi-level B-pyramid. Scientific Atlanta encoders use two-level B-pyramid with 7 B-frames max.
IPCM blocks. Used by x264 and Ateme encoders at high bitrates.
Motion vectors that go really far off the frame--that is, more than the distance that would be considered useful (needlessly long). This has caught a few decoders in the past.
Motion vectors longer than the max range specified in the standard. Not spec compliant, but even some Blu-rays have very long MVs, so it's something that has to be dealt with for streams in the wild.
Open-GOP + seek points instead of IDR frames.
High Profile levelcodes in CAVLC.

[tetsuo55]

the killa sample might help, dont know where to find it though

[benwaggoner]

Are you looking for 1080p@59.94fps 50MBps samples :) ?
Nah, we're targeting web, so I don't think that we'll see much more than 720p during this version. It's more tricky or unusual stuff that could trip up our decoder, particuarly in performance.

[benwaggoner]

@Dark Shakiri,

Great feedback, thanks. I'll pass it on to our test team.

As for lossless, I'll check to see if we support it. For security reasons, Silverlight tends to turn of features we can't test; we're appropriately paranoid about shipping any feature that we don't have a full test plan and threat model for.

Is there any good source for lossless H.264 files as well?

[Dark Shikari]

@Dark Shakiri,

Great feedback, thanks. I'll pass it on to our test team.

As for lossless, I'll check to see if we support it. For security reasons, Silverlight tends to turn of features we can't test; we're appropriately paranoid about shipping any feature that we don't have a full test plan and threat model for.

Is there any good source for lossless H.264 files as well?There are two types of H.264 lossless in the wild.

One is generated with older versions of x264 (still in use, and of course existing files), and I think there is at least one other H.264 encoder that supports it.

This mode is specified in the 2005 H.264 spec and is extremely simple: the DCT is removed and instead the residuals from each 4x4 or 8x8 block are directly coded to the bitstream (transform_bypass).

Samples of this can be generated trivially with x264 revisions prior to r994 (http://git.videolan.org/?p=x264.git;a=commit;h=b35a044b95c7eab2c91f55a3ac4100ca26a29d92) by using --qp 0.

The new lossless mode is the same (specified in the 2007 spec as High 4:4:4 Predictive profile), except that in intra prediction (for 4x4, 8x8, 16x16, and 8x8 chroma), the "horizontal" and "vertical" prediction modes use pixel prediction. E.g., for horizontal, each pixel is predicted to be the one to its left, and for vertical, each pixel is predicted to be the one to its top. This replaces the normal prediction methods, of course.

To generate this new lossless, use a newer x264 with --qp 0. Note that despite the profile names, x264 does not support more than 4:2:0 in either.

The two modes can be distinguished by looking at the profile_idc.

Back to the topic of "weird things that need to be noted", one thing that broke libavcodec decoding at one point is that CAVLC+8x8dct has a different ordering of nnz values for deblocking than for residual coding. This is important and forgetting this can lead to enormous amounts of artifacting on some samples. This is actually something that may be seen in the wild often, as 8x8dct does not increase decoding requirements but CABAC does, so people trying to minimize decoding speed cost for their videos may use CAVLC and 8x8dct.