x264 Known Hardware accelleration problems and solutions

[valnar]

Originally Posted by Atak_Snajpera

PS3 profile

Easy enough! Thanks.
Robert

[UsedUser]

Originally Posted by Atak_Snajpera

PS3 profile

Originally Posted by valnar

Easy enough! Thanks.
Robert

I'm seeing the following settings for the PS3 profile:

--level 4.1 --ref 3 --mixed-refs --bframes 3 --b-pyramid

It includes >2 b-frames, adaptive b-frames, and b-frame pyramids, all of which have been noted for their potential to break DXVA. Unless someone has already tested them @ 1080p + num_ref_frames < 5?

[tetsuo55]

someone will have to test them, but from what i can find on hardware players it seems that simply using profile 4.1 will fix the problems

[cscxk]

my test is for 1920*1080,only need to set ref 3 and no b-pyramid.for 1280*720,ref to 6 and no b-pyramid.so i think the first is disable b-pyramid.

[UsedUser]

Originally Posted by cscxk

my test is for 1920*1080,only need to set ref 3 and no b-pyramid.for 1280*720,ref to 6 and no b-pyramid.so i think the first is disable b-pyramid.

My suspicion is that b-pyramid is involved.

B-pyramid allows b-frames to be used as reference frames for other b-frames. So, if B-pyramid is "on", then the b-frames you have set are counted in the total num_ref_frames.

You can see this formula holds (partially) true in the AVInaptic output. With B-pyramids, num_ref_frames = ref_frames + b-frames (i.e., num_ref_frames = (3 ref_frames) + (3 b-frames) = 6). However, with B-pyramids, it must allow for extra references even at 2 b-frames, as num_ref_frames = (2 ref_frames) + (2 b-frames) = 5.

I'm currently testing the hypothesis that without b-pyramids, you can push the ref_frames up to the max allowed num_ref_frames for L4.1, e.g., 4 @ 1080p, 9 @ 720p.

[UsedUser]

Well, I made a bit of an error in thinking the ref value could be set to the max allowed num_ref_frames, because that wouldn't allow for the B-frames. So, I have to reject that hypothesis.

However, like cscxk, I have found that, with B-pyramids off, the magic value for ref is 3. num_ref_frames = (3 ref) + (3 B-frames) + (no-B-pyramids) = 4. With num_ref_frames = 4, DXVA works @ 1080p.

I'm thinking what we need, though, is more information about how to predict num_ref_frames.

My new hypothesis is that num_ref_frames won't always equal 4 when ref = 3 and B-frames = 3, if adaptive b-frames is ON.

With adaptive b-frames the encoder decides the number of (consecutive) B-frames, and the B-frames value is then used as a MAXIMUM, rather than a forced number of B-frames. So, my prediction is that for the clips I/we have tested, only 1 B-frame is actually being used with adaptive B-frames turned on. If I'm right, then num_ref_frames will be 6 if ref = 3, B-frames = 3, and B-adapt = no.

If this is true, then to force DXVA compatibility @ 1080p, we really need the following settings to use at most 1 consecutive B-frame and limit num_ref_frames to 4:

--level 4.1 --ref 3 --mixed-refs --bframes 1 --b-adapt

[UsedUser]

Well, reject that hypothesis as well.

I just tested a clip using the following settings, expecting num_ref_frames = 6:

--level 4.1 --ref 3 --mixed-refs --bframes 3 --no-b-adapt

Adaptive b-frames made no difference. num_ref_frames = 4 in both cases.

So, I still don't know what the magic formula is to arrive at num_ref_frames = 4, but with any clip I've encoded thus far, using the following settings has worked and resulted in successful DXVA @ 1080p:

--level 4.1 --ref 3 --mixed-refs --bframes 3 --b-adapt --no-b-pyramid

So, the only things that need to be changed from the PS3 profile for 1080p are to disable B-pyramids and increase the bitrate to your desired quality/filesize.

As for 720p, I've tested a number of clips with num_ref_frames from 4 to 8, all of which used DXVA, but with choppy playback. I used MP4 instead of MKV, so it isn't the 20fps bug I'm seeing. I need to do more 720p testing.

[CruNcher]

in some tests of this problem i found out that you can workaround it by useing drect temporal prediction instead of auto or spatial but im not sure why this was the case but it worked and yes b-pyramid shouldn't be used ever for DXVA compatibility and 2 ref frames btw is also the standard for example in Sonic Cinevision. Also something strange is with X264 bitstreams since the begining it seems the bitrate declared flag is wrong you can see this by renameing a .h264 bitstream to .mpv and playback it via PowerDVD then activate the OSD and you gonna see a non changeing bitrate value (also in the information tab the Value is to high for example you encoded AVG 7500 and Max is 10000 kbits it shows you 36 Mbits but it never reached that and only if you rename the bitstream to .mpv you gonna see it or analyzeing it with Elecard Stream analyzer there the Bitrate Declared Field is also visible) in the Counter wich is to high most of the times. With other Encoded Bitstreams from Nero (Ateme) and Mainconcept this value is changeing acordingly to the stream bitrate. Now if you use h264info on such a X264 bitstream without changeing any parameter just run it throug you gonna see suddenly that the counter is moveing but with extreme High values even higher then what it showed as the Hard value before useing h264info on it for example in the case above you gonna suddenly see 2200 Mbits in the Counter hehe, this doesn't seem right to me.

[cscxk]

on my system,this could play with DXVA.Job commandline:
--level 4.1 --keyint 999 --min-keyint 1 --ref 3 --mixed-refs --no-fast-pskip --bframes 8 --b-rdo --bime --weightb --direct auto --filter -3,-2 --subme 6 --trellis 1 --analyse p8x8,b8x8,i4x4,i8x8 --8x8dct --me umh --threads auto --thread-input --progress --no-dct-decimate --no-psnr

and you see,b-frames is 8

[valnar]

@UsedUser.

Keep those tests coming! Great work!

Robert

[UsedUser]

Originally Posted by cscxk

on my system,this could play with DXVA.Job commandline:
--level 4.1 --keyint 999 --min-keyint 1 --ref 3 --mixed-refs --no-fast-pskip --bframes 8 --b-rdo --bime --weightb --direct auto --filter -3,-2 --subme 6 --trellis 1 --analyse p8x8,b8x8,i4x4,i8x8 --8x8dct --me umh --threads auto --thread-input --progress --no-dct-decimate --no-psnr

and you see,b-frames is 8

I concur. B-frames can be pushed all the way up to 16 without breaking DXVA, as long as (num_ref_frames < 5 @ 1080p) or (num_ref_frames < 10 @ 720p) and B-pyramids are OFF.

[valnar]

Originally Posted by UsedUser

I concur. B-frames can be pushed all the way up to 16 without breaking DXVA, as long as (num_ref_frames < 5 @ 1080p) or (num_ref_frames < 10 @ 720p) and B-pyramids are OFF.

Supposedly if you keep bframes below 3 always, you add PS3 and other H.264 device compatibility too. That's what I'm looking for - the ubiquitous configuration that'll work with everything that can handle hi-def up to level 4.1 and hopefully futureproof the files.

-Robert

[UsedUser]

I'm fairly confident now that the feature breaking DXVA really is compliance with Profile High@L4.1, which is to say the num_ref_frames value.

What I didn't initially understand, and still don't fully, is the formula that is used to determine the num_ref_frames value, which is distinct from the "ref" value set when encoding. The "ref" value, the "bframes" value, and the "b-pyramid" value all factor into the final "num_ref_frames" value. The "ref" value seems to contribute its value directly, while adding b-frames only contributes 1 (no matter the "b-frames" value), and b-pyramid seems to contribute 2.

For example:

ref=3 adds 3 to num_ref_frames
b-frames=3 adds 1 to num_ref_frames
b-pyramid adds 2 to num_ref_frames
num_ref_frames = 3 + 1 + 2 = 6 (2 more than are allowed @ 1080p)

It is the num_ref_frames value that must comply with High@L4.1, where (num_ref_frames < 5 @ 1080p) and (num_ref_frames < 10 @ 720p).

B-pyramids, adaptive b-frames, mixed ref frames, weighted prediction --- all can be enabled, as long as the resulting encode stays under the limit of num_ref_frames.

The easiest way to reliably meet this requirement while encoding is to set the "ref" value one under the max allowed num_ref_frames and to turn off B-pyramids, because they may unpredictably contribute to num_ref_frames. The B-frames value can be anything you want, but with adaptive b-frames, I believe "3" is still the recommended value, unless you're encoding animation, where "8" or more could be justified.


The following encoding settings allow DXVA to work @ 1080p:

num_ref_frames=4 ref=1 b-frames=3 b-pyramid=on
--level 4.1 --ref 1 --bframes 3 --b-pyramid --b-adapt --b-rdo --bime --weightb --direct auto --subme 6 --trellis 1 --analyse p8x8,b8x8,i4x4,i8x8 --8x8dct --vbv-maxrate 25000 --me umh --merange 12

num_ref_frames=4 ref=3 b-frames=3 b-pyramid=off
--level 4.1 --ref 3 --mixed-refs --bframes 3 --no-b-pyramid --b-adapt --b-rdo --bime --weightb --direct auto --subme 6 --trellis 1 --analyse p8x8,b8x8,i4x4,i8x8 --8x8dct --vbv-maxrate 25000 --me umh --merange 12

num_ref_frames=4 ref=3 b-frames=3 b-pyramid=off b-adapt=off
--level 4.1 --ref 3 --mixed-refs --bframes 3 --no-b-pyramid --no-b-adapt --b-rdo --bime --weightb --direct auto --subme 6 --trellis 1 --analyse p8x8,b8x8,i4x4,i8x8 --8x8dct --vbv-maxrate 25000 --me umh --merange 12

num_ref_frames=4 ref=3 b-frames=16 b-pyramid=off
--level 4.1 --ref 3 --mixed-refs --bframes 16 --no-b-pyramid --b-rdo --bime --weightb --direct auto --subme 6 --trellis 1 --analyse p8x8,b8x8,i4x4,i8x8 --8x8dct --vbv-maxrate 25000 --me umh --merange 12

num_ref_frames=4 ref=4 b-frames=0 b-pyramid=off b-adapt=off
--level 4.1 --ref 4 --mixed-refs --bframes 0 --no-b-pyramid --no-b-adapt --b-rdo --bime --weightb --direct auto --subme 6 --trellis 1 --analyse p8x8,b8x8,i4x4,i8x8 --8x8dct --vbv-maxrate 25000 --me umh --merange 12


The following encoding settings BREAK DXVA @ 1080p:

num_ref_frames=5 ref=2 b-frames=2 b-pyramid=on
--level 4.1 --ref 2 --mixed-refs --bframes 2 --b-pyramid --b-adapt --b-rdo --bime --weightb --direct auto --subme 6 --trellis 1 --analyse p8x8,b8x8,i4x4,i8x8 --8x8dct --vbv-maxrate 25000 --me umh --merange 12

num_ref_frames=6 ref=3 b-frames=3 b-pyramid=on
--level 4.1 --ref 3 --mixed-refs --bframes 3 --b-pyramid --b-adapt --b-rdo --bime --weightb --direct auto --subme 6 --trellis 1 --analyse p8x8,b8x8,i4x4,i8x8 --8x8dct --vbv-maxrate 25000 --me umh --merange 12

num_ref_frames=5 ref=4 b-frames=3 b-pyramid=off
--level 4.1 --ref 4 --mixed-refs --bframes 3 --no-b-pyramid --b-adapt --b-rdo --bime --weightb --direct auto --subme 6 --trellis 1 --analyse p8x8,b8x8,i4x4,i8x8 --8x8dct --vbv-maxrate 25000 --me umh --merange 12


The following encoding settings allow DXVA to work @ 720p:

num_ref_frames=9 ref=9 b-frames=0 b-pyramid=off b-adapt=off
--level 4.1 --ref 9 --mixed-refs --bframes 0 --no-b-pyramid --no-b-adapt --direct auto --subme 6 --trellis 1 --analyse p8x8,b8x8,i4x4,i8x8 --8x8dct --vbv-maxrate 25000 --me umh --merange 12

num_ref_frames=9 ref=8 b-frames=3 b-pyramid=off
--level 4.1 --ref 8 --mixed-refs --bframes 3 --no-b-pyramid --b-adapt --b-rdo --bime --weightb --direct auto --subme 6 --trellis 1 --analyse p8x8,b8x8,i4x4,i8x8 --8x8dct --vbv-maxrate 25000 --me umh --merange 1

num_ref_frames=8 ref=7 b-frames=3 b-pyramid=off
--level 4.1 --ref 7 --mixed-refs --bframes 3 --no-b-pyramid --b-adapt --b-rdo --bime --weightb --direct auto --subme 6 --trellis 1 --analyse p8x8,b8x8,i4x4,i8x8 --8x8dct --vbv-maxrate 25000 --me umh --merange 12

num_ref_frames=7 ref=6 b-frames=3 b-pyramid=off
--level 4.1 --ref 6 --mixed-refs --bframes 3 --no-b-pyramid --b-adapt --b-rdo --bime --weightb --direct auto --subme 6 --trellis 1 --analyse p8x8,b8x8,i4x4,i8x8 --8x8dct --vbv-maxrate 25000 --me umh --merange 12

num_ref_frames=9 ref=6 b-frames=3 b-pyramid=on
--level 4.1 --ref 6 --mixed-refs --bframes 3 --b-pyramid --b-adapt --b-rdo --bime --weightb --direct auto --subme 6 --trellis 1 --analyse p8x8,b8x8,i4x4,i8x8 --8x8dct --vbv-maxrate 25000 --me umh --merange 12


The following encoding settings BREAK DXVA @ 720p:

num_ref_frames=10 ref=9 b-frames=3 b-pyramid=off
--level 4.1 --ref 9 --mixed-refs --bframes 3 --b-pyramid --b-adapt --b-rdo --bime --weightb --direct auto --subme 6 --trellis 1 --analyse p8x8,b8x8,i4x4,i8x8 --8x8dct --vbv-maxrate 25000 --me umh --merange 12

[cscxk]

i think you are right."num_ref_frames=4 ref=3 b-frames=16 b-pyramid=off" is power than "num_ref_frames=4 ref=3 b-frames=3 b-pyramid=off b-adapt=off".so "b-adapt=off" is not need.but "num_ref_frames=4 ref=4 b-frames=0 b-pyramid=off" should allow DxVA?

and allow dxva only be limited by num_ref_frames,not other.did you think so?

[UsedUser]

Originally Posted by cscxk

"num_ref_frames=4 ref=4 b-frames=0 b-pyramid=off" should allow DxVA?

Yes --- ref=4 b-frames=0 worked with DXVA for me.

Originally Posted by cscxk

and allow dxva only be limited by num_ref_frames,not other.did you think so?

Yeah, the bottom line, I believe, is that allowing DXVA is only limited by the num_ref_frames value.

[UsedUser]

One other reasonably important note --- I tested all of the encodes using the Cyberlink H.264 decoder on both an Nvidia 8800GT and an ATI HD2600PRO.

I got a gray screen during playback with every clip using the Nvidia card --- it NEVER successfully displayed a picture.

I got perfect DXVA playback with every compliant clip on the ATI card.

[cscxk]

Originally Posted by UsedUser

One other reasonably important note --- I tested all of the encodes using the Cyberlink H.264 decoder on both an Nvidia 8800GT and an ATI HD2600PRO.

I got a gray screen during playback with every clip using the Nvidia card --- it NEVER successfully displayed a picture.

I got perfect DXVA playback with every compliant clip on the ATI card.

I had enable DXVA on G98 and 2600pro

[CruNcher]

UsedUser im useing the 8800 GT G92 VP2 too with Cyberlinks Decoder before that a 7600 GS G71 VP1 and all complaint streams are working with the latest PowerDVD Decoder Filter and give me actually almost 0% Cpu Utilization on the G92 now without Sound im really amazed by that. Had no problems yet with it staying in the Parameters (even tried bitstreams with over 200 Mbits ok they where not playing fluid but they played) ( Forceware 192.21 WHQL and 192.28 Beta), but thats true for now only with tested raw bitstreams useing the .mpv extension with PowerDVD.
Those Bitstreams also had no playback problems in .mp4 but in .mkv they gave me often problems with the G71 for example if you have a .mkv where the fps lowers suddenly after some seconds of playback dramaticly down, my experience is that it lowers from normal playback framerate constantly down this constant 20 fps problem i didn't experienced yet (but i watch no anime or vfr stuff that might force timecode problems here with Cyberlinks Decoder that's mostly only used with anime and mkv). Demuxing such lowering fps framerate streams out of the .MKV containter and playback those Bitstreams with PowerDVD (7600 GS G71) worked for me so it must be something else here that has todo with this phenomenon except Hardware Limitations of the Stream itself. I asked Haali about a problem that might be in the way PowerDVD filter parses the container, but haali has not looked into this yet as he doesn't use PowerDVD and it's filters personaly. For Nvidia tough Cyberlinks Decoder seems to be the most compatible for WinXP at least in the meaning of Hardware Playback, so i would advise as hard as it sounds in terms of compatibility screw Matroska .MKV (there is so many Matroska stuff out that plays wrong with Hardware Playback, sometimes its also just stuttering for some frames) and use the standard advised .MP4, for hopefully no Interoperability problems @ the moment with Cyberlinks or any other Decoder on the market and correct Hardware DXVA Playback

At the moment also PowerDVD is the only choice for WinXP 8800GT VP2 PureVideo users as it is allways allowing to use Hardware accellerated Playback even with Subtitles and both HD-DVD and Blu-Ray and it makes no difference wich Codec Mpeg-2, VC-1 or H.264 everything works as it should be and is full Hardware accellerated to the capabilities of the Chip (VC-1 has higher CPU utilization as we know on Nvidia).
I tested every Software HD-DVD and Blu-Ray Player on the Market today except Corel WinDVD 9 Plus as it isn't available yet and all of them Show problems with Hardware Playback and the 8800 GT even Arcsoft Digital Theatre 2 wich has problems with Hardware accellerated VC-1 Playback doesn't showing the Video @ all only Software Mode Playback works here (and even then HD-DVD menus are flickering on screen). Nero Showtime does in General not work when the Medium uses Subtitles Hardware Accelleration is disabled on XP (not sure if it's a bug or Limitation but i try to make it known to Ahead and Nero for a long time and now im almost sure it's a Limitation of their Video Rendering usage on XP (VMR9 usage)). The only one that works with Mpeg-2 and H.264 in those regards are PowerDVD and Arcsoft Digital Theatre 2 with Arcsoft haveing the fastest BDJ Engine i saw nowdays but more Problems with HD-DVD and VC-1 then PowerDVD (it's BDJ Engine is not as fast as Arcsoft but works and also Menu Interoperability with HD-DVD and VC-1 is better, no flickering Menus and full Accelleration to the capabilities of the G92) But Arcsofts Player Software nevertheless has great Potential if they once squashed this bugs and is allready better then Showtime in Hardware Accelleration regards being used allways even with Subtitles, see http://forum.doom9.org/showthread.php?t=133278: for a inside look into there Software Player

[kumi]

@Cruncher:

A friendly tip: if you don't add any punctuation at all to your sentences, most people are not going to read past the first 2 or 3 dozen words.

It's unreadable.

[foxyshadis]

So basically x264 needs a new parameter to specify the absolute maximum refs any frame can use, to keep b-frame efficiency? Sounds like a good patch if Dark Shikari is interested =D