x264 Known Hardware accelleration problems and solutions

[tetsuo55]

Hello tetsuo and thanks for keeping this thread updated, I think it's one of the most important if we want to maximise the number of compliant streams "out there".

I've just one question regarding your settings : I notice you specify --merange 12, but I don't see how that is a "hard" constraint. I have always used the default of 16. Besides, since it is itself an option of an analysis parameter, I dont see how it should affect profile / level compliance. Am I wrong ?

cheers
audyovydeo

You are not wrong, these presets are old and at the time where made to be balanced in speed.

The only "Restrictions" are max bitrate and max ref frames

When the new MeGui profiles are finalised i will update the presets

[lexor]

Well I can tell you that on my psp fat, --b-pyramid produces quite noticeable artifacts on either 480x272 or 720x480. And the later resolution is not "beyond the capability of the PSP device" as you claim, PSP can decode streams of upto 20 Mbps!, I'm sure it can handle a few more pixels at a more common/mild bitrate. Sony just didn't bother to implement the feature (or implement it correctly) in the decoder.

[tetsuo55]

Well I can tell you that on my psp fat, --b-pyramid produces quite noticeable artifacts on either 480x272 or 720x480. And the later resolution is not "beyond the capability of the PSP device" as you claim, PSP can decode streams of upto 20 Mbps!, I'm sure it can handle a few more pixels at a more common/mild bitrate. Sony just didn't bother to implement the feature (or implement it correctly) in the decoder.

I found the problem, the PSP supports Main@L3.0 and not HIGH@L3.1

Besides that the PSP has a lot of other limitations. Full SD resolution supports seems to be more like a hack than something the PSP was designed to decode. Enabling b-pyramids seems to be hitting some hardware limit with the free settings most people here are using.(probably the added cpu overhead that comes with b-pyramids)

Accurate settings for problem free b-pyarmids playback from memory stick would have a resolution of 320*240 at most, and a very low bitrate to go with that.

It would be interesting to see if the UVD videos used b-pyramids

[lexor]

Umm... and? I don't think 3.0 precludes b-pyramid. I'm pretty sure it's just the crappy decoder implementation. MeGUI usually whines like a baby if you pick something that breaks profile, and I can pick b-pyramid just fine.

[tetsuo55]

Umm... and? I don't think 3.0 precludes b-pyramid. I'm pretty sure it's just the crappy decoder implementation. MeGUI usually whines like a baby if you pick something that breaks profile, and I can pick b-pyramid just fine.

main3.0 is severely limited when compared to high3.1:

main3.0:MAX DPB=2073600,MAX BITRATE=10MBIT/s(max ref frame at dvd resolution is 5 or 6)
high3.1:MAX DPB=4608000,MAX BITRATE=17.5Mbit/s(max ref frame at dvd resolution is 11 or 13)

Also with main 8×8dct cannot be used

The PSP's official spec lower those settings even more

The official maximum resolution is 368x208, theoretically b-pyramids should only work on this resolution or lower


(if b-pyarmids are indeed broken in the decoder a bug should be reported to sony to have it fixed)

[lexor]

main3.0 is severely limited when compared to high3.1:

main3.0:MAX DPB=2073600,MAX BITRATE=10MBIT/s(max ref frame at dvd resolution is 5 or 6)
high3.1:MAX DPB=4608000,MAX BITRATE=17.5Mbit/s(max ref frame at dvd resolution is 11 or 13)

Also with main 8×8dct cannot be used

The PSP's official spec lower those settings even more

The official maximum resolution is 368x208, theoretically b-pyramids should only work on this resolution or lower

I'll test that last statement a bit later. As for bitrate, profile may prohibit it, but psp certainly not bound by that bitrate and it needs no vbv buffer restrictions.

[tetsuo55]

I guess we should thank sony for completely not understanding how to implement levels then hehehe.

Awaiting your result!

[Sagekilla]

You know the reason why the PSP doesn't obey the bitrate limit (that it can go up to 25 mbps) is because it has a hardware based bitstream decoder, right?

[lexor]

You know the reason why the PSP doesn't obey the bitrate limit (that it can go up to 25 mbps) is because it has a hardware based bitstream decoder, right?

That's not the reason why it doesn't obey, that's the reason it is capable of supporting higher profiles. The reason it doesn't obey any set profile is because half the programming positions are filled pigeons.

[shon3i]

main3.0 is severely limited when compared to high3.1:

main3.0:MAX DPB=2073600,MAX BITRATE=10MBIT/s(max ref frame at dvd resolution is 5 or 6)
high3.1:MAX DPB=4608000,MAX BITRATE=17.5Mbit/s(max ref frame at dvd resolution is 11 or 13)

Also with main 8×8dct cannot be used

The PSP's official spec lower those settings even more

The official maximum resolution is 368x208, theoretically b-pyramids should only work on this resolution or lower


(if b-pyarmids are indeed broken in the decoder a bug should be reported to sony to have it fixed)
precisely :)

Level 3.0 Main Profile:

--vbv-bufsize 10000 --vbv-maxrate 10000

Level 3.0 High Profile:

--vbv-bufsize 10000 --vbv-maxrate 12500

Level 3.1 Main Profile:

--vbv-bufsize 14000 --vbv-maxrate 14000

Level 3.1 High Profile:

--vbv-bufsize 14000 --vbv-maxrate 17500

[lexor]

The official maximum resolution is 368x208, theoretically b-pyramids should only work on this resolution or lower


Nope, no b-pyramid support even at that resolution. Admit it, this has nothing to do with standards, and everything to do with incompetence.

[tetsuo55]

Nope, no b-pyramid support even at that resolution. Admit it, this has nothing to do with standards, and everything to do with incompetence.

If i understand correctly you just tested a file right?

In that case PSP decoding is indeed broken :(

Could you upload that sample?

[lexor]

Why? Any sample will work: use Avisynth to resize to any resolution that PSP supports (there aren't many of such resolutions as PSP has no freehand resize), use MeGUI with Shark's PSP profile (just enable b-pyramid). This isn't a specific video issue, they all exhibit horrid artifacts.

[tetsuo55]

Why? Any sample will work: use Avisynth to resize to any resolution that PSP supports (there aren't many of such resolutions as PSP has no freehand resize), use MeGUI with Shark's PSP profile (just enable b-pyramid). This isn't a specific video issue, they all exhibit horrid artifacts.

My cpu is very slow :'(

[CruNcher]

Hey guys we don't have to argue it's not working on the PSP and so it is, so be kind to each other tough someone should try it on the Slim to see if it's fixed if it indeed is then it seems to be more then a Firmware problem (but i guess it is a Firmware problem by Sony tough it could also have something todo with the Custom Firmware)
So wee need people with different PSPs (Firmware) to identify the problem for sure :), but if it is indeed Sonys problem i guess it wont be easy to make them understand that :P, most probably they dont even care, someone could write them a nice e-mail telling them the problem or just wait for the next Firmware and see if its fixed then.
Imho i guess it isnt even a real Firmware problem but Sony just didn't implement it i even dont remember to have read something about b-pyramid (hirachical b-frames) support in the official UMD Spec @ all.
So to be on the safe interoperable side dont use --b-pyramid when you trying to hit the PSP with your content, or tell Sony about it and hope they gonna care and ever support it, wich i highly doubt :(

[Sharktooth]

@tetsuo, PSP decoding isnt broken... it has its own specs.

[tetsuo55]

@tetsuo, PSP decoding isnt broken... it has its own specs.

Yeah, they simply decided to break the standard and not support it. Not entirely strange with the psp's relatively slow cpu and/or memory limitations. Although it would have helped UMD with its limited storage space.

Updated first post

[shon3i]

@tetsuo55, update aslo this on first post http://forum.doom9.org/showthread.php?p=1166537#post1166537.

[tetsuo55]

My research has revealed some new interesting things about hardware accelleration.

-I read a lot of DXVA and H264 docs
-I ran a lot of edge-case samples on my ATI HD2600
-I used google a lot

Commercially there is only L4.1 media available(the only exception being L3.0 UMD's)

All standalone players will attempt to play back any stream that meets the L4.1 limits

The only place where levels lower than 4.1 is being used is in digital broadcasting and webcasts.


Pending more verification this means we can completely ignore all levels below 4.1. Except for handheld systems.

[The_Rebel]

Is there any HW decoders that can handle 5.1 yet?

[Sulik]

Someone mentioned that the NVidia Cuda 2.0 beta drivers (177.xx) do enable support for level 5.1 encodes, but I haven't tried it myself on my 8800GT (sticking to official drivers).

[The_Rebel]

Someone mentioned that the NVidia Cuda 2.0 beta drivers (177.xx) do enable support for level 5.1 encodes, but I haven't tried it myself on my 8800GT (sticking to official drivers).

Coolness, now i have a real excuse to get a new GPU.

[Ranguvar]

EDIT: Never mind... bye bye, Windows.

[Dark Shikari]

My research has revealed some new interesting things about hardware accelleration.

-I read a lot of DXVA and H264 docs
-I ran a lot of edge-case samples on my ATI HD2600
-I used google a lot

Commercially there is only L4.1 media available(the only exception being L3.0 UMD's)

All standalone players will attempt to play back any stream that meets the L4.1 limits

The only place where levels lower than 4.1 is being used is in digital broadcasting and webcasts.


Pending more verification this means we can completely ignore all levels below 4.1. Except for handheld systems.I know of STBs that only support 4.0.

[valnar]

I know of STBs that only support 4.0.

Which ones? I'll make a point of staying away from them.

[The_Rebel]

EDIT: Never mind... bye bye, Windows.

ROFL, what happened?

I was really looking forward to your results.

[tetsuo55]

Major rewrite of the first post.

Old post archived here:


I have completely re-written this post to update things to the current situation

All the hardware limitations are clear at this this point:

To encode a hardware compatible file you need to keep the following limits in mind:

-The height and width of the file need to be mod16
-Every device supports different levels. each level has different limits

I will be focusing on level 3.1(Standard definition) and level 4.1(high definition)

Level 3.1 can be used for all resolutions from 16*16 to 720*576, however at these settings the lower resolution files will not work on portable players expecting lower levels.

Level 3.1 limits are:
MAX bitrate: 17.5 Mbit/s
MAX ref frame limit: 4608000 / (Height X Width) = max number of ref frames, this value can never be higher than "16"

Level 4.1 can be used for any resolution higher than 3.1 with an upper limit of 1920*1088
MAX bitrate: 40.0 Mbit/s (standard defines 50 as the limit, if you're hardware players supports 50 please report it in this thread)
MAX ref frame limit: 8355840 / (Height X Width) = max number of ref frames, this value can never be higher than "16"


If you have a file that was already encoded, but it doesn't work there are 4 options:
1.Try the file in MPC-HC if it supports you videocard
2.Use h264info to change the header to level3.1 or level4.1 and try your regular player
3.Use a software decoder like coreavc
4.Transcode the file to be Hardware level compliant.

Non-working files usually means that it was encoded with an old version of x264, has too many ref-frames for its resolution/level or was muxed with an old version of mkvmerge.

TO DO:
-Prove that players do or do not all support 50Mbit/s for level 4.1.
-the real-life max bitrate and max CPB have not yet been determined, if you want to help please encode some files and try to find the upper limit.
-Confirmed hardware limits for levels below 3
-Update this post with level information for levels below 3

DONE:
-b-pyramids works on all hardware devices including portable ones since x244 SVN721
NOTE: the PSP does not support b-pyramids on a hardware level, do not use it if you want your file to work on a PSP. This was a delibarate choice by Sony.


Below are the pre-sets from the old post for encodingThese are very old i will update them when the new megui profiles are out:

Settings for HD
--level 4.1 --ref 3 --mixed-refs --bframes 3 --b-rdo --bime --weightb --direct auto --subme 6 --trellis 1 --analyse p8x8,b8x8,i4x4,i8x8 --8x8dct --vbv-maxrate 25000 --me umh --merange 12 --b-pyramid

Settings for SD
--level 3.1 --ref 3 --mixed-refs --bframes 3 --b-rdo --bime --weightb --direct auto --subme 6 --trellis 1 --analyse p8x8,b8x8,i4x4,i8x8 --8x8dct --vbv-maxrate 25000 --me umh --merange 12 --b-pyramid

Extreme settings (this will be extremely slow):
--level 4.1 --ref * --mixed-refs --bframes 3 --b-rdo --bime --weightb --direct auto --subme 7 --trellis 1 --8x8dct --TESA --merange 24 --no-fast-pskip --partitions al --b-pyramid

Here are some settings for max DBP size

L4.1
1920x1088: 4 (--ref 4)
1920x864: 5 (--ref 5)
1920x720: 6 (--ref 6)

L4.1
1280x720: 9 (--ref 9)
1280x648: 10 (--ref 10)
1280x588: 11 (--ref 11)
1280x540: 12 (--ref 12)
1280x498: 13 (--ref 13)
1280x462: 14 (--ref 14)
1280x432: 15 (--ref 15)
1280x405: 16 (--ref 16)

L3.1
720x(any): 8 (--ref 8)
720x576: 11 (--ref 11)
720x480: 13 (--ref 13)

Level 3.0 Main Profile:

--vbv-bufsize 10000 --vbv-maxrate 10000

Level 3.0 High Profile:

--vbv-bufsize 10000 --vbv-maxrate 12500

Level 3.1 Main Profile:

--vbv-bufsize 14000 --vbv-maxrate 14000

Level 3.1 High Profile:

--vbv-bufsize 14000 --vbv-maxrate 17500

Original referenced threads:
http://www.avsforum.com/avs-vb/showthread.php?t=972503
http://www.avsforum.com/avs-vb/showthread.php?p=12402455
http://forum.doom9.org/showthread.php?p=1088894

[Audionut]

STEP 3: Make sure use these commands and never cross the limits, shown here are the highest settings(not marked) or mandatory settings(marked with *):

--VBVMaxBitrate 40000
--KeyframeInterval 24*
level-idc = 4.1*
Profile = High*




When using dgavcdec on "Juno" region B for the missus, I got these results.

Bitrate: 0.031
Bitrate (Avg): 31.760
Bitrate (Max): 44.249

[tetsuo55]

When using dgavcdec on "Juno" region B for the missus, I got these results.

Bitrate: 0.031
Bitrate (Avg): 31.760
Bitrate (Max): 44.249

Can you post the full encoding settings you used?

[Audionut]

Can you post the full encoding settings you used?

That was ripped from a store bought blu-ray.

My point being that if studios are selling blu-ray's with bit rates above 40 MB/s, I think it's safe to assume that the we can use those bit rates for our level 4.1 encodings.

[shon3i]

Well maximum bitrate for high and main profiles @ level 4.1 profile is 50000, and buffer size should be 62500 acording Annex A specification of H264 document. But someone on this forum (i think some of x264 devs) say that using high possible settings not good because is too much.

[Sharktooth]

That was ripped from a store bought blu-ray.

My point being that if studios are selling blu-ray's with bit rates above 40 MB/s, I think it's safe to assume that the we can use those bit rates for our level 4.1 encodings.
those are out of specs BDs. no guarantee they will play in spec compliant devices.
if studios are idiots we dont need to be idiots like them.

[tetsuo55]

That was ripped from a store bought blu-ray.

My point being that if studios are selling blu-ray's with bit rates above 40 MB/s, I think it's safe to assume that the we can use those bit rates for our level 4.1 encodings.



The settings i posted are SAFE, guaranteed to work,
Depending on who you talk to thhe spec allows for WAY higher limits than has been posted above. But there are litte changes left and right. Sony uses its own limits, bluray does, hd-dvd and even apple.

Using the settings in the first post makes sure it works on all devices and from all media.

the actual bluray-decoder-on-a-chip is supposed to be able to handle 54mbps, the limit is based on the media playback speeds and FastForward things

[Sharktooth]

the OFFICIAL BLU-RAY SPECS say 40mbps for blu-ray DISCS.
that means if someone produce a full spec compliant player it may not able to read more than 40mbps FROM THE DISC, it will stutter or wont play the streams...
specs exist for a reason.

[tetsuo55]

the OFFICIAL BLU-RAY SPECS say 40mbps for blu-ray DISCS.
that means if someone produce a full spec compliant player it may not able to read more than 40mbps FROM THE DISC, it will stutter or wont play the streams...
specs exist for a reason.

Exactly this is why the settings are limited to 40mbps.

i have heard a lot of reports of Bluray disks going over 40mbps, it seems that "Quality Control" is not a priority over at the blu-ray camp

[valnar]

Does anyone have any data on how high blue-ray movies go? I'm trying to imagine the scene that would require 40mb. A pan of an explosion then fading to black?

[tetsuo55]

According to this guy over at avsforums:
http://www.avsforum.com/avs-vb/showthread.php?p=14229004#post14229004

Forceware 177.41 supports H.264 720p L5.1.
Now you can play most of H.264 720p files with DxVA.
So you don't need to IDC Changer !

Can anyone with a Nvidia card confirm?

(To confirm you will need a file with the following: 1920*1080, 6 ref frames, bitrate over 60Mbps)

[pandy]

Which ones? I'll make a point of staying away from them.

AFAIK STM decoders have some problem due architecture flaws (bottlenecks on external bus and arbiters - h.264 decoder itself is OK)

[tetsuo55]

Updated the settings.

-Minimal/maximal values are now defined better
-Added "min-keyint 4" because lower values break HD-DVD players and possible others as well

[CruNcher]

tetsuo55 HD-DVD is officialy dead it shouldn't be taken into account for anything anymore (not if it lowers compression efficiency)

[DarkZell666]

tetsuo55 HD-DVD is officialy dead it shouldn't be taken into account for anything anymore (not if it lowers compression efficiency)

So you deny the existence of all those people who actually bought a HD-DVD player ? :p I know if I had bought one, I wouldn't just toss it right away, those costed quite a bit ... x_x

But I do get your point, it doesn't need to be marked as mandatory, a friendly warning is enough imho :)

[CruNcher]

@DarkZell666
Sorry but im not sorry for those earlyadopters like you seem to be, everyone knows how the Game is played especialy those who call themselves earlyadopters so they knew the risks (and that their player might endup in the recycle chain sooner than hoped).
Tough im sorry for those who belived all the Marketing and didn't wait and knew that this was a battle between 2 formats, but the Media informed them quite good about the ongoing format War and that only 1 would survive it, so even Avg Joe was aware of the risks.

[Sharktooth]

Blu-ray can and will die too. if it won the format war it doesnt mean it will survive (infact it's actually dying). so?
there are devices around, so it's worth taking them into account.

[tetsuo55]

Although i partially agree.

The settings here should be universal, if it reduces compression by a very small percentage then so be it.

However, further discussion in that thread has revealed that:

-x264 already has this rule applied internally(it never breaks compatibility with default settings)
-the problem appears to be caused by the muxer

Setting removed

[marc99]

According to this guy over at avsforums:
http://www.avsforum.com/avs-vb/showthread.php?p=14229004#post14229004


Originally Posted by P.J
Forceware 177.41 supports H.264 720p L5.1.
Now you can play most of H.264 720p files with DxVA.
So you don't need to IDC Changer !


Can anyone with a Nvidia card confirm?

(To confirm you will need a file with the following: 1920*1080, 6 ref frames, bitrate over 60Mbps)

Driver change nothing. New version of Cyberlink decoder doesn't have 20fps bug with mislabeled files. New version of Cyberlink decoder doesn't need to change level 5.1 -> 4.1 with IDC Changer or h264info.

[Audionut]

the OFFICIAL BLU-RAY SPECS say 40mbps for blu-ray DISCS.



So I see. http://www.blu-ray.com/faq/

And yet, the 10 movies I've looked at so far have all had video bitrates in excess of 40MB/s.

I can't recall which one, but it had a bitrate of 50MB/s.

[kemuri-_9]

that faq does have this stipulation in it:
According to the Blu-ray Disc specification, 1x speed is defined as 36Mbps. However, as BD-ROM movies will require a 54Mbps data transfer rate the minimum speed we're expecting to see is 2x (72Mbps)

so as most players already need 2x for their own standardized bitrates of combined audio&video data, going all the up to 72Mbps overall seems viable as well

[Audionut]

Section 2.4.

Video bit rate (max) 40.0Mbps


But with, Data transfer rate (video/audio) 54.0Mbps (1.5x), that's allowing 14Mbps for audio and subs etc. 5.1 LPCM is only about 7Mbps.

[Guest]

AFAIK STM decoders have some problem due architecture flaws (bottlenecks on external bus and arbiters - h.264 decoder itself is OK) That's FUD. Please provide a test stream and state the chip you are using. I can quickly test it here to prove you wrong. I'm a digital video engineer at STM.

You need to be able to support scurrilous disparagement.

[UsedUser]

According to this guy over at avsforums:
http://www.avsforum.com/avs-vb/showthread.php?p=14229004#post14229004



Can anyone with a Nvidia card confirm?

(To confirm you will need a file with the following: 1920*1080, 6 ref frames, bitrate over 60Mbps)
I did all my previous testing with the Cyberlink PDVD7 decoder. I will test the new Nvidia drivers with both PDVD7 and PDVD8 decoders to isolate the issue/resolution.

Previously, ref=5 @ 1920x1080 @ any bitrate would break DXVA. Why is a ref=6, 60Mbps stream required to test?

Driver change nothing. New version of Cyberlink decoder doesn't have 20fps bug with mislabeled files. New version of Cyberlink decoder doesn't need to change level 5.1 -> 4.1 with IDC Changer or h264info.
But does the new Nvidia driver work with the old PDVD7 decoder? It could be both the drivers and the decoder had fixes applied.

Question: The new Cyberlink decoder plays actual L5.1 streams, or just L4.1 streams with the IDC flag set to 5.1?