Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 AVC / H.264

Reply
 
Thread Tools Search this Thread Display Modes
Old 20th December 2007, 16:21   #1  |  Link
tetsuo55
MPC-HC Project Manager
 
Join Date: Mar 2007
Posts: 2,317
x264 Known Hardware accelleration problems and solutions

Below you will find the settings to create a file that will work on any hardware decoder.
Current problems will be explained and finally i will explain how you can try to convince an older file to work.

What settings to use to create a universally(L4.1) working fileNOTE1:
Follow these 3 steps

STEP 1: Determine the REF frame in DPB limit:
Code:
8388608 / (Height X Width) = nREF
The value can never be higher than 16 so round it down if its higherNOTE2

STEP 2: Decide if you want to use B-pyramidsNOTE3:
Code:
If B-Pyramids = YES then nREF -1
If B-Pyramids = NO then add "--no-b-pyramid"
STEP 3: Make sure use these commands and never cross the limits, shown here are the highest settings(not marked) or mandatory settings(marked with *):
Code:
--VBVMaxBitrate 40000 (highest possible value)
--KeyframeInterval 24*
level-idc = 4.1*
Profile = High*

NOTE1
If a file created with these settings does not work on your hardware it is either not capable of L4.1 decoding or there is something wrong with the universal settings.
Handheld devices do not support L4.1 i have created an alternative setting below.
NOTE2
Currently there is a bug with DXVA(videocards) on several of the decoders, a REF value higher than 11 will not work on all systems or with all players. (if you want to help fix the bug or want more information see the thread here http://forum.doom9.org/showthread.php?p=1170371)
NOTE3
B-pyramids are "broken" in x264, this is why you need to reduce ref frames by 1 to guarantee that it works, and even then it might not be stable at all. if you want to know more and/or help fix it see this thread: http://forum.doom9.org/showthread.php?t=140223


Not all hardware devices support L4.1, these devices need L3.0 settings(A lot of portable devices use even lower settings, please check the manual and use the correct MEgui profile):
Follow these 2 steps

STEP 1: Determine the REF frame in DPB limit:
Code:
2073600 / (Height X Width) = nREF
The value can never be higher than 16 so round it down if its higherNOTE2

STEP 2: Make sure use these commands and never cross the limits, shown here are the highest/lowest settings(not marked) or mandatory settings(marked with *):
Code:
--vbv-bufsize 10000 (highest possible value)
--vbv-maxrate 12500 (highest possible value)
--KeyframeInterval 24*
level-idc = 3.0*
Profile = High*
--no-b-pyramid*
B-pyramids does not work on most portable devices, so its best to disable it completely.

---

If the file is already encoded but you cannot get it to work you can try the following:

If you have a file that was already encoded, but it doesn't work there are 4 options:
1.Try the file in MPC-HC if it supports you videocard
2.Use h264info to change the header to level3.1 or level4.1 and try your regular player
3.Use a software decoder like coreavc
4.Transcode the file to be Hardware level compliant.

Non-working files usually means that it was encoded with an old version of x264, has too many ref-frames for its resolution/level or was muxed with an old version of mkvmerge.

(The previous post i had here has been archived here: http://forum.doom9.org/showthread.ph...69#post1170369)

EDIT:1 (Added min-keyint 4 to the required settings thanks to the findings in this thread: http://forum.doom9.org/showthread.php?t=140135 )
EDIT:2 (min-keyint 4 removed, x264 already respects these limits by default)
EDIT:3 (Corrected fullhd formula(the incorrectness of the old formula did not effect ref frame caluclation))

Last edited by tetsuo55; 27th September 2008 at 19:09. Reason: Completely re-written to update to current status
tetsuo55 is offline   Reply With Quote
Old 20th December 2007, 18:09   #2  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 6,087
- slower encoding
+ very small improvements in quality

Quote:
Maybe people creating the frontends for encoding could do the same so files become more hardware-compliant?
I will make necessary changes in my GUI. Thanks for info

Last edited by Atak_Snajpera; 20th December 2007 at 18:28.
Atak_Snajpera is offline   Reply With Quote
Old 20th December 2007, 21:40   #3  |  Link
Dark Shikari
x264 developer
 
Dark Shikari's Avatar
 
Join Date: Sep 2005
Posts: 8,690
Note the improvement in quality is much greater on anime/cartoon footage, where you can get a 1% increase in quality per bitrate per reference frame even at high numbers.
Dark Shikari is offline   Reply With Quote
Old 20th December 2007, 21:55   #4  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 6,087
What's the point of encoding cartoons with 16ref and not be able to watch them on PS3/X360 or PC with hardware acceleration? In this case I choose compatibility over small increase in quality.
Atak_Snajpera is offline   Reply With Quote
Old 20th December 2007, 21:58   #5  |  Link
akupenguin
x264 developer
 
akupenguin's Avatar
 
Join Date: Sep 2004
Posts: 2,393
Is it number of reference frames, or is it DPB size (i.e. a Level restriction like every other limited HW decoder under the sun)?
akupenguin is offline   Reply With Quote
Old 20th December 2007, 22:01   #6  |  Link
arfster
Registered User
 
Join Date: Jun 2006
Posts: 169
Quote:
Originally Posted by akupenguin View Post
Is it number of reference frames, or is it DPB size (i.e. a Level restriction like every other limited HW decoder under the sun)?
Copy/pasted this from somewhere that did tests to nail down the cyberlink decoder+dxva limitations:

"1080p - Number of Reference frames must be equal to or less than 4
720p - Number of Reference frames must be be equal to or less than 8
Mixed Reference Frames must be false.
B Frames must equal 2
Adaptive B frames must be false"

Apparently not meeting any of the above causes the 720p 20fps bug, or the 1080p black screen.
arfster is offline   Reply With Quote
Old 20th December 2007, 22:04   #7  |  Link
Dark Shikari
x264 developer
 
Dark Shikari's Avatar
 
Join Date: Sep 2005
Posts: 8,690
Quote:
Originally Posted by arfster View Post
Copy/pasted this from somewhere that did tests to nail down the cyberlink decoder+dxva limitations:

"1080p - Number of Reference frames must be equal to or less than 4
720p - Number of Reference frames must be be equal to or less than 8
Mixed Reference Frames must be false.
B Frames must equal 2
Adaptive B frames must be false"

Apparently not meeting any of the above causes the 720p 20fps bug, or the 1080p black screen.
That's a very heavy set of restrictions...
Dark Shikari is offline   Reply With Quote
Old 20th December 2007, 22:09   #8  |  Link
akupenguin
x264 developer
 
akupenguin's Avatar
 
Join Date: Sep 2004
Posts: 2,393
That's strange. If you go by memory used, 4 frames of 1080p is equivalent to 9 frames of 720p, not 8.

If Level 4.1 were the only restriction, I might recommend people to restrict refs for hardware compatibility. But non-adaptive B-frames and no mixed refs? that's too harsh. Plus it's just plain a decoder bug, not a legitimate limitation. (As a codec developer, I can confidently say that adaptive B-frames do not require any explicit support. You just implement the standard, and any sequence of frame types is decodable.)

I question the DPB limit too... 1080p 16ref takes 50MB of RAM. Any video card recent enough to decode HD h264 will come with at least 256MB. Unless the decoder's memory is separate from the video memory? in which case it seems wasteful to dedicate 5% (12MB) to only h264 decoding when it could be reused.

Last edited by akupenguin; 21st December 2007 at 01:31.
akupenguin is offline   Reply With Quote
Old 20th December 2007, 22:15   #9  |  Link
Sulik
Registered User
 
Join Date: Jan 2002
Location: San Jose, CA
Posts: 215
The real restriction is the HD/BD profile & level restriction, ie: high profile, level 4.1.
For the number of references, this means that the maximum is 12288KB/(W*H*1.5):
1920x1080 -> max num_ref_frames = 4
1280x720 -> max num_ref_frames = 9
720x480 -> max num_ref_frames = 16 (max)

The mixed_ref/adaptive B-frames restriction is not a HW restriction, but more likely a bug in the Cyberlink decoder (or demux<->decoder interaction problem, or possibly though unlikely, encoder compliance issue).

I can confirm that 1080p content with Adaptive B-frames + mixed_ref with the proper number of references plays just fine with HW acceleration on my 8600GT.

Last edited by Sulik; 20th December 2007 at 22:21.
Sulik is offline   Reply With Quote
Old 21st December 2007, 03:18   #10  |  Link
arfster
Registered User
 
Join Date: Jun 2006
Posts: 169
To make clear: I've no idea if the above list is right, it was just something I copied off a forum somewhere for my own reference for future encodes. Haven't actually tested it at all.

If anyone needs to do some tests, feel free to point to some samples and I'll play them with my 2600.
arfster is offline   Reply With Quote
Old 21st December 2007, 07:54   #11  |  Link
NanoBot
Registered User
 
Join Date: Sep 2003
Posts: 164
Hi,

at least with my 8600gts I found another restriction: I have to uncheck "b-pyramids" during the encode to achieve a playback with full GPU decoding.

C.U. NanoBot
NanoBot is offline   Reply With Quote
Old 21st December 2007, 15:31   #12  |  Link
tetsuo55
MPC-HC Project Manager
 
Join Date: Mar 2007
Posts: 2,317
Well i would be willing to test encodes.

Maybe if we had a few test files, that where encoded in the same format as hddvd/bluray, so they can be easily tested in all hardware compliant software players i could try them and see which ones break on which players.

This way we could find out where the bug really lies and file a good bug-report to the responsible party.

As the issues effect both ATI and NVIDIA i think its safe to say that its not the hardware
tetsuo55 is offline   Reply With Quote
Old 22nd December 2007, 03:21   #13  |  Link
UsedUser
Registered User
 
Join Date: Sep 2006
Posts: 117
Quote:
Originally Posted by Sulik View Post
The real restriction is the HD/BD profile & level restriction, ie: high profile, level 4.1.
For the number of references, this means that the maximum is 12288KB/(W*H*1.5):
1920x1080 -> max num_ref_frames = 4
1280x720 -> max num_ref_frames = 9
720x480 -> max num_ref_frames = 16 (max)
I would agree, from my experience troubleshooting, that the profile/level compliance is the issue for DXVA.

Quote:
Originally Posted by Sulik View Post
The mixed_ref/adaptive B-frames restriction is not a HW restriction, but more likely a bug in the Cyberlink decoder (or demux<->decoder interaction problem, or possibly though unlikely, encoder compliance issue).
I would also agree that the mixed ref / adaptive b-frames issue is with the decoder, and is the source of the 20fps bug, as DXVA will work with the proper number of ref frames, but the 20fps bug persists with adaptive b-frames. I haven't tested mixed ref without adaptive b-frames to isolate it further, but the bug persists when using adaptive b-frames without mixed ref. So at a minimum, adaptive b-frames alone can cause the 20fps bug.
UsedUser is offline   Reply With Quote
Old 22nd December 2007, 05:22   #14  |  Link
valnar
Registered User
 
Join Date: Dec 2001
Location: Cleveland
Posts: 498
As somebody who wants to use DXVA and hardware players (like the Sage HD Extender), keeping to these "standards" is important. Not every playing device has 256MB of RAM.

Since most people use front ends, as tetsuo55 pointed out, it would be great if they could give a warning or perhaps change the default recommendations.

-Robert
valnar is offline   Reply With Quote
Old 22nd December 2007, 18:20   #15  |  Link
tetsuo55
MPC-HC Project Manager
 
Join Date: Mar 2007
Posts: 2,317
Well maybe the following tests will help:

-Play a b-frame encoded file with a different decoder (mainconcept and or nero)

-Encode a b-frame file with mainconcept, then try that file on cyberlink, mainconcept and nero.
tetsuo55 is offline   Reply With Quote
Old 22nd December 2007, 18:34   #16  |  Link
arfster
Registered User
 
Join Date: Jun 2006
Posts: 169
Quote:
Originally Posted by tetsuo55 View Post
Well maybe the following tests will help:

-Play a b-frame encoded file with a different decoder (mainconcept and or nero)
Afaik neither of those supports VLD acceleration, which is when the problem pops up :-(

Arcsoft seems to support it, but I couldn't get the h264 decoder to work outside the player itself, and the player only plays certain types of video.
arfster is offline   Reply With Quote
Old 22nd December 2007, 19:58   #17  |  Link
tetsuo55
MPC-HC Project Manager
 
Join Date: Mar 2007
Posts: 2,317
Quote:
Originally Posted by arfster View Post
Afaik neither of those supports VLD acceleration, which is when the problem pops up :-(

Arcsoft seems to support it, but I couldn't get the h264 decoder to work outside the player itself, and the player only plays certain types of video.
Again stressing my point that the file needs to be in a stand-alone player type container, so mkv is obviously out of the question
tetsuo55 is offline   Reply With Quote
Old 24th December 2007, 15:43   #18  |  Link
Crisidelm
Registered User
 
Join Date: Oct 2006
Posts: 31
Quote:
Originally Posted by arfster View Post
Afaik neither of those supports VLD acceleration, which is when the problem pops up :-(

Arcsoft seems to support it, but I couldn't get the h264 decoder to work outside the player itself, and the player only plays certain types of video.
Within Dvb software Dvbviewer the Arcsoft H.264 works fine, and Dvbviewer can be used to play other files, no only for live DVB viewing...
Crisidelm is offline   Reply With Quote
Old 29th December 2007, 22:38   #19  |  Link
valnar
Registered User
 
Join Date: Dec 2001
Location: Cleveland
Posts: 498
Is there a megui profile that is recommended to not only give the best quality, but have the highest compatibility for DXVA or hardware players like the Sage HD Extender?

What would those options be for HD or DVD rips?

Robert
valnar is offline   Reply With Quote
Old 29th December 2007, 22:42   #20  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 6,087
PS3 profile
Atak_Snajpera is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 02:58.


Powered by vBulletin® Version 3.8.9
Copyright ©2000 - 2017, vBulletin Solutions, Inc.