x264 Known Hardware accelleration problems and solutions

[ohropax]

@juGGaKNot

For me the video is played nicely using DXVA with both PowerDVD and MPC-HC decoders. I am using a NVidia Quadro NVS 140M chip.

Cheers,
Marcus

[juGGaKNot]

@juGGaKNot

For me the video is played nicely using DXVA with both PowerDVD and MPC-HC decoders. I am using a NVidia Quadro NVS 140M chip.

Cheers,
Marcus

THANKS man !

MPC-HC doesn't work for me, only MPC, tried core AVC 1.6.5 but the same.

Probably codec problems ( have k-lite codec pack and CCCP and KMPlayer installed )

K so DXVA-HD-HQ with sharktoot's matrix is the way to go!

Now : I also want the video to be compatible with standalone BLU-RAY players, what do i have to change to make my DXVA footage BLU-RAY ?

Is 24P the FPS i must use ?

I also have some 48P footage, is it BLU-RAY compatible ? or DXVA compatible ?

Cheers.

juGGaKNot.

LINK DOWN , new link :

http://files.filefront.com/final+rendermkv/;10021753;/fileinfo.html

[ericab]

juGGaKNot;
off topic, sorry;
what HL:S mod is that footage from ?

[tetsuo55]

It seems that the Pixel Aspect Ratio is not a problem after all, i have now seen several examples of working DXVA compatible files that have PAR's that should be incompatible

[canuckerfan]

so what's the general consensus about ref frames greater than 4? is any combination of GPU+CPU able to handle high ref frame material?

[UsedUser]

so what's the general consensus about ref frames greater than 4? is any combination of GPU+CPU able to handle high ref frame material?

First off, the ref frame limit of 4 only applies to 1080p content. What we're really looking at is compliance with H.264 Profile: High, Level: 4.1. Lower resolutions can handle more ref frames without breaking L4.1 compliance (important for DXVA across platforms). The OP has details.

Secondly, The consoles have L4.1 limitations, but on the PC, it is the Cyberlink decoder that enforces L4.1.

PCs with just about any CPU + NVIDIA 8000 or ATI 2000/3000 GPU can handle more ref frames beyond L4.1 when using the MPC decoder.

[tetsuo55]

No hardware decoder supports more than 4 ref frames with 1920x1080 content, that includes videocards.

[Avenger007]

No hardware decoder supports more than 4 ref frames with 1920x1080 content, that includes videocards.
Just curious, why does that include graphics cards? Is it a technical issue that can be rectified with a future revision of GPU architecture or are graphics cards (GPU(s) + memory) somehow just physically incapable of handling more than 4 ref frames for 1920x1080 content or is it a software issue? :confused:

[UsedUser]

No hardware decoder supports more than 4 ref frames with 1920x1080 content, that includes videocards.
Hmm... I know we've both been along for the full ride of figuring this out, but IIRC, the L5.1 (ref > 4) encodes I made @ 1080p played smoothly with the MPC decoder using DXVA. I'll go back to test those again.

[foxyshadis]

In decoding, the reference frame count only affects the amount of memory needed to decode. More memory means more frames possible. Video cards as they are now don't allow you to use the card's full memory though.

[tetsuo55]

They need to enfore the level rules to be certified for Bluray playback.

The only reason we are even discussing this is because some software decoders do not enforce levels, this should actually be a bug that needs to be fixed but in fact we found that "out of level spec" files are sometimes better, the best example being anime.

However most experts will say that more than 5 ref frames is usually pointless, so going down 1 notch for full HD should not matter that much.

[nm]

They need to enfore the level rules to be certified for Bluray playback.
Really? That doesn't sound reasonable since the same decoder could also be used to play other sources. Why would the Blu-ray certification restrict the decoder to a certain level without allowing it to support higher bitrates and buffer sizes?

Technical reasons like limited on-board memory on low-end graphics cards would be a more understandable explanation.

The only reason we are even discussing this is because some software decoders do not enforce levels, this should actually be a bug that needs to be fixed but in fact we found that "out of level spec" files are sometimes better, the best example being anime.
I don't see any point in enforcing levels in a software decoder unless it has been allocated with a limited amount of memory.

[tetsuo55]

Well when i say bluray i mean h264 standard.

The reason why you do not want to create out of spec files (and remove all support for them) is basically to keep the videostream compatible with all players.

The playback method has been standerdised in h264, every decoder must show the same image.

This is a better explenation:

For some reason the buffer size has been limited to 12MB, this means that all hardware decoders that want 100% support for their IDC level need to have a 12MB buffer.

I imagine that level 5.1 files will only be able to use 1 or 2 ref frames.

Now a more interesting question is:

Is the 12MB limit an overal limit for all h264 levels or.. is it the maximum for level 4.1. IF its a 4.1 limit and 5.1 supports a bigger limit then all we need is level 5.1 support for the videocards and more ref frames would be possible for pc use (but these would then be level 5.1 files and will not be compatible with 4.1 limited devices anymore)

It's probably best to always stay within the limits of the codec for maximum playback support

[nm]

Is the 12MB limit an overal limit for all h264 levels or.. is it the maximum for level 4.1.
Only for level 4.1. According to the Wikipedia article (http://en.wikipedia.org/wiki/H.264#Levels), level 5.1 allows 1080p120 with 16 reference frames.

IF its a 4.1 limit and 5.1 supports a bigger limit then all we need is level 5.1 support for the videocards and more ref frames would be possible for pc use (but these would then be level 5.1 files and will not be compatible with 4.1 limited devices anymore)
Exactly.

[tetsuo55]

Hey someone updated that article

Then the solution would simply be asking all videocard vendors to support level 5.1, still i all for levels. sticking to levels means your file will work no matter where you try to watch it.

[UsedUser]

The bottom line, so it seems, is that commercial decoders support up to L4.1 because that is the upper limit for the BD / HD DVD specs.

It's clear that the hardware decoders (and commercial software decoders meant to use DXVA, i.e., Cyberlink's) were designed to decode BD / HD DVD content. It may be that they are capable of decoding above L4.1 (from a memory/bandwidth/throughput standpoint), but that they weren't designed for it, so they don't support it. Not a foreign concept in software or hardware design to build in room for future expansion, but limit current functionality to targeted specs.

This is not to say, however, that any of these decoders couldn't support higher than L4.1, as BD / HD DVD only requires that you support at least L4.1. Any individual decoder supporting L5.1 wouldn't break L4.1 compatibility, for that decoder or any other.

Where is it written that a decoder can at most decode L4.1 content in order to sport some sort of BD / HD DVD certification/logo?

[valnar]

I'm not seeing what the issue is. Just encode to L4.1 specs and you'll be fine for today and in the future. Has anyone done any comparative analysis of how much space you waste using REF 3 vs REF 15? Is it that much? The only reason to use L5 or better is if you need to encode to a higher resolution than 1920x1088 or need more than 30fps at that resolution. If so... then I can understand an argument against it.

[Dark Shikari]

Is the 12MB limit an overal limit for all h264 levels or.. is it the maximum for level 4.1.Its just a Level 4.1 maximum (and even then, only for specific profiles).

[UsedUser]

I'm not seeing what the issue is. Just encode to L4.1 specs and you'll be fine for today and in the future.
For anyone paying attention here, I don't think there's an issue getting them to encode to L4.1. It's addressing encodes done by others that's at issue.

For 720p, the constraints are loose enough that problems with encodes are unusual. For 1080p, encodes beyond L4.1 are still common.

It's a matter of determining the specific limitations imposed by the various decoders to figure workarounds, if available.

For example, determining that having a L4.1 encode still requires have the IDC set to L4.1 for the Cyberlink decoder. Or that MKVs muxed with older versions of mkvmerge can cause issues.

[lexor]

Is the 12MB limit an overal limit for all h264 levels or.. is it the maximum for level 4.1.
It's the overall limit. In context of video cards, 12mb is not related to levels (DXVA solutions don't check the level). Here is how it came to be, graphics cards manufacturers wanted to label their products as HD-DVD/Blu-Ray compliant, so they had to decode 1080p, 12mb is how much space they need to store 4 uncompressed frames (max ref at 1080p). So if your frames are smaller, i.e. 720p or 480p etc you can fit more of them in the 12mb.

This storage space is fixed for all streams regardless of any profiles and settings.

[nm]

It's the overall limit. In context of video cards, 12mb is not related to levels (DXVA solutions don't check the level).
Sure it is related and some DXVA-compatible decoders (like Cyberlink) do check the level.

Here is how it came to be, graphics cards manufacturers wanted to label their products as HD-DVD/Blu-Ray compliant, so they had to decode 1080p, 12mb is how much space they need to store 4 uncompressed frames (max ref at 1080p).
4 frames is maximum for 1080p at level 4.1.

This storage space is fixed for all streams regardless of any profiles and settings.
...for all level 4.1 streams (like Blu-ray and HD-DVD). Higher levels require larger buffers.

[foxyshadis]

It would be an interesting exercise for someone to create a tool that selectively re-encodes by keeping track of which frames are referenced but 'stale' (would be missing in L4.1), and re-encode any macroblock referencing one of those frames to one of the good frames instead, using whatever motion search you like, then fixing up fixable references (such as future predicted vectors). You just have to live with some minimal corruption of any frames that reference those blocks, but it beats complete corruption.

[UsedUser]

It's the overall limit. In context of video cards, 12mb is not related to levels (DXVA solutions don't check the level). ... This storage space is fixed for all streams regardless of any profiles and settings.
That A) doesn't make a whole lot of sense, and B) stands in stark contrast to Dark Shikari's statement just 2 posts prior to yours.

A) If it were an overall limit, and not related to L4.1, why would the hardware designers pick a 4 refs @ 1080p limit? The H.264 levels support specs well beyond those. They picked the limits precisely because they are those of L4.1, to which the BD / HD DVD specs comply. Complying with BD / HD DVD means you support L4.1 specifically.

Higher levels require larger buffers, and video cards have buffers to spare for decoding.

B) Dark Shikari seems to be just the code monkey (with affection) we need to answer such questions.

[lexor]

A) If it were an overall limit, and not related to L4.1, why would the hardware designers pick a 4 refs @ 1080p limit? The H.264 levels support specs well beyond those. They picked the limits precisely because they are those of L4.1, to which the BD / HD DVD specs comply. Complying with BD / HD DVD means you support L4.1 specifically.

I'll answer you and nm here. Not a contradiction to what I said. DXVA doesn't check levels. Cyberlink's player (in nm's example) only uses DXVA, what other software does is irrelevant to what DXVA does itself. Also MPC doesn't check level of stream, I have level5.1 streams playing with full acceleration in MPC-HC, ref count is what matters, not the level stamp on the stream.


Higher levels require larger buffers,
Which did not enter into consideration of hardware designers who only cared about getting their cards certified for HDDVD/Blu-Ray compatibility.


and video cards have buffers to spare for decoding.
They have more RAM, but the memory for DXVA's use is fixed, it cannot be changed on current cards.


B) Dark Shikari seems to be just the code monkey (with affection) we need to answer such questions.
My statements come from devs of MPC-HC whom I asked the question about ref limits. As good as DS is and for all the awesome work he has done for x264, he's not the one writing DXVA enabled decoder.

[MatMaul]

They have more RAM, but the memory for DXVA's use is fixed, it cannot be changed on current cards.
it is driver related and we can't bypass that kind of limitation.
Anyway L4.1 is enough to reach high quality with x264, so everybody should use that now.

[Dark Shikari]

It's the overall limit.

...

This storage space is fixed for all streams regardless of any profiles and settings.Your statements here seem to be almost designed to be horribly confusing ;)

The DPB size is a limit set by the H.264 standard. Any decoder compliant with a given level must have a buffer of at least the DPB size for that level.

The graphics card doesn't magically change the buffer it has available, so when playing lower-level streams, it still has the full buffer, it just doesn't get used. It doesn't get used because, by definition, if the stream is a lower level, it won't use up as much DPB on decoding.

Its very misleading to say "its the overall limit" because that implies that all hardware devices with H.264 playback, regardless of level (iPod? PSP? 8600GT?), have the same buffer size, which is patently false. Saying all L4.1-compliant hardware decoders have at least a certain buffer size is much more accurate.

[lexor]

Its very misleading to say "its the overall limit" because that implies that all hardware devices with H.264 playback, regardless of level (iPod? PSP? 8600GT?)

The only way it is confusing is if you start a new thread that is not about DXVA, or did someone get DXVA to run on iPod or PSP?

My statement is that 12mb is the overall limit DXVA has available to it and it cannot be changed on current cards. In context of this thread it has one and only interpretation, that all streams get 12mb for the DXVA decoder to work with at all times and it cannot be changed without buying some future card that's not on the market right now.

My statement is neither confusing nor misleading. Wishful thinking of others (mostly those who look at the 512mb or even 1gb of memory their cards have and ask, why can't I use it) notwithstanding.

it is driver related and we can't bypass that kind of limitation.
Anyway L4.1 is enough to reach high quality with x264, so everybody should use that now.
Oh, I didn't say I wanted more, I'm quite happy with the way things are.

[nm]

The only way it is confusing is if you start a new thread that is not about DXVA, or did someone get DXVA to run on iPod or PSP?
This thread and tetsuo55's question was not just about DXVA.

My statement is neither confusing nor misleading.
Well, I also think it was misleading until you clarified what you were talking about.

[tetsuo55]

The only way it is confusing is if you start a new thread that is not about DXVA, or did someone get DXVA to run on iPod or PSP?



Let me be clear about this:

This discussion is about ALL devices that support H.264

Yes DXVA compliant encodes do work on ipod and PSP as long as they are encoded at the correct level.

We talk mostly about DXVA because its the easiest target for testing/troubleshooting

Also the only reason MP-HC does not check level is because i explicitly asked Casimir to do so. Instead of checking levels MPC-HC uses a calculation of ref frames to decide wether the file is 4.1 or lower compatible or not. This is a "workaround" for inaccurate encodes with wrong level_IDC settings.

--------------------------------------------------

So we need a definative answer here:

How big a difference is 4 ref frames and 15 ref frames for 1080p?
And at which point do more ref frames have zero to no effect, from using search it seems that going over 5 ref frames has very little effect.

So i fail to see how using 4 ref frames would be a huge problem

[nm]

How big a difference is 4 ref frames and 15 ref frames for 1080p?
Depends on the source. It's probably not significant for most real life footage, but animated stuff may gain quality with more reference frames (>10 or so).

So i fail to see how using 4 ref frames would be a huge problem
It's not huge unless you already have a lot of encodes with more refs.

[UsedUser]

So we need a definative answer here:

How big a difference is 4 ref frames and 15 ref frames for 1080p?
And at which point do more ref frames have zero to no effect, from using search it seems that going over 5 ref frames has very little effect.
Depends on the material, of course, so we can't come up with anything definitive.

But, anecdotally, I generally saw a 5th ref frame was used at less than 3% of opportunities. IIRC, the graph for additional ref frames is hyperbolic, limit approaching 0% use.

ref 5: 3%
ref 6: 2%
ref 7: 1.5%


Something like that, I forget. The math isn't right, there.

What happened to the x264 log output that shows the ref frame usage?, e.g.:

x264 [info]: ref P 68.0% 17.8% 6.6% 4.6% 2.9%
x264 [info]: ref B 83.3% 10.7% 2.9% 1.9% 1.2%

[foxyshadis]

And that's only part of the equation. The more important part is, how much better is a higher reference match than a lower reference? (In aggregate.) So if on average 5% of the frames get 1% better with another reference, you only have a .05% total improvement to the video.

[lexor]

hmm... that's an interesting point, when you set --ref in x264, is it guaranteed to use all of them? is there a way to check if it did?

[tetsuo55]

And that's only part of the equation. The more important part is, how much better is a higher reference match than a lower reference? (In aggregate.) So if on average 5% of the frames get 1% better with another reference, you only have a .05% total improvement to the video.

again strenghtening my point..

Ideally the use of 5 or more ref frames should be proven to have less than 20% effect on the total image quality.

If so a max limit of 4 for dxva compatibilty in all cases can be used, this would ensure dxva compatibility on all levels and on all devices (including levels above and below 3 and 4)

Also i wonder if the improvement in anime is really due to the more ref frames or some other reason. if so maybe that other reason could be added to x264 as a patch.

[lexor]

/EDIT: sorry, wrong thread

[Turtleggjp]

Has anyone been able to get interlaced video from x264 working with DXVA? My video camera's footage (Canon HF100) seems to be quite unstable at seeking, and I was looking into re-encoding it with x264, but so far I haven't been able to play back the re-encodes correctly. I am able to play back the original clips and get 30i -> 60p de-interlacing using MPC and PowerDVD 8's decoder using my Radeon 3450, but not with anything I have encoded with x264 using the --interlaced option.

I haven't played with x264 for almost 2 years now, so I don't know how far along its support for interlaced video is. I became interested again when I was able to follow the directions in this thread to make some 1080p @ 23.976 clips that do playback accelerated. I am interested in accelerating interlaced video because:

1. My video camera footage is interlaced, and
2. Most of the 1080p video I encode (currently with XviD) is from 1080i broadcasts, and I was interested in seeing how well these would look encoded with x264 and accelerated during playback.

Thanks!

Matt

[nm]

You'll need a build with x264_hrd_pulldown.04_interlace.diff, like the builds at x264.nl and x264.tk. Then encode with parameters --nal-hrd and --interlaced (or --tff and --bff to specify the correct field order), and specify VBV buffer size and maximum rate.

Related discussion: http://forum.doom9.org/showthread.php?t=137432

[Turtleggjp]

Thanks! Will give it a try tonight

[Turtleggjp]

The interlaced video works great now. Now another question.

When I play back a video encoded at 1920x1080 with DXVA (PowerDVD 8), I get a blurry line near the bottom, probably the last 8 lines. Since a mod16 size was mentioned as a requirement, I tried encoding it at 1920x1088 and the blur went away. The only problem is that now it no longer fills my 1920x1080 plasma tv perfectly. Is there a way to encode the video at 1920x1088, but crop the last 8 lines during playback? I see that CoreAVC seems to have this option, but can this also be done either by a setting in the stream, or through some DXVA configuration? Thanks.

Matt

[foxyshadis]

Update PowerDVD (http://www.cyberlink.com/multi/download/patches_1_ENU.html). Update your video drivers if that doesn't fix it.

[P.J]

Awesome topic !
Now I can play my mkv files w/ DxVA (8600M GT DDR2 177.41)

For some 720p: IDC Changer 0.3
For 1080p and a few 720p: MeGUI w/ DXVA HQ Profile

But H264info was useless for me.


MPC-HC EVR Custom w/ subtitle, PDVD8 H.264, PDVD8 audio decoder MD3 (doesn't work for DTS :( WHY?), Vista HP x86 w/ Aero
Overall, I don't like MPC-HC's GUI :(

[shon3i]

Is there any news about nvidia DXVA problem?

[P.J]

Yes, PV3 which exists in GeForce 9 series. I think it has better post-processing for VC-1/H.264 !

EDIT: Forceware 177.41 supports H.264 720p L5.1.
Now you can play most of H.264 720p files with DxVA.
So you don't need to IDC Changer !

[P.J]

The interlaced video works great now. Now another question.

When I play back a video encoded at 1920x1080 with DXVA (PowerDVD 8), I get a blurry line near the bottom, probably the last 8 lines. Since a mod16 size was mentioned as a requirement, I tried encoding it at 1920x1088 and the blur went away. The only problem is that now it no longer fills my 1920x1080 plasma tv perfectly. Is there a way to encode the video at 1920x1088, but crop the last 8 lines during playback? I see that CoreAVC seems to have this option, but can this also be done either by a setting in the stream, or through some DXVA configuration? Thanks.

Matt

I also find this problem. Let me try it ;)

[tetsuo55]

I have updated the first post to be a little more informative.

Also i have rewritten a lot of the information to reflect the current status of hardware compatibility.

What we still need though is:

-Prove that players do or do not all support 50Mbit/s for level 4.1.
-the real-life max bitrate and max CPB have not yet been determined, if you want to help please encode some files and try to find the upper limit.
-Confirmed hardware limits for levels below 3

[CruNcher]

DONE:
-b-pyramids works on all hardware devices including portable ones since x244 SVN721


Are you sure of that i get heavy problems with PSP (FAT) 720x480 Full SD and b-pyramid for all Encoders ?, i can see with every Encoder strange temporal distortions (it's playable but it doesn't look nice when it happens highering Mhz also doesn't help).

[valnar]

I agree. I still think encoding without B-pyramids is the way to go, especially for future compatibility too. It doesn't gain you that much.

[shon3i]

@tetsuo55, how you calculate MAX ref frame limit = 6531840 when MaxDPB for level 3.1 are 6750 so 6750*1024/1.5 = 4608000

Formula for max ref frame is 4608000 / (Height X Width) for level 3.1

[tetsuo55]

Are you sure of that i get heavy problems with PSP (FAT) 720x480 Full SD and b-pyramid for all Encoders ?, i can see with every Encoder strange temporal distortions (it's playable but it doesn't look nice when it happens highering Mhz also doesn't help).

Now i see what you mean, You are using a resolution that is beyond the capability of the PSP device.
I See that you want to use full resolution videos on your PSP, and in your testing disabling b-pyramids allows you to do so.
To confirm this could you please try a file at 480x272 resolution (PSP screen size, and target for accurate hardware profile) with b-pyramids enabled?

I agree. I still think encoding without B-pyramids is the way to go, especially for future compatibility too. It doesn't gain you that much.
B-Pyramids are 100% standard compliant since x264 has fixed the encoding bug. Also the gains are substantial enough according to the x264 devs


@tetsuo55, how you calculate MAX ref frame limit = 6531840 when MaxDPB for level 3.1 are 6750 so 6750*1024/1.5 = 4608000

Formula for max ref frame is 4608000 / (Height X Width) for level 3.1

You are right, i made a miscalculation.

i updated the formula, thanks!

[audyovydeo]

Hello tetsuo and thanks for keeping this thread updated, I think it's one of the most important if we want to maximise the number of compliant streams "out there".

I've just one question regarding your settings : I notice you specify --merange 12, but I don't see how that is a "hard" constraint. I have always used the default of 16. Besides, since it is itself an option of an analysis parameter, I dont see how it should affect profile / level compliance. Am I wrong ?

cheers
audyovydeo