x264 Known Hardware accelleration problems and solutions [Archive] - Page 5

highjds

10th February 2008, 04:50

Help me T..T

--------------------------------------------------------------------------------

Hi !!

I wishes that we should understand although my English is awkward.

I'm living in South-Korea and use the megui currently.

I do ripping for collection and store.

I read DXVA encoding article some time ago.
(x264 + more than 4 ref frames = no DXVA or 20fps bug (ati avivo, nvidia purevideo)

Change the option in the megui is difficult to me.
I want to turn off the option not to be display in the information for a 'me-prepass = 0'.
And, We are marked by qcomp=1.00 after encoding.

Will I get DXVA Profile's file though I am sorry?

I want the profile of the option like below.
Your aid is urgent me. Please!!!

My E-mail : highjds@naver.com

[ About H.264 encoding ]

User data: x264
User data: core 56 svn-681
User data: H.264/MPEG-4 AVC codec
User data: Copyleft 2005
User data: http://www.videolan.org/x264.html
User data: cabac=1
User data: ref=5
User data: deblock=1:-3:-3
User data: analyse=0x3:0x133
User data: me=umh
User data: fpel_cmp=sad
User data: subme=6
User data: me-prepass=0
User data: brdo=1
User data: mixed_ref=1
User data: me_range=16
User data: chroma_me=1
User data: trellis=1
User data: 8x8dct=1
User data: cqm=2
User data: deadzone=10,10
User data: chroma_qp_offset=0
User data: threads=3
User data: nr=0
User data: decimate=0
User data: mbaff=0
User data: bframes=3
User data: b_pyramid=1
User data: b_adapt=1
User data: b_bias=0
User data: direct=1
User data: wpredb=1
User data: bime=1
User data: keyint=250
User data: keyint_min=25
User data: scenecut=40(pre)
User data: rc=2pass
User data: bitrate=5621
User data: ratetol=1.0
User data: rceq='blurCplx^(1-qComp)'
User data: qcomp=0.60
User data: qpmin=10
User data: qpmax=51
User data: qpstep=4
User data: cplxblur=20.0
User data: qblur=0.5
User data: ip_ratio=1.40
User data: pb_ratio=1.30
User data: aq=1:0.7:15.0
SPS id: 0
Profile: High@L5.1
Num ref frames: 8
Chroma format idc: YUV 4:2:0
PPS id: 0 (SPS: 0)
Entropy coding type: CABAC
Weighted prediction: No
Weighted bipred idc: B slices - implicit weighted prediction
8x8dct: Yes
Custom intra4X4 luma:
6 7 10 16
7 7 11 17
10 11 12 20
12 13 20 16
Custom intra4X4 chromau:
16 16 16 16
16 16 16 16
16 16 16 16
16 16 16 16
Custom inter4X4 luma:
10 13 28 41
13 14 32 84
28 32 41 111
41 46 111 16
Custom inter4X4 chromau:
16 16 16 16
16 16 16 16
16 16 16 16
16 16 16 16
Custom intra8X8 luma:
9 9 10 10 11 16 21 29
9 9 10 10 11 16 21 29
10 10 10 10 11 16 22 31
10 10 10 11 11 17 23 33
11 11 11 11 11 19 25 36
12 12 13 13 14 20 27 40
15 15 16 16 21 27 31 45
20 20 21 22 36 40 45 16
Custom inter8X8 luma:
12 13 15 18 20 70 163 255
13 13 16 18 20 72 170 255
15 16 17 19 21 81 190 255
18 18 19 21 23 96 228 255
20 20 21 23 25 120 255 255
33 34 36 39 45 32 255 255
64 66 71 80 164 255 255 255
151 155 169 192 255 255 255 10

tetsuo55

10th February 2008, 14:57

Has anyone tested non standard mod16 width's??

Something like 1600x720?

So to create a 100% compatible file:

-use the latest svn revision of x264
-use a SAR of 1:1/4:3/5:4/16:9
-For resolutions higher than 720x576
--mod16 width X mod16 height X ref frames <= 8355840
-For resolutions equal or lower than 720x576
--mod16 width X mod16 height X ref frames <= 4561920

other settings do not seem to matter

UsedUser

11th February 2008, 03:04

im trying to change the ref frames to 7 on this height edited sample

The point here is to find out why 90-100% compliant files do not get DXVA acceleration, most of the files i am testing here conform to all the DXVA specs but do not get accelerated, i'm simply trying to find out why. By changing certain things with h264info the problem can either be solved or the real issue found. This might even lead to another incompatible feature being found, even if it doesnt, the guide to fixing existing encodes can be updated with more fixes.

Ok, I get your point. But you need to get mine, too. Simply changing the label is not going to make the content the chips receive any different than before changing it. Saying in the label the frame is now 1280x544 does not change the fact the chip receives a 1280 x 536 frame to decode. Saying in the label the stream uses only 7 reference fames does not change the fact the chip receives a stream which has more than 7 reference frames in certain spots. If the chip did not like it before, it's not going to like it now either despite what your advertisement (label) said.
While there is value in determining what in a stream's header makes a particular decoder refuse to decode the stream, it is important to note that figuring that out, in some situations, may be fruitless.

What I mean to say is, changing the num_ref_frames by editing the header doesn't actually change the DPB required by the stream, so (as KoD pointed out), it doesn't ultimately help anything. While you may discover that the decoder is checking the stream header for num_ref_frames, it is doing so for a reason, because even if you get the decoder to successfully decode the stream, you get errors because it is, in actuality, out of spec with L4.1. The actual num_ref_frames for a stream cannot be changed by simply editing the header.

On the other hand, --level is not often specified when using x264, and as a result, the encoder defaults to labeling the stream L5.1 (presumably because it is safe to say any stream under L5.1 will also comply with L5.1). So, you get streams that comply with L4.1 in DPB size and other parameters, but are labeled L5.1. Editing the header to change the level_idc label to L4.1 is all that is required to get the decoder to decode successfully without errors.

Overall, you may expose what parameters the decoder is checking for, but it doesn't change the fact that level_idc is editable, and it is on you to ensure the stream complies with the label you give it, while num_ref_frames is not editable. DPB size is a hard requirement.

tetsuo55

11th February 2008, 10:00

snip

I'm sorry but you are wrong, except for those clips i have that have ref frames that are way out of bounds, all my previously non working files work now. (although some have a line of artifacts below the rendered image, but this does not always appear, and its basically a dark grey line)

My testing reveals that decoders only check the packet header, and then let the videocard decode whatever is inside if it passes the required checks. after that it becomes obvious that only a too high number of ref frames causes any real problems with the playback (artifacts, stuttering, 20fps)

CruNcher

11th February 2008, 16:32

I'm sorry but you are wrong, except for those clips i have that have ref frames that are way out of bounds, all my previously non working files work now. (although some have a line of artifacts below the rendered image, but this does not always appear, and its basically a dark grey line)

My testing reveals that decoders only check the packet header, and then let the videocard decode whatever is inside if it passes the required checks. after that it becomes obvious that only a too high number of ref frames causes any real problems with the playback (artifacts, stuttering, 20fps)

Tough the Dark Grey line is because that file wasn't mod16 encoded and you forced it to be mod16 ;)
And i allready made clear that changing this num_ref_frames for what tetsuo55 describes as "out of bounds" encoded streams will couse visual problems in most hardware decoders and even in software you gonna see the same problems then changing the num_ref_frames, but this was just thougt as a workaround for acceptable Visual error rates (and maybe hardware decoder that can handle it) and if one can't live with this he could still retranscode it (wich is of course the prefered method to handle such out of spec streams) :)

tetsuo55

11th February 2008, 17:53

Indeed, the best solution is to re-encode, especially with a new AQ patched SVN build ;)

By the way, does anyone know a solution to this related DXVA problem? its a problem with remuxing MPEG2 streams: http://forum.doom9.org/showthread.php?t=134615

(i think MKV and haali splitter might still have a lot of DXVA incompatibility issues)

UsedUser

12th February 2008, 08:01

My testing reveals that decoders only check the packet header, and then let the videocard decode whatever is inside if it passes the required checks. after that it becomes obvious that only a too high number of ref frames causes any real problems with the playback (artifacts, stuttering, 20fps)
That's exactly what I said. You can get the decoder to decode the stream, but it's still out of spec, so you get errors (artifacts, stuttering, 20fps, etc). Which is in contrast to tightening the IDC label to inform the decoder the stream is actually in spec.

So, if you don't get a properly decoded stream, again, it's essentially fruitless. Sure you can get DXVA, but how is that superior to software decoding if it's fraught with errors? Perhaps some are minor, but at least for me, the goal here was to get DXVA for an intact stream.

tetsuo55

18th February 2008, 20:30

On this topic, what is the actual buffer size and max bitrate required for hardware 4.1 support?

I'm seeing vastly conflicting information everywhere; the H.264 standard says the L4.1 CPB size is 62500kbits, while the MeGUI profiles have both 9000 and 14475, and max bitrate seems to vary wildly between 25000 and 50000 (!?).

What are the actual correct numbers?

this is indeed very interessting info

Could someone encode some test samples??

20 seconds video or something

1 with a CPB size of 62500kbits, other settings same as we use for DXVA compatibility
2 with a CPB size of 14475kbits, other settings same as we use for DXVA compatibility
3 with a CPB size of 9000kbits, other settings same as we use for DXVA compatibility
4. bitrate of 50000, other settings same as we use for DXVA compatibility

if 4 doesn't work try 25000, if that does we could up it until the limit is found

What i know about blu-ray/HDDVD benchmarks 50000 is the max for these disc's standard. some discs have already been released which are well over the 40000, it would be interesting to find the real upper limit, keep going up if 50000 works

-------------------

Again, its clear now that as long as you keep SAR. ref frame limit and mod16 x264 creates a DXVA compatible file, the only thing thats not completely clear is that if pyramids does or doesn't break compatibily on standalones and consoles(ps3/xbox360)

It would be nice to see an option in x264 or frontends that auto-selects the maximum number of ref frames based on the resolution with a -hardware or -dxva switch

akupenguin

18th February 2008, 22:18

You have to determine max bitrate before you can experiment with CPB. Because if you disagree with the decoder about what speed the CPB is filling, then you'll disagree about its fullness. But you don't need to know CPB to test bitrate, assuming your sample is long enough to overflow any reasonable CPB if bitrate is too high.

shon3i

18th February 2008, 23:39

It would be nice to see an option in x264 or frontends that auto-selects the maximum number of ref frames based on the resolution with a -hardware or -dxva switch

Or automaticly make decision based on resolution and framerate. I don't see point to somebody now encode with high level than should.

CruNcher

24th February 2008, 19:32

Mod please sticky this and rename it to something like (Known Hardware accelleration problems with X264)

ggab

25th February 2008, 08:29

and could anybody make a litle sumary post? (collecting the facts about you should care about doing your own DXVA enable's encodes)

thnks

tetsuo55

25th February 2008, 18:55

and could anybody make a litle sumary post? (collecting the facts about you should care about doing your own DXVA enable's encodes)

thnks

I updated the first post.

However i still think that x264 or the frontends should auto calculate the correct settings now that we know all the facts (besides max bitrate and CPB)

CruNcher

25th February 2008, 23:11

Also everyone should be aware that X264s VBV is not 100% standard complaint yet so it could fail in alot of situations so even if you take care there's no guarante that it works 100% with every device in every situation im afraid.

tetsuo55

26th February 2008, 13:17

what VBV? and what seems to be the problem with it?

CruNcher

26th February 2008, 16:04

it's the Video Buffer Verifier it's there to control to heavy bitrate spikes and hold everything constant so Hardware Decoders have no problems to decode a stream (complexity) and it's clearly in the Spec how a stream has to look like in terms of bitrate complexity for every level (some devices have their own VBV also it's called CPB i think Coded Picture Buffer specs, HD-DVD and Blu-Ray)

UsedUser

2nd March 2008, 22:09

I have a little time to execute some test cases, but I'm not very knowledgeable about CPB. I've picked up from CruNcher that VBV is synonymous with CPB. I'm not sure, though, if we're talking about max bitrate or VBV max bitrate, or if they should correspond.

Are these what I should be testing? What am I looking for (breaking DXVA (software decoding), stuttering, picture corruption)?

Baseline:

--pass 2 --stats ".stats" --level 4.1 --bframes 3 --weightb --direct auto --analyse p8x8,b8x8,i4x4,i8x8 --8x8dct

Plus these additional settings:

Establishing max bitrate:
1. --bitrate 62500 --vbv-maxrate 62500
2. --bitrate 50000 --vbv-maxrate 50000
3. --bitrate 25000 --vbv-maxrate 25000
4. [lower or higher values to nail down max bitrate]

Establishing max VBV size (let [maxrate] = the max bitrate determined above):
1. --bitrate [maxrate] --vbv-maxrate [maxrate] --vbv-bufsize [maxrate]
2. --bitrate [maxrate] --vbv-maxrate [maxrate] --vbv-bufsize 25000
3. --bitrate [maxrate] --vbv-maxrate [maxrate] --vbv-bufsize 14475
4. --bitrate [maxrate] --vbv-maxrate [maxrate] --vbv-bufsize 9000

akupenguin

2nd March 2008, 23:37

Establishing max bitrate:
1. --bitrate 62500 --vbv-maxrate 62500

You have to specify some bufsize, but you don't know the right value yet. So pick something small, like 5000.

Establishing max VBV size (let [maxrate] = the max bitrate determined above):
1. --bitrate [maxrate] --vbv-maxrate [maxrate] --vbv-bufsize [maxrate]

There's no reason bufsize couldn't exceed maxrate, that just means it's >1 second.

UsedUser

3rd March 2008, 06:43

You have to specify some bufsize, but you don't know the right value yet. So pick something small, like 5000.
To be clear, it works without specifying a bufsize, it simply gives the following warning:

x264 [warning]: VBV maxrate specified, but no bufsize.

I'm just wondering if there is a default that was used that is suitable for this test, or do I need to re-encode the files?

There's no reason bufsize couldn't exceed maxrate, that just means it's >1 second.
OK, I will crank up the bufsize past the maxrate, once that is found.

So, what am I looking for? How do I know when I've found the max bitrate, and how do I know when I've found the max bufsize?

If "smooth playback" is the criterion, then my preliminary tests indicate the max bitrate is somewhere around 40000.

akupenguin

3rd March 2008, 07:26

To be clear, it works without specifying a bufsize, it simply gives the following warning:

x264 [warning]: VBV maxrate specified, but no bufsize.

That means it didn't do anything. VBV is simply meaningless if you don't specify a bufsize. What default could there possibly be?
Of course you also have to check that it uses near your specified bitrate (checking average is enough). This isn't a given at 62mbit.

So, what am I looking for? How do I know when I've found the max bitrate, and how do I know when I've found the max bufsize?
Smooth playback. But to know that it's choppy due to VBV rather than decoding power, set all the other options to minimize decoding cost. Mainly that means CAVLC and no partitions.

Now, it could be that the device doesn't have any strict VBV maxrate, and only says it does to limit decoding complexity. But if that's the case then you're screwed: there would be no consistent number which you'll be safe if you stay below, there's only a very fuzzy limit depending on content and other options.

UsedUser

4th March 2008, 09:46

When I encode with --bitrate 200000, I still only get ~65000 in the output.

Thus far, I get smooth playback on every stream, no matter how big or small I set combinations of the bitrate, VBV bitrate, or VBV bufsize. I stopped testing at --bitrate 200000 --vbv-bufsize 200000 --vbv-maxrate 200000, but since I'm only getting ~65000 kb/s, are the other settings even valid?

One note:

If you specify any VBV bufsize, the value used in the encoding must be a minimum of 10% of the VBV bitrate, correct? This is why I asked about a default for the bufsize: When I tried to set bufsize low, i.e., 5000, it was always bumped up to 10% of the bitrate (i.e., with bitrate 90000, x264 [warning]: VBV buffer size too small, using 9009 kbit).

So what do I need to do? Find a very complex scene (i.e., flock of birds from Planet Earth ep. 1) that will require a higher bitrate, or is it meaningless to test beyond 62500-65000?

"C:\Program Files\MeGUI\tools\x264\x264.exe" --b
itrate 200000 --level 4.1 --bframes 3 --no-b-adapt --direct auto --no-cabac --su
bme 1 --analyse none --vbv-bufsize 200000 --vbv-maxrate 200000 --me dia --thread
s auto --thread-input --sar 1:1 --progress --no-psnr --no-ssim --output "D:\Vide
o\[2007-02-04] Superbowl XLI\Prince - Superbowl Halftime Show (HD OTA) 1080p L4.
1 max200000 buf200000.mp4" "D:\Video\[2007-02-04] Superbowl XLI\Prince - Superbo
wl Halftime Show (HD OTA) 1080p.avs"
avis [info]: 1920x1080 @ 29.97 fps (1001 frames)
x264 [info]: using SAR=1/1
x264 [warning]: VBV bitrate (200000) > level limit (50000)
x264 [warning]: VBV buffer (200000) > level limit (62500)
x264 [info]: using cpu capabilities: MMX MMXEXT SSE SSE2 SSE3 3DNow!
mp4 [info]: initial delay 1001 (scale 30000)
x264 [info]: slice I:11 Avg QP:10.39 size:25588900:00
x264 [info]: slice P:720 Avg QP:10.44 size:296301
x264 [info]: slice B:270 Avg QP:11.08 size:219575
x264 [info]: mb I I16..4: 26.2% 0.0% 73.8%
x264 [info]: mb P I16..4: 43.1% 0.0% 0.0% P16..4: 56.3% 0.0% 0.0% 0.0% 0
.0% skip: 0.6%
x264 [info]: mb B I16..4: 10.5% 0.0% 0.0% B16..8: 68.2% 0.0% 0.0% direct:
16.1% skip: 5.2%
x264 [info]: direct mvs spatial:70.0% temporal:30.0%
x264 [info]: kb/s:65972.9

encoded 1001 frames, 7.95 fps, 65973.23 kb/s

"C:\Program Files\MeGUI\tools\x264\x264.exe" --b
itrate 66000 --level 4.1 --bframes 3 --no-b-adapt --direct auto --no-cabac --sub
me 1 --analyse none --vbv-bufsize 5000 --vbv-maxrate 66000 --me dia --threads au
to --thread-input --sar 1:1 --progress --no-psnr --no-ssim --output "D:\Video\[2
007-02-04] Superbowl XLI\Prince - Superbowl Halftime Show (HD OTA) 1080p L4.1 ma
x66000 buf05000.mp4" "D:\Video\[2007-02-04] Superbowl XLI\Prince - Superbowl Hal
ftime Show (HD OTA) 1080p.avs"
avis [info]: 1920x1080 @ 29.97 fps (1001 frames)
x264 [info]: using SAR=1/1
x264 [warning]: VBV bitrate (66000) > level limit (50000)
x264 [info]: using cpu capabilities: MMX MMXEXT SSE SSE2 SSE3 3DNow!
x264 [warning]: VBV buffer size too small, using 6606 kbit
mp4 [info]: initial delay 1001 (scale 30000)
x264 [info]: slice I:11 Avg QP:11.49 size:22608100:00
x264 [info]: slice P:720 Avg QP:11.03 size:269783
x264 [info]: slice B:270 Avg QP:12.00 size:187602
x264 [info]: mb I I16..4: 28.7% 0.0% 71.3%
x264 [info]: mb P I16..4: 43.2% 0.0% 0.0% P16..4: 56.2% 0.0% 0.0% 0.0% 0
.0% skip: 0.6%
x264 [info]: mb B I16..4: 10.5% 0.0% 0.0% B16..8: 67.3% 0.0% 0.0% direct:
16.8% skip: 5.4%
x264 [info]: direct mvs spatial:68.5% temporal:31.5%
x264 [info]: kb/s:59253.4

encoded 1001 frames, 8.05 fps, 59253.71 kb/s

Dark Shikari

4th March 2008, 09:56

When I encode with --bitrate 200000, I still only get ~65000 in the output. Set --qpmin 0.

Moondust

4th March 2008, 22:10

When using HA there is some kind of macroblocking/dithering/corruption on a large scale (almost every scene). It also flickers in those areas where the picture is corrupted. It's not watchable though since it's not a smooth picture. Does any of you know what this is and how I can fix it?

I am using AC3Filter, DirectSound, ATI display driver v7.11, VMR9 renderless (only one working with HA) and the Cyberlink H264 decoder.

UsedUser

5th March 2008, 02:33

UsedUser

5th March 2008, 02:40

Set --qpmin 0.
Did the trick, thanks.

Update: It appears 80Mb is the practical limit for bitrate. Just past 80Mb stuttering begins, and gets worse as the bitrate gets higher.

Now I'll go to work on the VBV bufsize.

This stream plays back smoothly:

"C:\Program Files\MeGUI\tools\x264\x264.exe" --b
itrate 81000 --qpmin 0 --level 4.1 --bframes 3 --no-b-adapt --direct auto --no-c
abac --subme 1 --analyse none --vbv-bufsize 5000 --vbv-maxrate 81000 --me dia --
threads auto --thread-input --sar 1:1 --progress --no-psnr --no-ssim --output "D
:\Video\[2007-02-04] Superbowl XLI\Prince - Superbowl Halftime Show (HD OTA) 108
0p L4.1 max81000 buf05000 qpmin0.mp4" "D:\Video\[2007-02-04] Superbowl XLI\Princ
e - Superbowl Halftime Show (HD OTA) 1080p.avs"
avis [info]: 1920x1080 @ 29.97 fps (1001 frames)
x264 [info]: using SAR=1/1
x264 [warning]: VBV bitrate (81000) > level limit (50000)
x264 [info]: using cpu capabilities: MMX MMXEXT SSE SSE2 SSE3 3DNow!
x264 [warning]: VBV buffer size too small, using 8108 kbit
mp4 [info]: initial delay 1001 (scale 30000)
x264 [info]: slice I:11 Avg QP: 9.16 size:32411500:00
x264 [info]: slice P:720 Avg QP: 8.02 size:364740
x264 [info]: slice B:270 Avg QP: 9.74 size:236751
x264 [info]: mb I I16..4: 31.7% 0.0% 68.3%
x264 [info]: mb P I16..4: 42.4% 0.0% 0.0% P16..4: 57.0% 0.0% 0.0% 0.0% 0
.0% skip: 0.6%
x264 [info]: mb B I16..4: 9.9% 0.0% 0.0% B16..8: 69.0% 0.0% 0.0% direct:
15.9% skip: 5.3%
x264 [info]: direct mvs spatial:70.0% temporal:30.0%
x264 [info]: kb/s:79066.0

encoded 1001 frames, 7.51 fps, 79066.53 kb/s

This stream stutters slightly on playback:

"C:\Program Files\MeGUI\tools\x264\x264.exe" --b
itrate 82000 --qpmin 0 --level 4.1 --bframes 3 --no-b-adapt --direct auto --no-c
abac --subme 1 --analyse none --vbv-bufsize 5000 --vbv-maxrate 82000 --me dia --
threads auto --thread-input --sar 1:1 --progress --no-psnr --no-ssim --output "D
:\Video\[2007-02-04] Superbowl XLI\Prince - Superbowl Halftime Show (HD OTA) 108
0p L4.1 max82000 buf05000 qpmin0.mp4" "D:\Video\[2007-02-04] Superbowl XLI\Princ
e - Superbowl Halftime Show (HD OTA) 1080p.avs"
avis [info]: 1920x1080 @ 29.97 fps (1001 frames)
x264 [info]: using SAR=1/1
x264 [warning]: VBV bitrate (82000) > level limit (50000)
x264 [info]: using cpu capabilities: MMX MMXEXT SSE SSE2 SSE3 3DNow!
x264 [warning]: VBV buffer size too small, using 8208 kbit
mp4 [info]: initial delay 1001 (scale 30000)
x264 [info]: slice I:11 Avg QP: 9.40 size:32648300:00
x264 [info]: slice P:720 Avg QP: 7.92 size:368847
x264 [info]: slice B:270 Avg QP: 9.62 size:240689
x264 [info]: mb I I16..4: 32.5% 0.0% 67.5%
x264 [info]: mb P I16..4: 42.4% 0.0% 0.0% P16..4: 57.0% 0.0% 0.0% 0.0% 0
.0% skip: 0.6%
x264 [info]: mb B I16..4: 9.8% 0.0% 0.0% B16..8: 69.1% 0.0% 0.0% direct:
15.8% skip: 5.3%
x264 [info]: direct mvs spatial:70.7% temporal:29.3%
x264 [info]: kb/s:80035.1

encoded 1001 frames, 8.12 fps, 80035.65 kb/s

CruNcher

5th March 2008, 04:07

how about with cabac hehe :D
and what happens if you use i frames only so --keyint 1 and no b-frames

did you also made sure that the file you playback from is either in memory or fully defraged?

UsedUser

5th March 2008, 10:00

how about with cabac hehe :D
and what happens if you use i frames only so --keyint 1 and no b-frames
Those will have to be tests for another time. My patience has run out for now.

did you also made sure that the file you playback from is either in memory or fully defraged?
Umm, no.

Results with NVIDIA 8800GT + Cyberlink H264 decoder using DXVA:

VBV max bitrate: 80000-82000 kb/s
VBV max bufsize: 80000-162000 kb

Once i found 80-82mb as a reliable max bitrate, I tested VBV bufsize up to 200% of VBV max bitrate, then quit.

To reliably play back a file smoothly, VBV max bitrate = 80000 and VBV bufsize = 80000.

To intermittently play back a file smoothly, VBV max bitrate up to 80000, VBV bufsize up to 162000.

When I say intermittently, if I looped a 30-sec clip, sometimes it played smoothly, sometimes it stuttered.

Perhaps (or maybe even likely) this has more to do with Cruncher's questions about defragging and loading a clip into memory. Frankly, I just don't care to test the limits that much. As far as I went, for kicks, I encoded a clip at VBV max bitrate/bufsize = 200mb. It plays at about half speed.

If the H.264 spec states L4.1 limits VBV max bitrate = 50000 and VBV bufsize = 62500, then I've been able to show that DXVA works lovely beyond those values. Beyond that, it's really just getting ridiculous.

But, it's also important to keep in mind that this is with very low encoding complexity, per aku's recommendations.

I tried encoding a couple clips with the PS3 profile that is best for L4.1 compatibility. At VBV max bitrate/bufsize = 80000, the decoder chokes and I get about 1 frame every 10 seconds. With VBV max bitrate = 50000 and VBV bufsize = 62500, I get very choppy playback. With b-pyramid disabled, I get some stuttering, but at least it plays back in real time.

What the practical VBV bitrate/bufsize limits are for the PS3 profile, I haven't yet tested. Given that I've seen reports of some HD DVD / BD discs going beyond 40mb, I'm assuming they're largely substituting bitrate for encoding complexity.

CruNcher

5th March 2008, 14:32

jep crazy how far Nvidias solution goes i think they made it even intermediate Proof for future AVC Intra Production workflows that's why i asked if you could try with --keyint 1 :) in that case 80 mbit would be the lower range and somewhere @ 120 mbit the medium and then it goes higher for very professional quality use :)

Moondust

5th March 2008, 22:31

Those would be the tell-tale signs of exceeding L4.1. Have you checked the stream to see that it complies? If so, is the level_idc declared as High@L4.1?
Yes, it's High@L4.1. All recent L4.1 compliant scene encodes show this corruption. I don´t know where to look. Registry settings, drivers, codecs? I hope anyone can point me in the right direction.

CruNcher

5th March 2008, 23:03

upload a sample

UsedUser

6th March 2008, 00:27

Yes, it's High@L4.1. All recent L4.1 compliant scene encodes show this corruption. I don´t know where to look. Registry settings, drivers, codecs? I hope anyone can point me in the right direction.
You may try upgrading your driver; there have been good reports about 8.2 for HTPC usage. Also, I would always recommend applying the standard reg tweaks (http://home.comcast.net/~exdeus/ati-hd2x00/) for DXVA.

Uploading a sample would allow us to verify it is L4.1 compliant, but troubleshooting a specific issue is better left in the AVS Forum thread (http://www.avsforum.com/avs-vb/showthread.php?p=13302808#post13302808) where you've posted. If you've followed the guidance there, specifically on the driver version, Cyberlink decoder version, and reg settings, then that's really the best advice there is.

tetsuo55

6th March 2008, 17:20

So can i conclude that the following does no longer apply to create compliant files?

--vbv-maxrate 25000

I can remove this setting from the first post right?

However there could still be playback problems on devices like xbox, 360 and psp

I added an extreme setting, anyone willing to try it, i would test a 10 or 20 second clip.

--level 4.1 --ref * --mixed-refs --bframes 3 --b-rdo --bime --weightb --direct auto --subme 7 --trellis 1 --8x8dct --TESA --merange 24 --no-fast-pskip --partitions al--b-pyramid

i wonder if this will still work with DXVA, and if the quality is improvement is actually worth the slower encoding speed

To get the most out of these extreme settings a patched build should be tested (another good test to see if the patches break DXVA)

rack04

6th March 2008, 17:35

Has any MeGUI video profile been updated to provide a DXVA compliant file? FYI, I don't see a PS3 only PD-PS3-XBOX360.

tetsuo55

6th March 2008, 17:53

I don't use megui so i don't know, you safest bet is manual settings, the one's in the first post will lead to dxva compatible files.

Sharktooth

6th March 2008, 18:09

i set up 4 profiles (ill publish them later in the auto-update... waiting for DXVA compliancy confirmation):

DXVA-HD-HQ: --level 4.1 --ref 4 --mixed-refs --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -2,-1 --subme 6 --trellis 1 --analyse p8x8,b8x8,i4x4,i8x8 --8x8dct --me umh

DXVA-HD-Fast: --level 4.1 --ref 3 --mixed-refs --bframes 3 --b-pyramid --weightb --direct auto --filter -2,-1 --subme 5 --trellis 1 --analyse p8x8,b8x8,i4x4,i8x8 --8x8dct --merange 12

DXVA-SD-HQ: --level 3.1 --ref 8 --mixed-refs --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -2,-1 --subme 6 --trellis 1 --analyse p8x8,b8x8,i4x4,i8x8 --8x8dct --me umh

DXVA-SD-Fast: --level 3.1 --ref 3 --mixed-refs --bframes 3 --b-pyramid --weightb --direct auto --filter -2,-1 --subme 5 --trellis 1 --analyse p8x8,b8x8,i4x4,i8x8 --8x8dct --merange 12

Please check if there are any parameters that screws DXVA compatibility. All profiles are 2pass.

tetsuo55

6th March 2008, 18:19

i set up 4 profiles (ill publish them later):

DXVA-HD-HQ: --level 4.1 --ref 4 --mixed-refs --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -2,-1 --subme 6 --trellis 1 --analyse p8x8,b8x8,i4x4,i8x8 --8x8dct --me umh

DXVA-HD-Fast: --level 4.1 --ref 3 --mixed-refs --bframes 3 --b-pyramid --weightb --direct auto --filter -2,-1 --subme 5 --trellis 1 --analyse p8x8,b8x8,i4x4,i8x8 --8x8dct --merange 12

DXVA-SD-HQ: --level 3.1 --ref 8 --mixed-refs --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -2,-1 --subme 6 --trellis 1 --analyse p8x8,b8x8,i4x4,i8x8 --8x8dct --me umh

DXVA-SA-Fast: --level 3.1 --ref 3 --mixed-refs --bframes 3 --b-pyramid --weightb --direct auto --filter -2,-1 --subme 5 --trellis 1 --analyse p8x8,b8x8,i4x4,i8x8 --8x8dct --merange 12

Please check if there are any parameters that screws DXVA compatibility. All profiles are 2pass.

Those look safe to me, but ref3 in the last one, is it real worth it?

Sharktooth

6th March 2008, 18:21

both "Fast" profiles have --ref 3.
i could lower it to 1 though...

tetsuo55

6th March 2008, 18:24

what i mean is, the encoding time is only marginally faster from what i understand, when lowering the number of ref frames.

the higher ref frames could lead to a better image, with only a tiny bit slower encode times, or i am completely wrong here?

Sharktooth

6th March 2008, 18:25

ehrr... 16 refs is damn slow... and more than 5 refs is usually useless (unless the source is toons/anime)

tetsuo55

6th March 2008, 18:31

ehrr... 16 refs is damn slow... and more than 5 refs is usually useless (unless the source is toons/anime)

Okay would be nice if we had a quality scale.

How much extra quality/compression does 16 refs offer over 15, and is that quality improvement linear or not? so would the same improvemt apply for going from 1 ref to 2 refs?

If there is hardly any improvement we might be able to reduce the options list to a more basic HD and SD profile that just works for everything (but retains quality within a 5% margin )

Sharktooth

6th March 2008, 18:33

it's not linear, it's more like a descending curve. most gain comes going from 1 to 2... until about 3 refs the gain is good. on 4 and 5 there's a small gain... over 5 refs the gain is negligible (unless encoding toons/anime).

rack04

6th March 2008, 18:33

Sharktooth

6th March 2008, 18:38

you mean the DXVA-HQ profiles? then yes, more or less.
the Fast profiles are more comparable to HQ-Fast.

rack04

6th March 2008, 18:41

you mean the DXVA-HQ profiles? then yes, more or less.
the Fast profiles are more comparable to HQ-Fast.

Yes, DXVA-HQ. Thanks for these profiles. Can't wait to test.

Sharktooth

6th March 2008, 18:43

so, if the DXVA profiles are ok, i'll do some other changes to non-DXVA profiles and will put the package on the megui auto-update.

EDIT: Profiles are up. enjoy.

shon3i

6th March 2008, 19:03

--b-pyramid is not safe option, can easly brake compatability.

-level 3.2 aslo can be used on SD material such as 1024x576 or 853x480

-level 4.0 is safer option for HD

Moondust

6th March 2008, 19:04

You may try upgrading your driver; there have been good reports about 8.2 for HTPC usage. Also, I would always recommend applying the standard reg tweaks (http://home.comcast.net/~exdeus/ati-hd2x00/) for DXVA.
Already did that, no effect.

Uploading a sample would allow us to verify it is L4.1 compliant, but troubleshooting a specific issue is better left in the AVS Forum thread (http://www.avsforum.com/avs-vb/showthread.php?p=13302808#post13302808) where you've posted. If you've followed the guidance there, specifically on the driver version, Cyberlink decoder version, and reg settings, then that's really the best advice there is.
I tried about 10 recent 1080p scene encodes and they all show the same corruption pattern. I am unable to upload the sample files, but you can all find them on usenet (sample is included in all scene encodes). Nothing special about those encodes. Only thing I haven't thought of is upgrading the Cyberlink decoder. I am using Powerdvd 7.3319a. ATM my HTPC is broken (won't start, probably mainboard, have to send it back), but as soon as my htpc is up and running I will try that. Thanks for your response, CruNcher as well. :-)