x264 cli bugreports

[Revgen]

Okay, I'll let you know what I find.

[Revgen]

After a lot of testing and head scratching, it seems that the vbv issues are being caused by --b-adapt 2. However, sometimes I can use --b-adapt 2 and it works just fine. Using --b-adapt 1 in the first pass and 2nd pass never results in vbv errors.

The vbv errors always happen in the last 419 frames.

This section is 474mb, so it'll take me awhile to upload it.

[Dark Shikari]

After a lot of testing and head scratching, it seems that the vbv issues are being caused by --b-adapt 2. However, sometimes I can use --b-adapt 2 and it works just fine. Using --b-adapt 1 in the first pass and 2nd pass never results in vbv errors.

The vbv errors always happen in the last 419 frames. These frames are 474mb, so it'll take me awhile to upload it.You can recompress it in x264 lossless if it helps.

By "last 419 frames", you mean that if you encode only those 419 frames, the error still happens, right (not that if you encode all the frames, the errors happen in the last 419 frames)?

Also, can you post the minimal commandline you found that produces the issue? If it only happens with --b-adapt 2, I suspect a bug in VBV lookahead.

[Revgen]

^Yep, only in the last 419 frames. I can encode those frames alone and get the errors.


Minimal Settings.

1st pass

x264 --profile high --level 4.1 --ssim --pass 1 --bitrate 40000 --stats "E:\King.stats" --thread-input --deblock -2:-2 --keyint 24 --min-keyint 2 --b-adapt 2 --direct auto --ref 4 --ipratio 1.1 --pbratio 1.1 --vbv-bufsize 30000 --vbv-maxrate 40000 --qcomp 0.5 --me dia --subme 2 --partitions none --trellis 0 --no-mixed-refs --mvrange 511 --nal-hrd --aud --sar 1:1 -o "g:\island7.264" "g:\island.avs"

2nd pass

x264 --profile high --level 4.1 --ssim --pass 2 --bitrate 40000 --stats "E:\King.stats" --thread-input --deblock -2:-2 --keyint 24 --min-keyint 2 --b-adapt 2 --direct auto --ref 4 --ipratio 1.1 --pbratio 1.1 --vbv-bufsize 30000 --vbv-maxrate 40000 --qcomp 0.5 --me umh --subme 9 --psy-rd 1.0:0.15 --partitions all --trellis 2 --no-mixed-refs --mvrange 511 --nal-hrd --aud --sar 1:1 --fullrange off --colorprim bt709 --colormatrix bt709 --transfer bt709 -o "g:\island7.264" "g:\island.avs"

[Dark Shikari]

Wait, it underflows on the first pass, second pass, or both?

And those aren't minimal settings--minimal means that you've tried to take out as many options as you can without causing it to stop underflowing. More options makes it harder for me to figure out what's causing the problem. I can do it myself, of course, but I can't get started until I have said 419 frames, so the more I know before I start working, the faster I can pinpoint the problem.

[Revgen]

Underflows on 2nd pass.

When you say minimalist settings, you mean when can I use --b-adapt 2 without causing errors? I'll look into that.

[Dark Shikari]

Underflows on 2nd pass.

When you say minimalist settings, you mean when can I use --b-adapt 2 without causing errors? I'll look into that.I mean you drop every single setting that doesn't have any relation to the problem.

So you drop a setting, check for underflow--if the underflow goes away, put the setting back. If it doesn't go away, drop the setting. Repeat until the commandlines are as short as possible.

A second pass underflow is very interesting; it means there's a bug in Gabriel's 2-pass VBV algorithm, not the VBV lookahead.

[Revgen]

Here's the last 419 frames.

http://www.megaupload.com/?d=RM8TF0R0


Finding minimal settings is difficult. Sometimes the settings bring vbv errors. Sometimes they don't. I have to run a test 3 times to make sure. The only certainty I get is that no matter what, --b-adapt 2 causes vbv errors sometimes and --b-adapt 1 never does. The rest is a mystery. I'll try to find the most minimal that I can, but I can't guarantee that they'll be exact.

[Dark Shikari]

Here's the last 419 frames.

http://www.megaupload.com/?d=RM8TF0R0


Finding minimal settings is difficult. Sometimes the settings bring vbv errors. Sometimes they don't. I have to run a test 3 times to make sure. The only certainty I get is that no matter what, --b-adapt 2 causes vbv errors sometimes and --b-adapt 1 never does. The rest is a mystery. I'll try to find the most minimal that I can, but I can't guarantee that they'll be exact.Downloading; I'll see what I can do.

[Dark Shikari]

I've run the settings a few times and I can't seem to get any VBV underflow to occur, using 3 threads on a Core 2 Duo.

[Revgen]

My cpu is a quad-core. I'll try 2 threads again and see what happens.

[Revgen]

Still getting errors.

I guess it's a problem on my end.

[Boolsheet]

If I try to encode the last 419 frames of The Island trailer I get those warnings too (late edit: Yes, only in second pass):
"x264 [warning]: VBV underflow (-75256 bits)"

They showed up with these settings:
--pass 1 and 2 (Same settings for both)
--b-adapt 0 or 2 (like Revgen I didn't get warnings with 1)
--bitrate 40000 --vbv-bufsize 30000 --vbv-maxrate 40000
--threads 8 or higher (I'm on a Core i7 HT on - makes 8 threads)

There are no warnings with fewer threads, no bframes or no-mbtree. I'm guessing it's those white frames beginning at frame 285 that causes problems.
Can the lookahead be disabled? It appears --rc-lookahead 0 does not have any effect. ;)

(I used JEEB's modified 1222 x64 build)

[Revgen]

I'm not alone afterall! :D

Now if only we can get a quad-core or better shipped out to DS or Aku.

Using less threads still gets me errors, but it takes a lot more tries to get it to happen. Most times it doesn't happen at all.

[Boolsheet]

Using less threads still gets me errors, but it takes a lot more tries to get it to happen. Most times it doesn't happen at all.
You're right, now I got 1 underflow with 7 threads. Maybe the frame-based threaded encoding gets sometimes the right frames and sometimes not.

[Sagittaire]

--bitrate 40000 --vbv-bufsize 30000 --vbv-maxrate 40000

Not really a bug with bitrate at max bitrate limite. With that you make buffering CBR encoding (aka VBR with only variable bitrate from buffer). It's really stressing setting for VBV.

You can probaly obtain better overall quality with --bitrate 30000 --vbv-bufsize 30000 --vbv-maxrate 40000 simply because encoder will have more buffer for really high complexity part. Higher is not always better and in this particular example encoding at 30 Mbps will certainely produce better quality than 40 Mbps encoding.

[Dark Shikari]

Hah, I ran it with 6 threads and I was able to replicate it. More analysis tomorrow :)

In general, underflows with 2-pass VBV should NEVER EVER HAPPEN, so bug reports like this are very important.

@Boolsheet, are you getting underflows in the first pass, or just second?

[Dark Shikari]

... and it seems I completely forgot, when writing VBV lookahead and MB-tree, that when VBV is on, the second pass still runs rc_analyse_slice. I guess it's a testament to how good Gabriel's algorithm is that it despite my stupid mistake... ;)

It seems that the naive fix doesn't exactly work though, so there might be a bit more to this.

Edit: .... and fixed locally. Commit coming today.

[Revgen]

You can probaly obtain better overall quality with --bitrate 30000 --vbv-bufsize 30000 --vbv-maxrate 40000 simply because encoder will have more buffer for really high complexity part. Higher is not always better and in this particular example encoding at 30 Mbps will certainely produce better quality than 40 Mbps encoding.

That's why this is an X264 Bug thread. :)

Edit: .... and fixed locally. Commit coming today.

Nice. Glad we could help. This Island trailer has managed to cause bugs in 2 programs already.

[Revgen]

Used Komisar's new generic build 1232, and I'm still getting the same VBV errors.

[Dark Shikari]

Used Komisar's new generic build 1232, and I'm still getting the same VBV errors.With pass 1 or 2?

Again, I need specifics, because I can't replicate the issue anymore.

[Revgen]

Pass 2. Same issues as before. Same 419 frames.

[Dark Shikari]

Settings used:

x264 --profile high --level 4.1 --ssim --pass 1 --bitrate 40000 --thread-input --deblock -2:-2 --keyint 24 --min-keyint 2 --b-adapt 2 --direct auto --ref 4 --ipratio 1.1 --pbratio 1.1 --vbv-bufsize 30000 --vbv-maxrate 40000 --qcomp 0.5 --me dia --subme 2 --partitions none --trellis 0 --no-mixed-refs --mvrange 511 --aud --sar 1:1 -o encode.h264 input.avs --threads 8

x264 --profile high --level 4.1 --ssim --pass 2 --bitrate 40000 --thread-input --deblock -2:-2 --keyint 24 --min-keyint 2 --b-adapt 2 --direct auto --ref 4 --ipratio 1.1 --pbratio 1.1 --vbv-bufsize 30000 --vbv-maxrate 40000 --qcomp 0.5 --me umh --subme 9 --psy-rd 1.0:0.15 --partitions all --trellis 2 --no-mixed-refs --mvrange 511 --aud --sar 1:1 --fullrange off --colorprim bt709 --colormatrix bt709 --transfer bt709 -o encode.h264 input.avs --threads 8 --verboseRan twice, no VBV issues.

[ajp_anton]

Using 1232 from x264.nl:

C:\Users\ajp_anton\Desktop\test>x264 --pass 1 --b-adapt 2 --bitrate 40000 --vbv-bufsize 30000 --vbv-maxrate 40000 --threads 12 -o NUL test.avs
avis [info]: 1920x1080 @ 23.98 fps (419 frames)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 FastShuffle SSE4.2
x264 [info]: profile Main, level 4.1
x264 [info]: slice I:39 Avg QP:13.22 size:248741
x264 [info]: slice P:168 Avg QP:16.17 size:208505
x264 [info]: slice B:212 Avg QP:16.59 size:193248
x264 [info]: consecutive B-frames: 12.6% 28.4% 31.6% 27.4%
x264 [info]: mb I I16..4: 63.4% 0.0% 36.6%
x264 [info]: mb P I16..4: 71.7% 0.0% 0.0% P16..4: 27.7% 0.0% 0.0% 0.0% 0.0% skip: 0.5%
x264 [info]: mb B I16..4: 35.3% 0.0% 0.0% B16..8: 28.6% 0.0% 0.0% direct:34.0% skip: 2.0% L0:26.6% L1:26.7% BI:46.7%
x264 [info]: coded y,uvDC,uvAC intra:80.4% 94.9% 67.3% inter:68.4% 80.5% 15.5%
x264 [info]: kb/s:39230.6

encoded 419 frames, 9.68 fps, 39231.15 kb/s

C:\Users\ajp_anton\Desktop\test>x264 --pass 2 --b-adapt 2 --bitrate 40000 --vbv-bufsize 30000 --vbv-maxrate 40000 --threads 12 -o NUL test.avs
avis [info]: 1920x1080 @ 23.98 fps (419 frames)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 FastShuffle SSE4.2
x264 [info]: profile High, level 4.1
x264 [warning]: VBV underflow (-1161622 bits)/s, eta 0:00:13
x264 [warning]: VBV underflow (-641179 bits)b/s, eta 0:00:13
x264 [warning]: VBV underflow (-645571 bits)b/s, eta 0:00:13
x264 [warning]: VBV underflow (-647619 bits)b/s, eta 0:00:13
x264 [warning]: VBV underflow (-36379 bits)kb/s, eta 0:00:12
x264 [warning]: VBV underflow (-344443 bits)b/s, eta 0:00:12
x264 [warning]: VBV underflow (-34931 bits)kb/s, eta 0:00:12
x264 [warning]: VBV underflow (-41579 bits)kb/s, eta 0:00:12
x264 [warning]: VBV underflow (-481859 bits)b/s, eta 0:00:12
x264 [warning]: VBV underflow (-34987 bits)kb/s, eta 0:00:12
x264 [warning]: VBV underflow (-20115 bits)kb/s, eta 0:00:12
x264 [warning]: VBV underflow (-206035 bits)b/s, eta 0:00:12
x264 [warning]: VBV underflow (-46475 bits)kb/s, eta 0:00:11
x264 [warning]: VBV underflow (-43891 bits)kb/s, eta 0:00:11
x264 [info]: slice I:39 Avg QP:15.56 size:210048
x264 [info]: slice P:168 Avg QP:16.75 size:216575
x264 [info]: slice B:212 Avg QP:16.90 size:201904
x264 [info]: consecutive B-frames: 12.6% 28.4% 31.6% 27.4%
x264 [info]: mb I I16..4: 17.9% 49.5% 32.7%
x264 [info]: mb P I16..4: 3.5% 39.3% 6.9% P16..4: 23.1% 13.2% 12.6% 0.0% 0.0% skip: 1.6%
x264 [info]: mb B I16..4: 1.9% 29.3% 4.6% B16..8: 28.8% 3.6% 5.0% direct: 8.8% skip:17.9% L0:39.9% L1:41.6% BI:18.5%
x264 [info]: 8x8 transform intra:74.3% inter:66.2%
x264 [info]: coded y,uvDC,uvAC intra:93.5% 96.8% 78.6% inter:58.6% 71.7% 22.5%
x264 [info]: ref P L0 58.1% 24.8% 17.1%
x264 [info]: ref B L0 76.7% 23.3%
x264 [info]: kb/s:40000.5

encoded 419 frames, 9.09 fps, 40001.03 kb/sSo far this has happened 9 times out of 9.
The numbers are always different, but it always happens around frame 290.

[Dark Shikari]

x264 [warning]: specified frame type is not compatible with max B-frames
x264 [error]: slice=P but 2pass stats say BI think you have bigger problems than VBV...

[Revgen]

Settings used:

Ran twice, no VBV issues.

Oh well. <shrugs>

I doubt this will ever be a problem on a real movie anyway.

[Dark Shikari]

Oh well. <shrugs>

I doubt this will ever be a problem on a real movie anyway.Sure it will... because I just replicated it with the settings above (12 threads).

2-pass VBV clearly needs work.

[Revgen]

^ So now you tell me. :)

If this Island trailer continues to give you trouble, you can just blame Ben. It's his fault anyway. ;)

[ajp_anton]

I think you have bigger problems than VBV...Sorry that was probably caused by me aborting it. So i re-ran it and edited the post hoping to do so before you read it but I was too slow =)

[Chevy787]

I'm getting a "unsupported input format (DIB )" error when running scripts through the 64 bit version of r1232 x264. If I run the same .avs through the 32 bit version, no errors appear.
Here is the script that is not working.

AVISource("N:\rag\sora\7\filter.avi")
FreezeFrame(15422,15422,15423)
Textsub("N:\rag\sora\7\miles\LogoEP7.ass")
Textsub("N:\rag\sora\7\miles\Sora_no_Mani_Mani_OP_EP7[kana-timed-v2].ass")
Textsub("N:\rag\sora\7\miles\Sora_no_Manimani_ED_EP7[kana-timed].ass")
ConverttoYV12()

I tried a slightly edited script without textsub, and this enabled the 64bit to run.

AVISource("N:\rag\sora\7\filter.avi")
FreezeFrame(15422,15422,15423)
ConverttoYV12()

After further testing, I found Loadplugin() and Tweak() also caused the same error even without the presence of textsub()

Here is my x264 log (using the the first avs mentioned above)

x264.exe --pass 1 --stats Doverlay.avs.264stat --bitrate 1850 --profile high --bframes 16
--deblock 0:0 --subme 7 --merange 8 --keyint 300 --b-adapt 2 --psy-rd 0.8:0.4 --aq-strength0.6
--no-mbtree --min-keyint 10 --weightb --partitions none --me hex --direct auto
--threads auto --thread-input --fps 23.976 --output NUL Doverlay.avs
avis [error]: unsupported input format (DIB )
x264 [error]: could not open input file 'Doverlay.avs'
----------------
x264.exe --pass 2 --stats Doverlay.avs.264stat --bitrate 1850 --profile high --bframes 16
--deblock 0:0 --subme 10 --merange 24 --keyint 300 --ref 16 --trellis 2 --psy-rd 0.8:0.4
--b-adapt 2 --aq-strength 0.6 --no-mbtree --no-dct-decimate --min-keyint 10 --me tesa
--partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --weightb --mixed-refs --no-fast-pskip
--direct auto --threads auto --thread-input --fps 23.976 --output X264vefour.mkv Doverlay.avs
----------------
Done.
Press any key to continue . . . avis [error]: unsupported input format (DIB )
x264 [error]: could not open input file 'Doverlay.avs'

[LoRd_MuldeR]

I'm getting a "unsupported input format (DIB )" error when running scripts through the 64 bit version of r1232 x264.

Is the 64-Bit version of Avisynth installed on your system ???

[Chevy787]

Is the 64-Bit version of Avisynth installed on your system ???

At one point, but I uninstalled it.

[LoRd_MuldeR]

At one point, but I uninstalled it.

So how do you think 64-Bit x264 should be able to use Avisynth input, when there's no 64-Bit Avisynth installed? :rolleyes:

[Chevy787]

So how do you think 64-Bit x264 should be able to use Avisynth input, when there's no 64-Bit Avisynth installed? :rolleyes:

I didn't think it was possible to install both the 32bit and the 64bit version at the same time....is it?
But yes, you do have quite a good point there.

[LoRd_MuldeR]

I didn't think it was possible to install both the 32bit and the 64bit version at the same time....is it?

Should be possible. The 32-Bit DLL goes to "%WINDIR%\SysWow64", while the 64-Bit DLL goes to "%WINDIR%\System32".

(Yes, the 64-Bit system directory is called System32 on 64-Bit Windows ^^)

But yes, you do have quite a good point there.

There is a way to use 64-Bit x264 with 32-Bit Avisynth, but Avisynth must run in a separate 32-Bit process and the video data must be piped into the 64-Bit x264 process.

I made a tool for that purpose: http://forum.doom9.org/showthread.php?t=144140

[Chevy787]

Thanks a bunch for the help

[Dark Shikari]

^ So now you tell me. :)

If this Island trailer continues to give you trouble, you can just blame Ben. It's his fault anyway. ;)Well the result of this is that it appears to be a very odd situation: 2-pass VBV breaks no matter how perfect the input information is--i.e. even multi-multi-pass doesn't fix the problem. Debugging suggests something very weird must be going on internally, so I'm trying to contact Gabriel to see what he can figure out.

Also, in the meantime, 1-pass works great for CBR ;)

[G_M_C]

I am really impressed at the rate of change in x264. The number of patches wich come out and that improve quality or speed is astonishing.

I do hope that that doesnt cause what i often encounter in my hobby-attempts at programming; The more i add or change, the messier the programm tends to get as a whole. I often end up having to go over the whole program to rewrite the basic structure at some point. Hopefully that's not the case now with x264.

Also in this post: I want to giive many cudo's to all developers, but to DS goes even more praise; Cause he almost seems to be the "spokesperson on all things x264" here on the board.

[juGGaKNot]

--pass 1 --preset veryslow --level 3.2 --bitrate 4500 --ref 5 --bframes 3 --ratetol 2.0 --merange 32 --direct temporal --aq-mode 2 --no-fast-pskip --no-dct-decimate --sar 1:1 --aud

300 frames source, uncompressed, avisynth 2.5.8 :

AVIsource("C:\x264\movie.avi")
ConvertToYV12()
LoadPlugin("C:\x264\bin\autocrop.dll")
AutoCrop(0, 16, 16, threshold=0)

error :

avis [info]: 1184x656 @ 30.00 fps (300 frames)
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Slow
x264 [info]: profile High, level 3.2
avis [info]: 1184x656 @ 30.00 fps (300 frames), eta 0:00:00
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Slow
x264 [error]: ratecontrol_init: can't open stats file
x264 [error]: x264_encoder_open failed

gets to the last frame, stalls, x264 windows error box.

jeebs : stalls for 2-3 minutes after first pass, works, output k
techouse : crashes.

[rack04]

300 frames source, uncompressed, avisynth 2.5.8 :



error :



gets to the last frame, stalls, x264 windows error box.

jeebs : stalls for 2-3 minutes after first pass, works, output k
techouse : crashes.

Can you post the source so I can test?

[juGGaKNot]

i will upload on friday

i got the same error with 1222

seems to be GCC 4.4.1 or march=core2 problem

i have to test all 1232 builds first.

[Mtz]

O have this error using Megui:
Assertion failed: dpb_output_delay < pow( 2, sps->vui.nal_hrd_parameters.i_dpb_output_delay_length ), file encoder/set.c, line 679
Posted also in megui thread (http://forum.doom9.org/showthread.php?p=1370933#post1370933) because I don't know who is giving the error.

enjoy,
Mtz

[Underground78]

I think this error was caused by an old version of the nal-hrd patch. Komisar uses this version of the patch for his builds which are included in Megui, I think he should use the newer version that can be found here (http://forum.doom9.org/showthread.php?t=152127) ...

[horvathd]

Hi!

My problem with x264 started when the new source options were introduced. The problem is, when I try to encode a video with non integer fps to mp4, the result play much faster then it should using Windows Media Player.

The problem doesn't come up:
- with Media Player Classic HC
- when I try to encode a video with integer framerate
- encoding to mkv
- using the --fps option in the command line

I'm not sure it's a bug in x264, but with r1376 there is no problem, but it's appears in builds with and without the new input options.

Please look into it. Thanks.

[kieranrk]

I'm not sure it's a bug in x264, but with r1376 there is no problem, but it's appears in builds with and without the new input options.


Can you upload a sample of such a file that doesn't play right?

[horvathd]

Here is a compilation of different versions: http://www.mediafire.com/?ythjzznkd25

The problem is with x264bug_23.976-video.mp4 is I use Windows Media Player.
Now I tried play them with mplayer, and I got the following error with all mp4 files, except the one made with r1376 (x264bug_23.976_1376_-video.mp4): [h264 @ 00B2DC90]reference picture missing during reorder

[VFR maniac]

Hi, horvathd.
Did you install Haali Media Splitter?
If you did this, I think probably that problem caused from Haali's splitter bug.

From rev1379, I implmented EditBox/edts writer for eliminating initial delay from mp4 splitter side.
Haali's splitter has a bug about EditBox, because he misunderstanded timescale of media_time. (MP4Box still misunderstands about this in import function side.)
MPC-HC's internal splitter doesn't have the same bug because this splitter cannot deal with EditBox.

Here is test build from Haali.
http://haali.su/mkv/mkx.e.7.exe
This build had been fixed about media_time issue.
Instead of this correction he broke blank clip function but this is not problem here.
Problem causes only when giving plus delay to mp4.

[horvathd]

Thanks!

With the new Haali Media Splitter there is no problem using Windows Media Player.
But with mplayer the problem is still exists (as it doesn't use the new splitter), so probably they splitter is broken too.

Since it's a problem on the decoding site, to resolve it everybody has to update the splitter but there is no stable build with the fix. Is there any better option for making a video which can be played back without problem than encoding to raw h264 and muxing to mp4 with an external program?

[Dracaena]

Hello there x264 gurus

I had not been using x264 for a number of months, and recently started using it again, through meGUI.
meGUI reports that the version I have is "1376 - Jeeb's patched build v2". The CLI itself reports "x264 core:80 r1376M 3feaec2".

The problem I am having is that x264.exe throws an exception, just at the moment before it would normally return me to the command prompt. Here is a screenshot of the point at which thiss happens:

http://www.users.on.net/~drallim/pic/x264_error.png

The error only occurs when outputting to .mp4, however the resulting file is perfectly usable and does not appear to contain any errors. Outputting to .mkv, .flv, or .264 works normally.
I have tried:

Using a very simple command line (with only output and input specified)
using very simple avs script as input
breaking the process before encoding is finished (ctrl+c)
various different switches - eg. disabling CPU optimisations, quiet mode etc.

In all cases the error occurs right before it would normally report "encoded x frames, x.x fps, x.x kb/s".
In all cases the resulting .mp4 file was usable and did not appear corrupt in any way.

Needless to say I will be using one of the other output formats until this is fixed because it prevents meGUI from proceeding to the next job in the queue. Anyway I thought it might be helpful to make somebody aware of it.

[nurbs]

Try replacing your x264.exe with a current build. Maybe it was fixed somewhere along the way or the issue is only with this build for whatever reason. None of the options you use have changed since r1376 so there shouldn't be any problems.

Another thing: --b-pyramid strict is only needed for blu-ray and you are obviously not encoding blu-ray. --b-pyramid normal is set by default in recent builds.