Help! Can't use x264 --b-pyramid without ffdshow crashing instantly

[Tox|k]

i'm having a problem with x264 and ffdshow that for the life of me i cannot figure out.

Whenever i encode a video using x264 and the --b-pyramid option, I can't play it back without ffdshow instantly crashing on me no matter what.

I figured it might be some container issues, so I tried encoding to the MP4 container directly from x264 as well as placing the raw h.264 stream in MP4 with mp4box after. no go.

I've tried running x264 with it's gui generating the command as well as making my own. still nothing.

The strange thing is, i can still playback stuff i encoded with a prior revision (<285 i think) of x264 using the EXACT same options... it just doesn't seem to work now for some reason.

I currently have r287 installed from x264.nl, but i've also tried revision 286 from the official thread here and have the same problem.
My version of ffdshow is 20050803 (also from x264.nl)

My options are

x264.exe -p 1 --bitrate 750 --ref 3 --bframes 3 --b-pyramid --subme 6 --weightb --analyse all --8x8dct --qpmin 5 --qpstep 1 --me umh --progress --no-psnr --stats "x264.stats" --output NUL %1.avs
x264.exe -p 3 --bitrate 750 --ref 3 --bframes 3 --b-pyramid --subme 6 --weightb --analyse all --8x8dct --qpmin 5 --qpstep 1 --me umh --progress --no-psnr --stats "x264.stats" --output NUL %1.avs
x264.exe -p 2 --bitrate 750 --ref 3 --bframes 3 --b-pyramid --subme 6 --weightb --analyse all --8x8dct --qpmin 5 --qpstep 1 --me umh --progress --no-psnr --stats "x264.stats" --output %1.264 %1.avs
(The %1 is just to make batch processing easier)

I've tried to, but can't narrow it down to either x264 or ffdshow as the problem.

like i said, as soon as i remove the --b-pyramid option, it plays back fine.
I suppose i could just not use it, but that would irritate me to no end for as long as i used it.

[bond]

does it crash when being played in mplayer or videolan?
does it crash with the nero avc decoder?

can you upload a small snip showing this crash (you can cut .mp4 with mp4box)

[Tox|k]

Right now i'm using MPC for playing. I've tried the clip in the old and new windows media player with the same result.

I don't have the nero avc decoder installed, the whole point of using haali splitter + ffdshow is so that i don't have to.

BTW, i don't think i mentioned it, but i also tried using different avi clips for my source... i was wondering if the fact it was compressed (for what ever reason) was the problem, so i started using uncompressed AVI.

I've been using the THX intro to test settings and try to find the source of the problem, so i can easily post post the entire thing. I've also attached a picture of the error i get.

The crash error, mp4 file and raw h.264 should be available at http://reimconsulting.com/x264crash/

Thanx

[cuban]

I can confirm crashes. I tried to change the container to mkv, but it still crashes MPC (ffdshow). In mplayer everything works fine. Seems like a bug in ffdshow...

[bond]

i now encoded with your commandline and sharktooth's latest compile and it worked out fine with latest haali and latest ffdshow
no problem

my source is a .vob decoded via dgdecode (the standard procedure), but as long as your .avs input is fine (eg shown fine in virtualdub) the source shouldnt be the cause for the problem


but the raw .264 file you linked to indeed crashes the two filters when remuxed to .mp4 (with both mp4box and mp4creator)

[cuban]

Ok, You can reproduce this error with simple avs script:

ColorBars()
ConvertToYV12()
Trim(0,777)

The .264 file plays without problems in mplayer, but muxed to mp4 with newest (18/08/2005) mp4box crashes ffdshow. Mp4 produced by x264 CLI (v286) is also not playable with ffdshow but playable in mplayer.

Is it correct (i'm not sure anymore...)?

[bond]

Ok, You can reproduce this error with simple avs script:

ColorBars()
ConvertToYV12()
Trim(0,777)

The .264 file plays without problems in mplayer, but muxed to mp4 with newest (18/08/2005) mp4box crashes ffdshow. Mp4 produced by x264 CLI (v286) is also not playable with ffdshow but playable in mplayer.

Is it correct (i'm not sure anymore...)?indeed, i could also reproduce this with r281

as it also happens if you output to raw and remux to .mp4 with mp4creator i dont think its a gpac related problem

videolan and mplayer play the stream fine, anyone with nero filters wanting to try it?

[Tox|k]

Alright, so let me see if i got this straight. The bug is not in x264 but is a problem with ffdshow and certain players (MPC, WMP)? The bug only occurs after the h.264 stream has been muxed into MP4? muxed with any tool?
The bug occurs even with back revisions of x264 (seeing as bond tried r281)?

the only thing i'm confused about then is
i now encoded with your commandline and sharktooth's latest compile and it worked out fine with latest haali and latest ffdshow
no problem
was this encode in the mp4 container or not? if so, was it played with MPC? and it worked?

...
just trying to get my head around what works with what, and what's crashing whatever else.

Thanx

[bond]

according to haali this was a bug in ffdshow already fixed in cvs

we will have to wait for a new compile of ffdshow

[Tox|k]

Alright, that i guess i can live with.

Thanx for all your help guys.

[cuban]

the only thing i'm confused about then is

was this encode in the mp4 container or not? if so, was it played with MPC? and it worked?


I guess that ffdshow currently doesn't handle bframes used as references properly. From my knowledge bframes kept as refrences are very rare in mpeg4 streams, so this bug does not have to apply in every file produced with --b-pyramid option. But i might be wrong... Can anyone confirm this?

[bond]

i think you can check this yourself by checking the 2pass log file created by x264. you can see b-frames marked with "b" and "B". one of those means b-reference, dunno which one

i am not sure but i think x264 uses b-ref always when at least 2 b-frames in a row are used!?

[cuban]

i am not sure but i think x264 uses b-ref always when at least 2 b-frames in a row are used!?
You are right as usual :) (if B really means b-frame kept as reference).

I did additional test and ffdshow fails even if there is not a single b-frame in a stream :confused: In this case i dont know how --b-pyramid can make ffdshow crash... But if it is solved, it doesnt matter anymore. Thanks Bond for Your help!

[bond]

seems to be fixed in the latest build (22 aug)

[cuban]

Yes, i checked it a few days ago and it works perfectly. So we have another problem solved :)

[cuban]

Heh... It was too fast answer !

Can someone confirm that the file I attached to this post is valid mp4 stream and that it crashes ffdshow (22aug)? Its a part of the file I created with x264 (> r270, i dont remember exactly), and it used to play on ffdshow. Now it plays only in mplayer...

--EDIT-- I added second (bigger) sample. Do You need bigger (or longer) file?

For more details: i use latest Haali splitter, ffdshow 22aug. and the file was cut out of the original file with latest mp4box.

[bond]

Can someone confirm that the file I attached to this post is valid mp4 stream and that it crashes ffdshow (22aug)? Its a part of the file I created with x264 (> r270, i dont remember exactly), and it used to play on ffdshow. Now it plays only in mplayer...hm that file is too small to check anything imho
upload a bigger one plz, like with 5secs

[celtic_druid]

Also make sure that you test with the MSVC7.1 build and not just the ICL one.

[cuban]

OK, the samples work with msvc7.1 build. What's wrong with ILC build? Am i missing something? Shouldnt' it work in both compiles?

[Tox|k]

OK, the samples work with msvc7.1 build. What's wrong with ILC build? Am i missing something? Shouldnt' it work in both compiles?

I know this doesn't add much in way of content to the thread... but I too am curious why :confused:
since it's msvc7.1, does this mean those who *nix it are screwed?

[celtic_druid]

Well the dshow in ffdshow stands for directshow, so it isn't intended for *nix anyway.

That said as far as I know it isn't possible to produce a working binary with gcc even though it should be possible to cross compile.

That said, this is only the second sample that I have seen able to crash the ICL build.
Does it crash if you use a splitter other than haali's? Because in the other case I believe with Nero splitter + ffdshow there was no crash.

[ian8210]

OK,I got crashed with ffdshow 22 Aug , whenever B-frames is enabled. I don't know exactly the difference between the Max consecutive 1,and 2. But I got crashed anyway, so I set this to zero, and it work out fine!

[cuban]

That said, this is only the second sample that I have seen able to crash the ICL build.
For me (with Haali splitter) both samples made ffdshow (ICL) crash... FFdshow (MSVC) seem to handle these streams. Even more: no matter which part of the original mp4 stream i cut out, it always made ffdshow crash. I guess it might be some flag in mp4 stream, not the stream (video) itself.


Does it crash if you use a splitter other than haali's? Because in the other case I believe with Nero splitter + ffdshow there was no crash.
Are You suggesting that Haali's splitter might be the problem? I dont have Nero splitter. Is there a way to install Nero splitter only - i dont want to instal the whole Nero package?

Are ICL compile and MSVC made from the same source? If it's true how can it be, that one of them is crashing and the other is not?

[celtic_druid]

No, I'm not suggesting that it is haali's fault.

Yep, exact same source.