x264 bug: slower encoding with mp4 output on avi input with strange framerates

[sasam]

I can encode smaller videos ok.
When I encode videos longer then ~30min x264 starts at 15fps and somewhere in the middle it starts slowing down so when it finishes it is only 2fps and even then I cant play the .mp4 video that it produces.

I encode with x264 cli and megui...

input.avs
A=DirectShowSource("!CMP.avi")
A=ConvertToYV12(A)
A

This is what I use with x264.exe
encode.bat
start /low /b /w x264.exe --progress --pass 1 --bitrate 300 --subme 6 --ref 5 --threads 1 --filter 0:0 --keyint 250 --min-keyint 25 --scenecut 40 --qpmin 10 --qpmax 51 --qpstep 4 --direct temporal --me hex --merange 16 --sar 1:1 --bframes 3 --weightb --b-bias 0 --ipratio 1.40 --pbratio 1.30 --qcomp 0.60 --analyse all --8x8dct -o "x264.mp4" "input.avs"

start /low /b /w x264.exe --progress --pass 3 --bitrate 300 --subme 6 --ref 5 --threads 1 --filter 0:0 --keyint 250 --min-keyint 25 --scenecut 40 --qpmin 10 --qpmax 51 --qpstep 4 --direct temporal --me hex --merange 16 --sar 1:1 --bframes 3 --weightb --b-bias 0 --ipratio 1.40 --pbratio 1.30 --qcomp 0.60 --analyse all --8x8dct -o "x264.mp4" "input.avs"

With megui I use slow profile.

I have Avisynth_256 and newest ffdshow. I also tried older avisynth and ffdshow.
This is not something new. I could never encode large files with x264.
I have this problem on my 3Ghz P4 512Ram and a laptop P M 1.7Ghz 512Ram.
What is going on?

[DryFire]

Have you tried raw or .mkv outputs?

I wonder if .mp4 output is still borked.

[sasam]

Have you tried raw or .mkv outputs?

I wonder if .mp4 output is still borked.


No but I will

[sasam]

It works with .mkv.
encoding long videos in mp4 never worked for me but encoding smaller always works.
Why doesn't it work with mp4?

[Sergejack]

It works with .mkv.
encoding long videos in mp4 never worked for me but encoding smaller always works.
Why doesn't it work with mp4?

Are you using a recent version of X264 ?

[puffpio]

are you overclocked?

[sasam]

I am not overclocked and I am using rev 385.

[bond]

I am not overclocked and I am using rev 385.which compile? try sharktooths

another possible issue:

whats the framerate of the source? try this (http://forum.doom9.org/showthread.php?t=100443)

[bob0r]

If you did not use Sharktooth's version, and used the x264.nl version:

Please try this version: http://files.x264.nl/x264-385-test-install.exe

(My gpac refused to "cvs up", so my gpac lib was out to date)

Please test and let us know!

[sasam]

If you did not use Sharktooth's version, and used the x264.nl version:

Please try this version: http://files.x264.nl/x264-385-test-install.exe

(My gpac refused to "cvs up", so my gpac lib was out to date)

Please test and let us know!
This one works. I used Sharktooth's version, and that didnt work.

[bond]

This one works. I used Sharktooth's version, and that didnt work.interesting, can you try again with sharktooth's latest compile from just a few hours ago too?

and may i ask what framerate the avi has?

[sasam]

sorry i was wrong.
fps is 23.976

Both http://files.x264.nl/x264-385-test-install.exe and Sharktooth's version 385C don't slow down during the second pass but produce 1kb mp4 file???

[bond]

Both http://files.x264.nl/x264-385-test-install.exe and Sharktooth's version 385C don't slow down during the second pass but produce 1kb mp4 file???strange, i tried a small sample with sharktooths and it produced a normal .mp4 file

are you sure your .avs is correct?

[sasam]

It is strange. I use the same script to encode a small file and it works with 384 and doesnt with 385C???

[bond]

It is strange. I use the same script to encode a small file and it works with 384 and doesnt with 385C???no problem here with your exact cmdl

try another source

do you see the video when opening the avs in virtualdub?

[quake74]

I am experiencing the slowdowns during the second pass too. I am encoding a sticom episode and about a third through it slows down a lot. The episode is about 24 mins long, and it's taking (using the psp profile) more than 4 hours. I am attaching two screenshots of the megui window taken on different days with different x264 builds. I will try too different builds and report back.

[bond]

well for sasam, 385C fixed the slowdown

[sasam]

fixed the slowdown but produced 1kb file, but 385F works with big and small files...
I dont know what Sharktooth does to his compiles but rev384 x264.exe was ~800KB, 385C ~700KB and 385F is again ~800KB...

[Sharktooth]

it was caused by GPAC...

[bond]

it was caused by GPAC...in what way? afaik there werent any code changes in gpac?

[Sharktooth]

well, updating GPAC fixed it...

[bond]

well, updating GPAC fixed it...maybe you simply mixed up some sources ;)

[Sharktooth]

never... i always do a fresh checkout:P

[bond]

jaja, still there was no update in the gpac sources between your builds ;)

[Sharktooth]

r384....

[quake74]

I am puzzled, I am trying now the 387 build from sharktooth and it looks like it takes 4.5 hours ("looks like" because I aborted the encoding after 3 hours) to encode the 30min clip I have (about 3.5FPS). I tried with Nero recode (using 2 ref and 3 bframes, but I don't know how to set mixed ref frames or adaptive bframes) and it took just about 1.5hr. Is this normal on a Pentium M 1.83Ghz?

[nm]

And your x264 settings were...?

[quake74]

Sorry, I always forget that because I am only using these settings these days: the PSP profile from Sharktooth ( 2 pass, 2 ref, mixed, 3 bframes, adapt, no pyr).

[nm]

For those settings and a PSP resolution, 3.5 fps sounds a bit too low. Was it faster with some previous x264 build and are you sure there were no other heavy processes taking the CPU time? Comparing to Nero is not very helpful, but comparing to another x264 build would be.

[Shinjite]

I am currently using x264 r387 too, with HQ-Insane, a 1hr 41 minute movie, 1st pass with turbo (6.04fps) take like 7 hours, then 2nd pass now, at 68%(1.02fps), left 12 hours 4x mins more to go (68% took up like more than 12 hours already), then still has 3rd pass to finish. Uber super slow with Insane settings even with my P4 2.4 OCed to 3.24Ghz. But the output is superb so far ;)

[Clown shoes]

(The title has been changed to reflect the subject of the thread) formerly (Big problems with MeGUI and PSP compatibility)


Hi guys,

I am having serious problems making avc MP4s for my PSP. I am producing a .avs file to contain my 23.976 xvid (duration 00.03.07) with the following script;

AviSource("E:\Transporter\Test\Test23.976.avi")
Lanczos4Resize(368,144)
Addborders(0,32,0,32)
ChangeFPS(29.97)

I am then processing the .avs in MeGUI using Sharktooth's psp profile. If I select raw output the log seems fine and displays as follows;

avis [info]: 368x208 @ 29.97 fps (5606 frames)
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2
x264 [info]: slice I:23 Avg QP:11.00 size: 15478
x264 [info]: slice P:2467 Avg QP:11.86 size: 8534
x264 [info]: slice B:3116 Avg QP:13.95 size: 2945
x264 [info]: mb I I16..4: 45.4% 0.0% 54.6%
x264 [info]: mb P I16..4: 3.2% 0.0% 10.1% P16..4: 20.3% 12.7% 13.6% 2.7% 2.7% skip:34.8%
x264 [info]: mb B I16..4: 0.3% 0.0% 1.3% B16..8: 31.2% 8.0% 12.7% direct: 2.4% skip:44.1%
x264 [info]: ref P 89.7% 10.3%
x264 [info]: ref B 92.7% 7.3%
x264 [info]: kb/s:1308.0

Actual bitrate after encoding without container overhead: 1308.08
desired video bitrate of this job: 1308 kbit/s - obtained video bitrate: 1308.08250104717 kbit/s

However when I mux the raw .264 file with my 48khz aac .mp4 file produced in belight with YAMB 1.3.3. the resulting log reads;

Adjusting AVC SizeLength to 16 bits
AVC-H264 import - frame size 368 x 208 at 25.000 FPS

IsoMedia import - track ID 1 - Audio (SR 48000 - 2 channels)

Saving to E:\Transporter\Test\Test_1.mp4: 0.500 secs Interleaving


Somehow the file is detected as 25fps!! I tested the file on my PSP just to see if it is being incorrectly detected, but lo and behold it is badly out of sinc.

I then tried producing a .mp4 instead of a raw file in MeGUI but the file that is produced cannot be played by Quick Time as it comes up with an error;

Error -2014: the movie contains an incorrect duration

I tried muxing it in YAMB and got this log;

IsoMedia import - track ID 1 - Video (size 368 x 208)

IsoMedia import - track ID 2 - Audio (SR 48000 - 2 channels)

IsoMedia import - track ID 1 - Audio (SR 48000 - 2 channels)

Saving to E:\Transporter II\Test\Test_1_2.mp4: 0.500 secs Interleaving

It seems to have 2 tracks of audio as well. The file is playable in Quick Time, but guess what?

Movie FPS: 25.00
Duration: 00.03.44.07

I would be hugely grateful if someone could point me in the right direction here. I have spent my entire weekend trying to sort this out and if it takes me much longer to work out, I might well end up breaking my PSP into very small pieces.

Regards

ClownShoes

[quake74]

As far as I know, mp4box (of which yamb is just a gui) muxes by standard at 25 fps. If you want it at 29.97 you have to tell it by adding "-fps 29.97" to the command line (there should be an option somewhere in yamb too). I followed your same route many times (but not using yamb, just plain mp4box) and all the files play on my psp (although I never used such a high bitrate, I usually use 256kbps).

[Doom9]

why not use the muxer in megui? it allows you to set a framerate when muxing, and if you use auto-mode, it's automatically set to the source fps.. so the proper 29.97 you want in this case.
And it isn't done with that.. Sony uses a MP4 perversion so you need to run the atomchanger after creating the MP4 or the PSP will never play your files.

[Clown shoes]

Thanks for the responses guys. Interestingly enough I just tried loading the file I produced using MeGUI outputing .MP4 instead of a raw file. Although it is unplayable in Quick Time and atomchanger detects it as -64.97fps !! It amazingly works on my PSP and is in sync as well!. It did seem a little stuttery and I wonder if that is the normal look of a conversion from 23.976fps on a PSP, or is it my conversion problems?

(slightly OT, I read in another post on this site yesterday that PSP software version 2.60 is now able to play back AVCs at 23.976fps. Can anyone confirm this?

Ok update time;

I installed the full version of MeGUI with the intention of using the muxer functions, however it imediately closes after it opens. The status window says done, but the log reads;

Next job job3 is a mux job. mp4box commandline:
"E:\Downloads\mp4muxer\needed\MP4Box.exe" -add "E:\Transporter \Test\Newtest.mp4" -add "E:\Transporter\Test\Test23.976.mp4":lang=eng -new "E:\Transporter\Test\mux.mp4"
successfully set up muxer and callbacks for job job3
-----------------------------------------------------------------------
Log for job job3

Option -new unknown. Please check usage
an exception ocurred when trying to read from stdout: Could not find file "E:\Transporter\Test\mux.mp4".

Another thing I noticed is if I encode the video to a MP4 file, the frame rate drop down box in the muxer is greyed out. It is only selectable if I use a raw file. The muxer closes down in both cases. Is this intentional?

[Clown shoes]

Damn it, I'm now getting the progresively slower encoding with MeGUI problem. This only seems to occur when I encode as .mp4 and not raw .264 files. The problem is it's only the .mp4 files that I can currently get working. When I try to mux the raw files they still come out as 25fps!

My encoding started at about 40fps and is now at 6fps with only 10% of the 90min movie completed. It currently says 7hours remaining, but the last time I tried doing this It slowed so much that I gave up when it said 16hours still to go with only 19% completed.

I'm really not having much luck here, but it would be a real shame if I have to go back to using pspvideo9!

Anyone have any thoughts on possible solutions?

[Shinjite]

What settings did you use that slows down your encoding??
If it is extreme kind of settings, no doubt it will be slow at the cost of quality

[Clown shoes]

No, it seems that the problem only occurs when encoding to large .mp4 files. Raw .264 is fine.

[Doom9]

When I try to mux the raw files they still come out as 25fps!How do you mux? Megui has an mp4 muxer, press Control-4 in the main menu to access it (or it's also accessible via Tools - Muxer - MP4 muxer).. in that muxer you can set the fps that the mp4 is going to signal.
Check your CPU usage while encoding.. x264 should be taking about the same amount of CPU time over time.. if the cpu usage goes back, check if another application is eating it up.

[bond]

mp4box only needs -fps when muxing from raw. when muxing from .mp4 it should use the framerate of the input.mp4

so why didnt you encode directly to .mp4 but to raw in megui?

[Doom9]

so why didnt you encode directly to .mp4 but to raw in megui?Probably because the auto-mode using x264.exe directly spits out an mp4 and just adds audio to that.. it's more efficient than going raw and then to mp4.

[bond]

Probably because the auto-mode using x264.exe directly spits out an mp4 and just adds audio to that.. it's more efficient than going raw and then to mp4.right, but the guy did output to raw

[Doom9]

I'm always getting the feeling people use yamb for raw muxing.. it's a good tool, but the megui muxer forces you to enter an fps for raw streams.. it doesn't let you create a job otherwise and that's the only way to make sure..

[Clown shoes]

How do you mux? Megui has an mp4 muxer, press Control-4 in the main menu to access it (or it's also accessible via Tools - Muxer - MP4 muxer).. in that muxer you can set the fps that the mp4 is going to signal.


As I mentioned in my post above;

I installed the full version of MeGUI with the intention of using the muxer functions, however it imediately closes after it opens. The status window says done, but the log reads;

Next job job3 is a mux job. mp4box commandline:
"E:\Downloads\mp4muxer\needed\MP4Box.exe" -add "E:\Transporter \Test\Newtest.mp4" -add "E:\Transporter\Test\Test23.976.mp4":lang=eng -new "E:\Transporter\Test\mux.mp4"
successfully set up muxer and callbacks for job job3
-----------------------------------------------------------------------
Log for job job3

Option -new unknown. Please check usage
an exception ocurred when trying to read from stdout: Could not find file "E:\Transporter\Test\mux.mp4".


mp4box only needs -fps when muxing from raw. when muxing from .mp4 it should use the framerate of the input.mp4

so why didnt you encode directly to .mp4 but to raw in megui?


again as mentioned in my previous post;

I'm now getting the progresively slower encoding with MeGUI problem. This only seems to occur when I encode as .mp4 and not raw .264 files.


So in summary, MeGui is having problems for me when encoding .mp4 files. Excess of 24hours to encode 90mins of footage on the second pass, compared to only 3hours to encode the raw .264 file! MeGUI is also failing to mux anything I provide it. (Don't get me wrong, I don't think MeGUI is at fault. I know this must be some kind of setting issue or conflict on my part, I just can't work out where.) The work around I have found is to produce a raw .264 and .mp4 audio file in MeGUI and then mux them with psp.bat. It's just a shame I can't get it all working with MeGUI, as that would really simplify things.

Regards

Clown Shoes

[Doom9]

actually, I have to check the latest mp4box to see.. mp4box not recognizing the -new flag concerns me.. perhaps something has changed in mp4box.. I hate it when they change commandline options.

I'd still like to hear about the cpu usage of the various processes when things get slow. And what happens if you use the commandline shown in the log (the one for x264.exe) and run it from a command prompt (start - run, type "cmd", press enter, then paste the commandline and press enter again)

[bond]

Damn it, I'm now getting the progresively slower encoding with MeGUI problem.according to this thread (http://forum.doom9.org/showthread.php?t=103829&page=1&pp=20) this has been fixed with the latest compiles, make sure to upgrade

[Clown shoes]

That's strange. I think I downloaded and installed the latest build yesterday.

[Sharktooth]

uhm... post your commandline and the exact versions of MeGUI and x264 and mp4box.

[Clown shoes]

MeGUI is 0.2.3.1023b

Mp4box is GPAC version 0.4.1 -DEV

Which commandlines are you after Sharktooth?

[Sharktooth]

the x264 commandline used for encoding

[Clown shoes]

Okay, here's the command line for the first pass;

Next job job1-1 is a video job. encoder commandline:
"E:\Downloads\x264-Lite_r387\x264.exe" --pass 1 --bitrate 1067 --stats "I:\Transporter\Test\video.stats" --bframes 3 --subme 1 --analyse none --me dia --zones 145620,157534,q=40 --progress --no-psnr --output NUL "I:\Transporter\Test\video.avs"
successfully set up video encoder and callbacks for job job1-1


I shall post the second pass in about half an hour when it begins again.