x264 first pass aborting (was MeGUI 0.2.3.2060 - "0 Kbit/s" problem)

[xyloy]

I've got a very annoying problem. It's the first time when everything I try is helpless.


Let's resume the facts shortly:
Gordian Knot 0.35 setup2 installed (AviSynth 2.5.5 - DGIndex 1.4.3)
AviSynth plugins correctly installed
the AviSynth script(done by GordianKnot) works in Media Player Classic 6.4.8.7 and VirtualDubMod 1.5.10.2
XviD v1.1.0 finale is the YV12 decoder used
MeGUI v0.2.3.2060 (found here (http://www.chronocrossdev.com/apps/megui/)) is used, since the MeGUIx264 SVN v0.2.3.2050 given with x264 rev.410 (http://x264.nl/) suddenly needed some 'AviSynthWrapper.dll' (It worked perfectly before without it)
All of the others encodes(14 so far) from the same DVDs went fine(MP4 output) with exactly the same settings, the same AviSynth script and apps/files versions
The 3 pass seems to work fine(progress is shown, and a .stats.temp file of a few MB is done), but I have a 0 KB .MP4 file output
I have tried again with a demuxed video stream (demux done by DVD Decrypter), wich has given me .M2V files instead of .VOB files. It didn't change anything
I had the same problem with MeGUI v0.2.3.2055. I didn't try MeGUI v0.2.3.2058; since 0.2.3.2060 seems to be the newest version

Finally, here is the log:
Starting job job1-1 at 07:35:52
Job is a video job. encoder commandline:
--pass 1 --bitrate 1133 --stats "F:\****\Episode7\Episode7.stats" --bframes 3 --b-pyramid --subme 1 --analyse none --me dia --cqm "jvt" --progress --no-psnr --output NUL "F:\****\Episode7\Episode7.avs"
successfully started encoding
Processing ended at 08:18:11
----------------------------------------------------------------------------------------------------------

Log for job job1-1

avis [info]: 528x384 @ 25.00 fps (72202 frames)
x264 [info]: using cpu capabilities MMX MMXEXT SSE 3DNow!

----------------------------------------------------------------------------------------------------------
job job1-1 has been processed. This job is linked to the next job: job1-2
Starting job job1-2 at 08:18:11
Job is a video job. encoder commandline:
--pass 3 --bitrate 1133 --stats "F:\****\Episode7\Episode7.stats" --ref 5 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --bime --weightb --analyse all --8x8dct --cqm "jvt" --progress --no-psnr --output "F:\****\Episode7\Episode7.mp4" "F:\****\Episode7\Episode7.avs"
successfully started encoding
Processing ended at 08:18:16
----------------------------------------------------------------------------------------------------------

Log for job job1-2

desired video bitrate of this job: 1133 kbit/s - obtained video bitrate: 0 kbit/s
----------------------------------------------------------------------------------------------------------
job job1-2 has been processed. This job is linked to the next job: job1-3
Starting job job1-3 at 08:18:16
Job is a video job. encoder commandline:
--pass 3 --bitrate 1133 --stats "F:\****\Episode7\Episode7.stats" --ref 5 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --bime --weightb --analyse all --8x8dct --cqm "jvt" --progress --no-psnr --output "F:\****\Episode7\Episode7.mp4" "F:\****\Episode7\Episode7.avs"
successfully started encoding
Processing ended at 08:18:16
----------------------------------------------------------------------------------------------------------

Log for job job1-3

desired video bitrate of this job: 1133 kbit/s - obtained video bitrate: 0 kbit/s

And the script:
# Created with Gordian Knot
#
# http://gknot.doom9.org

# PLUGINS
LoadPlugin("E:\PROGRA~1\GORDIA~1\DGMPGDec\DGDecode.dll")
LoadPlugin("E:\PROGRA~1\GORDIA~1\AviSynthPlugins\decomb.dll")
#LoadPlugin("E:\PROGRA~1\GORDIA~1\AviSynthPlugins\KernelDeInt.dll")
LoadPlugin("E:\PROGRA~1\GORDIA~1\AviSynthPlugins\UnDot.dll")
#LoadPlugin("E:\PROGRA~1\GORDIA~1\AviSynthPlugins\dgbob.dll")
#LoadPlugin("E:\PROGRA~1\GORDIA~1\AviSynthPlugins\Convolution3d.dll")
#LoadPlugin("E:\PROGRA~1\GORDIA~1\AviSynthPlugins\FluxSmooth.dll")
#LoadPlugin("E:\PROGRA~1\GORDIA~1\AviSynthPlugins\TomsMoComp.dll")
#LoadPlugin("E:\PROGRA~1\GORDIA~1\AviSynthPlugins\VSFilter.dll")
#LoadPlugin("E:\PROGRA~1\GORDIA~1\AviSynthPlugins\SimpleResize.dll")

# SOURCE
mpeg2source("F:\****\Episode7\Episode7.d2v")

# TRIM
#trim(startframe,endframe)

# IVTC
#Telecide(order=1,guide=1).Decimate()
# or use
#IVTC(44,11,95)
#GreedyHMA(1,0,4,0,0,0,0,0)

# DEINTERLACING (1)
FieldDeinterlace()
#FieldDeinterlace(blend=false)
#TomsMoComp(1,5,1)

# DEINTERLACING (2)
#KernelDeInt(order=1,sharp=true)
# or maybe
#DGBob(order=1,mode=0)

# DEINTERLACING (3) - special requests
#GreedyHMA(1,0,0,0,0,0,0,0)
#Telecide()
#SeparateFields()

# CROPPING
crop(0,0,718,572)

# SUBTITLES
#VobSub("FileName")

# RESIZING
LanczosResize(528,384)

# DENOISING: choose one combination (or none)
Undot()

# 1) little noise
#Temporalsoften(2,3,3,mode=2,scenechange=6)
#mergechroma(blur(1.3))
#FluxSmoothST(5,7)

# 2) medium noise
#Temporalsoften(3,5,5,mode=2,scenechange=10)
#Convolution3d("moviehq")
#FluxSmoothST(7,7)

# 3) heavy noise
#Temporalsoften(4,8,8,mode=2,scenechange=10)
#Convolution3d("movielq")
#FluxSmoothST(10,15)

# BORDERS
#AddBorders(left,top,right,bottom)

# COMPRESSIBILITY CHECK
# !!!!Snip Size now has to be 20 for use in GKnot!
#SelectRangeEvery(400,20)

# FOOL CCEnc
#empty = BlankClip()
#AudioDub(last,empty)

I guess I could use VirtualDubMod and x264's VFW frontend for this time as a workaround since I don't use the missing options(except for Bidirectionnal ME and Chroma ME). But usually I do use them...

[dimzon]

Seems like problem is in x264,.exe
Can you run x264.exe manualy from command propmpt using such arguments:
x264.exe --pass 1 --bitrate 1133 --stats "F:\****\Episode7\Episode7.stats" --bframes 3 --b-pyramid --subme 1 --analyse none --me dia --cqm "jvt" --progress --no-psnr --output NUL "F:\****\Episode7\Episode7.avs"
x264.exe --pass 3 --bitrate 1133 --stats "F:\****\Episode7\Episode7.stats" --ref 5 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --bime --weightb --analyse all --8x8dct --cqm "jvt" --progress --no-psnr --output "F:\****\Episode7\Episode7.mp4" "F:\****\Episode7\Episode7.avs"
x264.exe --pass 3 --bitrate 1133 --stats "F:\****\Episode7\Episode7.stats" --ref 5 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --bime --weightb --analyse all --8x8dct --cqm "jvt" --progress --no-psnr --output "F:\****\Episode7\Episode7.mp4"

and post here what exatly occurs (including all encoder output)

[xyloy]

Thank you for your help.

I launched the first test(seems to work fine so far..).

I must go now, but the 3 results and logs will be posted tonight.

[Doom9]

the whole start/end time thing comes in really handy. We can see that the second pass takes but 5 seconds and the third pass takes less than a second.. I'll have to recheck the x264 encoder code.. I know for encraw I put in some workarounds that make sure all the output from the encoder is being read (I tested mostly abort cases).. I think we're missing a few crucial lines from the encoder here that would tell us what is going wrong.

[dimzon]

the whole start/end time thing comes in really handy. We can see that the second pass takes but 5 seconds and the third pass takes less than a second..
that's why I ask xyloy to run command manually and post all dignostics here :D

[xyloy]

I really don't know why, but the first test has stopped at 67.6%

Here is the entire log left by x264.exe in the CLI window:
E:\Program Files\x264>x264.exe --pass 1 --bitrate 1133 --stats "F:\***\Episode7\Episode7.stats" --bframes 3 --b-pyramid --subme 1 --analyse none --me dia --cqm "jvt" --progress --no-psnr --output NUL "F:\****\Episode7\Episode7.avs"

avis [info]: 528x384 @ 25.00 fps (72202 frames)
x264 [info]: using cpu capabilities MMX MMXEXT SSE 3DNow!
encoded frames: 48809/72202 (67.6%), 19.19 fps, eta 0:20:18

No one touched the computer during the time of encoding.
Besides, it should have ended three hours ago while I was away. This is very odd.

But it makes some sense with the previous log: the first pass didn't last 1 hour(as estimated) neither.

I relaunch it now... Haven't any hope it will success this time, but I don't think I can do the two others manual encoding tests if the 1st pass goes bonko like that.. O_o

The only output is a .stats.temp file, with the same size as earlier(4496 KB)
But I don't know if you have any x264 stats file reader. I've never found anyone..
5378
5379
5380
5381

[dimzon]

Seems like this is x264 OR AviSynth issue

[xyloy]

Sorry, but I don't understand very well.
If AviSynth was the cause, none of the previous/next episodes would have been encoded successfully, right ?

Or you mean the source .M2V files maybe corrupted ?
They are played with no errors on my computer, and the DVD(not dirty nor used at all, by the way) is played correctly.

the first pass test has stopped again at the same percentage(67.6%)

I think we're missing a few crucial lines from the encoder here that would tell us what is going wrong.
The first log shown previously, if that's what you are talking about, was complete.

[Doom9]

well.. source corruption is one reason for an abort, a problem in x264 is another. In any case, this doesn't look like a megui problem so I'm moving it to the avc forum. I'll still have a look at the encoder though. Could you run the second pass commandline and post the full results? I suspect there should be some error (I'm kinda expecting a message telling you that the number of frames of the stats file doesn't match the number of frames in the source).

[xyloy]

OK, testing...

Well, with or without renaming the .stats.temp file to .stats, I get the same error right after I've pressed [Enter] with the 2nd pass(no encoding at all):
avis [info]: 528x384 @ 25.00 fps (72202 frames)
x264 [error]: 2nd pass has more frames than 1st pass (72202 vs 48821)
x264_encoder_open failed

[Doom9]

just what I expected.. now I just need to make sure that megui reads all those lines and reports 'em back. You'd still be having the problem, but we'd have been slightly the wiser from the getgo (though a 5 second pass is a very strong indicator that encoding didn't really start).

[HOLiC]

hmm same problem here.. well, similar problem..

I wanted to report this after i try encoding with x264 using virtualdub..

The problem is that it just stop at 10% during encoding when i use megui.

At first, i thought the avs script might be wrong, but i just finished encoding

using xvid 1.1 final via virtualdub, so i don't think its the script.


*MeGUI-0.2.3.2058

*x264-rev.410

*MP4 output

*d2v is made with 1.4.5 version/also tried 1.2.0

*avisynth 2.56 latest build (not beta)/ also tried 2.56 alpha


log for automated 2 pass:
[1st pass is fine, but during 2 pass, it just stops at like 8%..for constant quantizer, it stop as well]
Starting job job1-1 at 오후 5:17:58
Job is a video job. encoder commandline:
--pass 1 --bitrate 2141 --stats "C:\DVDrip\Scrapped03\test.stats" --bframes 3 --b-pyramid --filter 1,1 --subme 1 --analyse none --qcomp 0.7 --scenecut 60 --me dia --threads 2 --progress --no-psnr --output NUL "C:\DVDrip\Scrapped03\test.avs"
successfully started encoding
Processing ended at 오후 11:50:30
----------------------------------------------------------------------------------------------------------

Log for job job1-1

avis [info]: 704x528 @ 23.98 fps (32836 frames)
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2
x264 [info]: slice I:287 Avg QP:12.03 size: 67769
x264 [info]: slice P:13824 Avg QP:14.08 size: 18623
x264 [info]: slice B:18725 Avg QP:15.65 size: 5064
x264 [info]: mb I I16..4: 18.0% 0.0% 82.0%
x264 [info]: mb P I16..4: 9.8% 0.0% 0.0% P16..4: 75.7% 0.0% 0.0% 0.0% 0.0% skip:14.5%
x264 [info]: mb B I16..4: 0.8% 0.0% 0.0% B16..8: 57.3% 0.0% 0.0% direct:15.4% skip:26.5%
x264 [info]: final ratefactor: 16.01
x264 [info]: kb/s:2171.3

Actual bitrate after encoding without container overhead: 2172.35

----------------------------------------------------------------------------------------------------------
job job1-1 has been processed. This job is linked to the next job: job1-2
Starting job job1-2 at 오후 11:50:30
Job is a video job. encoder commandline:
--pass 2 --bitrate 2141 --stats "C:\DVDrip\Scrapped03\test.stats" --ref 16 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --filter 1,1 --subme 6 --analyse p8x8,b8x8,i4x4,p4x4 --qcomp 0.7 --scenecut 60 --me umh --threads 2 --progress --no-psnr --output "C:\DVDrip\Scrapped03\test.mp4" "C:\DVDrip\Scrapped03\test.avs"
successfully started encoding
Processing ended at 오전 12:10:50
----------------------------------------------------------------------------------------------------------

Log for job job1-2

avis [info]: 704x528 @ 23.98 fps (32836 frames)
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2
mp4 [info]: initial delay 250 (scale 2997)
desired video bitrate of this job: 2141 kbit/s - obtained video bitrate: 79.9355809438421 kbit/s


and virtualdub (using x264) is already 30% encoding,
but i'll see if it finish the job or not.. i'll also try to use older version of x264 and megui after this.. thanks!
[sorry, i couldn't tell what is the problem by just looking at the logs..]

[xyloy]

just what I expected.. now I just need to make sure that megui reads all those lines and reports 'em back. You'd still be having the problem, but we'd have been slightly the wiser from the getgo (though a 5 second pass is a very strong indicator that encoding didn't really start).
If you need some more infos, let me know.

I think I'll have to wait for the next version of x264. I'm not even sure if the old VFW encoder would not crash also, and I don't like very much the video quality it gives. The CLI is much better.

Besides, I'm quite used to MeGUI now...

[Doom9]

well.. I debugged such an abort and I'm afraid megui cannot show you any more.. since it's a separate process that I first have to start and then redirect its stdout and stderr, either x264 is so incredibly fast to report the error that it falls between the start of the process and the two lines immediately there after where the redirection is done, or it just acts differently when its outputs are being redirected.. no matter what I tried, I seem to be unable to get any output whatsoever in this abort case :(

@HOLiC: please run your second pass commandline from the commandline and see if you get more output from the encoder.

[xyloy]

I've just tried with VirtualDubMod 1.5.10.2 and the VFW of x264. It crashed at the same state(source corruption ??).

I'm trying with XviD's 1.1.0 VFW. If it does not crash(nor I get a crappy output at frame number 48821), it would mean there is no source corruption.

Next test after that: x264 rev.411 with MeGUI X264-SVN 0.2.3.2061 (given with x264 rev.411. It does not yell at me anymore about AviSynthWrapper.dll - but still does if I delete it - when I open the .AVS file. Great! :D ).

(AviSynthWrapper.dll is still only in the MeGUIs versions available here (http://www.chronocrossdev.com/apps/megui/), not in the x264 rev.411 installer)

[Doom9]

AviSynthWrapper.dll is still only in the MeGUIs versions available here, not in the x264 rev.411 installerSomebody needs to tell bob0r about that.. the megui developers have no control over how megui is being bundled.

[xyloy]

With XviD 1.1.0 finale and VDubMod, everything went fine.
No problem at all with the .avi output, except the bad video quality(but that's normal with 0.224 BPF and only 1134 Kbit/s).

Using x264 rev.411 with MeGUI SVN x264 0.2.3.2061 didn't change anything.

Should I try an older version of x264 ?

Are the x264's developers aware of this problem ?
(Well, I mean now I don't know what else could cause this, unless if it's a bug in x264.exe)

[Doom9]

Are the x264's developers aware of this problem ?Since it's in the proper forum it'll be picked up. It definitely looks like a problem in x264.

[HOLiC]

but for me, using virtualdubmod 1.5.10.2 with x264 rev.410, I had no problem.
(single quantizer and 2nd pass encoding, both worked fine)

And i already ran 2nd pass commandline from cmd before and all i got
was something like this:

avis [info]: 704x528 @ 23.98 fps (32836 frames)
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2
encoded frames: 889/32835 (2.8%)...

However, like i said previously in my post, i'm having same problem with the
constant quantizer... So using exactly same avs and same encoding setting but slightly different builds, this is what happens:


*x264-rev.409

*megui-x264 v0.2.3.2035 (from the 409 build)

*MP4 output

*no audio

*d2v is made with 1.4.5 version

*avisynth 2.56 latest build (not beta)


log from megui:
(started 3:38:24 ended 4:20:15)
Starting job job1 at 오후 3:38:24
Job is a video job. encoder commandline:
--crf 15 --ref 16 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --filter 1,1 --subme 6 --analyse all --8x8dct --qcomp 0.7 --scenecut 60 --me umh --threads 2 --progress --no-psnr --output "C:\DVDrip\otome\otome01.mp4" "C:\DVDrip\otome\otome01.avs"
successfully started encoding
Processing ended at 오후 4:20:15
----------------------------------------------------------------------------------------------------------

Log for job job1

avis [info]: 704x528 @ 23.98 fps (34430 frames)
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2
mp4 [info]: initial delay 250 (scale 2997)
desired video bitrate of this job: 15 kbit/s - obtained video bitrate: 86.3386860128957 kbit/s
----------------------------------------------------------------------------------------------------------

log from cmd:

C:\Program Files\x264>x264.exe --crf 15 --ref 16 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --filter 1,1 --subme 6 --analyse all --8x8dct --qcomp 0.7 --scenecut 60 --me umh --threads 2 --progress --no-psnr --output "C:\DVDrip\otome\otome01.mp" "C:\DVDrip\otome\otome01.avs"
avis [info]: 704x528 @ 23.98 fps (34430 frames)
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2
mp4 [info]: initial delay 250 (scale 2997)
encoded frames: 2858/34430 (8.3%), 1.07 fps, eta 8:10:23

I guess, i'll have to wait for the next build of x264...
but if it was a bug from x264.exe, how did vdub finished my encoding?
perhaps because it didn't use th options that are included in new cli?
well, i'll try the older versions now..

[xyloy]

@HOLiC:
x264.exe is the CLI used by MeGUI
x264.dll is the VFW used by VDubMod (no longer updated: some useful options are missing, like Bidirectionnal ME, Chroma ME, PSNR, Trellis and some others)


My problem has occured again with the #17th episode... Older versions do not suit me. Wating for the next version. :(

[dimzon]

Wating for the next version. :(
I believe it's really hard to find what's wrong in this case (error is based on source material)
Contact to x264 developers directly, ask them to send you special build with extended logging support to investigate where and why this error occurs.
Another proposal - maybe something wrong when x264 working with AVISynth? Try another AviSynth versions for it...

[xyloy]

Contact to x264 developers directly, ask them to send you special build with extended logging support to investigate where and why this error occurs.
If you think that's appropriate, I will right now.

[xyloy]

I've contacted akupenguin by PM.

Hope it will be not too much difficult to track this bug down..

[xyloy]

Everyting seems to work fine again with x264 rev.413.

First pass OK. ;-)

I'm sure the two next passes will have no problem at all.

Thanks!

[xyloy]

Same crash happened at 6% of the first pass(same commandline) with x264 rev.417, 418 and 420 on another episode(saison 2, this time - but it is again the 7th! O_o)...

I've contaced again akupenguin by PM.

I believed it was over. I was just lucky.. :(

[Quarkboy]

Just like to say that I am experiencing the same type of problem with newer versions of x264 (later than 408, anyway). First pass stops with no warning at various places (66%, 20%, etc etc) but second pass works fine (well, it encodes fine, but using the wrong or a partial stats file so quality isn't any good).

I've had to downgrade to 408 till someone figures something out. I'd post my avs script but because I prefilter to a huffy the script is simply and AviSource a Wavsource, and Audiodub and a bunch of textsubs. I've tried doing a full quality 1st pass instead of the "turbo" options that megui does (ref = 1 search dia, analyze none), but to no avail. It really seems to be a -pass 1 with -b xxx problem. (Note, I don't use me-gui, just batch files )

[xyloy]

Akupenguin managed to fix it in the new revision build of x264.

Welcome to x264 rev.427 (http://x264nl/) :)

r427 | pengvado | 2006-02-10 02:16:40 +0100 (Fri, 10 Feb 2006) | 3 lines

lowres can reuse the normal mv cost table

;)