running x264.exe without running avisynth in it's own thread

[pyrates]

I've been trying to encode this movie for about a week now. The source is 1080i and I'm converting it over to 1080p. Now I found that it's been crashing alot it seems.

I've seemed to nail it down to the option, Run AviSynth in its own thread. It always does this when I'm trying to do something else on my computer. The command line for it is --thread-input. Unfortunately it's not an option in the megui program. I've also noticed that it seems to speed things up if I don't enable it.

I'm running build 558 and are using the MT plugin version of avisynth 2.5.6a. Which greatly helps to speed up encoding I find.

I never came across this when encoding to XViD before. It probably doesn't have that option anyways, which might explain why it didn't crash. I also know x264 is alpha software. So it could be a bug I just found too.

Has anyone else come across this?

[check]

you can add the option in megui in x264->config->zones, or you can copy the commandline out of megui and run it manually from a batch file

[Sharktooth]

MeGUI automatically enables --thread-input if threads are set (or autodetected) to 2 or more.
Crashing means your system is not stable, so before doing any other thing, test your hardware for possible problems.

[pyrates]

Originally Posted by Sharktooth

MeGUI automatically enables --thread-input if threads are set (or autodetected) to 2 or more.
Crashing means your system is not stable, so before doing any other thing, test your hardware for possible problems.

Did a test for my hardware, all passed pretty much. Ram test was fine. And my system is pretty stable most of the time. But when I said it crashed, I meant that x264.exe crashed. Everything else was fine. So if this doesn't crash, I'll know where the problem lies.

[bob0r]

Infact Pengvado told me: When you run --threads 2 --thread-input is also used by x264.

--threads 1: 1 thread doing encoding and avs
--threads 1 --thread-input: 1st thread doing encoding and 2nd thread doing avs
--threads 2: 1st and 2nd thread doing encoding, 3rd thread doing avs
--threads 2 --thread-input: same
--threads 3: 3 threads doing encoding, 4th thread doing avs

[pyrates]

Originally Posted by bob0r

Infact Pengvado told me: When you run --threads 2 --thread-input is also used by x264.

--threads 1: 1 thread doing encoding and avs
--threads 1 --thread-input: 1st thread doing encoding and 2nd thread doing avs
--threads 2: 1st and 2nd thread doing encoding, 3rd thread doing avs
--threads 2 --thread-input: same
--threads 3: 3 threads doing encoding, 4th thread doing avs

Well from what I can tell, if the --thread-input and --threads 2 is in their, then x264.exe crashes if it can't get the frame from avisynth. But without --thread-input, if avisynth can't get the frame, x264.exe just waits. Also the encode is slightly slower from what I can tell. But it doesn't crash and that's all I care about. http://forum.doom9.org/images/smilies/biggrin.gif


Oh and how many threads can I put? I have a dual core cpu. Am I only supposed to put 2 then? I'm assuming so.

[MeteorRain]

--threads 1 --thread-input: 1st thread doing encoding and 2nd thread doing avs

[foxyshadis]

Sounds like an undesirable interaction between MT avisynth and MT x264, more than anything. Odd, not something I've experienced, but it might have something to do with certain filters' bad behavior under multithreading.

[pyrates]

Originally Posted by foxyshadis

Sounds like an undesirable interaction between MT avisynth and MT x264, more than anything. Odd, not something I've experienced, but it might have something to do with certain filters' bad behavior under multithreading.

Well here is the script I'm using:

SetMTMode(2,0)
# Modify the path so that it finds your decomb dll file #
LoadPlugin("G:\DeComb\decomb521.dll")

# Modify the path so that it finds your MPEG2DEC dll file #
LoadPlugin("G:\DGMPGDec\DGDecode.dll")


LoadPlugin("G:\MKVMagic\exe\filter\Convolution3D.dll")
LoadPlugin("G:\MKVMagic\exe\filter\deen.dll")
LoadPlugin("G:\MKVMagic\exe\filter\MSharpen.dll")
LoadPlugin("G:\MKVMagic\exe\filter\tdeint.dll")
LoadPlugin("G:\MKVMagic\exe\filter\UnFilter.dll")
LoadPlugin("G:\MKVMagic\exe\filter\undot.dll")

import("G:\MvBob\mvbob.avs")
import("G:\AVS Scripts\convert60ito24p.avsi")
Import("G:\MKVMagic\exe\filter\HybridFuPP.avsi")

# Modify the path so that it finds the d2v file you created using dvd2avi #
MPEG2Source("N:\1080i\Amityville.Horror\movie\movie.d2v")
#audio=wavSource("E:\24.1.wav")
#audiodub(video,audio)
#Crop(left, top, -right, -bottom)
Crop(0, 0, -0, -0)

# Assume Top Field
#AssumeTFF().SeparateFields()
Telecide(order=1)
# Assume Bottom Field
#AssumeBFF().SeparateFields()
#Telecide(order=0)

# Reduce down to 23.976 fps
Decimate()

Undot()
Convolution3d(preset="movieHQ")

HybridFuPP(preset="High")

I use hybridfupp because it is the best at cleaning up the video and has a builtin resizer to resize the parts of each frame with the proper resizing method. Just is very cpu intensive. I wish it wrote out some kind of stats file though the first time it goes through. Because right now it has to calculate what settings to use for each and every frame each encoding pass. My encoding times tripled with enabling this. But still it's worth it in the end.

[Manao]

Before anything else, simplify the avs script to a bare minimum ( i.e, mpeg2source only ), and try again.

MT isn't the most stable thing. HybridFuPP relies on MaskTools 1.5 which aren't that stable eithers, and on MVTools which I definitely wouldn't call stable, especially when used with MT.

BTW, answering on the question you asked on the HybridFuPP thread, you can save the script with a lossless encoding, that way the script is only processed once.

[pyrates]

Originally Posted by Manao

Before anything else, simplify the avs script to a bare minimum ( i.e, mpeg2source only ), and try again.

MT isn't the most stable thing. HybridFuPP relies on MaskTools 1.5 which aren't that stable eithers, and on MVTools which I definitely wouldn't call stable, especially when used with MT.

BTW, answering on the question you asked on the HybridFuPP thread, you can save the script with a lossless encoding, that way the script is only processed once.

Wouldn't making a lossless encoding, even temporarily make a huge file? Not sure if I have the disk space for that. Got around 50 GB or so free to do this encoding.

[Manao]

Yes, it would ( 1 MB / frame, i'd say, for 1080p, so 1 hour @ 25 fps would give 90 GB ). But the only other thing that can be speeded up by saving a stat file between both passes in HybrippFupp would be the result of MVAnalysis, and that definitely not accounts for much in the slowliness of HybridFupp

[pyrates]

Originally Posted by Manao

Yes, it would ( 1 MB / frame, i'd say, for 1080p, so 1 hour @ 25 fps would give 90 GB ). But the only other thing that can be speeded up by saving a stat file between both passes in HybrippFupp would be the result of MVAnalysis, and that definitely not accounts for much in the slowliness of HybridFupp

Well every little bit helps. How would I save a stat file that would show the result of MVAnalysis and get the avisynth script to read that back on the 2nd pass? Or if you could pass me to a website or another forum topic that would show me how to do this.

[Manao]

You can't. I tried to make the MVTools output a stat file if asked too, but I soon realised the file would be quite big ( ~700 MB ), so I dropped the idea. Anyway, as I said, it doesn't help that much ( you'd win something like 2-3% of the overall encoding time )

@mods : sorry to have somehow hijacked the thread, I should have answered the HybridFuPP part on the dedicated thread.

[pyrates]

Originally Posted by Manao

You can't. I tried to make the MVTools output a stat file if asked too, but I soon realised the file would be quite big ( ~700 MB ), so I dropped the idea. Anyway, as I said, it doesn't help that much ( you'd win something like 2-3% of the overall encoding time )

@mods : sorry to have somehow hijacked the thread, I should have answered the HybridFuPP part on the dedicated thread.

700 MB isn't that bad considering I'm doing 1080p encoding here. So how did you do it? The reason why that hybridfupp is quite cpu intensive is because of all the calculating it has to do. If it knew what to do with each frame right away, it would be faster.

[Manao]

I contributed to the MVTools and the MaskTools, so I added the needed code. I deemed the results useless, so I dropped it. Believe me when I say you greatly overestimate the gain you'd have by not computing twice the motion vectors.

[pyrates]

Originally Posted by Manao

I contributed to the MVTools and the MaskTools, so I added the needed code. I deemed the results useless, so I dropped it. Believe me when I say you greatly overestimate the gain you'd have by not computing twice the motion vectors.

Ah well maybe I'll try that lossless idea you had their. See how much that reduces my time by and how much actual space it takes up since cimpression is used though. Just no detail loss occurs.

Oh and 2-3% time saved when an encoding takes 27 hours does help. Thanks for all your help though.

Oh and the results are in. It didn't crash at all for once for me. Putting it in an avi container now. So now I know to not use the option --thread-input from now on. The next encode shouldn't take a week now.