X264 r1353+ Breaks Avisynth Multithreading

[CarlEdman]

The avisynth improvements of x264 r1353 seem to have broken multi-threading of avisynth MT. The scripts still work, but multi-threading seems to be quietly gone. This can be a *massive* loss for those of us using demanding avisynth scripts.

For example, with the degraining avisynth script I have been using for years (following below), x264 encoding goes from typically 10 fps to 3 fps. Is there anything that can be done about this?

#Opts passed automatically
#x264opts=--tune film --crf 19 --preset slower --sar 32:27 --profile high --level 3.1
SetMTMode(2,0)
DGDecode_mpeg2source("VTS_01_PGC_01_1.d2v", info=3, idct=3, cpu=6)
ColorMatrix(d2v="VTS_01_PGC_01_1.d2v", interlaced=true)
#Deinterlace 0.808000 FILM
tfm().tdecimate(hybrid=1)
#Crop
autocrop(threshold=30,wMultOf=4, hMultOf=2,samples=51, mode=3) # crop(left,top, -right, -bottom)
#Denoise
Interleaved2Planar()
super = MSuper(planar=true)
bv1 = MAnalyse(super, isb = true, delta = 1, overlap=4)
fv1 = MAnalyse(super, isb = false, delta = 1, overlap=4)
bv2 = MAnalyse(super, isb = true, delta = 2, overlap=4)
fv2 = MAnalyse(super, isb = false, delta = 2, overlap=4)
bv3 = MAnalyse(super, isb = true, delta = 3, overlap=4)
fv3 = MAnalyse(super, isb = false, delta = 3, overlap=4)
MDegrain3(super,bv1,fv1,bv2,fv2,bv3,fv3,thSAD=400,planar=true)
Planar2Interleaved()

[burfadel]

Have you tried changing SetMTMode(2,0) to SetMTMode(2,4), where '4' is of course for a quad core, change it to your requirement.

Just setting it to 0 is for automatic, it may work still if set manually?... Its still something that should be looked at, but since MT isn't an official feature of Avisynth yet...

[TheRyuu]

This is a known problem with x264 and the new way avisynth input works.

[CarlEdman]

Have you tried changing SetMTMode(2,0) to SetMTMode(2,4), where '4' is of course for a quad core, change it to your requirement.

Good idea. Unfortunately, setting the number of threads explicitly (or mucking with thread-input) does not seem to help.

This is a known problem with x264 and the new way avisynth input works.

Good to know that this is a known bug even though I did not find any discussion of it when I searched before I posted (I swear!).

Any idea when/if this bug will be fixed?

I've used the denoising script quoted above on hundreds of different input materials yielding better-than-transparent results and substantially enhanced compressibility at a performance hit which is tolerable IF avisynth multithreading is enabled.

Having to choose between this powerful denoising and the enhanced features appearing in every new x264 version would be a major loss for me. :(

[CarlEdman]

This is a known problem with x264 and the new way avisynth input works.

Actually, thanks for the reference to Doom10 in your signature. I searched those forums and a contemporaneous thread there (http://doom10.org/index.php?topic=55.0) discusses the same issue and suggests a solution: Use piping and avs2yuv. That should work, regardless of what x264 does, and would allow use of 64-bit x264 as a side benefit.

I'll try that as soon as I've added proper piping to my custom mp4 builder python program. Still, it would be nice if avisynth MT incompatibility with X264 was fixed.

[kemuri-_9]

I still need you to try the following build of x264 and see if there's still a problem with AVS MT to confirm my suspicions about what is going on with AVS MT:
x264_avs_mt_test.exe

use the exact same commandline you posted originally for the test.

[CarlEdman]

I still need you to try the following build of x264 and see if there's still a problem with AVS MT to confirm my suspicions about what is going on with AVS MT:
x264_avs_mt_test.exe (http://kemuri9.net/dev/x264/experimental/x264_avs_mt_test.exe)

You may be confusing me with somebody else, but I did try the version of x264 you posted. It still has the same avisynth slowness as all the post r1352 official releases (i.e., 3-4 fps rather than than 10+ fps for MT-heavy avisynth scripts).

use the exact same commandline you posted originally for the test.

Exact same command line, except switched to raw .264 as your build does not support .mp4 output.

[kemuri-_9]

Exact same command line, except switched to raw .264 as your build does not support .mp4 output.

post the commandline you used then.

[CarlEdman]

post the commandline you used then.

No problem. The only change was to the extension of the output file from "mp4" to "264":

x264_avs_mt_test.exe --tune film --crf 19 --preset slower --sar 8:9 --profile high --level 3.1 --output "VTS_01_PGC_01_1.264" "VTS_01_PGC_01_1.avs"

[kemuri-_9]

start reading from http://doom10.org/index.php?topic=55.msg393#msg393
to see the cause of the problem and a known fairly easy workaround/solution

[IanB]

Summary :- there is no Distributor() call at the end of the script without VFW so there is no multithreading.

:search: Search for the same issue with HCEncoder.

[kemuri-_9]

thanks for the tip IanB.
A fix for this has been applied and will commit with the next spree.

[CarlEdman]

With Distributor() added everything works fine again with the latest and greatest x264.

:thanks:

[Dreadkillz]

Hi, I just wanted summarize everything that was discussed here. I have x264 build 1360 from the official site. I also have sET's 2.5.8 MT patch. So this means that I should add "Distributor()" at the end of my scripts now for setmtmode to work? Where can I download a patch of x264 so I don't have to do this?