Avisynth 2.6 MT

[SubJunk]

That's a weird problem to have. Hopefully someone can help you.

[Boulder]

Does MeGUI need the Distributor() call?

[SubJunk]

Originally Posted by Boulder

Does MeGUI need the Distributor() call?

I use MeGUI a lot and have never needed Distributor

[defalopii]

For masters who want to experiment with this monster regarding to the MeGUI, Avisynth and x264 optimizing on multicore system You can PM me, i will give You access to this system

[Gavino]

Originally Posted by SEt

With so many cores you sure hitting 32-bit address space limits with default number of threads. Try manually specifying different number of used threads in first SetMTMode call.

Originally Posted by SubJunk

Just change the first line in the script from:
SetMTMode(3)
to:
SetMTMode(3, 32)

What's the point of that? - it's just the same.
Since he has 32 cores, the default is to use 32 threads anyway.

SEt's advice was to use fewer threads, eg
SetMTMode(3, 8)

[SEt]

Yes, I was suggesting using smaller numbers like SetMTMode(3, 8).
Also, CPU usage is only a hint if you are doing something completely wrong or not – only speed matters in the end and best speed in certain situations doesn't come with 100% CPU usage.

[defalopii]

Originally Posted by Gavino

What's the point of that? - it's just the same.
Since he has 32 cores, the default is to use 32 threads anyway.

SEt's advice was to use fewer threads, eg
SetMTMode(3, 8)

The MT doesnt work. I've tried many configuration, from 1 to hundred value but still no different between using MT or not

The best way for me is split the video source using mkvmerge GUI into 8-20 parts than encode it simultaneously. This is awesome, i can encode a video 1080p (1920x1080) with total duration 1h 57mn only take not more than 2 hours using above x264 profile. And the CPU usage can reach from 60% to 100% on the 1st pass, more part means more CPU usage can be maximized

But splitting and re-merging that so many pieces of course annoying. Thats why i still want the developer of MeGUI, Avisynth 2.6 MT or x264 can make it more easy-without manually split the video source into many parts-to maximize the multi core system

[Keiyakusha]

defalopii
Do note that having that much threads can degrade quality of your video. Because it is achieved by starting encoding of the next frame while previous is not yet finished (or something like that) or by using slices, but this also bad if you have so many of them. The difference is usually unnoticeable, but with this much threads you probably want to check this.
UPD: also, looking on your spitted files 60% load i recall that 1st pass never was very thread-efficient by itself so u should make sure that load during 2nd pass is maximized. Not sure if something about it was changed lately, i don't use 2-pass encodes for years now, only crf.

[Boulder]

Does the amount of utilized threads by the encoder executable change in Task Manager when you change the number of threads?

[SubJunk]

Originally Posted by Gavino

Since he has 32 cores, the default is to use 32 threads anyway.

Often the MTMode functions only use half the available threads, especially on Intels, so it's common for it to use 16 threads on a 32-thread CPU.
That's not this guy's problem though, I'm just replying to the question

[defalopii]

Originally Posted by Keiyakusha

defalopii
Do note that having that much threads can degrade quality of your video. Because it is achieved by starting encoding of the next frame while previous is not yet finished (or something like that) or by using slices, but this also bad if you have so many of them. The difference is usually unnoticeable, but with this much threads you probably want to check this.

There is no quality degrade for the video result if You split it first into many parts than encde it simultaneously. I dont know if using MT, it probably degrade the quality.

Originally Posted by Boulder

Does the amount of utilized threads by the encoder executable change in Task Manager when you change the number of threads?

Yes its increase along with MT value. But if too much it will error "Process exits with error code: -1073741819 "

[pbristow]

Should this discussion be moved to the usage forum, perhaps? It doesn't seem to be a 2.6 MT development issue, more a general question of how to use Avisynth MT effectively.

[Keiyakusha]

Originally Posted by defalopii

There is no quality degrade for the video result if You split it first into many parts than encde it simultaneously. I dont know if using MT, it probably degrade the quality.

Sure, this is exactly what i mean. I tried to say that your way of splitting files to 8 or more parts may be better idea after all.


Does anyone knows if it possible that with this much threads x264 just reaches some point when previous frame doesn't have enough information to start encoding new frame so new thread is not created? Then maybe higher resolution video will use more threads... Makes any sense?

[Boulder]

Originally Posted by defalopii

Yes its increase along with MT value. But if too much it will error "Process exits with error code: -1073741819 "

Have you tried to do something that is more CPU-demanding than a simple resize and Convolution3D at low resolution?

[defalopii]

Originally Posted by Boulder

Have you tried to do something that is more CPU-demanding than a simple resize and Convolution3D at low resolution?

yes i've tried it on 720x400, thread increase but the FPS just a little different with no MT. Even if using too high value its become slower

[Overdrive80]

Originally Posted by defalopii

The MT doesnt work. I've tried many configuration, from 1 to hundred value but still no different between using MT or not

You can try SetMTMode(5, 8), with my cpu i have problems too using MT in mode 3. Intel core i7

[defalopii]

Originally Posted by Overdrive80

You can try SetMTMode(5, 8), with my cpu i have problems too using MT in mode 3. Intel core i7

Only got about 70 FPS 1st pass on 720x400 videos, still same with No MT/another MT value arround 65-70 FPS, no significant increase

[SEt]

If SetMTMode doesn't work – you are doing something wrong. I can't guarantee that 32 threads will work flawlessly (never tested such case), but something around 8 definitely works well. And I'm talking about Avisynth speed, not something like MeGUI to x264.

There is no quality degradation with SetMTMode. Either filters are MTMode-friendly and you will get exactly the same result as in non-MTMode, or you'll get crash, glitches, whatever but graceful quality degradation.

Originally Posted by SubJunk

Often the MTMode functions only use half the available threads, especially on Intels

What? First time I hear such issue. How about making proper bug report?

[defalopii]

Originally Posted by SEt

If SetMTMode doesn't work – you are doing something wrong. I can't guarantee that 32 threads will work flawlessly (never tested such case), but something around 8 definitely works well. And I'm talking about Avisynth speed, not something like MeGUI to x264.

Whats iwrong ?? how is the proper installation of Avisynth 2.6 MT ??
What i did, installed the Avisynth from here avisynth.org, than replace the avisynth.dll in the System32 folder with avisynth.dll from Avisynth 2.6 MT, thats it, is that wrong ??

Actually guys, how many percent increasing the FPS with or without Avisynth 2.6 MT ?? Is Avisynth 2.6 MT utilize the all cores up to arround 100% of CPU Usage ??

Do You ever trying like me splitting the videos into many parts and encode it simultaneously ?? than see how awesome it utilize the all core and CPU Usage and increasing the total encoding speed. Maybe Avisynth MT, or MeGUI or even x264 should try this methode, spliting the videos rather than playing with threads/frames

I dont know, am i still on my way.,,??
or sould move it to another sub-forum, there is so much, i am confused where must write this

[Keiyakusha]

Originally Posted by SEt

There is no quality degradation with SetMTMode. Either filters are MTMode-friendly and you will get exactly the same result as in non-MTMode, or you'll get crash, glitches, whatever but graceful quality degradation.

It degrades on x264 level, not avisynth. Unless for some reason you use 32 threads for avisynth but only like 2 threads for x264.
As for me, I don't see the reason to use many threads in avisynth. Most of the time the actual encoding is a more CPU-intensive task than pre-processing. Of course if we use only good optimized filters, and our source is not some old self-recorded VHS that needs a lot of processing. But many "heavy" filters can have their own threads anyway.