Processor never reaches %100 use with MeGUI

[saint-francis]

I have encoded about 50 -60 movies with MeGUI and I have never seen my processor maxed out. In some cases this is nice because I can browse the web and so on while encoding but I usually do my encoding at night and when I'm at work. I am encoding with x.264 and with the HQ insane profile and I basically follow the instructions in the wiki since I don't yet have a firm grasp on all of the processes MeGUI is running. I have the sneaking suspicion that this could be the AVS filters not being multi threaded. I have heard tell of the MT plugin but I have little or no understanding of it or how to experiment with it. But I assume that this must be an issue people have encountered before. Any thoughts?

[BigDid]

I have encoded about 50 -60 movies with MeGUI and I have never seen my processor maxed out.
Hi,

A link to a very interesting thread discussing testing x264/MeGUI/xxcores: http://forum.doom9.org/showthread.php?t=122318

...I have the sneaking suspicion that this could be the AVS filters not being multi threaded. I have heard tell of the MT plugin but I have little or no understanding of it or how to experiment with it.
The MT thread: http://forum.doom9.org/showthread.php?t=94996
and the Wiki Support page: http://avisynth.org/mediawiki/MT_support_page

Did

[graysky]

@SF: what are the specs on your hardware and O/S?

[JohnnyMalaria]

There is a common misconception that multithreading automatically means maximum CPU use on a multiprocessor system.

The key to maximum CPU use is parallel processing. Multithreading is a means to achieve it. However, the software has to be explicity written to process data in parallel and, to achieve 100% use on all processors, 100% of the entire data processing would have to be able to run in parallel. There are very few real-world scenarios that achieve this. At some point, there will be parts of the processing that can only occur serially. e.g., reading a GOP sequence, a single frame etc. Also, as processors get faster and faster, the slower parts of the system - hard drive, RAM etc - begin to hit performance dramatically - leading to less than 100% use. If you did get 100% on all processors, I'd be very suspicious of the programming - it may merely indicate sloppy coding.

With multithreading comes additional overhead for the operating system (whether Windows, Linux, OS X....) and - the real bane of programmers - cache pollution. It is quite common to code something to be high parallelized (horrid word!) and have it run slower than the serial/single threaded variety.

"Multithreaded" seems to have become the buzzword of the year and somehow it is spun as all-new. Multithreaded applications have been around for many years (esp. on Windows). Anyone compiling Windows applications with Microsoft's development tools automatically generates a multithreaded app. Generating efficient parallel code is a whole other matter....

[saint-francis]

@ graysky:

E6600 2.4 OC'ed to 3.5 on air Orthos stable for 15 hours.
2 GB Corsair DDR2 800
DFI Infinity 975 x/g (Awesome board BTW. I HIGHLY recommend it to any one. Easy as pie to overclock to anything air or water can handle and at a good price)
Windows XP SP2
620 watt corsair psu

I usually see about 75% CPU usage. Is this normal? I'm getting about 8 fsp. I tried one encoding on HQ slowest and my first pass was about 120 fps. I forget what the second was but it was much slower.
As far as MT goes I think I am in over my head. I can barely understand what the AVS means when I look at the ones MeGUI makes. At this point in time there is no way I am able to figure out how to implement it into the AVS. I need to figure out the basics of AviSynth first.

[carlo_0000]

the mine goes from 96-100%

on athlon x2 4200 with windows 2003

encoding @ slowest setting in hd 1920x1080 @ 8500kbps

[saint-francis]

the mine goes from 96-100%

on athlon x2 4200 with windows 2003

encoding @ slowest setting in hd 1920x1080 @ 8500kbps

Are you using MT? Can you please elaborate on your setting?

[Adub]

I get 100% using latest x264, megui, MT avisynth on occasion but not always. The multithreaded x264 is the key in my opinion, it works like a charm.

Here are my specs.
Core 2 duo e6600 @3.06ghz
2 gigs of ram
500gb raid0 array

MT 0.7, avisynth 2.5.7, x264 rev650 with ssse3 support.

[saint-francis]

I get 100% using latest x264, megui, MT avisynth on occasion but not always. The multithreaded x264 is the key in my opinion, it works like a charm.

MT 0.7, avisynth 2.5.7, x264 rev650 with ssse3 support.

If I posted an AVS could you give me a rough idea of where to begin using MT? It sounds like you are on to something important and maybe MeGUI should be updated. :confused:

[Adub]

The problem really isn't MeGUI, but by all means, post your script and lets see what may be slowing you down.

[saint-francis]

# Set DAR in encoder to 37 : 20. The following line is for automatic signalling
global MeGUI_darx = 37
global MeGUI_dary = 20
DGDecode_mpeg2source("D:\Ong Bak\VTS_01_1.d2v")
#Not doing anything because the source is progressive
crop( 6, 8, -6, -12)

LanczosResize(720,464) # Lanczos (Sharp)
Undot() # Minimal Noise


This is basically what they all look like. I pretty much use the same settings for all of the movies I encode. I am rarely encoding anything that it interlaced or cartoon (although I did just encode waking life and it was very confusing because it is cartoon on top of film.)

[graysky]

You must not have x264.exe setup to use auto threads. I have a q6600 and always get 100 % usage on the 2nd pass. Can you post one of your log files?

To give you an idea of what is maxing out my quad:

Log for job job73

Film percentage: 96.75 meets force film thresholdSuccessfully applied force film
----------------------------------------------------------------------------------------------------------
Starting postprocessing of job...
Job completed successfully and deletion of intermediate files is activated
Postprocessing finished!
Looking for job processor for job...
Processor found!
Starting job job37-1 at 2:06:24 AM
Starting preprocessing of job...
Preprocessing finished!
encoder commandline:
--pass 1 --bitrate 1895 --stats "D:\work\e8\benchmark.stats" --bframes 3 --b-pyramid --direct auto --subme 1 --analyse none --vbv-maxrate 25000 --me dia --threads auto --thread-input --progress --no-dct-decimate --no-psnr --no-ssim --output NUL "D:\work\e8\benchmark.avs"
successfully started encoding
Processing ended at 2:17:05 AM
----------------------------------------------------------------------------------------------------------

Log for job job37-1

avis [info]: 704x480 @ 23.98 fps (77694 frames)
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2
x264 [warning]: VBV maxrate specified, but no bufsize.
x264 [info]: slice I:717 Avg QP:15.79 size: 46604
x264 [info]: slice P:24312 Avg QP:17.61 size: 17140
x264 [info]: slice B:52665 Avg QP:18.81 size: 6256
x264 [info]: mb I I16..4: 32.4% 0.0% 67.6%
x264 [info]: mb P I16..4: 18.3% 0.0% 0.0% P16..4: 67.9% 0.0% 0.0% 0.0% 0.0% skip:13.8%
x264 [info]: mb B I16..4: 1.7% 0.0% 0.0% B16..8: 26.6% 0.0% 0.0% direct:33.4% skip:38.3%
x264 [info]: final ratefactor: 18.76
x264 [info]: direct mvs spatial:99.9% temporal:0.1%
x264 [info]: kb/s:1924.6

encoded 77694 frames, 121.69 fps, 1924.64 kb/s

----------------------------------------------------------------------------------------------------------
Starting postprocessing of job...
Job completed successfully and deletion of intermediate files is activated
Postprocessing finished!
Looking for job processor for job...
Processor found!
Starting job job37-2 at 2:17:05 AM
Starting preprocessing of job...
Preprocessing finished!
encoder commandline:
--pass 2 --bitrate 1895 --stats "D:\work\e8\benchmark.stats" --ref 5 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --subme 6 --trellis 1 --analyse all --8x8dct --vbv-maxrate 25000 --me umh --threads auto --thread-input --sar 40:33 --progress --no-dct-decimate --no-psnr --no-ssim --output "D:\work\e8\benchmark.mp4" "D:\work\e8\benchmark.avs"
successfully started encoding
Processing ended at 2:58:42 AM
----------------------------------------------------------------------------------------------------------

Log for job job37-2

avis [info]: 704x480 @ 23.98 fps (77694 frames)
x264 [info]: using SAR=40/33
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2
x264 [warning]: VBV maxrate specified, but no bufsize.
mp4 [info]: initial delay 2002 (scale 24000)
x264 [info]: slice I:717 Avg QP:15.58 size: 43993
x264 [info]: slice P:24312 Avg QP:16.93 size: 17338
x264 [info]: slice B:52665 Avg QP:18.30 size: 5976
x264 [info]: mb I I16..4: 24.2% 64.7% 11.1%
x264 [info]: mb P I16..4: 5.3% 11.3% 1.6% P16..4: 36.2% 20.5% 13.4% 0.5% 0.3% skip:10.9%
x264 [info]: mb B I16..4: 0.4% 0.6% 0.1% B16..8: 32.8% 2.5% 6.2% direct: 6.2% skip:51.2%
x264 [info]: 8x8 transform intra:61.7% inter:62.9%
x264 [info]: direct mvs spatial:97.4% temporal:2.6%
x264 [info]: ref P 53.7% 22.9% 10.4% 7.7% 5.4%
x264 [info]: ref B 68.8% 17.8% 6.3% 4.1% 3.0%
x264 [info]: kb/s:1895.5

encoded 77694 frames, 31.20 fps, 1895.60 kb/s

desired video bitrate of this job: 1895 kbit/s - obtained video bitrate (approximate): 1897 kbit/s
----------------------------------------------------------------------------------------------------------
Starting postprocessing of job...
Job completed successfully and deletion of intermediate files is activated
Found intermediate output file 'D:\work\e8\benchmark.stats', deleting...
Deletion succeeded.Postprocessing finished!

[saint-francis]

You must not have x264.exe setup to use auto threads. I have a q6600 and always get 100 % usage on the 2nd pass. Can you post one of your log files?

I will do so asap. Right now I am having a more serious problem with MeGUI that I am trying to get under control, but when I have it up and running again (a day or so) I will post an average log.

[graysky]

Try this under tools>settings:

http://img178.imageshack.us/img178/2692/threadspj6.gif

Also make sure the jobs files it makes have the following:

<NbThreads>0</NbThreads>

[saint-francis]

Well I have managed to get MeGUI using 100% of the CPU. I just set the number of threads to 4 manually. I set it to 3 on a whim and noticed an increase in processor utilization so I raised the number of threads to 4 and it now rocks. Even when using a bunch of avisynth plugins it still stays at 100% So why isn't MeGUI setting the number of threads to 4 by itself?

[legoman666]

I have noticed the same problem on my 2.6ghz AMD x2. I got 85-90% cpu utilization. While although this is nice while I'm using the computer, I want it to use 100% while I'm sleeping and afk. I have it set to 3 threads and am using the HQ-slower profile encoding 1080i hd content to 720p h264.

The weird thing is, when I pass the settings to x264.exe directly, it can max out the cpu, but megui can't.