x264 revision 610 - sliceless threading - what's new

[bob0r]

------------------------------------------------------------------------
r610 | pengvado | 2006-12-16 01:32:38 +0100 (Sat, 16 Dec 2006) | 2 lines

more win32threads -> pthreads

------------------------------------------------------------------------
r609 | pengvado | 2006-12-16 00:08:57 +0100 (Sat, 16 Dec 2006) | 2 lines

cosmetics: rename list operators to be consistent with Perl, and move them to common/

------------------------------------------------------------------------
r608 | pengvado | 2006-12-16 00:06:21 +0100 (Sat, 16 Dec 2006) | 2 lines

win32: use pthreads instead of win32threads. for some reason, pthreads is much faster.

------------------------------------------------------------------------
r607 | pengvado | 2006-12-16 00:03:36 +0100 (Sat, 16 Dec 2006) | 13 lines

New threading method:
Encode multiple frames in parallel instead of dividing each frame into slices.
Improves speed, and reduces the bitrate penalty of threading.

Side effects:
It is no longer possible to re-encode a frame, so threaded scenecut detection
must run in the pre-me pass, which is faster but less precise.
It is now useful to use more threads than you have cpus. --threads=auto has
been updated to use cpus*1.5.
Minor changes to ratecontrol.

New options: --pre-scenecut, --mvrange-thread, --non-deterministic

------------------------------------------------------------------------


when multithreading "--threads auto" recommended


Here are some results showing cpu*1.5:
x264_610.exe --threads # -B5000 -m6 -r5 --direct=temporal --me=hex -b2 -w --qcomp=0.10 -A"p8x8,i8x8,i4x4" -8 --fps=25 --output NUL 720p50_mobcal_ter.yuv 1280x720
x264_606.exe --threads # -B5000 -m6 -r5 --direct=temporal --me=hex -b2 -w --qcomp=0.10 -A"p8x8,i8x8,i4x4" -8 --fps=25 --output NUL 720p50_mobcal_ter.yuv 1280x720

Intel D930 3.0GHz (2 cores, no HT):
610 2 threads: 3.89
610 3 threads: 5.41
606 2 threads: 4.63
606 3 threads: 4.46

P4 xeon 3.06GHz (2 cores, HT):
610 4 threads: 4.24
610 5 threads: 4.63
606 4 threads: 3.98
606 5 threads: 4.01

AMD X2 3800+ 2.55GHz (2 cores):
610 2 threads: 4.07
610 3 threads: 5.24
606 2 threads: 4.85
606 3 threads: 4.78

C2D E6400 3.4GHz (2 cores):
610 2 threads: 8.78
610 3 threads: 12.35
606 2 threads: 9.52
606 3 threads: 8.34

I have more results, of earlier patches:
http://x264.nl/results.txt (cv = pthreads)

Not sure why, but Intel Core 2 Duo users can be happy with the big speed up.

Note as the changelog says, New options: --pre-scenecut, --mvrange-thread, --non-deterministic,
Go MeGUI .... and other guis!

[Thunderbolt8]

what about the aspect akupenguin once mentioned that, with more splitting up into more threads the quality also decreases (slightly) ? does this still apply now ?

[akupenguin]

Yes, but not as much as before. Hence the "reduces the bitrate penalty".

cpu: 4 core Xeon 5160

threads speed psnr loss
r606 r611 r606 r611
1: 1.000x 1.000x 0.000 0.000
2: 1.540x 1.739x -0.036 -0.004
3: 1.838x 2.384x -0.065 -0.002
4: 2.043x 3.224x -0.077 -0.005
5: 2.028x 3.512x -0.110 -0.009
6: 2.034x 3.629x -0.132 -0.009
7: 1.988x 3.680x -0.151 -0.015
8: 1.953x 3.702x -0.188 -0.017
9: 2.016x 3.729x -0.210 -0.020
10: 1.995x 3.742x -0.233 -0.031
11: 1.954x 3.749x -0.255 -0.030
12: 1.909x 3.765x -0.268 -0.040
13: 1.895x 3.770x -0.286 -0.045
14: 1.936x 3.759x -0.313 -0.046
15: 1.897x 3.781x -0.335 -0.045
16: 1.845x 3.765x -0.349 -0.046

scaling efficiency (speed / #cores):
r606: 51%
r611: 94%

The above was measured on Linux. We had much trouble getting efficient thread synchronization on Windows, and the performance still might be not quite as good.

[Sharktooth]

So... the megui automatic no. of threads detection should be updated.
also what's the exact usage for the new options? what do they actually do?
is the haali's AQ patch working correctly after those changes (it applies almost correctly btw...)?

for builders: pthread for win32 is here -> ftp://sources.redhat.com/pub/pthreads-win32

[bob0r]

@builders
or here:
cvs -d :pserver:anoncvs@sources.redhat.com:/cvs/pthreads-win32 checkout pthreads
make clean GC-static
copy pthreads/*.h to include dir
copy libpthreadGC2.a to lib dir

@sharktooth:
You can just use --threads auto for megui, as x264.exe will detect the # of cpu to use, and set threads by *1.5 (Using HT might give one or to extra threads, but the quality will not be affected much at all)

Exact usage of new options no clue here, from x264.exe:
--pre-scenecut Faster, less precise scenecut detection.
Required and implied by multi-threading.

--mvrange-thread <int> Minimum buffer between threads [-1 (auto)]

--non-deterministic Slightly improve quality of SMP, at the cost of repeatability

[akupenguin]

Short answer: you shouldn't need to use any of the new options.
--pre-scenecut might possibly be useful for fast 1pass single-threaded encodes, but is mostly just so I can compare the two scenecut algorithms without invoking the other threading stuff.

[Sharktooth]

im just updating megui... i removed the ability to set the number of threads within the x264 profile (updown control grayed out).
so --threads auto will be enforced when "Automatically set the number of threads" global option is enabled in megui settings.
It's just a workaround but should work as expected.

[Sharktooth]

Im reorganizing the MeGUI version numbers. A new MeGUI build is coming. Just be patient...:)

[Adub]

Uh, sharktooth? I just downloaded the new version, and those settings aren't greyed out. In addition MeGUI still sets the threads manually. Like I am using 2 threads right now according to the log for the first pass.

Edit: look at the post times! That should explain it.

[Audionut]

Thanks. This build is alot faster on a core 2.

[Zerofool]

Theoretically will this thing have any (positive) effect on single core CPUs (set at 2 threads) ? Does it make any sense using it that way?
(I'll try it tonight anyway :).)

[Sharktooth]

megui 0.2.3.2193 is up and running. Enjoy.
Tomorrow i'll update the Trunk version (0.2.4.0000) with a more elegant solution ;)

@bobor and other builders: DO A SVN CHECKOUT and use the sources in Tags/2193 for building.

[Audionut]

And working like a charm. Thanks.

[Adub]

I can't update to 2193. I mean I downloaded it, it asks me to restart the program. It does, but it still says 2192 at the top and the x264 options aren't greyed out.

[Sharktooth]

uhm... reboot. .NET cache is doing something wrong.
now im going to sleep (5:21AM) ;)
if you have still problems unistall megui and install a fresh version. then let it autoupdate.

[woah!]

i use ELDER now for my encodes but i see this "bug" in the release i thought might need a look at:

x264.nl REV611: http://mirror01.x264.nl/x264/revision611/x264.exe

H:\MCHD encodes\ELDER RATIO.1.77.1\905029>..\x264.exe -o test3.264 --ref 5 --8x8
dct --mixed-refs --no-fast-pskip --bframes 3 --bime --weightb --direct auto --me
range 12 --subme 6 --analyse all --me umh --colormatrix bt709 --filter -2:-1 --p
ass 2 --stats stageC-3.stats --progress -B 1377 stageA-3.avs
avis [info]: 720x400 @ 23.98 fps (357 frames)
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2
x264 [info]: slice I:5 Avg QP:20.60 size: 16678 PSNR Mean Y:46.04 U:47.30
V:47.27 Avg:46.40 Global:45.94
x264 [info]: slice P:268 Avg QP:22.98 size: 8262 PSNR Mean Y:44.48 U:46.08
V:45.76 Avg:44.89 Global:44.22
x264 [info]: slice B:84 Avg QP:23.46 size: 3583 PSNR Mean Y:44.30 U:46.71
V:46.87 Avg:44.96 Global:44.21
x264 [info]: mb I I16..4: 22.0% 63.3% 14.7%
x264 [info]: mb P I16..4: 15.7% 25.7% 4.5% P16..4: 30.2% 7.1% 2.0% 0.2% 0
.1% skip:14.5%
x264 [info]: mb B I16..4: 1.7% 4.0% 2.2% B16..8: 31.7% 1.6% 2.5% direct:
2.8% skip:53.5%
x264 [info]: 8x8 transform intra:56.0% inter:50.3%
x264 [info]: direct mvs spatial:0.0% temporal:100.0%
x264 [info]: ref P 73.7% 13.5% 8.0% 2.7% 2.2%
x264 [info]: ref B 86.8% 6.1% 3.7% 1.9% 1.5%
x264 [info]: SSIM Mean Y:0.9766733
x264 [info]: PSNR Mean Y:44.460 U:46.243 V:46.042 Avg:44.927 Global:44.239 kb/s:
1396.13

encoded 357 frames, 5.57 fps, 1396.55 kb/s
profiling:F:\msys\home\user\x264/x264.gcda:Cannot open
profiling:F:\msys\home\user\x264/matroska.gcda:Cannot open
profiling:F:\msys\home\user\x264/muxers.gcda:Cannot open
profiling:F:\msys\home\user\x264/encoder/encoder.gcda:Cannot open
profiling:F:\msys\home\user\x264/common/common.gcda:Cannot open
profiling:F:\msys\home\user\x264/common/mdate.gcda:Cannot open
profiling:F:\msys\home\user\x264/encoder/set.gcda:Cannot open
profiling:F:\msys\home\user\x264/common/macroblock.gcda:Cannot open
profiling:F:\msys\home\user\x264/common/set.gcda:Cannot open
profiling:F:\msys\home\user\x264/encoder/ratecontrol.gcda:Cannot open
profiling:F:\msys\home\user\x264/common/frame.gcda:Cannot open
profiling:F:\msys\home\user\x264/common/pixel.gcda:Cannot open
profiling:F:\msys\home\user\x264/encoder/macroblock.gcda:Cannot open
profiling:F:\msys\home\user\x264/common/cpu.gcda:Cannot open
profiling:F:\msys\home\user\x264/encoder/analyse.gcda:Cannot open
profiling:F:\msys\home\user\x264/common/mc.gcda:Cannot open
profiling:F:\msys\home\user\x264/common/cabac.gcda:Cannot open
profiling:F:\msys\home\user\x264/encoder/cavlc.gcda:Cannot open
profiling:F:\msys\home\user\x264/encoder/cabac.gcda:Cannot open
profiling:F:\msys\home\user\x264/common/dct.gcda:Cannot open
profiling:F:\msys\home\user\x264/common/quant.gcda:Cannot open
profiling:F:\msys\home\user\x264/common/csp.gcda:Cannot open
profiling:F:\msys\home\user\x264/common/predict.gcda:Cannot open
profiling:F:\msys\home\user\x264/encoder/eval.gcda:Cannot open
profiling:F:\msys\home\user\x264/common/i386/predict-c.gcda:Cannot open
profiling:F:\msys\home\user\x264/encoder/me.gcda:Cannot open
profiling:F:\msys\home\user\x264/common/i386/mc-c.gcda:Cannot open

H:\MCHD encodes\ELDER RATIO.1.77.1\905029>echo.1>stageC-result3.ready




===========================================



Sharktooths REV611: http://mirror05.x264.nl/Sharktooth/?dir=./x264


H:\MCHD encodes\ELDER RATIO.1.77.1\311401>..\x264.exe -o test3.264 --ref 5 --8x8
dct --mixed-refs --no-fast-pskip --bframes 3 --bime --weightb --direct auto --me
range 12 --subme 6 --analyse all --me umh --colormatrix bt709 --filter -2:-1 --p
ass 2 --stats stageC-3.stats --progress -B 1377 stageA-3.avs
avis [info]: 720x400 @ 23.98 fps (357 frames)
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2
x264 [info]: slice I:5 Avg QP:20.60 size: 16678 PSNR Mean Y:46.04 U:47.30
V:47.27 Avg:46.40 Global:45.94
x264 [info]: slice P:268 Avg QP:22.98 size: 8262 PSNR Mean Y:44.48 U:46.08
V:45.76 Avg:44.89 Global:44.22
x264 [info]: slice B:84 Avg QP:23.46 size: 3583 PSNR Mean Y:44.30 U:46.71
V:46.87 Avg:44.96 Global:44.21
x264 [info]: mb I I16..4: 22.0% 63.3% 14.7%
x264 [info]: mb P I16..4: 15.7% 25.7% 4.5% P16..4: 30.2% 7.1% 2.0% 0.2% 0
.1% skip:14.5%
x264 [info]: mb B I16..4: 1.7% 4.0% 2.2% B16..8: 31.7% 1.6% 2.5% direct:
2.8% skip:53.5%
x264 [info]: 8x8 transform intra:56.0% inter:50.3%
x264 [info]: direct mvs spatial:0.0% temporal:100.0%
x264 [info]: ref P 73.7% 13.5% 8.0% 2.7% 2.2%
x264 [info]: ref B 86.8% 6.1% 3.7% 1.9% 1.5%
x264 [info]: SSIM Mean Y:0.9766733
x264 [info]: PSNR Mean Y:44.460 U:46.243 V:46.042 Avg:44.927 Global:44.239 kb/s:
1396.13

encoded 357 frames, 6.19 fps, 1396.55 kb/s

H:\MCHD encodes\ELDER RATIO.1.77.1\311401>echo.1>stageC-result3.ready


Sharktooth's version works well but the other one as you can see isnt built correctly (i think)

anyways...

[bob0r]

Updated revision 611 coming right up

wrong 611 x264.exe md5: 2d738a38420fb2d92154556a1aa29c70
correct 611 x264.exe md5: b64eb9012bd24e2cd4f616be482c5a31

In about 30 minutes a proper version should be online, without the half-as*-ed make fprofiled version :)

[huang_ch]

Great job, thanks all the x264 developers for this. Below is a quick test from me:
Hardware: Conroe E6600@2.4GHz

FPS Kbps PSNR
b606(1t) 9.76 1200.92 47.015
b606(2t) 14.28 1214.10 47.008
b611(1t) 9.37 1229.90 47.203
b611(2t) 15.30 1242.54 47.248
b611(auto) 17.45 1242.29 47.248
b611(4t) 17.97 1241.65 47.247

It's good to see no PSNR penalty now and the CPU is mostly >90% usage. :thanks:

[Adub]

your getting >90%. I am only reaching about 83%. But that is still first pass. Plus I am Folding in the background.

[Sharktooth]

Updated revision 611 coming right up

wrong 611 x264.exe md5: 2d738a38420fb2d92154556a1aa29c70
correct 611 x264.exe md5: b64eb9012bd24e2cd4f616be482c5a31

In about 30 minutes a proper version should be online, without the half-as*-ed make fprofiled version :)

my build is fprofiled ...

@Merlin: .NET cache sometimes does weird things... and when it happens it's always a PITA :)

[MatMaul]

Hello !
Thanks for your great job.
On a single core CPU (pentium M), I noticed a little decrease of the speed, also with one thread.

command line :
x264 --bitrate 1000 --threads nb_of_threads --ref 3 --bframes 2 --direct auto --subme 5 --partitions all --8x8dct --me hex --progress --output NUL "dd.avs"

v606 :

1 threads : 11.56 fps, 1006.55 kb/s, PSNR 48.655
2 threads : 10.61 fps, 1007.08 kb/s, PSNR 48.626
3 threads : 10.63 fps, 1007.75 kb/s, PSNR 48.599

v611 :

1 threads : 11.20 fps, 1008.27 kb/s, PSNR 48.683
2 threads : 11.26 fps, 1007.95 kb/s, PSNR 48.687
3 threads : 11.20 fps, 1008.29 kb/s, PSNR 48.686

Why the 1 thread's ouputs are not identical ?

[Sharktooth]

what build you used?

[MatMaul]

bobor builds, I can't find rev606 of your builds

[Sharktooth]

http://megui.org/auto/x264_r606.zip
bobor's 611 is not fprofiled... that could be the cause of the slowdown vs 606....

[huang_ch]

Sharktooth, in my previous post, I also met a slight performance loss with 1 thread mode(9.76fps->9.37fps), all uses your builds.

[Sharktooth]

uhm... interesting...
with 1 thread speed is slightly lower but PSNR is slightly higher... (from 0.03 to 0.19db according to your results)

[MatMaul]

same results with your builds :

v606 :

1 threads : 11.57 fps, 1006.55 kb/s, PSNR 48.655
2 threads : 10.55 fps, 1007.08 kb/s, PSNR 48.626

v611 :

1 threads : 11.30 fps, 1008.27 kb/s, PSNR 48.683
2 threads : 11.37 fps, 1007.95 kb/s, PSNR 48.687

the PSNR is higher but the bitrate too.

[Sharktooth]

Well, however the speed difference is not so big :)
Could you please test if AQ is working properly?

[MatMaul]

AQ seems to works good (test with --aq-strength 0.5 --aq-sensitivity 15, video is ok).
The speed isn't very problematic (just 2%).

[huang_ch]

Well, however the speed difference is not so big :)
Could you please test if AQ is working properly?

I'm trying AQ this whole day to fight again those blocks(which is posted in another thread), but since I didn't use AQ before, so what I could tell is that AQ seems to really work, the output is really much bigger than no-AQ, and no obvious error.:)

[bob0r]

@Sharktooth,
No you got it wrong, i was propering my own build, and yes all my x264.exe files are make fprofiled, just the wrong 611, x264.exe was the one used by x264 source to collect it's information, then the final x264.exe failed (i need to fix gtk), and thus the half-as*-ed version of fprofiled was online :)

@MatMaul,
Yes, others have reported the same problem also, must be said, so far i have only noticed this with .avs input.

@akupenguin,
Should 606 and 611 with 1 thread give the same speed?
Or is 606 using 1 win32 thread and 611 is using 1 pthreads thread?

[vanger]

your getting >90%. I am only reaching about 83%. But that is still first pass. Plus I am Folding in the background.
On my AMD x2 3800+ 2500Mhz the cpu load is near 80% (85% max) at 1st pass... And near 95% (max 99%) at 2nd...

[MatMaul]

@MatMaul,
Yes, others have reported the same problem also, must be said, so far i have only noticed this with .avs input.


You talk about speed or not identical outputs with 1 threads ?

[bob0r]

On a single core, also try --threads 2. --threads 3, --threads 4, see how the speed fps and cpu load change, still on single core, it seems avisynth can limit an encode.

All test with big .yuv files show huge improvements on many computers.

[deets]

holy crap!!!

us PSP owners will be eternally thankful :) now i can use my centrino duo to its full power and after a quick test, i think my encoding fps speed has just gone up by a crazy amount!!

this is amazing, thanks very much for all your work, bobor and sharktooth

edit: i shall have to update my site, this is the best news for PSP owners since AVC was added :D

[Dreassica]

hmm switched from build 600 to 611 and i get artifacts and blocking on playback now.

command line used on both:

C:\x264.exe --qp 17 --ref 16 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter 1,2 --subme 6 --trellis 2 --analyse all --8x8dct --me esa --threads auto --thread-input --progress --no-dct-decimate --no-psnr --output "E:\sample.mkv" "E:\sample.avs"

[Thunderbolt8]

is it actually possible to use 4 threads with a core 2 duo ?

[emmel]

Very impressive.
Thanks.

[bob0r]

...

this is amazing, thanks very much for all your work, bobor and sharktooth

...

Since the x264 developers now get paid to work on x264, i guess Sharktooth and me can take full credit of Loren "pengvado" Merritt and co their work :devil:

So here goes: No problem, it was our pleassure!

akupenguin :sly:

[bob0r]

is it actually possible to use 4 threads with a core 2 duo ?

yes use --threads 4 (instead of --threads auto)
new threads limit is 16.

[Thunderbolt8]

yes use --threads 4 (instead of --threads auto)
new threads limit is 16.
so I could make my own new default with 16 threads each time then and it would be fastest ? is the loss of quality with 16 threads really as low as it seems on paper (i mean o,35% is like nothing)?

[akupenguin]

On a single core, also try --threads 2. --threads 3, --threads 4, see how the speed fps and cpu load change, still on single core, it seems avisynth can limit an encode.
If speed is limited by reading a big yuv file from hdd, then --thread-input will fix it. Load could also be limited if you put x264 on low priority, and then threads would be counterproductive. There is no other reason for load to not be 100% on a single core, and --threads should never be used on a single core.

Should 606 and 611 with 1 thread give the same speed?
Yes. But that's "should" as in a failure to do so would be a regression, not as in I'm sure its true.
Or is 606 using 1 win32 thread and 611 is using 1 pthreads thread?
1 thread invokes neither win32threads nor pthreads.

so I could make my own new default with 16 threads each time then and it would be fastest ?
I've seen quality loss of up to 1% for 16 threads. But yes, you could use lots.

[Adub]

wait x264 devs are getting paid now to work on the project? When did this happen?

[MatMaul]

@ akupenguin : is it normal I obtain a different output than with rev606 with 1 thread (or with multithreading disabled at the compilation) ?

v606, 1 thread : 1006.55 kb/s, PSNR 48.655

v611, 1 thread (or multithreading disabled at the compilation) : 1008.27 kb/s, PSNR 48.683

[akupenguin]

@MatMaul
Yes, there were minor changes in ratecontrol and B-adapt.
In your case, +.17% bitrate and +.028 psnr is a net gain, since as a rule of thumb .05 psnr is worth 1% bitrate.

[MatMaul]

Thanks, good news !

[emmel]

Small test on a four-core smp (two dual core Opterons) with default settings for x264 (--threads N -o test_720x576.264 test_720x576.y2m):

1 threads: 40.835 dB, 13.61 fps, 1743.59 kb/s
2 threads: 40.829 dB, 24.55 fps, 1740.32 kb/s
3 threads: 40.829 dB, 34.20 fps, 1740.32 kb/s
4 threads: 40.829 dB, 44.19 fps, 1740.25 kb/s
8 threads: 40.829 dB, 44.53 fps, 1739.77 kb/s

Scales almost linearly up to four threads without any significant loss in quality. Very nice indeed.

[Zero1]

hmm switched from build 600 to 611 and i get artifacts and blocking on playback now.

command line used on both:

C:\x264.exe --qp 17 --ref 16 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter 1,2 --subme 6 --trellis 2 --analyse all --8x8dct --me esa --threads auto --thread-input --progress --no-dct-decimate --no-psnr --output "E:\sample.mkv" "E:\sample.avs"
Same here.

Also, impressive improvements this time, my cores are 100% for a lot of the time with the occasional dip. It's nice that both cores are being put to good use (A64 Turion x2 TL-60).

[DeeGee]

Artifacting here too. But after ironing out those bugs, this is most exellent update. My Core 2 Duo is liking pthreads very much :)

[bob0r]

Can you guys post some screenshots of this artifacting?
Or even better, video samples.

[Dreassica]

I already posted 2 shots on page 2

[Sirber]

updating RA5 for it. cool!

[bob0r]

I already posted 2 shots on page 2

Hmm, can you test like only where the bug happens, a 5-10 seconds sample, and then keep removing options untill the problem is gone? (if ever)

Or give us a small source sample and we can do it ourselfs.
Dont forget to post your .avs file contents.

Next time post screenshots of identical frames, good and bad version.

Thanks.

[Dreassica]

Seems it's teh number of reference frames set causing this, at 13 ( mixed and not mixed) i dont see artifacting, when set to 14+ i get the blocking.

[Sirber]

Isn't max ref frames like 15 or something?

[Dreassica]

No it's 16

3 samples included now.

avs was just avisource loading ffdshow made huffie file, none of artifacts in source.

http://www.bigupload.com/d=3058EF54

[Sirber]

I hope this will make in mencoder :D

[akupenguin]

@Sirber:
pthread-win32-static will make it not work in mencoder, since it needs some initialization on application startup which mencoder doesn't do. If you link to pthreadGC2.dll or cygwin's pthread instead it will work, but then you need the dll.
I could fix that for mencoder by putting the pthread-win32-static initialization in libx264 instead of in x264cli, but that might break any applications (like x264cli) that do use threads themselves.
Someone who knows and cares more about windows than I do will have to resolve that.

@Dreassica:
On your broken sample, libavcodec says "reference picture missing during reorder", so at least I know it's not a macroblock-level error.
I can reproduce the error. (transcoding from your working sample, the exact source doesn't matter much as long as it uses refs a lot)

[Sirber]

@akupenguin

Thanks for the hints :)

[Thunderbolt8]

im not sure if I did it right (winxp sp2). I have 611 and medui 0.2.4.1007 and a core 2 duo, but somehow i dont seem to get a speed increase. i tried it 2 times: 1st time I just set threads in megui to 4 and 2nd time I set threads to 4 again and also added the command --enable-pthread in that custom command line. but speed stayed the same as it formerly was at 606 with 2 threads :/

a 2 pass test with some trimed file gave me a 10-15 fps increase for the 1st (turbo) pass, but 2nd pass even went down~half the value it should have with no other filters than just normal encoding.

[Sharktooth]

--enable-pthread is a configure switch useed for compilation only, it's not a x264 option.
What is your CPU?

[Thunderbolt8]

got a 6300er Core 2 Duo @3,2 GHz. the value of ~7fps was almost the same as with 606 when I had degrainmedian and/or fft3d filter inserted for example, that would make ~no speed increase at 2nd pass. but it also stays ~7fps when I use none of these filter settings, and then I had 15-18 fps with 606. so its either 0 increase or even a reduction.

[Mutant_Fruit]

I was just looking at the results.txt file and i'm a bit confused about the results.

It seems that PThreads are slower than Win32 threads if you use 1 thread per core, but if you use more than 1 thread per core you get a speed boost with PThreads as compared to Win32 threads.

The question i have is this: Isn't PThreads just a wrapper around Win32 threads? So why is that performance difference there? Shouldn't it be negligible at best?

[Sharktooth]

got a 6300er Core 2 Duo @3,2 GHz. the value of ~7fps was almost the same as with 606 when I had degrainmedian and/or fft3d filter inserted for example, that would make ~no speed increase at 2nd pass. but it also stays ~7fps when I use none of these filter settings, and then I had 15-18 fps with 606. so its either 0 increase or even a reduction.
Tonight i'll check.

[Thunderbolt8]

did another try, this time with 3 threads. but result is the same ~60 fps for 1st pass (when it was ~45-50 in 606), but only 7fps at 2nd, when it was >15fps with 606.

[Sharktooth]

it seems it's using only one thread for second pass.
can you post the logfile?

[Thunderbolt8]

apparently megui cant save my settings I do when configuring x264. this was the command line I had before I pressed ok:

--pass 2 --bitrate 1331 --stats ".stats" --ref 5 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -2,-1 --subme 6 --trellis 2 --analyse all --8x8dct --vbv-maxrate 25000 --me umh --threads 4 --thread-input --progress --no-dct-decimate --no-psnr --no-ssim --output "D:\DVD\Projects\OLD_BOY_VTS_01_PGC1\test trim.mp4"

and this I got at the beginning of the log:
Starting preprocessing of job...
Preprocessing finished!
Starting job job1-1 at 15:48:18
encoder commandline:
--pass 1 --bitrate 46031 --stats "D:\DVD\Projects\OLD_BOY_VTS_01_PGC1\test trim.stats" --bframes 3 --b-pyramid --direct auto --filter -2,-1 --subme 1 --analyse none --vbv-maxrate 25000 --me dia --threads 2 --thread-input --progress --no-dct-decimate --no-psnr --no-ssim --output NUL "D:\DVD\Projects\OLD_BOY_VTS_01_PGC1\test trim.avs"
successfully started encoding

bitrate 46031 was because its a trimed file, adjusted to 1/3 dvd size. I got 43~fps with that in 1st pass. usually, when not using the bitrate calculator, megui apparently used the standart 1000 bitrate, which then resulted in the ~60 fps for 1st pass.
already tried to attrib -s -h -r /s my megui folder in case theres any kind of file writing protection enabled, but nothing happened :S

[shon3i]

@Thunderbolt8, confirmed me to, but that is megui bug, only way to change this is, before ok button, to press update profile.

[Thunderbolt8]

which profile, you mean currently I have to create a new profile for each time im alternating one of the existing profiles, which I want to use then ?

[Sharktooth]

yes, create your own profile or update one that already comes with MeGUI.

[deets]

ive updated megui and it no longer sets the threads to auto, it just sets it to 2 and the threads are no longer greyed out.

im sure this wasnt the case on an earlier release?

[Thunderbolt8]

^^ I tried it now with the profile saving, he keeps the bitrate I enter now, but still sets threads automatically to 2, although I told him to use 4.

and btw. recently the small encoding status window keeps dissapearing. I just have the normal megui frontend and have to read the percentage values from the headline, but of course cant see any fps or time values any more (I mean I can fool him and cancel/retry encoding after 1st pass, but thats not the way it should work -.-)
with 2 threads he has ~10 fps now, which is still slower than it should be and also used to be in 606.

edit: that was fast, shark :p

edit: but he still doesnt keep all the settings in 1008 -.- and also sticks to 2 threads when having entered 4 threads in my own profile.

[squid808]

Even though multiple slice encoding introduces a slight bitrate penalty, isn't it still the threading method of choice when for example doing live encoding? What is the reason for removing it completely?

[Thunderbolt8]

maybe a stupid question, but is my/a C2D cpu able to use 4 threads at all (I mean if not, it would explain my problem) :S
but if not, why was akupenguin able to have 16 threads working with a dual xeon (4cpus) ?
Do I maybe have to dl some files enabling windows to use the 4 threads and I havent done that yet ?

[Sharktooth]

ive updated megui and it no longer sets the threads to auto, it just sets it to 2 and the threads are no longer greyed out.

im sure this wasnt the case on an earlier release?
read the notes: http://forum.doom9.org/showthread.php?p=918705#post918705

[Sharktooth]

maybe a stupid question, but is my/a C2D cpu able to use 4 threads at all (I mean if not, it would explain my problem) :S
but if not, why was akupenguin able to have 16 threads working with a dual xeon (4cpus) ?
Do I maybe have to dl some files enabling windows to use the 4 threads and I havent done that yet ?
a single core can just execute an indefinite number of threads... but all those threads will be executed in one core (serial processing).
if you have a multicore CPU those threads can be subdivided into each core and you will get parallel processing.

[LoRd_MuldeR]

maybe a stupid question, but is my/a C2D cpu able to use 4 threads at all (I mean if not, it would explain my problem) :S
but if not, why was akupenguin able to have 16 threads working with a dual xeon (4cpus) ?
Do I maybe have to dl some files enabling windows to use the 4 threads and I havent done that yet ?

You can run any number of threads on any type/number of CPU.
The big question is: What number of threads give best performance on your specific system?
Dualcores will need at least 2 threads for max. performance, Quadcores will need at least 4.
If you run more than one thread on Singlecore, it won't boost performance - maybe even hurt it.

[Thunderbolt8]

edit: trying, mom

edit2: :S seems like I was too fast and overlooked that.
1st pass now gives me the usual 60fps, even 65, which is allright. lets see what 2nd pass does...

2nd pass now also says 4 threads, but the speed doesnt increase, its again ~10 fps, which was equal to 1-2 threads (?) and definately too slow for no additional filtering options.

[Manao]

Even though multiple slice encoding introduces a slight bitrate penalty, isn't it still the threading method of choice when for example doing live encoding? What is the reason for removing it completely?The new threading algorithm is compatible with live streaming, but not with low delay. However, x264 itself, since it doesn't regulate its quality level to reach a certain speed, isn't really suitable for livestreaming.

[Thunderbolt8]

im encoding 2nd pass 4 threads with fft3d filter denoinsing, shaping and 3xdegrainmedian with only 2 frames less (~8 fps)than as with none (!) of these filter settings. theres something serioulsly wrong :S

[Sharktooth]

remove ANY filtering... and try again.
are you using Avisynth MT?

[Thunderbolt8]

remove ANY filtering... and try again.
are you using Avisynth MT?
did not use any filtering in all my previous attempts.

edit: guess MT means avysynth multithreading. so I have or better should not use that now for 4 threads with a C2D ? if yes, tell me as a complete newb of that MT part what to dl and type in the aviscript editor, so I could try it.

[Thunderbolt8]

hm, I updated my avisynth to 2.5.7. alpha/beta something
and dled MT 0.5 and added these 2 lines to my avisynth script:

LoadPlugin("C:\Programme\VIDEOBEARBEITUNG\AviSynth 2.5\plugins\MT.dll")
SetMTMode(2,4)

but I lost ~8 fps on 1st pass and 1,5 fps for 2nd pass -.-


is it really necessary to use avisynth MT to achieve that bonus if slice encoding now in 611?

edit: awwww, sry doublepost

[708145]

The new threading algorithm is compatible with live streaming, but not with low delay. However, x264 itself, since it doesn't regulate its quality level to reach a certain speed, isn't really suitable for livestreaming.

Adding that ain't difficult. It took me less than 2 hours to put that into ReStream264. ;)
So it's probably less than 10seconds for Loren :sly:

bis besser,
T0B1A5

[mean]

If i don't do

Index: encoder.c
===================================================================
--- encoder.c (revision 611)
+++ encoder.c (working copy)
@@ -1574,8 +1574,14 @@
goto do_encode;
}
}
-
- x264_encoder_frame_end( thread_oldest, thread_current, pp_nal, pi_nal, pic_out );
+ /* The first frames are delayed. The first non delayed frame will have an empty thread as oldest
+ * frame, so we cannot use them (null dereferencing).
+ * */
+ if(thread_oldest->fenc)
+ x264_encoder_frame_end( thread_oldest, thread_current, pp_nal, pi_nal, pic_out );
+ /*x264_encoder_frame_end( thread_oldest, thread_current, pp_nal, pi_nal, pic_out );*/
return 0;
}



I have a segfault when actually encoding the 1st frame, the thread_oldest being not initialised yet at that time.
With rev611

But it seems not a good fix as it should crash for everyone else too.

Any hint on what could be the problem ?

[giandrea]

There is a problem on Mac OS X Intel... the new multi threading doesn't work...
This command line works fine:
x264 -o harry_611_no_2t.mp4 --threads 1 --fps 25 --crf 20 --progress --sar 64:45 temp.yuv 720x576
This one doesn't work:
x264 -o harry_611_no_2t.mp4 --threads 2 --fps 25 --crf 20 --progress --sar 64:45 temp.yuv 720x576
I get this message:
encoded frames: 2/249 (0.8%), 39.87 fps, eta 0:00:06
and it is stuck, forever, I have to force quit the process in Process Viewer.
It may be something to do with pthreads?
I'm using the latest revision from svn (611) compiled with mp4 support...

[akupenguin]

I have a segfault when actually encoding the 1st frame, the thread_oldest being not initialised yet at that time.
thread_oldest is always initialized. Yes, the first frames will call x264_encoder_frame_end with thread_oldest->fenc == NULL. But it checks for thread_oldest->out.i_nal == 0, which should be an equivalent condition.

[Thunderbolt8]

well, what can I do about my problem now ?
the 2nd pass speed is even a bit slower than before.

[Audionut]

well, what can I do about my problem now ?
the 2nd pass speed is even a bit slower than before.

Using the same build. Must be a problem with your comp.

[Thunderbolt8]

hm, but others than using the autoupdate feature I did nothing :S

[Sharktooth]

:script: :logfile:

[Thunderbolt8]

next day, time to sleep now :p
I also tried it with a clean install now, I dled the files from sourceforge net again in another directory and let megui autoupdate itself. but the result was the same. after that I exchanged the dgindex&dgencode with v1.4.9.7, because I already made 1 video with that version, while the rest was still made with 1.4.8, but it gave me the same result. so that cant be the reason.

well, heres the log of the movie i made with dgindex etc. 1497 (not that it matters): I aborted the 2nd pass at 1st then when I saw it still doesnt work, but then let it run again, in order to have a complete log here (result is the same of course, as when letting the 2nd pass complete directly after the 1st one without having first interrupted , of course):

avisynth script:

# Set DAR in encoder to 4 : 3. The following line is for automatic signalling
global MeGUI_darx = 4
global MeGUI_dary = 3
DGDecode_mpeg2source("D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\HAPPY_TOGETHER.d2v",info=3)
ColorMatrix(hints=true)
trim(50000,55000)
#blank deinterlace line
#crop
#resize
#denoise


megui log:

Starting preprocessing of job...
Preprocessing finished!
Starting job job2-1 at 04:33:28
encoder commandline:
--pass 1 --bitrate 2008 --stats "D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\test.stats" --bframes 3 --b-pyramid --direct auto --filter -2,-1 --subme 1 --analyse none --vbv-maxrate 25000 --me dia --threads 4 --thread-input --progress --no-dct-decimate --no-psnr --no-ssim --output NUL "D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\test.avs"
successfully started encoding
Processing ended at 04:34:50
----------------------------------------------------------------------------------------------------------

Log for job job2-1

avis [info]: 720x576 @ 25.00 fps (5001 frames)
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2
x264 [warning]: VBV maxrate specified, but no bufsize.
x264 [info]: slice I:24 Avg QP:19.33 size: 45920
x264 [info]: slice P:1647 Avg QP:20.45 size: 17084
x264 [info]: slice B:3330 Avg QP:22.94 size: 6291
x264 [info]: mb I I16..4: 42.0% 0.0% 58.0%
x264 [info]: mb P I16..4: 18.1% 0.0% 0.0% P16..4: 59.1% 0.0% 0.0% 0.0% 0.0% skip:22.7%
x264 [info]: mb B I16..4: 1.5% 0.0% 0.0% B16..8: 20.7% 0.0% 0.0% direct:31.4% skip:46.4%
x264 [info]: final ratefactor: 21.29
x264 [info]: direct mvs spatial:98.9% temporal:1.1%
x264 [info]: kb/s:2007.2

encoded 5001 frames, 61.23 fps, 2007.28 kb/s

----------------------------------------------------------------------------------------------------------
Job completed successfully and deletion of intermediate files is activated
Starting postprocessing of job...
Postprocessing finished!
job job2-1 has been processed. This job is linked to the next job: job2-2
Starting preprocessing of job...
Preprocessing finished!
Starting job job2-2 at 04:34:50
encoder commandline:
--pass 2 --bitrate 2008 --stats "D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\test.stats" --ref 5 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -2,-1 --subme 6 --trellis 2 --analyse all --8x8dct --vbv-maxrate 25000 --me umh --threads 4 --thread-input --sar 16:15 --progress --no-dct-decimate --no-psnr --no-ssim --output "D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\test.mp4" "D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\test.avs"
successfully started encoding
Job aborted, deleting output file...Deletion failed.
Processing ended at 04:35:13
----------------------------------------------------------------------------------------------------------

Log for job job2-2

avis [info]: 720x576 @ 25.00 fps (5001 frames)
x264 [info]: using SAR=16/15
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2
x264 [warning]: VBV maxrate specified, but no bufsize.
mp4 [info]: initial delay 2 (scale 25)

----------------------------------------------------------------------------------------------------------
The current job was aborted. Stopping queue mode
The current job contains errors. Skipping chained jobs
Starting preprocessing of job...
Preprocessing finished!
Starting job job2-2 at 04:35:16
encoder commandline:
--pass 2 --bitrate 2008 --stats "D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\test.stats" --ref 5 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -2,-1 --subme 6 --trellis 2 --analyse all --8x8dct --vbv-maxrate 25000 --me umh --threads 4 --thread-input --sar 16:15 --progress --no-dct-decimate --no-psnr --no-ssim --output "D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\test.mp4" "D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\test.avs"
successfully started encoding
Processing ended at 04:43:50
----------------------------------------------------------------------------------------------------------

Log for job job2-2

avis [info]: 720x576 @ 25.00 fps (5001 frames)
x264 [info]: using SAR=16/15
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2
x264 [warning]: VBV maxrate specified, but no bufsize.
mp4 [info]: initial delay 2 (scale 25)
x264 [info]: slice I:24 Avg QP:18.46 size: 47672
x264 [info]: slice P:1647 Avg QP:20.17 size: 16629
x264 [info]: slice B:3330 Avg QP:22.38 size: 6521
x264 [info]: mb I I16..4: 26.2% 52.3% 21.4%
x264 [info]: mb P I16..4: 3.8% 6.1% 1.5% P16..4: 42.1% 18.5% 7.5% 0.5% 0.2% skip:20.0%
x264 [info]: mb B I16..4: 0.2% 0.4% 0.1% B16..8: 33.8% 1.6% 2.7% direct:11.1% skip:50.1%
x264 [info]: 8x8 transform intra:53.5% inter:69.6%
x264 [info]: direct mvs spatial:91.4% temporal:8.6%
x264 [info]: ref P 45.5% 25.6% 12.8% 9.7% 6.3%
x264 [info]: ref B 70.5% 17.5% 5.7% 4.1% 2.3%
x264 [info]: kb/s:2009.4

encoded 5001 frames, 9.75 fps, 2009.52 kb/s

desired video bitrate of this job: 2008 kbit/s - obtained video bitrate (approximate): 2011 kbit/s
----------------------------------------------------------------------------------------------------------
Job completed successfully and deletion of intermediate files is activated
Found intermediate output file 'D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\test.stats', deleting...
Deletion succeeded.Starting postprocessing of job...
Postprocessing finished!


compare it...61fps in 1st pass, which is really nice and an increase compared to 606, but the only 9 fps in 2nd...used to be >15 fps in 606.

[Sharktooth]

really weird. try using threads = 0 (auto) and see if it gets better.

[Audionut]

Threads setting in megui is broken. See megui bug thread.

[mean]

thread_oldest is always initialized. Yes, the first frames will call x264_encoder_frame_end with thread_oldest->fenc == NULL. But it checks for thread_oldest->out.i_nal == 0, which should be an equivalent condition.

In most cases yes, but i tend to think that when x264_encoder_headers is used to get the specific atom, the first_thread=thread_oldest->out->i_nal might not be zero
and thus leads to the aforementionned segfault.
Thanks.

[Thunderbolt8]

same movie as above, auto threads in settings disabled, in config = 0


Starting preprocessing of job...
Preprocessing finished!
Starting job job1-1 at 13:13:47
encoder commandline:
--pass 1 --bitrate 2008 --stats "D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\test.stats" --bframes 3 --b-pyramid --direct auto --filter -2,-1 --subme 1 --analyse none --vbv-maxrate 25000 --me dia --threads auto --thread-input --progress --no-dct-decimate --no-psnr --no-ssim --output NUL "D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\test.avs"
successfully started encoding
Processing ended at 13:15:09
----------------------------------------------------------------------------------------------------------

Log for job job1-1

avis [info]: 720x576 @ 25.00 fps (5001 frames)
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2
x264 [warning]: VBV maxrate specified, but no bufsize.
x264 [info]: slice I:24 Avg QP:19.33 size: 45920
x264 [info]: slice P:1647 Avg QP:20.45 size: 17087
x264 [info]: slice B:3330 Avg QP:22.94 size: 6291
x264 [info]: mb I I16..4: 42.0% 0.0% 58.0%
x264 [info]: mb P I16..4: 18.1% 0.0% 0.0% P16..4: 59.2% 0.0% 0.0% 0.0% 0.0% skip:22.7%
x264 [info]: mb B I16..4: 1.5% 0.0% 0.0% B16..8: 20.7% 0.0% 0.0% direct:31.4% skip:46.4%
x264 [info]: final ratefactor: 21.29
x264 [info]: direct mvs spatial:99.0% temporal:1.0%
x264 [info]: kb/s:2007.3

encoded 5001 frames, 60.83 fps, 2007.39 kb/s

----------------------------------------------------------------------------------------------------------
Job completed successfully and deletion of intermediate files is activated
Starting postprocessing of job...
Postprocessing finished!
job job1-1 has been processed. This job is linked to the next job: job1-2
Starting preprocessing of job...
Preprocessing finished!
Starting job job1-2 at 13:15:09
encoder commandline:
--pass 2 --bitrate 2008 --stats "D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\test.stats" --ref 5 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -2,-1 --subme 6 --trellis 2 --analyse all --8x8dct --vbv-maxrate 25000 --me umh --threads auto --thread-input --sar 16:15 --progress --no-dct-decimate --no-psnr --no-ssim --output "D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\test.mp4" "D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\test.avs"
successfully started encoding
Processing ended at 13:23:40
----------------------------------------------------------------------------------------------------------

Log for job job1-2

avis [info]: 720x576 @ 25.00 fps (5001 frames)
x264 [info]: using SAR=16/15
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2
x264 [warning]: VBV maxrate specified, but no bufsize.
mp4 [info]: initial delay 2 (scale 25)
x264 [info]: slice I:24 Avg QP:18.42 size: 47918
x264 [info]: slice P:1647 Avg QP:20.15 size: 16670
x264 [info]: slice B:3330 Avg QP:22.39 size: 6498
x264 [info]: mb I I16..4: 26.1% 52.2% 21.7%
x264 [info]: mb P I16..4: 3.8% 6.1% 1.5% P16..4: 42.1% 18.5% 7.5% 0.5% 0.2% skip:19.9%
x264 [info]: mb B I16..4: 0.2% 0.4% 0.1% B16..8: 33.8% 1.6% 2.7% direct:11.0% skip:50.1%
x264 [info]: 8x8 transform intra:53.4% inter:69.6%
x264 [info]: direct mvs spatial:91.3% temporal:8.7%
x264 [info]: ref P 45.5% 25.6% 12.8% 9.7% 6.3%
x264 [info]: ref B 70.5% 17.6% 5.6% 4.1% 2.3%
x264 [info]: kb/s:2009.4

encoded 5001 frames, 9.80 fps, 2009.45 kb/s

desired video bitrate of this job: 2008 kbit/s - obtained video bitrate (approximate): 2011 kbit/s
----------------------------------------------------------------------------------------------------------
Job completed successfully and deletion of intermediate files is activated
Found intermediate output file 'D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\test.stats', deleting...
Deletion succeeded.Starting postprocessing of job...
Postprocessing finished!



threads in settings enabled, threads=0


Starting preprocessing of job...
Preprocessing finished!
Starting job job1-1 at 13:24:49
encoder commandline:
--pass 1 --bitrate 2008 --stats "D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\test.stats" --bframes 3 --b-pyramid --direct auto --filter -2,-1 --subme 1 --analyse none --vbv-maxrate 25000 --me dia --threads 2 --thread-input --progress --no-dct-decimate --no-psnr --no-ssim --output NUL "D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\test.avs"
successfully started encoding
Processing ended at 13:26:14
----------------------------------------------------------------------------------------------------------

Log for job job1-1

avis [info]: 720x576 @ 25.00 fps (5001 frames)
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2
x264 [warning]: VBV maxrate specified, but no bufsize.
x264 [info]: slice I:24 Avg QP:19.29 size: 46145
x264 [info]: slice P:1647 Avg QP:20.45 size: 17080
x264 [info]: slice B:3330 Avg QP:22.94 size: 6292
x264 [info]: mb I I16..4: 42.0% 0.0% 58.0%
x264 [info]: mb P I16..4: 18.1% 0.0% 0.0% P16..4: 59.1% 0.0% 0.0% 0.0% 0.0% skip:22.8%
x264 [info]: mb B I16..4: 1.5% 0.0% 0.0% B16..8: 20.7% 0.0% 0.0% direct:31.4% skip:46.4%
x264 [info]: final ratefactor: 21.29
x264 [info]: direct mvs spatial:98.9% temporal:1.1%
x264 [info]: kb/s:2007.2

encoded 5001 frames, 59.55 fps, 2007.25 kb/s

----------------------------------------------------------------------------------------------------------
Job completed successfully and deletion of intermediate files is activated
Starting postprocessing of job...
Postprocessing finished!
job job1-1 has been processed. This job is linked to the next job: job1-2
Starting preprocessing of job...
Preprocessing finished!
Starting job job1-2 at 13:26:14
encoder commandline:
--pass 2 --bitrate 2008 --stats "D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\test.stats" --ref 5 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -2,-1 --subme 6 --trellis 2 --analyse all --8x8dct --vbv-maxrate 25000 --me umh --threads 2 --thread-input --sar 16:15 --progress --no-dct-decimate --no-psnr --no-ssim --output "D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\test.mp4" "D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\test.avs"
successfully started encoding
Processing ended at 13:35:55
----------------------------------------------------------------------------------------------------------

Log for job job1-2

avis [info]: 720x576 @ 25.00 fps (5001 frames)
x264 [info]: using SAR=16/15
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2
x264 [warning]: VBV maxrate specified, but no bufsize.
mp4 [info]: initial delay 2 (scale 25)
x264 [info]: slice I:24 Avg QP:18.46 size: 47701
x264 [info]: slice P:1647 Avg QP:20.15 size: 16682
x264 [info]: slice B:3330 Avg QP:22.40 size: 6490
x264 [info]: mb I I16..4: 26.3% 51.6% 22.1%
x264 [info]: mb P I16..4: 3.8% 6.1% 1.5% P16..4: 42.1% 18.5% 7.5% 0.5% 0.2% skip:19.9%
x264 [info]: mb B I16..4: 0.2% 0.4% 0.1% B16..8: 33.7% 1.6% 2.7% direct:11.0% skip:50.2%
x264 [info]: 8x8 transform intra:53.4% inter:69.6%
x264 [info]: direct mvs spatial:91.3% temporal:8.7%
x264 [info]: ref P 45.6% 25.6% 12.7% 9.7% 6.4%
x264 [info]: ref B 70.5% 17.5% 5.7% 4.1% 2.3%
x264 [info]: kb/s:2008.8

encoded 5001 frames, 8.62 fps, 2008.90 kb/s

desired video bitrate of this job: 2008 kbit/s - obtained video bitrate (approximate): 2011 kbit/s
----------------------------------------------------------------------------------------------------------
Job completed successfully and deletion of intermediate files is activated
Found intermediate output file 'D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\test.stats', deleting...
Deletion succeeded.Starting postprocessing of job...
Postprocessing finished!

[Sharktooth]

Audionut: the log shows threads work as expected in megui. check the notes about version 0.2.4.1008 in the development thread.

Thunderbolt8: try using x264 from the command prompt using the same exact settings (copy/paste the command line options). if the results are identical to what you get in MeGUI then it's something about x264 or your system.

[Thunderbolt8]

any idea how I can do this best ? I have to type everything manually in command window (link with exe and copy&paste doesnt work, space is too short) and doskey also doesnt work. if I just do 1 little mistake in that mega line I'll have to type everything again.

[deets]

create a batch file.

just put it in a text file with x264.exe at the start, then rename test.bat, put in folder with x264.exe and double click

[Thunderbolt8]

copied the whole line

--pass 2 --bitrate 2008 --stats ".stats" --ref 5 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -2,-1 --subme 6 --trellis 2 --analyse all --8x8dct --vbv-maxrate 25000 --me umh --threads auto --thread-input --progress --no-dct-decimate --no-psnr --no-ssim --output "D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\test.mp4" "D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\test.avs"

but I get an x264 error saying:

[warning] vbv maxrate specified, but no bufsize.
[error] ratecontrol_init_ can't open stats file
[error] x264_encoder_open failed

[deets]

x264.exe --pass 1 --bitrate 2008 --stats "D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\test.stats" --bframes 3 --b-pyramid --direct auto --filter -2,-1 --subme 1 --analyse none --vbv-maxrate 25000 --me dia --threads 2 --thread-input --progress --no-dct-decimate --no-psnr --no-ssim --output NUL "D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\test.avs"

x264.exe --pass 2 --bitrate 2008 --stats ".stats" --ref 5 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -2,-1 --subme 6 --trellis 2 --analyse all --8x8dct --vbv-maxrate 25000 --me umh --threads auto --thread-input --progress --no-dct-decimate --no-psnr --no-ssim --output "D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\test.mp4" "D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\test.avs"

try that. you didnt have the first pass it seems?

[Thunderbolt8]

x264.exe --pass 1 --bitrate 2008 --stats "D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\test.stats" --bframes 3 --b-pyramid --direct auto --filter -2,-1 --subme 1 --analyse none --vbv-maxrate 25000 --me dia --threads 2 --thread-input --progress --no-dct-decimate --no-psnr --no-ssim --output NUL "D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\test.avs"

x264.exe --pass 2 --bitrate 2008 --stats ".stats" --ref 5 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -2,-1 --subme 6 --trellis 2 --analyse all --8x8dct --vbv-maxrate 25000 --me umh --threads auto --thread-input --progress --no-dct-decimate --no-psnr --no-ssim --output "D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\test.mp4" "D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\test.avs"

try that. you didnt have the first pass it seems? I have automated 2nd pass in megui selected, 1st pass is turbo. dunno what the command line made of it, I just copied it.

when I enter that as: C:\Programme\VIDEOBEARBEITUNG\MeGUI\tools\x264\x264.exe x264.exe --pass 1 --bitrate 2008 --stats "D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\test.stats" --bframes 3 --b-pyramid --direct auto --filter -2,-1 --subme 1 --analyse none --vbv-maxrate 25000 --me dia --threads 2 --thread-input --progress --no-dct-decimate --no-psnr --no-ssim --output NUL "D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\test.avs"

x264.exe --pass 2 --bitrate 2008 --stats ".stats" --ref 5 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -2,-1 --subme 6 --trellis 2 --analyse all --8x8dct --vbv-maxrate 25000 --me umh --threads auto --thread-input --progress --no-dct-decimate --no-psnr --no-ssim --output "D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\test.mp4" "D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\test.avs"

then I get just the same error. when entering it that way:

C:\Programme\VIDEOBEARBEITUNG\MeGUI\tools\x264\x264.exe x264.exe --pass 1 --bitrate 2008 --stats "D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\test.stats" --bframes 3 --b-pyramid --direct auto --filter -2,-1 --subme 1 --analyse none --vbv-maxrate 25000 --me dia --threads 2 --thread-input --progress --no-dct-decimate --no-psnr --no-ssim --output NUL "D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\test.avs"

C:\Programme\VIDEOBEARBEITUNG\MeGUI\tools\x264\x264.exe x264.exe --pass 2 --bitrate 2008 --stats ".stats" --ref 5 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -2,-1 --subme 6 --trellis 2 --analyse all --8x8dct --vbv-maxrate 25000 --me umh --threads auto --thread-input --progress --no-dct-decimate --no-psnr --no-ssim --output "D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\test.mp4" "D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\test.avs"

then I get another error message after both pass' command lines saying:
x264 [error]: rawyuv input requires a resolution.

[emmel]

You have "x264.exe" twice in the beginnig of your line. Take one of them away. Or is the first one the folders' name?

Anyways, maybe simplifying the script by removing most of the options would help in the first place to make it work. Then, add them gradually to get the desired result.

[Thunderbolt8]

the ..tools\x264\x264.exe is the folder name in megui.
I cant take the options away, since then I'd lose the basis of my comparism.

[emmel]

the ..tools\x264\x264.exe is the folder name in megui.

I think the folder is called tools\x264 and in there you have x264.exe At least here it is so. Try removing the second x264.exe from your line.

[desta]

Sorry for asking a question which has already been answered, but it seemed some people's answers contradicted others...

Anyway, for a single core cpu, would "--threads auto" be better, worse, or no different than just using 1 thread?

[Dreassica]

Tried same revision on my core duo laptop on a different source, no filters either, just lossless avi loaded through avs, same artifacting.

[Thunderbolt8]

I think the folder is called tools\x264 and in there you have x264.exe At least here it is so. Try removing the second x264.exe from your line.
rofl, sorry, didnt see my error at 1st. of course I had the .exe twice in :S
its encoding right now, results come soon...

edit: 1st pass gave me 58,5 fps for threads = auto, but I got the error message for 2nd pass again, this one:

[warning] vbv maxrate specified, but no bufsize.
[error] ratecontrol_init_ can't open stats file
[error] x264_encoder_open failed
and the windows error message

maybe it has to be --stats "test.stats" instead of just --stats ".stats" for 2nd pass ? *testing*

edit: nope, same error -.-

[emmel]

You'll have to specify the stats file name as in the first pass.

[Thunderbolt8]

asdfölasjkdf
didint notice deets just shortened the path there

be gentle with me and not so fast please, im a command line newb :p

edit: its working, but im getting my usual 9 fsp :(( as with megui

edit2: changed to "--threads 4" for both pathes now, 1st pass is allright with ~60 fps, but 2nd pass I only get my 10 fps again ://


so its either an internal x264 error or an error of my comp now ?

[DeathTheSheep]

Keyframe autodetection is completely broken. Not a single keyframe was properly autodetected, and all of my file's resulting keyframes were placed exactly 1500 frames apart (I set this ridiculous maximum so x264 can autodetect keyframes in my high-scenecut video).

Is this just my problem? I'm using a simple avisource() with a video file on which x264 autodetects keyframes fine with r606.

[Sharktooth]

asdfölasjkdf
didint notice deets just shortened the path there

be gentle with me and not so fast please, im a command line newb :p

edit: its working, but im getting my usual 9 fsp :(( as with megui

edit2: changed to "--threads 4" for both pathes now, 1st pass is allright with ~60 fps, but 2nd pass I only get my 10 fps again ://


so its either an internal x264 error or an error of my comp now ?
Then megui is not related with your problem.

[Thunderbolt8]

so what can I do ?
I already dled the bins from sourceforge and let it update again in another directory, but same problem with that version. maybe its an internal x264 problem ?

[Sharktooth]

i already said it's not related to megui... so downloading megui binaries from SF wont fix your issue.
the problem is related to some other thing (maybe your system or x264).

[experttech]

im just updating megui... i removed the ability to set the number of threads within the x264 profile (updown control grayed out).
so --threads auto will be enforced when "Automatically set the number of threads" global option is enabled in megui settings.
It's just a workaround but should work as expected.

Even after unchecking this option I am not able to set the number of threads - its disabled. I am using your latest build of x264 and latest version of MeGUI.

And I am unable to go above 3 fps for any encodes. Something wrong :confused:

And my CPU utilization does not go beyong 50% so I am pretty sure the dual cores are not being used.

Oh yes I figure this is an issue with this build as I have another VFW build (http://gabextreme.googlepages.com/x264vfwunited) which I use with Virtualdub and it gives me ~40 fps for the same video and settings.

[Dreassica]

uploaded a sample of the video which has corruption

http://rapidshare.com/files/8271637/sample_corruption.mp4.html

As i said before, avs script was just a single avisource call on a lossless ffdshow made huffy, but same thing happened on 3 different sources lossles and with dgdecode andf filters applied, on my core duo laptop as well as on my core 2 duo desktop.

[bob0r]

@Dreassica

The reference frames problem should be solved in revision 614.
Thanks for testing and reporting!

[DeathTheSheep]

Any updates in regard to scene change detection (here, planned, or in planning)?

[Dreassica]

@Dreassica

The reference frames problem should be solved in revision 614.
Thanks for testing and reporting!

THanks, that seemde to have done the trick, no more artifacting. Thanks :)
btw, i seem to get a warnign msg with latest few builds of x264 when its starts, namely:
"x264 [warning]: threads are not yet compatible with ESA", do I have to be worried over this?

[Thunderbolt8]

with 614 my 2nd pass even dropped to 7,5 fps -.-

[experttech]

with 614 my 2nd pass even dropped to 7,5 fps -.-

I am facing the same issues (slowness). Try it with XVid instead of x264..what do you get?

[Thunderbolt8]

cant compare, ive never ever used xvid so i dont have any reference values

[experttech]

cant compare, ive never ever used xvid so i dont have any reference values

Ok, I think *may* be a MeGUI problem as my fps with XVid is also too low.

Whats your processor/mobo?

[experttech]

See this as well:

http://forum.doom9.org/showthread.php?t=119600

[Thunderbolt8]

sharktooth said some pages before its most likely not a megui problem.
ive got a core 2 duo 6300 @3,2GHz, gigabyte DQ6

[experttech]

sharktooth said some pages before its most likely not a megui problem.
ive got a core 2 duo 6300 @3,2GHz, gigabyte DQ6

Well, from my findings:

x264 (614) + megui (latest) = low fps = BAD
XVid + megui (latest) = low fps = BAD

x264 (VFW version) + VirtualDub 1.7.0 = high fps = GOOD
XVid + VirtualDub 1.7.0 = high fps = GOOD

If you need any specifics please let me know. The profiles do not matter - happens with most.

I am confused as to where the problem is. I think there might be a bug in setting the threads automatically.

What's your CPU usage?

[Sharktooth]

x264VFW is way OUTDATED.
MeGUI IS NOT THE CAUSE. Try with x264 CLI ... NOT VFW, NO VIRTUALDUB and NO SH!T!!!!!!!!!!!!

[Thunderbolt8]

does our problem actually apply to all c2d users ?

[Sharktooth]

He just has an outdated version of MeGUI.

[Thunderbolt8]

but I dont have :S

only thing I still could try to reinstall my windows soon, since I just copied my primary partition when I got a new comp, instead of formating. but on the other hand, it also worked well until 606, so I dont know :S

[Sharktooth]

if the number of threads are correctly set by megui (as shown in your log) then the problems lies in another place.
while experttech said (in another thread) the x264 config threads option is disabled (grayed out) and that true only for ver. 0.2.3.2193, that's outdated. So he must be using an older MeGUI version and still insisting on that phantomatic MeGUI problem...

[experttech]

if the number of threads are correctly set by megui (as shown in your log) then the problems lies in another place.
while experttech said (in another thread) the x264 config threads option is disabled (grayed out) and that true only for ver. 0.2.3.2193, that's outdated. So he must be using an older MeGUI version and still insisting on that phantomatic MeGUI problem...

Ok thanks, I'll try updating MeGUI and try again.

[experttech]

I am using MeGUI 0.2.4.1009

I am going to do another auto update and post my findings.

[experttech]

pass1:

encoder commandline:
--pass 1 --bitrate 1000 --stats "I:\Tape No 1\WC.stats" --bframes 3 --b-pyramid --subme 1 --analyse none --vbv-maxrate 25000 --me dia --threads 2 --thread-input --progress --no-psnr --no-ssim --output NUL "I:\Tape No 1\WC.avs"
successfully started encoding


Still getting the same startign fps which will drop to 3-4 fps in sometime. CPU usage is still around 50%

See screenshot.

[experttech]

FPS is down to 7 as I type this. It settles around 3 fps and remains at that. So it takes about 10 hours for 2 passes - whereas it should take not more than 1.5 hrs at this setting.

Video is DV AVI ~30 mins.

[Sharktooth]

i dont care about the FPS. Post a MeGUI log like the Thunderbolt8 one (with and without the "Automatically set the number of frames" option).

[experttech]

Log so far: (Pass1 is in progress) - Automatically set threads is checked in MeGUI settings (Even then I can manually set the # of threads but I didnt)

Generating jobs. No desired size.
User doesn't care what the filesize is. Leaving bitrate/qp/crf at the profile's valueStarting preprocessing of job...
Preprocessing finished!
Starting job job1-1 at 1:28:12 PM
encoder commandline:

successfully started encoding
Processing ended at 1:30:36 PM
----------------------------------------------------------------------------------------------------------

Log for job job1-1

Channels=2, BitsPerSample=16, SampleRate=48000Hz
D:\Utils\neroaacenc\win32\neroAacEnc.exe -ignorelength -lc -br 96000 -if - -of "I:\Tape No 1\Wine_Country.mp4"*************************************************************
* *
* Nero Digital Audio Reference MPEG-4 & 3GPP Audio Encoder *
* Copyright (C) 2006 Nero AG *
* All Rights Reserved Worldwide *
* *
* Package build date: May 26 2006 *
* *
* See -help for a complete list of available parameters. *
* *
*************************************************************




----------------------------------------------------------------------------------------------------------
Job completed successfully and deletion of intermediate files is activated
Starting postprocessing of job...
Postprocessing finished!
job job1-1 has been processed. This job is linked to the next job: job1-2
Starting preprocessing of job...
Preprocessing finished!
Starting job job1-2 at 1:30:36 PM
encoder commandline:
--pass 1 --bitrate 1000 --stats "I:\Tape No 1\WC.stats" --bframes 3 --b-pyramid --subme 1 --analyse none --vbv-maxrate 25000 --me dia --threads 2 --thread-input --progress --no-psnr --no-ssim --output NUL "I:\Tape No 1\WC.avs"
successfully started encoding

[Sharktooth]

--pass 1 --bitrate 1000 --stats "I:\Tape No 1\WC.stats" --bframes 3 --b-pyramid --subme 1 --analyse none --vbv-maxrate 25000 --me dia --threads 2 --thread-input --progress --no-psnr --no-ssim --output NUL "I:\Tape No 1\WC.avs"
That indicates MeGUI automatically set the number of threads and that's not a good option for x264.
Uncheck "Automatically set the number of threads" option and manually set threads = 0 i the x264 GUI (if you leave it checked the threads option will be ignored).

[foxyshadis]

Are you comparing with exactly the same avisynth script?

[experttech]

Ok, unchecked that option in MeGUI settings and clicked on x264 Config and set number of threads to zero, clicked on Update (Note: If Update button is not clicked the 'number of threads' rolls back to 1)

Log:

Log for job job1-2

avis [info]: 720x480 @ 29.97 fps (25335 frames)
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2 3DNow!
x264 [warning]: VBV maxrate specified, but no bufsize.

----------------------------------------------------------------------------------------------------------
The current job was aborted. Stopping queue mode
The current job contains errors. Skipping chained jobs
Starting preprocessing of job...
Preprocessing finished!
Starting job job1-2 at 2:01:03 PM
encoder commandline:
--pass 1 --bitrate 1000 --stats "I:\Tape No 1\WC.stats" --bframes 3 --b-pyramid --subme 1 --analyse none --vbv-maxrate 25000 --me dia --threads 2 --thread-input --progress --no-psnr --no-ssim --output NUL "I:\Tape No 1\WC.avs"
successfully started encoding



So it still sets threads to 2??

[experttech]

Are you comparing with exactly the same avisynth script?

Yes, I just aborted the job and queued it again.

[pinkie_1]

I'm sorry for this intrusion into the world of x2/c2d owners, but I'm need of some feedback/confirmation.
Is there any (good) reason for switching from rev 602 to rev 610+ if I only have a lousy Barton-cored Athlon XP ?
-quality wise : minor penalty (but still a penalty)
-speed wise : ???

As for myself, I will try to do a more thorough testing using different revs... but only God knows when I'll be able to.

[Sharktooth]

Ok, unchecked that option in MeGUI settings and clicked on x264 Config and set number of threads to zero, clicked on Update (Note: If Update button is not clicked the 'number of threads' rolls back to 1)

Log:

Log for job job1-2

avis [info]: 720x480 @ 29.97 fps (25335 frames)
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2 3DNow!
x264 [warning]: VBV maxrate specified, but no bufsize.

----------------------------------------------------------------------------------------------------------
The current job was aborted. Stopping queue mode
The current job contains errors. Skipping chained jobs
Starting preprocessing of job...
Preprocessing finished!
Starting job job1-2 at 2:01:03 PM
encoder commandline:
--pass 1 --bitrate 1000 --stats "I:\Tape No 1\WC.stats" --bframes 3 --b-pyramid --subme 1 --analyse none --vbv-maxrate 25000 --me dia --threads 2 --thread-input --progress --no-psnr --no-ssim --output NUL "I:\Tape No 1\WC.avs"
successfully started encoding



So it still sets threads to 2??
The update button MUST be clicked or the profile will not be updated (that's a wanted behaviour).
For what concerns the threads number im assuming the megui config was not updated.
Please stop your encoding, verify the Auto set... etc. option is disabled, clean up your queue, close MeGUI, restart MeGUI, verify the Auto set.. blahblah.. is disabled (again), restart encoding and post the logfile (again).

@pinkie_1: actually no. but there could be new features, optimizations and bugfixes in future versions.

[experttech]

Did everything you said and verified settings are indeed reflected in settings.xml before restarting. Added jobs to queue again

profile settings:

--pass 2 --bitrate 1000 --stats ".stats" --ref 3 --mixed-refs --bframes 3 --b-pyramid --b-rdo --bime --weightb --subme 6 --trellis 1 --analyse p8x8,b8x8,i4x4,p4x4 --vbv-maxrate 25000 --threads auto --thread-input --progress --no-psnr --no-ssim --output "I:\Tape No 1\WC.mp4" "I:\Tape No 1\WC.avs"


Log:

Starting preprocessing of job...
Preprocessing finished!
Starting job job1-1 at 2:32:56 PM
encoder commandline:
--pass 1 --bitrate 1000 --stats "I:\Tape No 1\WC.stats" --bframes 3 --b-pyramid --subme 1 --analyse none --vbv-maxrate 25000 --me dia --threads auto --thread-input --progress --no-psnr --no-ssim --output NUL "I:\Tape No 1\WC.avs"
successfully started encoding


So it seems to be setting the threads fine now.

However CPU utilization is still ~ 50% and fps is still ~10

Edit: fps down to 7 now and will go down till 2-3 in sometime. Very peculiar behavior. No one else facing something similar?

[Sharktooth]

ok... now you have the same problem as Thunderbolt8... and that is no longer related to MeGUI.
Maybe it's related to x264 or maybe the x264 build...
Can you try using the x264.nl build of x264CLI in place of the one "bundled" with megui?

EDIT: Wait.. maybe i found the problem...

[experttech]

x264.nl build is slightly faster (very slightly, only a few fps) but same issue - 50% CPU utilization and fpt is ~8 - 13.

What else can we try?

[Sharktooth]

how much is that "slightly"?

[experttech]

how much is that "slightly"?

About 2-3 fps better with the nl build. But with so much variation int he fps this might not be a good criteria. What I mean to say is that there isnt a vast difference and the pattern is: FPS at the start is ~13 and then comes down to ~3-7 and stays there. I'm watching this (.nl) encode and its @8fps for a few mins now.

[Sharktooth]

try updating MeGUI again... there's a new x264 build...

[experttech]

Updated, it got rev614-1

Not much difference, fps is at 9 and CPU usage is ~50 - 60 - same as what was before.

[Sharktooth]

weird...

[experttech]

Yes its weird. And the log is:

encoder commandline:
--pass 1 --bitrate 1000 --stats "I:\Tape No 1\WC.stats" --bframes 3 --b-pyramid --subme 1 --analyse none --vbv-maxrate 25000 --me dia --threads auto --thread-input --progress --no-psnr --no-ssim --output NUL "I:\Tape No 1\WC.avs"
successfully started encoding

Could it be because of the video content? This is DV avi and not MPEG2 or it does not matter?

[foxyshadis]

Yes, I just aborted the job and queued it again.

I actually meant between megui and virtualdub. Your first pass speeds indicate it may be limited by the avisynth script. (If it's using the tivtc/eedi2 high quality deinterlacing, that's probably the case.)

[experttech]

I actually meant between megui and virtualdub. Your first pass speeds indicate it may be limited by the avisynth script. (If it's using the tivtc/eedi2 high quality deinterlacing, that's probably the case.)

Oh Ok, no I have never used AVisynth with virtualdub. Good point, however I can try. Will the AVS work with Virtualdub or do I need Virtualdubmod?

I dont encode too may videos so I am not a guru - but even with the hi-q de-interlacing, should it take 60 mins for a 1 pass for 25335 frames of video.

[Sharktooth]

get process explorer (:search:), launch it.
right click on the right pan on the columns title areas.
click on "select columns..." and go to the "Process Performance" tab. Add "threads" and start encoding.
Look for x264 and watch the number of threads. they should be at least 3. Maybe post a screenshot...

[experttech]

Its actually 7 threads for x264 and CPU utilization is ~50% (54 in the screenshot)

See screenshot

[Sharktooth]

everything seems ok then...

[experttech]

everything seems ok then...

So

1. Despite 7 threads, both CPU cores are not being utilized? Shouldnt CPu usage be at 100%?

2. Does it really take that long to encode a DV with that # of frames?

[MatMaul]

Do you use FFT3DGPU in your avs script ?

[ChronoCross]

There's something wrong with your setup experttech. I can easily max out both cores on my encoding box.

[experttech]

This is my avs file:


DirectShowSource("I:\Tape No 1\Wine_Country.avi",fps=29.97000002997,audio=false)
edeintted = last.AssumeBFF().SeparateFields().SelectEven().EEDI2(field=-1)
TDeint(order=0,full=false,edeint=edeintted)
#crop
#resize
#denoise

ConvertToYV12()

[experttech]

There's something wrong with your setup experttech. I can easily max out both cores on my encoding box.

Maybe and that's what I am trying to understand. Here are my x264 settings (see screenshots)

[Thunderbolt8]

1st pass ok as usual


2nd pass:

4 threads: ~8,5 fps
8 threads: ~8,5 fps
16 threads: ~8,5 fps
threads = auto (=2): ~7,5 fps

apparently threads = auto always seems to be =2. apart from that the problem is not solved for me, but according to task manager at least my cpu usage is ~100% all the time -.-

oh, and that status window STILL disappears sometimes when starting the 2nd pass. only thing I can do it to get it back is to cancel & restart the 2nd pass.

[experttech]

Here's the speed of pass 1 attached

[experttech]

Ok I tried to encode the same video on my laptop. Its a Core2 Duo. Again its very slow - same settings, same everything. Its a slower machine (1.66 Ghz) and the fps is about 2-3 !

So could it be the source thats creating the slowness? Could it be that the encoder is not able to max out the cores while encoding a DV AVI?

Can someone post experience of encoding a DV AVI?

[Thunderbolt8]

mb akupeguin did the adjustment for sliceless encoding with a quad core cpu and mb this now affects the c2d with 2 threads really badly ?
ot mb its a general windows problem thats happening now ?

[Sharktooth]

dunno... try with another possibly progressive source

[Thunderbolt8]

progressive = PAL with 25 fps ?
my test video was already pal :S

[experttech]

progressive = PAL with 25 fps ?
my test video was already pal :S

progressive = not interlaced

[Thunderbolt8]

so pal = not interlaced or is it more difficult ? :S

[experttech]

so pal = not interlaced or is it more difficult ? :S

Both NTSC and PAL can be interlaced or progressive. We are deviating from the topic of discussion so I'd recommend reading the stickies.

In short, try getting a video that does not comes up as Interlaced when you do the 'Analyze' in AVISynth.

[Thunderbolt8]

tests with a definately progressive source gave ~ same results (+1 fps) yay

[Sharktooth]

something weird is happening... x264 ignores the --threads parameter...
i tried passing --threads 1 and --threads 16.... the number of x264 threads reported by process explorer was unchanged...

[Thunderbolt8]

ye, thats what Im also thinking, because the speed of 2nd pass is always ~same as with auto = 2 threads.
1st pass is allright though, its only a 2nd pass problem.

i tried passing --threads 1 and --threads 16.... the number of x264 threads reported by process explorer was unchanged...
if you mean the log window, my number changes there, when entered manually, but as already said not the speed :S

[Sharktooth]

i tried it on a 1 pass encode... maybe the problem lies o the FINAL pass.

[experttech]

If its ignoring the --threads param it should be for both passes I think

Nice to see we found out something.

[Thunderbolt8]

as I already said..yes :p
as I already said 1st pass (in 2 pass mode) has always been alright and improved with > 606 for me

[experttech]

ye, thats what Im also thinking, because the speed of 2nd pass is always ~same as with auto = 2 threads.
1st pass is allright though, its only a 2nd pass problem.


if you mean the log window, my number changes there, when entered manually, but as already said not the speed :S

He's referring to the Task manager Thread count for the x264 process

[Sharktooth]

however i updated megui so the OK button in the codec config window does something now... :D
you no longer need to click the Update button before the OK button... MeGUI will ask if you want to update the selected profile...

[experttech]

however i update megui so the OK button in the codec config window does something now... :D
you no longer need to click the Update button before the OK button... MeGUI will ask if you want to update the selected profile...

Much more intuitive, thanks!

[Thunderbolt8]

thats rather anoying, cause he also asks when you have changed nothing.

[giandrea]

I tried to encode your interlaced source with x264 and crf = 20. If I do not use --interlaced I get a higher quality (SSIM and PSNR) and a slightly lower bitrate.

Additionally the interlaced encode is not decoded properly (it makes some strange stripes) with mplayer.

I encoded it with last rev (614) with 3 threads on Mac OS X.

[Sharktooth]

thats rather anoying, cause he also asks when you have changed nothing.
well... one more click wont kill you.. ;)

[experttech]

I tried to encode your interlaced source with x264 and crf = 20. If I do not use --interlaced I get a higher quality (SSIM and PSNR) and a slightly lower bitrate.

Additionally the interlaced encode is not decoded properly (it makes some strange stripes) with mplayer.

I encoded it with last rev (614) with 3 threads on Mac OS X.

Whats ur processor/# of cores and the fps you get in 1st pass and 2 pass? Try to put only the deinterlacing filter (default (best) in avisynth) and tell us?

[foxyshadis]

I knew you were using the HQ deinterlace. Please, use the pre-rendering option in MeGUI, if you have the disk space; that's what it's there for. HQ deinterlace is single-threaded and about 1/10 to 1/5 the speed of the regular deinterlacing, it's by far the biggest bottleneck.

[experttech]

I knew you were using the HQ deinterlace. Please, use the pre-rendering option in MeGUI, if you have the disk space; that's what it's there for. HQ deinterlace is single-threaded and about 1/10 to 1/5 the speed of the regular deinterlacing, it's by far the biggest bottleneck.

Ok, will do it now. THanks. didnt know about that one. SO I keep all settings exactly the same and add the pre-rendering job right? Doing it now...

[experttech]

Jobs look like this now with Pre rendering

http://img239.imageshack.us/my.php?image=meguivb1.png

[experttech]

Holy shit...it just added 3 hrs of another job at 2 fps???

http://img86.imageshack.us/my.php?image=megui2xj0.png

[Sharktooth]

use http://www.imageshack.us/ for images otherwise we should wait until a mod approves the attachments

[experttech]

Done!

[Sharktooth]

The avs script was definatly slowing down the whole encoding.
Once the pre-rendering has finished the same pre-encoded file will be used for both passes and the speed should be higher.

[experttech]

The avs script was definatly slowing down the whole encoding.
Once the pre-rendering has finished the same pre-encoded file will be used for both passes and the speed should be higher.

Ok, but still 90 minutes pre-encoding time for a 30 min video - is that normal? Original video size is 3 GB

CPU utilization again is ~50% for the job

Settings:

http://img113.imageshack.us/my.php?image=mg1ex6.png
http://img221.imageshack.us/my.php?image=mg2lp7.png
http://img222.imageshack.us/my.php?image=mg3rp2.png
http://img222.imageshack.us/my.php?image=mg4zh3.png

[Thunderbolt8]

nothing changed here, with another progressive video 62 fsp 1st pass and ~10 fps 2nd pass with 4 threads -.-

[Sharktooth]

dont know what to say... try contacting akupenguin and debug with him

[giandrea]

Whats ur processor/# of cores and the fps you get in 1st pass and 2 pass? Try to put only the deinterlacing filter (default (best) in avisynth) and tell us?

1 processor with 2 cores. It'a an Intel Core Duo 2 GHz, but I'm running on Mac OS X, not Windows, so I don't have avysynth. I don't have problems of speed between the first and second pass either.

A test with your DVD source and x264 (no settings, all default, except 3 threads and bitrate=1000) show that first pass speed is 21.36 fps and second pass speed is 25.22 fps, but it misses the target by 80 kbps. In another case I tested the target was missed by 250 kbps... there might be something buggy there.

[akupenguin]

@anyone having problems on windows: no, I can't help you.

apparently threads = auto always seems to be =2.
threads=auto will never pick 2. If you have 1 core it uses 1 thread, if you have 2 cores it uses 3 threads.

mb akupeguin did the adjustment for sliceless encoding with a quad core cpu and mb this now affects the c2d with 2 threads really badly ?
I have a c2d and a quad core. The c2d gets 2.1x speedup from threads at hq settings. (Yes, that's super-linear, probably because using both cores provides extra cache in addition to extra cycles.)

[experttech]

@anyone having problems on windows: no, I can't help you.


threads=auto will never pick 2. If you have 1 core it uses 1 thread, if you have 2 cores it uses 3 threads.


I have a c2d and a quad core. The c2d gets 2.1x speedup from threads at hq settings. (Yes, that's super-linear, probably because using both cores provides extra cache in addition to extra cycles.)

What about Sharktooths point of it not picking up the command line argument for threads?

[Thunderbolt8]

threads=auto will never pick 2. If you have 1 core it uses 1 thread, if you have 2 cores it uses 3 threads.

when I leave it to automatically set number of threads in settings and choose 0 threads in x264 options, the logfile ALWAYS (!!!!!) tells the use of 2 threads in my case

[Sharktooth]

Try this:
Please stop your encoding, verify the Auto set... etc. option is disabled, clean up your queue, close MeGUI, restart MeGUI, verify the Auto set.. blahblah.. is disabled (again), restart encoding and post the logfile (again).

[Thunderbolt8]

whats the difference of both settings anyway, dont they stand for the same in the end ?

anyway, when disabling it in settings and setting it to 0 in config I still get the usual 10 fps. no change.

[Sharktooth]

look at the post above...
megui wont set the optimal number of threads for x264... so the threads = 0 option in the config window is a better choice (coz will force --threads auto)

http://mewiki.project357.com/wiki/MeGUI_FAQ#How_does_the_number_of_threads_affect_quality
http://mewiki.project357.com/wiki/Settings_window
and http://mewiki.project357.com/wiki/X264_configuration

[Thunderbolt8]

did it with all that closing & reopening, with =0 (threads auto) and =4. in both cases ~10,5 fps.

i want 606 back :S

[bob0r]

@Thunderbolt8

Can you download a 700MB yuv file?
Then we can do a speed test on:

x264.606.exe --threads 2 -B5000 -m6 -r5 --direct=temporal --me=hex -b2 -w --qcomp=0.10 -A"p8x8,i8x8,i4x4" -8 --fps=25 --output NUL 720p50_mobcal_ter.yuv 1280x720
x264.606.exe --threads 3 -B5000 -m6 -r5 --direct=temporal --me=hex -b2 -w --qcomp=0.10 -A"p8x8,i8x8,i4x4" -8 --fps=25 --output NUL 720p50_mobcal_ter.yuv 1280x720

x264.614.exe --threads 2 -B5000 -m6 -r5 --direct=temporal --me=hex -b2 -w --qcomp=0.10 -A"p8x8,i8x8,i4x4" -8 --fps=25 --output NUL 720p50_mobcal_ter.yuv 1280x720
x264.614.exe --threads 3 -B5000 -m6 -r5 --direct=temporal --me=hex -b2 -w --qcomp=0.10 -A"p8x8,i8x8,i4x4" -8 --fps=25 --output NUL 720p50_mobcal_ter.yuv 1280x720

You probably already answered one of these but:
What CPU do you have (name - # cores - HT yes/no - Ghz?)
What Operation System are you using?

If these results show positive towards 614, and you have problems with an .avs encode.
I have learned that people using MultiThreaded avisynth at first also had the slowdown, but then changed settings from setmode 5 to setmode 2 for example, and all of a sudden their encode went a lot faster.

Let's test this and show once again avisynth can be a huge bottleneck.

[experttech]

@Thunderbolt8

Can you download a 700MB yuv file?
Then we can do a speed test on:

x264.606.exe --threads 2 -B5000 -m6 -r5 --direct=temporal --me=hex -b2 -w --qcomp=0.10 -A"p8x8,i8x8,i4x4" -8 --fps=25 --output NUL 720p50_mobcal_ter.yuv 1280x720
x264.606.exe --threads 3 -B5000 -m6 -r5 --direct=temporal --me=hex -b2 -w --qcomp=0.10 -A"p8x8,i8x8,i4x4" -8 --fps=25 --output NUL 720p50_mobcal_ter.yuv 1280x720

x264.614.exe --threads 2 -B5000 -m6 -r5 --direct=temporal --me=hex -b2 -w --qcomp=0.10 -A"p8x8,i8x8,i4x4" -8 --fps=25 --output NUL 720p50_mobcal_ter.yuv 1280x720
x264.614.exe --threads 3 -B5000 -m6 -r5 --direct=temporal --me=hex -b2 -w --qcomp=0.10 -A"p8x8,i8x8,i4x4" -8 --fps=25 --output NUL 720p50_mobcal_ter.yuv 1280x720

You probably already answered one of these but:
What CPU do you have (name - # cores - HT yes/no - Ghz?)
What Operation System are you using?

If these results show positive towards 614, and you have problems with an .avs encode.
I have learned that people using MultiThreaded avisynth at first also had the slowdown, but then changed settings from setmode 5 to setmode 2 for example, and all of a sudden their encode went a lot faster.

Let's test this and show once again avisynth can be a huge bottleneck.

I can do this too. Where can I download the file?

[vanger]

This is my avs file:


DirectShowSource("I:\Tape No 1\Wine_Country.avi",fps=29.97000002997,audio=false)
edeintted = last.AssumeBFF().SeparateFields().SelectEven().EEDI2(field=-1)
TDeint(order=0,full=false,edeint=edeintted)
#crop
#resize
#denoise

ConvertToYV12()
TDeint+EEDI2 is too slow... With my x2 3800+ (oced to 2500Mhz) the CPU load is near 50% (same as yours)...
Try using other deinterlacer, e.g. TomsMoComp or LeakKernelDeinterlace...

I have another problem with x264 rev.611/612/614 - the encoding stops or slows down fps to 0 without any errors after 13800-14600 frames of encoding...
I tried different sources such as mpeg-2 (.d2v) or Avi (huffyuv) with filters and without...

I tried such avs scripts:
DGDecode_mpeg2source("D:\capture\video\edw.d2v",info=3)
ColorMatrix(hints=true)

crop(16,0,0,0)

or
DGDecode_mpeg2source("D:\capture\video\edw.d2v",info=3)
ColorMatrix(hints=true)

crop(16,0,0,0)

SelectRangeEvery(140, 14)
or
AVISource("rtr.avi")
FFT3DFilter(sigma=6)
or just
AVISource("rtr.avi")
With the last script the encoding stops near 14500 frame, with adding FFT3DFilter(sigma=6) it stops near 13800 frame.

The MeGui log says nothing:
Starting job job2-1 at 8:41:32
encoder commandline:
--pass 1 --bitrate 1950 --stats "D:\capture\video\zxxxz2.stats" --bframes 3 --b-pyramid --direct auto --filter -2,-1 --subme 1 --analyse none --vbv-maxrate 25000 --me dia --threads auto --thread-input --progress --no-psnr --output NUL "D:\capture\video\zxxxz2.avs"
successfully started encoding
Processing ended at 8:48:16
----------------------------------------------------------------------------------------------------------

Log for job job2-1

avis [info]: 768x576 @ 25.00 fps (16143 frames)
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2 3DNow!
x264 [warning]: VBV maxrate specified, but no bufsize.

----------------------------------------------------------------------------------------------------------
The current job contains errors. Skipping chained jobs
Starting x264 in cmd - the result is the same...

The encoding of video fragment (13000frames) works fine...

[Thunderbolt8]

I have learned that people using MultiThreaded avisynth at first also had the slowdown, but then changed settings from setmode 5 to setmode 2 for example, and all of a sudden their encode went a lot faster.Let's test this and show once again avisynth can be a huge bottleneck.
im not using MT avisynth at all.

[bob0r]

Hence the slowdown, avisynth can't feed the frames fast enough for x264's sliceless threading.

[experttech]

TDeint+EEDI2 is too slow... With my x2 3800+ (oced to 2500Mhz) the CPU load is near 50% (same as yours)...
Try using other deinterlacer, e.g. TomsMoComp or LeakKernelDeinterlace...

You are the man! I used TomsMoComp and now get CPU usage of ~100 :) and ~60 fps!

Thanks a lot!

[Thunderbolt8]

so I definately have to use MT avisynth now in order to get the speed increase ?
what script lines should I enter ?

[experttech]

Using AVISynth will ony increase your encoding time. I just replaced an AVISynth filter. The other deinterlacing filter was not optimized for multithreading.

You said that you were getting slowness even from the command line and skipping avisynth. You also said that a progressive source was slowing down. So using Avisynth wont help you.

Thats all I can say.

[nm]

Using AVISynth will ony increase your encoding time. I just replaced an AVISynth filter.With another AviSynth filter.

You said that you were getting slowness even from the command line and skipping avisynth.He still used AviSynth through x264.

So using Avisynth wont help you.
Using the right version in the right way might help. But don't ask me what that way is; I only use MEncoder ;)

@Thunderbolt8:
If switching to multithreaded AviSynth won't help and you still want to find out what is going on, take look at the experiment that bob0r suggested. Get both 606 and the latest x264 build and run test encodes on the command line: first with YUV input and then with the simplest possible AviSynth script and default x264 parameters. If that works fine, add filters and parameters until the problem surfaces.

[Thunderbolt8]

@Thunderbolt8:
If switching to multithreaded AviSynth won't help and you still want to find out what is going on, take look at the experiment that bob0r suggested. Get both 606 and the latest x264 build and run test encodes on the command line: first with YUV input and then with the simplest possible AviSynth script and default x264 parameters. If that works fine, add filters and parameters until the problem surfaces.
where can I find the old builds (cli) of x264 ?

[Sharktooth]

x264.nl or http://megui.org/auto

[Thunderbolt8]

btw. which avisynt version is recommended to be used atm, 256 or 257 rc1 ?

[Blue_MiSfit]

Awesome results.

Working with 1080p24 HuffYUV (filtered result of fft3dfilter etc on MPEG2 HDV 25Mbps source), and setmtmode(3,0) with automatic threading on a MacPro 2.66 2x dual core, I get ~6.5 - 7 fps on both first and second passes. Utalization is constantly over 85%, and generally in the high 90s. Well done guys, this is a tremendous improvement!

@Thunderbolt8
I have learned that people using MultiThreaded avisynth at first also had the slowdown, but then changed settings from setmode 5 to setmode 2 for example, and all of a sudden their encode went a lot faster.

Let's test this and show once again avisynth can be a huge bottleneck.

MT Avisynth's mode 5 can be slower than single threading :)

I usually use 3, because I've found it works well with almost everything. 2 is a bit faster but is less compatible.

~MiSfit

~MiSfit

[Thunderbolt8]

aerf, just dled and installed megui from x264.nl and let it autoupdate, but it messed up all the profiles, because they are not there, although the auto update feature says all are up-to-date. in the update selection window, there were also no profiles to be seen I could have selected to import :S

[experttech]

I notice that Pass 2 is slower than Pass1 but I think its supposed to be that way. With the HQ-Slow profile i get ~50 fps in Pass1 and ~20fps in Pass2 and I am happy. CPU utilization is ~100% always. Nice.

Oh yeah the input is NTSC DV AVI 29.97 fps 720x480i

[Thunderbolt8]

its definately supposed that way.
the 20fps for 2nd pass is definately what it should be. i just reinstalled windows, lets see if something has changed for me

edit: nothing has changed -.-

[emmel]

Thunderbolt8, you could try simplifying your avs-script and x264 command line for test purposes. The avs-script does not have to include all the fancy stuff, it suffices to load the source and for this you only need one line, e.g. MPEG2source("clip.d2v"). Then, try x264 with default parameters:
x264.exe --progress --threads 1 -o "outfile.mp4" "script.avs"
x264.exe --progress --threads 2 -o "outfile.mp4" "script.avs"
If this works, try something a little more complex. In this way, it is much easier to figure out wheres the problem.

[Thunderbolt8]

took out colour correction in avisynth script. 1st pass got even increased by 13 fps, 2nd pass still stays ~the same..


edit: ok, this is funny now. colour correction still out in avs, I shortened the command line down to that:

C:\Programme\VIDEOBEARBEITUNG\MeGUI\tools\x264\x264.exe --pass 1 --bitrate 2008 --stats "D:\DVD\Projects\OLD_BOY_VTS_01_PGC1\testtrim.stats" --threads 16 --thread-input --progress --output NUL "D:\DVD\Projects\OLD_BOY_VTS_01_PGC1\testtrim.avs"

C:\Programme\VIDEOBEARBEITUNG\MeGUI\tools\x264\x264.exe --pass 2 --bitrate 2008 --stats "D:\DVD\Projects\OLD_BOY_VTS_01_PGC1\testtrim.stats" --threads 4 --thread-input --progress --output "D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\testtrim.mp4" "D:\DVD\Projects\OLD_BOY_VTS_01_PGC1\testtrim.avs"


1st pass got slowed down now to ~38 fps (why the hell is that possible, when I actually shortened the options), 2nd pass therefore got increased to ~40 fps. i dont have any clue whats going on :S

[emmel]

Did you try the default x264 options? Did you try other video clips? Try at least once something else.

[Thunderbolt8]

well what are the "default" options ?

edit: found the button -.-

same clip with default:

threads auto, 2, 4, 8, 16: all within ~2 fps difference


ahhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh

[experttech]

Thunderbolt can you post your avs?

[Thunderbolt8]

# Set DAR in encoder to 37 : 20. The following line is for automatic signalling
global MeGUI_darx = 37
global MeGUI_dary = 20
DGDecode_mpeg2source("D:\DVD\Projects\OLD_BOY_VTS_01_PGC1\OLD_BOY.d2v",info=3)
trim(84000,89000)
#blank deinterlace line
#crop
#resize
#denoise


avisynth 256, dgindex & decode 148

[emmel]

well what are the "default" options ?
I mean no options, except the input and output files.


threads auto, 2, 4, 8, 16: all within ~2 fps difference
Now how does that compare to --threads 1?

[nm]

Well, Thunderbolt8 already tried with (almost) default parameters and that seems to work as it should (to know for sure, he would need to do the same with r606).


edit: ok, this is funny now. colour correction still out in avs, I shortened the command line down to that:

C:\Programme\VIDEOBEARBEITUNG\MeGUI\tools\x264\x264.exe --pass 1 --bitrate 2008 --stats "D:\DVD\Projects\OLD_BOY_VTS_01_PGC1\testtrim.stats" --threads 16 --thread-input --progress --output NUL "D:\DVD\Projects\OLD_BOY_VTS_01_PGC1\testtrim.avs"

C:\Programme\VIDEOBEARBEITUNG\MeGUI\tools\x264\x264.exe --pass 2 --bitrate 2008 --stats "D:\DVD\Projects\OLD_BOY_VTS_01_PGC1\testtrim.stats" --threads 4 --thread-input --progress --output "D:\DVD\Projects\HAPPY_TOGETHER_VTS_01_PGC1\testtrim.mp4" "D:\DVD\Projects\OLD_BOY_VTS_01_PGC1\testtrim.avs"


1st pass got slowed down now to ~38 fps (why the hell is that possible, when I actually shortened the options), 2nd pass therefore got increased to ~40 fps. i dont have any clue whats going on :S
That's how it works with default parameters -- both passes run at the same speed. To get a speed difference between the passes, you would need to lower the parameters for the first pass (to get so-called turbo or fast first pass), and you would probably want to use better quality (and slower) settings for the second pass. Just like MeGUI does for you.

Now just get the 606 revision and try the same thing with that.

[Thunderbolt8]

1 tread is ~20 fps less

[huang_ch]

Bring another report on a P4 3.0G w/ HT machine:
--threads 1: ~17fps
--threads auto: ~20fps
Pretty good performance gain. :)

[Sirber]

auto works on mplayer! :D

[experttech]

I tried the HQ Insane profile modified with 3000 kbps bitrate and thats giving me 3 fps or so :(


DirectShowSource("I:\Tape No 3\C.avi",fps=29.97000002997,audio=false)
ConvertToYV12()
TomsMoComp(0,5,1)
crop( 0, 0, -2, -2)
Lanczos4Resize(640,480) # Lanczo4 (Sharp)
Convolution3D("movielq") # Heavy Noise



Starting job job1-2 at 7:13:31 AM
Starting preprocessing of job...
Preprocessing finished!
encoder commandline:
--pass 1 --bitrate 3000 --stats "I:\Tape No 3\C.stats" --ref 16 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -2,-1 --subme 7 --trellis 2 --analyse all --8x8dct --vbv-maxrate 25000 --me umh --threads auto --thread-input --progress --no-dct-decimate --no-psnr --no-ssim --output NUL "I:\Tape No 3\C.avs"
successfully started encoding

[Thunderbolt8]

there you go, I also tried it all the time with the modified HQ profile (-5 ref frames, bitrate to reach 1/3 dvd size with audio), and I also only got 8-10 fps in 2nd pass, while I got ~15-17 with 606 -.-

[experttech]

there you go, I also tried it all the time with the modified HQ profile (-5 ref frames, bitrate to reach 1/3 dvd size with audio), and I also only got 8-10 fps in 2nd pass, while I got ~15-17 with 606 -.-

So it could depend on the profile settings maybe?

Try using CE-Mainprofile or HQ-Slow - what fps do you get with these?

Surprisingly CPU utilization is ~100% with HQ-Insane so maybe some setting is taking too much time.

Any ideas folks?

[Thunderbolt8]

CE main profile:

606 (2 threads): 1st pass: ~52 fps; 2nd pass: 19,5 fps
614 (0 = auto): 1st pass: ~60 fpd; 2nd pass: 23 fps

[foxyshadis]

Yes, fps depends on profile! After all, there's a reason they're named HQ-Slow, HQ-Slower, HQ-Slowest, and HQ-Insane! Please pick the one that best matches your speed requirements, and before assuming that you just must have the absolute top quality, ensure that you can actually see a difference with a few test-encodes. The top few quality options can enormously degrade speed while providing a nearly invisible quality difference on most video.

[Thunderbolt8]

my problem is more that with 606 I got like ~15-17 frames on HQ slowest with 5 ref frames and with 614 I only get like 8-11 fps, although the sliceless thing is meant to speed things up.

[foxyshadis]

If you suspect references of causing this, which is quite possible since reference threading is how all this works, try sticking with a single profile and changing only the reference count. Try 1, 6, and 16, for instance. That way you're controlling the variable, intead of changing a whole grab bag of options at once, and you can either confirm or rule it out and go on to the next.

I don't have time now or that's just what I'd be doing.

[vanger]

Trying rev614A and still have such problem:
http://forum.doom9.org/showthread.php?p=919862#post919862
any ideas?

[Thunderbolt8]

no, i dont suspect ref. frames causing this, I only meant I generally use the HQ slowest profile with 5 instead of 10 reference frames, because imho thats enough (I used same number for both versions of course). I have just no idea whats causing my problem.


edit: oh great, now my megui doesnt even start any more. I sometimes had an error when trying to delete some file in the qeue, i got an error screen each time I tried to delete that job or even when trying to close megui. closing via taskmanager was the only solution. but as said, now megui doesnt start any more, even after having already restarted windows. i click on the button, but nothing comes :/

edit: allright, after reinstall stupid megui has gone so far to mess up its own exe file after engaging the auto update so that I cant even start it any more ffs

[Sharktooth]

Trying rev614A and still have such problem:
http://forum.doom9.org/showthread.php?p=919862#post919862
any ideas?
overclocked CPU?

[Thunderbolt8]

could it be possbile that with overclocked cpus the overclocking is somehow not correctly identified and therefore mb kinds of performance distributions to different threads are based on an assuming lower cpu speed or something like that ?

[Dreassica]

Unlikely, sicne mine is overclocked and i managed to finish encodings longer then 34k frames no prob.

[Manao]

No. Overclocking may render your system unstable, leading to some encoding errors such as the one shown on this page, hence the question. It may be the cause of errors whether you're using threads or not.

[vanger]

overclocked CPU?
Yes, but it is stable! With 1 thread it works fine.
Without overclocking I have this error too...
Can I get any debug info from x264, when it stops?

P.S.: MC encoder works fine too (cpu load 100% without any interrupt for 3-7 hours of encoding)

[Romario]

I agree with vanger. Overclocked CPU IS stable if user know how to overclock CPU.

For example, mine Athlon 64 3500+ singlecore from default 2.2 GHz works WITHOUT any problem on 2.7 GHz, and I heavily use x264 encoding without any issue.

[Thunderbolt8]

I agree with vanger. Overclocked CPU IS stable if user know how to overclock CPU.

For example, mine Athlon 64 3500+ singlecore from default 2.2 GHz works WITHOUT any problem on 2.7 GHz, and I heavily use x264 encoding without any issue.
same for me, encoding processes work without any problems, apart from that >606 speed thing that its all about.

[Atak_Snajpera]

I sugest to use this tool if you are not sure if your cpu is stable

http://s24.quicksharing.com/d/6528656/1166992024/orthos_exe_20060420.cab

Go-Ortos is even better than SuperPi !!!

[Adub]

Yes, overclocking can be successful. Like my rig that I am typing this on is overclocked to 3ghz from 2.4ghz. Rock solid. And I am no where near being done.

[woah!]

i am still finding elder is giving me more frames at 100% cpu power ?

even tho both ways are showing 100% cpu power being used, elder gives me a 10fps gain using the same avs script and x264 settings on the 2nd Pass

still much better than before with the older x264 rev606

[Manao]

The concept is that debugging a program is complicated, so a programmer doesn't want to debug uselessly. When using an overclocked CPU, the probability for your system to be unstable and to be the cause of the apparent bug is quite higher than with a non overcloked one, so I find normal to discard bug report on possibly unstable system.

Moreover, an unstable system doesn't mean that it randomly crashes every now and then. It may very crash for one program only, with a specific configuration only. So saying "but only x264 multithreaded has the bug" isn't a justification at all.

Finally, just as a reminder. I haven't encountered yet a perfect test for checking hardware. Even if your hardware passes every known hardware checker, it doesn't mean it is stable. I once had some RAM that was passing every test, except that XviD was failing to encode some specific file with it. Replacing the RAM fixed the issue, so it wasn't XviD fault. Yet, only XviD was crashing.

[dvd_maniac]

How would I get this version to work with Staxrip?

[Sharktooth]

staxrip is becoming obsolete and unless it supports custom commandline options and manual editing you simply cant take advantage of the new x264 versions.

[dvd_maniac]

What would be the easiest GUI to use that will take advantage of it?

I tried MeGUI but I'm looking for something a little easier. I encode a couple of hours worth of video per day and like that Staxrip only takes a couple of minutes to queue them up and encode.

[Adub]

Something a little easier? MeGUI is pretty freakin' easy. Have you tried the One Click encoder, Autoencode, and so on?

[dvd_maniac]

Staxrip is much easier (At least for me). I can open multiple files at once and it will create a project for each of them. Then open each project and add them to queue then start queue.

[buzzqw]

even automkv is easier to use (and support queue, new x264.exe options, plus a lot of filters).

MeGui is more like a big furnace, put anything into and got anything out

BHH

[vanger]

I solved my problem with ecoding's stops. The overclocking was not the reason.
The problem was in Outpost Firewall. At first I reinstalled the OS and the encodings goes well, after installing Outpost it brakes again.
Now I'm using ZoneAlarm and have no problem with x264. :)

[Thunderbolt8]

i also have outpost firewall, but what encoding stops do you refer to ?

[vanger]

i also have outpost firewall, but what encoding stops do you refer to ?
http://forum.doom9.org/showthread.php?p=919862#post919862

[Hans Ohlo]

i noticed in process explorer that when i set x264 to auto threading x264 used 22 threads during first and second pass. i found that very strange and set it hard to 3 threads but it still used 22 threads.

i use the latest megui, latest x264 build on a c2d e6600 2gb ram and winxp x64.

[foxyshadis]

x264 also hosts a thread for avisynth, and avisynth can spawn as many threads as it wants. Certain filters, such as vsfilter, will use 2-3 threads each time they're called.

[Hans Ohlo]

x264 also hosts a thread for avisynth, and avisynth can spawn as many threads as it wants. Certain filters, such as vsfilter, will use 2-3 threads each time they're called.
my script was very simple:

DirectShowSource("", fps=25.0, seek=false)
Crop(...)
BicubicResize(1280,720,...)

i tested this before, but never saw 22threads...

[Thunderbolt8]

http://forum.doom9.org/showthread.php?p=919862#post919862
I dont have any interruption problems or something like that. just still that my 2nd pass is slower than it should be :X