x264farm: distributed encoding

[SpAwN_gUy]

Yeah, that's exactly the problem I'm having sometimes. One agent is stuck and others are just waiting.
let me guess.. maybe i'm having the sam sort of a problem..
it goes after al agents are ("agentname close_connection") and than controller just waits.. for something.
and all connection to source files are closed, and controller just waits..
it's not happening all the time, but some times..

this happens to me on both passes.. fist and second,.. in the end.. just before the MUXing starts :( ...

so my 5crnts: i think this loop occurs when agent ended encoding sended job, but something happened during sending of MKV, or Controlled didn't got it, or agent could not send it..

So a small solution,.. after a "timer" (~2 minutes) controller could ReCheck Working(as C thinks) Agents if they are actually working... timer could start after beginning of closing connections and disconnecting agents.. (well, theese are just my thoughts)

[SpAwN_gUy]

If the job is never returned, or if there is a constant cycle (the same frames get re-encoded over and over) then that's a problem. i have now end of second-pass with 60% bitrate .. and it just reencodes something again and again..
2007-03-20~10:43:59.65 #1 7 < #2 8 (read #2)
and
2007-03-20~10:44:03.46 Number of GOPs done: 719 / 719
2007-03-20~10:44:03.46 Looks like the second pass was already done; using 0 for the number of GOPs computed and :) now i've found 770 copies of GOP #658 in tempDir.. so i deleted them.. and it encodes again something.. on 100%total and 60% of Quality..

... ADD(few mins later): Yeppie.. it did.. :) ... two deleted GOPs(manually) and retcontrol(i guess this is the proper naming) works again.

I didn't find anything wrong with the MKV files you sent me. I checked them with the official mkvinfo tool and they parsed correctly. I used the same Matroska-decoder as in x264farm to merge them and everything went well... so I don't know where the problem is. yes, mkvMerge did that also.. but not the farm :(
Are you still able to recreate it? If so, I can send you a new build which should at least give you a more descriptive error message. yes i do,.. i thought about it,.. that you would like to recreate it again :) ...

... testings... so i am... it is still..
in the DOSBox
#0299: "299 - 56204 56453 5654cd.mkv"
#0300: "300 - 56454 56559 148fc1.mkv"
#0301: "301 - 56560 56617 befbc3.mkv"
Fatal error: exception Invalid_argument("Char.chr")

in the out-dump.txt
2007-03-20~11:08:53.92 Merging 88 / 302
2007-03-20~11:08:54.03 Merging 89 / 302
2007-03-20~11:08:54.17 Merging 90 / 302
2007-03-20~11:08:54.29 Merging 91 / 302
2007-03-20~11:08:54.62 Merging 92 / 302

[omion]

[edit: Oops. It looks like SpAwN_gUy wrote a bunch of stuff after I started writing this. I'll get to those responses later]

First off, I'd like to mention that I updated my x264 builds to r634. links: 32-bit (http://omion.dyndns.org/x264/32/x264-634_omion.rar) 64-bit (http://omion.dyndns.org/x264/64/x264x64-634_omion.rar)

Also, I don't know if I mentioned this, but you only need my builds if you are using controller-based encoding or piped input on the agent. If you are using non-piped input to an agent-based encoder, you may use any x264 build you want.


Yeah, that's exactly the problem I'm having sometimes. One agent is stuck and others are just waiting.Does the agent show any errors? What's the CPU usage like when this happens?

One problem that I have had is that if an agent stops doing anything without informing the controller that something's wrong, then the encoding will never finish. If you restart either the controller or agent it should complete, but there's not really a good answer right now. I have been trying to think of a nice, clean way to handle this since long before I released x264farm...

Perhaps it would work even without removing the temp files. I'll try not to delete them next time.

I'm not sure if it's possible to detect a DELFIRST dead loop, but I'd suggest putting a limit to consecutive DELFIRST messages received from an agent. If it's reached, then the controller assumes that the agent is stuch and sends a signal to drop the job and restart. Of course, this is only a quick&dirty workaround. Can't think of anything else.

Well... I really don't think DELFIRST is related to the problem you're having. The DELFIRST condition is a product of the controller so doing something with the agents won't get anywhere.

Also, if it gets into a loop, the agents will still be encoding. The loops occur when the controller throws out everything that an agent does and tells the agent to encode the same thing again. As long as --batch is set to larger than x264's --keyint option, I don't think it's possible to get in a loop.

As to what is causing the problem you're having, I don't really know. Is it doing it a lot, or was it just that one time? If multiple times, was it always the same agent that was having the problem?

[foxius]

Does the agent show any errors? What's the CPU usage like when this happens?


I don't really remember. I think there was no error displayed on the agent computer. I'll check that next time.


One problem that I have had is that if an agent stops doing anything without informing the controller that something's wrong, then the encoding will never finish. If you restart either the controller or agent it should complete, but there's not really a good answer right now. I have been trying to think of a nice, clean way to handle this since long before I released x264farm...


I think the controller has to "ping" the agents to get their status (I think you already do that to get the work progress). If it's stuck then you send a "restart" signal. The agents do their work in worker threads, so the main thread is always responsive.



Well... I really don't think DELFIRST is related to the problem you're having. The DELFIRST condition is a product of the controller so doing something with the agents won't get anywhere.


But whenever that agent is stuck in a loop I see the DELFIRST message on the controller. I guess they're related in some way.


As to what is causing the problem you're having, I don't really know. Is it doing it a lot, or was it just that one time? If multiple times, was it always the same agent that was having the problem?

I had it at least for each movie i recoded. For the last movie (harry potter, part 1) i had several DELFIRST & dead loops. It wasn't always the same agent.

[foxius]

omion,

does x264farm have any kind of IPC that lets third party applications to get some feedback from it? it'd make easier to develop a GUI or even may be integrate with MeGUI.

fox

[SpAwN_gUy]

does x264farm have any kind of IPC that lets third party applications to get some feedback from it? it'd make easier to develop a GUI or even may be integrate with MeGUI.
i think not.. well, sources are opened..(in first post)..
as omion is not familiar with C i'm not familiar with OCaml :) .. i can read through code, but.. i can't programm(i've never tried in OCaml) and i don' have tools.. 'cause i'm short in traffic..

BTW... if something goes wrong.. i'm offline. :( ... for some time.. my boss told me that i'm not working.. and disabled iNet for me.. (eMail still works)

[foxius]

i think not.. well, sources are opened..(in first post)..
as omion is not familiar with C i'm not familiar with OCaml :) .. i can read through code, but.. i can't programm(i've never tried in OCaml) and i don' have tools.. 'cause i'm short in traffic..


knowledge of C programming is not required for implementing IPC (interprocess communication).

x264farm will allow monitoring of the encoding process via IPC (named pipes, shared memory or, ideally, RPC). Then anyone could write a third party software in almost any language to communicate with x264farm.

I've googled for "ocaml RPC" and found some implementations. I'm not sure which is more suitable because I'm not familiar with the language. RPC (remote procedure calls) will allow you to define a set of functions in IDL (interface description language). IDL then can be compiled into ocaml, c++, delphi, etc. x264farn will act as an RPC server and third party software will act as a client which will be able to call functions defined in IDL.

Given that x264farm already has functions for monitoring the agents, imlementing RPC will take a few simple steps:

1. Define a set of published functions (an interface) via IDL
2. Compile IDL into ocaml
3. Map the interface to the implementation (i.e. the function defined in IDL will call the function inside x264farm)
4. Distribute the IDL file so anyone can compile it to other programming languages and use it.

RPC server and clients will be able to communicate over various protocols, such as tcp/ip, ipx, named pipes, etc.

[Wishbringer]

Can't get an agent to run properly on a Vista32 machine.
Can start agent, but when receiving jobs I allways get a windows error message that there is something wrong with nice.exe.

[SpAwN_gUy]

Can't get an agent to run properly on a Vista32 machine.
Can start agent, but when receiving jobs I allways get a windows error message that there is something wrong with nice.exe.
yeah,.. me too... i've reported that.. i have sources of newNice.exe ... and i think i'll look though a bit later, i think..
and now i'm f___ing around of HDDVD-BackUp,.. and i think AgentBased encoding will not work :( ...
currently i'm short on 40mins from a movie, but everythig seems fine,..

[foxius]

Can't get an agent to run properly on a Vista32 machine.
Can start agent, but when receiving jobs I allways get a windows error message that there is something wrong with nice.exe.

Could you please copy/paste the windows error message you're getting? Unfortunately, I don't have Vista and can't test it myself. Once I have the error message I'll try to fix it.

[SpAwN_gUy]

@foxius ... my old problem shows:
Леша-К7, 16.03.2007 13:24 :
Problem signature:
Problem Event Name: BEX
Application Name: nice.exe
Application Version: 0.0.0.0
Application Timestamp: 45f8df57
Fault Module Name: nice.exe
Fault Module Version: 0.0.0.0
Fault Module Timestamp: 45f8df57
Exception Offset: 00005183
Exception Code: c000000d
Exception Data: 00000000
OS Version: 6.0.6000.2.0.0.256.1
Locale ID: 1033
Additional Information 1: 5fd9
Additional Information 2: de9895fb4090bedd1a27a6ff8cfedad1
Additional Information 3: e9b8
Additional Information 4: 8079ce0cad7bc845eec58faba3ac0031
on Vista-Ultimate 64bit PC.
after doing this:
nice -n 20 avs2yuv -raw -seek 1250 -frames 250 -o - "\\controllerpc\x264farm-2\CHECH-Small\nazdravicko.avs" | nice -n 20 \\controllerpc\x264farm\!AGENT\x264x64.exe --ref 16 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -2,-1 --subme 7 --trellis 2 --analyse all --8x8dct --vbv-maxrate 25000 --me umh --threads 2 --thread-input --progress --no-dct-decimate --no-psnr --no-ssim --qcomp 0.6 --cplxblur 20. --qblur 0.5 --qpmin 10 --qpmax 51 --qpstep 4 --ipratio 1.4 --pbratio 1.3 --bitrate 292 --fps 25/1 --pass 3 --stats "temp\stats d40faa.txt" -o "temp\output 83d1d8.mkv" - 688x400

[Wishbringer]

@foxius

Curiously it's only a window that opens, that says that nice.exe works anymore, then another one that asks if i want to send infos to Microsoft.

But no further details what infos are available. So I can't give you a hint, sorry.

out-dump.txt from agent doesn't contain useable infos too.

[omion]

@all who have a problem with nice vs vista:
Does it give the error message when you try to use the nice command without x264farm? Like if you type this into the terminal:
nice -n 10 help
does it work?

@all:
Sorry about not responding in a while. I got sick again, and all the students just got back to school (I work at the UCSD bookstore), and I've been doing a major overhaul of my other program (mp3packer (http://www.hydrogenaudio.org/forums/index.php?showtopic=32379)) so I haven't had much time for x264farm. I'll get to work on it as soon as I can, though.

[legoman666]

is there a way to set the amount of frames sent to each node for each job? I have 5 computers in my network, 1 is a 1.8ghz laptop, 1 is a 1.6ghz amd xp, 1 is a p2 350mhz, 1 is an AMD x2 2.6ghz (the controller), and the last is a 1.6ghz amd 2800+.

Now of course, the p2 is a slow piece of crap. Looking at the command line output during encoding, it looks like it sends a minimum of 5250 frames for each job. After all my clients finish their work, my p2 350 is still chugging away at its job. Is there a way to set it so that each job is smaller? like 250 frames?

I somehow bugged it out so that its doing 250 frames per job, but i have no idea what I did to make it do that. any ideas?

[legoman666]

ah nevermind, i see. for the first pass its doing ~5000 frames per job, but for the second pass its a much lower number.

:D silly me.

[Wishbringer]

nice -n 10 help
does it work?


@omion:
no crash, help page was displayed.

I don't really know why it crashes when starting x264.exe

[legoman666]

this little program is amazing, thanks for the hard work!

although I do have a little question, what options do i use only for the first pass and which do i use only for the second? Or do you use them all for both. I'm using megui to get the command line instructions and just copying them for use with controller.exe. At the moment, my megui command line says (for the HQ-slowest profile):

--pass 2 --bitrate 918 --stats ".stats" --ref 10 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -2,-1 --subme 6 --trellis 2 --analyse all --8x8dct --vbv-maxrate 25000 --me umh --threads auto --thread-input --progress --no-dct-decimate --no-psnr --no-ssim --output "I:\example.mp4" "I:\example.avs"

and so for controller.exe im doing

controller.exe -B 918kbps --first "--ref 10 --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -2,-1 --subme 6 --analyse all --merange 12 --no-psnr --trellis 2 --progress " --second "--ref 10 --no-fast-pskip --bframes 3 --b-rdo --bime --b-pyramid --weightb --direct auto --filter -2,-1 --subme 6 --trellis 2 --analyse all --8x8dct --me umh --merange 12 --no-psnr --no-dct-decimate --no-ssim --no-psnr --vbv-maxrate 25000 --progress" --avs "I:\examlpe.avs" -o C:\example.mkv

is there anything im missing/doing wrong? with these setting and 8 physical cores working on the encoding, it took ~6 hours to encode a dvd thats about 1.5 hours long, which seems reasonable considering the quality setting. Something seems to be wrong with the first pass settings, because when I used megui alone to do the encoding, the first pass was the quickest (~30fps) and the second pass was ~3.5. With my current settings for controller.exe, the first pass took ~4 hours and the 2nd ~2hours.

list of computers workin on encoding (all over 100mb/s lan)
AMD x2 3800+ @2.6ghz (controller) 2 cores (local)
AMD 1800+ (catherine)
P2 350mhz (server)
AMD A64 2800+ (basement)
Centrino 1.6ghz (mom's laptop)
2xP3 500mhz Xeon (staircase)

Speeds at which they work

Agent | Now | GOPs | Frames | FPS |
Local 1 | 584 | 191 | 35515 | 9.11 |
Local 2 | 395 | 176 | 35362 | 9.61 |
basemnet 1 | 610 | 139 | 24938 | 6.18 |
server 1 | 644 | 7 | 1448 | 0.33 |
mom's laptop 1 | x | 40 | 7095 | 3.87 |
catherine 1 | x | 106 | 18011 | 4.59 |
staircase 1 | 659 | 32 | 6050 | 1.43 |
staircase 2 | 583 | 26 | 5110 | 1.13 |

[Wishbringer]

@legoman666

could it be that megui uses lesser quality settings for the first pass?
normally you don't need all options in first pass (like -b-rdo).
best is to look at megui's log to see, what commandline it used for first pass.

[legoman666]

ah thanks for the suggestion, that helps :D

controller.exe -B 8079kbps --first "--bframes 3 --b-pyramid --direct auto --filter -2,-1 --subme 1 --analyse none --vbv-maxrate 25000 --me dia --progress --no-dct-decimate --no-psnr --no-ssim" --second "--ref 10 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -2,-1 --subme 6 --trellis 2 --analyse all --8x8dct --vbv-maxrate 25000 --me umh --progress --no-dct-decimate --no-psnr --no-ssim" --avs "I:\movie.avs" -o C:\movie.mkv

theres my next project, 720p uber quality :D. should take.... 12-14 hours for my 6 computers.

[SpAwN_gUy]

although I do have a little question, what options do i use only for the first pass and which do i use only for the second? Or do you use them all for both. I'm using megui to get the command line instructions and just copying them for use with controller.exe. At the moment, my megui command line says (for the HQ-slowest profile):
okay.. i'm back,.. and i don't have iNet.. :) ...
so,.. i have few suggestions for you ..
1) i've noticed that most powerfull PCs sould be first in controller's config (i have something like 10PCs Farm ;) ade powers vary alot)
2) .. well.. if you add Job in MeGUI and then close it.. you'll get 2 job.xml files .. look through them and you'll find something like <Commandline> .... copy the values from both files and you'll see the difference..
2b) if someone wants i can send a beta-controllerGUI prog (and agentGUI) ... main thing is that controllerGUI can generate commandline for the controller (but don't even try to start it for now ;) ... it'll give you a headake)..

peace to all, i'm waiting for something new ... ;)

[legoman666]

hmm, im getting an error with my next encoding project,

2007-04-13~10:05:50.00 x264farm version 1.09-163
2007-04-13~10:05:50.00 Using config file ".\config.xml"
2007-04-13~10:05:50.03 CONFIG:
2007-04-13~10:05:50.03 temp_dir: "I:\\hdtv\\x264farm\\temp\\controller"
2007-04-13~10:05:50.03 agent_list 1:
2007-04-13~10:05:50.03 "Local 1" = 127.0.0.1 50700-50703
2007-04-13~10:05:50.03 "Local 2" = 127.0.0.1 50700-50703
2007-04-13~10:05:50.03 "basemnet 1" = 192.168.1.13 50700-50703
2007-04-13~10:05:50.03 "server 1" = 192.168.1.4 50700-50703
2007-04-13~10:05:50.03 "mom's laptop 1" = 192.168.1.167 50700-50703
2007-04-13~10:05:50.03 "catherine 1" = 192.168.1.215 50700-50703
2007-04-13~10:05:50.03 "staircase 1" = 192.168.1.9 50700-50703
2007-04-13~10:05:50.03 "staircase 2" = 192.168.1.9 50700-50703
2007-04-13~10:05:50.03 agent_list 2:
2007-04-13~10:05:50.03 "Local 1" = 127.0.0.1 50700-50703
2007-04-13~10:05:50.03 "Local 2" = 127.0.0.1 50700-50703
2007-04-13~10:05:50.03 "basemnet 1" = 192.168.1.13 50700-50703
2007-04-13~10:05:50.03 "server 1" = 192.168.1.4 50700-50703
2007-04-13~10:05:50.03 "mom's laptop 1" = 192.168.1.167 50700-50703
2007-04-13~10:05:50.03 "catherine 1" = 192.168.1.215 50700-50703
2007-04-13~10:05:50.03 "staircase 1" = 192.168.1.9 50700-50703
2007-04-13~10:05:50.03 "staircase 2" = 192.168.1.9 50700-50703
2007-04-13~10:05:51.54
WARNING: avs2yuv exited with error 1; avs2yuv responded:

2007-04-13~10:05:51.54 GOT NORMAL INFO "\013"
2007-04-13~10:05:51.54
ERROR: AVS file 'I:\CASINO\casino.avs' does not seem to be valid



my avs file is pretty simple and worked just fine in megui when I encoded the first 200 frames just to see if it would work. heres the contents of it:


LoadPlugin("C:\PROGRA~2\GORDIA~1\DGMPGDec\DGDecode.dll")
LoadPlugin("C:\PROGRA~2\GORDIA~1\AviSynthPlugins\UnDot.dll")
DirectShowSource("I:\CASINO\CASINO\BDMV\STREAM\00000.m2ts")
crop(0,144,1920,800)
LanczosResize(1280,528)
Undot()
ConvertToYV12()

my guess is the ConvertToYV12() at the bottom? megui told me to add it so I'm not really sure. any ideas?

[foxius]

I found that you get the best speed when using agent-based encoding. avs2yuv happens to consume too much CPU & bandwidth, but obviously useful when you can't use agent based encoding.

My setup is something like this:

1. All source files are located in network shares.
2. D2V files refer to network shares.
3. AVS files also refer to sources on network shares.
4. I also put AVS plugins on the network share
5. Thus, the agents will read the source without requiring avs2yuv to decode the source and send raw frames over your network.
6. It is easier to find(and eventually solve) problems with AVS scripts
7. I keep the sources on one HDD, write the encoded file to another.

Sample AVS script:


LoadPlugin("\\foxxp\_recoding\DGMPGDec\DGDecode.dll")
LoadPlugin("\\foxxp\_recoding\DGMPGDec\Undot.dll")

MPEG2Source("\\foxxp\_recoding\ka\ka.d2v")

# CROPPING
crop(0,0,720,576)

# RESIZING
LanczosResize(720,400)

Undot()


I also have a batch file (.bat) based on the MeGUI "Slower" profile. I just substitute


controller.exe --bitrate 1200kbps --avs "\\foxxp\_recoding\ka\ka.avs" --first "--bframes 3 --b-pyramid --direct auto --filter -2,-1 --subme 1 --analyse none --vbv-maxrate 25000 --me dia --no-psnr --no-ssim" --second "--ref 5 --mixed-refs --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -2,-1 --subme 6 --trellis 1 --analyse all --8x8dct --vbv-maxrate 25000 --me umh --no-psnr --no-ssim" -o "c:\_recoding\ka\ka.mkv"


I currently have 4 agents (including local) on a 100mbit network, 2 more over 100mbit uplink. I enable the second core on the local for the second pass (each core gives 5-7fps), and put one more job for HT processors on the network (gives 1-2 more fps per CPU). It takes 3-4 hours to encode an average movie.

I also found that the second pass never goes beyond 13-14fps, no matter how many agents/cores I add. I think it should have something to do with the bandwidth.

Hope this would be useful.

[foxius]

my guess is the ConvertToYV12() at the bottom? megui told me to add it so I'm not really sure. any ideas?

try removing it :) i think you only need that filter when you have some unusual (RGB?) source (i might be wrong). if it doesn't help:

1. try opening the .avs file in VirtualDub
2. try agent-based encoding

[omion]

WARNING: avs2yuv exited with error 1; avs2yuv responded:

2007-04-13~10:05:51.54 GOT NORMAL INFO "\013"


Try opening it up with avs2yuv. That's what x264farm uses to check the info. Run the following:
avs2yuv -frames 1 -o NUL AVSFILE.avs and see what it returns.

@all:
Again, sorry about the lack of programming. I'm pretty swamped right now :(

[legoman666]

apparently avs2yuv doesn't like the directshowsource? Which is weird because megui is converting it right now with no problems (at a appalingly slow 1.4fps I might add, wish i could use x264farm :P )


I:\hdtv\x264farm\controller>avs2yuv.exe -frames 1 -o NUL "I:\CASINO\casino.avs"

Avisynth error:
DirectShowSource: Could not open as video or audio.

Video returned: "DirectShowSource: couldn't create filter graph:
CoInitialize has not been called. "

Audio returned: "DirectShowSource: couldn't create filter graph:
CoInitialize has not been called. "

(I:\CASINO\casino.avs, line 18)

I:\hdtv\x264farm\controller>

its just the m2ts file from a blu ray backup (Casin Royale)

[chipzoller]

Might this project (sometime in the near or far future) see a GUI at some point? This being considered or worked on? Great tool, btw...

[rhodges]

@all who have a problem with nice vs vista:
Does it give the error message when you try to use the nice command without x264farm? Like if you type this into the terminal:
nice -n 10 help
does it work?

@all:
Sorry about not responding in a while. I got sick again, and all the students just got back to school (I work at the UCSD bookstore), and I've been doing a major overhaul of my other program (mp3packer (http://www.hydrogenaudio.org/forums/index.php?showtopic=32379)) so I haven't had much time for x264farm. I'll get to work on it as soon as I can, though.

I downloaded the source and ran the debug version and it was crashing due to a buffer size problem. The problem is the IsWinXPorLater function returns false, so max_command_size gets set to MAX_PATH which is too small.

I commented out the IsWinXPorLater check and it works like a champ on Vista 64 Ultimate.

[Edit]
I didn't try this, but I found on microsoft's site how you'd change the function:
http://msdn2.microsoft.com/en-us/library/ms724451.aspx

from:
return (version.dwMajorVersion >= 5 && version.dwMinorVersion >= 1);

to:
return ((version.dwMajorVersion > 5) || (version.dwMajorVersion == 5 && version.dwMinorVersion >= 1));

[omion]

[Edit]
I didn't try this, but I found on microsoft's site how you'd change the function:
http://msdn2.microsoft.com/en-us/library/ms724451.aspx

from:
return (version.dwMajorVersion >= 5 && version.dwMinorVersion >= 1);

to:
return ((version.dwMajorVersion > 5) || (version.dwMajorVersion == 5 && version.dwMinorVersion >= 1));

Ah. That would be a problem. Vista is version 6.0, so dwMinorVersion < 1...

I put a new nice build up. Download it here (http://omion.dyndns.org/x264farm/nice-20070415.rar) (src (http://omion.dyndns.org/x264farm/nice-20070415_src.rar))

[harissa]

I spent 24 h trying to make this work and I couldn't
Here s my setup

my avs
loadplugin("DGDecode.dll")
Mpeg2Source("carnavale.d2v")
I dont't know where to put nice.exe , avs2yuv.exe and x264-634_omion.exe
both controller and agent scrollin like hell , and showing this

E:\x264farm\temp\controller
2007-04-15~16:30:46.00 x264farm version 1.09-163
2007-04-15~16:30:46.00 Oops! File " G:\9\ka.mkv" does not exist
2007-04-15~16:30:46.00 WARNING: Second pass options are not set; this will use the default x264 settings
2007-04-15~16:30:46.00 Using config file ".\config.xml"
2007-04-15~16:30:46.00 CONFIG:
2007-04-15~16:30:46.00 temp_dir: "E:\\x264farm\\temp\\controller"
2007-04-15~16:30:46.00 agent_list 1:
2007-04-15~16:30:46.00 "Local 1" = 169.254.25.121 50700-50703
2007-04-15~16:30:46.00 "Local 2" = 169.254.25.121 50700-50703
2007-04-15~16:30:46.00 "Linux 1" = 169.254.23.92 50700-50703
2007-04-15~16:30:46.00 "Apple 1" = 169.254.25.121 50700-50703
2007-04-15~16:30:46.00 "Apple 2" = 169.254.25.121 50700-50703
2007-04-15~16:30:46.01 agent_list 2:
2007-04-15~16:30:46.01 "Local 1" = 169.254.25.121 50700-50703
2007-04-15~16:30:46.01 "Local 2" = 169.254.25.121 50700-50703
2007-04-15~16:30:46.01 "Linux 1" = 169.254.23.92 50700-50703
2007-04-15~16:30:46.01 "Linux 2" = 169.254.23.92 50700-50703
2007-04-15~16:30:46.01 "Apple 1" = 169.254.25.121 50700-50703
2007-04-15~16:30:46.01 "Apple 2" = 169.254.25.121 50700-50703
2007-04-15~16:30:46.01 "Apple 3" = 169.254.25.121 50700-50703
2007-04-15~16:30:46.15 GOT NORMAL INFO "G:\\dvb\\x264.avs: 1280x720, 60000/1001 fps, 23247 frames\013"
2007-04-15~16:30:46.26 GOT FIRST INFO "G:\\dvb\\x264.avs: 1280x720, 60000/1001 fps, 23247 frames\013"
2007-04-15~16:30:46.37 GOT FAST INFO "G:\\dvb\\x264.avs: 1280x720, 60000/1001 fps, 23247 frames\013"
2007-04-15~16:30:46.39 INFO:
2007-04-15~16:30:46.39 i_res_x = 1280
2007-04-15~16:30:46.39 i_res_y = 720
2007-04-15~16:30:46.39 i_fps_n = 60000
2007-04-15~16:30:46.39 i_fps_d = 1001
2007-04-15~16:30:46.39 i_fps_f = 59.940060
2007-04-15~16:30:46.39 i_num_frames = 23247
2007-04-15~16:30:46.39 i_bytes_y = 921600
2007-04-15~16:30:46.39 i_bytes_uv = 230400
2007-04-15~16:30:46.39 i_bytes_per_frame = 1382400
2007-04-15~16:30:46.39 multiplier for kbps to fbits = 0.059940
2007-04-15~16:30:46.39 fast_res_x = 1280
2007-04-15~16:30:46.39 fast_res_y = 720
2007-04-15~16:30:46.39 Oops. Writing the args failed with: Sys_error("E:\\x264farm\\temp\\controller\\option_log.txt.temp: No such file or directory")2007-04-15~16:30:46.39 AVS temp dir is E:\x264farm\temp\controller\132FA1D98D68B80231C0CECCD0F61FAF x264.avs
2007-04-15~16:30:46.40 First string is "-2\n„•¦¾\000\000\000\001\000\000\000\000\000\000\000\000\000\000\000\000@\n0+0\n0\n3hœ)Z\013ûø‚¾\127ܺx\027\020 = G:\\dvb\\x264.avs\n", with MD5 "F51A9C869FA15B80F7D0F2202ABBDF98"
2007-04-15~16:30:46.60 Second string is "\n„•¦¾\000\000\000\001\000\000\000\000\000\000\000\000\000\000\000\000@\n0+0\n0\n", with MD5 "A25B8F0CCB96F56266CFA33B07D0F828"
2007-04-15~16:30:46.81 not (not (is_something first_dir) && is_file first_stats) [first pass after making dir]
2007-04-15~16:30:46.81 not (is_dir first_dir)
2007-04-15~16:30:46.81 11111111 FIRST PASS 11111111
2007-04-15~16:30:46.81 From file:
2007-04-15~16:30:46.81 ( 0,23246) Empty
2007-04-15~16:30:46.81 Local 1 starting on 169.254.25.121 (id 1)
2007-04-15~16:30:46.81 Local 2 starting on 169.254.25.121 (id 2)
2007-04-15~16:30:46.81 Linux 1 starting on 169.254.23.92 (id 3)
2007-04-15~16:30:46.81 Apple 1 starting on 169.254.25.121 (id 4)
2007-04-15~16:30:46.81 Apple 2 starting on 169.254.25.121 (id 5)
2007-04-15~16:30:46.81 Splitter starting
2007-04-15~16:30:46.82 Split points at 5,3,1,1
2007-04-15~16:30:49.87 queue_beginning found the beginning; split to Q(0,5062) :: (5063,23246)
2007-04-15~16:30:49.87 Ranges at beginning:
2007-04-15~16:30:49.87 ( 0, 5062) Queued
2007-04-15~16:30:49.87 ( 5063,23246) Empty
2007-04-15~16:30:49.87 Splitter: fewer queues than max_queues_after_full
2007-04-15~16:30:49.87 Splitter: fewer queues than max_queues_split
2007-04-15~16:30:49.87 split_largest_range found the biggest empty range to be 18184 frames long
2007-04-15~16:30:53.29 split_largest_range found start I frame 11460 in (11405,11655)
2007-04-15~16:30:56.14 split_largest_range split to (5063,11460) :: (11461,16617) :: (16618,23246)
2007-04-15~16:30:56.14 Splitter: split_largest_range did something; re-run
2007-04-15~16:30:56.14 Splitter: fewer queues than max_queues_after_full
2007-04-15~16:30:56.14 Splitter: fewer queues than max_queues_split
2007-04-15~16:30:56.14 split_largest_range found the biggest empty range to be 6629 frames long
2007-04-15~16:30:58.62 split_largest_range found start I frame 19947 in (19807,20057)
2007-04-15~16:30:58.62 Splitter: split_largest_range did something; re-run
2007-04-15~16:30:58.62 Splitter: fewer queues than max_queues_after_full
2007-04-15~16:30:58.62 Splitter: fewer queues than max_queues_split
2007-04-15~16:30:58.62 split_largest_range found the biggest empty range to be 6398 frames long
2007-04-15~16:31:02.35 split_largest_range found start I frame 8307 in (8137,8387)
2007-04-15~16:31:02.35 Splitter: split_largest_range did something; re-run
2007-04-15~16:31:02.35 Splitter: fewer queues than max_queues_after_full
2007-04-15~16:31:02.35 Splitter: fewer queues than max_queues_split
2007-04-15~16:31:02.35 split_largest_range found the biggest empty range to be 3330 frames long
2007-04-15~16:31:02.35 but that's not long enough! (5250)
2007-04-15~16:31:02.35 Splitter: split_largest_range did nothing, but there are enough queues
2007-04-15~16:31:02.35 ( 0, 5062) Queued
2007-04-15~16:31:02.35 ( 5063, 8307) Empty
2007-04-15~16:31:02.35 ( 8308,11460) Queued
2007-04-15~16:31:02.35 (11461,16617) Queued
2007-04-15~16:31:02.35 (16618,19947) Empty
2007-04-15~16:31:02.35 (19948,23246) Queued
2007-04-15~16:31:54.68 using compression type 0 (no compression)
2007-04-15~16:31:54.68 Local 2 found no ranges after full
2007-04-15~16:31:54.68 Local 2 check_out_job got frames 0 - 5062
2007-04-15~16:31:54.68 Splitter: fewer queues than max_queues_after_full
2007-04-15~16:31:54.68 using compression type 0 (no compression)
2007-04-15~16:31:54.68 Local 2 sent full AVS info 33689C295A0DFBF882BE7FDCBA781B14 = "G:\/dvb/x264.avs"
2007-04-15~16:31:54.68 Splitter: fewer queues than max_queues_split
2007-04-15~16:31:54.68 Local 2 sent video info 1280x720 @ 60000/1001
2007-04-15~16:31:54.68 split_largest_range found the biggest empty range to be 3330 frames long
2007-04-15~16:31:54.68 Local 2 sent zone string ""
2007-04-15~16:31:54.68 but that's not long enough! (5250)
2007-04-15~16:31:54.68 Local 2 sent range (0,5062)
2007-04-15~16:31:54.68 Splitter: split_largest_range did nothing, but there are enough queues
2007-04-15~16:31:54.68 ( 0, 5062) Working (Local 2)
2007-04-15~16:31:54.68 ( 5063, 8307) Empty
2007-04-15~16:31:54.68 ( 8308,11460) Queued
2007-04-15~16:31:54.68 (11461,16617) Queued
2007-04-15~16:31:54.68 (16618,19947) Empty
2007-04-15~16:31:54.68 (19948,23246) Queued
2007-04-15~16:31:54.68 Local 2 agent-based encoding; waiting for agent to finish
2007-04-15~16:31:54.68 Apple 1 found no ranges after full
2007-04-15~16:31:54.68 Apple 1 check_out_job got frames 8308 - 11460
2007-04-15~16:31:54.68 Splitter: fewer queues than max_queues_after_full
2007-04-15~16:31:54.70 Apple 1 sent full AVS info 33689C295A0DFBF882BE7FDCBA781B14 = "G:\/dvb/x264.avs"
2007-04-15~16:31:54.70 Splitter: fewer queues than max_queues_split
2007-04-15~16:31:54.70 Apple 1 sent video info 1280x720 @ 60000/1001
2007-04-15~16:31:54.70 split_largest_range found the biggest empty range to be 3330 frames long
2007-04-15~16:31:54.70 Apple 1 sent zone string ""
2007-04-15~16:31:54.70 but that's not long enough! (5250)
2007-04-15~16:31:54.70 Apple 1 sent range (8307,11460)
2007-04-15~16:31:54.70 Splitter: split_largest_range did nothing, but there are enough queues
2007-04-15~16:31:54.70 ( 0, 5062) Working (Local 2)
2007-04-15~16:31:54.70 ( 5063, 8307) Empty
2007-04-15~16:31:54.70 ( 8308,11460) Working (Apple 1)
2007-04-15~16:31:54.70 (11461,16617) Queued
2007-04-15~16:31:54.70 (16618,19947) Empty
2007-04-15~16:31:54.70 (19948,23246) Queued
2007-04-15~16:31:54.70 Apple 1 agent-based encoding; waiting for agent to finish
2007-04-15~16:31:54.73 Local 2 process_job failed with ("The operation completed successfully.\013\n",recv,); putting job (0,5062) back
2007-04-15~16:31:54.73 Apple 1 process_job failed with "End_of_file"; putting job (8308,11460) back
2007-04-15~16:31:54.73 put back range
2007-04-15~16:31:54.73 ( 0, 5062) Queued
2007-04-15~16:31:54.73 ( 5063, 8307) Empty
2007-04-15~16:31:54.73 ( 8308,11460) Working (Apple 1)
2007-04-15~16:31:54.73 (11461,16617) Queued
2007-04-15~16:31:54.73 (16618,19947) Empty
2007-04-15~16:31:54.73 (19948,23246) Queued
2007-04-15~16:31:54.73 put back range
2007-04-15~16:31:54.73 ( 0, 5062) Queued
2007-04-15~16:31:54.73 ( 5063, 8307) Empty
2007-04-15~16:31:54.73 ( 8308,11460) Queued
2007-04-15~16:31:54.73 (11461,16617) Queued
2007-04-15~16:31:54.75 (16618,19947) Empty
2007-04-15~16:31:54.75 (19948,23246) Queued
2007-04-15~16:31:54.73 Local 2 close connection
2007-04-15~16:31:54.75 Apple 1 close connection

E:\x264farm\temp\agent
2007-04-15~16:31:54.00 x264farm version 1.09-163
2007-04-15~16:31:54.00 Using config file ".\config.xml"
2007-04-15~16:31:54.00 Temp dir: "E:\x264farm\temp\agent"
2007-04-15~16:31:54.00 Ports: 50700 - 50703
2007-04-15~16:31:54.00 x264: "x264.exe"
2007-04-15~16:31:54.00 nice: 10
2007-04-15~16:31:54.00 agents: 1
2007-04-15~16:31:54.00 agent bases:
2007-04-15~16:31:54.00 [exact]
2007-04-15~16:31:54.00 pipe: false
2007-04-15~16:31:54.00 compressions: 0001
2007-04-15~16:31:54.00 deleting old files: after 604800 seconds
2007-04-15~16:31:54.00 trying port 50700
2007-04-15~16:31:54.00 STALE ignoring file "agent.exe"
2007-04-15~16:31:54.01 bound to port 50700
2007-04-15~16:31:54.01 STALE ignoring file "config.xml"
2007-04-15~16:31:54.01 listening...
2007-04-15~16:31:54.01 STALE not time for "out-dump.txt"
2007-04-15~16:31:54.01 working!
2007-04-15~16:31:54.01 STALE ignoring file "x264.exe"
2007-04-15~16:31:54.07 Got a connection from 169.254.25.121:4519
2007-04-15~16:31:54.07 working!
2007-04-15~16:31:54.07 Got a connection from 169.254.25.121:4522
2007-04-15~16:31:54.07 working!
2007-04-15~16:31:54.07 2 first pass frames 0 to 5062
2007-04-15~16:31:54.07 Testing file "G:\dvb\x264.avs": FOUND!
2007-04-15~16:31:54.07 2 agent-based encoding
2007-04-15~16:31:54.07 2 doing the following:
nice -n 10 x264.exe -2 --qcomp 0.6 --cplxblur 20. --qblur 0.5 --qpmin 10 --qpmax 51 --qpstep 4 --ipratio 1.4 --pbratio 1.3 --fps 60000/1001 --pass 1 --stats "E:\x264farm\temp\agent\stats b4efcd.txt" --seek 0 --frames 5063 -o NUL "G:\dvb\x264.avs"
2007-04-15~16:31:54.09 3 first pass frames 8307 to 11460
2007-04-15~16:31:54.09 Testing file "G:\dvb\x264.avs": FOUND!
2007-04-15~16:31:54.09 3 agent-based encoding
2007-04-15~16:31:54.09 3 doing the following:
nice -n 10 x264.exe -2 --qcomp 0.6 --cplxblur 20. --qblur 0.5 --qpmin 10 --qpmax 51 --qpstep 4 --ipratio 1.4 --pbratio 1.3 --fps 60000/1001 --pass 1 --stats "E:\x264farm\temp\agent\stats a79e8d.txt" --seek 8307 --frames 3154 -o NUL "G:\dvb\x264.avs"
2007-04-15~16:31:54.12 2 exited 1
2007-04-15~16:31:54.12 Thread received exception "Failure(\"ERROR: nonzero return code from x264 (1)\")"
2007-04-15~16:31:54.12 3 exited 1
2007-04-15~16:31:54.12 Thread received exception "Failure(\"ERROR: nonzero return code from x264 (1)\")"
2007-04-15~16:31:54.15 Got a connection from 169.254.25.121:4527
2007-04-15~16:31:54.17 working!
2007-04-15~16:31:54.17 4 first pass frames 0 to 5062
2007-04-15~16:31:54.17 Testing file "G:\dvb\x264.avs": FOUND!
2007-04-15~16:31:54.17 4 agent-based encoding
2007-04-15~16:31:54.17 4 doing the following:
nice -n 10 x264.exe -2 --qcomp 0.6 --cplxblur 20. --qblur 0.5 --qpmin 10 --qpmax 51 --qpstep 4 --ipratio 1.4 --pbratio 1.3 --fps 60000/1001 --pass 1 --stats "E:\x264farm\temp\agent\stats 71045f.txt" --seek 0 --frames 5063 -o NUL "G:\dvb\x264.avs"
2007-04-15~16:31:54.20 4 exited 1
2007-04-15~16:31:54.20 Thread received exception "Failure(\"ERROR: nonzero return code from x264 (1)\")"
2007-04-15~16:31:54.20 Got a connection from 169.254.25.121:4529
2007-04-15~16:31:54.20 working!
2007-04-15~16:31:54.21 5 first pass frames 0 to 5062
2007-04-15~16:31:54.21 Testing file "G:\dvb\x264.avs": FOUND!
2007-04-15~16:31:54.21 5 agent-based encoding
2007-04-15~16:31:54.21 5 doing the following:
nice -n 10 x264.exe -2 --qcomp 0.6 --cplxblur 20. --qblur 0.5 --qpmin 10 --qpmax 51 --qpstep 4 --ipratio 1.4 --pbratio 1.3 --fps 60000/1001 --pass 1 --stats "E:\x264farm\temp\agent\stats 1a9ec3.txt" --seek 0 --frames 5063 -o NUL "G:\dvb\x264.avs"
2007-04-15~16:31:54.25 5 exited 1
2007-04-15~16:31:54.25 Thread received exception "Failure(\"ERROR: nonzero return code from x264 (1)\")"
2007-04-15~16:31:54.25 Got a connection from 169.254.25.121:4530
2007-04-15~16:31:54.25 working

[omion]

E:\x264farm\temp\controller
2007-04-15~16:30:46.00 x264farm version 1.09-163
2007-04-15~16:30:46.00 Oops! File " G:\9\ka.mkv" does not exist


This looks a bit fishy. What was the command line that you ran the controller with?

2007-04-15~16:30:46.39 Oops. Writing the args failed with: Sys_error("E:\\x264farm\\temp\\controller\\option_log.txt.temp: No such file or directory")This is sort of odd, too, but I don't think it matters.

E:\x264farm\temp\agent
2007-04-15~16:31:54.17 4 doing the following:
nice -n 10 x264.exe -2 --qcomp 0.6 --cplxblur 20. --qblur 0.5 --qpmin 10 --qpmax 51 --qpstep 4 --ipratio 1.4 --pbratio 1.3 --fps 60000/1001 --pass 1 --stats "E:\x264farm\temp\agent\stats 71045f.txt" --seek 0 --frames 5063 -o NUL "G:\dvb\x264.avs"
2007-04-15~16:31:54.20 4 exited 1
2007-04-15~16:31:54.20 Thread received exception "Failure(\"ERROR: nonzero return code from x264 (1)\")"

This probably means that the agent can't find x264. If you just type "x264.exe" in the agent directory, what does it say?

If it can't find x264, you need to make sure that it's on the path (in the agent's dir is a good place) and set the <x264> line in the agent's config.xml file to the right executable (that is, if you are using "x264-634_omion.exe", make sure config.xml has the line <x264>x264-634_omion.exe</x264> in it)

[harissa]

I run controller
controller.exe --bitrate 3000kbps --avs "G:\dvb\x264.avs" --first --second -o "G:\ka.mkv"
after that on the same pc , even on another pc , i run agent.exe , and it go mad scrollin

i ve tried to put the x264.exe in both dir , dosen t change anything ,

is there any other setting except the config.xlm
I dont't know where to put nice.exe , avs2yuv.exe and x264-634_omion.exe
cauz I can t find how to make this run

[foxius]

I run controller
controller.exe --bitrate 3000kbps --avs "G:\dvb\x264.avs" --first --second -o "G:\ka.mkv"
after that on the same pc , even on another pc , i run agent.exe , and it go mad scrollin

i ve tried to put the x264.exe in both dir , dosen t change anything ,

is there any other setting except the config.xlm
I dont't know where to put nice.exe , avs2yuv.exe and x264-634_omion.exe
cauz I can t find how to make this run

1. nice.exe goes to agent's folder
2. avs2yuv.exe goes to controller's folder
3. x264-634_omion.exe should be renamed to x264.exe and put into agent's folder (unless you've changed x264 path in agent's config.xml)

[Wishbringer]

@omion:
is your x264 build a plain compile from source with MS Visual Studio,
or did you some changes in source?
I just tried to compile build 650 with Studio 2005, no errors, but i can't get controller based encode to run.

[harissa]

i fixed evrything still not workin i m postin all info this time
E:\x264farm\temp\controller
has avs2yuv.exe controller.exe config.xml
<config>
<temp>E:\x264farm\temp\controller</temp> <!-- Some directory for storing temp data -->
<agents>
<agent name="Local"> <!-- The name of the agent. This is only informational -->
<ip>169.254.25.121</ip> <!-- IP address of the agent (127.0.0.1 is the computer the controller is running on) -->
<port from="50700" to="50703"/> <!-- A port range. This MUST be identical to the setup of the agent -->
<number>2</number> <!-- How many jobs to send to the agent at one time. Should generally be equal to the number of cores on the target machine -->
</agent>
<agent name="Linux">
<ip>169.254.23.92</ip>
<port from="50700" to="50703"/>
<number pad="1">1</number>
<!--
The pad="1" -->
</agent>
<agent name="Apple">
<ip>169.254.25.121</ip>
<port from="50700" to="50703"/>
<number pad="1">2</number>
</agent>

</agents>

</config>

here the log from the controller
controller.exe --bitrate 3000kbps --avs "G:\dvb\x264.avs" --first --second -o "G:\ka.mkv"

after this point it start scrollin non stop once the agent open
2007-04-16~08:33:18.00 x264farm version 1.09-163
2007-04-16~08:33:18.00 WARNING: Second pass options are not set; this will use the default x264 settings
2007-04-16~08:33:18.00 Using config file ".\config.xml"
2007-04-16~08:33:18.00 CONFIG:
2007-04-16~08:33:18.00 temp_dir: "E:\\x264farm\\temp\\controller"
2007-04-16~08:33:18.00 agent_list 1:
2007-04-16~08:33:18.00 "Local 1" = 169.254.25.121 50700-50703
2007-04-16~08:33:18.00 "Local 2" = 169.254.25.121 50700-50703
2007-04-16~08:33:18.00 "Linux 1" = 169.254.23.92 50700-50703
2007-04-16~08:33:18.00 "Apple 1" = 169.254.25.121 50700-50703
2007-04-16~08:33:18.00 "Apple 2" = 169.254.25.121 50700-50703
2007-04-16~08:33:18.01 agent_list 2:
2007-04-16~08:33:18.01 "Local 1" = 169.254.25.121 50700-50703
2007-04-16~08:33:18.01 "Local 2" = 169.254.25.121 50700-50703
2007-04-16~08:33:18.01 "Linux 1" = 169.254.23.92 50700-50703
2007-04-16~08:33:18.01 "Linux 2" = 169.254.23.92 50700-50703
2007-04-16~08:33:18.01 "Apple 1" = 169.254.25.121 50700-50703
2007-04-16~08:33:18.01 "Apple 2" = 169.254.25.121 50700-50703
2007-04-16~08:33:18.01 "Apple 3" = 169.254.25.121 50700-50703
2007-04-16~08:33:18.39 GOT NORMAL INFO "G:\\dvb\\x264.avs: 1280x720, 60000/1001 fps, 23247 frames\013"
2007-04-16~08:33:18.50 GOT FIRST INFO "G:\\dvb\\x264.avs: 1280x720, 60000/1001 fps, 23247 frames\013"
2007-04-16~08:33:18.62 GOT FAST INFO "G:\\dvb\\x264.avs: 1280x720, 60000/1001 fps, 23247 frames\013"
2007-04-16~08:33:18.62 INFO:
2007-04-16~08:33:18.62 i_res_x = 1280
2007-04-16~08:33:18.62 i_res_y = 720
2007-04-16~08:33:18.62 i_fps_n = 60000
2007-04-16~08:33:18.62 i_fps_d = 1001
2007-04-16~08:33:18.62 i_fps_f = 59.940060
2007-04-16~08:33:18.62 i_num_frames = 23247
2007-04-16~08:33:18.62 i_bytes_y = 921600
2007-04-16~08:33:18.62 i_bytes_uv = 230400
2007-04-16~08:33:18.62 i_bytes_per_frame = 1382400
2007-04-16~08:33:18.62 multiplier for kbps to fbits = 0.059940
2007-04-16~08:33:18.62 fast_res_x = 1280
2007-04-16~08:33:18.62 fast_res_y = 720
2007-04-16~08:33:18.62 Oops. Writing the args failed with: Sys_error("E:\\x264farm\\temp\\controller\\option_log.txt.temp: No such file or directory")2007-04-16~08:33:18.64 AVS temp dir is E:\x264farm\temp\controller\132FA1D98D68B80231C0CECCD0F61FAF x264.avs
2007-04-16~08:33:18.64 First string is "--second\n„•¦¾\000\000\000\001\000\000\000\000\000\000\000\000\000\000\000\000@\n0+0\n0\n3hœ)Z\013ûø‚¾\127ܺx\027\020 = G:\\dvb\\x264.avs\n", with MD5 "83936EFFC345B4F1C55FA600066A3830"
2007-04-16~08:33:18.84 Second string is "\n„•¦¾\000\000\000\001\000\000\000\000\000\000\000\000\000\000\000\000@\n0+0\n0\n", with MD5 "A25B8F0CCB96F56266CFA33B07D0F828"
2007-04-16~08:33:19.04 not (not (is_something first_dir) && is_file first_stats) [first pass after making dir]
2007-04-16~08:33:19.04 11111111 FIRST PASS 11111111
2007-04-16~08:33:19.04 #1 3 > #2 2 (read #1)
2007-04-16~08:33:19.04 From file:
2007-04-16~08:33:19.04 ( 0,23246) Empty
2007-04-16~08:33:19.04 Local 1 starting on 169.254.25.121 (id 1)
2007-04-16~08:33:19.04 Local 2 starting on 169.254.25.121 (id 2)
2007-04-16~08:33:19.04 Apple 1 starting on 169.254.25.121 (id 4)
2007-04-16~08:33:19.04 Linux 1 starting on 169.254.23.92 (id 3)
2007-04-16~08:33:19.04 Apple 2 starting on 169.254.25.121 (id 5)
2007-04-16~08:33:19.04 Splitter starting
2007-04-16~08:33:19.06 Split points at 5,3,1,1
2007-04-16~08:33:21.73 queue_beginning found the beginning; split to Q(0,5062) :: (5063,23246)
2007-04-16~08:33:21.73 Ranges at beginning:
2007-04-16~08:33:21.75 ( 0, 5062) Queued
2007-04-16~08:33:21.75 ( 5063,23246) Empty
2007-04-16~08:33:21.75 Splitter: fewer queues than max_queues_after_full
2007-04-16~08:33:21.75 Splitter: fewer queues than max_queues_split
2007-04-16~08:33:21.75 split_largest_range found the biggest empty range to be 18184 frames long
2007-04-16~08:33:24.60 split_largest_range found start I frame 11460 in (11405,11655)
2007-04-16~08:33:27.07 split_largest_range split to (5063,11460) :: (11461,16617) :: (16618,23246)
2007-04-16~08:33:27.09 Splitter: split_largest_range did something; re-run
2007-04-16~08:33:27.09 Splitter: fewer queues than max_queues_after_full
2007-04-16~08:33:27.09 Splitter: fewer queues than max_queues_split
2007-04-16~08:33:27.09 split_largest_range found the biggest empty range to be 6629 frames long
2007-04-16~08:33:29.15 split_largest_range found start I frame 19947 in (19807,20057)
2007-04-16~08:33:29.15 Splitter: split_largest_range did something; re-run
2007-04-16~08:33:29.15 Splitter: fewer queues than max_queues_after_full
2007-04-16~08:33:29.15 Splitter: fewer queues than max_queues_split
2007-04-16~08:33:29.15 split_largest_range found the biggest empty range to be 6398 frames long
2007-04-16~08:33:32.35 split_largest_range found start I frame 8307 in (8137,8387)
2007-04-16~08:33:32.35 Splitter: split_largest_range did something; re-run
2007-04-16~08:33:32.35 Splitter: fewer queues than max_queues_after_full
2007-04-16~08:33:32.35 Splitter: fewer queues than max_queues_split
2007-04-16~08:33:32.35 split_largest_range found the biggest empty range to be 3330 frames long
2007-04-16~08:33:32.35 but that's not long enough! (5250)
2007-04-16~08:33:32.35 Splitter: split_largest_range did nothing, but there are enough queues
2007-04-16~08:33:32.35 ( 0, 5062) Queued
2007-04-16~08:33:32.35 ( 5063, 8307) Empty
2007-04-16~08:33:32.35 ( 8308,11460) Queued
2007-04-16~08:33:32.35 (11461,16617) Queued
2007-04-16~08:33:32.35 (16618,19947) Empty
2007-04-16~08:33:32.35 (19948,23246) Queued
2007-04-16~08:33:55.40 using compression type 0 (no compression)
2007-04-16~08:33:55.40 Local 2 found no ranges after full
2007-04-16~08:33:55.40 Local 2 check_out_job got frames 0 - 5062
2007-04-16~08:33:55.40 Splitter: fewer queues than max_queues_after_full
2007-04-16~08:33:55.40 Local 2 sent full AVS info 33689C295A0DFBF882BE7FDCBA781B14 = "G:\/dvb/x264.avs"
2007-04-16~08:33:55.40 Splitter: fewer queues than max_queues_split
2007-04-16~08:33:55.40 Local 2 sent video info 1280x720 @ 60000/1001
2007-04-16~08:33:55.40 split_largest_range found the biggest empty range to be 3330 frames long
2007-04-16~08:33:55.40 Local 2 sent zone string ""
2007-04-16~08:33:55.40 but that's not long enough! (5250)
2007-04-16~08:33:55.40 Local 2 sent range (0,5062)
2007-04-16~08:33:55.40 Splitter: split_largest_range did nothing, but there are enough queues
2007-04-16~08:33:55.40 ( 0, 5062) Working (Local 2)
2007-04-16~08:33:55.40 ( 5063, 8307) Empty
2007-04-16~08:33:55.40 ( 8308,11460) Queued
2007-04-16~08:33:55.40 (11461,16617) Queued
2007-04-16~08:33:55.42 (16618,19947) Empty
2007-04-16~08:33:55.42 (19948,23246) Queued
2007-04-16~08:33:55.40 Local 2 agent-based encoding; waiting for agent to finish
2007-04-16~08:33:55.60 using compression type 0 (no compression)
2007-04-16~08:33:55.60 Apple 2 found no ranges after full
2007-04-16~08:33:55.60 Apple 2 check_out_job got frames 8308 - 11460
2007-04-16~08:33:55.60 Splitter: fewer queues than max_queues_after_full
2007-04-16~08:33:55.60 Apple 2 sent full AVS info 33689C295A0DFBF882BE7FDCBA781B14 = "G:\/dvb/x264.avs"
2007-04-16~08:33:55.60 Splitter: fewer queues than max_queues_split
2007-04-16~08:33:55.60 Apple 2 sent video info 1280x720 @ 60000/1001
2007-04-16~08:33:55.60 split_largest_range found the biggest empty range to be 3330 frames long
2007-04-16~08:33:55.60 Apple 2 sent zone string ""
2007-04-16~08:33:55.60 but that's not long enough! (5250)
2007-04-16~08:33:55.60 Apple 2 sent range (8307,11460)
2007-04-16~08:33:55.60 Splitter: split_largest_range did nothing, but there are enough queues
2007-04-16~08:33:55.60 ( 0, 5062) Working (Local 2)
2007-04-16~08:33:55.60 ( 5063, 8307) Empty
2007-04-16~08:33:55.60 ( 8308,11460) Working (Apple 2)
2007-04-16~08:33:55.60 (11461,16617) Queued
2007-04-16~08:33:55.60 (16618,19947) Empty
2007-04-16~08:33:55.60 (19948,23246) Queued

[harissa]

E:\x264farm\temp\agent , on the same computer
agent.exe x264.exe nice.exe config.xml
<config>
<temp>E:\x264farm\temp\agent</temp>

2007-04-16~08:33:49.00 x264farm version 1.09-163
2007-04-16~08:33:49.00 Using config file ".\config.xml"
2007-04-16~08:33:49.00 Temp dir: "E:\x264farm\temp\agent"
2007-04-16~08:33:49.00 Ports: 50700 - 50703
2007-04-16~08:33:49.00 x264: "x264.exe"
2007-04-16~08:33:49.00 nice: 10
2007-04-16~08:33:49.00 agents: 1
2007-04-16~08:33:49.00 agent bases:
2007-04-16~08:33:49.00 [exact]
2007-04-16~08:33:49.00 pipe: false
2007-04-16~08:33:49.00 compressions: 0001
2007-04-16~08:33:49.00 deleting old files: after 604800 seconds
2007-04-16~08:33:49.00 trying port 50700
2007-04-16~08:33:49.00 STALE ignoring file "agent.exe"
2007-04-16~08:33:49.01 bound to port 50700
2007-04-16~08:33:49.01 STALE ignoring file "config.xml"
2007-04-16~08:33:49.01 listening...
2007-04-16~08:33:49.01 STALE ignoring file "nice.exe"
2007-04-16~08:33:49.01 working!
2007-04-16~08:33:49.01 STALE not time for "out-dump.txt"
2007-04-16~08:33:49.01 STALE ignoring file "x264.exe"
2007-04-16~08:33:55.57 Got a connection from 169.254.25.121:1624
2007-04-16~08:33:55.57 working!
2007-04-16~08:33:55.57 2 first pass frames 0 to 5062
2007-04-16~08:33:55.57 Testing file "G:\dvb\x264.avs": FOUND!
2007-04-16~08:33:55.57 2 agent-based encoding
2007-04-16~08:33:55.57 2 doing the following:
nice -n 10 x264.exe --second --qcomp 0.6 --cplxblur 20. --qblur 0.5 --qpmin 10 --qpmax 51 --qpstep 4 --ipratio 1.4 --pbratio 1.3 --fps 60000/1001 --pass 1 --stats "E:\x264farm\temp\agent\stats 45b9ad.txt" --seek 0 --frames 5063 -o NUL "G:\dvb\x264.avs"
2007-04-16~08:33:55.78 Got a connection from 169.254.25.121:1627
2007-04-16~08:33:55.78 working!
2007-04-16~08:33:55.78 3 first pass frames 8307 to 11460
2007-04-16~08:33:55.78 Testing file "G:\dvb\x264.avs": FOUND!
2007-04-16~08:33:55.78 3 agent-based encoding
2007-04-16~08:33:55.78 3 doing the following:
nice -n 10 x264.exe --second --qcomp 0.6 --cplxblur 20. --qblur 0.5 --qpmin 10 --qpmax 51 --qpstep 4 --ipratio 1.4 --pbratio 1.3 --fps 60000/1001 --pass 1 --stats "E:\x264farm\temp\agent\stats 5d3a0f.txt" --seek 8307 --frames 3154 -o NUL "G:\dvb\x264.avs"
2007-04-16~08:33:56.00 Got a connection from 169.254.25.121:1625
2007-04-16~08:33:56.00 working!
2007-04-16~08:33:56.01 4 first pass frames 19947 to 23246
2007-04-16~08:33:56.01 Testing file "G:\dvb\x264.avs": FOUND!
2007-04-16~08:33:56.01 4 agent-based encoding
2007-04-16~08:33:56.01 4 doing the following:
nice -n 10 x264.exe --second --qcomp 0.6 --cplxblur 20. --qblur 0.5 --qpmin 10 --qpmax 51 --qpstep 4 --ipratio 1.4 --pbratio 1.3 --fps 60000/1001 --pass 1 --stats "E:\x264farm\temp\agent\stats 9f4a62.txt" --seek 19947 --frames 3300 -o NUL "G:\dvb\x264.avs"
2007-04-16~08:33:56.17 Thread received exception "Sys_error(\"nice -n 10 x264.exe --second --qcomp 0.6 --cplxblur 20. --qblur 0.5 --qpmin 10 --qpmax 51 --qpstep 4 --ipratio 1.4 --pbratio 1.3 --fps 60000/1001 --pass 1 --stats \\\"E:\\\\x264farm\\\\temp\\\\agent\\\\stats 45b9ad.txt\\\" --seek 0 --frames 5063 -o NUL \\\"G:\\\\dvb\\\\x264.avs\\\": No error\")"
2007-04-16~08:33:56.18 Thread received exception "Sys_error(\"nice -n 10 x264.exe --second --qcomp 0.6 --cplxblur 20. --qblur 0.5 --qpmin 10 --qpmax 51 --qpstep 4 --ipratio 1.4 --pbratio 1.3 --fps 60000/1001 --pass 1 --stats \\\"E:\\\\x264farm\\\\temp\\\\agent\\\\stats 5d3a0f.txt\\\" --seek 8307 --frames 3154 -o NUL \\\"G:\\\\dvb\\\\x264.avs\\\": No error\")"
2007-04-16~08:33:56.18 Thread received exception "Sys_error(\"nice -n 10 x264.exe --second --qcomp 0.6 --cplxblur 20. --qblur 0.5 --qpmin 10 --qpmax 51 --qpstep 4 --ipratio 1.4 --pbratio 1.3 --fps 60000/1001 --pass 1 --stats \\\"E:\\\\x264farm\\\\temp\\\\agent\\\\stats 9f4a62.txt\\\" --seek 19947 --frames 3300 -o NUL \\\"G:\\\\dvb\\\\x264.avs\\\": No error\")"
2007-04-16~08:33:56.20 Got a connection from 169.254.25.121:1628
2007-04-16~08:33:56.20 working!
2007-04-16~08:33:56.21 Got a connection from 169.254.25.121:1629
2007-04-16~08:33:56.21 working!
2007-04-16~08:33:56.21 Got a connection from 169.254.25.121:1630
2007-04-16~08:33:56.21 working!
2007-04-16~08:33:56.23 5 first pass frames 0 to 5062
2007-04-16~08:33:56.23 Testing file "G:\dvb\x264.avs": FOUND!
2007-04-16~08:33:56.23 5 agent-based encoding
2007-04-16~08:33:56.23 5 doing the following:
nice -n 10 x264.exe --second --qcomp 0.6 --cplxblur 20. --qblur 0.5 --qpmin 10 --qpmax 51 --qpstep 4 --ipratio 1.4 --pbratio 1.3 --fps 60000/1001 --pass 1 --stats "E:\x264farm\temp\agent\stats 5484d5.txt" --seek 0 --frames 5063 -o NUL "G:\dvb\x264.avs"
2007-04-16~08:33:56.25 7 first pass frames 19947 to 23246
2007-04-16~08:33:56.25 6 first pass frames 8307 to 11460
2007-04-16~08:33:56.25 Testing file "G:\dvb\x264.avs": FOUND!
2007-04-16~08:33:56.25 Testing file "G:\dvb\x264.avs": FOUND!
2007-04-16~08:33:56.25 7 agent-based encoding
2007-04-16~08:33:56.25 6 agent-based encoding
2007-04-16~08:33:56.25 7 doing the following:
nice -n 10 x264.exe --second --qcomp 0.6 --cplxblur 20. --qblur 0.5 --qpmin 10 --qpmax 51 --qpstep 4 --ipratio 1.4 --pbratio 1.3 --fps 60000/1001 --pass 1 --stats "E:\x264farm\temp\agent\stats b883ee.txt" --seek 19947 --frames 3300 -o NUL "G:\dvb\x264.avs"
2007-04-16~08:33:56.25 6 doing the following:
nice -n 10 x264.exe --second --qcomp 0.6 --cplxblur 20. --qblur 0.5 --qpmin 10 --qpmax 51 --qpstep 4 --ipratio 1.4 --pbratio 1.3 --fps 60000/1001 --pass 1 --stats "E:\x264farm\temp\agent\stats e89baf.txt" --seek 8307 --frames 3154 -o NUL "G:\dvb\x264.avs"
2007-04-16~08:33:56.29 Thread received exception "Sys_error(\"nice -n 10 x264.exe --second --qcomp 0.6 --cplxblur 20. --qblur 0.5 --qpmin 10 --qpmax 51 --qpstep 4 --ipratio 1.4 --pbratio 1.3 --fps 60000/1001 --pass 1 --stats \\\"E:\\\\x264farm\\\\temp\\\\agent\\\\stats 5484d5.txt\\\" --seek 0 --frames 5063 -o NUL \\\"G:\\\\dvb\\\\x264.avs\\\": No error\")"
2007-04-16~08:33:56.31 Thread received exception "Sys_error(\"nice -n 10 x264.exe --second --qcomp 0.6 --cplxblur 20. --qblur 0.5 --qpmin 10 --qpmax 51 --qpstep 4 --ipratio 1.4 --pbratio 1.3 --fps 60000/1001 --pass 1 --stats \\\"E:\\\\x264farm\\\\temp\\\\agent\\\\stats b883ee.txt\\\" --seek 19947 --frames 3300 -o NUL \\\"G:\\\\dvb\\\\x264.avs\\\": No error\")"
2007-04-16~08:33:56.32 Thread received exception "Sys_error(\"nice -n 10 x264.exe --second --qcomp 0.6 --cplxblur 20. --qblur 0.5 --qpmin 10 --qpmax 51 --qpstep 4 --ipratio 1.4 --pbratio 1.3 --fps 60000/1001 --pass 1 --stats \\\"E:\\\\x264farm\\\\temp\\\\agent\\\\stats e89baf.txt\\\" --seek 8307 --frames 3154 -o NUL \\\"G:\\\\dvb\\\\x264.avs\\\": No error\")"
2007-04-16~08:33:56.34 Got a connection from 169.254.25.121:1631
2007-04-16~08:33:56.34 working!
2007-04-16~08:33:56.34 Got a connection from 169.254.25.121:1632
2007-04-16~08:33:56.34 working!
2007-04-16~08:33:56.34 Got a connection from 169.254.25.121:1633
2007-04-16~08:33:56.34 working!
2007-04-16~08:33:56.35 8 first pass frames 0 to 5062
2007-04-16~08:33:56.35 Testing file "G:\dvb\x264.avs": FOUND!
2007-04-16~08:33:56.37 8 agent-based encoding
2007-04-16~08:33:56.37 8 doing the following:
nice -n 10 x264.exe --second --qcomp 0.6 --cplxblur 20. --qblur 0.5 --qpmin 10 --qpmax 51 --qpstep 4 --ipratio 1.4 --pbratio 1.3 --fps 60000/1001 --pass 1 --stats "E:\x264farm\temp\agent\stats b989cd.txt" --seek 0 --frames 5063 -o NUL "G:\dvb\x264.avs"
2007-04-16~08:33:56.39 9 first pass frames 8307 to 11460
2007-04-16~08:33:56.39 10 first pass frames 19947 to 23246
2007-04-16~08:33:56.39 Testing file "G:\dvb\x264.avs": FOUND!
2007-04-16~08:33:56.39 Testing file "G:\dvb\x264.avs": FOUND!
2007-04-16~08:33:56.39 9 agent-based encoding
2007-04-16~08:33:56.39 10 agent-based encoding
2007-04-16~08:33:56.39 9 doing the following:

[foxius]

harissa, please ensure the following folders exist prior to running the controller and agent:

E:\x264farm\temp\controller
E:\x264farm\temp\agent

it seems to me that the controller is not able to write to the temp folder.

[foxius]

harissa, please ensure the following folders exist prior to running the controller and agent:

E:\x264farm\temp\controller
E:\x264farm\temp\agent

it seems to me that the controller is not able to write to the temp folder.

here's my x264 folder tree for reference.


x264farm
│ gpl.txt
│ x264farm.html
│
├───agent-win32
│ │ agent.exe
│ │ config.xml
│ │ gpl.txt
│ │ nice.exe
│ │ out-dump.txt
│ │ x264.exe
│ │
│ └───temp
└───controller
│ avs2yuv.exe
│ config.xml
│ controller.exe
│ enc.bat
│ out-dump.txt
│
└───temp


and here's my agent config:


<config>
<temp>.\temp</temp>

<port from="50700" to="50703"/>

<x264>x264.exe</x264>

<nice>20</nice>

<base>d:\</base>

<agentpipe>0</agentpipe>

<compression>
<type>0</type>
<type>1</type>
</compression>

</config>


and here's controller config:


<config>
<temp>.\temp</temp> <!-- Some directory for storing temp data -->

<agents>
<agent name="Local">
<ip>127.0.0.1</ip>
<port from="50700" to="50703"/>
<number>1</number>
</agent>

<agent name="agent1">
<ip>192.168.13.1</ip>
<port from="50700" to="50703"/>
<number>1</number>
</agent>

<agent name="agent2">
<ip>192.168.13.15</ip>
<port from="50700" to="50703"/>
<number>1</number>
</agent>
</agents>
</config>


as you can see, i keep temp folders inside agent's and controller's folder and use a relative path in the config. this allows me to copy them to any path without modifying the config at all.

[harissa]

i ve tried that
i have E:\x264farm\temp
4 dir inside
controller and agent are temp dir ,
and controller2 dir, contain all exe
and agent2 dir contain all exe

[harissa]

I ve done the same setting as u do , i have some improvement less error msg , but both windows are scrollin like hell , can i post again the log for 2 agent so u can take a look

[foxius]

I ve done the same setting as u do , i have some improvement less error msg , but both windows are scrollin like hell , can i post again the log for 2 agent so u can take a look

sure. do you get the errors only in agents' windows now? controller is ok?

[harissa]

same dir as you same commande line
2007-04-16~10:09:46.00 x264farm version 1.09-163
2007-04-16~10:09:46.00 WARNING: Second pass options are not set; this will use the default x264 settings
2007-04-16~10:09:46.00 Using config file ".\config.xml"
2007-04-16~10:09:46.00 CONFIG:
2007-04-16~10:09:46.00 temp_dir: "E:\\x264farm\\temp\\controller\\temp"
2007-04-16~10:09:46.00 agent_list 1:
2007-04-16~10:09:46.00 "Local 1" = 169.254.25.121 50700-50703
2007-04-16~10:09:46.00 "Local 2" = 169.254.25.121 50700-50703
2007-04-16~10:09:46.00 "pc2 1" = 169.254.23.92 50700-50703
2007-04-16~10:09:46.00 "pc3 1" = 169.254.25.121 50700-50703
2007-04-16~10:09:46.00 "pc3 2" = 169.254.25.121 50700-50703
2007-04-16~10:09:46.00 agent_list 2:
2007-04-16~10:09:46.00 "Local 1" = 169.254.25.121 50700-50703
2007-04-16~10:09:46.00 "Local 2" = 169.254.25.121 50700-50703
2007-04-16~10:09:46.01 "pc2 1" = 169.254.23.92 50700-50703
2007-04-16~10:09:46.01 "pc2 2" = 169.254.23.92 50700-50703
2007-04-16~10:09:46.01 "pc3 1" = 169.254.25.121 50700-50703
2007-04-16~10:09:46.01 "pc3 2" = 169.254.25.121 50700-50703
2007-04-16~10:09:46.01 "pc3 3" = 169.254.25.121 50700-50703
2007-04-16~10:09:46.12 GOT NORMAL INFO "G:\\dvb\\x264.avs: 1280x720, 60000/1001 fps, 23247 frames\013"
2007-04-16~10:09:46.23 GOT FIRST INFO "G:\\dvb\\x264.avs: 1280x720, 60000/1001 fps, 23247 frames\013"
2007-04-16~10:09:46.32 GOT FAST INFO "G:\\dvb\\x264.avs: 1280x720, 60000/1001 fps, 23247 frames\013"
2007-04-16~10:09:46.34 INFO:
2007-04-16~10:09:46.34 i_res_x = 1280
2007-04-16~10:09:46.34 i_res_y = 720
2007-04-16~10:09:46.34 i_fps_n = 60000
2007-04-16~10:09:46.34 i_fps_d = 1001
2007-04-16~10:09:46.34 i_fps_f = 59.940060
2007-04-16~10:09:46.34 i_num_frames = 23247
2007-04-16~10:09:46.34 i_bytes_y = 921600
2007-04-16~10:09:46.34 i_bytes_uv = 230400
2007-04-16~10:09:46.34 i_bytes_per_frame = 1382400
2007-04-16~10:09:46.34 multiplier for kbps to fbits = 0.059940
2007-04-16~10:09:46.34 fast_res_x = 1280
2007-04-16~10:09:46.34 fast_res_y = 720
2007-04-16~10:09:46.34 Oops. Writing the args failed with: Sys_error("E:\\x264farm\\temp\\controller\\temp\\option_log.txt.temp: No such file or directory")2007-04-16~10:09:46.34 AVS temp dir is E:\x264farm\temp\controller\temp\132FA1D98D68B80231C0CECCD0F61FAF x264.avs
2007-04-16~10:09:46.34 First string is "--second\n„•¦¾\000\000\000\001\000\000\000\000\000\000\000\000\000\000\000\000@\n0+0\n0\n3hœ)Z\013ûø‚¾\127ܺx\027\020 = G:\\dvb\\x264.avs\n", with MD5 "83936EFFC345B4F1C55FA600066A3830"
2007-04-16~10:09:46.54 Second string is "\n„•¦¾\000\000\000\001\000\000\000\000\000\000\000\000\000\000\000\000@\n0+0\n0\n", with MD5 "A25B8F0CCB96F56266CFA33B07D0F828"
2007-04-16~10:09:46.75 not (not (is_something first_dir) && is_file first_stats) [first pass after making dir]
2007-04-16~10:09:46.76 11111111 FIRST PASS 11111111
2007-04-16~10:09:46.76 #1 1 < #2 2 (read #2)
2007-04-16~10:09:46.76 From file:
2007-04-16~10:09:46.76 ( 0,23246) Empty
2007-04-16~10:09:46.76 Local 1 starting on 169.254.25.121 (id 1)
2007-04-16~10:09:46.76 Local 2 starting on 169.254.25.121 (id 2)
2007-04-16~10:09:46.76 pc2 1 starting on 169.254.23.92 (id 3)
2007-04-16~10:09:46.76 pc3 1 starting on 169.254.25.121 (id 4)
2007-04-16~10:09:46.76 pc3 2 starting on 169.254.25.121 (id 5)
2007-04-16~10:09:46.76 Splitter starting
2007-04-16~10:09:46.76 Split points at 5,3,1,1
2007-04-16~10:09:49.90 queue_beginning found the beginning; split to Q(0,5062) :: (5063,23246)
2007-04-16~10:09:49.90 Ranges at beginning:
2007-04-16~10:09:49.90 ( 0, 5062) Queued
2007-04-16~10:09:49.90 ( 5063,23246) Empty
2007-04-16~10:09:49.90 Splitter: fewer queues than max_queues_after_full
2007-04-16~10:09:49.90 Splitter: fewer queues than max_queues_split
2007-04-16~10:09:49.90 split_largest_range found the biggest empty range to be 18184 frames long
2007-04-16~10:09:53.43 split_largest_range found start I frame 11460 in (11405,11655)
2007-04-16~10:09:56.29 split_largest_range split to (5063,11460) :: (11461,16617) :: (16618,23246)
2007-04-16~10:09:56.29 Splitter: split_largest_range did something; re-run
2007-04-16~10:09:56.29 Splitter: fewer queues than max_queues_after_full
2007-04-16~10:09:56.29 Splitter: fewer queues than max_queues_split
2007-04-16~10:09:56.29 split_largest_range found the biggest empty range to be 6629 frames long
2007-04-16~10:09:58.78 split_largest_range found start I frame 19947 in (19807,20057)
2007-04-16~10:09:58.78 Splitter: split_largest_range did something; re-run
2007-04-16~10:09:58.78 Splitter: fewer queues than max_queues_after_full
2007-04-16~10:09:58.78 Splitter: fewer queues than max_queues_split
2007-04-16~10:09:58.78 split_largest_range found the biggest empty range to be 6398 frames long
2007-04-16~10:10:02.56 split_largest_range found start I frame 8307 in (8137,8387)
2007-04-16~10:10:02.57 Splitter: split_largest_range did something; re-run
2007-04-16~10:10:02.57 Splitter: fewer queues than max_queues_after_full
2007-04-16~10:10:02.57 Splitter: fewer queues than max_queues_split
2007-04-16~10:10:02.57 split_largest_range found the biggest empty range to be 3330 frames long
2007-04-16~10:10:02.57 but that's not long enough! (5250)
2007-04-16~10:10:02.57 Splitter: split_largest_range did nothing, but there are enough queues
2007-04-16~10:10:02.57 ( 0, 5062) Queued
2007-04-16~10:10:02.57 ( 5063, 8307) Empty
2007-04-16~10:10:02.57 ( 8308,11460) Queued
2007-04-16~10:10:02.57 (11461,16617) Queued
2007-04-16~10:10:02.57 (16618,19947) Empty
2007-04-16~10:10:02.57 (19948,23246) Queued
2007-04-16~10:10:07.57 using compression type 0 (no compression)
2007-04-16~10:10:07.57 Local 2 found no ranges after full
2007-04-16~10:10:07.57 Local 2 check_out_job got frames 0 - 5062
2007-04-16~10:10:07.57 Splitter: fewer queues than max_queues_after_full
2007-04-16~10:10:07.57 Local 2 sent full AVS info 33689C295A0DFBF882BE7FDCBA781B14 = "G:\/dvb/x264.avs"
2007-04-16~10:10:07.57 Splitter: fewer queues than max_queues_split
2007-04-16~10:10:07.57 Local 2 sent video info 1280x720 @ 60000/1001
2007-04-16~10:10:07.57 split_largest_range found the biggest empty range to be 3330 frames long
2007-04-16~10:10:07.57 Local 2 sent zone string ""
2007-04-16~10:10:07.57 but that's not long enough! (5250)
2007-04-16~10:10:07.57 Local 2 sent range (0,5062)
2007-04-16~10:10:07.57 Splitter: split_largest_range did nothing, but there are enough queues
2007-04-16~10:10:07.57 ( 0, 5062) Working (Local 2)
2007-04-16~10:10:07.57 ( 5063, 8307) Empty
2007-04-16~10:10:07.57 ( 8308,11460) Queued
2007-04-16~10:10:07.57 (11461,16617) Queued
2007-04-16~10:10:07.57 (16618,19947) Empty
2007-04-16~10:10:07.57 (19948,23246) Queued
2007-04-16~10:10:07.57 Local 2 agent-based encoding; waiting for agent to finish
2007-04-16~10:10:07.96 Local 2 process_job failed with "End_of_file"; putting job (0,5062) back
2007-04-16~10:10:07.96 put back range
2007-04-16~10:10:07.96 ( 0, 5062) Queued
2007-04-16~10:10:07.96 ( 5063, 8307) Empty
2007-04-16~10:10:07.96 ( 8308,11460) Queued
2007-04-16~10:10:07.96 (11461,16617) Queued
2007-04-16~10:10:07.96 (16618,19947) Empty
2007-04-16~10:10:07.96 (19948,23246) Queued
2007-04-16~10:10:07.96 Local 2 close connection
2007-04-16~10:10:07.98 using compression type 0 (no compression)
2007-04-16~10:10:07.98 Local 2 found no ranges after full
2007-04-16~10:10:07.98 Local 2 check_out_job got frames 0 - 5062
2007-04-16~10:10:07.98 Splitter: fewer queues than max_queues_after_full
2007-04-16~10:10:07.98 Local 2 sent full AVS info 33689C295A0DFBF882BE7FDCBA781B14 = "G:\/dvb/x264.avs"
2007-04-16~10:10:07.98 Splitter: fewer queues than max_queues_split
2007-04-16~10:10:07.98 Local 2 sent video info 1280x720 @ 60000/1001
2007-04-16~10:10:07.98 split_largest_range found the biggest empty range to be 3330 frames long
2007-04-16~10:10:07.98 Local 2 sent zone string ""
2007-04-16~10:10:07.98 but that's not long enough! (5250)
2007-04-16~10:10:07.98 Local 2 sent range (0,5062)
2007-04-16~10:10:07.98 Splitter: split_largest_range did nothing, but there are enough queues
2007-04-16~10:10:07.98 ( 0, 5062) Working (Local 2)
2007-04-16~10:10:07.98 ( 5063, 8307) Empty
2007-04-16~10:10:07.98 ( 8308,11460) Queued
2007-04-16~10:10:07.98 (11461,16617) Queued
2007-04-16~10:10:07.98 (16618,19947) Empty
2007-04-16~10:10:07.98 (19948,23246) Queued
2007-04-16~10:10:07.98 Local 2 agent-based encoding; waiting for agent to finish

and agent log
2007-04-16~10:10:02.00 x264farm version 1.09-163
2007-04-16~10:10:02.03 Using config file ".\config.xml"
2007-04-16~10:10:02.03 Temp dir: "E:\x264farm\temp\agent\temp"
2007-04-16~10:10:02.03 Ports: 50700 - 50703
2007-04-16~10:10:02.03 x264: "x264.exe"
2007-04-16~10:10:02.03 nice: 10
2007-04-16~10:10:02.03 agents: 1
2007-04-16~10:10:02.03 agent bases:
2007-04-16~10:10:02.03 [exact]
2007-04-16~10:10:02.03 pipe: false
2007-04-16~10:10:02.03 compressions: 0001
2007-04-16~10:10:02.03 deleting old files: after 604800 seconds
2007-04-16~10:10:02.03 trying port 50700
2007-04-16~10:10:02.04 bound to port 50700
2007-04-16~10:10:02.04 listening...
2007-04-16~10:10:02.04 working!
2007-04-16~10:10:07.10 Got a connection from 169.254.25.121:1751
2007-04-16~10:10:07.10 working!
2007-04-16~10:10:07.10 2 first pass frames 0 to 5062
2007-04-16~10:10:07.10 Testing file "G:\dvb\x264.avs": FOUND!
2007-04-16~10:10:07.10 2 agent-based encoding
2007-04-16~10:10:07.10 2 doing the following:
nice -n 10 x264.exe --second --qcomp 0.6 --cplxblur 20. --qblur 0.5 --qpmin 10 --qpmax 51 --qpstep 4 --ipratio 1.4 --pbratio 1.3 --fps 60000/1001 --pass 1 --stats "E:\x264farm\temp\agent\temp\stats 40ebd3.txt" --seek 0 --frames 5063 -o NUL "G:\dvb\x264.avs"
2007-04-16~10:10:07.50 Thread received exception "Sys_error(\"nice -n 10 x264.exe --second --qcomp 0.6 --cplxblur 20. --qblur 0.5 --qpmin 10 --qpmax 51 --qpstep 4 --ipratio 1.4 --pbratio 1.3 --fps 60000/1001 --pass 1 --stats \\\"E:\\\\x264farm\\\\temp\\\\agent\\\\temp\\\\stats 40ebd3.txt\\\" --seek 0 --frames 5063 -o NUL \\\"G:\\\\dvb\\\\x264.avs\\\": No error\")"
2007-04-16~10:10:07.51 Got a connection from 169.254.25.121:1756
2007-04-16~10:10:07.51 working!
2007-04-16~10:10:07.51 3 first pass frames 0 to 5062
2007-04-16~10:10:07.51 Testing file "G:\dvb\x264.avs": FOUND!
2007-04-16~10:10:07.51 3 agent-based encoding
2007-04-16~10:10:07.51 3 doing the following:
nice -n 10 x264.exe --second --qcomp 0.6 --cplxblur 20. --qblur 0.5 --qpmin 10 --qpmax 51 --qpstep 4 --ipratio 1.4 --pbratio 1.3 --fps 60000/1001 --pass 1 --stats "E:\x264farm\temp\agent\temp\stats 26cabf.txt" --seek 0 --frames 5063 -o NUL "G:\dvb\x264.avs"
2007-04-16~10:10:07.54 Got a connection from 169.254.25.121:1750
2007-04-16~10:10:07.54 working!
2007-04-16~10:10:07.54 4 first pass frames 8307 to 11460
2007-04-16~10:10:07.54 Testing file "G:\dvb\x264.avs": FOUND!
2007-04-16~10:10:07.54 4 agent-based encoding
2007-04-16~10:10:07.54 4 doing the following:
nice -n 10 x264.exe --second --qcomp 0.6 --cplxblur 20. --qblur 0.5 --qpmin 10 --qpmax 51 --qpstep 4 --ipratio 1.4 --pbratio 1.3 --fps 60000/1001 --pass 1 --stats "E:\x264farm\temp\agent\temp\stats 124d23.txt" --seek 8307 --frames 3154 -o NUL "G:\dvb\x264.avs"
2007-04-16~10:10:07.59 Thread received exception "Sys_error(\"nice -n 10 x264.exe --second --qcomp 0.6 --cplxblur 20. --qblur 0.5 --qpmin 10 --qpmax 51 --qpstep 4 --ipratio 1.4 --pbratio 1.3 --fps 60000/1001 --pass 1 --stats \\\"E:\\\\x264farm\\\\temp\\\\agent\\\\temp\\\\stats 26cabf.txt\\\" --seek 0 --frames 5063 -o NUL \\\"G:\\\\dvb\\\\x264.avs\\\": No error\")"
2007-04-16~10:10:07.60 Thread received exception "Sys_error(\"nice -n 10 x264.exe --second --qcomp 0.6 --cplxblur 20. --qblur 0.5 --qpmin 10 --qpmax 51 --qpstep 4 --ipratio 1.4 --pbratio 1.3 --fps 60000/1001 --pass 1 --stats \\\"E:\\\\x264farm\\\\temp\\\\agent\\\\temp\\\\stats 124d23.txt\\\" --seek 8307 --frames 3154 -o NUL \\\"G:\\\\dvb\\\\x264.avs\\\": No error\")"
2007-04-16~10:10:07.60 Got a connection from 169.254.25.121:1757
2007-04-16~10:10:07.62 working!
2007-04-16~10:10:07.62 Got a connection from 169.254.25.121:1758
2007-04-16~10:10:07.62 working!
2007-04-16~10:10:07.62 5 first pass frames 0 to 5062
2007-04-16~10:10:07.62 Testing file "G:\dvb\x264.avs": FOUND!
2007-04-16~10:10:07.64 5 agent-based encoding
2007-04-16~10:10:07.64 5 doing the following:
nice -n 10 x264.exe --second --qcomp 0.6 --cplxblur 20. --qblur 0.5 --qpmin 10 --qpmax 51 --qpstep 4 --ipratio 1.4 --pbratio 1.3 --fps 60000/1001 --pass 1 --stats "E:\x264farm\temp\agent\temp\stats a7e4e6.txt" --seek 0 --frames 5063 -o NUL "G:\dvb\x264.avs"
2007-04-16~10:10:07.64 6 first pass frames 8307 to 11460
2007-04-16~10:10:07.65 Testing file "G:\dvb\x264.avs": FOUND!
2007-04-16~10:10:07.65 6 agent-based encoding
2007-04-16~10:10:07.65 6 doing the following:
nice -n 10 x264.exe --second --qcomp 0.6 --cplxblur 20. --qblur 0.5 --qpmin 10 --qpmax 51 --qpstep 4 --ipratio 1.4 --pbratio 1.3 --fps 60000/1001 --pass 1 --stats "E:\x264farm\temp\agent\temp\stats 37aaac.txt" --seek 8307 --frames 3154 -o NUL "G:\dvb\x264.avs"
2007-04-16~10:10:07.70 Thread received exception "Sys_error(\"nice -n 10 x264.exe --second --qcomp 0.6 --cplxblur 20. --qblur 0.5 --qpmin 10 --qpmax 51 --qpstep 4 --ipratio 1.4 --pbratio 1.3 --fps 60000/1001 --pass 1 --stats \\\"E:\\\\x264farm\\\\temp\\\\agent\\\\temp\\\\stats a7e4e6.txt\\\" --seek 0 --frames 5063 -o NUL \\\"G:\\\\dvb\\\\x264.avs\\\": No error\")"
2007-04-16~10:10:07.70 Thread received exception "Sys_error(\"nice -n 10 x264.exe --second --qcomp 0.6 --cplxblur 20. --qblur 0.5 --qpmin 10 --qpmax 51 --qpstep 4 --ipratio 1.4 --pbratio 1.3 --fps 60000/1001 --pass 1 --stats \\\"E:\\\\x264farm\\\\temp\\\\agent\\\\temp\\\\stats 37aaac.txt\\\" --seek 8307 --frames 3154 -o NUL \\\"G:\\\\dvb\\\\x264.avs\\\": No error\")"
2007-04-16~10:10:07.71 Got a connection from 169.254.25.121:1759
2007-04-16~10:10:07.71 working!
2007-04-16~10:10:07.71 Got a connection from 169.254.25.121:1760
2007-04-16~10:10:07.71 working!
2007-04-16~10:10:07.73 7 first pass frames 0 to 5062
2007-04-16~10:10:07.73 Testing file "G:\dvb\x264.avs": FOUND!
2007-04-16~10:10:07.73 7 agent-based encoding
2007-04-16~10:10:07.73 7 doing the following:
nice -n 10 x264.exe --second --qcomp 0.6 --cplxblur 20. --qblur 0.5 --qpmin 10 --qpmax 51 --qpstep 4 --ipratio 1.4 --pbratio 1.3 --fps 60000/1001 --pass 1 --stats "E:\x264farm\temp\agent\temp\stats 0c36c2.txt" --seek 0 --frames 5063 -o NUL "G:\dvb\x264.avs"
2007-04-16~10:10:07.75 8 first pass frames 8307 to 11460
2007-04-16~10:10:07.75 Testing file "G:\dvb\x264.avs": FOUND!
2007-04-16~10:10:07.75 8 agent-based encoding
2007-04-16~10:10:07.75 8 doing the following:

[foxius]

harissa

Assuming you're using the following command line, you're not providing options for the "--first" argument. I think you should remove it if you don't want to pass any first pass options (i might be wrong though)


controller.exe --bitrate 3000kbps --avs "G:\dvb\x264.avs" --first --second -o "G:\ka.mkv"

[harissa]

great its working
but with a simple avs . i got 4 therad running at 25% each = 100%
i have a quad core
once i input some dll in the avs , i got only 2 thread of x264 runnin at 25% each = 50% , how can i push it to use 4 thread

[harissa]

nevermind i found it
but i wonder if there s an option like in megui called turbo , that really boost second pass speed ?

[foxius]

nevermind i found it
but i wonder if there s an option like in megui called turbo , that really boost second pass speed ?

afaik it boosts only the first pass.

just select your options in MeGUI and use the command line it generates with x264farm. But you should remove some of the options - consult the manual or this thread, i've seen an advice regarding this subject.

[harissa]

with a quad core, the setting in config.xlm , says
<number>4</number> <!-- How many jobs to send to the agent at one time.

so it s supposed to run 4 x264.exe
but when i use this avs :
loadplugin("DGDecode.dll")
Mpeg2Source("carnavale.d2v")
he really load 4 x264 at once and the cpu run at 100%
, but when i add
loadplugin("Select2.dll")
it drop to 2 thread of x264.exe , and it run the first pass at 50% ,
only the second pass load all 4 x264.exe

with selecteven() it run only 3 x264.exe , instad of 4

[legoman666]

Hate to keep bugging you all, but i have a small feature request :D

Im on a 10/100MB lan, using a farm of 6 computers. During the first pass, my bandwidth usage is at 99-100%, if i enable compressions on the 2 clients that are the fastest (and thus demand the most data), my lan usage drops to ~30-50%.

but during the second pass, which is much more cpu intensive pre frame, my lan usage is never more than 10%, and spikes whenever a client finishes a job. But the controller is still compressing the data it sends to the 2 fastest clients. So the controller is wasting cpu time compressing the data during the second pass that need not be compressed.

put simply, can you add an option to only enable compression during the first pass? It'd be extremely useful for people in my position :D Thanks :D

[SpAwN_gUy]

put simply, can you add an option to only enable compression during the first pass? It'd be extremely useful for people in my position :D Thanks :Dwell, you can workaround that...
1) finish First pass WITH compression
2) on the start of Second pass terminate "controller"
3) add "--nocomp" option to the controller cmdLine
4) start controller withOUT the compression :) ...

for me.. agent-based encodings are the greatest :)

[omion]

@legoman666:
That'd be an easy addition for the next version. It makes sense too. SpAwN_gUy's workaround will work in the meantime.
[edit] By the way, how does setting compression affect the overall encoding speed?

@harissa:
That's a bit odd. How long is the video, and large is the --batch parameter? Also, does the controller think it's sending out 4 jobs?

@all:
Sorry about the complete lack of upgrades. I've been working full-tilt on my other program, so I haven't had any time to do anything with x264farm... I should be getting back to x264farm sometime next week.

[legoman666]

@legoman666:
That'd be an easy addition for the next version. It makes sense too. SpAwN_gUy's workaround will work in the meantime.
[edit] By the way, how does setting compression affect the overall encoding speed?



Using compression on 2/6 clients, controller.exe cpu usage jumps from ~10% to ~30%, but lan usage drops way down and all the clients that were being choked by lack of bandwidth all get a nice speed increase.

Anyway, I got a bug report :D check the thread here: http://forum.doom9.org/showthread.php?t=124950

Im getting green block artifacts on keyframes (I think they're keyframes, hard to say) when I use x264farm but not when I use megui.