DualDVDRB: Tool for DVDRB to support Hyperthreading/dual processors (RC 3)

[Rumbah]

DualDVDRB - A program to make use of two CPU cores in DVD-RB

DualDVDRB is an extension for DVD-RB (www.dvd-rb.com). It makes DVD-RB to
encode two different pieces of the DVD simultaneously. By doing that, DVD-RB
uses the power of two virtual (HT) or real CPU cores even if the encoder
only supports single thread encoding. So this will give you a performance
boost for Pentium 4 HT processors and of course for the new dual core
processors of AMD and Intel.
Currently supported are HC and QuEnc in the pro and in the free version of
DVD-RB.

You can grab the latest version of this software at
http://sourceforge.net/projects/dualdvdrb/

You can find an experimenal version with Procoder support at
http://forum.doom9.org/showthread.php?p=776956#post776956

You can post your thoughts and suggestions at sourceforge or in this doom9
thread: http://forum.doom9.org/showthread.php?p=692474


Installation
------------

Just copy the DualDVDRB and DualDVDRBWatcher exe files in a folder you want.
You need two instances of the encoder you want to use, so you have to make a
copy of your encoder into a different folder. For example something like this:
C:\encoders\HC1\hcbatch.exe
C:\encoders\HC2\hcbatch.exe


Usage
-----

If you simply execute the DualDVDRB file, the configuration window will pop up. Here
you have to set the different locations of your encoders. The first two
locations are for the HC encoder in the Pro version of DVD-RB. The two last
are for QuEnc in the Pro version or QuEnc and HC in the free version.
Then you can choose if the encoders should be executed minimized as
DualDVDRB does not take this setting of DVD-RB.
Now you have to setup DVD-RB to use the DualDVDRB.exe file as encoder for it
to work. For example if you want to use QuEnc you open the options setup
window and for "Path to QuEnc" you simply select the DualDVDRB file. That's
all. Now DVD-RB calls DualDVDRB to encode and DualDVDRB starts the encoding
job.


History
-------
Changes from RC2
-better detection of the last file to encode, one and two clock modes should work now


Changes from RC1
-support for ILVU, every project should work with DualDVDRB now
-no problems with pressing the abort button in DVD-RB anymore
-watcher executable to make the corrections that have to be done if you pressed the abort button in DVD-RB


Changes from the Beta 1:
-GUI for easy configuration
-support for QuEnc and therefor support for the free version with QuEnc and HC
-some bug fixed

[Carpo]

suppose i could give it a go on my p4 ht northwood :)

[k2002rahmani]

sounds good. when will dual CCE SP be available?

[Rumbah]

CCE SP already uses multiple threads for encoding so it is not necessary to start it twice to make use of HT or a dual CPU. So it would not make much sense to support it in my program.

[k2002rahmani]

cool. i didnt know this.

[jptheripper]

so 2.70 does HT? hmm, guess its time to make the switch from 2.5

[maksa]

I finally got the time to write a small program to start two encoders simultaneously when using DVDRB. That is the solution for people who want multithreading encoders to use their processor at max. At the moment it only supports HC and DVDRB Pro. The next days I'll add QuEnc and HC support for the non pro version and write a little manual to package with the exe file.

Actually, Jdobbs disabled this option - I don't remember when, but I know I was able to runt 2 DVD-RB simultaneously (and 2 CCE), but he even mentioned here that he is disabling this otion. Now it won't allow you to run 2 tohether, why, I don't know?

I was encoding 2 movies at the same time @ 50% CUP usage on PIV 3 GHz (HT). On AMD Athlon 2.8 with multithreading, CCE uses 100% CPU just for one DVD-RB. I guess PIV is more powerful (or 2 CUPs inside).

[jdobbs]

The reason it was disabled is because when you run two, the settings you change in either will affect both. This can cause numerous problems. Think for example, what might happen if you are in the middle of encoding with one instance of the program, and changed the working directory with another.

[k2002rahmani]

jdobbs: this makes a lot of sense.

maksa: I think what Rumbah is saying is that CCE will utilize the dual processors or HT capability. A single segment can be encoded using the capabilities of the HT or dual processors in a single window. There is no reason to have more than one window open.

[Rumbah]

New version released
Major changes
Support for QuEnc in Pro and free version of DVD-RB and support for HC in the free version added
A GUI for easy configuration. It's not beautiful, but funcional ;) .
Some bugs fixed, now spaces in path names are hopefully treated corectly.

I'm happy about reports if it works for you or not and suggestions or questions you may have.

[jarthel]

The reason it was disabled is because when you run two, the settings you change in either will affect both. This can cause numerous problems. Think for example, what might happen if you are in the middle of encoding with one instance of the program, and changed the working directory with another.

Hi jdobbs. I have 2 copies of rebuilder located in rebuilder1 and rebuilder2 folder.

Rebuilder2 is running a CCE encoding right now. If I open rebuilder1, the working directory is still the previous one. According to your reply, rebuilder1's working directory should be the same as rebuilder2's. right?

Thanks for the reply.

[jdobbs]

That wouldn't be true if you have two copies running out of two distinct DVD-RB executable directories. Each would keep its settings in its own .INI file Then those values would apply only to the individual instance. But you'd have to be careful when you opened a shared project file or were working in the same working/source/output directories.

[jarthel]

That wouldn't be true if you have two copies running out of two distinct DVD-RB executable directories. Each would keep its settings in its own .INI file Then those values would apply only to the individual instance. But you'd have to be careful when you opened a shared project file or were working in the same working/source/output directories.

thanks for the reply. what great relief it brought! :)

[Rumbah]

Big Update

DualDVDRB sould work with every DVD-RB project as it now supports ILVM.
In addition to that you can use the Stop/Pause button in DVD-RB with no harm. There is a program running in the backgroud now that makes the changes that are needed in such a case.

It is not really tested in the QuEnc (respectively with the free version of DVD-RB) mode but it shouldn't make any problems.

[SpazzHH]

@Rumbah,

Thanks so much for this fantastic tool. I just tried it for the first time today, and did a full backup(everything minus the trailers) of Dark Water with HC in 98 mins. :D

While my Hyperthreaded P4 only shows modest gains, my Dual Core Pentium D, truly runs twice as fast. :D

Just a few small things I've noticed that you might want to look at. I currently am running a project and have about 22 green "H" symbols in my system tray. For some reason, when a new instance of an encoder gets launched, the new DualDVDRB icon shows up, but the old one won't go away until you wave your mouse over it.

Secondly, you might take a look at how you notify RB that an encoder is finished with a segment. For some reason, while doing another project, I noticed that RB was showing a 100% status on encoding, when there were still two instances of HC working. On that one, RB actually started the "Rebuild" phase(with the new 2-click mode) before all segments were finished. I had to run the Rebuild again to eliminate the missing frames.

Thanks again for everything, Spazz

[Rumbah]

Just a few small things I've noticed that you might want to look at. I currently am running a project and have about 22 green "H" symbols in my system tray. For some reason, when a new instance of an encoder gets launched, the new DualDVDRB icon shows up, but the old one won't go away until you wave your mouse over it.
Well, everytime a new instance of the encoder gets launched, a new instance of DualDVDRB is launched, too. But there is only one running at a time. I noticed this behavior but I do not know whether it is the fault of AutoHotkey or Windows. The only option I have is to completely hide the icon. If you think that's better I'll add that.

Secondly, you might take a look at how you notify RB that an encoder is finished with a segment. For some reason, while doing another project, I noticed that RB was showing a 100% status on encoding, when there were still two instances of HC working. On that one, RB actually started the "Rebuild" phase(with the new 2-click mode) before all segments were finished. I had to run the Rebuild again to eliminate the missing frames.
There is some code to prevent that problem. It's the same as for ILVU sections. But somehow it seems it does not work for your DVD. I ran into this problem once but I deleted the files and forgot which film it was. So I could not reproduce the error. It would be great if you could try it a second time and see if the same problem appears again. If it's reproducable then I'll send you a debug version so that I can look into the log (or if you tell me the title, perhaps I own the film and can reproduce the error for myself if the NTSC and PAL versions are similar).
The percentage gauge will never be really accurate because of the way DualDVDRB "cheats" DVD-RB (but it should stop before 100% and wait for all open encodings to end).

[jptheripper]

slightly noob and slightly OT question, on the dual core processors that support hyperthreading.. does that mean 4 simultaneous cces?

[Rumbah]

My program does not run with CCE. And it only starts two instances of an encoder. So dual core cannot benefit of HT with it.

[bourtzovlakas]

Ηello,
Is it possible to use two instances of HC in the free version of RB?
If it is possible, how can i make sure that RB will "call" HC, through DualDVDRB?
Thanks...

[SpazzHH]

@Rumbah

The DVD was Lost in Translation-R1 NTSC, and yes the error is reproducable. I would be more than happy to help in any way possible.


@bourtzovlakas

Yes, DualDVDRB works also with the freeware version of RB. You can DL it from the link at the top of this thread. There is a "read me.txt" for you to follow, to help with setup.

[bourtzovlakas]

Yes...
The readMe file contains the same information as the first post...
If i understood correctly, in order to make DVDRB use DualDVDRB, a modification of the path of the encoder is required...
...but in free version, there is no such otion for HC...

Did i misread/misundrstood something?
As it is presented right now the only possible solution for free version seems to be, the combined use of QuEnc and HC encoders...
Am i totally wrong?

[SpazzHH]

Yes...
The readMe file contains the same information as the first post...
If i understood correctly, in order to make DVDRB use DualDVDRB, a modification of the path of the encoder is required...
...but in free version, there is no such otion for HC...

In RB Free, you enter the path to HC or DualDVDRB where it asks for the path to Quenc.

Did i misread/misundrstood something?
As it is presented right now the only possible solution for free version seems to be, the combined use of QuEnc and HC encoders...
Am i totally wrong?

As it states in the read me, you have to have 2 copies of your encoder. Manually launch DualDVDRB the first time and it will have you select the paths to your 2 versions.

[hank315]

While my Hyperthreaded P4 only shows modest gains, my Dual Core Pentium D, truly runs twice as fast.Hmm, this is interesting.
My experience running two HC instances using my HT P4 don't show a big speed increase also.
Both instances have to use the same CPU extensions which slows it down.

Does it really run twice as fast on dual core?
If so it's time to upgrade and implement multi-threading :D :D

[Rumbah]

On my P4 3,2 Ghz HT I get a speedgain of about 15% by just using DualDVDRB with HC. That's why I wrote it ;) .

@SpazzHH: I own that film. Tomorrow I'll have no time but I'll try it on wednesday with my RC2 Version. Perhaps I'll get the error, too, then I can debug it. If not I'll build a debug version of DualDVDRB for you.

@hank315: Another question, can I bundle my Cuttermaran provider with your encoder?

[SpazzHH]

Hmm, this is interesting.
My experience running two HC instances using my HT P4 don't show a big speed increase also.
Both instances have to use the same CPU extensions which slows it down.

Does it really run twice as fast on dual core?
If so it's time to upgrade and implement multi-threading :D :D

On my Pentium D it does run twice as fast. Each instance shows the exact same speed, with no slowdown, as I get with running only one instance. With my HT P4 3.2g, there is only about a 15% increase in overall speed, but my 3.0g Pentium D just loves it. CCE on the Pentium D goes at a 33% faster clip than on the P4, but Prepare phase and Rebuild phase go 25% slower.

[hank315]

@SpazzHH
Each instance shows the exact same speed, with no slowdown, as I get with running only one instance.Thanks for the info, seems there's a big difference between HT procs and real SMP.

@Rumbah
can I bundle my Cuttermaran provider with your encoder?Of course, no problem at all.

[bourtzovlakas]

In RB Free, you enter the path to HC or DualDVDRB where it asks for the path to Quenc.

:thanks:

[gilligan2]

Yes,this runs twice as fast on my dual Xeon system and my dual core Opteron system than just one instance of HC Enc so thanks very much Rumbah for developing this proggie !! Obviously since 2 physical processors are being used you will see a much greater benefit versus HT and it is really nice.

Thanks to Hank also for the nice encoder !!

g:):)

[levi]

I know it was mentioned earlier, but I didn't see it addresesed definitively. It seems CCE 2.5 could benefit from this program. Do you ever intend to add support for it?

[jptheripper]

id second a request for cce 2.5

[Whitespliff]

Just a few small things I've noticed that you might want to look at. I currently am running a project and have about 22 green "H" symbols in my system tray. AutoHotkey?
Been experimenting with it lately, nice proggie :D

[l8nights]

sorry,but I haven't got to test it out yet I'm guessing that it does not support procoder2 is pc2 already multi threaded??

[l8nights]

I have spoke to soon! I'm assuming everybody has got this rc2 version set-up and running,using the way the readme describes??

I have tried to get it setup and have copied the hc 0.16 folder in the same directory as described, started dualdvdrb and set the path's to each copy,

then when pointing dvd-rb @ the dualdvdrb file it was found right away through the quenc path but hc path in rebuilder did not want to recognize the prog til I renamed it w/ an .exe extension ?? (is this normal)
after running the prepare stage it gives me an error saying it failed to find & run encoder, path may be incorrect now I've rechecked everything and I can't seem to figure out where it is that I'm going astray ???

[jdobbs]

That may be caused by the version check. DVD-RB tries to get the version number from the file referenced by "HC=" in the INI... if it can't get one, it assumes it isn't legal and disables HC.

[Rumbah]

but hc path in rebuilder did not want to recognize the prog til I renamed it w/ an .exe extension ?? (is this normal)
What did you have to rename with an .exe extension? You should have DualDVDRB.exe and DualDVDRBWatcher.exe and the readme.txt out of the zip file.

[l8nights]

I renamed DualDVDRB to DualDVDRB.exe probably one of windows tricks I have no idea? but once I tried it I was able to select the file!

@ jdobbs is there away around/to disable that feature I'll look into the ini.file but I alway's tread softly in there!

I have previously used beta's before rebuilder installer included them and they worked well enough???

quenc path recognised the file before the renaming so I'm guessing it doesn't incorporate this version check?

[jdobbs]

There's really nothing to disable. You just need to make sure it points to an EXE file so Rebuilder can get the version... it really doesn't care what version it is -- as long as there is one.

[Rumbah]

Perhaps your download was damaged or you accidentally downloaded the source.
Try to download it again as the DualDVDRB file in the zip file should have the .exe extension without any renaming.

[SpazzHH]

Perhaps he has chosen to "Hide file extensions" within Windows.

[l8nights]

okay this time it took just deleted the files extracted the first time an re-extracted them into a folder a little closer to the encoders!

the only thing I can assume is it does not like to be over three subdirectories deep! the last time it was four!

[SpazzHH]

Mine is actually 5 subdirectories deep with no problem.

[Sophoclesdrf]

I just did a run using Dual-RB with HC encoder and the speed increase had to be about 30% or better. I've done only one 7 gig movie so far but I was impressed with the gains.

http://img36.imageshack.us/img36/5264/drb25vz.jpg

[Rumbah]

Found one error in the last file detection. I'll try to fix it later today. Perhaps it solves the problem with Lost in Translation.
DualDVDRB does not support Procoder or CCE and I do not know if Procoder or CCE 2.5 are multithreaded. The problem is that I do not have these encoders so developing and testing is a p.i.t.a. . I tried to support Procoder with my Cuttermaran provider and it did not work as intended but I could not find the error because I could not test it myself.

[SpazzHH]

Found one error in the last file detection. I'll try to fix it later today. Perhaps it solves the problem with Lost in Translation.

That's good news. Thanks. I sure hope it does solve it. It's not been just the one disc, that was just the first one I happened to try. It has happened with almost every disc I've done, making it impossible to do batch encodes, or use the "two-click" method. As soon as Rebuilder gets that "event complete", it starts rebuilding while the last segments are still encoding.

[l8nights]

Mine is actually 5 subdirectories deep with no problem.

then I really have no idea what the source of my little glitch was?? chock it up to clumsy installation!
this time I still had to add the .exe extension for it to be recognized while browsing but it blew through a test film!

as fir the starting rebuild before the last instance was finished I thought that was by design caused no issue's just multi-tasked in my case! (funny thing) the resulting rebuild was shown as 103% still no issue's perfect back-up!

[Sophoclesdrf]

I would be really interested in seeing more hyper threading results. If there are any gains they must be minimal since a single thread will demand the use of the entire CPU which leaves very little to run a second thread.

[SpazzHH]

I would be really interested in seeing more hyper threading results. If there are any gains they must be minimal since a single thread will demand the use of the entire CPU which leaves very little to run a second thread.

If HC is maxing out your CPU, I wouldn't recommend this program. One instance of HC on my machine only takes up about 50%.

[Rumbah]

I would be really interested in seeing more hyper threading results. If there are any gains they must be minimal since a single thread will demand the use of the entire CPU which leaves very little to run a second thread.
As mentioned here a few times the gains are about 15% with HT.

I ran into a bigger problem with the last file detection, it'll take a little longer until the problem is solved.

[l8nights]

sophocles I have posted some screenshot's here (http://www.dvdhounds.com/forums/showthread.php?goto=newpost&t=225) I know how you love them screen shot's
but to summarise using a p4 630 3.0w/ ht & hcenc 0.16 I saw nearly a 18% decrease in encoding times! which is not your 30% but it is effective!

great add-on:thanks:

[Sophoclesdrf]

I wasn't trying to insult anyone's Intel PC but I would like to see a few screen shots of individual results. "Seeing something once is better than hearing about it a 1000 times" Lao Tzu I tend to be Leery of claims without difficult to alter visual support and it has nothing to do with any particular individual's claims. Most encoders use virtually all of any CPU regardless of speed unless one has a dual core system or better and the encoder is single threaded but I have to admit that my use of HC encoder is somewhat limited.

I read a claim on another thread stating that an individual averaged 45 to 50 minutes to encode a movie using CCE with a 3.2 Intel processor, call me skeptical but I don't believe it. Especially when I know of several people who can't do that with an Intel processor running at 3.7 Ghz. I will however stand to be corrected if I can see a screen shot to verify the results.

BTW, I said greater than 30% because I didn't take time to do the math or to try it on a single processor but I'll do that tonight and calculate the difference.

Cheers