Processors vs Memory

[KaoS]

I find myself doing alot of DVD2SVCD rips (I probably do at least one/day).

I have a seperate server machine setup to do the job for me. It is currently a 1.3ghz Athlon with 256mb of memory.

My question is, how memory dependent is this process? Since I do so many, I want to upgrade the system to help me perform them as fast as possible.

What would I benefit from more.. more memory or maybe a dual processor motherboard?

Thanks

[Kedirekin]

More memory probably won't help. The only thing in DVD2SVCD that's a real memory hog is CCE, and 256 is adequate for CCE. In fact, since you're cancelling before DVD2SVCD starts video encoding, more memory is doubly not necessary.

Dual processors might help you quite a lot. I know from experience that you can run multiple instances of DVD2SVCD for audio encoding. I believe the limiting factor becomes hard-drive thoughput, so you'll want the fastest hard-drive subsystem you can get. You might concider IDE RAID (or even real RAID) if you're really serious about getting the most performance.

And TMPg should be twice as fast on a dually.

[Edit: I must be getting old - I must have mixed this up with another post I read earlier - why did I think you were using DVD2SVCD just for audio encoding then cancelling out and using TMpg for video encoding? Anyway, I leave the above as a tribute to my humanity. Here's a better answer.

You won't get much performance improvement from adding memory. 256 should be adequate.

You also won't get much performance improvement from a dual processor system. While CCE is MP enabled, the frame serving software (AviSynth or VFAPI) is not, so you only see a 2-15% performance boost in the video encoding (which is the most time consuming part). I don't believe any of the other portions (audio encoding, muxing, authoring, etc.) are MP enabled at all, so you'll see no improvement in them.

I think your best bet is to go with the fastest single-processor system you can get. A faster hard-drive subsystem (using two separate hard drives) might improve performance a bit too.]

[Slick]

Originally posted by Kedirekin
You might concider IDE RAID (or even real RAID)

I assume by the latter you mean Raid using SCSI?

Raid is Raid regardless of whether it it is IDE. it is just faster using Scsi.

bit OT. but for anyone who might be interested:

Scsi is faster on it's own than IDE is with or without using raid.

if using raid for performance, use it striped.
if using raid for security, use it mirrored.
if using raid for security & performance, use it striped & mirrored (but is not as fast as just striped)

Raid requires two Hdds the same size.

when set up for striping your operating system will see just one drive, slightly smaller than the total of both.
very basically, striping can & does write to both drives at the same time, thereby increasing performance. (but it is not twice as fast).

when setup for mirroring your operating system will again see just one drive, but will only be equal to the size of one of your drives.
mirroring does just that, it writes the same info to both drives, but if one drive gets corrupted or fails you still have a copy of the data.

striping & mirroring I will let you work that one out :)

no doubt I will get flamed for this, but hey who cares. :devil:

[markrb]

@Slick I disagree with a couple of points you said. No Flame here.

You say that SCSI is faster then IDE.
Not always true, especially now. The only reason this tail exhists is that faster drives usually hit the market first in SCSI. Such as rotational speed and more cache. Real world tests have proven that a Worstation running the same speed drive shows no performance improvement using SCSI.
SCSI has a serious advantage in servers where it's out of order data retrieval method gives it much faster real world performance, but worstations don't usually need the feature.
Anyway this is a very long topic. Bottom line is that most people will not see much,if any, real world performance gains using SCSI drives.

Raid is Raid regardless of using SCSI.
True, but I think he might have been talking hardware RAID and I do not know of any real hardware IDE raid solutions for home use.

if using raid for security & performance, use it striped & mirrored (but is not as fast as just striped)
Yes and no. When done right read performance can have a serious gain while write performance can take a hit. Basically if you can have each set of drives on a different controller then you use a round robin read algorythm to half the reads on each chain giving you a boost in read performance.

Just to be a real technoweenie: Striping, although commonly considered RAID, is not technically RAID. The R stands for redundent and there is no redundacy(protection) with just striping.

@KaoS Now back to the original subject. I can only comment on DVD2SVCD as using these programs seperately or differently would change the way I think. CCE supports Dual CPU's, but the frameserver does not. So I cannot suggest dual CPU's unless you plan to use the computer while encoding.
I always suggest as much RAM as possible.
As far as DVD2SVCD 256mb is minimum and higher will not produce much in the way of gains, but will allow you to be able to use the computer while encoding much more easily.
Fastest CPU you can afford. Right now the best bang for the buck is the AMD XP line of CPU's.
Motherboard should be a solid well respected brand. I have a Soltek Via based 266a solution and it's pretty good, except it has problems with higher bus speeds if you plan to OC. The Epox 266a solution is also a good choice for the lower price point. The other good choices are Abit, Soyo and Asus in that order. Of course now the VIA 333 chipset has just come out.

Memory bandwidth seems to have the biggest effect on overall performance. DDR(for AMD CPU's) and Rambus(for Intel CPU's) have much better performance with CCE then PC133 memory. I have seen many reports that show that the new Intel CPU's (Northwood) and DDR do not perform as well as with Rambus.

IDE RAID is not very important in my mind to what we do. A couple of good fast IDE drives are good enough. I like the new Maxtor Diamond MAX plus D740 series of drives for the bang for the buck and overall I like the special edition WD drives for overall speed. They have 8mb of cache on each drive and are very fast, but expensive for IDE.

If I had a good deal of money and was putting together a system today this is what I would buy for encoding:

AMD XP 2000+ CPU
512MB PC2700 DDR (Be very careful to get a good brand)
Abit 266a solution or the best of the new Via 333 solution.
Any good video board (if you get the new Geforce 4 make sure it fits the motherboard)
2 X WD Special edition 60gb or 2 X Maxtor D740X 60GB drives
Lite-ON 32X CDR drive
Toshiba or Pioneer DVD ROM
Any decent sound card

Hope this helps.

Mark

[Slick]

Cheers Markrb.

We can always rely on you for the "techie bits".
how do you remember all that stuff.

I stand corrected. (gracefully of course) :p

Few! got away lightly there, I was convinced I'd get a flaming on that one. :D

P.S.

for what it's worth I got my new components today & will be re-building my machine tomorrow. so if you want i'll let you know the resulting increase in encoding speed when its finished.

current m/c.
P4 1700
1gb pc133 ram
abit BL7 m/board
3 x 40gb hdds
radeon 8500

new m/c
P4 2.0gb (northwood core)
512mb ddr 2100 ram
abit BD7 m/board
2 x 40gb hdds
radeon 8500

Cheers

[markrb]

How do I remember all this stuff? Short answer: I have no life and love this computer crap way too much.

It's not the stuff I remember that kills me in the end it's the important stuff I forget. I have been a computer engineer since 1987. I am starting to have this stuff pour out of my head. You should see my friends and I when we have dinner the wives and girlfriends always end up yelling at us for talking techno babble they have no clue about.
For the most part you were pretty correct and I don't blame anyone for believing what is the common way of thinking. I always thought SCSI drives were always faster too until I starting reading some really good articles on the subject.

You should get a nice little boost there. The Northwood is a ton better CPU and DDR blows away PC133.

Mark

[Slick]

@ Markrb
funny old world aint it. I always had you tagged as a spotty little toe-rag until i started reading your replies properly & then you became a Mod.
I don't know why but you seemed to make more sense then. :D

yes I know where you are coming from with the techie bit.

I am a company director who just happens to run the entire I.T. dept
including the servers, networks etc.
I'm considered a geek with my limited knowledge.

amazing how many friends you didnt know you had until they have a computer problem aint it.

I make a few bob as a side line with P.Cs though.;)
thats where my new bits are from.

C u Later

[badbert]

I am a newbie here. And I don't mean to step on any toes. Please don't Flame me... I have limited experience with CCE and I did not see CCE mentioned anywhere in his post. I have been Using TMPGenc for over a year. (I have version 12e) I started on a celeron 500 with 512mb. When I switched to Dual celeron 500s. It made a drastic(double) difference. Adding IDE Raid, Promise FastTrax66 did not. I then moved up to a P4 with 512mb Rdram. The Difference was as drastic as "dualling" the processors. But was not multi-task(able?). Multi-tasking was actually better on the dual celerons. I could burn CDs while encoding. The P4 could not. Now i have a Dual AthlonMP1600 with 1gb of DDR. Speed doubled again. And it is definitely multi-taskable (Nero's "finished" chime, crashes my Q3 server..Yes while encoding with TMPGenc)
I have made a discovery (by accident of course), there is an option in tmpgenc that enables or disables the program from using your system cache. This option has a huge effect on multi-tasking. Option>environmental setting>General tab>General settings You can also set the CPU settings for your machine here.

In closing, each machine started with 256. Adding 256 more didn't make a difference. Adding Raid didn't. Faster Processors made a HUGE difference. And dualling the processors Also makes a huge difference. If you want fast and muli-taskable, use Your money for Duals. If you want fast and don't care about multi-taskin(you mention a dedicated encoding machine) Then use your money on the Fastest processor you can afford. If you want to be able to "Capture Video" in Large format Striped Raid "Rules".

[Kedirekin]

Even though RAID was brought up as a result of my loosing my mind, I have to chime in here.

Most of the things we do with DVD2SVCD are processor limited, and true, massive hard-drive performance will not improve overall performance much.

However, if you run multiple instances of DVD2SVCD to do parallel audio encoding (which I sometimes do), then the process is hard-drive limited. When I do this, with 3, 4, or even 6 instances of BeSweet running, I generally only see about 38% of processor usage. I believe that RAID would improve overall performance significantly in that situation. That's why I mentioned it in my 'loosing my mind' post, but not once I came to my senses.

[markrb]

Kedirekin in that case the faster the drive system the better. Multiple reads and writes all going at once definately will put a strain on a single drive. I have never come close to putting that much stress on my PC when encoding.

Slick by spotty little toe-rag did you mean computer newbie who thinks he knows it all or just a plain idiot? In my posts I often type in a way I would understand and sometimes people are confused since I take certain things for granted. I guess everybody does, but not everyone posts as much as I do so I am seen here and remembered more. If I didn't have this hobby I would be going nuts since I have been out of work for almost a year. That's why I have the time to post. I used to be a second level support for EMC until they laid off 2300 people in this state alone.

badbert nice machine. Man those MP CPU's must of set you back a bit. They are harder to come by and more expensive then the XP's from what I have seen. I agree with just about everything you said. My suggestion was for a computer that while encoding can do other things, but definately not a game of Quake, while I am pretty sure yours could do both at the same time.

One little hint. If you want to use your computer while encoding for not very heavy things like surfing the net, email and even some other stuff make sure to set the priority to Idle in DVD2SVCD. I have found it doesn't effect my encode speed, but I can run other apps much easier.

Mark

[badbert]

Kedirekin; raid has various other advantages as well. I have graduated to a FastTrax100 raid with my 2x60gb 7200 rpm Maxtors.
My programs load twice as fast. My computer starts and shuts down faster. I can access my disks faster...etc. I was just saying it has little to do with "normal" encoding. BUT, wonder what would happen if I created two DVD2SVCD directories and 2 tmpgenc directories and ran them at the same time....hmmm. Heck, It would be the same as running batch mode...LOL

markrb check out the place where I purchased mine EssentialComputers (http://www.essencompu.com/index.asp?rvid=1148)

[Bigbucks1959]

Hmmmmmmmmmmmm..The only big change I see is your RAM speed...I personally don't feel that any of your other changes will impact encoding that much.

I could be wrong, won't be first time..LOL

PK


Originally posted by Slick
Cheers Markrb.

We can always rely on you for the "techie bits".
how do you remember all that stuff.

I stand corrected. (gracefully of course) :p

Few! got away lightly there, I was convinced I'd get a flaming on that one. :D

P.S.

for what it's worth I got my new components today & will be re-building my machine tomorrow. so if you want i'll let you know the resulting increase in encoding speed when its finished.

current m/c.
P4 1700
1gb pc133 ram
abit BL7 m/board
3 x 40gb hdds
radeon 8500

new m/c
P4 2.0gb (northwood core)
512mb ddr 2100 ram
abit BD7 m/board
2 x 40gb hdds
radeon 8500

Cheers

[markrb]

Bigbucks1959 don't you know that the NorthWood is a far superior CPU to the original P4?
Plus 300Mhz CPU increase in speed as well.
Mark

[Bigbucks1959]

What OS u using with that?

Thx,
PK


Originally posted by badbert
Kedirekin; raid has various other advantages as well. I have graduated to a FastTrax100 raid with my 2x60gb 7200 rpm Maxtors.
My programs load twice as fast. My computer starts and shuts down faster. I can access my disks faster...etc. I was just saying it has little to do with "normal" encoding. BUT, wonder what would happen if I created two DVD2SVCD directories and 2 tmpgenc directories and ran them at the same time....hmmm. Heck, It would be the same as running batch mode...LOL

markrb check out the place where I purchased mine EssentialComputers (http://www.essencompu.com/index.asp?rvid=1148)

[Bigbucks1959]

No, I did not..
Darn you gonna see big increase in encoding speed..
I am now very jealous.

I am gonna research that cpu...I am trying to get my encoding speed down to 6 hours on a 2hr movie...

Looking at building a new pc for just using DVD2SVCD on..
ALso looking forward to results with new system...


Thx,
PK

Originally posted by markrb
Bigbucks1959 don't you know that the NorthWood is a far superior CPU to the original P4?
Plus 300Mhz CPU increase in speed as well.
Mark

[djc]

Hello.

If anybody is using Windows 2000 Pro, there is a very good software called SuperCache 2000, that uses memory as "disk" cache. The company is www.superspeed.com but for some reason they have cut the demos. If you want a demo try www.sunbelt.co.uk

It doesn't speed much the 1st stages (audio/avi) but it does help a lot when the encoders come to play, and after, until the end. It also generally help you with the use of your pc.

This is the best way to use your memory.

djc

[fuct]

Originally posted by markrb

When done right read performance can have a serious gain while write performance can take a hit. Basically if you can have each set of drives on a different controller then you use a round robin read algorythm to half the reads on each chain giving you a boost in read performance.

Mark [/B]

I dont understand what you mean by this, is it better (for write purposes) to have both drives on a single controller or have them on different? If using seperate controllers allows a boost in read performance, does it hurt the write performance?