Cache size and its effect on encoding

[ToiletDuck]

How much does cache matter when encoding? Does cache really matter? Would a machine with 1mb cahce out encode a machine with 256k? And when I ask that I mean by a good means. Not like %3 more or anything like that. Would a 1.7ghz celeron for instance be a lot slower than a 1.7 ghz P4

[killingspree]

yes a celeron 1.7 would be slower than a PIV at the same speed, but not only because of the cache (i don't really know how this affects encoding) but because of the missing multimedia commands (SSE etc) generally speaking a celeron will always be slower than the equal pentium...

regards
steVe

[ToiletDuck]

I was just using that for refferance. I have dual XP's and the new Barton XP's have double the cache. I was just wondering if it would be a noticable differance to have 1mb combined cache over 512 combined.

[wmansir]

Check out Anand's Barton Review (http://www.anandtech.com/cpu/showdoc.html?i=1783&p=18) for some encoding benchmarks, you can see at the same clock speed the Barton is 3% faster at divx and Quicktime, and negligably faster with WME.

[int 21h]

Originally posted by killingspree
yes a celeron 1.7 would be slower than a PIV at the same speed, but not only because of the cache (i don't really know how this affects encoding) but because of the missing multimedia commands (SSE etc) generally speaking a celeron will always be slower than the equal pentium...

regards
steVe

Newer Celerons have the same optimizations as the P4, the only difference is FSB and cache.

[ToiletDuck]

Cool thanks for the link. However you can take the XP 3000+ to 2.5 ghz easily with just stock air cooling. And since I run water cooling I could probably push her more than that. So in that case she would crush the other ones.

[markrb]

If you are talking overclocking the Bartons will not overclock nearly as well as the T'Breds. Larger die size = more current needed = greater heat.

When I first heard about the Barton's I was excited, but after reading the reviews I will hold off for awhile. It does not have the bigger punch that happened when the P4 increased it's cache size.

Right now the Best Value for overclockers in AMD is the XP 2100+ B.

In the original question will a larger cache help in encoding and the short answer is no. A cache is most benificial for applciations where data is reused over and over again. In video and audio encoding the data is streamed and not reused. So having data in cache is of little use.
Anandtech goes into further detail as to why the larger cache is of little use in encoding vs. other applications.

Mark

[tiki4]

I just quickly went over the Barton review at Anandtech. It's a little bit sad that all of them use DivX for their benchmark needs. I think it is clearly visible how well DivX was 'tuned' for the P4 processors. Actually I'd like to see the same benchmark made with the Avisynth/VirtualDub/XviD combo that more users here are using than Xmpeg/DivX. If I remember correctly DXN had help from Intel when they tuned their DivX 5 for speed.

Something else: Here at work I deal with the difference between Athlon MP processors with 256 kB cache vs. Alpha CPUs with 2 MB cache. It is usually impressive how well the Athlon 1200 MHz CPUs perform against the 667 MHz Alphas. This changes however when the computed arrays are small enough to fit into the CPU cache of the Alphas. The speed boost is incredible.
Frames in video encoding are usually quite large, but wouldn't it help to get (more than one) full frames into the Level 2 cache, e.g. for b-frame encoding, where future and past frames are referenced?

Regards,

tiki4

[int 21h]

Originally posted by tiki4
I just quickly went over the Barton review at Anandtech. It's a little bit sad that all of them use DivX for their benchmark needs. I think it is clearly visible how well DivX was 'tuned' for the P4 processors. Actually I'd like to see the same benchmark made with the Avisynth/VirtualDub/XviD combo that more users here are using than Xmpeg/DivX. If I remember correctly DXN had help from Intel when they tuned their DivX 5 for speed.


AMD has had the same opportunities to tune DivX 5 and optimize it. The problem is that 3DNow instructions aren't nearly as useful as SSE2 instructions in Video Compression applications.


Something else: Here at work I deal with the difference between Athlon MP processors with 256 kB cache vs. Alpha CPUs with 2 MB cache. It is usually impressive how well the Athlon 1200 MHz CPUs perform against the 667 MHz Alphas. This changes however when the computed arrays are small enough to fit into the CPU cache of the Alphas. The speed boost is incredible.
Frames in video encoding are usually quite large, but wouldn't it help to get (more than one) full frames into the Level 2 cache, e.g. for b-frame encoding, where future and past frames are referenced?


The entire video frame is not referenced, only parts of the video frame are referenced. The speed problems here are not related to I/O bottlenecks (in most cases) but raw processing power. It also depends greatly on your transcoding source, compare for instance, your FPS from transcoding a huffyuv AVI and an Avisynth script with Mpeg2Source, and you'll see that they vary greatly.

The only way to speed up Mpeg2->Mpeg4 transcoding effectively, would be to tear down the existing decoders and build something from scratch in assembly, using any and all optimizations available to specific architectures.

[ToiletDuck]

Well I did see that they ran the 3000+ Barton at 2.5ghz on stock air cooling. So the chip has some potential. However I have to disagree on the best price ratio chip to be the 2100+. You can get an XP 1700b+ for $55 and overclock it to +2600 if you have watercooling. +2400 if you have air cooling.

[markrb]

According to the CPU database at www.overclockers.com you are far more likely to hit above 2600+ levels with the $85 XP 2100+.

I have seen many actually well above 2.5Ghz which is above a XP 3000+ level with water cooling.

I just think that an $84 CPU that can realisticly do 2.2-2.3Ghz on air and some are getting over 2.5Ghz real speed and not PR rating is the better bargian. 2.5Ghz is well over a XP 3000+ level.

There are some on that database hitting 2.7. The average for the 2100+ is 2.38Ghz.

The average for the 1700+ is 2.03Ghz or about 380Mhz less then the 2100+.

It comes down to at what point does it matter to you. For me I will pay the extra $30 for 380Mhz real speed.

Mark

[ToiletDuck]

http://forum.oc-forums.com/vb/showthread.php?s=&threadid=169141

look at poopygood. He is basically running the same system except for processors.

[ToiletDuck]

mind you those are averages. And a lot of those people don't know anything about overclocking. Also we are using dual boards which aren't as good for Ocing. yet poopygood still got it goin on.

[Acaila]

Has he/she tried running VirtualDub on it though? Because this IS an encoding board, so who cares if you can run SETI or whatever or it and it remains stable. In my experience VirtualDub makes an OC'ed board crash way sooner than any of those other "benchmarking apps". Personally I couldn't get past 138 FSB without VDub crashing on air cooling (144 before anything else started crashing). Maybe watercooling will get you a bit farther, but don't expect miracles there if you still want to encode.

Not that you really need that extra boost at that level.

[Zhnujm]

Originally posted by Acaila
In my experience VirtualDub makes an OC'ed board crash way sooner than any of those other "benchmarking apps".

try prime95 for a good overclocking/stability test.
i can encode with vdub/cce/tmpeg for days, but prime95 will show you an error in 5 minutes.
so why care about that if anything else runs stable you may ask ? because i didnt like a computer that calculates wrong, not even in worst case situations. :D

[ToiletDuck]

Seti@home is just as hard as any other benchmarking app or encoding. It uses %100 of your processors all the time. My board is overclocked to 144mhz and I encode just fine. If you can run a seti on each processor then you can encode a movie.

[int 21h]

I can make a simple for loop that will use 100% of your CPU, does that mean its producing valid data?

[ToiletDuck]

no. But seti does.

[Zhnujm]

thats also the point of prime95, long before your pc would crash it shows you that your cpu calculates wrong.
but i wouldnt agree that all progs with 100% cpu utilisation are the same, its also a matter of wich instructions are used.
so better dont rely on only 1 program.

[sam_b]

@Toiletduck
Memory usage will be very different, and as Zhnujm said, different apps use different optimisations (CPU instructions such as SSE2,3dnow). Different areas of the CPU are used, different bits of RAM etc... Seti surely cannot test every result, and then test every test......

Oh, and 100% CPU time is just non idle-loop time, it does not give any information about instruction throughput.

Edit: Removed silly bit about double calculations

[int 21h]

Originally posted by ToiletDuck
no. But seti does.

How do you know the process by which SETI checksums data? Isn't one of the reasons SETI is closed-source is the fact that the developers aren't confident enough in their methods of error-checking that releasing the source would expose this and make them vulnerable to 'bad-data' and erroneous results?

[ToiletDuck]

Isn't one of the reasons SETI is closed-source is the fact that the developers aren't confident enough in their methods of error-checking that releasing the source would expose this and make them vulnerable to 'bad-data' and erroneous results?

Ummmmm nope. Who told you that? What gave you that idea. The reason that it's closed source is because it's a set standard. It isn't Xvid where you can have all the different versions you want. Its sole purpose is to process data that is sent from the observatory. The whole purpose of the program is to have as much computing power possible checking and rechecking data. There is nothing to expose. The fact is that it's for research. It wouldn't do any good if everyone had different versions running. It's for scientific research. And the developers feel absolutely fine with how it works. That's why computers test several times over each star and planet. If there are any differances from what one computer got in europe and one computer got in Texas. Then they go and redo it many times over to find out what the stats of the planet/star are. I'd sudjest you read up some before you make comments about people having programs that do nothing, but make your computer look pretty.

[int 21h]

That's odd considering the following in the FAQ:


Why don't you release the source code?

We decided not to make source code available for security reasons and for science reasons as well. We have to have everyone do the exact same analysis, or we can't have any control over our research and be confident in our results. We were also worried that there may be a few people that want to deliberately try to screw up our database and server.


What do you think these 'security reasons' would be? I would interpet this paragraph as a confession of 'We know there are issues, but if we keep our source obscure, no one will know any better' Security through obscurity at its best.

Don't get me wrong, I'm all for not-for-profit educational distributed projects, but unless you know the exact process to which the data is subject to, you cannot say for certain how valid the results are.

I'd suggest checking out their FAQ (http://setiathome.ssl.berkeley.edu/faq.html#q1.9).

Cheers.

[theReal]

If your system is not perfectly stable, it's possible you get a lot of Seti workunits that seem to be completely done after about 2 minutes. That can't be right (and it never happened to me on a perfectly stable system - that is a Prime Torture Test-tested system)

In my experience, Seti is very forgiving with unstable systems. Prime95 will report errors a lot earlier, and even VDub/Divx encoding will crash before Seti is showing any serious problems.

[ToiletDuck]

int you might as well just reread you FAQ post. It basically backs everything I said lol. Thanks for posting that.

[int 21h]

So what do you interpet the security reasons as? You obviously have the inside track on SETI, and know more than me. Perhaps, you can impart some of this wisdom upon us.

[ToiletDuck]

So what do you interpet the security reasons as? You obviously have the inside track on SETI, and know more than me. Perhaps, you can impart some of this wisdom upon us.

Well... First of all. As mentioned on their website SETI stands for Search for Extraterristrial intelligence. It isn't like there is some guy out there on top of his roof pointing his satellite dish at the sky. This is science. Think of what is involved here. You have the biggest dish in the world that cost billions being used. And it's not just being used for one thing. It's doing tons of things. So any time people get on it is precious. So what they do is that take this billion dollar dish and let it swing around recieving tons of data. Then the data is recorded and then later analyzed many times over by many computers. They do that to test the integrity of the signal. Some places get tested hundreds of times. Now imagine that you have the source code and are able to fiddle with it. So you decide to play a prank and alter it to say that artificial patterns were known in the radio waves recieved. Not only have you A) Wasted valuable time on a billion dollar dish that they have to bid for, B)Wasted the time of thousands of computing hours from people world wide, C) Taken the integrity of the entire program, which is mocked by some anyway as foolish, and made matters for them even worse, when they are trying to be professional about it, but you have also D) altered code that was designed to run on a piece of government hardware meaning that they way this harddware ran could be similar to that of orbiting satellites or other scientific instruments.... Why don't you just ask for the sourcecode that the spaceshuttle instruments use? Or the code used to program laser guided missles while your at it? Why not ask Microsoft for their code??? I'm sure they would say for "security reasons".... but that thought probably hadn't occured.

[int 21h]

This isn't true. Whether or not I submit false results, every day Berkeley will get their 35GB high-density tape (from the telescope) to be chunked out on the SETI@Home server. If I submit false results, the only time I've wasted is the human interaction to verify the results of my false submission.

Please realize that SETI@Home runs at Berkeley on Sun hardware and doesn't go anywhere near government hardware, satellites or other scientific instruments. Also realize that I don't need the source code to fiddle with it (RE: Olli the German and Microsoft SETI team scandals)

Even Microsoft is finally allowing 3rd party (governments at the moment) security audits of its codebase :) But we aren't referring to the security of the code, rather the security and authenticity of the data.

[ToiletDuck]

Please realize that SETI@Home runs at Berkeley on Sun hardware and doesn't go anywhere near government hardware, satellites or other scientific instruments. Also realize that I don't need the source code to fiddle with it (RE: Olli the German and Microsoft SETI team scandals)

SETI get's their information from the observatories which is a massive scientific instrument. And your right. If you faked something it would take some time to retest it. But it would have to be tested many times over to get the percentage of correctness back. And that's just you. If someone made a bot or anything else that could flood the server with invalid information then within a very small amount of time the program could be brought to it's knees. And microsoft might allow code audits. But that is for the purpose of them not infringing on our rights which they try to do whenever they can. Their next operating system codenamed Longhorn is a giantic step in that direction of screwing us over. And if it gets it's way then all this won't be allowed.

[int 21h]

Legacy applications (such as SETI@Home) will still run on Longhorn, even if the 'optional' Palladium extensions are installed (and enabled).

Palladium is the first step in securing the x86 environment, stopping the arbitrary buffer overflows, stack manipulations, real-time debugging (i.e. Nandub), and other things that have plagued the software community since its inception. But, alas, this thread isn't about Palladium and its impact on secure computing.

This thread is about how anyone (besides the researchers at Berkeley)can be sure whether or not the data that SETI@Home generates is actually valid (at Runtime). If the calculations are wrong, the user will recieve no indication of such, and while SETI@Home's data safeguards may weed out the 'bad' data, how does this indicate to you that your overclocking was successful or unsuccessful?

SETI@Home is an interesting initiative, but it is not suitable for testing the viability of overlocked hardware's calculations, no more than a hammer is suitable for use on screws.

[ToiletDuck]

The actual meaning of this thread was to see if more cache would give faster incode speeds. And the main enemy of overclocking is heat. Running programs that use %100 of your processors generate heat. Whether or not they actually use everything. If both my processors can run at %100 and maintain an overclock temp of 30 degrees C then I know it's stable. I've ran prime 95 on each chip for 5 days and still nothing wrong.

[theReal]

The actual meaning of this thread was to see if more cache would give faster incode speeds.

A friend of mine has an old Pentium Pro 200MHz with 1MB Cache - that thing is pretty fast for a 200MHz chip. Crunching Seti on this PPro is about as fast as on a PII 400 - so I guess the 1MB (or even 2MB? I'm not sure which one he has) cache is pretty good for the CPU speed in general.

[ToiletDuck]

Yea in general it is. I was interested in encoding though. Someone said it wouldn't matter because cache is only so a processor can fetch that data faster. And that encoding video's is live streaming so it wouldn't make a differance. But then again MAC's have 1mbyte and they are huge on working with video.

[tiki4]

MACs are doing that far longer than PC's, remember?

Also that kiddies have a totally different CPU architecture, anyway, did someone a comparison of MAC vs. PC recently?

tiki4