Hardware interrupts through the roof

[odysseus]

I was playing around with trying to make DGIndex go faster, looking at various performance counters.

I first noticed that when you increase the priority, the CPU utilization of DGIndex never goes higher than 10%.

Then I looked at the counters and saw the hardware interrupts were occasionally up to 99% of CPU time (and DGIndex itself was 1%)

First, is this normal or am I doing something weird without knowing it? (I leave all the settings to the defaults, basically, and I use DGIndex to do clipping)

Would there be any way to get DGIndex to use more memory & lower interrupt processing? I have 1 GB of ram on this machine.

[jstelly]

Interrupts are a function of, or are really triggered by your hardware and the drivers for the hardware. I've never looked at the DGIndex source code, but I've got a good idea of what it does...

What filesystems are you reading from/writing to? Is your source or destination a network share? I'm guessing it might be, and if so then it's probably your network card generating the interrupts. Or maybe it's your hard drive controller?

[odysseus]

(don't confuse me with another odysseus, I'm changing my login to ody_seuss ASAP)

Neither the source or destination are on a network.
Source and destination are NTFS

copying a file from the same source to destination as a DGIndex run generates almost zero interrupts (to answer your question whether it's the disk controllers)

I can make the Hardware Interrupt jump at any time, just by increasing or decreasing the priority of the DGIndex executable.

Priority up, interrupts up. Priority down, interrupts down.

I don't really need to increase DGIndex's speed (it takes 2 hours for a large set of VOBs when my system's heavily loaded, 30 minutes when it's lightly loaded).

I was just curious what was going on, this interrupt behavior is really quite rare to me & I do a lot of performance optimization work on database machines. I remember reading that some x86 programming can mimic hardware interrupts and I thought maybe that's what DGIndex does.

Interrupts are a function of, or are really triggered by your hardware and the drivers for the hardware. I've never looked at the DGIndex source code, but I've got a good idea of what it does...

What filesystems are you reading from/writing to? Is your source or destination a network share? I'm guessing it might be, and if so then it's probably your network card generating the interrupts. Or maybe it's your hard drive controller?

[Guest]

What utility are you using to monitor hardware interrupt counts?

[odysseus]

First, thanks for some great software Mr. Graft. I use your vDub filters too.

The tool I'm using is

process explorer

www.sysinternals.com

Hardware Interrupts is the 2nd item down when you first start up the program. Actually and unfortunately it doesn't tell you counts or where the interrupts are coming from, just the total CPU time spent in the interrupt handlers.

All my DGIndex tests over the last few days have been done with virtualdub going in the background (a long set of batched jobs). I'll repeat the tests when the current job finishes up.

What utility are you using to monitor hardware interrupt counts?

[odysseus]

I tested it out with nothing running in the background.
The high hardware interrupts happen on some VOBs and not others.

For the VOBs where the high interrupt is happening the processing time is around 25 minutes, for the regular VOBs it's around 4 minutes.

I cannot find anything different in the VOBs.

high interrupt and low interrupt VOBs are all interlaced, same frame rate, same aspect ratios, they're all using ac3 (I thought for a few minutes maybe the high interrupt ones are the VOBs with MPA or LPCM, but they're all AC3)

I then noticed one high-interrupt VOB had field order of 'bottom' and thought that could be the issue, but another VOB showed up with high interrupts and field order of 'top'.

I'll run some other tests / look for other differences if anyone suggests differences to look for.

I was hoping someone knew what's going on off the top of their heads. The performance is not so critical even on the biggest VOB sets that it's worth spending too much time worrying about it.

[JuanC]

... process explorer ... www.sysinternals.com ... Hardware Interrupts is the 2nd item down when you first start up the program. ... Could there be a hardware problem? I saw a similar behavior with process explorer on my old P3, with a difference: interrupts were using only just 20-25% of CPU, especially during and after using CPU intensive applications. The problem was with a DIMM. There was a lot of dust accumulated on top of it, and after cleaning it, I used Memtest86+ (http://www.memtest.org/) to test memory, to find out it had developed bad spots. The problem was gone once I replaced the bad DIMM. Hope this helps.

[jstelly]

(this is just a guess) It could be virual memory. If you monitor page faults/sec in perfmon while monitoring interrupts, do the numbers sync up between the two? On my Athlon64 3400 I maxed out at about 20K interrupts a second (as seen in perfmon) and they accounted for about 20-25% CPU Usage. That was reading from/writing to a network share.

[odysseus]

If it was a hardware problem I would expect that one VOB would show up with high interrupts one time, and low interrupts another time. But in fact the same VOB had the same behavior over 5 tests over 5 days.

Memory pressure wasn't raising interrupt handling times either. One of the tests was with 2 programs taking up almost all the physical ram (2 separate instances of 7zip, from 7zip.org). The memory page faults and the hardware interrupts are completely unrelated (the DGIndex priority toggling turned interrupts up & down the same as when I had 1 gig free, with no memory-hogging programs running.).

(this is just a guess) It could be virual memory. If you monitor page faults/sec in perfmon while monitoring interrupts, do the numbers sync up between the two? On my Athlon64 3400 I maxed out at about 20K interrupts a second (as seen in perfmon) and they accounted for about 20-25% CPU Usage. That was reading from/writing to a network share.