How to benchmark video playback?

[shae]

Does anyone know a flexible tool for benchmarking of video (and audio) decoding/playback?

I tried toying a bit with GraphEdit along with an Elecard FPS counter filter, but GraphEdit is awkward and there are quite a few things I can't measure.

[foxyshadis]

Haali's timecodec (http://haali.cs.msu.ru/mkv/timeCodec.exe) should fit the bill.

[shae]

I should've mentioned I tried that one too. :) While it's more streamlined, it reports obscure figures (183234737s?) and not FPS, there's no way to see/control the filter graph, etc.

[reepa]

MPlayer is perfect for this. With the following commandline MPlayer decodes video as fast as possible while ignoring audio decoding and video rendering:
mplayer -benchmark -vo null -nosound pathtofile

When it's done with the video it reports a summary like this:
BENCHMARKs: VC: 97.821s VO: 0.000s A: 0.000s Sys: 7.679s = 105.500s
BENCHMARK%: VC: 92.7213% VO: 0.0000% A: 0.0000% Sys: 7.2787% = 100.0000%

You can get the average FPS by dividing the decoding time by the length of the video. Also I think MPlayer shows a continuous FPS counter in the command prompt in Linux (it's disabled in Windows for some reason). There might be a way to save the value for each frame in a text file somehow.

[shae]

Last time I checked it didn't look like there's a friendly non-pre-alpha MPlayer for Windows. But even if there were, can it use DShow filters and not the internal ones?

[Awatef]

The way I do it is the following:

Open Throttlewatch (http://www.softpedia.com/get/Tweak/CPU-Tweak/ThrottleWatch.shtml), start the video on your favourite player, then click on the throttlewatch window and hit F5. Wait a minute, then hit F5 again. You'll find a "journal.txt" file in the same directory where you put ThrottleWatch. At the end of this file, you'll find the average CPU load.

The lower the value, the faster the decoder ;)
For example, I just benched libavcodec and DivX for XviD decoding. Libavcodec gives 20% average, DivX gives 25%, you got your champion ;)

[foxyshadis]

I should've mentioned I tried that one too. :) While it's more streamlined, it reports obscure figures (183234737s?) and not FPS, there's no way to see/control the filter graph, etc.

Oops, I missed this. For what it's worth, I always get values like this:

User: 52s, kernel: 0s, total: 52s, real: 63s, fps: 41.0, dfps: 33.7

The very last one is the only one that's important for benchmarking. The first four are the raw amount of wall clock time spent decoding. User/kernel is meaningless unless you're a developer. Total means the time spent in the decoder, real includes reading, splitting, decoding, and display, along with any color conversions that happen along the way, so it benchmarks the entire playback chain. (With null renderer, it benches the splitting+decoder only.) The fps and dfps are based on the total and real time respectively.

[zambelli]

If you're benchmarking playback, you could try GraphEdit with Graph | Use Clock disabled and then check the video and audio renderers for stats when the source is finished playing. Note that you'll need to start the graph once and then stop it before disabling the clock.

If you're strictly benchmarking decoding, you could try wrapping the source into DirectShowSource() via Avisynth and then running it through avs2avi.exe with -c null -o n parameters. The Avisynth/avs2avi layers will introduce some overhead, but if you're benchmarking for relative figures, it won't matter.