I Need Help Testing Codec Performance (encoding fps)

[LastQuestion]

My goal is to do high fps capture of video games. I'd like to achieve at least a consistent 120fps.

Unfortunately I've run into a bottleneck which is preventing this. I need to find out where this bottleneck occurs - whether it is due to h/w or s/w limitations.

Before I go spending more money on h/w it seems best to determine whether there is even a lossless codecs capable of such performance.

Specifically, I would like to determine what video codecs can encode 1920x1080 content at 120fps+(real-time) on a PC running W7-64.

Any information, guidance, or advise that seems relevant would be greatly appreciated.

[fumoffu]

I'm curious - why you want 120fps? 60 seams more reasonable and fluid enough, also nobody with 60Hz monitor will be able to tell a difference and even with 120Hz or more it would be difficult I imagine.

(notice this is different than in games because every decent player uses vsync so if monitor is 60Hz it just won't display half of the frames at all, also 60fps video is more fluent than 60fps in games because in games frames usually aren't rendered in even, identical intervals)

btw. It might be more doable to capture in 60 and then interpolate to 120fps if you really need this.

From technical standpoint I doubt you can go lossless because at that speed you would have to write 700MB to HDD every second if you just use uncompressed avi, and I don't know how efficient lossless video codec are but even half of this is too much.
With codec like x264, encoding this even on fastest preset and playing the game at the same time, you would need 2 CPUs.
I know there are special hardware cards for that I think they can do FullHD at 60fps so this is probably your best option.
But again I don't know what do you need this for and what kind of budget you have for this.

Did you try using OBS (Open Broadcaster Software), try encoding with this and see how much fps can you get. This should give you an idea how much more would you need.

[LastQuestion]

I'm curious - why you want 120fps? 60 seams more reasonable and fluid enough, also nobody with 60Hz monitor will be able to tell a difference and even with 120Hz or more it would be difficult I imagine.

Editing clips for slow-motion is much easier with more frames. I hope to do some interesting things with twixtor using 120fps footage.

From technical standpoint I doubt you can go lossless because at that speed you would have to write 700MB to HDD every second if you just use uncompressed avi, and I don't know how efficient lossless video codec are but even half of this is too much.
With codec like x264, encoding this even on fastest preset and playing the game at the same time, you would need few CPUs.
I know there are special hardware cards for that I think they can do FullHD at 60fps so this is probably your best option.
But again I don't know what do you need this for and what kind of budget you have for this.

Did you try using OBS (Open Broadcaster Software), try encoding with this and see how much fps can you get. This should give you an idea how much more would you need.

I wish to determine what codecs are even capable of rendering 1080@120fps+ in real-time. I don't want to go buying tons of h/w eliminating every imaginable bottleneck only to find out the codecs are whats holding it back.

I'll try out OBS to see if it has different results.

Edit: OBS basically breaks when set to 120fps.

[fumoffu]

I did quick test and OBS seems to work perfectly fine with 120fps.
What do you mean breaks? Maybe it doesn't have enough processing power... Did you configure it at all?
go to advanced and check
"Allow 61-120fps entry in video settings" and change x264 preset to ultrafast.
In "Video" try downscalling resolution and see if this helps.
In "Encoding" check "use custom buffer size" and set it to 0
Max Bitrate set to something large just in case like 80000
(for better quality in "custom x264 Encoder settings" type crf=18)

[raffriff42]

...it seems best to determine whether there is even a lossless codecs capable of such performance.

Specifically, I would like to determine what video codecs can encode 1920x1080 content at 120fps+(real-time) on a PC running W7-64.The fastest codec is no codec - uncompressed. Of course, this requires tons of storage bandwidth. To eliminate storage as a bottleneck, set up [and write to] a RAM disk (http://memory.dataram.com/products-and-services/software/ramdisk). For short recordings of course. After performing this test, you will find if your bottleneck lies elsewhere. (hint: it does)

[foxyshadis]

Run some monitoring in the background on how much CPU, memory, disk load you're using. Something like ManageEngine's Windows Health Monitor, or Systemometer, or whatever can record a log. Asking this question is silly, because with enough hardware almost any codec would be able to do what you want, so you have to figure out your bottleneck first.

Along with that, "OBS basically breaks when set to 120fps." is completely unhelpful. How does it break? Error messages? Too slow? Crash?

Oh, and make sure you write recordings to a separate disk (or raid array). You obviously can't write to the same disk you're loading the game from.

[LastQuestion]

I did quick test and OBS seems to work perfectly fine with 120fps.
What do you mean breaks? Maybe it doesn't have enough processing power... Did you configure it at all?
go to advanced and check
"Allow 61-120fps entry in video settings" and change x264 preset to ultrafast.
In "Video" try downscalling resolution and see if this helps.
In "Encoding" check "use custom buffer size" and set it to 0
Max Bitrate set to something large just in case like 80000
(for better quality in "custom x264 Encoder settings" type crf=18)

Breaks, as in at a quality setting that doesn't look terrible there is rapid variance in framerate, and frame drops are very visible.
There's really no point in high fps capture if the video quality is that terrible.

However, I didn't realize the x264 field would accept CRF values. I'll do more extensive testing. Also, by fine, do you mean you counted the frames to insure there are, in fact 120fps?

[LastQuestion]

So, as suggested I tweaked the x264 settings in OBS and monitored CPU usuage in order to be thorough, and fair. Settings as follows: Quality Balance 0, Bitrate 80k, Buffer 0, CRF 20, preset Ultrafast, profile High.

The quality is awful. Whats more is that it has less frames than fraps/dxtory capture. While OBS was capturing my 2500k@4.6Ghz, with unparked cores, it peaked at 70% load, briefly, before returning to what looks like an average of 50-60%.

I came here with a specific question, for very specific information. Tests like these are not my goal at this point in time as I have already done extensive testing that has lead me to question the performance of these codecs.

Since so many of you are adamant that they are not the bottleneck I would appreciate it if someone would care to elaborate as to why that is.

[fumoffu]

1. You should set quality balance to 10 not 0, although this shouldn't matter if you use crf command later, if quality 20 is not enough go to 16 or even lower (the lower the better).

2. In advanced also check "Use CFR" - this means constant frame rate.

3. It's possible that OBS is working fine but when you try to watch the video the player you use doesn't play it correctly. While I recorded my 120fps test video correctly, most players including VLC did not play it back smoothly - it lagged every second or two, like there were frame drops. The only player that played it correctly was MPC-HC (latest version). Also it might be good idea to disable DXVA in player settings (or rather in internal LavFilters in this case) because most likely 1080p @120fps is too much for hardware decoder build in your graphic card, better use software decoding.

4. I doubt 1 cpu with 4 cores will be enough to play any modern game and record at such high speed. You probably need 6 cores minimum or server motherboard with 2 CPUs. 60-90fps sounds more reasonable and should be more than enough for slow-motion.

5. I came here with a specific question, for very specific information. Tests like these are not my goal at this point in time as I have already done extensive testing that has lead me to question the performance of these codecs.

Since so many of you are adamant that they are not the bottleneck I would appreciate it if someone would care to elaborate as to why that is.
I'll just repeat what foxyshadis already said:
"Asking this question is silly, because with enough hardware almost any codec would be able to do what you want, so you have to figure out your bottleneck first."
So unless you CPU usage is close to 100% there is no reason why codec would not encode as fast as you want.

[LastQuestion]

1. You should set quality balance to 10 not 0, although this shouldn't matter if you use crf command later, if quality 20 is not enough go to 16 or even lower (the lower the better).

The lower the CRF, the lower the fps encode. Lowering the CRF won't help here.

2. In advanced also check "Use CFR" - this means constant frame rate.
"This will force OBS to output at a constant frame rate, duplicating frames if necessary to achieve this." (http://obsproject.com/forum/viewtopic.php?f=18&t=642)


3. It's possible that OBS is working fine but when you try to watch the video the player you use doesn't play it correctly.

I imported the video into Sony Vegas Pro 12, set the project settings to 60fps, then set playback on the clip to 50%. I counted the number of frames that were missing. There were a lot of missing frames - meaning the frame rate dropped below 60fps, repeatedly.


4. I doubt 1 cpu with 4 cores will be enough to play any modern game and record at such high speed.

The games I've been testing use no more than 25% of my CPU, even with the framereate unlocked and running at 250fps+. When capturing with lossless codecs my CPU stayed below 70%.

I didn't come here thinking these codecs have to be at fault. My personal view has been that this is most likely a h/w bottleneck.

Before I go throwing more money at this problem I need to know, for a fact, these codecs aren't at fault. All I want to know is how to test them accurately.

[raffriff42]

All I want to know is how to test them accurately.Run some test recordings while playing the game, comparing the performance of each candidate codec. For the most accurate measurement, use Fraps' benchmark feature (you don't have to capture with Fraps to use the benchmark feature, if you don't want to) and create .CSV log files of individual frame render times. Frame time measurement has been widely discussed in the last few years on techreport.com (http://techreport.com/review/21516/inside-the-second-a-new-look-at-game-benchmarking) and elsewhere.

Run a benchmark for each candidate codec (capturing a repeatable sequence in the game for consistency) and compare them. You can view benchmark results in a spreadsheet like any .CSV file, but I have written a viewer application for Fraps benchmark files ('https://frapsforum.com/threads/new-tool-for-viewing-fraps-benchmark-files.2121/') which is very easy to use.

I think I would write to a RAM drive as I suggested above (http://forum.doom9.org/showthread.php?p=1656142#post1656142) (free for up to a 4GB drive; write speed ~ 5000 MB/s) for maximum FPS, at least in the early stages. Once you get consistent 120 fps writing to a RAM drive, you can move to a HDD or SSD. However, doing this makes the higher compression codecs look worse than they would be if writing to a HDD - compression is slow, but so are disk drives, and a more compressed file writes faster. So I guess you need to test using your actual hardware as well.

[fumoffu]

"This will force OBS to output at a constant frame rate, duplicating frames if necessary to achieve this." (http://obsproject.com/forum/viewtopic.php?f=18&t=642)



I imported the video into Sony Vegas Pro 12, set the project settings to 60fps, then set playback on the clip to 50%. I counted the number of frames that were missing. There were a lot of missing frames - meaning the frame rate dropped below 60fps, repeatedly.


OBS by default uses variable framerate so if you don't enable CFR it might droop frames if they are very similar. This option will duplicate the frames only if the the input frames are already duplicated - as in the game didn't have time to render next frame.
btw. Make sure you are using "game capture" not "window capture" or "monitor capture" in OBS, because the last two can limit your frame rate, especially if you have aero enabled or don't have monitor with >60 refresh rate.
If you can't get even 60 stable fps with OBS (and CPU is not your limiting factor) there is something wrong with your settings.

here is my 120fps test video, count the frames in this ;-)
http://www.multiupload.nl/LCWUSIQZ20
this is in 720p (and crf 16) but today I tried in 1680x1050, crf 20 (can't go higher - that is my monitor resolution) and it works perfectly well with cpu usage 50-70% on i5 3570 (4cores) @4Ghz

[LastQuestion]

here is my 120fps test video, count the frames in this ;-)
http://www.multiupload.nl/LCWUSIQZ20
this is in 720p (and crf 16) but today I tried in 1680x1050, crf 20 (can't go higher - that is my monitor resolution) and it works perfectly well with cpu usage 50-70% on i5 3570 (4cores) @4Ghz

I originally encountered this issue several months ago. One of the first tests I did was to see what the results were in 720p. In 720p every single one of the codecs I suspect might be bottlenecked with 1080p content captured at a steady 120fps with no frame drops.

Extensive testing meant extensive testing. Please stop patronizing me.

[fumoffu]

OK this is a waste of my time. I honestly don't understand what are you even trying to say.
In 720p every single one of the codecs I suspect might be bottlenecked with 1080p content captured at a steady 120fps with no frame drops.
This sentence don't make any sense. Are you using google translate or something?
You just keep talking about some imaginary bottlenecks in codecs while there is no such thing. The only thing that can bottleneck you is your hardware - CPU and RAM.
I'm out :/

[LastQuestion]

OK this is a waste of my time. I honestly don't understand what are you even trying to say.

This sentence don't make any sense. Are you using google translate or something? My wording was not the best. I'll will rephrase that statement.

Each of the lossless codecs I have tested have been able to capture 720p content at 120fps. The capture had no dropped frames. I suspect the substantially increased data of 1080p content is too much for these codecs to handle. Since I could find no information online testing such content I would need to perform such tests myself.


You just keep talking about some imaginary bottlenecks in codecs while there is no such thing.

This has been stated multiple times by you and others. If you have evidence which can demonstrate these codecs are not, in any way, a bottleneck then please provide it.

[mariush]

Let's do the numbers man ...

1920 x 1080 x 3 bytes per pixel x 120 frames per second = 746,496,000 bytes /s = 711 MB/s

Some capture applications do conversion in hardware from RGB to YV12 so then there's only 2 bytes per pixel instead of 3.. but you're still looking at 497,664,000 bytes/s or about 470 MB/s

This is a lot of data to be copied from the video card in the first place. Then, your hard drives can't sustain this speed, unless you have a RAID5 made out of SSDs or something like that. It needs to be compressed.
Very fast lossless codecs like the ones in Fraps will compress with a poor ratio, maybe they'll reduce the content to about 65-70% but you're still looking at about 200-250 MB/s out of the 470 MB/s of uncompressed data.
Codecs like x264 or other non-lossless codecs will compress much better, maybe they'll manage to do 1920x1080 120fps at around 200 mbps or about 20-30 MB/s but they'll use a lot of processing power, which in turn they'll take cpu cores from the game and lower your framerate.

At 1280x720 it's much easier so it's no wonder codecs don't have problems with encoding 120 frames per second without problems ... you're dealing with 1280x720 x 2 x 120 = 221.184.000 bytes or 210MB/s ... note that it's less than HALF the size you have for 1920x1080 @ 120fps. Also note that even the crappiest codec will encode this down to about 150 MB/s or less, which any modern hard drive can sustain, at least for the duration of a test. With higher resolutions and more data, your hard drives may choke resulting in lower framerates.
In theory, the encoding effort for 1280x720@120fps should be comparable to 1920x1080 @ 50 fps but in practice it should actually be comparable to about 1920x1080 @ 30-35 fps. Any codec can do this with a modern computer so it's no wonder it works at 1280x720.

Get back to us with info about your hardware .. where do you store the captured data (what hard drives or ssds, what configuration raid or not etc) , what's the bitrate of the videos you capture now in your experiments, what's your memory bandwidth etc etc

[vivan]

Some capture applications do conversion in hardware from RGB to YV12 so then there's only 2 bytes per pixel instead of 3.. Actually it's 12 bits instead of 24. But it would be 16 for 4:2:2.

[foxyshadis]

LastQuestion, a dedicated capture drive is an absolute prerequisite, you still haven't mentioned whether you're using a second dedicated hard drive and whether you've measured the data rate on it. That's actually much more important than the CPU monitoring unless you start really pushing x264 harder. For 120FPS at good quality, you need the fastest possible, in fact a RAID 0 deck of drives to capture to would be even better but might be overkill. (You could also capture to a large high-end SSD, but that's extremely expensive.) Similarly, if it's an external drive, make sure it's on USB 3 or eSATA, not USB 2. DXTory is the only one that can split writing across multiple drives without making a RAID 0.

Fraps will require 250MB/s in regular (YV12) mode or 500MB/s in RGB mode; even the regular mode precludes all but top-end external drives like the Angelbird SSD2Go, and even a lot of internal hard drives. OBS and DXTory, while not as fast at raw capturing as Fraps, can at least lower the bitrate to more manageable levels with compression. (In DXTory, you can choose an MJPEG codec like FFDShow's; its internal codec is lossless. Even its compressed YUV420 mode will likely be at least 200MB/s.)

Disk is usually the first place you should look for a capturing bottleneck. Check things like write speed and I/O queue length to find out if you're getting as much onto it as you should be. You'll probably find that you're disk bound, since you apparently aren't CPU bound, and if you can get it to work at all with bad settings.

If you do become CPU bound, you can try switching x264 to intra-only mode by adding the custom option keyint=1. It's faster, but much higher bitrate, so you have to worry about a disk bottleneck again. That's the difficulty of pushing your system near its limit.

[LastQuestion]

Run some test recordings while playing the game, comparing the performance of each candidate codec. For the most accurate measurement, use Fraps' benchmark feature (you don't have to capture with Fraps to use the benchmark feature, if you don't want to) and create .CSV log files of individual frame render times. Frame time measurement has been widely discussed in the last few years on techreport.com (http://techreport.com/review/21516/inside-the-second-a-new-look-at-game-benchmarking) and elsewhere.


Let's do the numbers man...
...
Get back to us with info about your hardware ..

Sorry for the delayed response raff. Since you took the time to directly link the TR article I reread it to see if I had forgotten anything important. Figuring out how to get perfmon and process explorer to monitor and log the right information took time as well. I also ran the Fraps frametime benchmark you suggested and I'm curious why you'd ask for this type of information.

As to performance logs of i/o and cpu usage the results are unlikely to be much use at this point in time. Explaining why I believe this is the case involves my h/w configuration - this post might be somewhat lengthy.

Okay. Numbers.

I have two 3tb Seagate .14s in Raid 0 with a sustained write speed of 380MB/s, which peaks at 410MB/s while the heads are close to the platters center. This array is dedicated entirely to capture. My Asus p8z68-v/Gen3 has an Intel raid controller and a PCI-E 3.0 bus. The raid has been configured to use the controller - it is not software based via windows.

When capturing gameplay with Fraps/Lags/Utvideo/Xtory in 1080@60fps the bitrate is ~700Mbps. So, twice the framerate at 120fps would be ~1400Mbps, which is ~180MB/s. When I capture with uncompressed full RGB via Dxtory the average bitrate (via mediainfo) is ~3200Mbps.

Now, in terms of performance the codecs with the highest compression have the lowest fps. This seems to indicate a CPU issue rather than HDD write speed.

The game, when framelocked to 120fps, use's ~15% CPU. When capturing at 120fps via Dxtory CPU usage averages to about 80%, with some peaks to 90%(rapid scene changes with fast camera movement). So, it might be a CPU bottleneck right? That's what such data seems to indicate. I'm not so sure. Why that is has to do with my GPU and the overhead from how Fraps/Dxtory capture gameplay.

My GTX 570 has 1.2GB of vram. Playing this game in 1080p fills it up. I did some tests monitoring vram usage by freezing the gameworld so that no events were occurring; even effects such as fire stop their animation. When I began capturing with Dxtory vram usage doubled.

So, how could this possibly not be a CPU bottleneck? Well, funny thing about GPUs with vram issues - they have to constantly get data from RAM, which places strain on the CPU while also saturating memory bandwidth. How much of that CPU load is from the codec; how much of it is from game assets being constantly loaded from RAM; does the data Dxtory sends to the CPU/RAM, and then out again to write to the HDD, significantly impact the frame latencies seen in the Fraps benchmark?

Figuring this stuff out, doing these tests, defining the limits of the tools I need to use, is part of what I find interesting about game capture. All I need to know in order to continue that process is how to accurately test these codecs. Obtaining that information is the only reason I posted this thread.

[raffriff42]

Thank you. This information should have been in your first post - it would have saved everyone some time.

>I also ran the Fraps frametime benchmark you suggested and I'm curious why you'd ask for this type of information.
I wanted you to compare the differences in average and worst-case frame times with the various codecs you are considering. You asked how to test accurately, and this is the most accurate test that I know of - without some sort of high-end hardware analyzer (http://teledynelecroy.com/protocolanalyzer/protocolstandard.aspx?standardID=3).

>So, how could this possibly not be a CPU bottleneck?
It is - in a manner of speaking. The CPU usage you see is from I/O operations, not codec number-crunching, which is relatively trivial in the case of Huffman-type codecs such as Fraps and DXT.

During video capture, a video frame has to traverse the RAM buss, passing through the CPU, several times - I'm not sure how many - as it is transferred from vram (accounting for your doubled vram I/O) to the CPU, compressed, write-cached and finally sent to the HDD. This adds to the already heavy RAM>VRAM traffic for texture & vertex loading that you mention.

...If you are looking for some sort of measurement of codec performance alone, without the load of the game, you *could* capture some relatively non-demanding DX source such as my Frafs Test Pattern (https://sourceforge.net/projects/frafstestpatt/)program, which easily runs at > 1200 fps. This would not be as useful as testing with the actual game, IMO.

[LastQuestion]

I got my new SDD yestereday. Today I installed W7-64 SP1 on it, as well as UTvideo 13.3, Lagarith 1.3.27, AMV2mt/AMV3, Dxtory, Fraps, Steam, Killing Floor, Teracopy, and Nvidia's Geforce 331.65 (custom:driver+physx).

I did the same tests I had done earlier and found no difference in performance. I then reverted back to 314.22 as it has proved to be the most stable release for 500 series cards.

FRAPS and AMV3 were able to capture at a steady 120fps to the Raid 0 Array. Dxtory almost could, in 4:2:0(compress), but the fps varied rapidly between 110-120. UTvideo/LAGS were marginally improved, but still far below 120fps. Capturing with UTvideo/Lags/Dxtory used ~75% CPU with 314.22; which is ~10% less than 331.65.

AMV3 capturing at 120fps, with the S1 preset (1:4 compression), used ~50% CPU.
FRAPS could also capture at 120fps when using 314.22.
For all of these tests the performance was the same when capturing to the SSD (450MB/s sustained write speed)
Locking the framerate to 120fps or leaving it unlocked did not affect capture performance. Killing Floor uses approximately ~10% more CPU with an unlocked framerate.


A CPU or HDD bottleneck seems highly unlikely at this point. The GPU does indeed seem to contribute to this problem, but the disparate performance between AMV3/FRAPS and UTvideo/LAGS demonstrate that some codecs are more efficient than others for 1080@120fps capture. While stating they are bottlenecks may not be entirely accurate, it is, nonetheless, not far from the truth.

This issue is clearly far more complex than it initially appears.

I won't be able to test the GPU's influence further until I upgrade from my GTX 570. Unfortunately, that won't occur until r9 290/X's with aftermarket coolers are released and reviewed.