Smaller File Size With Higher FPS(?)

[Danette]

Starting point: two identical AVI segments processed through VD 1.10, with one segment being converted to 59.94 fps (via QTGMC avs filter) and the other to 29.97 fps (native capture).

The 59.94 conversion is double the size of the 29.97 ...as you would expect. However, after I compress them via H264 (Ripbot), the 59.94 file is 15% smaller in size than the compressed 29.97.

This makes no sense, to me, and I'm wondering if this is as it should be or is there something wrong that I should be investigating.

[LoRd_MuldeR]

It's impossible to give any advice unless you tell us what encoder settings you used. Especially what rate-control mode!

For example: If you use ABR or 2-Pass mode, files of the same duration will always come out at the same size - regardless of the resolution or frame rate - unless you change the target bitrate.

[Danette]

True. H264 settings are CQ mode at 16 CRF.

[LoRd_MuldeR]

True. H264 settings are CQ mode at 16 CRF.

"CQ" (constant quantizer) and "CRF" (constant rate-factor) mode are mutually exclusive, so either you have been using "CQ" or "CRF" mode ;)

The different RC modes are explained here:
http://git.videolan.org/?p=x264.git;a=blob_plain;f=doc/ratecontrol.txt

However, if you were using CRF mode, your observations can be explain by the fact that x264's rate control implementation is "frame rate aware". Put simply, at higher frame rate, you get lower "per frame" quality at the same CRF value. This accommodates for the fact that the loss is less visible at the higher frame rate, i.e. when the frame is displayed for a shorter period. This way you save a lot of bits and still get approximately the same "perceived" quality (at playback speed).

See also:
http://git.videolan.org/?p=x264.git;a=commit;h=c583687fab832ba7eaf8626048f05ad1f861a855

[Danette]

Interesting. So you're saying (along with the links) that when set to CRF, the higher the fps, the higher the compression (lower per frame quality) which results in the smaller file size than lower fps?

I wasn't so much focused on the CQ aspect, having heard that it's of little practical use, but it is the only way to get at a CRF quality setting in Ripbot.

Would you think that perceived quality would be higher at the lower 29.97 fps at the same CRF level as the 59.94 fps?

Conversely, if I lower the CRF level (gaining quality) at the higher 59.94 fps and match the file size of the 29.97 fps higher CRF level, would perceived quality of the lower CRF 59.94 be higher than the 29.97 higher CRF level? That's a mouthful!

[foxyshadis]

Normally higher fps means higher bitrate (scales by about 40-50% more for every doubling, so efficiency increases), but only for the same video. If they're two different videos, then you'd never expect them to come out to the same bitrate no matter what their framerates are.

The 60p one is probably just much easier to encode.

[CarlEdman]

Is your source by any chance interlaced? A lot of 29.97 fps material is and that you would bother to do a frame-rate doubling also suggests the same.

An unprocessed interlaced frame consists out of two fields which are slightly off-set in time. If there was substantial movement (either object or camera) that will result in a lot of spurious big horizontal jaggies that I've seen x264, when processing the frame as progressive, waste a lot of bits on.

If so, go with the rate-doubled encode or, for further reductions in file size with little quality loss, sample the result of that back down to 29.97 fps using MFlowFPS(), for example.

[LoRd_MuldeR]

Interesting. So you're saying (along with the links) that when set to CRF, the higher the fps, the higher the compression (lower per frame quality) which results in the smaller file size than lower fps?

I wasn't so much focused on the CQ aspect, having heard that it's of little practical use, but it is the only way to get at a CRF quality setting in Ripbot.

Would you think that perceived quality would be higher at the lower 29.97 fps at the same CRF level as the 59.94 fps?

Conversely, if I lower the CRF level (gaining quality) at the higher 59.94 fps and match the file size of the 29.97 fps higher CRF level, would perceived quality of the lower CRF 59.94 be higher than the 29.97 higher CRF level? That's a mouthful!

If CRF mode did not take the frame rate into account, the 60 fps version would probably come out a lot larger than the 30 fps version at the same CRF value. Maybe not exactly twice in size, for the reasons foxyshadis mentioned, but still a lot larger. Also, the 60 fps version would probably come out at significant better quality than the 30 fps version. That's because the 60 fps would have approximately the same "per frame" quality as the 30 fps - but twice the frames per second!

By taking the frame rate into account, which x264 does for a long time now, you get much more consistent results among different frame rates. Lowering the "per frame" quality for higher frame rates means that the same CRF value will now give roughly the same perceived quality for the 30 fps and the 60 fps version, instead of making the 60 fps significant better quality. Or, as the commit message says: "This will make --crf closer to constant perceptual quality than previously".

Last but not least: Did you actually look at the results? Unless the 60 fps version looks noticeably worse than the 30 fps version, there is no reason to worry about the smaller size ;)

[raffriff42]

I did a few tests. Surprisingly, for equal video content, in CRF mode, doubling the frame rate does [not] make a 100% larger file - it might be only 5% to 50% larger. I think this is because there is less difference between adjacent frames, therefore less to encode (in all frames other than I-frames). Motion compensation works better too, as pixel displacement between frames is less.

[Danette]

Yes, my source (VHS) is interlaced. I used two approaches in creating this problem/question, although not initially intentionally. The capture was at 29.97 fps. I processed a single segment of this source two ways and both using VD 1.10. The first using Donald Graft’s “Smart Deinterlace” filter, which retains the 29.97 source fps. The second using the QTGMC deinterlacer for Avisynth (which I find cleans up/presents video fantastically - even non-interlaced video). QTGMC doubles the fps to 59.94. The result of the QTGMC output is the doubling of the file size vs. the 29.97, as you would expect. I will try the MFlowFPS() ...and thanks for that suggestion.

Then, to compress the deinterlaced products, I used Ripbot264. That’s when I noticed the odd file size difference where I expected the doubled 59.94 fps compression to be larger than the lower 29.97 fps compression. The CRF mode (at least in Ripbot) seems to take the frame rate into account because, rather than a doubled file size resulting from the h264 compression, the file size of the 59.94 fps segment is actually smaller, by 15%, than the same CRF application to the 29.97 h264 compression. The same CRF value was applied to both.

LoRd_MuldeR’s explanation of h264 treating higher frame rates to lower quality compression seems to explain this behavior and may indicate that, if I wish to go for a larger file size (up to the size of the 29.97 fps compression, for example), I can impart even more theoretical quality by using an even lower CRF level on the 59.94 compression.

I have looked at both results and cannot say that I can see a difference in quality. It is more a matter of trying to settle upon the best possible quality for personal satisfaction reasons. You know how that goes.

Thanks to all for the discussion and explanations - very helpful.

[LoRd_MuldeR]

LoRd_MuldeR’s explanation of h264 treating higher frame rates to lower quality compression seems to explain this behavior...

Just to make it clear: This has nothing do with h.264 in general. It's how rate-control, or more specifically "qcomp" (quantizer curve compression), has been implemented in the x264 encoder software.

Some basic info on how "qcomp" works:

Quantizer curve compression, or “qcomp”, is a very old and simple idea. Lower the quality in areas of the video with high complexity and raise the quality in areas with low complexity. The original implementation dates at least back to the original libavcodec ratecontrol. There are three justifications for this:

1. High-complexity scenes generally have high motion, and one is less likely to notice quality loss in a high motion scene, since fine detail is impossible to see in motion anyways.

2. High-complexity scenes are extraordinarily costly bit-wise, so even if it makes quality somewhat worse, it might be worth it to save those bits to use elsewhere.

3. Each frame in a high-complexity scene is not referenced very far into the future since there is a great deal of change between each frame, so even from a PSNR perspective, one should allocate fewer bits to those frames as opposed to frames which are nearly static.

x264 uses qcomp pretty much unchanged except that it performs a gaussian blur over the frame complexities to avoid too much local fluctuation between quantizers.

[STaRGaZeR]

I processed a single segment of this source two ways and both using VD 1.10. The first using Donald Graft’s “Smart Deinterlace” filter, which retains the 29.97 source fps. The second using the QTGMC deinterlacer for Avisynth (which I find cleans up/presents video fantastically - even non-interlaced video).

Without looking at the filtered results I'd say the mystery lies here, not in x264. You know that clean sources, having little to no random noise, compress a lot more for a given quality (I mean same CRF here) than noisy sources. So even if you have to compress more frames, if they are cleaner then you can perfectly get smaller file sizes. And your source is VHS, which is usually full of difficult to compress artifacts.

[Asmodian]

I agree. QTGMC usually results in very easy to compress video. The stabilization and cleaning done by QTGMC v.s. Smart Deinterlace probably accounts for the smaller size more than the frame rate awareness of x264. The frame rate awareness of x264 does decrease the size of higher frame rate video but not below the size of a lower frame rate version, at least I have never seen a 60 fps video actually be smaller then its 30 fps version if both were processed in the same way.

A better comparison would be to set QTGMC to output 30 fps video instead of using Smart Deinterlace and compress the result with the same x264 settings used for the 60 fps encode. The 60 fps will probably be larger, though much less than double the size for all the reasons already mentioned.

I have seen the output of QTGMC compress to half the size required for an unprocessed version with the same x264 settings. I am also doing encodes from VHS sources.

[benwaggoner]

Normally higher fps means higher bitrate (scales by about 40-50% more for every doubling, so efficiency increases), but only for the same video. If they're two different videos, then you'd never expect them to come out to the same bitrate no matter what their framerates are.

Bear in mind that with a 60p source with typical motion, a 30p encode will have twice as much temporal change between frames as a 60p encode of the same source. So even at the same QP, PSNR, SSIM, whatever, we'd expect doubling the frame rate to less than double the bitrate.

This effect will be stronger with smoother content, and smaller with grainy/noisy content.

[hello_hello]

I've compared Yadif de-interlacing with QTGMC lots of times, in respect to file size. Yadif at 25fps tends to produce larger file sizes for the same CRF value than QTGMC at 50fps. Yadif at 50fps would be a little larger again. QTGMC makes the video easier to compress.

I assume the source was interlaced and being de-interlaced when it was captured? If so, what's the quality of the de-interlacing like?

I still have some old comparison encodes sitting on my hard drive from back when I first started playing with QTGMC. 10 minutes of PAL DVD video, resized to 640x480.

Yadif 25fps 1738kbps
Yadif 50fps 1921kbps
QTGMC 25fps 1539kbps
QTGMC 50fps 1660kbps

Wait till you discover using QTGMC for cleaning progressive video. Something like

QTGMC(InputType=1, Ezdenoise=1)

It tends to make the video way easier to compress too, but it's kind of disappointing when you feel a need to run QTGMC on almost everything, given how slow it is. :)