crf and 8x8dct

[PhilR]

I encoded the first 10,000 frames of The Matrix at a range of crf settings both with and without 8x8dct. My specific settings are at the end of this post.

At crf 20, 8x8dct resulted in smaller files, but at crf 21 and up it resulted in bigger files. My expectation had been that there would either be no disadvantage to enabling 8x8dct or, at the very worst, a small penalty if bits had to be spent signaling that it wasn't being used. At crf 24, though, the file was 6% larger with than without.

Is this expected?

crf with without % change from enabling
20 172267108 175441379 -1.81
21 143936753 143302417 0.44
22 118643848 115593620 2.64
23 96456703 92349353 4.45
24 77615229 73064606 6.23

Version:
$ x264 --version
x264 0.65.999M cc51047

I've also added hrd_pulldown.09_interlace.diff, y4m_input.patch, and the version of rate_estimate_qscale from x264-devel on October 22nd.

Base settings in addition to variable crf and optional 8xdct. (Some of these settings are required by a hardware platform I'm targetting): -I 30 --aud --level 3 --vbv-bufsize 8000 --vbv-maxrate 10000 --ref 5 --nal-hrd --trellis 2 --psy-rd 1.0:1.0 --bframes 3 --b-adapt 2 --weightb --direct auto --partitions all --mixed-refs --subme 9 --me umh

[LoRd_MuldeR]

You can't judge a CRF encode by size only! If it became 6% bigger, then the question is: Was 6% increase in size worth the improvement in quality ???
So in fact you have to judge "quality per size", anything else is useless. And the only proper way to do this would be a 2-Pass encode!

If you want to find out whether 8x8dct is a good or a bad thing, make two 2-Pass encodes of the same size. One with 8x8dct and one without it.
Then compare the quality of both encodes and decide...

[PhilR]

What is the point of CRF, then?

[PhilR]

Or, more precisely (and less unintentionally inflammatory-sounding), what is it that CRF is trying to hold constant?

[wyti]

the quality, but it's not exactly constant, it may be a little better or worse depending on your settings (and the source too)

[LoRd_MuldeR]

Settings in x264 are tuned in a way that "slower" settings will improve quality as well as compression.
In 2-Pass mode with fixed target filesize "better compression" equals "better quality", so using slower settings will only give better quality.

But in CRF mode it's not that easy. Using slower settings in CRF mode will always improve the "quality per size" ratio.
However the total size of the CRF encode may become either bigger or smaller...

Conclusion: Don't judge a CRF encode solely by it's size! The only valid way to test "8x8dct" is a quality comparison in 2-Pass mode!
The same applies to Trellis and Co ;)

[Disabled]

I wonder if there is an explanation how crf is doing what its doing? I searched about it a little while ago, but didn't find anything.
Is the only thing crf does, change the quantiser based on the frame complexity (compared to const quant)?

[LoRd_MuldeR]

I wonder if there is an explanation how crf is doing what its doing? I searched about it a little while ago, but didn't find anything.
Is the only thing crf does, change the quantiser based on the frame complexity (compared to const quant)?

Read this:
http://git.videolan.org/gitweb.cgi?p=x264.git;a=blob_plain;f=doc/ratecontrol.txt

Documentation from the x264 developer. The last statement for "all modes" isn't true anymore, thanks to VAQ.

[Sagekilla]

Also, VAQ doesn't apply to CQ (constant quantizer) by pure virtue of what CQ does... constant QP for every frame, which means no AQ can take place.

[LoRd_MuldeR]

Also, VAQ doesn't apply to CQ (constant quantizer) by pure virtue of what CQ does... constant QP for every frame, which means no AQ can take place.

Therefore I said the statement isn't true anymore for "all modes". For CQP mode it's stilll true, of course...

[Disabled]

Read this:
http://git.videolan.org/gitweb.cgi?p=x264.git;a=blob_plain;f=doc/ratecontrol.txt
Thanks a lot, that pretty much explains what I wanted to know.
Two questions though:
If all my frames in the source would be of "average complexity" (in the sense of the 1 pass estimation alogrithm), would a crf x encode give "about the same" result as an cq x encode? (Assuming the frame decision in cq mode is the same as in crf mode... and probably a lot more thats not mentioned there.)
Is it possible to write a stats file during an crf encode to use with a second pass? I usually do crf encodes, but if it just takes a little less space to fit on DVD I would like to do a second pass to do so.

[LoRd_MuldeR]

If all my frames in the source would be of "average complexity" (in the sense of the 1 pass estimation alogrithm), would a crf x encode give "about the same" result as an cq x encode? (Assuming the frame decision in cq mode is the same as in crf mode... and probably a lot more thats not mentioned there.)

It's extremely unlikely that you have a source where all frames have the same complexity. Don't think such thing exists in reality.

Also there's another difference: CRF uses adaptive quantization now, while CQP assigns the same quantizer to all macroblocks in the frame.

Is it possible to write a stats file during an crf encode to use with a second pass?

Yup.

[Disabled]

It's extremely unlikely that you have a source where all frames have the same complexity.
It was more a theoretical question to get a feeling how it behaves.

Also there's another difference: CRF uses adaptive quantization now, while CQP assigns the same quantizer to all macroblocks in the frame.
Ah ok, didn't think about that.
Yup.
Just via --stats filename? Youre sure? I just tried it and didn't find a stats file...

[LoRd_MuldeR]

Just via --stats filename? Youre sure? I just tried it and didn't find a stats file...

I think it should work like this:
x264.exe --crf 22 --pass 1 --stats foobar.stat --output NUL "C:\Input.avs"