Comparing results from different x264 versions

[junglemike]

Hi All.
I've been using x264 core 68 r1195 version to encode all my camera videos. I know it's an old version but the results were good enough for me and I just didn't change anything.

Recently I decided to try x264 core 102 r1666 version.
I used the same source file and same options which are:
for %f in (*.avs) do start "encode" /b /low /wait c:\x264.exe --profile high --preset slower \
--crf 23 --output "%f_crf=23.mkv" "%f"

The difference in size was more than I expected. "old" x264 produced average bitrate of 499kbps , while new x264 did 388 kbps. visually both files were very close, and I couldn't tell which one's better.

Hence the question: IIRC crf mode is supposed to represent more or less constant quality, in my case with quality parameter crf 23. Maybe crf=23 means different quality levels in old and new versions?
For example crf=23 in old version would be equal(quality-wise) to ,say, crf=24.5 in new version.

thanks.

[Dark Shikari]

CRF has changed its meaning a couple times in x264's history. It's not a constant offset change though -- if it was, we would compensate for it. Rather, the algorithm itself changes, which means that some things may get more bitrate and some less, so there's no single offset one can apply to fix it.

This may happen in the future as well.

[junglemike]

Thanks for quick reply. I understand what you said, but i'm still confused.
You said crf's meaning was changed. So does --crf x in new builds mean different quality than --crf x in old version? or does it "try" to produce similar quality?

I understand that a lot of things changed from old version, like bitrate distribution, etc.
But the difference between these 2 versions seem to be too good to be true. I know that you guys constantly work on x264 and there are new builds daily.
But 30% reduction in size for same quality is too unbelievable. Is it really possible that it got that much better?
Or maybe it's a combination of the two? I.E. new x264 got "somewhat" better , but also --crf 23 in new version aims at lower quality than the old version did, and hence the huge difference in size?

[Dark Shikari]

Thanks for quick reply. I understand what you said, but i'm still confused.
You said crf's meaning was changed. So does --crf x in new builds mean different quality than --crf x in old version? or does it "try" to produce similar quality?

I understand that a lot of things changed from old version, like bitrate distribution, etc.
But the difference between these 2 versions seem to be too good to be true. I know that you guys constantly work on x264 and there are new builds daily.
But 30% reduction in size for same quality is too unbelievable. Is it really possible that it got that much better? Yes, MB-tree, psy-RD, and all these other features can help that much.Or maybe it's a combination of the two? I.E. new x264 got "somewhat" better , but also --crf 23 in new version aims at lower quality than the old version did, and hence the huge difference in size?Yes, that too.

But again, when the definition of CRF changes, it's not guaranteed to give the same quality in all cases as the old definition.

Suppose we have three videos, A, B, and C. We add a new feature to x264 that does the following:

Video A: 10% higher bitrate, 20% higher quality
Video B: 20% lower bitrate, 10% lower quality
Video C: 5% lower bitrate, 5% higher quality

Obviously, this new feature is a big win in every sense of the word. But there's no magical way to make an x264 with this new feature "agree" quality-wise with the old x264 on all three videos at the same time.

[junglemike]

Ok. It clears up my confusion quite a bit.
Looks like i'll stick with the new version - that's for sure :-)

[kypec]

But again, when the definition of CRF changes, it's not guaranteed to give the same quality in all cases as the old definition.

Suppose we have three videos, A, B, and C. We add a new feature to x264 that does the following:

Video A: 10% higher bitrate, 20% higher quality
Video B: 20% lower bitrate, 10% lower quality
Video C: 5% lower bitrate, 5% higher quality

@DS - you keep using a term "X% higher/lower quality" in all your explanations yet it's been stated many times that there is no simple objective metric parameter which could be used to compare a quality of encoded video objectively (namely SSIM & PSNR are out of the question obviously). How should I interpret those percent numbers of yours then? :confused:

[Dark Shikari]

@DS - you keep using a term "X% higher/lower quality" in all your explanations yet it's been stated many times that there is no simple objective metric parameter which could be used to compare a quality of encoded video objectively (namely SSIM & PSNR are out of the question obviously).SSIM and PSNR are perfectly valid metrics as long as everything else remains orthogonal to them. That is, it's perfectly reasonable, particularly when not working with psy optimizations, for a PSNR increase of 5% to be correlated highly with an "actual quality increase" of 5%.

They only begin to disconnect when dealing with psy optimizations. Usually.

[Sharktooth]

uhm... usually... but not exclusively. a further visual check is always needed.

[MasterNobody]

kypec
You need to understand difference between using % for simple explanation and real use of objective metrics for quality measurement.