Encoding CRF - Same file size with L4.1 (5 ref frames)/L5.1 (16 ref frames)

[Furiousflea]

Hi there,

Simple question,

I'm encoding a film from Bluray source, cropped to it's proper aspect ratio 1920x816, that's rather grainy at L4.1 with...

--level 4.1 --preset veryslow --tune film --aud --nal-hrd vbr --b-pyramid normal

This is how I always encode my stuff for storing on my NAS using CRF encoding.

I thought I'd give L5.1 a try since there's no reason not to except longer encoding time and my hardware on my XBMC setup is up to the job, using same profile except L5.1.

--level 5.1 --preset veryslow --tune film --aud --nal-hrd vbr --b-pyramid normal

In both cases the resulting output was more or less exactly the same size for a 2 hour movie, the L5.1 encode was about 20MB bigger, which is less than 0.1%.

When looking at the resulting properties of the 2 files, the L4.1 one showed as using 5 B-Frames, the L5.1 one showed as using 16 B-Frames.

Visually, I can't distinguish between the 2, they were both set to CRF16.5 for quality target.

Would it be silly to use L5.1 going forward, or was this simply a case of bad returns due to the amount of grain present, and hence B-Frames not being very useful?

Thanks.

[J_Darnley]

Very slow only uses 8 B-frames. Are you getting B-frames confused with ref-frames? You can see how "useful" the larger number of refs were when the encoding is finished.

[Furiousflea]

Very slow only uses 8 B-frames. Are you getting B-frames confused with ref-frames? You can see how "useful" the larger number of refs were when the encoding is finished.

Sorry I'm a fool,

Yes I mean ref frames. mediainfo is reporting that the video is using 16 ref frames for the L5.1 profile and 5 ref frames for the L4.1 profile, all other settings being equal.

Both finished encodes are basically exactly the same size.

[sneaker_ger]

Visually, I can't distinguish between the 2

This is basically your answer: since they came out visually the same, you should save time by going for level 4.1. Most people agree that 16 ref frames barely have any advantage on most real life content anyways. This also has the side effect of much better hardware compatibility in case you want to play it on something else than your current setup.

You can also cut the other options:
1.) --aud
2.) --nal-hrd vbr
3.) --b-pyramid normal

1.) Why would you need this?
2.) Why would you need this? Also: it doesn't do anything without vbv parameters anyways
3.) this is the default anyways

[Furiousflea]

Thanks for the advice.

OK that's my inquisitiveness inquisitivized...

In regards to those other switches being set, well I just took the default ripbot264 settings and removed the vbv/maxrate bit because there is no bandwidth limitation, and read that with CRF encoding, they shouldn't be used.

I know the author of Ripbot264 is quite vocal in defending his alleged ideal settings, so I won't go there :)

Thanks again for both of your times.

[sneaker_ger]

I just took the default ripbot264 settings and removed the vbv/maxrate bit because there is no bandwidth limitation, and read that with CRF encoding, they shouldn't be used.

vbv settings are in no way depending on the CRF/n-pass ABR mode, but only on the playback device and reading/streaming bandwidth.

[TheRyuu]

Most people agree that 16 ref frames barely have any advantage on most real life content anyways.

You're really only going to start to see an advantage with 16 references frames on stuff like anime or other types with more static content in which case they can actually become more important than bframes.

[LoRd_MuldeR]

Yes I mean ref frames. mediainfo is reporting that the video is using 16 ref frames for the L5.1 profile and 5 ref frames for the L4.1 profile, all other settings being equal

What does "is using 16 ref frames" mean ???

In fact it means that 16 decoded frames will be kept in the decoder's DPB (Decoded Picture Buffer) and thus potentially could be used as references for the current frame.

It certainly does not mean that every single frame in your encoded stream references to all the 16 frames in the DPB to which it theoretically could reference!

It's still up to the encoder to decide which frames are used as references in order to get the most efficient coding.

Consequently "using" more reference frames (--ref option) only means that the encoder will have more frames to choose from - which makes encoding slower but not necessarily better ;)

Example:

If you allow up to 8 ref frames, but in 99% of all cases the encoder uses "only" the last 5 frames (which is not unusual!), going from 8 to 16 refs probably makes no difference at all...

[hello_hello]

Is it possible to see reference frame usage my looking at the x264 log file. I think I understand the consecutive b-frames entry, but not the ref ones.

For example here's the result of 2 quick encodes, the first using b-frames 3, ref frames 5.

-[NoImage] x264 [info]: frame I:133 Avg QP:13.79 size: 30716
-[NoImage] x264 [info]: frame P:2022 Avg QP:16.99 size: 8291
-[NoImage] x264 [info]: frame B:3592 Avg QP:21.59 size: 2768
-[NoImage] x264 [info]: consecutive B-frames: 3.5% 38.5% 3.4% 54.6%
-[NoImage] x264 [info]: mb I I16..4: 19.1% 78.6% 2.3%
-[NoImage] x264 [info]: mb P I16..4: 4.3% 12.2% 0.5% P16..4: 34.6% 15.5% 10.9% 0.0% 0.0% skip:22.0%
-[NoImage] x264 [info]: mb B I16..4: 0.3% 2.4% 0.1% B16..8: 31.8% 5.5% 1.3% direct: 5.0% skip:53.6% L0:33.7% L1:41.4% BI:24.9%
-[NoImage] x264 [info]: 8x8 transform intra:75.9% inter:82.3%
-[NoImage] x264 [info]: direct mvs spatial:99.8% temporal:0.2%
-[NoImage] x264 [info]: coded y,uvDC,uvAC intra: 66.4% 68.4% 37.7% inter: 19.9% 20.4% 3.1%
-[NoImage] x264 [info]: i16 v,h,dc,p: 72% 13% 10% 5%
-[NoImage] x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 21% 24% 28% 3% 4% 4% 5% 5% 6%
-[NoImage] x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 18% 20% 11% 5% 10% 9% 11% 7% 8%
-[NoImage] x264 [info]: i8c dc,h,v,p: 42% 26% 22% 9%
-[NoImage] x264 [info]: Weighted P-Frames: Y:4.0% UV:2.0%
-[NoImage] x264 [info]: ref P L0: 71.8% 11.5% 11.0% 3.3% 2.3% 0.1% 0.0%
-[NoImage] x264 [info]: ref B L0: 91.7% 6.2% 1.7% 0.5%
-[NoImage] x264 [info]: ref B L1: 98.2% 1.8%
-[NoImage] x264 [info]: kb/s:1071.59
-[NoImage] encoded 5747 frames, 73.68 fps, 1071.62 kb/s

Same encode, this time with ref frames reduced to 3.

-[NoImage] x264 [info]: frame I:133 Avg QP:13.79 size: 30716
-[NoImage] x264 [info]: frame P:2022 Avg QP:16.98 size: 8290
-[NoImage] x264 [info]: frame B:3592 Avg QP:21.57 size: 2764
-[NoImage] x264 [info]: consecutive B-frames: 3.5% 38.5% 3.4% 54.6%
-[NoImage] x264 [info]: mb I I16..4: 19.1% 78.6% 2.3%
-[NoImage] x264 [info]: mb P I16..4: 4.4% 12.3% 0.5% P16..4: 34.5% 15.5% 10.8% 0.0% 0.0% skip:22.0%
-[NoImage] x264 [info]: mb B I16..4: 0.3% 2.4% 0.1% B16..8: 31.9% 5.5% 1.3% direct: 5.0% skip:53.6% L0:33.4% L1:41.3% BI:25.3%
-[NoImage] x264 [info]: 8x8 transform intra:75.8% inter:82.4%
-[NoImage] x264 [info]: direct mvs spatial:99.8% temporal:0.2%
-[NoImage] x264 [info]: coded y,uvDC,uvAC intra: 66.3% 68.4% 37.8% inter: 19.9% 20.3% 3.1%
-[NoImage] x264 [info]: i16 v,h,dc,p: 72% 13% 10% 5%
-[NoImage] x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 21% 24% 28% 3% 4% 4% 5% 5% 6%
-[NoImage] x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 18% 20% 11% 6% 10% 9% 11% 7% 8%
-[NoImage] x264 [info]: i8c dc,h,v,p: 42% 26% 22% 9%
-[NoImage] x264 [info]: Weighted P-Frames: Y:4.0% UV:2.0%
-[NoImage] x264 [info]: ref P L0: 73.0% 11.6% 11.5% 3.7% 0.1%
-[NoImage] x264 [info]: ref B L0: 93.3% 5.7% 1.0%
-[NoImage] x264 [info]: ref B L1: 98.1% 1.9%
-[NoImage] x264 [info]: kb/s:1070.99
-[NoImage] encoded 5747 frames, 80.96 fps, 1071.02 kb/s

The main difference between the log files is the lines highlighted in blue, but I'm not sure how to interpret them.

[LoRd_MuldeR]

It shows you how often the available reference frames have actually been used (chosen), separately for P- and B-frames.

I'm not quite sure why you got 7 and 5 values in the "ref P L0" line, because I would have expected 5 and 3 values here with 5 and 3 ref frames. So I assume you used 5 and 7.

Anyway, it shows, as expected, that the "first" few reference frames have been used most of the time while the "latter" ones have rarely been used.

Even with 5 ref frames (second log), in 96% of all cases the first three ref frames have been used. That doesn't change much when going to 7 ref frame (first log). The 7-th ref frames even was used in less than 0.05% of all cases, while the first three refs are still used in 94.3% of all cases...

Conclusion: Going from 5 to 7 ref frames probably didn't make a noteworthy difference for the output. But it made encoding somewhat slower (and increased the decoder requirements). Also, for this particular clip, you could have gotten away with 3 ref frames (maybe 4) without much loss. Anything above 4 is used only in homeopathic cases anyway.

(With the B-Frames it's pretty much the same, except that we have a "L0" and "L1" line here, because the B-Pyramid feature was enabled)

[MasterNobody]

I'm not quite sure why you got 7 and 5 values in the "ref P L0" line, because I would have expected 5 and 3 values here with 5 and 3 ref frames. So I assume you used 5 and 7.
You should take into account --weigthp 2 gotcha. --weightp 2 adds 2 more refs (dups) into P-refs and --weightp 1 adds 1 more ref (dup). So IIRC first 3 refs are all refs to directly previous frames (original, smart weight, blind weight -1) in --weightp 2 case.

[LoRd_MuldeR]

You should take into account --weigthp 2 gotcha. --weightp 2 adds 2 more refs (dups) into P-refs and --weightp 1 adds 1 more ref (dup). So IIRC first 3 refs are all refs to directly previous frames (original, smart weight, blind weight -1) in --weightp 2 case.

Thanks for clearing that up :)

[hello_hello]

Ah, yes..... it's all clear to me now..... :)

I might have to wait for the brain to create a few neural pathways dedicated to understanding the way --weigthp settings effect the P-refs values being displayed in log files and come back and read it again, but I think I get the gist if it.