Using crf, followed by a two-pass ABR using the bitrate

[dyscoria]

I've so far been using a single pass crf=17 very slow encode as I am a believer in a collection with constant quality throughout. However, as encode-time is not a big issue and my hard drives are filling rather quickly, i'm probably going to start trying to get lower bitrates, but maintain constant quality using this method:

Single pass crf=25.5 average speed encode, then use that bitrate to do a very slow two pass encode.

I've heard this method mentioned a few times (and i happen to be a big fan of crf), and now i'm interested in the habits of people that actually currently do it. Do you use a low crf followed by a percentage of that bitrate or a high crf and exactly that bitrate? Do you make the very first crf pass as quick as possible?

Many thanks.

[fibbingbear]

I asked this question when I first came to doom9, and the answer is: it doesn't matter. The differences are negligible between 2-pass and CRF (if the bitrate is the same for both).

Here's the thread, if it helps: http://forum.doom9.org/showthread.php?p=1097255#post1097255

[dyscoria]

Thanks for the link (and apologies to moderators for not finding that thread :P). This will save quite a lot of encoding time! Guess i'll just increase the crf value until i'm happy.

[fibbingbear]

Just remember --- CRF 18 is generally "transparent" (can't tell the difference between the original and the encoded version unless you really try hard). DeathTheSheep has stated (in one of his guides) that anything under 15 is probably overkill. Animated material might be able to go with a higher CRF than live action. Values between 20-24 seem to be popular (although I think most people agree going above 22 results in noticeable degradation).

Sharktooth has mentioned that for lower CRF values, it's better to disable some of the compression techniques (like using an extreme value for trellis), since those might cause artifacts (even with a higher bitrate)... look at his CRF_18 profile to get an idea. If you're going to higher CRFs, you might want to start enabling some of those compression options... not that I'm an expert on this subject.

[dyscoria]

crf=17 that i'm currently using will certainly have to be increased then (if 18 is considered transparent). I'm thinking crf=20.4 (60%) would be a nice value that will cut filesize down a bit without too much difference from DVD quality, but i'm going to go do some visual tests and see how keen my eyes really are. Might be able to make do with an even higher crf.

Thanks again for your help :)

[fibbingbear]

but i'm going to go do some visual tests and see how keen my eyes really are.

That's the ultimate way to figure out your best compression rate :) Don't forget that CRF_18 works for most people, but not for everyone.

On a side note, you might want to look into SSIM and other computational mechanisms to determine how "close" the encoded version is to the original. Anything lower than 80% similarity is going to be terrible, around 95% to 97% is when things start becoming transparent (again, differs from person to person, this is just a general guideline).

[ggab]

dyscoria, have a look at this thread too :)

need an advice with CRF encoding - Doom9's Forum
http://forum.doom9.org/showthread.php?p=1107795


you are welcome to post there ;)

[qyqgpower]

patched785

crf16 merange16 ref5

umh trellis1
-[NoImage] x264 [info]: slice I:9 Avg QP:11.65 size: 16477
-[NoImage] x264 [info]: slice P:179 Avg QP:13.56 size: 15884
-[NoImage] x264 [info]: slice B:313 Avg QP:15.71 size: 2330
-[NoImage] x264 [info]: SSIM Mean Y:0.9923041
-[NoImage] encoded 501 frames, 14.34 fps, 1425.82 kb/s

tesa trellis1
-[NoImage] x264 [info]: slice I:8 Avg QP:11.78 size: 17201
-[NoImage] x264 [info]: slice P:181 Avg QP:13.55 size: 15515
-[NoImage] x264 [info]: slice B:312 Avg QP:15.81 size: 2311
-[NoImage] x264 [info]: SSIM Mean Y:0.9923361
-[NoImage] encoded 501 frames, 7.86 fps, 1405.11 kb/s

tesa trellis2
-[NoImage] x264 [info]: slice I:8 Avg QP:11.78 size: 16352
-[NoImage] x264 [info]: slice P:181 Avg QP:13.55 size: 14772
-[NoImage] x264 [info]: slice B:312 Avg QP:15.99 size: 2077
-[NoImage] x264 [info]: SSIM Mean Y:0.9924763
-[NoImage] encoded 501 frames, 6.11 fps, 1323.04 kb/s

even at a low crf value, using trellis2 leads to a much better compression, and a slight improvement in SSIM. So I doubt sharktooth's conclusion.

[fibbingbear]

So I doubt sharktooth's conclusion.

Please note that it's much more likely that I'm misquoting Sharktooth :confused: He usually has excellent advice.

I can't find the original post where Sharktooth talked about it, but he gave a very convincing argument for why you don't want to max out your compression options for low CRF. Instead of trellis, maybe he wrote about subpelrefinement?

Basically, he talked about how if the optimizations are set too aggressively, x264 may try to heavily compress an area, even if it has plenty of bitrate to spare, and thus result in lower quality (the heavily compressed area will take up less space but may contain compression artifacts).

Look at his CRF_18 profile to see what he really endorses :p

[Dark Shikari]

I know there are some options for x264 where you can crank it up to full (I think 6 for subpelrefinement) and it'll be worse for low values of CRF than something faster (like 4 or 5 for subpelrefinement).I find this doubtful :p

Perhaps trellis; probably not subme.

[fibbingbear]

I find this doubtful :p

Perhaps trellis; probably not subme.

Man, I am just tripping up on my own words today :confused: :confused: :confused:

I tried to search for the thread where Sharktooth talks about it, but couldn't find it. All I could find was this esoteric entry in my notes about compression:

"You cant simply use RDO and expect to be able to keep grain... (with x264, from sharktooth's notes)"

With that, I'll stop talking... I'm clearly not an expert in this area, and mis-remembering what other people have written is not helping


Edit: found the post! Google search FTW:

http://forum.doom9.org/showthread.php?p=953771#post953771

[Dark Shikari]

"You cant simply use RDO and expect to be able to keep grain... (with x264, from sharktooth's notes)"

With that, I'll stop talking... I'm clearly not an expert in this area, and mis-remembering what other people have written is not helpingMore likely, Sharktooth is just wrong; RDO does not inherently stop grain retention. I have dozens of perfectly grainy movies on my hard disk as proof...

[dyscoria]

I have a feeling that I should be putting as much thought into this as you guys :D

I'll probably go for a crf in the low 20's, and will probably not go for any form of trellis. From what i've read, it seems that trellis is much more useful for a multipass encode. The use of trellis doesn't inspire confidence in me so I think if I need a smaller file size I could just raise the crf value. Maybe I should go test trellis=1 (trellis=2 seems too agressive for my needs). I've only just worked out how to display the SSIM (log=3) so off I go to do some objective tests. It's doubtful that i'll be able to visually see the difference between no trellis and with trellis. Guess i'll post up some form of results when i'm done.

[fibbingbear]

I'll probably go for a crf in the low 20's, and will probably not go for any form of trellis.

Okay, I know I said I won't post again, but I don't want to be misleading either.

From what I've read, having some form of trellis is going to be beneficial --- the argument is over what value it should be.

[dyscoria]

Okay, I know I said I won't post again, but I don't want to be misleading either.

From what I've read, having some form of trellis is going to be beneficial --- the argument is over what value it should be.

Hmm ok thanks for the clarification. I'll definately try out trelllis in that case and see what effects the values have.

What I don't understand though is from what qyqgpower posted up. How can trellis improve compression _and_ increase the SSIM value? Shouldn't it just keep the same SSIM but improve compression?

[Dark Shikari]

Hmm ok thanks for the clarification. I'll definately try out trelllis in that case and see what effects the values have.

What I don't understand though is from what qyqgpower posted up. How can trellis improve compression _and_ increase the SSIM value? Shouldn't it just keep the same SSIM but improve compression?You can improve quality and reduce bitrate at the same time.