x264 CRF Filesize

[Puncakes]

Hi, more recently I've been having problems getting a reasonable filesize with CRF. I understand that the implementation of mb-tree "redefines" CRF based on the source, but I'm getting filesizes of 50-60MB on unfiltered encodes straight from the DVDs at CRF 18. Turning off mb-tree does not solve the problem either. This obviously provides pretty low quality, but I have to set it to CRF 10 to get close to 200MB, and even then, the quality seems lower than it should be. I suppose it should be noted that my source is an episode of animation. I'd just like to know if this is normal behavior or if there's something I've done wrong.

I'm using build 1251 and by x264 settings (taken from megui log) are:
--profile high --level 4.1 --preset fast --crf 18.0 --thread-input --deblock 1:1 --bframes 16 --b-adapt 2 --b-pyramid --direct auto --b-bias 0 --scenecut 40 --ref 16 --rc-lookahead 80 --aq-mode 1 --aq-strength 0.6 --merange 32 --me umh --subme 10 --partitions all --trellis 2 --psy-rd 0.20:0 --no-dct-decimate --no-fast-pskip

Help, suggestions, ideas, etc. are all greatly appreciated.

[Dark Shikari]

Quality lower than it should be... at CRF 10?

Sample input, output, anything? :confused: We're not psychic here...

[Sagekilla]

I mean sure, 50 MB might make sense if this straight-from-DVD was a short clip (Like 30 seconds?)

But unless you give us an unprocessed sample, there's little we can do to help.

Find a small clip (~30 seconds) that you can reproduce this small file size issue on and upload it to mediafire. That should help a lot more with figuring this out.

[Puncakes]

Oops, my apologies for the late reply, and for not posting a sample. I just didn't think it could be source-specific since it's the case for more than one DVD.

In any case, I've uploaded a 32 second sample here: http://www.mediafire.com/?4wu2mwzer2y

When I encode that with the settings I posted above, I get a ~500KB file.

[Dark Shikari]

I just encoded your clip at --crf 15 --preset fast (after IVTCing); it came out at 532kbps (2 megabytes total), which doesn't seem unreasonable given how clean the source is. The source is by and large capped out by the default qpmin of 10, which is preventing it from reaching a higher bitrate.

I'm not sure what the issue is...?

[Puncakes]

Well, I've just tested it on a 3rd DVD, with the same results. If it helps, I'm using megui set to update with kurtnoise's server. I suppose I could try rolling back to earlier versions of things.

I also tried encoding to lossless and going from there, but that didn't help either...

Edit: Well, after reinstalling megui, and using crf 15 + const. qual fast, which should be the same as --crf 15 --preset fast, unless megui is hiding something from me, I got the same results as you.

Apparently the problem is in my settings after all. Now I just need to figure out where.

Edit2: Looks like I jumped the gun. I got similar results on the sample, but moving back to the entire episodes brought the problem back with it. Using the const. qual fast preset, I'm getting around 60MB for one DVD, and 700MB for another. The other is significantly more grainy, but I don't think it's 640MB more grainy...

*ahem*Edit3: I've uploaded larger samples for the one that appears to be problematic (60MB), and the one that appears to be encoding properly (700MB) here (http://www.mediafire.com/?emdkuuizojm) and here (http://www.mediafire.com/?nmwzmmkuoji) respectively.

Edit4:
Sample logs:
[NoImage] Standard error stream
-[NoImage] avis [info]: 720x480 @ 23.98 fps (1343 frames)
-[NoImage] x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
-[NoImage] x264 [info]: profile High, level 3.0
-[NoImage]
-[NoImage] x264 [info]: frame I:26 Avg QP:13.38 size: 12805
-[NoImage] x264 [info]: frame P:440 Avg QP:16.04 size: 3809
-[NoImage] x264 [info]: frame B:877 Avg QP:18.04 size: 840
-[NoImage] x264 [info]: consecutive B-frames: 6.2% 12.3% 8.7% 72.9%
-[NoImage] x264 [info]: mb I I16..4: 37.9% 46.4% 15.7%
-[NoImage] x264 [info]: mb P I16..4: 3.6% 5.6% 1.5% P16..4: 32.3% 7.8% 5.6% 0.0% 0.0% skip:43.7%
-[NoImage] x264 [info]: mb B I16..4: 1.6% 1.5% 0.2% B16..8: 9.4% 0.7% 0.3% direct: 3.8% skip:82.5% L0:51.9% L1:44.6% BI: 3.5%
-[NoImage] x264 [info]: 8x8 transform intra:49.1% inter:71.0%
-[NoImage] x264 [info]: coded y,uvDC,uvAC intra:40.1% 51.4% 18.7% inter:5.4% 12.3% 0.3%
-[NoImage] x264 [info]: ref P L0 77.0% 23.0%
-[NoImage] x264 [info]: kb/s:392.1
-[NoImage] encoded 1343 frames, 51.86 fps, 392.24 kb/s

[NoImage] Standard error stream
-[NoImage] avis [info]: 720x480 @ 29.97 fps (1784 frames)
-[NoImage] x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
-[NoImage] x264 [info]: profile High, level 3.0
-[NoImage]
-[NoImage] x264 [info]: frame I:29 Avg QP:16.22 size: 43622
-[NoImage] x264 [info]: frame P:1240 Avg QP:20.18 size: 19786
-[NoImage] x264 [info]: frame B:515 Avg QP:21.32 size: 5997
-[NoImage] x264 [info]: consecutive B-frames: 47.7% 35.9% 10.4% 5.9%
-[NoImage] x264 [info]: mb I I16..4: 22.9% 63.3% 13.8%
-[NoImage] x264 [info]: mb P I16..4: 6.2% 21.2% 4.3% P16..4: 33.7% 12.0% 9.7% 0.0% 0.0% skip:12.8%
-[NoImage] x264 [info]: mb B I16..4: 3.5% 5.0% 0.9% B16..8: 30.1% 2.2% 1.7% direct:15.1% skip:41.4% L0:34.4% L1:42.1% BI:23.5%
-[NoImage] x264 [info]: 8x8 transform intra:65.2% inter:68.8%
-[NoImage] x264 [info]: coded y,uvDC,uvAC intra:61.9% 88.1% 63.5% inter:33.2% 51.6% 16.6%
-[NoImage] x264 [info]: ref P L0 79.5% 20.5%
-[NoImage] x264 [info]: kb/s:3882.5
-[NoImage] encoded 1784 frames, 27.31 fps, 3882.68 kb/s

- Again, respectively.

[Puncakes]

I suppose all that editing wasn't going to bring any attention to the thread, and in any case, I think I need to clarify a bit

I'm not sure what the issue is...?

The problem is that, while you say the size sounds right, the visual quality is significantly lacking, even at crf 15.

Source: http://i26.tinypic.com/2qlv1hu.jpg
--crf 15 --preset fast: http://i31.tinypic.com/69peo1.jpg

[Firebird]

Use 2 pass mode then... And crf 15 looks pretty good by the way.

[Dark Shikari]

Source: http://i26.tinypic.com/2qlv1hu.jpg
--crf 15 --preset fast: http://i31.tinypic.com/69peo1.jpgI can't even tell the difference between those two images...

... again, like I said before, but apparently you didn't bother reading, the "problem" is that your encode is capped out by qpmin.

[creamyhorror]

I can't even tell the difference between those two images...
The differences are there, but minor. The sign in the top-left, for example.


... again, like I said before, but apparently you didn't bother reading, the "problem" is that your encode is capped out by qpmin.
Hmm, does this imply that some anime sources are going to need lowered qpmin? Or is it uncommon that they do?

[froggy1]

I can't see the difference in the "sign in the top left"

[Puncakes]

I can't even tell the difference between those two images...

Aside from the obvious increased banding, look at the blocking in the background buildings, or the blurring of the windows on the closest bulding on the left. Parts of the fence slightly right of the center are also being ignored. It's clear that the differences are subtle when at native resolution but when going fullscreen for actual viewing, those small differences become large.

... again, like I said before, but apparently you didn't bother reading, the "problem" is that your encode is capped out by qpmin.

I read, and understood what you wrote, but, what I don't understand is why the qpmin is capping out while the quality is still abnormally low. Even if I throw in an addgrainc(5,5), it doesn't retain barely any of it at crf 15, and all I'm left with is a fluctuating, noisy image.

[nakTT]

I suppose all that editing wasn't going to bring any attention to the thread, and in any case, I think I need to clarify a bit



The problem is that, while you say the size sounds right, the visual quality is significantly lacking, even at crf 15.

Source: http://i26.tinypic.com/2qlv1hu.jpg
--crf 15 --preset fast: http://i31.tinypic.com/69peo1.jpg
To be honest, I failed to see the different between the source and the output pics that you gave. All I can say is that it was a very good encode indeed since both are lookalike. Anyway, at least to my eyes.

[Dark Shikari]

I read, and understood what you wrote, but, what I don't understand is why the qpmin is capping out while the quality is still abnormally low. Even if I throw in an addgrainc(5,5), it doesn't retain barely any of it at crf 15, and all I'm left with is a fluctuating, noisy image.Adding grain works great here, and since you refuse to provide an output sample, I'm going to drop this as "worksforme".

But thanks for providing a sample for which I found significant inefficiencies in ultra-low quantizer encoding--expect an improvement in bitrate of up to ~5-10% at low CRF values on this clip in a few days ;)

[creamyhorror]

I can't see the difference in the "sign in the top left"
It's very minimal, you have to zoom in to see it, and now on my laptop the difference is even smaller.

I read, and understood what you wrote, but, what I don't understand is why the qpmin is capping out while the quality is still abnormally low.
The point (as I understand it) is that the default qpmin of 10 is too high for your source, so you should reduce it to allow for even higher quality at the expense of a higher bitrate.

I'm wondering if psy-rd and AQ are playing into this. Would disabling psy-rd improve the appearance?

[Puncakes]

Adding grain works great here, and since you refuse to provide an output sample, I'm going to drop this as "worksforme".

After testing it with --crf 15 --preset fast, I wasn't able to replicate the same problem I had when using the supposedly superior settings in my opening post. Which I guess puts me back at square one in terms of fixing my problem. Guess we know why I'm in the newbie forum.

In any case, I've uploaded a output sample with with those settings to show what I'm experiencing here:
http://www.mediafire.com/?zhqhtvyecjj

However, it appears to be much more severe in this sample clip: http://www.mediafire.com/?t2kz3d35bzy -- Which I've also uploaded an unprocessed version of here: http://www.mediafire.com/?zrymkj2wntd

But thanks for providing a sample for which I found significant inefficiencies in ultra-low quantizer encoding--expect an improvement in bitrate of up to ~5-10% at low CRF values on this clip in a few days ;)

Always happy to inadvertently help. Maybe you'll get some insight out of that other sample as well, seeing as I'm having the same problem with it.

[poisondeathray]

What was your script for 3.vob ? There is no way the 3.mkv would look like that unless you added grain (did you add grain to try to get rid of banding?)

[Puncakes]

What was your script for 3.vob ? There is no way the 3.mkv would look like that unless you added grain (did you add grain to try to get rid of banding?)


Even if I throw in an addgrainc(5,5)

Adding grain works great here, and since you refuse to provide an output sample, I'm going to drop this as "worksforme".

In any case, I've uploaded a output sample with with those settings to show what I'm experiencing here:
http://www.mediafire.com/?zhqhtvyecjj


Yes, yes I did add grain, to show that even at CRF 18, with as high quality settings as I can get,

all I'm left with is a fluctuating, noisy image.

[poisondeathray]

It looks way better without grain on that specfic source. The banding is minor compared to the noise you've added. Notice your filesize ballooned up to 8-10x of what it would be without grain in your script.

Also try with psy off, and AQ off - it might be personal preference, but I find it works better on this type of anime.

For example, I tried MeGUI with the default old anime preset (only changes were psy off, AQ off, and crf15), and it ended up at 765kb/s compared to your 392kb/s in the 1.vob example above, and looked slightly better

--profile high --preset fast --crf 15.0 --thread-input --deblock 1:1 --bframes 16 --b-pyramid --no-weightb --b-bias 0 --scenecut 40 --ref 8 --rc-lookahead 40 --no-mbtree --merange 16 --me umh --subme 7 --partitions all --trellis 2 --psy-rd 0:0

[Puncakes]

It looks way better without grain on that specfic source. The banding is minor compared to the noise you've added. Notice your filesize ballooned up to 8-10x of what it would be without grain in your script.

I didn't add the grain to dither it, like I said in an earlier post, I added it to show that CRF 18 with insane quality settings (and then some) isn't actually retaining any of it and, if anything, the higher quality settings were lowering the quality. If I wanted to dither, I'd use gradfun2dbmod.

Also try with psy off, and AQ off - it might be personal preference, but I find it works better on this type of anime.

For example, I tried MeGUI with the default old anime preset (only changes were psy off, AQ off, and crf15), and it ended up at 765kb/s compared to your 392kb/s in the 1.vob example above, and looked slightly better

--profile high --preset fast --crf 15.0 --thread-input --deblock 1:1 --bframes 16 --b-pyramid --no-weightb --b-bias 0 --scenecut 40 --ref 8 --rc-lookahead 40 --no-mbtree --merange 16 --me umh --subme 7 --partitions all --trellis 2 --psy-rd 0:0

I'll try it (as soon as my processor is free in ~40 minutes), but again, my problem is that things like increasing reference frames, subme, lookahead, etc. are only making the quality worse, which perplexes me.

[Dark Shikari]

I didn't add the grain to dither it, like I said in an earlier post, I added it to show that CRF 18 with insane quality settings (and then some) isn't actually retaining any of it and, if anything, the higher quality settings were lowering the quality. If I wanted to dither, I'd use gradfun2dbmod.

I'll try it (as soon as my processor is free in ~40 minutes), but again, my problem is that things like increasing reference frames, subme, lookahead, etc. are only making the quality worse, which perplexes me.You can't compare quality between files at different bitrates. Stop doing it.