Unusually small x264 output [Archive]

asarian

3rd June 2010, 17:14

I recently upgraded from x264 version 1093 to the latest 1627 build from two days ago. Some parameters had been removed, others slightly changed. No big deal, I thought, so I used the following parameters on several encodings:

--crf 16 --sar 1:1 --level 4.1 --aud --nal-hrd none --vbv-bufsize 70000 --vbv-maxrate 35000 --filter 0,0 --ref 3 --b-pyramid normal --mixed-refs --bframes 8 --b-adapt 2 --weightb --no-fast-pskip --direct auto --subme 9 --trellis 1 --partitions all --8x8dct --me tesa --threads auto --merange 16 --thread-input

The result is phenomenal; a bit too phenomenal, it would seem, as the resultant output file is exceedingly small compared to what I'm used to. For example, re-coding my entire Blu-Ray 2010 disc (VC-1) results in a file length of < 5 GB! (of the original 19 GB). Used to be around the same size, also ~19 GB.

So, what's happening here? Did I mess up a new parameter? Does x264 mess up? (less likely, but not impossible, of course). Or is the result really all that spectacularly better? Only thing I truly changed is that I set bframes to 8 (instead of 3), as my PS3 can apparently handle it (as long as you limit refs to 3).

I prepared a 2 minutes sample from the output here (below). The low bit-rate and VERY small output length notwithstanding, the output video itself really looks fine to me: SAMPLE (http://rapidshare.com/files/394854108/2010-sample.rar.html)

creamyhorror

3rd June 2010, 17:38

1. What was the measured SSIM of the encode? That should give some assurance that what you got was of sufficient quality.

2. Do zoomed frame-matched comparisons with AvsP(mod) and see if you can tell the differences between the source and encode.

Ultimately, if it looks good to you even under close inspection, you shouldn't worry.

Keiyakusha

3rd June 2010, 17:58

actually I think 19 -> 5 gb is fine and if it "Used to be around the same size" then there was some problems before and now they gone

P.S.
you probably want to adapt your commandline to new preset system and get rid of stuff like --me tesa

nurbs

3rd June 2010, 18:05

Or at least you can get rid of the stuff that's on by default anyway:
--nal-hrd none --filter 0,0 --ref 3 --b-pyramid normal --mixed-refs --weightb --trellis 1 --8x8dct --threads auto --merange 16 --thread-input

asarian

3rd June 2010, 18:07

1. What was the measured SSIM of the encode? That should give some assurance that what you got was of sufficient quality.

I'm not entirely sure how useful computing SSIM in this case woul be, as I used a hefty TemporalDegrain filter to clean up the picture quality (for a Blu-Ray, the source is actually rather crappy, tbh).

The x264 log doesn't seem to have skipped any specified features, though, so I'm still both surprised (and a little worried) about the small output size. Yesterday I reduced a ~25 GB Anime to around also ~ 5 GB (although sizes like that are not necessarily uncommon for certain Anime's).

dstln

3rd June 2010, 18:47

This doesn't seem overly unreasonable for certain sources that are being denoised.

Blue_MiSfit

3rd June 2010, 21:45

Does it look good? If so, don't worry about it :)

If it doesn't look good, use a lower CRF number.

/thread

burfadel

3rd June 2010, 21:58

Keep --b-adapt 2 and change --subme to 10 instead of 9 :). Also use --me UMH instead of tesa.

Although tesa is minisculally more precise than umh, its speed degradation isn't worth it. I see in your command line using subme 10 to be more worthwhile than subme 9, and --aq-mode 2 instead of 1 (default)

asarian

3rd June 2010, 23:28

If it doesn't look good, use a lower CRF number.

I won't profess to be an x264 expert here (far from it, actually), but this advice doesn't seem to make a lot of sense. There's nothing of value to be gained from lowering CRF below 16, really. Even using 'tesa,' as people pointed out, is already more in the realm of being somewhat OCD about the highest possible setttings, rather than doing something real useful.

Also, keep in mind that I'm basically comparing the output to the result I received from using x264 1093, on which I also always use CRF 16.

Keep --b-adapt 2 and change --subme to 10 instead of 9 :). Also use --me UMH instead of tesa.

Doesn't 'subme 10' require trelis 2? I thought trelis 2 was still listed as 'experimental'?

I see in your command line using subme 10 to be more worthwhile than subme 9, and --aq-mode 2 instead of 1 (default)

Interesting tip on the aq-mode; I'll use that. :)

Dark Shikari

3rd June 2010, 23:48

Doesn't 'subme 10' require trelis 2? I thought trelis 2 was still listed as 'experimental'?Never has been. And really, just use the presets.

asarian

4th June 2010, 00:42

Never has been. And really, just use the presets.

Yep, appears I read it wrong; it's '--psy-rd 2' (which requires Trellis) which is listed as 'experimental'. Sorry.

And I'm leaning towards 'me umh'. Maybe my eyes just aren't good enough, but I have yet to detect a real difference to esa or tesa (except that the latter two increase encoding speed manifold).

Thanks for your time, Dark.

MatLz

4th June 2010, 01:14

subme 10 requires trellis == 2 and AQ on

psy-rd x:y :
to be used x requires m>=6 and y requires trellis in use (1 or 2)

All is in the --fullhelp, good way to start.

Blue_MiSfit

4th June 2010, 03:34

I won't profess to be an x264 expert here (far from it, actually), but this advice doesn't seem to make a lot of sense. There's nothing of value to be gained from lowering CRF below 16, really. Even using 'tesa,' as people pointed out, is already more in the realm of being somewhat OCD about the highest possible setttings, rather than doing something real useful.

Also, keep in mind that I'm basically comparing the output to the result I received from using x264 1093, on which I also always use CRF 16.

Doesn't 'subme 10' require trelis 2? I thought trelis 2 was still listed as 'experimental'?

Interesting tip on the aq-mode; I'll use that. :)

It's pretty simple, actually..

I'd suggest just taking a step back and asking yourself "does my output look good enough?"

If it doesn't look good enough, you can either spend more bits (reduce CRF number, or use more bitrate in 2pass mode), or you can "increase" encoding settings. You're already near the slowest possible settings, so there's not much more to do there.

SO, if you're not happy with the quality, increase bitrate :)

BUT!! You said you ARE happy with the quality, so I'd say leave it as-is :)! For academic purposes, you might try re-encoding with more typical CRF numbers - say 18 or so, and see if your eyes are happy with it.

Also, please do use presets and tunings. They make life infinitely simpler and are literally impossible to screw up.

Try JUST this:

x264.exe input.avs --level 41 --crf 18 --vbv-maxrate 35000 --vbv-bufsize 70000 --ref 3 --tune animation --preset veryslow --output output.mp4

BTW, those VBV settings seem a little unusual. Are these PS3 specific settings for USB playback? You would want different values for BD25/BD50 or BD5/BD9 (BluRay on a DVD+R(DL))

L4 limits maxrate to 25mbps, and L4.1 allows up to 62.5mbps.

Also.. going from x264 rev 1093 to 1627... well... there's been a LOT of improvement, to say the very least!!

Derek

Lam3rD

4th June 2010, 05:07

The source doesn't have any grain and no details past 720p -> the size is pretty normal. And yea you should check the preset/tune combo and maybe drop tesa in favor of umh

asarian

4th June 2010, 13:15

I'd suggest just taking a step back and asking yourself "does my output look good enough?"

That's always a bit of a tricky call, of course. For instance, in rev r1155 I read "Broke --me esa/tesa due to forgetting to add handling for x264_cost_mv_fpel." So it's always possible something inadvertently dropped back to a lower setting on the new version somehow (wasn't 'tesa' here, because it ran at tesa speed, LOL). Something that may not be immediately obvious. I did some more testing, though; and I'm now convinced the new x264 simply really is a LOT better than the old 1093 release.

So, if you're not happy with the quality, increase bitrate :)

Well, you can admire the output in the sample above. :) I really can't detect anything out of the ordinary with it, except that the output is much smaller now.

Try JUST this:

x264.exe input.avs --level 41 --crf 18 --vbv-maxrate 35000 --vbv-bufsize 70000 --ref 3 --tune animation --preset veryslow --output output.mp4

Those presets sound like a good idea. Thanks. And it seems I can still override certain requirements for my PS3 (like limit ref frames to 4 on 1920x1080p). So it's all good.

BTW, those VBV settings seem a little unusual. Are these PS3 specific settings for USB playback? You would want different values for BD25/BD50 or BD5/BD9 (BluRay on a DVD+R(DL))

L4 limits maxrate to 25mbps, and L4.1 allows up to 62.5mbps.

Yeah, I know; but I figured 35Mb/s for a x264 output file is already unusually high as it is; but I can crank it up a bit, for sure.

Also.. going from x264 rev 1093 to 1627... well... there's been a LOT of improvement, to say the very least!!

It would appear so, yes. I thought improvement had been pretty much maxed out already at 1093; but this newer version really seems to kick some major ass. :) Looks like I really should upgrade more often.

nurbs

4th June 2010, 14:09

Those presets sound like a good idea. Thanks. And it seems I can still override certain requirements for my PS3 (like limit ref frames to 4 on 1920x1080p). So it's all good.
You don't need to do that. As long as you don't manually specify the number of reference frames the --level 4.1 command will automatically limit the number used by the preset to the maximum that's allowed at the level you requested according to the resolution.

Trahald

4th June 2010, 19:20

The degrain filter definately didnt hurt, more b-frames than the original would have had to work on this degrained input, longer gops. All of that helps. Having said that, x264 of course is an excellent encoder(imho)

Blue_MiSfit

4th June 2010, 23:15

Yep, I'd suggest upgrading as frequently as possible. Little bugs get fixed all the time, and you missed out on some really good things like excellent 1 pass VBV, mb-tree (which is huge for anime) etc..

Derek

asarian

12th June 2010, 15:50

I'll say! :) Just re-encoded my entire Macross Frontier collection (9 Blu-Rays); this time also with max settings, but properly tuned for Anime with the new x264 settings. The result? I shaved over 20 GB off the total!

And last night I re-encoded Evangalion 1.11 again; also with '--tune animation' and otherwise max settings. The resultant h264 file is -- hang onto something! -- only 2 GB!! (out of ~25 GB). Looks like I'm just going to have to re-encode everything, as the results are simply too spectacular to ignore. Dark Shikari, you're the man!!

creamyhorror

12th June 2010, 16:51

Blue_MiSfit

13th June 2010, 00:21

Chances are, you probably used too much bitrate and chose bad seettings last time :) Honestly x264 is farking amazing, but it was also farking amazing back at rev ~1000.

IMO, x264 became the "best" option for most folks doing dvd / bluray backup when D_S coded in VAQ, and later Psy RDO. These eliminated the nasty issues of blocking static backgrounds which had plagued every H.264 implementation I'd seen until then (and kept me on Xvid for far too long). I think this all happened about 2.5 years ago..

Derek

mp3dom

13th June 2010, 13:27

And last night I re-encoded Evangalion 1.11 again; also with '--tune animation' and otherwise max settings. The resultant h264 file is -- hang onto something! -- only 2 GB!! (out of ~25 GB).

Allow me to highly doubt that 2 GB is enough to store that movie (at least if your intention was to re-encode the movie in a transparent way, keeping it also in FullHD)

Blue_MiSfit

14th June 2010, 01:30

Perfectly doable, especially with compressible anime!

mp3dom

14th June 2010, 02:10

Keiyakusha

14th June 2010, 07:13

If you mean small grain that is not visible easily, than this is matter of preference probably. Without turning of psy optimizations, there should be enough of that grain. So far I never saw any anime source that needs 15-20 mbps to look good with x264. Something around 10 max usually works for me (if source true HD and not some lame 720p->1080p upscale). Also even if some of that grain will be lost, I see no problem in that, because for those who like it there is gradfun, by using which you can get similar look or even better. So there is no need in wasting bitrate for some noise/grain. Also there is people who dislikes this grain. Personally, if there was a way to turn luma into something as flat as this (http://imgur.com/bK75O.png) without destroying some of the edges and increasing processing time very much - thats would be the first that I would done.

asarian

14th June 2010, 07:31

Allow me to highly doubt that 2 GB is enough to store that movie (at least if your intention was to re-encode the movie in a transparent way, keeping it also in FullHD)
Which brings me back to the title of this thread: "Unusually small x264 output." :) For the (transparent, 1080P) re-encoding of Evangalion 1.11 I used the exact following x264 parameters:

--crf 16 --sar 1:1 --level 4.1 --aud --nal-hrd none --vbv-bufsize 70000 --vbv-maxrate 60000 --filter 0,0 --ref 4 --b-pyramid normal --mixed-refs --bframes 8 --b-adapt 2 --weightb --no-fast-pskip --direct auto --subme 10 --trellis 2 --partitions all --8x8dct --me tesa --threads auto --merange 24 --thread-input --aq-mode 2 --psy-rd 0.4:0 --aq-strength 0.6 --deblock 1:1

The part in bold is just the equivalent of '--tune animation' (but without allowing x264 to double the ref frames: a thing my PS3 wouldn't like). Those settings yield a ~2 GB output file. It has, at times, a ridiculously low birate (< 1 Mb/s), but I can't detect *anything* wrong with it, visually.

mp3dom

14th June 2010, 09:10

So far I never saw any anime source that needs 15-20 mbps to look good with x264. Something around 10 max usually works for me (if source true HD and not some lame 720p->1080p upscale).
No doubt about your statement, but it depends on what's your final goal. Look good is a bit different from 'transparent'. At those bitrates the results is no transparent at all.

Also even if some of that grain will be lost, I see no problem in that, because for those who like it there is gradfun, by using which you can get similar look or even better.

This means that you'll only watch the movie on PC. Standalone players are not supposed to have gradfun built-in.

So there is no need in wasting bitrate for some noise/grain. Also there is people who dislikes this grain.
I think the main goal for a good work is to be as close as possible to the master source for every user with every equipment. So, basically, you prefer to denoise a movie if it contains grain.

The part in bold is just the equivalent of '--tune animation' (but without allowing x264 to double the ref frames: a thing my PS3 wouldn't like). Those settings yield a ~2 GB output file. It has, at times, a ridiculously low birate (< 1 Mb/s), but I can't detect *anything* wrong with it, visually.
Since I've that movie, I'm just curious to use the same settings to see the results. Are you watching on an LCD monitor or CRT? Is it calibrated? Because too often the artefacts are no visible due to wrong luma/contrast settings on the monitor.

asarian

14th June 2010, 09:42

Since I've that movie, I'm just curious to use the same settings to see the results. Are you watching on an LCD monitor or CRT? Is it calibrated? Because too often the artefacts are no visible due to wrong luma/contrast settings on the monitor.
I'm watching the result on my 40" Sony Bravia LCD TV (1080P). It's very well calibrated; I'm very fussy when it comes to quality (I tend to encode with higher x264 settings than most people, I think).

creamyhorror

14th June 2010, 11:43

--crf 16 --sar 1:1 --level 4.1 --aud --nal-hrd none --vbv-bufsize 70000 --vbv-maxrate 60000 --filter 0,0 --ref 4 --b-pyramid normal --mixed-refs --bframes 8 --b-adapt 2 --weightb --no-fast-pskip --direct auto --subme 10 --trellis 2 --partitions all --8x8dct --me tesa --threads auto --merange 24 --thread-input --aq-mode 2 --psy-rd 0.4:0 --aq-strength 0.6 --deblock 1:1

The part in bold is just the equivalent of '--tune animation' (but without allowing x264 to double the ref frames: a thing my PS3 wouldn't like).
Does anyone else think that these psy-rd and aq-strength values might be a little low for detailed/grainy animation? They make sense for clean anime, but (film) grain does well with higher aq-strength and psy-rd, so grain on anime movies might as well...

mp3dom

14th June 2010, 11:55

Keiyakusha

14th June 2010, 16:31

No doubt about your statement, but it depends on what's your final goal. Look good is a bit different from 'transparent'. At those bitrates the results is no transparent at all.
Well maybe but this depends on situation. I can't tell the difference, maybe someone can. So for me they transparent.

This means that you'll only watch the movie on PC. Standalone players are not supposed to have gradfun built-in.
Oops. For some reason I forgot about those who prefer standalone players over HTPC. By the way, do you know what is film grain modeling, which is part of h264? Isn't this is something that can give results similar to gradfun? (this is actually a question, I have very little knowledge about this FGT)

I think the main goal for a good work is to be as close as possible to the master source for every user with every equipment. So, basically, you prefer to denoise a movie if it contains grain.

I think this is doubtful. This noise added artificially. There is no such thing when anime just created/drawn. For example what if picture will be badly upscaled? Like... lets say AiR? We also don't want to leave it like it is. Also the "flat" image I posted is actually a histogram which shows luma of the vector-based image that I created. It uses colors and have some gragients. By converting it back to RGB, in looks exactly like just rendered RGB source (except chroma subsampling of course) but histogram still loks like have banding. Because this is normal, because there is also chroma which makes gradients look good and with adding this noise, result actually looks farther from the original. So as long as our chroma is not killed (by overcompression or something) and looks OK, maybe we don't need this additional noise, thats why it looks different but transparent.

No I don't prefer to denoise. I'm saying that if some of that noise is lost due to encoding (not prefiltering) this is fine. Generally I do denoise only if grain easily visible and not when I need to go frame by frame to see it.

Does anyone else think that these psy-rd and aq-strength values might be a little low for detailed/grainy animation? They make sense for clean anime, but (film) grain does well with higher aq-strength and psy-rd, so grain on anime movies might as well...

Thats depends on situation AQ - yes. I prefer mode 2 with default settings which should adjust its strength on frame basis. But PSY - no. No, if you want to keep source close to the original. Because at higher values it is possible to see distortion. For example recently I saw like at strength 0.8 small objects like falling leaves started to change their shape.

mp3dom

14th June 2010, 16:58

Well maybe but this depends on situation. I can't tell the difference, maybe someone can. So for me they transparent.

Ok, but here we go in a personal-subjective choice. Anyway it's quite irrealistic that a 3mbps avg and true 1080p could be really objectively transparent. x264 is good above all in the low-bitrate field but it's not a holy grail.

By the way, do you know what is film grain modeling, which is part of h264? Isn't this is something that can give results similar to gradfun? (this is actually a question, I have very little knowledge about this FGT)

I don't know but I don't think that film grain modeling adds dithering grain to the picture. Probably its use is to preserve an already-dithered image.

This noise added artificially. There is no such thing when anime just created/drawn.

It's called dithering. Due to the facts that the master comes in 10bit 4:2:2 (and up), if you don't do a proper dithering you'll end up with an image with absurd color banding (especially in flat/cg/color gradients) due to the fact that you're truncating data from 10bit to 8bit and going from 4:2:2 to 4:2:0. If the banding is too strong there's no gradfun here that could effectively resolve the problem. It's the same as 24bit audio to 16bit. You add dithering noise. If the dithering is made in the proper way, the noise is nearly as invisible and you preserve the smooth gradients (with proper bitrate). Otherwise you have an image completely damaged by posterization/banding which deviate even more from the original source. Try to encode the first 1000 frames of Elephants Dream from uncompressed PNG and you'll see what I'm saying.

For example what if picture will be badly upscaled? Like... lets say AiR? We also don't want to leave it like it is.

The difference here is that dithering is the right step when you're working on a master, upscale is the *wrong* step to do something :)

Keiyakusha

14th June 2010, 17:15

Probably its use is to preserve an already-dithered image.
Well as I understand FGT is something that created on the fly during decoding. Yes, it probably uses some additional info, maybe something like SBR in AAC but isn't this even better? Anyway grain is grain, so by using FGT we all (HTPC and Hardware guys) should be happy - thats was my idea :)

And about dithering, I don't want to go deeper in it now. By the way I can't seem to find Elephants Dream in PNG on xiph.org :(

mp3dom

14th June 2010, 17:22

The sources of Elephants Dream are here (http://media.xiph.org/ED/)

14th June 2010, 17:50

Well as I understand FGT is something that created on the fly during decoding. Yes, it probably uses some additional info, maybe something like SBR in AAC but isn't this even better? Anyway grain is grain, so by using FGT we all (HTPC and Hardware guys) should be happy - thats was my idea :)

If properly implemented, film grain extraction and synthesis might work, but Thomson's FGT died with HD DVD (at least for now) and it's not supported by current hardware decoders.

Blue_MiSfit

14th June 2010, 20:39

Correct. FFM / FGT isn't implemented anywhere. Some decoders may support it, but no encoders do AFAIK :)

Derek

asarian

15th June 2010, 16:27

That combination to me is low even for clean anime. I've found out that often it creates macroblocks on uniform colors even at high bitrates so in general I raise psy-rd to something near 1.0 and aq-strength to 0.7/0.8
Hmm, your findings are a mite disturbing. :( Maybe I should forget about "--psy-rd 0.4:0 --aq-strength 0.6" altogether for Anime, and just revert to a regular max profile. As the saying goes: when something looks too good to be true, it usually is.

For regular moviev, btw, does '--aq-mode 2' make me lose detail too? Or is it simply always better than 1?

asarian

15th June 2010, 18:22

Could it be, but that movie is not that compressible. It has lots of subtle grain practically in every scene to keep the color gradients as smooth as possible. With that verylow bitrate (what's the average, 3 Mbps? For a 1080p?) I bet that all the smooth gradients goes away. To start to preserve something you need to go at least at 15-20 Mbps on that movie (and it could be not enough).
Okay, I did some close-up testing; and, regret to say, you're right: the RESULT (http://rapidshare.com/files/399354618/goodbad.jpg.html) is bad. Very bad, actually. Can't believe I didn't see it at first (wishing is believing, I guess), but the resultant 2GB output is simply horrific, and completely unacceptable. So much gradient is gone; and so much macro-blocking in return! I'll chalk it up as a good experiment; but no more running with '--tune animation' for me. I'm sure it may work better on cleaner sources, but Evangelion is sadly not among those.

asarian

17th June 2010, 23:33

Okay, I'm getting a little worried now. :( Thinking "--psy-rd 0.4:0 --aq-strength 0.6 --deblock 1:1" was the culprit in my earlier rendition of Evangelion 1.11, which turned out quite bad, this time I re-encoded without that stuff:

--crf 16 --sar 1:1 --level 4.1 --aud --nal-hrd none --vbv-bufsize 70000 --vbv-maxrate 60000 --aq-mode 2 --rc-lookahead 60 --filter 0,0 --ref 4 --b-pyramid normal --mixed-refs --bframes 8 --b-adapt 2 --weightb --no-fast-pskip --direct auto --subme 10 --trellis 2 --partitions all --8x8dct --me tesa --threads auto --merange 24 --thread-input

And low and behold, I get the same crappy output! A ~2.3 GB file, that just looks horrible (borked gradients, huge macro-blocks). Look at this still photo sample (http://rapidshare.com/files/400185702/alsobad.jpg.html), please. It's very bad!

I attached the x264 log for inspection. I'm telling ya, something is fishy.

AlekseiV

18th June 2010, 00:46

Alt-host of the second image here (http://sovietpride.su/alsobad.jpg). First image is down.

I don't see anything wrong but my monitor is covered in dust and fingerprints.

What is "--filter 0,0"? I've never seen that before and the x264 Settings wikiarticle doesn't seem to have it.

mp3dom

18th June 2010, 00:47

To me there's too big difference between I frame QP (~12) and B frame QP (~20). Try to use tune grain and overwrite the psy-rd to 0.6 and aq-strength to 1.0. Also set qpmin to 0. Dithering are really fine details and qpmin=10 (default) is not enough. Also set weightp=2 if you can (your standalone supports it) and --no-dct-decimate . Also I would change --me tesa since it's too slow to --me umh and I would raise --merange to 64. Anyway keep in mind that preserving dithering is very difficult and above all, it require a lot of bitrates. If your goal is to 'shrink' the movie from 25 GB down to 3-4 GB AND keep all the smooth gradients, you're probably wasting your time. There's a reason why the original movie has an average bitrate of ~32 Mbps and it's not because the encoder used sucks.

I don't see anything wrong but my monitor is covered in dust and fingerprints.
It's simply awful :). The hairs, the car and the seat are all full of color banding/posterization

kemuri-_9

18th June 2010, 01:43

What is "--filter 0,0"? I've never seen that before and the x264 Settings wikiarticle doesn't seem to have it.

--filter is an alias for --deblock

asarian

18th June 2010, 02:15

Mp3dom, your priceless suggestions are much appreciated! :) I've implemented your changes, and am now running a new process with the following settings:

--level 4.1 --aud --nal-hrd none --vbv-bufsize 70000 --vbv-maxrate 60000 --ref 4 --b-pyramid normal --mixed-refs --bframes 8 --b-adapt 2 --weightb --no-fast-pskip --direct auto --subme 10 --trellis 2 --partitions all --8x8dct --me umh --threads auto --merange 64 --pbratio 1 --aq-mode 2 --qcomp 0.8 --deadzone-inter 6 --deadzone-intra 6 --rc-lookahead 60 --deblock -2:-2 --ipratio 1.1 --psy-rd 0.6:0.25 --aq-strength 1.0 --no-dct-decimate --qpmin 0 --weightp 2 --thread-input

I had encoded this a few days earlier (but didn't like the size of the subs, after all), so I decided to re-encode. For that earlier encode I had used the following params:

--level 4.1 --aud --nal-hrd none --vbv-bufsize 70000 --vbv-maxrate 60000 --filter 0,0 --ref 4 --b-pyramid normal --mixed-refs --bframes 8 --b-adapt 2 --weightb --no-fast-pskip --direct auto --subme 9 --trellis 1 --partitions all --8x8dct --me tesa --threads auto --merange 16 --aq-mode 1 --rc-lookahead 40 --thread-input

It's somewhat less heavy; and most noticeble difference is '--aq-mode 1' instead of 2, and 'trellis 1' instead of 2.

Thing of it is, though, that with those settings, the same movie looks much, much better (same still photo sample (http://little-albatross.net/notsobad.jpg), looking much better); it's not perfect yet, but hugely better than what I got from the 2 GB output file (this old one was around 10 GB) on the encode that was supposed to be even better. So, I'm still thinking there's something in the 'heavy' settings that causes the extreme deterioration. Dunno what, of course; could be the --aq-mode 2.

Audionut

18th June 2010, 03:14

What horrible command lines.
I'm also fairly positive that your vbv settings exceed level 4.1.
Are you using 4 refs for hardware compatibility? Again, I don't think you'll find a device on the planet that will take those bitrates.
Also, with the lowish bitrates you're getting, the bitrate isn't going to get anywhere near those vbv values. And even, if you had a crazy scene that "maybe" could reach those bitrates, imo, it would be frames of just grain like the hbo logo and would be better being limited in bitrate to spare more for the rest of the movie.

--qcomp 0.0???? Leave it alone!!!!!!!

That 1st command line can be reduced to this, with a few tweaks I think might help your case.

--preset placebo --tune grain --level 4.1 --ref 4 --me umh --merange 64 --aq-mode 2 --aq-strength 0.6 --psy-rd 0.4 --qpmin 0 --aud --nal-hrd none --vbv-bufsize 40000 --vbv-maxrate 40000

edit: what are you using to capture the still. Compared to the second still, the first one is de-saturated and has less brightness.
edit2: if you don't need to limit the refs for hardware compatibility, just remove the --refs 4 from the command line I posted.

asarian

18th June 2010, 03:46

What horrible command lines.
I'm also fairly positive that your vbv settings exceed level 4.1.
Dunno about that. L4.1 allows for max 62,500kb/s; and I'm fair certain --vbv-bufsize is still within bounds, too.

Are you using 4 refs for hardware compatibility?
Yes, for my PS3. Max allowed value for 1080P content is 4.

--qcomp 0.0???? Leave it alone!!!!!!!
That was me making a typo copying it erroneously from the '--tune grain' line. Should be 0.8 that.

edit: what are you using to capture the still. Compared to the second still, the first one is de-saturated and has less brightness.
The somewhat desaturated stills were taken from KMplayer; the last one with GOM player (as KMplayer kept choking on it)

asarian

18th June 2010, 14:14

Some news.

Mp3dom, I tried it with your settings, and there's still significant colorbanding:

still colorbanding (http://little-albatross.net/aq2.jpg)

Then I disabled something which I suspected all along: aq-mode 2, and let it use mode 1 instead. The result? Already a vast improvement! Here:

huge improvement with aq-mode 1 (http://little-albatross.net/aq1.jpg) (!)

So, I still say there's something fishy with --aq-mode 2; at least for this type of Anime.

Lastly, I cranked psy-rd up to 0.9; even slightly better, I think:

doing --psy-rd 0.9:0.25 (http://little-albatross.net/psy09.jpg)

It's not perfect yet, though. Are there other things I can do to stay closest to the original?

mp3dom

18th June 2010, 15:25

burfadel

18th June 2010, 15:31

asarian

18th June 2010, 15:36

Is there any difference in the filesize (>0.5%) between --aq-modes 1 and 2?
Yes, there's exactly a 17.57% difference; which is to say, the aq-mode 1 file is 17% larger than its aq-2 variant.

asarian

18th June 2010, 15:53

The difference with psy-rd is not that high (assuming the frame is exactly the same). Since too high psy could create other kind of artefacts (ringing or some image distortion) I would prefer to keep psy a bit lower.
Are you still keep CRF at 16?
I'll keep the psy-rd comment in mind; thanks. :)

Yes, CRF is still at 16. I'm tempted to set it to 0 (lossless), though, LOL.

mp3dom

18th June 2010, 16:24

You can try to set it to 10/12. The bitrate probably will jump to something near the range of pressed BD (25-35 Mbps)

burfadel

18th June 2010, 16:25

Yes, there's exactly a 17.57% difference; which is to say, the aq-mode 1 file is 17% larger than its aq-2 variant.

I suspected that would be the case, a lower CRF seems to be needed at times with aq mode 2 I have found.

Try raising the CRF of aq mode 1 until the output filesize is approximately the same as from aq mode 2. Remember CRF can have a decimal too, like 17.6 etc.

A comparison considering the same file size will truly show the quality differences between the aq modes. Of course this shouldn't need to be done, but it is classed as 'experimental' and probably needs some tweaking to suit all scenarios effectively.

Audionut

18th June 2010, 18:16

Yes, there's exactly a 17.57% difference; which is to say, the aq-mode 1 file is 17% larger than its aq-2 variant.

And you think there is something wrong with aq-mode 2 :confused:

As suggested, raise the bitrate of the aq-mode 2 encode to that of the aq-mode 1 encode before you make such claims.

asarian

18th June 2010, 19:52

And you think there is something wrong with aq-mode 2 :confused:

As suggested, raise the bitrate of the aq-mode 2 encode to that of the aq-mode 1 encode before you make such claims.
Why, yes, actually: if just switching to aq-mode 2 causes a severe deterioration, then yes, I'd say there's something 'wrong' with it; or, at the very least, that it should come with stiff warnings about having to compensate for it by adding bitrate (like lowering CRF). Otherwise people use it and find their output to be of worse quality than that of mode 1.

nurbs

18th June 2010, 20:02

Every option potentially has that effect. Turning off mbtree will often inflate the filesize, turning off psy can lead to much lower filesizes. CRF isn't really constant quality, especially not if you change options.

asarian

18th June 2010, 20:57

You can try to set it to 10/12. The bitrate probably will jump to something near the range of pressed BD (25-35 Mbps)
I think we're really getting somewhere here. :) Your continual good suggestions have been a big help! With --aq-mode 2 back in, and CRF 10, all of a sudden I got the best output so far:

aq-mode 2 + crf 10 (http://little-albatross.net/crf10.jpg)

And here with CRF 8, which is even slightly better:

aq-mode 2 + crf 8 (http://little-albatross.net/crf8.jpg)

I was 'raised' (ahem) how anything under CRF 16 was ridiculous. And you'd be surprised how many people on this forum as well have told me even CRF 16 is waay too low. But lowering it even further definitely seems the answer for -aq-mode 2 here.

I think I'm getting close to the best I can make it here. :)

mp3dom

18th June 2010, 21:24

Not bad, but as I've already seen on grainy source, x264 tends to make some small blocks of no-grain in a grainy source (look at the lowest left part of the seat in the crf10 image). This is not so good to the eye. I think here it could be preferable (at least to the eye) to have a very slighty blur on the grain but preserve the 'complexity' (so rather than have blocks with crispy noise as the source + small smoothed blocks it could be better to have all blocks with a slighty bit blurred grain but with all uniform grainy blocks)

Anyway, the lower the crf is, the more the accuracy is to the source :). What's the average bitrate now? (with crf 0 or qp 0 I think you're in the mathematically lossless)

You could also try to disable trellis 2 (set it to 0) and set deadzone-intra and deadzone-inter both to 0. It could enhance the dithering more (but I think it depends on the source)

Audionut

19th June 2010, 01:58

To add to what nurbs posted.

at the very least, that it should come with stiff warnings about having to compensate for it by adding bitrate (like lowering CRF)

So the option should be,

--aq-mode-2-I'm-expecting-the-same-bitrate-as-aq-mode-1-so-warn-me-if-the-bitrate-isn't-the-same-cause-I-don't-understand-how-crf-mode-works-and-need-assistance-so-I-don't-go-to-message-boards-and-post-false-information-based-on-my-own-lack-of-information.

How is supposed to know your shooting for the same bitrate. It's an option that provides a function.

Do a search for crf by Dark_Shikari and learn how it works.

I think we're really getting somewhere here. :)With --aq-mode 2 back in, and CRF 10, all of a sudden I got the best output so far:

And what's the bitrate of the new encode compared to the others!! Can you not see how increasing bitrate, increases quality??????

I was 'raised' (ahem) how anything under CRF 16 was ridiculous. And you'd be surprised how many people on this forum as well have told me even CRF 16 is waay too low. But lowering it even further definitely seems the answer for -aq-mode 2 here.

--aq-mode has got nothing to do with it. "Generally", yes, --crf 16 is normally overkill. But, it's also known, that to fix banding issues, you generally have to throw bitrate at it. ie: lower --crf.

(with crf 0 or qp 0 I think you're in the mathematically lossless)

Yes --crf 0 is lossless as stated by the --help option.

You could also try to disable trellis 2 (set it to 0) and set deadzone-intra and deadzone-inter both to 0. It could enhance the dithering more (but I think it depends on the source)

The only reason to disable trellis is for speed. You could add some psy-trellis to try keep the grain, --psy-rd x.x:0.25

deadzone settings of 0 :confused: Why do you advise settings you're not even sure could help?

asarian

19th June 2010, 09:33

So the option should be,

--aq-mode-2-I'm-expecting-the-same-bitrate-as-aq-mode-1-so-warn-me-if-the-bitrate-isn't-the-same-cause-I-don't-understand-how-crf-mode-works-and-need-assistance-so-I-don't-go-to-message-boards-and-post-false-information-based-on-my-own-lack-of-information.

How is supposed to know your shooting for the same bitrate.
Nobody is talking about shooting for the same bitrate -- except you.

There's many options in x264 that lower bitrate, one way or the other. Usually that's good thing (in a nutshell: x264 takes away from scenes that need it less, and gives to scenes that need it more). It only becomes a problem when lowering the bitrate also comes with an overall, severe deterioration of the image quality, as is the specific case here with --aq-mode 2. In cases like that, yes, a simple line somewhere, like: "Please, be sure you substantially increase bitrate when using --aq-mode 2, or you may lose image quality!" wouldn't have been amiss.

And what's the bitrate of the new encode compared to the others!! Can you not see how increasing bitrate, increases quality??????
You really don't get it, do you? :)

It's all about the benefit/penalty ratio. --aq-mode 2 attempts to adapt quantization strength per-frame; and I think we can agree on this happening for the purpose of making things look better, right? In order to achieve the desired result, it apparently (often, it would seem) needs a higher bitrate. That's, in itself, not unreasonable; except that it needs not just a bit more here, but a lot more: i.c., for this movie you need to go from CRF 16 to at least CRF 10. To quote Dark Shikari himself: "Dropping CRF by 6 will double filesize. Dropping CRF by 1 will raise filesize by 12.5%." There you have it.

So the question then is simply: is going from --aq-mode 1 to --aq-mode 2 worth doubling your output file size for? I say no: the penalty here far outweighs any possible, minute visual improvement you may gain. There may well be instances where --aq-mode 2 works miracles at the same bitrate (or with just slightly higher rates); but so far, in earnest, I'm not so sure about it. I've had similar experiences with --aq-mode 2 with "The 13th Floor" and "Avatar." So, I may well revert to using mode 1 on my future encodes. Haven't decided yet.

The only reason to disable trellis is for speed. You could add some psy-trellis to try keep the grain, --psy-rd x.x:0.25

deadzone settings of 0 :confused: Why do you advise settings you're not even sure could help?
His advice, your confusion despite, is dead accurate! And no, "The only reason to disable trellis is for speed" is incorrect. Basically, trellis and the --deadzone settings are incompatible:

--deadzone-inter and --deadzone-intra tweak a parameter of the deadzone algorithm. (The parameter in question is "deadzone", that's where the algorithm got its name.) --trellis determines when to use the trellis algorithm (0=never, 1=sometimes, 2=almost always); any dcts that don't use trellis must use deadzone instead. So, the fact that --deadzone* still has some minor effect when you enable trellis does not contradict the fact that deadzone and trellis are incompatible. Don't do that. If you enable trellis, then deadzone is only used as a fast estimate of trellis, so a reduced value of --deadzone does not provide any grain retention, it just reduces the precision of that estimate and thus of the mode decision process.

So, Mp3dom was spot-on right suggesting I disable trellis when using the --deadzone tweaks. And, indeed, the image quality improved as a result!

mp3dom

19th June 2010, 10:49

deadzone settings of 0 :confused: Why do you advise settings you're not even sure could help?

I've see improvements setting deadzone to 0 and trellis to 0 in some fadein/fadeout prior of the fade-improvement patch (which is anyway not committed actually, but only available as a separate patch) even when using weightp 2. In Fadein/Fadeout you need to preserve dithering/subtle grain if you don't want to have posterization during the fades so that setting worked for me. I have not tested it deeply for every kind of footage so I've suggested to try and see if it could help to improve preserving the dithering a lot more.

Audionut

19th June 2010, 11:16

Ok sparky.

Nobody is talking about shooting for the same bitrate -- except you.

Yes, because it's the only way to compare 2 different encodes with different settings. crf setting doesn't matter!!!!

In cases like that, yes, a simple line somewhere, like: "Please, be sure you substantially increase bitrate when using --aq-mode 2, or you may lose image quality!" wouldn't have been amiss.

There's no magical way for x264 to figure out in advance what effect a given setting will have on filesize at a given CRF, particularly for some types of settings. It varies wildly from source to source; if there was an easy way to compensate, we would have already done it.
http://forum.doom9.org/showthread.php?p=1386676#post1386676

Of course, that's beside the fact, that any sane person would see that the bitrate has dropped significantly and would lower their crf setting to compensate before saying this or that option is bad. You cannot compare different settings at different bitrates. crf does not compare, equal, mean bitrate. Expect to say, that lowering crf, increases bitrate and raising crf, lowers bitrate, with the same settings.

And also besides the fact that doing just that, ie: lowering crf to compensate, which you have tested, increased the quality with aq-mode 2.

I think we're really getting somewhere here. :) Your continual good suggestions have been a big help! With --aq-mode 2 back in, and CRF 10, all of a sudden I got the best output so far:

Fancy that!!!!!!!!!!!!!!!!!!:stupid:
Are those encodes about the same bitrate? Cause the fact that they have different crf settings, doesn't mean anything.

You really don't get it, do you? :)

It's all about the benefit/penalty ratio. --aq-mode 2 attempts to adapt quantization strength per-frame; and I think we can agree on this happening for the purpose of making things look better, right? In order to achieve the desired result, it apparently (often, it would seem) needs a higher bitrate. That's, in itself, not unreasonable; except that it needs not just a bit more here, but a lot more: i.c., for this movie you need to go from CRF 16 to at least CRF 10. To quote Dark Shikari himself: "Dropping CRF by 6 will double filesize. Dropping CRF by 1 will raise filesize by 12.5%." There you have it.

Full of fail. --crf doesn't mean you will get a certain bitrate with different options as you have found out. How do you know that aqmode 2 needs a higher bitrate to maintain the same quality? Can you post you command line results proving this, or are you basing the fact that aqmode 2 requires crf 10 to equal aqmode 1 at crf 16 on this source. Again, --crf xx.xx doesn't mean you will get the same bitrate or same quality when changing options.
There's no magical way for x264 to figure out in advance what effect a given setting will have on filesize at a given CRF
So how about making an encode with aq-mode 1 at crf 16. Note down the resulting bitrate of the encode, and then make another encode with aq-mode 2 while adjusting crf to get an encode with the same bitrate. The fact that each encode uses a different crf doesn't mean diddly squat. If both encodes hit the same bitrate or very close, only then can you compare aq-mode 1 and aq-mode 2 on this source and make a claim that 1 is better than the other.

So, Mp3dom was spot-on right suggesting I disable trellis when using the --deadzone tweaks. And, indeed, the image quality improved as a result!

do,

x264.exe --preset slower -o x:\output.mkv input.avs

x264.exe --preset slower --trellis 0 --deadzone-inter 0 --deadzone-intra 0 -o x:\output.mkv input.avs

Adjust the crf of each to get an equal bitrate encode from both and then judge quality.

Trellis has the benefit of psy-trellis, so you could also do,

x264.exe --preset slower --psy-rd 1.00:0.25 -o x:\output.mkv input.avs

Again adjust the crf to get the same bitrate as the 2 encodes you get above and see the result.

Audionut

19th June 2010, 11:25

I have not tested it deeply for every kind of footage so I've suggested to try and see if it could help to improve preserving the dithering a lot more.

I do apologize for this coming across rather nasty in advance.

I would suggest, particular as you yourself have not thoroughly tested the settings, that you don't offer settings advice such as --deadzone to someone who clearly doesn't have a grasp on how to compare 2 different settings.

Also, you advise him to adjust based on your experience with fades, and I don't recall the op mentioning fades.

Also, as you rightly point out, fade-compensate patch helps with fades and will mean that the op will probably still be using funky deadzone parameters when the patch has been committed.

That setting is only adjusted once out of every preset and tune option provided by the developers of x264 and only down to 6.

asarian

19th June 2010, 12:34

Of course, that's beside the fact, that any sane person would see that the bitrate has dropped significantly and would lower their crf setting to compensate before saying this or that option is bad.
You wilfully don't get it, do you? If you pay X amount for quality Y with option Z, and you have to pay X x 2 for same quality Y with option Z2, then any sane person would shy away from using option Z2. Why is this concept so difficult to grasp for you?

Full of fail. --crf doesn't mean you will get a certain bitrate with different options as you have found out.
Again with the strawman! Nobody staked such a claim about getting a certain bitrate.

How do you know that aqmode 2 needs a higher bitrate to maintain the same quality? Can you post you command line results proving this,
Don't need to: already did. I posted two samples, one with aq-mode 1, and the other with aq-mode 2 (and otherwise same settings). Clearly the aq-mode 2 sample was worse, and could only be 'cured' by increasing bitrate drastically (= lowering CRF, in this case).

The fact that each encode uses a different crf doesn't mean diddly squat. If both encodes hit the same bitrate or very close, only then can you compare aq-mode 1 and aq-mode 2 on this source and make a claim that 1 is better than the other.
As I've tried to explain to you, for the umptiest time, 'better' is not just a matter of final output quality, but also a matter of cost. If it costs me twice as much in file-size to achieve the same result with aq-mode 2 as it does for aq-mode 1, then, in this case, however loud you decry the fact, aq-mode 2 is bad.

Audionut

19th June 2010, 13:05

Don't need to: already did. I posted two samples,

Screen shots are not samples.

MokrySedeS

19th June 2010, 13:12

Why is this concept so difficult to grasp for you?

I'm sorry, but it is for YOU, obviously. Audionut is absolutely right.

asarian

19th June 2010, 13:27

I'm sorry, but it is for YOU, obviously. Audionut is absolutely right.
All y'all can agree with each other until you see blue in the face, LOL, but I, for one, am not prepared to double the output file size with aq-mode 2, to get the same result with what aq-mode 1 offers me for half the size. If that makes me wrong, then I don't want to be right. :)

Audionut

19th June 2010, 13:40

It's not a matter of wrong or right, simply that you seem to be confused about a few things.

Bitrate and file-size are directly comparable. Increase bitrate, increase file-size. decrease bitrate, decrease file-size. No, ifs, buts or maybe's.

Lower crf, increase file-size, raise crf, lower bitrate. Unless you change other command line options. You've experienced it yourself. Your original aq-mode 2 vs aq-mode 1 encodes at the same crf had completely different bitrates and file-sizes.

This doesn't make aq-mode 2 bad. It simply means that x264 is not smart enough to maintain the same bitrate (file-size) when enabling aq-mode 2 as what happened when it encoded it with aq-mode 1.

This also is not a bad thing. It just means that the user has to manually adjust the crf setting if he wants to hit the same bitrate (file-size) as he got with aq-mode 1.
edit2: I guess what you are saying, is that x264 should get the same quality with the same crf regardless of other options. It doesn't work like that, and afaik, won't work like that any time soon. There are simply to many variables to take into consideration.

Please, for the love of god, conduct this little experiment.

Make one encode with aq-mode 1. Now make another encode with aq-mode 2 and adjust no other option except --crf to make the aq-mode 2 encode equal the same bitrate (file-size) as the aq-mode 1 encode.

Now have a look at both encodes and decide which you like the best. If you conduct this experiment, I don't care if you decide aq-mode 2 sucks balls, and aq-mode 1 is best. Atleast I will know that you have conducted an analysis of aq-mode 1 vs aq-mode 2 correctly.

edit: all other things being equal, bitrate and quality are also directly comparable. Increase bitrate, increase quality.

mp3dom

19th June 2010, 14:00

Audionut

19th June 2010, 14:08

Using CRF for that kind of test I think is too a "hit-or-miss". It could require a lot of time to hit the same bitrate since with CRF you don't know the final used bitrate. Probably is better to do a 2pass encode with fixed bitrate/maxrate/bufsize.

Yes and no. But most importantly, the op doesn't seem to quite understand the relationship between, crf, different settings and bitrate (file-size)

He seems to think that a lower crf automatically means larger file-size. Which in turn bases his opinion that aq-mode 2 is crap on the fact that aq-mode 2 requires a lower crf to hit the same quality and/or file-size.

@asarian

You can add --frames 1500 to the command line. This will make each encode shorter and faster to get the correct crf settings for both encodes.
Or, upload a small sample of the original source somewhere, and I'll do the test for you and post all the results here.

edit: just for a bit of clarity around something you posted earlier and I missed.
To quote Dark Shikari himself: "Dropping CRF by 6 will double filesize. Dropping CRF by 1 will raise filesize by 12.5%." There you have it.

This is true for encodes with the same settings. If you change settings, for example, aq-mode, then that statement by Dark Shikari is meaningless.

creamyhorror

20th June 2010, 06:58

It's all about the benefit/penalty ratio. --aq-mode 2 attempts to adapt quantization strength per-frame; and I think we can agree on this happening for the purpose of making things look better, right? In order to achieve the desired result, it apparently (often, it would seem) needs a higher bitrate.
How is this supported by what you posted? Your aq-mode 1 encode had a bitrate 17.6% higher than your aq-mode 2 encode, and the quality of the former was (expectedly) better. The unequal bitrates is a confounding factor to the question of whether aq-mode 1 or 2 is better: you have to eliminate it if you want to make a pronouncement.

CRF is not an absolute scale tied to a particular bitrate curve. It changes for many combinations of options. Under certain options (and across different sources) you may have to use lower or higher RF values than normal to achieve the level of quality you want. aq-mode is probably one of those options.

So the question then is simply: is going from --aq-mode 1 to --aq-mode 2 worth doubling your output file size for? I say no: the penalty here far outweighs any possible, minute visual improvement you may gain. There may well be instances where --aq-mode 2 works miracles at the same bitrate (or with just slightly higher rates); but so far, in earnest, I'm not so sure about it.
On what basis? Have you actually tested the same-bitrate case?

Simply do a 2-pass encode under aq-mode 2 with the same bitrate as a CRF encode under aq-mode 1, and compare those. That is a fair test.

I think you had an expectation that you should be able to use the same RF values irrespective of your other settings, to obtain (roughly) the same bitrates. This is simply not justifiable in light of how CRF and bit distribution works. Audionut may have been rather blunt, but he's right, in a nutshell. Even mp3dom wouldn't disagree.

asarian

20th June 2010, 09:28

Originally Posted by asarian
To quote Dark Shikari himself: "Dropping CRF by 6 will double filesize. Dropping CRF by 1 will raise filesize by 12.5%."
This is true for encodes with the same settings. If you change settings, for example, aq-mode, then that statement by Dark Shikari is meaningless.
And this is exactly what I've done all along: I only lowered CRF (from 16 to 10), both with aq-mode 2 and everything else the same; and, lo and behold, my output file size doubled.

Your aq-mode 1 encode had a bitrate 17.6% higher than your aq-mode 2 encode, and the quality of the former was (expectedly) better. The unequal bitrates is a confounding factor to the question of whether aq-mode 1 or 2 is better: you have to eliminate it if you want to make a pronouncement.
People seem to be under the impression (introduced by Audionut) that I don't understand this. Well, I do. :)

What people seem to gloss over here, en masse, is that I've repeatedly said I already needed to go way beyond equal bitrates for -aq-mode 2 to get the same visual quality as I did for aq-mode 1. In this case, on my aq-mode 2 encoding, I had to lower CRF all the way to 10 (down by 6) to even get something looking similar to what I got from aq-mode 1. So, for the aq-mode 2 encode, I wound up with a file that was 164.8% the size of that of the aq-mode 1 file (remember, with both at CRF 16, the aq-mode 2 file was 17.6% smaller to begin with).

Also, I've never sad aq-mode 2 sucks in general. I've continually said that, in this partical case it makes things much worse, and that I needed to lower CRF too much for my taste to compensate for the quality loss.

For the sake of curiosity, I'll gladly do some experimenting (after the current encoding job is done) with roughly equal bitrates between aq-mode 2 and 1. For this particular movie (Evangelion 1.11), I already know the result: it's worse; but there are some other movies I can try, like "The 13th Floor," with good dark scenes, that will perhaps give me a fairer idea of what I can expect from aq-mode 2 for regular movie content at comparable bitrates. Anime, encoding wise, simply remains a category of its own.

EDIT: Current bitrate for aq-mode 2 sample with CRF 10 hovers between ~40-60Mb/s.

asarian

1st July 2010, 17:07

... Also, with the lowish bitrates you're getting, the bitrate isn't going to get anywhere near those vbv values. And even, if you had a crazy scene that "maybe" could reach those bitrates, imo, it would be frames of just grain like the hbo logo and would be better being limited in bitrate to spare more for the rest of the movie.
Hmm, I don't think I fully understand this. I thought CRF encoding was kinda 'open-ended' that way, in that it doesn't matter whether or not one scene needs a whole lot of bitrate, as it won't be subtracted from some sort of total amount.

After several encodes with the new x264 and settings, indeed, I'm seeing lower average bitrates and higher max rates (vs. higher average rates vs. lower max rates, as it used to be). If I wanted to allow bitrate intense scenes to keep their high bitrate, without this going at the expense of the rest, would lowering qpmin help?

1st July 2010, 17:20

Hmm, I don't think I fully understand this. I thought CRF encoding was kinda 'open-ended' that way, in that it doesn't matter whether or not one scene needs a whole lot of bitrate, as it won't be subtracted from some sort of total amount.

Correct.

asarian

1st July 2010, 21:58

Correct.
In that case, how should I interpret Audionut's comment about sparing bitrate for the rest of the movie? :confused:

nurbs

1st July 2010, 22:33

You should interpret them as only being applicable for when you use bitrate based encoding and VBV.

1st July 2010, 22:39

Yep.

Btw, I'd use Blu-ray compatible settings if those encodes are archived for later use. That way you may not need to re-encode if you get another standalone device at some point.

asarian

1st July 2010, 23:01

You should interpret them as only being applicable for when you use bitrate based encoding and VBV.
Oh good, thanks. :) Trying to eliminate the possible reason(s) for getting consistently lower bitrates with my new encodes; so, with my CRF encodes, at least I can strike off my unusually high VBV setiings as the cause.

Btw, I'd use Blu-ray compatible settings if those encodes are archived for later use. That way you may not need to re-encode if you get another standalone device at some point.
I currently stream my encodes from my TwonkyMedia media server; so I can lax a bit on absolute scrictest Blu-Ray compliance. I don't go over the amount of specified reference frames for a certain resolution; but b-frames and bitrate and such, I tend to give myself some allowance in that area. :)