settings for animated video (BD5/9 / AVCHD) compliant

[moviefan]

Hi guys,

I have read somewhere that the settings for film content and animated content should not be the same, e.g. it is recommended not to use --weightb. Are there any other options I should use/not use for optimal compression of animated videos? The clip I have is very clean, so not noise or grain at all, only banding has been an issue so far. Encoding speed, in the first place, doesn't matter.

Thanks for advice!

Regards
moviefan

[Mr VacBob]

There is not much difference between settings for film and animation; higher numbers of B-frames and refs are more effective for animation, though, and maybe using slower subq/me settings isn't as useful. AQ and Psy-RD might lower quality of lines, but help
with banding just like they do with grain.

And weight_b is safe these days, and recommended with b-adapt 2.

[professor_desty_nova]

With b-adapt 2 --weightb is safe, and you might only need 6 B-frames. Test with various short clips of your source and see the line in x264: "consecutive B-frames" - it gives the % of consecutive B-frames used in the format 0 1 2... consecutive B-frames. If most of the clips only use for example 6 B-frames, you don't need to use a higher values when you encode the final clip (of course you could use 16 B-frames and let x264 decide, but with b-adapt 2 it's incredibly slow). You can also up the reference frames closer to the level limit (4 for 1080p; 9 for 720p @ L4.1). You can use AQ (--aq-strength) but use it at 0.5 or lower so it doesn't affect the lines. If your source is clean, you can try with Psy-RD off (--psy-rd 0.0:0.0) or with a lower value than the default (--psy-rd 0.5:0.0) and compare them. Then encode using the setting you like more.

P.S.: as I don't encode for AVCHD/etc. I don't know if there are limitations to the number of B-frames.

[moviefan]

Okay, I'll try that out. Just one question concerning AQ: higher aq-strengths assign more bits to flat areas. Since my animated video consists of many flat areas, should a higher aq-strength avoid banding? aq-strength 0.5 is pretty low isn't it? So could you please explain your reasons for a smaller aq-strength a bit more in detail? By the way, max. b-frames for BD5 etc. compliancy is 3.

Regards

[Sagittaire]

Okay, I'll try that out. Just one question concerning AQ: higher aq-strengths assign more bits to flat areas. Since my animated video consists of many flat areas, should a higher aq-strength avoid banding? aq-strength 0.5 is pretty low isn't it? So could you please explain your reasons for a smaller aq-strength a bit more in detail? By the way, max. b-frames for BD5 etc. compliancy is 3.

Regards

Really important part for anime are contour ... and AQ from x264 use complexity mask. AQ with x264 produce high ringing level.

[moviefan]

Hm, I have tested with aq-strength 1.0 and already get clearly visible banding... and really annoying... I use some psy-trellis, but that's it. Will aq-strength 0.5 not increase banding?

[poisondeathray]

This is sort of related, but some of the anime guys use filtering for banding (eg. gradfun2db) which you may want to explore

[Sagekilla]

If you do use GradFun2DB, make sure you use AQ too. Otherwise, that dither is going to disappear since x264 normally decimates that.

[Snowknight26]

If you do use GradFun2DB, make sure you use AQ too. Otherwise, that dither is going to disappear since x264 normally decimates that.

Wouldn't you use --no-dct-decimate for that?

[moviefan]

Does --no-dct-decimate reduce banding at the cost of not zeroing near-zero coefficients?

[egrimisu]

i have the same problem when encoding anime's, tried aq form 0.5 to 1.5 with psy-rd on and off but the banding is still visible. i do not use GradFun2DB anymore since i realised that if the source is very detailed and blurym,GradFun2DB ruins the picture. But if the source is clean sharp and witout grains the result is excelent

[OvejaNegra]

I tested and tested with some animes (hellsing right now) and AQ produces ringin on the lines when the picture has dark areas or large flat areas.

The banding comes from using VagueDenoiser or frfun3b. But those filters actually kill all the noise in the way i want. But produce banding.
After the encoding, the bading is even higher.


Any suggestions for noise filtering (fft3d behaves allmost like Vaguedenoiser but is more agressive for me).
Please some sugfestions for x264, for example, trellis or not? what is trellis? and Psy-RD?

Some time ago i remember people telling that 16 bframes is faster and reccomended (x264 wont use 16 anyway but it has more freedom to choose). but now i read that is reccomended 6 and the second mode of adaptive b-frames. Why?

thanks

[Sagekilla]

Regarding the question earlier about --no-dct-decimate: No, you don't use that to prevent x264 from obliterating dither. You use AQ for that. Turning off DCT decimation actually increases the bitrate by a significant amount for practically no benefit.