Limiting the quality of a "Fast-forward" scene

[gamebox]

Hi everybody. :) I need a small advice about x264 encoding.

I have a Full-HD movie which is (sadly) done using H.263 @ 8Mbps, and I'm transcoding it to X264 @ 5Mbps, aiming to keep quality as close to original as possible by using the most advanced options X264 has to offer. Although I've done this a few times with very good results, the movie I need to process this time needs somewhat "special" treatment.

The issue is that there is a segment, several minutes in length, where movie is using some "fast-forward" like effect - that is, a regular scene is presented at speed several times over normal (many original frames are discarded). H.263 codec used originally goes all "blocky" here, probably limited by VBV, bitrate or something similar, but things on the screen are "happening too fast" for the blockiness to be obvious, and the low visual quality of the scene becomes visible only when the video is stopped. X264, on the other hand, treats this section the same way as the rest of the video, meaning it throws unnecessary high amount of bits to this scene, where the quality is already low, and it's "visual integrity" is not that important too. How can I "correct" X264s behavior so it increases quants or limits the bitrate in some way here, but not in the remaining of the video where short segments of increased complexity normally appear and need to be kept intact? I want to take away bits from that scene and redistribute them throughout the movie, where it really counts. Thanks in advance! :)

My options are:
cabac=1 ref=3 deblock=1:-1:-1 me=umh subme=9 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=32 chroma_me=1 trellis=2 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=0 chroma_qp_offset=1 threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=2 b_bias=0 direct=3 weightb=1 open_gop=0 weightp=2 keyint=300 keyint_min=25 scenecut=40 intra_refresh=0 mbtree=0 bitrate=5000 ratetol=1.0 qcomp=0.60 qpmin=10 qpmax=51 qpstep=4 ip_ratio=1.40 pb_ratio=1.30 aq=1:1.00

[huhn]

you can use a zone.

you can read about it here: https://en.wikibooks.org/wiki/MeGUI/x264_Settings#zones

just change the min QP at that place to 23 or something terrible bad like that.

[foxyshadis]

Aside from zones, you should turn mbtree on. That automatically lowers the quality of unreferenced blocks, while raising the ones that do help. In a blockfest like that, I'd assume most of the blocks won't be referenced by anything.

In the fast zone, you might use a high --nr as well, that'll help make it a little less ugly.

[LoRd_MuldeR]

Actually x264 does lower the quality in "high complexity" scenes. This is called quantizer curve compression, or “qcomp”, and I think it dates back to original libavcodec rate-control. MB-Tree simply expands this idea to the per-block level.

So you could try to adjust the "--qcomp" parameter a bit, if you think the effect of quantizer curve compression (or MB-Tree) is not strong enough for your particular source.

See also:
https://web.archive.org/web/20120910092729/http://x264dev.multimedia.cx/archives/98#more-98

[gamebox]

Thanks for fast replies. :)

I remember zones from Xvid encoding times (good old trick of increasing the quantizer on simple-text movie end-credits ;) ), but I remember I didn't see it in X264 I used. I looked better, and true - that option exists but is for some reason grayed out?! AviDemux 2.6.8 is my software of choice...

MB tree is off deliberately as the material is of "a special kind", focusing viewers' attention to small fast-moving parts of the scene. Now, that was a nice and polite way to describe the ...content, was it? :D

How far do you recommend to go with -qcomp, which direction? I use a single-core machine with this essentially "slower" preset on a lengthy 1080p material, so encoding does take a while and testing options are limited...

[LoRd_MuldeR]

How far do you recommend to go with -qcomp, which direction? I use a single-core machine with this essentially "slower" preset on a lengthy 1080p material, so encoding does take a while and testing options are limited...

If I'm not mistaken, the closer to 1.0 you set "qcomp", the more you go towards constant quantizers - which is the opposite effect of what you want. So you would probably want to lower the "qcomp" value a bit.

See also:
https://en.wikibooks.org/wiki/MeGUI/x264_Settings#qcomp

[F J Walter]

I would generally advise against adjust qcomp because it has far reaching ramifications for encoder efficiency and how well it deals with motion, even short bursts of it.

It sounds like what you really do want is VBV. This can be safely used with CRF to simply provide a maximum rate for scenes of high complexity.

The thing about VBV is it's not just a maximum rate but also a buffer size. If you set the buffer size high enough, then small periods of high complexity won't be affected while longer periods will. So for example if you set the vbv-bufsize to about 3 times the vbv-maxrate, then your video can exceed the vbv-maxrate *by 3 times* for a period of one second (or by 1.5 times for 2 seconds, or by 6 times for 0.5 seconds, and so on) without being affected. Set it to 5 times the vbv-maxrate, for example, and it should only affect very long or very severe periods of high complexity while leaving short normal bursts all but unaffected.

If you're encoding for a particular device, you need to keep the VBV buffer and rate within those supported by the device, but otherwise, you could use VBV to achieve the sort of thing you are talking about.

Keep mbtree on, and use a reasonable rc-lookahead, say 20 or more, for this to work well.

cabac=1 ref=3 deblock=1:-1:-1 me=umh subme=9 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=32 chroma_me=1 trellis=2 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=0 chroma_qp_offset=1 threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=2 b_bias=0 direct=3 weightb=1 open_gop=0 weightp=2 keyint=300 keyint_min=25 scenecut=40 intra_refresh=0 mbtree=0 bitrate=5000 ratetol=1.0 qcomp=0.60 qpmin=10 qpmax=51 qpstep=4 ip_ratio=1.40 pb_ratio=1.30 aq=1:1.00

I'd leave qpmin and qpmax at defaults, to let the ratecontrol do its job, and unless you have a good reason, also leave mbtree on, as that will HELP with your issue not hinder it.