x264: Constant quality vs constant file size

[ArtDen]

What case would be better for best quality of the resulting video if I want to encode 2-hour average dynamic 720x384 film into 700-1000 Mb file? Is there difference in quality of two equal sized files which are encoded whith CQ and CFS?

PS: all quality options are setted to maximum. CFS method works with 2-passes of course.

[check]

2pass gives superior output to a identically sized 1pass output.

[TechMage89]

If you are storing it on the hard drive (or any other medium with high transfer speeds) and don't need predictable file size, CQ is what most people prefer.

CQ doesn't choke on high-motion, high-detail sequences (it just increases the bitrate for that sequence), so nitpickers can't complain that the fast-motion scenes are very blocky (or blurry). On the whole, CQ provides a consistent level of quality across the video, which many people find more natural-looking.

The main drawback to CQ is that both the overall file size and the maximum bitrate are unpredictable, so it doesn't work well on any storage medium slower than about 30% of the uncompressed bitrate (because, depending on settings the bitrate could get that high (it could be higher, but it would not be worth it))

@check: that's only true if you are using average bit rate (abr) CQ is different entirely.

[ArtDen]

Is your assertions "CFS is better than CQ" confirmed by real testing of these modes (with comparing PSNR value for example) or it's only private opinion?

[foxyshadis]

"Better" is too ambiguous to ask for proof of. But based on the human psychovisual system, average brightness still scenes are by far the most important, and fast-moving scenes can suffer quite a bit of damage before it becomes obvious. All two-pass systems exploit this, as does x264's crf mode. Single-pass bitrate can only approximate it with its imperfect information. That's why crf is generally superior to cq unless you really need superb high-motion quality (screenshots, huge hdtv, whatever).

Better considering psnr and ssim, usually not, unless you also weight them for motion/brightness/etc. But it's also usually small enough that it's not statistically significant; psnr differences aren't usually visually reliable unless they're more than 2dB.

[Manao]

All two-pass systems exploit thisNo. x264 uses only the complexity of the scene, without any nothing of luminosity or motion. It will raise the quantizer on complex scenes, lower it on simple ones. That is mostly done for coding efficiency purposes ( ie, PSNR at a given filesize ) : Raising the quantizer on a complex scene allows to lower it on a lot of simple scenes.

Since complex scenes tends to hide artifacts better than simple scenes, the side effect is that visually the quality does seem to be somehow constant with CRF.

CRF is more efficient than CQP only if you consider the final size.

[DocAliG]

No. x264 uses only the complexity of the scene, without any nothing of luminosity or motion.


Mmmmh.That might be somehow interesting to consider those parameters as well.... Regarding the human visual system..... :p

[foxyshadis]

No. x264 uses only the complexity of the scene, without any nothing of luminosity or motion. It will raise the quantizer on complex scenes, lower it on simple ones. That is mostly done for coding efficiency purposes ( ie, PSNR at a given filesize ) : Raising the quantizer on a complex scene allows to lower it on a lot of simple scenes.
Of course, that's exactly what I meant, in another way. 2-pass is visually better for most people than cq at a given file size because it allows higher quality in the frames they'll be able see more detail in, even though the codec only thinks in terms of complex and simple frames. (If people could take in every frame equally, no matter how fast it flashed by, it would be a bad tradeoff for the codec to make no matter how much it saved; that's why I brought up hvs. It's the ultimate underpinnings of what the codecs do.) There are exceptions, like an unstable camera panning over a field, that'll be considered complex and get a raised quant even though to the eye there's not much going on, but in movies the correlation usually holds. And of course exceptions because some people have amazing eyes or use it for more than just watching at normal speed.

I only mentioned brightness because I was talking about human vision, even though it's AQ that exploits that (among other things) and not the svn x264. Or rather, it's mostly xvid's AQ and kopernicus's patch. =p

[akupenguin]

CRF is more efficient than CQP only if you consider the final size.
You *always* consider the final size. Because if size didn't matter, you wouldn't be compressing it. And how else would you define efficiency, anyway?

[ArtDen]

Yes, 2-pass better than constant quality. I tested two files PSNR and was surprised: 2-pass gives better quality (slightly larger PSNR and slightly more details on screen with 4x zoom). But it advantage is very small with quality=25 and nearly insignificant with quality=21. Also SSIM test shows identical values for both method. So if file size doesn't matter, constant quality is quite usable.

[delacroixp]

I've been using AutoMKV with the H264 (=> MKV container) ConstantQuality-CRF profile.

I used AVInaptic on an episode of 'Band of Brothers' with the following Results (http://forum.doom9.org/showpost.php?p=973478&postcount=1469) ... which would indicate that although I used Q23... the Average Quantizer was 25.5 with a swing between Q10 and Q34... how wierd is that ???


IEEE Xplore (http://ieeexplore.ieee.org/Xplore/login.jsp?url=/iel5/10533/33339/01577167.pdf?isnumber=33339&arnumber=1577167)

PSNR control for GOP-level constant quality in H.264 video coding
De Vito, F.; De Martin, J.C.
Signal Processing and Information Technology, 2005. Proceedings of the Fifth IEEE International Symposium on
Volume , Issue , 18-21 Dec. 2005 Page(s): 612 - 617
Digital Object Identifier 10.1109/ISSPIT.2005.1577167

Summary: For some multimedia applications, like storage or surveillance, it may be desirable to encode the content at constant quality level; the encoder should be aware of media content and set suitable parameters. In this work, we analyze the case of video. We propose an algorithm to adapt the quantization, to obtain constant PSNR. This algorithm selects each frame's quantizer level by comparison with the average of PSNR over N preceding frames, and modifying the quantizer accordingly. This system proved to obtain the target average with a variance in the order of 0.3 in most of the cases. Furthermore, the application may require a per-GOP constant PSNR, like in semantic-based video coding. To obtain this effect, we propose a method to track a given PSNR pattern. The proposed algorithm is able to achieve the target PSNR for each single shot, with a convergence time of only few frames after shot boundary.


According to this article on IEEE Explore a modified Quantizer Strategy is desirable...

What gives... I want a real quantizer... all the profiles (Sharktooth, etc... seam to support Variable Quantizer settings).


:) :D :eek:
Pascal

[nurbs]

Why don't you do a constant quantizer encode then?

[weaver4]

The problem with two-pass with a set filesize is that you have to set the filesize. You are much more likely to pick a filesize that is too small or too big. For example you might have a movie that 1hr 30 minutes long so you decide 900k might be the right value (0.21 bits per pixel). Well half the time that will be OK and half the time it will be not good enough. But if you pick a constant quality value and then encode you don't need to guess.

I guess you could encode once in CQ to get a filesize and then do a two pass encode to the same filesize and then if you look really, really, really hard you might see a difference between the two pass and single pass.

So IMHO you are much more likely to get a "poor quality" movie by picking a filesize on your own than running CQ.

[weaver4]

We propose an algorithm to adapt the quantization, to obtain constant PSNR. This algorithm selects each frame's quantizer level by comparison with the average of PSNR over N preceding frames, and modifying the quantizer accordingly.

Somebody correct me if I am wrong, but I don't see that working very well where scenes go from inactive to very active in a single frame. Like: two men are talking and then a bomb goes off with fire, wood, bricks and body parts flying everywhere.

[smok3]

so crf in x264 is basically constant psnr? or...?

[buzzqw]

a simply but effective definition by check http://forum.doom9.org/showthread.php?p=881943

BHH

[akupenguin]

You don't want constant PSNR. I can't think of any application where it would be desirable.

Ratecontrol targets can be order on a scale of constant bitrate to constant PSNR, as follows. The visually optimal mode is somewhere in the middle, and is definitely not below CQP. Of course, these are not the only possible modes, there's a whole continuum of others in between, as well as different algorithms to implement these targets.

CBR
ABR
CRF/2pass
CQP
CPSNR

(CRF and 2pass are at the same point on the CBR-to-CPSNR continuum, because they have the same target bitrate distribution. 2pass just ends up slightly closer to that target.)

[smok3]

ok, so CRF is 'simply' like 1pass ABR with no bitrate limits?

[delacroixp]

Why don't you do a constant quantizer encode then?

I guess you're right... eventhough check gives a beautifull description with the advantages of CRF on megui/x264 ABR question (http://forum.doom9.org/showpost.php?p=882928&postcount=9).
However, I think there are very real concerns about CQ-CRF as pointed out by weaver4 above and omion x264 constant quality first pass (http://forum.doom9.org/showthread.php?t=101551) back in 2005.

I suppose, like all encoding... there is no single solution and only by becoming more expert will we get closer to the Holy Grail of Encoding.


:) :D :eek:
Pascal

[akupenguin]

However, I think there are very real concerns about CQ-CRF as pointed out by weaver4 above and omion x264 constant quality first pass (http://forum.doom9.org/showthread.php?t=101551) back in 2005.

I don't see any concerns about CRF... omion only posted concerns about CQP, which were answered by implementing CRF. weaver4 posted concerns about CQP and 2pass, for which his solution was to use CRF. What's not to like about CRF?

[PuzZLeR]

If I may continue this thread after a week. (Sorry, had to wait the 5 day period, which is probably a "newbie impatience" control mechanism.)

I'm a FIRM believer in 2-pass over the 1-pass quality modes, given the same file size, in terms of quality. I guess, I'm basing it on my eyesight, but have noticed a bit of a difference.

However, I'm in agreement with weaver4. The 1-pass modes determine a proper file size, which is near impossible to guess without them. I believe in running one of these passes just to get an accurate file size so you won't under/over encode with 2-pass in your final encode.

But I write to share a tip for those that are in this belief, but don't want to wait the long AVC encode times for testing.

Use DivX quantizers, which are excellent at determining a quality level and bitrate needed. Just run a quick pass in fast mode and use that final bitrate/file size value as an index to your 2-pass encoding for AVC.

You will find they are proportional to each other if you test them both and with a bit of experience you will find your "sweet spot" for what DivX determines, and what to input into AVC.

Hope it helps.

PS: Speaking of DivX, greetings to two "old" friends, and cool freaks, from the DivX forum: weaver4 and delacroixp. :-)

[check]

Instead of encoding the whole first pass, just encode a 1 or 2% selection with the avisynth command selectrangeevery(period,number) (ie sre(600,6) for 1%).

[PuzZLeR]

Hey check, I agree with you when you mention that 2-pass mode is superior given an equal file size.

But I would assume here that picking out a handful of frames to base a conclusion, even if they are scattered rather randomly, may yield a high error rate when it comes to video.

Video is a rather erratic beast that can change dramatically at any point.

Forgive me if I'm off the boat, since I personally don't use AVISynth, but are you sure you could use a 1% or 2% sample of the entire file as inference for determining quality levels for the whole?

Have you actually tested this? Just curious.

[nm]

It is possible, see this thread: http://forum.doom9.org/showthread.php?t=116773

[delacroixp]

It is possible, see this thread: CRF translated to DABR graph and discussion (http://forum.doom9.org/showthread.php?t=116773)
Awesome tread... pretty meaty...

I started encoding 2-pass with Dr DivX 1 and then discovered 'constant-quality 1-pass' with Dr DivX 2 which offered so many advantages... I avoided 2-pass like the plague...

Now with 1-pass CQ-CRF it's an entirely different ballgame... there is no '1 perfect encode' and more passes can actually improve upon an already excellent effort...
However, I tried the 2-pass Insane profile (AutoMKV) with an episode of Band of Brothers @ 'original' resolution and at the same bitrate as produced by Q22 CQ-CRF...
The results: ... my fps dropped from 6 fps downto 2 fps and combined with 2-pass... the entire encode took 6 times longer with no significant difference.

I'm sure there are 'ways and means' including filters... to get some pretty awesome results... possibly even getting Q20 results with a Q22 filesize... but I just don't know enough to achieve it myself...


:):D:eek:
Pascal