Yet another one-pass vs. two-pass question

[JeanMarc]

I am no expert, but I have tried to decide if I would use a one-pass or two-pass process for xvid encoding. I am posting this in case someone wants to confirm, critique, or bring a different point of view.

I am generally using ffmpeg libxvid (ffmpeg version N-36193-gf514695, Copyright (c) 2000-2011 the FFmpeg developers
built on Dec 26 2011 17:50:37 with gcc 4.6.2).
I ran ffmpeg with a short clip consisting of four random subclips of 100 frames from the final video I want encode, with -vcodec libxvid -vtag XVID. Here are the three sets of video options I used:
(1 pass) -b:v 607K -g 240 -trellis 2 -mbd rd -flags +mv4+aic
(2 passes) -b:v 635K -g 240 -trellis 2 -mbd rd -flags +mv4+aic
(1 pass) -qscale 6 -g 240 -trellis 2 -mbd rd -flags +mv4+aicI then tried to evaluate the quality using avisynth compare() and SSIM() against the original clip, while keeping an eye on the actual bitrate, and execution time.
Filesize (B) PSNR (dB) SSIM (%) Exec time(s) Bitrate(kbps)
1-pass bitrate 1,294,604 43.52 80.90 66 607,242
2-pass bitrate 1,261,388 43.48 80.99 128 591,482
1-pass fixed quant 1,220,306 43.44 81.13 66 571,972
Running the same process with different clips gives in fact slightly varying results with a very small advantage to either the 2-pass fixed bitrate or the 1-pass fixed quantizer. But they are always so close to each other in terms of quality vs bitrate (within a few percents) that it seems meaningless to try to name a consistant winner.

However, the two-pass encoding taking almost twice as long as the one-pass methods, I would certainly discard it.

Given the often published generic recommendation in favor of always using two passes, does that make sense?

[LoRd_MuldeR]

Here are a few remarks:

1. Objective metrics, like SSIM (or even worse: PSNR), can be very misleading. There are enough examples where "psychovisual" optimizations clearly improved the subjective quality, but worsened the objective metrics at the same time.

2. A mistake often done by people who try to compare different settings is using a target bitrate that is much too high. If you pick a bitrate at which even the setting that is supposed to be the "worst" gives "good" output, then the settings that are supposed to be "better" simply can't look much better anymore. Using a target bitrate that is too low is bad too - if all settings just look "horrible", you can't conclude much either.

3. Remember that 2-Pass mode is mainly intended to be used when you need to hit a fixed (pre-defined) file size. You can't do that with fixed quantizer mode. Well, you can try different quantizers until you find the "right" one. But this would usually take much more time than running a 2-Pass encode.

[Dark Shikari]

For long, relatively uniform complexity videos, 2-pass's advantage over 1-pass is lower.

[JeanMarc]

Thanks for the comments.
1. Objective metrics, like SSIM (or even worse: PSNR), can be very misleading. There are enough examples where "psychovisual" optimizations clearly improved the subjective quality, but worsened the objective metrics at the same time.
I have been using primarily SSIM to define what quality I want. I am really struggling when I look at two videos and try to decide which one is better, unless the difference is large. Looking at individual snapshots is even more meaningless. And my understanding is that the SSIM development team was attempting to take into account the way visual perception works, making it way superior to PSNR. Now, if I still want to somewhat automate the quality optimization, I would be more than happy to use better measurement methods, if anyone cares to suggest one.
Well, you can try different quantizers until you find the "right" one. But this would usually take much more time than running a 2-Pass encode
It doesn't take that long for me. My objective is to reach a good quality at the smallest bitrate. I wrote a program that concatenates a few clips from the video to create a small test clip, to encode it and to run an iteration to reach the SSIM I have selected -- generally between 81% and 86%, depending on the quality of the original. It converges very quickly because I have an estimate of the slope of the bitrate vs ssim (or qscale vs ssim). It may take less than 5-10 minutes to run, while one pass on the whole video can take between 2 and 4 hours on my PC. I then know what bitrate or qscale I should pick to get that quality, what size the video will be and how long it will take (approximately).
2. A mistake often done by people who try to compare different settings is using a target bitrate that is much too high. If you pick a bitrate at which even the setting that is supposed to be the "worst" gives "good" output, then the settings that are supposed to be "better" simply can't look much better anymore. Using a target bitrate that is too low is bad too - if all settings just look "horrible", you can't conclude much either. I think I can avoid that issue with the method described above. It is easy to see if you are near the bottom or near the top when you look at a couple of points on an SSIM vs bitrate curve. When that happens, I move my quality target (SSIM) either lower, or higher.

[LoRd_MuldeR]

I have been using primarily SSIM to define what quality I want. I am really struggling when I look at two videos and try to decide which one is better, unless the difference is large. Looking at individual snapshots is even more meaningless. And my understanding is that the SSIM development team was attempting to take into account the way visual perception works, making it way superior to PSNR. Now, if I still want to somewhat automate the quality optimization, I would be more than happy to use better measurement methods, if anyone cares to suggest one.

PSNR and SSIM (and most other objective metrics) follow the same basic idea: They assume the original video has "perfect" quality and any deviance from the original is an "error" that causes quality degradation. So quality is actually measures as the opposite of "error". SSIM only differs from PSNR in the way it calculates/weights the error. And indeed SSIM is commonly known to give a better prediction of the "perceived" quality than PSNR does. But this doesn't mean SSIM is an ideal quality measure. Most important, some "psycho-visual" optimizations, that clearly improve the "perceived" quality, will hurt SSIM (and PSNR too). Sometimes you need hard numbers to measure quality, e.g. to tune certain aspects of a video encoder. But for judging the overall quality, you should trust your eyes first. If you can't see the difference, it is probably too small to worry about. Either that, or you testing methodology is wrong...

It doesn't take that long for me. My objective is to reach a good quality at the smallest bitrate. I wrote a program that concatenates a few clips from the video to create a small test clip, to encode it and to run an iteration to reach the SSIM I have selected -- generally between 81% and 86%, depending on the quality of the original. It converges very quickly because I have an estimate of the slope of the bitrate vs ssim (or qscale vs ssim). It may take less than 5-10 minutes to run, while one pass on the whole video can take between 2 and 4 hours on my PC. I then know what bitrate or qscale I should pick to get that quality, what size the video will be and how long it will take (approximately). I think I can avoid that issue with the method described above. It is easy to see if you are near the bottom or near the top when you look at a couple of points on an SSIM vs bitrate curve. When that happens, I move my quality target (SSIM) either lower, or higher.

Your method seems to be a workaround for the lack of something like x264's CRF mode in Xvid. You'd need a mode that can give the same level of quality (roughly) for different source and use "as many bits as needed". Still, the purpose of 2-Pass mode is getting the best possible quality out of a predetermined target file size (or target average bitrate). 1-Pass mode cannot deliver this, because it cannot "look into the future" - it only knows the frames it has processed so far. 2-Pass mode "knows" the whole clip, because the second pass can rely on the stats file from the first pass. If, for example, the movie has a "high complex" scene at the end while the rest is relatively "low complex", then it of course makes sense to move more bits towards the end of the movie. At least if you want to achieve a consistent quality throughout the whole movie. 2-Pass mode can do this fine, but 1-Pass mode can not (or only in a very limited way) - for obvious reasons. But, as Dark Shikari said, the more uniform the "complexity" of the movie is, the less 2-Pass mode will be able to show its advantage (simply because the defects of 1-Pass mode become less meaningful).

[JeanMarc]

Thanks a lot for the insight

[henryho_hk]

.... generally between 81% and 86%, depending on the quality of the original.

81% ~ 86% SSIM?

Why not something like MSP@6, VHQ@4 and CQ@3 with the custom matrices EQM v3 LR or EQM v3 HR.