how to predict quaity using x264

[MarcioAB]

@MarcioAB
In case it isn't obvious why graph A is ridiculous:
It says that lossless compression at 0.3 bpp is low compressibility? As is CRF28 at 0 kbps?

Humm, that open up another perspective (example):

Assuming lossless compression is represented by CRF=0 (the X-axis), then assume a lossless compression of film1 will get 2.0 BPP and a lossless compression of film2 will get 5.0 BPP.

We can say that film1 is High Compressible but film2 is Low compressible. So, the division curves should touch the X-axis.

On the other side, anything finishing near 0 BPP is very-High compressible, so the High/Mid division curve must be asymptote to Y-axis like this graph.
Does it make sense ?

http://pws.prserv.net/marcioab/compressibility3.gif

EDIT: The division curves are asymptote to X-axis on CRF=0, but maybe Lossless is something higher than 0, lets say CRF=5 and that is the reason why the division curves touch the "X-axis" (so the X-axis in fact is CRF=5). Make sense ?

[nurbs]

The division curves are asymptote to X-axis on CRF=0, but maybe Lossless is something higher than 0, lets say CRF=5 and that is the reason why the division curves touch the "X-axis" (so the X-axis in fact is CRF=5). Make sense ?

I think if you enter crf 0 it automatically encodes constant quantizer. Only 0 can be lossless in AVC.

[foxyshadis]

that's what I always use here. First pass in CRF @ 19, then second pass with a bitrate. Doing only 1-pass CRF encodes is too unpredictable in terms of file size

In that case you could probably save yourself a lot of time by first running DDogg's tool to see how large CRF 19 is likely to be.

Of course, if you run it while you're asleep or at work, it really doesn't matter how much time is 'wasted'.

[mitsubishi]

Of course, if you run it while you're asleep or at work, it really doesn't matter how much time is 'wasted'.

Hmm time maybe, but wasting all that energy on a regular basis will make Al Gore cry.

[jellysandwich]

In that case you could probably save yourself a lot of time by first running DDogg's tool to see how large CRF 19 is likely to be.

Of course, if you run it while you're asleep or at work, it really doesn't matter how much time is 'wasted'.

little lost here, but where might i find ddoggs tool?

[SealTooGreat]

That's CBR (i.e. ABR with one or more of: VBV, low --ratetol, or low --qcomp).
Lax ABR (i.e. --ratetol inf) tries to be CRF. It adjusts the RF value periodically to approximate the given filesize, but only the first few scenes have really variable quality. After that the variations in RF are progressively damped. ABR --ratetol inf still isn't as good as CRF, but there isn't such a huge difference. Otoh, --ratetol inf not only compromises between ABR and CRF in quality, it also compromises in filesize accuracy, being as much as +/-10% off.
Could you explain how does ABR --ratetol behave with low/mid values?

[microchip8]

In that case you could probably save yourself a lot of time by first running DDogg's tool to see how large CRF 19 is likely to be.

Of course, if you run it while you're asleep or at work, it really doesn't matter how much time is 'wasted'.

Well, I don't think there's DDogg for Linux and I haven't used Windows in over 6 years now (I also don't have XP here).

As I wrote somewhere else in another thread, I want to be able to squeeze up to 5 (usually 4) encodes on one 4.7 GB disc (still don't have a dual-layer DVD here) and I usually scale down my DVDs to 624x384 or 608x384, and use as bitrate for the second pass somewhere between 1050 and 1150 kbps

Only a few days ago, I encoded Sin City (the Uncut version) with a bitrate of 1060 kbps (2-pass encoding) and I couldn't tell the difference between the DVD and the encoding. For me, always doing a 1-pass CRF encoding is just too unpredictable when you want to take file size into account

[foxyshadis]

little lost here, but where might i find ddoggs tool?

This thread (http://forum.doom9.org/showthread.php?t=116773), tyee's tool (http://tplay.lunarpages.net/tiny/eFLiN) is near the end of it. (Several versions throughout the thread.)

froggy, if you almost always end up doing two pass, why not use a normal turbo first pass instead? Or if you use crf to proportion out the bitrate each movie is going to have, you can still get as good of an estimate using 5% and often even 2% of a movie, as outlined above. If Delphi works on wine, that gui might. Admittedly, if not, it'd probably be too much trouble to work the results out manually each time.

[microchip8]

@foxyshadis

I always do turbo=2 for the first pass, it takes me about 30-50 minutes (depending on the movie) to finish the first pass. The second pass, again depending on the movie, takes ~3-4 hours to finish, which is acceptable for me

I guess me always doing a 2-pass encoding has also to do with my mindset, since I'm accustomed to do a 2-pass encoding when I used Xvid in the past (and I used it for many years). At the moment, I'm happy with a 2-pass encode for H.264 so there's no real need for me to change anything :)

[foxyshadis]

No, I totally agree with you that 2-pass is the best way to reliably maximize the quality in a restricted space, like dvds. I just misunderstood and thought you were doing a full, not fast first pass. Sorry!

[MarcioAB]

How to predict quality using x264 ?

Just one idea based on the graph below:

http://pws.prserv.net/marcioab/compressibility4.gif

workflow: (just one idea, no tested at all)

1. encode film with any reasonable CRF and find BPP.
2. Plot this (CRF, BPP) pair in the graph.
3. By this point, plot your film curve, parallel to the guide curves (division curves)
4. Find your new (CRF, BPP) pair, more or less in the middle of the good-quality "work area".
5. You have BPP for the size estimation and CRF for the encode.

Obs: This graph has just example values. Need to collect as many (CRF, BPP) pairs as possible to make a real one.

[MarcioAB]

A graph comparing 3 films:

- Signs (Mel Gibson): the film I found to be most compressible.
- Casablanca (black&white): the film I found to be less compressible.
- Madagascar (anime): To compare with Signs.

The black rectangle is the point where I found image quality became bad for my taste.

http://pws.prserv.net/marcioab/compress2.png

Graph is an open source application found here (http://www.padowan.dk/graph/)
The graph data I used can be found here (http://pws.prserv.net/marcioab/crf.grf)

"Quick and dirt" encodes with MeGUI ( --crf nn --analyse none)
First 3000 frames of each film, right after the credits.

"Quick and dirt" Avisynth script like:
LoadPlugin("d:\PRGM\DGDecode\DGDecode.dll")
MPEG2Source("VTS.d2v").trim(4086,7085)

One can check his values against those in the graph using "Insert a point series".

Any comparison is welcome. I learn here that "Bad Boys 2" is also a low compression film but I do not have it (would be nice to check if it is even lower than Casablanca).

[MarcioAB]

Comparing const-Quality (--crf) with const-Quantizer (--qp) both in the same bitrate (or BPP) regard exponential-SSIM ( 100^SSIM, kind of zoom over SSIM )

http://pws.prserv.net/marcioab/ssim.png

"Quick and dirt" encodes of "Signs" film (3000 frames), but the same parms for both modes.

For this case, --CRF is much better than --QP in lower bitrates and trending to became equal at high bitrates.

[MarcioAB]

Another view (closer to the central "work area") using CRF 21 and 0.2 BPP as axis.
Included an interlaced film (The Promise): Reasonable fast action but I guess the reason for such low compressibility (even with --interlaced parm) is the interlace itself.

The pair (CFR=21, BPP=x) with subme=5 (to be faster) seems to be good to identify the relative compressibility between films.

http://pws.prserv.net/marcioab/compress3.png

[MarcioAB]

http://pws.prserv.net/marcioab/ssim_bpp1.png

A better view, now SSIM in Y and BPP in X and each discrete point correspond to an integer CRF.
The limit in the right (higher CRFs) are where x264 BPP match the original film BPP. In the left is the behavior near the "work area".
The more toward left the CRFs are, the more compressible is the film.
The smaller the distance between CRFs in the same curve, the better the quality is retained during compress.

"Eragon" is a very compressible film and retain quality during compress.
"Promess" has low compression and does not retain very well quality during compression.
"Casablanca" is bit more compressive than "Promess" but also does not retain quality during compression as "Promess".
"Signs" (movie) and "Madagascar" (anime) behave very much similar.

A close look into the graph varying x264 parms reveal some interisting points (next post).

Marcio

[MarcioAB]

http://pws.prserv.net/marcioab/ssim_bpp2.png

The 2 blue curves are "extrema" compressions of the same film (Eragon in the case).
The higher is using the Insane profile (adjusted to CRF) and the lower using the default profile (almost no parms).
The process speed for the higher curve was near 8 fps and for the lower 60 fps. Quality (SSIM) and compression (BPP) are in the graph.

Varying x264 parms it is possible to find parallel curves between these 2 with corresponding speeds. Follow the most significants (displayed in the graph):

1) Insane profile with subme 6, ref 3, bframes 3: speed 15 fps and quality/compression close to full-insame.

2) Insane profile with subme 6, ref 1, bframes 3: speed 19 fps and quality/compression close to (1).

3) Insane profile with subme 5, ref 2, bframes 3: speed 24 fps and quality/compression close to (2).

4) Insane profile with subme 5, ref 1, bframes 1: speed 30 fps and quality/compression close to (3).

5) subme 5, ref 1, bframes 1 --analyse p8x8,b8x8,i8x8 --8x8dct: speed 43 fps and quality/compression close to (4).

6) subme 5, ref 1, bframes 1 --analyse i4x4 --8x8dct: speed 51 and quality/compression not so close to (5).

I tested dozens of other combinations an check them against the graph and the profile above are the best. Look forward to see if I forgot something.

obs: as CRF increase the speed reduce somehow in all curves.

Next task is to proof (or not) how much and where CRF is better than 2-pass.

Marcio

[MarcioAB]

http://pws.prserv.net/marcioab/signs.png

oPSNR and SSIM comparison of CRF, Qp, 2Pass and 1Pass with low compression (almost no parms) and medium compression (Insane profile with subme 5, ref 1, bframes 1) over 3000 frames of SIGNS. Obs: SSIM was scaled to 60-380(1-SSIM) to fit near oPSNR in the graph.

Recomendations:

Low bitrate = up to 0.2 BPP for lite compression parms or 0.15 BPP for heavy compression parms
CRF is better (SSIM: constant higher and oPSNR: equal Qp).
obs for 2Pass: SSIM in the middle and oPSNR lower than all.

Medium bitrate = from Low bitrate up to 0.4 BPP
CRF better on lower edge and Qp better on higher edge.

CRF: SSIM: from higher to equal. oPSNR: from equal to lower.
Qp: SSIM: from lower to equal. oPSNR: from equal to higher.
2Pass: SSIM: same as Qp. oPSNR: lower than all.

High bitrate = from Medium bitrate up to 1 BPP
Qp is better (SSIM: equal for all and oPSNR constant higher than CRF)
obs for 2Pass: oPSNR from lower to equal Qp

Very High bitrate = above 1 BPP
Qp or 2Pass ( SSIM and oPSNR are almost equals)
obs for CRF: not available.

[Manao]

oPSNR and SSIM aren't suited to compare different rate control algorithm. They should only be used to compare short sequences.

[MarcioAB]

oPSNR and SSIM aren't suited to compare different rate control algorithm.They should only be used to compare short sequences.

The way I understood H264 so far is that in the end it is the selection of coefficients (Q) to be used to squeeze the difference between the real frame and the (motion) estimated frame.

So for the exactly same short sequence (as is the case), the comparison should be how much oPSNR+SSIM varies regarding the chosen Q selections (made by the Rate Control in some caser or by myself in others) and the efficiency of the (motion) estimation engine.

I would like to further understand why oPSNR+SSIM can not be used to compare different Rate Controls.

Thank you

[Manao]

It can't be used to compare rate controls because neither PSNR nor SSIM have an absolute meaning. For example, PSNR 42 doesn't amount to the same quality for two different sequences. So, when merging PSNR/SSIM scores of two different sequences ( which you do when you compute PSNR/SSIM of a whole movie ), you end up with a number that has lost almost its meaning.

[MarcioAB]

If I correctly understood, the issue is to use PSNR/SSIM to analyze more then 1 "sequence" ? In my case the extracted segment is only 3000 frames long, but for sure there are many "sequences" there.

But then, if analyzing "merged sequences" with PSNR/SSIM is the issue, it will be an issue even if analyzing it with just one single Rate Control.

So, when to use PSNR/SSIM ?

Thank you.

[Manao]

With a single rate control, i.e if you test quality settings, it's possible to use a whole video, because usually those settings have the same effect on the PSNR/SSIM of all the frames. So, for example, if you compare subme 5 with subme 7, you'll see that the PSNR of all the sequences of your video raises by something like 0.2/0.3 dB, and so the single PSNR figure you get at the end also raises by 0.2/0.3 dB.

However, if you were playing with a setting that would raise the quality of some scene, and reduce the quality of others, it would not be good to use PSNR/SSIM. With x264, such settings are all related to rate control.

Said otherwise, if you consider a video made with two sequences. With constant quantizer, the first one gets a PSNR of 42 and the second a PSNR of 41. With a CRF encoding, the PSNRs are 43 and 40.5 ( for the same size ). Can you say which rate control is the best ?

BTW, the same happen on single frames whenever you use adaptive quantization. That can be seen as a rate control algorithm on a single frame, and it tends systematically to lower the PSNR, while drastically improving the visual quality. That's another case where PSNR/SSIM are ill suited.

[MarcioAB]

Can you say which rate control is the best ?
Thank you to point that. You are all correct.
Btw, a superficial analysis on 3000-frames sequence (SIGNS) processed with 3 rate controls (Qp, CRF and 2Pass) with almost same resulted bitrate, almost same mean PSNR, with low motion (in the begining) and high motion (in the end), indicated better PSNR results for Qp in the high motion scenes and better PSNR results for CRF / 2Passes in the low motion scenes.

BTW, the same happen on single frames whenever you use adaptive quantization. That can be seen as a rate control algorithm on a single frame, and it tends systematically to lower the PSNR, while drastically improving the visual quality. That's another case where PSNR/SSIM are ill suited.

May I ask how to use "adaptive quantization" ? (in MeGUI ?).

Thank you.

[Manao]

May I ask how to use "adaptive quantization" ? (in MeGUI ?).You need a build of x264 with the adaptive quantization patch. There are plently of such builds available ( check the sticky )
indicated better PSNR results for Qp in the high motion scenes and better PSNR results for CRF / 2Passes in the low motion scenes.And that is normal, for two reasons :
- the human vision system is less sensitive to details/artifacts in high motion scenes, so it makes sense to reduce their quality
- slightly degrading a very complex scene allows to improve a lot of simple scenes ( because complex scenes have a high bitrate, hence degrading slightly their quality still gives a lot of bitrate to play with in comparison to the bitrate of the simple scenes ).

[Nikos]

From wiki:
SSIM (Structural SIMilarity)
The Structural SIMilarity (SSIM) index is a method for measuring the similarity between two images. The SSIM index is a full reference metric, in other words, the measuring of image quality based on an initial uncompressed or distortion-free image as reference.
The SSIM index is a decimal value between 0 and 1.
0 would mean zero correlation with the original image, and
1 means the exact same image.
0.95 SSIM, for example, would imply half as much variation from the original image as 0.90 SSIM. Through this index, image and video compression methods can be effectively compared.

Is there a mathematical formula to associate SSIM values and quality variation?
For example if i have 0.88 and 0.82 SSIM values what's the difference in quality?

[Manao]

No.

Metrics such a PSNR / SSIM are mostly relevant when used on short sequences. You should try to get, for two different settings, the same PSNR or SSIM. The two settings will have a different bitrate, and the difference between those two bitrate is the only objective figure that has a meaning. 10% more quality means nothing, while 10% less bitrate does mean something.

[Nikos]

Thanks Manao for the reply, you are clear about relevant.
According to wiki:
0.95 SSIM, for example, would imply half as much variation from the original image as 0.90 SSIM
my question is:
0.88 SSIM, for example, would imply ??? as much variation from the original image as 0.82 SSIM.

[akupenguin]

0.88 SSIM, for example, would imply ??? as much variation from the original image as 0.82 SSIM.
It doesn't matter. Sure you can look at the formula for SSIM and say that 0.88 is X times as much variation than 0.82, but what would you do with that result? All you need to know about SSIM is that higher scores are better. Any quantitative comparison should be phrased in units of bitrate, not units of SSIM. Furthermore, you can't derive a change in bitrate from a change in SSIM, because that mapping is not always the same for different video content and different codecs. At best you could come up with a formula specific to some types of content and one codec, and develop an intuition for when to trust it.

[MarcioAB]

And that is normal, for two reasons
I see ... I just proved to myself that America was discovered.

Still intrigued regard those metrics, follow a graph:
http://pws.prserv.net/marcioab/ssim_psnr_frame.png

Horiz is the frames, Vert is SSIM(CRF-Qp) is red and PSNR(CRF-Qp) in blue, scaled to better view.
They both seems similar, despite PSNR seems to be more focused in luminance (dark and light) and SSIM seems to add motion on top that.

Visually inspecting the videos is difficult to judge where each scene is better/worst.
By far the most annoying issue to me is the blockness that seems none of these metrics (PSNR or SSIM) can caught/consider.

[akupenguin]

And that is normal, for two reasons :
- the human vision system is less sensitive to details/artifacts in high motion scenes, so it makes sense to reduce their quality
- slightly degrading a very complex scene allows to improve a lot of simple scenes ( because complex scenes have a high bitrate, hence degrading slightly their quality still gives a lot of bitrate to play with in comparison to the bitrate of the simple scenes ).
More precisely, it's normal because it's what CRF/2pass explicitly try to do. The above reasons are why I wrote it that way.
(Clarifying because otherwise a reader might think that x264 has some generic optimization algorithm that only usually gives such a result.)

[Nikos]

Thanks akupenguin for the useful reply.
I want to know the SSIM formula just because i am a physicist, but i can't find it.

If i want to compare same video, same codec, same bitrate with different settings, the SSIM value does have any meaning?

[akupenguin]

If i want to compare same video, same codec, same bitrate with different settings, the SSIM value does have any meaning?
Yes, but only insofar as it is an approximation of the real experiment, which is: same video, same codec, same SSIM, different settings (but same ratecontrol method), and measure bitrate.

You may ask, how do I pick a SSIM and measure bitrate when x264 has a --bitrate but not a --ssim? Encode at 2 or more bitrates/qps/crfs bracketing your target SSIM value, and interpolate between them. You can save some cpu time at the expense of approximation and complexity, by making multiple encodes for a few of the settings, then fitting your SSIM->bitrate predictor function to those results, then encode the rest of the settings at just one bitrate each and extrapolate their bitrate at the target SSIM.
On the other hand, you can spend some extra cpu time to simplify the process and provide a more complete picture, in the form of an RD plot: Encode each setting at several different bitrates, plot the SSIM vs bitrate of all encodes with a curve between the points of a given setting, and the distance between the curves is the relative goodness of the settings.

I want to know the SSIM formula just because i am a physicist, but i can't find it.
from http://www.cns.nyu.edu/~zwang/files/papers/ssim.html
http://akuvian.org/images/ssim_eqn.png
The equation is defined for a window of a few pixels (originally a gaussian weighted window, but we use unweighted 8x8 blocks because its faster). The SSIM score of an image is the average of the scores of all possible windows in the image.
x and y are the windows in the two frames to be compared.
mu_x is the average pixel value in window x.
sigma_x is the standard deviation of pixel values in window x.
sigma_xy is the covarience.
C1=1e-4 and C2=9e-4 if pixel values are normalized to the range [0,1].

[Nikos]

Thank you very much akupenguin, i feel happy.
I will follow your tips in my experiments encodes.

[Dark Shikari]

By far the most annoying issue to me is the blockness that seems none of these metrics (PSNR or SSIM) can caught/consider.
Use a blocking metric then. MSU's Video Quality program (freeware) has a blocking metric.

[MarcioAB]

Use a blocking metric then. MSU's Video Quality program (freeware) has a blocking metric.

I'm using the MSU tool for PSNR and SSIM frame-by-frame measures, but the "MSU Blocking (MPEG-4) BETA" results does not align even close to my perception of blocky.

To better view the blocks, I'm scaling gamma with avisynth "Levels(0, 2, 255, 0, 255)". To match the same blockness of the source MPEG-2 (in the dark and low motion areas of course) I need almost double the bitrate in x264. I know it is a known issue of x264 ... maybe someday.

[Dark Shikari]

I'm using the MSU tool for PSNR and SSIM frame-by-frame measures, but the "MSU Blocking (MPEG-4) BETA" results does not align even close to my perception of blocky.

To better view the blocks, I'm scaling gamma with avisynth "Levels(0, 2, 255, 0, 255)". To match the same blockness of the source MPEG-2 (in the dark and low motion areas of course) I need almost double the bitrate in x264. I know it is a known issue of x264 ... maybe someday.
Use AQ if you have issues with blocking...

[mav1]

What are you think about this profile for test compression bitrate?

"x264.exe" --qp 18 --bframes 3 --b-pyramid --direct auto --filter -2,-1 --subme 1 --partitions none --me dia --threads auto --thread-input --sar 1:1 --progress --no-psnr --no-ssim --output "file.mkv" "file.avs" --aq-strength 0.5

[Blue_MiSfit]

thread-ro-mancy!

Also, if you're going to post settings, don't post them like that. Post them in the form of a command line!

~MiSfit