X-QuaSaT - x264 settings quality/speed/bitrate/cpuload analysation tool (download)

[mogobime]

Hi guys,

i coded a small tool to fully automatic change parameters of x264 settings to test them for speed, filesize and the resulting quality.
The tool changes parameters by itself within the ranges you define:

--->>> X-QuaSaT <<<---
[x264 Quality and Speed analyse Tool]

X-QuaSaT is a tool for automatic analysation of speed and quality that
different constellations of x264 settings result in, that shall help you to
find your optimal x264 settings.
The idea is to continuously let it raise the parameters of the x264-settings
- quantization
- reference-frames
- b-frames
- motion estimation
- Psy-rate distortion/trellis
in loop.

All of those settings can (if you want to, depends on your settings)
automatically be tested in all thinkable parameter-constellations with a
bunch of test-clips you put in X-QuaSaT's "VIDEOFILES" folder.
It will create a statistic-file, that shows you the effects of the combinated
parameters to:
- SSIM
- PSNR
- bitrate
- encoding speed
- quantitative usage of B-frames (BF_USED)
- quantitative usage of P-frames (PF_USED)
- quantitative usage of I-frames (IF_USED)
- effective usage of the CPU

It will build the average out of all used test clips and log every
parameter-constellation in one line.

You can also keep the temporary generated videos, to have a closer look on
some of them after having a look to the statistic file.

Other configurable parameters are:
- Pixel motion
- Pixel search range
- Trellis quantization
- Encodingmode
- Adaptive quantization mode
- Adaptive quantization strength

I added a screenshot of the main menue because a picture can say more than a thousand words

http://www4.picfront.org/picture/hDupj8Cdob6/thb/menue-screenshot.jpg (http://www.picfront.org/d/8Q0s)

DOWNLOAD HERE:
http://forum.selur.de/topic193-xquasat-x264-settings-quality-speed-analyse-tool-download-here.html

Greetz to Selur and thanks for his testing-support and suggestions in the time of developing

Would be happy to get some feedback,

Cu

mogobime

[mogobime]

I added a netload-link because some user replied, that he couldn't use the ftp-link. But the ftp-link still works and is faster.

[mogobime]

Changelog 1.6.10 (28.10.2012):

- Added 32 bit x264 support
- CPU-load measurement will now terminate correctly under Windows XP
- some more x264 errors will be detected

[pandy]

Thx mogobime - sounds like lot of work done and new, nice tool created!

[mogobime]

NEW 31.10.2012:

Some diagrams and graphical worked up x-QuaSaT statistics can be viewed at my new website:
http://x-quasat.redirectme.net
The Open Office ods-source tables and the csv-files they were created from are available, too.

[mogobime]

NEWS 02.11.2012:
Pooh, this was some hard work for me and my pc.
By the way, the big trellis comparison with html-tables and diagrams, based on X-QuaSaT statistic is ready on my website!
the diagrams and tables refer to quantizer 19+20 with trellis 0-2 (respectively reference-frames 5-9 and b-frames 3-11).
I marked at least the best three candidates with the help of a estimation matrix depending on quality, speed and bitrate.
Additionally there are bar diagrams showing quality / speed / bitrate of all tested combinations. Combinations that for example achived high quality because of a bitrate higher than average were ignored (they're marked grey in the tables).

I hope somebody has some usage for this.

Cu

mogobime

[mogobime]

Argh! I made a really bad mistake with the html tables and diagrams on the website. They were displayed unreadable inside the frame they were clicked.
Problem is solved now:
http://x-quasat.redirectme.net

[CommonMortal]

Thank you! Very useful! I tested 2 files of my own, from Blue Ray films and the results are similar to yours, although i used less settings (those close to what i use). It appears that from a quality-speed perspective 5 ref+5 b frames is usually the best. Higher b-frames lower the size, but also SSIM. Mor ref frames don't seem to do any good to SSIM.

I used settings similar to Tune Film. Also changed AQ mode to default 1:1, umh to 24, subme 9, Trellis 1.

1st clip (clean source):

x264.exe --crf 18 --ref 5 --keyint 500 --min-keyint 0 --scenecut 40 --bframes 5 --b-bias 0 --b-pyramid normal --direct auto --b-adapt 2 --sync-lookahead 25 --partitions all --8x8dct --no-fast-pskip --me umh --merange 24 --mvrange -1 --subme 9 --cabac --trellis 1 --psy-rd 1.00:0.15 --weightp 2 --aq-mode 1 --aq-strength 1 --nr 0 --deadzone-inter 21 --deadzone-intra 11 --cqm flat --threads auto --sar 1:1 --deblock -1:-1 --psnr --ssim --colormatrix undef

[NR]1 [QUANTIZATION]18 [SUBME]9 [REF]5 [BFRAMES]5 [PSY-RD]1.00:0.15 [ME]umh [MERANGE]24 [TRELLIS]1 [CPULOAD]95 [AvgPSNR]45.367 [SSIM]17.554db [FPS]6.1 [BR]15367 [BF_USED]463 [PF_USED]146 [IF_USED]25 [ENCODINGMODE]crf [AQ-MODE]1 [AQ-STRENGTH]1 [CPUNAME]AMD Phenom(tm) II X6 1090T Processor [CPUCLOCK]3200





x264.exe --crf 18 --ref 5 --keyint 500 --min-keyint 0 --scenecut 40 --bframes 6 --b-bias 0 --b-pyramid normal --direct auto --b-adapt 2 --sync-lookahead 25 --partitions all --8x8dct --no-fast-pskip --me umh --merange 24 --mvrange -1 --subme 9 --cabac --trellis 1 --psy-rd 1.00:0.15 --weightp 2 --aq-mode 1 --aq-strength 1 --nr 0 --deadzone-inter 21 --deadzone-intra 11 --cqm flat --threads auto --sar 1:1 --deblock -1:-1 --psnr --ssim --colormatrix undef

[NR]1 [QUANTIZATION]18 [SUBME]9 [REF]5 [BFRAMES]6 [PSY-RD]1.00:0.15 [ME]umh [MERANGE]24 [TRELLIS]1 [CPULOAD]94 [AvgPSNR]45.362 [SSIM]17.549db [FPS]5.9 [BR]15313 [BF_USED]463 [PF_USED]147 [IF_USED]22 [ENCODINGMODE]crf [AQ-MODE]1 [AQ-STRENGTH]1 [CPUNAME]AMD Phenom(tm) II X6 1090T Processor [CPUCLOCK]3200




x264.exe --crf 18 --ref 6 --keyint 500 --min-keyint 0 --scenecut 40 --bframes 5 --b-bias 0 --b-pyramid normal --direct auto --b-adapt 2 --sync-lookahead 25 --partitions all --8x8dct --no-fast-pskip --me umh --merange 24 --mvrange -1 --subme 9 --cabac --trellis 1 --psy-rd 1.00:0.15 --weightp 2 --aq-mode 1 --aq-strength 1 --nr 0 --deadzone-inter 21 --deadzone-intra 11 --cqm flat --threads auto --sar 1:1 --deblock -1:-1 --psnr --ssim --colormatrix undef

[NR]1 [QUANTIZATION]18 [SUBME]9 [REF]6 [BFRAMES]5 [PSY-RD]1.00:0.15 [ME]umh [MERANGE]24 [TRELLIS]1 [CPULOAD]95 [AvgPSNR]45.366 [SSIM]17.554db [FPS]5.6 [BR]15353 [BF_USED]463 [PF_USED]146 [IF_USED]26 [ENCODINGMODE]crf [AQ-MODE]1 [AQ-STRENGTH]1 [CPUNAME]AMD Phenom(tm) II X6 1090T Processor [CPUCLOCK]3200



x264.exe --crf 18 --ref 6 --keyint 500 --min-keyint 0 --scenecut 40 --bframes 6 --b-bias 0 --b-pyramid normal --direct auto --b-adapt 2 --sync-lookahead 25 --partitions all --8x8dct --no-fast-pskip --me umh --merange 24 --mvrange -1 --subme 9 --cabac --trellis 1 --psy-rd 1.00:0.15 --weightp 2 --aq-mode 1 --aq-strength 1 --nr 0 --deadzone-inter 21 --deadzone-intra 11 --cqm flat --threads auto --sar 1:1 --deblock -1:-1 --psnr --ssim --colormatrix undef

[NR]1 [QUANTIZATION]18 [SUBME]9 [REF]6 [BFRAMES]6 [PSY-RD]1.00:0.15 [ME]umh [MERANGE]24 [TRELLIS]1 [CPULOAD]95 [AvgPSNR]45.361 [SSIM]17.549db [FPS]5.4 [BR]15301 [BF_USED]463 [PF_USED]147 [IF_USED]22 [ENCODINGMODE]crf [AQ-MODE]1 [AQ-STRENGTH]1 [CPUNAME]AMD Phenom(tm) II X6 1090T Processor [CPUCLOCK]3200



2nd clip (more grainy):


x264.exe --crf 18 --ref 5 --keyint 500 --min-keyint 0 --scenecut 40 --bframes 5 --b-bias 0 --b-pyramid normal --direct auto --b-adapt 2 --sync-lookahead 25 --partitions all --8x8dct --no-fast-pskip --me umh --merange 24 --mvrange -1 --subme 9 --cabac --trellis 1 --psy-rd 1.00:0.15 --weightp 2 --aq-mode 1 --aq-strength 1 --nr 0 --deadzone-inter 21 --deadzone-intra 11 --cqm flat --threads auto --sar 1:1 --deblock -1:-1 --psnr --ssim --colormatrix undef

[NR]1 [QUANTIZATION]18 [SUBME]9 [REF]5 [BFRAMES]5 [PSY-RD]1.00:0.15 [ME]umh [MERANGE]24 [TRELLIS]1 [CPULOAD]95 [AvgPSNR]44.473 [SSIM]16.293db [FPS]5.6 [BR]20340 [BF_USED]626 [PF_USED]175 [IF_USED]20 [ENCODINGMODE]crf [AQ-MODE]1 [AQ-STRENGTH]1 [CPUNAME]AMD Phenom(tm) II X6 1090T Processor [CPUCLOCK]3200





x264.exe --crf 18 --ref 5 --keyint 500 --min-keyint 0 --scenecut 40 --bframes 6 --b-bias 0 --b-pyramid normal --direct auto --b-adapt 2 --sync-lookahead 25 --partitions all --8x8dct --no-fast-pskip --me umh --merange 24 --mvrange -1 --subme 9 --cabac --trellis 1 --psy-rd 1.00:0.15 --weightp 2 --aq-mode 1 --aq-strength 1 --nr 0 --deadzone-inter 21 --deadzone-intra 11 --cqm flat --threads auto --sar 1:1 --deblock -1:-1 --psnr --ssim --colormatrix undef

[NR]1 [QUANTIZATION]18 [SUBME]9 [REF]5 [BFRAMES]6 [PSY-RD]1.00:0.15 [ME]umh [MERANGE]24 [TRELLIS]1 [CPULOAD]94 [AvgPSNR]44.474 [SSIM]16.293db [FPS]5.5 [BR]20341 [BF_USED]626 [PF_USED]175 [IF_USED]17 [ENCODINGMODE]crf [AQ-MODE]1 [AQ-STRENGTH]1 [CPUNAME]AMD Phenom(tm) II X6 1090T Processor [CPUCLOCK]3200





x264.exe --crf 18 --ref 6 --keyint 500 --min-keyint 0 --scenecut 40 --bframes 5 --b-bias 0 --b-pyramid normal --direct auto --b-adapt 2 --sync-lookahead 25 --partitions all --8x8dct --no-fast-pskip --me umh --merange 24 --mvrange -1 --subme 9 --cabac --trellis 1 --psy-rd 1.00:0.15 --weightp 2 --aq-mode 1 --aq-strength 1 --nr 0 --deadzone-inter 21 --deadzone-intra 11 --cqm flat --threads auto --sar 1:1 --deblock -1:-1 --psnr --ssim --colormatrix undef

[NR]1 [QUANTIZATION]18 [SUBME]9 [REF]6 [BFRAMES]5 [PSY-RD]1.00:0.15 [ME]umh [MERANGE]24 [TRELLIS]1 [CPULOAD]96 [AvgPSNR]44.471 [SSIM]16.290db [FPS]5.2 [BR]20338 [BF_USED]626 [PF_USED]175 [IF_USED]20 [ENCODINGMODE]crf [AQ-MODE]1 [AQ-STRENGTH]1 [CPUNAME]AMD Phenom(tm) II X6 1090T Processor [CPUCLOCK]3200






x264.exe --crf 18 --ref 6 --keyint 500 --min-keyint 0 --scenecut 40 --bframes 6 --b-bias 0 --b-pyramid normal --direct auto --b-adapt 2 --sync-lookahead 25 --partitions all --8x8dct --no-fast-pskip --me umh --merange 24 --mvrange -1 --subme 9 --cabac --trellis 1 --psy-rd 1.00:0.15 --weightp 2 --aq-mode 1 --aq-strength 1 --nr 0 --deadzone-inter 21 --deadzone-intra 11 --cqm flat --threads auto --sar 1:1 --deblock -1:-1 --psnr --ssim --colormatrix undef

[NR]1 [QUANTIZATION]18 [SUBME]9 [REF]6 [BFRAMES]6 [PSY-RD]1.00:0.15 [ME]umh [MERANGE]24 [TRELLIS]1 [CPULOAD]94 [AvgPSNR]44.473 [SSIM]16.292db [FPS]5.1 [BR]20340 [BF_USED]626 [PF_USED]175 [IF_USED]17 [ENCODINGMODE]crf [AQ-MODE]1 [AQ-STRENGTH]1 [CPUNAME]AMD Phenom(tm) II X6 1090T Processor [CPUCLOCK]3200

[Sharc]

Nice work :)
I would suggest to add the resulting file size in the LOG as this is essential for any comparison.

Edit:
Ooops! I see you record the bitrate in the log. Perfect.

[nm]

Thank you! Very useful! I tested 2 files of my own, from Blue Ray films and the results are similar to yours, although i used less settings (those close to what i use). It appears that from a quality-speed perspective 5 ref+5 b frames is usually the best. Higher b-frames lower the size, but also SSIM. Mor ref frames don't seem to do any good to SSIM.

Since SSIM in your first ref=6/bf=5 clip is the same as in that ref=5/bf=5 encode but bitrate is lower, 6 references achieved higher compression (better quality/bitrate ratio). Therefore you could lower the CRF value slightly and get better quality at the same bitrate.


Overall, comparing different settings at the same CRF doesn't make much sence since both bitrate and quality change between the encodes. Most of the time you can't draw conclusions about which setting is better. Use 2-pass mode for these kind of comparisons instead!

[CommonMortal]

Since SSIM in your first ref=6/bf=5 clip is the same as in that ref=5/bf=5 encode but bitrate is lower, 6 references achieved higher compression (better quality/bitrate ratio). Therefore you could lower the CRF value slightly and get better quality at the same bitrate.


Overall, comparing different settings at the same CRF doesn't make much sence since both bitrate and quality change between the encodes. Most of the time you can't draw conclusions about which setting is better. Use 2-pass mode for these kind of comparisons instead!

Right, that's why i wrote from a quality-speed perspective. I left size aside. I now encode with CRF, so i don't mind about size anymore.

The program unfortunately, unless i missed something, doesn't allow 2-pass comparison.

[nm]

Right, that's why i wrote from a quality-speed perspective. I left size aside. I now encode with CRF, so i don't mind about size anymore.

So why don't you encode with CRF=10 then? Better quality regardless of settings, right? Or how about CRF=1?

The point is that you can't announce setting X better than setting Y based on higher quality at different bitrates. What if setting Y halves the bitrate, in which case you could use a couple points lower CRF to get both higher quality and smaller (or same) file size than setting X.

[CommonMortal]

So why don't you encode with CRF=10 then? Better quality regardless of settings, right? Or how about CRF=1?

Because that would be also regardless of speed and i do care about speed. Also, if you do CRF=10, you might as well not encode at all. There has to be a sense in using lossy format.



The point is that you can't announce setting X better than setting Y based on higher quality at different bitrates. What if setting Y halves the bitrate, in which case you could use a couple points lower CRF to get both higher quality and smaller (or same) file size than setting X.

I don't want to annouce anything. It's just what i understand from these results. You have all settings same except for 2: ref and B-frames. And you wait and see what happens.

What i observed is that higher than 5 b-frames, does help in size but not in SSIM. Same for ref 6.

I would be happier if a ref 6, b 6 was leading to higher SSIM and lower size at the same time, but i didn't see it. Or at higher SSIM at the same size, but didn't see that either. What i saw is that you get same SSIM for less size at best. Luck? Coincidence? Maybe. I will try more clips and see if the patterns repeates itself.

You can draw your own conclusions, i am not trying to convince you. Or you may just say that all this is casual and no conclusion can be drawn, i am not trying to impose my opinion on you.

[Sharc]

Agree, 2-pass would be nice to see in future for comparisons on the basis of equal bitrates/filesizes.

Secondly, the possibility of editing the entire x.264 command line (not looping all the parameters of course) would be useful.

Carry on the good work.

[CommonMortal]

Agree, 2-pass would be nice to see in future for comparisons on the basis of equal bitrates/filesizes.

Secondly, the possibility of editing the entire x.264 command line (not looping all the parameters of course) would be useful.

Carry on the good work.

In fixed size, isn't it rather obvious that the one that can compress more, also gets higher SSIM at the same bitrate? I have already done that watching Vidcoder's log in the past and that's why i had ended up using ref 6, b 6 when i was using 2-pass. It was better than ref 5 b 5.

[Sharc]

In fixed size, isn't it rather obvious that the one that can compress more, also gets higher SSIM at the same bitrate? I have already done that watching Vidcoder's log in the past and that's why i had ended up using ref 6, b 6 when i was using 2-pass. It was better than ref 5 b 5.
I doubt if it is so obvious.
If size counts then we should compare on equal bitrate (aka file size) to investigate the impact (which depends on the source as well) of the other parameters.
For Blu-ray compatibility for example, ref 6 is compliant for 720p only, so I need to "play around" with other parameters to get an impression how they effect the quality. I like the possibility to keep the encodes for visual inspection rather than rely on SSIM only.

[nm]

Because that would be also regardless of speed and i do care about speed. Also, if you do CRF=10, you might as well not encode at all. There has to be a sense in using lossy format.

Ok, so how about CRF=17 and a faster preset? Better quality and higher speed at ~20 % higher bitrate? Wouldn't that make sense if you don't care about size (much).


I would be happier if a ref 6, b 6 was leading to higher SSIM and lower size at the same time, but i didn't see it. Or at higher SSIM at the same size, but didn't see that either. What i saw is that you get same SSIM for less size at best.

That can be just as useful as higher SSIM at lower size. Note that you are looking at sizes here, and yet you claim that size doesn't matter.

You can draw your own conclusions, i am not trying to convince you. Or you may just say that all this is casual and no conclusion can be drawn, i am not trying to impose my opinion on you.

I'm trying to explain that you can't use this method of comparison to decide which setting is actually better for you.

This flaw of comparing settings without fixing neither bitrate nor quality has been discussed numerous times here. I'll stop spamming this thread and crawl back under my stone now. :)

[CommonMortal]

I doubt if it is so obvious.
If size counts then we should compare on equal bitrate (aka file size) to investigate the impact (which depends on the source as well) of the other parameters.
For Blu-ray compatibility for example, ref 6 is compliant for 720p only, so I need to "play around" with other parameters to get an impression how they effect the quality. I like the possibility to keep the encodes for visual inspection rather than rely on SSIM only.

Ok. i am not going to enter subject of what it BR-compliant, it goes beyond my reasoning, you can test these things if you want. What i have seen in the past, is that more B-frames and ref frames, increase compression (reduce size). Imagine you have 2 walls of the same surface (size) to paint and equal amount of paint (bitrate). But one brush (setting), needs less paint to cover well a unit of surface (compresses more). At the end, which wall will be painted better? The one painted with the "good brush" or the "bad brush"? The good one, no?


Ok, so how about CRF=17 and a faster preset? Better quality and higher speed at ~20 % higher bitrate? Wouldn't that make sense if you don't care about size (much).

You could try to see CRF-17 and faster preset, i just don't want to use faster preset, i believe subme 9 is important, because of the psy-rdo and psy aware deblocking that Dark Shikari had implemented.

It's not that i can't try and see what's the effect of one change on settings that i want to choose, can't i?


That can be just as useful as higher SSIM at lower size. Note that you are looking at sizes here, and yet you claim that size doesn't matter.

I am telling you what i see, since you insist on size. If i didn't tell about size, you would then reply "but more b and ref frames, increase compressibility".

I'm trying to explain that you can't use this method of comparison to decide which setting is actually better for you.
This flaw of comparing settings without fixing neither bitrate nor quality has been discussed numerous times here. I'll stop spamming this thread and crawl back under my stone now.


I am trying to see what varies at the variation of 1 setting. How this affects the end result. Inevitably, 3 things will change. SSIM, speed and size. Out of this, i may have the peculiarity to be more fond of one or two or even want to see what's the best compromise i want to choose between the 3.

I don't know why this is flawed, i actually found it useful and quite logical. If i wanted too tired to go grab my old statistics book, i would bring it down to a statistical test and find the levels of confidence.

See, statistics doesn't need to compare what varies at these settings, with what varies at crf_17 with higher preset.

You can work on a hypothesis you want. Dark Shikari would protest that anything with psy-rd and SSIM isn't measurable and visually it may not be, but technically it is.

A hypothesis could be: "At my settings 6 ref frames do not increase SSIM over 5 b-frames".

http://en.wikipedia.org/wiki/Null_hypothesis

Anyway, i am sure that even if i did that, with calculating the probability, it would still be insignificant as result here.

So, back under my rock too. Thank you for the discussion. At the end, each person has beliefs. If i believe that i saw something useful out of that program, it will be difficult to convince me otherwise, won't be...

[mogobime]

[NR]1
[NR]1
...


You know that the tool can automatically raise the usage of:
-quantizer
-reference-frames
-b-frames
-subme mode
-psy-rd
for you?

You just have to set the parameter(s) to "STEP" you want the tool let modify for you.
Then set the range at th corresponding "LOOP"-Parameters.

An example: To test quantizer 18+19, reference frames 4-6 and b-frames 3-5, type:

4 step -> for raising quantizer automatic instead of setting a fixed value
5 18 1 19 -> start with quantizer 18, raise by one until 19

0 step -> for raising ref-frames automatic instead of setting a fixed value
1 4 1 6 -> start with ref-frame 4, raise by one until 6

2 step -> for raising b-frames automatic instead of setting a fixed value
3 3 1 5 -> start with b-frame 3, raise by one until 5

Now you can have a beer and the tool will test all constellations.

Furthermore if you test a lot of constellations, it's better to save the results in a .csv-table (if you can read them with OpenOffice/MS Office for example).


The program unfortunately, unless i missed something, doesn't allow 2-pass comparison.

You cann use qp mode instead (constant quantizer) for exactly comparing speed/quality of different parameter constellations.


Thanks for your feedback!

Cu mogobime

[woah!]

hi. i dont seem to be able to get this to see video files in the folder? shows 0 in there when i know i have added a mkv/avi/avs to make sure it wasnt a codec thing.

probably missing something stupid, i am on win 7 32bit

update: moving complete folder to C drive and now they show?

[mogobime]

1. Run with admin-account
2. Did you hit return after adding some files?
3. Try to change config and have a look if files are found after that
-> Files are only detected after some user action occurs in program

4. Try starting tool after adding files to folder

The behaviour of the cmd interpreter which interprets batch can differ in details from OS to OS. I have no Win7 x32 here, but i don't think it differs from win7 x64.
Are you sure now that it only runs on c: when using admin rights?
Would be important for me to know that

thanks for feedback

mogobime

[woah!]

ok i seem to have found the issue, i use a folder named !bluray so the folder is at the top of the folder list for me. if i remove the ! from of a folder name it works fine at seeing the video files. with ! it shows 0 video files. so its my fault really then i suppose.

thx again.

[mogobime]

"!" "%" and "&" are signs that can be problematic, because they describe variable names / the end of an argument...
I will have a look, if i can solve the problem without producing a new one :D

thx for answering

[mogobime]

Changelog 1.6.11 (23.11.2012):

- bugfix: statistic file write error isn't reported falsely anymore in non-german windows versions.