The Not Quite But Almost Exhaustive Deblocking Comparison Test (TNQBAEDBCT)

[Sagekilla]

I'll be setting up over 144 encodes to test how deblocking effects a source from -3:-3 all the way up to 3:3 with all the videos available for download. Naturally, It'll take quite a while to accomplish. The video I'll be using with be a short (720 frames, 30 seconds) clip from the first episode of Battlestar Galactica encoded at 864x480. I'll use the following avisynth script to pipe it to x264:

MPEG2Source("G:\Movies\Battlestar Galactica\source.d2v")
Trim(28000,28719)
Lanczos4Resize(864,480)

The settings used for all encodes will be based off my normal encoding settings:

x264.exe --qp varies --ref 4 --mixed-refs --no-fast-pskip --bframes 16 --b-pyramid --weightb --b-rdo --bime --direct auto --filter varies --partitions all --8x8dct --subme 7 --me umh --trellis 0 --progress --threads auto --thread-input --cqm "G:\Movies\Matrices\eqm_avchr.cfg" --output "G:\Movies\Battlestar Galactica\Video1.264" "G:\Movies\Battlestar Galactica\source.avs"

All video files will be made available upon completion and I'll provide screencaps of each video along with the SSIM, PSNR, bitrate and encoding speed. Naturally, due to the extreme nature of my settings as well as how much information will be provided this is going to take quite a while. If you have any suggestions on how I can improve this test, please feel free to voice your opinion and provide (constructive) criticism!


File List:
qp 30 encodes: http://www.megaupload.com/?d=X8D6JB0T
qp 25 encodes: TBA
qp 20 encodes: TBA
qp 15 encodes: TBA
No deblock encodes: http://www.megaupload.com/?d=KE7V9GAB

[Sagekilla]

I just had second thoughts and wondered: Would it be better for me to do a crf or a 2-pass for comparison? I just remembered crf is a "constant quality" so that might end up interfering with my intentions, does anyone have any recommendations on what to do?

[Manao]

It's better if all your videos end up having the same size. Else visual comparison is imho pointless. So either use 2 passes ( or three for such a short video ), or change the CRF factor in order to match the size of all the encodings.

[Dark Shikari]

I just had second thoughts and wondered: Would it be better for me to do a crf or a 2-pass for comparison? I just remembered crf is a "constant quality" so that might end up interfering with my intentions, does anyone have any recommendations on what to do?CRF is good because it'll express how the changes in deblocking affect both bitrate and quality, rather than testing the encoder's ratecontrol ability.

One problem with your testing idea is that deblocking acts differently at different CRFs/QPs.

[Sagekilla]

Got it. Switching to 2-pass now, since it does take quite a while even for such a short clip.

[Sagekilla]

CRF is good because it'll express how the changes in deblocking affect both bitrate and quality, rather than testing the encoder's ratecontrol ability.

One problem with your testing idea is that deblocking acts differently at different CRFs/QPs.

In other words, the only true "exhaustive" test would be to test literally all 144 deblocking settings across every possible crf value?

[Dark Shikari]

In other words, the only true "exhaustive" test would be to test literally all 144 deblocking settings across every possible crf value?Well personally I wouldn't test "absurd" deblocking values like -6:7... limit it to values that make some sort of sense. Then test each one at a variety of QPs.

--qp 15
--qp 20
--qp 25
--qp 30

perhaps.

[Sagekilla]

Well personally I wouldn't test "absurd" deblocking values like -6:7... limit it to values that make some sort of sense. Then test each one at a variety of QPs.

--qp 15
--qp 20
--qp 25
--qp 30

perhaps.

So limit to... -3:-3 to 3:3 perhaps? That'd leave 48 encodes per qp value, or 192 encodes over those 4 qps I think.

[Sagekilla]

Anyone know a way to reliably capture screen caps from specified frames and save them from a .mkv file? I have --qp 30 fully encoded for all deblocking from -3:-3 to 3:3 but I have no idea how I can grab screenshots from each one at the same frame and save it.

Edit: I've uploaded the results from testing the --qp 30. It's here: http://www.megaupload.com/?d=X8D6JB0T

[Episodio1]

*click*

Thanks.


But maybe you should encode without deblocking as well. ^_^

[Sagekilla]

*click*

Thanks.


But maybe you should encode without deblocking as well. ^_^

Good idea, I'll upload an archive with no deblocking used soon. I'll have qp 25 up in a little while, maybe tomorrow.

Edit: Here's the No deblocking sample, http://www.megaupload.com/?d=KE7V9GAB Just looking at the qp 30 example, it's very bad to turn it off.

[Episodio1]

Would the stream be bit-identical to turn it OFF in encoding than turn it OFF while decoding?

"it's very bad to turn it off."

We'll see at qp15 or 20. ^_^


PD: I appreciate your effort.

[Sagekilla]

Would the stream be bit-identical to turn it OFF in encoding than turn it OFF while decoding?

"it's very bad to turn it off."

We'll see at qp15 or 20. ^_^


PD: I appreciate your effort.

No, it wouldn't I think.. At encode time, removing deblocking would give a different bitstream than you would get having it on since it's smearing some details and therefore the stream is not identical to the no-deblock. Also, I have reason to believe that enabling @ encode but disabling while decoding would look better then not having it at all. And yes, it looks better at qp 15 and 20, but I still see some extremely minor blocking at 20, very very tiny amounts.

Edit: And you are most welcome, I hope this thread will be helpful for some people.

[akupenguin]

Also, I have reason to believe that enabling @ encode but disabling while decoding would look better then not having it at all.
No, that's very bad. It produces random corruption of the image (sometimes taking the form of blocks, but any kind of artifact is possible). To see it, try lavcopts skiploopfilter=all on any video that uses deblocking. If by some chance you don't see blatant ugly artifacts everywhere, it just means your video was sufficiently low qp that deblocking didn't make any difference.
The related but sane idea is: have the decoder first display the image and then deblock it. That way you see the unfiltered image, but there's no desync errors and motion compensation is happy.

[Sagekilla]

Apologies to anyone who was looking forward to a full test according to my original post, but unfortunately schoolwork has caught up to me at the moment and I'll be quite busy over the next two or three days

Hopefully by Wednesday evening I'll have it all sorted out though!

[DeathTheSheep]

So now it's TNQBAEDBCTTWBBOW. :D

(TWBBOW=That Will Be Back On Wednesday).

[Sagekilla]

So now it's TNQBAEDBCTTWBBOW. :D

(TWBBOW=That Will Be Back On Wednesday).

Indeed it will, as I've managed to work out the issues with homework at the moment so I can resume encoding come tomorrow. (Don't you just love how it's becoming such an enormous and unpronounceable acronym?)

[DeathTheSheep]

Nice!
Don't you just love how it's becoming such an enormous and unpronounceable acronym?
I do, I really do. ;)