What is really behind Smart Crop All?

[duartix]

Well I've been debating with myself on whether to send TheWEF a private message or to post this as a new thread. Since I for once rarely read my own private messages, I went for the second. Throw me a stone, it will make us feel better.

Now, there is this post about getting better compression with a careful alignment of the cropping window:
http://forum.doom9.org/showthread.php?s=&threadid=18277&perpage=20&pagenumber=1

In the page 3 of the very same post:
http://forum.doom9.org/showthread.php?s=&threadid=18277&perpage=20&pagenumber=3

LotionBoy says:if you crop off all the black in GKnot, then select Smart Crop All, and make sure both the hor. and vert. are mod32, you will get the highest quality file. which to my understanding means the optimal placement of the cropping window, and thus the lower bitrate.

@theWEF or anyone else who helped to develop GK:
Please drop in that post or maybe here, to answer this 3 questions:
Will it really produce the best alignment?
How?
Why?

[DaveEL]

Originally posted by duartix
LotionBoy says: which to my understanding means the optimal placement of the cropping window, and thus the lower bitrate.

Please drop in that post or maybe here, to answer this 3 questions:
Will it really produce the best alignment?
How?
Why? [/B]

If you crop all the black borders off there is less image to compress (black borders may take very little space but still take some and the edges where the transition to black is relatively hard to encode.)

As you have to make the width a multiple of 32 and the height multiple of 16 if you just crop exactly to the borders the aspect ratio may be quite far off. GordianKnots smart crop options crop additional pixels from the source in order to make the aspect ratio match the original source as closely as possible

DaveEL

[LotionBoy]

Alrightie, since you're talking about me, I'll see if I can answer. I cannot actually prove that Smart Crop All is going to give you the best quality because I haven't actually tested this enough. Maybe if I have some time this weekend (ha! like that ever happens) I'll do some testing. What I can tell you for sure is the following.
Cropping off all the black bars makes the file more compressible because it kills the need to encode border pixels. Smart Crop All works well because it allows the image to be a 32x32 resolution without having the black bars present and while preserving as much as possible of the picture. If you check out my tests using Smart Crop All and 32x32 resolutions (in the same thread), these files always come out with the smallest size/best Average DRF. So Smart Crop All and Mod32 give the best quality of any of the options I tested. What I did not take into account was the effect that moving the cropping window has on encoding. This could very well effect my results and lead to my answer not being correct at all (well, it will be correct in context, but not overall). So I'll try doing something like what follows when I get a chance. I'll take a file, smart crop all and set it to mod32. Then I'll move around the crop window a little to test the effects of that on the size. This test will combine my tests with all the other tests that were being done in the thread and should hopefully bring us a bit closer to an answer. Hope that clears things up.

LotionBoy

[duartix]

@DaveEL:

You probably haven't read the post I was refering to. There is a lot more to that than just taking out the borders as you will find if you read it.

@LotionBoy:

Well if were to be proved that GK allways chooses the right cropping alignment then I think it would be mere than just a coincidence, that is why I was pleading for a developers comment...

Anyway let's wait and see.

[TheWEF]

Originally posted by duartix
...were to be proved that GK allways chooses the right cropping alignment...

i have no idea what you mean with "cropping alignment"? ;)

the only thing smartcrop does is keep 99.99% correct aspect ratio. it just crops off a few more pixels depending on output resolution.
(on the other hand: what "correct ar" really is can be discussed eternally: ITU or not, mastering errors, how can we find out...)

why it changes compressibility?:

lets say with and without smartcrop you get the same output-ar (= the same resolution). but with smartcrop your input frame is smaller and has less different pixels, therefor carries less information and compresses better.
but if smartcrop crops at the sides and decreases output-ar you can get the opposite effect.

wef.

[LotionBoy]

@theWef. What we are discussing here is the placement of the cropping frame. Say you are cropping 5 off the left and the right. Say you then switch that to 4 off the left and 6 off the right (assuming this does not include black bars in the video). Your AR is the same. Your filesize (if you do a constant quant) will not be.

LotionBoy

[TheWEF]

hmmm... this is not a surprise since you are not encoding the same content. but i guess the difference should not be too much though, if you are moving the content 1 pixel sideways.

or are you saying that e.g. 4/6 will always give a smaller/bigger filesize than 5/5?
i'd think that it depends on the content, one time it will be a little more and another time (another movie) a little less...?

wef.

[LotionBoy]

See, the thing that was discovered in the thread linked above (worth the read if you have a chance) is that shifting the crop window by one pixel can result in huge file size difference in constant quant. encodes, which translate into quality differences in normal encodes. To just quickly repost one of my test here.

720x416 t32 b32 : 432,812,032 bytes
720x416 t31 b33 : 411,508,736 bytes
720x416 t30 b34 : 412,196,864 bytes
720x416 t29 b35 : 408,639,488 bytes
720x416 t28 b36 : 412,862,464 bytes
720x416 t27 b37 : 409,436,160 bytes
720x416 t26 b38 : 414,685,184 bytes
720x416 t25 b39 : 413,167,616 bytes
720x416 t24 b40 : 432,678,912 bytes

you can see that there is a difference of almost 30MB between some of these files, even though they are all exactly the same (I was only cropping out black, so there is no change in the actual image, just in the location and size of the black bars). So basically, there is more to cropping that just killing the black bars.

LotionBoy

[TheWEF]

again, this doesn't surprise me.

if you LEAVE IN black bars, of course it makes a difference where the borders lie an if an additional macroblock is touched or not (= completely black).
that's exactly what we've been saying all the time: crop away all black, it can eat up a lot of bandwith.

you could try this, use an avs:

crop away all black, then resize.
e.g.:
crop(3,69,715,438)
BicubicResize(640,272,0,0.75)
encode.

now use the same script, but with
AddBorders(0,0,0,32)
add 32 black lines at the bottom.
encode.
although the frame is bigger now (640x304)
you should get almost the same filesize.

now change to
AddBorders(0,1,0,31)
encode.
the framesize is still the same (640x304)
but i'm sure filesize is much bigger now
an additional row of 40 macroblocks is not completely black anymore...

wef.

[duartix]

@TheWEF:

We have tested way beyond what you are suggesting. These people have posted a shait load of tests, they have discovered a complex phenomenon, and are commited to discovering the reasons behind it, as you can verify if you just read the post thoroughly. ;)

[TheWEF]

Originally posted by duartix
...they have discovered a complex phenomenon, and are commited to discovering the reasons behind it...

highly appreciated! :)
this thread is a little to much for me and i will wait for an interpretation of all the data before i say another word.

wef.

[ScarMouth]

Originally posted by LotionBoy
you can see that there is a difference of almost 30MB between some of these files, even though they are all exactly the same (I was only cropping out black, so there is no change in the actual image, just in the location and size of the black bars). So basically, there is more to cropping that just killing the black bars.

LotionBoy Yes, I'm a junior nOOb however...

Perhaps I don't understand how the compression actually works but I imagined that each screen is compressed by using an Overlay mechanism similar to JPG compression.

Like, pick a point then examine all the points around it.

If I got this right then changing the crop window shifts the overlay schema so that even though your tests were manipulating the black area, the actual "stuff" of the frame to be compressed moved.

So in one case, test point (1,1) is black but in another (1,1) is the actual data. The overlay hyothetically centered at (2,2) will have a different compression aspect since not only did its neighbors change but also the reference point data of (2,2).

Do I grock?