How much to crop to preserve correct AR?

[Comatose]

OBJECTIVES
Help Coma resize cropped video without butchering it

DESCRIPTION
Previously, I haven't resized correctly and skewed the AR a little, pretty much because I was too lazy D:

I have a 1440x1080 video (flagged 16:9), and need to crop 4 pixels from left and 4 from right to remove the overscan.
Just cropping and resizing skews the AR a little, so obviously I need to crop a bit more to preserve the correct AR, but I'm not sure how to calculate this.

Note that there's no need for compensate for strange PARs, since this is HDTV material.

Everything I've found online goes very deep into compensating non-square PARs and makes it very hard to "fish" out the basic math D:

REWARD
Avisynth Necklace of Wisdom
+10 Spirit

3 Gold

+500 reputation with People Who Have No Clue

[thetoof]

LOL I want the reward!!!

If you crop too much, you'll lose precious video information. So, since you most likely have a YV12 source, crop in even numbers until you have removed all the overscan while preserving as much video as you can.

You can simply resize to mod16 with the correct AR after (in this case, 1920x1080, 1440x816, 528x304, etc etc). Simply choose a mod16 value for the width or the height and match the other by using the AR. Example: 528/? = 16/9, ?=297, Closest mod16=304

OR crop, resize to the closest mod16 (in this case 1440x1080), encode and use an anamorphic flag in mkvmerge to set the Display Aspect Ratio to 16/9

[Comatose]

Let me give you an example.

Doing all of that will be in vain, as it will still result in the wrong AR.
The correct AR is maintained when resizing *with* the overscan, however, if you crop and then resize, it'll be slightly skewed.

I had originally thought of resizing first then cropping out the overscan to find the correct resolution, noting that, then cropping and resizing to get a mod16 resolution and set the resolution in the MKV to 1276*720 or something to achieve the correct AR.

The only problem with this is that I only have an old Celly laptop connected to the HDTV, and it's simply not fast enough to resize at that resolution. (and if you're curious, yes, I encode to xvid for the same reason >.>)

The example (screenshots) I'm going to give you aren't from what I was talking about earlier (different anime... the other one is upscaled SD and looks horrible, so... :p), but the problem still applies.

Resized without cropping (http://xs227.xs.to/xs227/08180/correct540.png)
Cropped and resized (http://xs227.xs.to/xs227/08180/cropped527.png)

Finding the 16:9 height for the new width is often impossible.
In this case, the new width is 1276, and the 16:9 height for it is 1276*(9/16)=717.75.
Since it will still be a bit off, it feels kinda pointless. I might as well just crop and resize, since I still won't have the correct AR.

So unless I'm missing something here, the only way seems to be to crop a bit more.

It seems that Japanese people are doing this - I've compared my cropped and resized with theirs, and while mine has a bit more video on the left and right, theirs has the correct AR.

I'm just not sure how they calculate the amount they need to crop.

Thanks!

[thetoof]

Oh, right, now I get it.

If you crop the height by some value, you'll need to crop the same amount of the width relatively to the AR (not sure if that was clear... lemme give you an example :p)

The width is 1.777777777777777 bigger than the height for 16/9 AR, so if you crop x of the height, you'll have to crop 1.7777777777777777x of the width (and if you crop x of the width, you'll need to crop 0,5625x of the height)

For your example : If you crop a total of 8 pixels to the width, you'll need to crop 4,5 pixels to the height. However, since you're cropping 8 pixels to 1440 that appears as 1920 because of the anamorphic flag, you're actually cropping 10,6666 pixels (8/1440 x 1920 = 10,666). So, you'll need to crop 6 pixels of the height. For all that, I think crop(4,4,-4,-2) would be the best choice for your clip.

As I am very tired atm, there might be some kind of error in this post, but I did this not so long ago and the AR was preserved, so I'm pretty confident about my answer... I'll reread after a good night of sleep just to be sure :p

Then, resize to any mod16 16/9 resolution you want

[Wilbert]

I would leave the black borders in and resize to a 16:9 resolution. If you still want to remove the black borders you also need to crop a bit of the image itself.

You just need to calculate what happens with the black borders. For example:

source: 1440x1080 (with 2x4 black pixels)

real ar: 1080*16/9 x 1080 = 1920 x 1080

or (1080-2*w)*16/9 x 1080 = (1920-2*x) x 1080 with w=4 and x=w*16/9=7.1111

resize to any other 16:9 clip (z is height):

let a = 1080/z then

real ar: 1080*16/9 x 1080 = 1920 x 1080 -> z*16/9 x z

and (1080-2*w)*16/9 x 1080 = [(1080-2*w)]/a*16/9 x [1080/a] = [z-2*w/a]*16/9 x z

thus for example: z=720 and a=1.5 and
[z-2*w/a]*16/9 x z = [720-2*4/1.5]*16/9 x 720 = [1280-9.4815] x 720 = 1270 x 720 [no black borders]

mod 16 -> 1264 x 720 [no black borders]

In other words:
1) resize to 1280x720
2) crop the black pixels: 1270x720 [shouldn't contain black borders]
3) to get mod16: crop further (6 pixels) to 1264x720. ar should be correct.

[Comatose]

So, you'll need to crop 6 pixels of the height.
I don't understand how you came to that number oO

Wilbert: Thanks, but that leaves me with a "non-standard" resolution, and my TV doesn't like some of those =/ It looks funky.
1280x720 is guaranteed to work properly, which is why I'm trying to stick to it with all my might =p

[thetoof]

(and if you crop x of the width, you'll need to crop 0,5625x of the height)

1 / 1,7777777777 = 0,5625

0,5625 x 10,66666666 = 6

[Comatose]

Facepalm D:

Thanks!

[Alex_ander]

I think you'll get the same result if you ignore the flagging and simply keep pixel proportion after cropping. If you crop A pixels horizontally and want to find X pixels you have to crop vertically for keeping the proportion:

before cropping 1440/1080=4/3

after cropping (1440-A)/(1080-X)=4/3

this gives X=3A/4 (for A=8 X=6)

[2Bdecided]

I'm amazed your TV displays these borders.

If connected as a "PC" I suppose it will.

However, if connected as any other source, it's common to still overscan a little. Few displays (even those where you'd expect 1:1 pixel mapping) keep the very edge of the image. That's why broadcasters take so little care! (or maybe the displays are like this because broadcasters take so little care).

Cheers,
David.

[RedDwarf1]

scrap that, it does not work properly for hi def stuff

[smok3]

if you need to keep the PAR, but still get mod16 resolution then:
http://somestuff.org/resizeCalc.php?ssmw=1440&sar=1.33&sar2=&ssmh=1080&CT=4&CL=8&CR=8&CB=4&cropmode=normal&trw=1424&dar=1.33&dar2=&mod=&padw=&padh=&css=&doit=true
if you need to go to square 1280x720 then:
http://somestuff.org/resizeCalc.php?ssmw=1440&sar=1.33&sar2=&ssmh=1080&CT=4&CL=4&CR=4&CB=4&cropmode=normal&trw=1280&dar=1&dar2=&mod=&padw=&padh=&css=&doit=true