Review Resize Calculations

[Seraphic-]

Hi,

Working with a 720x480 anamorphic widescreen 59.94p video here.
When cropped to visible pixels, its size is 640x448.
Now a simple mod16 widescreen resize of that would be 796x448.
However, if I resize to 852x480 first, then crop, total is 757x448.
Was interested on getting some other thoughts on which of the above two would be the recommended process.

Also, should I be using anything other then Spline36Resize for this?

Thanks

[Keiyakusha]

Post some unprocessed sample, post your script. This is movie or what? If movie, you probably should know its original aspect ratio.
If you do mod16 resize, aspect ratio can be not exactly right

[10L23r]

Do you mean that there are black borders on all four sides? That would be weird.
But assuming that you have black borders, you should crop then resize.

That is, crop to 640x448 and resize to 758x448 to maintain aspect ratio. 758 isn't mod16 though, but it preserves aspect ratio. The closest mod16 resolution is 752.

Spline36 is fine. If you really need quality, use nnedi2 and downsize, but the difference will probably be marginal, since the upsize is so minor.

edit: you could also resize after encoding.

[Seraphic-]

Source is a game from a console with a 16:9 aspect ratio (anamorphic widescreen)
Yes, it has black boarder on all four sides.

[Keiyakusha]

Then you should know how this file was created, why its anamorphic and why there is black borders.

[Seraphic-]

Then you should know how this file was created, why its anamorphic and why there is black borders.

I do not understand what you mean.
Yes, I know why it is anamorphic and why it has black boarders.
But my question was which of the two resize methods would be recommended here.

[Keiyakusha]

Sorry, for some reason I thought you asking about how you should crop+resize correctly. If you only want to know about resize method, MOD16 or not, then the answer is obvious - do what you like more. There is nothing too bad in MOD8 resolution for example. The same for Spline36Resize. Other resizers can be softer or sharper (but with more ringing) so pick the one you like.

[Seraphic-]

Sorry, for some reason I thought you asking about how you should crop+resize correctly.

Well yeah, I was.

720x480>640x448>796x448
or
720x480>852x480>757x448>752x448

[Keiyakusha]

I still have feeling that I not completely understand you.
If you asking about order of applying filters, then resize after cropping, as 10L23r said. If you wondering about final resolution - without knowing how file was created we can't tell much, but since you sure that source originally was 16x9 then 796x448 is right. But 757x448 can be right too since there is monitors with such aspect ratios. Try to find some objects in your video that should be perfectly round and see with what resolution they looks better.

[Seraphic-]

What do you mean how the file was created?
And yes it was 16:9 without a doubt.
Thanks for the help.
I'll do some more testing (like objects in the video).

[Keiyakusha]

Seraphic-
I mean anything! How do you get this file? This is gameplay recording or something else? Its created by you or ripped exact from the game in native format? If this file is not created by you but ripped from game then it can be 16x9 but i don't see the reason why it can't be 16x10 too, since this is common aspect ratio on modern monitors (and the one you get if you do resize first).

[10L23r]

I guess it was recorded on a ntsc dvd (that was recording in widescreen)

Since the pixel aspect ratio is 40/33 for widescreen ntsc dvd's, the aspect ratio of the visible things is: (640/448)*(40/33)=400/231=1.7316... what a weird number

These are all assumptions... if i'm wrong, then the below is useless.

List of possible mod16 resolutions within 0.3% error for the aspect ratio, generated by avsp:
304*176
416*240
608*352
720*416
832*480
912*528
944*544
...

edit:
[B](and the one you get if you do resize first).

rly? I think you calculated wrong.

[Mug Funky]

resizing to keep aspect ratios should be the same numbers you'd use for any NTSC video.

if your capture was 720x480, the "visible area" of that used to determine aspect ratio is 704x480.

so if our target is square pixels, and the source is 16:9 (i assume grabbed via component out of a wii or something considering it's 59.94p), you'll want:

480*16/9 = 853.33... from 704 input pixels.

using the full width of your input you'll actually have 872.7272... width. (480*16/9)/704 * 720

using avisynth's resizers you can handle cropping and resizing in floating point, so don't fear non-integer pixels. analog cares not for pixels. if you want mod16 you can just make sure the resizer outputs this.

resizer syntax does my head in, so i'll let you sit with the avs docs for a few mins and do that with your video at hand, so you can test it to make sure it works. i'm in PAL land, so all the numbers are different.

your resize line will look like a big bucket of numbers, but it should work perfectly in theory. in practice, who knows how the console is outputting this video - because a console does care about pixels, so going on "principle" like above might actually yield the wrong aspect. if you can get the console to make a perfect circle for you to capture, that's the best way to determine correct aspect - you can just sit in photoshop and line up another perfect circle and tweak until they fit each other.

[edit]

@10L23r: 832x480? that sounds very computery. i think we have a winner as far as the original size the console may have rendered to.

[Keiyakusha]

rly? I think you calculated wrong.
Ohh, maybe I rounded it wrong. But its close enough, I think ^_^

if your capture was 720x480, the "visible area" of that used to determine aspect ratio is 704x480
Who tells that? Specifications? If so, on practice this is not allways true. At least with japanese DVDs. Anyways we talking about some-game-we-know-nothing-about and we don't know how file was created.

[Seraphic-]

@10L23r
Don't think any of those resolution apply here.
Because with anamorphic video, only the horizontal resolution changes, not the vertical.

@Keiyakusha
The file was "created" by me.
In the sense that is a gameplay recording I made using the console's native output.

@Mug Funky
I am not sure how you are getting 704x480 from 720x480.
The visible pixels from the 720x480 recording is 640x448 (which is the exact frame buffer for this console)
Yes, the intended widescreen display resolution is 853x480 (before cropping and is also an odd number)
Again, my whole point was not being sure on the method to achieve the correct final picture.
1. Capture 720x480, crop 640x448, resize to mod16/9 = 796x448
2. Capture 720x480, resize to 853x480, crop to visible pixels around 758x448 (then crop extra to mod16 or mod4)

[10L23r]

If you only want to resize the horizontal resolution, then use 758*448, as I mentioned earlier.


1. Capture 720x480, crop 640x448, resize to mod16/9 = 796x448
2. Capture 720x480, resize to 853x480, crop to visible pixels around 758x448 (then crop extra to mod16 or mod4)

You see, there is quite a difference in 796 and 758. The first method distorts the aspect ratio (because the game might not have outputed 16/9). The second preserves aspect ratio, but blurs the black borders, resulting in artifacts near the edges of the cropped video. So what you should do is crop your 720*480 video to 640*448, and then resize to xyz*448, with xyz being what ever number that corrects the aspect ratio. I suggested 776, but it's really up to you to decide what works.


Yes, the intended widescreen display resolution is 853x480

This is where you are wrong. The intended resolution before cropping is 872.7272... width. (480*16/9)/704 * 720. You will also arrive at this number if you multiply 720 by 40/33.


and what's wrong with changing both horizontal and vertical resolution, especially if doing so yields mod16?

Ohh, maybe I rounded it wrong. But its close enough, I think ^_^


1.73 isn't 1.6... but the game may have intended to output 1.6. We don't know.

[Keiyakusha]

1.73 isn't 1.6... but the game may have intended to output 1.6. We don't know.

Where did you get 1.73? 757:448=1.69 And 0.9 error is acceptable.

This is where you are wrong. The intended resolution before cropping is . You will also arrive at this number if you multiply 720 by 40/33.
This is what specifications tells us, but in real world this is can or can not be true. It was always true in analogue TVs era. The only way to decide what is right is your eyes.

[10L23r]

(640/448)*(40/33)=1.73, assuming an aspect ratio of 1.818

1.69 is correct if the 720*480 is has a aspect ratio of 16/9.


This is what specifications tells us, but in real world this is can or can not be true. It was always true in analogue TVs era. The only way to decide what is right is your eyes.

That's true. I guess the only reason the specs would be wrong would be for hd>sd downsizing. But maybe the dvd recorder was weird...

maybe the game was trying to output 5:3? 1.66 is pretty close to 1.69

[Keiyakusha]

(640/448)*(40/33)=1.73

OK i see now but I said "the one you get if you do resize first" so I just taked existing numbers on that moment.

And i can rephrase myself. 40:33 SAR not always true if we talking about DVDs. I don't have much experience with games, but I assume this can happens everywhere. So the "best" solution is to find round objects and then we can see how we should calculate resolution. Also if we know what final aspect ratio we should get, we can crop all borders and just resize to appropriate resolution.

P.S.
Sorry, I know that my English is bad...

[smok3]

Seraphic-: what is the intended final format? web? does it have to be PAR=1?

[Seraphic-]

@smok3 - Web. PAR=1?
@10L23r- I don't know, math checks out, but 872 seems way too much. Always thought that widescreen 480p was 853x480.
@Keiyakusha - Per your suggestion, 796x448 seems to have the correct appearance of round objects.

I did some tests myself and posted the results.
Posted raw bmp single frame images if anyone else wants to mess around.
To crop to visible pixels, use 38,16,-42,-16
Used Spline36Resize for the three final images.

--------------------------------------

RAW 4:3 Mode - 720x480
http://www.seraphicgate.com/test/raw43.bmp

RAW 16:9 Mode - 720x480
http://www.seraphicgate.com/test/raw169.bmp

Crop 4:3 Mode - 640x448
http://www.seraphicgate.com/test/crop43.bmp

Crop 16:9 Mode - 640x448
http://www.seraphicgate.com/test/crop169.bmp

Resize Then Crop 16:9 Mode - 720x480->853x480->758x448
http://www.seraphicgate.com/test/758.bmp

Crop Then Resize 16:9 Mode - 720x480->640x448->776x448
http://www.seraphicgate.com/test/776.bmp

Crop Then Resize 16:9 Mode - 720x480->640x448->796x448
http://www.seraphicgate.com/test/796.bmp

[Keiyakusha]

For me cropped 4:3 result without any resizing looks best...
EDIT: Crop Then Resize 16:9 Mode - 720x480->640x448->796x448 alsо fine exсept the HUD but this is probably normal since it overlayed on top of other things.

[Seraphic-]

For me cropped 4:3 result without any resizing looks best...
EDIT: Crop Then Resize 16:9 Mode - 720x480->640x448->796x448 alsо fine exсept the HUD but this is probably normal since it overlayed on top of other things.

Well, yeah, the 4:3 image doesn't need anything to it at all.
Was posted to reference how round objects look.

Out of the three tests I did 796x448 looked the most correct.
But 10L23r said it distorts the aspect ratio.

[10L23r]

the 4:3 640*448 is definitely the best.
the 796*448 is good, but the HUD is distorted (check button)

actually, when comparing the 796*448 to the 4:3, i think 796 is a tiny bit too low. (or my eyes are lying, but the circle isn't completely circular)

But 10L23r said it distorts the aspect ratio.

Well, I was wrong... probably

[Keiyakusha]

Well, I was wrong... probably
You right. But you shouldn't rely on specs so much because they not always respected in SAR cases.

[10L23r]

That's where I was wrong :)

measuring the length btwn the rightmost pixel of the left orange blob and the leftmost of the right for the 4:3 yields 410. for the 796*448 it's 416. for the 16:9 crop it's 335.
so the aspect ratio is: (410/335)*(640/448)=1.7484. 796*448 is approx. correct

[Seraphic-]

That's where I was wrong :)

measuring the length btwn the rightmost pixel of the left orange blob and the leftmost of the right for the 4:3 yields 410. for the 796*448 it's 416. for the 16:9 crop it's 335.
so the aspect ratio is: (410/335)*(640/448)=1.7484. 796*448 is approx. correct


Yeah, 796x448 looks about right. But is isn't perfect.
But as far as keeping correct aspect ratio, looks like it is a good as it will get.