Video Upscaling and Resizing

[Seraphic-]

Hi,

I'm hoping to get an interesting discussion going on the cost/benefit of upscaling PS2 16x9 console video to wide-screen definition.

For a long time now, I have been unwilling to resize video from it's default resolution. Which, in my case, would be a capture at 720x480 and then just a crop down to 636x444 to remove the black box outline. But, over the past few days, I've really become interested in trying upscaling video to wide-screen. I would really like to see what could be done with minimal video artifacts.

For the below wide-screen demo video I used VirtualDub's Lanczos3 resize filter. However, I posted here because I'm interested to know what I might be able to do with Avisynth that I couldn't do with VirtualDub.

I think the wide-screen demo turned out really well. But I'm really interested in getting some feedback on what other people think on it. Along with suggestions on improvements as at this time, I haven't experimented too much with upscaling video.

Thanks

[Guest]

Do a search; there are several vibrant threads about upconversion. Why do you have to start another one?

[*.mp4 guy]

Also you should post samples compressed with a high quality avi codec or mpeg2 compressor, dealing with RMVB files is a pain in the ass, and people are less likely to help you if you make it a pain in the ass to do it.

[Seraphic-]

@neuron2, didn't mean to cause any problems. This is my first post in several months and even though there are a few other posts, I didn't want to crowd them with my specific issues and questions. Rather, thought it would have been best to start my own. I guess not.

@*.mp4 guy, no one likes .rmvb . I posted those in .rmvb format as that is what I work with and thought most people have real/quicktime alternative installed to view. RV10 is high quality too, but i'll have to re-encode with something else then.

[*.mp4 guy]

RV10 is high quality, but it is hard to work with, the only way is to decode it to a lossless file via a directshow filter graph then work on the lossless file and pray that the RV10 file wasn't vfr.

[HeadBangeR77]

Originally Posted by Seraphic-

@neuron2, didn't mean to cause any problems. This is my first post in several months and even though there are a few other posts, I didn't want to crowd them with my specific issues and questions. Rather, thought it would have been best to start my own. I guess not.

Don't worry - the most "vibrant" ones seem to have ended in one large discussion about ITU vs generic resizing, instead of sticking to the main matter. In one of them the discussion was sooo hot, that they even accused the thread's founder of hijacking their "own thread"

I'm for upscaling, if one has enough time and hdd space, though I'm doing it just for fun, and for a better presentation of my really favourite movies - otherweise that wouldn't make much sense to me.

[Seraphic-]

Originally Posted by *.mp4 guy

RV10 is high quality, but it is hard to work with, the only way is to decode it to a lossless file via a directshow filter graph then work on the lossless file and pray that the RV10 file wasn't vfr.

I didn't know you were trying to decode the video. It was posted only to view the difference between the default and wide-screen versions.

Originally Posted by HeadBangeR77

Don't worry - the most "vibrant" ones seem to have ended in one large discussion about ITU vs generic resizing, instead of sticking to the main matter. In one of them the discussion was sooo hot, that they even accused the thread's founder of hijacking their "own thread"

I'm for upscaling, if one has enough time and hdd space, though I'm doing it just for fun, and for a better presentation of my really favourite movies - otherweise that wouldn't make much sense to me.

Thanks

I've become really interested in upscaling to wide-screen just to see what can be done before there are too many artifacts. I mean, after all, you are in fact stretching the video a fair amount. My question is how far is too far and to resize before or after cropping out black outline boxes. Also, what size the standard for wide-screen and if it must be in multiples of 16. I mean, would it be better to crop first it I.E - 636x444 => 848x480 or to resize 720x480 => to a much higher resolution and then crop down to 848x480. That is, if 848x480, is the optimal resolution.

I'm sure I'll have more questions, but that is good start.

Thanks

[HeadBangeR77]

Originally Posted by Seraphic-

I've become really interested in upscaling to wide-screen just to see what can be done before there are too many artifacts. I mean, after all, you are in fact stretching the video a fair amount. My question is how far is too far and to resize before or after cropping out black outline boxes. Also, what size the standard for wide-screen and if it must be in multiples of 16. I mean, would it be better to crop first it I.E - 636x444 => 848x480 or to resize 720x480 => to a much higher resolution and then crop down to 848x480. That is, if 848x480, is the optimal resolution.

I'm sure I'll have more questions, but that is good start.

Thanks

There are some folks out there, who think such a small upsizing doesn't make much sense.
In your case I don't think you would get any artifacts, but I'm not sure which resizing method (e.g. bicubic, Lanczos3, Lanczos4, Spline16/32 etc.) would be the best for your type of contenct, since I don't encode anything apart from regular movies and videoclips.

I'm a sample-freak, but it's worth the time imo: whatever you do, just encode a few samples (for instance with your favourite scenes), one pass, the same settings, to analyze them and to come to conclusion, which one looks the best for you (since those are your eyes, and nobody's elses ).

As to cropping and resizing - I always crop at the very beginning. That's the first thing I do, since then it's easier for me to choose the apropriate resolution, so that proportions will look ok. The only thing I could think of before cropping, is deblocking, if it's needed.
I know it's considered to be outdated, but Gordian Knot it's a good tool for cropping /resizing /getting the proper aspect ratio. If you're into avisynth, you could just copy and paste the values from GK into your own script. Plus there's a preview window, so that you can see the proportions etc.

Sorry, it's 4 AM by me, so good night

[*.mp4 guy]

There isn't going to be much harm done upsizing the video's aslong as you don't overcompress them. You would get better looking video if you ran it through a good antialiaser (before resizing).

lanczos based resizers preserve the most of the sources detail, but leave behind the most artifacts.

Spline based resizers preserve the second highest amount of detail, and have the least artifacts of any "simple" resizer.

Bicubic has no ringing artifacts (unlike spline which has a very small amount, and lanczos which has a very large amount) but has much worse aliasing, and preserves the least detail of any resizer besides bilinear.

lanczos has less aliasing and less bluring the more taps you use, but has more ringing and overemphasized frequencies aswell (looks like oversharpening).

Spline only benefits from higher taps settings, but is also very slow with them.

Bicubic does not have adjustable taps, neither does bilinear, but thats doesn't really matter since they are both crap anyway.

[Seraphic-]

Hmmm, why would I need "Anti-aliasing" before resizing? Wouldn't it need it after, since the video was captured correctly at 720x480? Also, I take it when you say "lanczos", it's the same as Lanczos3?

A video a friend of mine posted (sample image below), in which he was showing his 720x480/16x9 wide-screen/upscaling test results is what got me interested in this (it looks really good, as text and the overall detail remain sharp ~ mine looks kind of blurry, but I'm still new to this). I have also just captured video at 720x480 and cropped out black-bars down to 636x444.

[*.mp4 guy]

The reason for the antialiaser is because your source is aliased (because its from a video game), this will make the interpolators job harder and make it cause more artifacts then if the source didn't have aliasing, which is why I suggested using an antialiaser before you resize. However an antialiaser will soften the picture (which I'm guessing now would bother you more then the aliasing) so if your goal is to get the sharpest results possible you don't need to worry about antialiasing. It looks like your friend might have used a sharpener after he resized, you could try limitedsharpenfaster to get similarly sharp results.

lanczos3, lanczos4, etc. are all just alternative names for lanczos(taps=3/4/8/256/whatever).

[Ð.Sp!dér]

If I were you, I would consider using AA (Antialiasing). This is a common problem in video games where you can clearly see the edges being choppy MS Paint-style. Even though it could result in a small loss of detail, more like in the backround and such, at a far away distance, but this will considerably improve the overall visual quality.

[Chainmax]

One thing I'd do before using antialiasing is to upsize the cropped image with EEDI2 in this manner:

EEDI2().TurnRight().EEDI2().TurnLeft()

which would bring it from 636x444 to 1272x888 and eliminate most (and probably all) of the aliasing. Then I'd use Spline36Resize and bring it down to the final resolution. This way, you are basically doing two antialiasing procedures (EEDI2 and upsizing+downsizing) with almost no blurring .

[Seraphic-]

Originally Posted by Ð.Sp!dér

If I were you, I would consider using AA (Antialiasing). This is a common problem in video games where you can clearly see the edges being choppy MS Paint-style. Even though it could result in a small loss of detail, more like in the background and such, at a far away distance, but this will considerably improve the overall visual quality.

Wouldn't you agree that most, if not all, of that aliasing in that image would be caused or worsened from the sharpening filter? I mean, if you take a look at my 636x444 image, you don't find anything too bad in relation to the edges (or maybe some scenes could just have edges that way).

But since this has been brought up, it has made me wonder if I've been missing a step this whole time. I take it you this isn't the same as aligning the fields to remove interlacing, as seen below?

Originally Posted by Chainmax

One thing I'd do before using anti-aliasing is to upsize the cropped image with EEDI2 in this manner:

EEDI2().TurnRight().EEDI2().TurnLeft()

which would bring it from 636x444 to 1272x888 and eliminate most (and probably all) of the aliasing. Then I'd use Spline36Resize and bring it down to the final resolution. This way, you are basically doing two anti-aliasing procedures (EEDI2 and upsizing+downsizing) with almost no blurring .

So 636x444 to 1272x888 (with EEDI2) and the back down to 848x480 (or is there another recommended resolution)? Also, do you have a script code for the Spline36Resize downsizing?

[check]

spline36resize(destx,desty)

Aliasing is where the edges are not straight, but jagged. Since everyone is having fun with paint I'll get in on the action.

The left line is not aliased, the right line is aliased (ie, jaggy). As you can see, while the left line is 'the real thing', the right line is just a bunch of blocks lined up to approximate a real line.
The encoder will be trying to encode an approximation, rather than the real thing, which is bad because the aliased version will prove to be less compressible than the original.

[Chainmax]

Originally Posted by Seraphic-

...
So 636x444 to 1272x888 (with EEDI2) and the back down to 848x480 (or is there another recommended resolution)? Also, do you have a script code for the Spline36Resize downsizing?

There isn't really a recommended resolution, it all depends on what you want and/or need.

[Seraphic-]

Originally Posted by Chainmax

There isn't really a recommended resolution, it all depends on what you want and/or need.

Doesn't it and shouldn't it be in multiples of 16? I mean, there must be a recommended one if you're going down from 1272x888 to keep the correct ratio. The reason I said 848x480, was because it was in multiples of 16.

Originally Posted by check

spline36resize(destx,desty)

Aliasing is where the edges are not straight, but jagged. Since everyone is having fun with paint I'll get in on the action.

The left line is not aliased, the right line is aliased (ie, jaggy). As you can see, while the left line is 'the real thing', the right line is just a bunch of blocks lined up to approximate a real line.
The encoder will be trying to encode an approximation, rather than the real thing, which is bad because the aliased version will prove to be less compressible than the original.

I was wondering why there is some in one part of my below test image, but not in another.



-----------------------------

Also, if anyone wants to do some testing, here is a raw avi clip from the above scene (ten seconds long). What was done: Cropped down from 720x480 to 636x444 to remove black bars and field alignment to remove interlacing. Download Here

[Chainmax]

Originally Posted by Seraphic-

Doesn't it and shouldn't it be in multiples of 16? I mean, there must be a recommended one if you're going down from 1272x888 to keep the correct ratio. The reason I said 848x480, was because it was in multiples of 16.
...

Other than keeping width and height multiples of 16 (which has to do with MPEGx shortcomings IIRC), there is not an ideal resolution. The suggestion to use EEDI2 was based on the fact that it does some pretty good antialiasing without blurring the picture too much. Upon using it, you end up with 2x the original framesize which in your case would be 1272x888. Since that's a higher resolution than the one you intended, I suggested a downsize. If it's any help, the closest exact 16:9 resolution to your source's is 768x432.

[Seraphic-]

Originally Posted by Chainmax

Other than keeping width and height multiples of 16 (which has to do with MPEGx shortcomings IIRC), there is not an ideal resolution. The suggestion to use EEDI2 was based on the fact that it does some pretty good antialiasing without blurring the picture too much. Upon using it, you end up with 2x the original framesize which in your case would be 1272x888. Since that's a higher resolution than the one you intended, I suggested a downsize. If it's any help, the closest exact 16:9 resolution to your source's is 768x432.

Alright, thanks, I'll give that a try. Also, when you say "6:9 resolution to your source's is 768x432": Do mean 636x444 to 768x432 or 1272x888 down to 636x444?

Check, is "spline36resize(destx,desty)" part of a .dll plugin? If yes, could I have the link please?

------------------------------------

For those interested in testing on their end:

Cropped down from 720x480 to 636x444 to remove black bars and field alignment to remove interlacing.
Download HERE ~ 206MB

Raw capture at 720x480, no cropping and no field alignment.
Download HERE ~ 115MB

[Ð.Sp!dér]

Originally Posted by Seraphic-

is "spline36resize(destx,desty)" part of a .dll plugin? If yes, could I have the link please?

It's part of the AviSynth script. You know, those .avs files...
Wich you can Edit in Notepad to see some really cool magical rainbows.

The 6th (sixth) line in a normal AviSynth scripts is:
#resize

Wich you can replace with:
Spline36Resize(768,432)

After a # symbol you can add as many lines as you wish and it won't make a difference.
It would just be skipped by AviSynth.