Chroma Resampling - 2D games - image attached

[zerowalker]

I was playing around with the Chroma Resampling settings in the YUY conversions and noticed that when games are very pixel like, and resized using point resize (nearest neighboor), it seems that using point as resampling does the best jobb.

But, i do have problems with it.
It seems that it drags all the colors down 1-2 pixels, giving ever edge a halo of sorts just at the bottom.

I know that downsampling will smooth it out and make a halo, but this is only at the bottom, similar to analogue capture where you can solve it with ShiftChroma.


Look at the picture i attached.

Zoom in on the hearts, and look right under the black edge around it. Everything it gray right under (if it´s a black edge).

[TheSkiller]

We cannot see your image yet because it is still pending approval – better use image hosting services. ;)

So you are probably trying to convert RGB to YV12, right?

Wild guess, maybe this helps.

ConvertToYV24(chromaresample="point")
MergeChroma(PointResize(width, height, 0, 1))
ConvertToYV12(chromaresample="point")

[Gavino]

See the discussion on chromaresample="point" in this thread, especially post #17.

[zerowalker]

It doesn´t seem to solve it.
I tried the other version (1,1) in post 17 aswell.

The result is:

Only convert with point = chroma is out of place with 1-2 pixels down

MergeChroma(PointResize(width, height, 0, 1))
ConvertToYV12(chromaresample="point")

Chroma is out of place with 1-2 pixels up.

MergeChroma(PointResize(width, height, 1, 1))

ConvertToYV12(chromaresample="point")

Chroma is out of place both up and left, it´s just worse than before.


TheSkiller: Yes it´s RGB to YV12 with Rec709.

[poisondeathray]

Which version of avisynth ? Which method of converting back to RGB for the screenshot ?

(Note the method in which you convert back to RGB for the screenshot will alter your image)

[zerowalker]

I don´t convert it to RGB, as it´s RGB from the start.

I am using 2.60 MT version, i think it´s: 2012.05.16 that version or 2012.04.03. As newer than that causes some scripts to not work for me (QTGMC)

[StainlessS]

return version

Would remove the guesswork.

[zerowalker]

Thanks, it´s 2012.04.0.

AviSynth 2.60, build:Apr 3 2012

[poisondeathray]

I don´t convert it to RGB, as it´s RGB from the start.



You are converting to YUV.

The screenshot is in RGB.

(You don't view YUV directly, you view a RGB representation of it)

If you use one algorithm (e.g. point) to convert YUV to RGB, it will look different than if you used another algorithm (e.g. lanczos) to do the same conversion to RGB. If you are using something like youtube, you have no control over how it converts to RGB for display

The reason for the avisynth version, is the shift values are different between 2.58 and 2.6

There is no "perfect" way if you are downsampling chroma, there will always be a tradeoff somewhere

[zerowalker]

Okay so there is no use in trying other resampling methods as i don´t know what resampling is used in the end?

Is there any point of having the chroma resampling then, if it only matters when doing the conversion back to RGB?

[poisondeathray]

Okay so there is no use in trying other resampling methods as i don´t know what resampling is used in the end?


It can make a difference. A certain method might be better for the conversion to YUV


Is there any point of having the chroma resampling then, if it only matters when doing the conversion back to RGB?

I never said that.

It makes a difference what you use for each conversion . Just because you might not be able to control the conversion back to RGB, you can control the conversion to YUV

[zerowalker]

Okay, but i don´t get why RGB needs a chroma resampling.
Shouldn´t it just be able to get data that YUV has?

Cause it feels weird that 2 resamplings of the chroma needs to be done for it to show.

When i look at in through Avspmod without converting to RGB, what kind of resampling is done then?


EDIT:

But in still, i would like to be able to make the chroma in the correct place with point.
As it still becomes better quality with easy chroma stuff (like pixel games, 2D.)
I am playing around but i still can´t make it good, it´s always out of place.

I uploaded an example.

Try to make the 3 triangles on the altar look correct and symmetric with point resampling.
For me, it´s not possible, it goes to the left making it become "unsymmetrical", and i can´t fix it.

[poisondeathray]

Okay, but i don´t get why RGB needs a chroma resampling.
Shouldn´t it just be able to get data that YUV has?

Cause it feels weird that 2 resamplings of the chroma needs to be done for it to show.


All displays work in RGB . If you don't want any chroma sampling stay in RGB, or YV24

Going from RGB to YV12 4:2:0 is chroma downsampling . RGB to YV24 4:4:4 won't have any downsampling (full color resolution)

Do you understand what chroma subsampling is ?
http://en.wikipedia.org/wiki/Chroma_subsampling

When you convert to YUV 4:2:0 , you throw away lots of color information. For color, horizontal and vertical dimensions are cut in half. e.g. a 1920x1080 RGB image would only have 960x540 in the U,V planes . That's why it's impossible to go back to RGB and have it look "good" . You can't possibly get 960x540 to look as good as the 1920x1080 original - You either tradeoff chroma aliasing, or blurring (stretch symetrically)


When i look at in through Avspmod without converting to RGB, what kind of resampling is done then?


I think it uses bicubic for the RGB conversion

[TheSkiller]

The way the YV12 video is sampled back to RGB for display can have a huge impact on what you're seeing.

Try this for example:


ConvertToYV12(chromaresample="point")
#now we have YV12, let's convert it back to RGB

ConvertToYV24(chromaresample="point")
MergeChroma(PointResize(width, height, 0, 1))
ConvertToRGB32()

This is as accurate as it gets for point-resampling to YV12 and then back to RGB. In other words, if that doesn't satisfy you, consider it a YV12 limitation.
However the problem remains – most of the time you don't have control over the YV12 to RGB conversion.

[zerowalker]

Yes i know what dowsampling chroma is, and i know that there will be loss of color information one way or the other.
And correct, it uses Bicubic, did some testing.

But what i don´t understand is why it goes out of place with Point.

TheSkiller:

So the limitation with YV12 (you don´t get 1/1 pixel accuracy) is what makes point resampling out of place?
And the only way to solve it is by converting it back to RGB (with some stuff going on before) to make the chroma come into the right place?

That´s really to bad, it was very good results with point, the only thing that fails is that it´s always out of place.
But i guess it was too good to be true, even though it´s only good for 2D stuff like this.

[Gavino]

What is the complete script you are using (or at least the chain of conversions used) to get the results you are seeing?

[zerowalker]

It´s just

Clip

Converttoyv12(matrix="rec709,chromaresample="point")

[Asmodian]

Isn't it just that point for chromaresample uses the top left color values for four Y pixels? This is why the color looks like it moved down and to the left, you copied the color value from the top left pixel to the other three pixels in the 2x2 square.

When converting back to RGB it has no idea what the colors of those pixels were, it only has the color of the top left pixel in the 2x2 square.

If you point resize the RGB to 2x in both dimensions and then do Converttoyv12(matrix="rec709,chromaresample="point") it should be perfect (but also pointless, just use YV24).

[zerowalker]

Probably so.
And if i do make it with:

PointResize(width*2,height*2)
Converttoyv12(matrix="rec709",chromaresample="point")

It does look better.
But if i then use

PointResize(width/2,height/2)

It looks the same, which is fair enough as it´s the same size.

[Gavino]

it was very good results with point, the only thing that fails is that it´s always out of place.
A 'normal' conversion to YV12 (eg with the default chromaresample="bicubic") will construct each new chroma pixel from a weighted average of the source pixels in an area around the point in question, while chromaresample="point" just copies a single pixel (the top left in each block of four).

So with a 'blocky' source like yours, "point" will preserve both the blockiness and the original colours better, because it's not blending the source pixels. However, the positions of block edges will in general not be preserved because the new pixels are both further apart and in different vertical positions.

I think that explains why you got "very good results" but "out of place".

[zerowalker]

yes that´s seems to be the case.
To bad that it went out of place, otherwise it was extremely good results at this particular case.

I guess it´s not possible to fix by shifting the luma afterwards or something?

[Gavino]

I guess it´s not possible to fix by shifting the luma afterwards or something?
No, I don't think so.
For one thing, the chroma shift is not the same for all pixels, it depends on whether the block edges occur at odd or even pixels in the source.

Even if they were all the same (all even, say), I think the required luma shift would be a subpixel one (0.5 pixels), which would introduce a blur, sort of defeating the object.

[zerowalker]

Ah, well good to know.
I tried playing around and well, the only way to make it look good, is by doubling the resolution, convert to yv12 point, then convert to RGB and then resize it back (which defeats the purpose as you get RGB in the end).