converttoyv12() chroma layers too high

[10L23r]

Take a look at this:

original image
http://i.imgur.com/YFKEA.png

(magnified 4x)
http://i.imgur.com/u6v1Z.png

imagesource("test.png").converttoyv12()
http://i.imgur.com/OJPTo.png

imagesource("test.png") > virtualdub > xvid > avisource("test.avi")
http://i.imgur.com/mC566l.png

The chroma layers in the second image seem to be too high by a little. Encoding in xvid without converttoyv12() doesn't cause this problem o.O

[poisondeathray]

I'm getting different results. Your image #2 and #3 don't look right; they look like the chroma has been blurred

[Gavino]

I haven't tried #3, but I get the same results as 10L23r for #2.
You have to pointresize back to 80x80 before doing the conversion (assuming that's what he means by magnified x4)

ImageSource("test.png")
PointResize(80,80)
ConvertToYV12()
PointResize(320,320)

[poisondeathray]

ok, I was applying converttoyv12() to the 1st image as presented

[Gavino]

It's possible this is the result of the chroma shift on RGB->YUY2 conversion (conversion to YV12 goes via YUY2).

Just to confuse things further, using ConvertToYUY2 instead of YV12 gives a different result as there is no horizontal chroma subsampling.

[Wilbert]

The chroma layers in the second image seem to be too high by a little.
It's a result of the conversion to YV12 and the text being 1 pixel wide (remember chroma will be averaged). Try converting to YV24 instead of YV12 and the problem won't show up.

Encoding in XviD should give the same as pic2 though, so i'm surprised that's not the case.

@Gavino, the latest csv (with the fix for that chroma shift included) gives the same result (as pic2).

[Gavino]

Encoding in XviD should give the same as pic2 though, so i'm surprised that's not the case.
Exactly. There must be some difference between XviD's conversion to YV12 and Avisynth's method (which is why I thought maybe the shift bug was to blame).

[10L23r]

I updated the original post with the test.png file.

this makes the chroma's exactly centered:

function manual_yv12(clip a) {
y=a.converttoyv24()
half=y.spline36resize(a.width/2,a.height/2)
YtoUV(half.UtoY(),half.VtoY(),y)
}

[Wilbert]

@10L23r,

Are you sure XviD won't result in the same problem? In my case it will. Could you explain how you created the XviD file and post it?

[10L23r]

test.png is the 320 x 240 image in the first post

test.avs
imagesource("test.png")

open test.avs in VirtualDubMod

File > Save As
File Name: test.avi
Compression: change to xvid

xvid.avs
avisource("test.avi")

[Didée]

@10L23r: Seems so trivial, but even that simple description is not clear beyond doubt. :p

Vdub's default mode is "full processing mode". Did you use that, or did you change to another mode?

[10L23r]

Yes I used the default, full processing mode.

[Zarxrax]

There's your problem.
That does a conversion to RGB ;p

Wait no, I'm dumb. The source is already RGB.
But I think vdub may be doing some conversion in that case.

[10L23r]

I tried fast recompress, normal, and full processing. All give the same results