Interlaced YV12 vs YUY2 & Deinterlacing

[Asmodian]

I decided to check what happens when deinterlacing a YV12 vs YUY2 interlaced capture using a worst case test image of red bottom field with a cyan top field. The results are odd to me but are probably expected by anyone who understands it (not me :confused:).

I used this avisynth script:
red=ImageSource("H:\stills\red.bmp", use_DevIL=true, pixel_type="RGB24").PointResize(80, 30)
cyan=ImageSource("H:\stills\cyan.bmp", use_DevIL=true, pixel_type="RGB24").PointResize(80, 30)
Interleave(red, cyan)
AssumeFieldBased()
Weave()
PointResize(320,240)

http://i1222.photobucket.com/albums/dd496/asmodian3/redcyanRGB.png

Adding:
ConvertToYV12()
Before the final resize gives:
http://i1222.photobucket.com/albums/dd496/asmodian3/redcyanYV12_convert.png
This is as if captured in YUY2 and converted to YV12 after deinterlacing.

Using ConvetToYV12(interlaced=true) instead gives:
http://i1222.photobucket.com/albums/dd496/asmodian3/redcyanYV12.png
Notice the chroma is really off every other line but the luma is correct. This is as if captured in YV12 and kept in YV12. ConvertToYUY2(interlaced=true) gives correct colors, further encoding in YV12 will give "correct" grey image, this does blur the color vertically but that cannot be seen in this test image except the faint color at the top and bottom of the grey image gets even fainter.

Is this expected behavior? Is there something wrong with my testing method?

Bobbing the grey video gives full frame dark gray+full frame light gray and bobbing the final case gives full frame red+full frame cyan video.

I assume grey is more correct. The average of the two rows should be grey but the third gives me the fields I started with if I bob. They both seem correct for their situations. :confused:

Does this mean you need to bob deinterlace if you capture in YV12? It seems like IVTC would be less than ideal if you captured in YV12. Edit: Maybe do ConvertToYUY2(interlaced=true) before IVTC? This will blur the chroma across four lines when converted back to YV12.. hmm :p

note: All other methods I have tried to do these color conversions always came out the same. This is not the only way I have produced these results, just the simplest. I have always used ImageSource(), Interleave(), ConvertToYV12() to generate my "interlaced video" rather than a real YV12 capture, does this cause an issue? I used QTGMC() to do the bobbing.

[Didée]

First of all, set the "interlaced=true" parameter in all of the ConvertToXXX() calls. Then check again. :)

[Gavino]

Is this expected behavior? Is there something wrong with my testing method?
It is expected behaviour for the script you used.
However, your method is wrong since default RGB<->YV12 conversions will mix up the chroma of an interlaced source.
You need to use interlaced=true where appropriate, ie for all conversions done after the Weave() - both cases for your ConvertToRGB24() and the first case for ConvertToYV12().

[Asmodian]

It is expected behaviour for the script you used.
However, your method is wrong since default RGB<->YV12 conversions will mix up the chroma of an interlaced source.
You need to use interlaced=true where appropriate, ie for all conversions done after the Weave() - both cases for your ConvertToRGB24() and the first case for ConvertToYV12().

The ConvertToRGB24() was just to be able to save it using imagewriter().

It works great in avisynth with ConvertToRGB24(interlaced=true). However if I encode to x264 with --tff set (or without it) and watch it (I am using Lav 0.42 and MadVR 0.39) I still get the chroma mismatch.

If I open this script in virtualdub for example:
red=ImageSource("H:\stills\red.bmp", use_DevIL=true, pixel_type="RGB24").PointResize(80, 30)
cyan=ImageSource("H:\stills\cyan.bmp", use_DevIL=true, pixel_type="RGB24").PointResize(80, 30)
Interleave(red, cyan)
AssumeFieldBased()
Weave()
ConvertToYV12(interlaced=true)


Is there a way to tell others outside avisynth weather or not the YV12 is interlaced?

It doesn't seem like there is a way to convert interlaced YV12 to progressive YV12 without vertically bluring the chroma.

Sorry if I am just being dense. :o

[Asmodian]

I edited the first post after feedback and a better understanding of what the issue is.

This isn't a problem of any of the tools used, it is a problem of interlaced YV12 video. I think ;)

[Gavino]

It works great in avisynth with ConvertToRGB24(interlaced=true). However if I encode to x264 with --tff set (or without it) and watch it (I am using Lav 0.42 and MadVR 0.39) I still get the chroma mismatch.
You need to watch it on something that deinterlaces it, or at least takes interlacing into account when converting to RGB for display.
Is there a way to tell others outside avisynth weather or not the YV12 is interlaced?
No. VirtualDub always treats YV12 as progressive when converting to RGB, so you should convert interlaced YV12 to RGB in your script (with interlaced=true) before passing it to VDub.
It doesn't seem like there is a way to convert interlaced YV12 to progressive YV12 without vertically bluring the chroma.
Chroma will be vertically blurred because of the subsampling, but if done correctly the operation will not mix chroma from different fields together. Even Bob() gets this right.

[Asmodian]

Chroma will be vertically blurred because of the subsampling, but if done correctly the operation will not mix chroma from different fields together. Even Bob() gets this right.

But you couldn't IVTC it and save it as progressive? If you had a progressive frame encoded in interlaced YV12 you couldn't restore it to progressive YV12 right?

[Gavino]

Not sure if I understand your question correctly, but telecined YV12 can be IVTC'd to perfectly recover the original progressive YV12 frames, including chroma. The process is just the same as for YUY2 or RGB.

[Asmodian]

Here is another example that shows the problem as I understand it now:

The interleaves+weaves are just to generate the "worst case" image, four lines of cyan, four of red. This is a camcorder on a tripod looking at some ugly wallpaper from just the right distance, or a much weaker effect on more realistic material. :p

This script:
red=ImageSource("H:\stills\red.bmp", use_DevIL=true, pixel_type="RGB24").PointResize(80, 10)
cyan=ImageSource("H:\stills\cyan.bmp", use_DevIL=true, pixel_type="RGB24").PointResize(80, 10)
a=Interleave(red, cyan).AssumeFieldBased().Weave()
b=Interleave(a, a).AssumeFieldBased().Weave()
Interleave(b,b).AssumeFieldBased().Weave()
PointResize(320,320)
ImageWriter("H:\stills\redcyan", type="png")

http://i1222.photobucket.com/albums/dd496/asmodian3/redcyanRGB3-1.png

This script:
red=ImageSource("H:\stills\red.bmp", use_DevIL=true, pixel_type="RGB24").PointResize(80, 10)
cyan=ImageSource("H:\stills\cyan.bmp", use_DevIL=true, pixel_type="RGB24").PointResize(80, 10)
a=Interleave(red, cyan).AssumeFieldBased().Weave()
b=Interleave(a, a).AssumeFieldBased().Weave()
Interleave(b,b).AssumeFieldBased().Weave()
ConvertToYV12(interlaced=true)
ConvertToRGB24(interlaced=true)
PointResize(320,320)
ImageWriter("H:\stills\redcyan", type="png")

http://i1222.photobucket.com/albums/dd496/asmodian3/redcyanYV12_3i-1.png

And:
red=ImageSource("H:\stills\red.bmp", use_DevIL=true, pixel_type="RGB24").PointResize(80, 10)
cyan=ImageSource("H:\stills\cyan.bmp", use_DevIL=true, pixel_type="RGB24").PointResize(80, 10)
a=Interleave(red, cyan).AssumeFieldBased().Weave()
b=Interleave(a, a).AssumeFieldBased().Weave()
Interleave(b,b).AssumeFieldBased().Weave()
ConvertToYV12()
ConvertToRGB24()
PointResize(320,320)
ImageWriter("H:\stills\redcyan", type="png")

http://i1222.photobucket.com/albums/dd496/asmodian3/redcyanYV12_4.png

[Gavino]

That's different to your original test.
Now you have both red lines and cyan lines present in each field, so it's inevitable you will get blurring between them following chroma resampling (whether treated as interlaced or not).

A further complication in the interlaced case is that there is a bug in Avisynth's conversion of interlaced YV12 to YUY2 (which is used during conversion to RGB). This bug causes a chroma shift which affects each field differently. See this thread (which also touches on the VirtualDub issue). The bug is often difficult to see in practice but is clearly shown up by extreme examples like yours.

[Asmodian]

Ok thanks! That is a very useful thread on this topic and helps clears it up for me. :thanks:

That's different to your original test.
Now you have both red lines and cyan lines present in each field, so it's inevitable you will get blurring between them following chroma resampling (whether treated as interlaced or not.

Yes, this was the idea. It shouldn't be blurred without anti-aliasing during the chroma resize (which is normally a good thing, just not for this test image). I wanted to show that using interlaced YV12 blurs chroma more than progressive YV12 once you have gotten it to progressive RGB for display.

It seems like this is a well known and understood issue that still exists (and it has to exist, other bugs aside) with the current Avisynth 2.6. It is a minor defect when converting real life material once. I am also bad at designing test images. :o

All this convinces me to make sure to keep capturing in YUY2 and convert to YV12 near the end of processing, after deinterlacing. While avisynth can handle 4:2:0 with interlaced chroma and progressive luma the YV12 standard does not allow for this so it gets weird when exporting to Vdub (if you want to use fast recompress) or x264.

It is currently not possible to get progressive luma + interlaced YV12 into x264 and have it treated as interlaced YV12 through playback, right? I guess you could open it in avisynth and do a ConvertToYUY2(interlaced=true) but short of that?

Is it still advised to use your YV12ToYUY2_26 script when doing YV12 to YUY2, interlaced or not, or has avisynth 2.6 incorporated this method?

[Gavino]

Is it still advised to use your YV12ToYUY2_26 script when doing YV12 to YUY2, interlaced or not, or has avisynth 2.6 incorporated this method?
The behaviour of a simple ConvertToYUY2() or ConvertToYUY2(interlaced=true) has not changed in 2.6, but I believe the new parameters work correctly in 2.6 Alpha 3, so you can use ConvertToYUY2(chromaresample="bilinear") (plus interlaced=true where appropriate) in place of my workaround functions. (You could also use some other resizer, eg chromaresample="bicubic".)

It is currently not possible to get progressive luma + interlaced YV12 into x264 and have it treated as interlaced YV12 through playback, right?
Not sure - perhaps someone with more knowledge of x264 can answer this?

[Asmodian]

Thanks for all your help understanding this and sorry to keep working with these test images, I don't know another way to test my theories.

This shows the chroma offset bug when doing YV12->YUY2? They look like my previous images using bilinear but I wanted to use point resize to keep the color mixing only due to subsampleing. Edit: Oops, point resize means there is no color mixing period, it just uses the color of the top pixel in the pair when subsampleing.

ConvertToYV12(interlaced=false, chromaresample="point")
ConvertToRGB24(interlaced=false, chromaresample="point")
http://i1222.photobucket.com/albums/dd496/asmodian3/redcyanYV12point2.png

ConvertToYV12(interlaced=true, chromaresample="point")
ConvertToRGB24(interlaced=true, chromaresample="point")
http://i1222.photobucket.com/albums/dd496/asmodian3/redcyanYV12ipoint2.png

From what I understand both these cases would be lossless (given this test image (http://i1222.photobucket.com/albums/dd496/asmodian3/redcyanRGBsmall.png)) minus the YV12->YUY2 offset bug.

Edit: I can confirm that both of these videos (minus the ConvertToRGB24s) when encoded with x264 look perfect using point resize in MadVR.

[Gavino]

From what I understand both these cases would be lossless (given this test image (http://i1222.photobucket.com/albums/dd496/asmodian3/redcyanRGBsmall.png)) minus the YV12->YUY2 offset bug.
I too would expect both these pairs of conversions to be lossless when using "point", and I thought that the offset bugs affecting chromaresample had been fixed in 2.6a3. Your results suggest there is still a problem.

I will try to investigate further (but I'm not sure I will have time today).

[Mounir]

I find these tests informative and not reassuring i thought the v2.6 solved entirely the issue but there is work on the table it appears
I'd be interested to see the result of the following conversions... if you can with the latest v2.6:

video source: YUY2
converttoyv12()
converttorgb32()
converttoyv12()

[Asmodian]

These tests were actually made with Avisynth 2.6 MT. version() tells me: AviSynth 2.60, build:Sep 13 2011 [03:56:35]

I hope this isn't an issue, I assumed this wouldn't be different in MT vs ST but assumptions are not safe. I will retest with alpha3 [110525] but I am running a very long QTGMC() atm.

ImageSource("H:\stills\redcyanRGBsmall.png", use_DevIL=true, pixel_type="RGB24").ConvertToYUY2()
ConvertToYV12()
ConvertToRGB32()
ConvertToYV12()
ConvertToRGB24()
PointResize(last.width*4,last.height*4)
ImageWriter("H:\stills\redcyan", type="png")

http://i1222.photobucket.com/albums/dd496/asmodian3/redcyanYUY2YV12.png
I swear it looks the same if I delete everything below the last ConvertToYV12() and encode with x264.

This issue is basically impossible to notice with a real image. Only a weird test image like this will show so much change when you blend two lines' color together with bilinear resize (red+cyan=grey).

Edit:
Interestingly if I use pointresize in MadVR to play said video (using bilinear gives the above image):
http://i1222.photobucket.com/albums/dd496/asmodian3/RedCyanMadVR_2.png

This is what I would want to happen with YV12. The only issue I have now is the odd offset when doing YV12->YUY2 that I can only notice if I use point resize. I wonder if it happens using other resize methods?

Also now I am sure that capturing interlaced YV12 and encoding progressive YV12 is not as good as capturing YUY2 and encoding progressive YV12 though it might be hard to actually spot the difference with a real image.

Here is an interesting image to test with, it exaggerates most chroma issues so you can see them:
http://i1222.photobucket.com/albums/dd496/asmodian3/multicolor2.png