strange interlacing artifacts

[raymod2]

I captured some video off a friend's DV tape. I ran it through the following Avisynth filters to see if it is TFF or BFF:

AssumeBFF().SeparateFields()

Some of the video looks like it was shot in progressive mode since every other field looks nearly identical to the field preceding it. Other parts of the video look interlaced (BFF) because moving objects track smoothly from field to field. The strange thing is that every other FIELD looks like it has interlacing artifacts. This doesn't make sense because only a FRAME should have interlacing artifacts. To demonstrate this I have attached the screenshots from four successive fields. Notice how everything looks fine in the 1st and 3rd fields but the 2nd and 4th fields are blurred (look at the windblades in the water and notice they each have a ghost to their right). Am I doing something wrong? Is this BFF interlaced video or something else?

http://img111.imageshack.us/img111/2041/field1zk5.png
http://img341.imageshack.us/img341/1181/field2uv8.png
http://img105.imageshack.us/img105/8798/field3hz7.png
http://img105.imageshack.us/img105/5513/field4jt1.png

[henryho_hk]

It looks more incorrect chroma upsampling. What DV codec are you using?

[raymod2]

How do I determine that? I captured the video using Adobe Premiere. Then I wrote an Avisynth script that opened the file using avisource(). Finally I opened the .avs file in virtualdub to take the screenshots.

[henryho_hk]

Please load your DV AVI file in VirtualDub directly. Then under File -> File Information, it will tell which DV decoder is running and what colorspace the output is.

[raymod2]

It says the Decompressor is "ffdshow Video Codec" but it I don't see where it mentions the colorspace.

[henryho_hk]

It says the Decompressor is "ffdshow Video Codec"

Try
avisource("your_dv_file.avi",pixel_type="YUY2",audio=false)
or
avisource("your_dv_file.avi",pixel_type="YV12",audio=false)

and see if anything changes.

[raymod2]

The problem is present when I use YV12 but it goes away when I use YUY2.

EDIT: I also tried the following script..

AviSource("speed.avi",pixel_type="YUY2",audio=false)
ConvertToYV12()
AssumeBFF().SeparateFields()

This time I see color bleeding in both fields (one field shows bleeding to the left of the windblades and the next bleeds to the right).

[henryho_hk]

Try ConvertToYV12(Interlaced=true)

It looks like a YV12 output bug of ffdshow's DV decoder in NTSC mode. The chroma shifts horizonally. Try Cedocida DV VFW codec.

[scharfis_brain]

It looks like a YV12 output bug of ffdshow's DV decoder in NTSC mode.

No. It is not a bug.

NTSC-DV is YV12. BUT its chroma is sampled totally different compared to PAL-DV-YV12 or MPEG-YV12.

NTSC-DV is YUV 4:1:1 while PAL-DV is YUV 4:2:0.

AVISynth neither supports NTSC-DV-YV12 nor supports PAL-DV-YV12 natively.

[GodofaGap]

4:1:1 is not YV12. The YV12 fourcc is used for a specific way of storing YUV 4:2:0.

[scharfis_brain]

I know at least three variants of YUV subsampling that are carried using the YV12 fourCC:
http://www.mir.com/DMG/chroma.html

MPEG1 - YUV 4:2:0
MPEG2 - YUV 4:2:0
PAL-DV - YUV 4:2:0

NTSC-DV is YUV 4:1:1 and contains the same amount of information but placed in a different way. So it is possible to transfer it using the YV12 FourCC (Of course I don't know of an application that is capable to interpret it).

Besides this do you have sources dening YUV 4:1:1 being a type of YV12?

the fourcc YV12 means to me that we have 12 bits/pixel in average. and this is valid for all four types.

[GodofaGap]

Ah, so for you I420 is also YV12. The ball doesn't roll that way.

The confusion comes from MPEG codecs using a 4:2:0 format that is decodable to YV12, they don't necessarily use the YV12 format internally. 4:2:0 does not equal YV12, there are many ways in which a 4:2:0 format could be stored. (I420 is one for example). The page you linked to doesn't mention YV12 one time.

http://www.fourcc.org/yuv.php#YV12
http://forums.virtualdub.org/index.php?act=ST&f=15&t=11044&hl=&

YV12 is a four-character code (FOURCC) used with Windows bitmaps and with AVI that denotes a very specific non-interlaced 4:2:0 YCbCr format. It is not a general designation for a particular color space or chroma subsampling pattern. MPEG-2 doesn't use YV12 because it doesn't use FOURCCs to denote color encodings — it uses its own color subsampling field which chooses one of the several defined MPEG-2 formats. Some of these can then be decoded to YV12 format.

[raymod2]

ConvertToYV12(Interlaced=true) seems to work right (no color bleeding).

So what is the correct way to handle this video? I will be deinterlacing it with interp() and TDeint(), resizing it with LanczosResize() and then I need to output in YV12 format (x264 requires this).

[henryho_hk]

NTSC-DV is YV12.

I know PAL DV & NTSC DV are different. But I think FFDShow has a bug when decoding NTSC DV in YV12 mode. The chroma plane(s) seem shifted for about 8 pixels from the luma plane. I observed it for a few DV clips already. Maybe it's the chroma put in a wrong temporal position, I dunno. I would definitely call it a bug.

[scharfis_brain]

henryho_hk: no. it is not a bug. ffdshow just puts out YUV411 flagged as YV12. But YV12 always is interpreted as YUV420. So you're likely to get these strange 'errors'.

An AVS-function like convert411to422() for sure could be done.

GodofaGap: Probably you are right, cause only 420 formats give 'viewable' results (with more or less chroma shifting depending on the actual sampling layout)
But how else can one transfer the unaltered chroma to an application?
YV12 already allows us to do so (even if the results aren't satisfying yet), because all oy these types have one in common: the amount of subsampling.

Also I interpret the sentence you made bold in the quote this way: YV12 just says that you got 4 luma pixels sharing one 'pixel' of chromacity. The sampling layout (NTSC-DV, PAL-DV, MPEG2, MPEG1) doesn't matter.

[henryho_hk]

ConvertToYV12(Interlaced=true) seems to work right (no color bleeding)

1) Install Cedocida DV codec. Remember to set its YV12 output to MPEG2 Interlaced.
2) avisource(...... pixel_type="YV12", ....)
3) tdeint(... order=0....)
4) [some filters you like]
5) lanczosresize(640,480,8,0,704,480)
6) [mild denoiser if needed]

[GodofaGap]

Also I interpret the sentence you made bold in the quote this way: YV12 just says that you got 4 luma pixels sharing one 'pixel' of chromacity. The sampling layout (NTSC-DV, PAL-DV, MPEG2, MPEG1) doesn't matter.
Im sorry, you still dont get it. YV12 is a specific way of reading 4:2:0 data as described by fourcc.org. The fourcc "YV12" is used to indicate that specific way of reading data. You cannot (I repeat cannot) bend it into something else. The layout of the data is exactly the point of the fourcc. (Imagine the same fourcc be used by On2, MPEG4 and WMV. How much sence would make that?) And yes, interlaced YV12 as done by Avisynth is a hack. Of course you can have interlaced 4:2:0, but the layout of YV12 does not support interlaced content.

You still confuse the sampling 4:2:0 with YV12. This is understandable because in practice they are very often synonyms, but it is wrong. YV12 is a specific way of storing 4:2:0 content.

[henryho_hk]

scharfis_brain, shouldn't ffdshow give MPEG2 YV12 output, since it has a DV output option also... ?

Edit: I discovered that ffdshow's DV output option is disabled under VFW mode. It is available only under video decoder mode.

[henryho_hk]

GodofaGap, how should we name the colorspace of interlaced MPEG2 and MPEG4?

[GodofaGap]

There is no appropriate fourcc for interlaced 4:2:0. If there was, non of these problems would exist.

[Wilbert]

henryho_hk: no. it is not a bug. ffdshow just puts out YUV411 flagged as YV12.
I thought YUV411 was resampled to YV12?

Also I interpret the sentence you made bold in the quote this way: YV12 just says that you got 4 luma pixels sharing one 'pixel' of chromacity. The sampling layout (NTSC-DV, PAL-DV, MPEG2, MPEG1) doesn't matter.
GodofaGap is completely correct, but for completeness i wanted to add the following:

PAL-DV, MPEG2 (in DVD), MPEG1 are all YV12, and YV12 is a 4:2:0 format. NTSC-DV is YUV411, which is a 4:1:1 format. Both are 12 bit.

There is no appropriate fourcc for interlaced 4:2:0. If there was, non of these problems would exist.
The problem is that interlaced is a per-frame property. Some frames can be flagged as interlaced and others as progressive within the same stream. Besides that, interlaced 4:2:0 itself is an ill-defined concept. How do you upsample, for example, progressive footage which is flagged as interlaced?

[GodofaGap]

The problem is that interlaced is a per-frame property. Some frames can be flagged as interlaced and others as progressive within the same stream. Besides that, interlaced 4:2:0 itself is an ill-defined concept. How do you upsample, for example, progressive footage which is flagged as interlaced?
You could even take it further and say interlacing is a property of only the moving areas within one frame. But yes the concept of interlaced 4:2:0 is broken. Let's hope that in the future of digital formats people step aside from the resolution race for a moment and use the extra bandwidth available on new storage media to go to 4:2:2 sampling.

[scharfis_brain]

GodofaGap: After reading the things you wrote again during some calm minutes I understood what you wanted to say.
Indeed YV12 is for one type of YUV420 only. (the one that MPEG2 uses)
But this type IMO contains the interlaced sampling, too.

You could even take it further and say interlacing is a property of only the moving areas within one frame. But yes the concept of interlaced 4:2:0 is broken.

I won't actually call it broken. It is just a matter how to treat it 'correctly' .

All current implementations are broken. Like mostly all implementation of LCD-TVs are broken to display interlaced contents.
The problem is: there aren't suitable deinterlacers built into many displays and/or receivers/DVDplayers.

Some time ago I implemented an motion adaptive YV12 -> YUY2 upsampler (colorspace conversion is nothing else than a a special kind of resizing operation).
Its results on interlaced footage are damn good compared to a simple converttoyuy2(interlaced=true).

It consisted mainly of these AVIsynth commands:


Input=Last

# All included converttoyuy2() commands won't convert any chroma information here.
# They are only abused to 'assume' the YUY2 colorspace
Y=Input.GreyScale().ConvertToYUY2()

#Bob-deinterlace chroma with ELA interpolation, motion adaption and Pulldown detection
U=Input.UtoY().TDeint(mode=1, full=false, mode=3).ConvertToYUY2()
V=Input.VtoY().TDeint(mode=1, full=false, mode=3).ConvertToYUY2()

#Upsample the Chroma to full frame height. Maybe ELA-Resizing will perform a little better
U=U.BicubicResize(Y.Width()/2, Y.Height)
V=V.BicubicResize(Y.Width()/2, Y.Height)

#Reinterlace the Chroma
U=U.SeparateFields.Selectevery(4,0,3).Weave()
U=U.SeparateFields.Selectevery(4,0,3).Weave()
Output=YtoUV(U,V,Y)


So interlaced YUV420 not only is a per frame property. It also is a per pixel property (is pixel moving or not?)

[GodofaGap]

But this type (=YV12) IMO contains the interlaced sampling, too.
(I added the italics for clarification)

This is where I disagree (and apparantly the author of VirtualDub too). Having said that, I cannot deny that unfortunately people are using it in this way. Cramming multiple formats into one fourcc, eliminates the purpose of having a fourcc (which is to distinguish between formats, and interlaced YV12 is interpret different from progressive so strictly speaking it is not the same format) to begin with, but so be it.

[EDIT]Actually I disagree completely. MPEG2 does not use YV12. The link that has been given (http://www.mir.com/DMG/chroma.html) seems to describe a packed format while YV12 is a planar format. But I am not sure how to interpret the graphs there correctly.

[Wilbert]

[EDIT]Actually I disagree completely. MPEG2 does not use YV12. The link that has been given (http://www.mir.com/DMG/chroma.html) seems to describe a packed format while YV12 is a planar format.
No, it doesn't. It doesn't say that the formats are packed or planar.

The important issue though, is that the mpeg-2/dvd specs don't mentioned YV12, they are just referring to progressive 4:2:0 and interlaced 4:2:0. The former happens to be YV12. (In AviSynth the former is defined as progressive YV12 and the latter as interlaced YV12, but that's really AviSynth's definition, or a hack as GodofaGap called it in this thread.)

I hope it's clear now :)

[henryho_hk]

What about a new fourcc "IY12"?