Why YV12 is bad for your Anime

[wanton]

I'm writing this because i think many people won't know how yv12 version of mpeg2dec.dll decodes to yv12. If you have 29.97fps mpeg-2 material then you use force film for normal movie that is 100% FILM it's good chance video probably encoded with progressive_frame on meaning yuv data comes out as normal YV12 progressive data. But if you don't have force film one those extra frames made by repeat first field flags in mpeg2 data allways are in interlaced YV12...meaning if you don't use force film there will be wrong chroma decoding for those extra frames. In anime many video is encoded as hybrid with real NTSC and FILM data mixed together so YV12 data isn't very practical to use with those since you get lot of wrong chroma decoding since avisynth doesn't have any kind of flags for YV12 data if it's interlaced or not. Only way this would work is that every frame had an interlaced flags.

But that also has it's problem if you use something like decomb since it assumes you are using interlaced YV12. Author original of original DVD2AVI made DVD2AVI just because of flaskmpeg decoded chroma wrong for interlaced frames here is a link

http://arbor.ee.ntu.edu.tw/~jackei/dvd2avi/interlace/

You can test this yourself. Don't use Force film for pure film material and you can see that interlaced frames has wrong chroma. If you are an programmer you might want to check original mpeg2dec.dll sourcecode since you can see there that it converts chroma diffrently from YV12->YUY2 depending if frame has progressive_frame flag on or not.

If you material is true film and you use force film it's should be fine or material is truly interlaced mpeg-2 then you might want to do

ConvertToYUY2( interlaced=true )
ConvertToYV12()

to turn interlaced YV12 to progressive one if you want to encoded to example XVID since it assumes progressive YV12 input like probably everything else too since i think YV12 format in general is progressive format atleast in AVI's since you can't specify anywhere it to be interlaced atleast in microsoft site YV12 seems to specified as progressive one only. Also there should be any quality loss if you encode mpeg-2 file with YV12 progressive converted to YUY2 and then back to YV12 if you encode progressive material (example 23.976fps with pulldown).

And last note...i bet most YV12 filters asume progressive YV12 since avisynth doesn't specify if it's interlaced or not and most filters don't have setting to use interlaced video. Since if something assumes progressive chroma and it's actually interlaced one then filter gets wrong chroma value for every second line.

What i would suggest for hybrid material like anime is to use YUY2 version of mpeg2dec.dll and do decomb and then use ConvertToYV12 after it to run filters and such. Because frames are combined to progressive you can use normal ConvertToYV12() to turn them to progressive YV12 to use directly with XVID example.

If you use avi source then you probably doesn't have to worry at all since if they are decoded to YV12 they are in progressive YV12 mode allready.

I made this text because after transfering from avisynth 2.0 to 2.5 i had hard time with hybrid anime material so this might help other people. Now i think i should go and compile my self an old MPEG2DEC.DLL with YUY2 support to use with this kind of material.
Ofcourse it would be better if author of MPEG2DEC3 would add YUY2 decoding support since it has lot of nice options.

P.S. Didn't much check what i wrote so i hope you people got the idea :)

[mf]

Thanks for spreading a useless rumour. (that size tag was for users just reading the thread quickly, so that they will notice this.)

The interlaced tag is part of the YV12 "bitstream" (dunno if they call it this with colorspaces). And also AVISynth recognizes and stores it. Any chroma errors you might see on anime DVDs are from bad mastering, and using or not using YV12 will not matter. The DVDs themselves are stored as YV12, and converting to YUY2 will not help you. I have a hybrid anime DVD here, and using MPEG2Decoder and Decomb works perfectly in YV12. No chroma ghosting. I have a filter with a bug though, FieldMatch(), which incorrectly handles chroma, which DOES cause ghosting. Now this is not AVISynth's fault, not YV12's fault, or anybody's fault except the programmer, who overlooked something. Make sure you don't have any of these filters, and if you do, and if they are supposed to work on interlaced material (most filters except deinterlacers don't so it's not wise to use them before a deinterlacer), you should contact the author about it. Do anything else actually, than spreading strange rumours.

Doing unnecessary color conversions is bad. Don't do it.

[Kika]

@mf
You'r right. But: If you mix Filters which need YUV2 or RGB and some of them, using YV12, you CAN produce visible Noise in Animes. It's not a good idea to make more Colorspace changings than 1.

[mf]

Originally posted by Kika
@mf
You'r right. But: If you mix Filters which need YUV2 or RGB and some of them, using YV12, you CAN produce visible Noise in Animes. It's not a good idea to make more Colorspace changings than 1.
That's what I said.

[ssjkakaroto]

thx for that size tag mf ;)

[sh0dan]

I wanted to respond, but didn't have the energy - thanks mf!

Stick to one colorspace. If you are able to keep away from doing ANY colorconversions from decode to encode you are still better off with YV12 chroma than resampled (and in effect blurred) chroma.

If there are any problems with the chroma you are getting using YV12, a MergeChroma(blur(1.5)) will give you the chroma you want.

These issues have been INTENSELY discussed here anyway. There is no 100% correct solution, as the one you describe fail on all material, where the interlaced flags does NOT correspond with the chroma placement (and yes, there are a lot of these cases, unfortunately).

[trbarry]

I won't rehash the details but it is still my belief that either UnComb or Decomb will give better and more accurate color detail when remaining in YV12.

- Tom

[MasterYoshidino]

nani?
yeesh another useless thread.
defiantely agree on the decomb in YV12 ;)
it's "ok" to convert to YUY2 but only do it after the deinterlacers/deblockers.