YV12 capture codec. Which one?

[acrespo]

I have search for YV12 codecs for analog capture. I found this:

VBLE - I can use to capture because it's fast, but the file size is very big.

FFV1 - The file size is great, ideal for me, but it's very CPU intensive and I can't use to capture.

Loco - The same of FFV1

Morgan JPEG - It's good but encode in YUY2. Only decode in YV12.

There are other options comments?
I want to use YV12 because the file size is less than YUY2.

[WildCelt]

From what I have read, capturing in YV12 is not the way to go, mainly because of Chroma Upsampling Errors (http://members.aol.com/ajaynejr/vidbug2.htm)

[acrespo]

I didn't know this problem. Tanks WildCelt. I will make tests for verify this problem.

[Arachnotron]

I was under the impression the chroma upsampling error is a MPEG2 related problem, nothing to do with YV12, which relates to how Y,U and V data are stored in an array.:confused: :confused:

[edit] ok, it's because with YV12 you share U and V within a 4 pixel macropixel square, i.e. over two lines, while yuv2 shares U and V only in a 2 pixel macropixel on the same line.

I wonder why avisynth is moving towards YV12? It must be a lot more difficult to deinterlace if your source is YV12.

even more :confused: :confused: :confused:

[trevlac]

Originally posted by Arachnotron
I was under the impression the chroma upsampling error is a MPEG2 related problem, nothing to do with YV12, which relates to how Y,U and V data are stored in an array.:confused: :confused:

YV12 is a storage format for 4:2:0 (1/2 width samples & 1/2 height). MPEG2 is also a YUV 4:2:0 format. I belive IYUV and I420 are also 4:2:0 with a different order of info.

Here is my guess... :D

YUV (more appropriately YCbCr) is most likely 4:2:2 comming from a capture card, like YUY2. If you ask for YV12 you get 4:2:0 YCbCr, which means for progressive, every other line shares color. For interlaced, the issue is not clear. I believe it would be proper to sample interlaced fields seperately. That way every other line for each field shares color.

Now you can see the confusion when you convert from YV12 to a higher sample like 4:4:4 RGB or 4:2:2 YUV. The converter needs to know how the color was sampled and how to go back up.

So, it seems to me that if you capture in YV12, you need to know how your source deals with color (like a dvd player), and what your filters and codecs do. For example, if you open an YV12 source in Virtualdub, your default YV12 codec will convert it to RGB 4:4:4.

The big confusion I have is why do the avisynth people want to work in 4:2:0? Maybe their source is PAL DV and their destination is MPEG2. In that case, if they are careful, they don't need to change color spaces / subsample rates, but they'd have to be very careful and know what does what.

If however, they use analog capture, then non-component input is always YUV/YIQ which 'naturally' converts to YCbCr 4:2:2.

Here is a quote from a microsoft 'capture guide'

It is important to note that if you capture to an .avi file with a 4:2:0 pixel format (for example I420, YV12, or IYUV), you will not be able to maintain the interlacing in the video because the 4:2:0 pixel formats do not carry chroma information on the alternate lines that contain the data for the second field. If you are working with progressive-scan source video or if you are working with interlaced source video that you plan to deinterlace, then a 4:2:0 pixel format is appropriate because the output file is smaller than an .avi file with a 4:2:2 pixel format.

They seem to imply that you lose the color in 1 field if you do YV12.

Trev


edit

From a quick read, I guess the YV12 support in avisynth is for mpeg2 input without conversions. That way you can go to mpeg4?

So it seems most 4:2:0 codecs are probably related to mpeg4 (divx/xvid) stuff, which I know very little about. :)

[Arachnotron]

Hi Trev,

I thought this subject would bring you in :D

My guess, but I didn't know enough about this issue to be sure, was that if you want to work with an analogue source like tape or TV, your best bet would be to work yuv2 (or RGB24) throughout, and only convert to yv12 as a last step for DVD production (if ever at all)

Even if your codec stores the fields separately, you are throwing away vertical information before deinterlacing. (or rather, when interlaced, your macropixel must be 2 hor x 4 vert)

But wat kept me from this conclusion was the emphasis put on YV12 by the avisynth people and the number of people developing and searching for yv12 capture codecs. So maybe I am missing something here.

Maybe Wilbert, as the maintainer of the YV12 faq, can shed some light on this issue?

[edit] see this (http://forum.doom9.org/showthread.php?s=&threadid=42708&perpage=20&pagenumber=1) thread for the deinterlaced/interlaced info

[trevlac]

Hey,

I'm with you on the capture thing. Yuy2 and then I'd like to work in RGB because it is easier for me to understand (what the heck should a color difference bit look like anyway).

The problem I encountered with YUY2 to RGB is that most codec/converts give no options on how they map ranges and how they 'upsample'. YCbCr has a normal range of 16-235 and RGB wont look right unless it has a range of 0-255. So to go from 4:2:2 YCbCr to 4:4:4 RGB, the code usually just doubles the CbCr and maps 16-235 to 0-255. It could be way better. There is possibly some good data in the YCbCr 236-253 range. With a direct conversion, this is lost. I doubt the 2-16 has much. Also, I don't see why the CbCr can't be upsampled using the info in the Luma to enhance it. I'd guess hi end dvd players do this.

Anyhow, Wilbert pointed me to this (http://www.avisynth.org/index.php?page=ColorSpaces) link a while ago. Since then, I've learned alot about colorspaces. I think the like is just a tad off on a few terms, but very informative.

Maybe Wilbert can set us straight....:D


edit After having re-read wilberts faq, I think YV12 support is mainly for an mpeg4 target. Going from PAL DV or MPEG2 would require less space and be faster without losing any color info (it was never there). I can't see why you'd use it for analog capture, unless you are going right to mpeg4.

I'd use a smaller frame size if I wanted a smaller file and less cpu usage. :D

[Arachnotron]

(Very off-topic ;) )

Why not map directly to RGB without converting? While in RGB mode everything will look wrong, but if you convert back to yuy2 at the end in the same way, the result should be ok and you loose the conversion rounding errors.

Of course, that means writing your own conversion routine... :)


As to yv12: I can understand the advantage for coding; having three 2D arrays is much easier to work with than the YUY2 packed 4:2:2 1D array.

But why YV12 and not say YV16? i.e. a planar format with the same amount of info as YUV2?

But perhaps this should be in the avisynth forum. :)

[trevlac]

Originally posted by Arachnotron
(Very off-topic ;) )

Why not map directly to RGB without converting? While in RGB mode everything will look wrong, but if you convert back to yuy2 at the end in the same way, the result should be ok and you loose the conversion rounding errors.

Of course, that means writing your own conversion routine... :)


That c++ book paid off. :) I wrote an avisynth filter that converts YUY2->RGB32, without changing ranges. It takes 3 YUY2 clips. The 1st is normal. The 2nd & 3rd have the UV in the Y spot. I use a script that has UtoY(),VtoY() to pull the UV out of the 1st and make the 2nd and 3rd. A bit of a hack, but that way I can upsample/filter the U/V before I effectively pass it in as 4:4:4 YCbCr. I want to make this internal, but I'm still trying to see if any particular method makes a difference. Most converters just double the samples.

Once in RGB (vdub) I can then see the xtra info and decide if and where to clip it. If I do this as a 1st step, I'm really making the video look 'normal' and I don't worry from there.

[Wilbert]

I will throw in some random comments ...

My guess, but I didn't know enough about this issue to be sure, was that if you want to work with an analogue source like tape or TV, your best bet would be to work yuv2 (or RGB24) throughout, and only convert to yv12 as a last step for DVD production (if ever at all)
True. If you are capturing an analogue source, you should not use a yv12 codec because of the reason trevlac mentioned.

But wat kept me from this conclusion was the emphasis put on YV12 by the avisynth people and the number of people developing and searching for yv12 capture codecs. So maybe I am missing something here.
The reason is dvd (mpeg2) to divx/xvid (mpeg4) conversions. It gives a nice speed up if you stay in the yv12 colorformat.

So to go from 4:2:2 YCbCr to 4:4:4 RGB, the code usually just doubles the CbCr and maps 16-235 to 0-255. It could be way better. There is possibly some good data in the YCbCr 236-253 range. With a direct conversion, this is lost. I doubt the 2-16 has much.
I can't follow you here, because there is no YCbCr 2-16 and 236-255.

Although in my case the capture chip/driver captures YCbCr 2-255 (from a 16-235 source, the tv). This implies you have to scale it to 16-235 and then convert it to rgb (at least, if you want to convert it). Here, you won't loose any information.

I don't know in which case your source is trully YCbCr 2-255 (DV maybe ?). In that case you will loose information.

Why not map directly to RGB without converting? While in RGB mode everything will look wrong, but if you convert back to yuy2 at the end in the same way, the result should be ok and you loose the conversion rounding errors.
Assuming that the conversion 'interlaced yuy2' to 'interlaced rgb' is done correctly by the codec, this is possible.

However, you might want to denoise your clip. Doing this in yuy2 will be a lot faster. Besides this rgb->yuy2 conversion takes also time.

But why YV12 and not say YV16? i.e. a planar format with the same amount of info as YUV2?
In the AviSynth dev. forum they are talking about a YUV 15 bit format for AviSynth v3.0 (first post of Bidoche:

http://forum.doom9.org/showthread.php?s=&threadid=67451#postid418184

[trevlac]

Originally posted by Wilbert

I can't follow you here, because there is no YCbCr 2-16 and 236-255.

Although in my case the capture chip/driver captures YCbCr 2-255 (from a 16-235 source, the tv). This implies you have to scale it to 16-235 and then convert it to rgb (at least, if you want to convert it). Here, you won't loose any information.

I don't know in which case your source is trully YCbCr 2-255 (DV maybe ?). In that case you will loose information.


I think This is where we are out of sync. YCbCr can contain info outside of 16-235. 16-235 is the 'nominal' range. Look at the 601 spec and see what they say about conversion from YCbCr to RGB. They don't say change the range.

This is why I am begining to think this:

Your Analog comes in as YUV. I'll skip composite because that confuses the matter. Let's assume s-video. The Y signal ranges from 0 to 0.7 mVolts. The mixed UV signal is the same. For the conversion to YCbCr, your card mapps 0mV to 16 and 0.7mV to 235. The trick is that the signal may not be 0 to 0.7 which is IRE 0 to IRE 100 using a generic scale (because voltages differ). It may actually be IRE -6 to IRE 108. AND your card may boost the brightness, contrast, do some coring of the black, etc.

So You will/can (and I do) get analog stuff in the 235-253 range. The black (2-15) is not as much of a problem, because making shades of black pure black acutally reduces noise more than any loss of picture.

The outside ranges of 16-235 are there for precisely the problem of messing with the video and not losing info. That is why RGB 16-235 exists.

I believe 0/1 and 254/255 are reserved for info, but you can get a fuller tonal range by using 2-253 IF your signal had a fuller range to start.

To be honest, I'm not sure it really matters. But I'm trying to figure that out. ;)

[Wilbert]

I think This is where we are out of sync.
I don't think we are contradicting each other. Yes YCbCr can contain info outside 16-235. However, pal and ntsc follow the range: 16-235 (level 16 is black and 235 is the peak white level). Like you said your card may be upscaling this to 2-253. Thus you have to scale it back again (in AviSynth for example) at some stage.

Are we still out of sync?

[Arachnotron]

My 2 cents:

If you have an 8 bit ADC and have blanking level at zero, it will have 8 bits available between blanking level and white. This will put black at a higher value. So upscaling later would not add information, since the bitrange available between black and white in the raw captured data is not 8 bits but less.

Of cource, for the newer 9 and 10 bit ADC's, you may have enough range between black and white to fill up 255 steps.

@Wilbert:

Thanks for clearing this up, I really wondered about this. :thanks:

Having avisynth do YV15 or 16 would be wonderfull (though I suspect not a non-trivial thing to do). But all those poor filter developers... :)

For now, I will be capping using the shiny new version of huffyuf Shodan released last december.

[trevlac]

Originally posted by Wilbert
I don't think we are contradicting each other. Yes YCbCr can contain info outside 16-235. However, pal and ntsc follow the range: 16-235 (level 16 is black and 235 is the peak white level). Like you said your card may be upscaling this to 2-253. Thus you have to scale it back again (in AviSynth for example) at some stage.

Are we still out of sync?

Yes.

Assuming there are no card/driver changes to brightness,contrast,etc. A PAL and NTSC signal does not have to be in the range of 16-235. Those numbers equate to IRE0 and IRE100. A signal may well be IRE-6 to IRE108. If you look at the test results of some DVD players, you will see what I mean. When a device outputs NTSC/PAL, it probably does not hit 0 to 100. The stuff above 100 is not pad. It is part of the signal.

But your card may muck it up more. So, PC->TV lets you get to it, but squishes the original signal and loses data. Besides, It's hard for me to see it ahead of time. :)

I think most stuff does not hit 0 to 100 because that gives it less room for loss. The 'fuller'/stronger a singal, the better the signal to noise ratio.

Maybe Arachnotron can tell us what his VCR/DVD player does. :)

Anyway, I think you are think in a normal fashion, and I'm looking at the edges. What I'm looking at may not make 1 bit of difference.

[Arachnotron]

Ok, a small detour in SMPTE-170M:

The NTSC signal ranges between -23 IRE (minimal excursion with chroma level) and +131 IRE (maximum excursion with chroma)
blanking level is 0 ire
black is 7.5 ire
white is 100 ire

(100-7.5) / (131 + 23) = 60% of total range is between black and white
so only 153 steps left assuming 8 bits sampling

It would seem that 16-235 is already upsampled for composite...

It becomes even less if you take sync level into account (-40 IRE) and set this to zero.

What I mean with all this is that using an 8 bit ADC, you simply don't have the resolution between white and black to fill the 16-235 range....

[edit] I think most devices determine the gain/ calibrate the voltage range from the difference between sync level (-40 IRE) and blanking level (0 IRE). Even cap cards do this (automatic gain control, AGC)
Macrovision operates by disturbing this difference by putting spikes in the blanking signal following sync.

[edit2] So if you define the difference between blanking and sync as 40 IRE, it does not matter what the absolute voltage is. Only the voltahe relative to sync/blanking is important

[trevlac]

Originally posted by Arachnotron
It would seem that 16-235 is already upsampled for composite...

It becomes even less if you take sync level into account (-40 IRE) and set this to zero.

What I mean with all this is that using an 8 bit ADC, you simply don't have the resolution between white and black to fill the 16-235 range....



I think you are mixing things up here. IRE is a scale (like voltage) that can contain less than interger values. So the IRE scale has many more than 153 steps. What about IRE 10.625 ?

[Arachnotron]

I think you are mixing things up here.

No, I think you don't follow me :)

What I mean is this: you have 8 bits in your ADC for the full range. The full range is wider than the area between white and black level, because of the chroma addition. Therefore, since you have only 255 quantization levels for the whole signal range, less are left for the area between white and black (luma)

So the 8 bits/ 255 levels 0r 220 quantization levels of luma you get in the end are in effect already upscaled from about 150. An 8 bit ADC used on a real world video signal does not have the precision required to put 220 quantization levels between black and white.


The second point I was trying to make was about your remark about voltages sometimes being higher in reality than 0.7 V.

The height of the voltage can vary, but what is important is the height relative to the voltage differens between blanking and sync. So if white level would be 0.8 V, as long as the blankin/sync difference is increased accordingly, the ADC would output the same level at this point.

[edit] chip/TV/VCR will compensate for this by adjusting the gain BEFORE digitizing

[acrespo]

Ok, I will capture in YUY2 because interlacing (this is the mainly reason as I see in the comments). But after deinterlace, can I change from YUY2 to YV12 for noise (convolution,mipsmooth,fluxsmooth) and resize (lanczos) or is better stay in YUY2 and convert to YV12 only when Xvid start?

[WildCelt]

Originally posted by Arachnotron
For now, I will be capping using the shiny new version of huffyuf Shodan released last december.

Zuh? I missed that one! Where can I find it?

[Arachnotron]

The codecs section of the downloads on Doom9. The previous version (CCSE patch 0.23) was replaced by the new one made by Shodan (v2.2.0)

see this (http://forum.doom9.org/showthread.php?s=&threadid=67121) thread.

[Wilbert]

The previous version (CCSE patch 0.23) was replaced by the new one made by Shodan (v2.2.0)
Small detail. CCE patch 0.22 is replaced by v2.2.0. I don't think Sh0dan included the changes in patch 0.23 yet.

Ok, I will capture in YUY2 because interlacing (this is the mainly reason as I see in the comments). But after deinterlace, can I change from YUY2 to YV12 for noise (convolution,mipsmooth,fluxsmooth) and resize (lanczos)
Yes. If you do it, the best is to do it after deinterlacing. You won't gain much, unless you want to use filters which require YV12.

[trevlac]

Originally posted by Arachnotron
No, I think you don't follow me :)


Probably :D

Well, sometimes you make me work too hard.

So do you think the process is:

- Sample
- Quantization
- Decode the signal
- Make Pixels

Makes sense. So if they only use 8 bits in step #2, there are fewer than 224 steps in the chroma? I can't see how they can quantizie the full signal (blanking/sync/etc). I think they take IRE-6 to IRE108, which I guess is about .05mV to .75mV assuming a signal runs only from 0 to .7mV (I think it really runs in the nominal range of .3 to 1.0).

Honestly, I'd have to look at a book/link with a composite signal to understand. However, I do not think they use sync samples nor any sample above 108IRE. That would give a much fuller range. That is why they do 2-253 not 16-235.

-----------------
What does the new huffyuv do? YV12? I want it to output RGB 16-235. :)

edit oh, 'reduced resolution' now i remember. :)

[WildCelt]

Originally posted by Arachnotron
The codecs section of the downloads on Doom9. The previous version (CCSE patch 0.23) was replaced by the new one made by Shodan (v2.2.0)

see this (http://forum.doom9.org/showthread.php?s=&threadid=67121) thread.

Thanks. I would have never looked in that forum for it!

[Arachnotron]

Small detail. CCE patch 0.22 is replaced by v2.2.0. I don't think Sh0dan included the changes in patch 0.23 yet

I know. I tried to say the 0.23 version in the downloads section was replaced by the Shodan V2.2.0 version.

[vigi_lante]

But how bad it would be to capture from an analogue source using a YV12 codec ? Damn, just when I was very happy with the speed increase for keeping everything in YV12.

[trevlac]

Originally posted by vigi_lante
But how bad it would be to capture from an analogue source using a YV12 codec ? Damn, just when I was very happy with the speed increase for keeping everything in YV12.

I don't think it's really that bad. You'd be the judge of what looks best. :)

[Arachnotron]

@ trevlac

here (http://tuner.ixbt.com/soft/soft_tv_test_001.html)

at the bottom of the page you can download a program which can show the VBI capped (raw) date for a 878 or 7134 chipset. It should give some idea how the raw voltages are distributed across the ADC range and give those of us without an oscilloscope some idea what a video signal looks like. :D

The page is in russian, but the links are at the bottom.

[trevlac]

@Arachnotron

Thanks for the link. I have not had a chance to check it out yet.

However, I did want to share something I just stumbled across. :)

There is no rounding loss of the luma info if you convert from YUV 16-235 to RGB 16-235. One argument given for not converting color spaces is the loss of info due to rounding. Well, if you use the standard ITU 601 calculations, there is no loss in the luma as long as you stay at 16-235. If you go to 0-255 you loose fractions.

Here are the calcs ignoring the chroma parts:
Y = .299R + .587G + .114B
R = Y
G = Y
B = Y

Run any number 0-255 thru and you will see that there is ZERO rounding.

Now, seeing that YUV is 4:2:2, 'color shifts' due to the conversion calcs may also be less significant. I'm not sure.

Moral of the story:

If you cap YUV and want to edit RGB, you should stick with 16-235 except to view. Maybe I need vdub view filter.

[Inc]

@ Trevlac

I did some tests on XVID codec for capturing and it worked nice, fast detailed ... EXEPT exactly these croma upsampling errors showed in the link of the first page of this thread in interlaced mode.

Even if I select "interlaced" in the XVid settings that bug in the resolution of chroma occurs.
Does something in the Xvid settings like "fix chroma ...." will fix that? Ok you explained that this is not a codec issue but a 4:2:0 issue. I hope I didn't miss here something but does that mean when encoding interlaced material to mpeg 4:2:0 these chroma upsampling errors do also occur?

I try to see the logic in this if its yiust a capture codec issue or 4:2:0 in general. As you got in your explanations into the "deep" ;-)

How about using FFvfw Xvid codec?? as there you also can integrate avisynth commands like to convert the YUY2 signal from the card using ConverttoYV12(interlaced=true) in the avisynth section of ffvfw codec so the signal from the card YUY2 which will be provided to the xvid encoder engine will be correct converted to YV12 before.

And what about mpeg2 capturing apps. or even the Haupauge Win PVR 250 which captures directly in mpeg2 (also interlaced!)YV12 4:2:0 as it gots the encoder hardwarebased onboard???

Im not in these colorspace "deep" things, so I still got these questions. ;-)

Thanx for answer in advance.

Inc.

[Inc]

.. sorry replied instead of edited

[trevlac]

@incredible

Funny thing is that I don't know diddly about mpeg4, do I can't say much about those codecs.

My understand is:

If you don't see a problem before you encode to 4:2:0, then the problem is probably when you decode the mpeg when you play it. This is because the flavors of mpeg remove some color info. If you have interlaced source, recreating the removed color is tricky. The 'chroma bug' was 1st discussed because dvd players were not doing the decode correctly.

You can test the above by capping to huffyuv (YUY2) 1st, and then encoding (as a test).

So...for interlaced source. If no problem before encode, and you pick interlaced, the problem is the player. But like I said, i don't know divx. I probably should learn .... ;)


Hope this helps.

[Sulik]

The problem with YV12 and other 4:2:0 formats is that 4:2:0 progressive is different than 4:2:0 interlaced (the chroma sampling positions are not the same).

This means that once you convert from 4:2:2 to 4:2:0, you HAVE to make a decision on weither you're converting to interlaced 4:2:0 (two interleaved fields) or progressive 4:2:0 (progressive frame).

If you try encoding 4:2:0 progressive as interlaced or vice versa, you WILL get chroma artifacts.

[Inc]

@ Sulik

Well these days I bought a Typhoon 7134 chipset card to get some experiences with XVID capturing. I still got my DC30 running under Win2k which uses the onboard mjpeg YUY2 encoder and always a fieldbased avi come out.

I did choose "interlaced" in Xvid as the captures came out interlaced with combing artefacts (sports documentations), so I think "interlaced " was the correct choice in XVID.

Mybe my build of XVID is too old, I don't know, but yesteday I downoaded the latest build rev 1.0xx from koepi. mAybe that one behaves different.

Also I should mention that I have to choose RGB32 or RGB24 as inputsignal to the Xvid codec as that are the ones it seems my card supports. The Typhoon seems to got the same hardware as the cinergy 400.

BTW: When opening that "interlace" XVID avi in Avisynth, the info() command still say "fieldbased = no"! A bug or what?
Because if no matter what set the stream will be framebased I also should try one capture with interlaced=off even the broadcast is interlaced?? Strange ... but tonight I will keep testing.

Maybe I did not catch some statements of you well, sorry in advance! BTW I do capture in PAL.

[Wilbert]

BTW: When opening that "interlace" XVID avi in Avisynth, the info() command still say "fieldbased = no"! A bug or what?
I have to document that properly ...

You have to wait for AviSynth v3.0 for this feature. Until now it means: "field based (you applied SeparateFields) or frame based (you didn't apply SeparateFields)".

[Inc]

Yesterday I mad some interesting Experiences on a analogue testing capture using the newest XVID build downloaded from Koepi's Webspace and my Typhoon 7134 chip based PCI capture card, captured in Vdub at 352x576 using that XVID mpeg4 codec.

I choosed "interlacing" in the options, CBR at 8000kbit and shurely on an interlaced capture I got these artifacts, as known:

The capture is scaled 300% using just Pixeldoubling mode in Photoshop, not Bicubic etc.
http://home.arcor.de/the.incredible/Chroma2.jpg

I opened that Avi via "Open AVI via Avisynth" in VdubMod and added after the Avisource("....") line just a

ConverttoYUY2(interlaced=true)

And this came out:

http://home.arcor.de/the.incredible/Chroma1.jpg

WOW!

So does that mean, converting that YV12 4:2:0 capture to 4:2:2 YUY2 does "blend" or "interpolate" the chroma informations so they don't look blocky anymore.

Maybe that issue was already explained here, but I couldn't find it, so explanations are very welcome.

Thanks in advance!

Inc.

BTW: When my sytem decodes that orig capture, Vdub says that exactly the XVID is also used for decoding, as recommended in here.