ColorYUV and Histogram

[Trephin]

When I use ColorYUV to adjust my video, I get strange patterns in the Histogram display. Can anyone explain why I get these vertical lines in classic display and the checkers when in color display? Are they rounding errors? Is this bad? In the color diagram, it seems like certain colors are eliminated entirely.

http://scheffer.se/files/classic.pnghttp://scheffer.se/files/color.png

This is my script:

# --- IMPORTS ------------------------------------------------------------------
import("MVbob.avs")
# ------------------------------------------------------------------------------


# --- SOURCE -------------------------------------------------------------------
movie = OpenDMLSource("tape.1.dv.avi", audio=true, pixel_type="RGB24", fourCC="CDVC")
# ------------------------------------------------------------------------------


# --- FIX DV SOURCE ------------------------------------------------------------
movie = movie.ConvertToYUY2(matrix="PC.601")
movie = movie.ReInterpolate420(interlaced=true)
movie = movie.reYV12()
# ------------------------------------------------------------------------------


# --- DEINTERLACE --------------------------------------------------------------
movie = movie.MVbob()
movie = movie.SelectEven()
# ------------------------------------------------------------------------------


# --- ALTER COLORS -------------------------------------------------------------
movie = movie.ColorYUV(off_y=-6.0, gain_y=-14.0, cont_u=128, cont_v=128, opt="")
# ------------------------------------------------------------------------------


# --- CROP ---------------------------------------------------------------------
movie = movie.Crop(8, 0, -8, -0, true)
# ------------------------------------------------------------------------------


# --- RESIZE -------------------------------------------------------------------
#movie = movie.Lanczos4Resize(704, 528)
# ------------------------------------------------------------------------------


# --- FOR TESTING --------------------------------------------------------------
#movie = movie.Histogram(mode="classic")
movie = movie.Histogram(mode="color")
movie = movie.Trim(12600, 12600)
# ------------------------------------------------------------------------------


return movie

[scharfis_brain]

I cannot say much about the histogram, but about this:


movie = movie.ReInterpolate420(interlaced=true)
movie = movie.reYV12()

knock out reinterpolate420, if you use reYV12 and vice versa.

they should not be used together. they are not intended for this.

I recommend reYV12 alone here.

[Trephin]

Aah, thanks Scharfi! I did some additional tests now, and I can see that reYV12 alone looks more correct than combined with ReInterpolate420. My source is PAL DV. Do you think using the Canopus decoder and then reYV12 is the best thing to do here, or should I try YV12 decoding with ffdshow?

Anyone have anything on the patterns in the histogram?

[scharfis_brain]

should I try YV12 decoding with ffdshow

jep.
it *should* be be faster.

[Wilbert]

Could you post a screenshot of the same frame (of which you took the histogram), just taken after resizing? Also a screenshot of the same frame before any processing.

But, i really don't know. Perhaps Sh0dan can say something about this?

[Trephin]

I actually skipped resizing for those histograms. The linked screenshots might be useful. I saved them with image sequence from VirtualDub. I guess the chroma have to be stretched in one way or another when increasing contrast, but those gaps just looks scary. And the spikes in luma too.

Before (657 KB) (http://scheffer.se/files/before.png)
After (689 KB) (http://scheffer.se/files/after.png)

Used script:

# --- IMPORTS ------------------------------------------------------------------
import("MVbob.avs")
# ------------------------------------------------------------------------------


# --- SOURCE -------------------------------------------------------------------
movie = OpenDMLSource("tape.1.dv.avi", audio=true, pixel_type="RGB24", fourCC="CDVC")
# ------------------------------------------------------------------------------


# --- FIX DV SOURCE ------------------------------------------------------------
movie = movie.ConvertToYUY2(matrix="PC.601")
movie = movie.reYV12()
# ------------------------------------------------------------------------------


# --- DEINTERLACE --------------------------------------------------------------
movie = movie.MVbob()
movie = movie.SelectEven()
# ------------------------------------------------------------------------------


# --- FOR TESTING --------------------------------------------------------------
movie = movie.Trim(12600, 12600)
befor = movie.Histogram(mode="levels")
after = movie.ColorYUV(off_y=-6.0, gain_y=-14.0, cont_u=128, cont_v=128, opt="").Histogram(mode="levels")
movie = Interleave(befor, after)
# ------------------------------------------------------------------------------


return movie

[Arachnotron]

I think this is normal. Since you are working in 8 bit digital, you only have 255 discrete values vor Y, U and V to fall in. As long as you plot those values next to each other in a histo the plot looks continuous, but in effect it is a bar-graph. So the moment you stretch out the distribution and redistribute the values over a wider range, there will be values in between you will not get. Hence the gaps in the histo.

An example: suppose your luma is between 30 and 50. that leaves 20 possible luma values in the picture. Next you stretch the luma range to 0-255. You now have 20 luma values to distribute over a range of 255 possible values. The histo will look like 20 white spikes with huge black gaps in between.

Isn't the term for this quantization error? Anyone?

[Guest]

I long ago wrote histogram equalization filters for VirtualDub. I called that effect banding. The banding is actually visible as color stepping and so I provided an anti-banding feature. This adds a small random component to the equalized pixel to fill in the gaps. See here to see an example:

http://neuron2.net/histogram.html

[Trephin]

Thanks guys, for explaining the banding effect. What about the spikes in the luma histogram? Here I am compressing the range. Are these spikes from rounding?

[Arachnotron]

I suppose that is the reverse effect. Suppose you have 6 luma values and compress the range to 5 values. That means that at least two values will be rounded off and end up as one single value, while the others will map to a single new value. So one luma value in the resulting histo will contain contributions of the luma values of two sets of pixels, and spike out above the rest.

[Trephin]

Simple as that... All right. Thanks!