ColorYUV(levels="TV->PC") vs colormatrix for DVD video?

[mahsah]

I'm not sure which one I should use... should I use one, or both?

[Sagekilla]

I've tested between letting the player convert TV levels to PC levels, and having the video converted to PC before encoding then playback at native levels, and found virtually no difference. The only difference for me was that the corrected version (before encoding) took up more bitrate because of the increased color range. In the end I gave up trying because it was a hassle switching to and from TV --> PC levels vs no conversion on the player.

Also, Colormatrix is for converting between color standards. It's useful if you wanna use only one standard (BT.709 for example) and have your player use the correct color conversion for that standard. Most of the time though, you won't notice it until you're shown the correct and incorrect images.

Good luck.

[canuckerfan]

Forgive me, but I have to retouch this subject just because I am very confused now after much searching and reading....

Don't Colormatrix and ColorYUV(levels="PC->PC") perform different actions. Colormatrix is supposed to preserve colours when feeding your video into the appropriate encoder, isn't it? But I'm not sure when the "PC->TV" conversion is supposed to be done. Isn't it for converting levels for viewing on a different device (in this case either a TV or PC)?

For example when I perform ColorYUV(levels="PC->TV") on a DVD of mine everything appears a bit brighter and details before that were lost in extreme bright or extreme dark become apparent. But a visual pitfall of all this is the black levels are now off and I don't think black is trully "black" anymore. Seems to make sense thatthe very dark and the very bright were "chopped off" in the range conversion.

But to my knowledge almost all YV12 content is already clipped to a TV range of 16-235. So why is it that my footage (which I'm re-encoding to DVD) is being affected by ColorYUV(levels="PC->TV") if the levels should already be in TV range? Or are they in the PC range of 0-255?

Oh god:confused:

[manono]

You're just reencoding DVD for DVD, right? Then you don't want to use either one.
But to my knowledge almost all YV12 content is already clipped to a TV range of 16-235.
No. While most luma values are between 16 and 235, it's perfectly OK to have some outside those ranges. Unfortunately, many, if not most, of the Indian movies you and I both love are incompetently prepared for DVD and they often wind up crushing the blacks and blowing out the whites. If you'd like to see how much is outside of the 16-235 range, add these to the bottom of a script:

ColorYUV(Analyze=True)
Limiter(Show="Luma")

The red dots (sometimes masses of red) show what's below 16 and the green dots show what's above 235.

Histogram()

can show similar things.

[canuckerfan]

You're just reencoding DVD for DVD, right? Then you don't want to use either one.

No. While most luma values are between 16 and 235, it's perfectly OK to have some outside those ranges. Unfortunately, many, if not most, of the Indian movies you and I both love are incompetently prepared for DVD and they often wind up crushing the blacks and blowing out the whites. If you'd like to see how much is outside of the 16-235 range, add these to the bottom of a script:

ColorYUV(Analyze=True)
Limiter(Show="Luma")

The red dots (sometimes masses of red) show what's below 16 and the green dots show what's above 235.

Histogram()

can show similar things.
That's a pretty handy script. Thanks for that.

Well as far as colormatrix is concerned, over here (http://forum.doom9.org/showthread.php?t=82217&page=18) it's suggested that it is required when going from DVD to DVD (if the coefficients are incorrect for your encoder, of course).

I see green and red dots pretty consistently... which means the luma is 0-255. Is there a way to "brighten up" details that were hidden in the dark and bright areas like ColorYUV(Analyze=True) did (maybe to a lesser extent) while keeping these red and green dots?

[Wilbert]

I see green and red dots pretty consistently... which means the luma is 0-255.
No, not necessarily. Usually black is centered at 16 and white at 235. Is that the case for you?

[canuckerfan]

No, not necessarily. Usually black is centered at 16 and white at 235. Is that the case for you?
I'm not sure how to determine that....

[manono]

Well as far as colormatrix is concerned, over here (http://forum.doom9.org/showthread.php?t=82217&page=18) it's suggested that it is required when going from DVD to DVD

And it's dead wrong. That was back when most here thought DVDs used Rec.709. I was as guilty as everyone else. Sometime after that thread we found out they use Rec.601. As far as I know, the only time you need to change from one to the other is when going Hi-Def<->Standard-Def.

Many times Indian DVDs stretch the 16-235 to 0-255. Then you get the masses of red and the solid blocks of green when using that script I gave above. And you lose the shadow detail. What should have been 16-32 showing black and many shades of dark grey becomes 0-16 and solid black. Same with the whites, except it's worse for the whites. You lose all sense of 3-dimensionality. You lose the wrinkles and textures and the varying shades of dark grey or light grey from the light bouncing off at different angles. It's just one of the many things wrong with many Indian DVDs. Like I said, it's OK to have some red dots and green dots. It's when you see the big blobs of red and/or green that you know they screwed it up again.

[manono]

Is there a way to "brighten up" details that were hidden in the dark and bright areas...

I use a combination of Tweak and YLevels. A typical script might go:

Tweak(Bright=10,Cont=0.9,Coring=False)
YLevelsS(0,1.5,255,0,255)

That's for a DVD where the blacks are crushed and the whole thing looks kind of 'murky', or generally dark. If it's a color film I might boost the saturation in Tweak (Sat=1.2, for example). There are other ways to do these things also. However, once the damage is done nothing can really fix it.

[canuckerfan]

hmmm. it's all making sense now. i'm kind of in the middle of many lagarith passes and I really would rather not start again. could I invoke colormatrix to reverse the colorimetry change?

and would you recommend to keep the levels at 0-255 and not "core" them like Tweak (by default) does? I know alot of SAPs and HDTVs support the full PC range now.


Many times Indian DVDs stretch the 16-235 to 0-255. Then you get the masses of red and the solid blocks of green when using that script I gave above. And you lose the shadow detail. What should have been 16-32 showing black and many shades of dark grey becomes 0-16 and solid black. Same with the whites, except it's worse for the whites. You lose all sense of 3-dimensionality. You lose the wrinkles and textures and the varying shades of dark grey or light grey from the light bouncing off at different angles. It's just one of the many things wrong with many Indian DVDs. Like I said, it's OK to have some red dots and green dots. It's when you see the big blobs of red and/or green that you know they screwed it up again.
I'm guessing in this case there's no way to really recover those details...

[manono]

could I invoke colormatrix to reverse the colorimetry change?
I suppose, but since I don't use it I don't know for sure.
and would you recommend to keep the levels at 0-255 and not "core" them like Tweak (by default) does
Other filters clip to 16-235 by default also, including Levels and ColorMatrix. It's a huge mistake, in my opinion. I don't think a video should be effected in that way, 'behind the back' so to speak. Ordinarily I leave in the full range, but sometimes (rarely) will use Limiter afterwards if I want to clamp it to 16-235.

I don't think my views as expressed in this thread are universally accepted and I'd more than welcome any contrary opinions. I'm far from an expert on this subject.

[canuckerfan]

Well, universally accepted or not, I appreciate your input :)

Unfortunately I think my indian DVD falls into this range expansion problem that you described. There is virtually no shadow detail and whites seems blown out. Alot of green and red dot patches! Although, it may not be as bad as I make it seem.

I've uploaded a sample which I think best shows this: http://www.sendspace.com/file/0wvtxn

[canuckerfan]

ok. I think i've finally settled with something that looks good to my eyes.

my only question is if I keep the 0-255 range and don't clamp all the way through (even after converttoyuy2 for cce), will it still be fully compliant with all SAPs+TVs? In other words, will older sets that do 16-235 play these without any major problems? I'm assuming newer TVs will be ok since alot of them do 0-255.

edit: I think i answered my own question. since the source itself is 0-255 and there are many DVDs like this than it must be compliant with consumer appliances:p

[manono]

That sample is a good example (but nowhere near as bad as many) of the kind of thing that's typical of Indian DVDs. That DVD has so much more wrong with it than just the luma levels. Compare it sometime to a decent Hollywood studio DVD.
ok. I think i've finally settled with something that looks good to my eyes.
You understand that if it looks good on a computer monitor it'll most likely be too 'bright' or 'contrasty' on a TV set, don't you? If it's going to look good on a properly calibrated TV, it should look slightly 'dull' or 'dim' or 'dark' (don't know the right word) on the monitor.
I think i answered my own question. since the source itself is 0-255 and there are many DVDs like this than it must be compliant with consumer appliances
Yeah, it'll play, no matter what. The problems might arise if the DVD player (and there are many) doesn't pass Blacker Than Black and/or Whiter Than White. But if you have a good home theater setup, what do you care what someone else watches on? Do you want to 'dumb down' your encodes to suit someone else's inferior equipment?

[canuckerfan]

^i've compensated for the brightness on a tv. i'll probably test it before fully encoding it. and i agree with the hardware situation. luckily for me, I have a good HT setup :)

[2Bdecided]

Most broadcasters clip at 16 and 235 - though you'll always see values beyond this because MPEG encoding is lossy, and (by definition) doesn't give the exact values at the output of the decoder that were present at the input of the encoder.

It's really bad practice to have black at 0 or white at 255 in YUV or YV12 colour space.It's true that most DVD players support whiter-than-white, many DVD players support blacker-than-black, and all analogue TVs support both - but that's not the point. A DVD authored with blacker-than-black and whiter-than-white will, by default, get horribly clipped on PC playback, give unpredictable results elsewhere, and not look the same as "good" broadcast content.

(whiter-than-white is ubiquitous in the output of consumer camcorders - it's good practice to correct this. Blacker-than-black is quite rare).

Cheers,
David.

[manono]

It's really bad practice to have black at 0 or white at 255 in YUV or YV12 colour space.
Sure, you know that; I know that; we all here know that. Now try telling it to the Indian DVD production companies that make a practice of stretching the luma out to 0-255. I could feed you samples showing it from now until Christmas. It's more the rule than the exception to the rule. It's seems almost like they release garbage intentionally.

[2Bdecided]

I wasn't doubting that it existed - I was challenging your suggestion that it's "dumbing down" your encodes to correct it...

Yeah, it'll play, no matter what. The problems might arise if the DVD player (and there are many) doesn't pass Blacker Than Black and/or Whiter Than White. But if you have a good home theater setup, what do you care what someone else watches on? Do you want to 'dumb down' your encodes to suit someone else's inferior equipment?

...it seems pretty essential to correct it to me. Even if you can watch blacker-than-black and whiter-than-white without clipping on your system, it can only be calibrated to correctly display 16-235 or 0-255 - it's not going to display both "correctly" with the same setting, and who wants to keep messing around with the contrast, brightness etc?

Cheers,
David.

[manono]

Oh, I see. Well I do correct it when it's 0-255. Parts of my posts in this thread are about doing just that. But I'm not one to blindly clip to 16-235 since even the best DVDs have some luma outside the 16-235 range. And it's meant to be there.

[Keiyakusha]

This is a really good thread! But I'm a bit confused now. Should I correct THIS (http://img4.imageshack.us/img4/3237/11184255.png)? And how much, until levels will be in 16-235 range or some correcting + some clamping?
EDIT: when I watching any kind of video, TV->PC conversion usually done on playback.

[Emulgator]

I would say, yes.
Even if it is a dark scene.
If your source is encoded into ultra black like this throughout its duration and DVD is your goal:
I would correct min Luma up to 16 until only a few pixels (less than 0.05%, better 0.02%) are below Y=16.
(Like I would do with any gradation correction while working on stills).
But not clamp.
Black clamping is like white clamping a form of limiting,
so applying a non-linear conversion where you lose shadow (light) information forever.
In case you use CCE there is a provision to assure CCE that (input video is (already) 0-255). Tick this.
Mainconcept has a tick box to assure that input video is 16-235. Untick this.
This should solve the luma correction.

[manono]

In case you use CCE there is a provision to assure CCE that (input video is (already) 0-255). Tick this.
Works only if the input is RGB. Does nothing otherwise. I don't know about MainConcept.