Help understand BD video Aka is everybody blind or is it just me?

[DoctorM]

I've encoded, decoded, pulled apart, rebuilt and rewritten DVDs 4 million ways by now.

I'm just sticking my toe in the BD pond and have run into something a couple times that I just don't get.

NTSC DVD video should have the blackest black at 16 and the whitest white at 235.
From my reading, BD keeps the same peak levels.

Now here is the problem. I've seen a few discs now, described by professional reviewers as looking fantastic, but I think look dingy and grey.

Below is an original screenshot and the Waveform graph:

The script feeding VDubMod is DirectshowSource("BLU-RAY\BDMV\STREAM\00001.m2ts", fps=23.976, framecount=186648, audio=false)
crop(220,0,-224,0)
http://thumbnails102.imagebam.com/27411/99dd56274109002.jpg (http://www.imagebam.com/image/99dd56274109002)
http://thumbnails101.imagebam.com/27411/dfc009274108938.jpg (http://www.imagebam.com/image/dfc009274108938)

Now if this were a DVD, I'd say it was poorly done as if someone accidentally converted from 0-255 -> 16-235 twice and ended up with 32-215. I would think it should look like this (added "Levels(12,1,235,0,255)" (decided 12 was a better number than 16 to prevent crushing)):

http://thumbnails102.imagebam.com/27411/81c0f7274109093.jpg (http://www.imagebam.com/image/81c0f7274109093)
http://thumbnails103.imagebam.com/27411/e545e0274108952.jpg (http://www.imagebam.com/image/e545e0274108952)

Am I looking at this the wrong way? I know the first thought is to assume I've done something wrong, but the disc looks like I'm watching it through a dirty window on my TV.

Do Blu-rays not try to achieve 16-235 for the best contrast?

[Brazil2]

http://avisynth.org.ru/docs/english/externalfilters/colormatrix.htm

[raffriff42]

Have you calibrated your playback signal chain from disk to display? http://forum.blu-ray.com/showpost.php?p=47055&postcount=207

As you may know, Sony movies on BD have calibration tests when you key in 7669 from the main menu. How do we use it to calibrate our HDTVs using these screens? ...

Second question: how do you know the Waveform graph is correct, taking into account the processing that occurs between the compressed file data and the point of measurement? To answer that, it seems to me you need a calibrated BD test pattern set, which must be converted to .mts using the same workflow used on your working footage. Checking the waveform on some other source means nothing, as DirectShowSource may load a very different codec. My point here is that I doubt the BD was mastered at 32-215. But - stranger things have happened.

For quick level checking (to see what the Source statement is outputting), I find this script snippet very useful, as it can be applied right after the Source statement, before any (possibly implicit) conversions: #XXXSource(...
(Last.IsRGB)
\ ? RGBAdjust(analyze=true).Subtitle("RGB")
\ : ColorYUV(analyze=true).Subtitle("YUV")

[Didée]

DirectshowSource
There, the culprit. :)

[DoctorM]

There, the culprit. :)

That sounds like a conversation you need to have with jdobbs.

@Brazil2: I am not converting to another format. Colormatrix isn't needed.

@raffriff42: Huh?
And with regards to your script... huh? It's a fragment that doesn't work. Where were you going with that?

@All - I try get post a clip later so you guys can see directly.

[Guest]

It's all talk and guessing until you post an unprocessed source sample. My money is on Didée's conclusion.

[raffriff42]

>@raffriff42: Huh?
1. I would urge you to follow the this link (http://forum.blu-ray.com/showpost.php?p=47055&postcount=207) and calibrate your monitor as explained.
2. I would also urge you to use FFmpegSource (http://ffmpegsource.googlecode.com/svn/trunk/doc/ffms2-avisynth.html), not DirectShowSource, where possible.

[hello_hello]

Now here is the problem. I've seen a few discs now, described by professional reviewers as looking fantastic, but I think look dingy and grey.

To be honest. If I had to pick a winner between the two pics it'd be the first one. At least the way each displays on my TV from within the browser (the TV's set to expect 0-255).
To me the second pic has too much contrast, or it's too dark. I don't think there'd be anything in the pic which would actually be black. Outside on a nice sunny day.

Having said that.... I'm certain I've seen DVDs which went through a levels conversion one time too may, or at least episodic DVDs where just a couple of episodes apparently went back for seconds (comparing the opening sequences confirmed I wasn't imagining it), and I can think of at least one DVD where the extras must have been converted twice. That was fairly obvious when comparing them to the same scenes in the movie, then again after applying ColorYUV(Levels="TV->PC").
I've seen TV shows which I'd bet were broadcast using the wrong levels and I'd be willing to bet a fairly large amount of money I've seen at least one Bluray movie which was the same. It wasn't mine, it was probably a year ago and I can't even remember which movie it was. I watched it at my other half's house. I remember it was the second movie for the night and I recall commenting on it looking very washed out the minute it started.

[DoctorM]

I calibrated the monitor I'm using a long time ago. While it may not be perfect, it's quite good.
That also doesn't explain why it look terrible on my HDTV as well. (Which is what started me digging deeper.)

I usually prefer FFmpegSource. In this case directshowsource was a convenience... and most likely beside the point.

Here is a 1 minute clip. It was chopped out with tsMuxeR. No funny stuff.
It's a representative sample. A supposedly bright outdoor shot, the darkest blacks are grey, and everything looks like their lens was dirty.
Anyone testing the level should not forget the left/right black borders are plenty black.

http://www.mediafire.com/download/9wfuz7c77478lqb/TQMClp.part1.rar
http://www.mediafire.com/download/mbpdqyto94v56br/TQMClp.part2.rar
http://www.mediafire.com/download/m0xtv5r0vkzz6fl/TQMClp.part3.rar

Apologies for using a split file, but MediaFire would not behave today.

[raffriff42]

I have to agree about the limited luma range - it seems to be intentional.
LooseMinimum, LooseMaximum between 34 and 217 throughout the clip, with the pillar boxes at 16. Not what I expected to see.

https://www.dropbox.com/s/3tjvmfc6no1wbls/TQMClp%2BColorYUV%2Bcrop-720x404.jpg?raw=1

https://www.dropbox.com/s/uhgjobhdi6tprtw/TQMClp%2BVideoScope%2Bcrop.jpg?raw=1

This shot has way too much contrast - it looks like it's transferred from a print, not the negative. Maybe the restricted luma range is trying to hide that?
https://www.dropbox.com/s/vadhmgzqui6ndjp/TQMClp%2Bcrop%2BShowHistograms-720x404.jpg?raw=1

[ChiDragon]

I think your only mistake is expecting "professional reviewers" to offer much informed criticism.

One milky transfer is the SE Blu-ray of Total Recall (1990): http://www.avsforum.com/t/1406134/total-recall-special-edition-1990/390#post_22282072

[DoctorM]

I believe it is scanned from a camera negative... which sounds different to me than what is traditionally done.

@raffriff42, you said 'too much contrast'. Wouldn't that mean little grey with strong delineation between whites and blacks. It seems to me this is the opposite of what is going on.

So then a simple 'levels(12,1,235,0,255)' really is the best idea?

Taking this, Total Recall (did they go orange and teal there?), and some other discs I've seen now into account, it seems like BD technology is a lot less mature than they want us to believe.

[raffriff42]

Either negative or positive may be used, or an even higher generation. IIRC (OK I used Google to refresh my memory) the normal film generations are called camera negative, interpositive (https://en.wikipedia.org/wiki/Interpositive), internegative (https://en.wikipedia.org/wiki/Internegative), distribution print. Camera negatives look the cleanest and have the most dynamic range. Each further generation adds some grain and loses some dynamic range. Normally the digital film restorer / telecine colorist tries to get film elements from the earliest generation possible. Sometimes elements from different generations must be used if (for example) a reel of film goes missing. That last image above shows what I *think* is a higher-generation source than the rest of the clip.

As for color correction, I don't think anything can be done, but I'm no colorist. I'm curious what someone like @foxyshadis would have to say.

[DoctorM]

Hmm. After adjusting the levels things still looked wrong.
I think the problem isn't that they did an inappropriate conversion to 32-215, I think they crushed the levels to it.

Too much shadow detail is missing. Making the blacks darker steals even more detail. Whatever was 16-31 is just gone leaving banding behind (I assume the same is true for the whites as well).

Ah well, I had hoped to make this look better but I don't think that is possible.

[Emulgator]

SmoothAdjust should be your friend.
You still get a steady histogram afterwards, no banding.

BTW I don't expect all blu-rays/dvds to be mastered perfectly.
It is YV12 for both DVD/BD, the range suggestions 16-235 are similar.
And nothing wrong with letting some pixels under- or overshoot by a few steps.
Just never let something clip.
I do frequently get encodes to judge and often these are far from ok.
Just had to judge a poor PAL-DVD 32-235 encode from a 1920x1080x25 ProRes Source by FCP/Compressor
It is nothing about the format(s) themselves, is is more about the skills/beliefs of the ones who grade/encode the footage.
(And the assumptions some software makes, FCP/Compressor being one of the main culprits)

If something looks milky, I suggest to watch the histograms and just process it to taste.

[Lyris]

I think your only mistake is expecting "professional reviewers" to offer much informed criticism.

+1. I doubt most "professional reviewers" are even using calibrated displays.