Black picture areas of my encoded DVD need improving

[jclampy]

I noticed when playing a test DVD encode on one of the LCD TV sets that the videos black areas did not look very black. It didn't look blocky or have misquito shimmering but was maybe too bright and had specks of other colours in it.

Does someone know a method or filter I can use in my avs script to correct this. Would prefer a 'proper way' but if need be I will accept a 'quick and nasty' fix as well.

When initially tweaking the colour calibration I used SmoothLevels and I noticed that the lowest colour on the histogram was at 16 and the highest was at 230. I then stretched it to fill 16 to 235. I did this by setting a 'fake' input as 16/235 then locking the bottom and using gamma at 2.0 with output as 16/235. This made it fill the TV colorspace area of the histogram completely without clipping.

I had to use 2x rainbow denoisers which ended up washing out the colours a bit so I also used SmoothTweak to set saturation at 1.3 and offset hue1 by 3 towards blue and hue2 by 3 towards red.

I tested the black areas on a PC LCD and on a small Analog TV set and the issue was not noticeable. On a LCD TV set though the issue with the black areas stuck out like a sore thumb.

Please can somebody give me some advice. Should I drop whatever is at the bottom '16' of the histogram so that it goes below the TV colorspace and gets 'clipped'? Or what should I do? :scared:

[Lyris]

Out of curiosity, what model of LCD TV is this?

And have any attempts been made to get it showing accurate video? A lot of them will have wacky gamma tracking and contrast stretching turned on by default.

Sorry, that doesn't answer your question; what I'm saying is that unless it's accurate, blame the TV :)

[jclampy]

Hi Lyris,

It's a Sony Bravia, I can get the model number tomorrow if you need it as I should be asleep now.

I usually colour calibrate displays although I need to try and improve my methods beyond just using colorbars. :)

I remember that LCD TV had given me the most trouble of all displays whilst calibrating though. I am still receiving Analog broadcasts at the moment though and the reception isn't the best. :o It does have seperate set of settings for DVD playback to those for broadcasted channels though.

I'll see if I can test a few more displays tomorrow but no guarantees.

Still, if I can enhance the black areas of my video encode I would like to. Surely someone will offer some advice or methods they have used themselves in the past?

Thanks in advance.

[manono]

I'd try and make sure the TV is calibrated properly first also. Do you have any Star Wars DVDs available to use? They have a THX calibration feature included and can help you get the brightness, contrast, and levels set correctly. And with them done right (given that LCDs don't really have true blacks anyway), about 90% of the battle is won.

[jclampy]

I have tried some different ideas with changing the colour settings of the video when encoding and made a few tests for the LCD TV. The second test was so wrong from what I was expecting I decided to recalibrate the LCD TV again.

Well, it was way off for some reason. I guess I must have calibrated it in the middle of the day in a sun filled room, it was that shocking.

I also realised how difficult it is to get that LCD TV perfectly calibrated. I'm still not convinced it is perfect but it is closer to what my LCD PC and CRT TV look like.

I need to further tweak my encoding color settings now, hopefully one more test run and then I will have something more concrete.

I don't have any Star Wars on DVD but good idea mentioning THX because I think I might have a DVD with that on it, I'll have a look tomorrow. ;)

[Lyris]

Hi Lyris,

It's a Sony Bravia, I can get the model number tomorrow if you need it as I should be asleep now.

I usually colour calibrate displays although I need to try and improve my methods beyond just using colorbars. :)

You're out of luck there I'm afraid - Sony is (I think!) the last manufacturer to have no advanced colour controls at all. But the colour gamut is usually pretty correct on them.

In this case it's the gamma you need to look out for - but they have no advanced gamma controls either. Make sure the Advanced Contast Enhancer is shut off because it'll probably be exaggerating shadow details.

If you'd like to get some idea of how the Bravias perform, I've measured and reviewed some at www.hdtvtest.co.uk. Basically, make sure to use the Cinema mode and disable the video processing enhancements and it'll be pretty accurate.



Putting the display aside, what encoder are you using? I think I'm in the minority using Cinema Craft SP3, but it has an option that lets you bias quality towards dark or light areas in the Picture menu. The open source encoders I'm sure have something similar?

[jclampy]

Hi Lyris,

Thanks for that information, I'll read some of those similar LCD TV reviews to get a general idea of how the Sony LCD/Bravia are in regards to color characteristics.

Before I do that though I need to recalibrate all the different TV's around the house because I just read online that there are different color bars for PAL and NTSC and apparently I was using the NTSC ones to calibrate PAL TVs.

I am trying to see if I can find some color bars .iso files for VCD/DVD or something so theres no chance of them getting screwed up via colorspace conversions or anything else if I made them myself.

Also, I'm using HCenc at the moment and I think the lumgain setting does what you are refering to.

I have tweaked the videos color settings a bit more; got the whites and blacks sorted I hope, I just need to sort out the hue's which is proving difficult.

I think my problem as I originally posted is noise in the blacks of the source. I am starting to think there is no way to fix this, any ideas?
Is it possible to denoise just the black areas only?

Edit1:
Atleast I can confirm my PC LCD is calibrated perfectly; I used and highly recommend this website here: http://www.lagom.nl/lcd-test/

Edit2:
For giggles I calibrated an old CRT TV in NTSC mode with the NTSC color bars and I am hoping/wondering that if I calibrate a CRT or LCD TV using that method that it also means that the TV will still be calibrated when I switch it to PAL mode. In other words when the TV makes whatever internal adjustments to switch between NTSC and PAL that if it is calibrated in one mode then it should also be calibrated in the other mode without needing re-adjustment. Does this sound correct?

Edit3:
I have found a website where you can download a PAL SD DVD calibration in VOB format: http://merifon.altervista.org/TestDVD.html

[jclampy]

I'd try and make sure the TV is calibrated properly first also. Do you have any Star Wars DVDs available to use? They have a THX calibration feature included and can help you get the brightness, contrast, and levels set correctly. And with them done right (given that LCDs don't really have true blacks anyway), about 90% of the battle is won.

Hi Manono, just following on from this...

After exhausting all the 'free options' from the internet for trying to calibrate PAL TV's I decided to try the THX calibration feature on the Terminator 2 Ultimate Edition PAL DVD. Amazing stuff, I was able to get the most accurate results so far. Basically you can skip through to a test about 2/3 of the way through. The test description says that the test is not just for 'sharpness' but also technicians/engineers can use it for so much more. That one test I was able to setup everything; brightness control, picture contrast, color saturation, and sharpness. At the moment I will use that test for all further testing references.

Ok, so back to calibrating my video encoding footage...

I have found that it is really hard to 'eyeball' calibration because PC is 0-255 and my output format is 16-235. Does anyone know any tricks on how to do this? I am wondering if at the end of my script I use something like colormatrix to convert the displayed video image as 0-255 just for 'eyeball' calibration and then remove the colormatrix filter from my script after I have calibrated.

*'eyeball' calibration is just for the color/chroma calibration. I would calibrate the black/white by the edges of histogram at 16 and 235.

Another idea I have had is maybe convert my video footage to 0-255 before calibration and then create a DVD with 0-255 flagged and then the 'DVD player' could automatically convert it if needed to 16-235. If so would also make the video footage more future proof, what do you think?

My video footage will be 1/2 D1 resolution DVD if that matters?

I appreciate all advice! :cool:

[TheSkiller]

Another idea I have had is maybe convert my video footage to 0-255 before calibration and then create a DVD with 0-255 flagged and then the 'DVD player' could automatically convert it if needed to 16-235.
This won't work, first there is no flag for this with MPEG2/DVD and second a DVD-Player always expects video to be in TV range. If you put a 0-255 video on DVD it will have the blacks and white clipped during playback.

I don't know what's the problem with "eyeballing" video levels on a PC, such 16-235 video would have to be converted to 0-255 for PC viewing. If you want to preview your Avisynth script correct in that regard then just add ConvertToRGB(interlaced=true/false) at the end and remove it prior to encoding.

[hello_hello]

I have found that it is really hard to 'eyeball' calibration because PC is 0-255 and my output format is 16-235. Does anyone know any tricks on how to do this? I am wondering if at the end of my script I use something like colormatrix to convert the displayed video image as 0-255 just for 'eyeball' calibration and then remove the colormatrix filter from my script after I have calibrated.

How are you previewing the video? I have my video card set to output PC levels to the monitor and TV levels to the TV. Opening a script using MPC-HC the levels are as they should be when displayed on either device, as they are when playing all video.
Alternatively you could probably use the MPC-HC "TV to PC levels" shader to fix the levels while previewing.

[jclampy]

This won't work, first there is no flag for this with MPEG2/DVD and second a DVD-Player always expects video to be in TV range. If you put a 0-255 video on DVD it will have the blacks and white clipped during playback.


Are you sure, HCenc has a setting called 'colorimetry' where you can set a flag for either BT709 or BT470 (BT601). Usually BT709 is for HD but I have read on this forum that some posters said they had commercial DVDs that were in BT709 format, unless they were posting erroneously. If you mean that a DVD player won't take notice of that flag anyway then I guess I can't do that then.


I don't know what's the problem with "eyeballing" video levels on a PC, such 16-235 video would have to be converted to 0-255 for PC viewing. If you want to preview your Avisynth script correct in that regard then just add ConvertToRGB(interlaced=true/false) at the end and remove it prior to encoding.

Well all PC software DVD players I have tested display DVD video at colorimetry of 16-235. But looking at the avisynth colorbars() in AVSPmod it looks to me that it is previewing 16-235 at 0-255. So if that is the case then it is hard to 'eyeball' calibration of uncalibrated video footage to 16-235 pallete when you are looking at a 0-255 display.

I have heard that it is possible to calibrate by matching skin tones as their position doesn't change too much on a histogram between BT601 & BT709 if you can get it to the right 'area'. Unfortunately I don't know what the people's skin tone in the video footage looks like in real life as I have never met them in person. :)

PS: video footage is the same as what I posted about in the other thread I started http://forum.doom9.org/showpost.php?p=1547214&postcount=18 source is RGB32 and I am converting to YV12 for filtering and then encoding in YV12.

Edit:
The script I have posted there is not 100% up-to-date as I have been tweaking it a bit. I will post a newer script in this thread once I get the calibration done.

[jclampy]

How are you previewing the video? I have my video card set to output PC levels to the monitor and TV levels to the TV. Opening a script using MPC-HC the levels are as they should be when displayed on either device, as they are when playing all video.
Alternatively you could probably use the MPC-HC "TV to PC levels" shader to fix the levels while previewing.

I'm not using a TV Output on video card, maybe I could look into that if it is the only viable solution.

I did have a look at VLC and MPC-HC but wasn't having luck. Thanks for replying about the "TV to PC levels" shader in MPC-HC. It works if I choose 16-235 to 0-255 [SD] shader. (I put the avisynth colorbars() through to MPC-HC to test that it works).

I'll do that if I have to. I'll just have a look and see if I can come up with a way to get AVSPmod to look like this as well. Thanks.

Edit:
This looks like a good place to get NTSC and PAL still images for calibrating displays; http://www.belle-nuit.com/testchart.html although I haven't tested them yet myself. There are instructions for how to make sure that when you have encoded the still images to a DVD that you can check the color values are correct for the colorspace your have output too. Hmm, not too sure about the color's on the testchart being at the same levels for NTSC and PAL though, as I thought they might be slightly different. Maybe someone can elaborate on that?

[hello_hello]

I did have a look at VLC and MPC-HC but wasn't having luck. Thanks for replying about the "TV to PC levels" shader in MPC-HC. It works if I choose 16-235 to 0-255 [SD] shader. (I put the avisynth colorbars() through to MPC-HC to test that it works).

Some renderers, such as Haali and EVR (I think) expand TV to PC levels by default, but generally video cards should have a video specific setting to expand TV levels to PC levels so video looks correct on a PC monitor (in my case the option can be found in the Nvidia Control Panel). If so, it should expand the levels regardless of the player being used.

[jclampy]

Some renderers, such as Haali and EVR (I think) expand TV to PC levels by default, but generally video cards should have a video specific setting to expand TV levels to PC levels so video looks correct on a PC monitor (in my case the option can be found in the Nvidia Control Panel). If so, it should expand the levels regardless of the player being used.

Ok, I will elaborate here a little as I retested everything;
VLC player can't play .avs files.
VLC player correctly displays DVD's at 16-235 by default on PC even if PC in 0-255 desktop.
MHC-PC player play 16-235 .avs file displayed as 0-255 which is wrong, but with 16-235 to 0-255 [SD] shader filter it looks fine.
MHC-PC player correctly displays DVD's at 16-235 by default on PC even if PC in 0-255 desktop, so don't use any shader filter!
PowerDVD player can't play .avs files.
PowerDVD player correctly displays DVD's at 16-235 by default on PC even if PC in 0-255 desktop.

Well all DVD's I have tried to play on PC regardless of player (ie; VLC, MPC-HC, PowerDVD), by default the colors are being displayed as 16-235 format which is correct in the way that what I am seeing on my monitor is same as what would be on TV. This is because those players use an 'overlay', even though my desktop is shown as 0-255. Whether it is being done in hardware by the video card or by software player defaults or both doesn't really matter. What I am wanting to do is display .avs files or 16-235 video footage in the 'video preview' in AVSPmod as 16-235 because by default it is being shown at my desktop 0-255 colorspace which is skewing everything and make calibration difficult.

In regards to video card, mine is an ATI card. Hmm, looking through Catalyst Control Center I can't see any settings for changing the video playback colorspace from TV to PC range. Unless they are using some obscure name?

Since I don't have a setting for my video card to make it go into TV colorspace mode, I guess the next best thing is to make AVSPmod display the video footage as 0-255 for 'eyeball' calibration of the color/chroma but at encoding time use 16-235. If I am correct then is just a matter of inserting a color change filter at end of script and then removing that filter before encoding.

If I can't do that then I am happy that MPC-HC has that shader filter so I can preview the color calibration that way. Thanks for pointing that out to me! :cool:

[jclampy]

I finally give up trying to get the AVSPmod 'preview window' to display 16-235 as 16-235 on my PC. It just ain't gonna happen. So my only option is to preview changes through MPC-HC with the 16-235 to 0-255 [SD] shader filter.
:eek:

Edit:
Just found out that MPC-HC is not viable in default setup because it will only open .avs files of RGB32 but not YV12 or YV24. I haven't got around to check whether this can be circumvented or not yet.

Oh well, in AVSPmod I'll just use a filter to change brightness -16 and contrast +16 at the end of my script and then remove that line after colour calibration. Just something I noticed in an mpeg2 decoder options, not tested myself so don't know of accuracy,etc.

Also, I noticed VirtualDub can use a .fx shader filter for output display, so if anyone knows where I can find one for 16-235 to 0-255 let me know. ;)

PS: There is some interesting reading on a post here which I haven't gone completely through yet;
http://www.avsforum.com/avs-vb/showthread.php?t=523614