Correcting a video with huge contrast?

[HappyLee]

Hi. I'm dealing with a terrible source again, and this time it's the Bad Tour Wembley DVD of Michael Jackson (released in 2012). There are lots of problems with the "enhancing" job of the VHS tape, like over-sharpened, but the biggest problem is the contrast of the video was adjusted way large. It's too black on places that aren't supposed to be that black, especially with his eyes. Same problem with white areas but lighter compared to the black problem.

Here's a comparison. On the left is the video source I'm trying to improve, and on the right is the normal color.
http://image14.poco.cn/mypoco/myphoto/20130701/22/6581821420130701225121018.jpg

Levels and Tweak won't make it look much better, because there are some places like the background that's truly supposed to be black, and they only makes the picture looks more grey.

My idea is to make gradual change of luma. Use Levels to make black parts not so black at first, detect dark areas and gradually make the middle part darker. For small black areas like his eyes, the gradual change won't make much different, but for large black areas like the background, it could make it's middle part darker like it's supposed to be. This, I think, could make the picture looks better in some degree.

But unfortunately I haven't found any filters doing that, and I'm not able to make my idea into script so far. So, could anyone here help me find a way to solve that problem? Thanks in advance! :)

[Guest]

What is the source on the right that you call normal, and why don't you use that one?

[raffriff42]

Looks like a job for SGradation (http://avisynth.nl/index.php/SGradation) (discussed here (http://forum.doom9.org/showthread.php?p=1606598#post1606598))
# output luma with gamma < 1.0
# ^
# 255+ +
# | +
# | +
# | +
# | +
# | +
# | +
# 128+ +++
# | +
# | +
# | +
# | +
# |+
# |+
# 0+-------------+-------------+> input luma
# 0 128 255

[johnmeyer]

AVISynth and its plugins are a marvelous tool for many jobs, but if I were facing this problem, AVISynth would be the last thing I would think of using. In my video editing program (Vegas), I have a number of "plugins" that the program provides, including a histogram correction feature, similar to what "raffriff42" shows above. I'd use that, along with color correction, in order to match different scenes.

[johnmeyer]

I did a screen grab of the "bad" still shot you posted, and then put that in my video editing program (Vegas). It took three plugins, adjusting the controls interactively using the real-time GUI feedback that any full-fledged editing program provides, but in the end I think I came up with something that is pretty close to matching the "good" still shot you posted above:

http://i177.photobucket.com/albums/w208/johnmeyer/Jackocorrected_zps722056b4.jpg

So, once again, try the controls in your video editing program and I think you'll get better results than anything you are likely to be able to do with AVISynth.

[raffriff42]

Meh, not great, but better than before. EDIT - agreed, a GUI would have been helpful.

https://www.dropbox.com/s/fsy03era8140nzd/sgrad-test-compare.jpg?raw=1
#avisynth

C1=ImageSource("sgrad-test-1.jpg")
C1=C1.Crop(0, 0, 512, 384).ConvertToYV12()

## original image
#return C1

#@func SGradation2(clip clp, val "gammalite", val "gammadark")

## less midrange contrast
C2=C1.SGradation2(0.3, 0.3)

## more overall contrast
C2=C2.Levels(42, 1.0, 192, 16, 235)

## stage one complete
#return C2

### optional ###

LoadModule("avslib", "filters", "channels")

## clamp red, hackishly (EDIT: should have used Limiter)
V=C2.ShowV
V=V.Levels(0, 1.0, 160, 0, 255, coring=true)
V=V.Levels(0, 1.0, 255, 0, 160)

C3=MergeYUV(C2.ShowY, C2.ShowU, V)

## stage two complete
#return C3

### compare all ###

return StackVertical(
\ C1.Histogram(mode="levels").Subtitle("before"),
\ C2.Histogram(mode="levels").Subtitle("SGrad+Levels"),
\ C3.Histogram(mode="levels").Subtitle("clamp red"))

[HappyLee]

What is the source on the right that you call normal, and why don't you use that one?
It's not the same tape, not even the same show, just a comparison.

Looks like a job for SGradation (http://avisynth.org/mediawiki/SGradation) (discussed here (http://forum.doom9.org/showthread.php?p=1606598#post1606598))
Yeah, you're right. SGradation really seems to be the answer. :)

Meh, not great, but better than before. EDIT - agreed, a GUI would have been helpful.
That looks much better. Thank you so much for it, and I'll try adjusting it with that method right away. :)

[HappyLee]

SGradation works great, but this happens sometimes (on the right, where you can see the luma blocks):
http://image14.poco.cn/mypoco/myphoto/20130702/21/6581821420130702215848060.png

What should I do? Is there a better way than MaskTools + MedianBlur, without ruining useful details? Or is there a smart luma smoother that can handle it (I tried SmoothHiQ but it doesn't work)?

[creaothceann]

Well, you could try the Dither (http://avisynth.org/mediawiki/External_filters#Deepcolor_Filters) functions.

[HappyLee]

Well, you could try the Dither (http://avisynth.org/mediawiki/External_filters#Deepcolor_Filters) functions.

I used GradFun3() but nothing I see is changed. I'm not doing it right am I?
Edit: Oh, sorry, should have increased the thr value.

[raffriff42]

There is only so much that can be done. See the HDR AGC thread (http://forum.doom9.org/showthread.php?t=93571) for a good discussion of the issues involved.

This call to HDRAGCHDRAGC(max_gain=2.0, max_sat=1.0, black_clip=10.0)looks pretty similar toSGradation2(0.3, 0.55)but one or the other may look better in the actual video.

[TheSkiller]

I used GradFun3() but nothing I see is changed. I'm not doing it right am I?
You should try SmoothCurve() (part of SmoothAdjust (http://forum.doom9.org/showthread.php?t=154971)) instead of trying to smooth out SGradation2() afterwards using GradFun3().

SmoothCurve() will give you plenty control. Make use of it's debug view show=true.

[Mug Funky]

i'm going to post in defence of colourists everywhere and say perhaps the producers wanted it that way? :sly:

it's a bit heavy-handed i'll agree.

also, having graded live shows before, you can't be sure those two pics are even slightly comparable (his pose is identical, but he's perhaps the best dancer in the last century so that makes sense). if the flare abates for a few frames you'll find the black-level crashes through the floor...

with crushed blacks, there's no recovering them... best just let them go. personally i would never have smashed them that hard unless johnny-big-nuts-producer insisted and i wanted to placate him or shut him up...

[HappyLee]

You should try SmoothCurve() (part of SmoothAdjust (http://forum.doom9.org/showthread.php?t=154971)) instead of trying to smooth out SGradation2() afterwards using GradFun3().

SmoothCurve() will give you plenty control. Make use of it's debug view show=true.

I tried to use SmoothLevels() instead of Levels() in the SGradation function, but I guess they can't replace each other. I don't really understand what SmoothLevels() and SmoothCurve() do. I tried SmoothCurve() after SGradation just now and I didn't see any difference. Could you teach me how to use it further? Thanks. :)

[raffriff42]

SmoothAdjust is pretty neat. I found I could stretch the blacks to an extreme degree without banding or blotchiness. #avisynth

C1=ImageSource("sgrad-test-2.jpg")
C1=C1.Crop(0, 128, 512, 256).ConvertToYV12()

## SGrad
C2=C1.SGradation2(0.6, 0.6)

## Smooth - moderate
#C3=C1.SmoothCurve(Ycurve="0-0;4-0;8-12;36-66;245-235;255-255", smooth=200, dither=100, show=false)
#C3=C3.SmoothLevels(23, 1.1, 255, 0, 255)

## Smooth - extreme
C4=C1.SmoothCurve(Ycurve="0-0;4-0;8-26;36-66;245-235;255-255", smooth=200, dither=100, show=false)
C4=C4.SmoothLevels(30, 1.35, 255, 0, 255)

S=StackVertical(
\ C4.Subtitle("Smooth"),
\ C2.Subtitle("SGrad"))

## exaggerate the artifacts
return S.Levels(0, 1.9, 255, 0, 255)
Note the "exaggerate " line is just to show off the smoothing.

Agree w/ Mug Funky on the futility of trying to rescue this particular video, especially if the stage lights are constantly changing.

https://www.dropbox.com/s/8zv65t9hpfkhw1e/sgrad-test-compare4.jpg?raw=1

[HappyLee]

I improved SGradation with SmoothAdjust. Finally the color banding problem is solved. Here's my code for SGradation2S (S for smooth):
function SGradation2S(clip clp, val "gammaW", val "gammaB") {
gW=float(default(gammaW,1.0))
gB=float(default(gammaB,1.0))
gc=clp.GreyScale()
dk=gc.SmoothLevels(16,1.0/gB,127,16,127,smooth=200,limiter=1)
lt=gc.SmoothLevels(235,1.0/gW,128,107,0,smooth=200)
r=Overlay(dk,lt,mode="add")
r.MergeChroma(clp)
}