How to improve an overexposed slightly out-of-focus footage?

[Underground78]

Hello,

I am currently editing some footage which was unfortunately badly recorded.

It is quite severely overexposed and I think also slightly out-of-focus.

I want to try improving it as much as possible, even if that ends up being not much, but I am lacking AviSynth knowledge to do so, hence this post.

I have already made some attempts, mostly based on scripts I found on the Internet. I got some results regarding overexposure but it is probably far from perfect. I also tried FQRestore hopping I would be able to remove a bit the out-of-focus blur but without any luck so far.

Here is a screenshot comparison between the source and the best I could do so far: http://screenshotcomparison.com/comparison/133920.

I have also uploaded a small sample: http://www.mediafire.com/file/c3vmxs84w4wuc80/sample.mkv.

I hope you will have a look and share some advises. :thanks:

Underground78

[MWilson]

https://i.imgur.com/7VLGqZ1.png

autolevels(1).invert().hdragc(10,10).invert().autolevels(1)

[wonkey_monkey]

How did you restore the overexposure?

[MWilson]

All I did was try to improve it. That's what I got with the script below the image.

[Underground78]

@MWilson : Thanks for your input.

How did you restore the overexposure?

In case the question was for me, I used the script from this post (https://forum.doom9.org/showthread.php?p=1606598#post1606598) uncommenting the first two lines of the AutoExpose function.

Here is a screenshot comparison of my attempt and MWilson's: http://screenshotcomparison.com/comparison/133938.

I feel like filtering is too strong, you get a bit more details but the hue seems kinda off and people on the video looks weird like even blurrier.

[MWilson]

I feel like filtering is too strong, you get a bit more details but the hue seems kinda off and people on the video looks weird like even blurrier.

I agree, but you can use mergechroma() and overlay() or merge() if it's too strong.
:)

[Underground78]

Overlay at 50% does improve things quite a lot. I cannot really decide whether the colors look more natural than with my first attempt.

[raffriff42]

I can't beat your color correction.
For sharpening, I like NonlinUSM (http://avisynth.nl/index.php/NonlinUSM) on the edges only. Something like this:
https://www.dropbox.com/s/xlfwaia2kdfpx5b/focus-nonlinusm1.jpg?raw=1
Import("NonlinUSM.avsi")

(source & color correction go here...)

S=NonlinUSM(z=6, str=1.5, rad=0.6).Sharpen(0.3)
EdgeMerge2(S, maskthr=40, showmask=false)
return Last

##################################
### merge 2 filters with edge mask
##
## @ C - source clip for non-edges
## @ E - source clip for edges in result as determined by edge mask
## @ M - edge mask; overrides internally generated mask
## (if supplied, 'maskthr' and 'maskblur' have no effect)
## @ maskthr - edge mask threshold (0~255, default 128)
## @ maskblur - amount of mask blur (0.0~1.0, default 0.6)
## @ strength - strength of clip 'E' blend (0.0~1.0, default 1.0)
## @ showmask - if true, return the edge mask
##
function EdgeMerge2(clip C, clip E, clip "M",
\ int "maskthr", float "maskblur",
\ float "strength", bool "showmask")
{
maskthr = Min(Max(0, Default(maskthr, 128)), 255)
maskblur = Min(Max(0.0, Default(maskblur, 0.6)), 1.0)
strength = Min(Max(0.0, Default(strength, 1.0)), 1.0)
showmask = Default(showmask, false)

M = (Defined(M) && IsClip(M)) ? M
\ : C.mt_edge("hprewitt", thY1=maskthr, thY2=maskthr, chroma=String(-128))
\ .Blur(maskblur)

return (showmask) ? M
\ : Overlay(C, E, mask=M, opacity=strength)
}

[Underground78]

After tweaking a bit MWilson's attempt, I got this: http://screenshotcomparison.com/comparison/134041 using :
overlay(autolevels(1).invert().hdragc(10,10).invert(), opacity=0.55)
(added overlay at 55% opacity and dropped the last "AutoLevels" call).

I think colors look much better than in my first version with AutoExpose while not being as strong as MWilson's first attempt.

If I add raffriff42's sharpening to this I get: http://screenshotcomparison.com/comparison/134052.

I think I'm getting somewhere. :)

If you have more ideas or comments, feel free to share. :thanks:

[Underground78]

I have made a test encoding of 15s to try out the filtering: http://www.mediafire.com/file/wgzz9l89kehs3at/colorfix_sharpen.mp4.

I am quite happy with it but it seems like there are some flickering artifacts in the sky. Anything I can do about those?

[WorBry]

Can't say I see any flickering in the skies there myself....just a bird at the top right towards the end.

BTW - another scripted function that does a pretty decent job with the backlight exposure 'correction', in terms of balanced 'pulldown' of near-blown highlight detail, is javlak's HighlightLimiter, which uses the classic 'multiply blend' approach.

https://forum.doom9.org/showthread.php?p=1574765#post1574765

Original

http://i.imgur.com/BuYLmgxm.png (https://imgur.com/BuYLmgx)



SmoothLevels(0,1.0,255,16,235)
HighlightLimiter()

http://i.imgur.com/e93sARYm.png (https://imgur.com/e93sARY)



SmoothLevels(0,1.0,255,16,235)
HighlightLimiter(twopass=true)

http://i.imgur.com/ReRKmJum.png (https://imgur.com/ReRKmJu)



SmoothLevels(0,1.0,255,16,235)
HighlightLimiter(twopass=true)
Autolevels()

http://i.imgur.com/PSHcFM9m.png (https://imgur.com/PSHcFM9)


Bearing in mind that the original clip has full range (0-255) luma and the compression to limited range (16-235) itself contributes some 'pulldown' of highlight detail - although the same can be said of the above 'autolevels/inverted hdragc' based methods. Some traces of posterization there in the clouds, but much better colour rendition IMHO. To be honest, in the filtered clip you posted it looks like they are dancing in front of some painted theatrical backdrop...OTT really.

And with RaffRiff42's sharpening routine (as is) to top it off:

SmoothLevels(0,1.0,255,16,235)
HighlightLimiter(twopass=true)
Autolevels()
S=last.NonlinUSM(z=6, str=1.5, rad=0.6).Sharpen(0.3)
EdgeMerge2(S, maskthr=40, showmask=false)


http://i.imgur.com/ypHQt4Dm.png (https://imgur.com/ypHQt4D)

[raffriff42]

I would expect any filter with auto levels or AGC to flicker somewhat, unless the gains can be locked down somehow... like StainLessS does:
https://forum.doom9.org/showthread.php?p=1824198#post1824198

[poisondeathray]

I'd probably boost and spread out the shadows and low midtones more. There is little gradation and crushed shadow detail in those examples instead of a gradual falloff. There are some missing details relevant to the dancers/clothing that are recoverable. For example, you should be able to see the 2nd guy is wearing a green kilt, 3rd guy is wearing a blue kilt. You have more non linear control with something like curves, but I suppose a lot of it is subjective and personal preference

I'm assuming the "subject" of this is the dancers, not the BG buildings or sky ? I wouldn't go overboard on the BG recovery, because you can get an unnatural looking "burn" effect. A graceful blow out is usually more preferrable than a burn effect for most people. But I suppose there is variation and opinion on that too. Some people might even add a vingnette or fake/post shallow DoF to obscure the BG more to draw viewer attention to the FG dancers , and also to hide flaws.

But to answer the question about "noise" , one way is to use a denoiser and limit the damage by apply through a luma mask based - so only brighter areas are filtered more and you don't damage the foreground areas

[WorBry]

I'd probably boost and spread out the shadows and low midtones more. There is little gradation and crushed shadow detail in those examples instead of a gradual falloff. There are some missing details relevant to the dancers/clothing that are recoverable. For example, you should be able to see the 2nd guy is wearing a green kilt, 3rd guy is wearing a blue kilt. You have more non linear control with something like curves, but I suppose a lot of it is subjective and personal preference

Not sure if you are referring to Underground78's filtered clip there or the examples I posted. For sure there's a lot more you could do with an NLE/grading suite that has a comprehensive set of color tools. Certainly I'd be looking at curves, qualifiers and localized gradients if I were to be tackling it that way, especially if the desire is to retain full luma range. But my understanding was that he wants to use only AVISynth filters. I guess one could have a go at it with the VDub Gradation Curves filter (1.46 beta for Win7 and above) and import the curves string into AVS script with LoadVirtualdubPlugin but that's a lot of back and to tweaking.

I wouldn't go overboard on the BG recovery, because you can get an unnatural looking "burn" effect.

Yes, that's what I was getting at by:

To be honest, in the filtered clip you posted it looks like they are dancing in front of some painted theatrical backdrop.

And the 'invert/hdragc' approach does have that tendency.

Anyhow, I only happened upon that HighlightLimiter function when I was browsing the same thread that spawned NonLinUSM.

Hitherto I'd had a stab at it using various combinations of a 'blend mode' function (Blend_MT_alpha3) I concocted years back (primarily for film look emulations), and RaffRiif42's equivalent, to apply the 'multiply blend' pull-down..

https://forum.doom9.org/showthread.php?p=1677913#post1677913

..coupled with hdragc to bring up the shadows - with a degree of success, but HighlightLimiter seemed to produce an acceptably decent result without all of the tweaking.

[Underground78]

Thank you for all your answers! :)

First about what I called "flickering", I think I got it wrong but poisondeathray still managed to understand what I meant.

But to answer the question about "noise" , one way is to use a denoiser and limit the damage by apply through a luma mask based - so only brighter areas are filtered more and you don't damage the foreground areas

I think I am just seeing compression artifacts that are made visible by the color corrections.

I am not sure how to phrase this correctly in English but it feels like the sky is pulsating (might be because of the I-frames).

I think your suggestion might be worth a try. Any help on how to apply it would be appreciated.

BTW - another scripted function that does a pretty decent job with the backlight exposure 'correction', in terms of balanced 'pulldown' of near-blown highlight detail, is javlak's HighlightLimiter, which uses the classic 'multiply blend' approach.

(...)

Some traces of posterization there in the clouds, but much better colour rendition IMHO. To be honest, in the filtered clip you posted it looks like they are dancing in front of some painted theatrical backdrop...OTT really.

I agree with you. The color rendition is somewhere in between my first attempt (less washed-out) and my attempt based on MWilson's filtering (and its artificially bright colors).

I'd probably boost and spread out the shadows and low midtones more. There is little gradation and crushed shadow detail in those examples instead of a gradual falloff. There are some missing details relevant to the dancers/clothing that are recoverable. For example, you should be able to see the 2nd guy is wearing a green kilt, 3rd guy is wearing a blue kilt. You have more non linear control with something like curves, but I suppose a lot of it is subjective and personal preference
I would be interested in bringing out more details (like on the dancers' clothing) but really I don't have any knowledge about all this.

I'm assuming the "subject" of this is the dancers, not the BG buildings or sky ? I wouldn't go overboard on the BG recovery, because you can get an unnatural looking "burn" effect. A graceful blow out is usually more preferrable than a burn effect for most people.
Yes, you are perfectly right about this. However it is also meant to convey a message about the location, namely Paris so it is kinda meaningful to ensure that one could recognize Notre-Dame.

:thanks:

Edit: I went ahead and encoded a 15s sample using WorBry's filtering: http://www.mediafire.com/file/c461l6d9lkok8a6/colorfix_sharpen2.mp4.

[WorBry]

Edit: I went ahead and encoded a 15s sample using WorBry's filtering: http://www.mediafire.com/file/c461l6d9lkok8a6/colorfix_sharpen2.mp4.

Still looks a bit like the dancers are superimposed - I guess that's down to the edge sharpening - better, I think though.

Would you mind uploading the original (unprocessed) sample of that particular 15s segment ?

[Underground78]

Sure, here it is: http://www.mediafire.com/file/ymhafqjch8m6gij/sample2.mkv

[WorBry]

Thanks. Yes, for my taste at least, I think I'd ease off some on the edge sharpening:

SmoothLevels(0,1.0,255,16,235)
HighlightLimiter(twopass=true)
Autolevels()
S=last.NonlinUSM(z=6, str=0.9, rad=0.6).Sharpen(0.3)
EdgeMerge2(S, maskthr=40, showmask=false)

or maybe even:

S=last.NonlinUSM(z=3, str=2.5, rad=0.6)
EdgeMerge2(S, maskthr=40, strength=0.9, showmask=false)

About the best I can suggest with that particular approach. Maybe others with higher knowledge of local contrast manipulation (AVISynth) methods could tease more detail out of it. Colour saturation control also.

Edit: I'd probably boost and spread out the shadows and low midtones more.

I'd have to agree and you could maybe get more mileage out of SmoothLevels using the 'limiting' parameters - but really:


You have more non linear control with something like curves

On that note - RaffRiff42 has just posted a helper function for SmoothCurves:

http://forum.doom9.org/showthread.php?t=175323

Yet to try it myself.

[poisondeathray]

I think I am just seeing compression artifacts that are made visible by the color corrections.

I am not sure how to phrase this correctly in English but it feels like the sky is pulsating (might be because of the I-frames).

I think your suggestion might be worth a try. Any help on how to apply it would be appreciated.




You can filter selectively with masks. It's typical to "reveal junk" from making levels manipulations, you will see that too in the shadows if you manipulate shadows. If using avisynth you would use mt_merge() or overlay() . To create the mask, there is a lumamask() function, or there are dozens of ways to create masks. A luma mask makes sense, because you're going to be filtering the sky (which is "bright") in the original. Areas that are 100% white in the mask are 100% affected. Areas that are 100% black are 0% affected, intermediate "grey" shades are intermediate. The idea is you don't want to damage some areas , or limit the filter to specific areas. If there was no camera movement, you could even draw out a static mask in gimp or photoshop. Or even just replace the sky and parts of the BG (but in your clip there is a bird flying, that would be "eliminated" unless you rotoed it out).

So you can apply filters through the mask, for example you might use smdegrain() , or mctemporaldenoise() or some temporal smoothing like ttempsmooth(). You might use stronger settings if you filter the sky /brights more, or use weaker settings if you don't push it as much, adjust it to what you feel is appropriate

Mask usage with overlay() and mt_merge() is covered in many posts , but if you're still lost after looking at some examples, just ask and someone will help you

It would look something like this
overlay(background, filtered, mask=mymask)






I would be interested in bringing out more details (like on the dancers' clothing) but really I don't have any knowledge about all this.



The concept is you're spreading out the tonal range instead of having them clumped in groups. Right now there is a lot of data "squished" in the shadows, so it looks like 1 shade of black (ok it's an overstatement, but you get the idea) . So the idea is to increase the shadow contrast so you can see finer gradations in the shadows. You've done the same thing for the sky and brights, it only makes sense to do that for the shadows too. HDRAGC is supposed to do that, but there is limited control. In this footage you have details in the shadows that you can bring out, just like the brights (the shadow detail is "crushed", not "clipped, unlike some of the highlights which are clipped and not recoverable). You want to do this non linearly, which helps to spread out the values. It analgous to photoshop curves, but instead of a traditional "s-curve" which increases contrast by pushing the ends out , you would use a "reverse s-curve" to bring the ends in. There should be many tutorials on curves. If you need more help after spending some time on it, just ask. Essentially it's remapping input to output, and you can insert as many control points as you want.

This was compared to "colorfix_sharpen2.mp4" which is too "contrasty" . I "overdid" this example to illustrate what details are "left on the table" in the shadows. Normally you wouldn't go this far because it's "low contrast, flat, or washed out" , illustrating the opposite end. So most people would do something in-between. But you can see colors in the kilts , pouch/sporran, patterns like checkers in the 4th girl's dress (instead of "black"), hair highlights etc.. A side effect of pushing it this far is you reveal more noise and "squish" midtones, and it looks flatter. So with curves , you can adjust individual areas, and if you want even more control you can combine with masks, a lot of it is subjective and personal preference. But objectively, you're crushing some shadow details with colorfix_sharpen2.mp4, and colorfix_sharpen.mp4. It's "ok" if you 're going for a special type of "look", but in general you want to spread out the tones more.

"colorfix_sharpen2.mp4" - crushed shadow detail
https://s9.postimg.org/5pbutyb0f/colorfix_sharpen2.png

shadow boost - low contrast to illustrate shadow detail
https://s9.postimg.org/xcok89e9b/boost_shadows.png

One difference in video is you're usually working in YUV . There are RGB and YUV curves, but most programs use RGB curves. If you work in RGB, make sure you convert using full range first. Ideally you would want to work at higher bit depth because you're able to make finer adjustments and have higher precision. Ideally you would want to work linearly as well, because sRGB has a curve baked in. Basically 1+1 does not equal 2 unless you work in linear.

WorBry mentioned gradation curves in vdub, rr42 has a new helper function for smoothcurve in avisynth (smoothpoints), there are other tools like blender, natron that have curves. Most NLE's and compositing tools have curves too

Use monitoring scopes like waveforms, histograms to see the distribution of the data. That will help guide you. In avisynth , histogram() is actually a Y' waveform





This is probably too much for this topic, but if the BG is important, some people would "fix" the BG with compositing. It's analgous to masking. Your sample clip had the camera locked off, but if there is camera motion you could motion track and apply that motion data to the "patches" - it's know as a "patch repair". This is a tourist site and you should be able to find many photos on google and such . Or if you had accompanying photos from the day (many videographers do this for this very reason, photos on the day of the shoot for compositing to embellish the video). You're basically covering up the problem areas , adjusting them to blend in. The patches are usually higher resolution, much higher quality - you have to "dumb" them down to match, otherwise they will stick out like sore thumb. You can't have the BG sharper and higher resolution than the FG dancers and objects. So on a shot like this you'd actually blur and blow out the patches a bit. There are ways to adjust for perspective, lenses, etc.. also, and more complex methods like camera projection for better matches. Static areas like buildings, walls, are relatively easy to "fix" . Some types of motion can be compensated for by motion tracking , but complex objects people etc.. very difficult. You have to mask out (rotoscope) the FG objects if they occlude the BG replacements. For example, if a dancer's head goes up & down and in front of a building like Notre Dame cathedral, you need to mask out that occlusion layer. You could even completely replace the sky with whatever you want (sky replacement is quite common in visual effects) , but it's more difficult if you want to keep the bird, and cloud movement). Ask if you want more info

[Underground78]

Thanks. Yes, for my taste at least, I think I'd ease off some on the edge sharpening

Thank you, I gave this a try: http://www.mediafire.com/file/hk861bzys46hqm2/2.mp4 and http://www.mediafire.com/file/10hnneeqxxbnxpu/3.mp4 (to be compared with colorfix_sharpen2.mkv).

I am not even sure I can see a difference... :o

@poisondeathray: Thanks, that's a lot of information to process.

I feel like trying all this so I will keep you posted. First I will have to set up AviSynth+ apparently to use this new curve helper.

Regarding the sky's replacement, that seems like an overkill here but it still interesting to read about it.

Edit : Figured how to create a luma mask: https://i.imgur.com/SRwu3es.png (just for illustration purposes, not saying this is a proper mask for what I want to do).

[WorBry]

I am not even sure I can see a difference... :o


There is, albeit subtle ;) You could also tweak the sensitivity of the edge mask a tad with the maskthr setting (set showmask=true, to see it), but 40-50 does look around optimum for this case.

[Underground78]

I've done some tests with the denoising and luma mask and got some results although I feel like I don't know what I'm doing. :p

I have used "AutoExpose(0.9)" for luma correction in this test, just because it is faster and kinda close to your proposal.

# Dumb mask creation for the first test but probably not that bad after all...
msk=LumaMask(last, last, 230, 240, show=true)

# Denoising (also dumb)
filtered=SMDegrain(tr=2,thSAD=250,refinemotion=true)
# OR
# filtered=MCTemporalDenoise(settings="medium")
# OR
# filtered=TTempSmooth()

# Apply masking (dumb copy-paste from LumaMask function, not sure what's the difference with Overlay)
mt_merge(filtered, msk, luma=true, U=3,V=3)


No denoiser: http://www.mediafire.com/file/m4b3rwtz21jdtbi/src_denoise.mp4
SMDegrain: http://www.mediafire.com/file/jff25aes866l9hb/SMDegrain.mp4
MCTemporalDenoise: http://www.mediafire.com/file/atm823hf3p44c5i/MCTemporalDenoise.mp4
TTempSmooth: http://www.mediafire.com/file/3pgg3r9774jz88h/TTempSmooth.mp4

I think the best results so far is with MCTemporalDenoise but that's probably because the settings for the others are not strong enough (and possibly for MCTemporalDenoise too).

[Underground78]

I haven't had time to work on this again but I have a few questions:

1) What is the actual difference between "mt_merge" and "overlay"?

2) About using SmoothPoints or any similar tool to further improve the color rendition of the video, would it replace completely the current filtering to improve the overexposure or would it be like a second pass?

[raffriff42]

1) they do similar jobs. Overlay controls the overlay strength not only through the mask, like mt_merge, but also has the opacity argument. There's also the mode argument, and a lot more, which you should read up on when you can.

2) SmoothPoints is only another tool; use it if you like it. You can get similar results with various combinations of Levels, ColorYUV, Overlay, MaskTools filters, SmoothAdjust filters, and lots of others (http://avisynth.nl/index.php/External_filters#Levels_and_Chroma). I use SmoothPoints a lot, but not exclusively.

Multiple passes are fine - if you work at high bit depth - 12 bits or more. Color correcting in 8 bit should not be done at all now that we have a choice in the matter.

Also, it needs to be stated in every thread like this, AviSynth is not an ideal color correction tool. Serious video film makers need to look at tools like Premiere pro, Vegas pro, Davinci Resolve, ShotCut etc.

[Underground78]

1) I'm aware of "Overlay" numerous modes and opacity argument. I have also looked up "mt_merge" documentation. My conclusion was that my case, both functions would work more or less identically.

2) Sure, but since I don't know the other tools, seems like SmoothPoints is a good starting point. Also I get the EQ idea so I feel like it might be easier for me to experiment with this particular filter.

Note I'm no serious colorist. I'm editing this video because I'm the only person in the association that has the basic knowledge to get the work done. If I can squeeze some more details out of the footage I would be happy but I'm aware it won't be anywhere near professional work. ;)

Edit : I am not sure my question #2 was clear so I will rephrase it. Should I try to use SmoothPoints or any similar tool on top of the color corrections WorBry suggested (it's the one I like best so far) or should it replace it completely?

[raffriff42]

You can use it on top of the other corrections, yes. It would be good for making a final overall levels adjustment.

[Underground78]

Ok, thanks. I guess I have to install AviSynth+ now and start experimenting. (Edit: Done, the upgrade went smoothly as far as I can tell.)

I did some more tests and I'm pretty happy with MCTemporalDenoise to denoise the sky. My very basic luma mask seems to do the trick.

[Underground78]

Use as few points as possible; if more control is needed, "upgrade" to the filter with the next larger amount of points -- starting with SmoothPoints2, to SmoothPoints3, to SmoothPoints5 etc.

I tried doing that but I ended up using SmoothPoints7 or SmoothPoints9 (using less control points I had the impression the adjustments were not selective enough, not sure if I'm making any sense)... :scared:

I feel like I not doing that right. Here is what I get:

Source:
https://i.imgur.com/34yg2O3l.jpg (https://i.imgur.com/34yg2O3.jpg)

SmoothPoints9(0, +10, 0, 0, 0, 0, 0, 0, 0):
https://i.imgur.com/4nXNvmfl.jpg (https://i.imgur.com/4nXNvmf.jpg)

It seems like it's already too strong (see the girls' hair for example), yet the kilt of the second guy is still too dark.

[poisondeathray]

It seems like it's already too strong (see the girls' hair for example), yet the kilt of the second guy is still too dark.


You're not going to have enough control if you want to affect the kilt but not the hair at all. Because they have overlapping values.

If you want to affect kilt only, you're going to have to rotoscope (precise masking, moving masks). Affecting by Y range or RGB values precisely is impossible , even with specialized programs, because of the overlap in values.



In reality, if you were standing right there, your eyes would see more "shades" or values. It wouldn't be 1 "shade" of black (again an exagerration) . Both hair and kilt would demonstrate more "shades". Not just the kilt. The videographer would have used lights ideally and an ND filter

[Underground78]

Ok, so maybe there isn't so much I can do. I feel like the girls' hair, the kilt of the first guy on the right and some of the guys' shoes are looking a bit weird when I try to reveal some of the shadow details.

Edit : Missed your edit, I see your point. I need to have a closer look...

[WorBry]

Should I try to use SmoothPoints or any similar tool on top of the color corrections WorBry suggested (it's the one I like best so far) or should it replace it completely?

The problem with using SmoothPoints, or another 'curves' tool, to try and offset the 'highlight pulldown' introduced by HighlightLimiter and boost/expand the shadows/lower mid-tones, is that you are fighting with a somewhat irregular curve (see below).

As PDR explained above, what you are aiming for ideally is smooth asymmetric 'reverse S-curve':


The concept is you're spreading out the tonal range instead of having them clumped in groups. Right now there is a lot of data "squished" in the shadows, so it looks like 1 shade of black (ok it's an overstatement, but you get the idea) . So the idea is to increase the shadow contrast so you can see finer gradations in the shadows. You've done the same thing for the sky and brights, it only makes sense to do that for the shadows too. HDRAGC is supposed to do that, but there is limited control. In this footage you have details in the shadows that you can bring out, just like the brights (the shadow detail is "crushed", not "clipped, unlike some of the highlights which are clipped and not recoverable). You want to do this non linearly, which helps to spread out the values. It analgous to photoshop curves, but instead of a traditional "s-curve" which increases contrast by pushing the ends out , you would use a "reverse s-curve" to bring the ends in

Something along the lines of:

http://i.imgur.com/tHDfli0m.png (https://imgur.com/tHDfli0)

That was created in DaVinci Resolve 14.3 (free version) with the Custom Curves tool (YRGB colorspace) using a greyscale ramp source; I combined the waveform plots with and without the curve applied as one image. The Resolve scopes are scaled for 10-bit depth, so think of 0-1023 as equivalent to 0-255 range in 8-bit.

I spent a good while last evening attempting to shape the output of that HighlightLimiter script into something similar using SmoothPoints, without satisfactory results. Here are some AVISynth 'waveform' plots (i.e. Histogram(mode='classic') produced with a full range (0-255) YV12 ramp as source, compressed in the first instance to 16-235 with SmoothLevels:

http://i.imgur.com/2S1pg10m.png (https://imgur.com/2S1pg10)


As you can see, your SmoothPoints9(0, +10, 0, 0, 0, 0, 0, 0, 0) has little impact. Better (although not perfect) results could be achieved by plotting the entire curve - but that entailed a lot of tweaking also:

SmoothLevels(0,1.0,255,16,235)
SmoothPoints9(0, +6, +6, -14, -48, -50, -36, -18, 0)
Autolevels()
S=last.NonlinUSM(z=6, str=0.8, rad=0.6).Sharpen(0.3)
EdgeMerge2(S, maskthr=40, strength=0.8, showmask=false)

Note: I ended up adding Autolevels there because I couldn't quite get the 'upper-highlights' portion of the curve smooth enough with SmoothPoints on its own.

Quick Disclaimer: I'm not at all suggesting those settings as optimum for your particular sample footage. It was just to illustrate the basic shape of curve to go for and how to go about it. If you find it useful, the script used for preparing the greyscale gradient and visualizing the curves was:

colorbars().converttoyv12().mt_lutspa(relative=true,expr="x 255 *").greyscale().Turnright().LanczosResize(360,360)
....applied filters.....
Histogram(mode="classic")

Really though (and not to deny the "Power of AVIsynth") it's a lot easier to do this in an NLE/color grading suite with a curves tool and videoscopes. The Curves 'effect' in KDenLive (of which there is a Windows version) is actually quite useable, for 8-bit processing at least - there's a Bezier curves set with adjustable spline handles also:

https://kdenlive.org/

[Underground78]

Thanks for your input. I had got this completely wrong... I did try using SmoothPoint9 on the source but I thought it was meant to replace SmoothLevels while keeping HighlightLimiter.

I will give this another try now that I understand better how all this works.

Not sure I will have time fiddling with Kdenlive but I will keep that in mind.

[WorBry]

Thanks for your input. I had got this completely wrong... I did try using SmoothPoint9 on the source but I thought it was meant to replace SmoothLevels while keeping HighlightLimiter.


Which will produce a curve like this:

http://i.imgur.com/SqFX4kfm.png (https://imgur.com/SqFX4kf)

Again, not really what you want. That pesky 'mid-tone hump' is a big obstacle to get around.

Edit: Less so if you set HighLightLimiter for 'single pass' (i.e. one pass of the 'multiply blend' computation) but then the highlight detail pulldown will be less effective:

http://i.imgur.com/yyNkHXNm.png (https://imgur.com/yyNkHXN)


I will give this another try now that I understand better how all this works.


Just don't end up with a symmetric reverse (negative) S-surve (i.e. with the axis/pivot intersect at the mid-point) - all that will do is flatten the image.

[WorBry]

SmoothLevels(0,1.0,255,16,235)
SmoothPoints9(0, +6, +6, -14, -48, -50, -36, -18, 0)
Autolevels()
S=last.NonlinUSM(z=6, str=0.8, rad=0.6).Sharpen(0.3)
EdgeMerge2(S, maskthr=40, strength=0.8, showmask=false)

.... I'm not at all suggesting those settings as optimum for your particular sample footage.


Nah, too much loss in mid-tone detail with that curve. Needs lower pivot point. You'd need to play with it.

[raffriff42]

I happen to be working on a smooth S-curve filter, that doesn't have the "crossover" issues I've seen with others; here it is.
Not as completely tested as I'd like, but it seems to work.
Requires AviSynth+ (http://avisynth.nl/index.php/AviSynth+), Utils-r41.avsi (https://github.com/raffriff42/AvisynthPlusUtilities/blob/master/Utils-r41.avsi) and Utils-r42.avsi (https://github.com/raffriff42/AvisynthPlusUtilities/blob/master/Utils-r42.avsi) (no MaskTools required; it's using Expr (http://avisynth.nl/index.php/Expr)).

### simple gamma adjustment (more = brighter lowlights)
#@ function GammaCurve(clip C, float gamma, bool "tvrange")

### simple 'anti-gamma' adjustment (more = darker highlights)
#@ function AntigammaCurve(clip C, float antigamma, bool "tvrange")

### merge GammaCurve and AntigammaCurve
#@ function GammaScurve(clip C, float gamma, float antigamma, bool "tvrange")

## @ tvrange - if true, anchor gamma curve on TV black (16d) and TV white (235d);
## default false for RGB source, true for YUV;
## for Y-only sources, if 'tvrange' not specified, an error is raised.

https://www.dropbox.com/s/3hm3qvpmd2dvfcj/GammaScurve-01.png?raw=1 https://www.dropbox.com/s/ng73konh6907txc/GammaScurve-02.png?raw=1

https://www.dropbox.com/s/pqka96c30qies60/GammaScurve-03.png?raw=1 https://www.dropbox.com/s/fu39eevhy7ej6d8/GammaScurve-04.png?raw=1

[Underground78]

Which will produce a curve like this (...)
Well I didn't use SmoothPoints9(0, +10, 0, 0, 0, 0, 0, 0, 0) in this case but something similar to what you did (i.e. arguments starting around +10 and ending around -30 with a progressive decrease).

I happen to be working on a smooth S-curve filter, that doesn't have the "crossover" issues I've seen with others; here it is.
Thanks, I will have a look.

[Underground78]

So far the best I could come up with is:
SmoothLevels(0,1.0,255,16,235)
HighlightLimiter(twopass=true)
Autolevels()
GammaScurve(1.3, 1)

So from:
https://i.imgur.com/34yg2O3l.jpg (https://i.imgur.com/34yg2O3.jpg)
to:
https://i.imgur.com/lsPDazwl.jpg (https://i.imgur.com/lsPDazw.jpg)

Not sure if I could/should drop AutoLevels() but I couldn't get any good results without HighlightLimiter(twopass=true).

[WorBry]

Not sure if I could/should drop AutoLevels() but I couldn't get any good results without HighlightLimiter(twopass=true).

Yes, it's the 'multiply blend' component of the curve that makes it so effective for pull-down of near-blown highlight detail in this context - it's a 'classic' approach to 'correcting' overexposure in post, bearing in mind that once highlights are blown (clipped) there is no detail to 'recover'.

The problem, as I indicated above though, is the 'mid-tone' bump that HighLightLimiter produces, which is effectively flattening the mid-tones and reducing local contrast. Here's what impact GammaSCurve (1.3,1.0) has on the resulting curve, with and with Autolevels():

http://i.imgur.com/rQs8QlRm.png (https://imgur.com/rQs8QlR)


As you can see, yes it does boost the shadows and lower mid-tones some, which is what you want, but if you remove AutoLevels, the white point is 'pulled-in' (or 'down' depending how you view it) reducing global contrast. On the other hand, if you leave AutoLevels in, it lifts the broader mid-tones and accentuates that 'mid-tone bump'. So it's 'swings and roundabouts'. Still, that's definitely a big improvement.

Maybe a better approach would be to apply a multiply-blend to the entire curve (at optimal opacity) and then examine methods for bringing back the shadows/lower-mids - something I was aiming for before I stumbled upon HighlightLimiter:


Hitherto I'd had a stab at it using various combinations of a 'blend mode' function (Blend_MT_alpha3) I concocted years back (primarily for film look emulations), and RaffRiif42's equivalent, to apply the 'multiply blend' pull-down..

https://forum.doom9.org/showthread.php?p=1677913#post1677913

..coupled with hdragc to bring up the shadows - with a degree of success

Don't really have the time to spend more on that, with all the tweaking involved. But here are two results where that approach was applied in DaVinci Resolve, the first based on a single pass multiply blend:

http://i.imgur.com/4Nh8WTym.png (https://imgur.com/4Nh8WTy)

and then using a two-pass multiply blend (with lowered opacity)

http://i.imgur.com/IPUCdOPm.png (https://imgur.com/IPUCdOP)

No 'Custom Curves' were used and the only "mask" was an end-stage qualifier on the skin tones, to reduce saturation a tad (needs a bit more I think). Resolve also has a nice tool for boosting 'mid-tone detail' (local contrast), but I finished off by applying some edge sharpening to the rendered outputs with RaffRiff's EdgeMerge2.

Could definitely pull some more detail out of the foreground lower-mids/upper-shadows (including the kilt patterns) with 'curves' and add a gradient on the cloudy sky to give it more depth, but I left it there as it was more to gauge whether the basic multiply-blend/shadow boost (HDR) approach is worth pursuing. I prefer the two-pass multiply version (better highlight pulldown), but bringing back the shadows involves more work. Maybe too dark for some, but possibly closer to what the scene looked like standing there with the partial shade.

Your last result definitely brings out more background detail (of Notre Dame) though, which you indicated is desirable.

[Underground78]

That last screenshot does like really good. The color rendition is really nice, I love that the sky is blue and not grey.

I have asked one of the dancers and for what she remembers, the background part really looked like this that day.

I see that DaVinci Resolve is free so I might try using it. I feel like the AviSynth-only approach would take me way too much time being that I don't really understand what I'm doing and I don't really have time to learn about it.

[WorBry]

For sure Resolve is a powerful, pro-level software with more than enough tools for a task like this, but bear in mind there's a learning curve there also and it pays to have a firm grasp of color correction/grading concepts and techniques for it to be of value.

If you're interested there's a good introductory tutorial series on YouTube produced by Goats Eye View:

https://www.youtube.com/playlist?list=PLyHr4tji72rUvEooOCYTJVyjIzRcJp3b0

It was created for Resolve 12/12.5 specifically, but it mostly still applies to Resolve 14. You'll need to be on Windows 10 BTW. One other thing - the quality of the native H264 encoder in Resolve is nothing to write home about, so you'd likely be looking at exporting to an 'visually lossless' intermediate (DNxHD, Cineform) for encoding to x264, with whatever.

Anyhow, should you get up and running on Resolve, you could PM me and I'll send you a 'Power Grade' file for that particular grade to play with.

Meanwhile there are a couple of AVISynth possibilities I'm looking at, as and when I have time. One is applying RaffRiff42's GammaCurve through a luma range mask. If it turns up anything magical, I'll post.

[Underground78]

For sure Resolve is a powerful, pro-level software with more than enough tools for a task like this, but bear in mind there's a learning curve there also and it pays to have a firm grasp of color correction/grading concepts and techniques for it to be of value.

Yeah, I'm aware of that and even more so after I installed Resolve and opened it. :D

I did find the curve tool and also this "nodes" thingy that I used to multiply-blend the input with itself at lowered opacity (using the "Gain" setting from the "Key" panel). I'm not sure at all it is what you really did though but I wanted to give it a try.

Meanwhile there are a couple of AVISynth possibilities I'm looking at, as and when I have time. One is applying RaffRiff42's GammaCurve through a luma range mask. If it turns up anything magical, I'll post.
I thought about something like that after I used the luma mask to denoise the sky. In fact I even tried it with the previous color correction filtering but I probably did that wrong because it ended up looking really weird.

[WorBry]

I did find the curve tool and also this "nodes" thingy that I used to multiply-blend the input with itself at lowered opacity (using the "Gain" setting from the "Key" panel). I'm not sure at all it is what you really did though but I wanted to give it a try.

I've sent you a PM.

[Underground78]

Thanks a lot! :)

I'm trying to understand exactly what you've done:

Node 1 does nothing as far as I can tell.
Node 2 applies some brightness correction but I cannot find where this setting was modified.
Node 3 is where the multiply-blend magic happens:

pass1 = multiply(input, input @ 60% opacity)
pass2 = multiply(pass1, pass1 @ 60% opacity)

Node 4 applies a 2-step "levels"-like correction to fix the really dark shadows caused by the previous step. Again I cannot find where you edited the settings.
Node 5 is 3-step saturation correction with some masking at step 2 to focus on the skin tone. Once again I cannot find how you applied the "levels" filter. I do see that the masking stuff is happening in the "Qualifiers" tab. I also see that you used the "Curves" tool with mode "Lum vs Sat" but I cannot see what it does (I tried disabling that part and playing with that curve).
Node 6 also doesn't do anything as far as I can tell.

[WorBry]

Seeing as this is an AVISynth Usage thread, I'll elaborate on your points by PM.

Suffice it to say the real 'magic' in that Resolve grade was the method used to bring up the Shadows (Stage 4) - that's what I'm striving to replicate with AVISynth. The 'multiply blend' part is straightforward. The Saturation control could also be done with AVISynth - I concocted a function SelSah way back when for that type of thing:

https://forum.doom9.org/showthread.php?p=1405481#post1405481

Basically Tweak applied with a luma range mask. You can get reasonable isolation of skin tones with it (in YV12), although no way near as good as Resolve or other pro-level NLE's. Haven't used it in years.

I also see that you used the "Curves" tool with mode "Lum vs Sat" but I cannot see what it does (I tried disabling that part and playing with that curve).


Partial desaturation of the (deeper) shadows - principle illustrated here:

https://youtu.be/mUmogRJOlnI
https://youtu.be/EUBqwnWcFUg

Very useful tip, not just for 'cinematic look'. Creating gradients in RGB colorspace can profoundly affect saturation levels, as you can see from the output of the multiply blend.

[WorBry]

Meanwhile there are a couple of AVISynth possibilities I'm looking at, as and when I have time. One is applying RaffRiff42's GammaCurve through a luma range mask. If it turns up anything magical, I'll post.

Tried various options for pulling up the shadows after applying a multiply blend (1 and 2-pass), including passing HDRAGC or RaffRiff42's GammaCurve through a luma range mask with a soft roll off, but just couldn't get the curve shape right. They all ended up too flat or reduced the effectiveness of the highlight pulldown.

If you are happy with the results of the Resolve grade, I'd say go with that, in this case. I added a tweaked version that pulled a bit more detail out of the foreground upper shadows/lower-mids (e.g the kilt patterns). I'd still finish it off with a pass through RaffRiff42's EdgeMerge2 though; Resolve's 'mid-tone detail' tool helps to bring some local contrast, but it's still left a bit fuzzy around the edges. There is a Resolve technique for creating a "High Pass" filter intended to sharpen focus, but it requires a lot of tweaking and doesn't look that great. This get's it much better.

[Underground78]

Sorry I couldn't answer earlier, I don't have much time currently to work on this.

However I think I will go with Resolve to fix the color and then use AviSynth to apply RaffRiff42's EdgeMerge2 since I'm really happy with that part. I was going to edit the footage with AviSynth anyway being that I don't know Resolve enough.

Thanks a lot for your help!

[WorBry]

Sounds good. Can't think of a better way to do it, other than with a 'custom curves' tool maybe.

Edit: In retrospect though, I would reduce the global saturation more (and tweak the skin tone mask accordingly) in that '2-pass multiply' grade. It's nice to see the colours 'pop' but it's a bit too vibrant, IMHO.

[Underground78]

Sorry I couldn't answer earlier. I kinda agree with you about saturation, I think I will reduce it a bit more.

I have another (hopefully last) question about the second grade though. I know for sure that the sky was blue that day which makes me think the final result looks much better without the first "levels" step. If I keep it enabled, it is like I'm looking at the footage through a sort of brown filter (though it does loose some details from the shadow if I disable it).

[WorBry]

You don't want to delete that first 'levels adjustment' as it's crucial to the process. As I explained in my PM, your clip was recorded with full range (0-255) luma. That first step compresses the full range data to what Resolve calls 'Video Levels' - consider it the equivalent of SmoothLevels(0,1.0,255,16,235) in YV12. Resolve actually processes with 32bit floating point at full data levels, so recorded data outside of the 'Video Levels' range is not lost and can be pulled into range. Simplest way to do that is to raise the black point with Lift and lower the white point with Gain, but in this case I used the 'Highlight' tool (in place of Gain) which is tailored for 'recovering' near-blown highlight detail (up to a point) - not to be confused with the 'Highlights' control in the 3-way 'Log Wheels'.

If you delete that first step it will effectively crush that outlying data and what detail could be recovered from it will be lost when the multiply blend is applied i.e loss of virtually all of the cloud relief and much of the recoverable detail of 'Notre Dame' in the background. Likewise what detail can be recovered from the deep shadows when the 'shadow pull-up' is applied will be lost.

Now you could instead process at Full Data Levels (change the Data Level in the Clip Attributes to 'Full'), which will preserve the full range of your source clip. In that case you would definitely delete that first 'levels adjustment' step. But I think you get a better highlight detail recovery applying this treatment at 'Video Levels'.

If the issue is that the blue sky showing between the clouds is not 'blue enough' for you (whether the tint or saturation) that's easily remedied with a qualifier on those blue patches. For sure though it was not a cloudless blue sky that day - and to make it so would require a more involved 'sky replacement' strategy as PDR outlined above.

If I keep it enabled, it is like I'm looking at the footage through a sort of brown filter

If you mean it's a little on the 'warm' side, well yes I'd agree the patches of light (compared to the original) in the foreground do give it a slight 'golden hour' feel - but that was not by design, its purely the way the colours came out.

If that doesn't appeal to you (or it presents issues for matching with other clips) you could always adjust the white balance, as a final step. Resolve 14 now has an 'auto white balance' tool - use the 'white balance' picker (same tool set as the 'Highlight' control, below the color wheels) to select a white/neutral grey object on the Preview screen. Or you could do it manually. One way (which I favour) is to zoom in and/or key on the chosen white/neutral grey object, pull up the Vectorscope and use the Temp and Tint controls to manipulate the plot (dot) to the centre cross-hairs on the scope. Or else, just adjust the Temp (negative values > cooler) until it looks right to you.

Actually (because it's an interesting exercise) I'd also been looking at 'Custom Curves' in place of the 'Multiply-Blend' approach, both in Resolve and using the VirtualDub Gradation Curves filter (imported to AVISynth). One advantage is that it achieves a more natural (i.e. less 'HDR') look and with less colour saturation issues to deal with....but it's harder to get it right. I'll maybe post some results when I'm done.

[Underground78]

My bad, I forgot to switch to video levels in "Clip attributes" before applying the grade... :(

So I was in fact exactly in this situation:

Now you could instead process at Full Data Levels (change the Data Level in the Clip Attributes to 'Full'), which will preserve the full range of your source clip. In that case you would definitely delete that first 'levels adjustment' step.