AutoLevels v0.10

[StainlessS]

AutoLevels v0.10 (avs v2.58, avs/+ v2.6 x86 & x64)
Req VS 2008 Runtimes.
8 Bit Only.

This thread continued from thread in Avisynth Development forum, here:-
http://forum.doom9.org/showthread.php?p=1876311#post1876311

AutoLevels only has thread in devs forum, and probably a good idea that it has one in Usage forum, so posting this here.

Includes 3 dll's, + source + Full VS2008 project files for easy rebuild.

See HOSTED directory @ MediaFire in sig below this post.

Now looks like this. Top line shows Frame number: and averaging range start and end frames
https://i.postimg.cc/TYCdsxVV/Auto-Levels-0-10.jpg (https://postimages.org/)

[StainlessS]

AutoLevels() v0.10, update see above post.

BugFix

// 11/06/2019 Ver 0.10 + Oops, limited gamma max in adjust to 1.0 instead of 10.0 (so only ever reduced gamma, never increased).
// Mods to debug output, shows output Y frame average.

Below in BLUE was 1.0

gamma = min(max(gamma,0.1f),10.0f); // ssS: Added sane range limiting (& avoid div by zero on gamma)


I dont thinkg AutoLevels works as well as it could, but I'm just too stupid to fix it properley. :(

As it stands it calcs input midpoint relative 0->255, and output midpoint same 0->255 (after input coring if required, and before output coring).

I think it would produce better results if we could figure out a way to calc required gamma based on input and output ranges,
any maths geniuses (or should that be genii) knocking about round here ?
I cant even figure out how to do it disregarding coring, and if coring then even bigger problem.

Here source.

// use user specified input high/low points, if specified.
// if coring is specified but input high/low aren't specified,
// we must do the inverse mapping to counteract the mapping that
// gets done inside adjust().
double in_low = (force_input_low) ? double(input_low)
: (do_coring) ? inv_coring_func(ymin_avg)
: ymin_avg;
double in_high = (force_input_high) ? double(input_high)
: (do_coring) ? inv_coring_func(ymax_avg)
: ymax_avg;

// pick a gamma curve
double do_gamma = (autogamma) ? log(ymean_avg/255.0) / log(gamma_midpoint) : gamma;
// double do_gamma = (autogamma) ? log((ymean_avg-ymin_avg) / (in_high-in_low) ) / log(gamma_midpoint) : gamma;
// IDEALLY GAMMA MIDPIOINT COMPUTED FOR INPUT RANGE (REQUIRES MORE WEIRD METHOD WHERE CORING)
// NEED ALSO ALLOW FOR SETTING MIDPOINT RELATIVE OUTPUT RANGE (WELL WEIRD WHERE CORING)
// HOW ???

// use levels() code to do remapping
adjust(frame,dst, int(in_low+0.5), float(do_gamma), int(in_high+0.5),int(output_low+0.5), int(output_high+0.5), do_coring);

////////////////////////////////////////
////////////////////////////////////////
////////////////////////////////////////


void Autolevels::adjust( PVideoFrame &frame, PVideoFrame &dst,
int in_min, float gamma, int in_max, int out_min, int out_max,
bool coring)
{

BYTE map[256], mapchroma[256];

// ----- this section is taken from: ------
//Levels::Levels( PClip _child, int in_min, double gamma, int in_max, int out_min, int out_max, bool coring,
// IScriptEnvironment* env )


gamma = min(max(gamma,0.1f),10.0f); // ssS: Added sane range limiting (& avoid div by zero on gamma)

gamma = 1/gamma;
int divisor = (in_max == in_min) ? 1 : (in_max - in_min); // avoid zero divide

if (vi.IsYUV())
{
for (int i=0; i<256; ++i)
{
float p;

if (coring)
p = ((i-16)*(255.0f/219.0f) - in_min) / divisor;
else
p = float(i - in_min) / divisor; // range 0.0 -> 1.0 of input range

p = pow(min(max(p, 0.0f), 1.0f), gamma); // gamma
p = p * (out_max - out_min) + out_min; // output range with out_min offset
int pp;

if (coring)
pp = int(p*(219.0f/255.0f)+16.5f);
else
pp = int(p+0.5f); // round to nearest luma level

map[i] = min(max(pp, (coring) ? 16 : 0), (coring) ? 235 : 255);

int q = ((i-128) * (out_max-out_min) + (divisor>>1)) / divisor + 128;
mapchroma[i] = min(max(q, (coring) ? 16 : 0), (coring) ? 240 : 255);
}
} else if (vi.IsRGB()) {
for (int i=0; i<256; ++i)
{
float p = float(i - in_min) / divisor;
p = pow(min(max(p, 0.0f), 1.0f), gamma);
p = p * (out_max - out_min) + out_min;
int z=int(p+0.5f);
map[i] = (z < 0) ? 0 : (z>255) ? 255 : z;
}
}
}

[Cary Knoop]

I would convert the code values to float, then convert gamma to linear then determine the midpoint and put it all back to the original gamma.

You might want to add an input gamma parameter (Rec.709 2.2, Rec.709 2.4, etc) to determine how to convert to linear and scale the float values to limited or data as required if the output range is different. I would at least have an option to have a float32 in and out as opposed to 8-bit only.

[StainlessS]

Thankyou Mr Knoop, now I just have to figure out how to do that. :)

It would seem easiest (if not the only way [it did not even remotely occur to me]).

I had intended (If Pinterf did not just jump in) to attempt 10, 12 14, 16 bit source, not sure that I would bother with Float, but I guess that would now make sense too.

Not gonna attempt just yet, would like to apply stuff from this mod to GamMac, soon.


Again, thanx, you be a superstar.

[SnillorZ]

Not gonna attempt just yet, would like to apply stuff from this mod to GamMac, soon.

Ohh Mr StainlessS, your a tease :p

What are your thoughts on how you would classify the next version of GamMac - bugfix / performance improvements release / Quality of results release ?

Are you able to give rough timescales?

........Asking the above for a friend (who's a serial procrastinator and needs to stop p*$$ing about waiting to see what the next best version of plugin might bring and actually get on with something productive and process some actual footage to use.)

[StainlessS]

Is gonna be Exploratory, timescale, Hmm, When all the lights go out in the Universe.
I would prefer if somebody else did it, you game ?

Also, just twigged I think, for the AutoLevels thing, would need convert to RGB, then linear, then do midpoint, and then back pedel to original colorspace.
Not that easy (for me) really.

[SnillorZ]

When all the lights go out in the Universe.
I would prefer if somebody else did it, you game ?

Now that would be a long wait.

All good though, I can get on with some proper work.

[fenomeno83]

Hi. Thanks! I use a lot autolevels 0.7. What are your improvements? Thanks

[StainlessS]

Some bug fixes[crashes, access violations], Avs v2,60 colorspaces, and x64 bit.

I found a file that shows lots of flickering (head bobbing back and forth in front of a small sun lit window), so am currently
trying to improve that. Same clip also fades to black at end and where very dark almost black, is amped so much that
it looks awful [still not managed to find fix for that]. (autogamma=true).

Will try up a v0.11 test dll, say if any better than 0.7.

[StainlessS]

Here v0.011Beta (Beta really as two new args that may or may not survive as is to next update).
See MediaFire in sig below this post, HOSTED folder.



// 01-09-2010 Ver. 0.6 + added support for the "coring" flag
// Jim Battle + revised/improved the readme document
//
// StainlessS
// 31/05/2019 Ver 0.7 + Convert for avs v2.6/+ x86 and x64. revised a bit.
// 06/05/2019 Ver 0.9 + Switch off debugging in v0.8, oops.
// 11/06/2019 Ver 0.10 + Oops, limited gamma max in adjust to 1.0 instead of 10.0 (so only ever reduced gamma, never increased).
// Mods to debug output, shows output Y frame average.
// 08/07/2019 Ver 0.11Beta1 + Basic scene change detect also now takes into account diffLumaMean((n,n-1))>=(sceneChgThresh*0.5), as well as min and max luma diff.
// + Added sc2Th=8, and sc2Perc=33.33, a check and possible override to sceneChgThresh detected scene change.
// + Changed default filterRadius from 5 to 7.


Only avialable doc for diffs is this


AviSource(".\ALTEST.AVI").Trim(74,11154).crop(0,46,0,-46)
######################
ORG=Last
AUTOGAMMA=True
DEBUG=true
# DEFAULT
SCENECHGTHRESH=20 # SC if diff of min or max Y to prev frame is more than this : NEW additional SC if DiffMeanY(n,n-1) is more than 1/2 this. [If either, recheck for override to non SC via SC2 stuff]
# NON DEFAULT
MIDPOINT=0.42 # Default stays 0.5
# NEW, If Scene change detected via SceneChgThresh, then re-check and override to NOT SC if not at least SC2PERC(33.33%) of luma pixels differ by more than SC2TH(8) when compared with prev frame.
SC2TH=8
SC2PERC=33.33
######################
A=AutoLevels(autogamma=AUTOGAMMA,midpoint=MIDPOINT,debug=true, sceneChgThresh=sceneChgThresh,FilterRadius=5,sc2Th=256) # Nearly same as 0.7. sc2Th=256 switch off new sc overrides [still does 1/2 mean SC chk]
B=AutoLevels(autogamma=AUTOGAMMA,midpoint=MIDPOINT,debug=DEBUG,sceneChgThresh=sceneChgThresh,FilterRadius=7,sc2Th=SC2TH,sc2Perc=SC2PERC) # New Settings
LFT=StackVertical(ORG,A)
RGT=StackVertical(ORG,B)
StackHorizontal(LFT,RGT)


EDIT: Ignore debug info on bottom of frame when DEBUG, is really just debug and of no use to you.

EDIT:
From above source docs

// 11/06/2019 Ver 0.10 + Oops, limited gamma max in adjust to 1.0 instead of 10.0 (so only ever reduced gamma, never increased).

Maybe upwards, or downwards gamma limiting might be a pretty good user arg to add. [it worked pretty good when above bug existed, bug prevented upwards 'overkill']

[StainlessS]

V0.11Beta2, In sig below this post, HOSTED folder.

// 08/07/2019 Ver 0.11Beta1 + Basic scene change detect also now takes into account diffLumaMean((n,n-1))>=(sceneChgThresh*0.5), as well as min and max luma diff.
// + Added sc2Th=8, and sc2Perc=33.33, a check and possible override to sceneChgThresh detected scene change.
// + Changed default filterRadius from 5 to 7.
// 08/07/2019 Ver 0.11Beta2 + Added GamMax, GamMin.


Earlier (today Beta) version removed, this considerably more stable (than ALL prev versions) where autolevels(autogamma=true)
[Allowing Gamma max of 10.0 a bit monsterous, most clips require nothing at all like that, even current default of 1.01 is a lot of correction allowed [together with min/max corrections].





AviSource(".\ALTEST.AVI").Trim(74,11154).crop(0,46,0,-46)
######################
ORG=Last
AUTOGAMMA=True
DEBUG=true
# DEFAULT
SCENECHGTHRESH=20 # SC if diff of min or max Y to prev frame is more than this : NEW additional SC if DiffMeanY(n,n-1) is more than 1/2 this. [If either, recheck for override to non SC via SC2 stuff]
# NON DEFAULT
MIDPOINT=0.45 # Default stays 0.5
# NEW, If Scene change detected via SceneChgThresh, then re-check and override to NOT SC if not at least SC2PERC(33.33%) of luma pixels differ by more than SC2TH(8) when compared with prev frame.
SC2TH=8
SC2PERC=33.33
# NEW NEW
GAMMAX = 1.01 # Default 1.01, : 1.0 <= GamMax <= 10.0 : Limit upwards Gamma corrections
GAMMIN = 1.0/GAMMAX # Default 1.0/GAMMAX : 1.0/10.0 <= GamMim <= 1.0 : Limit Downwards gamma corrections
######################
# Nearly same as 0.7. sc2Th=256 switch off new sc overrides [still does 1/2 mean SC chk] : GammMax Unrestricted (well max 10.0)
A=AutoLevels(autogamma=AUTOGAMMA,midpoint=MIDPOINT,debug=true, sceneChgThresh=sceneChgThresh,FilterRadius=5,sc2Th=256,Gammax=10.0)
# New Settings
B=AutoLevels(autogamma=AUTOGAMMA,midpoint=MIDPOINT,debug=DEBUG,sceneChgThresh=sceneChgThresh,FilterRadius=7,sc2Th=SC2TH,sc2Perc=SC2PERC,gammax=GAMMAX,gammin=GAMMIN)
######################
LFT=StackVertical(ORG,A)
RGT=StackVertical(ORG,B)
StackHorizontal(LFT,RGT)


Still prob with fade to black, but much better with autogamma limited stuff.

"And I commend this budget to the House".
[Come on Wonkey, explain it to me again - EDIT: no that was " I refer the honourable gentleman to the reply I gave some moments ago" thing].

EDIT: For AutoLevels and similar, scenechange detect is really whether or not colors change much rather than real scene cut.

EDIT: Perhaps for AutoGamma(autolevels=false) we need not limit gamma corrections, but for the moment I'm just playing with autoLeves(AutoGamma=true),
exact same code just diff args submitted.

EDIT: The two step Scenchange stuff is because 1st step is pretty fast and almost insignificant to speed, 2nd test a check on whether or not 1st test got it right, a bit slower but not often occurs.

EDIT: Also, default Ignore is 1.0/256, suggest better 1.0/512, ie half a luma level rather than a whole one. [I will not change default for that], but suggest may be better.

EDIT: Maybe I was a bit cautious, perhaps more sane GamMax would be eg 1.05 or 1.1 rather than 1.01, clip I'm testing with has wildly changing levels,
some frames with lots of white, others with almost none.

[StainlessS]

V0.12Beta3, In sig below this post, HOSTED folder. [v2.58, avs/+ x86 & x64]


// 11/07/2019 Ver 0.12Beta_3 + Changes to debug metrics.



AviSource("D:\Parade.AVI")
#AviSource(".\ALTEST.AVI").Trim(74,11154).crop(0,46,0,-46)
#AviSource(".\ALTEST2.avi")
#convertToRGB32
#ConvertToY8
ORG=Last
######################
TITLES = True
DEBUG_A = true
DEBUG_B = true
DEBUG_C = true
# DEFAULT
SCENECHGTHRESH=20 # SC if diff of min or max Y to prev frame is more than this : NEW additional SC if DiffMeanY(n,n-1) is more than 1/2 this. [If either, recheck for override to non SC via SC2 stuff]
#
MIDPOINT = 0.42 # Default stays 0.5
IGNORE_A = 1.0/256 # 1.0/256 # OLD Default
IGNORE_B = 1.0/512 # 1.0/512 # NEW Default
IGNORE_C = 1.0/512 # 1.0/512 # NEW Default
# If Scene change detected via SceneChgThresh, then re-check and override to NOT SC if not at least SC2PERC(33.33%) of luma pixels differ by more than SC2TH(8) when compared with prev frame.
SC2TH=12 # 12 Default
SC2PERC=33.33 # 33.33 Default
# NEW NEW
GAMMAX_A = 10.0 # OLD Default for AutoLevels(AutoGamma=true)
GAMMIN_A = 1.0/GAMMAX_A # OLD Default for AutoLevels(AutoGamma=true)
GAMMAX_B = 1.5 # Default 1.5, : 1.0 <= GamMax <= 10.0 : Limit upwards Gamma corrections
GAMMIN_B = 1.0/GAMMAX_B # Default 1.0/GAMMAX : 1.0/10.0 <= GamMim <= 1.0 : Limit Downwards Gamma corrections
GAMMAX_C = 1.25 #
GAMMIN_C = 1.0/GAMMAX_C #
######################
MINRNG_B=100 # Default 100 : Input Y range less than this will hae corrections reduced.
MINRNG_C=100 # Default 100
######################
# A, Nearly same as 0.7. sc2Th=256 switch off new sc overrides [still does 1/2 mean SC chk], also switch off MinRng : GammMax Unrestricted (well 10.0)
A=AutoLevels(autogamma=true,midpoint=MIDPOINT,sceneChgThresh=sceneChgThresh,ignore=IGNORE_A,debug=DEBUG_A,FilterRadius=5,gammax=GAMMAX_A,gammin=GAMMIN_A,sc2Th=256,MinRng=0)
B=AutoLevels(autogamma=true,midpoint=MIDPOINT,sceneChgThresh=sceneChgThresh,ignore=IGNORE_B,debug=DEBUG_B,FilterRadius=7,gammax=GAMMAX_B,gammin=GAMMIN_B,sc2Th=SC2TH,sc2Perc=SC2PERC,MinRng=MINRNG_B)
C=AutoLevels(autogamma=true,midpoint=MIDPOINT,sceneChgThresh=sceneChgThresh,ignore=IGNORE_C,debug=DEBUG_C,FilterRadius=7,gammax=GAMMAX_C,gammin=GAMMIN_C,sc2Th=SC2TH,sc2Perc=SC2PERC,MinRng=MINRNG_C)
######################
ORG=TITLES?ORG.TSub("ORG",True):ORG
A =TITLES?A.TSub("A: "+String(GAMMAX_A," Gammax=%.3f")+String(IGNORE_A," Ignore=%.3f")+" SC2=OFF MinRNG=OFF"):A
B =TITLES?B.TSub("B: "+String(GAMMAX_B," Gammax=%.3f")+String(IGNORE_B," Ignore=%.3f")+String(MINRNG_B," MinRng=%.3f")):B
C =TITLES?C.TSub("C: "+String(GAMMAX_C," Gammax=%.3f")+String(IGNORE_C," Ignore=%.3f")+String(MINRNG_C," MinRng=%.3f")):C
RGT=StackVertical(A,C)
LFT=StackVertical(ORG,B)
Return StackHorizontal(LFT,RGT)

# Stack Overhead Subtitle Text, with optional FrameNumber shown.
Function TSub(clip c,string Tit,Bool "ShowFrameNo",Int "Col"){
c.BlankClip(height=20,Color=Default(Col,0))
(Default(ShowFrameNo,False))?ScriptClip("""Subtitle(String(current_frame,"%.f] """+Tit+""""))"""):Trim(0,-1).Subtitle(Tit)
Return StackVertical(c).AudioDubEx(c)
}


GamMax, 1.0->10.0. Default 1.5 where AutoLevels(AutoGamma=true) and Default 10.0 Where AutoGamma(). Limits amount of gamma correction, maybe only want to darken clip sor something.
GamMin, 0.1->1.0. Default 1.0/GammMax. Darken Limit for gamma.
Sc2th, Default 12. 0->256. 256=OFF. Secondary scene change detection, a check and possible override to primary detector which is quick but not so good.
Sc2Perc, Default 33.33. Range 0.0 <= Sc2Perc < 100.0. Secong arg to secondary scene change detector.
If Scene change detected via SceneChgThresh, then re-check and override to NOT SC if not at least Sc2PErc(33.33%) of luma pixels differ by more than Sc2Th(12) when compared with prev frame.
MinRng, Default 100. 64->128 and also 0=OFF. If luma range less than this, then amount of autolevel/autogamma is reduced, more so where range approaches 0. [avoid horrible fade to black autolevels]

Debug
226:224,233] LHG+sSF : sc2=38.8%
ymin= 3 ymin_avg = 3.5 { 3.5}
ymax=217 ymax_avg =224.2 {224.2}
mean= 79.7 ymean_avg= 82.2
gamm=1.250 out_mean =105.80(0.41)

Top line.
226:224,233] ... Current frame: Frames sample range, limited by arg FilterRadius and detected Scene Changes.

Flags, 'LHG+sSF', where hi-lited:
L ... Low input to levels adjustment is changed from that of ymin.
H ... High input to levels adjustment is changed from that of ymax.
G ... Gamma to levels adjustment is not 1.0.
+ ... (Actually a single character with '+' appearing above '-'), Gamma levels adjustment is limited by GamMin, GamMax or low range fading.
s ... Scene change detection via primary scene change detector arg sceneChgThresh.
S ... Scene change detection via secondary scene change detector args sc2Th and sc2Perc. [Only hilited if primary detector also fired]
F ... Low Luma Range detected via arg MinRng, levels inputs of ymin, ymax and gamma are adjusted to have lesser effect [F=Fade].

sc2=38.8%
Secondary detector percentage of luma pixel difference greater than arg Sc2th (WRT corresponding pixels of prev frame).

ymin=3, Luma Minimum of current frame ignoring at most arg ignore_low extreme (noise) pixels [as for YPlaneMin(threshold)].
ymax=217, Luma Maximum of current frame ignoring at most arg ignore_high extreme (noise) pixels [as for YPlaneMax(threshold)].
mean=1.250 AverageLuma of current frame.
ymin_avg=3.5, Average of all luma ymin values of the current range of frames (range limited by FilterRadius and scene change detection).
{ 3.5}, in this case same as ymin_avg, but will be hilited & different if levelling strength reduction due to arg MinRng.
ymax_avg=224.2, Average of all luma ymax values of the current range of frames.
{224.2}, in this case same as ymax_avg, but will be hilited & different if levelling strength reduction due to arg MinRng.
ymean_avg=82.2, Average of all AverageLuma values of the current range of frames.
Gamm=12.250, Gamma input to levels adjustment.
out_mean=105.80, Resulting AverageLuma, on current frame after levelling.
(0.41), in this case 105.80/255.0, ie equivalent to the full PC levels range midpoint.
At some future stage, will be implemented as eg (out_mean-out_min)/(out_max-out_min).



EDIT: Now looks like this.
EDIT: The below hi-lited flag +/- symbol indicates that Gamma has been restricted by the GamMax arg [in this case 1.25].
Also, the sc2=38.8% in status line, indicates that number of pixels with a difference greater than Sc2Th(12) is 38.8%, which is greater than arg Sc2Perc(33.33),
but as primary scene change detector is not fired, is not a scene change.(neither s nor S flag is hi-lited) [The scene is panning]
https://i.postimg.cc/9FPtgkMn/ALTEST-01.png (https://postimages.org/)

[StainlessS]

Here a demo of the arg MinRng fade effect on movie intro

The effect reductions shown via orange hi-lited values in metrics of lower two images.
https://i.postimg.cc/DW0tYW4k/ALTEST-04.png (https://postimg.cc/DW0tYW4k)

Nuther one
https://i.postimg.cc/tnFNTKDk/ALTEST-05.png (https://postimg.cc/tnFNTKDk)

And another random pick (just because I liked it)
https://i.postimg.cc/QVvxg3Pd/ALTEST-03.png (https://postimg.cc/QVvxg3Pd)

EDIT: That whole PAL format movie is a bit horrible, very dark, probably to hide the appalling blending and
other problems that it has.

[Micheal813]

I'm using AviSynth Plus. What (if any) is the proper MT Mode to use?

[StainlessS]

I'm kinda curious about that too (dont know, I dont usually use MT).
Maybe someone has done tests.

I'm trying to get back to this plugin, but have other things I must do first.

[StainlessS]

The only time that I've ever used AutoLevels with MT, I've used this in setMTMode.avsi

SetFilterMTMode("RoboCrop", MT_SERIALIZED)
SetFilterMTMode("AutoLevels", MT_SERIALIZED)

And I did Auto cropping first, then AutoLevels, and then whatever else. (worked fine and I got ~ 99% CPU on my Quad Q9950 2.83GHz whotsit, with some MC stuff later in script)

EDIT: RoboCrop samples entire clip (by default 40 samples spread throughout clip) and you want this done first, pretty much before anything else.
RoboCrop Scanning is done in constructor before frame serving even starts, and you dont want to be scanning heavy proceed frame eg MC processed
frames in RoboCrop constructor, real bad idea, so have it as first filter after source filter. During Frame serving, RoboCrop is not much different to crop, ie fairly light-weight.

Any kind of auto leveling, auto color correction, ideally requires borders cropped off beforehand so as not to mess up auto corrections.
AutoLevels (or eg AutoAdjust) would also best be done prior to any heavy duty processing, ideally 2nd filter after auto crop filter.

[Music Fan]

Hi,
I tried you filter today, the result is quite impressive.
I first tried Autolevels(), then Autogamma(), then both together.
Are they supposed to be used together ? If yes, is there a preferable order ?
After reading the doc, I believed that Autolevels().Autogamma() would produce the same result than Autogamma().Autolevels() or Autolevels(autogamma=true) but this is not the case.
Here is an exemple ;
https://nsa40.casimages.com/img/2020/04/13/mini_200413011919242253.jpg (https://www.casimages.com/i/200413011919242253.jpg.html)
From left to right (darkest to clearest) ;
-the original (just avisource, this is Lagarith yuy2 captured from a Hi8 tape with a S-video cable)
-avisource(...).crop(24,2,-20,-14).Autolevels(autogamma=true)
-avisource(...).crop(24,2,-20,-14).Autogamma().Autolevels()
I also tried Autolevels().Autogamma() which is a little bit clearer than Autogamma().Autolevels().
And finally Autogamma(autolevel=true) which is much clearer than Autolevels(autogamma=true).
At first sight, I prefer Autolevels(autogamma=true) (in the middle of the image). Is this the correct way to use it ?

[StainlessS]

You can use both together (IF CLIP REQUIRES IT).
AutoLevels has arg AutoGamma=False by default.
AutoGamma has arg AutoLevel=False by default.

AutoLevel() and AutoGamma() are pretty much the same with slightly different defaults [each of them can do both].



Autolevels(clip c,
int "filterRadius" = 7, [* Original plugin = 5 *]
int "sceneChgThresh" = 20,
string "frameOverrides" = "",
float "gamma" = 1.0, [* Auto if autogamma *]
bool "autogamma" = false, # <<<<<<<<<<<<<<<<<<
float "midpoint" = 0.5,
bool "autolevel" = true, # <<<<<<<<<<<<<<<<<<
int "input_low" = undefined, [* measured by statistics *]
int "input_high" = undefined, [* measured by statistics *]
int "output_low" = (c.IsYUV)?16 :0, [* 16 for yuv, 0 for rgb *]
int "output_high", = (c.IsYUV)?235:255 [* 235 for yuv, 255 for rgb *]
bool "coring" = false,
float "ignore" = 1.0/256.0, # 1.0/512 in currrent beta
float "ignore_low" = ignore,
float "ignore_high" = ignore,
int "border" = 0,
int "border_l" = border,
int "border_r" = border,
int "border_t" = border,
int "border_b" = border,
bool "debug" = false

[* Additional to original args *]

int "sc2th" = 12
Float "sc2Perc" = 33.33 [* Where scene change via sceneChgThresh, and % of pixels different by at least sc2th is less than sc2Perc, then Override to NOT scene change *]
Float "GamMax" = 1.5 [* Max limit on Gamma correct *]
Float "GamMin" = 1.0/GamMax [* Min limit on Gamma correct *]
int "minRng" = 100 [* If dynamic range of luma is less than minRng then corrections are reduced *]
)



It is not my filter, was written by Theodor Anschütz, and then updated by Jim Battle[Frustum]
I've only really ever use AutoLevels(AutoGamma=Whatever,AutoLevel=Whatever).
Most clips would not need gamma adjustment.

On use, read authors intro.

EDIT: Some clips do exhibit flickering, I want to try reduce or eliminate that.

If flickering is a problem, then maybe could try this:- https://forum.doom9.org/showthread.php?p=1757661#post1757661

[Music Fan]

AutoLevels has arg AutoGamma=False by default.
AutoGamma has arg AutoLevel=False by default.
Yes, that's why I thought they were complementary (one excluding the other by default).

Most clips would not need gamma adjustment.
Ok, but my sample looks really different (and better) with the gamma adjustment.

EDIT: Some clips do exhibit flickering, I want to try reduce or eliminate that.

If flickering is a problem, then maybe could try this:- https://forum.doom9.org/showthread.php?p=1757661#post1757661
Ok, it seems ok on my sample but wanted to try anyway AutoContrast (with an avsi extension) and GScript.dll (both in the plugins+ folder) and got this error message ;
There is no function named 'RT_FunctionExist'.
:confused:

edit : I didn't see that RT_Stats was also needed, it works now.

edit 2 : it works but I didn't notice any difference with AutoContrast().

[StainlessS]

I didn't notice any difference
Then it did not need it.
Mainly of use where global [ie clip wide] levels are badly set, eg PC->TV levels on already TV levels clip results in real bad contrast.
Strength=1.0 = full adjustment, default 0.75 = 75% of full adjustment to levels [might be deliberately low contrast dark clip so we dont want to overdo it].
Also, if levels are eg PC range then will (by default) compress to TV levels 16->235, strength 0.75 will also only partly adjust that too.
Long clip, maybe set SAMPLES = Round(clip.Framecount * Required_Scan_Percent / 100.0), will take time where lots of frames scanned, default 40.

Most clips would not need both levels and gamma adjust.

[Music Fan]

Ok thanks, thus it's useful to make verification if there's a doubt about contrast.

[patul]

Hello,

First of all I'm a newbie, currently I'm trying to improve the brightness/contrast/banding/block/noises of an old video of TV series from my childhood for better viewing experience. The video itself is from YouTube (link (https://www.youtube.com/watch?v=MDDW8PufU-k)). I'm trying to use AutoLevels to enhance it, but since the scenes were varying (day, night, indoor, outdoor, shades etc), I cant get that better viewing experience.. Some low light scenes are still too dark, on the other hand, if I force gamma correction to entire video then some daylight scenes will be too bright, hence I'm trying to use AutoLevels here.

Here's my current script:


ORG = Last
IGNORE_C = 1.0/512
C = AutoLevels(ignore=IGNORE_C)
ConvertBits(16)
#
C.AutoAdjust()
C.flash3kyuu_deband(sample_mode=2,dither_algo=3)
C.Deblock(quant=25, aOffset=0, bOffset=0, planes="yuv")
C.neo_f3kdb(y=64, cb=64, cr=64, grainy=0, grainc=0)
C.Spotless(2)

ConvertBits(8)

D = Overlay(C, ORG, mode="difference", pc_range=false)
E = Compare(ORG, C)
S = StackHorizontal(ORG,C)
K = StackHorizontal(E,D)
RTN = StackVertical(S,K)

#return Compare(ORG, C)
Return RTN


Result on dark scene,

https://i.imgur.com/OTZ0TJG.png

What should I do to improve the viewing experience specifically on the brightness/contrast and the noises?

Thanks in advance.

Regards,
Patul

[StainlessS]

Its the bright logo that prevents auto level.

[patul]

Its the bright logo that prevents auto level.

Thank you StainlessS for your reply,

Does it mean that I have to take that logo out? Or I can set some parameter to ignore the bright logo?

I saw the AutoLevels has ignore parameters, while I assume that the parameters are intended to ignore percentage of dark and bright, I tried to play round with those

float "ignore" = 1.0/256.0, # 1.0/512 in currrent beta
float "ignore_low" = ignore,
float "ignore_high" = ignore,


But haven't got any better result. Is my understanding correct?

[StainlessS]

Nope, from original docs

* By default, statistics are gathered over the entire image, and these
statistics drive the autolevel and autogamma behavior. The border
parameters can be used to specify that a band of pixels on one or more sides
of the image should not be included in the statistics. Whether a border is
specified or not, the entire image is always processed. Specifying
"border=N" indicates that the N pixels from the left, right, top, and bottom
edges are to be ignored. "border_l" specifies the zone on the left edge to
ignore; "border_r" does the same for the right edge, "border_t" for the top
edge, and "border_b" for the bottom edge. If both "border" and a specified
border, say "border_t" are specified, the larger value is used.

This feature is useful if the edges of the frame contain defects or
brightness rolloff that aren't representative of the image in whole and
might adversely affect the algorithm.


You can avoid left hand side incl logo or top.
No idea if will work at all.

[patul]

Thank you StainlessS for your reply,

I have tried you suggestion, there's a bit improvement, but it wasnt a drastic one, I guess it will be impossible, considering the quality of the video itself. Thanks again for your suggestion.

[StainlessS]

At some future date I may add detect clip arg, where you could eg crop a part from other area of frame, and overlay over the logo, so
would not disrupt overall statistics too much but avoid the logo altogether.

EDIT: Also, a DC.Blur(0.25) or thereabout applied to detect clip would avoid some flickering in result [due to occasional 'sparkles' in source].

[poisondeathray]

@patul -
In terms of levels, this is mainly compressed shadows in all scenes (night, day, dark, bright)

You can use hdragc or smoothcurve to boost the shadows without blowing out the brights in all scenes. (If you wanted to, you could still manually tweak each scene, but that's an automatic way for an improved starting point compared to the original)

e.g.

hdragc(shift=-1, coef_sat=0.75, max_gain=3)


hdragc works ok in avs+ x64 with mp_pipeline

[patul]

Hi StainlessS, I would welcome such feature, and will try the Blur(). Thanks


You can use hdragc or smoothcurve to boost the shadows without blowing out the brights in all scenes.

hdragc works ok in avs+ x64 with mp_pipeline

Hi poisondeathray, thank you for your reply.

Following up your suggestion on hdragc with mp_pipeline, I can report following result:
-it is bit heavy on CPU
-outdoor scene (bright) is generally okay, but dark scenes were horrible, blocky artifact destroyed the image. Even subsequent deblock, deband and denoising filters couldn't handle it. AutoLevels certainly generates more pleasing result on dark scenes, at least from the settings which I tried, however I haven't been playing much with hdragc.

I tried Smoothcurve as well, but I'm too dumb to understand the number representation of a curve, I got nice contrast tho, but couldn't figure out how to boost the shadow without getting 'foggy' effect.

Bit of topic, on debanding/deblocking/denoising, what is the suggested order of them? Currently it's done in this order debanding/deblocking/denoising.

Thank you for your help.

[StainlessS]

Given how dark it is, I suspect even AutoLevels() would produce blocky appearance when/if without the logo, similar to hdragc().
The blocks are in your original soruce, the encoder/compressor judged differences between blocky areas to be 'invisible' so saved bitrate,
but in using any massive levels adjuster [auto or by hand] the difference between these blocks will be amplified, hence nasty blocky-ness.
Just curious, is the frame shown in a fade to/from black ? [I have no solution, whether is or isn't]

[patul]

Hi StainlessS,

You're correct. I was comparing apples and oranges. For AutoLevels, I did ConvertBits(16) BEFORE further processing, while for hdragc I didn't do the same, so the deband/deblock/denoise filters were working on different bit-depths.

This is the result after doing the same processing for both hdragc and autolevels.

https://i.imgur.com/Tkdykv4.png

The left side is hdragc, notice the blocky-ness, although this is most probably due to different strength of level adjustments on the darks/shades just like your suspicion.

On whether the frame is shown in a fade to/from black, I'm not really sure how to answer that, but I don't think such fade is there, except for few frames at the beginning of the video. Some of the frames on dark scenes were technically difficult to handle, there're people wearing whites while the background is almost black.

@poisondeathray,
I have to correct my earlier statement about hdragc, after same follow up process (as applied for AutoLevels), hdragc produced better result. Thanks again for recommending it.

[Cary Knoop]

In this case I would recommend using an NLE, like DaVinci Resolve, to lighten it up a bit, remove some of the red and overal desaturate it a bit.
But don't expect much the shadows are just bad compression blocks.

[patul]

Hi Cary Knoop, thank you for your reply.

While probably it will yield better result, since my objective is only to improve my viewing experience and I'm not the producer of the video, I wouldn't push myself into NLE world. To make it worse,I just don't have necessary skill to do that :D.

[StainlessS]

If you get rid of that damn logo you stand a much better chance.
https://forum.doom9.org/showthread.php?t=176860

EDIT: Removed some rubbish from below, aint had my coffee yet. [I was thinking that I'de already implemented Detect clip].

[Cary Knoop]

Hi Cary Knoop, thank you for your reply.

While probably it will yield better result, since my objective is only to improve my viewing experience and I'm not the producer of the video, I wouldn't push myself into NLE world. To make it worse,I just don't have necessary skill to do that :D.
I created a "lighten up" LUT you could apply to the video:
https://www.dropbox.com/s/98kmi7sl008fs7o/LightenUp_1.Mbangun%20Desa%20%20Nggayuh%20Kamukten.cube?dl=0

I seldom use LUTs myself but I think you can apply a 3D LUT in Avisynth,

[patul]

If you get rid of that damn logo you stand a much better chance.


For the last picture, I specified border_l already, and it didn't help that much.

@Cary Knoop,

Thank you for your help with the cube, this is the result of it vs the original.

https://i.imgur.com/LhJBS5j.png

using the following script
LoadPlugin("D:\App\AVS\plugins64+\vscube.dll")
LoadPlugin("D:\App\AVS\plugins64+\LSMASHSource.dll")
LoadPlugin("D:\App\AVS\plugins64+\avsresize.dll")
LoadPlugin("D:\App\AVS\plugins64+\AutoLevels_x64.dll")

LSMASHVideoSource("D:\App\NggayuhKamukten.mp4")

ConvertBits(16)
z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020ncl:st2084:2020:l=>rgb:linear:2020:f", dither_type="none")
ORG = Last
CUB = Cube("D:\App\AVS\AVSMeter290\LightenUp_1.Mbangun Desa Nggayuh Kamukten.cube", fullrange = false)
StackHorizontal(ORG, CUB)
z_ConvertFormat(pixel_type="YUV420P16", colorspace_op="rgb:linear:2020:f=>2020ncl:st2084:2020:l", dither_type="none")
ConvertBits(8)

[Cary Knoop]

That does not look right. You need to use Rec709 (or Rec601) colorspace.

This is what i get after applying the LUT:

https://www.dropbox.com/s/blycqoysbko9mre/indonesia-0_1.1.jpg?dl=0
https://www.dropbox.com/s/blycqoysbko9mre/indonesia-0_1.1.jpg?dl=0

[patul]

That does not look right.

Probably I did it wrongly, although I followed up the sample script from avisynth.nl for Cube, since I'm not really familiar with it. I suspected the conversion was affecting it, but after I saw the original frames were okay, then I concluded probably the script worked as intended. Thanks for your help. :thanks:

[Cary Knoop]

Probably I did it wrongly, although I followed up the sample script from avisynth.nl for Cube, since I'm not really familiar with it. I suspected the conversion was affecting it, but after I saw the original frames were okay, then I concluded probably the script worked as intended. Thanks for your help. :thanks:
ST2084 is for HDR10 you need to use a Rec709 base.

[patul]

ST2084 is for HDR10 you need to use a Rec709 base.

Thank you, I just realized how dumb I was haha..

This is the result

https://i.imgur.com/6g2VVZy.png

Notice the "fog"/"haze", just like when we bump up overall gamma.

I think I will just give up on this, at least I have 3 choices to pick from, hdragc, autolevels and your LUT.

Thanks StainlessS, poisondeathray & Cary Knoop

[kedautinh12]

Thank you, I just realized how dumb I was haha..

This is the result

https://i.imgur.com/6g2VVZy.png

Notice the "fog"/"haze", just like when we bump up overall gamma.

I think I will just give up on this, at least I have 3 choices to pick from, hdragc, autolevels and your LUT.

Thanks StainlessS, poisondeathray & Cary Knoop

Can you share your scripts code

[patul]

Can you share your scripts code

Sure. Here it is

LSMASHVideoSource("D:\App\NggayuhKamukten.mp4")

#ConvertToYV12().ConvertBits(16).ConvertToPlanarRGB()
z_ConvertFormat(pixel_type="RGBP16", colorspace_op="709:709:709:f=>rgb:709:709:f", dither_type="random")

ORG = Last
CUB = Cube("D:\App\AVS\AVSMeter290\LightenUp_1.Mbangun Desa Nggayuh Kamukten.cube", fullrange = false)
StackHorizontal(ORG, CUB)
z_ConvertFormat(pixel_type="YUV420P16", colorspace_op="rgb:709:709:f=>709:709:709:l", dither_type="random")


My mistake was on the colorspace_op. It should be specified based on the video colorspace itself, I used MediaInfo to know the colorspace. It needs to be converted to Planar RGB16 since it is the only color space that is supported by Cube. I tried to do the conversion using internal AVS+ functions and it also worked.

ConvertToYV12().ConvertBits(16).ConvertToPlanarRGB()

Please be informed that this script is incomplete, I didn't handle the audio yet, you need to add it by yourself or use source filter of your liking.

[kedautinh12]

Thanks

[age]

Sure. Here it is

LSMASHVideoSource("D:\App\NggayuhKamukten.mp4")

#ConvertToYV12().ConvertBits(16).ConvertToPlanarRGB()
z_ConvertFormat(pixel_type="RGBP16", colorspace_op="709:709:709:f=>rgb:709:709:f", dither_type="random")

ORG = Last
CUB = Cube("D:\App\AVS\AVSMeter290\LightenUp_1.Mbangun Desa Nggayuh Kamukten.cube", fullrange = false)
StackHorizontal(ORG, CUB)
z_ConvertFormat(pixel_type="YUV420P16", colorspace_op="rgb:709:709:f=>709:709:709:l", dither_type="random")


My mistake was on the colorspace_op. It should be specified based on the video colorspace itself, I used MediaInfo to know the colorspace. It needs to be converted to Planar RGB16 since it is the only color space that is supported by Cube. I tried to do the conversion using internal AVS+ functions and it also worked.

ConvertToYV12().ConvertBits(16).ConvertToPlanarRGB()

Please be informed that this script is incomplete, I didn't handle the audio yet, you need to add it by yourself or use source filter of your liking.
Hi!
Shouldn't be like this?

LSMASHVideoSource("D:\App\NggayuhKamukten.mp4")

#ConvertToYV12().ConvertBits(16).ConvertToPlanarRGB()
z_ConvertFormat(pixel_type="RGBP16", colorspace_op="709:709:709:l=>rgb:709:709:f", dither_type="random")

ORG = Last
CUB = Cube("D:\App\AVS\AVSMeter290\LightenUp_1.Mbangun Desa Nggayuh Kamukten.cube", fullrange = true)
StackHorizontal(ORG, CUB)
z_ConvertFormat(pixel_type="YUV420P16", colorspace_op="rgb:709:709:f=>709:709:709:l", dither_type="random")

[bruno321]

Hi StainlessS. Suppose I use AutoLevels(), but I don't want to levels to change throughout the video, actually: I go to one frame, I like the way it looks, and I'd like this adjustment to be done throughout the film. How can I achieve that? I guess if I could "export" the levels it's using in that frame so I can just use Levels() with those parameters, that'd do it? I tried "debug=true" but I don't know what to do with the info on screen.

[StainlessS]

See the debug stuff at bottom of frame, ie
Where Adjust(minin,gamma,maxin,minout,maxout,coring=False)

try Levels(minin,gamma,maxin,minout,maxout,coring=False)

[Josué]

Dear mister Stainless,
I was using Autoadjust for years, but for some reason, LaTo has stopped working on it, and issues are appearing with the latest Avisynth+ versions.
So I am using Autoleves now, which works great.
But I have hard times with the settings.
Is there a way to adjust the autogain settings ?
For example, with autolevels on a dark scene, the dark pixels become grey.

Example :
https://ibb.co/WvPvz89
https://ibb.co/WvPvz89

Is there a way to adjust the dark_limit, as in AutoAdjust ?

dark_limit [default: 1.50]
--------------------------
Amount of allowed change for dark pixels
1.0 = no gain
10.0 = no limit

bright_limit [default: 1.50]
----------------------------
Amount of allowed change for bright pixels
1.0 = no gain
10.0 = no limit

Thanks for your work and your patience

[StainlessS]

Hi Josué,
Which version autoLevels are you using ?
I Suggest
AutoLevels_25&26_x86_x64_dll_v0.12Beta3_20190711.zip
Here:- https://www.mediafire.com/file/8k2vubr57yw1yp3/AutoLevels_25%252626_x86_x64_dll_v0.12Beta3_20190711.zip/file

Here a text that I forgot to put in the zip, posted here:- https://forum.doom9.org/showthread.php?p=1907484#post1907484

Autolevels(clip c,
int "filterRadius" = 7, [* Original plugin = 5 *]
int "sceneChgThresh" = 20,
string "frameOverrides" = "",
float "gamma" = 1.0, [* Auto if autogamma *]
bool "autogamma" = false, # <<<<<<<<<<<<<<<<<<
float "midpoint" = 0.5,
bool "autolevel" = true, # <<<<<<<<<<<<<<<<<<
int "input_low" = undefined, [* measured by statistics *]
int "input_high" = undefined, [* measured by statistics *]
int "output_low" = (c.IsYUV)?16 :0, [* 16 for yuv, 0 for rgb *]
int "output_high", = (c.IsYUV)?235:255 [* 235 for yuv, 255 for rgb *]
bool "coring" = false,
float "ignore" = 1.0/256.0, # 1.0/512 in currrent beta
float "ignore_low" = ignore,
float "ignore_high" = ignore,
int "border" = 0,
int "border_l" = border,
int "border_r" = border,
int "border_t" = border,
int "border_b" = border,
bool "debug" = false

[* Additional to original args *]

int "sc2th" = 12
Float "sc2Perc" = 33.33 [* Where scene change via sceneChgThresh, and % of pixels different by at least sc2th is less than sc2Perc, then Override to NOT scene change *]
Float "GamMax" = 1.5 [* Max limit on Gamma correct *]
Float "GamMin" = 1.0/GamMax [* Min limit on Gamma correct *]
int "minRng" = 100 [* If dynamic range of luma is less than minRng then corrections are reduced *]
)


Also maybe post a short sample clip.

I'm kind of stuck a bit with this filter, I dont really like the way it works at all but I'm stuck with what was implemented before I
touched it. I would prefer to be able to drop it altogether and start again.

[bruno321]

Just as a word of support, I've used it a couple of times (often modifying the output/input high/low), so there would definitely be humble interest on my side in seeing this filter being developed further :)

[takla]

How feasible would it be to apply an average to a whole scene?

Lets say that scene is 10 seconds long. Or is filterRadius=240 what would achieve this? But the thing is, this parameter goes into both directions (previous and next frames). Wouldn't be analyzing the whole scene (effectively filterRadius=∞, but only forwards seeking) and reset when scenechange treshold is triggered be much better?

Ultimately, I'd want it to analyze as many frames as the scene is long, only interrupted by the scene change treshold,and apply an average based on all the scenes frames.