Findtweak histogram matcher

[jmac698]

This script finds the equivalent tweak statement to match two images. It's useful if there's a known transformation which is constant for the whole clips, such as two captures of the same video with different procamp settings.
It currently has a lot of problems and I'm open to comments.

#findtweak 0.1 by jmac
#Find brightness and contrast transformation
#requires masktools.  Doesn't work on real clips due to clipping. Crashes on monocolored clips.
#Two clips must be geometrically aligned. Will be confused by gamma.
#works by partitioning the clips into dim and bright pixels
#By using two measurements between original and tweaked you can find contrast and brightness
blankclip(length=99,width=64,pixel_type="YV12")
a=mt_lutspa(last,mode="absolute",expr="x 16 +")#ramp 16-79, avg=47.5
b=a.tweak(bright=15,cont=2.0,coring=false)#31-157, avg=94
cont=findtweak(a,b,0)
bright=findtweak(a,b,1)
stackhorizontal(b,a.tweak(bright=bright,cont=cont))
subtitle("Cont="+string(cont))
subtitle(y=20," Bright="+string(bright))

function findtweak(clip a, clip b, int parm){
  current_frame=0
  ma=a.averageluma
  mb=b.averageluma
  x1=trueavg(a,"<=",ma)
  x2=trueavg(a,">",ma)
  y1=trueavg(b,"<=",mb)
  y2=trueavg(b,">",mb)
  #in the test clip, looking for true avg of 31.5 (16-47), but half the pixels are 0 (or 1)
  #upper half is avg=63.5 (48-79)
  #tweaked values gives 62,127
  k=(y2-y1)/(x2-x1)
  b=(y1-16)-(x1-16)*k
  parm==0?k:b
}

function trueavg(clip a, string op, float luma){
#Return average of all pixels op luma *while ignoring others in the calculation*
#Do this by finding avg(x op luma ? x:1)-avg(x op luma ? x:0) etc.
  current_frame=0
  m0=mt_lut(a,expr="x "+string(luma)+" "+op+" x 0 ?").averageluma
  m1=mt_lut(a,expr="x "+string(luma)+" "+op+" x 1 ?").averageluma
  m0/(m1-m0)
}

[Gavino]

I think your trueavg function is wrong.
It seems to me it returns (total luma of pixels meeting condition)/(no of pixels not meeting condition).
I think it should be

function trueavg(clip a, string op, float luma){
  current_frame=0
  m0=mt_lut(a,expr="x "+string(luma)+" "+op+" x 0 ?").averageluma
  m1=mt_lut(a,expr="x "+string(luma)+" "+op+" 1 0 ?").averageluma
  m0/m1
}

[jmac698]

Good point. I was finding sum/n/maskedcount/n and if half the pixels are masked I get sum/n*2=trueavg so in testing if gives the right answer (in any case the pixels are partitioned in half). You are just finding unmaskedcount/n which is direct (sum/n/unmaskedcount/n=sum/unmaskedcount=trueavg)
e.g. if you have

x x x x 0 0 0 0

obviously you always have half the trueavg.

[StainlessS]

Moved here from this thread:- http://forum.doom9.org/showthread.ph...56#post1795556

JMac698,

Been playing with your FindTweak(), was driving me nuts trying to figure it out, finally twigged that the initial test conversion was done
with Tweak(...,coring=false) and result was for Tweak(...,coring=true) [which accounts for all of the seemingly wrong numbers in comments,
which refer to input Y values after reduction by 16 due to coring].
Was very perplexing having Tweak(bright=15,cont=2.0,coring=false) and Tweak(bright=31,cont=2.0) coming out identical, until I realized was due the default coring=true [I know it looks obvious there, but comparing result clip variable of first with second some distance away kind of obscured it].
Anyways, thought it might be a good idea to point that out in the thread, save someone else going bananas.
Also would need some modification to work (at all) with RGB as suggested in your above post [EDIT: in above linked thread].
EDIT: Not to mention that RGB does not directly work with Tweak anyway, would need conversion to eg Y8 and would then suffer from
the coring stuff.

EDIT: I also tried with Gavino suggested mod and got identical results to your script.

EDIT: Could YPlaneMedian be in some way of use in your script ?
Implemented as YPlaneMin(Threshold=50.0),
Lowest pixel value whose cumulative population (including population of all lower value pixels) is greater than 50.0%.

EDIT: Your script with some debug stuff left in, perhaps of use if you plan to mod the script.

# findtweak 0.1 by jmac, https://forum.doom9.org/showthread.php?t=161960
# Find brightness and contrast transformation
# requires masktools.  Doesn't work on real clips due to clipping. Crashes on monocolored clips.
# Two clips must be geometrically aligned. Will be confused by gamma.
# works by partitioning the clips into dim and bright pixels
# By using two measurements between original and tweaked you can find contrast and brightness
blankclip(length=99,width=64,pixel_type="YV12")
a=mt_lutspa(last,mode="absolute",expr="x 16 +",chroma="128") #ramp 16-79, avg=47.5
RT_DebugF("A) YMin=%d YMax=%d YAve=%f",a.RT_YPlaneMin(0),a.RT_YPlaneMax(0),a.RT_AverageLuma(0))  # 16,79,47.5
b=a.tweak(bright=15,cont=2.0,coring=false)                   #31-157, avg=94
RT_DebugF("B) YMin=%d YMax=%d YAve=%f",b.RT_YPlaneMin(0),b.RT_YPlaneMax(0),b.RT_AverageLuma(0))  # 47,173,110.0
cont=findtweak(a,b,0)
bright=findtweak(a,b,1)
stackhorizontal(b,a.tweak(bright=bright,cont=cont))
subtitle("Cont="+string(cont))
subtitle(y=20," Bright="+string(bright))

function findtweak(clip a, clip b, int parm){
  current_frame=0
  ma=a.averageluma
  mb=b.averageluma
  x1=trueavg(a,"<=",ma)     
  x2=trueavg(a,">",ma)
  y1=trueavg(b,"<=",mb)
  y2=trueavg(b,">",mb)
  #in the test clip, looking for true avg of 31.5 (16-47), but half the pixels are 0 (or 1)
  #upper half is avg=63.5 (48-79)
  #tweaked values gives 62,127
  k=(y2-y1)/(x2-x1)
  b=(y1-16)-(x1-16)*k
  RT_DebugF("X1=%f X2=%f Y1=%f Y2=%f K=%f B=%f",x1,x2,y1,y2,k,b)
  parm==0?k:b
}

function trueavg(clip a, string op, float luma){
#Return average of all pixels op luma *while ignoring others in the calculation*
#Do this by finding avg(x op luma ? x:1)-avg(x op luma ? x:0) etc.
  current_frame=0
  m0=mt_lut(a,expr="x "+string(luma)+" "+op+" x 0 ?").averageluma
  m1=mt_lut(a,expr="x "+string(luma)+" "+op+" x 1 ?").averageluma
  RT_DebugF("Op='%s' M0=%f M1=%f : m0/(m1-m0)=%f",op,m0,m1, m0/(m1-m0))  m0/(m1-m0)
# Gavino Mod
#  m1=mt_lut(a,expr="x "+string(luma)+" "+op+" 1 0 ?").averageluma
#  RT_DebugF("Op='%s' M0=%f M1=%f : m0/m1=%f",op,m0,m1, m0/m1)  m0/m1
}

And the Debug output

00000003    0.40231544  [3640] RT_DebugF: A) YMin=16 YMax=79 YAve=47.500000 
00000004    0.40702322  [3640] RT_DebugF: B) YMin=47 YMax=173 YAve=110.000000   
00000005    0.41002613  [3640] RT_DebugF: Op='<=' M0=15.750000 M1=16.250000 : m0/(m1-m0)=31.500000  
00000006    0.41226628  [3640] RT_DebugF: Op='>' M0=31.750000 M1=32.250000 : m0/(m1-m0)=63.500000   
00000007    0.41454145  [3640] RT_DebugF: Op='<=' M0=39.000000 M1=39.500000 : m0/(m1-m0)=78.000000  
00000008    0.41686803  [3640] RT_DebugF: Op='>' M0=71.000000 M1=71.500000 : m0/(m1-m0)=142.000000  
00000009    0.41699862  [3640] RT_DebugF: X1=31.500000 X2=63.500000 Y1=78.000000 Y2=142.000000 K=2.000000 B=31.000000   

REPEATED:
00000005    0.41002613  [3640] RT_DebugF: Op='<=' M0=15.750000 M1=16.250000 : m0/(m1-m0)=31.500000  
00000006    0.41226628  [3640] RT_DebugF: Op='>' M0=31.750000 M1=32.250000 : m0/(m1-m0)=63.500000   
00000007    0.41454145  [3640] RT_DebugF: Op='<=' M0=39.000000 M1=39.500000 : m0/(m1-m0)=78.000000  
00000008    0.41686803  [3640] RT_DebugF: Op='>' M0=71.000000 M1=71.500000 : m0/(m1-m0)=142.000000  
00000009    0.41699862  [3640] RT_DebugF: X1=31.500000 X2=63.500000 Y1=78.000000 Y2=142.000000 K=2.000000 B=31.000000

[vcmohan]

My HistograAdjust plugin for avisynth 2.6 or Histogram function of modPlus plugin for newer avisynth+ has a facility to match with a specified frame of any (other ) clip.

[jmac698]

@stainlessS
Thanks for pointing that out. I really need to finish this.
Also I realized I couldn't do this in rgb afterwards, but perhaps it could be done in saturation domain as well.

@vcmohan
I have a specific purpose in mind for this. It's not just for matching, but matching under a particular model, that is the classical analog tint/sat/bright/cont model. A very particular use is matching two recordings with different procamp settings. I looked at the datasheets of some decoder chips and tested this, and found the digital model. Besides matching captures, you can of course find the best constrained match with old video edited sources. I'm assuming the edit equipment of the time used this model. There were some other ones I read about for primitive colour grading at the time. They were based on simple mixing models (multiplies) in certain transformed domains (like y pb pbr, rgb, etc.).