mt_lutxy maximum expression size

*.mp4 guy · 10th July 2011, 07:17

I've been trying to get mt_lutxy to work with some very large expressions, but I am getting odd behavior that doesn't fit what I would expect from a bug in the expression. The expressions being as large as they are, I want to try to rule out any limitations mt_lutxy may have.

for example the following always returns 0 for luminance regardless of how much is added to the output at the end of the expression:

Mt_LutXY(B1_D, B2_D, " X 128 - Y 128 - 3.75 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * > 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * 3.75 / 2 ^ 0.0045813 * 0.0360768 + 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * 3.75 / 2 ^ * 0.2659732 + 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * 3.75 / 2 ^ * 1.2067492 + 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * 3.75 / 2 ^ * 3.0899424 + 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * 3.75 / 2 ^ * 3.5156229 + 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * 3.75 / 2 ^ * 1 + 3.75 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * / 0.00392377 * -0.01647633 + 3.75 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * / * 0.02635537 + 3.75 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * / * -0.02057706 + 3.75 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * / * 0.00916281 + 3.75 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * / * -0.00157565 + 3.75 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * / * 0.00225319 + 3.75 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * / * 0.01328592 + 3.75 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * / * 0.39894228 + 3.75 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * / exp 3.75 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * / sqrt / * ? 3.75 A "+string(thresh)+" * > y 128 - abs x 128 - abs / "+string(thresh)+" * 3.75 / 2 ^ 0.0045813 * 0.0360768 + y 128 - abs x 128 - abs / "+string(thresh)+" * 3.75 / 2 ^ * 0.2659732 + y 128 - abs x 128 - abs / "+string(thresh)+" * 3.75 / 2 ^ * 1.2067492 + y 128 - abs x 128 - abs / "+string(thresh)+" * 3.75 / 2 ^ * 3.0899424 + y 128 - abs x 128 - abs / "+string(thresh)+" * 3.75 / 2 ^ * 3.5156229 + y 128 - abs x 128 - abs / "+string(thresh)+" * 3.75 / 2 ^ * 1 + 3.75 y 128 - abs x 128 - abs / "+string(thresh)+" * / 0.00392377 * -0.01647633 + 3.75 y 128 - abs x 128 - abs / "+string(thresh)+" * / * 0.02635537 + 3.75 y 128 - abs x 128 - abs / "+string(thresh)+" * / * -0.02057706 + 3.75 y 128 - abs x 128 - abs / "+string(thresh)+" * / * 0.00916281 + 3.75 y 128 - abs x 128 - abs / "+string(thresh)+" * / * -0.00157565 + 3.75 y 128 - abs x 128 - abs / "+string(thresh)+" * / * 0.00225319 + 3.75 y 128 - abs x 128 - abs / "+string(thresh)+" * / * 0.01328592 + 3.75 y 128 - abs x 128 - abs / "+string(thresh)+" * / * 0.39894228 + 3.75 y 128 - abs x 128 - abs / "+string(thresh)+" * / exp 3.75 y 128 - abs x 128 - abs / "+string(thresh)+" * / sqrt / * ? / * + 12800 + ", u=1, v=1)

the code tag was omitted for readability.

wOxxOm · 10th July 2011, 09:08

as the string parameter is longer than 2048 characters after substituting thresh & N, it could be an overlooked limit somewhere in masktools/avisynth

btw, such a long sequence is better written this way for the sake of forum readability:

Code:

TH=string(thresh)
N=string(N)
Mt_LutXY(B1_D, B2_D, " 
X 128 - Y 128 - 
3.75 1 "+N+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+TH+" * * > 	
1 "+N+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+TH+" * * 3.75 / 2 ^ 0.0045813 * 0.0360768 +
1 "+N+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+TH+" * * 3.75 / 2 ^ * 0.2659732 + 
1 "+N+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+TH+" * * 3.75 / 2 ^ * 1.2067492 + 
1 "+N+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+TH+" * * 3.75 / 2 ^ * 3.0899424 + 
1 "+N+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+TH+" * * 3.75 / 2 ^ * 3.5156229 + 
1 "+N+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+TH+" * * 3.75 / 2 ^ * 1 + 
3.75 1 "+N+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+TH+" * * / 0.00392377 * -0.01647633 + 
3.75 1 "+N+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+TH+" * * / * 0.02635537 + 
3.75 1 "+N+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+TH+" * * / * -0.02057706 + 
3.75 1 "+N+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+TH+" * * / * 0.00916281 + 
3.75 1 "+N+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+TH+" * * / * -0.00157565 + 
3.75 1 "+N+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+TH+" * * / * 0.00225319 + 
3.75 1 "+N+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+TH+" * * / * 0.01328592 + 
3.75 1 "+N+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+TH+" * * / * 0.39894228 + 
3.75 1 "+N+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+TH+" * * / exp 
3.75 1 "+N+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+TH+" * * / sqrt / * ? 
3.75 A "+TH+" * > 
y 128 - abs x 128 - abs / "+TH+" * 3.75 / 2 ^ 0.0045813 * 0.0360768 + 
y 128 - abs x 128 - abs / "+TH+" * 3.75 / 2 ^ * 0.2659732 + 
y 128 - abs x 128 - abs / "+TH+" * 3.75 / 2 ^ * 1.2067492 + 
y 128 - abs x 128 - abs / "+TH+" * 3.75 / 2 ^ * 3.0899424 + 
y 128 - abs x 128 - abs / "+TH+" * 3.75 / 2 ^ * 3.5156229 + 
y 128 - abs x 128 - abs / "+TH+" * 3.75 / 2 ^ * 1 + 
3.75 y 128 - abs x 128 - abs / "+TH+" * / 0.00392377 * -0.01647633 + 
3.75 y 128 - abs x 128 - abs / "+TH+" * / * 0.02635537 + 
3.75 y 128 - abs x 128 - abs / "+TH+" * / * -0.02057706 + 
3.75 y 128 - abs x 128 - abs / "+TH+" * / * 0.00916281 + 
3.75 y 128 - abs x 128 - abs / "+TH+" * / * -0.00157565 + 
3.75 y 128 - abs x 128 - abs / "+TH+" * / * 0.00225319 + 
3.75 y 128 - abs x 128 - abs / "+TH+" * / * 0.01328592 + 
3.75 y 128 - abs x 128 - abs / "+TH+" * / * 0.39894228 + 
3.75 y 128 - abs x 128 - abs / "+TH+" * / exp 3.75 
y 128 - abs x 128 - abs / "+TH+" * / sqrt / * ? 
/ * + 12800 +", u=1, v=1)

Gavino · 10th July 2011, 09:17

Quote:

Originally Posted by *.mp4 guy

... string(thresh)+" * * / sqrt / * ? 3.75 A "+string(thresh)+" * > ...

Could that stray "A" be the source of your problem?

*.mp4 guy · 10th July 2011, 16:15

Good catch, thanks, I was trying to rule out anything on avisynth's part before trying to seriously debug that thing.

[edit]hmm, that doesn't change the behavior at all, still blank output. Here is a more readable version that still exhibits the same behavior.

Code:

A = string("y 128 - abs x 128 - abs /")
	C = string("1 "+string(N)+" 15 /  1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * *")
        D = string("y 128 - abs x 128 - abs /  "+string(thresh)+" *")
        G = string("1 "+string(N)+" 15 /  1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * 3.75 / 2 ^")
        H = string("3.75 1 "+string(N)+" 15 /  1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * /")
        I = string("y 128 - abs x 128 - abs / "+string(thresh)+" * 3.75 / 2 ^")
        J = string("3.75 y 128 - abs x 128 - abs / "+string(thresh)+" * /")

Mt_LutXY(B1_D, B2_D, " X 128 - Y 128 - 3.75 "+string(C)+" > "+string(G)+" 0.0045813 * 0.0360768 + "+string(G)+" * 0.2659732 + "+string(G)+" * 1.2067492 + "+string(G)+" * 3.0899424 + "+string(G)+" * 3.5156229 + "+string(G)+" * 1 + "+string(H)+" 0.00392377 * -0.01647633 + "+string(H)+" * 0.02635537 + "+string(H)+" * -0.02057706 + "+string(H)+" * 0.00916281 + "+string(H)+" * -0.00157565 + "+string(H)+" * 0.00225319 + "+string(H)+" * 0.01328592 + "+string(H)+" * 0.39894228 + "+string(H)+" exp "+string(H)+" sqrt / * ? 3.75 "+string(D)+" > "+string(I)+" 0.0045813 * 0.0360768 + "+string(I)+" * 0.2659732 + "+string(I)+" * 1.2067492 + "+string(I)+" * 3.0899424 + "+string(I)+" * 3.5156229 + "+string(I)+" * 1 + "+string(J)+" 0.00392377 * -0.01647633 + "+string(J)+" * 0.02635537 + "+string(J)+" * -0.02057706 + "+string(J)+" * 0.00916281 + "+string(J)+" * -0.00157565 + "+string(J)+" * 0.00225319 + "+string(J)+" * 0.01328592 + "+string(J)+" * 0.39894228 + "+string(J)+" exp "+string(J)+" sqrt / * ? / * + 128 + ", u=1, v=1)

Code:

Mt_LutXY(B1_D, B2_D, " X 128 - Y 128 - 
3.75 "+string(C)+" > "+string(G)+" 0.0045813 * 0.0360768 + "+string(G)+" * 0.2659732 + "+string(G)+" * 1.2067492 +
 "+string(G)+" * 3.0899424 + "+string(G)+" * 3.5156229 + "+string(G)+" * 1 + "+string(H)+" 0.00392377 * -0.01647633 +
 "+string(H)+" * 0.02635537 + "+string(H)+" * -0.02057706 + "+string(H)+" * 0.00916281 + "+string(H)+" * -0.00157565 +
 "+string(H)+" * 0.00225319 + "+string(H)+" * 0.01328592 + "+string(H)+" * 0.39894228 + "+string(H)+" exp 
"+string(H)+" sqrt / * ? 3.75 "+string(D)+" > "+string(I)+" 0.0045813 * 0.0360768 + "+string(I)+" * 0.2659732 + 
"+string(I)+" * 1.2067492 + "+string(I)+" * 3.0899424 + "+string(I)+" * 3.5156229 + "+string(I)+" * 1 + 
"+string(J)+" 0.00392377 * -0.01647633 + "+string(J)+" * 0.02635537 + "+string(J)+" * -0.02057706 + "+string(J)+" * 
0.00916281 + "+string(J)+" * -0.00157565 + "+string(J)+" * 0.00225319 + "+string(J)+" * 0.01328592 + "+string(J)+" * 
0.39894228 + "+string(J)+" exp "+string(J)+" sqrt / * ? / * + 128 + ", u=1, v=1)

I'm not sure if abusing strings like that is alright, but it has the same issue.

10th July 2011, 07:17	#1 \| Link
*.mp4 guy Registered User Join Date: Feb 2004 Posts: 1,348	mt_lutxy maximum expression size I've been trying to get mt_lutxy to work with some very large expressions, but I am getting odd behavior that doesn't fit what I would expect from a bug in the expression. The expressions being as large as they are, I want to try to rule out any limitations mt_lutxy may have. for example the following always returns 0 for luminance regardless of how much is added to the output at the end of the expression: Mt_LutXY(B1_D, B2_D, " X 128 - Y 128 - 3.75 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * > 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * 3.75 / 2 ^ 0.0045813 * 0.0360768 + 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * 3.75 / 2 ^ * 0.2659732 + 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * 3.75 / 2 ^ * 1.2067492 + 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * 3.75 / 2 ^ * 3.0899424 + 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * 3.75 / 2 ^ * 3.5156229 + 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * 3.75 / 2 ^ * 1 + 3.75 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * / 0.00392377 * -0.01647633 + 3.75 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * / * 0.02635537 + 3.75 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * / * -0.02057706 + 3.75 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * / * 0.00916281 + 3.75 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * / * -0.00157565 + 3.75 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * / * 0.00225319 + 3.75 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * / * 0.01328592 + 3.75 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * / * 0.39894228 + 3.75 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * / exp 3.75 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * / sqrt / * ? 3.75 A "+string(thresh)+" * > y 128 - abs x 128 - abs / "+string(thresh)+" * 3.75 / 2 ^ 0.0045813 * 0.0360768 + y 128 - abs x 128 - abs / "+string(thresh)+" * 3.75 / 2 ^ * 0.2659732 + y 128 - abs x 128 - abs / "+string(thresh)+" * 3.75 / 2 ^ * 1.2067492 + y 128 - abs x 128 - abs / "+string(thresh)+" * 3.75 / 2 ^ * 3.0899424 + y 128 - abs x 128 - abs / "+string(thresh)+" * 3.75 / 2 ^ * 3.5156229 + y 128 - abs x 128 - abs / "+string(thresh)+" * 3.75 / 2 ^ * 1 + 3.75 y 128 - abs x 128 - abs / "+string(thresh)+" * / 0.00392377 * -0.01647633 + 3.75 y 128 - abs x 128 - abs / "+string(thresh)+" * / * 0.02635537 + 3.75 y 128 - abs x 128 - abs / "+string(thresh)+" * / * -0.02057706 + 3.75 y 128 - abs x 128 - abs / "+string(thresh)+" * / * 0.00916281 + 3.75 y 128 - abs x 128 - abs / "+string(thresh)+" * / * -0.00157565 + 3.75 y 128 - abs x 128 - abs / "+string(thresh)+" * / * 0.00225319 + 3.75 y 128 - abs x 128 - abs / "+string(thresh)+" * / * 0.01328592 + 3.75 y 128 - abs x 128 - abs / "+string(thresh)+" * / * 0.39894228 + 3.75 y 128 - abs x 128 - abs / "+string(thresh)+" * / exp 3.75 y 128 - abs x 128 - abs / "+string(thresh)+" * / sqrt / * ? / * + 12800 + ", u=1, v=1) the code tag was omitted for readability.