mt_lutxy maximum expression size

[*.mp4 guy]

I've been trying to get mt_lutxy to work with some very large expressions, but I am getting odd behavior that doesn't fit what I would expect from a bug in the expression. The expressions being as large as they are, I want to try to rule out any limitations mt_lutxy may have.

for example the following always returns 0 for luminance regardless of how much is added to the output at the end of the expression:

Mt_LutXY(B1_D, B2_D, " X 128 - Y 128 - 3.75 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * > 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * 3.75 / 2 ^ 0.0045813 * 0.0360768 + 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * 3.75 / 2 ^ * 0.2659732 + 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * 3.75 / 2 ^ * 1.2067492 + 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * 3.75 / 2 ^ * 3.0899424 + 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * 3.75 / 2 ^ * 3.5156229 + 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * 3.75 / 2 ^ * 1 + 3.75 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * / 0.00392377 * -0.01647633 + 3.75 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * / * 0.02635537 + 3.75 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * / * -0.02057706 + 3.75 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * / * 0.00916281 + 3.75 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * / * -0.00157565 + 3.75 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * / * 0.00225319 + 3.75 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * / * 0.01328592 + 3.75 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * / * 0.39894228 + 3.75 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * / exp 3.75 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * / sqrt / * ? 3.75 A "+string(thresh)+" * > y 128 - abs x 128 - abs / "+string(thresh)+" * 3.75 / 2 ^ 0.0045813 * 0.0360768 + y 128 - abs x 128 - abs / "+string(thresh)+" * 3.75 / 2 ^ * 0.2659732 + y 128 - abs x 128 - abs / "+string(thresh)+" * 3.75 / 2 ^ * 1.2067492 + y 128 - abs x 128 - abs / "+string(thresh)+" * 3.75 / 2 ^ * 3.0899424 + y 128 - abs x 128 - abs / "+string(thresh)+" * 3.75 / 2 ^ * 3.5156229 + y 128 - abs x 128 - abs / "+string(thresh)+" * 3.75 / 2 ^ * 1 + 3.75 y 128 - abs x 128 - abs / "+string(thresh)+" * / 0.00392377 * -0.01647633 + 3.75 y 128 - abs x 128 - abs / "+string(thresh)+" * / * 0.02635537 + 3.75 y 128 - abs x 128 - abs / "+string(thresh)+" * / * -0.02057706 + 3.75 y 128 - abs x 128 - abs / "+string(thresh)+" * / * 0.00916281 + 3.75 y 128 - abs x 128 - abs / "+string(thresh)+" * / * -0.00157565 + 3.75 y 128 - abs x 128 - abs / "+string(thresh)+" * / * 0.00225319 + 3.75 y 128 - abs x 128 - abs / "+string(thresh)+" * / * 0.01328592 + 3.75 y 128 - abs x 128 - abs / "+string(thresh)+" * / * 0.39894228 + 3.75 y 128 - abs x 128 - abs / "+string(thresh)+" * / exp 3.75 y 128 - abs x 128 - abs / "+string(thresh)+" * / sqrt / * ? / * + 12800 + ", u=1, v=1)

the code tag was omitted for readability.

[wOxxOm]

as the string parameter is longer than 2048 characters after substituting thresh & N, it could be an overlooked limit somewhere in masktools/avisynth

btw, such a long sequence is better written this way for the sake of forum readability:
TH=string(thresh)
N=string(N)
Mt_LutXY(B1_D, B2_D, "
X 128 - Y 128 -
3.75 1 "+N+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+TH+" * * >
1 "+N+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+TH+" * * 3.75 / 2 ^ 0.0045813 * 0.0360768 +
1 "+N+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+TH+" * * 3.75 / 2 ^ * 0.2659732 +
1 "+N+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+TH+" * * 3.75 / 2 ^ * 1.2067492 +
1 "+N+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+TH+" * * 3.75 / 2 ^ * 3.0899424 +
1 "+N+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+TH+" * * 3.75 / 2 ^ * 3.5156229 +
1 "+N+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+TH+" * * 3.75 / 2 ^ * 1 +
3.75 1 "+N+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+TH+" * * / 0.00392377 * -0.01647633 +
3.75 1 "+N+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+TH+" * * / * 0.02635537 +
3.75 1 "+N+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+TH+" * * / * -0.02057706 +
3.75 1 "+N+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+TH+" * * / * 0.00916281 +
3.75 1 "+N+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+TH+" * * / * -0.00157565 +
3.75 1 "+N+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+TH+" * * / * 0.00225319 +
3.75 1 "+N+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+TH+" * * / * 0.01328592 +
3.75 1 "+N+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+TH+" * * / * 0.39894228 +
3.75 1 "+N+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+TH+" * * / exp
3.75 1 "+N+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+TH+" * * / sqrt / * ?
3.75 A "+TH+" * >
y 128 - abs x 128 - abs / "+TH+" * 3.75 / 2 ^ 0.0045813 * 0.0360768 +
y 128 - abs x 128 - abs / "+TH+" * 3.75 / 2 ^ * 0.2659732 +
y 128 - abs x 128 - abs / "+TH+" * 3.75 / 2 ^ * 1.2067492 +
y 128 - abs x 128 - abs / "+TH+" * 3.75 / 2 ^ * 3.0899424 +
y 128 - abs x 128 - abs / "+TH+" * 3.75 / 2 ^ * 3.5156229 +
y 128 - abs x 128 - abs / "+TH+" * 3.75 / 2 ^ * 1 +
3.75 y 128 - abs x 128 - abs / "+TH+" * / 0.00392377 * -0.01647633 +
3.75 y 128 - abs x 128 - abs / "+TH+" * / * 0.02635537 +
3.75 y 128 - abs x 128 - abs / "+TH+" * / * -0.02057706 +
3.75 y 128 - abs x 128 - abs / "+TH+" * / * 0.00916281 +
3.75 y 128 - abs x 128 - abs / "+TH+" * / * -0.00157565 +
3.75 y 128 - abs x 128 - abs / "+TH+" * / * 0.00225319 +
3.75 y 128 - abs x 128 - abs / "+TH+" * / * 0.01328592 +
3.75 y 128 - abs x 128 - abs / "+TH+" * / * 0.39894228 +
3.75 y 128 - abs x 128 - abs / "+TH+" * / exp 3.75
y 128 - abs x 128 - abs / "+TH+" * / sqrt / * ?
/ * + 12800 +", u=1, v=1)

[Gavino]

... string(thresh)+" * * / sqrt / * ? 3.75 A "+string(thresh)+" * > ...
Could that stray "A" be the source of your problem?

[*.mp4 guy]

Good catch, thanks, I was trying to rule out anything on avisynth's part before trying to seriously debug that thing.

[edit]hmm, that doesn't change the behavior at all, still blank output. Here is a more readable version that still exhibits the same behavior.

A = string("y 128 - abs x 128 - abs /")
C = string("1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * *")
D = string("y 128 - abs x 128 - abs / "+string(thresh)+" *")
G = string("1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * 3.75 / 2 ^")
H = string("3.75 1 "+string(N)+" 15 / 1 - 2 ^ - sqrt y 128 - abs x 128 - abs / "+string(thresh)+" * * /")
I = string("y 128 - abs x 128 - abs / "+string(thresh)+" * 3.75 / 2 ^")
J = string("3.75 y 128 - abs x 128 - abs / "+string(thresh)+" * /")

Mt_LutXY(B1_D, B2_D, " X 128 - Y 128 - 3.75 "+string(C)+" > "+string(G)+" 0.0045813 * 0.0360768 + "+string(G)+" * 0.2659732 + "+string(G)+" * 1.2067492 + "+string(G)+" * 3.0899424 + "+string(G)+" * 3.5156229 + "+string(G)+" * 1 + "+string(H)+" 0.00392377 * -0.01647633 + "+string(H)+" * 0.02635537 + "+string(H)+" * -0.02057706 + "+string(H)+" * 0.00916281 + "+string(H)+" * -0.00157565 + "+string(H)+" * 0.00225319 + "+string(H)+" * 0.01328592 + "+string(H)+" * 0.39894228 + "+string(H)+" exp "+string(H)+" sqrt / * ? 3.75 "+string(D)+" > "+string(I)+" 0.0045813 * 0.0360768 + "+string(I)+" * 0.2659732 + "+string(I)+" * 1.2067492 + "+string(I)+" * 3.0899424 + "+string(I)+" * 3.5156229 + "+string(I)+" * 1 + "+string(J)+" 0.00392377 * -0.01647633 + "+string(J)+" * 0.02635537 + "+string(J)+" * -0.02057706 + "+string(J)+" * 0.00916281 + "+string(J)+" * -0.00157565 + "+string(J)+" * 0.00225319 + "+string(J)+" * 0.01328592 + "+string(J)+" * 0.39894228 + "+string(J)+" exp "+string(J)+" sqrt / * ? / * + 128 + ", u=1, v=1)

Mt_LutXY(B1_D, B2_D, " X 128 - Y 128 -
3.75 "+string(C)+" > "+string(G)+" 0.0045813 * 0.0360768 + "+string(G)+" * 0.2659732 + "+string(G)+" * 1.2067492 +
"+string(G)+" * 3.0899424 + "+string(G)+" * 3.5156229 + "+string(G)+" * 1 + "+string(H)+" 0.00392377 * -0.01647633 +
"+string(H)+" * 0.02635537 + "+string(H)+" * -0.02057706 + "+string(H)+" * 0.00916281 + "+string(H)+" * -0.00157565 +
"+string(H)+" * 0.00225319 + "+string(H)+" * 0.01328592 + "+string(H)+" * 0.39894228 + "+string(H)+" exp
"+string(H)+" sqrt / * ? 3.75 "+string(D)+" > "+string(I)+" 0.0045813 * 0.0360768 + "+string(I)+" * 0.2659732 +
"+string(I)+" * 1.2067492 + "+string(I)+" * 3.0899424 + "+string(I)+" * 3.5156229 + "+string(I)+" * 1 +
"+string(J)+" 0.00392377 * -0.01647633 + "+string(J)+" * 0.02635537 + "+string(J)+" * -0.02057706 + "+string(J)+" *
0.00916281 + "+string(J)+" * -0.00157565 + "+string(J)+" * 0.00225319 + "+string(J)+" * 0.01328592 + "+string(J)+" *
0.39894228 + "+string(J)+" exp "+string(J)+" sqrt / * ? / * + 128 + ", u=1, v=1)

I'm not sure if abusing strings like that is alright, but it has the same issue.