weird nnedi3 artifacts (bug?) along with "eval ()"

[feisty2]

I'm working on a modification of a nnedi3 based upscale script, try to extend it to a multi EDI selectable script.
so I modified the original "rpow2" function and things get freaking weird

here's the original "rpow2" part

Function nnedi3_resize16_rpow2(clip input, int "vct", int "hct", int "vfield", int "hfield", bool "Y", bool "U", bool "V",
\ int "nsize", int "nns", int "qual", int "etype", int "pscrn", int "threads", bool "honly")
{
vct = Default(vct, 1 )
hct = Default(hct, 1 )
vfield = Default(vfield, 1 )
hfield = Default(hfield, 1 )
Y = Default(Y, True )
U = Default(U, False )
V = Default(V, False )
honly = Default(honly, False )


input

hct >= 1 ? \
Eval("""
(honly) ? last
\ : Eval(" try { fturnright(chroma=U||V, mt=threads!=1) } catch(error_msg) { TurnRight() } ")
nnedi3(hfield, True, Y, U, V, nsize, nns, qual, etype, pscrn, threads)

hct = hct - 1
honly = hct >= 1
hfield = 0

(honly) ? last
\ : Eval(" try { fturnleft (chroma=U||V, mt=threads!=1) } catch(error_msg) { TurnLeft () } ")
""") : NOP()

vct >= 1 && !honly ? \
Eval("""
nnedi3(vfield, True, Y, U, V, nsize, nns, qual, etype, pscrn, threads)

vct = vct - 1
vfield = 0
""") : NOP()


return Y||U||V ? vct <= 0 && hct <= 0 ? last
\ : last.nnedi3_resize16_rpow2(vct, hct, vfield, hfield, Y, U, V, nsize, nns, qual, etype, pscrn, threads, honly)
\ : input
}


here's my modified version

Function hybrid_edi_resize16_rpow2(clip input, int "vct", int "hct", int "vfield", int "hfield", bool "Y", bool "U", bool "V", string "edimode",
\ float "alpha", float "beta", float "gamma", int "nrad", int "mdis", bool "hp", bool "ucubic", bool "cost3", int "vcheck", float "vthresh0", float "vthresh1", float "vthresh2", clip "sclip", clip "mclip", int "nsize", int "nns", int "qual", int "etype", int "pscrn", int "threads", int "repmode", int "repmodeu", int "repmodev", bool "honly")
{
/* parameters for test
edimode = Default(edimode, "eedi3" )
# Parameters for eedi3
alpha = Default(alpha, 0.2 )
beta = Default(beta, 0.125 )
gamma = Default(gamma, 60.0 )
nrad = Default(nrad, 3 )
mdis = Default(mdis, 40 )
hp = Default(hp, False )
ucubic = Default(ucubic, True )
cost3 = Default(cost3, True )
vcheck = Default(vcheck, 2 )
vthresh0 = Default(vthresh0, 32.0 )
vthresh1 = Default(vthresh1, 64.0 )
vthresh2 = Default(vthresh2, 4.0 )
sclip = Default(sclip, UnDefined() )
mclip = Default(mclip, UnDefined() )
# Parameters for nnedi3
nsize = Default(nsize, 3 )
nns = Default(nns, 4 )
qual = Default(qual, 2 )
etype = Default(etype, 1 )
pscrn = Default(pscrn, 1 )
# Common parameter
threads = Default(threads, 0 )
# Repair Modes
repmode = Default(repmode, 13 )
repmodeu = Default(repmodeu, repmode)
repmodev = Default(repmodev, repmode)
*/




vct = Default(vct, 1 )
hct = Default(hct, 1 )
vfield = Default(vfield, 1 )
hfield = Default(hfield, 1 )
Y = Default(Y, True )
U = Default(U, False )
V = Default(V, False )
honly = Default(honly, False )


input

hct >= 1 ? \
Eval("""
(honly) ? last
\ : Eval(" try { fturnright(chroma=U||V, mt=threads!=1) } catch(error_msg) { TurnRight() } ")
(edimode=="eedi3") ? eedi3_dh(hfield, Y, U, V, alpha, beta, gamma, nrad ,mdis, hp, ucubic, cost3, vcheck, vthresh0, vthresh1, vthresh2, sclip, mclip, threads)
\ : (edimode=="nnedi3") ? nnedi3_dh(hfield, Y, U, V, nsize, nns, qual, etype, pscrn, threads)
\ : (edimode=="eedi3+nnedi3") ? hedi_dh(hfield, Y, U, V, alpha, beta, gamma, nrad, mdis, hp, ucubic, cost3, vcheck, vthresh0, vthresh1, vthresh2, mclip, nsize, nns, qual, etype, pscrn, threads)
\ : (edimode=="eedi3_repaired") ? eedi3_dh_r(hfield, Y, U, V, alpha, beta, gamma, nrad, mdis, hp, ucubic, cost3, vcheck, vthresh0, vthresh1, vthresh2, sclip, mclip, nsize, nns, qual, etype, pscrn, threads, repmode, repmodeu, repmodev)
\ : (edimode=="eedi3+nnedi3_repaired") ? hedi_dh_r(hfield, Y, U, V, alpha, beta, gamma, nrad, mdis, hp, ucubic, cost3, vcheck, vthresh0, vthresh1, vthresh2, mclip, nsize, nns, qual, etype, pscrn, threads, repmode, repmodeu, repmodev)
\ : Assert (1>2,"Invalid edimode !")


hct = hct - 1
honly = hct >= 1
hfield = 0

(honly) ? last
\ : Eval(" try { fturnleft (chroma=U||V, mt=threads!=1) } catch(error_msg) { TurnLeft () } ")
""") : NOP()

vct >= 1 && !honly ? \
Eval("""
(edimode=="eedi3") ? eedi3_dh(vfield, Y, U, V, alpha, beta, gamma, nrad ,mdis, hp, ucubic, cost3, vcheck, vthresh0, vthresh1, vthresh2, sclip, mclip, threads)
\ : (edimode=="nnedi3") ? nnedi3_dh(vfield, Y, U, V, nsize, nns, qual, etype, pscrn, threads)
\ : (edimode=="eedi3+nnedi3") ? hedi_dh(vfield, Y, U, V, alpha, beta, gamma, nrad, mdis, hp, ucubic, cost3, vcheck, vthresh0, vthresh1, vthresh2, mclip, nsize, nns, qual, etype, pscrn, threads)
\ : (edimode=="eedi3_repaired") ? eedi3_dh_r(vfield, Y, U, V, alpha, beta, gamma, nrad, mdis, hp, ucubic, cost3, vcheck, vthresh0, vthresh1, vthresh2, sclip, mclip, nsize, nns, qual, etype, pscrn, threads, repmode, repmodeu, repmodev)
\ : (edimode=="eedi3+nnedi3_repaired") ? hedi_dh_r(vfield, Y, U, V, alpha, beta, gamma, nrad, mdis, hp, ucubic, cost3, vcheck, vthresh0, vthresh1, vthresh2, mclip, nsize, nns, qual, etype, pscrn, threads, repmode, repmodeu, repmodev)
\ : Assert (1>2,"Invalid edimode !")

vct = vct - 1
vfield = 0
""") : NOP()


return Y||U||V ? vct <= 0 && hct <= 0 ? last
\ : last.hybrid_edi_resize16_rpow2(vct, hct, vfield, hfield, Y, U, V, edimode, alpha, beta, gamma, nrad, mdis, hp, ucubic, cost3, vcheck, vthresh0, vthresh1, vthresh2, sclip, mclip, nsize, nns, qual, etype, pscrn, threads, repmode, repmodeu, repmodev, honly)
\ : input
}

Function hedi_dh_r (clip input, int "field", bool "Y", bool "U", bool "V", float "alpha", float "beta", float "gamma", int "nrad", int "mdis", bool "hp", bool "ucubic", bool "cost3", int "vcheck", float "vthresh0", float "vthresh1", float "vthresh2", clip "mclip", int "nsize", int "nns", int "qual", int "etype", int "pscrn", int "threads", int "repmode", int "repmodeu", int "repmodev")
{
nnedi3=input.nnedi3_dh (field=field, Y=Y, U=U, V=V, nsize=nsize, nns=nns, qual=qual, etype=etype, pscrn=pscrn, threads=threads)
dh=input.eedi3 (field=field, dh=true, Y=Y, U=U, V=V, alpha=alpha, beta=beta, gamma=gamma, nrad=nrad, mdis=mdis, hp=hp, ucubic=ucubic, cost3=cost3, vcheck=vcheck, vthresh0=vthresh0, vthresh1=vthresh1, vthresh2=vthresh2, sclip=nnedi3, mclip=mclip, threads=threads).repair (nnedi3, mode=repmode, modeu=repmodeu, modev=repmodev)
return dh
}

Function eedi3_dh_r (clip input, int "field", bool "Y", bool "U", bool "V", float "alpha", float "beta", float "gamma", int "nrad", int "mdis", bool "hp", bool "ucubic", bool "cost3", int "vcheck", float "vthresh0", float "vthresh1", float "vthresh2",clip "sclip", clip "mclip", int "nsize", int "nns", int "qual", int "etype", int "pscrn", int "threads", int "repmode", int "repmodeu", int "repmodev")
{
nnedi3=input.nnedi3_dh (field=field, Y=Y, U=U, V=V, nsize=nsize, nns=nns, qual=qual, etype=etype, pscrn=pscrn, threads=threads)
dh=input.eedi3 (field=field, dh=true, Y=Y, U=U, V=V, alpha=alpha, beta=beta, gamma=gamma, nrad=nrad, mdis=mdis, hp=hp, ucubic=ucubic, cost3=cost3, vcheck=vcheck, vthresh0=vthresh0, vthresh1=vthresh1, vthresh2=vthresh2, sclip=sclip, mclip=mclip, threads=threads).repair (nnedi3, mode=repmode, modeu=repmodeu, modev=repmodev)
return dh
}

Function hedi_dh (clip input, int "field", bool "Y", bool "U", bool "V", float "alpha", float "beta", float "gamma", int "nrad", int "mdis", bool "hp", bool "ucubic", bool "cost3", int "vcheck", float "vthresh0", float "vthresh1", float "vthresh2", clip "mclip", int "nsize", int "nns", int "qual", int "etype", int "pscrn", int "threads")
{
nnedi3=input.nnedi3_dh (field=field, Y=Y, U=U, V=V, nsize=nsize, nns=nns, qual=qual, etype=etype, pscrn=pscrn, threads=threads)
dh=input.eedi3 (field=field, dh=true, Y=Y, U=U, V=V, alpha=alpha, beta=beta, gamma=gamma, nrad=nrad, mdis=mdis, hp=hp, ucubic=ucubic, cost3=cost3, vcheck=vcheck, vthresh0=vthresh0, vthresh1=vthresh1, vthresh2=vthresh2, sclip=nnedi3, mclip=mclip, threads=threads)
return dh
}

Function nnedi3_dh (clip input, int "field", bool "Y", bool "U", bool "V",int "nsize", int "nns", int "qual", int "etype", int "pscrn", int "threads")
{
dh=input.nnedi3 (field=field, dh=true, Y=Y, U=U, V=V, nsize=nsize, nns=nns, qual=qual, etype=etype, pscrn=pscrn, threads=threads)
return dh
}

Function eedi3_dh (clip input, int "field", bool "Y", bool "U", bool "V",float "alpha", float "beta", float "gamma", int "nrad", int "mdis", bool "hp", bool "ucubic", bool "cost3", int "vcheck", float "vthresh0", float "vthresh1", float "vthresh2", clip "sclip", clip "mclip", int "threads")
{
dh=input.eedi3 (field=field, dh=true, Y=Y, U=U, V=V, alpha=alpha, beta=beta, gamma=gamma, nrad=nrad, mdis=mdis, hp=hp, ucubic=ucubic, cost3=cost3, vcheck=vcheck, vthresh0=vthresh0, vthresh1=vthresh1, vthresh2=vthresh2, sclip=sclip, mclip=mclip, threads=threads)
return dh
}



the script can run without any error, but produces weird artifacts if set "edimode" to "nnedi3" or "eedi3+nnedi3"

let's do a 4x upscale and see how the artifacts pops out, I'll crop the image to show the part with artifacts

first, this works perfectly with out any artifacts

nnedi3_dh ()
fturnleft ()
nnedi3_dh ()
fturnright ()
nnedi3_dh ()
fturnleft ()
nnedi3_dh ()
fturnright ()
crop (1000,500,-1000,-1000)

result:http://thumbnails112.imagebam.com/34255/497bdd342543234.jpg (http://www.imagebam.com/image/497bdd342543234)


and with the "rpow2" function

hybrid_edi_resize16_rpow2 (vct=2,hct=2,y=true,u=true,v=true,edimode="nnedi3")
crop (1000,500,-1000,-1000)

result:http://thumbnails112.imagebam.com/34255/83facd342543192.jpg (http://www.imagebam.com/image/83facd342543192)

what the heck is this? I don't know how and why this artifacts appears, with edimode="eedi3", everything seems nice, no weird artifacts, is it a nnedi3 bug or something?
:stupid:

[Asmodian]

Is the red square the artifact? That is the only difference I can see in the two pictures.

I assume not and either I just cannot see the artifact or you posted the wrong "good" picture. This is probably an example of NNEDI3's neural net making an incorrect connection. NNEDI3 is very different from other scalers and can sometimes get "confused" and generate strange artifacts by connecting details that shouldn't be connected.

[feisty2]

Is the red square the artifact?

yeah, look at the eyelashes i squared out, the first image and latter one used nnedi3 with exactly the same parameters, but the first one is clean, and there's weird white artifacts on the latter one, please click the pictures and compare them in original size

I'm trying to say nnedi3 works perfectly fine if use it directly, but makes artifacts when use it in an "eval" part, and eedi3 works fine in both ways

[foxyshadis]

Honestly, I've gone back and forth a couple dozen times, I've diff'd the pictures, and I still can't find the artifacts you're talking about. Maybe a clip would help?

Also, there's a few different versions of NNEDI3. jpsdr's is the most recent (https://github.com/jpsdr/NNEDI3), have you tried that?

[mp3dom]

There are some "vertical stripes" inside the black eyeslash. You see them only at original size.

[feisty2]

@foxyshadis
I wrapped everything together: the source image, the enlarged images, the function script, and scripts that produced the enlarged images
https://www.sendspace.com/file/aqtkmg
all images captured by AvsPmod 2.5.1

EDIT: yes, I already tried jpsdr's version, nnedi3 v 0.9.4.7, the issue remains, I'm currently using avs+ r1576, the scripts that produced both images should be identical, but give different results

[Reel.Deel]

Maybe this would be easier to solve if the problem could be reproduced with the bare minimums (without a convoluted script).

[feisty2]

@Reel.Deel

function test (clip input, int "vfield", int "hfield", bool "Y", bool "U", bool "V",
\ int "nsize", int "nns", int "qual", int "etype", int "pscrn", int "threads", bool "honly")
{
vfield = Default(vfield, 1 )
hfield = Default(hfield, 1 )
nsize = Default(nsize, 3 )
nns = Default(nns, 4 )
qual = Default(qual, 2 )
etype = Default(etype, 1 )
pscrn = Default(pscrn, 1 )
threads = Default(threads, 0 )
honly = Default(honly, False )

hct=1
vct=1

input

Eval("""
(honly) ? last
\ : Eval(" try { fturnright(chroma=U||V, mt=threads!=1) } catch(error_msg) { TurnRight() } ")
nnedi3(hfield, True, Y, U, V, nsize, nns, qual, etype, pscrn, threads)

hct = hct - 1
honly = hct >= 1
hfield = 0

(honly) ? last
\ : Eval(" try { fturnleft (chroma=U||V, mt=threads!=1) } catch(error_msg) { TurnLeft () } ")
""")
}


the lite version of the function that produces artifacts

all this function does is get the input clip and do "turnleft"/"turnright" and "nnedi3" inside an eval () part
and the function makes artifacts

[feisty2]

if strip down the function even more aggressively


function test (clip c)
{
hfield=1
y=true
u=true
v=true
nsize=3
nns=4
qual=2
etype=1
pscrn=1
threads=0
c
eval ("""nnedi3(hfield, True, Y, U, V, nsize, nns, qual, etype, pscrn, threads)""")
}


the artifacts will be gone

[feisty2]

now I found out the reason: if you turnright the image before applying a nnedi3 to it, the artifacts will pop out
the following script makes artifacts

xxxsource ("xxx")
hfield=1
y=true
u=true
v=true
nsize=3
nns=4
qual=2
etype=1
pscrn=1
threads=0
turnright ()
nnedi3(hfield, True, Y, U, V, nsize, nns, qual, etype, pscrn, threads)

[feisty2]

okay, problem solved, i tested all nnedi3 parameters and found out the artifacts is there for too aggressive nsize value, set nsize to 0, everything is perfect now
i guess the artifacts gets worsened if turnright first is because the horizontal direction for this source is not so reliable like vertical :)

[Asmodian]

I thought more neurons would be less likely to have errors? :confused:
I can see the artifacts in the original images after all but I have to get really close to the screen.

They also don't seem like "normal" NNEDI3 artifacts to me, is that true?

[feisty2]

@Asmodian
a stupid typo, I already corrected it

[colours]

I thought more neurons would be less likely to have errors? :confused:

"Maxing out" the nsize/nns/qual parameters for nnedi3 can be counterproductive on sources that do not have a lot of aliasing; the nnedi3 documentation itself mentions this, too. My very unscientific hypothesis is that nnedi3 tries too hard to reconcile the aliased training data with your not-aliased input and ends up doing unexpected things.