nnedi2/nnedi3/eedi3

[tritical]

nnedi3 v0.9.4 (http://bengal.missouri.edu/~kes25c/nnedi3.zip)
nnedi3 v0.9.2 - 64-bit (http://bengal.missouri.edu/~kes25c/nnedi3_64.dll)
eedi3 v0.9.1 (http://bengal.missouri.edu/~kes25c/eedi3.zip)
nnedi2 v1.6 (http://bengal.missouri.edu/~kes25c/nnedi2.zip)

Filter operation is the same as nnedi... how it accomplishes the task is improved - hopefully. Syntax is almost the same - removed one parameter, added two new ones. 4x enlargement examples with spline36, nnedi, and nnedi2 are here: (nnedi/nnedi2 were used on r/g/b planes, nnedi2 used qual=3, spline36 was used on rgb)

http://bengal.missouri.edu/~kes25c/nnedi2_images

Differences are best seen if you quickly flip back and forth between images. Improvement over nnedi isn't huge by any means, but I thought it was enough to finally warrant a new release.

[7ekno]

Wow, pretty impressive difference :)

Thanks, will give it a whirl!

7ek

[Adub]

Holy Cow Batman! I would definitely say that there is a noticeable difference. Atleast with still images anyways.

[Inspector.Gadget]

:eek:

Good work, as always.

[aegisofrime]

It should be relatively easy to retrofit this into TempGaussMC, right?

[thewebchat]

It should be relatively easy to retrofit this into TempGaussMC, right?

The readme says it's syntax compatible so you can just search/replace all nnedi with nnedi2. Or you can make some lame function "nnedi(blah) { nnedi2(blah) }"

[LaTo]

Good work, thanks! ;)

[lansing]

really nice stuff, how is the speed compare to the old one?

[Terka]

just added
# Create spatially interpolated bob-clips
dbob = clp.bob(0,0.5)
edi = (EdiMode=="NNEDI") ? clp.nnedi(field=-2)
\ : (EdiMode=="EEDI2") ? clp.SeparateFields().EEDI2(field=-2, maxd=EEDI2maxd)
\ : (EdiMode=="NNEDI2")? clp.nnedi2(field=-2)
\ : (EdiMode=="Yadif") ? clp.Yadif(mode=1)
\ : dbob
speed of TGMC looks +- the same, maybee a bit faster.
!good work tritical!

[Terka]

for my DV source the TGMC difference between using nnedi and nnedi2 is invisible.

[Leak]

What are the chances of this being made into a GIMP-plugin? ;)

np: T.Raumschmiere - Brenner (I Tank U)

[Gavino]

Thanks for this, Tritical - good work!

One question which has puzzled me about nnedi (and now nnedi2):
With YV12 clips, simply interpolating between the original pixels introduces a chroma shift of 1/4 pixel.
Is this too small to worry about?

[tritical]

really nice stuff, how is the speed compare to the old one?
On my computer (Q6600) with default settings nnedi2 is 25-30% faster than nnedi.

What are the chances of this being made into a GIMP-plugin?
I have no experience with GIMP. I'll see how tough it would be before answering :).

With YV12 clips, simply interpolating between the original pixels introduces a chroma shift of 1/4 pixel.
Is this too small to worry about?

You mean when using dh=true? I hadn't thought about it, but you are right. There will be a 1/4 on the original scale, 1/2 on the new scale, shift upwards with field=1, or downwards with field=0, because vertically the chroma samples are half way between 2 luma lines. So just copying the original chroma to every other line of the enlarged image doesn't maintain the positioning relative to the luma.

If you follow that with a turnleft or turnright and another dh=true it gets more complicated. Since horizontally the chroma samples are aligned with the left pixel in each pair in the original image, everything should be fine if you use field=1 in the subsequent nnedi2(dh=true) call. However, another shift would be introduced if you use field=0. That also applies to yuy2 since it has the same horizontal subsampling as yv12.

From that point if you rotate the image back, so that you have a 2x enlargement, there will be a vertical shift in chroma. If instead you rotate the image again in the same direction and make another dh=true call, I think there is the possibility to cancel part of the vertical shift or introduce even more vertical shift depending on how you set 'field'.

I'll try to modify the avs functions in the help file to minimize chroma shift when enlarging yv12/yuy2.

[Gavino]

You mean when using dh=true? I hadn't thought about it, but you are right. There will be a 1/4 on the original scale, 1/2 on the new scale, shift upwards with field=1, or downwards with field=0, because vertically the chroma samples are half way between 2 luma lines. So just copying the original chroma to every other line of the enlarged image doesn't maintain the positioning relative to the luma.
Yes, that's what I meant.
If you follow that with a turnleft or turnright and another dh=true it gets more complicated. Since horizontally the chroma samples are aligned with the left pixel in each pair in the original image, everything should be fine if you use field=1 in the subsequent nnedi2(dh=true) call. However, another shift would be introduced if you use field=0. That also applies to yuy2 since it has the same horizontal subsampling as yv12.

From that point if you rotate the image back, so that you have a 2x enlargement, there will be a vertical shift in chroma. If instead you rotate the image again in the same direction and make another dh=true call, I think there is the possibility to cancel part of the vertical shift or introduce even more vertical shift depending on how you set 'field'.
That's the conclusion I came to thinking further about it. I think you can minimise the accumulation of shift by alternating between field=0 and field=1 on each further vertical doubling. Then you are always left with just 1/2 final pixel of shift.

Conversely, for the horizontal resizings, you need to use field=0 if doing TurnLeft.Nnedi2().TurnRight() and field=1 for TurnRight().Nnedi2().TurnLeft(). In either of these cases, there will be no shift at all.

EDIT:
Arghh, the vertical doubling case is not so simple once you move beyond 4x. It turns out that for a chain of vertical doublings, the way to minimise the shift is to have field=0 for the first one and field=1 for all the others, or vice versa (start with field=1 and use field=0 for the others). So for 8x, use fields (0, 1, 1) or (1, 0, 0).

[tritical]

That's the conclusion I came to thinking further about it. I think you can minimise the accumulation of shift by alternating between field=0 and field=1 on each further vertical doubling. Then you are always left with just 1/2 final pixel of shift.

Arghh, the vertical doubling case is not so simple once you move beyond 4x. It turns out that for a chain of vertical doublings, the way to minimise the shift is to have field=0 for the first one and field=1 for all the others, or vice versa (start with field=1 and use field=0 for the others). So for 8x, use fields (0, 1, 1) or (1, 0, 0).

Hm, what I came up with in the 4x case using nnedi2(dh=true).turnright() four times in a row is to use the same field value for both vertical shifts, nnedi2 calls 1/3, and to use field=1 for call 2, and field=0 for call 4.

Say you start with field=1, that introduces an upwards shift. turnright() and use field=1 because chroma is aligned with the top field luma... so that doesn't introduce any shift. turnright() again, and the image is now upside down (and flipped horizontally). Since the first call introduced an upwards shift we want to introduce a downwards shift by using field=1 to shift the upside down image upwards. turnright() again, and use field=0 because chroma is aligned with bottom field luma, which introduces no shift. turnright() the final time and the image is enlarged 4x with chroma shifted 1/2 pixel (at 4x scale) upwards relative to luma.

Right or wrong :confused:

[Gavino]

Right or wrong :confused:
Right. Indeed, we're both right. :)

You are essentially using the same approach as me, but doing things in a different order. I was applying the vertical doublings without rotation, so using a different field for the second one. In your case, the second doubling is done on an upside-down image, so you use the same field. And changing the field used for the horizontal doubling is consistent with my description, since the field to use depends on whether you have turned left or right.

For the 8x case using your approach, if you started with field 1, you would continue with
nnedi(0, dh=true).turnright().nnedi(1, dh=true).turnleft

[Leak]

Wouldn't it be possible for your filter to have a dedicated function that'll internally enlarge the image a given number of times in both directions with non-subsampled chroma?

np: Lali Puna - B-Movie (Faking The Books)

[roozhou]

Where can i get the source code?

[*.mp4 guy]

Wouldn't it be possible for your filter to have a dedicated function that'll internally enlarge the image a given number of times in both directions with non-subsampled chroma?

np: Lali Puna - B-Movie (Faking The Books)

nnedi is basid on training a neural net to interpolate a specific spatial position very well, If nnedi2 is based on the same principles, it would be quite dificult to change its interpolation operation. It would also most likely end up two times slower.

[tritical]

Right. Indeed, we're both right.
Yep, I missed that you weren't rotating all the way around. Your way is easier to follow.

Where can i get the source code?
You can't right now.

Wouldn't it be possible for your filter to have a dedicated function that'll internally enlarge the image a given number of times in both directions with non-subsampled chroma?
I don't follow. *.mp4 guy is right that changing the location of interpolation isn't possible. However, your suggestion did give me the idea to automate this scripting by making an internal function to take care of the turnright/turnleft sequencing.

On that note, v1.1 changes (zip file is at the same location).
+ added RGB24 support
+ added nnedi2_rpow2 function
+ faster SSE2 routines

This version should be about 25% faster for those with SSE2 but not SSE3. Syntax for nnedi2_rpow2 is

nnedi2_rpow2(int rfactor, int qual, bool pscrn, string cshift, int fwidth, int fheight, int threads, int opt)

It will use nnedi2 to enlarge by a power of 2 (rfactor) while minimizing relative luma/chroma shifting as much as possible. With a secondary goal of minimizing image center shifting when possible. It can then automatically correct for any image center shifting using the resizer you specify with 'cshift', and if you do specify 'cshift' it also gives you the option to set a new output width/height (fwidth/fheight). It works with yv12, yuy2, rgb24. Examples:

enlarge image by 4x, don't correct for center shift.

nnedi2_rpow2(rfactor=4)

enlarge image by 2x, correct for center shift using spline36resize.

nnedi2_rpow2(rfactor=2,cshift="spline36resize")

enlarge image by 8x, correct for center shift and downsample from 8x to 7x using lanczosresize.

nnedi2_rpow2(rfactor=8,cshift="lanczosresize",fwidth=width*7,fheight=height*7)

[Gavino]

Excellent, tritical!

Well done for getting the new function out so quickly.
:thanks:

[Undead Sega]

this is wicked! :D

but can someone help me combine this fuction for the NNEDI+MCBob_v03c.avs??

[thewebchat]

Undead Sega: Open your mcbob.avs and search/replace all nnedi with nnedi2. Done.

[Undead Sega]

# MCBob v0.3:
# nnedi and nnedibob was made possible by tritical and the fellow Doom9 community who contributed CPU cycles.
# Another approach to motion compensated bobbing, build by Didée.
#
# ( Between-all-chairs version with some quick hacks )
# ( v0.3c: as stated above, but worse ;-) )
# ( v0.3u (unofficial): use new nnEDI interpolater by tritical, modded by Terranigma)
# Features:
#
# - No residual combing, due to STT (Shape Transposition Technology)
# - Works without thresholds (with adaptive thresholds instead of fixed ones)
# - Motion Search between fields of same parity, for maximum flicker/bob reduction in motion areas
# - Motion Masking adaptive to local complexity, for maximum flicker/bob reduction in static areas
# - spatial Interpolation overweights spatio-temporal interpolation
# ( in areas where the information obtained from temporal neighbors in itself was only spatially
# interpolated, use a mix of spatial and spatio-temporal interpolation )
# - error correction for temporal interpolation is fully self adaptive
#
# Prerequisites:
#
# - MVTools, preferably v1.4.13 (or newer)
# - MaskTools v2.0
# - nnEDI 1.3 +
# - RemoveGrain/Repair package
# - ReduceFlicker (if temp-NR for ME is used)
# - MedianBlur by tsp

function MCBob(clip clp, float "EdiPre", int "EdiPost", int "blocksize", int "MEdepth", float "sharpness", int "mtnmode", float "mtnth1", float "mtnth2", float "errth1", float "errth2", float "MEspatNR", float "MEtempNR")
{
EdiPre = default( EdiPre, 1.0 ) # What bob to start with: 0.0 = dumbbob, 1.0 = nnEdiBob, inbetween = mix of both
EdiPost = default( EdiPost, 2 ) # 0 = no nnEDI PP / 1 = Framesized nnEdi PP / Average two Fieldbased nnEdi PP's
bs = default( blocksize, 16 ) # Blocksize for motion search
me = default( MEdepth, 2 ) # Search effort of motion search
sharpness = (EdiPost==2)
\ ? default( sharpness, 0.7 )
\ : default( sharpness, 1.0 ) # use slight sharpening before STT routine

mtnmode = default( mtnmode, 1 ) # 0 = use only same-parity motion check, 1|2 use an additional
# inter-parity check: 1 = on vertical edges / 2 = not on horizontal edges
mtnth1 = default( mtnth1, 0.20 ) # below this %age of local min/max is static
mtnth2 = default( mtnth2, 0.40 ) # above this %age of local min/max is motion
errth1 = default( errth1, 0.40 ) # similar for error detection
errth2 = default( errth2, 0.60 ) # of motion interpolation errors
MEspatNR = default( MEspatNR, 0.00 ) # amount of spatial NR (for motion search only)
MEtempNR = default( MEtempNR, 0.00 ) # amount of temporal NR (for motion search only)

order = (clp.GetParity == True) ? 0 : 1
ORDR = (order==0) ? "TFF" : "BFF"

ox = clp.width()
oy = clp.height()
ERTH1 = string(errth1)
ERTH2 = string(errth2)
MNTH1 = string(mtnth1)
MNTH2 = string(mtnth2)
SSTR = string(sharpness)
idx_1 = 10
idx_2 = (MEspatNR==0.0 && MEtempNR==0.0) ? idx_1 : idx_1+2
idx_3 = idx_2 + 2


# Create basic operations that we will work with
# ==============================================

# Basic Field & Bob clips
# -----------------------
flatbob = clp.Bob(1,0)
normbob = clp.Bob(0.0,0.5)
ofields = clp.SeparateFields()
oweave = clp.DoubleWeave()
nnedibobbed = clp.nnEDIbob()
bobbed = (EdiPre == 0.0) ? normbob
\ : (EdiPre == 1.0) ? nnedibobbed
\ : normbob.merge(nnedibobbed,EdiPre)


# Mask to check if motion compensation has delivered only the neighbor's spatial interpolated part
# ------------------------------------------------------------------------------------------------
black = Blankclip(ofields).mt_lut("0").Trim(1,1).Loop(Framecount(clp))
white = Blankclip(ofields).mt_lut("255").Trim(1,1).Loop(Framecount(clp))
interpol = Interleave(black,white,white,black).AssumeFieldbased().AssumeParity(ORDR).Weave()


# Vertical Edge mask, needed for more safe motion masking
# -------------------------------------------------------
Vedge = bobbed.mt_Edge("1 0 -1 2 0 -2 1 0 -1",0,255,0,255,U=1,V=1)
Vedge2 = Vedge.mt_Inpand(mode="vertical").mt_Inpand(mode="vertical").mt_Expand(mode="vertical").mt_Expand(mode="vertical")
Vedge = mt_Lutxy(Vedge,Vedge2,yexpr="y 2 - 2 * x > x y 2 - 2 * ?") #.mt_Expand()

Hedge = bobbed.mt_Edge("1 2 1 0 0 0 -1 -2 -1",0,255,0,255,U=1,V=1)
Hedge = Hedge.mt_logic(Hedge.temporalsoften(1,255,0,255,2),"max")


# If requested, do flicker reduction before searching motion vectors
# -------------------------------------------------------------------
(MEspatNR==0.0) ? bobbed : bobbed.Merge(bobbed.minblur(2,uv=3),MEspatNR)
(MEtempNR==0.0) ? last : last.Merge(reduceflicker(2),MEtempNR)
srch=last


# Perform Motion Search
# ---------------------
lmbda = 128
pnw = 40
bw_vec2 = srch.SelectEven().MVAnalyse(isb=true, truemotion=false,delta=1,lambda=lmbda,pel=2,searchparam=me,sharp=2,blksize=bs,overlap=1*bs/2,pnew=pnw,idx=idx_1)
fw_vec2 = srch.SelectEven().MVAnalyse(isb=false,truemotion=false,delta=1,lambda=lmbda,pel=2,searchparam=me,sharp=2,blksize=bs,overlap=1*bs/2,pnew=pnw,idx=idx_1)
bw_vec3 = srch.SelectOdd() .MVAnalyse(isb=true, truemotion=false,delta=1,lambda=lmbda,pel=2,searchparam=me,sharp=2,blksize=bs,overlap=1*bs/2,pnew=pnw,idx=idx_1+1)
fw_vec3 = srch.SelectOdd() .MVAnalyse(isb=false,truemotion=false,delta=1,lambda=lmbda,pel=2,searchparam=me,sharp=2,blksize=bs,overlap=1*bs/2,pnew=pnw,idx=idx_1+1)


# Create RAW motion interpolation
# -------------------------------
alt_1 = bobbed.SelectEven().MVFlowInter(bw_vec2,fw_vec2,time=50.0,thSCD1=64*18,thSCD2=227,idx=idx_2)
alt_2 = bobbed.SelectOdd() .MVFlowInter(bw_vec3,fw_vec3,time=50.0,thSCD1=64*18,thSCD2=227,idx=idx_2+1).DuplicateFrame(0)
alt = Interleave(alt_2,alt_1)


# Create motion interpolation of "nothing new" mask
# -------------------------------------------------
interpol_1 = interpol.SelectEven().MVFlowInter(bw_vec2,fw_vec2,time=50.0,thSCD1=64*8,thSCD2=127,idx=idx_3)
interpol_2 = interpol.SelectOdd() .MVFlowInter(bw_vec3,fw_vec3,time=50.0,thSCD1=64*8,thSCD2=127,idx=idx_3+1).DuplicateFrame(0)
interpol_comp= Interleave(interpol_2,interpol_1)
nothing_new = mt_lutxy(interpol,interpol_comp,"x y * 255 / 255 / 1 2 / ^ 160 *")


# Error check of motion interpolation
# ===================================
# Errors that are neutralized by errors in direct vertical neighborhood are not considered, because bob-typical.
# Remaining error is checked against [min,max] of local error to decide if it's valid or not.
#
# Build error mask, neutralize vertical-only errors
# ---------------------------------------------------
altD = mt_Makediff(bobbed,alt,U=3,V=3)
altDmin = altD.mt_Inpand(mode="vertical",U=3,V=3)
altDmin = altDmin.mt_Deflate().mt_Merge(altDmin,Vedge,U=4,V=4)
altDmax = altD.mt_Expand(mode="vertical",U=3,V=3)
altDmax = altDmax.mt_Inflate().mt_Merge(altDmax,Vedge,U=4,V=4)
altDmm = mt_Lutxy(altDmax.mt_Expand(mode="horizontal",U=3,V=3),altDmin.mt_Inpand(mode="horizontal",U=3,V=3),"x y -",U=3,V=3)
altDmm = altDmm.mt_Inflate().mt_Merge(altDmm,Vedge,U=4,V=4)
altD1 = altD .mt_Lutxy(altDmin,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?",U=3,V=3)
altD1 = altD1.mt_Lutxy(altDmax,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?",U=3,V=3)
altD2 = altD.Repair(altD1,1)


# Build correction mask by combining: error mask + "nothing new" mask + a scenechange mask
# ---------------------------------------------------------------------------------------------
corrmask = mt_Lutxy(altD2,altDmm,"x 128 - abs 2 - y 2 + / "+ERTH1+" - "+ERTH2+" "+ERTH1+" - / 255 *",U=3,V=3).mt_Expand(U=3,V=3)
sc = corrmask.BilinearResize(64,64)
sc = mt_LutF(sc,sc,mode="average",expr="x 255 0.6 * > 255 0 ?").PointResize(ox,oy)
corrmask = corrmask.mt_Logic(nothing_new,"max",U=2,V=2)
corrmask = corrmask.mt_Logic(sc,"max",U=2,V=2)


# Create a first bob from motion interpolation, not yet error corrected ...
# -------------------------------------------------------------------------
# ***( temporarily changed ... yet unsure what works best )***

Interleave(bobbed,alt).AssumeParity(ORDR)
SeparateFields().SelectEvery(8,0,3,5,6).Weave()
naked= last
naked2 = last.vinverseD(1.6) # flatbob #

naked_mm = naked.mt_Edge("min/max",0,255,0,255,U=1,V=1)
edibb_mm = nnedibobbed.mt_Edge("min/max",0,255,0,255,U=1,V=1).mt_Expand(mode="vertical")
check2 = mt_LutXY(naked_mm,edibb_mm,"x y / 3 - 5 3 - / 255 *")
corrmask = corrmask.mt_Logic(check2,"max",U=2,V=2)


# ... and build a motion mask from this one.
# ------------------------------------------
# ***( temporarily changed ... tickertapes might suffer. )***

stc = bobbed .removegrain(2)# oweave.removegrain(11)
mm = stc.mt_Edge("min/max",0,255,0,255,U=3,V=3)
# mm = mm .mt_Logic(mm.DuplicateFrame(0),"max",U=3,V=3).mt_Logic(mm.DeleteFrame(0),"max",U=3,V=3)
# max = stc.mt_expand(U=3,V=3)
# max = max.mt_logic(max.Duplicateframe(0),"max",U=3,V=3).mt_logic(max.Duplicateframe(0).Duplicateframe(0),"max",U=3,V=3)
# min = stc.mt_inpand(U=3,V=3)
# min = min.mt_logic(min.Duplicateframe(0),"min",U=3,V=3).mt_logic(min.Duplicateframe(0).Duplicateframe(0),"min",U=3,V=3)
# mm = mt_LutXY(max,min,"x y -",U=3,V=3)
diff2prev1 = mt_LutXY(stc,stc.DuplicateFrame(0),"x y - abs",U=3,V=3)
diff2prev2 = mt_LutXY(stc,stc.DuplicateFrame(0).DuplicateFrame(0),"x y - abs",U=3,V=3)

diff2prev12 = (mtnmode==0) ? diff2prev2 :
\ (mtnmode==1) ? diff2prev2 .mt_Merge(diff2prev1,Vedge,U=2,V=2)
\ : diff2prev1 .mt_Merge(diff2prev2,Hedge,U=2,V=2)

motn = diff2prev12.mt_Logic(diff2prev12.DeleteFrame(0),"max",U=3,V=3).mt_Logic(diff2prev12.DeleteFrame(0).DeleteFrame(0),"max",U=3,V=3)
notstatic = mt_LutXY(motn,mm,"x 1 - y 1 + / "+MNTH1+" - "+MNTH2+" "+MNTH1+" - / 255 *",U=3,V=3).mt_Expand(U=3,V=3).mt_Inpand(U=3,V=3)
# notstatic = notstatic.mt_Logic(notstatic.RemoveGrain(4),"max",U=3,V=3).mt_Expand(U=3,V=3).mt_Inpand(U=3,V=3)


# Now do the error correction of the "naked" MC-bob
# -------------------------------------------------
naked .mt_Merge(nnedibobbed,corrmask,luma=false,U=3,V=3) .VinverseD(2.7-sharpness)
repaired = last


# If requested, sharpen the corrected MC-bob up a little
# ( pre-sharpen for EdiPost = 0 | 1 )
# ------------------------------------------------------
shrpbase = last#.MinBlur(1,1).Merge(RemoveGrain(12,-1),0.23)
shrp = mt_LutXY(shrpbase,shrpbase.RemoveGrain(11,-1),"x x y - abs 16 / 1 1 x y - abs 1 4 / ^ + / ^ 16 * "+SSTR+" * x y - x y - abs 1.3 + / * 1 x y - abs 16 / 1 4 / ^ + / +",U=2,V=2)
# \ .Repair(repaired,1,0)
shrpD = mt_Makediff(shrpbase,shrp)

(sharpness==0.0 || EdiPost==2) ? last : last .mt_Makediff(MergeLuma(shrpD.MinBlur(1,uv=1),shrpD.RemoveGrain(12,-1),0.24),U=2,V=2)


# If requested, do additional PP via nnEDI2
# ----------------------------------------
oweave.mt_merge(last,notstatic,luma=false,U=3,V=3)
AssumeTFF()
edisingle = nnedi(dh=true,field=1).LanczosResize(ox,oy,0,-0.5,ox,2*oy+0.001,taps=3)
edidouble = merge(nnedi(field=1),nnedi(field=0),0.5)
edidoubleD = mt_makediff(last,edidouble,U=3,V=3)
(EdiPost==1) ? edisingle : \
(EdiPost==2) ? edidouble : last

# ( post-sharpen for EdiPost = 2 )
# ------------------------------------------------------
edidoubleshrpD = mt_makediff(edidouble,sharpness==1.0?edidouble.removegrain(20):edidouble.removegrain(20).merge(edidouble,1.0-sharpness),U=3,V=3)
edidoubleshrpD = edidoubleshrpD.repair(edidoubleD,13)
(EdiPost==2) ? edidouble.mt_adddiff(edidoubleshrpD,U=3,V=3) : last


# STT (Shape Transposition Technology) Routine:
# =============================================
# Simply weaving the corrected output with the original fields is bad, because the risk of
# creating unwanted residual combing is too high.
# Instead, the vertical "shape" is taken off the corrected output, and transposed
# onto the fixed "poles" of the original fields' scanlines. Et Voila.
# ----------------------------------------------------------------------------------------
synthbob = last.AssumeParity(ORDR).SeparateFields().SelectEvery(4,0,3).Weave().Bob(1,0)
mapped_new = flatbob.mt_makediff(mt_makediff(synthbob,last,U=3,V=3),U=3,V=3)
newfields = mapped_new.AssumeParity(ORDR).SeparateFields().SelectEvery(4,1,2)
mappedbob = Interleave(ofields,newfields).SelectEvery(4,0,1,3,2).AssumeParity(ORDR).Weave()


# Finally, for static areas use just original fields
# --------------------------------------------------
mappedbob
#bobbed

oweave.mt_merge(last,notstatic.mt_inpand(Y=2,U=2,V=2),luma=false,U=3,V=3)


# Lastly, set correct parity for the bobbed clip
# ----------------------------------------------
(order==0) ? AssumeTFF() : AssumeBFF()

return(last)
}

# ===============================================

############################
# Helper functions below #
############################


## Function nnEDIbob, courtesty of tritical:

# slow, but accurate nnEDI-bob, always dumb ;)

Function nnEDIbob(clip Input)
{
Input.nnedi(field=-2)
}


# Helper to simplify script
function AssumeParity(clip clp, string "order")
{
order == "TFF" ? clp.assumeTFF() : clp.assumeBFF()
return(last)
}

# Kill Combing Function
function VinverseD(clip clp, float "sstr", int "amnt", int "uv")
{
uv = default(uv,3)
sstr = default(sstr,2.7)
amnt = default(amnt,255)
uv2 = (uv==2) ? 1 : uv
STR = string(sstr)
AMN = string(amnt)
vblur = clp.mt_convolution("1","50 99 50",U=uv,V=uv)
vblurD = mt_makediff(clp,vblur,U=uv2,V=uv2)
Vshrp = mt_lutxy(vblur,vblur.mt_convolution("1","1 4 6 4 1",U=uv2,V=uv2),expr="x x y - "+STR+" * +",U=uv2,V=uv2)
VshrpD = mt_makediff(Vshrp,vblur,U=uv2,V=uv2)
VlimD = mt_lutxy(VshrpD,VblurD,expr="x 128 - y 128 - * 0 < x 128 - abs y 128 - abs < x y ? 128 - 0.25 * 128 + x 128 - abs y 128 - abs < x y ? ?",U=uv2,V=uv2)
mt_adddiff(Vblur,VlimD,U=uv,V=uv)
(amnt>254) ? last : (amnt==0) ? clp : mt_lutxy(clp,last,expr="x "+AMN+" + y < x "+AMN+" + x "+AMN+" - y > x "+AMN+" - y ? ?",U=uv,V=uv)
return(last)
}

# Nifty Gauss/Median combination
function MinBlur(clip clp, int r, int "uv")
{
uv = default(uv,3)
uv2 = (uv==2) ? 1 : uv
rg4 = (uv==3) ? 4 : -1
rg11 = (uv==3) ? 11 : -1
rg20 = (uv==3) ? 20 : -1
medf = (uv==3) ? 1 : -200

RG11D = (r==1) ? mt_makediff(clp,clp.removegrain(11,rg11),U=uv2,V=uv2)
\ : (r==2) ? mt_makediff(clp,clp.removegrain(11,rg11).removegrain(20,rg20),U=uv2,V=uv2)
\ : mt_makediff(clp,clp.removegrain(11,rg11).removegrain(20,rg20).removegrain(20,rg20),U=uv2,V=uv2)
RG4D = (r==1) ? mt_makediff(clp,clp.removegrain(4,rg4),U=uv2,V=uv2)
\ : (r==2) ? mt_makediff(clp,clp.medianblur(2,2*medf,2*medf),U=uv2,V=uv2)
\ : mt_makediff(clp,clp.medianblur(3,3*medf,3*medf),U=uv2,V=uv2)
DD = mt_lutxy(RG11D,RG4D,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?",U=uv2,V=uv2)
clp.mt_makediff(DD,U=uv,V=uv)
return(last)
}

every one of them???

[Gavino]

every one of them???
Yes.
An alternative is to do what thewebchat suggested at post #6, create a wrapper function called nnedi that just calls nnedi2.

In the longer term, it might be useful to use the new function nnedi2_rpow2 where appropriate instead of nnedi2 directly. But that requires more thought and care.

[Chainmax]

Thanks for a new release of such a great plugin, tritical :)

From examining the comparison pics, nnedi2 v2 looks a wee bit softer and with far less artifacts (if any) than nnedi v1.3. The white fences and house walls in the second lighthouse comparison are a really good indicator of the lack of artifacting, it's pretty amazing.

I especially like that now nnediresize2x.avs is no longer needed and we can simply use nnedi2_rpow2(qual=3,rfactor=2,cshift="spline36resize") instead. By the way, how does the resizer choice affect the output of the cshift parameter?


One thing, though: there were some cases (I think they were with animated content, not sure though) where EEDI2 would yield better results than NNEDI v1.3. Would it be possible for you to incluse EEDI2 in that comparison?

[tritical]

By the way, how does the resizer choice affect the output of the cshift parameter?
Exactly the same way it affects any other resizing operation. If you specify cshift, this line gets added to the end of the constructed filter chain with the enlarged result from nnedi2 as the input:

xxxresize(fwidth,fheight,hshift,vshift,rfactor*owidth,rfactor*oheight)

Where 'xxxresize' is the resizer you specified with 'cshift', and owidth/oheight are the original input image width/height. fwidth/fheight are set equal to rfactor*owidth and rfactor*oheight if you don't specify them. hshift/vshift are the shifts to cancel the center image shift introduced by nnedi2. So if you don't set fwidth/fheight there wont be much, if any, difference between resizers because the scaling factors are so close to 1.

Also, I just realized that how I coded it wont work for bicubicresize :eek: because the syntax is

bicubicresize(clip clip, int target_width, int target_height, float b, float c, float src_left, float src_top, float src_width, float src_height)

It also has the limitation that you can't pass extra arguments to the resizer ala gauss/blackman/lanczos/bicubic. I'll fix these issues in the next version.

One thing, though: there were some cases (I think they were with animated content, not sure though) where EEDI2 would yield better results than NNEDI v1.3. Would it be possible for you to incluse EEDI2 in that comparison?
I'll add eedi2 images when I get a chance. However, I think its unlikely that eedi2 could beat nnedi/nnedi2 for image enlargement. Sometimes it is better for deinterlacing interpolation because it is more aggressive at connecting lines - especially thin high contrast lines. However, that comes at the price of lots of artifacts. Whereas, nnedi/nnedi2 are trained to minimize mse, which doesn't always equate to connecting every line or smoothing every edge depending on how other similar neighborhoods need to be handled. eedi2 is also limited from the standpoint that it forms the output value as an average of one pixel from the line above and one pixel from the line below, both at equal distances from the center. nnedi2 formulates the output as a linear function of all pixels in the local neighborhood (2 lines above/below, 5 pixels left, 6 pixels right, so 48 pixels total). nnedi formulated the output as a non-linear function of a local neighborhood involving 100 pixels... it had significantly more problems with overfitting the training data. nnedi also had spatial continuity problems due to the use of hard k-means clustering, with each cluster having a separate small network. nnedi2 simply uses one large network for all patterns.

[th3obr0]

What to do if it says height is not mod2? I used to get this error with the previous version. How to solve that?

Nice plugin, sir. Would come in handy ;).

[mikenadia]

nnedi2_rpow2(rfactor=8,cshift="lanczosresize",fwidth=width*7,fheight=height*7)

Nice plug-in. I was wondering if they will be a difference between the above line and applying three times ( after or before each iteration of NNEDI2 ) lanczos resize with the resize factor of (7/8)^(1/3).

Edit: Initial Size=Size0. If downsizing after enlarging.
Enlarge by 2. Resize to Size1=int(Size0*2*(7/8)^(1/3)).
Enlarge by 2. Resize to Size2=int(Size0*4*(7/8)^(2/3))
Enlarge by 2. Resize to Size0*7.

[Gavino]

I was wondering if they will be a difference between the above line and applying three times ( after or before each iteration of NNEDI2 ) lanczos resize with the resize factor of (7/8)^(1/3).
Well, you couldn't do that anyway, because you can only resize to integer sizes and 7^(1/3) is not a rational number. ;)

[Archimedes]

I have problems with the cpu optimizations.

LoadPlugin("plugins\NNEDI2\nnedi2.dll")

ImageSource("clown__original.png", end=0, use_DevIL=False, pixel_type="RGB24")
nnedi2_rpow2(rfactor=2)

Opening with VirtualDub i received the following error code:

VirtualDub crash report -- build 31536 (release)
--------------------------------------

Disassembly:
0227a380: 0000 add [eax], al
0227a382: 0f59af50010000 mulps xmm5, [edi+150]
0227a389: 0f59b710020000 mulps xmm6, [edi+210]
0227a390: 0f59bfd0020000 mulps xmm7, [edi+2d0]
0227a397: 0f58c4 addps xmm0, xmm4
0227a39a: 0f58cd addps xmm1, xmm5
0227a39d: 0f58d6 addps xmm2, xmm6
0227a3a0: 0f58df addps xmm3, xmm7
0227a3a3: 0f28a1a0000000 movaps xmm4, [ecx+a0]
0227a3aa: 0f28ec movaps xmm5, xmm4
0227a3ad: 0f28f4 movaps xmm6, xmm4
0227a3b0: 0f28fc movaps xmm7, xmm4
0227a3b3: 0f59a7a0000000 mulps xmm4, [edi+a0]
0227a3ba: 0f59af60010000 mulps xmm5, [edi+160]
0227a3c1: 0f59b720020000 mulps xmm6, [edi+220]
0227a3c8: 0f59bfe0020000 mulps xmm7, [edi+2e0]
0227a3cf: 0f58c4 addps xmm0, xmm4
0227a3d2: 0f58cd addps xmm1, xmm5
0227a3d5: 0f58d6 addps xmm2, xmm6
0227a3d8: 0f58df addps xmm3, xmm7
0227a3db: 0f28a1b0000000 movaps xmm4, [ecx+b0]
0227a3e2: 0f28ec movaps xmm5, xmm4
0227a3e5: 0f28f4 movaps xmm6, xmm4
0227a3e8: 0f28fc movaps xmm7, xmm4
0227a3eb: 0f59a7b0000000 mulps xmm4, [edi+b0]
0227a3f2: 0f59af70010000 mulps xmm5, [edi+170]
0227a3f9: 0f59b730020000 mulps xmm6, [edi+230]
0227a400: 0f59bff0020000 mulps xmm7, [edi+2f0]
0227a407: 0f58c4 addps xmm0, xmm4
0227a40a: 0f58cd addps xmm1, xmm5
0227a40d: 0f58d6 addps xmm2, xmm6
0227a410: 0f58df addps xmm3, xmm7
0227a413: f20f db 0fh <-- FAULT
0227a415: 7cc1 jl 0227a3d8
0227a417: f20f db 0fh
0227a419: 7cd3 jl 0227a3ee
0227a41b: f20f db 0fh
0227a41d: 7cc2 jl 0227a3e1
0227a41f: 0f2900 movaps [eax], xmm0
0227a422: 81c700030000 add edi, 00000300
0227a428: 83c010 add eax, 10h
0227a42b: 83ea04 sub edx, 04h
0227a42e: 0f857dfdffff jnz 0227a1b1
0227a434: 8b442410 mov eax, [esp+10h]
0227a438: 8b542414 mov edx, [esp+14h]
0227a43c: 33c9 xor ecx, ecx
0227a43e: 0f280488 movaps xmm0, [eax+ecx*4]
0227a442: 0f58048f addps xmm0, [edi+ecx*4]
0227a446: 0f290488 movaps [eax+ecx*4], xmm0
0227a44a: 83c104 add ecx, 04h
0227a44d: 83ea04 sub edx, 04h
0227a450: 75ec jnz 0227a43e
0227a452: 5f pop edi
0227a453: c3 ret
0227a454: cc int 3
0227a455: cc int 3
0227a456: cc int 3
0227a457: cc int 3
0227a458: cc int 3
0227a459: cc int 3
0227a45a: cc int 3
0227a45b: cc int 3
0227a45c: cc int 3
0227a45d: cc int 3
0227a45e: cc int 3
0227a45f: cc int 3
0227a460: 57 push edi
0227a461: 8b4c2408 mov ecx, [esp+08h]
0227a465: 8b7c240c mov edi, [esp+0ch]
0227a469: 8b442410 mov eax, [esp+10h]
0227a46d: 8b542414 mov edx, [esp+14h]
0227a471: 0f2801 movaps xmm0, [ecx]
0227a474: 0f28c8 movaps xmm1, xmm0
0227a477: 0f28d0 movaps xmm2, xmm0
0227a47a: 0f28d8 movaps xmm3, xmm0
0227a47d: 0f5907 mulps xmm0, [edi]

Built on Aegis on Sat Mar 28 15:17:50 2009 using compiler version 1400

Windows 5.1 (Windows XP x86 build 2600) [Service Pack 3]

EAX = 02389a50
EBX = 00000000
ECX = 014d2350
EDX = 00000004
EBP = 024dfef8
ESI = 02389a50
EDI = 014d3640
ESP = 024dfecc
EIP = 0227a413
EFLAGS = 00010206
FPUCW = ffff027f
FPUTW = ffffffff

Crash reason: Illegal instruction

Crash context:
An SSE2 (Pentium 4/Athlon 64) instruction not supported by the CPU was executed in module 'nnedi2'.

Pointer dumps:

EAX 02389a50: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
ECX 014d2350: bd582d8e bbbb6667 3b0b3536 3c91e728 3b0b3536 bd582d8e 3b0b3536 3c91e728
ESI 02389a50: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
EDI 014d3640: bf489640 3de50f20 bc95bba8 3ee5be34 bf1d7202 beb635a4 bed43c6b bc04f6ba
ESP 024dfec8: 00244c20 014d3640 0227b768 014d2350 014d3640 02389a50 00000004 022e03dc
024dfee8: 014d2278 00000000 0227c9d3 022e0217 000000e0 0227c9ff 02389a50 7c802530
024dff08: 024dff74 7c80a0db 014d2350 02335664 022e03d0 02389a50 014d3640 00000005
024dff28: fffffe40 00000204 00000180 000000cc 42957556 00000000 00055294 000000c0
EBP 024dfef8: 000000e0 0227c9ff 02389a50 7c802530 024dff74 7c80a0db 014d2350 02335664
024dff18: 022e03d0 02389a50 014d3640 00000005 fffffe40 00000204 00000180 000000cc
024dff38: 42957556 00000000 00055294 000000c0 00000380 000001c0 014d9c60 000001c0
024dff58: 000001c0 0227cb52 014d0000 7c920222 014db4c0 00000000 7c920222 024dffac

Thread call stack:
0227a413: nnedi2!0000a413
0227b768: nnedi2!0000b768
0227c9d3: nnedi2!0000c9d3
0227c9ff: nnedi2!0000c9ff
0227cb52: nnedi2!0000cb52
7c920222: ntdll!RtlAllocateHeap [7c910000+100c4+15e]
7c920222: ntdll!RtlAllocateHeap [7c910000+100c4+15e]
02271555: nnedi2!00001555
7c920222: ntdll!RtlAllocateHeap [7c910000+100c4+15e]
022715fa: nnedi2!000015fa
7c920222: ntdll!RtlAllocateHeap [7c910000+100c4+15e]
7c80b729: kernel32!GetModuleFileNameA [7c800000+b56f+1ba]
7c920222: ntdll!RtlAllocateHeap [7c910000+100c4+15e]

-- End of report
Changing the opt parameter to 2 (use sse routines) helps.

[tritical]

I have problems with the cpu optimizations.
Thanks for the report. You must have been the only person with a cpu without sse3 to try it because the automatic optimization detection wasn't correct. For opt=0 it always used sse3 routines. For right now you'll have to set opt manually for processors without sse3. Fixed in next version.

What to do if it says height is not mod2? I used to get this error with the previous version. How to solve that?
Are you saying you get that error when height is actually mod 2? Or just that you get that error on images that don't have mod 2 height? In the second case, just crop a line or add a line and remove it later. I'll remove that restriction for dh=true operation in the next version.

I was wondering if they will be a difference between the above line and applying three times ( after or before each iteration of NNEDI2 ) lanczos resize with the resize factor of (7/8)^(1/3).
Maybe, you are free to try :p. I think it's unlikely to make any useful difference.

[Gavino]

nnedi2_rpow2(int rfactor, int qual, bool pscrn, string cshift, int fwidth, int fheight, int threads, int opt)

It will use nnedi2 to enlarge by a power of 2 (rfactor) while minimizing relative luma/chroma shifting as much as possible. With a secondary goal of minimizing image center shifting when possible. It can then automatically correct for any image center shifting using the resizer you specify with 'cshift', and if you do specify 'cshift' it also gives you the option to set a new output width/height (fwidth/fheight).
Do you think it's worth adding an option to correct the remaining chroma shift as well (using MergeChroma with a specified resizer)?

[mikeytown2]

Also, I just realized that how I coded it wont work for bicubicresize :eek: because the syntax is
bicubicresize(clip clip, int target_width, int target_height, float b, float c, float src_left, float src_top, float src_width, float src_height)


To make a successful xxxresize, simply name them src_left, src_top, src_width, src_height. That's what I call for in KBE (http://forum.doom9.org/showthread.php?t=135776). You can see the code in ZoomBoxIt().

[Archimedes]

Here another comparison.

Source:
http://img199.imageshack.us/img199/9968/1070730original.th.jpg (http://img199.imageshack.us/i/1070730original.jpg/)

Spline36Resize:
http://img191.imageshack.us/img191/6266/1070730original1200x089.th.png (http://img191.imageshack.us/i/1070730original1200x089.png/)

EEDI2:
http://img26.imageshack.us/img26/6266/1070730original1200x089.th.png (http://img26.imageshack.us/i/1070730original1200x089.png/)

NNEDI:
http://img44.imageshack.us/img44/6266/1070730original1200x089.th.png (http://img44.imageshack.us/i/1070730original1200x089.png/)

NNEDI2:
http://img189.imageshack.us/img189/6266/1070730original1200x089.th.png (http://img189.imageshack.us/i/1070730original1200x089.png/)

NNEDI and NNEDI2 after strong sharpening and decent denoising (using identical parameters in both cases).

NNEDI + SuperSlowSharpen + dfttest:
http://img199.imageshack.us/img199/6266/1070730original1200x089.th.png (http://img199.imageshack.us/i/1070730original1200x089.png/)

NNEDI2 + SuperSlowSharpen + dfttest:
http://img199.imageshack.us/img199/5945/1070730original1200x089h.th.png (http://img199.imageshack.us/i/1070730original1200x089h.png/)

[tritical]

Do you think it's worth adding an option to correct the remaining chroma shift as well (using MergeChroma with a specified resizer)?
I did some tests, and on images with bright colors it is worth it. So I decided to make that correction automatic (always invoke spline36resize at the end of the chain to correct vertical chroma shift when enlarging yv12).

To make a successful xxxresize, simply name them src_left, src_top, src_width, src_height.
That's what I did. Before I was passing without names because I thought they all took the same first 6 parameters. Didn't notice bicubicresize had b/c in the middle instead of at the end.

New version should be ready soon.

[Blue_MiSfit]

:D :D :D

Nice work, as always, tritical!!!

Very very impressive, a nice step up from NNEDI.

As soon as things stabilize I will be using this on everything!

~MiSfit

[tritical]

I updated the zip archive to v1.2, changes:

+ remove mod2 height restriction when dh=true
+ more assembly optimizations/tweaks
- fixed automatic cpu detection
- fixed issues with nnedi2_rpow2
- always correct yv12 vertical chroma shift
- use arg names when invoking resizers
- allow users to pass optional resizer arguments (ep0/ep1 parameters)

Will try to update the example images later. I'd like to get nnedi2/nnedi/eedi2/spline36 enlargements of each, with all properly centered. If anyone wants to have another resizer included please say so.

[shoopdabloop]

Lanczos4 maybe?

[Archimedes]

tritical, thank you for updating, but the problem with cpu detection still exists.

nnedi2_rpow2(rfactor=2) results in:

VirtualDub crash report -- build 31536 (release)
--------------------------------------

Disassembly:
01e7b640: 8d348e lea esi, [esi+ecx*4]
01e7b643: 660f6f2e movdqa xmm5, [esi]
01e7b647: 660f6f5e10 movdqa xmm3, [esi+10h]
01e7b64c: 660f db 0fh
01e7b64e: 3a0f cmp cl, [edi]
01e7b650: dd05660f6f14 fld qword ptr [146f0f66]
01e7b656: 4e dec esi
01e7b657: 660f6f644e10 movdqa xmm4, [esi+ecx*2+10h]
01e7b65d: 660f db 0fh
01e7b65f: 3a0f cmp cl, [edi]
01e7b661: e205 loop 01e7b668
01e7b663: e91d030000 jmp 01e7b985
01e7b668: eb06 jmp 01e7b670
01e7b66a: 8d9b00000000 lea ebx, [ebx+00]
01e7b670: 660f6f1e movdqa xmm3, [esi]
01e7b674: 660f6f4610 movdqa xmm0, [esi+10h]
01e7b679: 660f db 0fh
01e7b67b: 3a0f cmp cl, [edi]
01e7b67d: c3 ret
01e7b67e: 06 push es
01e7b67f: 660f6f144e movdqa xmm2, [esi+ecx*2]
01e7b684: 660f6f4c4e10 movdqa xmm1, [esi+ecx*2+10h]
01e7b68a: 660f db 0fh
01e7b68c: 3a0f cmp cl, [edi]
01e7b68e: ca068d retf 8d06
01e7b691: 348e xor al, 8eh
01e7b693: 660f6f2e movdqa xmm5, [esi]
01e7b697: 660f6f5e10 movdqa xmm3, [esi+10h]
01e7b69c: 660f db 0fh
01e7b69e: 3a0f cmp cl, [edi]
01e7b6a0: dd06 fld qword ptr [esi]
01e7b6a2: 660f6f144e movdqa xmm2, [esi+ecx*2]
01e7b6a7: 660f6f644e10 movdqa xmm4, [esi+ecx*2+10h]
01e7b6ad: 660f db 0fh
01e7b6af: 3a0f cmp cl, [edi]
01e7b6b1: e206 loop 01e7b6b9
01e7b6b3: e9cd020000 jmp 01e7b985
01e7b6b8: eb06 jmp 01e7b6c0
01e7b6ba: 8d9b00000000 lea ebx, [ebx+00]
01e7b6c0: 660f6f1e movdqa xmm3, [esi]
01e7b6c4: 660f6f4610 movdqa xmm0, [esi+10h]
01e7b6c9: 660f db 0fh <-- FAULT
01e7b6cb: 3a0f cmp cl, [edi]
01e7b6cd: c3 ret
01e7b6ce: 07 pop es
01e7b6cf: 660f6f144e movdqa xmm2, [esi+ecx*2]
01e7b6d4: 660f6f4c4e10 movdqa xmm1, [esi+ecx*2+10h]
01e7b6da: 660f db 0fh
01e7b6dc: 3a0f cmp cl, [edi]
01e7b6de: ca078d retf 8d07
01e7b6e1: 348e xor al, 8eh
01e7b6e3: 660f6f2e movdqa xmm5, [esi]
01e7b6e7: 660f6f5e10 movdqa xmm3, [esi+10h]
01e7b6ec: 660f db 0fh
01e7b6ee: 3a0f cmp cl, [edi]
01e7b6f0: dd07 fld qword ptr [edi]
01e7b6f2: 660f6f144e movdqa xmm2, [esi+ecx*2]
01e7b6f7: 660f6f644e10 movdqa xmm4, [esi+ecx*2+10h]
01e7b6fd: 660f db 0fh
01e7b6ff: 3a0f cmp cl, [edi]
01e7b701: e207 loop 01e7b70a
01e7b703: e97d020000 jmp 01e7b985
01e7b708: eb06 jmp 01e7b710
01e7b70a: 8d9b00000000 lea ebx, [ebx+00]
01e7b710: 660f6f1e movdqa xmm3, [esi]
01e7b714: 660f6f4610 movdqa xmm0, [esi+10h]
01e7b719: 660f db 0fh
01e7b71b: 3a0f cmp cl, [edi]
01e7b71d: c3 ret
01e7b71e: 08660f or [esi+0fh], ah
01e7b721: 6f outsd
01e7b722: 144e adc al, 4eh
01e7b724: 660f6f4c4e10 movdqa xmm1, [esi+ecx*2+10h]
01e7b72a: 660f db 0fh
01e7b72c: 3a0f cmp cl, [edi]
01e7b72e: ca088d retf 8d08
01e7b731: 348e xor al, 8eh
01e7b733: 660f6f2e movdqa xmm5, [esi]
01e7b737: 660f6f5e10 movdqa xmm3, [esi+10h]
01e7b73c: 660f db 0fh
01e7b73e: 3a0f cmp cl, [edi]

Built on Aegis on Sat Mar 28 15:17:50 2009 using compiler version 1400

Windows 5.1 (Windows XP x86 build 2600) [Service Pack 3]

EAX = 020aff28
EBX = 0149afb8
ECX = 00000150
EDX = 01e7b6c0
EBP = 00000000
ESI = 01eb02f0
EDI = 020afee0
ESP = 020afee0
EIP = 01e7b6c9
EFLAGS = 00010206
FPUCW = ffff027f
FPUTW = ffffffff

Crash reason: Illegal instruction

Crash context:
An instruction not supported by the CPU was executed in module 'nnedi2'.

Pointer dumps:

EAX 020aff28: 7c91e920 00000280 00000138 00000005 0006ffd4 0000012c 01eb0590 00000540
EBX 0149afb8: 00000001 00000001 00000001 00000000 00000000 01eb0050 01ed56f0 01efad90
EDX 01e7b6c0: 1e6f0f66 466f0f66 3a0f6610 6607c30f 4e146f0f 4c6f0f66 0f66104e 07ca0f3a
ESI 01eb02f0: e3e1e0c6 dad3d6d6 b4ccd5dc d5ccb4bc d6d3dadc e0e1e3d6 b8e0c5c6 bcc5b3a3
EDI 020afee0: 000002a0 01eb059c 01e7f8c4 01eb02f7 00000150 0149b090 020aff28 7c802530
ESP 020afee0: 000002a0 01eb059c 01e7f8c4 01eb02f7 00000150 0149b090 020aff28 7c802530
020aff00: 0149afb8 020aff74 7c80a0db 0149b090 fffffd60 00000150 0149b180 01492140
020aff20: 000001c4 00000005 7c91e920 00000280 00000138 00000005 0006ffd4 0000012c
020aff40: 01eb0590 00000540 000002a0 000002a0 000002a0 0149a160 01e7fa35 01f20564

Thread call stack:
01e7b6c9: nnedi2!0000b6c9
01e7f8c4: nnedi2!0000f8c4
01e7fa35: nnedi2!0000fa35
7c92005d: ntdll!RtlFreeHeap [7c910000+ff2d+130]
01e71752: nnedi2!00001752
7c92005d: ntdll!RtlFreeHeap [7c910000+ff2d+130]
01e717f7: nnedi2!000017f7
7c80b729: kernel32!GetModuleFileNameA [7c800000+b56f+1ba]
7c92005d: ntdll!RtlFreeHeap [7c910000+ff2d+130]

-- End of report
However nnedi2_rpow2(rfactor=2, opt=3) works fine.

[tritical]

What processor and version of Avisynth are you using? If you run debugview when loading the script nnedi2 should output a line like:

nnedi2: auto-detected opt setting = %d

what value does it give? It works correctly on my laptop that only has sse2.

EDIT: It looks like Avisynth's GetCPUFlags() function is off by 1 bit when checking ecx for SSSE3/SSE4.1/SSE4.2. For SSSE3 on intel cpus it is actually checking the Thermal Monitor 2 bit, which happens to be 1 on my Q6600 and 0 on my laptop so it ends up working correctly here :p. I'll just make auto-detection only go up to SSE3, the one SSSE3 routine didn't really make any difference speedwise.

EDIT2: Actually, it doesn't crash on my laptop because I had an old version of Avisynth on it that didn't check beyond SSE3 support.

[mikenadia]

Originally posted by tritical:
Maybe, you are free to try . I think it's unlikely to make any useful difference.
A gain in speed without noticeable loss of quality?

[Alex_ander]

Thank you for the new version, it works faster for me (5 years old laptop, Celeron-M 2.4, XP SP3, new AviSynth).
I have similar behavior to the mentioned by Archimedes: unless I set opt=3 in nnedi2_rpow2, it crashes in AVSP or VDub at the moment I use time slider. Other parameters don't matter at that. Maybe another default opt value would be more safe?

[lansing]

will nnedi2_rpow2(rfactor=2,cshift="spline36resize") have the same effect as TurnLeft().nnedi2(dh=true).TurnRight().nnedi2(dh=true) on antialiasing?

[10L23r]

well of course not, cus of that cshift thing

[tritical]

A gain in speed without noticeable loss of quality?
For which case? 7x enlargement?

I have similar behavior to the mentioned by Archimedes: unless I set opt=3 in nnedi2_rpow2, it crashes in AVSP or VDub at the moment I use time slider. Other parameters don't matter at that. Maybe another default opt value would be more safe?
This time there isn't technically a problem in nnedi2, but in Avisynth's GetCPUFlags() function which returns the supported instruction sets on the current processor. For opt=0 nnedi2 just checks that functions returned value, and chooses the largest opt value that is supported by the processor. Turns out that GetCPUFlags()'s checks for SSSE3/SSE4.1/SSE4.2 aren't right. Older versions of Avisynth 2.5 (those prior to Dec 2007) don't check beyond SSE3, and therefore wont cause a problem. Anyways, I'm just gonna make it not automatically set to opt=5 (just 4 or below) since the one SSSE3 routine doesn't make any real difference (that function doesn't take much time overall).

The initial problem with opt=0 (in v1.0/v1.1) was from my passing the opt value to some worker threads when they were created. However, I had the code that switches opt=0 to opt=x (the automatically detected value) after the thread creation code.

[Dark Shikari]

For which case? 7x enlargement?


This time there isn't technically a problem in nnedi2, but in Avisynth's GetCPUFlags() function which returns the supported instruction sets on the current processor. For opt=0 nnedi2 just checks that functions returned value, and chooses the largest opt value that is supported by the processor. Turns out that GetCPUFlags()'s checks for SSSE3/SSE4.1/SSE4.2 aren't right. Older versions of Avisynth 2.5 (those prior to Dec 2007) don't check beyond SSE3, and therefore wont cause a problem. Anyways, I'm just gonna make it not automatically set to opt=5 (just 4 or below) since the one SSSE3 routine doesn't make any real difference (that function doesn't take much time overall).

The initial problem with opt=0 (in v1.0/v1.1) was from my passing the opt value to some worker threads when they were created. However, I had the code that switches opt=0 to opt=x (the automatically detected value) after the thread creation code.Why not just call cpuid yourself? It's pretty easy to parse the output.

[tritical]

That is possible as well. Actually, I had forgotten that I already have cpuid code in there to check up to sse2. We shall see what happens.

[mikenadia]

For which case? 7x enlargement?
Yes. I do not know if we can deduct something from it but it seems that compressibility and speed may vary
if we split the resize after each iteration of NNEDI2 (my eyes are not that good to see a change in quality ,but I am sure there is one :)).
From a 240-160 source, using HC at CQ=5.

a) nnedi2_rpow2(rfactor=2)
nnedi2_rpow2(rfactor=2)
Spline36resize(672,448)
Size= 1.64 Mg (3mn 41s)

b) nnedi2_rpow2(rfactor=2)
Spline36resize(336,224)
nnedi2_rpow2(rfactor=2)
Size= 1.65 Mg (2 mn 48s)

NNEDI2 being much better than Spline36Resize, a) has to be better quality than b).
If you think in the future for 2 presets, "quality" and "speed".

Thanks a lot for your great plug-in.

[Archimedes]

What processor and version of Avisynth are you using?
Intel Pentium M (Centrino) with 1.5 GHz. With the new update, now it works fine on my "old" Intel Pentium 4 (2 GHz). In both cases i have installed the latest official release of AviSynth (version 2.58).

If you run debugview when loading the script nnedi2 should output a line like:

nnedi2: auto-detected opt setting = %d

what value does it give?
DebugView returns:
nnedi2: auto-detected opt setting = 5

[CorruptNova]

Intel Pentium M (Centrino) with 1.5 GHz. With the new update, now it works fine on my "old" Intel Pentium 4 (2 GHz). In both cases i have installed the latest official release of AviSynth (version 2.58).


I am also getting the same problem and I have basically the same system as Archimedes, a Intel P4 3.0ghz, AviSynth 2.5.8

DebugView gives the same output as Archimedes as well

[powerPC]

a) interp = nnedi(field=-2)
tdeint(mode=2,edeint=interp)

b) interp = nnedi2(field=-2)
tdeint(mode=2,edeint=interp)

a=b:confused:

[tritical]

I put up v1.3. It should fix the remaining opt=0 problems. Changes:
+ added nsize parameter
+ updated neural network weights
- fixed problem with opt=0 and SSSE3 detection

nsize switches between the old predictor neural network architecture, nsize=1 (what was used in v1.0-1.2), and a smaller architecture, nsize=0, which is faster. The only real difference is that the number of neurons in each layer is cut in half. On my Q6600 nsize=0 gives ~50% speed increase w/ qual=1. Speed difference is larger as qual is increased. There isn't that much difference between nsize=0/nsize=1, but nsize=1 should be better quality.

Should have new example images up soon. Will probably make them lossless avis instead of single images.

@powerPC
Those two should do the same thing.

[Archimedes]

Perfect! Now it works! Thank you very much for the update.

[Lorax2161]

Hi tritical,

Thanks for nnedi2 and your other contributions.

Having a problem with version v1.3 relating to:

nnedi2_rpow2(rfactor=4)
or
nnedi2_rpow2(rfactor=2,cshift="spline36resize")

Did the required syntax change for this? Previous versions have worked fine. I rolled back to v1.2 after trying v1.3, and v1.2 still works fine.

Processor: Athlon 64 X2 Dual Core 4200+ 2.19GHz 2GB of RAM

Avisynth: v2.58

DebugView: "nnedi2: auto-detected opt setting = 5"

VirtualDub: "An instruction not supported by the CPU was executed in module 'nnedi2'

Please let me know if you require any more information.

Thanks.

[kemuri-_9]

Processor: Athlon 64 X2 Dual Core 4200+ 2.19GHz 2GB of RAM
Avisynth: v2.58
DebugView: "nnedi2: auto-detected opt setting = 5"
VirtualDub: "An instruction not supported by the CPU was executed in module 'nnedi2'

I'm getting the same issue on my Phenom II x4 940.
getting crashes on the palignr (SSSE3 instruction set) calls
use opt=4 until tritical can fix the instruction set detection.

[tritical]

I goofed on that one. Make sure this - [removed] - works before I post a new version again. :thanks:

[kemuri-_9]

I goofed on that one. Make sure this - nnedi2.dll (http://bengal.missouri.edu/~kes25c/nnedi2.dll) - works before I post a new version again. :thanks:

yes, it's correctly detecting as SSE3 (opt=4) here now.

[Lorax2161]

I goofed on that one. Make sure this - nnedi2.dll (http://bengal.missouri.edu/~kes25c/nnedi2.dll) - works before I post a new version again. :thanks:

Confirmed here as well, thanks.

DebugView: "nnedi2: auto-detected opt setting = 4 (127)"

[MadRat]

I'm afraid vdub is crashing and saying:
An SSE2 (Pentium 4/Athlon 64) instruction not supported by the CPU was executed in module 'nnedi2'.
I have an AMD Anthlon 64 3200+ and opt=3 works so yeah there's still an auto detect problem. Sorry to have to report bad news.

[10L23r]

pscrn -

Controls whether or not the prescreener neural network is used to decide which pixels
should be processed by the predictor neural network and which can be handled by simple
cubic interpolation.

why not spline or lanczos instead of cubic?

[Chainmax]

I goofed on that one. Make sure this - nnedi2.dll (http://bengal.missouri.edu/~kes25c/nnedi2.dll) - works before I post a new version again. :thanks:

I was experiencing VDubMod crashes with v1.3 too, although it would just crash down without giving me time to see the error message (I think it was the "oops" one, not sure though). This DLL works perfectly. Thanks :).


By the way, I have one example where it seems like EEDI2 gives better results than NNEDI2. The first screenshot is with the former, the second with the latter:

http://img514.imageshack.us/img514/5647/eedi2.png
http://img190.imageshack.us/img190/9713/nnedi2.png

Yep, it's that clip again :). I assume that due to the unique nature of both the source and the script, it's not just the switch from EEDI2 to NNEDI2 that causes the differences. In fact, the script might only work with EEDI2. In any case, I can provide you with a clip (or the whole video, it's ~160MB) and the script if you want.

[JohannesL]

EEDI2 smoothes edges (and the whole image) more. NNEDI2 is sharper and thus enhances detail (and noise) more.

[Chainmax]

That's not the only difference. Look closely at the microphone: the lines on the EEDI2 version are completely straight (and the edges themselves are sharper), whereas the ones in the NNEDI2 version aren't.

[10L23r]

what's the source?

imo, the mic in nnedi2 looks better...
nnedi2 also keeps more details; eedi2 feels more artificial.
the only place where i find eedi2 better is the outline of the face.

[Keiyakusha]

I like EEDI2 result. On NNEDI2 screenshot these additional details are less noticeable than microphone (where eedi2 shows better result). In motion these details probably can't be noticed at all, while jagged lines - easily. I wonder how NNEDI2 looks on Anime compared to eedi2...

[shoopdabloop]

EEDI2 is the smoother deinterlacer, so if you can deal with the slight loss of detail it is definitely the way to go.

NNEDI2, on the other hand, is the far superior upsizer, hands down.

[10L23r]

wait... what is the mic supposed to look like? cus the mic in eedi2 looks rly rly fake

[shoopdabloop]

Chainmax, could you upload a clip of the unfiltered source? A small one.

Unless there are other filters that are somehow causing the poor deinterlacing in both cases, perhaps it would be best to go with MCBob or TempGaussMC.
I would like to try out various deinterlacers on the source myself.

[scharfis_brain]

@chainmax: provide your unprocessed sample as well as both scripts (for eedi2 and nnedi2)

[Didée]

That's not the only difference. Look closely at the microphone: the lines on the EEDI2 version are completely straight (and the edges themselves are sharper), whereas the ones in the NNEDI2 version aren't.
It's give and take. The diagonal rods are more straight with EEDI2, yes. But then, the lattice on the microphone's corpus are correctly (non-)interpolated only by NNEDI ... there, EEDI2 creates an artificial geometric pattern that should not be there. (And it probably will warp-around when there is slight motion.)

It's okay if you like EEDI2 better in some parts, but one should note that at the same time EEDI2 *fails* in other parts.

[*.mp4 guy]

A good interpolator will only blur/antialias/lowpass to the extent that must in order to remove aliasing. EEDI2 is far too aggressive for even the most aliased source achievable using a competent processing chain. nnedi(1/2) are extremely good at processing sources containing medium to high aliasing that is still more or less recoverable, but are not very good on sources with low/no aliasing (too slow for, if anything, likely reduced quality).

I should note at this point that most good sources will have some aliasing. Killing all aliasing in a signal will take too much of said signal to the grave with it, wheather through blurring or ringing, the damage is more or less the same.

If your source is butchered : eedi2
If your source is on the sharp side of good quality : nnedi
If your source is blurry : high quality linear resampling

Obviously there are gaps between the categories, they fall to personal taste.

[tritical]

I put up version 1.4. Same as the standalone nnedi2.dll that I posted before, except that it adds another nsize value (nsize 0,1 in v1.3 are now 1,2).

why not spline or lanczos instead of cubic?It's faster. Also, it doesn't make any noticeable difference. In the areas where it gets used (non-edges) they all give the same result.

@Chainmax
If you can provide a sample that would be great. As mentioned already, there is always a tradeoff between eedi2/nnedi2.

[Chainmax]

[EDIT]Link to the full video (http://www.sendspace.com/file/jgi6by)

shoopdabloop: the source is progressive, the EDI interpolators are used as a resizing method and to diminish aliasing actually.

Here (http://www.mediafire.com/?sharekey=e1b5f41eed4d25e0c2b435915e8821d723312b95698279d75be6ba49b5870170)'s a clip of the source, which most of you guys will recognize :). It contains the intro and the scene where the screenshots were taken from. Bear in mind it's encoded on Microsoft Video 1, in order to be able to work with it you'll have to load the file in VDub and make a lossless encode out of it, as AFAIK Avisynth cannot load it.

The script used is, again, familiar to most of you guys:

f=4

AVISource("X:\wherever\MakingGK-LssLss.avi",audio=false)

a=last

a.converttoyv12()

nnedi2_rpow2(qual=3,rfactor=2,cshift="spline36resize")

pointresize(a.width*f,a.height*f)

vf=last.mvanalyse(isb=false,blksize=16, truemotion=true)
vb=last.mvanalyse(isb=true,blksize=16, truemotion=true)
vf1=last.mvanalyse(isb=false,delta=2,blksize=16, truemotion=true)
vb1=last.mvanalyse(isb=true,delta=2,blksize=16, truemotion=true)

xf1=last.mvflow(vf1,thSCD1=9999)
xf=last.mvflow(vf,thSCD1=9999)
xb=last.mvflow(vb,thSCD1=9999)
xb1=last.mvflow(vb1,thSCD1=9999)

interleave(xf1,xf,last,xb,xb1)
temporalsoften(2,40,50,mode=2)
x=selectevery(5,2)

x.converttorgb()
xa=pointresize(a.width*2,a.height*2).converttoyv12().nnedi2_rpow2(qual=3,rfactor=2,cshift="spline36resize")
xb=crop(1,1,0,0).addborders(0,0,1,1).converttoyv12().pointresize(a.width*2,a.height*2).nnedi2_rpow2(qual=3,rfactor=2,cshift="spline36resize")
x=merge(xa,xb)

xvf=x.mvanalyse(isb=false,blksize=16, truemotion=true)
xvb=x.mvanalyse(isb=true,blksize=16, truemotion=true)
xvf1=x.mvanalyse(isb=false,delta=2,blksize=16, truemotion=true)
xvb1=x.mvanalyse(isb=true,delta=2,blksize=16, truemotion=true)

xxf1=x.mvflow(xvf1,thSCD1=9999)
xxf=x.mvflow(xvf,thSCD1=9999)
xxb=X.mvflow(xvb,thSCD1=9999)
xxb1=x.mvflow(xvb1,thSCD1=9999)

interleave(xxf1,xxf,x,xxb,xxb1)
temporalsoften(2,255,255,mode=2)
selectevery(5,2)

Spline36Resize(512,384)

Levels(0,1,245,16,235)

FFT3DFilter(sigma=6,plane=3,bw=32,bh=32,bt=3,ow=16,oh=16)

Tweak(sat=1.3)

dull=last
sharp=dull.LimitedSharpenFaster(Smode=4,Strength=200)
Soothe(sharp,dull,25)

c=last
GrainU=c.UtoY().AddGrain(4,0,0)
GrainV=c.VtoY().AddGrain(4,0,0)
Return ytouv(GrainU,GrainV).MergeLuma(c).gradfun2db(thr=2.4).gradfun2db().AddGrain(10,0,0)

The EEDI2 version is the same, only substituting nnedi2_rpow2(qual=3,rfactor=2,cshift="spline36resize") for eedi2().turnleft().eedi2().turnright()

[tritical]

See if this: [removed] works on that processor. :thanks:

[Dark Shikari]

01eac208: 0fd4c1 paddq mm0, mm1 <-- FAULTWhat?!

PADDQ mm1,mm2/m64 ; 0F D4 /r [PENT,MMX]

paddq is part of MMX1, which should be on every chip since the Pentium 2 (and a few before).

[MasterNobody]

See if this: nnedi2.dll (http://bengal.missouri.edu/~kes25c/nnedi2.dll) works on that processor. :thanks:
Yes. It fixes the crash.
paddq is part of MMX1, which should be on every chip since the Pentium 2 (and a few before).
Probably, you are wrong and PADDQ is SSE2 instruction.

[Dark Shikari]

Yes. It fixes the crash.

Probably, you are wrong and PADDQ is SSE2 instruction.NASM documentation says it's MMX, but it could be wrong.

[MasterNobody]

http://www.intel.com/Assets/PDF/manual/253667.pdf says it's SSE2. So I would trust Intel documentation more than NASM.

[IanB]

And from the Fastwire defs (very reliable) :-

{"PADDQ", "mmreg,r/m64", "0F D4 /r", Instruction::CPU_WILLAMETTE|Instruction::CPU_SSE2},

So you need CPUF_SSE2 = 0x20

[g-force]

as for the nnedi2/eedi2 debate, why not something like this?

Function EDImod(clip input)
{# function by g-force v.1.00
#connects more lines than NNEDI2, without the blur bubbles and loss of detail of EEDI2

nn=input.NNEDI2(dh=true,field=1)
ee=input.EEDI2()

Interleave(ee,ee.RemoveGrain(12,-1),ee.RemoveGrain(4,-1))
Clense(reduceflicker=false,grey=true)
SelectEvery(3,1)

MT_MakeDiff(last,last.RemoveGrain(12,-1))
last.Repair(MT_MakeDiff(nn,ee),1,-1).MT_LUTxy(last,"x 128 - abs y 128 - abs < x y ?")

ee.MT_AddDiff(last,chroma="copy first")

last.Repair(nn,9)

Return(last)
}

-G

[Didée]

@g-force: Quick question.

The final repair(9) is the most stringuent of the "line-repairing" modes of removegrain. Assuming the scenario that

1) EEDI2 does connect a given diagonal
2) NNEDI2 does not connect the same diagonal

then, how can it be that the result should connect the diagonal, when NNEDI2 is the reference? If NNEDI2 didn't connect, shouldn't the "line-repair" mode remove the connection that EEDI2 did?

[g-force]

@g-force: Quick question.

The final repair(9) is the most stringuent of the "line-repairing" modes of removegrain. Assuming the scenario that

1) EEDI2 does connect a given diagonal
2) NNEDI2 does not connect the same diagonal

then, how can it be that the result should connect the diagonal, when NNEDI2 is the reference? If NNEDI2 didn't connect, shouldn't the "line-repair" mode remove the connection that EEDI2 did?

Agreed, but Repair(9) doesn't repair ALL lines, so, yeah, it basically undoes most of what EEDI2 did, just not all of it.

One could play around with different repair modes I suppose, and use more or less of the EEDI2 result, but then I suspect those darn blur bubbles would start showing up again.

-G

[Revgen]

EEDI2 is okay for alot of sources that aren't sports related.

However, most of what I deinterlace is sports material. IMO, EEDI2 simply doesn't cut it like NNEDI or NNEDI2 does when it comes to sports, especially the diagonal lines of the courts/fields and preserving the detail of the athletes. The cameras usually have far away views showing all of the players on the court/field so preserving details is important, otherwise the athletes look like blurry figments.

[Chainmax]

as for the nnedi2/eedi2 debate, why not something like this?

Function EDImod(clip input)
{# function by g-force v.1.00
#connects more lines than NNEDI2, without the blur bubbles and loss of detail of EEDI2

nn=input.NNEDI2(dh=true,field=1)
ee=input.EEDI2()

Interleave(ee,ee.RemoveGrain(12,-1),ee.RemoveGrain(4,-1))
Clense(reduceflicker=false,grey=true)
SelectEvery(3,1)

MT_MakeDiff(last,last.RemoveGrain(12,-1))
last.Repair(MT_MakeDiff(nn,ee),1,-1).MT_LUTxy(last,"x 128 - abs y 128 - abs < x y ?")

ee.MT_AddDiff(last,chroma="copy first")

last.Repair(nn,9)

Return(last)
}

-G

Thanks for thinking this up, and I apologize for the very late reply. Pending some reading on the Masktools docs (I am very rusty now in this regard), can someone explain in a few words what is this doing? Seems like it is mixing the EEDI2 and NNEDI2 results, then toying around with the differences, but I can't gander much beyond that. Also, I take it this approach would not benefit from using edge masks (i.é: using EEDI2 only on edges and NNEDI2 everywhere else)?

[Didée]

Briefly: Sharpen up EEDI, then limit the result by NNEDI2 via the line-repair mode 9.


Slighty more elaborative:

interleave(orig,rg12,rg4) - clense - selectevery
That combo does a sort of "blurry-clipping", i.e. damping-down of signal peaks. (The result is highly similar to what my "MinBlur" routine does. Same idea, different approch. This one probably is slightly faster.)

diff(last,last.rg12) - repair( diff(ee,nn) ).mt_lutxy(..)
This figures the sharpening kernel. From the two differences: diff{damped-EEDI,gauss1(damped-EEDI)} - and - diff{EEDI,NNEDI}, choose that difference that is smaller (i.e. makes less change).

ee.adddiff(..)
Apply the sharpening kernel.

repair(nn,9)
Limit the sharpened-up EEDI to not exceed the clamp spanned by those two neighbor pixels in the NNEDI clip that are most agreeing on their value. (repair, mode9)


Frankly, I'm not sure if the elaborotive version is understandable.:D - Tried about my best, but it's not easy to squeeze in just a few words.

[Chainmax]

Thanks for the explanation :). Sounds pretty cool, and I guess it would be more effective than my suggestion. I'll toy around with it and post a couple of screenshots here.

[Didée]

While you're at toying, compare also with

nn = theclip.NNEDI2(dh=true,field=1)
nn.sharpen(.6789012345).repair(nn,9)

and see if it's much of a difference. (Vary sharpen strength, can't predict what's a good value here.)

[Didée]

Maybe EDImod helps on the occasional pixel, if you search and look out hard. But, on the occasional noticeable errors of NNEDI2, it won't help anything worth ...

Case example, together with two other simple things one can try:

http://thumbnails22.imagebam.com/5450/9951e054498949.gif (http://www.imagebam.com/image/9951e054498949)

Neither method can rule out errors, because it's not possible to reckognize errors as such. (Not by such simple means, at least.)

[g-force]

The result is highly similar to what my "MinBlur" routine does. Same idea, different approch.

More accurately: "The idea was completely stolen from Didée and tweaked beyond recognition"

As a matter of fact, that's not the only thing that was stolen. Basically, the whole function applies a very dumbed-down version of Didée's Contrasharpening to EEDI (contrasharpen EEDI with NNEDI) then apply some bandages in the form of the repair function to help with the artifacts EEDI produces. Not the greatest function in the world, but it is what it is.

I wrote this function for a source that has some pretty bad aliasing due to a point-resize down applied at some point. I've since switched to the method that I think is the same as what scharfis_brain suggests here:

http://forum.doom9.org/showthread.php?t=150346

I think this is the same as:

NNEDI2(dh=true,field=1)
NNEDI2(dh=false,field=0)

-G

[scharfis_brain]

I think this is the same as:

NNEDI2(dh=true,field=1)
NNEDI2(dh=false,field=0)

-G

I am becoming old... I completely miss the easy stuff sometimes ;)

[g-force]

I am becoming old... I completely miss the easy stuff sometimes ;)

Yours makes logical sense, gets the job done, and probably doesn't take much more time.

I noticed in the post that I linked to, you call it a "very disgusting way...". This is no more disgusting that the crazy stuff that someone else has done to the video to get it to the point that you need to do something so drastic. I originally thought the same way, but thinking about it more i realize that when doing a "good" resize, you have all interpolated pixels anyway. Now if someone does a point resize that causes lots of aliasing, they could equally say, "well, at least I have some of the original pixels". Sounds good until you see what a hatchet job of the input they made. So, why not interpolate ALL of the pixels, just like the good resizer would have done anyway?

-G

[scharfis_brain]

I think the video in the thread you mentioned hasn't been butchered at all.

It is high speed footage taken with a high speed camera.

These cameras usually will drop lots of sensor pixels in order to achieve fast readout rates.

[g-force]

I think the video in the thread you mentioned hasn't been butchered at all.

It is high speed footage taken with a high speed camera.

These cameras usually will drop lots of sensor pixels in order to achieve fast readout rates.

Oh, I see. Well, whether it was done by a person or the camera, the video is still decimated instead of being interpolated/resampled. So my point is, why not do after the fact what should have been done to begin with.

-G

[Didée]

Regarding the source in "that other thread" - though not fully sure, I think the reason that a simple nnedi interpolation looks still jaggy is due to the camera's sampling. NNEDI assumes that there is *one* sample (scanline) missing between the present samples. But there, probably the camera dropped much more, and there are actually two or three, or evenmore samples missing between the actual samples. Which is a scenario that NNEDI is not really made for ... there is much more data missing than NNEDI assumes that it would!

[tritical]

I have a new filter almost ready that is much more an extension of eedi2 than nnedi2, but does use some ideas from nnedi2. It is much better at line/edge connection. In terms of subjective visual appearance I definitely prefer it for deinterlacing. Image enlargement is another story. Hopefully this weekend a rough first cut will be done. Just a warning, it will be quite slow since it takes into account how every pixel in the same line is being interpolated.

[scharfis_brain]

Oh, I see. Well, whether it was done by a person or the camera, the video is still decimated instead of being interpolated/resampled. So my point is, why not do after the fact what should have been done to begin with.
-G
With high-speed cameras it often is impossible to interpolate, cause the camera hadn't the time available to read out all necessary pixels.
The only solution would have been an exchangable optical low pass filter in front of the sensor. (Like the fixed ones in front of bayer based colour sensors to avoid colour moire).

[g-force]

I have a new filter almost ready that is much more an extension of eedi2 than nnedi2, but does use some ideas from nnedi2. It is much better at line/edge connection. In terms of subjective visual appearance I definitely prefer it for deinterlacing. Image enlargement is another story. Hopefully this weekend a rough first cut will be done. Just a warning, it will be quite slow since it takes into account how every pixel in the same line is being interpolated.

I look forward to checking it out!

-G

[tritical]

A little late, but here is an early version: eedi3 (http://bengal.missouri.edu/~kes25c/eedi3/eedi3.dll). Like I said, it is SLOW. Atm it isn't optimized at all... no asm, no threading, not even on the algorithmic level. It's just proof of concept stage.

Suggested usage is eedi3(sclip=nnedi2()) for same rate deinterlacing, and eedi3(field=-2,sclip=nnedi2(field=-2)) for double rate.

Some example images:
http://bengal.missouri.edu/~kes25c/eedi3/soccer_original.png
http://bengal.missouri.edu/~kes25c/eedi3/soccer_eedi2.png
http://bengal.missouri.edu/~kes25c/eedi3/soccer_nnedi2.png
http://bengal.missouri.edu/~kes25c/eedi3/soccer_eedi3+nnedi2.png

http://bengal.missouri.edu/~kes25c/eedi3/bbcourt_original.png
http://bengal.missouri.edu/~kes25c/eedi3/bbcourt_eedi2.png
http://bengal.missouri.edu/~kes25c/eedi3/bbcourt_nnedi2.png
http://bengal.missouri.edu/~kes25c/eedi3/bbcourt_eedi3+nnedi2.png

Haven't created a readme yet, but the parameter strings from the code are:

"eedi3", "c[field]i[dh]b[Y]b[U]b[V]b[alpha]f[beta]f[gamma]f[nrad]i[mdis]i[hp]b[ucubic]b[cost3]b[vcheck]i[vthresh]f[sclip]c"

"eedi3_rpow2", "c[rfactor]i[alpha]f[beta]f[gamma]f[nrad]i[mdis]i[hp]b[ucubic]b[cost3]b[vcheck]i[vthresh]f[cshift]s[fwidth]i[fheight]i[ep0]f[ep1]f"

Similar to nnedi2 calls, field/dh/y/u/v are the same as nnedi2. rfactor/cshift/fwidth/fheight/ep0/ep1 are the same as nnedi2_rpow2. The other parameters:

alpha/beta/gamma (defaults: 0.15,0.2,20.0):
These trade off line/edge connection vs artifacts created. alpha and beta must be in the range [0,1], and the sum alpha+beta must be in the range [0,1]. alpha is the weight given to connecting similar neighborhoods.. the larger it is the more lines/edges should be connected. beta is the weight given to vertical difference created by the interpolation... the larger beta is the less edges/lines will be connected (at 1.0 you get no edge directedness at all). The remaining weight (1.0-alpha-beta) is given to interpolation direction (large directions (away from vertical) cost more)... so the more weight you have here the more shorter connections will be favored. Finally, gamma penalizes changes in interpolation direction, the larger gamma is the smoother the interpolation field between two lines (range is [0,inf].

My findings have been that alpha in [0.08-0.2] and beta in [0.1-0.3] work well. If lines aren't getting connected than increase alpha and maybe decrease beta/gamma. Go the other way if you are getting unwanted artifacts.

hp/ucubic/cost3 (defaults: false, true, true):
These are speed vs quality options. hp=true, use half pel steps, hp=false, use full pel steps. ucubic=true, use cubic 4 point interpolation, ucubic=false, use 2 point linear interpolation. cost3=true, use 3 neighborhood cost function to define similarity, cost3=false, use 1 neighborhood cost function.

vcheck/vthresh/sclip (defaults: 2 range is [0-3], 32 range is (0,inf], NULL):
A inverse reliability measure in the range [0,255] is created at each interpolated location. The generated pixel value is averaged with the pixel value at the same location in sclip using:

a = min(rmeasure/vthresh,1.0)
(1.0-a)*generated_val+a*sclip_val

How rmeasure is generated is controlled by vcheck. vcheck=0, always gives rmeasure=0 (so no averaging with sclip). As you increase from 1 to 2 to 3 it gets stricter so rmeasure will always be greater for vcheck=2 than 1, etc...

If sclip is not given than cubic interpolation is used instead for those values.

Any testing would be appreciated, and any images that fail miserably. eedi3 isn't that great at enlargement, it's mainly for deinterlacing.

[elguaxo]

great! :thanks:

[poisondeathray]

Thanks, some nice improvements on a couple of sources that I have tried

Here is a DV-AVI sample where it definitely improves some sections, but worse on others
http://www.megaupload.com/?d=NZUTSNFD

I used the defaults settings, not any of the alpha/beta/gamma etc.... so maybe they could have been improved

In the 1st set (frame 75), the background ramp is improved, but there are some minor artifacts introduced in the
grass and the blue tube under the foreground ramp with the dog

In the 2nd set (frame 175), the white fence is "blurred", and this occurs in the entire sequence of frames that
have the fence

75 nnedi2
http://i35.tinypic.com/im5k09.jpg

75 eedi3+nnedi2
http://i34.tinypic.com/6zofup.png

175 nnedi2
http://i33.tinypic.com/30rw968.png

175 eedi3+nnedi2
http://i35.tinypic.com/e0iphk.png

[AVIL]

Hi:

In a quick test, i've compared yadifmod+nneedi2 with eedi3+nnedi2. IMHO eedi3 is a bit sharper (good) but it destroys crossed lines pattern and dont treat properly sobreimpressed progresive subtitles (it jumps). But is a good start.

A image (left yadifmod, right eedi3, child's face is blurred intentionally):


After, i've compared against nnedi2 pure. Now subtitles are equally processed but nnedi2 outperforms eedi3 in crossed lines pattern.

[canuckerfan]

^similar pattern in poisondeathray's picture with the white picket fence. just at the top the lines are connected where they shouldn't be. by chance did you try fooling around with the alpha/beta/gamma values?

[shoopdabloop]

Yeah, I think eedi3 is too watercolor-y at default values atm.

[tritical]

Thanks for the reports. eedi3 tries to recreate the missing line by connecting matching regions/pixels in the top line and bottom line through a smooth, non-decreasing (no crossing) warping. It finds the mapping with the least cost (cost is based on alpha/beta/gamma and the associated measures: neighborhood similarity, induced vertical variation, connection distance, change in direction) through a dynamic programming algorithm. This works really well as long as the main assumption holds: every pixel in line n-1 has a match in line n+1. This is not the case surprisingly often such as junctions, ends of lines/edges, etc. These are specifically the cases that nnedi2 handles much better than other edge directed algorithms since it doesn't make the assumption about pixels in n-1/n+1 connecting to form pixels in line n. The inverse reliability measure that eedi3 produces attempts to detect these areas, and then leans towards the sclip values.

Take the fence case, to correctly interpolate (not connect things that shouldn't be) the top/bottom parts of the fence you need to connect completely different regions (white to green) at certain points when similar (white to white) mappings exist that meet the non-decreasing warping criteria. You might be able to tweak alpha/beta/gamma to get it correct, but that would probably prevent it from connecting edges/lines in a lot of other cases. In the second case with the girls dress, I'm not sure why it is so blurry. The horizontal lines seem correct, but the vertical lines are completely gone. Of course they appear to be different frames, and one example is using yadifmod and the other is not. I'd have to see the source, but I assume you probably don't want to distribute it. Still these cases are tough since the interpolation direction is ambiguous without considering a much larger area.

I think some price has to be paid for better edge/line connectivity. nnedi2 is safe, it rarely produces artifacts that vertical interpolation would not; however, I have many sources similar to the two I posted before where nnedi2 fails to connect/smooth major lines/edges. For a while I thought this was due to inadequate complexity of the neural network, sub-optimal optimization of the parameters, insufficient neighborhood size, etc, but I've found it is primarily due to the metric being optimized for. Neither psnr nor ssim seem to favor what a person would favor close enough. In terms of acceptable artifacts (where the output is slightly to completely ambiguous given a local neighborhood) vs connected/smoothed major edges/lines (the ones a person notices immediately when looking at an image). Anyways, I will try a few more ideas I have for eedi3.

[AVIL]

Hi,

I've posted in rapidhsare a extract from the skirt (huffyuv 256x192, 10 secs).

The link:

http://rapidshare.com/files/307295248/test.avi.html

[akiduki]

Hi, when applying eedi3 to the following image, I found some strange effect around the boundary (see the attached files).
It seems eedi3 lacks sophisticated boundary check and handling process when filtering.

BTW eedi2 and needi2 are all right in this case.

Here's the original image and eedi3'd image:
above image is eedi3'd, using the suggest call syntax eedi3(sclip=nnedi2());
bottom image is the original one

[tritical]

For the borders I just pad the image by mirroring and run the algorithm. At the left/right edges the interpolation direction is forced to vertical (I don't allow directions that would result in using pixels outside of the original image in the interpolation). That isn't a problem however. What really causes the artifacts, or incorrect directions, at the borders is using the mirrored pixel values in the neighborhood similarity computation. This could be fixed by ignoring locations that fall off the image instead of using some sort of extension to generate them.... of course that takes more code and time to implement.

I also found a bug in cubic interpolation for non-vertical directions. That image also shows part of the downside to processing RGB planes separately... with default values B/G get interpolated correctly, but the difference between pixel values in the R plane is large enough that the vertical variation outways neighborhood similarity in the cost function and the lines aren't connected. Increasing alpha to 0.2 fixes that.

[akiduki]

Thanks for the info.

OK, so the problem is coming from the similarity computation that falsely took the off-image pixels, which is mirrored from inside in your implementation, into account of direction of interpolated pixels?
I noticed that the patterns in the upper and lower boundary are differed from the left and right one after interpolation. Is it due to what you are saying above?

And BTW, is it any other boundary treatment available besides mirroring pixels from inside? I am not on firm track with those related topics but I think mirroring is rather old-fashioned method that I remembered I was taught such treatment years ago in university.

[Revgen]

I was able to get to work somewhat with MCBob (couldn't get it to work with edisingle=), but it was slow and hogged up way too much memory and crashed after 4 frames so it's basically unusable with HDTV footage.

However, the dumb bobs came out nice. The EEDI3+NNEDI2 method looks great (for a dumb bobber anyway) for the sports footage that I work with.

EDIT:

Apparently, I don't need it for edisingle, as that stage doubles the height, which is apparently better suited for NNEDI2 to do alone.

[Revgen]

The biggest difference with EEDI3 is the improvement of straight diagonal lines. You can really see this by looking at the top of edge of the glass pane when comparing NNEDI2 and EEDI3 + NNEDI2.

Which pretty much mirrors what Tritical already showed.

http://img109.imageshack.us/img109/1636/hockeyprogressiveorigin.png (http://img109.imageshack.us/i/hockeyprogressiveorigin.png/)
Original Progressive Source

http://img195.imageshack.us/img195/2859/hockeytgmcnnedi2.png (http://img195.imageshack.us/i/hockeytgmcnnedi2.png/)

Interlaced From Progressive Source

http://img109.imageshack.us/img109/4542/hockeymcbobeedi3.png (http://img109.imageshack.us/i/hockeymcbobeedi3.png/)

MCBob + EEDI3 + NNEDI2
http://img195.imageshack.us/img195/7383/hockeymcbobnnedi2.png (http://img195.imageshack.us/i/hockeymcbobnnedi2.png/)

MCBob + NNEDI2

http://img509.imageshack.us/img509/9607/hockeytgmceedi3.png (http://img509.imageshack.us/i/hockeytgmceedi3.png/)

TGMC + EEDI3 + NEDI2

http://img8.imageshack.us/img8/2859/hockeytgmcnnedi2.png (http://img8.imageshack.us/i/hockeytgmcnnedi2.png/)

TGMC + NNEDI2

[Undead Sega]

Im somewhat starting to think (although i could be wrong) but is it me or is it that NNEDI2 isnt great at Antialiasing?

also, where can one obtain EEDI3? it really seems to perform really good combined with NNEDI2! :D

[markanini]

The downlad link is at the top of the page.

I mostly use NNEDI2 for upscaling still images. When I try EEDI3 I get the impression that it erroneously creates diagonals in some areas of complex detail like foliage and checker board patterns. It's better than NNEDI2 at dealing with halos from oversharpening and mosquito noise. Upping alpha to 0.2 does improve things overall but doesnt deal spcifically with the errors I'm seeing. So I'm gonna have to feel content with NNEDI2 for now and await further developments with EEDI3.

Many thanks to trictal for his work.

[Undead Sega]

Well i could be very wrong on this matter, can anyone prove to me that NNEDI2 is an excellent Antialiaser for video despite the screencaps in the previous posts? because surely if it is used for upscaling which is satisfied to most people, that itself would create jagged edges wouldnt it?

[shoopdabloop]

nnedi2 doesn't necessarily create jagged edges unless it can't connect some, and it usually can when upscaling progressive footage.

[LoRd_MuldeR]

http://img109.imageshack.us/img109/1636/hockeyprogressiveorigin.png (http://img109.imageshack.us/i/hockeyprogressiveorigin.png/)
Original Progressive Source

Sorry for the off-topic, but does anybody have a link to that sample clip? :o

[Revgen]

^I'll upload it for you.

[Revgen]

Here it is.

http://www.sendspace.com/file/jjq5hx

[LoRd_MuldeR]

Here it is.

http://www.sendspace.com/file/jjq5hx

:thanks:

[Chainmax]

New screenshot batch with the new suggestions made.


Source + Bilinear resize
http://img51.imageshack.us/img51/1931/frame12307v1.png

eedi2().turnleft().eedi2().turnright()
http://img192.imageshack.us/img192/1391/frame12307v2.png

nnedi2_rpow2(qual=3,rfactor=2,cshift="spline36resize")
http://img192.imageshack.us/img192/1170/frame12307v3.png

eedi3_rpow2(rfactor=2,cshift="spline36resize",hp=false)
http://img192.imageshack.us/img192/1170/frame12307v4.png

EDIMOD()
http://img188.imageshack.us/img188/5610/frame12307v5.png

Didée's quick suggestion (no toying with sharpen strength)
http://img192.imageshack.us/img192/5786/frame12307v6.png


In my opinion: Bilinear resize looks predictably awful, EEDI2 gives really straight lines but is a bit too soft, nnedi2 looks much sharper but borders are not as well rendered, eedi3 looks to combine the best aspects of EEDI2 and nnedi2, EDIMOD is very similar to EEDI2 but marginally sharper, and Didée's quick suggestion looks the sharpest but borders get really aliased. I might try lowering sharpen strengh, but I expect that to reverse to EEDI2-like results.

Thanks for the feedback so far, and once again I apologize for the huge delay.

[Undead Sega]

wow....EEDI3 looks really good...seems NNEDI2 has some competition :D

but looking at the previous caps NNEDI2 has some small advantage by retaining alot of the shapes but quite badly aliased, if only there was a way to combine both NNEDI2 and EEDI3 then it will be game over! :D

P.S. How can one use EEDI3? and why is there no thread dedicated to it?

EDIT: oh wait my bad, i havent seen a screencap of EEDI3 working on its down or with TGMC without NNEDI2.

[Varies]

and how about optimizations for Quad CPU\ CUDA\ OpenCL ?

EEDI3 looks really great but work's very SLOOOOOOOOOW :) (0,1 FPS on Core2Quad 9300@3GHz)

[AnnaFan777]

Is there a way for nnedi2 to output an edge mask image?
(bicubic area + edge interpolated area),

I am trying to replace bicubic with other resize methods
and sharpen only the non-edge areas.

[shoopdabloop]

Yes, an edge mask from nnedi2 would be nice.

[tritical]

I can have it output a mask indicating where it used cubic interpolation vs neural network interpolation, and/or I could have it run the prescreener neural network on all pixels of the input image and output that mask. However, that wouldn't be a true edge mask since most vertical edges (large gradient horizontally, small gradient vertically) will not be marked, only diagonal/horizontal edges.

[LaTo]

A little function to automate resizing with nnedi2:

# NNEDI2resize function by LaTo INV.
#
# PARAMETERS:
# -----------
# ox = destination width
# oy = destination height
# nsize,qual,pscrn,cshift = nnedi2 parameters
#
# ABOUT RESOLUTION:
# -----------------
# if output > input : use nnedi2 resizer + "cshift" resizer
# if output = input : do nothing
# if output < input : use "cshift" resizer
#
# EXAMPLE:
# --------
# NNEDI2resize(1280,720)
# NNEDI2resize(1920,1080,qual=1,cshift="spline16resize")

function NNEDI2resize(clip clp, int ox, int oy, int "nsize", int "qual", bool "pscrn", string "cshift")
{
nsize = default( nsize, 2 )
qual = default( qual, 3 )
pscrn = default( pscrn, true )
cshift = default( cshift, "spline36resize" )

ix = clp.width()
iy = clp.height()

rx = float(ox)/float(ix)
ry = float(oy)/float(iy)
rf = (rx > ry) ? rx : ry

clp = ( rf > 1.0 ) ? clp.nnedi2_rpow2(rfactor=ceil(rf/2.0)*2, nsize=nsize, qual=qual, pscrn=pscrn, cshift=cshift, fwidth=ox, fheight=oy)
\ : ( rf < 1.0 ) ? Eval( "clp." + cshift + "(" + string(ox) + "," + string(oy) + ")" )
\ : clp

return clp
}

Maybe it will help someone :)

[Gavino]

Thanks, LaTo.
... Eval( "clp." + cshift + "(" + string(ox) + "," + string(oy) + ")" )

You could also write this more simply as:
Apply(cshift, clp, ox, oy)

[Didée]

Perhaps I'm missing something, but that wrapper function seems like a placebo. A placebo with additional restrictions. For upsizing, there's nothing new to it. All functionality is directly available through nnedi2_rpow2. For the cases of same-size or smaller-size, it may seem coolio to do nothing, resp. to bypass nnedi2 and just use the kernel resizer. But, what if someone wants to use nnedi2 e.g. for anti-aliasing or feature connection, while keeping same size or doing a slight downsize? That's a valid aim to achieve, but nnedi2resize doesn't do. Well, in that case the answer is "don't use nnedi2resize, instead use nnedi2_rpow2 directly."

I just don't see the point in it. If the user does a call to nnedi2, then it is to be assumed that he does for a reason. I don't like the idea of making such decisions for the user. The blue statement is universally true.

(What if IanB decides that for downsizing, all of the available bicubic/lanczos/spline/blackman resizers are silently mapped to spline16resize? Just because IanB finds that it's a cool thing to do so? Joe user is a dumby anyway, so let's make that decision for him?)

[LaTo]

Perhaps I'm missing something, but that wrapper function seems like a placebo. A placebo with additional restrictions. For upsizing, there's nothing new to it. All functionality is directly available through nnedi2_rpow2. For the cases of same-size or smaller-size, it may seem coolio to do nothing, resp. to bypass nnedi2 and just use the kernel resizer. But, what if someone wants to use nnedi2 e.g. for anti-aliasing or feature connection, while keeping same size or doing a slight downsize? That's a valid aim to achieve, but nnedi2resize doesn't do. Well, in that case the answer is "don't use nnedi2resize, instead use nnedi2_rpow2 directly."

I just don't see the point in it. If the user does a call to nnedi2, then it is to be assumed that he does for a reason. I don't like the idea of making such decisions for the user. The blue statement is universally true.

(What if IanB decides that for downsizing, all of the available bicubic/lanczos/spline/blackman resizers are silently mapped to spline16resize? Just because IanB finds that it's a cool thing to do so? Joe user is a dumby anyway, so let's make that decision for him?)

It's just a helper function for resizing... :rolleyes:

[Didée]

Yeah, okay. But if it's a helper function, in what point does it actually help?

Perhaps make also a wrapper for tdeint, so that people can use tdeint_wrapper(params) instead of tdeint(params)? Yay!

Maybe it's the high amount of analgesics I'm currently on, but I don't see the deal. And I don't see how you can decide for the user that he doesn't need nnedi2 for same-or-smaller size. It usually will be so, but not always. And you're not the one who can make that decision safely. Me neither. It's a decision that cannot be done universally.

[LaTo]

Yeah, okay. But if it's a helper function, in what point does it actually help?

Perhaps make also a wrapper for tdeint, so that people can use tdeint_wrapper(params) instead of tdeint(params)? Yay!

Maybe it's the high amount of analgesics I'm currently on, but I don't see the deal. And I don't see how you can decide for the user that he doesn't need nnedi2 for same-or-smaller size. It usually will be so, but not always. And you're not the one who can make that decision safely. Me neither. It's a decision that cannot be done universally.

This:
NNEDI2resize(1280,720)

is faster than:
rx = float(1280)/float(last.width())
ry = float(720)/float(last.height())
rf = (rx > ry) ? rx : ry

nnedi2_rpow2(rfactor=ceil(rf/2.0)*2, cshift="spline36resize", fwidth=1280, fheight=720)

:D

[Chainmax]

tritical, is there any chance an optimized version of EEDI3 will come out someday?

[Revgen]

^He said that it's at a proof of concept stage right now. So basically, he plans to release an optimized version sometime in the future.

[Dogway]

Do you know if EEDI3 works with MT 0.7? it only reached 25% on a 4 cores system.

[Revgen]

^My guess is no. Since Tritical has already said that the current version is not optimized and is simply a proof of concept. He released it simply to see how it works and what needs to be improved fundamentally before doing speed optimization later.

[tritical]

eedi3 should work with mt (or setmtmode, i don't remember the details of each method), but it processes each line independently so any mt methods that break up the image and then reassemble it should be ok, as well as methods that process different frames in different threads using a separate filter instance per thread (don't think I have any changing globals in there). I don't have any plans to work on eedi3 further. When i get a chance i'll package up the source and release what's there. Threading it internally would be easy, probably can be done with one openmp line iirc, but it doesn't lend itself to other optimization as easily.

I'm actually close to an nnedi3. Compared to nnedi2 it has a slightly different predictor nn architecture, different local neighborhood sizes, different local neighborhood pre-processing before feeding to the nn, and uses a different nn weight optimization method. Progress has been rather slow lately, but i should have more time after this week. It wont be significantly faster than nnedi2 :)

[Revgen]

^Sorry to here that. Hopefully, somebody else can optimize it after you release the source.

[Undead Sega]

ohhh an NNEDI3! :D we all amazed by wat NNEDI2 can do, so wat will be the main difference in visual quality between the two? samples the whole image instead of edge or something? :D

[Undead Sega]

Apologize for that nonsensical post there, but i would like to ask, is it possible to use nnedi2_rpow2 on an MTmode or somesort?

[um3k]

I don't see why it wouldn't be. I imagine SetMTMode(2, 0) would work just fine, assuming you don't have any conflicting filters. Have you tried it?

[Undead Sega]

well my script so far is:

LoadPlugin("C:\Downloads\VirtualDub-1.8.6\Requirements\nnedi2\nnedi2.dll")
Import("C:\Downloads\VirtualDub-1.8.6\TempGaussMC_beta1mod.avsi")
AviSource("D:\***.avi")
ConvertToYV12()
nnedi2_rpow2(rfactor=4)
Spline36resize(720,576)

and i think i have the latest Avisynth installed, is there anything special i need to install or add perhaps?

[scharfis_brain]

nnedi2 already is MT. what's the point in trying to MT it further?

[Terka]

Tritical,
"I'm actually close to an nnedi3"...
Sorry for my bad english. Does this mean you will continue working on nnedi3?
Will be the final speed +- like nnedi2?
Will you work on nnedi3 to improve the results?
Thank you, Terka

[kool]

I apology for my very noobi questions.
is the nnedi resizer meant for upsizing & downsizing?
is it possible to use nnedi2_rpow2 resize for 720p & 1080p?
what would be the rfactor for 1080p (1920x816) out of (1920x1080 blu-ray)

[julius666]

I apology for my very noobi questions.
is the nnedi resizer meant for upsizing & downsizing?
is it possible to use nnedi2_rpow2 resize for 720p & 1080p?
what would be the rfactor for 1080p (1920x816) out of (1920x1080 blu-ray)

It's only for upsizing. And it's most useful when the resolution that you are aiming for is near to 2× of the original resolution.
In your case I would simply use spline36resize.
Oh, and :search:

[Inspector.Gadget]

kool, if you're keeping the hoizontal resolution the same, that suggests you want to crop, rather than resize.

[kool]

IG, do you mean that I don't resize at all only crop the 1920x1080 > 1920x816 and encode it?

[Inspector.Gadget]

Yup. 1920x1080 BDs are 1:1 PAR, no need to resize. Don't crop before deinterlacing/IVTC, though, if your source is 1080i true interlaced or hard telecine. To avoid taking this thread further off topic from nnedi2, make a new thread about anything you can't find an answer for and we'll help you there.

[kool]

OK, thank you very much.

[Dogway]

See if this: nnedi2.dll (http://bengal.missouri.edu/~kes25c/nnedi2.dll) works on that processor.

Thanks for that, if it wasnt for your input I would still be "guessing" whats wrong with my script... and my CPU is an intel core 2 duo (T9400)

By the way, do you know if theres a problem on TempGaussMC when using qual=above 1? It makes crash my encoding on 2nd pass...

[Stephen R. Savage]

@tritical: Since JoshyD seems to be serious about developing AVS64, perhaps you could make a 64-bit build of nnedi2? Since it's closed-source, I can't ask him to undertake the task! Then again, it seems that the use of threading is incompatible with JoshyD's AVS64.

[Terka]

Tritical,
are you still working on nnedi3, or is it a dead project?

[tritical]

I am still working on it. Training a 150000 parameter model on 50 million 288 dimensional vectors takes a little while even using stochastic gradient descent. I will post some pictures and a test dll sometime soon.

EDIT: Actually, I'm kind of exaggerating. The training only takes 4-5 days, and that is the largest model. The smaller models (smaller local neighborhood around center pixel) only take 1/6-1/2 that time. Most of the time since February (up to about 1 week ago) was been spent tweaking the model structure and making the training algorithm faster and more memory efficient in order to use more training data. Extension to 3D is only a trivial change at this point. I'm planning to go that route once I have all of the 2D models trained.

[Terka]

Thank you, T. :cool: 3D! :cool:

[wonkey_monkey]

Extension to 3D is only a trivial change at this point.

Oh, hey, what? Have I not been paying attention closely enough? Is a motion interpolating NNEDI in the pipeline?

David

[tritical]

It isn't that advanced. It's sort of like training a model to do motion-adaptive deinterlacing... giving it the 3D local neighborhood of a pixel instead of just the 2D neighborhood. Motion compensation could be performed separately beforehand to create a truly (large scale motion) motion compensated deinterlacer. I said trivial, but it will actually take a while to create new training/testing data sets, run experiments, etc... But I am fairly convinced that nnedi3 exploits the spatial domain for all it's worth (without introducing higher level processing like object recognition/scene understanding)

[tritical]

Here is a beta version: nnedi3.dll (http://bengal.missouri.edu/~kes25c/nnedi3.dll). I hacked it together quickly from nnedi2 so please report any problems.

Parameters are the same as nnedi2/nnedi2_rpow2, but named nnedi3/nnedi3_rpow2. There are two differences:

1.) 'nsize' now controls the size of the 2D neighborhood. Possible settings for now (default is 2):

0: 8x6
1: 16x6
2: 32x6
3: 48x6

First value is the horizontal diameter. Second is the vertical diameter. Larger horizontal diameter allows connecting lines of smaller slope, but also has higher probability of artifacts. I would recommend always using 0 for image enlarging.

2.) 'qual' can only be set to 1 or 2. It works the same as in nnedi2 (default is 1).

In the future there will be more neighborhood size options, and a new parameter 'nns' to select model complexity (the number of neurons) for a fixed 'nsize' value. All of the models in the beta dll use nns=256, which is the largest I will probably go. Smaller values in the future will allow trading quality for speed.

Further notes. There is only one nsize=3 model in the beta dll so qual=2 w/ nsize=3 will give the same result as qual=1. That nsize=3 model is also not the final version as it was only trained on a small data set for test purposes. It will be replaced in the future (which should give better results).

[aegisofrime]

tritical, I don't know if it's just me but I'm getting a 0 byte file when I download nnedi3.dll.

[AVIL]

I've downloaded a 0 byte dll too.

[D3C0D3R]

Huge RESPECT to Tritical. I've use NNEDI (EEDI) 1,2 + TFM + TDecimate about 5 years.
Now when i regged on forum, I would like to thank for your great work.

as link for nnedi3 still not works, I wanna ask How has the speed changed compare to nnedi2?

[tritical]

Sorry, should work now. I put up some image enlargements here http://bengal.missouri.edu/~kes25c/nnedi2_images (using nsize=0/qual=2). Right now speed is slower than nnedi2 except for maybe nsize=0. For reference nnedi2 uses a 12x4 local neighborhood. The speed is basically linearly dependent with local neighborhood size and with nns. It should get better when more nsize options are available and smaller nns values (64 and 128) are available. Going to nns=128 is twice as fast as 256, and shouldn't result in much if any quality loss based on tests so far.

[osgZach]

Those pics look pretty nice, wow..

I'm beginning to wonder if this might be useful for upsizing some Final Fantasy VII (PC) assets to try and correct some of the idiocy that happened when they ported it.. (low res movies.. not holding out much hope for those, but the low res rendered backdrops could maybe be helped)

[markanini]

Fantastic! Can't wait to use nnedi3 for image upsizing and deinterlacing.

[jpsdr]

I've made a test with nnedi3 (for upsizing), with nsize=0/qual=2, speed is around 3 time slower than nnedi2 with nsize=2 (default)/qual=3...

[D3C0D3R]

i've made quick tests too.
on lines looks like eedi2, but better (no artifacts, smoother in good meaning)
little bit sharper than nnedi2, faster than incredibly slow eedi3, but still - slow.

source - interlaced. i've deint it & upscale

nnedi2(field=-1,nsize=2,qual=1)
nnedi2_rpow2(rfactor=2, cshift="spline64resize",nsize=2,qual=1)
http://i43.tinypic.com/25a7ygw.jpg

nnedi3(field=-1,nsize=2,qual=1)
nnedi3_rpow2(rfactor=2, cshift="spline64resize",nsize=2,qual=1)
http://i41.tinypic.com/258wjyd.jpg

nnedi2
http://i40.tinypic.com/352ivd5.jpg

nnedi3
http://i40.tinypic.com/99gtc1.jpg

i've got 7 fps by nnedi2 script & about 1 fps by nnedi3.
on my C2Duo @ 4Ghz. but looks very good & finally on nnedi we have excelent lines.

[Frank K Abbott]

May i ask why you want to upscale an SD source by so much?

[D3C0D3R]

not so much - just only in 2 times.
read this thread. it upscales only powers of 2.
and in real script after nnedi I've always use spline64resize to downscale imgae to resonable resolution.

[julius666]

nnedi3(field=-1,nsize=2,qual=1)
nnedi3_rpow2(rfactor=2, cshift="spline64resize",nsize=2,qual=1)
http://i43.tinypic.com/25a7ygw.jpg

nnedi2(field=-1,nsize=2,qual=1)
nnedi2_rpow2(rfactor=2, cshift="spline64resize",nsize=2,qual=1)
http://i41.tinypic.com/258wjyd.jpg


I'm pretty sure that the pictures you posted are in wrong order.

[D3C0D3R]

I'm pretty sure that the pictures you posted are in wrong order.
yeah, you are right :thanks:

[Skauneboy]

How is nnedi3 coming along tritical? I'm looking forward to the addition of nns options.

[tritical]

I put up a new version at the same link. It adds an 'nns' parameter, which can be set to 0 (64), 1 (128), or 2 (256, only available setting in first version). Default is 1. Based on my observations the difference between 1 and 2 is usually small, and 1 is roughly twice as fast. 0 starts to noticeably sacrifice quality. In terms of MSE on my test data, 2 (at all nsize values) is better than 1, and 1 is better than 0, but my perceived visual quality doesn't always follow those numbers. I've started training new nsize options: 8x4, 16x4, 32x4, 48x4, which should be ~33% faster than the x6 versions. Not expecting much difference MSE wise or visually there.

[julius666]

Thanks a lot tritical, it's really faster with nns=1, and I can't see any difference between nns=2 and 1 (although I can't see between 0 and 1 either - I'm using nnedi3 for resizing). :cool:

Extension to 3D is only a trivial change at this point. I'm planning to go that route once I have all of the 2D models trained.

Will you do it? And what effects would it have? "Just" less aliasing artifacts, or would it be some kind of superresolution?

[thanhtu5013]

Nice job Tritical.
This one works a lot faster than your previous beta release.

Not fast as Nnedi2, obviously.

[Skauneboy]

Thanks for your awesome work tritical. Your interpolators have become quite essential for many of us Avisynthers. :thanks:

[Undead Sega]

Motion compensation could be performed separately beforehand to create a truly (large scale motion) motion compensated deinterlacer.

Does this exist in the Avisynth world?

TGMC, MCBob by any chance?

[tritical]

New version up... with help file and link on my site finally. I also released version 1.5 of nnedi2 which is just 1.4 with the paddq fix for sse.

The new version of nnedi3 adds another nns option: nns=0 (32). nns=0/1/2 from the previous version are now 1/2/3. It also adds nsize=4 (8x4). The defaults for nsize/nns when using nnedi3() are now 2/1 (so should be faster than previous version - 2/2). When using nnedi3_rpow2 the defaults are 0/3.

Will you do it? And what effects would it have? "Just" less aliasing artifacts, or would it be some kind of superresolution?
I will eventually. The basic idea is the same as that of tdeint or any other motion-adaptive deinterlacer. Namely, weave in areas of no motion and interpolate in areas of motion. The difference is that the weave/interpolate reasoning would be via neural network fed with the local 3D neighborhood instead of human coded rules/logic (like eedi2 vs nnedi3 for interpolation).

Does this exist in the Avisynth world?

TGMC, MCBob by any chance?
Not that I'm aware. I know that Terka was working on this idea or something similar a few months ago, but I don't know what came of it. The idea being to combine a motion-adaptive deinterlacer with mvtools to make it motion-compensated... the same way you can combine fft3dfilter/dfttest with mvtools to make a motion-compensated denoiser.

[LoRd_MuldeR]

New version up... with help file and link on my site finally. I also released version 1.5 of nnedi2 which is just 1.4 with the paddq fix for sse.

The new version of nnedi3 adds another nns option: nns=0 (32). nns=0/1/2 from the previous version are now 1/2/3. It also adds nsize=4 (8x4). The defaults for nsize/nns when using nnedi3() are now 2/1 (so should be faster than previous version - 2/2). When using nnedi3_rpow2 the defaults are 0/3.

:thanks:

[Terka]

Tritical, dont want to make this thread longer, but nevermind, i must say: thank you! Your filters are one of the top.

[Terranigma]

Thanks for the first official release. :thanks:
I guess there's no need to Update mcbob again at the moment... not just yet, but TempGaussMC otoh..... :)


btw, wouldn't it be fitting to update this thread's title to something more fitting such as "nnedi3" or "nnedi2/nnedi3" to place emphasis upon the release? :p

[Undead Sega]

Not that I'm aware. I know that Terka was working on this idea or something similar a few months ago, but I don't know what came of it. The idea being to combine a motion-adaptive deinterlacer with mvtools to make it motion-compensated... the same way you can combine fft3dfilter/dfttest with mvtools to make a motion-compensated denoiser.

Hmmm, thats very interesting, i wonder what has had happened to that. However, i thought that a full scale motion compensation deinterlacer existed (maybe as a fraction) in the Avisynth world, seems im wrong on that.

But if one was to make one, it could just put an end to all other deinterlacing filter for those who want high quality outputs. At the same time, (even though im not sure how it works) but the algorithm of this motion-compensated deinterlacer can be used for other purposes as well, one example could be that we could finally have a true 'Super-resolution' filter.

As ambitious i may be talking, the one thing i believe wat would let this down is the speed, however if this was made to fully work with 64-bit systems and even CUDA, this would be a small minor for some or many. So if one was to make one, i would suggest that it should be made standalone but based on Avisynth. I hope you get what i mean :D

[daimroc]

I am trying to use nnedi3, with the following parameters:

nnedi3(-1, false, true, true, true, 4, 3, 1, true, 0, 0)


But when I open the avs with virtualdubmod i get this error:

Invalid arguments to function nnedi3.

What am I doing wrong? I have readed the documentation and it seems to be all right.



Thanks.
Daimroc.

[Gavino]

I am trying to use nnedi3, with the following parameters:
nnedi3(-1, false, true, true, true, 4, 3, 1, true, 0, 0)
But when I open the avs with virtualdubmod i get this error:
Invalid arguments to function nnedi3.
When you get the "Invalid arguments ..." error and the arguments seem to be right, it's usually because the input clip is not explicitly specified, and there is no 'last' clip to use at that point in the script.

So the error is likely to be in the line(s) preceding the call to nnedi3.
As always, you need to :script:

[daimroc]

True. The line when I open the video was commented. I forget delete "#".



Thanks.
Daimroc.

[Bi11]

:thanks: for releasing an optimized version of nnedi3, tritical.

A little late, but here is an early version: eedi3 (http://bengal.missouri.edu/~kes25c/eedi3/eedi3.dll). Like I said, it is SLOW. Atm it isn't optimized at all... no asm, no threading, not even on the algorithmic level. It's just proof of concept stage.Is there a chance you could release an optimized version of eedi3 any time soon?

nnedi3 gives very similar results to eedi3, but I find eedi3 is much more aggressive at connecting straight lines and curves.

I've noticed eedi3 stomps out every last bit of combing and flickering, resulting in very smooth, fluid, flicker-free motion (both object motion and camera movement) when used with TGMC by inserting the following code (it works, but I'm not sure if it's the best way to do it):
\ : (EdiMode=="EEDI3") ? clp.eedi3(field=-2, sclip=clp.nnedi3(field=-2))
TGMC is already slow, but currently eedi3 seems at least an order of magnitude slower than nnedi3. At that rate, even a partially optimized version of eedi3 would make a big difference.

[Terka]

Bi11, can you post some examples with comparsion to nnedi2, where the
clp.eedi3(field=-2, sclip=clp.nnedi3(field=-2))
looks better then nnedi2?

[Undead Sega]

Hmmm, thats very interesting, i wonder what has had happened to that. However, i thought that a full scale motion compensation deinterlacer existed (maybe as a fraction) in the Avisynth world, seems im wrong on that.

But if one was to make one, it could just put an end to all other deinterlacing filter for those who want high quality outputs. At the same time, (even though im not sure how it works) but the algorithm of this motion-compensated deinterlacer can be used for other purposes as well, one example could be that we could finally have a true 'Super-resolution' filter.

As ambitious i may be talking, the one thing i believe wat would let this down is the speed, however if this was made to fully work with 64-bit systems and even CUDA, this would be a small minor for some or many. So if one was to make one, i would suggest that it should be made standalone but based on Avisynth. I hope you get what i mean :D

Anything wrong with what I've said? :(

[Bi11]

Bi11, can you post some examples with comparsion to nnedi2...From my observations, the examples posted by tritical accurately depict the aggressiveness of eedi3 towards connecting straight lines and curves. That difference extends to TGMC as well.


Some example images:
http://bengal.missouri.edu/~kes25c/eedi3/soccer_original.png
http://bengal.missouri.edu/~kes25c/eedi3/soccer_eedi2.png
http://bengal.missouri.edu/~kes25c/eedi3/soccer_nnedi2.png
http://bengal.missouri.edu/~kes25c/eedi3/soccer_eedi3+nnedi2.png

http://bengal.missouri.edu/~kes25c/eedi3/bbcourt_original.png
http://bengal.missouri.edu/~kes25c/eedi3/bbcourt_eedi2.png
http://bengal.missouri.edu/~kes25c/eedi3/bbcourt_nnedi2.png
http://bengal.missouri.edu/~kes25c/eedi3/bbcourt_eedi3+nnedi2.png

[Gser]

\ : (EdiMode=="EEDI3") ? clp.eedi3(field=-2, sclip=clp.nnedi3(field=-2))

Or perhaps
\ : (EdiMode=="EEDI3") ? clp.eedi3(field=-2, sclip=clp.nnedi3(field=-2,nsize=nsize,nns=nns,qual=qual))
Based on beta2u. Turns out that EEDI3 doesn't have a maxd value.

[Terka]

yes. these pictures are quite special cases.
the fence for example is better using nnedi2.
it depends on the source. nnedi3 looks lets say more sharp, but sometimes 'unnatural'

[thanhtu5013]

I could see a lot of guys using nnedi3 for deinterlacing or anything else but it seems there has not a lot of guys using it for anti-aliasing...

Does someone using nnedi3 for anti-aliasing function ?
On an another topic, there has one guy that help me to create a script for anti-aliaising.

Function santiag (clip c, int "strh", int "strv")
{
strh = Default (strh, 1)
strv = Default (strv, 1)

c
santiag_dir (strh)
TurnLeft ()
santiag_dir (strv)
TurnRight ()
}

Function santiag_dir (clip c, int strength)
{
c
w = Width ()
h = Height ()
santiag_stronger (strength)
Spline36Resize (w, h, 0, -0.5, w, h * 2)
}

Function santiag_stronger (clip c, int strength)
{
c
(strength <= 0)
\ ? NNEDI3 (dh=true, field=0)
\ : santiag_stronger (strength - 1)
\ . NNEDI3 (dh=false, field=(strength % 2))
}

AVISOURCE("C:\Video Test.avi")

santiag()
I'm satisfied with AA when the video doesn't running ( immovable ) and when it's about taking screenshots but when the video runs ( in movement ), it looks a bit strange...
Does someone could help me to improve anti-aliasing function if it's possible ? :)

[Mystery Keeper]

For some of my footage pure nnedi3 works better than TempGaussMC_beta2u(truemotion=true,EdiMode="NNEDI3").SelectEven(). But it has flickering that TGMC fights. Is there a way to soothe the flickering outside deinterlacing?

I could see a lot of guys using nnedi3 for deinterlacing or anything else but it seems there has not a lot of guys using it for anti-aliasing...

Does someone using nnedi3 for anti-aliasing function ?
On an another topic, there has one guy that help me to create a script for anti-aliaising.

The more sharp is interpolator - the less it is usable for antialiasing. That's why EEDI2 does it better than nnedi.

[thanhtu5013]

The more sharp is interpolator - the less it is usable for antialiasing. That's why EEDI2 does it better than nnedi.
nnedi3 is very efficient about anti-aliasing for screenshots (immovable) but on videos in movement, it looks a bit strange due to flickering. It's this point exactly what I'm looking for improve.
I never used EEDI2 and I'm still a newbie with AviSynth. Someone can write a script to me for anti-aliasing function ? :)

[Undead Sega]

Hmmm, thats very interesting, i wonder what has had happened to that. However, i thought that a full scale motion compensation deinterlacer existed (maybe as a fraction) in the Avisynth world, seems im wrong on that.

But if one was to make one, it could just put an end to all other deinterlacing filter for those who want high quality outputs. At the same time, (even though im not sure how it works) but the algorithm of this motion-compensated deinterlacer can be used for other purposes as well, one example could be that we could finally have a true 'Super-resolution' filter.

As ambitious i may be talking, the one thing i believe wat would let this down is the speed, however if this was made to fully work with 64-bit systems and even CUDA, this would be a small minor for some or many. So if one was to make one, i would suggest that it should be made standalone but based on Avisynth. I hope you get what i mean :D

Anyone at all may i ask? :(

[Didée]

i thought that a full scale motion compensation deinterlacer existed (maybe as a fraction) in the Avisynth world, seems im wrong on that.

What is a "full scale motion compensation deinterlacer"?

Or, what is it that e.g. TGMC (or MCBob, or MVBob, or..or..or..) doesn't qualify?


How to answer a question when the intend is not clear.

[Undead Sega]

It isn't that advanced. It's sort of like training a model to do motion-adaptive deinterlacing... giving it the 3D local neighborhood of a pixel instead of just the 2D neighborhood. Motion compensation could be performed separately beforehand to create a truly (large scale motion) motion compensated deinterlacer. I said trivial, but it will actually take a while to create new training/testing data sets, run experiments, etc... But I am fairly convinced that nnedi3 exploits the spatial domain for all it's worth (without introducing higher level processing like object recognition/scene understanding)

I think i might have misquoted, but it was something that tritical mentioned on April Fool's Day.

[Didée]

Tritical's post wasn't a joke, makes perfect sense, and I don't see how your answer is related to what I have asked you above.

"What color has the wind" is difficult to answer.

If you want an answer, pose a clear question.

[Undead Sega]

I originally asked if there was such thing as a large scale motion motion compensated deinterlacer which i thought was TGMC or MCBob, but it turns out they're not?

Therefore thats why i made this post:

Hmmm, thats very interesting, i wonder what has had happened to that. However, i thought that a full scale motion compensation deinterlacer existed (maybe as a fraction) in the Avisynth world, seems im wrong on that.

But if one was to make one, it could just put an end to all other deinterlacing filter for those who want high quality outputs. At the same time, (even though im not sure how it works) but the algorithm of this motion-compensated deinterlacer can be used for other purposes as well, one example could be that we could finally have a true 'Super-resolution' filter.

As ambitious i may be talking, the one thing i believe wat would let this down is the speed, however if this was made to fully work with 64-bit systems and even CUDA, this would be a small minor for some or many. So if one was to make one, i would suggest that it should be made standalone but based on Avisynth. I hope you get what i mean :D

probably talking ambitiously or nonesense (which i doubt), i was wondering if such thing could be done.

[Didée]

Perhaps nobody answers your question because you already made it into everybody's ignore list, so nobody is even seeing your posts at all.

And seriously,
I originally asked if there was such thing as a large scale motion motion compensated deinterlacer which i thought was TGMC or MCBob, but it turns out they're not?
you're not far from making it into mine, too.


What tritical posted basically refers to a method that we're using in Avisynth since many years already. Just that the usual "decision elements" (mostly threshold-driven) would be replaced by neural network decisions.

How come "large scale" into the game? I don't see what your talking about.


QUESTION:

What do you think that a "large scale motion compensated deinterlacing" should be?


Trick or treat. Answer or IgnoreList.

[Guest]

I think he's just saying that the weave/interpolate logic could be done by neural net, as suggested *ALREADY* by tritical. His small addition is to suggest using CUDA for that. We all know of the high difficulty of coding CUDA. Undead should stop banging this drum and implement it if he thinks it's really a feasible idea. His reposting and bumping the same message is getting tiresome and could incur strikes.

[markanini]

A nerual-net based de-interlacer sounds tasty!

Anyway, big thanks trictical for NNEDI3 and the nsize option which works great for tuning aliasing vs resolution.

[D3C0D3R]

Just that the usual "decision elements" (mostly threshold-driven) would be replaced by neural network decisions.
now i clearly understand, what tritical meant. thnx.
that means further improving mvTools alhorithms can make deinterlacers quality significantly better

[PhrostByte]

How would one correct for the center shift manually? I need to not only resize, but crop after using nnedi2 so I'd like to do it all in one go.

[daimroc]

I don't see a post for nnedi3, but I see here there is some questions about nnedi3, so I use this post.

Well, I am noticed in some videos that when I use nnedi3, in the top and in the bottom I see that crop an uncrop one or two pixels, and it is a bit disturb, because It's like the image tremble in the top and int the bottom. The rest of the image it's perfect.

I play the video without deinterlacing it, and this effect doesnt happend, so the problem it seems nnedi3.

If I use the internal yadif of ffdshow this effect doesnt happend.

I use nnedi3 in the following way:

nnedi3(field=3, dh=false, Y=true, U=true, V=true, nsize=4, nns=3, qual=1, pscrn=true, threads=0, opt=0)

I try with nsize = 0 too and field = 2, although I know the video is top field first.



Thanks.
Daimroc.

[foxyshadis]

How would one correct for the center shift manually? I need to not only resize, but crop after using nnedi2 so I'd like to do it all in one go.

For YV12:
mergechroma(spline36resize(w,h,src_left=-.5,src_top=-.5,src_width=w,src_height=h), \
spline36resize(w,h,src_left=-1.0,src_top=-1.0,src_width=w,src_height=h))

Modify the src_ values to do your cropping. w = final width after crop, same for h.

[Gavino]

mergechroma(spline36resize(w,h,src_left=-.5,src_top=-.5,src_width=w,src_height=h), \
spline36resize(w,h,src_left=-1.5,src_top=-1.5,src_width=w,src_height=h))
nnedi2_rpow2 already corrects the chroma shift automatically (see posts #36 and #38), so shouldn't this just be
spline36resize(w,h,src_left=-.5,src_top=-.5,src_width=w,src_height=h)

(And in any case I think the correct shift would be -1.0, not -1.5)

[foxyshadis]

He's not using chromashift, he specifically asked for a way to do it manually so it doesn't need a shift+crop or double shift.

Also, yes, you're correct about the 1.0, I'll fix that. Damn, I've been using -1.5 for weeks now.

[jpsdr]

Is an x64 version of the pluggin scheduled ?

[Gavino]

He's not using chromashift, he specifically asked for a way to do it manually so it doesn't need a shift+crop or double shift.
If using nnedi2 directly (with dh=true) rather than nnedi2_rpow2, note that the sign of the required shifts will depend on the value of the field parameter and, for the horizontal shift, whether you use TurnRight or TurnLeft before calling nnedi2 the second time.

The values you gave (assuming 1.5 is corrected to 1.0) correspond to using field=1 and intermediate TurnRight.

[Dogway]

Any news about eedi3 optimization, at least multithreading? It has been 7 months since last update, I like it because it works great for agressive (animation) aliasing.

[tritical]

I put up a new version of eedi3 http://bengal.missouri.edu/~kes25c/eedi3.zip. Source is included. It is multithreaded using openmp. I had made some changes to the defaults (alpha/beta) and reworked the reliability checking (vcheck). From the readme:
Info:

eedi3 works by finding the best non-decreasing (non-crossing) warping between two lines by minimizing a cost functional.
The cost is based on neighborhood similarity (favor connecting regions that look similar), the vertical difference created
by the interpolated values (favor small differences), the interpolation directions (favor short connections vs long), and
the change in interpolation direction from pixel to pixel (favor small changes).


Functions:


"eedi3", "c[field]i[dh]b[Y]b[U]b[V]b[alpha]f[beta]f[gamma]f[nrad]i
[mdis]i[hp]b[ucubic]b[cost3]b[vcheck]i[vthresh0]f[vthresh1]f[vthresh2]f[sclip]c[threads]i"

"eedi3_rpow2", "c[rfactor]i[alpha]f[beta]f[gamma]f[nrad]i[mdis]i[hp]b[ucubic]b
[cost3]b[vcheck]i[vthresh0]f[vthresh1]f[vthresh2]f[cshift]s[fwidth]i[fheight]i[ep0]f[ep1]f[threads]i"


Parameters:

field/dh/y/u/v are the same as nnedi2/nnedi3.

rfactor/cshift/fwidth/fheight/ep0/ep1 are the same as nnedi2_rpow2/nnedi3_rpow2.


alpha/beta/gamma (defaults: 0.2,0.25,20.0):

These trade off line/edge connection vs artifacts created. alpha and beta must be in the range [0,1], and the sum
alpha+beta must be in the range [0,1]. alpha is the weight given to connecting similar neighborhoods.. the larger
it is the more lines/edges should be connected. beta is the weight given to vertical difference created by the
interpolation... the larger beta is the less edges/lines will be connected (at 1.0 you get no edge directedness at all).
The remaining weight (1.0-alpha-beta) is given to interpolation direction (large directions (away from vertical)
cost more)... so the more weight you have here the more shorter connections will be favored. Finally, gamma penalizes
changes in interpolation direction, the larger gamma is the smoother the interpolation field between two lines
(range is [0,inf].

If lines aren't getting connected than increase alpha and maybe decrease beta/gamma. Go the other way if you are
getting unwanted artifacts.


nrad/mdis (defaults: 2,20):

nrad sets the radius used for computing neighborhood similarity. Valid range is [0,3]. mdis sets the maximum
connection radius. Valid range is [1,40]. If mdis=20, then when interpolating pixel (50,10) (x,y), the farthest
connections allowed would be between (30,9)/(70,11) and (70,9)/(30,11). Larger mdis will allow connecting lines
of smaller slope, but also increases the chance of artifacts. Larger mdis will be slower. Larger nrad will be
slower.


hp/ucubic/cost3 (defaults: false, true, true):

These are speed vs quality options. hp=true, use half pel steps, hp=false, use full pel steps. ucubic=true, use
cubic 4 point interpolation, ucubic=false, use 2 point linear interpolation. cost3=true, use 3 neighborhood cost
function to define similarity, cost3=false, use 1 neighborhood cost function.


vcheck/vthresh0/vthresh1/vthresh2/sclip (defaults: 2, 32.0, 64.0, 4.0, NULL):

vcheck settings:

0 - no reliability check
1 - weak reliability check
2 - med reliability check
3 - strong reliability check

If vcheck is greater than 0, then the resulting interpolation is checked for reliability/consistency. Assume
we interpolated pixel 'fh' below using dir=4 (i.e. averaging pixels bl and cd).

aa ab ac ad ae af ag ah ai aj ak al am an ao ap
eh el
ba bb bc bd be bf bg bh bi bj bk bl bm bn bo bp
fd fh fl
ca cb cc cd ce cf cg ch ci cj ck cl cm cn co cp
gd gh
da db dc dd de df dg dh di dj dk dl dm dn do dp

When checking pixel 'fh' the following is computed:

d0 = abs((el+fd)/2 - bh)
d1 = abs((fl+gd)/2 - ch)

q2 = abs(bh-fh)+abs(ch-fh)
q3 = abs(el-bl)+abs(fl-bl)
q4 = abs(fd-cd)+abs(gd-cd)

d2 = abs(q2-q3)
d3 = abs(q2-q4)

mdiff0 = vcheck == 1 ? min(d0,d1) : vcheck == 2 ? ((d0+d1+1)>>1) : max(d0,d1)
mdiff1 = vcheck == 1 ? min(d2,d3) : vcheck == 2 ? ((d2+d3+1)>>1) : max(d2,d3)

a0 = mdiff0/vthresh0;
a1 = mdiff1/vthresh1;
a2 = max((vthresh2-abs(dir))/vthresh2,0.0f)

a = min(max(max(a0,a1),a2),1.0f)

final_value = (1.0-a)*fh + a*cint


** If sclip is supplied, cint is the corresponding value from sclip. If sclip isn't supplied,
then vertical cubic interpolation is used to create it.


threads (default: 0):

Sets the number of threads used by openmp.

0 = default (environment variable OMP_NUM_THREADS)
> 0 = calls omp_set_num_threads(threads)

Any other optimization would not be very easy so I'm not going to try :).

[Gser]

AWESOME! I've been waiting for this.

[LoRd_MuldeR]

Are we still supposed to use it like this:
eedi3(sclip=nnedi2())
:confused:

[Bi11]

Thanks a lot for the multithreaded EEDI3! :thanks:

Now I can do fine deinterlacing using supersampling with TGMC (in reasonable time :devil:), like so:
\ : (EdiMode=="EEDI3_SS") ? clp.eedi3(field=-2, sclip=clp.nnedi3(field=-2)).eedi3_rpow2(rfactor=2, cshift="spline64resize").eedi3().spline64resize(clp.width, clp.height)

The output is very smooth, having continuous curves with no trace of combing, and is virtually flicker-free, which makes for an easily compressible encode.
It's as though the supersampled use of eedi3 acted as a "precision denoiser" - reconstructing lines if pixels can connect, else it blurs those "random" pixels - resulting in a rather stable clip.
Makes me wonder how to configure eedi3 into a general "edge reconstructor + denoiser".

Are we still supposed to use it like this:
eedi3(sclip=nnedi2())
:confused:

final_value = (1.0-a)*fh + a*cint


** If sclip is supplied, cint is the corresponding value from sclip. If sclip isn't supplied,
then vertical cubic interpolation is used to create it.
In the few videos I tested, sclip=nnedi3() gave less artifacts and connected lines better than not using it, though the actual difference is small.

[tritical]

Are we still supposed to use it like this: eedi3(sclip=nnedi2())

The defaults for vcheck/vthresh0/vthresh1/vthresh2 are configured with that mind. Probably the most important is vthresh2. That literally controls whether to trust eedi3 or sclip (cubic interpolation if sclip isn't given) for small directions (vertical/large slope lines). This part of the explanation for vcheck:

a2 = max((vthresh2-abs(dir))/vthresh2,0.0f)

a = min(max(max(a0,a1),a2),1.0f)
final_value = (1.0-a)*fh + a*cint


Here, dir is literally the inverse slope, as detected by eedi3, at the current pixel location. You can also think of it as the direction of interpolation. So if dir=3, eedi3 is going to form the current pixel by connecting the pixel at x+3 from the line above and x-3 from the line below. If dir was -6 it would be formed by connecting x-6 from the line above and x+6 from the line below. So with vthresh2=4.0 (the default), and ignoring a0/a1, eedi3's interpolated value will get all weight for directions >= 4.0. At 0.0 the sclip value will get all weight. For directions in between you get a weighted average. The reason is I trust nnedi2/nnedi3 more for small directions than eedi3 (the reason is most image structures that can't be fit by eedi3's interpolation model will have small directions), but I trust eedi3 more for large directions. If you don't use sclip=nnedi3/nnedi2 then vthresh2 should really be adjusted... for example if sclip isn't given and cubic interpolation is used I would lower it to 3 or 2 probably... maybe 1. Also the other reliability checks determine whether eedi3's detected direction is accurate or not. If it's determined not to be accurate then sclip is the follow over (it does a weighted average between the two based on how inaccurate eedi3 is)... so the more accurate sclip is the more accurate the final output will be.

[Terka]

\ : (EdiMode=="EEDI3_SS") ? clp.eedi3(field=-2, sclip=clp.nnedi3(field=-2)).eedi3_rpow2(rfactor=2, cshift="spline64resize").eedi3().spline64resize(clp.width, clp.height)
got error if itry it.
eedi3_rpow2: error using env -> invoke (function not found)!

[foxyshadis]

spline64resize is part of avisynth 2.6 iirc. Use spline36resize if you're still using 2.5, or install newresizers plugin (http://www.mediafire.com/?rmmw22yynnq).

[Gavino]

Spline64Resize is also present in Avisynth 2.58, but not in 2.57 or the MT version derived from it.

[Terka]

Thank you, spline36 works! but nnedi2 is still better for me, both for quality and of course for speed.

[markanini]

what can one expect when running nnediX in eediX compared to pure nnediX? Would results be in-between nnedi and eedi?

[Great Dragon]

Now I can do fine deinterlacing using supersampling with TGMC (in reasonable time ),
It is slow as hell. It takes 2 mins to process one frame on my Q6600 with all 4 cores loaded for 100%.
But if you hold your breath and wait till it finished you'll be happily surprised about quality.

Default TGMC
http://i.imagehost.org/t/0474/eedi2.jpg (http://i.imagehost.org/view/0474/eedi2)
EEDI3_SS with nnedi2
http://a.imagehost.org/t/0648/eedi3.jpg (http://a.imagehost.org/view/0648/eedi3)

[Didée]

2 minutes per frame? Cool. That makes a processing time of only 83 days for one 40-minutes PAL clip. :D
___

Since that's Anime, it can be assumed the input is not of the category "video"-interlaced. And when it is not, then the number of different motion states (on the field-separated level) is not equal in forward and backward direction. As a consequence of this unsymmetry, the internal motion search of TGMC is not working optimally. (Due to the temporally-symmetric preprocessing.)

TGMC is intended for video-type interlaced sources, and nothing else. In most cases the internal safeguarding is good enough to avoid "obvious" errors when feeding sources with other types of interlacing. But you can't expect an optimal result when using a tool for something it wasn't designed for.

[Bi11]

Spline64Resize is also present in Avisynth 2.58, but not in 2.57 or the MT version derived from it.
I am using Avisynth 2.5.8 MT (http://forum.doom9.org/showthread.php?t=144852). So I guess it already has Spline64Resize built-in.

It is slow as hell. It takes 2 mins to process one frame on my Q6600 with all 4 cores loaded for 100%.
Yes, it is impractically slow. Using avs2avi clip.avs -o n, on a Core i5 750 @3.4GHz, with a 720x540 clip gives 0.13fps with setMTmode(2). Using eedi3_ss outside of TGMC gives 0.3fps, again with setMTmode(2).

Have you tried doing only one call to eedi3, like so:
\ : (EdiMode=="eedi3") ? clp.eedi3(field=-2, sclip=clp.nnedi3(field=-2))

It is much faster (10x as fast @1.3fps) and looks close to eedi3_ss, but without connecting "micro" jagged edges.
Scaling that rate, on an overclocked Core i7 860 w/ HT, processing a 40-minute PAL clip with EdiMode="eedi3" in TGMC would take 6.4 hours. :)

I think it should be possible to get the same effect as eedi3_ss without the supersampling, with only two consecutive calls to eedi3, or maybe first call to nnedi3 then second call to eedi3. The second call would configure eedi3 to be more aggressive at connecting very short lines.
I'll have to keep experimenting...

Update:
After some experimenting, it seems eedi3_rpow2 is entirely responsible for anti-aliasing jagged edged. So nnedi3 could be used for deinterlacing, which would give less artifacts than eedi3, and still look nearly identical to eedi3_ss, like so
\ : (EdiMode=="nnedi3+eedi3_ss") ? clp.nnedi3(field=-2,nsize=nsize,nns=nns,qual=qual).eedi3_rpow2(rfactor=2,cshift="spline64resize",fwidth=clp.width,fheight=clp.height)

Runs @0.25fps.

[Terka]

on DV content is nnedi2 better, so i suppose also on all dvd/blueray also.
for anime you say there nnedi3 wins. anime has a lot of flat areas.
maybee nnedi3 could be optimised - for flat areas use some speedy algoritm, for details current nnedi3. is something like this possible, Tritical?

[Didée]

maybee nnedi3 could be optimised - for flat areas use some speedy algoritm, for details current nnedi3. is something like this possible, Tritical?
This feature is already implemented, and used by default. Dive into the readme, and see the "pscrn" parameter.

[Great Dragon]

Have you tried doing only one call to eedi3, like so:
Thanks to your script example I did and I have to say it's MUCH faster and still do the job well. Actually now it's practically useful.
http://i.imagehost.org/t/0835/eedi3_single.jpg (http://i.imagehost.org/view/0835/eedi3_single)
This is still frame in the end of scene where default TGMC fails.

There is no such big difference in visual quality on "normal" frames but with EEDI3+NNEDI3 these looks a little more cleaner/sharper.

TGMC default
http://i.imagehost.org/t/0417/01_7.jpg (http://i.imagehost.org/view/0417/01_7)
EEDI3+NNEDI3 one call
http://j.imagehost.org/t/0995/02_4.jpg (http://j.imagehost.org/view/0995/02_4)

[D3C0D3R]

This feature is already implemented, and used by default. Dive into the readme, and see the "pscrn" parameter.
2 Tritical
can you implement this useful thing for much faster EEDI2 or add speedy option to EEDi3 (ex. using networks of smaller sizes)?

[WorBry]

This is still frame in the end of scene where default TGMC fails.


The last frame is always a dud. Obviously, one way to avoid it is to pad the end of the source clip with a few extra frames and excise afterwards.

[tritical]

can you implement this useful thing for much faster EEDI2 or add speedy option to EEDi3 (ex. using networks of smaller sizes)?

eedi2 already does this, but not using a neural network (it builds an edge mask). I could replace eedi2's edge mask building with nnedi2/nnedi3's prescreener, but I don't really want to work on on eedi2 anymore. Prescreening for eedi3 isn't as useful since it must process entire lines. How each pixel is interpolated is dependent on how every other pixel in the line contributes to the cost function. I could run the prescreener and not process any lines where it detects all pixels can use cubic, but I don't think it will save any significant time. If you want eedi3 to be faster there are really four settings to change from the defaults: mdis/nrad/cost3/ucubic. (ucubic=false,cost3=false,nrad=1,mdis=10) is 4-5x faster than the defaults, but of course the result isn't quite as good (and with mdis=10 lines of very small slope: < 1/10 wont be connected). As for smaller networks, neither eedi2/eedi3 use neural networks so there are none to make smaller. Once I get nsizes 16x4 and 32x4 added to nnedi3 I'm going to work on improving the speed/accuracy of the prescreener. I haven't touched the prescreener since the original nnedi.

The temporal version of nnedi3 is on hold for now.

[Great Dragon]

Obviously, one way to avoid it is to pad the end of the source clip with a few extra frames and excise afterwards.
If there are many scenes with such bad frames in a source clip it's a real pain.

[WorBry]

The 'dud frame' will (should) only occur once at the end of a processed clip, not at (internal) scene changes. Of course, if you are processing each scene separately, it becomes more tedious, but so is splitting and splicing anyway.

[plonk420]

all i can say is that eedi3+nnedi2 is AMAZING! :O

[hartford]

I got an error when trying to load EEDI3 with Avisynth v2.5.7:

The instruction at "0x012198f4" referenced memory at "0x011c3ca8". The memory could not be "read".

So, I updated to Avisynth 2.5.8 and got a simple error:

"LoadPlugin: unable to load "c:\program files\avsplugins\eedi3.dll"
(c:\..\blah.avs, line blah")

I have checked paths and syntax several times.


Script:

loadplugin("c:\program files\avsplugins\nnedi2.dll")
loadplugin("c:\program files\avsplugins\eedi3.dll") # the path is correct

avisource("blah.avi") # the avs file resides in the same directory as the avi

eedi3(sclip=nnedi2())



Any idea about what is causing the error?

Other stuff:

Windows XP sp2
eedi3 v0.9 for Avisynth 2.5x; file version 0.9.0.0; 24Jun10
nnedi2 v1.5 for Avisynth 2.5x; 16May2010

[tritical]

I didn't mention it, but since the last version uses openmp, and I compiled it with vs2005, you will need to have the visual studio 2005 redistributable installed (for vcomp.dll).

If you already have the redistributable then I'm not sure.

[hartford]

I did a search on my harddrive for vcomp.dll and did not find it.

I installed visual studio 2005 redistributable. I did a search for vcomp.dll
and found it in this directory:

c:\windows\x86_Microsoft.VC80.OpenMP_1fc8b3b9a1e18e3b_8.0.50727.42_x-ww_0ee63867

but I got the same error (LoadPlugin: unable to load "c:\program files\avsplugins\eedi3.dll").

Yes, I did reboot after installing the redistributable from MicroSoft.

Odd thing was that I didn't find it when I did a search of "c:\" but I did find it when I did
a search of "c:\Windows"

I don't know what that means except that paths were not established.

I'm open for suggestions.

I copied the dll to the system32 directory but no joy.


*edit for missing a copy-paste character*

[Bi11]

Remove the loadplugin() calls and place eedi3.dll in Avisynth 2.5/plugins folder.

If that doesn't work then:

Do a clean uninstall of AviSynth and clear the plugins directory. Reboot.
Do a default install of SEt's AviSynth MT 2.5.8 (http://forum.doom9.org/showthread.php?p=1412274#post1412274).
Copy eedi3.dll to the default AviSynth 2.5\plugins directory.

Run the script below (without the loadplugin()) to see if it works :
BlankClip(pixel_type="YV12")
eedi3()

[hartford]

@Bi11

<i>Remove the loadplugin() calls and place eedi3.dll in Avisynth 2.5/plugins folder.</i>

That did not work.

<i>If that doesn't work then:

Do a clean uninstall of AviSynth and clear the plugins directory. Reboot.
Do a default install of SEt's AviSynth MT 2.5.8.
Copy eedi3.dll to the default AviSynth 2.5\plugins directory.</i>

Doesn't work. I get an "Import error."

After putting the AviSynth MT dll in the system32 directory I tried to
register it without success. I get the error:

LoadLibrary("C:\WINDOWS\system32\avisynth.dll") failed - the specified module could not be found.

I'm totally perplexed. I've installed previous versions of avisynth MT with no problem.
I've tried EEDI2 just to see what it could do and had no problem.

So now I have EEDI3 and Avisynth MT not working.

Problems are compounding, but thanks for the suggestions.

[hartford]

Remove the loadplugin() calls and place eedi3.dll in Avisynth 2.5/plugins folder.

That did not work.

If that doesn't work then:

Do a clean uninstall of AviSynth and clear the plugins directory. Reboot.
Do a default install of SEt's AviSynth MT 2.5.8.
Copy eedi3.dll to the default AviSynth 2.5\plugins directory.

Doesn't work. I get an "Import error."

After putting the AviSynth MT dll in the system32 directory I tried to
register it without success. I get the error:

LoadLibrary("C:\WINDOWS\system32\avisynth.dll") failed - the specified module could not be found.

I'm totally perplexed. I've installed previous versions of avisynth MT with no problem.
I've tried EEDI2 just to see what it could do and had no problem.

So now I have EEDI3 and Avisynth MT not working.

Problems are compounding, but thanks for the suggestions.

[Gavino]

nnedi3 does not interpret the field parameter correctly when the input is RGB24.

With this script:
b = BlankClip(height=240, pixel_type="RGB24")
w = BlankClip(b, color=color_white)
Interleave(b, w)
AssumeFieldBased().AssumeTFF()
Weave()
nnedi3(field=0) # same rate, keep bottom field
the result should be all-white (bottom field), but in fact an all-black frame is produced.
If pixel_type is changed to "YUY2", a white frame is correctly obtained.
Similar problems occur when dh=true.

I suspect that the implementation does not allow for Avisynth storing RGB frames 'upside-down', so interprets top fields as bottom and vice-versa.

Edit: The same behaviour is also present in nnedi2.

[tritical]

Yep. I'll fix it in the next release. Should be in the next few days.

@hartford
Did you ever get things sorted out? I really have no idea what the problem might be.

[Gavino]

Yep. I'll fix it in the next release. Should be in the next few days.
Thanks, tritical.

Will this fix also change the behaviour of nnedi2/3_rpow2?
I notice that this produces opposite shifts for RGB and YUY2 when called without the cshift parameter. This isn't necessarily wrong, but seems anomalous.

[tritical]

Yeah, it'll make the rpow2 functions produce the same shifts when called without cshift. Seems when I added rgb24 support I changed the cshift behavior in the rpow2 functions to account for rgb24 being stored upside down, but forgot that field=0/1 would be switched when using nnedi2/3 alone.

[hartford]

@hartford
Did you ever get things sorted out? I really have no idea what the problem might be.

Sort of.

The Sourceforge version messed-up my computer; still not straight as to fonts. Don't understsand how that happened.

Anyway, with the "SEt install version" by Atak_Snajpera I get normal errors such as:

"Script error: there is no function named "eedi3"

If I open an avi without eedi3.dll then everything is normal.

So on my system with the SEt install by Atak_Snajpera "eedi3" is seen as a function rather than a dll.

This is so wierd.

Any suggestions to fix this are welcome.

This is on a WinXp sp2 computer.

[tritical]

Posted a new version of nnedi3. Changes:
+ add nsize=5/6 (16x4,32x4)
+ add nns=0 (16 neurons). nns 0/1/2/3 from v0.9 are now 1/2/3/4.
- new defaults. nnedi3: nsize=6,nns=1. nnedi3_rpow2: nns=3.
- fix field=0/1 flipped with rgb24 input

I'll post a new nnedi2 with the rgb24 fix sometime soon.

@hartford
So when you get "Script error: there is no function named "eedi3"", you are able to open the script if you comment out the eedi3() line? or only if you remove eedi3.dll from the plugins directory? Which version of the 2005 redistributable do you have? I think there are multiple versions due to various service packs. eedi3 should require the first one (not sp1 or sp2). Hm, actually I need to check that.

[SubJunk]

Thanks for the update, tritical

[Archimedes]

Regarding NNEDI3, for image enlargement, I tend to use nsize=4, because with nsize=0 I often got strange results (see example below).

Source, 400x300 >> (http://img69.imageshack.us/img69/6127/z3hs1n8ns1t.png)
nnedi3_rpow2 (rfactor=4, nsize=0, nns=3, qual=2, pscrn=True, cshift="Spline36Resize"), 1600x1200 >> (http://img338.imageshack.us/img338/4208/z3hs1n8ns1tnsize0.jpg)
nnedi3_rpow2 (rfactor=4, nsize=4, nns=3, qual=2, pscrn=True, cshift="Spline36Resize"), 1600x1200 >> (http://img707.imageshack.us/img707/9967/z3hs1n8ns1tnsize4.jpg)

Look at the pillars in the windows.

[tritical]

It's a result of the training data used. The first few nsize/nns combinations I generated used a smaller training set. Later I increased the training set size by 50%. I think the combinations were nsize=0,1,2 with nns=2,3,4. I'm re-generating those models, but it will probably take a week or so.

Also, eedi3 has the same bug with rgb24 input as nnedi2 has and nnedi3 had. I'll post a fix for that as well.

One more thing, the new defaults for nnedi3 should be much closer to default nnedi2 in terms of speed. For reference, nnedi2 uses a 12x4 neighborhood, and nsize=2 in nnedi2 is somewhere between nns=1 and nns=2 in nnedi3.

[hartford]

@hartford
Which version of the 2005 redistributable do you have?

That was the problem; I thought that I had installed it but did not.

Works fine after installing it.

Sorry for all the hub-bub.

Oh, I should say that eedi3 is damn fast on anime and fixing jaggies.

Thanks much for your efforts. I can't thank you enough.

And thanks to all on this thread who provided assistance; I'm grateful.

[tritical]

I put up new versions of eedi3/nnedi2 that fix the field=0/1 problem with rgb24 input.

[Taurus]

Which version of the 2005 redistributable do you have? I think there are multiple versions due to various service packs. eedi3 should require the first one (not sp1 or sp2). Hm, actually I need to check that.

http://img248.imageshack.us/img248/7789/errorca.png

The 2005 redistributable is installed (verified).
The eventviewer shows the errors even when eedi3 is not called in any script.
Del eedi3.dll in avisynth\plugins directory - no more errors...
This is on WinXP Pro SP3 32bit.

[kai0n7]

Hi Tritical,

I am also having the ""Script error: there is no function named "eedi3" problem.

I installed the VS2005 redistributable here: http://www.microsoft.com/downloads/details.aspx?familyid=32bc1bee-a3f9-4c13-9c99-220b62a191ee&displaylang=en

Not sure what the problem is. The latest versions of NNEDI2 and NNEDI3 work fine.

[DeathWolf]

For all those getting an error, the version of vcomp needed is not one commonly found, it's a dev version with a security fix.
Here's the good redist to get:
http://www.microsoft.com/downloads/details.aspx?FamilyID=766A6AF7-EC73-40FF-B072-9112BAB119C2&displaylang=en

@tritical: you might want to put that in the readme:)

[tritical]

Thanks for posting that link DeathWolf.

Part of the problem was the first mt version of eedi3 I posted needed the first vs2005 redist, but between that release and this last one I finally got my desktop computer, which has all my avisynth projects, online - about a year after moving. I had just been moving things between it and my laptop with a usb stick as needed. In the process of running windows update it installed sp1 for vs2005 and some other security updates resulting in needing this newer vcomp.dll. Sorry about that.

[Taurus]

@DeathWolf:
Thank you for pointing it out.

@tritical:
Yeah, its working now without errors.

But I think there will be others who will come to complain.
So the readme should be updated as soon as possible.

[kai0n7]

For all those getting an error, the version of vcomp needed is not one commonly found, it's a dev version with a security fix.
Here's the good redist to get:
http://www.microsoft.com/downloads/details.aspx?FamilyID=766A6AF7-EC73-40FF-B072-9112BAB119C2&displaylang=en

@tritical: you might want to put that in the readme:)

That got it working for me, thanks!

[tritical]

@Archimedes

What do you think of this enlargement?

nnedi3_rpow2 (rfactor=4, nsize=0, nns=4, qual=2, cshift="Spline36Resize"), 1600x1200 (http://bengal.missouri.edu/~kes25c/castle_4x_new.png)

[akupenguin]

http://akuvian.org/src/x264/nnedi.tar.bz2
Same algorithm as nnedi3-0.9.1, and about 4x faster (per cputime. I haven't implemented threads).
This is just a C function, not an avisynth filter.
I included only the 8x6 mode because I'm only interested in scaling, not deinterlacing. But I assume the wider modes would benefit from the same optimizations.

Changed:
* Convert much of the float math to fixed-point.
* The max part of softmax was useless. Adding a constant before exp is equivalent to multiplying after, and the weighted average removes any such multiplicative constant.
* Edge mirroring was weird: nnedi3 kept 1 copy of the edge row on 2 sides of the frame, and 2 copies on the other 2 sides. imho 2 looks better.
* And a bunch of assembly optimizations.

@tritical
I have some further optimization ideas, but they would require retraining. Could you post sourcecode for that?
I tried to apply CMAES, and it sorta worked, but wasn't really satisfactory; I don't know if I'm doing something wrong or if I just haven't thrown enough cputime at it.

[tritical]

4x is impressive! Not to mention I guess you figured out the structure from just the dll? I'll admit it isn't the most complicated thing in the world though. I had actually spent some time rolling together separate functions in the prescreener network into one big asm function (to eliminate unneeded function call overhead and memory stores) and a few other minor things and only got maybe 20%. Doing an int16 implementation never crossed my mind. I would think that would only get you 2x faster at best? Then you got another 2x on top of that? And here I thought I knew how to program :p

Anyways, I haven't had time to look at your code in depth or compile it. Hopefully tomorrow or the day after. On the training, I no longer use CMA-ES just basic online gradient descent with a cooling schedule.

Oh, and the mirroring thing is a bug... it should only be keeping 1 copy of the edge row on all sides. On the softmax bit, softmax is invariant to a global offset as you point out. The subtracting out the max of all the net values before computing the exp() of them is simply to make exp() well behaved in training (avoid overflow and maintain accuracy of the softmax output). Definitely could be avoided after training, though I doubt it's taking up much cpu time.

[akupenguin]

I would think that would only get you 2x faster at best? Then you got another 2x on top of that? And here I thought I knew how to program :p
Other improvements:
* I can eliminate the horizontal sum from the end of each dotproduct by shuffling the inputs appropriately. If there were only 1 dotp that would be counterproductive, but with lots of them I can reuse the shuffled inputs and it's a net win.
* The prescreener (once you get past the first round of dotps) was latency bound, and out of order execution can't look far enough ahead to interleave multiple copies of the whole function. Manual interleave helped a bunch. (Just moving the first dotps into a separate loop from the tail of the prescreener also helped somewhat, because then out of order execution could do something. Still far from my end result though.)
* Some more scheduling improvements smaller than the above.
* There's a lot of spatial locality in the prescreener results. I subsampled it and then evaluated the in between samples only when the neighbors disagree. (This might be counterproductive on the really slow modes.)
* Your exp approximation was excessively precise. A 6th order polynomial iirc, whose error was less than floating-point precision. I got away with 2nd order (0.1% rmse).
* Mean removal can be factored into weights for free. (Noticed just now and updated the code.)

Oh, and the mirroring thing is a bug... it should only be keeping 1 copy of the edge row on all sides.
When I tried that, it tended to reverse the direction of diagonal lines that hit the edge of the frame. I'm assuming that's because a diagonal becomes a sharp angle in the mirrored image.

The subtracting out the max of all the net values before computing the exp() of them is simply to make exp() well behaved in training (avoid overflow and maintain accuracy of the softmax output).
Yep, I had to deal with that. (But the risk is only overflow/underflow, not loss of precision. Incrementing a floating point exponent doesn't lose any LSBs.) Still possible to munge the constants instead of the variables if you run the net several times between updates. And I wouldn't expect it to drift out of the allowed range very fast.

[aegisofrime]

@akupenguin and tritical: I assume that what akupenguin posted was the source code? Any chance of a compiled version for consumer usage?

[tritical]

He posted source code for a program that enlarges by 2, not an avisynth plugin. Not sure if he plans to make it one or not. I'm working on implementing some of his optimizations in nnedi3. Specifically, int16 dot products, mean removal factored into the weights, and faster exp function for the prediction neural network. Mean removal factored into the weights for the first stage of the prescreener. I probably wont get to anything else anytime soon, but for larger nsize areas (those besides nsize=0/4) and nns values (> 1) the computation is dominated by the predictor so this should give most of the speed up. Once I finish that I'll release a new version and my source code. I have plans to revamp the prescreener, but may or may not get to it anytime soon. Or someone else might do it first :).

I had also been experimenting with minimizing abs error instead of squared error, the last image I posted of the castle was using weights optimized for abs error. Since all that takes is me waiting for models to train I might add abs error optimized weights at some point.

[aegisofrime]

Thanks for the update tritical! I use NNEDI2 on a daily basis (mainly with TGMC), so having an NNEDI3 competitive speed wise would be great! :)

[Terka]

Tritical,
thank you for all your work!
Will theese speed improvements added to nnedi2 also?

[LoRd_MuldeR]

I had also been experimenting with minimizing abs error instead of squared error, the last image I posted of the castle was using weights optimized for abs error. Since all that takes is me waiting for models to train I might add abs error optimized weights at some point.

I wonder: Did you ever try other metrics than squared/absolute error, like SSIM for example? Or is the "sliding window" approach not suitable for your algorithm?

(BTW: Looking forward for any speed-optimization in NNEDI3)

[tritical]

I did try SSIM, but it's hard to effectively use it within the online gradient descent training scheme because the derivative for a single pixel depends on all of the other pixels within the gaussian weighted window around it. That means I can't calculate a single pixel result and then do gradient descent on the model weights unless I assume a value for all of the missing pixels (those which need to be interpolated by the network) and pre-compute the necessary statistics beforehand (so that during training I only have to modify a few values based on the current result to compute the gradient). What I tried was assuming all of the missing values were perfect... that gave results very close to mse training. The one idea I didn't try was starting by assuming all of the missing values were interpolated using cubic interpolation, and after each iteration (after presenting all of the training cases to the network) recalculating all of the ssim statistics that depend on the missing pixels based on the current state of the network. Like a lot of things I plan to try that in the future.

Now if you're not using online gradient descent (say CMA-ES or a fitness function based optimization) you could just run through the training set, interpolate all the necessary pixels, compute ssim for everything at the end. I actually tried that as well, but it takes A LOT more cpu time. Using online gradient descent I can train the 6x48x256 network on 60 million pixels in only a few days on my quadcore desktop. For nnedi2 I was using separable CMA-ES for training (using the scheme I just described) and to train on 1/5 the pixels was taking about 3x longer while using about 4x the cpu power. And nnedi2 is basically equivalent to 4x12x32 in nnedi3 terms. Those are rough numbers of course.

[LoRd_MuldeR]

Time to start a nnedi@home project :D

[akupenguin]

The one idea I didn't try was starting by assuming all of the missing values were interpolated using cubic interpolation, and after each iteration (after presenting all of the training cases to the network) recalculating all of the ssim statistics that depend on the missing pixels based on the current state of the network. Like a lot of things I plan to try that in the future.
Surely the correct algorithm involves computing the partial derivatives of ssim wrt all of its inputs, not treating the neighboring pixels as constants?

For nnedi2 I was using separable CMA-ES for training (using the scheme I just described) and to train on 1/5 the pixels was taking about 3x longer while using about 4x the cpu power.
Is that measuring just speed of convergence, or weighting by quality of solutions found?

[tritical]

Surely the correct algorithm involves computing the partial derivatives of ssim wrt all of its inputs, not treating the neighboring pixels as constants?

I compute the partial of ssim with respect to the network output, which does depend on neighboring pixel values... some of which are fixed, but some of which depend on the network. I assumed fixed values (from the very beginning of the optimization) for those pixels as well, as a rough, fast approximation. Recalculating the statistics depending on those pixels after each (or every few) complete pass through the training patterns using the current state of the network is more desirable, but I haven't done it yet.

EDIT: The partial of ssim with respect to the network output is only for ssim of the 11x11 gaussian weighted window centered on the current pixel. I'm not explicitly taking into account other ssim windows that depend on the network output at this point.

Is that measuring just speed of convergence, or weighting by quality of solutions found?
Time to equal quality of solution, approximately. In all of my tests gradient descent achieved equal or lower training error in significantly less time. Of course given more time cmaes might do better, but for my time/cpu point gradient descent was much better.

[akupenguin]

BTW, modes with more weights than fit in L1d cache are bottlenecked by cache misses, not arithmetic throughput. I didn't rectify this (except insofar as int16 reduces cache footprint too), but you probably want to if you're tuning for nsize>8x6.

[Forteen88]

@tritical: Could you please make a comparison of NNEDI3, with the same source as these:
http://forum.doom9.org/showthread.php?p=1343668#post1343668
Thanks

[Archimedes]

@Archimedes

What do you think of this enlargement?

nnedi3_rpow2 (rfactor=4, nsize=0, nns=4, qual=2, cshift="Spline36Resize"), 1600x1200 (http://bengal.missouri.edu/~kes25c/castle_4x_new.png)
Looks much better, than with the current version of NNEDI3 with the same parameter.

[yup]

Hi all!
Please explain how work sclip parameter? It is first iteration for finding solution?
yup.

[Skauneboy]

I can't get EEDI3 to work with this AA-function:

o=last
AssumeBFF().SeparateFields()
dbl = mt_Average(SelectEven().EEDI3(field=0),SelectOdd().EEDI3(field=1),U=3,V=3)
dblD = mt_MakeDiff(o,dbl,U=3,V=3)
shrpD = mt_MakeDiff(dbl,dbl.RemoveGrain(11),U=3,V=3)
DD = shrpD.Repair(dblD,13)
dbl.mt_AddDiff(DD,U=3,V=3)

Repair gives the error "clips must be of equal type". It works fine with your other interpolation filters though.

[Didée]

Oops, I don't have eedi3.dll at hand ... give a try if it works with the following change?

dbl = mt_Average(SelectEven().EEDI3(field=0),SelectOdd().EEDI3(field=1),U=3,V=3).AssumeFrameBased()

[Skauneboy]

Thanks for the suggestion Didée but it didn't work. Something awry with the field-parameter of EEDI3? Or perhaps Repair.dll is at fault.

[Didée]

Ah, reading documentation helps. EEDI3 works framebased like NNEDIx, not fieldbased like EEDI2. Try like so:


o=last
AssumeBFF() # should not be needed ...
EEDI3(field=-2)
dbl = merge( selecteven(), selectodd() )
dblD = mt_MakeDiff(o,dbl,U=3,V=3)
shrpD = mt_MakeDiff(dbl,dbl.RemoveGrain(11),U=3,V=3)
DD = shrpD.Repair(dblD,13)
dbl.mt_AddDiff(DD,U=3,V=3)

[Skauneboy]

Ah, that did the trick. :thanks:

[Chainmax]

I did some more comparisons using Gabriel Knight's "Making Of" video:


eedi3_rpow2(rfactor=2,cshift="spline36resize",hp=false)
http://img827.imageshack.us/img827/5200/fr12307eedi3v091.png

nnedi3_rpow2(rfactor=2,nsize=3,nns=4,qual=2,cshift="spline36resize")
http://img137.imageshack.us/img137/8916/fr12307nnedi3v091.png

eedi3_rpow2(alpha=0.3,beta=0,rfactor=2,cshift="spline36resize",hp=true,vcheck=3)
http://img825.imageshack.us/img825/3504/fr12307nnedi3v091try2.png


Original screen batch here (http://forum.doom9.org/showpost.php?p=1356658&postcount=122). In my opinion, on the particular filterchain used for this particular source, the first version of EEDI3 gives the more pleasing results.

[markanini]

Thanks for the comparison chainmax. I'd go with nnedi3 myself in this case but I wouldn't argue with your preference for eedi3 as it's a matter of taste.

I'm eagerly awaiting for Triricals next nnedi3 release.

[Usedocne]

Thanks also Chainmax. I couldn't choose between them though. Each is good on different areas (parts of mic, side of face), but struggle on others. They are pretty much equal imho, in as far as weighing up the positives and negatives effects.

[tritical]

So I finally got back to working on this, and almost have the next nnedi3 release ready... but one question first. Does anyone using this not have an SSE2 capable processor? I'm considering dropping support for SSE/MMX since it's a pain to keep around and test.

[aegisofrime]

So I finally got back to working on this, and almost have the next nnedi3 release ready... but one question first. Does anyone using this not have an SSE2 capable processor? I'm considering dropping support for SSE/MMX since it's a pain to keep around and test.

Those are pretty old CPUs. We are talking about Pentium 3s and Athlon XPs and below... Judging by the processing power of those CPUs, I doubt anyone will use them for NNEDI3...

Of course, this is just my opinion.

[SubJunk]

Welcome back, Tritical. Looking forward to the release. I think you can safely drop SSE/MMX support

[Blue_MiSfit]

I'd say so as well. Anyone doing encoding on a P3 / Athlon XP probably won't mind having to use the previous release - especially if they're willing to tolerate everything else that comes with life on such old CPUs ;)

[mandarinka]

I have one Athlon XP at hand, but I'm okay with requiring SSE2. It is better when you have less obstacles to improve the filter (which I like a lot - I use daa() and the difference nnedi3 made was quite something, thanks!).

[tritical]

I put up the new version. I decided to drop sse/mmx, but v0.9.1 is still available from the old_stuff directory of my website. Changes:

+ clean up and release source code
+ speed improvements - (thanks Loren Merritt - akupenguin - for many ideas and code)
int16 dot products in prescreener/predictor neural networks
faster exp function approximation
mean removal factored into weights
lots of smaller changes
+ new prediction nn weights for certain nsize/nns combinations
+ added fapprox parameter
- fixed bug with border mirroring
- dropped support for sse/mmx - all asm code requires sse2. changed opt parameter accordingly.

The fapprox parameter is a bitmask that lets you switch between int16 and float dot products in the predictor or prescreener (&1 and &2). That part was mainly for testing. It also allows switching between 3 exp function approximations based on bits 2/3 (&12 = 0,4, or 8/12). 0 gets you what I was originally using - exp from intel approximate math library (6th order - slowest, most accurate). 4 gets you the exp from Loren's code (2nd order - faster, slightly less accurate). 8/12 get you an even faster, more inaccurate version. By default the fastest version is used - I couldn't tell any noticeable difference when flipping back and forth between frames when using it (and pixel changes are pretty much +-2 at max). Perhaps someone might want to change that setting.

Hope to have predictor weights trained to minimize abs(error) or sqrt(abs(error)) eventually.

[LoRd_MuldeR]

Thanks for the update!

One thing I noticed: Using NNEDI3_rpow2() with only 'refactor' set will complain that 'fapprox' must be in the [0,7] range.

What 'fapprox' value would you recommend as default for NNEDI3_rpow2() ???

[tritical]

Give me a sec and I'll fix that. Hopefully that's the only bug :).
The default of 15 is what I'd recommend or switching to the slightly slower exp() function - fapprox = 7. I personally can't tell a difference in the final result though. fapprox=0 would technically be the most accurate, but basically disables all of the major optimizations between this version and the last version.

Finished uploading the fix.

[Usedocne]

HUGE thanks tritical and akupenguin. *goes off to speed test TGMC*

[Usedocne]

Here's my TGMC results, if anyone cares:

1000 frames (720x480i)

1m.35s nnedi3 v0.9.2

1m.50s nnedi2 v1.6

2m.28s nnedi3 v0.9.1

[IanB]

@Tritical,

Just for interest it would have been nice to see the MMX/SSE source code before you junked it.

Also do you have any plans to release the source code for nnedi2?

[ajp_anton]

Now that you're back, could you please make a 64-bit version?

[aegisofrime]

Now that you're back, could you please make a 64-bit version?

With the source code released, it should be a lot easier. Somebody with the right skills can just compile a 64-bit version.

[tritical]

@Usedocne
Thanks for testing. What are the times if you use nsize=5 in nnedi3? That is more comparable to nnedi2.

@IanB
I think I still have the v0.9.1 source code around. I can post that, but really it is just the sse2 floating point code in nnedi3 minus any sse2 instructions + the necessary workarounds. nnedi2 is the same as nnedi3 except for some changes to the predictor nn. Specifically:

1.) nnedi2 always used 4x12 neighborhood
2.) nnedi2 didn't scale by the standard deviation of the neighborhood
3.) the second set of neurons (perceptrons) which use the elliott activation function and whose output is multiplied by 5.0 in nnedi3 used the linear activation function (essentially no activation function) in nnedi2.
4.) the first set of neurons which use a softmax activation in nnedi3 were actually the final layer of a three layer feed forward nn in nnedi2 (first two layers used elliott, connection architecture was the same as the prescreener - final layer connected to both previous layers, etc...)

I personally see nnedi2 as completely superseded by nnedi3. I know some people use nnedi2 because it is faster by default, but if you use equivalent settings in nnedi3 (nsize=5 instead of the default nsize=6) they are basically the same speed - and that was with v0.9.1 of nnedi3.

@ajp_anton
I might actually since I now have a computer (a new laptop) which has a 64-bit version of windows. Or if someone else does it first even better.

[Usedocne]

Updated TGMC speed test results:

1000 frames (720x480i)

1m.24s nnedi3-nsize=5 v0.9.2

1m.46s nnedi3-nsize=5 v0.9.1

1m.50s nnedi2 v1.6

[jpsdr]

Out of curiosity, i've tried to compile the source (with Visual Studio 2008), i've got the 6MB binary file. So, when i build the project, it said everything went fine, but... i've not the .dll result file. Has anyone tried to compile ???

[XhmikosR]

Everything works fine here. I built nnedi3 with MSVC2010 and ICL 11. Can someone compare them to the original build speed-wise?
nnedi3_MSVC2010_SSE2 (http://xhmikosr.1f0.de/avisynth/nnedi3_MSVC2010_SSE2.7z)
nnedi3_VS2008_ICL11 (http://xhmikosr.1f0.de/avisynth/nnedi3_VS2008_ICL11.7z)

[jpsdr]

Ok... My mistake, didn't take look on right place...

[aegisofrime]

Is that ICL version still biased against AMD CPUs?

[XhmikosR]

No idea, I don't have an AMD cpu and I don't care about AMD at all :p But it should work on AMD cpus.

[aegisofrime]

No idea, I don't have an AMD cpu and I don't care about AMD at all :p But it should work on AMD cpus.

Oh, I was referring to the whole fiasco of Intel's compiler taking the slower route, so to speak, when it detects a CPUID that is not Intel. It works, but slower than it should be.

Anyway, here's my test. Test content is 1000 frames of 720x480i DVD content, using QTGMC. avs2avi was used in null mode to eliminate encoder as a variable.

The PC's specs are

AMD Phenom II X4 955 @Stock speed, Undervolted @ 1.28V
4GB Corsair DDR2-800 RAM
Windows 7 Ultimate 64


SetMTMode(5,4)
DGDecode_mpeg2source("C:\Work\Test.d2v")
SetMTMode(2,4)
QTGMC(Preset="Slow",EDIMode="NNEDI3")

Distributor()

Trim(0,1000)

tritical's NNEDI3
* Pass 1/1: Finished in 00:00:50.171 (19.95 FPS)
* Frames: 1001 (1001 keyframes)
* Size: 494.88 MB

NNEDI3 ICL11
* Pass 1/1: Finished in 00:00:50.798 (19.71 FPS)
* Frames: 1001 (1001 keyframes)
* Size: 494.88 MB

NNEDI3 MSVC2010
* Pass 1/1: Finished in 00:00:50.607 (19.78 FPS)
* Frames: 1001 (1001 keyframes)
* Size: 494.88 MB

NNEDI2
* Pass 1/1: Finished in 00:00:51.873 (19.30 FPS)
* Frames: 1001 (1001 keyframes)
* Size: 494.88 MB

[XhmikosR]

Ok, thanks for your results. I'll wait for someone with Intel cpu to see if there's a difference there.

[jpsdr]

What is the Distributor() command in the script ?

[julius666]

Everything works fine here. I built nnedi3 with MSVC2010 and ICL 11. Can someone compare them to the original build speed-wise?
nnedi3_MSVC2010_SSE2 (http://xhmikosr.1f0.de/avisynth/nnedi3_MSVC2010_SSE2.7z)
nnedi3_VS2008_ICL11 (http://xhmikosr.1f0.de/avisynth/nnedi3_VS2008_ICL11.7z)

Could you build a 64-bit version too?

[ajp_anton]

avisource(".avi").trim(0,-500) #384x480, interlaced, YUY2
nnedi3(field=-1)
#nnedi3(field=-2)

Average of 4 runs:
Core i7 at 3.4GHz:
field= -1 -2
tritical's: 83.21 114.05
MSVC2010: 83.29 114.04
ICL11: 83.24 113.94

Athlon II x4 at 2.8GHz:
field= -1 -2
tritical's: 54.13 67.86
MSVC2010: 54.13 67.86
ICL11: 52.70 66.71
edit: Added AMD results.
nnedi3_rpow2 doesn't work for some reason... not with eedi3 or nnedi2 either (happened to have the .dll's), and they did a while ago, so I guess it's because of my computer (computers...).

[Usedocne]

Thanks for the compiles XhmikosR. Doesn't look like theres much of a difference between them though, going by the speed tests.

Be interesting to see a 32-bit versus 64-bit nnedi3 speed test done now.

[HeadlessCow]

There's no reason to average. Just take the fastest time.

[IanB]

@IanB
I think I still have the v0.9.1 source code around. I can post that, but really it is just the sse2 floating point code in nnedi3 minus any sse2 instructions + the necessary workarounds.
It was more just of academic interest to see how you did "the necessary workarounds".

[IanB]

Why do people still think the compiler will make any difference in speed to software that is cpu bound totally in the hand written SSE2 assembler part of the code.

Sure pure C++ code or idiotic code written using intrinsics might get a small boost, but the point of hand written SSE2 assembler is to squeeze the last erg of performance out of the cpu.

I personally find it insulting when people attempt to assert that a compiler might make any difference in performance to hand written assembler code.

[tritical]

It was more just of academic interest to see how you did "the necessary workarounds".

In the floating point path the only real thing to work around is the lack of parallel int->float and float->int conversions in sse taking 128-bit registers as source/dest. In sse2 you get cvtdq2ps/cvtps2dq. int->float is required in both the first/second stage extract() methods and two of the three exp() methods require float->int. You also have to give up pshufd in the prescreener, which can be used to eliminate horizontal reductions. Everything else is basically using shufps instead of some sse2 shuffling instructions. For the int16 path you'd basically have to replace all the 128-bit code with 64-bit mmx code since all of the integer instructions on 128-bit require sse2. Either way, it was just easier to only support sse2.

[XhmikosR]

I personally find it insulting when people attempt to assert that a compiler might make any difference in performance to hand written assembler code.

No problem, feel insulted then. Even if I gain 1% by changing the compiler is fine for me.

[jpsdr]

Be interesting to see a 32-bit versus 64-bit nnedi3 speed test done now.

Interesting only if all the ASM code is re-writen to take use of what 64 bits adds (more registers for exemple). If it's 'simply' recompiled, you 'only' gain a 64bits version, but no performances.
As there is a lot of ASM inline, porting to 64bits will not be easy. You have first to take out all the asm inline on external files, and re-write them...

[Usedocne]

@jpsdr

Hmm... that doesn't sound like too much fun. Still a 64-bit compile is needed for those using Avisynth64 (hopefully me someday).

[Dark Shikari]

@jpsdr

Hmm... that doesn't sound like too much fun. Still a 64-bit compile is needed for those using Avisynth64 (hopefully me someday).Well, start by fixing Avisynth64; its assembly code is also broken.

[Usedocne]

@Dark Shikari

Lol, wish I could, but I'm no programmer. Just a lowly VE hobbyist. :D

[tritical]

64-bit test build: http://bengal.missouri.edu/~kes25c/nnedi3_64.dll

This was built with avisynth.h from JoshyD's 64-bit 2.5.8 MT version. Any testing or reports are welcome :thanks:.

[Didée]

Quick dash with nnedi3(field=-2) on planet x64. It's breathing, it's running, it's reaching goal: seems to work fine. :)

About same speed as x86, perhaps slightly slower ... 3 seconds more for ~29000 output frames, but it was only one single test run.
Anyway, it's close to 100 fps on 720x576i, using 8 threads/4cores on an i7@3.33. (75 to 120 fps, scene dependent.)

average 95.9 fps with internal multithreading (8 threads)
average 113.8 fps with setmtmode(2,8)

[jpsdr]

@tritical : Do you plan to optimize your ASM code for 64 bit also ?

[tritical]

I already changed the few assembly functions that could benefit from the extra xmm registers. None of the sse2 routines were really limited by lack of registers, and the function call overhead is already small. I would not except any noticeable change in speed for the sse2 path going from 32 bit to 64 bit. The C code path is definitely faster. Of course, that is not intended for actual usage.

Here is a 64bit test version of eedi3_64 (http://bengal.missouri.edu/~kes25c/eedi3_64.dll) (I couldn't find a build by anyone else). It's about 15-20% faster than the 32-bit version on my laptop.

[Bi11]

Here is a 64bit test version of eedi3_64 (http://bengal.missouri.edu/~kes25c/eedi3_64.dll) (I couldn't find a build by anyone else). It's about 15-20% faster than the 32-bit version on my laptop.
:thanks:
Any speed improvement to eedi3 is welcomed!

Is there any chance of being able to do assembly-level optimizations to improve performance similar to the speed improvement seen with nnedi3?
If not, can nnedi3 be trained to connect edges as aggressively as eedi3?

Now the remaining plugins I need to complete my 64-bit workflow are, an updated version of dfttest_x64, and dctfilter_x64 for deblock_qed. :)

[jpsdr]

I already changed the few assembly functions that could benefit from the extra xmm registers.
....
It's about 15-20% faster than the 32-bit version on my laptop.
Very interesting and very good !
Nice, thanks for this.

[akupenguin]

Is there any chance of being able to do assembly-level optimizations to improve performance similar to the speed improvement seen with nnedi3?
My version gains 6% speed from 64bit. This is almost entirely due to better scheduling of dotproducts. I have no code that needs more than 8 registers for a naive scheduling, but out of order execution isn't perfect.

[Blue_MiSfit]

I had a wonderful experience with NNEDI3 tonight for an unusual task: Upscaling a 12 megapixel image from my Canon s90 (RAW -> Lightroom -> 16bpc ProPhoto TIFF -> 8 bit sRGB PNG) to ~48 megapixels for professional printing on 20" x 30" paper.

A 64 bit toolchain was mandatory :)


imagesource("...\felix_source.png")
trim(1,1)

red=showred.converttoyuy2(matrix="pc.709")
green=showgreen.converttoyuy2(matrix="pc.709")
blue=showblue.converttoyuy2(matrix="pc.709")

red=red.nnedi3(field=-1,dh=true).turnright.nnedi3(field=-1,dh=true).turnleft
green=green.nnedi3(field=-1,dh=true).turnright.nnedi3(field=-1,dh=true).turnleft
blue=blue.nnedi3(field=-1,dh=true).turnright.nnedi3(field=-1,dh=true).turnleft

mergeRGB(red,green,blue)

spline64resize(7949,6000)


I used a solid 2.8GB of RAM for this script on a single frame. Processing time was under a minute, using PNG output from VirtualDub x64.

The result was very good! When stacked with an image scaled by Photoshop's bicubic filter, it was noticeably better, especially after I staked an instance of Photoshop's "Smart Sharpen" filter on each image.

MARVELOUS work, akupenguin and tritical et al!

Now, if I had support for 16 bpc through the whole pipeline, I'd be REALLY happy :devil:

[dansrfe]

Trying that out now^

EDIT: AvsP starts throwing errors when I press F5. CPU usage does shoot up to 100% for about 1 min though.

EDIT EDIT: for rfactor = 4, rfactor = 2 works very snappily on my c2d.

[jpsdr]

I'm wondering if there is not some kind of memory leak in nnedi3_rpow ? I've made an avisynth script with only AVISource and nnedi3_rpow, absolutely nothing else, and noticed during VDub process that memory usage was always increasing regularly. I've noticed this with x64 version, not tested yet with x86 version. Doesn't anybody else noticed it ?

[bcn_246]

tritical,
I have been using your plug-ins for some time, thought I should say thanks for all your hard work.

I tried your 64-bit build of NNEDI3 using "NNEDI3_rpow2(rfactor=2,cshift="Spline64Resize")" and found the speed improvement to be vast. Using JoshyD's 2.5.8 MT x64 build, an i7 @ 4.00GHz w/6GB RAM, processing the same video (Gorillaz - Feel Good Inc. @ 720p), I got ~3fps on x86 and ~8fps on x64. Probably the most significant x86->x64 increase I have seen.

I tried your first x64 build of EEDI3 but it threw up a "unable to load plugin..." error. I just re-downloaded the file from your post and see the timestamp is different, updated today (08/12/2010), so I will have a go with that build of EEDI3 x64 once my current processing job is done.

Ben

[bcn_246]

Tried the EEDI3 version you posted today. Still refusing to load with JoshyD's AviSynth v2.5.8 MT x64 build 4/16/2010.

http://img80.imageshack.us/img80/8394/64552070.th.jpg (http://img80.imageshack.us/i/64552070.jpg/)

- Ben

[tritical]

@bcn_246
I haven't posted any new versions since my last post - not sure why the timestamps are different. Probably you need the visual studio 2010 runtime (http://www.microsoft.com/downloads/en/details.aspx?familyid=BD512D9E-43C8-4655-81BF-9350143D5867&displaylang=en) for the openmp dependency.

@jpsdr
nnedi2_rpow is just a wrapper around nnedi3() that automates resizing. It constructs an avisynth filter chain consisting of nnedi3(), turnleft/turnright, final resize filter, etc... It doesn't allocate any memory directly so if memory is leaking it is either Avisynth or nnedi3(). It's difficult to tell for sure just looking at memory usage due to the way Avisynth's cache operates... after you open a script Avisynth doesn't immediately gobble up all the memory it plans to use. It gradually fills the cache as frames are requested until it hits its memory limit. If you let it run for a while does the memory usage stabilize? I will look into this.

@dansrfe
rfactor=3 in nnedi2_rpow2 should throw an error. Only powers of two are supported.

[bcn_246]

My (stupid) mistake with the timestamps. My download manager wasn't set to copy the file's date from the server, so re-downloading modified the creation date.

I installed Microsoft Visual C++ 2010 (x64), and updated .NET to v4.0 while I was at it, EEDI3 x64 is still refusing to load.

I am using Windows 7 Ultimate x64, up-to-date with MS. Here is a list of other plug-ins I have in my 'plugins64' folder. I am guessing that there is something that should be here (or in SysWOW64/System...) that EEDI3 depends on.


\Documentation
\Scripts
fieldhint.cpp
Utilities.cpp
AddGrainC.dll
autocrop.dll
aWarpSharp.dll
colormatrix.dll
defreq.dll
DeGrainMedian.dll
dfttest.dll
DirectshowSource.dll
EEDI2.dll
eedi3_64.dll
ffms2.dll
FFT3DFilter.dll
FFT3dGPU.dll
fftw3.dll
FieldHint.dll
FluxSmooth.dll
gradfun2db.dll
hqdn3d.dll
LeakKernelDeint.dll
libfftw3f-3.dll
MipSmooth.dll
motion.dll
MT.dll
mt_masktools-25-x64.dll
mvtools2.dll
nnedi3_64.dll
RemoveGrain.dll
Repair.dll
TDeinterlace.dll
TelecideHints.dll
TIVTC.dll
TNLMeans.dll
TTempSmooth.dll
UnDot.dll
VerticalCleaner.dll
yadifmod.dll
ffmsindex.exe
avisynth.h
info.h
internal.h
Utilities.h
fft3dgpu.hlsl
FieldHint.icproj
FieldHint.sln
FieldHint.vcproj

- Ben

[jpsdr]

@jpsdr
If you let it run for a while does the memory usage stabilize? I will look into this.

Yes, it's stabilize at around 500MB after a long time. This seems a little high considering the fact :
- It begins at around 150MB and constantly increase.
- I'm working in input with 720x480 pictures, asking for an increase size of 2 (so output at 1440x960) + Spline36Resize.
I've not waiting all the time, i've just noticed the continuous increase on the begining, and noticed that several hours laters, and so dozen of file processed laters, memory usage seems stabilized at around 500MB.

Edit : Video is YV12.

Note : You've provided the source code of last version, thanks, nice. You've also made an experimental 64bits version, but without the source code. Is it possible you made this one avaible ?

[IanB]

For a system with 2+GB of ram this is more or less what I would expect. The default SetMemoryMax is 25% physical memory capped to 512Mb. You can of course change this with a SetMemoryMax() statement at the start of your script.

[jpsdr]

I don't understand the slow growing use of memory. Once a filter has done all the memory allocations it needs, if there is no memory leak, i don't understand why memory should slowly go bigger and bigger and bigger.... I use some others avisynth scripts (always simple basic things), and memory used is always stabilised in the few seconds, it doesn't still growing half an hour after starting the process.
This is not the case, but i think if on a script like (YV12 video) :

AVISource("File.avi")
Spline36Resize(1280,720)

I would see a memory use of 500MB !!! I thought it could be a problem................
Wait... I remember now IanB.... I've got similar issue with something totaly different, and problem was linked to the fact that source is YV12 in 720x480, and so some resolution not mod16, and so created some cache default or i don't remember what, resulting in avisynth cache always increasing...
At the time i thought there was a problem with avisynht or Lagarith...
I must look out for the solution you provided me, and test it and check if there is still the memory increase.

[jpsdr]

It's not what i thought...
I've replaced

AVISource("File.avi")
nnedi3_rpow2(rfactor=2,cshift="Spline36Resize",fwidth=960,fheight=720,nsize=0,nns=3,qual=2)

by

AVISource("File.avi",False,"YV12")
SetPlanarLegacyAlignment(True)
nnedi3_rpow2(rfactor=2,cshift="Spline36Resize",fwidth=960,fheight=720,nsize=0,nns=3,qual=2)

but nothing changed, memory is still begining at around 150MB, and is slowly growing until reaching around 500MB.
So, it behaves like there is some kind of memory leak somewhere...

[Didée]

Did you read IanB's post? Start your script with the line "SetMemoryMax(16)", and see Avisynth using only very little memory.

Avisynth has an internal frame buffer. If not used elsehow by internal (temporal) filters, Avisynth will fill that frame cache with "past output frames". (For example, that means that if you're running a very slow/complex "1fps" script, you can very quickly step backwards lots of frames that already have been rendered.)

[jpsdr]

After some test with only Spline36Resize, it seems it's my mistake, behavior is the same.

[IanB]

There are some heuristics involved with the cache memory usage that tries to fight growth. This involves cache instances trying to recognise non-usage cases and marking VFB's involved as returned for early reuse. If the memory manager has a VFB of the right size marked as returned it will issue that one (limit mode) instead of checking the SetMemoryMax values and issuing a new one if current usage is less (growth mode) or issuing one from the LRU chain (plateau mode). The heuristic code errs on the side of safety and won't return VFB's unless it is pretty sure they will never be needed again.

If the growth is a problem use SetMemoryMax() to constrain it. Also the OS had to have memory free at script compile time to set the initial SetMemoryMax value, so that memory was not being used for anything else anyway.

[Gavino]

Interesting stuff, thanks for the details, Ian.
The heuristic code errs on the side of safety and won't return VFB's unless it is 100% sure they will never be needed again.
Can it ever be 100% sure, given that a plugin could request frames in some totally arbitrary order?

[IanB]

Well no it can never be 100% sure.

[Amateur]

new user here. i am wanting to give this eedi3 filter a try but it won't load in megui. i keep getting an error saying "there is no function named eedi3". i put the dll into the plugin folder of avisynth so i don't know what i'm missing.

a secondary question:
i'm wanting to use this filter for some anti-aliasing. am i choosing the proper filter for this purpose? here (http://img225.imageshack.us/img225/634/21384130.jpg) a the scene that i see it the most. i tried to use aaa() but that didn't seem to do much. here is the script that i am using that has given me the best results so far

LoadPlugin("C:\Program Files (x86)\MeGUI\tools\dgindex\DGDecode.dll")
DGDecode_mpeg2source("E:\xv1e1\VTS_01_1.d2v", info=3)
LoadPlugin("C:\Program Files (x86)\MeGUI\tools\avisynth_plugin\ColorMatrix.dll")
ColorMatrix(hints=true, threads=0)

undot()

AnimeIVTC(mode=1, aa=1, precision=3, edimode="nnedi3")

Deen("a2d",7,11,13,min=0.5)

crop( 8, 0, -6, -4)

LanczosResize(640,480)

trim(4000,6000)
**notes on the parameters - i found that line for animeivtc in another thread so i just used it for mine. reading the documentation for that was like looking at chinese for me. i took that deen line from scintilla...maybe that can be tweaked to make my picture look better. and i read that undot is risk free compressibility so i just add that to all of my scripts...not really sure what it does.

here is what the scene looks like at 600kbps (http://img530.imageshack.us/img530/8035/32416345.jpg) bitrate (might bump that up more if i can't get filters to help clean up the image)

i've been trying to read up on filters for the last week or more but a lot of comments are beyond me so i very well might be implementing these filters in the wrong way. would love any help possible. kind of blindly using scintilla's filter guide as a baseline for quite a while but that is somewhat limited now that i've been snooping around on doom9

here is a vob clip (http://www.megaupload.com/?d=CKKDAZ8M) of the opening scene if anyone cares to fool around with it. source is a North American DVD (ntsc?)

just in case the opening scene is different from the actual episode, here (http://www.megaupload.com/?d=F7672659) is a one minute vob clip following the opening scene.

[yup]

Amateur!
try:
http://forum.doom9.org/showthread.php?p=1423090#post1423090
If You want antialiasing use function inside AnimeIVTC (ediaa, daa, maa). I prefer daa with nnedi3, and try setting for nnedi3 for improving accuracy.
yup.

[Amateur]

alright i clicked on the link from the response to your link that is supposed to be the fix and was given a page that had three download options
vcredist_IA64.exe
vcredist_x64.exe
vcredist_x86.exe
i am running a 64 bit operating system so i chose the first two. i went to install the first one and am told my processor isn't compatible (phenom ii x4) so i installed the second one only. but i still get the error there is no function named eedi3. i even restarted my computer after running that vcredist_x64.exe and no change in error

and how do i anti-alias from within animeivtc? i tried finding it in the info of animeivtc but again, it's kind of like reading another language with that much info that i'm not familiar with. i changed aa=2 because the filter had
aaed = aa==0 ? dec :
\ aa==1 ? dec.ediaa() :
\ aa==2 ? dec.daa() :
\ aa==3 ? dec.maa() :


i'm so lost...

also, could deen be tweaked in any way for some improvement?

[yup]

Amateur!
I try only Avisynt 2.58 and SEt 2.6 both 32 bit. I know that for 64 bit tritical write separate version.
Try start from 32 bit.
For realize 64 bit need know a lot of tip trick.
yup.

[bcn_246]

alright i clicked on the link from the response to your link that is supposed to be the fix and was given a page that had three download options

i am running a 64 bit operating system so i chose the first two. i went to install the first one and am told my processor isn't compatible (phenom ii x4) so i installed the second one only. but i still get the error there is no function named eedi3. i even restarted my computer after running that vcredist_x64.exe and no change in error

and how do i anti-alias from within animeivtc? i tried finding it in the info of animeivtc but again, it's kind of like reading another language with that much info that i'm not familiar with. i changed aa=2 because the filter had



i'm so lost...

also, could deen be tweaked in any way for some improvement?

I mentioned this issue also. I am running Visual Studio 2010, and have all the packs that came with that installed, as well as up-to-date Windows 7 Ultimate.

I am going to try running it on a VM'd Windows XP x64 later and see if I have any luck.

I will also have a look for a x64 dependency scanner and see if it can pick up any (missing) required binary(s). The one I have is 32-bit only.

Ben

[Amateur]

so how do i set anti-alias using nnedi3 within animeivtc?

[yup]

Amateur
daa(Your clip)
#Anti-aliasing with contra-sharpening by Didйe
function daa(clip c) {
nn = c.nnedi3(field=-2)
dbl = mt_average(selecteven(nn),selectodd(nn),U=3,V=3)
dblD = mt_makediff(c,dbl,U=3,V=3)
shrpD = mt_makediff(dbl,dbl.removegrain((width(c)>1100) ? 20 : 11),U=3,V=3)
DD = shrpD.repair(dblD,13)
return dbl.mt_adddiff(DD,U=3,V=3) }

I am only replace nnedi2 to nnedi3.
yup.

[Mystery Keeper]

so how do i set anti-alias using nnedi3 within animeivtc?

Do not. Do NOT antialias with nnedi3.

[ajp_anton]

alright i clicked on the link from the response to your link that is supposed to be the fix and was given a page that had three download options
vcredist_IA64.exe
vcredist_x64.exe
vcredist_x86.exe
i am running a 64 bit operating system so i chose the first two. i went to install the first one and am told my processor isn't compatible (phenom ii x4) so i installed the second one only.
Not going to help your problem, but IA64 is Intel's Itanium architecture. You will probably never encounter these, ever.
x64 and x86 are what you think they are.

[Didée]

Do not. Do NOT antialias with nnedi3.
For what reason? Has nnedi3 become bad, all of a sudden?

[Amateur]

so in order to do this i would change my script to

LoadPlugin("C:\Program Files (x86)\MeGUI\tools\dgindex\DGDecode.dll")
DGDecode_mpeg2source("E:\xv1e1\VTS_01_1.d2v", info=3)
LoadPlugin("C:\Program Files (x86)\MeGUI\tools\avisynth_plugin\ColorMatrix.dll")
ColorMatrix(hints=true, threads=0)

undot()

AnimeIVTC(mode=1, aa=1, precision=3, edimode="nnedi3")

daa(VTS_01_1.d2v)

Deen("a2d",7,11,13,min=0.5)

crop( 8, 0, -6, -4)

LanczosResize(640,480)

trim(4000,6000)

and that will do it? not on the computer to test it out currently.

Not going to help your problem, but IA64 is Intel's Itanium architecture. You will probably never encounter these, ever.
x64 and x86 are what you think they are.

this explains why it wasn't working for me since i use an AMD so thank you for that info. but why isn't that filter working still? eedi3 still gives me the "no function named eedi3" error even though it's in the avisynth plugin folder with the rest of my filters.

[Mystery Keeper]

For what reason? Has nnedi3 become bad, all of a sudden?

For anitaliasing? It was always bad for antialiasing. It is deinterlacing plugin. Quite sharp! EEDI2 is the only EDI deinterlacer that is good for antialising, because it blurs the edges while connecting them. NNEDI1/2/3 are completely different. And Amateur was asking about anitalizing anime too. Why did I remove EEDI2 from SharpAAMC? Because SangNom worked twice as fast with same result and didn't have the shift problem. EDIs are not intended for antialiasing. The sharper the EDI - the worse it is for antialiasing.

[Didée]

Might be all well and true. (Though still arguable.)

However, you must consider the basic working principle of the various AA functions. In SharpAAMC (and predecessors a/o cognates), the EDI interpolator basically interpolates between adjacent original pixels. (And btw, SangNom IS an EDI interpolator, too.) But, the daa() script works different! It averages two independent interpolations, where each interpolation set works between odd-distanced pixels. This on its own provides sufficient amount of blurring. Enough blurring that the script uses a contra-sharpening step to counteract the blurring.

Hence, you shouldn't say that EEDI/NNEDI are generally bad for antialiasing. Perhaps they're not suited for *your* preferred AA method. But they are well suited for AA methods other than that.

[Mystery Keeper]

Ah. Completely different method. Now I see. Got to compare for quality/speed ratio.

[Boulder]

Has anybody else had stability issues with the latest version and qual or nns higher than the default values?

Using this script I get a reboot quite soon to the beginning:
MPEG2Source("2x01.d2v",cpu=4,moderate_h=25,moderate_v=45)
#SetMTMode(2)
#ChangeFPS(last,last,true)
AssumeTFF()
Crop(24,8,-20,-4,true)
ColorMatrix(d2v="2x01.d2v",threads=1,interlaced=true)
r=YADIFMod(mode=1,edeint=NNEDI3(field=-2,qual=2,threads=1))
d=r.Blur(1.58)
SRestore(r,mode=-2,frate=29.97,thresh=22,cache=10,dclip=d)
The previous version works fine.

The problem is that the rebooting points to an unstable system but my system is 24/7 Prime95-stable. Rebooting is also a problem because it doesn't leave any tracks to follow.

[yup]

Boulder!
I try
nnedi3(field=-2,nsize=3,qual=2)
for 2.58 official and Set 2.6 build without problem.
yup.

[Boulder]

Which CPU do you have? I have an Intel E6750.

EDIT: using opt=1 doesn't work either :(

[Didée]

Is the CPU actually OC'ed? You know - if yes, you need to cross-check with Stock settings. "Prime stable" is only a hint, in last instance it doesn't mean much. You can only prove instability, but you can not prove stability. I've made that experience too - with some certain OC settings (probably borderline), the system was 24h Prime and LinX stable. Still, very-complex Avisynth scripts tended to crash occasionally. Which disappeared when I relaxed the OC just a little bit.

[Boulder]

It's possible that it is due to overclocking.. Funny thing that during the three years that I've had the system as it is, there have not been such problems. Possibly the latest NNEDI3 just tickles a certain register the right way to cause issues :)

[yup]

Hi all!
How can upscale YUY2 source
ConvertToRGB24(interlaced=false)
nnedi3_rpow2(rfactor=2,cshift="spline36resize")
ConvertBackToYUY2(interlaced=false)
or using separate plane for Y,U and V.
Please advice.
yup.

[Didée]

Why jump through loops when it's not necessary?

INFO:

nnedi3 [...] works with YV12, YUY2, and RGB24 input.

[yup]

Didée!
May be problem with Set 2.6 build. My script crash if I using nnedi3_rpow2 for YUY2 colorspace.
For YV12 work fine.
Script simple

SetMTMode(2,4)
DirectShowSource("bbb.avi")
ConvertToYUY2(interlaced=true)# for ConvertToYV12(interlaced=true) work fine
AssumeTFF()
QTGMC()#YUY2 version
Crop(16,16,-16,-16)
nnedi3_rpow2(rfactor=2,cshift="spline64resize",fwidth=720,fheight=576)
yup.

2.58 official release work without problem.

[tritical]

I put up a version of nnedi3 with weights trained to minimize absolute error instead of squared error (see first post). At the moment the following nns/nsize combinations are missing new weights (they are still using the squared error trained weights) nns=256 @ 16x6, 32x6, 48x6, and 32x4. The absolute error trained weights seem to be better for image enlargements - they eliminate some artifacts currently produced by nnedi3.

The castle image of Archimedes:

http://forum.doom9.org/showthread.php?p=1418087#post1418087
nnedi3_rpow2(rfactor=4,nsize=0,nns=4,qual=2,cshift="Spline36Resize") - ABS (http://bengal.missouri.edu/~kes25c/castle_abs.png)
spline36resize(width*4,height*4) (http://bengal.missouri.edu/~kes25c/castle_spline36.png)

Lighthouse image:

nnedi3_rpow2(rfactor=4,nsize=0,nns=4,qual=2,cshift="Spline36Resize") - SQUARE (http://bengal.missouri.edu/~kes25c/lighthouse_square.png)
nnedi3_rpow2(rfactor=4,nsize=0,nns=4,qual=2,cshift="Spline36Resize") - ABS (http://bengal.missouri.edu/~kes25c/lighthouse_abs.png)
spline36resize(width*4,height*4) (http://bengal.missouri.edu/~kes25c/lighthouse_spline36.png)

Nothing huge, but some difference. I did this just to see what would happen since it only requires running the training program and no actual work :). Am going to try minimizing sqrt(abs(error)) next. Based on some initial tests on a small amount of training data I think it may offer more improvements for image enlargement.

[markanini]

It's a noticeable improvement. :)

[jpsdr]

@tritical : Do you plan to update the 64bits version with ABS ?

[tritical]

Yes, but I'm thinking about changing it so that the weights are not compiled into the binary. There would be a string that points to a weight file. Then I don't have to have multiple versions of a 6MB dll on my web space (which is pretty limited) - only one copy of each set of weights. Although half of that 6MB size is due to the nns=256 weights, so I'm considering just dropping the 256 setting since it is too slow for video and does not offer much visual improvement over 128... metric wise the improvement is the same as 64->128. Also, the training takes a LONG time.

[Gser]

Yes, but I'm thinking about changing it so that the weights are not compiled into the binary. There would be a string that points to a weight file. Then I don't have to have multiple versions of a 6MB dll on my web space (which is pretty limited) - only one copy of each set of weights. Although half of that 6MB size is due to the nns=256 weights, so I'm considering just dropping the 256 setting since it is too slow for video and does not offer much visual improvement over 128... metric wise the improvement is the same as 64->128. Also, the training takes a LONG time.

I have web space to spare if you are in need.

[Archimedes]

Thanks for the update, tritical. Indeed an improvement.

Source (400x300) >> (http://img263.imageshack.us/i/sn850652.png/)

Spline36Resize(1600, 1200) >> (http://img13.imageshack.us/i/sn850652spline36resize.jpg/)
nnedi2_rpow2(rfactor=4, nsize=2, qual=2, cshift="Spline36Resize") >> (http://img508.imageshack.us/i/sn850652nnedi2.jpg/)
nnedi3_rpow2(rfactor=4, nsize=0, nns=4, qual=2, cshift="Spline36Resize"), SQR >> (http://img13.imageshack.us/i/sn850652nnedi3sqr.jpg/)
nnedi3_rpow2(rfactor=4, nsize=0, nns=4, qual=2, cshift="Spline36Resize"), ABS >> (http://img851.imageshack.us/i/sn850652nnedi3abs.jpg/)

[jpsdr]

@tritical : Any news about a final version (32 & 64 bits) of nnedi3 with abs ?

[mandarinka]

BTW It seems there is no nnedi3 (for non-sse2 cpus) in the old_stuff folder.

[Sapo84]

I've been toying with nnedi3 (abs version) for the past days and I've found a somewhat strange behaviour.

When using

nnedi3_rpow2(rfactor=2,nns=2,nsize=3,cshift="Spline36Resize")

I think the filter is not doing the chroma shift part quite correctly.
In fact I think the filter actually does something very similar to

nnedi3(1, dh=true,nns=2,nsize=3).turnright().nnedi3(1, dh=true,nns=2,nsize=3).turnleft()
a=last
Spline36Resize(OX,OY,-0.5,-0.5,OX*2,OY*2)
MergeChroma(a.Spline36Resize(OX,OY,-0.5,-1.00,OX*2,OY*2))

Which is obviously not correct, because the correct version should end with something like

MergeChroma(b.Spline36Resize(OX,OY,-1.00,-1.00,OX*2,OY*2))

If I'm correct the filter probably needs some kind of correction.

On another note it would be helpful (to me, at least) if nnedi3_rpow2 had more parameters, I'd like to have the option to disable the chroma planes processing and to skip the chroma shift, I know it can be easily done with two nnedi3 calls, but it would help in writing clearer scripts.

[Gavino]

I think the filter is not doing the chroma shift part quite correctly.
In fact I think the filter actually does something very similar to
nnedi3(1, dh=true,nns=2,nsize=3).turnright().nnedi3(1, dh=true,nns=2,nsize=3).turnleft()
a=last
Spline36Resize(OX,OY,-0.5,-0.5,OX*2,OY*2)
MergeChroma(a.Spline36Resize(OX,OY,-0.5,-1.00,OX*2,OY*2))

Which is obviously not correct, because the correct version should end with something like
MergeChroma(b.Spline36Resize(OX,OY,-1.00,-1.00,OX*2,OY*2))
Why do you think that is not correct?
Avisynth assumes MPEG-2 sampling (http://avisynth.org/mediawiki/Sampling#mpeg-1_versus_mpeg-2_sampling), so chroma pixels are vertically aligned with luma.
Hence, the horizontal shift correction is the same (-0.5) for both luma and chroma, and it is only the vertical one which differs (-0.5 for luma and -1.0 for chroma).

[Sapo84]

Why do you think that is not correct?
Avisynth assumes MPEG-2 sampling (http://avisynth.org/mediawiki/Sampling#mpeg-1_versus_mpeg-2_sampling), so chroma pixels are vertically aligned with luma.
Hence, the horizontal shift correction is the same (-0.5) for both luma and chroma, and it is only the vertical one which differs (-0.5 for luma and -1.0 for chroma).

Well, that may be true (and in fact, I have no idea why that should not work), but then it means there's something fishy going on.

UToY() applied to the clip before processing
http://www.cbland.net/images4/01azazel.png

UToY() applied to the clip after -0.5 -1.0 chroma resize
http://www.cbland.net/images4/01azazel050100.png

UToY() applied to the clip after -1.0 -1.0 chroma resize
http://www.cbland.net/images4/01azazel100100.png

The second one is different from 1 and 3 (which are basically the same), that'd why I assumed -1.0 -1.0 was the way to go.

Edit: Did try with a different source, same problem.

[Gavino]

UToY() applied to the clip before processing
http://www.cbland.net/images4/01azazel.png

UToY() applied to the clip after -0.5 -1.0 chroma resize
http://www.cbland.net/images4/01azazel050100.png

UToY() applied to the clip after -1.0 -1.0 chroma resize
http://www.cbland.net/images4/01azazel100100.png

The second one is different from 1 and 3 (which are basically the same), that'd why I assumed -1.0 -1.0 was the way to go.
Can you clarify the exact scripts used to produce those three results, please.
I can't see any difference visually between them, but using Compare shows they are all different, with 2 and 3 being (by a small margin) the most similar.
1 v 2: PSNR=51.99dB
1 v 3: PSNR=53.98dB
2 v 3: PSNR=55.88dB

[Sapo84]

Can you clarify the exact scripts used to produce those three results, please.
UToY()
___________
nnedi3(1, dh=true,nns=2,nsize=3).turnright().nnedi3(1, dh=true,nns=2,nsize=3).turnleft()
a=last
Spline36Resize(OX,OY,-0.5,-0.5,OX*2,OY*2)
MergeChroma(a.Spline36Resize(OX,OY,-0.5,-1.00,OX*2,OY*2))
UToY()
___________
nnedi3(1, dh=true,nns=2,nsize=3).turnright().nnedi3(1, dh=true,nns=2,nsize=3).turnleft()
a=last
Spline36Resize(OX,OY,-0.5,-0.5,OX*2,OY*2)
MergeChroma(a.Spline36Resize(OX,OY,-1.0,-1.00,OX*2,OY*2))
UToY()

I can't see any difference visually between them, but using Compare shows they are all different, with 2 and 3 being (by a small margin) the most similar.
Well, the difference is 1/4 of a pixel, you need to zoom in A LOT.
I did a quick crop and zoom.
http://www.cbland.net/images4/Azazel_01.png
http://www.cbland.net/images4/Azazel_02.png
http://www.cbland.net/images4/Azazel_03.png

While 1 and 3 are pretty much identical the 2 suffer from a little shift, and it's the one with the (-0.5,-1) chroma shift, which should be the correct one.

Edit: After a bit of thinking I may have the reason why -1 -1 is correct, in fact nnedy does never resize horizontally, it just does Turnleft() and then Turnright(), so the chroma does get shifted by 1 pixel in both directions.

[Gavino]

Are OX and OY the original clip width and height?
So you are upsizing with nnedi3 and downsizing back to the original size?
I'm confused about your original point now. Where does nnedi3_rpow2 come into it?

After a bit of thinking I may have the reason why -1 -1 is correct, in fact nnedy does never resize horizontally, it just does Turnleft() and then Turnright(), so the chroma does get shifted by 1 pixel in both directions.
But because of the difference between vertical and horizontal chroma/luma alignment (MPEG-2 sampling), there is no chroma shift from the horizontal resize. See the discussion starting at post #12 of this thread. As I said in post #14:
for the horizontal resizings, you need to use field=0 if doing TurnLeft.Nnedi2().TurnRight() and field=1 for TurnRight().Nnedi2().TurnLeft(). In either of these cases, there will be no shift at all.

[Sapo84]

Are OX and OY the original clip width and height?
Yeah, I should have wrote a.width() and a.heigth() for clarity sake.

So you are upsizing with nnedi3 and downsizing back to the original size?
I'm confused about your original point now. Where does nnedi3_rpow2 come into it?
nnedi3_rpow2(rfactor=2) and then resizing back to the original resolution is equal to the second script.
And I'm finding the chroma is shifted by 1/4 of a pixel (also I would like to have more control on the parameters, because I'd like to write a script like naa() and I would like to skip chroma processing (if I don't want to process chroma) or chroma shifting (I would like to do it afterwards), but these are just feature request).

But because of the difference between vertical and horizontal chroma/luma alignment (MPEG-2 sampling), there is no chroma shift from the horizontal resize. See the discussion starting at post #12 of this thread. As I said in post #14:
But horizontal chroma is never touched, nnedi only does vertical doubling, and vertical doubling has even shift (chroma line is not aligned vertically).
(well, that depends on what TurnLeft() and TurnRight() actually do to the chroma plane)

[Didée]

If anything can beat theory, it is practice.

loadplugin("nnedi3.dll")

r = blankclip(width=64,height=64,pixel_type="YV12",color_yuv=$FF0000)
g = blankclip(width=64,height=64,pixel_type="YV12",color_yuv=$00FF00)

stackhorizontal(r,g)
stackvertical(last,last.fliphorizontal())
o=last

nnedi3_rpow2(2,cshift="spline36resize",fwidth=128,fheight=128)
nnedi3_rpow2(2,cshift="spline36resize",fwidth=128,fheight=128)
nnedi3_rpow2(2,cshift="spline36resize",fwidth=128,fheight=128)
nnedi3_rpow2(2,cshift="spline36resize",fwidth=128,fheight=128)
nnedi3_rpow2(2,cshift="spline36resize",fwidth=128,fheight=128)
nnedi3_rpow2(2,cshift="spline36resize",fwidth=128,fheight=128)
nnedi3_rpow2(2,cshift="spline36resize",fwidth=128,fheight=128)
nnedi3_rpow2(2,cshift="spline36resize",fwidth=128,fheight=128)
nnedi3_rpow2(2,cshift="spline36resize",fwidth=128,fheight=128)
nnedi3_rpow2(2,cshift="spline36resize",fwidth=128,fheight=128)
nnedi3_rpow2(2,cshift="spline36resize",fwidth=128,fheight=128)
nnedi3_rpow2(2,cshift="spline36resize",fwidth=128,fheight=128)
nnedi3_rpow2(2,cshift="spline36resize",fwidth=128,fheight=128)
nnedi3_rpow2(2,cshift="spline36resize",fwidth=128,fheight=128)
nnedi3_rpow2(2,cshift="spline36resize",fwidth=128,fheight=128)
nnedi3_rpow2(2,cshift="spline36resize",fwidth=128,fheight=128)

interleave(o,last)

return(last)

To my eyes, that's a horizontal chroma shift, indeed.

[Gavino]

(well, that depends on what TurnLeft() and TurnRight() actually do to the chroma plane)
TurnLeft and TurnRight preserve all original pixels, just re-arranging their positions. Hence the combination TurnRight().Nnedi3(1, dh=true).TurnLeft() does not introduce a chroma shift.

Strictly speaking, you could argue that TurnLeft and TurnRight should resample chroma but that would mean they were no longer lossless operations.

[Sapo84]

TurnLeft and TurnRight preserve all original pixels, just re-arranging their positions. Hence the combination TurnRight().Nnedi3(1, dh=true).TurnLeft() does not introduce a chroma shift.

Strictly speaking, you could argue that TurnLeft and TurnRight should resample chroma but that would mean they were no longer lossless operations.
Then I don't really know why in practice the chroma shift is there ^^''
Back to square one, I guess.


To my eyes, that's a horizontal chroma shift, indeed.
Exactly.

Edit: Also
nnedi3(1, dh=true,nns=2,nsize=3).turnright().nnedi3(1, dh=true,nns=2,nsize=3).turnleft()
a=last
Spline36Resize(128,128,-0.5,-0.5,256,256)MergeChroma(a.Spline36Resize(128,128,-1.0,-1.0,256,256))
when used instead of nnedi3_rpow2 does not introduce any chroma shift.

[tritical]

Gavino or Didée, any other thoughts on the horizontal shifting?

In terms of mpeg2 chroma placement nnedi2_rpow2() does not introduce a horizontal shift of chroma relative to luma... which to me can only mean that Avisynth's resizers do not assume mpeg2 chroma placement. I have not looked at the code to confirm this, but if it is the case then chaining many calls of nnedi3 (which assumes mpeg2 placement) with one of Avisynth's resizers (which assume mpeg1 maybe?) together would produce a shift as shown by Didée's script. However, if I change these lines of Didée's script:

nnedi3_rpow2(2,cshift="spline36resize",fwidth=128,fheight=128)

to

nnedi3_rpow2(2).converttorgb24().spline36resize(128,128,-0.5,-0.5,256,256).convertbacktoyuy2().converttoyv12()

there is no horizontal chroma shift. Which I believe confirms my suspicion as converttorgb24() + convertbacktoyuy2().converttoyv12() uses mpeg2 placement.


Anyway, I'm planning to make a new release of nnedi3 with significantly faster prescreening + squared/abs weights in the same binary (with a parameter to switch between) and would like to get this issue cleared up.

EDIT: after looking at the code it is indeed what I suspected. Avisynth's resizers are assuming mpeg1 chroma placement when performing subpixel shifting on YV12/YUY2 - which results in a leftward shift of the chroma. I'll adjust nnedi3 to take this into account.

[poisondeathray]

EDIT: after looking at the code it is indeed what I suspected. Avisynth's resizers are assuming mpeg1 chroma placement when performing subpixel shifting on YV12/YUY2 - which results in a leftward shift of the chroma. I'll adjust nnedi3 to take this into account.


Does this apply to all versions of avisynth , or were there some changes made to 2.6 alpha 3 that might change this behaviour ?

[Gavino]

Avisynth's resizers are assuming mpeg1 chroma placement when performing subpixel shifting on YV12/YUY2 - which results in a leftward shift of the chroma.
Hmm, you're right.
I don't know why this hasn't occurred to me before - it's because the chroma sampling grid is not horizontally centred and is similar in nature to the problem in Bob that I deduced was present with interlaced YV12.

I don't know whether this is a performance shortcut, an oversight, or (dare I say it) a bug.

Does this apply to all versions of avisynth , or were there some changes made to 2.6 alpha 3 that might change this behaviour ?
It applies to all previous versions too.
The chroma is shifted right when upsizing and left when downsizing, and the amount depends on the scale ratio - probably imperceptible in most practical cases, but it definitely exists.

[IanB]

Ahhhh! gawd more chroma positioning bugs. At first glance it seems to me the code dumbly assumes the chroma is positioned top left of the macrocell, not mpeg1 :confused:

The HResize code :- if (vi.IsYUY2()) {
pattern_chroma = func->GetResamplingPatternYUV(
vi.width >> 1,
subrange_left / 2,
subrange_width / 2,
target_width >> 1,
false, tempUV, env );
}
else if (vi.IsPlanar() && !vi.IsY8()) {
const int shift = vi.GetPlaneWidthSubsampling(PLANAR_U);
const int div = 1 << shift;

pattern_chroma = func->GetResamplingPatternYUV(
vi.width >> shift,
subrange_left / div,
subrange_width / div,
target_width >> shift,
true, tempY, env );
}

The VResize code :- if (vi.IsPlanar() && !vi.IsY8()) {
const int shift = vi.GetPlaneHeightSubsampling(PLANAR_U);
const int div = 1 << shift;

resampling_patternUV = func->GetResamplingPatternRGB(
vi.height >> shift,
subrange_top / div,
subrange_height / div,
target_height >> shift,
env);
}

The centre correction code in the Resampling Pattern generators :- // the following translates such that the image center remains fixed
double pos;
...
if (fir_filter_size == 1) // PointResize
pos = subrange_start;
else
pos = subrange_start + ((subrange_width - target_width) / (target_width*2));

[tritical]

Hopefully Gavino will explain what it should be (and why upsizing results in right shift and downsizing results in left) because after thinking about it some more I realized that my initial thoughts were not correct.. and after thinking it about it a little longer I am just more confused :p.

So for right now I am releasing v0.9.3 of nnedi3, which has the new prescreener and abs/square weights (switch with 'etype' parameter). It is linked in the first post. I will fix the chroma placement for YV12/YUY2 in the future... though it isn't really a problem unless you chain a bunch of calls together. This version is still missing a few abs weights for nns=4 (I think nsize=1/2/3). The new prescreener is more aggressive, but could be made even more aggressive since I can't tell any difference between it and the old prescreener at this point. Planning on addressing these issues and then releasing v1.0, as well as updating the 64-bit version.

[Gavino]

Hopefully Gavino will explain what it should be (and why upsizing results in right shift and downsizing results in left) because after thinking about it some more I realized that my initial thoughts were not correct.. and after thinking it about it a little longer I am just more confused :p.
I'm not sure yet what the solution is, but the way the shift happens is as follows.

The resizer core is designed to preserve the position of the image centre, and it does this for both luma and chroma independently.
This works fine for vertical resizing, since in that case the centres of the luma and chroma sampling grids coincide for all formats (including YV12, where the chroma samples are vertically positioned between the luma samples).

But for horizontal resizing (except YV24), the placement is such that the luma and chroma centres do not coincide - in each case the chroma centre is to the left of the luma centre.
Eg for YV12 and YUY2:
luma centre
|
Luma: L L L L ... L L L L ... L L L L
Chroma: C C ... C C ... C C
|
chroma centre
Since the distance between the two centres is proportional to pixel spacing, it changes on a resize.
For upsizing, pixel spacing decreases, so the chroma centre moves closer to the luma centre (hence to the right), and conversely for downsizing it moves to the left. Since the output chroma is calculated based on a fixed centre, the visual result is a corresponding chroma shift, right for upsizing and left for downsizing.

We can quantify the shift as follows.
For YV12 and YUY2, the chroma centre is 0.5 luma pixels to the left of the luma centre.
When resizing from width Win to Wout, output pixel spacing in terms of input is multiplied by Win/Wout.
Hence the net (rightwards) chroma shift is a distance of 0.5*(1-Win/Wout) input luma pixels.
For a 2x upsize, this equals 0.25, for a 2x downsize it's -0.5 (ie 0.5 to the left).

Similarly, for YV411, the net shift is 1.5*(1-Win/Wout), ie 3 times as much as for YV12/YUY2.

[IanB]

So we need to maintain the absolute pixel offset of chroma centre to luma centre, i.e. 0.5 pixels left for YV12/YV16/YUY2 and 1.5 pixels left for YV411.

Structurally it may help to move the centre offset calculation up a level into the [HV]Resize constructors so as to avoid fighting that calculation in the generator.

[Gavino]

Structurally it may help to move the centre offset calculation up a level into the [HV]Resize constructors so as to avoid fighting that calculation in the generator.
Yes, I was thinking along those lines too.

[Gavino]

So we need to maintain the absolute pixel offset of chroma centre to luma centre, i.e. 0.5 pixels left for YV12/YV16/YUY2 and 1.5 pixels left for YV411.
I think that's the end effect, but an easier way to look at it is that for horizontal resize, we want the first pixel to have the same absolute offset for both luma and chroma.

I think the following should work.
First remove the centre adjustment code from the resample_functions.
Then for FilteredResizeH, it is simply a matter of inserting at the start:
if (fir_filter_size != 1) // preserve image centre, except for PointResize
subrange_start += ((subrange_width - target_width) / (target_width*2));
For FilteredResizeV, you need to treat YV12 specially (changes in blue):
if (vi.IsRGB())
subrange_top = vi.height - subrange_top - subrange_height;

// offset to preserve image centre, except for PointResize
const double offset = (fir_filter_size == 1) ? 0 : ((subrange_width - target_width) / (target_width*2));

resampling_pattern = func->GetResamplingPatternRGB(vi.height, subrange_top+offset, subrange_height, target_height, env);

if (vi.IsYV12()) {
resampling_patternUV = func->GetResamplingPatternRGB(
vi.height >> 1,
subrange_top / 2 + offset,
subrange_height / 2,
target_height >> 1,
env);
}
else if (vi.IsPlanar() && !vi.IsY8()) {
const int shift = vi.GetPlaneHeightSubsampling(PLANAR_U);
const int div = 1 << shift;

resampling_patternUV = func->GetResamplingPatternRGB(
vi.height >> shift,
(subrange_top+offset) / div,
subrange_height / div,
target_height >> shift,
env);
}
What do you think?

[IanB]

I think you might be right, the offset for chroma needs to be the same as for luma with co-located chroma pixels (YUY2, mpeg2 YV12, etc).

But I think it is FilteredResizeH (Horizontal) that needs the special treatment.

[Gavino]

I think you might be right, the offset for chroma needs to be the same as for luma with co-located chroma pixels (YUY2, mpeg2 YV12, etc).

But I think it is FilteredResizeH (Horizontal) that needs the special treatment.
It's true that FilteredResizeH is the one that fails with the existing code. However, the real special case is YV12 vertical resize, since all the others have co-located chroma.

By moving the centre correction up to the constructors, it can be calculated always in units of luma pixels. Then the horizontal case becomes uniform and only YV12 vertical needs to be treated differently. (At least, I think so.)

I am assuming that other planar formats with vertically subsampled chroma would have co-located chroma. Is this correct? (As far as I know, we don't actually have any yet.)

[IanB]

Okay, with the existing code.

The value of ((subrange_width - target_width) / (target_width*2)) is independent of chroma scaling.

So the luma and chroma samplers get the same numerical correction value, i.e. the initial value of "pos".

This results in the chroma and luma centre points moving with respect to each other with the FilteredResizeH as explained in post #385 above. There is currently no consideration given to the co-location of the chroma positioning for YUY2, YV12 and YV411.

For the FilteredResizeV this operation is correct, but only because the chroma and luma centre points are coincident.

In summary, for co-located samples, i.e. YUY2, YV12 & YV411, the correction values needs to be scaled by the chroma subsampling to keep the centre points relative relationship. When the centre points are already coincident, i.e. vertical and mpeg1 then the same correction value needs to be applied.

[Gavino]

In summary, for co-located samples, i.e. YUY2, YV12 & YV411, the correction values needs to be scaled by the chroma subsampling to keep the centre points relative relationship. When the centre points are already coincident, i.e. vertical and mpeg1 then the same correction value needs to be applied.
Yep, that's it.

Are you also saying that vertically subsampled chroma is always centred in all planar formats? If so, my suggested code changes can be simplified. Also, I had inadvertently used subrange_ and target_width instead of _height in the earlier post. So my revised FilteredResizeV changes are:
if (vi.IsRGB())
subrange_top = vi.height - subrange_top - subrange_height;

// offset to preserve image centre, except for PointResize
const double offset = (fir_filter_size == 1) ? 0 : ((subrange_height - target_height) / (target_height*2));

resampling_pattern = func->GetResamplingPatternRGB(vi.height, subrange_top+offset, subrange_height, target_height, env);

if (vi.IsPlanar() && !vi.IsY8()) {
const int shift = vi.GetPlaneHeightSubsampling(PLANAR_U);
const int div = 1 << shift;

resampling_patternUV = func->GetResamplingPatternRGB(
vi.height >> shift,
subrange_top / div + offset,
subrange_height / div,
target_height >> shift,
env);
}

[tritical]

Put up version 0.9.4. Some more optimizations, plus more aggressive prescreener. I still can't tell the difference between the most aggressive new prescreener level (level 0) and the old prescreener. However, it is much much faster and flags a lot less pixels for the predictor nn. For small nsize and nns combinations where the prescreener computation was taking a significant chunk of the processing time in v0.9.2 the speedup is pretty good. Will work on fixing the chroma issues in nnedi3_rpow2 next. I'm probably going to add a parameter which specifies horizontal chroma placement.

[poisondeathray]

thanks!

[wOxxOm]

@tritical: is it possible to train/guide nnedi3 for such a low-contrast thin double lines like on the pocket slit below?

http://img42.imageshack.us/img42/7055/pocketslit.png

prescreener enabled (all levels) on the left vs disabled on the right.

[henryho_hk]

Put up version 0.9.4.

Thank you, tritical!

[aegisofrime]

Thanks tritical. Any possibility of a 64-bit build? :)

[Undead Sega]

Sorry for asking such a noobish question, but despite me actually using it recently, what is NNEDI3 ABS???

[tritical]

wOxxOm, could you upload a short sample of that video (a part that includes the frame you showed) somewhere? :thanks:

Undead Sega, nnedi3_abs is nnedi3 with predictor network weights trained to minimize l1 norm (absolute error) instead of the original training which minimized l2 norm (squared error).

[Undead Sega]

ohhh right, that's great I suppose...

Can you tell me exactly what does this mean by any chance? :)

[tritical]

The predictor networks in nnedi3 take in an area around the pixel in question, and, based on those pixel values, produce a guess for the missing pixel value. Those predictor networks are just mathematical models with sets of parameters (weights). To learn the parameters I take lots of training examples (grabbed from a 1000 frame video sequence containing images from many different sources), start with a random initial guess for the weights, and then perform online gradient descent training. Which simply means that I repeatedly grab a single training pattern, run the pattern through the model, compute the error in guessing the output value, compute the partial derivative of the error with respect to each weight (term) in the model, and then update the weights. Keep repeating until a minimum in the error surface is reached (progressive stops).

L1 (absolute error) vs L2 (squared error) simply define how I compute the error term. L2 will weight large errors more - an error of 20 will count 16 times as much as an error of 5 - whereas L1 will weight errors proportionally - an error of 20 will count 4 times as much as an error of 5. So with the L2 error measure the model will spend more 'effort' trying to fit the hardest cases since it will make more sense to fit the hard cases just a little better at the cost of some accuracy in a lot of the easy cases (for example, drop a single error from 100 to 20 while increasing many errors of 5 to 8, etc...). L1 will tip the balance more toward fitting the easier cases better. For deinterlacing, I think L1 makes more sense than L2 because after a certain point (say a miss of 20 or whatever) it doesn't matter if you miss the true value by 50 or 100 or 150 they will all look bad. However, the model will spend a lot of effort trying to fit the 150 case just a little better. On the other hand, if you are close to the true value a small difference can make a big visual improvement (for example: miss by 2 vs 10). The results of nnedi3_abs seem to back this up. In fact, I think going even further, using L0.5 - sqrt(abs(error)) - produces better results based on my initial tests.

[mandarinka]

Just an impression from me:
I was using qtgmc/srestore on a fieldblended anime dvd (ova from 1991), and I *think* I got better results from the old version ('etype=1' now I think). It was similar to that issue posted earlier - if there were paralel lines running close to each other, they would get less sharp and defined, compared to teh old method. I think there were some places where the new version did better but IIRC they weren'T that noticeable.

I didn't spot this watching of course, it was when comparing using interleave() - but how else is one suppossed to spot anything, right :)
As I said, it's jsut my impression from a brief testing on one source and through qtgmc. But if you are using that (and similarly, probably daa() with nnedi3 for antialiasing anime), you might want to run your own comparisons.

[Dogway]

I really like how nnedi3 evolved. I compared it to eedi3 which I once had it as the ultimate solution for strong AA and surprised myself on how nnedi3 outperformed it, which says a lot. Anyhow I observed that under occasional circumstances eedi3 was doing better. Maybe it is just due to the nature of the filter itself, but wanted to take it noted.

source(x2) eedi3 nnedi3
http://i212.photobucket.com/albums/cc35/Dogway/Misc/1694copy.png

http://i212.photobucket.com/albums/cc35/Dogway/Misc/1694copy2.png

[hamsterstyle]

What advantage is there running NNEDI3 as a de-interlacer rather than tdeint()? What kind of math does it do differently? Does it still make decisions about where the combed areas are by using a motion mask? Or is it purely spatial?

Also,

Is there an advantage to running say:
Yadifmod(Edeint=nnedi3()) or Tdeint(edeint=nnedi3()) as opposed to just running nnedi3 by itself?

[SSH4]

Dogway i played with nnedi/eedi too. replace nnedi to eedi on daa(). and result made me happy too on some anime source. nnedi3(pscrn=0) made shrper result (on daa()) but eedi on it made more correct result.
so both plugins are really usefull :)

[Chainmax]

So, as of v0.9.4 the default etype is better for image enlargements and for deinterlacing we should switch to etype=1, right?

[huhuyaya]

if can be used to downsize~
just like:LoadPlugin("nnedi3.dll")
nnedi3_rpow2(rfactor=-2,cshift="spline36resize")
that will be great

[Bloax]

There's no reason to use something as hefty as an (artificial) neural network ("NN"EDI) interpolator for something that's easily handled by something like Bilinear and Lanczos.

And thing is, NNEDI, if memory serves, is indeed "Neural-Network Edge-Directed Interpolator"
With interpolation needed for upscaling, since we need new data to fill the gaps.

With downscaling, we already have all the data we need.
So yeah, use Lanczos/Spline, or hell - BilinearResize(x,y) for that.

[Portioli]

hello

i would like to do some kind of supersampling (1080p --> 4K) using needi3,
then Sharpen the image using LimitedSharpenFaster
and then downsampling back to 1080p using lanczos or spline64resize.

could you please help me write the avisynth script?

[PhrostByte]

hello

i would like to do some kind of supersampling (1080p --> 4K) using needi3,
then Sharpen the image using LimitedSharpenFaster
and then downsampling back to 1080p using lanczos or spline64resize.

could you please help me write the avisynth script?

LSF already super-samples the clip internally, so this won't be as brilliant as you hoped.

[SubPixie]

I'd be curious to see if it looks good anyway, don't hesitate to post your findings Portioli

[Pulp Catalyst]

just a quick confirmation,

Startrek DS9 (really weird deintelacing, many here probably already know that a lot of 90's scifi programs are difficult to handle)

anyway, for this particular source is there any better combination than

"Yadifmod+NNEDI3"

for the record "QTGMC always crashes Xvid and x264 when using with Staxrip and MeGUI"

i'm just after confirmation if yadifmod+nnedi3 is as good as it get or near as, any feed back i would really appreciate it.

EDIT

DVD to Xvid/x264 (no resizeing, only cropping....MAX quality takes priority, but there is still an upper limit on file size i won't go over)

[SubJunk]

Have you seen the MEECH DS9 releases? He encoded all seasons at max quality.

If you still want to do it yourself, I recommend QTGMC. I know you said it crashes but I use it with MeGUI all the time with no crashes.

[Pulp Catalyst]

when you look at each episode, the part where the defiant is just about to vanish in the background (intro credits), you will see some savere interlace problems (most deinterlacers have trouble with this section because it's so small), i have tried many..... sources shall we say... and all suffer the same issue.

I have just installed Xvid4PSP, has QTGMC built in, seems to work flawlessly with this program..... go figure.

seeing what it can do now, just running quick test and will compare with yadifmod+nnedi3

downside to Xvid4PSP though is having to put in

SetMemoryMax(512)
SetMTMode(5,3)
......
SetMTMode(2)
......
......


on every episode, i have asked in forum, there is no way around it...... the template system is very good and mature in development in Xvid4PSP, however not so good when wanting to insert certain lines of code permanently on all future projects (like doing DVD episodes)

if i recall, i also believe that i found it very hard to get 100% cpu usage when doing Xvid encoding to which was annoying, a symptom that Staxrip and MeGUI did not have with the above additions to the avisynth script.

i did try for sometime to findout why QTGMC crashed in MeGUI and Staxrip (although it was the encoder that crashed in MeGUI, however in Staxrip, it was the program itself that stopped responding), tricky to work out, encoder crashed in one, but application crashed in the other????

gave up in the end however....... i put the problem on the backburner (meaning will deal with the issue once and for all when i do my next round of PC tower upgrade cycle), what tests i did do however i realized quite quickly that my tripple core could not be used on large scale jobs with QTGMC.... just not realistic when talking about 24 x 7 episodes LOL

[Chainmax]

I have some homemade footage that shows artifacting on the right side of the frame, which I am trying to antialias away. The filterchain is as follows:

SetMemoryMax(900)

SetMTMode(5,4)

MPEG2Source("X:\wherever")

SetMTMode(2)

a=last

b=a.Crop(704,0,16,576,align=true).nnedi3_rpow2(rfactor=2,nsize=3,nns=4,qual=2,pscrn=4,cshift="spline36resize").nnedi3_rpow2

(rfactor=2,nsize=3,nns=4,qual=2,pscrn=4,cshift="spline36resize").Spline36Resize(16,576)

Overlay(a,b,704)

rest of the filtering

Distributor()

I am getting a crash that VDubMod attributes to an out-of-bounds error that might be originated in nnedi3. How can I modify this to be able to encode?


[edit] I just noticed that I was doing this previous to bobbing. However, that was not enough: in addition to placing it after bobbing, I had to sandwich the nnedi3 line(s) between turnright() and turnleft().

[plonk420]

is there a way i can NNEDI3 + EEDI2/3 while keeping 60fps? (e.g. nnedi3's field = -2)

i tried EEDI3(field = -2, sclip=nnedi3(field = -2)), (as well as 2) but it has the "poorly deinterlaced video bounce"

edit: and can this be done with just built in filters?

[NicolasRobidoux]

http://www.imagemagick.org/discourse-server/viewtopic.php?f=22&t=21435&start=15#p88094 (NNEDI3) VS http://www.imagemagick.org/discourse-server/viewtopic.php?f=22&t=21435&start=15#p88102 (Elliptical Weighted Averaging with slightly sharpened Jinc-windowed Jinc 3 with sigmoidal halo minimization).
P.S. I should have written "experimental" instead of "state of the art": "sigmoidization" as a halo minimization method with negative lobe filters is just over 2 weeks old.
P.S.2 Better sigmoidal result: http://www.imagemagick.org/discourse-server/viewtopic.php?f=22&t=21435&p=88156#p88155

[wOxxOm]

second picture has more aliasing and blur with just a few better scaled small spots.

[pbristow]

Hey tritical, you seem to be the go-to guy for EDI solutions in AviSYnth... Have you ever considered doing an implementation of "Fast EDI-1.5"?
http://www.eie.polyu.edu.hk/~wcsiu/paper_store/Conference/2010_Conf/2010_EUSIPCO_Wong-n-Siu_1569292991.pdf

The paper talks about it being both faster and more accurate for uscaling images by a factor of 1.5x (e.g. for going from 480p to 720p) than the traditional approach of using NEDI to go up to 2x, and then downsizing by 3/4. Could be a useful addition to the AviSynth upscaling armoury...?

[active1]

is the nnedi3 resizer better than ResampleHQ?

[pbristow]

is the nnedi3 resizer better than ResampleHQ?

In some ways yes, in some ways no... They're each tackling completely different problems with the resizing process: NNEDI fills in missing *detail* (sort of) to prevent staircase artefacts and broken lines, while Resample HQ works around the side-effects of non-linear sampling of the colour values of pixels. Which one you would chose to use depends on which is the bigger problem in the case you're dealing with.

Ideally we would have NNEDI-ised version of Resample HQ, (or to put it another way, a gamma-aware version of NNEDI3) to combine the benefits of both...

[active1]

In some ways yes, in some ways no... They're each tackling completely different problems with the resizing process: NNEDI fills in missing *detail* (sort of) to prevent staircase artefacts and broken lines, while Resample HQ works around the side-effects of non-linear sampling of the colour values of pixels. Which one you would chose to use depends on which is the bigger problem in the case you're dealing with.

thanx, but can you please explain what kinds of images that i should use NNEDI3 or ResampleHQ with?
for example, if i had a linear image and i want to resize it with gamma-correction then i should choose ResampleHQ, right? if it is, what about NNEDI3?

Ideally we would have NNEDI-ised version of Resample HQ, (or to put it another way, a gamma-aware version of NNEDI3) to combine the benefits of both...
that would be great :)

[pbristow]

thanx, but can you please explain what kinds of images that i should use NNEDI3 or ResampleHQ with?
for example, if i had a linear image and i want to resize it with gamma-correction then i should choose ResampleHQ, right? if it is, what about NNEDI3?


Well, no, if your image is *already* sampled linearly, and you want the output to stay linear, then the regular resamplers will work correctly. But the thing is most digital video *isn't* sampled linearly, which is why the regular resamplers/resizers tend to slightly darken any high-contrast details (such as stars in the night sky). Resample HQ converts the input to a high-quality linear version, resizes *that*, then converts back to the original sample format.

The difference from straightforward resizing of the non-linear version is small, though, and not noticable on most images: the biggest effect is on small *points* of detail (like stars or tiny specks of film dirt), where one or two pixels at one brightness level are surrounded by others at a very different level.

If your image has a lot of bold *edges* in it - especially straight (or nearly-straight) lines that continue for many pixels in each direction - then you can benefit from Edge Directed Interpolation, which is what NNEDI3 and it's stable-mates provide. It's a technique that recognises parts of the image that are supposed to be hard, straight-ish edges, and makes sure they *stay* hard straight-ish edges after up-sizing. Otherwise they either break up into stair-case effects, or develop a kind of "rippling blur" along the edges that looks almost as bad, especiially if the camera is moving because then the effects move along the edges, which is really eye-catching. Things like prizons bars, window blinds, and brick walls - and dare I say it? - fishnet tights look much better using EDI than normal resizers. :)

EDI is a slow process though: Even with the simplest, fastest, "do as little to help as possible" settings, NNEDI3 is much slower than a standard resizer. And it's only really worth it if you're up-sizing a long way, i.e. at least 40% enlargement. The actual EDI step always enlarges x2, and then you can use a normal resizer to nudge that up or down to your actual required size.

What I usually do is start out with a lightweight but decent quality resizer such as Lanczos, convert the whole video through using that, then watch the result and make a note of any sections that show obvious stair-casing on the edges. Then I re-process those particular scenes through NNEDI3 on moderate settings, compare them with the Lanczos versions, and if they look significantly better then I edit them in to place (or edit my script to do so automatically).

I've never yet found a video in my own collection where I really needed to use Resample HQ. Some where it would have helped, yes, but not where the difference would be enough make me bother with it, especially if the same image has lines that need NNEDI3 more.

[active1]

Well, no, if your image is *already* sampled linearly, and you want the output to stay linear, then the regular resamplers will work correctly. But the thing is most digital video *isn't* sampled linearly, which is why the regular resamplers/resizers tend to slightly darken any high-contrast details (such as stars in the night sky). Resample HQ converts the input to a high-quality linear version, resizes *that*, then converts back to the original sample format.

The difference from straightforward resizing of the non-linear version is small, though, and not noticable on most images: the biggest effect is on small *points* of detail (like stars or tiny specks of film dirt), where one or two pixels at one brightness level are surrounded by others at a very different level.

If your image has a lot of bold *edges* in it - especially straight (or nearly-straight) lines that continue for many pixels in each direction - then you can benefit from Edge Directed Interpolation, which is what NNEDI3 and it's stable-mates provide. It's a technique that recognises parts of the image that are supposed to be hard, straight-ish edges, and makes sure they *stay* hard straight-ish edges after up-sizing. Otherwise they either break up into stair-case effects, or develop a kind of "rippling blur" along the edges that looks almost as bad, especiially if the camera is moving because then the effects move along the edges, which is really eye-catching. Things like prizons bars, window blinds, and brick walls - and dare I say it? - fishnet tights look much better using EDI than normal resizers. :)

EDI is a slow process though: Even with the simplest, fastest, "do as little to help as possible" settings, NNEDI3 is much slower than a standard resizer. And it's only really worth it if you're up-sizing a long way, i.e. at least 40% enlargement. The actual EDI step always enlarges x2, and then you can use a normal resizer to nudge that up or down to your actual required size.

What I usually do is start out with a lightweight but decent quality resizer such as Lanczos, convert the whole video through using that, then watch the result and make a note of any sections that show obvious stair-casing on the edges. Then I re-process those particular scenes through NNEDI3 on moderate settings, compare them with the Lanczos versions, and if they look significantly better then I edit them in to place (or edit my script to do so automatically).

I've never yet found a video in my own collection where I really needed to use Resample HQ. Some where it would have helped, yes, but not where the difference would be enough make me bother with it, especially if the same image has lines that need NNEDI3 more.

thanks for the explaination, it was really benefit for me :)

EDI is a slow process though: Even with the simplest, fastest, "do as little to help as possible" settings, NNEDI3 is much slower than a standard resizer. And it's only really worth it if you're up-sizing a long way, i.e. at least 40% enlargement.

what if i want to down-size my video, is there any benefits of using EDI too?

[pbristow]

thanks for the explaination, it was really benefit for me :)

You're welcome! [TIPS HAT] :)

what if i want to down-size my video, is there any benefits of using EDI too?

Generally, no. The avisynth resizers already do a good job of using all the available information to calculate the new pixels, so there's no benefit to calculating *extra* pixels first. The one possible exception I can think would be anime: if the "pencil lines" around faces etc. are already a bit blurred or broken, then maybe EDI would help "reinforce" them before downsizing...? I don't know, I haven't tried it that way.

[Bloax]

nnedi3 (Do people really use the others?) Is already kind of blurry, so I can't really see how it would help to upscale blurry lines with a soft upscaler.

Though FastLineDarken (?) or something like that probably would help with what you're thinking about there. (Though of course not for video content.)

So yeah, there's no actual reason to do so. Unless you'd like to do a PointResize that isn't (1/2, 1/3, 1/4, etc.) Of the original.

[mzso]

Stupid question: NNEDI and EEDI are acronyms for what exactly?

[Reel.Deel]

By looking at the source code (eedi3.cpp) EEDI stands for enhanced edge directed interpolation. I'm not completely sure on this one but NNEDI stands for neural network edge directed interpolation.

Edit: Thanks for the clarification pbristow. Also thanks for the link. :)

[pbristow]

By looking at the source code (eedi3.cpp) EEDI stands for enhanced edge directed interpolation. I'm not completely sure on this one but NNEDI stands for neural network edge directed interpolation.

Yep, although there's also the word "New" in the documentation, i.e. "Neural Network New Edge Directed Interpolation"... So arguably NNEDI should be called NNNEDI! :)

"New Edge Directed Interpolation" is the name of a specific EDI algorithm (See: http://elynxsdk.free.fr/ext-docs/Demosaicing/more/news0/New%20Edge-Directed%20Interpolation.pdf ). If I understand correctly, NNEDI implements that same basic algorithm, but instead of using weightings/coefficients calculated by a theoretical analysis, or even by human trial-and-error, it uses the values selected by a neural network that was trained on a whole lot of different video clips to give the best results.

[Pulp Catalyst]

done quite a lot of reading, for standard Disney animation films,

what would a high quality setup be (but still keeping things within realistic reaches), i heard that QTGMC can cause issue at times with animation,
i've read in places that eedi3+nnedi3 would be brilliant for animation (albeit slow), i am using a 3770k so that should help a little.

i've just purchased a DVD boxset of disney animated films (yes exspensive, but it was going on a extremely good deal)

i use MeGUI + Avisynth MT (and QTGMC on interlaced stuff),
my issue is that reading things where QTGMC was not designed really for animation has given me reason to pause,

can someone give the the line that would be needed in avisynth for the above eedi3+nnedi3 combo..... or any other alternatives would be great too.... i'm looking for very high quality but of course without loosing sense of reality....LOL

20-25FPS minimum really (i get around 36fps using QTGMC with slow pre-set and select even to Xvid, same setup but going to x264 i get around 23-26fps (x264 set to slower))

i welcome any feedback on this, it's a big project for me (over 40-50 films i think, haven't actually counted) so i would like to get this right first time round if you know what i mean?

[Chikuzen]

done quite a lot of reading, for standard Disney animation films,

what would a high quality setup be (but still keeping things within realistic reaches), i heard that QTGMC can cause issue at times with animation,
i've read in places that eedi3+nnedi3 would be brilliant for animation (albeit slow), i am using a 3770k so that should help a little.

(I am not detailed since I hardly see TV and movies) Are the Disney works 60fps?

I think that you should do IVTC instead of Bob.

[paradoxical]

i've just purchased a DVD boxset of disney animated films (yes exspensive, but it was going on a extremely good deal)

Wouldn't it have just been better to buy them on BluRay at not have to fiddle with upscaling the DVDs? The BluRays have great restoration jobs. Also, which animated Disney films are interlaced that you would need QTGMC?

[kolak]

Almost none.

[Pulp Catalyst]

i've never seen a 48 film Disney box-set on bluray.... didn't know disney had converted all there backlog of disney films of to bluray..... did a quick search can't find anything, but i'm not saying your wrong however.

Wouldn't it have just been better to buy them on BluRay at not have to fiddle with upscaling the DVDs?

so i'm not sure how i could do that? and i never said i was upscaling either..... i'm keeping the same resolution, just converting straight to Xvid using MeGUI.

anyway onwards, yes they are PAL, and yes they do need deinterlacing, and usually QTGMC would be the way togo.... but like i said, i have read that QTGMC does not play nice with animation.... and some on this forum have suggested that QTGMC should not be used on animation.....

so, what alternative is there?
i found a topic suggesting that a combo of yadifmod+eedi3+nnedi3 would be really high quality and would actually work wonders with animation!

but the topic did not show an avisynth line, nor did it go into detail regarding how it should be implemented in avisynth, which is why i'm asking here if anyone knows the avisynth line i need to get this combo working?

[kolak]

Are you sure they are interlaced?
Most PAL animation are 25p, not 50i. Are they recent or older ones?
Sample, grab with interlacing?

Yes- you can use yadifmod+NNEDI3- it will be good.

some sourcefilter()
assumetff()
yadifmod(mode=0,edeint=nnedi3(field=1))

This will deinterlace 50i Top Field source into 25p file.

[Pulp Catalyst]

no, the combo i saw was either yadifmod+eedi3+nnedi3

or eedi3+nnedi3

nnedi3 is already used by QTGMC, besides nnedi3 used on animation is terrible.... (well, no, but because of the straight lines, still not very good)

but apparently eedi3+nnedi3 is suppose to be really good (even with Classic animation)

not sure if the DVD's are interlaced, just assumed they were as they are PAL?
will check tomoz on that, getting late now.

EDIT

just ripped and checked first one, your right, it's not Interlaced....
excellent, all that reading for nothing LOL

[Chortos-2]

What’s wrong with NNEDI3 on animation? What’s wrong with QTGMC on interlaced animation? Finally, there is QTGMC(EdiMode="EEDI3+NNEDI3").

[Pulp Catalyst]

What’s wrong with NNEDI3 on animation? What’s wrong with QTGMC on interlaced animation?

http://forum.doom9.org/showpost.php?p=1496803&postcount=649

Finally, there is QTGMC(EdiMode="EEDI3+NNEDI3")

maybe that was it, thanks.... i'll give it a go at some point in the future.

[mandarinka]

Well, that may be true but the problem is that nothing better is there.

So if you need to antialias, you don't have other options than stuff based on NNEDI3. Similarly, if your source is field-blended and you chose to bob it to use srestore, QTGMC with NNEDI3 is by far the best quality option we have.

The fact that it might not be optimally geared for animation is sad, but what is it worth if there actually aren't alternatives that perform better (or just on par)...

So no, nnedi3 is good for animation too...

P.S.:
There is no interlaced animation. It might look so, but in such cases it is merely telecine, which you must use IVTC filters on instead. The only exception is as I said, when this fundamentally progressive video (again, nothing is animated as interlaced) gets blended (old field-blended sources), or if the animation team compositions (overlays, crossfades, ...) telecined clips in such way that the results are not IVTCable cleanly anymore. In such cases, sometimes QTGMC/srestore is a workable solution (even if conceptually it is not correct). Sometimes it is not (often IVTC + postprocessing looks better).

[Chortos-2]

http://forum.doom9.org/showpost.php?p=1496803&postcount=649To mandarinka’s post I want to add that the linked post talks about QTGMC and has nothing to do with nnedi3. (And the other way round, it applies to QTGMC even when eedi3+nnedi3 is used for interpolation.)

There is no interlaced animation.Firstly, if you haven’t met it, it doesn’t mean it doesn’t exist. Then, as you note yourself, production companies unfortunately have the tendency to butcher telecined and progressive footage with interlaced mastering chains. Given enough butchering, the result becomes literally interlaced. (How about speeding up a clip by dropping one field from each frame? Perhaps they even properly alternate between dropping top and bottom fields to prevent bobbing.)

[Pulp Catalyst]

well i remember reading that if you get eedi3+nnedi3 working together (although at great cost in performance) this rare issue can be nullified to a certain extent,

i'm hoping that Finally, there is QTGMC(EdiMode="EEDI3+NNEDI3")

is just that, although haven't tested yet!

it was a long time ago i read somewhere that these two can work together, and one situation where that maybe the case would be in classic animation video due to the way classic animation is (a lot of straight lines i guess), anyway i do recall reading that classic animation can even cause issue with nnedi3 sometimes, however when combined with eedi3, that issue is now dealt with, although at great COST!

again, thanks Chortos-2, haven't confirmed it yet, but will give it a try at the weekend hopefully

i run a auto detector using MeGUI.... i know, not a reliable source, done it on 7 so far though, all reported to be progressive.

[Mystery Keeper]

Getting 3/4 of chroma zeroed on YV24. Also, any chance to have these plugins work on stacked 16 bit in near future?

[njahnke]

a bit off topic, but any tips for building nnedi3 with gcc, specifically the inline assembly? it seems to be the wrong format for this compiler ...

[Mystery Keeper]

If you're building for Windows - I would discourage you from building with GCC. MSVC produces faster code, and in AviSynth speed is rather critical. If you want to build it for AvxSynth to use in Linux - yes, you have to rewrite the assemblies for AT&T syntax. Read this. (http://www.ibiblio.org/gferg/ldp/GCC-Inline-Assembly-HOWTO.html)

[JEEB]

Just going to note it here that akupenguin ported NNEDI3 to something cross-platform quite some time ago, and it can be gotten either as a source archive (http://akuvian.org/src/x264/nnedi.tar.bz2), or as a git repository archive (http://akuvian.org/src/x264/nnedi.git.tar.bz2). This, methinks, is not an Avisynth plugin, though.

The main reason why Avisynth plugins aren't recommended to be built with GCC is because the C++ ABI differs between compilers, and at times even between versions of compilers, as it is not set (unlike C, which more or less by accident actually has an ABI specified -- which is why you can compile CPlugins with GCC just fine). So even if you got the library built, it might just not load up because of the ABI differences.

And yes, many things use MSVC-specifics which then will not compile on GCC or any other compiler that doesn't replicate its behavior.

[x265]

Should i use NNEDI3 or EEDI3 for removing the aliasing in anime?

[Sapo84]

Depends on how much aliasing is there, the speed you want to achieve and the type of aliasing script you're going to use (a daa() like script is more effective than supersampling and then spline36resizing back to the original resolution).
EEDI3 has a much stronger antialiasing effect (it connects more lines) but is a lot slower.
If you're going to process every frame of a film/episode you may want to use something faster.
If NNEDI3 effect is too weak you could try EEDI2, it's fast and it connects more lines than NNEDI.
Try and see what's better for you, I used both depending on the anime source (also, you may want to filter the edges only. I usually do, because I'm not too fond of the grainy looks that NNEDI3 sometimes has on uniform color parts).
(I also saw EEDI3 being used, but that was a monstrosity (4 eedi3 chained) that took 30s to render a single frame of B Gata H Key).

[pbristow]

[PERKS UP EARS]
...daa() ?

[GOOGLES "Avisynth DAA"]

Ooh! I must try that one. :)

[mandarinka]

It depends on the strength of the aliasing.

I would generally start with something like spline36resize(width,height*3/2).daa().spline36resize(width,height)
Naturally you must first modify daa() do use nnedi3 instead of old nnedi.

If your aliasing is mild and it is a product of computer rendering (pixelization) and not a result of interlacing, then maybe you might want to try this thing I posted recently: http://forum.doom9.org/showthread.php?t=167480
It is more detail-safe than daa().

[Selur]

btw. would be nice if eedi3&co could be recompiled with a newer Visual Studio, it's kind of annoying to always stumble over the missing visual studio 2005 runtime ;)

[StainlessS]

Yes, it's so much nicer to stumble over a missing 2008 or 2010 runtime. :)

[Selur]

You are right a static build would be the best alternative, but if static is not possible for some reason the change that I'm missing the 2010 runtime is far smaller for me. :D

[Reel.Deel]

Would be nice if eedi3&co could be recompiled with a newer Visual Studio...

A while back 06_taro recompiled EEDI3 with ICL12. Here you go:
[EEDI3_ICL12 (http://nmm.me/oe)], [EEDI3_ICL12_static (http://nmm.me/of)]

For NNEDI3 how about using -Vit-'s version? It's compiled with MSVC10.
-Vit- has a separate 2.5 (http://forum.doom9.org/showpost.php?p=1503034&postcount=691) / 2.6 (http://forum.doom9.org/showpost.php?p=1568142&postcount=1126) plugin package. I've been using the 2.6 one for a while and everything worked nicely.
I recently upgraded to AVS alpha4 and had to switch from his 2.6 to the 2.5. So far, everything seems to be working as good as it did.

[wOxxOm]

on default settings nnedi3_rpow2 is about 20% faster (39 -> 47 fps @ 480p -> 960p) when recompiled (https://dl.dropboxusercontent.com/u/40341141/nnedi3-tp7.7z) against FTurn (http://forum.doom9.org/showthread.php?p=1638416) instead of avisynth's built-in turnleft/turnright.

Edit1: nnedi3_rpow2(), not nnedi3()
Edit2: FTurn plugin should be (auto)loaded so that nnedi3_rpow2 linked in my post could use it, see the discussion below.

[Selur]

Nice, thanks for sharing :)

[TurboPascal7]

on default settings nnedi3 is about 20% faster (39 -> 47 fps @ 480p -> 960p) when recompiled (https://dl.dropboxusercontent.com/u/40341141/nnedi3-tp7.7z) against FTurn (http://forum.doom9.org/showthread.php?p=1638416) instead of avisynth's built-in turnleft/turnright.

Only nnedi3_rpow2, this has no effect on nnedi since it doesn't use turns.

[Groucho2004]

on default settings nnedi3 is about 20% faster (39 -> 47 fps @ 480p -> 960p) when recompiled (https://dl.dropboxusercontent.com/u/40341141/nnedi3-tp7.7z) against FTurn (http://forum.doom9.org/showthread.php?p=1638416) instead of avisynth's built-in turnleft/turnright.
I can't confirm that. I get 5% speedup at most with the "official" Avisynth 2.6 Alpha4 (i5 2500K).
When I rebuild avisynth and the original nnedi3 with ICL10 it's actually a bit faster than the DLL you posted.

[wOxxOm]

Sorry for the confusion, Groucho2004, I've fixed my post above, nnedi3_rpow2, of course.

[Selur]

@Groucho2004: would be nice if you could share the icl10 build of nnedi3.

[Groucho2004]

@Groucho2004: would be nice if you could share the icl10 build of nnedi3.

Here (http://www.mediafire.com/download/ft1ophbu6a7b38u/nnedi3_icl10.zip) you go.

[Groucho2004]

Sorry for the confusion, Groucho2004, I've fixed my post above, nnedi3_rpow2, of course.

That's what I was referring to as well.

[Selur]

Here you go.
Thanks :D

[Groucho2004]

on default settings nnedi3_rpow2 is about 20% faster (39 -> 47 fps @ 480p -> 960p) when recompiled (https://dl.dropboxusercontent.com/u/40341141/nnedi3-tp7.7z) against FTurn (http://forum.doom9.org/showthread.php?p=1638416) instead of avisynth's built-in turnleft/turnright.
OK, did some tests using "official" Avisynth 2.6 Alpha4:

The test clip is a short 720p avc video.

Script:
LoadPlugin("DGDecodeNV.dll")
LoadPlugin("nnedi3.dll")
DGSource("F:\Test\test.dgi")
NNEDI3_rpow2(rfactor = 2)

Results (AVSMeter):
nnedi3_tritical.dll : 26.70 fps (from tritical's website)
nnedi3_fturn.dll : 28.14 fps
nnedi3_icl10.dll : 28.28 fps

[TurboPascal7]

It looks like you didn't load fturn.dll. It would use avisynth's turn functions as usual in this case. "Compiled against" was an incorrect term. "Able to use if available" is.

[Selur]

So nnedi3_fturn.dll will use the fturn.dll if it is loaded in the script?

[wOxxOm]

So nnedi3_fturn.dll will use the fturn.dll if it is loaded in the script?Yes.
Or if fturn was autoloaded from plugins directory.

[TurboPascal7]

Let me just paste the code.
auto turnRightFunction = env->FunctionExists("FTurnRight") ? "FTurnRight" : "TurnRight";
auto turnLeftFunction = env->FunctionExists("FTurnLeft") ? "FTurnLeft" : "TurnLeft";
try
{
double hshift = 0.0, vshift = 0.0;
if (vi.IsRGB24())
{
for (int i=0; i<ct; ++i)
{
v = new nnedi3(v.AsClip(),i==0?1:0,true,true,true,true,nsize,nns,qual,etype,pscrn,threads,opt,fapprox,env);
v = env->Invoke(turnRightFunction,v).AsClip();
v = new nnedi3(v.AsClip(),i==0?1:0,true,true,true,true,nsize,nns,qual,etype,pscrn,threads,opt,fapprox,env);
v = env->Invoke(turnLeftFunction,v).AsClip();
}
hshift = vshift = -0.5;
}

Nothing fancy, one minute's job. If fast alternative is available - use it, otherwise go with default one. Optimizing this function in the core/writing nnedi3_rpow2 as a script would be better, but we get what we get. I'm surprised tritical didn't implement fast turns himself.

[Selur]

Okay, thanks, that clears it up. :)

[Groucho2004]

It looks like you didn't load fturn.dll. It would use avisynth's turn functions as usual in this case. "Compiled against" was an incorrect term. "Able to use if available" is.

Aaaahh, I see. Now that certainly changes things. With FTurn I get actually 57 fps (!) in the same scenario. Nice.

[Myrsloik]

Since we're on the subject of speed and turning...

Currently nnedi3_rpow2() does this internally:
nnedi3()
turnright()
nnedi3()
turnleft()

It should in theory be slightly faster if rearranged to:
turnright()
nnedi3()
turnleft()
nnedi3()

I didn't bother to test it myself but you'll be turning half the number of pixels so maybe it'll be a noticable improvement. Or maybe it'll look a lot different... who knows...

[PetitDragon]

Yes.
Or if fturn was autoloaded from plugins directory.

Hi wOxxOm,

If I do it with ffdshow in realtime, I just put both nnedi3_fturn.dll and fturn.dll in avisynth plugin folder, then nnedi3_rpow2 will use the new fturn automatically. right?

[wOxxOm]

PetitDragon, yeah. Anyway you can check in ffdshow if FTurn is autoloaded by explicit call: FTurnLeft().FTurnRight()

[PetitDragon]

PetitDragon, yeah. Anyway you can check in ffdshow if FTurn is autoloaded by explicit call: FTurnLeft().FTurnRight()

Got it! :thanks:

[aegisofrime]

Hi Groucho2004! While you are at it, do you think you could do a ICL10 compile of the Vapoursynth version of NNEDI3 as well?

[Groucho2004]

Hi Groucho2004! While you are at it, do you think you could do a ICL10 compile of the Vapoursynth version of NNEDI3 as well?
It won't make any difference. All the "speed-relevant" code is already ASM. A C compiler can't perform miracles. :)

[cretindesalpes]

I did some modifications on EEDI3 0.9.2 (https://ldesoras.fr/src/avs/eedi3-0.9.2.zip):
SSE/SSE2 optimizations, giving a nice 4× speedup.
Added an optional mask to process only the specified parts. It’s helpful when EEDI3 is used as an anti-aliasing processor on cartoon-like materials. Additional 1.2×–2× speedup, depending on the source.
I fixed a typo in the original code too, but I don’t know if it has any visible effect.
Be sure to have Visual C++ Redistributable for Visual Studio 2012 (http://www.microsoft.com/download/details.aspx?id=30679) installed (needed for vcomp11.dll).
The SSE2 code is 16-bit ready, although the glue code to pass stack16 clips hasn’t been implemented yet. It will probably come in a next update.

If you compare the C++ and SSE2 results, there are some rare and minor differences here and there, but I don’t know the exact reason at the moment (different rounding? bug?), and it’s difficult to tell which result is better.

[mandarinka]

Regarding eedi3... today I found that eedi3rpow2 displays the same chroma shift issue that nnedi3rpow2 has - see this post (http://forum.doom9.org/showthread.php?p=1499826#post1499826) and following discussion. Just so that people know.

Sadly I totally forgot that the issue is (still) there, only finding out today. Aw, I was using nnedi3rpow2 for antialiasing purposes...
I'm not sure how correct would Sapo84's scripted workaround (described back there) be, since I have no idea how exactly does nnedi3 implement the shifting mechanism in nnedi3rpow2.

Edit: Nice to see this update, cretindesalpes. Thanks!

[mawen1250]

Regarding eedi3... today I found that eedi3rpow2 displays the same chroma shift issue that nnedi3rpow2 has - see this post (http://forum.doom9.org/showthread.php?p=1499826#post1499826) and following discussion. Just so that people know.

Sadly I totally forgot that the issue is (still) there, only finding out today. Aw, I was using nnedi3rpow2 for antialiasing purposes...
I'm not sure how correct would Sapo84's scripted workaround (described back there) be, since I have no idea how exactly does nnedi3 implement the shifting mechanism in nnedi3rpow2.

Edit: Nice to see this update, cretindesalpes. Thanks!
nnedi3_rpow2 internally uses Spline36Resize to fix the vertical chroma shift introduced by nnedi3(dh=True, field=1).nnedi3(dh=True, field=0).nnedi3(dh=True, field=0)...
For horizontal scaling, nnedi3_rpow2 doesn't produce horizontal chroma shift for MPEG-2 chroma placement by nnedi3(dh=True, field=1).nnedi3(dh=True, field=1).nnedi3(dh=True, field=1)...(both of them are left-aligned)

When "cshift" is set, nnedi3_rpow2 uses AviSynth resizers to fix the center shift introduced by nnedi3. As for the chroma shift issues when "fwidth" is set, that's exactly the problem of AviSynth's resizers. For 4:2:x content, AviSynth's resizers always assume they are of MPEG-1 chroma placement and produce horizontal chroma shift.

Recently I wrote a script named Resize8, which fixes the chroma shift issues of AviSynth's resizers: http://www.nmm-hd.org/newbbs/viewtopic.php?f=7&t=1323
I have also written a script named nnedi3_resize16, which combines nnedi3 & Dither_resize16 for scaling & color space transferring, and it also correctly deal with center shift & chroma shift: http://www.nmm-hd.org/newbbs/viewtopic.php?f=7&t=1117

[Selur]

Be sure to have Visual C++ Redistributable for Visual Studio 2012 installed (needed for vcomp11.dll).
how about a statically compiled version?

[TurboPascal7]

how about a statically compiled version?
Out of curiosity: why?

[Selur]

Because having to keep track which avisynth dlls needs what runtime is a pain and atm. no other plugin is using the 2012 runtime, especially if you use Avisynth as a portable package.

[wOxxOm]

y8, yv16, yv24, please.

[zero9999]

I have also written a script named nnedi3_resize16, which combines nnedi3 & Dither_resize16 for scaling & color space transferring, and it also correctly deal with center shift & chroma shift: http://www.nmm-hd.org/newbbs/viewtopic.php?f=7&t=1117
here's (https://github.com/AviSynth/avs-scripts#eedi3_resize) a script for Firesledge's eedi3 mod that is based on mawen1250s nnedi3_resize16.

[feisty2]

here's (https://github.com/AviSynth/avs-scripts#eedi3_resize) a script for Firesledge's eedi3 mod that is based on mawen1250s nnedi3_resize16.

have you considered using eedi3+nnedi3 instead of pure eedi3/nnedi3, because eedi3 seems failed to connect small aliasing like something caused by downscaling with bicubicresize (b=-1,c=0) and nnedi3 failed to connect large aliasing like badly deinterlaced clips, but eedi3 (sclip=nnedi3 ()) can connect both kinds of aliasing

[zero9999]

have you considered using eedi3+nnedi3 instead of pure eedi3/nnedi3, because eedi3 seems failed to connect small aliasing like something caused by downscaling with bicubicresize (b=-1,c=0) and nnedi3 failed to connect large aliasing like badly deinterlaced clips, but eedi3 (sclip=nnedi3 ()) can connect both kinds of aliasing

eedi3_resize isn't supposed to be an antialiasing script, but more like a building block for the actual antialiasing script (http://forum.doom9.org/showthread.php?p=1657610). That said, i totally forgot about this one and will add it to HiAA

[feisty2]

eedi3_resize isn't supposed to be an antialiasing script, but more like a building block for the actual antialiasing script (http://forum.doom9.org/showthread.php?p=1657610). That said, i totally forgot about this one and will add it to HiAA

when upscaling, the possibility of small aliasing that eedi cannot connect contained in source clip is much bigger than large aliasing like badly deinterlaced clip, so it makes more sense to upscale with nnedi3 or eedi3+nnedi3

[zero9999]

when upscaling, the possibility of small aliasing that eedi cannot connect contained in source clip is much bigger than large aliasing like badly deinterlaced clip, so it makes more sense to upscale with nnedi3 or eedi3+nnedi3

do you have a sample at hand that demonstrates this issue?

[feisty2]

do you have a sample at hand that demonstrates this issue?

here
source (videos suffered from aliasing like this are very common, maybe caused by bicubicresize (b=-1,c=0))
http://thumbnails109.imagebam.com/29574/82f832295739191.jpg (http://www.imagebam.com/image/82f832295739191)
eedi3_rpow2 (rfactor=2) (aliasing is still there)
http://thumbnails112.imagebam.com/29574/9f9d1d295739370.jpg (http://www.imagebam.com/image/9f9d1d295739370)
nnedi3_rpow2 (rfactor=2) (aliasing is gone)
http://thumbnails110.imagebam.com/29574/56990e295739428.jpg (http://www.imagebam.com/image/56990e295739428)
eedi3 (dh=true,sclip=nnedi3 (dh=true))
TurnLeft()
eedi3 (dh=true,sclip=nnedi3 (dh=true))
turnright () (best result)
http://thumbnails111.imagebam.com/29574/aa74d6295739507.jpg (http://www.imagebam.com/image/aa74d6295739507)

[Bloax]

They're certainly very sharp edges, but I wouldn't really call that aliasing.

[jpsdr]

I've ported NNEDI3 v0.9.4 to x64. It's working with new API, so, only avisynth 2.6 or avs+. If some are interested to test (of course, it's intented/supposed to work, but maybe there still some issues i may have not encounter in my tests).

You can get binaries here (https://github.com/jpsdr/NNEDI3/releases/download/v0.9.4.1/NNEDI3_v0_9_4_1.7z).
Source files are here (https://github.com/jpsdr/NNEDI3).

[jpsdr]

Updated version (Add Y8, YV16 and YV24) of my ported NNEDI3. More here (http://forum.doom9.org/showthread.php?p=1661944#post1661944).

EDIT :
Another updated version (add fturn support). More here (http://forum.doom9.org/showthread.php?p=1662012#post1662012).

[jpsdr]

I've created specific thread for port of NNEDI3 v0.9.4. More here (http://forum.doom9.org/showthread.php?t=170083).

[hydra3333]

I am getting a crash that VDubMod attributes to an out-of-bounds error that might be originated in nnedi3. How can I modify this to be able to encode?

Hello, did you get a workaround for this ?

[BBA163]

Who can make eedi3 more fast...

[GMJCZP]

I thought it was my imagination, but it is true that eedi3 is super slow. Please could do it faster?

[Reel.Deel]

I thought it was my imagination, but it is true that eedi3 is super slow. Please could do it faster?

Are you aware of this update (http://forum.doom9.org/showpost.php?p=1652667&postcount=476)?

[GMJCZP]

Reel.Deel I can't to access to the link, my web browser reject it.

[Bloax]

Just add an exception, or if you're too scared then just
http://forum.doom9.org/showthread.php?p=1652667#post1652667

or alternatively just read the the previous page of this thread

[GMJCZP]

Thank you all for helping me, but I think it was not necessary so much mystery, only to tell me it was version 0.9.2 (cretindesalpes) have been enough. That is the version I already had.

I did some tests with Santiag and "Sangnom" (using Sangnom2) and the process would take approximately 43 min, in contrast to "eedi3" was almost 11 hours! And that I added to Santiag previously Fturn.

Could it be that is incompatible eedi3 with Fturn, as I read with another version of nnedi3? :thanks: again.

[feisty2]

nnedi3 works rather fast because it adopts the "prescreener" method, aka "only process the edi necessary parts like edges with the edi algorithm, and bicubic for other parts", and yet, prescreener does not (maybe never will) work on eedi3 cuz it requires every vertical line to calculate the edi similarity stuff, so, its just slow like hell, and no way outta it

[colours]

mclip (default: not set)

A mask to use edge-directed interpolation only on specified pixels. The clip should have the same format as the input clip. Pixels where the mask is 0 are generated using cubic linear or bicubic interpolation. The main goal of the mask is to save calculations.
This parameter does not exist in eedi3_rpow2.

This pretty much functions as a prescreener if you use an edge mask.

[GMJCZP]

Thank you for your observation, colours.

I applied the option mclip in Santiag (function santiag_eedi3) as follows (additions in red):

...
c_edge = c.mt_edge()
c.eedi3(dh=dh, field=field, threads=threads, vcheck=vcheck, sclip=sclip, mclip=c_edge)
}
...

I did another test and spent almost 11 hours to just over 6 hours, and the speed increased by approximately 40%. But I'm not satisfied, the turtle was put a rocket on the back, but is still tortoise.

The idea applied it correctly?

[colours]

I did another test and spent almost 11 hours to just over 6 hours, and the speed increased by approximately 40%. But I'm not satisfied, the turtle was put a rocket on the back, but is still tortoise.

The idea applied it correctly?

The latest version of santiag already sets mclip if you use santiag([other options],maskt=24). The value roughly corresponds to detail sensitivity.

Your initial observation that eedi3 is super slow is not wrong at all, and while using an edge mask for mclip does improve speed significantly, "somewhat faster than extremely slow" is still slow.

[GMJCZP]

The latest version of santiag already sets mclip if you use santiag([other options],maskt=24). The value roughly corresponds to detail sensitivity.

Your initial observation that eedi3 is super slow is not wrong at all, and while using an edge mask for mclip does improve speed significantly, "somewhat faster than extremely slow" is still slow.

I just noticed the update, but still cretindesalpes has not changed to Fturn, but to sangnom2.

[feisty2]

eedi2 already does this, but not using a neural network (it builds an edge mask). I could replace eedi2's edge mask building with nnedi2/nnedi3's prescreener, but I don't really want to work on on eedi2 anymore. Prescreening for eedi3 isn't as useful since it must process entire lines. How each pixel is interpolated is dependent on how every other pixel in the line contributes to the cost function. I could run the prescreener and not process any lines where it detects all pixels can use cubic, but I don't think it will save any significant time. If you want eedi3 to be faster there are really four settings to change from the defaults: mdis/nrad/cost3/ucubic. (ucubic=false,cost3=false,nrad=1,mdis=10) is 4-5x faster than the defaults, but of course the result isn't quite as good (and with mdis=10 lines of very small slope: < 1/10 wont be connected). As for smaller networks, neither eedi2/eedi3 use neural networks so there are none to make smaller. Once I get nsizes 16x4 and 32x4 added to nnedi3 I'm going to work on improving the speed/accuracy of the prescreener. I haven't touched the prescreener since the original nnedi.

The temporal version of nnedi3 is on hold for now.

prescreener won't help much
the slowest similarity calculation step gotta be performed on the whole image, it only works on edi interpolation step, so, won't rocket the speed up

[cretindesalpes]

It does help, but the mask should be sparse. It works decently on hand-drawn animated content. The “entire line” issue is simplified internally by horizontally extending the mask of mdis pixels. So if the processed picture contains a lot of vertical edges, it won’t help much.

[feisty2]

It does help, but the mask should be sparse. It works decently on hand-drawn animated content. The “entire line” issue is simplified internally by horizontally extending the mask of mdis pixels. So if the processed picture contains a lot of vertical edges, it won’t help much.

good to know, I tested the mclip stuff but it didn't improve the speed much, maybe my test clip contains too many edges

[GMJCZP]

cretindesalpes not have considered adding Fturn to Santiag?

[Reel.Deel]

cretindesalpes not have considered adding Fturn to Santiag?

It's really easy to add support yourself, just add an F to all TurnLeft/Right calls. Note that if you're already using AviSynth+, the optimized routines are already present, thus no need for FTurn.


Edit:
The temporal version of nnedi3 is on hold for now.
Temporal nnedi3?...hmm that sounds interesting, too bad tritical is no longer around...

[GMJCZP]

It's really easy to add support yourself, just add an F to all TurnLeft/Right calls. Note that if you're already using AviSynth+, the optimized routines are already present, thus no need for FTurn.

I spoke formally. Not all still use Avisynth+ ;)

[MysteryX]

There is an OpenCL version of NNEDI3 (nnedi3ocl) but it doesn't really perform any better than the standard version because the pre-screener doesn't work with OpenCL.

For EEDI3, the prescreener cannot be used anyway and that's why it's slow as hell.

Perhaps the way out of EEDI3 being slow as hell is to make it work with OpenCL?

[Kein]

Does it make sense to me use nnedi_rpow2 if I just need to upscale from H:576 to H:720 (1280x720)? Seems like the lowest factor is 2 so I will have to donwnscale which would lead to another artifacts. Lancsoz4 would suffice?

[colours]

Technically, it's not just that the "lowest factor" is 2, it's that the only legal factors are positive powers of two. If you want to scale to 3×, you can either do rfactor=2 or rfactor=4 (or any larger power of two); rfactor=3 is not an option.

Using nnedi3_rpow2 even for small resizing factors might make sense if you want to exploit the slight antialiasing effect nnedi3 has on your source. I've fooled with a couple of DVDs where this antialiasing was beneficial, and some where the difference was insignificant. You'll have to compare nnedi3 against Lanczos yourself to make the judgment.

[Kein]

Okay, I went through nnedi3 with some custom cshift resize algorithms and then also made some examples with AA mechanisms from this thread (http://forum.doom9.org/showthread.php?p=1711895#post1711895) but surprisingly enough I received best results with basic Lancsoz3 in Avidemux/Vdub. That being said, there is one artifact that was introduced during encoding and upscaling:

Original:
http://i.imgur.com/AQk7nFV.png
Final:
http://i.imgur.com/l8FuPSx.png

Also here: http://screenshotcomparison.com/comparison/114280

it looks like something similar to Chroma Shift by "U", but the step is too small, smaller then 1px even, because if I do -1 "U" shift it won't come even closer.

It is not a big deal but what can I do to somehow fix it?

[colours]

it looks like something similar to Chroma Shift by "U", but the step is too small, smaller then 1px even, because if I do -1 "U" shift it won't come even closer.

It is not a big deal but what can I do to somehow fix it?

:script:

Also, you can use noninteger values for src_left/src_top in the resizers, in case that helps any.

[Kein]

:script:

Also, you can use noninteger values for src_left/src_top in the resizers, in case that helps any.

There is not script, I do not do any resize through Avisynth, I use Avidemux Lancsoz3. Altho, in case I would need, I'd use Lancsoz4 in AVS.

My script is just video/audio source through FFmergSource2 plugin to preserve colorspace.

[foxyshadis]

swscale (the resizer avidemux uses) has a chroma shift that gets worse the smaller the image, and you can't avoid it. Avisynth resizers shift slightly too, but at least they're easy to correct with subpixel cropping.

[Kein]

foxyshadis
I went back to 2.5.8, it uses Mplayer resize but the shift if here, yeah.

How would you go about correcting with subpixel cropping avisynth?

[stax76]

It seems 64-Bit versions of nnedi3 and TIVTC are working fine on Win7 but crash on Win8/Win10.

[Reel.Deel]

It seems 64-Bit versions of nnedi3 and TIVTC are working fine on Win7 but crash on Win8/Win10.

Are you using jpsdr's 64bit nnedi3 (http://forum.doom9.org/showthread.php?t=170083)? If so, maybe he can help you out. Regarding TIVTC, JoshyD is no longer around and he never release the source, so I guess we're outta luck on that one.

[stax76]

Are you using jpsdr's 64bit nnedi3 (http://forum.doom9.org/showthread.php?t=170083)? If so, maybe he can help you out. Regarding TIVTC, JoshyD is no longer around and he never release the source, so I guess we're outta luck on that one.

:thanks:

[MysteryX]

Guys, I just found a way to double the execution of my script with EEDI3 and NNEDI3.

Using AviSynth 2.6 MT on a 8-core system, I was using MT with 2 threads to run EEDI3, NNEDI3, Interframe, 3x FF3DFilter and a few other things. This was giving me ~6fps.

Instead, calling it like this "eedi3_rpow2(2, Threads=1)" and setting MT to 16 threads.

The exact same script is running at 13fps, over twice the performance!!

[CruNcher]

@tritical\madshi
can his results be taken serious ?

http://www.resampling.narod.ru/

[Keiyakusha]

The exact same script is running at 13fps, over twice the performance!!

Not sure about over twice the performance, but what's the catch here? I thought its a common knowledge that you must disable internal threading in plugins if you use them i'm multithreaded avisynth/vapoursynth. As well as the other way - disable all avisynth threads if you have a gpu-filter in the chain. Even setting source filter threads to 1 sometimes makes slight difference.

[Groucho2004]

Not sure about over twice the performance
After a quick test with nnedi3_rpow2, that's about right. The internal OpenMP implementations in nnedi3 and eedi3 are quite inefficient.

I thought its a common knowledge that you must disable internal threading in plugins if you use them i'm multithreaded avisynth/vapoursynth. As well as the other way - disable all avisynth threads if you have a gpu-filter in the chain. Even setting source filter threads to 1 sometimes makes slight difference.
Common knowledge? Hm, getting on top of things with Avisynth MT, a long filter chain (sane or not) and all the obstacles that come with it takes time.
And don't forget that some people strongly dislike reading documentation and searching forums. :rolleyes:

[foxyshadis]

@tritical\madshi
can his results be taken serious ?

http://www.resampling.narod.ru/

Looks like good research, but unavailable software and the Russian site is long gone, not even in Internet Archive.

[vivan]

Looks like good research, but unavailable software and the Russian site is long gone, not even in Internet Archive.It seem to work fine here.

And it was kinda discussed here http://forum.doom9.org/showthread.php?p=1576518#post1576518

[foxyshadis]

It seem to work fine here.

And it was kinda discussed here http://forum.doom9.org/showthread.php?p=1576518#post1576518

The site links to a Russian version at http://resample.narod.ru/ and I was hoping that might have software, more details, or a description of the algorithm, but unfortunately it's just a holding page.

[Groucho2004]

There have been a number of updates to nnedi3 by jpsdr (http://forum.doom9.org/showthread.php?t=170083) but for those of you who want to use the original nnedi3 0.9.4, don't need the additional color spaces and also want to take advantage of the substantial speed increase that TP7's "FTurn" brings for nnedi3_rpow2() I have built a DLL that does just that.

Tritical's 0.9.4 code is unaltered except:

If TP7's fturn.dll (https://github.com/tp7/fturn/releases) is present in the Avisynth autoload directory, nnedi3 will use it's optimized TurnLeft/TurnRight functions
Replaced the utterly obsolete "memcpy_amd" with standard memcpy


The DLL is built with ICL10. Download here (https://www.sendspace.com/folder/10dsem).


Some performance tests on my i5 2500K (4 cores):

Script:
avsmt_threads = 1
edi_threads = 4
(avsmt_threads > 1) ? setmtmode(2, avsmt_threads) : nop()
colorbars(width = 1536, height = 1024, pixel_type = "yv12").killaudio().assumefps(50, 1).trim(0,999)
nnedi3_rpow2(rfactor = 2, nsize = 0, nns = 4, qual = 2, threads = edi_threads)


With "avsmt_threads = 1" and "edi_threads = 4" (using only nnedi's internal multi-threading):
[Runtime info]
Frames processed: 1000 (0 - 999)
FPS (min | max | average): 11.64 | 13.69 | 13.60
Memory usage (phys | virt): 476 | 488 MB
Thread count: 14
CPU usage (average): 77%
Time (elapsed): 00:01:13.539


With "avsmt_threads = 4" and "edi_threads = 1" (using only AVS MT multi-threading):
[Runtime info]
Frames processed: 1000 (0 - 999)
FPS (min | max | average): 3.231 | 149148 | 16.84
Memory usage (phys | virt): 572 | 626 MB
Thread count: 23
CPU usage (average): 97%
Time (elapsed): 00:00:59.375


The sweet spot is actually a combination of both, "avsmt_threads = 3" and "edi_threads = 3":
[Runtime info]
Frames processed: 1000 (0 - 999)
FPS (min | max | average): 5.305 | 255682 | 17.02
Memory usage (phys | virt): 560 | 600 MB
Thread count: 31
CPU usage (average): 96%
Time (elapsed): 00:00:58.751

[burfadel]

Apparently Fturn (the actual Fturn, not normal turn function) is built in, but used as normal turn with Avisynth+.

From the Avisynth+ changelog (24th November 2013):
Code from FTurn is now integrated into the core (with some additional optimizations and new RGB32 routines), making the plugin obsolete.

I realise not everyone wants to use Avisynth+, but it seems that the fturn code got further optimisations that isn't present in the separate plugin. I guess the code could be 'extracted' from the avisynth+ source, it might provide a little extra performance.

[Groucho2004]

Apparently Fturn (the actual Fturn, not normal turn function) is built in, but used as normal turn with Avisynth+.
Yes, I should probably have mentioned that this is useful mainly for users of the official or SEt's MT Avisynth versions.

I realise not everyone wants to use Avisynth+, but it seems that the fturn code got further optimisations that isn't present in the separate plugin. I guess the code could be 'extracted' from the avisynth+ source, it might provide a little extra performance.
Same script as above, AVS+ r1576, without fturn.dll in the autoload directory:
[Runtime info]
Frames processed: 1000 (0 - 999)
FPS (min | max | average): 11.82 | 13.45 | 13.35
Memory usage (phys | virt): 74 | 91 MB
Thread count: 25
CPU usage (average): 76%
Time (elapsed): 00:01:14.901

with fturn.dll in the autoload directory:
[Runtime info]
Frames processed: 1000 (0 - 999)
FPS (min | max | average): 11.83 | 13.49 | 13.39
Memory usage (phys | virt): 74 | 91 MB
Thread count: 25
CPU usage (average): 75%
Time (elapsed): 00:01:14.658

[burfadel]

Ah okay, so those results are well within error margin! I was just going by the changelog comment. I presume that in the test that the turn command was using the fturn dll in the fturn test :).

[Groucho2004]

I presume that in the test that the turn command was using the fturn dll in the fturn test :).
Yes, it did.

[MysteryX]

I have a question about turnleft/turnright. How does that work? If I'm applying EEDI3 + NNEDI3, should I call it in this order?
EEDI3, turnleft, NNEDI3, turnright

and if I'm calling NNEDI3(4), is it automatically calling turnleft/turnright?

In that case, NNEDI3(2) + NNEDI3(2) wouldn't give the same output as NNEDI3(4) ?

Anything else I should know about this?

[Desbreko]

TurnLeft/TurnRight rotates the video on its side so that nnedi3/eedi3's height doubling can be used to double the width of the image. If you use the rpow2 functions, they'll handle the turning to double both the width and height in one function call. If you use the base functions with dh=true, you need to call them twice with TurnLeft/TurnRight in between.

For example, the following two lines do basically the same thing, assuming you're using jpsdr's nnedi3.

nnedi3_rpow2(2)

TurnRight().nnedi3(1, dh=true, nsize=0, nns=3).TurnLeft().nnedi3(1, dh=true, nsize=0, nns=3)

[Desbreko]

After putting it off for a long time, seeing jpsdr make improvements to nnedi3_rpow2 in his builds finally gave me the motivation to write a better rpow2 function that also supports the other edi plugins, so here we go.

edi_rpow2 v1.0 (http://www.mediafire.com/view/pcmkxaauku375xd/edi_rpow2_v1.0.avsi)
(Also requires ResizeX (http://www.mediafire.com/download/lcbtb7uta4ta5pc/ResizeX_v1.0.avsi))

It supports jpsdr's nnedi3, nnedi3ocl, eedi3, and eedi2, and includes all the improvements jpsdr made plus more, such as...

Supports all AviSynth 2.6 colorspaces except RGB32 with all edi plugins
Uses FTurn if it's loaded
Separate rfactors for horizontal and vertical image enlargement
Corrects for the chroma shift caused by AviSynth's resizers and includes a cplace parameter
Can use Dither_resize16 instead of the internal resizers to output a stack16 clip
Allows disabling processing of individual YUV planes
Avoids distortion at the edges of the frame (http://i.imgur.com/QBDJIlD.png)
Supports eedi3's sclip and mclip parameters (sclip works a little differently; see the included documentation)

[MysteryX]

I mentioned that NNEDI3's multi-threading was inefficient. I just realized it's weirder than that.

I'm running these scripts with AvsMeter to see how many threads are created. I'm running in MT mode with 8 threads.

This creates 16 threads.
nnedi3_rpow2(2, cshift="Spline16Resize", threads=1)

This creates 32 threads.
nnedi3_rpow2(2, cshift="Spline16Resize", threads=2)

This creates 128 threads.
nnedi3_rpow2(2, cshift="Spline16Resize", threads=1)

When running without MT and asking NNEDI3 to use a single thread, it creates 2 more threads.

What's going on here?

[wonkey_monkey]

This creates 16 threads.
nnedi3_rpow2(2, cshift="Spline16Resize", threads=1)

This creates 32 threads.
nnedi3_rpow2(2, cshift="Spline16Resize", threads=2)

This creates 128 threads.
nnedi3_rpow2(2, cshift="Spline16Resize", threads=1)


The two I've bolded are identical. Is there a typo in one of them?

[MysteryX]

should be "threads=8" :) or not specified since it is the default on my system

[StainlessS]

should be "threads=8" :) or not specified since it is the default on my system

Might want to say which bolded one you refer to.

[MysteryX]

last one, of course

[luquinhas0021]

Desbreko, the edi_rpow2 v1.0 looks like fine! it's what I need.
I have seven questions about:
1) The jpsdr nnedi3 that edi_rpow2 v1.0 supports is nnedi3 v0.9.4.20, with all original parameters (Update maden in 5th September)?
2) Does eedi3, nnedi3ocl and eedi3, in your plug-in, have the same improvements that jpsdr made in nnedi3? More your improvements, course!
3) Your plug-in works only with nnedi3_rpow2, nnedi3ocl_rpow2, eedi3_rpow2 and eedi2_rpow2 functions, with all original parameters, or works with nnedi3, nnedi3ocl, eedi3 and eedi2 functions too, with all original parameters?
4) If works with nnedi3, nnedi3ocl, eedi3 and eedi2 too, this functions has the jpsdr's improvements and, of course, the yours?
5) There's no chroma and luma shift neither any type of chroma and luma flaws in your plug-in, in all functions and parameters it supports. Am I right?
6) Any future quality improvements plans?
7) You correct only AviSynth internal resizers chroma shift or correct chroma shift of some other resize, such Spline 144, Jinc,... (If yes, please, upload your corrected Avisynth external resizers)?
Please, answer all my questions.

[Desbreko]

1: Yes, the latest version of jpsdr's nnedi3 is recommended.
2: edi_rpow2 is a script, not a plugin. It can't change the main nnedi3ocl, eedi3, or eedi2 functions.
3: edi_rpow2 uses nnedi3, etc. directly and supports all of the main functions' parameters. It doesn't use the original rpow2 functions.
4: All of the improvements jpsdr made to nnedi3_rpow2 (not nnedi3 itself) are included in edi_rpow2 and also apply to the other edi types.
5: Not that I know of. If you find any, please let me know so I can fix them.
6: None currently, but I'm open to suggestions.
7: Currently, only AviSynth's internal resize kernels and Dither_resize16's kernels are supported. The SplineResize plugin's kernels (Spline144Resize, etc) are buggy and you should use Dither_resize16's generic spline kernel instead. I might add support for Jinc if a good anti-ringing algorithm like the one in madVR ever gets developed for AviSynth, but otherwise it doesn't seem worth using.

[luquinhas0021]

Sinc 16, 36, 64, 128, 256; Lanczos 16, 36, 64, 128, 256: do you appreciate some of them over Spline144?
Speaking about Spline 144 (And Spline 6 taps), why it's buggy, if it is a Wilbert plug-in, not an Avisynth internal resizer? And why aren't dither_16 resize's generic splines buggy?

[Desbreko]

Sinc and Lanczos are already supported. Use the taps parameter to set the number of sample points.

About the SplineResize plugin, see this thread.

[luquinhas0021]

The guys, in post you send me, was making many iterations with Spline! This was crazy!
Well, I will follow the sugestion you gave me. Anyway, how I put spline 6 taps (aka Spline 144) present in Dither_16 resize on nnedi3 cshift parameter? Like: nnedi3_rpow2 (... cshift = spline 6 taps present in Dither_16 resize) (I know this way isn't the right).
In my tests with Sinc and Lanczos, with every possible number of taps, no one outperformed Spline 144, in detail retention, sharpness and in minimal generation of artifacts. The computer where I made this test has a 3rd generation Intel i5 processor, so it's a relatively new device.
In test I made with Wilbert plug-in, using Spline144Resize and Spline taps=6, the Spline144Resize was a little, little better than Spline taps=6.

[Desbreko]

To have edi_rpow2 use Dither_resize16, set lsb=true. So to use 6-tap spline for the cshift correction, you'd use this:

edi_rpow2(cshift="Spline", taps=6, lsb=true)

[luquinhas0021]

Warning to you: the link you subscribe (EDI v1.0) isn't opening - it's on MediaFire server.
In EDI v1.0, there's most recent jpsdr nnedi3, eedi3, nnedi3 ocl and eedi2, including their _rpow2 functions. But, in code example you wrote, it's edi_rpow2. Why? Your program doesn't give the option to choose the edi algorithm the person want use? If gives, how I choose it?

[Desbreko]

The link works fine for me. Maybe try this link (http://www.mediafire.com/download/pcmkxaauku375xd/edi_rpow2_v1.0.avsi) instead to go directly to the download page instead of the view page?

edi_rpow2 is its own function; it doesn't replace the rpow2 functions of nnedi3, etc. It uses nnedi3 by default, but you can use its edi parameter to set which plugin to use for the image enlargement. Read the documentation at the top of the script to learn how to use its parameters.

[luquinhas0021]

Do you know about the core (Kernel) of nnedi3 and eedi3 (Including their _rpow2 functions)? Seemly, there's nothing more, unless do it compatible with RGB30 and RGB32, to do in peripherical aspects of edi algorithms. Maybe update kernel be the optimal choice... Although we will should train again the neural net. Here is some my ideas:
Deprecate the minimization of absolute error - L1 - and squared error - L2. Replacing it by the minimization of square root of absolute error - L0.5 (Idea of Tritical, but never was released (Go to page top of page 21 of this thread)).
Increase the number of neurons and conections between them. I think 512 or 1024 neurons will be excelent
Review the sets of image/frames Tritical has used in order to train the net.
Review the weights of neurons.
What do you think about my ideas?

[Desbreko]

I'm not a programmer, so I wouldn't even know where to begin in improving the core functions.

[luquinhas0021]

If you aren't a programmer, how you could modify and create edi v1.0?

[Katie Boundary]

What's the difference between Bob() and the nnedi family of filters? My OCPD is dying to know.

[pbristow]

What's the difference between Bob() and the nnedi family of filters? My OCPD is dying to know.

In a nutshell:

"Bob" is simple, fast... but often quite horrible to look at. Especially on scenes featuring strong straight or slightly curved edges; and even more so if those edges are moving. (For more details, read the documentation on Bob).

EDI-based bobbing (in general) is more complicated, slower, but very often less horrible to look at in the cases I've mentioned. (For more details, google the phrase "Edge directed interpolation".)

NNEDI is a way of doing EDI-based bobbing that, instead of being based on any one person's theory of "What's the best way to do EDI?", is based on a neural network (hence "NN") that has been trained with various sets of test cases, to give the best all-round results. (For more details, google the phrase "neural network").

[Katie Boundary]

In a nutshell:

"Bob" is simple, fast... but often quite horrible to look at. Especially on scenes featuring strong straight or slightly curved edges; and even more so if those edges are moving. (For more details, read the documentation on Bob).

EDI-based bobbing (in general) is more complicated, slower, but very often less horrible to look at in the cases I've mentioned. (For more details, google the phrase "Edge directed interpolation".)

NNEDI is a way of doing EDI-based bobbing that, instead of being based on any one person's theory of "What's the best way to do EDI?", is based on a neural network (hence "NN") that has been trained with various sets of test cases, to give the best all-round results. (For more details, google the phrase "neural network").

I literally had to read the post twice, then click "quote", then start copying and pasting things before I realized that "EDI" and "Edge-directed interpolation" were the same thing.

So really, you could have just said "Google edge-directed interpolation" and saved yourself a bit of time :) And I will Google it, thanks.

EDIT: I got as far as the phrase "Wiener filtering" before deciding that I didn't want to read any more...

[MysteryX]

Running this with the latest build of NNEDI3.

nnedi3_rpow2(2,nns=4,cshift="spline16")


AviSynth 2.6 gives me
nnedi3_rpow2: error using env->invoke (function not found)!
(it works if I remove 'spline16')

AviSynth+ v2022 gives me
System exception - Access Violation

Perhaps we're due for an updated build?

[Reel.Deel]

Running this with the latest build of NNEDI3.

nnedi3_rpow2(2,nns=4,cshift="spline16")


AviSynth 2.6 gives me
nnedi3_rpow2: error using env->invoke (function not found)!
(it works if I remove 'spline16')

AviSynth+ v2022 gives me
System exception - Access Violation

Perhaps we're due for an updated build?

That's because there's no function named Spline16, it needs to be Spline16Resize. As for the lastest Avs+, if your using nnedi3 v0.9.4.22 then it needs to be recompiled with the latest avs+ headers. For the mean time use an older version.

Here's the official thread for jpsdr's nnedi3 mod: http://forum.doom9.org/showthread.php?t=170083

[Katie Boundary]

I'm not sure yet what the solution is, but the way the shift happens is as follows.

The resizer core is designed to preserve the position of the image centre, and it does this for both luma and chroma independently.
This works fine for vertical resizing, since in that case the centres of the luma and chroma sampling grids coincide for all formats (including YV12, where the chroma samples are vertically positioned between the luma samples).

But for horizontal resizing (except YV24), the placement is such that the luma and chroma centres do not coincide - in each case the chroma centre is to the left of the luma centre.
Eg for YV12 and YUY2:
luma centre
|
Luma: L L L L ... L L L L ... L L L L
Chroma: C C ... C C ... C C
|
chroma centre
Since the distance between the two centres is proportional to pixel spacing, it changes on a resize.
For upsizing, pixel spacing decreases, so the chroma centre moves closer to the luma centre (hence to the right), and conversely for downsizing it moves to the left. Since the output chroma is calculated based on a fixed centre, the visual result is a corresponding chroma shift, right for upsizing and left for downsizing.

We can quantify the shift as follows.
For YV12 and YUY2, the chroma centre is 0.5 luma pixels to the left of the luma centre.
When resizing from width Win to Wout, output pixel spacing in terms of input is multiplied by Win/Wout.
Hence the net (rightwards) chroma shift is a distance of 0.5*(1-Win/Wout) input luma pixels.
For a 2x upsize, this equals 0.25, for a 2x downsize it's -0.5 (ie 0.5 to the left).

Similarly, for YV411, the net shift is 1.5*(1-Win/Wout), ie 3 times as much as for YV12/YUY2.

Are there any plans to fix this issue? If not, does converttorgb() make an effective workaround?

[jpsdr]

It's been done since a little time.
If you're interested only on nnedi3, you can get it here (http://forum.doom9.org/showthread.php?t=170083), but if you're interested by others of my plugins, you can get the "all in one" pack here (http://forum.doom9.org/showthread.php?t=174248).

[Katie Boundary]

It's been done since a little time.

what :confused:

[Groucho2004]

what :confused:
I believe it means "it was fixed recently".

[hello_hello]

Are there any plans to fix this issue? If not, does converttorgb() make an effective workaround?

Once again I offered alternatives in your other thread and once again Katie is still to childish to read my post. That's probably why the question wasn't answered a second time in the other thread. Nobody else is privy to who you're ignoring.

Maybe someone should tell Katie she should post like a grown-up so she'd know her question was answered. I guess that's what happens when you try to help her despite herself. Her old DVD thread was littered with posts calling her a troll, but those who persisted the longest and put up with the most abuse also ended up on Katie's ignore list for their trouble.

[Katie Boundary]

I believe it means "it was fixed recently".

Okay, cool. Can anyone confirm?



While I'm here, I might as well discuss a problem I've been having with nnedi3: AVIsynth refuses to acknowledge it. I've gone so far as to dump nnedi3.dll directly into the root c:\ drive and add loadplugin("c:\nnedi3.dll") to the script, which results only in a "module not found" error.

I'm using the win7, x86, VS2015 version of the dll, on a windows 7 computer with the x86 version of AVIsynth, so I don't think that's the problem.

:confused:

[Groucho2004]

While I'm here, I might as well discuss a problem I've been having with nnedi3: AVIsynth refuses to acknowledge it. I've gone so far as to dump nnedi3.dll directly into the root c:\ drive and add loadplugin("c:\nnedi3.dll") to the script, which results only in a "module not found" error.

I'm using the win7, x86, VS2015 version of the dll, on a windows 7 computer with the x86 version of AVIsynth, so I don't think that's the problem.

:confused:I told you in the other thread to use AVSMeter to find out what's wrong. Put nnedi3 in the plugin directory and run "AVSMeter -avsinfo".

[Katie Boundary]

Never mind. Installing Microsoft Visual Basic 2015 fixed the problem.

Hey, tritical: splitting up important information into multiple readme files is a bad idea.

[wonkey_monkey]

Okay, cool. Can anyone confirm?

Yes, you can, by downloading it and trying it...

[djonline]

Windows 10 x64, Avisynth+ 2420.
Trying to load nned3.dll x86.
Avisynth open failure: Cannot load file 'c:/Program Files (x86)/AviSynth+/plugins+/nnedi3.dll'. Platform returned code 193: %1 is not a valid Win32 application.
No information about this in wiki or requirements section.
If I missed some ddl, please provide link to download and add to faq.

[Groucho2004]

Windows 10 x64, Avisynth+ 2420.
Trying to load nned3.dll x86.
Avisynth open failure: Cannot load file 'c:/Program Files (x86)/AviSynth+/plugins+/nnedi3.dll'. Platform returned code 193: %1 is not a valid Win32 application.
No information about this in wiki or requirements section.
If I missed some ddl, please provide link to download and add to faq.Run "AVSMeter (https://forum.doom9.org/showthread.php?t=174797) avsinfo" and check if it returns any error.

[StainlessS]

Also maybe try load in HexAscii viewer, check that it aint an html file you downloaded. [the page to download the dll, might happen if right click link, SaveAs]

[djonline]

"AVSMeter avsinfo" output:
Cannot load file 'C:/Program Files (x86)/AviSynth+/plugins+/nnedi3.dll'. Platform returned code 193: is not a valid Win32 application.
Dependencies that could not be loaded: svml_dispmd.dll
I download latest update3 library from https://software.intel.com/en-us/articles/redistributable-libraries-for-intel-c-and-fortran-2019-compilers-for-windows (never put it only on readme.txt, clone this info on first page and wiki, so it will be seen by google).
Now:
Cannot load file 'C:/Program Files (x86)/AviSynth+/plugins+/nnedi3.dll'. Platform returned code 126: The specified module could not be found.
Dependencies that could not be loaded: svml_dispmd.dll
I have "Release_Intel_W7_Core2_SSE4.2" on i7-4930k. Also try Release_Intel_W7_Core2_AVX, the same luck.

[StainlessS]

Google: "svml_dispmd.dll"
https://www.google.com/search?source=hp&ei=tK3lXNyPLZGalwT84Ia4Cw&q=%22svml_dispmd.dll%22&oq=%22svml_dispmd.dll%22&gs_l=psy-ab.12..0l4j0i30l6.1518.2450..2763...0.0..0.116.320.1j2......0....1j2..gws-wiz.....0.TBLumo6FVBA

Developer Intel Corporation
Product Intel(R) CCFortran Compiler
Description SVML Library for Intel(r) Compilers (threadsafe)
Filename svml_dispmd.dll
Version 16.0.0
Size 13.6MB


What Is Svml_dispmd.dll?

Svml_dispmd.dll is a type of DLL file associated with Intel® C Compiler, Intel® C++ Compiler, Intel®
Fortran Compiler developed by SCIA for the Windows Operating System.
The latest known version of Svml_dispmd.dll is 11.1.11.2, which was produced for Windows.
This DLL file carries a popularity rating of 1 stars and a security rating of "UNKNOWN".


EDIT: SVML: Short Vector Math Library.

Where did you get the dll ?

EDIT: Maybe its some debug Neural Net thing for teaching Nnedi3.

[djonline]

About x64:
It requires libmmd.dll to put near nnedi3.dll, it not autoloaded from c:\Program Files (x86)\Common Files\Intel\Shared Libraries\redist\intel64_win\compiler\libmmd.dll

[djonline]

StainlessS, can you provide link to download ? There is the same link as above, but with 'fortran' ?

[StainlessS]

libmmd.dll

Above d in red looks like it might be a debug version.

I'll see if I've got x64 Nnedi3 dll. back in a bit.

[StainlessS]

Here tis x64 dll [nnedi3_x64_9.4.48.dll], dont know if I have used it. [looks ok in Dependency walker]
http://www.mediafire.com/file/51zby9k9ledtl59/nnedi3_x64_9.4.48.dll.7z/file

EDIT: Requires VCRUNTIME140.dll: Microsoft Visual C++ 2015 Redistributable Update 3 RC:- https://www.microsoft.com/en-us/download/details.aspx?id=52685

[jpsdr]

Actual version of my nnedi3 release is 9.4.51. Is there any specific reason or issue for you to still use a previous version ? (If it's my release you're using).

[StainlessS]

Thanx jpsdr, me gots new ver$, dont now why I did not already have it (maybe already in my cache of stuff to test, its huuuuuge).

If its non XP comp, then that is reason why (I only use XP compat unless eg Masktools2 of non XP Infix -> RPN, but then when all RPN, use XP ver$ only).

[jpsdr]

There is an XP version in the builds for each release. After, if you also uses others of my releases, it's best to use the all in one plugin version... :D

[wonkey_monkey]

Just trying to get nnedi3 to work on Windows 10. I have various copies of libmmd.dll floating around on my computer. Is there an official source for a latest version?

[ChaosKing]

What is libmmd.dll? I don't need this dll for nnedi3 on win10.

[Stereodude]

What is libmmd.dll? I don't need this dll for nnedi3 on win10.
Ditto. I just

LoadPlugin("C:\HDTV Tools\NNEDI3\x86\Release_W7_AVX\nnedi3.dll")

And it works fine.

[StainlessS]

About x64:
It requires libmmd.dll to put near nnedi3.dll, it not autoloaded from c:\Program Files (x86)\Common Files\Intel\Shared Libraries\redist\intel64_win\compiler\libmmd.dll

So that is only for Intel compiler dll.
[still think maybe the 'd on end may also mean debug library which will not be installed on non developer machine, if is debug dll, then should never have been released as is].


Math Library for Intel(r) Compilers (thread-safe)

The file libmmd.dll is a 32/64bit Windows DLL module for Intel(r) C Compiler, Intel(r) C++ Compiler, Intel(r) Fortran Compiler software or other related programs. The file and the associated Intel(r) C Compiler, Intel(r) C++ Compiler, Intel(r) Fortran Compiler software was developed by Intel Corporation. The file was licensed as Copyright (c) 1985-2004 Intel Corporation.

libmmd.dll file is a critical component for the software to work properly and without any dll error messages. You may receive an error message like the ones below that indicates the file is not found, damaged, or the wrong version.

This application failed to start because libmmd.dll was not found.
The file libmmd.dll is missing or corrupted.
A required component is missing: libmmd.dll. Please install the application again.
Error loading libmmd.dll. The specified module could not be found.
The program can't start because libmmd.dll is missing from your computer. Try reinstalling the program to fix this problem.

To fix the error, you need to copy the missing libmmd.dll file to your system or replace the corrupted dll file with a working one by following the instructions below. There are 3 different versions of the dll file available for download with sizes between 2.21MB - 3.88MB.

https://www.dll4free.com/libmmd.dll.html

EDIT: I use nnedi3 and know nothing of that dll [but mine is probably MSVC]

EDIT: For me and NNEDI3_v0_9_4_53.7z, the intel dlls all need "SVML_DISPMD.dll" in Dependency Walker [cannot find the file specified],
but that might also be because I dont have any Intel compiler stuff installed, and my machine is max SSE4.1 so not supported for that ver$ dll.
The MSVC version for Release_W7, Release_XP, Release_XP_SSE2 all req VCRUNTIME140.dll [Microsoft Visual C++ 2015 Redistributable]

[Groucho2004]

Just trying to get nnedi3 to work on Windows 10. I have various copies of libmmd.dll floating around on my computer. Is there an official source for a latest version?Have a look at this (https://forum.doom9.org/showthread.php?p=1889362#post1889362).

[wonkey_monkey]

Some Intel Compiler thing. It's needed for x64 but not x86.

Turns out all the variations I had were the same so I just copied one of those.

Edit: Photoshop installs it so it seems unlikely it's debug anything.

[StainlessS]

Well the 'd' at end of dll name is probably only an M$ convention that Intel dont use.

[real.finder]

since this bug in old nnedi3
https://github.com/jpsdr/NNEDI3/issues/9#issuecomment-652239329

is the VS port fix it?

[ChaosKing]

VS version looks ok to me.

EDIT
But znedi has a green line!

[jpsdr]

The green line seems to be only on pure white pixels... Maybe something left somewhere to not process 255 pixels value...
On the other hand... Colorbar YUV, white should be 235 not 255... Is Colorbar full range by default for YUV...?

[FranceBB]

On the other hand... Colorbar YUV, white should be 235 not 255...

Precisely.


Is Colorbar full range by default for YUV...?

Nope

https://i.imgur.com/GzkhcNL.png

[ChaosKing]

It seems to appear on non mod4 resolutions like 1920x1206

https://github.com/sekrit-twc/znedi3/issues/11

[jpsdr]

It's true, that theoricaly, you can't deinterlace not mod4 4:2:0 resolution... You have to define what to do for the missing line (the most logical and easy choice would be just to duplicate the last line...).

[jpsdr]

Can someone give me the VS github link ?
I've tried some quick ideas, but no succes...

[ChaosKing]

https://github.com/dubhater/vapoursynth-nnedi3

[jpsdr]

Tried to add more lines on the bottom of the copypad, but didn't work, so, the other quick and easy fix...;)
(Didn't find the proper cause...)

if (!vi.IsPlanar()) dstPF->copyTo(dst,vi);

if (vi.Is420() && ((vi.height & 3)!=0))
{
ptrdiff_t dst_offsetU=(dst->GetHeight(PLANAR_U)-2)*dst->GetPitch(PLANAR_U);
ptrdiff_t dst_offsetV=(dst->GetHeight(PLANAR_V)-2)*dst->GetPitch(PLANAR_V);

memcpy(dst->GetWritePtr(PLANAR_U)+(dst_offsetU+dst->GetPitch(PLANAR_U)),
dst->GetWritePtr(PLANAR_U)+dst_offsetU,dst->GetRowSize(PLANAR_U));
memcpy(dst->GetWritePtr(PLANAR_V)+(dst_offsetV+dst->GetPitch(PLANAR_V)),
dst->GetWritePtr(PLANAR_V)+dst_offsetV,dst->GetRowSize(PLANAR_V));
}

return dst;

[Katie Boundary]

What's the difference between "Release_W7", "Release_W7_AVX", and "Release_W7_AVX2"?

[wonkey_monkey]

Support for different instruction set extensions. If your CPU supports the AVX2 (https://en.wikipedia.org/wiki/Advanced_Vector_Extensions#Advanced_Vector_Extensions_2) nstruction set, use the W7_AVX2 one. If it only supports W7_AVX, use that one. If it's very old and supports neither, use W7.

[StainlessS]

@Katie,
What Wonkey says.


colorbars.killaudio
info

https://i.postimg.cc/VJZcbsQF/test-2-00.jpg (https://postimg.cc/VJZcbsQF)

[wonkey_monkey]

@Katie,
What Wonkey says.


Aww, you ruined it. KB blocked me years ago for... I don't know, saying something sensible or something. So now I like to get in first with an answer they'll never see :D

[StainlessS]

Katie, how do I block Wonkey ?