NNEDI - intra-field deinterlacing filter

[IanB]

field -
Possible settings:
...
-1 = same rate, uses avisynth's internal parity value
...
Default: -1 (int)Which field is kept odd or even?

dh -
...
Default: false (int) <- (bool)Wouldn't it have been simpler to fold double height mode into extra settings for field? And maybe name field as mode?

Y, U, V -
These control whether or not the specified plane is processed. Set to true to
process or false to ignore.What state are the unprocessed planes left? Copied, zerod, trashed, ...?


Also are you going to discuss the nitty gritty of the algorithm or are you constrained because you are doing your thesis?

[tritical]

Which field is kept odd or even?
If field is set to -1 then nnedi calls child->GetParity(0) during initilization. If it returns true then field is set to 1. If it returns false then field is set to 0. If field is set to -2 then the same thing happens, but instead of setting field to 1 or 0 it sets field to 3 or 2.

Wouldn't it have been simpler to fold double height mode into extra settings for field? And maybe name field as mode?

dh could have been rolled into 4 extra values for field, but I went with adding dh instead. field could be renamed mode... I named it field because eedi2 has the same parameter with the same values.

What state are the unprocessed planes left? Copied, zerod, trashed, ...?
Unprocessed planes are left trashed.

Also are you going to discuss the nitty gritty of the algorithm or are you constrained because you are doing your thesis?I'm not going to discuss the full details just yet. I have talked to my advisor about using it for my thesis, and he said that it would be ok. So I will probably use it for that.

I'll try and update the readme tommorrow sometime... thanks for pointing out areas lacking complete descriptions.

Also, if anyone is willing to donate cpu time again, network training for version 2 has started. Same irc channel as before (#editrain on irc.freenode.net)... files are in same directory on my website as before. Based on previous experience, I would say an x2 or core 2 duo is probably the minimum processor to complete the optimization runs in a reasonable amount of time. I'm also looking into offloading some processing onto the gpu, which should speed things up a good bit.

[sh0dan]

Results are very impressive. I had a look at some of the math you link to, and it seems like another case of "Math I'll never be able to understand", just like FFT math.

Anyway, this is very interesting. I'm thinking it could be very useful for a Bayer Grid de-mosaic, which has very similar problems compared to deinterlacing.

I hope the source will be available at some point. Keep up the good work!

[manolito]

One (probably stupid) question:

When I want to use nnedi with TDeint or Yadifmod, do I have to use the same syntax as for EEDI2

AssumeTFF (or BFF)
interp = separatefields().selecteven().nnedi(field=1,dh=true)
yadifmod(edeint=interp)or can I use

AssumeTFF (or BFF)
interp = nnedi()
yadifmod(edeint=interp)

I tried both, and the result (and the speed) looked pretty much the same to me.

And one small request:
While playing with different deinterlacers I found it very annoying to have two Yadif plugins with almost the same functionality, one does autoload, the other one doesn't. Is it possible to make one consolidated build of Yadif which has the "edeint" parameter, but which uses its own spatial interpolation when the edeint parameter is not specified (just like TDeint)?

Cheers
manolito

[tritical]

separatefields().selecteven().nnedi(field=-1,dh=true)

and

nnedi()

are equivalent. The second should be faster, but the speed difference probably wouldn't be enough to notice. As far as adding internal interpolation to yadifmod, I personally think yadif's default interpolation is far too artifact prone. So I probably wouldn't add exactly the same method that Fizick's filter uses. At the very least it would include capping the prediction to the max/min +-2 of the vertical neighbors.

[IanB]

... I had a look at some of the math you link to, and it seems ...I must have missed this, anyone got a post reference.

I hope the source will be available at some point. Keep up the good work!I second that emotion ;)

[sh0dan]

I must have missed this, anyone got a post reference.
http://forum.doom9.org/showthread.php?p=1038500#post1038500

You might also spend some time at wikipedia:

http://en.wikipedia.org/wiki/Artificial_neural_network

[IanB]

Thx! :cool:

So if I guess correctly, you classify and examine a stack of images to build a tables of how "the world" works. To do the interpolation you assume the target image is a member of "the world" with every 2nd line missing. You then search your table for "a good/best match" and insert the missing lines based on that instance of experience. This should be extensible for general times N upsizing by assuming N-1 of N lines are missing and need insertion.

Also I suspect with a little lateral thinking, the tables of how "the world" works, could be adapted to analyze interlaced images to classify the area that are static (good experience match) from those that have motion (bad experience match).....:D

[manolito]

@ tritical
separatefields().selecteven().nnedi(field=-2,dh=true)

and

nnedi()

are equivalent.
field=-2? Typo? Nnedi won't let me do that, it insists on field being 0 or 1 if dh=true is used. Also would not make much sense IMO because I need a same rate nnedi clip.

Cheers
manolito

[Terranigma]

@ tritical

field=-2? Typo?
I believe so. I believe he meant -1, as it refers to the parity ;)
Field=-2 should only be used in double-rate deinterlacing, like when using yadifmod with mode 1.

[tritical]

The -2 was indeed a typo, it should have been -1.

So if I guess correctly, you classify and examine a stack of images to build a tables of how "the world" works. To do the interpolation you assume the target image is a member of "the world" with every 2nd line missing. You then search your table for "a good/best match" and insert the missing lines based on that instance of experience. This should be extensible for general times N upsizing by assuming N-1 of N lines are missing and need insertion.
Here is basically how it works. In the first stage a small ann takes in some surrounding pixels and predicts whether or not cubic interpolation will be close to the true pixel value (basically a two class classifier). If it thinks it is then cubic is used. Otherwise, the point is matched up to the closest of 64 cluster prototypes. The pixel is then fed to the ann for that cluster prototype (there is a separate ann for each cluster prototype), which predicts (outputs) the missing pixel value using surrounding pixels as input (more this time than for the first stage classification). In this way no single ann has to learn to approximate the entire input->output space mapping, only a small piece of it. CMAES is used to find the weights for these anns (instead of a more usual training method like gradient descent, lev-mar, etc...). In the second stage CMAES is minimizing squared error.

[Mtz]

I made a small test on a DV source and I said: WOW!

Reasons:

Original:
http://i20.tinypic.com/mki1cm.jpg

Nnedi:
http://i23.tinypic.com/ok4q4x.jpg


Original (the camera was moving left-right or right-left):
http://i20.tinypic.com/2w65imx.jpg

Nnedi:
http://i24.tinypic.com/aafaxj.jpg

Cedocida encode & Nnedi:
http://i23.tinypic.com/avht29.jpg


Original (the camera was moving left-right or right-left and up-and-down):
http://i24.tinypic.com/15grwn8.jpg

Nnedi:
http://i22.tinypic.com/j6o568.jpg

Cedocida encode & Nnedi:
http://i24.tinypic.com/rthk79.jpg

enjoy,
Mtz

[Terranigma]

mtz, are those nnedi examples using just the nnedi filter, or as an external interpolator such as using nnedi with yadifmod?

Btw, those results are awesome! :)

[Mtz]

I used:
AVISource("I:\MiniDV\My Video-04\InterlacedSonyDCRHC96__DV.avi")
AssumeBFF()
nnedi()

The sample is at the bottom of this page. (http://www.ftyps.com/unrelated/interlace/)

enjoy,
Mtz

[Boulder]

Okay I've now looked at MVBob and MCBob, and it appears that NNEDI makes a definite difference on edges. With EEDI2 the edges display something I call "blur bubbles" on straight lines. Replacing EEDI2 with NNEDI seems to greatly reduce if not eliminate these artifacts.Would you mind posting the two functions, please?

[Terranigma]

Would you mind posting the two functions, please?
You can grab both modded bobbers here (http://www.zshare.net/download/404129650335d2/).

[Boulder]

Thanks :)

[ficofico]

You can grab both modded bobbers here (http://www.zshare.net/download/4016525fb0727e/)

Thanks for upload....... for me mcbob works well, mvbob... no, there's some problem.

[Terranigma]

for me mcbob works well, mvbob... no, there's some problem.

What kind of problem? It runs just fine here.
Make sure you have all the prerequisites.
I didn't include any of them for either scripts since that's only what boulder asked for.

[2Bdecided]

I've finally jumped in with this.

NNEDI seems to have different strengths from EEDI2 - another useful tool - thank you tritical!

However, I'm having problems with mcbob downloaded from Terranigma's link. The "old" EEDI2 MCBob_v03c works beautifully, but MCBob_v03u isn't right. It doesn't crash, and I get sensible output, but the deinterlacing is bad - on every second field (so bottom field on TFF content) moving diagonals look close to plain old bob() - much worse than v03c, and worse than NNEDI on its own. On the frame generated from top fields, it's not horrible, but not great – those bobbles on diagonals that nnedi reportedly avoided are present, whereas they aren't there with v03c.

What have I broken? I assume all plug-ins are OK since mcbob v03c works beautifully.

I can post samples and screen shots if it's not an obvious/known error.

Cheers,
David.

[krieger2005]

If the problem is not NNEDI why discussing in the Thread of NNEDI?

[Didée]

Well, the problem is about correct usage of NNEDI, so it does fit here somehow.

Wrong:
Function nnEDIbob(clip Input)
{
[...]
GetParity(Input) ? Input.SeparateFields().nnEDI(dh=true,Field = 1) : Input.SeparateFields().nnEDI(dh=true,Field = 0)
AssumeFrameBased()
GetParity(Input) ? AssumeTFF() : AssumeBFF()
}
NNEDI has slightly different operating modes compared to EEDI2, hence you can not just replace EEDI2 -> NNEDI in all circumstances.

Correct:
Function nnEDIbob(clip Input)
{
[...]
Input.SeparateFields()
GetParity(Input) ? Interleave( SelectEven().nnEDI(dh=true,Field = 1), SelectOdd().nnEDI(dh=true,Field = 0) )
\ : Interleave( SelectEven().nnEDI(dh=true,Field = 0), SelectOdd().nnEDI(dh=true,Field = 1) )
AssumeFrameBased()
GetParity(Input) ? AssumeTFF() : AssumeBFF()
}

It would be less irritating if modes -2...3 were working the same way in EEDI2 and NNEDI, agreed.

[Terranigma]

Well, the problem is about correct usage of NNEDI, so it does fit here somehow.

Wrong:
Function nnEDIbob(clip Input)
{
[...]
GetParity(Input) ? Input.SeparateFields().nnEDI(dh=true,Field = 1) : Input.SeparateFields().nnEDI(dh=true,Field = 0)
AssumeFrameBased()
GetParity(Input) ? AssumeTFF() : AssumeBFF()
}
NNEDI has slightly different operating modes compared to EEDI2, hence you can not just replace EEDI2 -> NNEDI in all circumstances.

Correct:
Function nnEDIbob(clip Input)
{
[...]
Input.SeparateFields()
GetParity(Input) ? Interleave( SelectEven().nnEDI(dh=true,Field = 1), SelectOdd().nnEDI(dh=true,Field = 0) )
\ : Interleave( SelectEven().nnEDI(dh=true,Field = 0), SelectOdd().nnEDI(dh=true,Field = 1) )
AssumeFrameBased()
GetParity(Input) ? AssumeTFF() : AssumeBFF()
}

Thanks Didee for pointing that out. :)

It would be less irritating if modes -2...3 were working the same way in EEDI2 and NNEDI, agreed.
Yes, I fully agree.

---
Link Updated.

[tritical]

Function nnEDIbob(clip Input)
{
Input.SeparateFields()
GetParity(Input) ? Interleave( SelectEven().nnEDI(dh=true,Field = 1), SelectOdd().nnEDI(dh=true,Field = 0) )
\ : Interleave( SelectEven().nnEDI(dh=true,Field = 0), SelectOdd().nnEDI(dh=true,Field = 1) )
AssumeFrameBased()
GetParity(Input) ? AssumeTFF() : AssumeBFF()
}

much simpler:Function nnEDIbob(clip Input)
{
Input.nnedi(field=-2)
}

It would be less irritating if modes -2...3 were working the same way in EEDI2 and NNEDI, agreed.They do work the same way... the only difference is that you don't have to call separatefields() or separatefields().selecteven()/selectodd() before nnedi.

EEDIbob() could be simplified as well:
Function EEDIbob(clip Input, int "maxd")
{
GetParity(Input) ? Input.SeparateFields().EEDI2(Field = 3, maxd = maxd) : Input.SeparateFields().EEDI2(Field = 2, maxd = maxd)
AssumeFrameBased()
GetParity(Input) ? AssumeTFF() : AssumeBFF()
}

to

Function EEDIbob(clip Input, int "maxd")
{
Input.SeparateFields().EEDI2(Field = -2, maxd = maxd)
}

Just make sure you're using at least v0.9.1 of eedi2, the original didn't call vi.SetFieldBased(false).

[Terranigma]

Thanks tritical for making things simpler and faster :D
I updated the bobbers once again, hopefully this'll be the last time :P
With mcbob, I switched the default interpolater with nnedi, changed mtnmode to 1, renamed Vinverse to VinverseD (D = Didée. Seriously Didée, I hope you're ok with this), and included triticals' much faster vinverse dynamic library file. Grab it here (http://www.zshare.net/download/404129650335d2/) or from the link above.

[2Bdecided]

I've always wondered with EEDI2, and this may apply even more to NNEDI...

When I use it to re-size, I call NNEDI twice, with turnright and turnleft after each call. Sometimes I do the whole process twice (so calling NNEDI four times - twice for each dimension).

Would it be possible / better if NNEDI worked in 2-D to start with, so it doubled in each dimension in one pass? I'm not worried about speed, but worried that the current two-pass method forces NNEDI to interpolate from what's already 50% interpolated data. I'm guessing that if it interpolated in 2-D in one pass, it would use only "real" data as a source for interpolation in both dimensions.


Second (easier?) question: with or without the above modification, could it be told to interpolate more than one pixel in a given dimension? I know for interlaced content, it's always 1-in-2 pixels "missing". However, when I process other things, sometimes it's 2-in-3 pixels missing (meaning I only have 1-in-3 pixels available). For now, I just call NNEDI twice, but it would be neater (and avoid interpolating from interpolation again!) if I could call it with a parameter to do 2x (like now), or 3x, or 4x etc interpolation.


I'm not asking or expecting you to do this tritical (I know you're busy), but if either part is easy/possible and you're updating NNEDI at any time anyway, please think about it. It might make it an even more interesting academic project!

Cheers,
David.

[IanB]

Yes, TurnLeft/Right by the nature of the algorithm are slow. They violate the "never look down" rule for fast processing. You either read the src across and write the dest down or vide versa.

[MfA]

Slow is a relative term, on a modern computer it shouldn't take more than a msec to transpose a SD yv12 frame from cache even with the naive C implementation in Avisynth (on mine it takes nearly 2, but that stuff ain't modern). Just a drop in the bucket really.

A bigger problem to me seems that processing U/V planes without data from the Y plane is a bit dodgy.

[2Bdecided]

It's not speed I'm worried about (though if it makes it faster, great!) - it's accuracy. With turnright turnleft NNEDI is being given it's own output to work from - surely it would be better if we could avoid this?

Cheers,
David.

[Leak]

It's not speed I'm worried about (though if it makes it faster, great!) - it's accuracy. With turnright turnleft NNEDI is being given it's own output to work from - surely it would be better if we could avoid this?
That's the way all other resizers in AviSynth work - scaling in two dimensions is done by scaling in one first and then scaling the result in the other.

Also, NNEDI is designed (or rather, trained) to interpolate between image's odd or even lines (which is about the same as doubling the height) - scaling in both directions would need a total rewrite and probably a lot more training.

[2Bdecided]

If it's a total re-write (that's what I feared), I guess I'll use the old workarounds to get the best out of it (i.e. doing both turnright turnleft AND turnleft turnright to give it two stabs at it!).

Of course the other AviSynth resizers are (AFAIK) linear in nature, so working in one dimension then the other, or the opposite, or both together, would all give identical results. That's certainly not the case with EEDI2 and NNEDI, since they're not linear, and not designed to work in both dimensions anyway.

Cheers,
David.

[wonkey_monkey]

would all give identical results.

<stickler>Nearly identical ;)</stickler>

David (a different one)

[Terka]

any news about the new version?

[Adub]

We finished training a lot of NN's for Tritical, so I assume he is running with those somewhere. He may be a little busy elsewhere with school work and what not, but be happy in the conclusion that steps are being taken in the direction of a new version.

[Terka]

how to correctly use yadifmod v1?
how to get the :edeint clip: in there?

[manolito]

Try this:

AssumeTFF (or BFF)
interp = nnedi()
yadifmod(edeint=interp)


Cheers
manolito

[Terka]

thank you, ill try; its working fine

[superuser]

thnks will try it out ... :thumbup:

[yup]

Hi manolito!
Advice script for make 50fps from 25fps interlaced source.
AssumeTFF (or BFF)
interp = nnedi()
yadifmod(edeint=interp)
give 25 fps output.
With kind regards yup.

[foxyshadis]

Read the manuals, they both tell you how to get double-rate output. (Hint: field and mode.) They're not just for decoration!

[tritical]

any news about the new version?
Don't fear, it is still being worked on :). I've just been busy working on some other projects the last week or two. A bit of good news is that I got permission to run the training program on my university's 512 cpu cluster, and I'm doing some test runs as I write this. A new version should be ready in a week or two (no promises though :p).

[Chainmax]

How does one enlarge an image by 2x in both directions using NNEDI v1.3?

[Adub]

turnright
turnleft?

function nnediresize2x(clip c, bool pY, bool pU, bool pV)
{
v = c.nnedi(dh=true,Y=pY,U=pU,V=pV).turnleft()
v = v.nnedi(dh=true,Y=pY,U=pU,V=pV).turnright()
return v
}

[Chainmax]

Doesn't that only need to be done with older versions (the ones you had to put PointResize before or something like that)?

[tritical]

That function is correct. With dh=true, nnedi will double the height (without the need to use pointresize before nnedi). So to resize 2x you need to do that for both the width and height... hence the need to rotate.

[Leak]

That function is correct. With dh=true, nnedi will double the height (without the need to use pointresize before nnedi). So to resize 2x you need to do that for both the width and height... hence the need to rotate.
Are you sure? The last time I tried it (using it to resize (http://www.blanklabelcomics.com/community/viewtopic.php?t=315&start=1100) an image (http://www.wapsisquare.com/d/20071026.html) then tracing it in Inkscape for further resizing and coloring it with GIMP... :D) it also wanted a "field=0" to work...

np: Savath & Savalas - Apnea Obstructiva (Golden Pollen)

[manolito]

it also wanted a "field=0" to work...
Yes, this is true. Whenever you specify "dh=true" then "field=0" or "field=1" has to be present, too. "field=-1" does NOT work in this case (contrary to what tritical says).

This is even true if the source only has one field (analog TV capture @720 x 288 PAL).

Cheers
manolito

[Chainmax]

...
There is a parameter called 'field' to control which field is kept and double vs same rate output (same as the field parameter in eedi2).
...

Sorry for my dumbness, but I wanted to update my IVTC/Deinterlacing/Bob lines I have as templates. Does the quote mean that the equivalent to these lines:


Interp = SeparateFields().SelectEven().EEDI2(field=0)

interp = separatefields().eedi2(field=3)


would be these:


Interp = SeparateFields().SelectEven().NNEDI(field=0)

interp = separatefields().NNEDI(field=3)

?

[tritical]

@Leak/manolito
Yep, you both are correct. The current error checking requires field=0 or field=1 with dh=true. That's a bug.

@Chainmax

Interp = SeparateFields().SelectEven().EEDI2(field=0)

interp = separatefields().eedi2(field=3)

is the same as

Interp = nnedi(field=0)

interp = nnedi(field=3)

[g_aleph_r]

Is there any chance we can see a multithreaded version like eedi2mt? or maybe Cuda enabled?
Pleeeease