Frame interpolation using NNEDI3?

[Soulvomit]

This idea occurred to me but I'm having a hard time executing it. When used as a deinterlacer, NNEDI3 works spatially to create new fields, but how can it be used temporally as a frame interpolator? I know it isn't intended for this purpose but I'm really curious of the results. So what would that script look like?

[Sharc]

Spatial and temporal interpolation are based on different principles.
I don't think you can 'abuse' nnedi3 for temporal interpolation just by means of a script.
For temporal interpolation see the 'Frame Rate Conversion' section in avisynth's External Filters.

[johnmeyer]

Spatial and temporal interpolation are based on different principles.Absolutely true, but don't the "better" deinterlacers, like QTGMC, do both? It uses MVTools2 to estimate a field at an intermediate point in time, shifted up or down by one scan line.

[Frank62]

I thought so, too. But I fear there will be no way to produce something "in between" two frames. You could simply produce pseudo-interlaced frames by taking a clip and the same shifted one frame, maybe like this:

source=xxsource("file")
shift=trim(source,1,0)
i=interleave(source,shift)
i=weave(i)

But if you deinterlace this, the interpolation will always be optimized to the one field you specify. If temporal methods are used like with QTGMC, then full motion-compensation is used to produce a second clip for interpolation, but still for the exact same point in time. If no temporal methods are used, interpolation is fully spatial, so the second field won't be used at all. The idea, though, is interesting somehow.

[Soulvomit]

Yeah, I know there are better, proper ways to interpolate frames. This is just strictly an experiment of using NNEDI3's way of creating new fields to create new frames instead. Let's assume the source is a 640x480 progressive RGB video. You would have to first separate it into individual fields or rows of pixels. For the first field/row you would crop vertically (crop(0,0,0,-479), then stack those rows vertically with one another, then double the height of that stack with point resize, then deinterlace that stack with NNEDI3(field=1) to get the new fields/rows, then separate/unstack those new fields/rows, before finally stacking them again with the other new ones after doing this 479 other times and interleaving that with the source. It's the stacking and unstacking I'm stumped on.

[Frank62]

I didn't want to show that there are better ways to interpolate. I only used QTGMC as an example, I know better. I don't know exactly how nnedi3 works, but I wanted to show that:

IF it does interpolation spatially then there can be no temporal benefit at all.
IF nnedi tries to interpolate temporally from neighbour frames then this must be done similar to QTGMC:
1. FULL motion compensation of the neighbour frames
2. Use these compensated pixels from neighbour frames in some way
The problem for your task is that pixels that are temporally BETWEEN frames will never be generated.
Can't say it better in English.

[Soulvomit]

I'm with you. I just used the word temporally very loosely just to refer to using NNEDI3 horizontally instead of vertically using adjacent frames for the creation of new ones. Technically this is still a spatial affair as that's the only way NNEDI3 works on its own and nothing else will be used, I'm very aware of that, but even so I'm curious to see the results of this experiment.