Best Motion Interpolation's software/plugin ?

[manolito]

From MysteryX

With FrameRateConverter I have a pretty good algorithm for detecting artifacts but there's just no other backup plan than blending frames in those areas

From lansing

Frame blending just doesn't look good on hand waving because on playback, it makes it looking like Matrix.

I believe that these artifacts have nothing to do with falling back to frame blending when artifacts are detected. I ran the source through the old original johnmeyer script from this post
https://forum.doom9.org/showthread.p...39#post1800439

and the artifacts were identical to the ones caused by FramerateConverter. And the johnmeyer script does not do any frame blending.

Cheers
manolito

[poisondeathray]

Originally Posted by johnmeyer

Which one is on the left? You didn't label them. The one on the left looks like standard ME, so I assume it is Twixtor, but maybe I'm wrong.

The order of presentation correlates with the order of text. So, left=dimitri , right=twixtor + track points. Note, that "automatic" or default twixtor looks a lot like mvtools2 results . Sometimes you need a bit of guidance for the motion estimation to make a shot usable.

Originally Posted by manolito

I believe that these artifacts have nothing to do with falling back to frame blending when artifacts are detected. I ran the source through the old original johnmeyer script from this post
https://forum.doom9.org/showthread.p...39#post1800439

and the artifacts were identical to the ones caused by FramerateConverter. And the johnmeyer script does not do any frame blending.

I agree, the "blending" is just mvtools2



I also tried ffmpeg minterpolate
https://ffmpeg.org/ffmpeg-filters.html#minterpolate

The default settings, are a lot like mvtools2, and you end up with frame blending, but occlusion and morphing artifacts with the default mode (There are many other settings you can play with, but only had time to play with a few)


And butterflow
https://github.com/dthpham/butterflow

All command line driven (well I guess ffmpeg minterpolate is too... ) but this can use OpenCL . This resulted in almost pure blending, so meh.. but I used the 0.2.4 dev version, not sure if something got broken, I'll revisit 0.2.3 stable when I have time



Blending might be "bad" for realtime playback; but sometimes blobby edge morphing occlusion artifacts are worse. At least blending is consistent. pros/cons

[zorr]

Originally Posted by MysteryX

I just use LWLibavVideoSource for everything and don't have any issue.

Thanks, now I just need to figure out which Libav binary distribution to install... do you have recommendation on that? I'm using 32bit AviSynth.

[EDIT] Or should it work without Libav installation? I got

AviSynth open failure:
[Fatal]: Failed to avformat_open_input.

when I tried it.

[EDIT2] I'm dumb. I had the file name wrong. Works perfectly now.

[MysteryX]

L-SMASH Source

[zorr]

I tried John Meyer's frame doubling script with that hand waving video. MediaInfo says

Frame rate mode                          : Constant
Frame rate                               : 23.976 (24000/1001) FPS

so I put 2 times 23.976 as the target frame rate.

For some reason the result doesn't have all the original frames. It goes "out of sync" after frame 55, every frame (or at least most of them) are interpolated after that. Using AssumeFPS doesn't help either. I also tried setting num=48000 and den=1001 but that made it go out of sync slightly faster, after frame 53.

Here's the script I tried, it should display nothing but black if the result has original frames. Can anyone spot an error?

LWLibavVideoSource("d:\process\hand wave.mkv")

orig = last
fps = jm_fps(last, 2*23.976, 32, 2)
orig_only = fps.SelectEven()
#orig_only.AssumeFPS(23.976)
return orig.Subtract(orig_only).Histogram("luma")

# Motion Protected FPS converter script by johnmeyer from Doom9
# Slightly modified interface by manolito
# Requires MVTools V2 and RemoveGrain
# Also needs fftw3.dll in the System32 or SysWOW64 folder for Dct values other than 0
function jm_fps(clip source, float "fps", int "BlkSize", int "Dct") {
	fps = default(fps, 25.000)
	fps_num = int(fps * 1000)
	fps_den = 1000
	BlkSize = default(BlkSize, 16)
	Dct = default(Dct, 0)

#	fps_num = 48000
#	fps_den = 1001

	prefiltered = RemoveGrain(source, 22)
	super = MSuper(source, hpad = 16, vpad = 16, levels = 1, sharp = 1, rfilter = 4) # one level is enough for MRecalculate
	superfilt = MSuper(prefiltered, hpad = 16, vpad = 16, sharp = 1, rfilter = 4) # all levels for MAnalyse
	backward = MAnalyse(superfilt, isb = true, blksize = BlkSize, overlap = 4, search = 3, dct = Dct)
	forward = MAnalyse(superfilt, isb = false, blksize = BlkSize, overlap = 4, search = 3, dct = Dct)
	forward_re = MRecalculate(super, forward, blksize = 8, overlap = 2, thSAD = 100)
	backward_re = MRecalculate(super, backward, blksize = 8, overlap = 2, thSAD = 100)
	out = MFlowFps(source, super, backward_re, forward_re, num = fps_num, den = fps_den, blend = false, ml = 200, mask = 2)

	return out
}

[poisondeathray]

Originally Posted by zorr

Here's the script I tried, it should display nothing but black if the result has original frames. Can anyone spot an error?
commment out this line, replace it with fps .

#return orig.Subtract(orig_only).Histogram("luma")

It should be fps, or return fps

orig = last
fps = jm_fps(last, 2*23.976, 32, 2)
orig_only = fps.SelectEven()
#orig_only.AssumeFPS(23.976)
#return orig.Subtract(orig_only).Histogram("luma")
fps

[manolito]

If you want to double your frame rate and make sure that every other frame will be an original frame then you should edit the johnmeyer script like this:

Before:

out = MFlowFps(source, super, backward_re, forward_re, num = fps_num, den = fps_den, blend = false, ml = 200, mask = 2)

return out

After:

out = MFlowFps(source, super, backward_re, forward_re, num = fps_num, den = fps_den, blend = false, ml = 200, mask = 2)

out=out.selectodd()
interleave(source,out)
return last

Cheers
manolito

[zorr]

Originally Posted by poisondeathray

#return orig.Subtract(orig_only).Histogram("luma")

It should be fps, or return fps

orig = last
fps = jm_fps(last, 2*23.976, 32, 2)
orig_only = fps.SelectEven()
#orig_only.AssumeFPS(23.976)
#return orig.Subtract(orig_only).Histogram("luma")
fps

That's what I tried first, but I noticed the above mentioned problem. This script is just demonstrating the problem clearly, by taking every second frame from the fps clip and comparing it to the original. They should be identical but they're not.

[poisondeathray]

Oh, I see what you're doing, you're testing it . I get all black, but I'm using FFMS2 . It's probably a source filter issue

FFVideoSource("hand wave.mkv")

[poisondeathray]

Yes, that's the problem. You ccan check with info()
LWLibavVideoSource returns an off framerate 23.9752 . Probably jitter in the timecodes. Add AssumeFPS(24000,1001). Framecount is correct

Or use FFMS2, which returns 24000/1001

[zorr]

Originally Posted by poisondeathray

Yes, that's the problem. You ccan check with info()
LWLibavVideoSource returns an off framerate 23.9752 . Probably jitter in the timecodes. Add AssumeFPS(24000,1001). Framecount is correct

Or use FFMS2, which returns 24000/1001

Thanks, using AssumeFPS(24000,1001) fixes it. Or using AssumeFPS(23.976).

I thought it couldn't be a problem with the source because I also converted the mkv to Huffyuv-encoded avi and the problem occurred with that file as well. But obviously the error in framerate just propagated there.

So is FFMS2 better than LWLibavVideoSource in every way?

[poisondeathray]

Originally Posted by zorr

So is FFMS2 better than LWLibavVideoSource in every way?

No, sometimes one is preferred over the other for some types of sources. And it depends on the specific version too.

I find you need to have both, and backup plan B,C,D,E

[zorr]

Originally Posted by manolito

If you want to double your frame rate and make sure that every other frame will be an original frame then you should edit the johnmeyer script like this:
...

That's certainly a way to achieve it, but it seems like it's just covering the real problem which is the incorrect frame rate reported by the source filter. I think the interpolation quality will suffer (perhaps not much, but who knows) when MFlowFps is working with incorrect frame rate.

[StainlessS]

Originally Posted by zorr

That's certainly a way to achieve it, but it seems like it's just covering the real problem which is the incorrect frame rate reported by the source filter. I think the interpolation quality will suffer (perhaps not much, but who knows) when MFlowFps is working with incorrect frame rate.

Manolito posted that before it becme clear that you had a source filter problem.

Also, AssumeFPS(24000,1001) is better than AssumeFPS(23.976),
as 24000/1001 is actually 23.97602398.

EDIT: To below,

AssumeFPS(23.976) is an alias for AssumeFPS(24000,1001) in Avisynth 2.57 and up, along with a number of other common framerates.

Thanks Foxy, I had seen in sources, detection and replacment of close to standard common framerates, indeed I do it myself in some scripts.

[foxyshadis]

Originally Posted by StainlessS

Manolito posted that before it becme clear that you had a source filter problem.

Also, AssumeFPS(24000,1001) is better than AssumeFPS(23.976),
as 24000/1001 is actually 23.97602398.

AssumeFPS(23.976) is an alias for AssumeFPS(24000,1001) in Avisynth 2.57 and up, along with a number of other common framerates. Still good practice to use the real fraction in case you need a non-standard framerate, though.

[zorr]

Originally Posted by StainlessS

Manolito posted that before it becme clear that you had a source filter problem.

That's true, I didn't mean to diss his answer. Especially when he was just trying to help me. Sorry Manolito!

Originally Posted by StainlessS

Also, AssumeFPS(24000,1001) is better than AssumeFPS(23.976),
as 24000/1001 is actually 23.97602398.

I started to wonder why is it reported as 23.976 (by MediaInfo and even by AviSynth) when that's not what the correct ratio is. Maybe it's because 32bit float can only represent about 7 decimals and the correct ratio needs 8.

[StainlessS]

and the correct ratio needs 8

Sorry, I just copied from my calculator, no idea how many digits required to accurately represent true result, if indeed it can be expressed in a finite number of digits.
Maybe I should have said something akin to "as 24000/1001 is actually more like 23.97602398."

EDIT: Using WinXP Calculator, Scientific mode, 23.976023976023976023976023976024, 239760 sequence recurring.

[Sparktank]

Originally Posted by StainlessS

Sorry, I just copied from my calculator, no idea how many digits required to accurately represent true result, if indeed it can be expressed in a finite number of digits.

Online calculators end up repeating "976023", even if use some unbound decimal points/etc.

23.976023976023976023976023976023etc

[StainlessS]

Actually sequence starts right at the beginning, ie 239760, etc

[Sparktank]

I see what happened. The coffee cup is always half empty.