I need some help with interpolation videos, lots of detail in post [Archive] - Page 2

Sparktank

25th July 2018, 22:58

What I am currently trying to achieve is to find the script that made that youtube video, regardless of how well that script would work with the rest of the film.

An impossible feat.
From the comments in the youtube video, the uploader doesn't know anything either. He just found it and uploaded it.

But, given that it was in 2014, the options were limited back then.
SVP/Spriton/mvtools.

Your best bet is to stick with the scripts posted.

There was probably some light artifact masking in it if they used SVP Pro.
Free vs Pro, you get to customize a lot in Pro (SVP 4).
back then, I think it was only SVP3 and still customizable before it went SVP 4 Pro.

You're not going to be able to 'guess' the settings.
If you can find the original person who created the video, they might still have the script.
Or they just might say use SVP.

But I tried the jm_fps script and the mod the other day on a ProRes trailer (1080p) and it came out incredibly smooth without much artefacts (except for a few complex scenes).

Your best bet is to play with SVP and its settings (if you paid for it).
The SVPFlow library uses some of MVtools settings, which you can read about here:
https://www.svp-team.com/wiki/Manual:SVPflow

With SVP 4 Pro, using the max settings for everything (encoding/playback) doesn't always guarantee "the best" settings.

There were days where I spent a couple hours going through various scenes before settling on an average to finally watch the whole movie.

lansing

25th July 2018, 23:04

Not really. Visually they are different. Blending is slightly smoother but gives a "strobey" or "ghosting" look. Repeated frames is the cleanest, but the least smooth - it's what most people would say exhibits the most "stutter"

For example try ChangeFPS vs. ConvertFPS . You're implying they give visually same result ?? They definitely don't.

Yes you are right about the ghosting with blending frames, but that does not make it smoother. When an object is moving from position A to position B, with the ghosting effect, visually you kind of see the object arrived to B before it actually happened, but the motion of the object does not get smoother, it still would be stuttering.

poisondeathray

25th July 2018, 23:15

Yes you are right about the ghosting with blending frames, but that does not make it smoother. When an object is moving from position A to position B, with the ghosting effect, visually you kind of see the object arrived to B before it actually happened, but the motion of the object does not get smoother, it still would be stuttering.

It's usually distinguished from "stuttering" which implies a pure repeat frame (a cadence of repeat frames , usually irregular , like AABBBCDDDDDD)

People usually call "blending" slightly smoother than pure repeats because edges are less aliased. It's along the same lines as motion blur the makes the appearance of motion smoother. The object hasn't really moved their either

Personally I think blending is terrible, but ask 10 people and 9/10 people will say it's slightly smoother than pure repeats . Personally I find the "strobey" look nauseating

bradwiggo

26th July 2018, 00:34

An impossible feat.
From the comments in the youtube video, the uploader doesn't know anything either. He just found it and uploaded it.

But, given that it was in 2014, the options were limited back then.
SVP/Spriton/mvtools.

Your best bet is to stick with the scripts posted.

There was probably some light artifact masking in it if they used SVP Pro.
Free vs Pro, you get to customize a lot in Pro (SVP 4).
back then, I think it was only SVP3 and still customizable before it went SVP 4 Pro.

You're not going to be able to 'guess' the settings.
If you can find the original person who created the video, they might still have the script.
Or they just might say use SVP.

But I tried the jm_fps script and the mod the other day on a ProRes trailer (1080p) and it came out incredibly smooth without much artefacts (except for a few complex scenes).

Your best bet is to play with SVP and its settings (if you paid for it).
The SVPFlow library uses some of MVtools settings, which you can read about here:
https://www.svp-team.com/wiki/Manual:SVPflow

With SVP 4 Pro, using the max settings for everything (encoding/playback) doesn't always guarantee "the best" settings.

There were days where I spent a couple hours going through various scenes before settling on an average to finally watch the whole movie.

Would the interpolation using SVP 3 look better than using SVP 4? Or if not necessarily better would there be a significant difference.

lansing

26th July 2018, 00:48

Thus, the OP's original desire to increase the frame rate in order to eliminate these visual disturbances is quite well founded, but the reality that he won't seem to acknowledge is that the technology does not exist to do this on all scenes. For a month he has posted that he thinks this is possible because he has seen examples where 24 fps has been increased in frame rate without introducing motion estimation artifacts. The problem is, these examples show scenes where ME works just fine, but it will always fail on scenes with attributes that I have described multiple times in previous posts.

Of course if you can come up with a solution for his video that works, my hat is off to you!

I thought he was only asking about how to get the result of the youtube video, which can easily be achieved by using the sample script provided by the svpflow package.

The only thing he needs to do now is to figure out how to get his amd intergrated graphic card to work with the script.

Sparktank

26th July 2018, 01:04

Would the interpolation using SVP 3 look better than using SVP 4? Or if not necessarily better would there be a significant difference.

They made a lot of improvements to 4, so I'd say 3 would be worse.
I did some testing after 4 came out and never went back to 3.

It's $24.99 (USD).
https://www.svp-team.com/wiki/Purchase

you can do just as good with the scripts posted in the forum.
you're not going to find better out there (or here).

Unless something significant happens like a whole new level to the mvtools code.
Which seems to be mostly just updates for avs+ and new-age colorspaces (beyond yv12/yv16/yv24).

Or wait until NVidia becomes more popular with it's cuDNN library and find a way into mainstream editors.
https://forum.doom9.org/showthread.php?p=1845068#post1845068

poisondeathray

26th July 2018, 02:26

This comparison is aligned (using trim()) and stacked; each is resized 1212x540 to keep approx. AR (the reason is so it can be stacked and viewed on a 1080 height screen).

They are not labelled on purpose - Which one is the YT video, which one is the jm_fps manolito mod (with blend=true, masking disabled) using your test1.mkv ? It's easy to tell from a compression standpoint (the YT version will have more compression artifacts)
http://www.mediafire.com/file/vmi3pm1h3o2bb6y/compare.mp4/file

And here is the same thing, just pillarboxed to get 1080p60 treatment on YT
https://www.youtube.com/watch?v=DUmHTLkEgec

Again, tell me which is which. Or which do you think is "smoother" ?

bradwiggo

26th July 2018, 12:48

And here is the same thing, just pillarboxed to get 1080p60 treatment on YT
https://www.youtube.com/watch?v=DUmHTLkEgec

Again, tell me which is which. Or which do you think is "smoother" ?

So is one of those the file I uploaded and one is the youtube video? Could you upload those 2 videos but separately (but with random names so I don't know which is which), as I struggle to see the fine details when it is only half the size of the screen (my laptop only has a 720p screen).

poisondeathray

26th July 2018, 15:23

So is one of those the file I uploaded and one is the youtube video? Could you upload those 2 videos but separately (but with random names so I don't know which is which), as I struggle to see the fine details when it is only half the size of the screen (my laptop only has a 720p screen).

Yes; but it will be easy to tell from the filesize difference

The "smoothness" is pretty much the same. There is nothing "special" about the youtube video. The only significant difference between any default script setting is the blended scene changes

When you stack them and use the same player, you eliminate all the other potential issues maybe you had a source decoding issue introducing duplicates ; or playback issues like different decoder, different player, different renderer. (Even the same player can use different decoding pathway)

You can download the YT version, and make the other one yourself. As mentioned earlier, the script is the jm_fps manolito mod (I just copied and pasted from your earlier post), with scene blending enabled, masking disabled

You can go frame by frame , or if you want compare them in different tabs in avspmod (if you resize to same dimensions, trim() to align the frames - they will be superimposed and you just hit the number keys to swap tabs back and forth - very easy to see frame differences this way)

Or if you're just looking at individual videos, separately, then you don't need to resize or align them. But it's easy to see they they are very similar in terms of "smoothness".

Or, if you still think they are different in terms of "smoothness" then we need to investigate farther - eg. maybe there was an encoding issue your end?, maybe the encoding settings you're using are causing problems? YT videos are encoded so they can be easily decoded across platforms. Maybe player dropping frames, etc...

FFVideoSource("test1.mkv")
jm_fps(59.94)

As mentioned earlier, only the very last line is changed (highlighted in red) ; to blend=true, and the masking disabled (rest of line commented out). You could do the same thing in SVPFlow or any of the dozen interpolation variants. They produce similar results when using similar settings

# Motion Protected FPS converter script by johnmeyer from Doom9
# Slightly modified interface by manolito
# Requires MVTools V2 and RemoveGrain
# Also needs fftw3.dll in the System32 or SysWOW64 folder for Dct values other than 0

function jm_fps(clip source, float "fps", int "BlkSize", int "Dct")
{
fps = default(fps, 25.000)
fps_num = int(fps * 1000)
fps_den = 1000
BlkSize = default(BlkSize, 16)
Dct = default(Dct, 0)

prefiltered = RemoveGrain(source, 22)
super = MSuper(source, hpad = 16, vpad = 16, levels = 1, sharp = 1, rfilter = 4) # one level is enough for MRecalculate
superfilt = MSuper(prefiltered, hpad = 16, vpad = 16, sharp = 1, rfilter = 4) # all levels for MAnalyse
backward = MAnalyse(superfilt, isb = true, blksize = BlkSize, overlap = 4, search = 3, dct = Dct)
forward = MAnalyse(superfilt, isb = false, blksize = BlkSize, overlap = 4, search = 3, dct = Dct)
forward_re = MRecalculate(super, forward, blksize = 8, overlap = 2, thSAD = 100)
backward_re = MRecalculate(super, backward, blksize = 8, overlap = 2, thSAD = 100)
out = MFlowFps(source, super, backward_re, forward_re, num = fps_num, den = fps_den, blend = true)#, ml = 200, mask = 2)

return out
}

bradwiggo

26th July 2018, 15:52

Yes; but it will be easy to tell from the filesize difference

The "smoothness" is pretty much the same. There is nothing "special" about the youtube video. The only significant difference between any default script setting is the blended scene changes

When you stack them and use the same player, you eliminate all the other potential issues maybe you had a source decoding issue introducing duplicates ; or playback issues like different decoder, different player, different renderer. (Even the same player can use different decoding pathway)

You can download the YT version, and make the other one yourself. As mentioned earlier, the script is the jm_fps manolito mod (I just copied and pasted from your earlier post), with scene blending enabled, masking disabled

You can go frame by frame , or if you want compare them in different tabs in avspmod (if you resize to same dimensions, trim() to align the frames - they will be superimposed and you just hit the number keys to swap tabs back and forth - very easy to see frame differences this way)

Or if you're just looking at individual videos, separately, then you don't need to resize or align them. But it's easy to see they they are very similar in terms of "smoothness".

Or, if you still think they are different in terms of "smoothness" then we need to investigate farther - eg. maybe there was an encoding issue your end?, maybe the encoding settings you're using are causing problems? YT videos are encoded so they can be easily decoded across platforms. Maybe player dropping frames, etc...

FFVideoSource("test1.mkv")
jm_fps(59.94)

As mentioned earlier, only the very last line is changed (highlighted in red) ; to blend=true, and the masking disabled (rest of line commented out). You could do the same thing in SVPFlow or any of the dozen interpolation variants. They produce similar results when using similar settings

# Motion Protected FPS converter script by johnmeyer from Doom9
# Slightly modified interface by manolito
# Requires MVTools V2 and RemoveGrain
# Also needs fftw3.dll in the System32 or SysWOW64 folder for Dct values other than 0

function jm_fps(clip source, float "fps", int "BlkSize", int "Dct")
{
fps = default(fps, 25.000)
fps_num = int(fps * 1000)
fps_den = 1000
BlkSize = default(BlkSize, 16)
Dct = default(Dct, 0)

prefiltered = RemoveGrain(source, 22)
super = MSuper(source, hpad = 16, vpad = 16, levels = 1, sharp = 1, rfilter = 4) # one level is enough for MRecalculate
superfilt = MSuper(prefiltered, hpad = 16, vpad = 16, sharp = 1, rfilter = 4) # all levels for MAnalyse
backward = MAnalyse(superfilt, isb = true, blksize = BlkSize, overlap = 4, search = 3, dct = Dct)
forward = MAnalyse(superfilt, isb = false, blksize = BlkSize, overlap = 4, search = 3, dct = Dct)
forward_re = MRecalculate(super, forward, blksize = 8, overlap = 2, thSAD = 100)
backward_re = MRecalculate(super, backward, blksize = 8, overlap = 2, thSAD = 100)
out = MFlowFps(source, super, backward_re, forward_re, num = fps_num, den = fps_den, blend = true)#, ml = 200, mask = 2)

return out
}

Did you cut the youtube video to the same size as the one I made? I will try cutting it and then go through frame by frame with the youtube one and my one on a different monitor.

poisondeathray

26th July 2018, 16:06

Did you cut the youtube video to the same size as the one I made? I will try cutting it and then go through frame by frame with the youtube one and my one on a different monitor.

Yes; as mentioned earlier, I aligned them temporally (using trim , so the start frame is the same) , and spatially by resized them. (so they are the same dimensions). This enables you to stack them or view in different tabs superimposed

Another way you can do this frame by frame on a single 720p monitor is split screen and in avspmod (e.g. left half YT, right half jm_fps) , but put the reverse in tab 2 (e.g. left half jm_fps, right half YT). So you could advance frame by frame with arrow keys and swap views by hitting number keys (1 for tab 1, 2 for tab 2)

It's important to just "watch" it normally for motion issues, but having them superimposed in different tabs and looking frame by frame reveals problems each version too. For example there are worse problems in YT version at the end zooming away from the tower. But overall, the motion "smoothness" is the same, and overall the motion artifacts are very similar; you can tell that some mvtools2 derivative was used for sure because of this

bradwiggo

26th July 2018, 16:34

Yes; as mentioned earlier, I aligned them temporally (using trim , so the start frame is the same) , and spatially by resized them. (so they are the same dimensions). This enables you to stack them or view in different tabs superimposed

Another way you can do this frame by frame on a single 720p monitor is split screen and in avspmod (e.g. left half YT, right half jm_fps) , but put the reverse in tab 2 (e.g. left half jm_fps, right half YT). So you could advance frame by frame with arrow keys and swap views by hitting number keys (1 for tab 1, 2 for tab 2)

It's important to just "watch" it normally for motion issues, but having them superimposed in different tabs and looking frame by frame reveals problems each version too. For example there are worse problems in YT version at the end zooming away from the tower. But overall, the motion "smoothness" is the same, and overall the motion artifacts are very similar; you can tell that some mvtools2 derivative was used for sure because of this

Something I have just noticed is the two clips in the compare.mp4 video you made are slightly different colours. I used xnView MP (my default image viewer) to look at a screenshot I took of the video. I got the RGB colour value of the same pixel on each video, and the top one was RGB(197, 167, 241, 255), and the bottom one was RGB(201, 169, 251, 255). Did you download the youtube video directly from youtube using a downloader, if so do you think this colour difference may be down to youtube's compression?

poisondeathray

26th July 2018, 16:47

Something I have just noticed is the two clips in the compare.mp4 video you made are slightly different colours. I used xnView MP (my default image viewer) to look at a screenshot I took of the video. I got the RGB colour value of the same pixel on each video, and the top one was RGB(197, 167, 241, 255), and the bottom one was RGB(201, 169, 251, 255). Did you download the youtube video directly from youtube using a downloader, if so do you think this colour difference may be down to youtube's compression?

Yes, a downloader.

It's probably compression differences. But you're not using exactly the same source as they did either, and there are multiple generation (rounding) differences

For example, even different studio release versions from the same blu-ray in the same year can have different colors.

bradwiggo

26th July 2018, 17:32

Yes, a downloader.

It's probably compression differences. But you're not using exactly the same source as they did either, and there are multiple generation (rounding) differences

For example, even different studio release versions from the same blu-ray in the same year can have different colors.

There is also a download link in the description I think. Does interpolation to 60fps cause problems as it was originally 24fps? In the interpolation script it said fps = 25, and I though that might have caused the original stuttering, so I later changed it to 24fps, but the output video was the same, does it automatically detect the framerate?

poisondeathray

26th July 2018, 17:38

There is also a download link in the description I think. Does interpolation to 60fps cause problems as it was originally 24fps? In the interpolation script it said fps = 25, and I though that might have caused the original stuttering, so I later changed it to 24fps, but the output video was the same, does it automatically detect the framerate?

For that script, you enter the desired framerate (59.94) . Other types of scripts have slightly different syntax. For example, SVPFlow uses a multiplier for numerator and denominator

FFVideoSource("test1.mkv")
jm_fps(59.94)

So if FFVideoSource is loading the video correctly at 23.976, jm_fps will interpolate that to 59.94

It interpolates from the fps avisynth "thinks" the source is. That is partially determined by the source filter

So if you have a source filter that returns an "off" frame rate, you could possibly get the wrong results

You can use info() to "see" what avisynth "thinks" the framerate is, with the video only . It will print out an overlay on top. Some source filters have problems with some sources or containers, and you might have to make adjustments . That might have caused some of the issues you were seeing earlier and in the videohelp thread . (But another difference for sure, is the scenechange blending vs. duplicates)

eg.

FFVideoSource("test1.mkv")
Info()

When you align clips, or compare them in different tabs, or stack them - it's very easy to see something is "off" right away . Or that they match.

bradwiggo

26th July 2018, 17:42

For that script, you enter the desired framerate (59.94)

FFVideoSource("test1.mkv")
jm_fps(59.94)

So if FFVideoSource is loading the video correctly at 23.976, jm_fps will interpolate that to 59.94

It interpolates from the fps avisynth "thinks" the source is. That is partially determined by the source filter

So if you have a source filter that returns an "off" frame rate, you could possibly get the wrong results

You can use info() to "see" what avisynth "thinks" the framerate is, with the video only . It will print out an overlay on top. Some source filters have problems with some sources or containers, and you might have to make adjustments . That might have caused some of the issues you were seeing earlier and in the videohelp thread . (But another difference for sure, is the scenechange blending vs. duplicates)

eg.

FFVideoSource("test1.mkv")
Info()

When you align clips, or compare them in different tabs, or stack them - it's very easy to see something is "off" right away . Or that they match.

Is that defining the framerate for this script:

# Motion Protected FPS converter script by johnmeyer from Doom9
# Slightly modified interface by manolito
# Requires MVTools V2 and RemoveGrain
# Also needs fftw3.dll in the System32 or SysWOW64 folder for Dct values other than 0

function jm_fps(clip source, float "fps", int "BlkSize", int "Dct")
{
fps = default(fps, 25.000)
fps_num = int(fps * 1000)
fps_den = 1000
BlkSize = default(BlkSize, 16)
Dct = default(Dct, 0)

prefiltered = RemoveGrain(source, 22)
super = MSuper(source, hpad = 16, vpad = 16, levels = 1, sharp = 1, rfilter = 4) # one level is enough for MRecalculate
superfilt = MSuper(prefiltered, hpad = 16, vpad = 16, sharp = 1, rfilter = 4) # all levels for MAnalyse
backward = MAnalyse(superfilt, isb = true, blksize = BlkSize, overlap = 4, search = 3, dct = Dct)
forward = MAnalyse(superfilt, isb = false, blksize = BlkSize, overlap = 4, search = 3, dct = Dct)
forward_re = MRecalculate(super, forward, blksize = 8, overlap = 2, thSAD = 100)
backward_re = MRecalculate(super, backward, blksize = 8, overlap = 2, thSAD = 100)
out = MFlowFps(source, super, backward_re, forward_re, num = fps_num, den = fps_den, blend = true)#, ml = 200, mask = 2)

return out
}

Also, what does jm mean? Is it just a variable that we set to certain values?

poisondeathray

26th July 2018, 17:46

Is that defining the framerate for this script:

We probably posted at the same time; have a look at the post above it explains some things

The function is defined in the stuff below in that script. You have to call the function to use it in the script. jm_fps(something) . If I used jm_fps(120) it would interpolate to 120 fps

You could have "named" the function anything like jm_fps2 or brads_function() or whatever

Also, what does jm mean? Is it just a variable that we set to certain values?

"jm" is for our buddy John Meyer :). He posted a few times in this thread. That manolito version is a variation on one he posted years ago (by you guessed it, manolito)

bradwiggo

26th July 2018, 17:56

We probably posted at the same time; have a look at the post above it explains some things

The function is defined in the stuff below in that script. You have to call the function to use it in the script. jm_fps(something) . If I used jm_fps(120) it would interpolate to 120 fps

You could have "named" the function anything like jm_fps2 or brads_function() or whatever

"jm" is for our buddy John Meyer :). He posted a few times in this thread. That manolito version is a variation on one he posted years ago (by you guessed it, manolito)

# Motion Protected FPS converter script by johnmeyer from Doom9
# Slightly modified interface by manolito
# Requires MVTools V2 and RemoveGrain
# Also needs fftw3.dll in the System32 or SysWOW64 folder for Dct values other than 0
LoadPlugin("C:\Users\bradw\Downloads\masktools2-v2.2.17\x86\masktools2.dll")
LoadPlugin("C:\Users\bradw\Downloads\mvtools-v2.5.11.22\mvtools2.dll")
LoadPlugin("C:\Users\bradw\Downloads\RgTools-0.97\x86\RgTools.dll")
LoadPlugin("C:\Users\bradw\Downloads\GRunT101\GRunT.dll")
LoadPlugin("C:\Users\bradw\Downloads\MeGUI-2836-32\tools\avisynth_plugin\svpflow1.dll")
LoadPlugin("C:\Users\bradw\Downloads\MeGUI-2836-32\tools\avisynth_plugin\svpflow2.dll")
PluginPath = "C:\Users\bradw\Downloads\MeGUI-2836-32\tools\lsmash\"
LoadPlugin(PluginPath+"LSMASHSource.dll")

Function jm_fps(clip source, float "fps", int "BlkSize", int "Dct") {
fps = default(fps, 23.976)
fps_num = int(fps * 1000)
fps_den = 1000
BlkSize = default(BlkSize, 16)
Dct = default(Dct, 0)
prefiltered = RemoveGrain(source, 22)
super = MSuper(source, hpad = 16, vpad = 16, levels = 1, sharp = 1, rfilter = 4) # one level is enough for MRecalculate
superfilt = MSuper(prefiltered, hpad = 16, vpad = 16, sharp = 1, rfilter = 4) # all levels for MAnalyse
backward = MAnalyse(superfilt, isb = true, blksize = BlkSize, overlap = 4, search = 3, dct = Dct)
forward = MAnalyse(superfilt, isb = false, blksize = BlkSize, overlap = 4, search = 3, dct = Dct)
forward_re = MRecalculate(super, forward, blksize = 8, overlap = 2, thSAD = 100)
backward_re = MRecalculate(super, backward, blksize = 8, overlap = 2, thSAD = 100)
out = MFlowFps(source, super, backward_re, forward_re, num = fps_num, den = fps_den, blend = true)#, ml = 200, mask = 2)
return out
}

DCT = 0 # EDIT: 1 is SLOW
BLKSIZE = 16

c=LSMASHVideoSource("C:\Users\bradw\Documents\source.mkv")

Return jm_fps(c,fps=c.FrameRate*2,blkSize=BLKSIZE,dct=DCT)

That is my current script, is that right, as it is slightly different to the one you posted. You had the number in blue as 25, I set it to 23.976 as I believe that is the framerate of the source.

If I used jm_fps(120) it would interpolate to 120 fps

If you wanted to do that would you change the text in red to that?

poisondeathray

26th July 2018, 18:13

<snip>

That is my current script, is that right, as it is slightly different to the one you posted. You had the number in blue as 25, I set it to 23.976 as I believe that is the framerate of the source.

If you wanted to do that would you change the text in red to that?

Not the way you have it set up in that script; the blue actually does nothing - it's just a default value. As explained in the post above, the source FPS is determined by what avisynth "thinks" it is from the source filter . Or you can manually override it with AssumeFPS(something) . You have to do that sometimes if the source filter isn't returning a proper value

The final fps is in the return line "fps=c.FrameRate*2"

"c" refers to the source video as loaded by LSmash in that script

So if LSMash returned the proper 23.976 fps for the source, it would be 23.976*2 or 47.952. If it "thought" the video was 30.0 fps, you would get 30.0*2 or 60.0fps

So if you wanted output 120.0 FPS for the script as you have it now, you would just change the last line to

Return jm_fps(c,fps=120, blkSize=BLKSIZE,dct=DCT)

But again , the interpolation also partially depends on source framerate (or what avisynth "thinks" the source framerate is). If it's not loading correctly , you can get off results

bradwiggo

26th July 2018, 18:50

Not the way you have it set up in that script; the blue actually does nothing - it's just a default value. As explained in the post above, the source FPS is determined by what avisynth "thinks" it is from the source filter . Or you can manually override it with AssumeFPS(something) . You have to do that sometimes if the source filter isn't returning a proper value

The final fps is in the return line "fps=c.FrameRate*2"

"c" refers to the source video as loaded by LSmash in that script

So if LSMash returned the proper 23.976 fps for the source, it would be 23.976*2 or 47.952. If it "thought" the video was 30.0 fps, you would get 30.0*2 or 60.0fps

So if you wanted output 120.0 FPS for the script as you have it now, you would just change the last line to

Return jm_fps(c,fps=120, blkSize=BLKSIZE,dct=DCT)

But again , the interpolation also partially depends on source framerate (or what avisynth "thinks" the source framerate is). If it's not loading correctly , you can get off results

So it should be interpolating my videos to (almost) 48 fps? I have checking using the Info() script, and it is running at 47.9520. The youtube video is running at 59.9401 fps (60000/1001), if I want to interpolate to that framerate do I just use:
Return jm_fps(c,fps=59.9401, blkSize=BLKSIZE,dct=DCT)

manolito

26th July 2018, 19:35

@ poisondeathray

That manolito version is a variation on one he posted years ago (by you guessed it, manolito)

Actually this is just a year and a half old. It started when I stumbled over this post:
https://forum.doom9.org/showthread.php?p=1788725#post1788725

This script did beat everything I had tried before, so I stuck with it and advertised it a little bit.

As mentioned earlier, only the very last line is changed (highlighted in red) ; to blend=true, and the masking disabled (rest of line commented out)

Are you sure that you are disabling masking by commenting out the "ml" and "mask" parameters? According to the documentation the defaults are 2 for "mask" and 100 for "ml", so commenting out these params will just change "ml" from 200 to 100.

Cheers
manolito

johnmeyer

26th July 2018, 19:53

Yes, apparently "jm" refers to the settings I posted in that thread manolito linked to. For the video that was the topic of that post, my script did indeed produce fewer artifacts than other scripts and as a result, in subsequent threads, it became somewhat of a benchmark. I was flattered but I can assure you that I did nothing more than spend a few hours "twiddling the knobs" in all of the MVTools2 functions (MAnalyze, MFlowFPS, etc.) until I got the best result for that video.

I've been doing this for a long time, having first used the old MotionPerfect to create slow motion more than fifteen years ago, and since having used just about every motion estimation program around. My conclusion from this experience? The technology simply can't handle some situations. I listed a few of these earlier in this thread.

The only solution is to look for ways to detect when the ME is failing, and use some other method for that frame, or via masking, for part of that frame. MysteryX spent a huge amount of time tackling this problem and made some progress, but IMHO (through no fault of his) his script still doesn't yet work in 99% of all scenes, or anything close to that.

I've already stated why this particular animated video is going to be particularly difficult to smooth out without making it look worse. My mantra when doing video restoration is similar to the doctor's mantra: "first of all, do no harm." If the cure is worse than the disease, then you should just forget about it. Which, of course, has been my advice on this one from the beginning.

bradwiggo

26th July 2018, 19:55

Yes, apparently "jm" refers to the settings I posted in that thread manolito linked to. For the video that was the topic of that post, my script did indeed produce fewer artifacts than other scripts and as a result, in subsequent threads, it became somewhat of a benchmark. I was flattered but I can assure you that I did nothing more than spend a few hours "twiddling the knobs" in all of the MVTools2 functions (MAnalyze, MFlowFPS, etc.) until I got the best result for that video.

I've been doing this for a long time, having first used the old MotionPerfect to create slow motion more than fifteen years ago, and since having used just about every motion estimation program around. The technology simply can't handle some situations (I already listed a few of these earlier in this thread). The only solution is to look for ways to detect when the ME is failing, and use some other method for that frame, or via masking, for part of that frame. MysteryX spent a huge amount of time tackling this problem and made some progress, but IMHO (through no fault of his) still didn't quite create something that will work in 99% of all scenes.

I've already stated why this particular animated video is going to be particularly difficult to smooth out without making it look worse. My mantra when doing video restoration is similar to the doctor's mantra: "first of all, do no harm." If the cure is worse than the disease, then you should just forget about it. Which, of course, has been my advice on this one from the beginning.

I don't think it looks that bad, the youtube video looks quite good, not too many artifacts. As I said in one of my other comments, I am not necessarily trying or expecting to get the whole film to look as good as that video, I am just trying to reproduce that video.

poisondeathray

26th July 2018, 20:06

So it should be interpolating my videos to (almost) 48 fps? I have checking using the Info() script, and it is running at 47.9520. The youtube video is running at 59.9401 fps (60000/1001), if I want to interpolate to that framerate do I just use:
Return jm_fps(c,fps=59.9401, blkSize=BLKSIZE,dct=DCT)

I think it was suggested to use 2x earlier just to examine the effect of frame doubling (every second frame synthesized). Yes you would use 59.94 (the exact rate should be 60000/1001, but it's not going to make a difference here; technically 59.94 or 59.9401 are approximations. 60000/1001 is exact)

Are you sure that you are disabling masking by commenting out the "ml" and "mask" parameters? According to the documentation the defaults are 2 for "mask" and 100 for "ml", so commenting out these params will just change "ml" from 200 to 100.

You're right - it's just the default settings for mask, not disabled

bradwiggo

26th July 2018, 20:38

I think it was suggested to use 2x earlier just to examine the effect of frame doubling (every second frame synthesized). Yes you would use 59.94 (the exact rate should be 60000/1001, but it's not going to make a difference here; technically 59.94 or 59.9401 are approximations. 60000/1001 is exact)

You're right - it's just the default settings for mask, not disabled

Do I need to disable mask?

If I wanted to reproduce the youtube video, should I use the fps = 59.94.

Also, can I write is as 60000/1001 in order to be as precise as possible?

poisondeathray

26th July 2018, 21:33

bradwiggo

27th July 2018, 14:09

I already said the difference between those framerates are not going to make a difference here. If you do the math, over the course of a 3 hour movie , the difference between 59.94 vs 59.9401 vs 60000/1001 is not even 1 frame. So you will end up getting the same results either way

There are only minor differences if you use that script. Some of the difference might be from multi generation compression artifacts too. The only major difference is blend=true for the scene change. You can preview the results and adjust the settings, eg. blocksize etc... Some frames will be better some worse. You don't have to encode anything. Preview things before you waste time encoding the whole thing. What I'm saying is you can answer these questions easily yourself . Just open it up in avspmod, preview, change settings, rinse , repeat

Or maybe you want to adjust the settings per scene.

Or if you want better results, then do some manual masking along with layers and different settings

I had changed the CRF settings a while ago in meGUI, I has the crf at 6 and the preset set to slower. I noticed this today and changed it back to 9.5 and medium, and then interpolated a small (2 minute long) video. The interpolation approximately doubled the file size. I then started to interpolate a larger (1 hour) file, and megui was reporting that the output file would be almost 4 times the size of the original. Why is this the case, as I am using the exact same settings for each of those two?

Also, as seen as I am interpolating 24fps to 60fps, does that mean the file size of the output should be at least 2.5 times that of the input to stop quality loss, or does it not necessarily work like that?

poisondeathray

27th July 2018, 15:13

I had changed the CRF settings a while ago in meGUI, I has the crf at 6 and the preset set to slower. I noticed this today and changed it back to 9.5 and medium, and then interpolated a small (2 minute long) video. The interpolation approximately doubled the file size. I then started to interpolate a larger (1 hour) file, and megui was reporting that the output file would be almost 4 times the size of the original. Why is this the case, as I am using the exact same settings for each of those two?

Also, as seen as I am interpolating 24fps to 60fps, does that mean the file size of the output should be at least 2.5 times that of the input to stop quality loss, or does it not necessarily work like that?

CRF doesn't necessarily work like that, and strictly speaking it's not a measure of "quality" ; it's just a rate control method

2.5x the number of frames generated by interpolation usually does not result in 2.5x the bitrate at a given CRF (everything else the same) ; because a) the relationship is not linear and b) the interpolated frames are usually more blurry and lower quality. When you do get much higher bitrate at a CRF value, it's usually indirect evidence that your have more interpolation artifacts than normal. Ugly edge morphing artifacts tend to "eat up" more bitrate, than a clean smooth interpolation. ie. The less differences between frames, the less bitrate at a given CRF value; roughly speaking - it's the differences between frames that are stored. The more differences, the more bitrate

bradwiggo

27th July 2018, 15:16

CRF doesn't necessarily work like that, and strictly speaking it's not a measure of "quality" ; it's just a rate control method

2.5x the number of frames generated by interpolation usually does not result in 2.5x the bitrate at a given CRF (everything else the same) ; because a) the relationship is not linear and b) the interpolated frames are usually more blurry and lower quality. When you do get much higher bitrate at a CRF value, it's usually indirect evidence that your have more interpolation artifacts than normal. Ugly edge morphing artifacts tend to "eat up" more bitrate, than a clean smooth interpolation

So my interpolated video will have loads of artefacts most likely? How should I fix this, as it has never done this before?

poisondeathray

27th July 2018, 15:27

So my interpolated video will have loads of artefacts most likely? How should I fix this, as it has never done this before?

It's just an educated guess. A clean interpolation will result in a much lower ratio. There can be other explanations too, maybe you messed up the settings, or it's not passing the correct commandline etc...

Take a look at the results first. Look frame by frame. It's the inbetween frames that have problems like edge morphing artifacts. A character might have 3 legs when walking , 3 arms when waving. (its the inbetween frame that isn't calculated cleanly). Because there are large differences, the bitrate will shoot up at a given CRF. But a clean interpolation where everything looks "normal" will have much lower bitrate

It's a matter of perspective or opinion; "is the glass half full or half empty" . For example you though the YT video was good. Many people here did not like the artifacts . Some people would rather have a duplicate frame or blend or other options instead of the ugly artifacts. It's a matter of opinion or what type of scenario you're working with

I think this was already discussed in some earlier posts - With current technology you can't fix these problems easily. They require a lot of manual work, masking, motion tracking, guiding motion estimation, compositing layers , patching areas. It's not like photoshoping each frame, but there is a steep learning curve on composting techniques tips and tricks. And it's tedious and takes a lot of time.

The MysteryX FrameRateConverter script is probably the closest thing that attempts to mask out problems or at least have options to address some of the issues automatically. Realistically it's far from perfect too.

There was a link referring to Nvidia AI - that looks very promising if you can believe demos

bradwiggo

27th July 2018, 15:35

It's just an educated guess. A clean interpolation will result in a much lower ratio. There can be other explanations too, maybe you messed up the settings, or it's not passing the correct commandline etc...

Take a look at the results first. Look frame by frame. It's the inbetween frames that have problems like edge morphing artifacts. A character might have 3 legs when walking , 3 arms when waving. (its the inbetween frame that isn't calculated cleanly). Because there are large differences, the bitrate will shoot up at a given CRF. But a clean interpolation where everything looks "normal" will have much lower bitrate

It's a matter of perspective or opinion; "is the glass half full or half empty" . For example you though the YT video was good. Many people here did not like the artifacts . Some people would rather have a duplicate frame or blend or other options instead of the ugly artifacts. It's a matter of opinion or what type of scenario you're working with

I think this was already discussed in some earlier posts - With current technology you can't fix these problems easily. They require a lot of manual work, masking, motion tracking, guiding motion estimation, compositing layers , patching areas. It's not like photoshoping each frame, but there is a steep learning curve on composting techniques tips and tricks. And it's tedious and takes a lot of time.

The MysteryX FrameRateConverter script is probably the closest thing that attempts to mask out problems or at least have options to address some of the issues automatically. Realistically it's far from perfect too.

There was a link referring to Nvidia AI - that looks very promising if you can believe demos

If the file size is bigger, does that suggest there may be more artefacts. In that case there will be a huge amount of artefacts if it is twice as big as it should be, if indeed that is what is causing the increase in file size.

I am not sure hw to preview videos. You mentioned avspmod, however, that does work for me, it gives me an error saying that masktools2.dll is not an avisynth 2.5 plugin, as I use the avisynth that is included with megui normally.

poisondeathray

27th July 2018, 15:42

If the file size is bigger, does that suggest there may be more artefacts. In that case there will be a huge amount of artefacts if it is twice as big as it should be, if indeed that is what is causing the increase in file size.

I am not sure hw to preview videos. You mentioned avspmod, however, that does work for me, it gives me an error saying that masktools2.dll is not an avisynth 2.5 plugin, as I use the avisynth that is included with megui normally.

Make sure you do the 1x test first. That's your baseline to compare to. Re-encode the video using the same settings, not interpolated. Don't compare to the original video file size. A low CRF can make the original video much larger too.

You need to fix your avisynth plugins and versions. Welcome to "dll hell", everyone goes through it.

Another way to compare is to use vdub2 . Or many media players can open avs scripts and go frame by frame.

bradwiggo

27th July 2018, 15:55

Make sure you do the 1x test first. That's your baseline to compare to. Re-encode the video using the same settings, not interpolated. Don't compare to the original video file size. A low CRF can make the original video much larger too.

You need to fix your avisynth plugins and versions. Welcome to "dll hell", everyone goes through it.

Another way to compare is to use vdub2 . Or many media players can open avs scripts and go frame by frame.

So I should use that script but set it to the original framerate?

How do I tell avspmod to use the megui avisynth?

poisondeathray

27th July 2018, 16:14

So I should use that script but set it to the original framerate?

Just load the video only. For example "test1.mkv" or whatever your test section was

The script should just be 1 line only, just the source filter that loads the video

eg. something like this

FFVideoSource("test1.mkv")

How do I tell avspmod to use the megui avisynth?

Not sure, I think some versions of megui have a "portable" avisynth version separate from the installed version. I don't really use it

But normally you would save the script from megui

e.g "myscript.avs"

You can open that script in avspmod, or vdub2 or something like mpchc to preview it.

But if you have a dll error, this means your installed avisynth plugins folder need to be cleaned up. You need to find proper versions (ie. "dll hell" :) )

bradwiggo

27th July 2018, 16:39

Just load the video only. For example "test1.mkv" or whatever your test section was

The script should just be 1 line only, just the source filter that loads the video

eg. something like this

FFVideoSource("test1.mkv")

Not sure, I think some versions of megui have a "portable" avisynth version separate from the installed version. I don't really use it

But normally you would save the script from megui

e.g "myscript.avs"

You can open that script in avspmod, or vdub2 or something like mpchc to preview it.

But if you have a dll error, this means your installed avisynth plugins folder need to be cleaned up. You need to find proper versions (ie. "dll hell" :) )

I don't really know what you mean by proper versions. I know one of my scripts just references a dll that is in a separate folder somewhere as I had to download a new one, but it works fine in megui. I think the best way to solve the problem is probably find out the version of avisynth that is included with megui and then download the separate version of that for avspmod to use. How do you tell avspmod to use a different avisynth install?

MeGUI is using avisynth+ 0.1

poisondeathray

27th July 2018, 18:11

I don't really know what you mean by proper versions.

The correct matching version. There are many different versions of dll's. Sometimes you might be using the wrong x86 vs x64, or one compiled for a different avisynth version e.g. 2.6 vs 2.5 or avsynth+ , sometimes there are specific versions that work with specific scripts

How do you tell avspmod to use a different avisynth install?

I don't think you can - it just uses the default installed version. If you use x86 version, it will initialize avisynth x86. If you use the x64 it will initialize the x64 version

MeGUI is using avisynth+ 0.1

I would say avisynth+ is stable enough now for general use. Almost all the commonly used plugins have working versions

bradwiggo

27th July 2018, 18:14

The correct matching version. There are many different versions of dll's. Sometimes you might be using the wrong x86 vs x64, or one compiled for a different avisynth version e.g. 2.6 vs 2.5 or avsynth+ , sometimes there are specific versions that work with specific scripts

I don't think you can - it just uses the default installed version. If you use x86 version, it will initialize avisynth x86. If you use the x64 it will initialize the x64 version

I would say avisynth+ is stable enough now for general use. Almost all the commonly used plugins have working versions

I will just have to use the megui preview then, the problem with that one is it doesn't play it properly, it plays it slowly.

poisondeathray

27th July 2018, 18:19

I will just have to use the megui preview then, the problem with that one is it doesn't play it properly, it plays it slowly.

Some scripts are too slow for realtime playback, or some hardware is insufficient to get realtime playback

You can use avsmeter to get diagnostics on how fast that script is processed on your computer setup under optimal conditions . If the output is supposed to be 59.94, and you can't meet that minimum, you will not get realtime playback on your system

Also, I don't think megui preview is meant to "playback" , as in it's probably not optimized for smooth playing like a player would be. I haven't used it for a few years but at least that's how it used to be; it's just meant to preview some frames here and there. But maybe something has changed

manolito

28th July 2018, 02:24

bradwiggo

28th July 2018, 16:38

FWIW I downloaded the source clip and converted it to double fps using 3 different settings:

The first conversion was plain jm_fps using default settings.
The second one used mx_fps (a mod of FrameRateConverter by MysteryX which adds artifact masking). I also used "DCT=1".
The third one also used mx_fps, but I also changed the default block size of 16 to 32.

Download here:
https://www.zeta-uploader.com/40449651

A few remarks about the source:
It seems to come from the Apple world, I had some problems to convert the VFR video to CFR without loosing audio sync. All my AviSynth source filters failed, only FFmpeg could do it. I needed to convert the HD source to SD, my slow computer does not like HD...

All the conversion results cannot handle the snow flakes, but at least to my eyes they all look pretty good. This is probably all you can get from MVTools based interpolators.

Cheers
manolito

It seems to come from the Apple world

You are correct, it is from iTunes. How were you able to tell, are the VFR issues common with apple?

Do you need to convert it to CFR before interpolation?

I wouldn't worry too much about the snowflakes, I remember people in the youtube comments of the video saying that the snow still looked 24fps, you don't really notice as your eyes focus past the snow on the scene behind. The snow effects normally get interpolated, but the snow that is just falling to the ground normally doesn't.

Would a blu ray movie interpolate better than an iTunes movie? Also does the resolution matter. I am using the 720p version, would the 1080p version interpolate better ?

Sparktank

28th July 2018, 17:09

bradwiggo

28th July 2018, 17:15

That would explain all the banding in that scene.
The bluray isn't nearly as banded.

Going CFR is better.

Bluray sources are CFR from the start, so that'll help a lot.
And they should have better quality than the iTunes movies. More clarity and details.

Results can vary if you use 1080p resolution (crop black bars first) compared to the 720p.
You can encode the bluray at 1080p and then do one at 720p and watch them both.
Some sources look better at 720p.

I don't have a blu ray to test, I was just wondering if a blu ray reader for my computer may be something I should get, as I am currently looking for a new dvd drive, so I could kill two birds with one stone. Where in the scene was the banding, I can't say I noticed any.

Did the youtube video look like it was from a blu ray, or is it too hard to tell?

johnmeyer

28th July 2018, 20:47

Blu-ray will make zero difference because the issues you have been trying to solve for the last two months are caused by frame rate, not resolution.

bradwiggo

28th July 2018, 22:29

Blu-ray will make zero difference because the issues you have been trying to solve for the last two months are caused by frame rate, not resolution.

If blu rays have a constant frame rate though isn't that better?

Sparktank

28th July 2018, 22:40

The skies looked banded the most.

If you get a bluray drive, you'd need to use something MakeMKV which is currently free while in BETA. It's been in beta for years, so probably a few years before it's out of beta.
MakeMKV will decrypt and remux to MKV for you.

But VFR to CFR isn't that big a deal. It's just another step.

For SVP, I'll downscale my blurays to 720p for watching and let MadVR upscale back to 1080p. It's less resource heavy for live playback if you downscale the resolution.

You'll still get issues from interpolation. The artifacts, etc.
That's the limitation we've hit with mvtools.

manolito

29th July 2018, 00:31

You are correct, it is from iTunes. How were you able to tell, are the VFR issues common with apple?

Do you need to convert it to CFR before interpolation?

I have no idea if those VFR issues are common with Apple, I hate Apple and don't use any of their stuff... :devil:

The MP4 seems to be peculiar, though. MediaInfo says this about the frame rate:
Frame rate mode : variabel
Frame rate : 23,976 FPS
Minimum frame rate : 23,077 FPS
Maximum frame rate : 24,000 FPS

I need to use CFR for my videos because AviSynth is always involved in my conversions, and AviSynth does not understand VFR. Converting the source to CFR is usually done by the source filter (DSS2Mod or FFmpegSource in my case), but for this source I always end up with messed up audio sync.

I also noticed that when I play back the file from within One-Drive with the latest Chrome browser the audio sync is also broken.

I did finally discover a method to fix this issue. First I needed to repack the source to MKV with mkvmerge. This already changed the fps to CFR. When I then used FFmegSource as the source filter for video and audio I got perfect audio sync. DSS2Mod did not work.

Cheers
manolito

johnmeyer

29th July 2018, 06:23

If blu rays have a constant frame rate though isn't that better?Your clip did not show any variable frame rate, even though, before I looked at the clip and only knew it was animation, I thought there might be frame repeats.

So, just to be perfectly clear: the video clip you posted IS constant frame rate.

When I say the issue is frame rate, I mean that the reason you think it is not smooth is that it is only 24 frames per second. I already explained in detail why 24 fps often appears jerky, and that you need to get to 60 fps, either progressive or via interlacing (29.97 interlaced) in order for the eye to perceive motion as being smooth.

The only way to get smooth motion without making the video fuzzy (which frame blending will do) is via motion estimation which, as has been discussed for two months (first five weeks at Videohelp and the last two weeks here), will never give you a pleasant outcome because you are going to create too many weird artifacts. The slower the initial frame rate, the worse the result, because there is such a big temporal gap between frames that the estimation can't bridge the gap without mistakes. This is why, if you start with 60 fps progressive, you can often get some remarkable super slow motion, without artifacts.

You are already going to lose snowflakes, and believe me, despite what you say, that will get very distracting.

bradwiggo

29th July 2018, 10:14

Your clip did not show any variable frame rate, even though, before I looked at the clip and only knew it was animation, I thought there might be frame repeats.

So, just to be perfectly clear: the video clip you posted IS constant frame rate.

When I say the issue is frame rate, I mean that the reason you think it is not smooth is that it is only 24 frames per second. I already explained in detail why 24 fps often appears jerky, and that you need to get to 60 fps, either progressive or via interlacing (29.97 interlaced) in order for the eye to perceive motion as being smooth.

The only way to get smooth motion without making the video fuzzy (which frame blending will do) is via motion estimation which, as has been discussed for two months (first five weeks at Videohelp and the last two weeks here), will never give you a pleasant outcome because you are going to create too many weird artifacts. The slower the initial frame rate, the worse the result, because there is such a big temporal gap between frames that the estimation can't bridge the gap without mistakes. This is why, if you start with 60 fps progressive, you can often get some remarkable super slow motion, without artifacts.

You are already going to lose snowflakes, and believe me, despite what you say, that will get very distracting.

The comment above yours from manolito mentions some weird info in media info, what causes that if not VFR?