I need some help with interpolation videos, lots of detail in post

[bradwiggo]

Originally Posted by poisondeathray

It's just an educated guess. A clean interpolation will result in a much lower ratio. There can be other explanations too, maybe you messed up the settings, or it's not passing the correct commandline etc...

Take a look at the results first. Look frame by frame. It's the inbetween frames that have problems like edge morphing artifacts. A character might have 3 legs when walking , 3 arms when waving. (its the inbetween frame that isn't calculated cleanly). Because there are large differences, the bitrate will shoot up at a given CRF. But a clean interpolation where everything looks "normal" will have much lower bitrate

It's a matter of perspective or opinion; "is the glass half full or half empty" . For example you though the YT video was good. Many people here did not like the artifacts . Some people would rather have a duplicate frame or blend or other options instead of the ugly artifacts. It's a matter of opinion or what type of scenario you're working with

I think this was already discussed in some earlier posts - With current technology you can't fix these problems easily. They require a lot of manual work, masking, motion tracking, guiding motion estimation, compositing layers , patching areas. It's not like photoshoping each frame, but there is a steep learning curve on composting techniques tips and tricks. And it's tedious and takes a lot of time.

The MysteryX FrameRateConverter script is probably the closest thing that attempts to mask out problems or at least have options to address some of the issues automatically. Realistically it's far from perfect too.

There was a link referring to Nvidia AI - that looks very promising if you can believe demos

If the file size is bigger, does that suggest there may be more artefacts. In that case there will be a huge amount of artefacts if it is twice as big as it should be, if indeed that is what is causing the increase in file size.

I am not sure hw to preview videos. You mentioned avspmod, however, that does work for me, it gives me an error saying that masktools2.dll is not an avisynth 2.5 plugin, as I use the avisynth that is included with megui normally.

[poisondeathray]

Originally Posted by bradwiggo

If the file size is bigger, does that suggest there may be more artefacts. In that case there will be a huge amount of artefacts if it is twice as big as it should be, if indeed that is what is causing the increase in file size.

I am not sure hw to preview videos. You mentioned avspmod, however, that does work for me, it gives me an error saying that masktools2.dll is not an avisynth 2.5 plugin, as I use the avisynth that is included with megui normally.

Make sure you do the 1x test first. That's your baseline to compare to. Re-encode the video using the same settings, not interpolated. Don't compare to the original video file size. A low CRF can make the original video much larger too.

You need to fix your avisynth plugins and versions. Welcome to "dll hell", everyone goes through it.

Another way to compare is to use vdub2 . Or many media players can open avs scripts and go frame by frame.

[bradwiggo]

Originally Posted by poisondeathray

Make sure you do the 1x test first. That's your baseline to compare to. Re-encode the video using the same settings, not interpolated. Don't compare to the original video file size. A low CRF can make the original video much larger too.

You need to fix your avisynth plugins and versions. Welcome to "dll hell", everyone goes through it.

Another way to compare is to use vdub2 . Or many media players can open avs scripts and go frame by frame.

So I should use that script but set it to the original framerate?

How do I tell avspmod to use the megui avisynth?

[poisondeathray]

Originally Posted by bradwiggo

So I should use that script but set it to the original framerate?

Just load the video only. For example "test1.mkv" or whatever your test section was

The script should just be 1 line only, just the source filter that loads the video

eg. something like this

FFVideoSource("test1.mkv")

How do I tell avspmod to use the megui avisynth?

Not sure, I think some versions of megui have a "portable" avisynth version separate from the installed version. I don't really use it


But normally you would save the script from megui

e.g "myscript.avs"

You can open that script in avspmod, or vdub2 or something like mpchc to preview it.

But if you have a dll error, this means your installed avisynth plugins folder need to be cleaned up. You need to find proper versions (ie. "dll hell" )

[bradwiggo]

Originally Posted by poisondeathray

Just load the video only. For example "test1.mkv" or whatever your test section was

The script should just be 1 line only, just the source filter that loads the video

eg. something like this

FFVideoSource("test1.mkv")

Not sure, I think some versions of megui have a "portable" avisynth version separate from the installed version. I don't really use it


But normally you would save the script from megui

e.g "myscript.avs"

You can open that script in avspmod, or vdub2 or something like mpchc to preview it.

But if you have a dll error, this means your installed avisynth plugins folder need to be cleaned up. You need to find proper versions (ie. "dll hell" )

I don't really know what you mean by proper versions. I know one of my scripts just references a dll that is in a separate folder somewhere as I had to download a new one, but it works fine in megui. I think the best way to solve the problem is probably find out the version of avisynth that is included with megui and then download the separate version of that for avspmod to use. How do you tell avspmod to use a different avisynth install?

MeGUI is using avisynth+ 0.1

[poisondeathray]

Originally Posted by bradwiggo

I don't really know what you mean by proper versions.

The correct matching version. There are many different versions of dll's. Sometimes you might be using the wrong x86 vs x64, or one compiled for a different avisynth version e.g. 2.6 vs 2.5 or avsynth+ , sometimes there are specific versions that work with specific scripts

How do you tell avspmod to use a different avisynth install?

I don't think you can - it just uses the default installed version. If you use x86 version, it will initialize avisynth x86. If you use the x64 it will initialize the x64 version

MeGUI is using avisynth+ 0.1

I would say avisynth+ is stable enough now for general use. Almost all the commonly used plugins have working versions

[bradwiggo]

Originally Posted by poisondeathray

The correct matching version. There are many different versions of dll's. Sometimes you might be using the wrong x86 vs x64, or one compiled for a different avisynth version e.g. 2.6 vs 2.5 or avsynth+ , sometimes there are specific versions that work with specific scripts


I don't think you can - it just uses the default installed version. If you use x86 version, it will initialize avisynth x86. If you use the x64 it will initialize the x64 version



I would say avisynth+ is stable enough now for general use. Almost all the commonly used plugins have working versions

I will just have to use the megui preview then, the problem with that one is it doesn't play it properly, it plays it slowly.

[poisondeathray]

Originally Posted by bradwiggo

I will just have to use the megui preview then, the problem with that one is it doesn't play it properly, it plays it slowly.

Some scripts are too slow for realtime playback, or some hardware is insufficient to get realtime playback

You can use avsmeter to get diagnostics on how fast that script is processed on your computer setup under optimal conditions . If the output is supposed to be 59.94, and you can't meet that minimum, you will not get realtime playback on your system

Also, I don't think megui preview is meant to "playback" , as in it's probably not optimized for smooth playing like a player would be. I haven't used it for a few years but at least that's how it used to be; it's just meant to preview some frames here and there. But maybe something has changed

[manolito]

FWIW I downloaded the source clip and converted it to double fps using 3 different settings:

The first conversion was plain jm_fps using default settings.
The second one used mx_fps (a mod of FrameRateConverter by MysteryX which adds artifact masking). I also used "DCT=1".
The third one also used mx_fps, but I also changed the default block size of 16 to 32.

Download here:
https://www.zeta-uploader.com/40449651

A few remarks about the source:
It seems to come from the Apple world, I had some problems to convert the VFR video to CFR without loosing audio sync. All my AviSynth source filters failed, only FFmpeg could do it. I needed to convert the HD source to SD, my slow computer does not like HD...

All the conversion results cannot handle the snow flakes, but at least to my eyes they all look pretty good. This is probably all you can get from MVTools based interpolators.


Cheers
manolito

[bradwiggo]

Originally Posted by manolito

FWIW I downloaded the source clip and converted it to double fps using 3 different settings:

The first conversion was plain jm_fps using default settings.
The second one used mx_fps (a mod of FrameRateConverter by MysteryX which adds artifact masking). I also used "DCT=1".
The third one also used mx_fps, but I also changed the default block size of 16 to 32.

Download here:
https://www.zeta-uploader.com/40449651

A few remarks about the source:
It seems to come from the Apple world, I had some problems to convert the VFR video to CFR without loosing audio sync. All my AviSynth source filters failed, only FFmpeg could do it. I needed to convert the HD source to SD, my slow computer does not like HD...

All the conversion results cannot handle the snow flakes, but at least to my eyes they all look pretty good. This is probably all you can get from MVTools based interpolators.


Cheers
manolito

It seems to come from the Apple world

You are correct, it is from iTunes. How were you able to tell, are the VFR issues common with apple?

Do you need to convert it to CFR before interpolation?

I wouldn't worry too much about the snowflakes, I remember people in the youtube comments of the video saying that the snow still looked 24fps, you don't really notice as your eyes focus past the snow on the scene behind. The snow effects normally get interpolated, but the snow that is just falling to the ground normally doesn't.

Would a blu ray movie interpolate better than an iTunes movie? Also does the resolution matter. I am using the 720p version, would the 1080p version interpolate better ?

[Sparktank]

That would explain all the banding in that scene.
The bluray isn't nearly as banded.

Going CFR is better.

Bluray sources are CFR from the start, so that'll help a lot.
And they should have better quality than the iTunes movies. More clarity and details.

Results can vary if you use 1080p resolution (crop black bars first) compared to the 720p.
You can encode the bluray at 1080p and then do one at 720p and watch them both.
Some sources look better at 720p.

[bradwiggo]

Originally Posted by Sparktank

That would explain all the banding in that scene.
The bluray isn't nearly as banded.

Going CFR is better.

Bluray sources are CFR from the start, so that'll help a lot.
And they should have better quality than the iTunes movies. More clarity and details.

Results can vary if you use 1080p resolution (crop black bars first) compared to the 720p.
You can encode the bluray at 1080p and then do one at 720p and watch them both.
Some sources look better at 720p.

I don't have a blu ray to test, I was just wondering if a blu ray reader for my computer may be something I should get, as I am currently looking for a new dvd drive, so I could kill two birds with one stone. Where in the scene was the banding, I can't say I noticed any.

Did the youtube video look like it was from a blu ray, or is it too hard to tell?

[johnmeyer]

Blu-ray will make zero difference because the issues you have been trying to solve for the last two months are caused by frame rate, not resolution.

[bradwiggo]

Originally Posted by johnmeyer

Blu-ray will make zero difference because the issues you have been trying to solve for the last two months are caused by frame rate, not resolution.

If blu rays have a constant frame rate though isn't that better?

[Sparktank]

The skies looked banded the most.

If you get a bluray drive, you'd need to use something MakeMKV which is currently free while in BETA. It's been in beta for years, so probably a few years before it's out of beta.
MakeMKV will decrypt and remux to MKV for you.

But VFR to CFR isn't that big a deal. It's just another step.

For SVP, I'll downscale my blurays to 720p for watching and let MadVR upscale back to 1080p. It's less resource heavy for live playback if you downscale the resolution.

You'll still get issues from interpolation. The artifacts, etc.
That's the limitation we've hit with mvtools.

[manolito]

Originally Posted by bradwiggo

You are correct, it is from iTunes. How were you able to tell, are the VFR issues common with apple?

Do you need to convert it to CFR before interpolation?

I have no idea if those VFR issues are common with Apple, I hate Apple and don't use any of their stuff...

The MP4 seems to be peculiar, though. MediaInfo says this about the frame rate:

Frame rate mode : variabel
Frame rate : 23,976 FPS
Minimum frame rate : 23,077 FPS
Maximum frame rate : 24,000 FPS

I need to use CFR for my videos because AviSynth is always involved in my conversions, and AviSynth does not understand VFR. Converting the source to CFR is usually done by the source filter (DSS2Mod or FFmpegSource in my case), but for this source I always end up with messed up audio sync.

I also noticed that when I play back the file from within One-Drive with the latest Chrome browser the audio sync is also broken.

I did finally discover a method to fix this issue. First I needed to repack the source to MKV with mkvmerge. This already changed the fps to CFR. When I then used FFmegSource as the source filter for video and audio I got perfect audio sync. DSS2Mod did not work.


Cheers
manolito

[johnmeyer]

Originally Posted by bradwiggo

If blu rays have a constant frame rate though isn't that better?

Your clip did not show any variable frame rate, even though, before I looked at the clip and only knew it was animation, I thought there might be frame repeats.

So, just to be perfectly clear: the video clip you posted IS constant frame rate.

When I say the issue is frame rate, I mean that the reason you think it is not smooth is that it is only 24 frames per second. I already explained in detail why 24 fps often appears jerky, and that you need to get to 60 fps, either progressive or via interlacing (29.97 interlaced) in order for the eye to perceive motion as being smooth.

The only way to get smooth motion without making the video fuzzy (which frame blending will do) is via motion estimation which, as has been discussed for two months (first five weeks at Videohelp and the last two weeks here), will never give you a pleasant outcome because you are going to create too many weird artifacts. The slower the initial frame rate, the worse the result, because there is such a big temporal gap between frames that the estimation can't bridge the gap without mistakes. This is why, if you start with 60 fps progressive, you can often get some remarkable super slow motion, without artifacts.

You are already going to lose snowflakes, and believe me, despite what you say, that will get very distracting.

[bradwiggo]

Originally Posted by johnmeyer

Your clip did not show any variable frame rate, even though, before I looked at the clip and only knew it was animation, I thought there might be frame repeats.

So, just to be perfectly clear: the video clip you posted IS constant frame rate.

When I say the issue is frame rate, I mean that the reason you think it is not smooth is that it is only 24 frames per second. I already explained in detail why 24 fps often appears jerky, and that you need to get to 60 fps, either progressive or via interlacing (29.97 interlaced) in order for the eye to perceive motion as being smooth.

The only way to get smooth motion without making the video fuzzy (which frame blending will do) is via motion estimation which, as has been discussed for two months (first five weeks at Videohelp and the last two weeks here), will never give you a pleasant outcome because you are going to create too many weird artifacts. The slower the initial frame rate, the worse the result, because there is such a big temporal gap between frames that the estimation can't bridge the gap without mistakes. This is why, if you start with 60 fps progressive, you can often get some remarkable super slow motion, without artifacts.

You are already going to lose snowflakes, and believe me, despite what you say, that will get very distracting.

The comment above yours from manolito mentions some weird info in media info, what causes that if not VFR?

[Sparktank]

Originally Posted by bradwiggo

The comment above yours from manolito mentions some weird info in media info, what causes that if not VFR?

I only ever see that when dealing with anything Apple.
If you remuxed it to mkv with FFMPEG, you probably won't see that in the MediaInfo readout.

Even with some tools if you remux to MP4 again with forcing Contstant Frame Rate, you still see that junk.

It's not a big thing to worry about.
You can just add AssumeFPS(24000, 1001) when importing and it'll work normally.

[johnmeyer]

Originally Posted by bradwiggo

The comment above yours from manolito mentions some weird info in media info, what causes that if not VFR?

Mediainfo often reports strange numbers. You can find all sorts of posts about this. As already advised, ignore the numbers for this video.