SVP-like frame interpolation?

[MysteryX]

Here is the parade clip
Source (https://mega.nz/#!nJZHEKab!H7TnhbXDzXV8lBxJs6th_hkrZQObgXFuvGIgb0kpxo4)


file="Motion Estimation Torture Clip.avi"
LWLibavVideoSource(file, cache=False)
ConvertToYV12()
FrameRateConverter(60, blksize=8, preset="slower")


BlkSize=8 (https://mega.nz/#!rB5EnLSA!gV_FthFn0EycbrO0471ajYzXavT-IVT1vnkHd2ez2Uc)

BlkSize=12 (https://mega.nz/#!SFoBUaBa!J8WIG81ZKvZ_M2DmH9yn0cd40x0Q5i64wjSHxsSwpKE)

Even though this doesn't yet work right with MT, running with a single thread already has internal multi-threading as it runs at 40%-50% CPU on my 8 cores. BlkSize=8 encoded at 6.45fps while BlkSize=12 encoded at 2.18fps... huge difference!

[manolito]

Are you serious? Did you even watch your BlkSize=8 conversion?

This one looks horrible. Look at the scene with the car. Didn't you notice that the people in the background are totally butchered?
The lower speed for the BlkSize=12 conversion certainly paid off. And still comparing it to my test conversion #3 I think mine looks better, even using Preset = "normal"...


Cheers
manolito

[manolito]

Alright, I decided that for me this plugin will be static from now on...

The issues I have are most likely caused by my older plugin versions, but that's fine because I think that the latest improvements which cause my issues are not too important. Generally I believe that development for this tool has reached its peak, it won't get much better than it is now.

My latest and probably final modded version can be found in this post:
https://forum.doom9.org/showthread.php?p=1805050#post1805050

I modified the user interface so this is a normal fps converter now. The only exposed parameters are fps, Preset and BlkSize.

I removed the StripeMask and the ConditionalFilterMT functions so the FrameRateConverter.dll plugin is no longer needed. Also removed all debug related code from the script.

Also Preset = "slower" was removed, and the CalcDiff routine is no longer there. BlkSizes are limited to 8, 16 and 32 again.


In my tests this version is mostly just as fast as the core jm_fps script. In many cases the output will be almost indistinguishable, but for some sources the artifact removal routines do make a real difference.


Thanks to MysteryX and all the other contributors... :thanks:

Cheers
manolito

[MysteryX]

Keep in mind that if you don't use StripeMask (which isn't an issue unless you have a source with stripes), it weakens the mask and you have to slightly lower the thresholds to make up for it.

And why would you remove the debug option? Before encoding a video, you may want to at least quickly review to make sure it's skipping and blending at the right tresholds.

[manolito]

Keep in mind that if you don't use StripeMask (which isn't an issue unless you have a source with stripes), it weakens the mask and you have to slightly lower the thresholds to make up for it.

You mean all the way down to the values of the latest script version without the stripe mask from 1-May-2017, like this?
MaskTrh = Default(MaskTrh, 100)
SkipTrh = 80
MaskOcc = MaskTrh > 0 ? Default(MaskOcc, 105) : 0
BlendOver = Default(BlendOver, 30)
SkipOver = Default(SkipOver, 60)


And why would you remove the debug option? Before encoding a video, you may want to at least quickly review to make sure it's skipping and blending at the right tresholds.

That's exactly what I do not want. I want the script to work just like ChangeFPS(), with default thresholds which work in the vast majority of cases right out of the box. The least thing I need is the requirement to adjust params for every different source. For a while I even thought about removing "Preset" and "BlkSize" from the interface... :p

I remember having exactly the same discussion with johnmeyer after I started promoting the jm_fps script. He insisted that his parameters would probably give bad results for a lot of sources, so the parameters needed to be tuned for each new source. I strongly disagreed because in my tests his params for the first time did work well for almost all sources I tried it with, so for me this script was a godsend.

And this is what I want to achieve for the FrameRateConverter script, too. I want defaults which work each and every time.


Cheers
manolito

[MysteryX]

Then contribute into improving the defaults. Removing options does nothing to help anyone.

Defaults work well so far, except with the parade with blksize=16 where whole-frame blending starts to happen too often.

[manolito]

Removing options does nothing to help anyone.

It sure does help users who don't have a clue about how this whole artifact removal thingy works. And where else do you have AviSynth plugins which feature a plethora of debug options? Debug code is for developers, release versions usually have the debug options removed.

For users who have all the required expertise, there always is your FrameRateConverter.avsi. My modification is for a different group of users...


Regarding my question from my previous post how much I should reduce the thresholds when not using the stripe mask, I did some test conversions using the parade clip and the str clip (Groucho's Japanese girl with the striped stockings), and using these different thresholds did not make any visual difference.

I also converted the parade clip again using jm_fps, and IMO only the moose scene shows a visual improvement when using FrameRateConverter (uses blending instead of interpolating). The other scenes look just fine with jm_fps, I see no need to use artifact removal for these other scenes.


My goal for FrameRateConverter is to have defaults which make any conversion look at least as good as a jm_fps conversion (looking worse than jm_fps is not acceptable whatsoever), but additionally improve the results of jm_fps for demanding scenes like the moose scene. I can probably achieve this by using rather high thresholds for the masks.


Cheers
manolito

[raffriff42]

So make a wrapper function (https://en.wikipedia.org/wiki/Wrapper_function) that exposes only the arguments you care about. Your preferred interface can coexist with MysteryX's.

[MysteryX]

So make a wrapper function (https://en.wikipedia.org/wiki/Wrapper_function) that exposes only the arguments you care about. Your preferred interface can coexist with MysteryX's.

Even there, a wrapper function wouldn't provide any value. You'd call the wrapper with 3 arguments, while those 3 first arguments are exactly the same as FrameRateConverter, so might as well call FrameRateConverter directly. Unless you want to use different defaults (eg: preset) for your needs, then you could write a wrapper to alter the defaults so you don't need to specify them every time.

As far as I'm concerned, development isn't done here.

Pinterf saw some issues with MvTools2 and MT

Inconsistent output of the same script under mt. These are binary algorithms and not some fuzzy logic with AI intuition and randomness.


Although SVP has been working that way for a while


StripeMask needs to be written in assembly and some bug fixed in it; Pinterf will work on that when he's done with MvTools.

I haven't gotten much feedback on StripeMask yet. Does it work correctly on bad clips? StripeMask normally should only alter the Skip value and not the mask, but in some cases it may slightly alter the mask. Are there cases where it's giving worse output?

Then there's the Diff mode where I also need feedback. Finding any good uses for it besides preset="slower"?

And then with the new block sizes, the defaults will need to be changed. It requires testing a variety of video formats and listing what block sizes work best, and building the defaults from there. And anyone sees anything to gain from non-square block sizes?

For the output, the changes are subtle. It always looks better to leave jm_fps as-is than masking with frame blending, except when there are bad artifacts. It looks best with small and precise masks. As for slower preset, it fixes several details and small artifacts, but it's still only details here and there. It's when you add up all those details properly tweaked that you get a considerably better result.

Slightly improved the Diff mask here (removed blockiness)
https://github.com/mysteryx93/FrameRateConverter/blob/master/FrameRateConverter.avsi

[manolito]

It always looks better to leave jm_fps as-is than masking with frame blending, except when there are bad artifacts.

That's exactly what I think... :devil:
So my motto for this kind of tasks is (taken from the old "Alchemist" thread): Try to compensate first, fall back to blending if compensate does not work.

I had been playing a lot with the old salFPS3 (Mug Funky and Didée), but it only worked in theory, the real results were not so good. See here: https://forum.doom9.org/showthread.php?p=899222#post899222


After doing many more tests I concluded that all the recent additions to FrameRateConverter don't do it for me. So I stripped down my modded version even further. Basically all additions since the 1-May-2017 version had to go.

The code is much simpler and clearer now, and all my tests so far did not show any degraded quality compared to the latest edition. The modded script can be found here:
https://forum.doom9.org/showthread.php?p=1805050#post1805050


Cheers
manolito

[burfadel]

I am only using normal as slow and slower seem overkill. I feel slower does produce the best results though, but the difference is probably much more noticeable in the encode stats than visually. If it weren't so much slower I'd probably be using it. I'm guessing using DCT currently means doing a whole second lot of calculations? If it could be used only on the part of the frame where there is artifacts and only when necessary (a threshold for example), I think normal + this would be the ultimate solution. I realise this isn't supported in the support tools currently. As it stands, slow seems a bit wasteful, and slower very wasteful since a large amount of the calculated stuff is thrown away, and the useful stuff is always around the areas where you intend the information to be used.

[hello_hello]

The code is much simpler and clearer now, and all my tests so far did not show any degraded quality compared to the latest edition. The modded script can be found here:
https://forum.doom9.org/showthread.php?p=1805050#post1805050

I don't use frame interpolation much but I stumbled into this thread and tried the script & it seems quite good, relative to Interframe at least.
I'll confess I changed the function name to xfps though. Life's too short to be typing underscores, and xfps seems kinda cool. ;)

A question though....

What's the intention with the speed presets? The script seems a bit undecided.

## @ Preset - The speed/quality preset [slower|slow|normal|fast]. (default=normal)

P_SLOWER = 0 P_SLOW = 1 P_NORMAL = 2 P_FAST = 3
Pset = Preset == "slow" ? P_SLOW : Preset == "normal" ? P_NORMAL : Preset == "fast" ? P_FAST : -1
Assert(Pset != -1, "mx_fps: 'Preset' must be slow, normal or fast {'" + Preset + "'}")

Should the Assert line be permitting Preset="slower"?

Cheers.

[Sharc]

After doing many more tests I concluded that all the recent additions to FrameRateConverter don't do it for me. So I stripped down my modded version even further. Basically all additions since the 1-May-2017 version had to go.

The code is much simpler and clearer now, and all my tests so far did not show any degraded quality compared to the latest edition. The modded script can be found here:
https://forum.doom9.org/showthread.php?p=1805050#post1805050


Cheers
manolito

Comparison here (default settings, except presets)
http://www.mediafire.com/file/oranq24vuccna10/FRC_comparison.mkv

Edit:
Somewhat better results I got with MaskThr=255 in MysteryX's script:
http://www.mediafire.com/file/4vwdtbc2gutvlw9/FRC_comparison2.mkv

Comments, suggestions for better tweaking?

[MysteryX]

Should the Assert line be permitting Preset="slower"?

Official script allows preset slower. (https://github.com/mysteryx93/FrameRateConverter/blob/master/FrameRateConverter.avsi)

Manolito took the diff feature out. DCT=1 is overkill, but preset="slower" isn't much slower, so I'd rather go with either Normal for HD or Slower for SD.


Somewhat better results I got with MaskThr=255
MaskThr=255 completely disables artifact masking.

It won't look good where there are actually artifacts, so I suggest comparing on high-artifacts scenes.

Since many high-artifacts scenes are being skipped altogether, it's easier to compare MaskTrh by setting BlendOver=0 and SkipOver=0, then you can look at what it's doing with ugly scenes.

[manolito]

What's the intention with the speed presets? The script seems a bit undecided.
Should the Assert line be permitting Preset="slower"?

I removed Preset = "slower", but obviously I wasn't thorough enough... :eek:

Script is fixed now.


There is a reason for removing "slower". Switching between DCT=1 and DCT=0 is a good idea in therory, but in my tests it did not work as intended. I have this anime clip where using DCT=1 removes tons of bad artifacts, but with the "slower" preset the script mostly picks the frames which use DCT=0. Which means that a conversion with Preset "slow" looks much better than with Preset = "slower" (and it is a little faster also).

Please note that for DCT=1 AND BlkSize=32 I disabled artifact masking completely (by forcing Output = "flow"). Looked better in my tests...


Cheers
manolito

[manolito]

Comparison here (default settings, except presets)
http://www.mediafire.com/file/oranq24vuccna10/FRC_comparison.mkv

Edit:
Somewhat better results I got with MaskThr=255 in MysteryX's script:
http://www.mediafire.com/file/4vwdtbc2gutvlw9/FRC_comparison2.mkv

Comments, suggestions for better tweaking?

The second test with MaskThr=255 shows that the core jm_fps script without any artifact treatment often looks better than the elaborate FrameRateConverter script. This is my experience, too.


The first test is interesting. The two Preset = "normal" conversions should look identical, but for some frames my script seems to be a little better. I can't explain this because the parameters are identical. The only differences are that my script has no stripe mask and that I do not use ConditionalFilterMT.

Comparing the two "slower" and "slow" conversions IMO it is a tossup. At the beginning the "slower" clip looks better, later in the clip I think my "slow" script gives better results.

For me the bottom line is that none of the scripts can get rid of the artifacts, but when watching the clip in real time I find the quality quite good. If I had encoded this scene with the "normal" preset I would never see the need to try a slower preset because of real bad artifacts.


Cheers
manolito


//EDIT//
It might be worth a try to encode the clip using my script and specifying Preset = "slow" and BlkSize = 32. In my experience this works well with objects like the fence.

[MysteryX]

Comparison here (default settings, except presets)
http://www.mediafire.com/file/oranq24vuccna10/FRC_comparison.mkv

Edit:
Somewhat better results I got with MaskThr=255 in MysteryX's script:
http://www.mediafire.com/file/4vwdtbc2gutvlw9/FRC_comparison2.mkv

Comments, suggestions for better tweaking?
Interesting. First, the fence doesn't get handled properly.

Then, a *lot* of frames are done with Blending; yet look different between my script and Manolito's script? It's as if whole-frame blending wasn't taking effect. Full-frame blending should look exactly the same.

Then, there are frames where the fence really doesn't look good with blending either; it would look better to skip it.

Can you upload your source so I can try with it?

[Sharc]

Interesting. First, the fence doesn't get handled properly.

Then, a *lot* of frames are done with Blending; yet look different between my script and Manolito's script? It's as if whole-frame blending wasn't taking effect. Full-frame blending should look exactly the same.

Then, there are frames where the fence really doesn't look good with blending either; it would look better to skip it.

Can you upload your source so I can try with it?
Here the source (originally provided by Selur):
http://www.mediafire.com/file/4xfo0poblri72su/forInterpolation.mp4

For my tests I resized it to 960x540.

[MysteryX]

OK. Good news is that StripeMask is accurately detecting the barrier.

The philosophy right now is that StripeMask is weak enough so that it gets counted for Skip value but doesn't alter the Mask (or very little), so most of the clip gets blended as expected.

Question is: for clips where only small areas have stripes, should StripeMask mask those areas? If so, then there's the risk that it's going to enlarge the mask in other areas where it isn't beneficial.

Darkening StripeMask so that it masks those areas would introduce a bunch of other problems. For example, those marks are the size of a block, so if we mask all full blocks in the mask, then the whole stripemask would trigger artifact removal which is bad.

This clip is like the moose scene: the best is to blend it all. In this particular case, setting BlendOver=41 would give better results, but I don't think I'd want to lower the default to 40. Since we only do frame blending, this isn't a good clip to test any of the other features.

There is also a bug where the last frames have a completely white mask. I saw this before and fixed it by deleting the last frame and repeating the other one before; but in this case the 3 last frames are white! Anyone knows what is the cause of this? Manolito, does the old version of MvTools2 have the same behavior?

I fixed a minor issue in my script to make StripeMask's strength relative to SkipTrh to get consistent output with various block sizes.

The difference between using StripeMask or not is that without it, you should get inconsistent frame blending even if you set BlendOver=41. Of course in this case, if you just set it low enough, it will all get blended, but then other scenes that don't need frame blending will get blended as well. With it, you get more consistent detection to blend the frames.

The other option would be to process the stripe mask entirely separately with a different logic to detect large chunks, let's say, 4x larger than blksize, and then merge that mask with the first mask. This would allow to mask only those areas and interpolate the rest of the image, and wouldn't require lowering BlendOver. I think that idea could work nice actually. Mask processing isn't heavy anyway compared to interpolation.

[MysteryX]

Alright I implemented masking of large patches of StripeMask :) It's working fine.

Try the latest script, and you'll get best results with BlendOver=0. The girl will be smooth the whole clip, there will be no serious artifacts (a little bit around her head only), and since we're focused on the pretty girl, we won't even notice the blended background.

There is still the issue with the last 3 frames having a white mask.

[hello_hello]

I removed Preset = "slower", but obviously I wasn't thorough enough... :eek:

Script is fixed now.

Cheers.

I thought as a tiny contribution I'd share my new found fondness for wrapper functions, brought on by typing laziness.
It just calls your script with preset frame rates by adding the frame rate to the function name. ie

xfps() = Double frame rate by default, or fps can be specified
x23fps() = 23.976fps
x24fps() = 24fps
x25fps() = 25fps etc.

It's easily edited to use a different preset or BlkSize by default.
Admittedly in this case it doesn't save a lot of typing, but every little bit helps. :)

# xfps - mx_fps wrappers
#
# Wrapper functions for mx_fps
# https://forum.doom9.org/showthread.php?p=1805050#post1805050
#
# xfps() = Double frame rate by default, or fps can be specified
# x23fps() = 23.976fps
# x24fps() = 24fps
# x25fps() = 25fps
# x29fps() = 29.97fps
# x50fps() = 50fps
# x59fps() = 59.94fps
# x60fps() = 60fps
#

# ========== xfps Wrapper Function - Double frame rate, or fps can be specified =====================

function xfps(clip c, float "fps", string "Preset", int "BlkSize")
{
fps = default(fps, c.framerate * 2.0)
Preset = default(Preset, "normal")
BlkSize = default(BlkSize, c.width>2000||c.height>1200 ? 32 : c.width>=720||c.height>=480 ? 16 : 8)
return c.mx_fps(fps, Preset, BlkSize)
}

# ========== x23fps Wrapper Function ====================================================

function x23fps(clip c, string "Preset", int "BlkSize")
{
fps = 24.0/1.001
Preset = default(Preset, "normal")
BlkSize = default(BlkSize, c.width>2000||c.height>1200 ? 32 : c.width>=720||c.height>=480 ? 16 : 8)
return c.mx_fps(fps, Preset, BlkSize)
}

# ========== x24fps Wrapper Function ====================================================

function x24fps(clip c, string "Preset", int "BlkSize")
{
fps = 24.0/1.0
Preset = default(Preset, "normal")
BlkSize = default(BlkSize, c.width>2000||c.height>1200 ? 32 : c.width>=720||c.height>=480 ? 16 : 8)
return c.mx_fps(fps, Preset, BlkSize)
}

# ========== x25fps Wrapper Function ====================================================

function x25fps(clip c, string "Preset", int "BlkSize")
{
fps = 25.0/1.0
Preset = default(Preset, "normal")
BlkSize = default(BlkSize, c.width>2000||c.height>1200 ? 32 : c.width>=720||c.height>=480 ? 16 : 8)
return c.mx_fps(fps, Preset, BlkSize)
}

# ========== x29fps Wrapper Function ====================================================

function x29fps(clip c, string "Preset", int "BlkSize")
{
fps = 30.0/1.001
Preset = default(Preset, "normal")
BlkSize = default(BlkSize, c.width>2000||c.height>1200 ? 32 : c.width>=720||c.height>=480 ? 16 : 8)
return c.mx_fps(fps, Preset, BlkSize)
}

# ========== x50fps Wrapper Function ====================================================

function x50fps(clip c, string "Preset", int "BlkSize")
{
fps = 50.0/1.0
Preset = default(Preset, "normal")
BlkSize = default(BlkSize, c.width>2000||c.height>1200 ? 32 : c.width>=720||c.height>=480 ? 16 : 8)
return c.mx_fps(fps, Preset, BlkSize)
}

# ========== x59fps Wrapper Function ====================================================

function x59fps(clip c, string "Preset", int "BlkSize")
{
fps = 60.0/1.001
Preset = default(Preset, "normal")
BlkSize = default(BlkSize, c.width>2000||c.height>1200 ? 32 : c.width>=720||c.height>=480 ? 16 : 8)
return c.mx_fps(fps, Preset, BlkSize)
}

# ========== x60fps Wrapper Function ====================================================

function x60fps(clip c, string "Preset", int "BlkSize")
{
fps = 60.0/1.0
Preset = default(Preset, "normal")
BlkSize = default(BlkSize, c.width>2000||c.height>1200 ? 32 : c.width>=720||c.height>=480 ? 16 : 8)
return c.mx_fps(fps, Preset, BlkSize)
}

[hello_hello]

Official script allows preset slower. (https://github.com/mysteryx93/FrameRateConverter/blob/master/FrameRateConverter.avsi)

Manolito took the diff feature out. DCT=1 is overkill, but preset="slower" isn't much slower, so I'd rather go with either Normal for HD or Slower for SD.

Cheers.

For funzies, I put together similar wrapper functions for FrameRateConverter as I did for manolito's version of the script. I didn't know which options others might want to change from their defaults, so I kept them all. This time it saves a bit more typing. FrameRateConverter Wrapper - xFRC.avsi (https://files.videohelp.com/u/210984/FrameRateConverter%20Wrapper%20-%20xFRC.avsi)

Unfortunately I couldn't test the script when encoding as it'd encode for a short period, then x264 would crash with an error apparently relating to FrameRateConverter.dll. I didn't make note of any details but I can try encoding again and provide what I can if you like.

Mind you it could be an XP thing, or my PC. Aside from the occasional reboot it's been running for about four years with zero maintenance, so it's due for a reformat. Or should I be using AVIsynth+ instead of the standard version?

I was using the script dated 18-May-2017.

PS MysteryX. The new version of the script is still dated 2017-05-18. I just thought I'd let you know in case you want to change it to avoid confusion.
Cheers.

[raffriff42]

MysteryX, pardon my ignorance, but where is the latest version of your script?

EDIT: thanks!

[Sharc]

MysteryX, pardon my ignorance, but where is the latest version of your script?
Good question. My guess is from here:
https://github.com/mysteryx93/FrameRateConverter/blob/master/FrameRateConverter.avsi

[hello_hello]

It says the last update was 7 hours ago, so I'm guessing:

https://github.com/mysteryx93/FrameRateConverter

The only place I could find FramerateConverter.dll was in the zip file here:

https://github.com/mysteryx93/FrameRateConverter/releases

[Sharc]

Alright I implemented masking of large patches of StripeMask :) It's working fine.

Try the latest script, and you'll get best results with BlendOver=0. The girl will be smooth the whole clip, there will be no serious artifacts (a little bit around her head only), and since we're focused on the pretty girl, we won't even notice the blended background.

There is still the issue with the last 3 frames having a white mask.
Here the update of the comparison with your latest script and BlendOver=0. It looks fine real-time on TV.
http://www.mediafire.com/file/mx4w2vuel255vfl/FRC_comparison4a.mkv

[MysteryX]

Here the update of the comparison with your latest script and BlendOver=0. It looks fine real-time on TV.
http://www.mediafire.com/file/mx4w2vuel255vfl/FRC_comparison4a.mkv

:)

Now I'm only updating the AVSI script on GitHub. I'll only release another package when there are changes to the DLL.

For the crashes, are you using ST or MT? It doesn't yet work with MT.

Admittedly in this case it doesn't save a lot of typing, but every little bit helps. :)

Sometimes I forget that not everybody types at 200 words per minutes, especially in an age of cellphones.

[pinterf]

Does it really work?

EMocc = MaskOcc > 0 ? C.ConvertToY8().MMask(bak, ml=MaskOcc, kind=2, gamma=1/gam, ysc=255, thSCD2=255).mt_inpand() : BlankClip(C, pixel_type="Y8", color_yuv=$000000)
EM = MaskOcc > 0 ? EM.Overlay(EMocc, opacity=.4, mode="lighten", pc_range=true) : EM

MMask is getting its format from the input clip, which is Y8 here.
But MMask does not support it (but it should, now I see).
The input clip format is not checked at all in this filter, I'm getting
"Filter error: GetPlaneWidthSubsampling not available on greyscale pixel type." when it tries to get subsampling for the non-existant chroma planes.

But when you happen to use a YV12 clip instead of Y8, either convert it to greyscale _before_ mt_inpand, or specify "process" mode (U=3, V=3) for mt_inpand, or else chroma planes will be garbage (This is important only if you ever want to use this mask for chroma later other than mt_merge(.. luma=true))

And the last one:
"pc_range" in Overlay is only used for RGB input or output, because Overlay "lighten" works in YUV colorspace

Then I have mentioned the mvtools MFlowFPS MT mode problem: I was getting different file sizes for each encoding.

There were two or three old bugs, but this is the nastiest one:
part of the bottom mask area was not cleared up and contained some random memory garbage in each run. The bug affected the bottom or bottom right part of the occlusion time mask. This mask is used by all MxxxxFPS when they are run in specific modes. It occured under specific circumstances, depending on the relationship between clip dimensions and blocksize and padding. Finding the culprit one single missing letter took me more than 30 hours, I was getting mad. Release later, I have to clean up all the debugging garbage I have put in the source.

[MysteryX]

Well... the script *is" working here so I'm not sure if something needs to be changed.

If I don't specify pc_range, then there were issues where the masks wouldn't go full black and full white, and instead the whole images would have partial blending (using 16 and 235 values instead of 0 and 255). It's the only way I found to fix it.

Those nasty spelling errors are the worst, I hear you! Undetectable to the naked eye! When bugs don't make any sense, I had taken the habit of looking for the stupidest causes.

[Sharc]

For the crashes, are you using ST or MT? It doesn't yet work with MT
ST only. Avisynth 2.60

[MysteryX]

ST only. Avisynth 2.60
There's a weird bug in StripeMask that causes crashes even in ST. I added code that checks memory boundaries on every assignment and I haven't been getting crashes since; but apparently you've got some anyway. Pinterf will look into it when he gets time; or I'll go deeper into debugging it.

[Sharc]

I didn't get crashes... must be someone else, unless you mean the last 3 frames issue ....

[MysteryX]

I didn't get crashes... must be someone else, unless you mean the last 3 frames issue ....
hello_hello got a crash in FrameRateConverter.dll

[MysteryX]

I also just got an access violation crash -- so definitely something still needs to be fixed in the DLL.

Script updated to improve the masking of stripe areas. The bicycle scene now looks very smooth.

[Sharc]

Updated comparison here:
http://www.mediafire.com/file/p8qccveqgvxtjgq/FRC_comparison4b.mkv

I did some blind tests on TV with "uneducated" viewers. Interestingly, nobody made comments about the fence. Only after I told them to focus on it, they noticed "some differences" but were still undecided about the ranking.

[MysteryX]

The latest version mostly has differences with the trees above the fence, resulting from more consistent and precise coverage of the fence. You have to know what to look for as these are details.

Also the fact that you're showing each version small makes it nearly impossible to see such details. Adding to the fact that this video only has 1 of 2 frames interpolated and the other half identical.

[hello_hello]

It occured under specific circumstances, depending on the relationship between clip dimensions and blocksize and padding. Finding the culprit one single missing letter took me more than 30 hours, I was getting mad. Release later, I have to clean up all the debugging garbage I have put in the source.

The 30 hours is appreciated. Thank you.

[MysteryX]

Another update. (https://github.com/mysteryx93/FrameRateConverter/blob/master/FrameRateConverter.avsi)

- Further improved the stripe mask to reduce false positives and give smoother results.
- Raw stripe mask will no longer affect artifact masking and will be processed separately. To reflect the slight weakening of the mask, MaskTrh default changed from 150 to 145.
- Adjusted default block sizes as follow:
Defaults for 4/3 video of height:
0-359: 8
360-749: 12
750-1199: 16
1200-1699: 24
1600-2160: 32

Note that stripe masking will cover some flag scenes in the parade. It may not look better (some frames are better, some are worse), but that's technically correct because that's what it's designed to do. It's not "worse" and it will give more consistent results across a range of videos.

[Sharc]

Error in Line 98
BlkSize = Default(BlkSize, DefH<360 ? 8 : DefH<750 ? 12 : DefH<1200 ? 24 : DefH<1600 ? 32)

should be (I think):
BlkSize = Default(BlkSize, DefH<360 ? 8 : DefH<750 ? 12 : DefH<1200 ? 16 : DefH<1700 ? 24 : 32)
But then I get an error in line 149 with the blocksizes x*y .....

[MysteryX]

Error in Line 98
BlkSize = Default(BlkSize, DefH<360 ? 8 : DefH<750 ? 12 : DefH<1200 ? 24 : DefH<1600 ? 32)

should be (I think):
BlkSize = Default(BlkSize, DefH<360 ? 8 : DefH<750 ? 12 : DefH<1200 ? 16 : DefH<1700 ? 24 : 32)
But then I get an error in line 149 with the blocksizes x*y .....
Uh... how did this one slip through?

After fixing that, I'm not getting more errors. What syntax are you using when getting error at line 149?

[Sharc]

I am getting:
MAnalyze: Block's size must be 4x4, 8x4,8x8,16x2,16x8,16x16,32x16,32x32
(FrameRateConverter.avsi, line 149

I don't get the error when I set BlkSize=8 or 16 or 32 explicitly. I get the error when I set BlkSize=24 (or leave it default)

[MysteryX]

and what block size are you using?

12 and 24 and only supported in Pinterf's latest version

[Sharc]

I left it at default. So I am guessing that it used 12 or 24 now (?).
In the previous version it picked 16 for the same clip.

Maybe I have to doublecheck with pinterf's (very)latest mvtools .... (will try later as I have to leave for now....)

[burfadel]

I get the error as well:
MAnalyze: Block's size must be 4x4, 8x4,8x8,16x2,16x8,16x16,32x16,32x32
(FrameRateConverter.avsi, line 149

The issue is this:
BlkSize = Default(BlkSize, DefH<360 ? 8 : DefH<750 ? 12 : DefH<1200 ? 16 : DefH<1600 ? 24 : 32)

If DefH is such that 12 or 24 is used, it seems Manalyse won't accept it as it states it only allows 4x4, 8x4,8x8,16x2,16x8,16x16,32x16,32x32, meaning 12x12 and 24x24 aren't supported.

To get the interim sizes this works, but probably stuffs up the masks and other things:
BlkSize = Default(BlkSize, DefH<360 ? 8 : DefH<750 ? 16 : DefH<1200 ? 16 : DefH<1600 ? 32 : 32)
BlkSizeV = Default(BlkSizeV, DefH<360 ? 8 : DefH<750 ? 8 : DefH<1200 ? 16 : DefH<1600 ? 16 : 32)

Instead of using invalid block sizes 12x12 and 24x24, it will use 16x8 and 32x16.

[Sharc]

With pinterf's mvtools2 2.7.19.22 from here (https://github.com/pinterf/mvtools/releases/tag/2.7.19.22) with depans it works now here even with BlkSize 12 and 24.
I am on avisynth 2.60 32bit (standard)

[burfadel]

Version 2.7.20.22 of mvtools2 just released. The bug has been fixed that caused the crash preventing me from using 2.7.19.22. The official framerateconverter script works fine now.

[Sharc]

I just tried it. Seems to work fine here and is even a bit faster.

[MysteryX]

New release: 2017-05-26 (https://github.com/mysteryx93/FrameRateConverter/releases/tag/v0.2-alpha)

The DLL access violation has been fixed, and performance improved. It now fully supports MT. It still only supports 8-bit with no SIMD.

[MysteryX]

I thought it would be interesting to show how the stripe mask works.

Frame 91, Interpolated image
https://s12.postimg.org/xxfs7e821/91flow.png (https://postimg.org/image/xxfs7e821/)

Scan blocks by calculating line averages and detecting contrast changes. Dynamic content blends into greys and show no such contrast.
https://s12.postimg.org/wwfji9r2h/91lines.png (https://postimg.org/image/wwfji9r2h/)

From these lines, detect recurring patterns and mark those areas.
https://s12.postimg.org/3wpqg0xu1/91patterns.png (https://postimg.org/image/3wpqg0xu1/)

Blur

\ .BicubicResize(Round(C.Width/BlkSize)*4, Round(C.Height/BlkSizeV)*4)
\ .FRC_GaussianBlur2(6)

https://s12.postimg.org/n9c18jy2x/91blur.png (https://postimg.org/image/n9c18jy2x/)

Binarize

\ .mt_binarize(90)

https://s12.postimg.org/wfubvu3bd/91binarize.png (https://postimg.org/image/wfubvu3bd/)

Blur again

\ .mt_expand(mode= mt_circle(zero=true, radius=5))
\ .FRC_GaussianBlur2(1.4)
\ .BicubicResize(C.Width, C.Height)

Final mask
https://s12.postimg.org/u3mby8qq1/91mask.png (https://postimg.org/image/u3mby8qq1/)

Final result
https://s12.postimg.org/oyl4gmds9/91auto.png (https://postimg.org/image/oyl4gmds9/)

[Sharc]

Very interesting demo. Thanks.

Edit:
Comparison "normal,flow" (interpolation) ..... "slower,auto" (masking). All other settings = default
http://www.mediafire.com/file/7z90z0l5b49t5e7/FRC_comparison4e.mkv
The masking works very well now IMO, also for higher resolutions of the picture.