FrameRateConverter (Official)

[hfrforever]

is that masktools2?

[StainlessS]

Yes, thats Masktools2.

Didnt we have a wonderful day, the day we went to Bangor, A beautiful day, we had lunch on the way, and all for under a pound you know,
and on the way back, I cuddled with Jack and opened a bottle of cider, singin' a few of our favourite songs as the wheels went round ...
https://www.cosgan.de/images/smilie/musik/e050.gif https://www.cosgan.de/images/smilie/musik/a075.gif https://www.cosgan.de/images/smilie/musik/c025.gif

EDIT: Fiddler's Dram - "Day Trip to Bangor (Didn't We Have a Lovely Time ... https://www.youtube.com/watch?v=wfwQi8L9h8M

[hfrforever]

it was mask tools 2. It still says that there is no function named 'RT_String'.

EDIT: I am officially to the ripping hair out stage.

thank you very much for your infinite patience

[StainlessS]

Comment out the FrameRateConvert line (precede with '#' character)
Try in VDub (NOT x64 version, not x64 version media player, MUST use ALL 32bit or ALL 64 bit), if complaining about RT_string, its because its the very first used external
function from dll, I'm guessin' because you are using x64 AVS via x64 player, and ALL plugins are 32 bit, so ALL will fail, its just that RT_string is first in line pending fail.
EDIT: Probably Vdub2 is your favourite bet, as has both x86 and x64 versions.

If works OK, then uncomment.

[hfrforever]

i tried it as is, and it seemed to work in vdub (at a really low framerate)

[hfrforever]

it also works commented out.

[hfrforever]

how can I make it work in megui then? I seem to be using 32bit megui (is in the x86 program files directory, and what I downloaded seems to be 32 bit.

[StainlessS]

Have you tried in MeGUI since now got it working ? Should work OK unless you have installed 64 bit MeGUI, (which is not quite stable as yet).

EDIT: And yes, motion compensated processing can be slow. (A few years back I spent 3 days rendering a movie [LOTR-ROTK]).

[hfrforever]

yes, same error as before.


https://i.postimg.cc/6QrKfB8N/error8.png (https://postimages.org/)

[StainlessS]

Check [expand] first line in log, Versions.

[hfrforever]

https://i.postimg.cc/j2SrT9Rw/error9.png (https://postimages.org/)

[StainlessS]

And the other stuff, eg redistributables (ie runtimes).
https://i.postimg.cc/2yVZVgtR/Me-GUI-VERS.jpg (https://postimages.org/)

EDIT: At bottom of page looks like MeGUI bug (somebody post it in MeGUI forum), says path to dll is Windows\system32\Avisnth.dll (on x64 system, should be showing SysWOW64),
the one its showing is the 64 bit dll (in 32 bit MeGUI). [On my system both dll's present in system32 & SysWOW64]

[hfrforever]

https://i.postimg.cc/bYVvJ04c/error10.png (https://postimg.cc/qgnrDKkj)

[StainlessS]

You seem to be missing the RT_Stats VS2008 redistributables (runtimes).
You will likely also need VS2012.
Not sure about VS2013, maybe VS2013 -> 2019 all some, cant remember.

See Here:- https://support.microsoft.com/en-gb/help/2977003/the-latest-supported-visual-c-downloads

You need 2008, 2012, 2013.

and from MS above link, looks like you best install vs2019.
Note Visual C++ 2015, 2017 and 2019 all share the same redistributable files.

For example, installing the Visual C++ 2019 redistributable will affect programs built with Visual C++ 2015 and 2017 also. However, installing the Visual C++ 2015 redistributable will not replace the newer versions of the files installed by the Visual C++ 2017 and 2019 redistributables.

This is different from all previous Visual C++ versions, as they each had their own distinct runtime files, not shared with other versions.


ABOVE, BOTH x86 and x64 REQUIRED [you will need all, sooner or later]

[hfrforever]

Where would I get them from?

[StainlessS]

The given link above.

[Groucho2004]

Where would I get them from?Didn't you already install the All-In-One MS runtime thingy? Also, if runtimes were missing, the Info Tool would complain.

Edit - Maybe you're using the "other" Avisynth in megui?

[StainlessS]

In my MeGUI image, shows VS2005 and VS2015, missing, but they are present in Programs And Features thing.
Maybe should check in Programs And Features, perhaps an additional bug in MeGUI [I had I think noticed that before, but ignored it].

EDIT: Or maybe MeGUI only shows runtimes that it relies upon(or its tools), dont know.

EDIT: I think hfrforever version MeGUI is current Stable (2896).

[StainlessS]

From Your Image

Avisynth Status: Ignored, as Portable build is FORCED.

We Got it. https://www.cosgan.de/images/smilie/froehlich/c010.gif

No idea how to unforce it, probably in Settings.

EDIT: Easy Peasy, first setting "Always Use Included Avisynth", Untick.

See just above Avisynth Portable, in the last image you posted (and also Avisynth Portable entries).

EDIT: I think that MeGUI now clutters up its directories with loads of dll's and stuff, real messy, perhaps its just showing the VS runtimes included within MeGUI Directories.

[hfrforever]

maybe, how would i know if I am??

[Groucho2004]

I am officially to the ripping hair out stage.Avisynth is quite complex and you (please don't take offense) are a complete noob and have not read any documentation. The spoon-feeding we're doing can only get you so far and most importantly, you're not learning anything, you just follow commands you don't understand (yet).

So, you may have a look at the tutorials on avisynth.nl:
http://avisynth.nl/index.php/Main_Page
("New to AviSynth – Start Here")

[hfrforever]

your probably right. I will come back if i cant figure it out in a few days.

[Groucho2004]

your probably right. I will come back if i cant figure it out in a few days.Well, we've been at this for almost 100 posts so let's at least get you going on this particular venture. :D

Did you find the megui switch off portable thing?

[StainlessS]

To be fair, he has encountered quite a few hurdles, and the Forced MEGUI thing would have tripped me up a bit, was not even aware of that setting.

Anyways, there is now it seems light at the end of the drain pipe, nothing but fun from here on in. https://www.cosgan.de/images/smilie/froehlich/a065.gif

EDIT:
Maybe a MODERATOR could separate off from post #192:- https://forum.doom9.org/showthread.php?p=1876263#post1876263
into a new thread with hfrforever as OP, and let MysteryX have his thread back.

Suggested title pending rename by hfrforever, "A Painless Intro to Avisynth.".

[hfrforever]

it works. I am soooo dumb. i was hitting the wrong *** button. this whole time...... argh. argh.... there are two queue buttons (now i know)

https://i.postimg.cc/K8MbFjKt/success2.png (https://postimages.org/)

thank you both very much for your help and patience working with me.

[StainlessS]

I personally use (almost) nothing but AutoEncode. [EDIT: And most use CRF (Const Qual) rather than two pass for x264, I usually use CRF 21.5, many use lower (higher qual) setting]

EDIT: Also, not sure but think some version of AAC would perform better with x264 MP4, than mp3. [also has more bang for you're buck]

EDIT: I have no idea which AAC flavour is best bet, I personally use Nero AAC, which I think is no longer available for download,
and most others use something else, I just stick with what I've always used. [AAC, half file size for same quality : compared to mp3]

[ChaosKing]

EDIT: I have no idea which AAC flavour is best bet, I personally use Nero AAC, which I think is no longer available for download,
and most others use something else, I just stick with what I've always used. [AAC, half file size for same quality : compared to mp3]

The best is currently the AAC encoder from Apple. Install /extract Itunes and download (or via megui) qaac wrapper, done.

[Dreamject]

Opus betteryoutube uses it

[videophile]

Hi, I am new to this forum and this is my first post. English is not my mother tongue so I apologize in advance for any mistake.

I have developed a 8 mm / Super 8 transfer bench, which is working quite well. I do some post-processing after capture, which includes denoising, color correction, image stabilization and frame rate interpolation from 18 to 36 fps.

I use an Avisynth script and I am struggling with frame rate interpolation.

I have first used this script, which, I think, comes from John Meyer:
super= processed.MSuper()
backward_vec= MAnalyse(super, blksize=block_size, blksizev= block_size_v, overlap=block_overlap, isb=true, truemotion=false, search=3)
forward_vec= MAnalyse(super,blksize=block_size, blksizev= block_size_v, overlap=block_overlap, isb= false, truemotion=false, search=3)
interpolated= processed.MFlowFps(super, backward_vec, forward_vec, num=36, den= 1, ml=200)

In this script, "processed" in the input clip, and "interpolated" is the output clip.

This script works quite well, except on fast motion where it produces artifacts.

Then I tried FrameRateConverter.

Here is the FRC script I used:
interpolated = processed.FrameRateConverter(Preset="slow", FrameDouble=true)

On very complex scenes with a lot of motion, FRC reverts back to frame blending, and the visual result is more pleasant than the JM script. However, I feel that the algorithm used by FRC to detect problematic areas and fill them with blended frames is way overzealous. As a result, FRC is sometimes better, but too often worst, which is not what I would expect.

Here is an example clip (FRC on the left, JM on the right): https://cp.sync.com/dl/7fa3fbd80/ncpe3rku-nv7j5qrq-vwccx56c-8apevkms

In this screenshot from the clip, we see that FRC has blended the dress, whereas JM has done a good job in spite of the repetitive pattern (which is always difficult for motion detection):
https://i.imgur.com/fFVMywJ.jpg

Here, FRC has blended the whole frame: we see the ghost hair of the lady (red circle), whereas JM has done a good interpolation, except on the eagle (blue circle):
https://i.imgur.com/fDaTgFR.jpg

This example shows how FRC often fails: although there was a moderate motion of the eagle's head, FRC has decided to blend (see the eye and the surrounding area on the left image), whereas JM has interpolated it properly:
https://i.imgur.com/ia94Q2b.jpg

This last example shows another situation where FRC has unnecessarily blended the whole frame:
https://i.imgur.com/v2qLz67.jpg

I have tried to play with the FRC parameters, but no luck. The parameters are quite numerous and the documentation is not very detailed, so I am not sure what to do.

I feel that FRC is not doing what it is expected to do.

Any advice?

[johnmeyer]

I wanted FRC to work. I was rooting for it to work. It is a much-needed piece of technology. The guy who wrote it put in a ton of time and effort, and obviously built something that can and does work, in certain circumstances.

However, I had the same experience as you, and finally gave up. I think it still needs a lot more work on both the masking and detection, the two key technologies that underlie its basic concept.

[videophile]

Are you THE John Meyer ?

If yes, then thank you for your great script which I am using since 2013! Is there somewhere an official/updated version?

Regarding FrameRateConverter, that's bad news :(

I think I will stick to my current technique: I create two clips at 36 fps, one with your method, the other one with frame blending, and then I manually select the one that works best on each scene, using my NLE Software (Edius).

[johnmeyer]

Are you THE John Meyer?I am one of thousands, but yes, I am the guy who spent half a day trying hundreds of combinations of MVTools2 settings, trying to understand what each one did, and figure out what impact they had on the typical motion estimation artifacts that all motion estimation tools seem to create. These problems are present even in the expensive commercial programs, such as Twixtor, After Effects, Motionperfect, etc.

I managed to find some sort of "sweet spot" that has held up pretty well for many people. The most important setting was block size. What I have found is that for frame rate conversion you want to use big block size, which makes perfect sense, since this keeps small details from producing "morphs." By contrast, when doing motion-estimated de-noising, you almost always want to use smaller block sizes.

The mask idea is a good one, and when I am doing really critical work, I'll create both a motion estimated version as well as a frame-blended version, but then manually go through each frame and either cut between them for that one frame, or create a manual mask, which is the equivalent of what this tool tries to do. I only do this for paid jobs, of course, and it is very, very tedious.

[kolak]

Best motion adaptive engines are in XFile (ex Alchemist) and in Tachyon. Both GPU based. Apparently Nvidia neural based engine is also good.

[johnmeyer]

Best motion adaptive engines are in XFile (ex Alchemist) and in Tachyon. Both GPU based. Apparently Nvidia neural based engine is also good.Thanks for listing those two. I need to look at the results they can produce. I'm always looking for better tools, even if they cost money.

[videophile]

The mask idea is a good one, and when I am doing really critical work, I'll create both a motion estimated version as well as a frame-blended version, but then manually go through each frame and either cut between them for that one frame, or create a manual mask, which is the equivalent of what this tool tries to do. I only do this for paid jobs, of course, and it is very, very tedious.

Yes, this is exactly what I am doing currently. I wished I could rely on a more automated - i.e. less time consuming - method.

[doxel]

I've found two john meyer scripts on this forum with only two lines that are different and I'm wondering which version people are using:

super = MSuper(source, hpad = 16, vpad = 16, levels = 1)
superfilt = MSuper(prefiltered, hpad = 16, vpad = 16)
super = MSuper(source, hpad = 16, vpad = 16, levels = 1, sharp = 1, rfilter = 4)
superfilt = MSuper(prefiltered, hpad = 16, vpad = 16, sharp = 1, rfilter = 4)

What does 'sharp' and 'rfilter' do? Should I use them?

[manolito]

Have a look at this post:
https://forum.doom9.org/showthread.php?p=1841952#post1841952

IMO these two params do make sense, but you need to test it for yourself...

[kolak]

Thanks for listing those two. I need to look at the results they can produce. I'm always looking for better tools, even if they cost money.


They cost serious money. Xfile is 10k$. Tachyon similar.

[poisondeathray]

Best motion adaptive engines are in XFile (ex Alchemist) and in Tachyon. Both GPU based. Apparently Nvidia neural based engine is also good.



You mentioned this a year or two ago . Is that marketing speak or actual testing ?

I don't know about Tachyon, but for Alchemist Ph.C / Ph.C high effort, it's terribly over rated in terms of the motion compensation conversions

I got a chance to test Alchemist at a trade show, then later in more detail where I know somebody at facility that has it . The artifacts were quite similar as to other solutions, and it would fail in exactly the same conditions and situations. I ran it though typical clips where most ME approaches fail in automatic mode, not some cherry picked marketing demo clips. It was slightly better in terms of the artifacts were maybe slightly smaller (there weren't massive fails, just slightly smaller edge morphing fails)

I think the product got taken over by another company ( Grass Valley now? ), so maybe it improved 100x . But when it was still Snell, it was similar to other solutions in pure automatic mode.

The Nvidia demos looks great and promising, but until it's a usable product under more a few cherry picked test scenarios, it's still unproven vapourware in my mind.

[johnmeyer]

I don't know about Tachyon, but for Alchemist Ph.C / Ph.C high effort, it's terribly over rated in terms of the motion compensation conversions

I got a chance to test Alchemist at a trade show, then later in more detail where I know somebody at facility that has it . The artifacts were quite similar as to other solutions, and it would fail in exactly the same conditions and situations. I ran it though typical clips where most ME approaches fail in automatic mode, not some cherry picked marketing demo clips. It was slightly better in terms of the artifacts were maybe slightly smaller (there weren't massive fails, just slightly smaller edge morphing fails)

I think the product got taken over by another company ( Grass Valley now? ), so maybe it improved 100x . But when it was still Snell, it was similar to other solutions in pure automatic mode.

The Nvidia demos looks great and promising, but until it's a usable product under more a few cherry picked test scenarios, it's still unproven vapourware in my mind.That's really interesting information and will save me some time trying to do my own tests. I too have found that the artifact generation seems to be an issue with the concept itself, and not with the specific implementation: if I take video of a picket fence while driving by (one of the best torture tests), really bad things are going to happen when I use any of these products.

The problems are also highly dependent on the original frame rate: if you apply ME to 10 fps material and try to take it to 60 fps, all heck will break loose. But, if you want to take 120 fps to 720 fps, it will look perfect, no matter what program you use.

I have always thought that somewhere in that last sentence lies the solution to the problem of getting better results, perhaps by doing multiple estimations sequentially, rather than all at once.

As (admittedly flimsy) support for that idea, when doing broadband audio noise reduction (e.g., single-ended hiss removal), you get much better results by reducing the noise only a little bit, and then doing a second (and third, fourth, fifth, etc.) pass, each one doing just a little reduction. You get far fewer artifacts this way.

However, I don't know if it is possible to only do a "little ME" in each pass.

I'm just trying to come up with ideas ...

[poisondeathray]

The problems are also highly dependent on the original frame rate: if you apply ME to 10 fps material and try to take it to 60 fps, all heck will break loose. But, if you want to take 120 fps to 720 fps, it will look perfect, no matter what program you use.

I have always thought that somewhere in that last sentence lies the solution to the problem of getting better results, perhaps by doing multiple estimations sequentially, rather than all at once.

As (admittedly flimsy) support for that idea, when doing broadband audio noise reduction (e.g., single-ended hiss removal), you get much better results by reducing the noise only a little bit, and then doing a second (and third, fourth, fifth, etc.) pass, each one doing just a little reduction. You get far fewer artifacts this way.

However, I don't know if it is possible to only do a "little ME" in each pass.

I'm just trying to come up with ideas ...



I don't think that's applicable here, but go ahead and test it out

The reason why higher FPS sources work better - in general - is you are starting with more frequent true motion samples. So in general, the distance that a given object moves between frame A and frame B is smaller for the higher FPS source (of course you have exceptions like whip pans, explosions etc..., but in general terms) . The larger the motion (and distance travelled by objects), the more difficult to interpolate.

Not only that, but complex motion paths, and things in nature are not linear. Someone walking in straight line does not mean their muscles bend and flex linearly. Where that FPS snapshot frame take of where an arm bend might miss the peak of the bend. You miss the trajectory and motion curves

And that only addresses some of the problems with ME. The other huge problem is occlusions and object boundaries . Objects passing over another. Objects deforming (yet the same object). This involves user intervention in the higher end solutions . If that could be done accurately, automatically, you have a winner. Some of the github projects look amazingly accurate for object/mask generation , the ones for superresolution too

[Matt Kirby]

Hi guys,

I have a problem with "framerateconverter" and "anime".
I have ugly blends or ghosts in my frames when I use it to slowmotion my anime video. My original video has 25 fps. I pump it up with interpolated frames to 37.5 fps. Then I slow down it to 25 fps. The video is factor 1.5 longer than the original. (I need it in this speed, don't ask why :D)

My script:
LoadPlugin("...\tools\lsmash\LSMASHSource.dll")
LWLibavVideoSource("...\Mach a S Test.mkv")
FrameRateConverter(NewNum=37500, NewDen=1000, Preset="Anime")
AssumeFPS(25)


The ugly result is : Look at "Attachment"

Here is the original testfile:

https://www.dropbox.com/sh/pl29vs4t20wanco/AAD3FPYO_TwyoR0kdinij6Q8a?dl=0


Is it possible to convert the video to 37.5 fps without these effects?

[manono]

Frame interpolation often doesn't work well with animations. You might be better off with frame duplication.

ChangeFPS(37.5)
AssumeFPS(25)

[Matt Kirby]

OK, it looks much better. Thank you!

[poisondeathray]

The Nvidia demos looks great and promising, but until it's a usable product under more a few cherry picked test scenarios, it's still unproven vapourware in my mind.


Mini review - Current status of available Optical Flow AI research methods (that are semi-usable right now and don't require a PhD in programming to use)

1) Super SloMo - This one was the biggest "letdown" for me :(

Remember the "Super SloMo" Nvidia AI demo awhile back ?
https://www.youtube.com/watch?v=MjViy6kyiqs
https://people.cs.umass.edu/~hzjiang/projects/superslomo/

It's going to be released for NGX for RTX cards eventually, the author said it could be implemented in pytorch
https://developer.nvidia.com/rtx/ngx

So there is a working pytorch implementation with pretrained model using the Adobe240fps dataset
https://github.com/avinashpaliwal/Super-SloMo

The results? Mediocre. Slightly better/slightly worse in areas than mvtools2. Basically the same problems as other optical flow methods - object/edge artifacts , picket fence problems etc.. It might be an issue with this implementation, or perhaps the dataset and training aren't as good as the other unavailable set. I tried several tests, including samples cut from Nvidia's own demo video and the results weren't as good as the demo. Their demo video had clean edges, almost no warping artifacts. But the trained model used was different, and there might be pytorch implementation differences. Hopefully the official NGX release will improve with different distributed trained models

There is a tensorflow implementation of Super-Slomo using the same Adobe240fps dataset, but the provided tensorflow model has motion problems. Produced results are definitely worse than gif bike demo and the pytorch implementation
https://github.com/rmalav15/Super-SloMo

2) CyclicGen, voxelflow based
I got CyclicGen to run, but results are poor. Lots of edge morphing artifacts. Tried both small and large models
https://github.com/alex04072000/CyclicGen

3) sepconv using "lf" model seems to do ok, similar quality to super slo mo; ie. Slightly better/slightly worse in some areas than mvtools2 based optical flow.
https://github.com/sniklaus/pytorch-sepconv


Of course, most of my tests were on "problematic" samples where OF typically "fails" or has problems, so it's probably not representative of general use

[johnmeyer]

I just tuned past a re-run of M*A*S*H on MeTV and they have used optical flow to make it look like it was shot on video rather than film. I assume they had access to whatever "best of breed" is available for professional use. It too doesn't look very good.

[StainlessS]

Anybody done any tests to see if hi bitdepth performs better in mvtools than lo bitdeth.
For lo BD and eg pel =4, so max vector length about 32, & so if movement
greater then cannot be
Used.

God I hate auto correct, Mobile.

Edit: Presumably vectors can be much longer in HBD.
Edit: Of greater importance where HD clip.

[Adam65]

Hi,
How to install FrameRateConverter on Windows with MPC-HC/BE ?

[Sharc]

Hi,
How to install FrameRateConverter on Windows with MPC-HC/BE ?
It would put too much load on the CPU. Playback would just stutter.

[Treaties Of Warp]

At the "best" settings (Preset = "slowest",Dct=1,DctRe=1), it's WAAAAY better than Interframe at not producing noticeable artifacts. But it's also WAAAAAY slower. :)

It does seem to have one problem... it doesn't do well with scrolling end credits, especially white text on a black background. "Boxes" or sections of credit text seem to "slide" around a little as they scroll. Has anyone encountered that and have any settings that might fix it?