Framerate Conversation - filling "between frames"

[color]

I have some video in 15 fps, I did use a script before for one off them to 30 fps.

Did try MSU Framerate Conversion "MSU_FRC.dll".
http://www.compression.ru/video/frame_rate_conversion/index_en_msu.html

It does what I want but it has some "artifacts", I can live with some but there is a little bit to mutch. I've seen some scripts for slow motion that adds some frames between for smoother playback. But It's hard to get them to work with normal speed.

Does it exist something else similar to this? :confused: I did try to change fps by the avisynth.nl, but they just double upp frames so It's still not smooth.

I used a script from a blog that has been deleted (even web.archive can nog even retrieve it), I loved that but I don't have it anymore. :(

[cork_OS]

FrameRateConverter (Official) (https://forum.doom9.org/showthread.php?p=1814652)

[manolito]

The latest FrameRateConverter version by MysteryX is this one:
https://forum.doom9.org/showthread.php?p=1817245#post1817245

It is based on a script by johnmeyer and adds artifact masking to this basic script. In many cases this much simpler script is all you need:
https://forum.doom9.org/showthread.php?p=1800439#post1800439

I have also made a simplified version of MysteryX's script which works with plain vanilla AviSynth 2.6, it works under WinXP and does not even need a SSE2 capable CPU. The following post also has a link to an AIO package with all the needed plugins (older but stable versions):
https://forum.doom9.org/showthread.php?p=1805050#post1805050


All the above scripts are based on MVTools2 and can be quite slow. For something faster you can try the old MotionProtectedFPS. It produces more artifacts, but it is a lot faster. Download here:
https://www.sendspace.com/file/if2o7s


Cheers
manolito

[color]

Oh thank you both. I will give them a try they look amazing. :)

[color]

I did try FramerateConverter but get this error:

MAnalyse: Block's size must be 4x4, 8x4, 8x8, 16x2, 16x8, 16x16, 32x16, 32x32
(FrameRateConverter.avsi, line 143)

Don't realy understand, the size is 720x480 on the video.

jm_fps, it accouly looks great, and I might even go with 25fps so its pal (like where I live). It do look amazing and It gives so little artifacts and runs smooth. (trying out doubble the frame) I do look one one scen where there is a guy walking in front of a pole, It does "split" and follow the guy for 2 frames.

SlowMo, I double up the frames and it worked with make the speed double so fast and It run smoothly. But its a bit slow. But It gives even less artifacts. I look on the same guy and this time it is giving artifacts but It's on 1 frame and It's faint, so playing it in speed its hard to see it if you don't know its there.

mx_fps, can not get it to work, It first says it needs to be YUV colorspace but when using ConvertToYV12() or 16 or 24 it says, unsuported colorspace. masktools only support planar YUV colorspaces (YV12, YV16, YV24) (mx_fps.avsi, line 142) looking on the line its:

EMocc = MaskOcc > 0 ? C.ConvertToY8().MMask(bak, ml=MaskOcc, kind=2, gamma=1/gam, ysc=255, thSCD2=255).mt_inpand() : Blank

[manolito]

FrameRateConverter needs the latest "modernized" PinterF versions of MVTools2 and MaskTools. The old standard MVTools2 by Fizick only supports block sizes of 8, 16 and 32.

SlowMo has not been mentioned in this thread so far. Can you give a link to this script?

mx_fps needs YV12 or YUY2 color space for the input. I have no explanation for your errors except maybe different versions of the helper plugins. For testing you should use the plugin versions from the AIO package. It works nicely here on 3 different machines under AVS 2.60 (WinXP and Win7 64-bit).

BTW did you also test MotionProtectedFPS?


Cheers
manolito

[manolito]

masktools only support planar YUV colorspaces (YV12, YV16, YV24) (mx_fps.avsi, line 142) looking on the line its:

EMocc = MaskOcc > 0 ? C.ConvertToY8().MMask(bak, ml=MaskOcc, kind=2, gamma=1/gam, ysc=255, thSCD2=255).mt_inpand() : Blank

The only call to masktools in this line is the "mt_inpand" command. And a test script like this works nicely with masktools-v2.0a48.
AnySourceFilter("mysource")
ConvertToY8()
mt_inpand()


Cheers
manolito

[color]

https://forum.doom9.org/showthread.php?t=174089

I don't know if that is the right one, I think I used:

https://forum.doom9.org/showthread.php?t=174089&page=2

I did use this with the latest one. I think I did it wrong but it works perfectly.
SlowMo(mul=2, ILeave=True)

[color]

I will read more what you wrote tomorrow, Its night here now. :D

[color]

Where do I find "AIO package"?

I did get them to work, but after reinstalling my pc (but didn't do backup before this I don't remember where to find them)

[Cary Knoop]

Going from 15 to 30p using interpolation is quite a stretch. Using cadences is in those cases often the better solution. In your case you would simply double the frames.

Here is an example of converting 18p to 60p using an (AAA, BBB, CCCC) pattern:
https://www.youtube.com/watch?v=9Jjb6vWo1MQ

Compare this to generated artifacts using flowFPS in Vapoursynth:
https://www.youtube.com/watch?v=SFE8QqVbJ_4

Or optical flow in Davinci Resolve:
https://www.youtube.com/watch?v=iIe3GyzsJ_A

Or using flownet2 (https://github.com/lmb-freiburg/flownet2-docker):
http://users.mur.at/ms/attachments/flownet2/flowtest30.mp4


A fragment of the source file:
https://www.dropbox.com/s/vx1wbiksagpjpa5/flow%20test.mov?dl=0

[Frank62]

Did you get flownet2 to work?

[manolito]

Where do I find "AIO package"?

I am hosting it permanently at VideoHelp.
https://files.videohelp.com/u/172211/mx_fps%20AIO.zip


Cheers
manolito

[color]

Cary Knoop, stupid question, flownet2 looks like it could do a mutch better job than the other options. But does it work on Windows 10, 64bit, if so, how? And if not is it only python in ubuntu it needs? I might give it a try in ubuntu terminal (in windows), I got phyton installed there.

EDIT: or can you do a tutorial?

Manolito. Thank you. I searched and searched but didn't find it here, but i think it where there (VideoHelp) I found it. Thank you for direct link. :)

[johnmeyer]

Cary,

That is a breathtakingly beautiful transfer, one of the best nitrate transfers I've ever seen. It is somewhat similar to the two films taken in San Francisco on Market Street just before and then again just after the '06 earthquake, but yours is in much better shape. I think the SF one was cranked at about 10-12 fps. The higher frame rate of yours definitely adds to the feeling of realism.

I seldom use any optical flow with my film transfers, but am intrigued with the artifact suppression I see in that bridge torture test. All those vertical lines, coupled with camera pan, people walking, people entering and leaving the frame, and people passing each other close to the camera pretty much cover all the bases of what breaks optical flow. There is some "goo" around people's heads, but the bridge support and railings stay intact.

I guess I need to try it out.

[edit]Never mind. I spent a little more time looking at all three optical flow results, and they all exhibit the usual motion estimation artifacts. The flownet2 result was better in that it did a better job at keeping vertical lines intact while the camera panned, but failed pretty badly as people walked in front of the camera. So, I will continue delivering most of my work with the tried-and-true telecine patterns.

[Cary Knoop]

That is a breathtakingly beautiful transfer, one of the best nitrate transfers I've ever seen. It is somewhat similar to the two films taken in San Francisco on Market Street just before and then again just after the '06 earthquake, but yours is in much better shape. I think the SF one was cranked at about 10-12 fps. The higher frame rate of yours definitely adds to the feeling of realism.


I had some brief conversation with Rick Prelinger to obtain a better quality digital scan of that particular San Francisco footage using the original framerate as the published material has compression artifacts and uses complicated cadences. I was able to cut the duplicate frames but the compression artifacts are still in the way. This reminds me to follow up.

[color]

Could you do a tutorial on how to get flownet2 to work? I don't really understand how to do this.

[poisondeathray]

I had some brief conversation with Rick Prelinger to obtain a better quality digital scan of that particular San Francisco footage using the original framerate as the published material has compression artifacts and uses complicated cadences. I was able to cut the duplicate frames but the compression artifacts are still in the way. This reminds me to follow up.

Were the frame drops in the original footage too ? Looks like ~3rd frame in that 18fps sample . Or where is the original published material ?

thanks

[Cary Knoop]

Were the frame drops in the original footage too ? Looks like ~3rd frame in that 18fps sample . Or where is the original published material ?

thanks
For the YouTube video, I had to clean up the New York footage to get 18fps and added a (AAA, BBB, CCCC) cadence afterward to get 60p. The samples were experiments to see if optical flow would be a better solution but as you can see it is not.

The published San Francisco material has duplicates as well but the compression is too high to be useful.
There is a 118GB version but downloads consistently fail. Hopefully, this version used an encoding that respects the original framerate.

[poisondeathray]

For the YouTube video, I had to clean up the New York footage to get 18fps and added a (AAA, BBB, CCCC) cadence afterward to get 60p. The samples were experiments to see if optical flow would be a better solution but as you can see it is not.

I meant the dropbox "source" MOV, not the YT video



Also, shameless bump for flownet2 - do you have any other general impressions /observations ?

Honestly it doesn't look that good , I was expecting better results, or maybe there are some other options or models that can be used ? I'd still like to play with it

[color]

I have trying to try flownet2, but I end up with error, but I came one step closer but I still get error. I'm also trying to do a tutorial, but It does not go so well. Any idea? seems like I'm not able to install docker, I tried a few different install scripts but It just don't want to install, or find the package. I don't know where its located online, otherwise that could be easier.

Step 1: UPDATE

$ sudo apt update
$ sudo apt upgrade # press Y to accept, this takes a while

Step 2: REMOVING OLD:

$ sudo apt install docker.io # press Y to accept, this takes a while
$ docker volume ls -q -f driver=nvidia-docker | xargs -r -I{} -n1 docker ps -q -a -f volume={} | xargs -r docker rm -f
$ sudo apt-get purge nvidia-docker

then:

$ sudo apt-get install nvidia-docker2
$ sudo pkill -SIGHUP dockerd

GET ERROR:
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package nvidia-docker2

//EDIT: I also did this:

curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | \
sudo apt-key add -
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | \
sudo tee /etc/apt/sources.list.d/nvidia-docker.list
sudo apt-get update


$ alias docker="sudo docker.io"
$ complete -F _docker docker

see if it works?
$ sudo docker info

Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

Even tried:

$ sudo service docker start

still get:
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

//END EDIT

Step 3: CHECK VERSION

$ python3 -V

It should be 3.6.6 or newer. Otherwise, just update with "sudo apt-get update xxx" replace xxx with the version.
Then install make:

$ sudo apt install make

Step 4: go to the folder:

$ cd /mnt/d/flownet2/flownet2

Step 5: MAKE

GET ERROR
docker build -f Dockerfile -t flownet2 .
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Makefile:7: recipe for target 'flownet2' failed
make: *** [flownet2] Error 1

And of course the script wont work:
Using GPU: 0
Running network: FlowNet2
Working dir: /flownet2/flownet2/scripts
First input: data/00000.png
Second input: data/00001.png
Output: flow.flo
./run-network.sh: line 165: nvidia-docker: command not found

[color]

After restarting twice, I got another error:

sudo apt install nvidia-docker2
Reading package lists... Done
Building dependency tree
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
nvidia-docker2 : Depends: docker-ce (= 18.06.1~ce~3-0~ubuntu) but it is not installable or
docker-ee (= 18.06.1~ee~3-0~ubuntu) but it is not installable
E: Unable to correct problems, you have held broken packages.