Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
|
10th March 2017, 13:22 | #1 | Link |
Registered User
Join Date: Jan 2002
Posts: 226
|
SVP-like frame interpolation?
I'm almost certain this has been answered elsewhere, but I had difficulty finding an answer. The software SVP (Smooth Video Project) does a pretty impressive job of generating frames based on motion in order to increase the framerate of content, however its focus is for real-time playback. Is there some rough equivalent of this for Avisynth or some other software to generate the frames as a new video; not in real-time, but as some sort of script to be processed?
So say I had a 30fps clip that I wanted to convert to 60fps (or slow down more for slow motion), but instead of doubling frames, I wanted the computer to use motion analysis to take its best guess in generating new ones. How would I go about doing that? Please feel free to dumb this down for me (like posting a sample script), I often get hung up on simple syntax errors. Thanks in advance. |
10th March 2017, 14:48 | #3 | Link | |
Registered User
Join Date: Sep 2003
Location: Berlin, Germany
Posts: 3,078
|
Here is a "poor man's" fps conversion function by johnmeyer.
jm_fps.avsi Quote:
Maybe you are interested in a standalone tool for slow motion using different methods for changing frame rates. Have a look here: https://forum.doom9.org/showthread.p...31#post1789031 Cheers manolito Last edited by manolito; 13th May 2018 at 21:34. Reason: improved params for MSuper |
|
10th March 2017, 21:06 | #4 | Link | |
Registered User
Join Date: Nov 2004
Location: Poland
Posts: 2,843
|
Quote:
|
|
10th March 2017, 22:07 | #5 | Link | |
Registered User
Join Date: Feb 2002
Location: California
Posts: 2,691
|
Quote:
You are absolutely correct, however, that it slows down the conversion (it is about 5x slower). |
|
24th March 2017, 17:46 | #6 | Link | |
Soul Architect
Join Date: Apr 2014
Posts: 2,559
|
Quote:
|
|
24th March 2017, 18:50 | #7 | Link |
Registered User
Join Date: Feb 2002
Location: California
Posts: 2,691
|
For the answer, read my post here.
That post also explains why some in this thread thought my script provided some good results: I was able to tweak some things which are not "exposed" with SVP and its Interframe front end. As for things said in this thread, I still am not convinced that any of the DCT settings are going to provide any substantial, real improvements, i.e., it won't produce differences that actually matter. Note I didn't say there wouldn't be differences, only that those differences won't really get at the reasons why motion estimation fails when doing frame rate changes or other operations which involve creating new frames from adjacent frames. The real issue is how to define "objects" and how to track them. The motion estimation done by any of these MVTools-derived filters relies on nothing more than tracking pre-determined blocks of pixels rather than pre-identifying actual objects in the frame. This is why block size is the most important variable to change when trying to get good results. Depending on the video and the size of the things being tracked (like people's legs, vertical fence posts, and other difficult-to-track items), different block sizes will work on some videos better than others. I find a block size of 16 to be a good starting point, but sometimes find 8 or 32 works better. The block overlap can then provide some fine tuning. In general, I only use this technology in conjunction with something else and then mix the two together. The reason is that other technologies, such as frame blending, never fail badly, but they also don't produce results that are as good as motion estimation, but only when motion estimation is behaving. Unfortunately, when motion estimation (including other tools like Twixtor) fails, it fails sepectacularly, ruining the viewing experience. You cannot rely on it. So, motion estimation is not a "set it, and forget it" tool, and if you use it that way for creating new frames, you will get burned, and it will be sooner rather than later. |
25th March 2017, 07:24 | #8 | Link | |
Registered User
Join Date: Apr 2002
Location: Germany
Posts: 4,926
|
Quote:
__________________
all my compares are riddles so please try to decipher them yourselves :) It is about Time Join the Revolution NOW before it is to Late ! http://forum.doom9.org/showthread.php?t=168004 Last edited by CruNcher; 25th March 2017 at 07:27. |
|
28th March 2017, 03:32 | #9 | Link |
Soul Architect
Join Date: Apr 2014
Posts: 2,559
|
I have made a quick test to compare Interframe(preset="smooth") with jm_fps. This is part of a script that does more processing, and does an extra frame double which makes the changes more visible. This is a test on very bad quality content to see how it behaves.
http://screenshotcomparison.com/comparison/204832 http://screenshotcomparison.com/comparison/204833 The difference is huge. It's much better than I expected. Worth investigating some more. Plus, it allows processing in YV24. Can the jm_fps script work in 16-bit, using either DitherTools hack or native AVS+ format? I did further tests in YV24, and strangely, it generates a LOT of strong artifacts!! This seems more like a bug. http://screenshotcomparison.com/comparison/204834
__________________
FrameRateConverter | AvisynthShader | AvsFilterNet | Natural Grounding Player with Yin Media Encoder, 432hz Player, Powerliminals Player and Audio Video Muxer Last edited by MysteryX; 28th March 2017 at 03:46. |
10th March 2017, 17:12 | #10 | Link |
Quality Checker
Join Date: Aug 2013
Posts: 284
|
and there this :
https://github.com/gdiaz384/frameTools
__________________
I love Doom9 |
11th March 2017, 00:17 | #11 | Link |
Registered User
Join Date: Nov 2004
Location: Poland
Posts: 2,843
|
It always produced less artefacts for me when doing fps conversion.
DCT=0 is also very bad on fades. You can use other modes if DCT=1 is to slow (eg. DCT=3). DCT=0 is the simplest mode and it's for speed not quality. |
11th March 2017, 09:26 | #12 | Link |
Registered User
Join Date: Sep 2003
Location: Berlin, Germany
Posts: 3,078
|
This made me curious so I dug out my old fps conversion torture clip.
I converted it to half speed using dct values of 0, 1 and 3. Download the resulting clips here: http://www23.zippyshare.com/v/L4HbbPzJ/file.html Conversion speed was 1.3 fps for dct=0, 0.3 fps for dct=1 and 1.0 fps for dct=3. Speed for dct=1 is forbiddingly slow, there is no way I will ever use this. And watching the results I have to agree with johnmeyer: Using a dct value of 0 produces less artifacts than using values of 1 or 3, and it is faster. Cheers manolito |
11th March 2017, 14:35 | #13 | Link | |
Registered User
Join Date: Nov 2004
Location: Poland
Posts: 2,843
|
Quote:
This is definitely not a case for me, but I'm using vs (but this should not matter). I also used different/simpler script, so maybe this is the reason (again shouldn't be). I've converted 500h worth of footage and done few days testing before this. Used DCT=3 at the end as a compromise. I will try this script, maybe it has some magic I have to find average settings for 500h worth of footage not ones which will work for e.g. 1min clip. I know DCT=0 will fail me on fades. Last edited by kolak; 11th March 2017 at 15:27. |
|
11th March 2017, 19:39 | #14 | Link |
Registered User
Join Date: Sep 2003
Location: Berlin, Germany
Posts: 3,078
|
Spent some more time tuning the script parameters for this particular test clip. Please keep in mind that this clip is extremely demanding, usually I would never recommend to use this script on anime.
What I found out is that a dct value of 3 does give better results on the control panel to the left, but it does not improve the vertical grille to the right. What does improve this grille is using a larger block size of 32 for MAnalyze. For MRecalculate I kept the original block size of 8, increasing it to 16 gave worse results. So for this particular clip my preferred settings are dct =3 and blksize = 32 for MAnalyse. The resulting file is here: http://www59.zippyshare.com/v/l6Ouh8fw/file.html In my experience any settings which work for this clip will also give good results for other sources. I need to test if the increased block size will indeed work for HD sources... Cheers manolito Last edited by manolito; 11th March 2017 at 20:17. |
11th March 2017, 20:29 | #15 | Link |
Registered User
Join Date: Nov 2004
Location: Poland
Posts: 2,843
|
I quickly tested this script on one of my samples and it's not any better than my one. Some frames are better, others worse.
I use block 32 with overlap 8 as this is better (again- for me) on HD sources. My sources are real videos- TV stuff. I also asked jackoneill to implement even bigger block sizes (for vs mvtools), but this seams to not improve quality as much as I expected. There are some other issues with this bigger block sizes which seams to be related to mvtools internals. Last edited by kolak; 12th March 2017 at 16:11. |
18th March 2017, 11:20 | #16 | Link | |
Registered User
Join Date: Apr 2002
Location: Germany
Posts: 4,926
|
Quote:
Though this indicates their must be a better result hiding in between if you split all the best parts and put them back together you would have a overall better result, from the timing this could even work. Temporal i can split this in 3 different parts that show different problems Wherby the amount of problems are different from each case. In this test case you could say you have 3 scene parts Beginning,Middle and the End part and depending on your setup now they come out different the beginning part never completely fixed the middle and end part with some very good overall results except in your latest result the end part comes out perceptually much worse then in any result before, therfore though the middle part is perceptually better (no fast texture failure). Logic now says take all the good parts and put them together and you have a pretty good overall result. Which also could indicate you need something more adaptively clever to get this result in 1 try,without putting it manually together My reasoning still stays dct 1 gives the overall best result except 1 difference now to the latest output therefore 1 part in the latest output is now totally ending up bad perceptually. i would also weight the temporal failure of the latest result in the end part higher even critical now then the small problem in the middle part on the dct 1 result before (fast texture failure).
__________________
all my compares are riddles so please try to decipher them yourselves :) It is about Time Join the Revolution NOW before it is to Late ! http://forum.doom9.org/showthread.php?t=168004 Last edited by CruNcher; 18th March 2017 at 12:17. |
|
11th March 2017, 11:30 | #17 | Link |
Registered User
Join Date: Apr 2002
Location: Germany
Posts: 4,926
|
dct 1_test_speed=50%.mkv
most stable temporal result on the first view in many problematic areas, most viewers would rate it the highest MOS of the 3 results 0 and 3 it becomes harder to percept the temporal difference but 1 is pretty obvious. I would predict everyone of the test viewers with ok visuals would rate dct 1 the most "annoying free result" That you don't percept the difference let me wonder about your overall script results Though overall all 3 have to much temporal problems and overall can only be rated BAD by everyone as such For Marketing i would present it side by side that way the difference would be very fast visible to even a higher amount of viewers and maybe at another 50 slowmo to make it even more obvious So overall you could also simplify it further in 2 presets Slow = 1 / Fast = 3 if Slow,Medium,Fast would be usable maybe but for the the overall testcase presented here i would say nope useless Which makes me wonder how AMDs,Intels and Nvidias FRC would compare here by now vs Mvtools or even Kronos
__________________
all my compares are riddles so please try to decipher them yourselves :) It is about Time Join the Revolution NOW before it is to Late ! http://forum.doom9.org/showthread.php?t=168004 Last edited by CruNcher; 11th March 2017 at 13:12. |
18th March 2017, 12:23 | #19 | Link |
Registered User
Join Date: Apr 2002
Location: Germany
Posts: 4,926
|
yes but overall this result could endup in a catastrophe now if it creates the same temporal issues for every source that heavy perceptual jitter is really annoying for moving objects like those angels it destroys everything and i really wonder how you can percept it now as better then anything before.
you fixed the texture problem yes but this is hyper annoying now in motion compared to that small glitch that was only visible mere milliseconds before. Good thing in the end you at last saw the problems on the panel with the light scan Also there are much much more problems to think about you using AVC currently as output you don't know yet how this will be perceived actually with HEVC/VP9 for example it could come out even worse (better perceptible). It partly though could be also get fixed in the other direction as internaly a hevc encoder is smarter then mvtools and you might throw out extra resources for nothing
__________________
all my compares are riddles so please try to decipher them yourselves :) It is about Time Join the Revolution NOW before it is to Late ! http://forum.doom9.org/showthread.php?t=168004 Last edited by CruNcher; 18th March 2017 at 13:00. |
19th March 2017, 15:06 | #20 | Link |
Registered User
Join Date: Sep 2003
Location: Berlin, Germany
Posts: 3,078
|
I don't know but I think you are being a little too critical. I showed the converted clips to some of my friends who are journalists working for Deutsche Welle TV. They do have trained eyes, but just like me they focus on the overall visual impression, and all of them agreed that the MVTools based conversions were quite pleasing to watch.
For this particular clip my ultimate conversion is here: http://www82.zippyshare.com/v/7bRTiwej/file.html dct=1 and blocksize for MAnalyse = 32. Way too slow, but I cannot find any annoying flaws. My main problem with these fps conversion methods are hard coded subs or movie credits. The letters mostly get warped to a point where they are painful to watch. But this is the only occasion where I would consider splitting up the source and convert the different parts separately... Cheers manolito |
Thread Tools | Search this Thread |
Display Modes | |
|
|