Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 10th March 2017, 13:22   #1  |  Link
chilledinsanity
Registered User
 
chilledinsanity's Avatar
 
Join Date: Jan 2002
Posts: 223
SVP-like frame interpolation?

I'm almost certain this has been answered elsewhere, but I had difficulty finding an answer. The software SVP (Smooth Video Project) does a pretty impressive job of generating frames based on motion in order to increase the framerate of content, however its focus is for real-time playback. Is there some rough equivalent of this for Avisynth or some other software to generate the frames as a new video; not in real-time, but as some sort of script to be processed?

So say I had a 30fps clip that I wanted to convert to 60fps (or slow down more for slow motion), but instead of doubling frames, I wanted the computer to use motion analysis to take its best guess in generating new ones. How would I go about doing that? Please feel free to dumb this down for me (like posting a sample script), I often get hung up on simple syntax errors.

Thanks in advance.
chilledinsanity is offline   Reply With Quote
Old 10th March 2017, 13:32   #2  |  Link
thecoreyburton
Registered User
 
Join Date: Jul 2015
Posts: 101
I think this might be what you're looking for.
thecoreyburton is offline   Reply With Quote
Old 10th March 2017, 14:48   #3  |  Link
manolito
Registered User
 
manolito's Avatar
 
Join Date: Sep 2003
Posts: 2,112
Here is a "poor man's" fps conversion function by johnmeyer.

jm_fps.avsi
Quote:
# Motion Protected FPS converter script by johnmeyer from Doom9
# Slightly modified interface by manolito
# Requires MVTools V2 and RemoveGrain
# Also needs fftw3.dll in the System32 or SysWOW64 folder for Dct values other than 0


function jm_fps(clip source, float "fps", int "BlkSize", int "Dct")
{
fps = default(fps, 25.000)
fps_num = int(fps * 1000)
fps_den = 1000
BlkSize = default(BlkSize, 16)
Dct = default(Dct, 0)

prefiltered = RemoveGrain(source, 22)
super = MSuper(source, hpad = 16, vpad = 16, levels = 1) # one level is enough for MRecalculate
superfilt = MSuper(prefiltered, hpad = 16, vpad = 16) # all levels for MAnalyse
backward = MAnalyse(superfilt, isb = true, blksize = BlkSize, overlap = 4, search = 3, dct = Dct)
forward = MAnalyse(superfilt, isb = false, blksize = BlkSize, overlap = 4, search = 3, dct = Dct)
forward_re = MRecalculate(super, forward, blksize = 8, overlap = 2, thSAD = 100)
backward_re = MRecalculate(super, backward, blksize = 8, overlap = 2, thSAD = 100)
out = MFlowFps(source, super, backward_re, forward_re, num = fps_num, den = fps_den, blend = false, ml = 200, mask = 2)

return out
}
Motion based fps conversion will always introduce some artifacts, but with these parameters johnmeyer really had a golden hand. I did test many different scripts for this, but this one gave by far the best results on all the clips I used it for.

Maybe you are interested in a standalone tool for slow motion using different methods for changing frame rates. Have a look here:
https://forum.doom9.org/showthread.p...31#post1789031


Cheers
manolito

Last edited by manolito; 7th October 2017 at 19:40.
manolito is offline   Reply With Quote
Old 10th March 2017, 17:12   #4  |  Link
amayra
Quality Checker
 
amayra's Avatar
 
Join Date: Aug 2013
Posts: 194
and there this :
https://github.com/gdiaz384/frameTools
__________________
I love Doom9
amayra is offline   Reply With Quote
Old 10th March 2017, 21:06   #5  |  Link
kolak
Registered User
 
Join Date: Nov 2004
Location: UK
Posts: 2,264
Quote:
Originally Posted by manolito View Post
Here is a "poor man's" fps conversion function by johnmeyer.

jm_fps.avsi


Motion based fps conversion will always introduce some artifacts, but with these parameters johnmeyer really had a golden hand. I did test many different scripts for this, but this one gave by far the best results on all the clips I used it for.

Maybe you are interested in a standalone tool for slow motion using different methods for changing frame rates. Have a look here:
https://forum.doom9.org/showthread.p...31#post1789031


Cheers
manolito
DCT=1 will definitely help further, but slow down conversion a lot also
kolak is offline   Reply With Quote
Old 10th March 2017, 22:07   #6  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 1,960
Quote:
Originally Posted by kolak View Post
DCT=1 will definitely help further, but slow down conversion a lot also
Are you sure? I've never found that it did anything to reduce artifacts when doing slow motion. I find it useful to reduce flicker, if that is a problem.

You are absolutely correct, however, that it slows down the conversion (it is about 5x slower).
johnmeyer is offline   Reply With Quote
Old 11th March 2017, 00:17   #7  |  Link
kolak
Registered User
 
Join Date: Nov 2004
Location: UK
Posts: 2,264
It always produced less artefacts for me when doing fps conversion.
DCT=0 is also very bad on fades. You can use other modes if DCT=1 is to slow (eg. DCT=3). DCT=0 is the simplest mode and it's for speed not quality.
kolak is offline   Reply With Quote
Old 11th March 2017, 09:26   #8  |  Link
manolito
Registered User
 
manolito's Avatar
 
Join Date: Sep 2003
Posts: 2,112
This made me curious so I dug out my old fps conversion torture clip.

I converted it to half speed using dct values of 0, 1 and 3. Download the resulting clips here:
http://www23.zippyshare.com/v/L4HbbPzJ/file.html

Conversion speed was 1.3 fps for dct=0, 0.3 fps for dct=1 and 1.0 fps for dct=3.

Speed for dct=1 is forbiddingly slow, there is no way I will ever use this.

And watching the results I have to agree with johnmeyer: Using a dct value of 0 produces less artifacts than using values of 1 or 3, and it is faster.


Cheers
manolito
manolito is offline   Reply With Quote
Old 11th March 2017, 11:30   #9  |  Link
CruNcher
Registered User
 
CruNcher's Avatar
 
Join Date: Apr 2002
Location: Germany
Posts: 4,949
dct 1_test_speed=50%.mkv

most stable temporal result on the first view in many problematic areas, most viewers would rate it the highest MOS of the 3 results 0 and 3 it becomes harder to percept the temporal difference but 1 is pretty obvious.

I would predict everyone of the test viewers with ok visuals would rate dct 1 the most "annoying free result"

That you don't percept the difference let me wonder about your overall script results

Though overall all 3 have to much temporal problems and overall can only be rated BAD by everyone as such

For Marketing i would present it side by side that way the difference would be very fast visible to even a higher amount of viewers and maybe at another 50 slowmo to make it even more obvious


So overall you could also simplify it further in 2 presets

Slow = 1 / Fast = 3

if Slow,Medium,Fast would be usable maybe but for the the overall testcase presented here i would say nope useless

Which makes me wonder how AMDs,Intels and Nvidias FRC would compare here by now vs Mvtools or even Kronos
__________________
all my compares are riddles so please try to decipher them yourselves :)

It is about Time

Join the Revolution NOW before it is to Late !

http://forum.doom9.org/showthread.php?t=168004

Last edited by CruNcher; 11th March 2017 at 13:12.
CruNcher is offline   Reply With Quote
Old 11th March 2017, 14:35   #10  |  Link
kolak
Registered User
 
Join Date: Nov 2004
Location: UK
Posts: 2,264
Quote:
Originally Posted by manolito View Post
This made me curious so I dug out my old fps conversion torture clip.

I converted it to half speed using dct values of 0, 1 and 3. Download the resulting clips here:
http://www23.zippyshare.com/v/L4HbbPzJ/file.html

Conversion speed was 1.3 fps for dct=0, 0.3 fps for dct=1 and 1.0 fps for dct=3.

Speed for dct=1 is forbiddingly slow, there is no way I will ever use this.

And watching the results I have to agree with johnmeyer: Using a dct value of 0 produces less artifacts than using values of 1 or 3, and it is faster. I know DCT=0 will fail me on fades.


Cheers
manolito
That's something odd if DCT=0 produces less artefacts (also- try fades with DCT=0).

This is definitely not a case for me, but I'm using vs (but this should not matter). I also used different/simpler script, so maybe this is the reason (again shouldn't be).

I've converted 500h worth of footage and done few days testing before this. Used DCT=3 at the end as a compromise.

I will try this script, maybe it has some magic I have to find average settings for 500h worth of footage not ones which will work for e.g. 1min clip. I know DCT=0 will fail me on fades.

Last edited by kolak; 11th March 2017 at 15:27.
kolak is offline   Reply With Quote
Old 11th March 2017, 19:39   #11  |  Link
manolito
Registered User
 
manolito's Avatar
 
Join Date: Sep 2003
Posts: 2,112
Spent some more time tuning the script parameters for this particular test clip. Please keep in mind that this clip is extremely demanding, usually I would never recommend to use this script on anime.

What I found out is that a dct value of 3 does give better results on the control panel to the left, but it does not improve the vertical grille to the right. What does improve this grille is using a larger block size of 32 for MAnalyze. For MRecalculate I kept the original block size of 8, increasing it to 16 gave worse results.

So for this particular clip my preferred settings are dct =3 and blksize = 32 for MAnalyse. The resulting file is here:
http://www59.zippyshare.com/v/l6Ouh8fw/file.html

In my experience any settings which work for this clip will also give good results for other sources. I need to test if the increased block size will indeed work for HD sources...


Cheers
manolito

Last edited by manolito; 11th March 2017 at 20:17.
manolito is offline   Reply With Quote
Old 11th March 2017, 20:29   #12  |  Link
kolak
Registered User
 
Join Date: Nov 2004
Location: UK
Posts: 2,264
I quickly tested this script on one of my samples and it's not any better than my one. Some frames are better, others worse.
I use block 32 with overlap 8 as this is better (again- for me) on HD sources. My sources are real videos- TV stuff.
I also asked jackoneill to implement even bigger block sizes (for vs mvtools), but this seams to not improve quality as much as I expected. There are some other issues with this bigger block sizes which seams to be related to mvtools internals.

Last edited by kolak; 12th March 2017 at 16:11.
kolak is offline   Reply With Quote
Old 18th March 2017, 10:24   #13  |  Link
chilledinsanity
Registered User
 
chilledinsanity's Avatar
 
Join Date: Jan 2002
Posts: 223
Thanks, I'll experiment with this.
chilledinsanity is offline   Reply With Quote
Old 18th March 2017, 11:20   #14  |  Link
CruNcher
Registered User
 
CruNcher's Avatar
 
Join Date: Apr 2002
Location: Germany
Posts: 4,949
Quote:
Originally Posted by manolito View Post
Spent some more time tuning the script parameters for this particular test clip. Please keep in mind that this clip is extremely demanding, usually I would never recommend to use this script on anime.

What I found out is that a dct value of 3 does give better results on the control panel to the left, but it does not improve the vertical grille to the right. What does improve this grille is using a larger block size of 32 for MAnalyze. For MRecalculate I kept the original block size of 8, increasing it to 16 gave worse results.

So for this particular clip my preferred settings are dct =3 and blksize = 32 for MAnalyse. The resulting file is here:
http://www59.zippyshare.com/v/l6Ouh8fw/file.html

In my experience any settings which work for this clip will also give good results for other sources. I need to test if the increased block size will indeed work for HD sources...


Cheers
manolito
This version introduced many new perceptable problems though you fought 1 problem and created a whole bunch of new ones which would most probably result now in a even lower mos score then before (predicted).

Though this indicates their must be a better result hiding in between

if you split all the best parts and put them back together you would have a overall better result, from the timing this could even work.


Temporal i can split this in 3 different parts that show different problems

Wherby the amount of problems are different from each case.

In this test case you could say you have 3 scene parts Beginning,Middle and the End part

and depending on your setup now they come out different the beginning part never completely fixed the middle and end part with some very good overall results except in your latest result the end part comes out perceptually much worse then in any result before, therfore though the middle part is perceptually better (no fast texture failure).

Logic now says take all the good parts and put them together and you have a pretty good overall result.

Which also could indicate you need something more adaptively clever to get this result in 1 try,without putting it manually together


My reasoning still stays dct 1 gives the overall best result except 1 difference now to the latest output therefore 1 part in the latest output is now totally ending up bad perceptually.

i would also weight the temporal failure of the latest result in the end part higher even critical now then the small problem in the middle part on the dct 1 result before (fast texture failure).
__________________
all my compares are riddles so please try to decipher them yourselves :)

It is about Time

Join the Revolution NOW before it is to Late !

http://forum.doom9.org/showthread.php?t=168004

Last edited by CruNcher; 18th March 2017 at 12:17.
CruNcher is offline   Reply With Quote
Old 18th March 2017, 11:53   #15  |  Link
manolito
Registered User
 
manolito's Avatar
 
Join Date: Sep 2003
Posts: 2,112
Thanks for looking into it...

Quote:
I need to test if the increased block size will indeed work for HD sources...
Well, I found that it does not...

The best results for this clip came from using dct=1, but this is so slow that it is not really usable for me.

Of course you can optimize the results by finetuning the params for each and every source, maybe even split the source into several parts. But this is not what I'm after. I want a script which works well for the vast majority of sources without tweaking the params.

And I believe that johnmeyer's original params do an excellent job for most sources (maybe use dct=3 instead of 0). Way better than MotionProtectedFPS. And keep in mind that I am not interested in watching still frames. I want to just watch the movie, and it should look better than using ChangeFPS or ConvertFPS. I do not mind some motion artifacts as long as they do not get too annoying. And I use this only on film, not on anime. Plus I mostly use this type of fps conversion for PAL <-> NTSC conversion, not for slow motion.


Cheers
manolito
manolito is offline   Reply With Quote
Old 18th March 2017, 12:23   #16  |  Link
CruNcher
Registered User
 
CruNcher's Avatar
 
Join Date: Apr 2002
Location: Germany
Posts: 4,949
yes but overall this result could endup in a catastrophe now if it creates the same temporal issues for every source that heavy perceptual jitter is really annoying for moving objects like those angels it destroys everything and i really wonder how you can percept it now as better then anything before.

you fixed the texture problem yes but this is hyper annoying now in motion compared to that small glitch that was only visible mere milliseconds before.

Good thing in the end you at last saw the problems on the panel with the light scan

Also there are much much more problems to think about you using AVC currently as output you don't know yet how this will be perceived actually with HEVC/VP9 for example it could come out even worse (better perceptible).

It partly though could be also get fixed in the other direction as internaly a hevc encoder is smarter then mvtools and you might throw out extra resources for nothing
__________________
all my compares are riddles so please try to decipher them yourselves :)

It is about Time

Join the Revolution NOW before it is to Late !

http://forum.doom9.org/showthread.php?t=168004

Last edited by CruNcher; 18th March 2017 at 13:00.
CruNcher is offline   Reply With Quote
Old 19th March 2017, 15:06   #17  |  Link
manolito
Registered User
 
manolito's Avatar
 
Join Date: Sep 2003
Posts: 2,112
I don't know but I think you are being a little too critical. I showed the converted clips to some of my friends who are journalists working for Deutsche Welle TV. They do have trained eyes, but just like me they focus on the overall visual impression, and all of them agreed that the MVTools based conversions were quite pleasing to watch.

For this particular clip my ultimate conversion is here:
http://www82.zippyshare.com/v/7bRTiwej/file.html

dct=1 and blocksize for MAnalyse = 32. Way too slow, but I cannot find any annoying flaws.

My main problem with these fps conversion methods are hard coded subs or movie credits. The letters mostly get warped to a point where they are painful to watch. But this is the only occasion where I would consider splitting up the source and convert the different parts separately...


Cheers
manolito
manolito is offline   Reply With Quote
Old 19th March 2017, 20:08   #18  |  Link
kolak
Registered User
 
Join Date: Nov 2004
Location: UK
Posts: 2,264
MvTools are almost as good as Alchemist or Tachyon which are used in braodcast and cost small fortune.
It just would be nice to have it ported to GPU, like svp does. SVP made to many speed shortcuts so quality is not as good.
kolak is offline   Reply With Quote
Old 22nd March 2017, 04:07   #19  |  Link
CruNcher
Registered User
 
CruNcher's Avatar
 
Join Date: Apr 2002
Location: Germany
Posts: 4,949
Quote:
Originally Posted by manolito View Post
I don't know but I think you are being a little too critical. I showed the converted clips to some of my friends who are journalists working for Deutsche Welle TV. They do have trained eyes, but just like me they focus on the overall visual impression, and all of them agreed that the MVTools based conversions were quite pleasing to watch.

For this particular clip my ultimate conversion is here:
http://www82.zippyshare.com/v/7bRTiwej/file.html

dct=1 and blocksize for MAnalyse = 32. Way too slow, but I cannot find any annoying flaws.

My main problem with these fps conversion methods are hard coded subs or movie credits. The letters mostly get warped to a point where they are painful to watch. But this is the only occasion where I would consider splitting up the source and convert the different parts separately...


Cheers
manolito
Slowly we getting there
beginning = nice
middle = nice (with a small new glitch instead of the complete texture fail we see a big shift now)
end = horrible (full of motion perceptible fails, heat haze effect)

__________________
all my compares are riddles so please try to decipher them yourselves :)

It is about Time

Join the Revolution NOW before it is to Late !

http://forum.doom9.org/showthread.php?t=168004

Last edited by CruNcher; 22nd March 2017 at 04:18.
CruNcher is offline   Reply With Quote
Old 22nd March 2017, 12:46   #20  |  Link
manolito
Registered User
 
manolito's Avatar
 
Join Date: Sep 2003
Posts: 2,112
Quote:
end = horrible (full of motion perceptible fails, heat haze effect)
Are you talking about the three small angels in the background?

The human brain works differently. Everybody just sees the two large angels in the foreground, and they do not display motion artifacts. The information in the background is pretty much discarded because the brain determines that this information is not important. (This is gone by the "Formatio Reticularis" which works like a spotlight).


Cheers
manolito
manolito is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 10:19.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.