1920x1080 50i to 1280x720 25p with some slomo

[smok3]

ok so what i have is 1920x1080 50i which will be basically converted to 1280x720 25p prior to editing, like probably simply:

ffmpegsource("file.mp4")
yadif(mode=0, order=1)
resize(1280,720)

now, some part of the footage would require small amount of slomo (the amount is not determined yet, but 2-times slower would be to slow),

any recommendations for the filters/scripts/approaches?

something like this, field based as it seems:
http://avisynth.org/mediawiki/AlterFPS

or should i change to 50p first and then do the frame-blending magic of some sort?

[Alex_ander]

For the parts with slow motion (separated with Trim) I'd use:
1. mode 1 in Yadif for 50p (the higher initial framerate, the less artifacts in next step) -> downsize
2. MVFlowFPS of MVTools for reducing 50p to a framerate between 25 & 50
3. AssumeFPS(25)

[smok3]

Alex_ander: i'll give that a try. thanks.

[Blue_MiSfit]

If you're doing production quality work, don't use YADIF :)

Time should be of almost no object, so do yourself a favor and use something really, really good - like TempGaussMC :)

We don't all have Alchemists or Teranex VC300s, but we can do damned well with AviSynth, at the cost of speed!

There are going to be artifacts with MVFlowFPS, without a doubt, but it shouldn't be a *huge* deal.

Alex_ander's idea of bobbing first, and then reducing frame rate, followed by an AssumeFPS is perfectly sound.

Here's an example script, with some guessing. I don't have any 1080i stuff laying around, so you will probably need to tweak things a bit, especially the MVTools stuff, since I'm making a lot of guesses. The new experimental MVTools by josey_wells will probably improve speed a lot on multicore systems, but I can't find good documentation on its syntax.


#Bob
AssumeTFF()
TempGaussMC_beta1()

#Ensure sanity
dirty=last

#Generate subpixel clip
subpel=nnedi(0,true).turnright.nnedi(0,true).turnleft

#Generate clean clip
clean=fft3dgpu(plane=4, sigma=4)

#Generate clean motion vectors
bv = clean.MVAnalyse(isb = true, overlap=4, pel=2, pelclip=subpel, idx=1)
fv = clean.MVAnalyse(isb = false, overlap=4, pel=2, pelclip=subpel, idx=1)

#Re-generate motion vectors
re_fv = MVRecalculate(dirty, fv, overlap=4, pel=2, pelclip=subpel, thSAD=100, idx=2)
re_bv = MVRecalculate(dirty, bv, overlap=4, pel=2, pelclip=subpel, thSAD=100, idx=2)


#Flow FPS to 37.5 as an example
MVFlowFps(re_bv, re_fv, num=75, den=2, ml=100, idx=2)

#Make PAL
AssumeFPS(25)


That might make your computer explode, but it might also work!

:D

~MiSfit

[smok3]

Blue_MiSfit, tnx, but, but just by looking at this i see it would take months to transcode... :(

[Blue_MiSfit]

Probably - it depends on how much footage needs the fancy-pants slow motion effects, and how fast your computer is :D

[smok3]

how about something less fancy;

ffmpegsource("file.mp4")
yadif(mode=1, order=1)

convertfps(40)
assumefps(25)

resize(1280,720)

or bit more advanced

ffmpegsource("file.mp4")
yadif(mode=1, order=1)
FFT3DGPU(sigma=6, sharpen=0.3)

ConvertFPS(33.5)
assumefps(25)

resize(1280,720)

[Alex_ander]

Convertfps() will drop frames here and since input/output frequencies are not in multiple relationship, motion may lose smoothness. In case of 25p -> 40 blended frames would be added, also not much regularly. While MVFlowFPS rebuilds all the frames for equidistant displaying (the example from its docs usually works fairly without adjustment).
To make bob work faster it's useful to previously downsize horizontally (which is safe) and just after bob - vertically.

[smok3]

ok, i guess i need to find some nasty pans to actually test the smoothness, i will report back.

[2Bdecided]

It's where objects move behind other objects, especially with some transparency, that mvtools stuff can fall apart IME.

You don't have to use sub-pixel motion estimation. You don't have to use the world's best bob.

However, convertfps really isn't up to re-timing smooth pans!

Cheers,
David.

[smok3]

this

ffmpegsource("film.mp4")
yadif(mode=1, order=1)
FFT3DGPU(sigma=6, sharpen=0.3)
source=last

# mvtools from manual
backward_vec = source.MVAnalyse(isb = true, truemotion=true, pel=2, idx=1)
# we use explicit idx for more fast processing
forward_vec = source.MVAnalyse(isb = false, truemotion=true, pel=2, idx=1)
source.MVFlowFps(backward_vec, forward_vec, num=33, den=1, ml=100, idx=1) # get 33 fps?
# end mvtools

assumefps(25)

resize(1280,720)

goes with a blazing speed of 0.44 fps (not sure if it is correct even), so in best scenario (if i find mt version), then i would still get less than 1fps (which is not acceptable).

p.s.1. pans can probably be 'corrected' with skipping them.

p.s.2. changing vector parameters to pel=1, truemotion=false doesn't change speed.

[Alex_ander]

Move the resizer here (mvtools will also benefit from this):

LanczosResize(1280, last.height)
yadif(mode=1, order=1)
LanczosResize(last.width, 720)

[smok3]

ok, now i'am here;

ffmpegsource("file.mp4")
resize(1280, 1080)
yadif(mode=1, order=1)
#FFT3DGPU(sigma=6, sharpen=0.3)
resize(1280,720)
source=last

# mvtools from manual
backward_vec = source.MVAnalyse(isb = true, truemotion=false, pel=1, idx=1)
# we use explicit idx for more fast processing
forward_vec = source.MVAnalyse(isb = false, truemotion=false, pel=1, idx=1)
source.MVFlowFps(backward_vec, forward_vec, num=33, den=1, ml=100, idx=1) # get 33 fps?
# end mvtools

assumefps(25)

2.55 fps, now thats supersonic :D

p.s.1. yes the change in bober/resize order did a big difference, geting the same speed with truemotion=true now (2.55 fps).
p.s.2. chaning pel to 2 goes to 2 fps.
p.s.3. will test this now on the other machine, which should be a bit faster.

p.s.4. note: additional problem here would be background noise, so maybe where kamera is relatively static the convertfps may work better?
p.s.5. this breaks pretty badly when motion is conflicting, say longangle camera following actor with some books and stuff in the back.

[Blue_MiSfit]

That's where better motion compensation helps :)

Are you working on a multicore machine? If so, you *need* to figure out how to get MVTools multithreaded!

~MiSfit

[2Bdecided]

Re-thinking this, since both 1x and 2x can be done perfectly, but you want something in between, maybe convertfps to give 1.5x - set somehow so that only half the frames are blended - won't look too bad. It'll be a bit stuttery and a bit blur/clear/blur/clear - but that might be preferable to mvflowfps artefacts.

So for an input of frames
1,2,3,4,5,6,7,8,9,etc
you'd get an output of frames
1,2+3,4,5+6,7,8+9,etc

where "+" = "blend"

Cheers,
David.

[smok3]

yes i have tried both, the blended version looks like the actor is drunk, the mv version looks like it is from another planet, hard to say what is better.

yes, i have dual-core opteron here.

so my options for shots that wont pass are:
a. skip the slomo completely, just 25fps them
b. do dual vector analisis and get old
c. somehow get very smart and set the mvtools to work multithreaded and then get old (edit: actually the easy way out would be to simply run two klips at once, solved)

p.s. is it possible to make mvtools to detect oppositing motion and skip to some softer mode there?

[Didée]

the blended version looks like the actor is drunk, the mv version looks like it is from another planet, hard to say what is better.

LOL, touche! The saying of today, thank you. :D

In MVFlowFPS there's the "ml" parameter to tweak the sensitivity against occlusion. On deteced occlusion areas MVFLowFPS should use frame blending instead of interpolation. Default is ml=100, lower switches to blending earlier, bigger switches later.

"thSAD" used to be another parameter to balance carefulness vs. aggressivity, but seemingly has been disabled since MVTools v1.8.


Also, browsing through the "Alchemist" thread (http://forum.doom9.org/showthread.php?t=113256&highlight=alchemist) will bring up a few scripts that [try to] put more effort into avoiding aliens, amoebic colitis and fractal art in the output. ;)

[smok3]

Didée, tnx i'll give ml a try, other than that i got one sequence together, edited it and it looks good.
(slomo done with my last script, unfortunatelly this 2nd machine (dual-core opteron) is no faster than 5 years old dual xeon, weird...).

[vampiredom]

If MVTools is too slow or otherwise impractical, you may simply want to try using BlendFPS (from the Motion plugin) for the slomo. I also suggest using a combination of TomsMoComp and YadifMod, which seems to me to give better stability than Yadif.

Something like:

ffmpegsource("file.mp4")
AssumeTFF() # or AssumeBFF(), as needed

# padding for mod16 width and height (seems to be required by TomsMoComp on some CPUs?)
w16 = ceil(float(width()) / 16.0) * 16
h16 = ceil(float(height()) / 16.0) * 16
bL = (w16 - width()) / 2
bR = (bL % 2 != 0) ? bL - 1 : bL
bL = (bL % 2 != 0) ? bL + 1 : bL
bT = (h16 - height()) / 2
bB = (bT % 2 != 0) ? bT - 1 : bT
bT = (bT % 2 != 0) ? bT + 1 : bT
(bL > 0 || bR > 0 || bT > 0 || bB > 0) ? AddBorders(bL, bT, bR, bB) : last

# do bobbing
yadifmod(mode=1, edeint=Interleave(TomsMoComp(-1, 0, 0), DoubleWeave().SelectOdd().TomsMoComp(-1, 0, 0)))

# remove any mod16 padding
(bL > 0 || bR > 0 || bT > 0 || bB > 0) ? Crop(bL, bT, bR * -1, bB * -1) : last

# do slomo and/or change framerate
BlendFPS(33).AssumeFPS(25) # or simply use SelectEven() instead for non-slomo 25 fps

# resize for final output
Spline36Resize(1280,720)

[Blue_MiSfit]

Agreed. YadifMod can be a lot better than vanilla Yadif. Interesting choice with TomsMoComp, I haven't played with that deinterlacer in years.

Other options would be TDeint, with or without TMM and / or NNEDI assistance..

Multithreading shouldn't be too hard to get working - just install the latest revision of it, and add SetMTMode(2) to the top of your script :) It will help a lot!

-Derek

[vampiredom]

Actually, Yadifmod + EEDI2 is my favorite "medium quality" deinterlacer these days. (TempGaussMC is just too slow for me). Unlike NNEDI, EEDI2 has the variable "maxd" parameter. Tweaking this can greatly help the quality -- and lower values (12 or so) can provide very good diagonals at a reasonable rendering time.

However, TomsMoComp (with mocomp disabled) does a fine job and is very quick. It's a little bit lacking for 1:1, but for reduced size cases such as 1080->720 or 1080->SD it is excellent.

Other options would be TDeint, with or without TMM and / or NNEDI assistance..