FrameRateConverter (Official)

[Seedmanc]

The video is already in YV12, it was only put there to demonstrate that.

[poisondeathray]

what avs version? check your plugins again or load them explicitly - maybe you have multiple versions

[StainlessS]

Run AvsMeter against your installation, and see what version, and what plugs you got.
Today, G2K4 got a nice shiney new version to play with.

[Groucho2004]

Originally Posted by StainlessS

Run AvsMeter against your installation, and see what version, and what plugs you got.

More precisely - Run "avsmeter avsinfo -log" and post the log file ("avsinfo_x86.log" or "avsinfo_x64.log").

[Seedmanc]

Avisynth 2.6, the one with SetMTmode.
Do masktools and mvtools have their own version() function? Such large plugin packs with lots of dependencies should have that so you don't have to check it manually in plugin directory every time to see which one got loaded.

When I just call MMask(last) (mmask is among the offending functions in that line that FRC is complaining about) I get a strange "Access violation /syswow64/ntdll.dll".

Here's log, compressed. Why does everything has to be so difficult here in 2018.

[StainlessS]

Seedmanc. dont think you log is likely to overflow Usage Forum post limit of 16KB (Dev forum=20KB).

Awaiting mods to approve.

[Groucho2004]

Originally Posted by Seedmanc

Here's log, compressed. Why does everything has to be so difficult here in 2018.

The size of the log is not the issue, the time it takes for a moderator to approve your attachment is. You'd be better off posting it on pastebin or similar.

[Seedmanc]

I tried it on a different OS with a fresh installation of AVS+, still getting errors. https://pastebin.com/5Kz7Wsft log
Depending on the masktools version the errors are different:
MT a48 : mt_inpand unsupported colorspace, only planar YUV, etc, line 164
MT pinterf : mt_merge clips should have identical bit depths, line 189
Was this even tested?

[Groucho2004]

Originally Posted by Seedmanc

I tried it on a different OS with a fresh installation of AVS+, still getting errors. https://pastebin.com/5Kz7Wsft log
Depending on the masktools version the errors are different:
MT a48 : mt_inpand unsupported colorspace, only planar YUV, etc, line 164
MT pinterf : mt_merge clips should have identical bit depths, line 189
Was this even tested?

Delete "mt_masktools-25.dll" and try again.

[Seedmanc]

That helped, thank you very much.

But it looks like this plugin just blends frames together most of the time. Is there a configuration example I can use to remove "safety locks" so it never resorts to blending and instead shows its best attempt at interpolation even if it's not passing some good metrics?

[StainlessS]

You wann give those that use this script, some idea what you are trying to do, low framerate up to super duper framerate, perhaps ?

[Lirk]

0 value doesn't disable BlendOver (and maybe SkipOver) parameters. 0 value just sets threshold to 0, but not disable this parameters.

[Seedmanc]

If merely doubling to 48-60fps counts as super for this script then my point still stands.

I'd rather see all the artifacts the script produces instead of the ghosting doubleframes, it would at least allow to compare to the artifacting other scripts do, to see which gets closer to the original (assuming I can take 60 fps, halve the framerate and try to reconstruct dropped frames). The way it is now the only measurement I can do is about its speed.

Here are the examples, one from the Sintel clip http://www.framecompare.com/image-co...rison/0J9MNNNU the other from idolmaster ingame footage http://www.framecompare.com/image-co...rison/WD7PNNNX for more 2D-ish kind of material. In both cases I had to use the "flow" output of FRC to get at least something similar to motion interpolation instead of pure doubling like ConvertFPS on auto. But even then the simple mvtools with recalculation did better. I tried setting various threshold params of FRC to 0 as the readme says but it didn't help.

[zorr]

Originally Posted by Seedmanc

...it would at least allow to compare to the artifacting other scripts do, to see which gets closer to the original (assuming I can take 60 fps, halve the framerate and try to reconstruct dropped frames). The way it is now the only measurement I can do is about its speed.

...

But even then the simple mvtools with recalculation did better. I tried setting various threshold params of FRC to 0 as the readme says but it didn't help.

You could try AvisynthOptimizer for this task, it can search optimal settings for you (semi)automatically. If you're interested I can provide you a frame-doubling MVTools script that AvisynthOptimizer can work with. Should be easy to make one for FramerateConverter as well. Warning: running this optimization takes a long time (hours, maybe days if you get really serious about it) and is very addictive.

[Seedmanc]

zorr, as a matter of fact yes I was going to try your tool for a while already, been watching your topic closely, waiting till it 'stabilizes' more or less about parameters and software involved. Maybe it's about time I give it a try.
I've been extensively testing various framerate conversion plugins recently,trying to figure out the best settings, but for now I've ran out of options (seems the pinterf's 64x64 does the trick better than the rest, though I had my hopes high for SVP). Particularly I'm interested in figuring out the usable truemotion parameters for blocksizes 32 and above, as they seem to be incompatible.

[zorr]

Originally Posted by Seedmanc

zorr, as a matter of fact yes I was going to try your tool for a while already, been watching your topic closely, waiting till it 'stabilizes' more or less about parameters and software involved. Maybe it's about time I give it a try.

While the optimizer is still officially beta I don't expect any major changes anymore. Please do try it and if you have any problems I will try to help. Your feedback on any issues you may have will be valuable.

Originally Posted by Seedmanc

I've been extensively testing various framerate conversion plugins recently,trying to figure out the best settings, but for now I've ran out of options (seems the pinterf's 64x64 does the trick better than the rest, though I had my hopes high for SVP). Particularly I'm interested in figuring out the usable truemotion parameters for blocksizes 32 and above, as they seem to be incompatible.

All right, here's a frame-doubling script using Pinterf's MVTools2.

TEST_FRAMES = 10
MIDDLE_FRAME = 25

# original framerate
FPS_NUM = 18
FPS_DEN = 1

# source clip
AVISource("d:\optimizer\bridge\bridge_cropped.avi")
AssumeFPS(FPS_NUM, FPS_DEN) 

#return last

# needed for some parameter combinations
ConvertToYV24()

orig = last

# preprocessing - blur and/or degrain
#searchClip = orig
#searchClip = last.Blur(1.58).Blur(1.58)
searchClip = RemoveGrain(orig, 22)		# optimize RemoveGrain(orig, _n_) | -1..24 | rmgrain


super_pel = 4					# optimize super_pel = _n_ | 2,4 | super_pel
super_sharp = 2					# optimize super_sharp = _n_ | 0..2 | super_sharp
super_rfilter = 2				# optimize super_rfilter = _n_ | 0..4 | super_rfilter
super_search = MSuper(pel=super_pel, sharp=super_sharp, rfilter=super_rfilter, searchClip)		
super_render = MSuper(pel=super_pel, sharp=super_sharp, rfilter=super_rfilter, orig, levels=1)	

blockSize = 32					# optimize blockSize = _n_ | 4,6,8,12,16,24,32,48,64 ; min:divide 0 > 8 2 ? ; filter:overlap overlapv max 2 * x <= | blockSize
searchAlgo = 5					# optimize searchAlgo = _n_ | 0..7 D | searchAlgo
searchRange = 2					# optimize searchRange = _n_ | 1..30 | searchRange
searchRangeFinest = 2				# optimize searchRangeFinest = _n_ | 1..60 | searchRangeFinest
lambda = 1000*(blockSize*blockSize)/(8*8)	# optimize lambda = _n_ | 0..20000 | lambda
lsad=1200					# optimize lsad=_n_ | 8..20000 | LSAD
pnew=0						# optimize pnew=_n_ | 0..256 | pnew
plevel=1					# optimize plevel=_n_ | 0..2 | plevel
overlap=0					# optimize overlap=_n_ | 0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32 ; max:blockSize 2 / ; filter:x divide 0 > 4 2 ? % 0 == | overlap
overlapv=0					# optimize overlapv=_n_ | 0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32 ; max:blockSize 2 / | overlapv
divide=0					# optimize divide=_n_ | 0..2 ; max:blockSize 8 >= 2 0 ? overlap 4 % 0 == 2 0 ? min | divide
globalMotion = true				# optimize globalMotion = _n_ | false,true | globalMotion
badSAD = 10000					# optimize badSAD = _n_ | 4..10000 | badSAD
badRange = 24					# optimize badRange = _n_ | 4..50 | badRange
meander = true					# optimize meander = _n_ | false,true | meander
temporal = false				# optimize temporal = _n_ | false,true | temporal
trymany = false					# optimize trymany = _n_ | false,true | trymany
dct = 0						# optimize dct = _n_ | 0,2,3,4,5,6,7,8,9,10 | dct

delta = 1
useChroma = true
bv = MAnalyse(super_search, isb = true, blksize=blockSize, search=searchAlgo, searchparam=searchRange, pelsearch=searchRangeFinest, 
\ chroma=useChroma, delta=delta, lambda=lambda, lsad=lsad, pnew=pnew, plevel=plevel, global=globalMotion, overlap=overlap, overlapv=overlapv, 
\ divide=divide, badSAD=badSAD, badrange=badRange, meander=meander, temporal=temporal, trymany=trymany, dct=dct)
fv = MAnalyse(super_search, isb = false, blksize=blockSize, search=searchAlgo, searchparam=searchRange, pelsearch=searchRangeFinest, 
\ chroma=useChroma, delta=delta, lambda=lambda, lsad=lsad, pnew=pnew, plevel=plevel, global=globalMotion, overlap=overlap, overlapv=overlapv, 
\ divide=divide, badSAD=badSAD, badrange=badRange, meander=meander, temporal=temporal, trymany=trymany, dct=dct)



threshold = 10000 
maskScale = 70					# optimize maskScale = _n_ | 1..300 | maskScale
mask_fps = 3					# optimize mask_fps = _n_ | 0..2 | mask_fps
inter = orig.MFlowFPS(super_render, bv, fv, num=FPS_NUM*2, den=FPS_DEN, mask=mask_fps, ml=maskScale, thSCD1=threshold, thSCD2=100)

# return this to look at the clip with doubled framerate
#return inter

fps_only = inter.SelectOdd()

# second pass
super_search2 = MSuper(pel=super_pel, sharp=super_sharp, rfilter=super_rfilter, fps_only)		
super_render2 = MSuper(pel=super_pel, sharp=super_sharp, rfilter=super_rfilter, fps_only, levels=1)	
bv2 = MAnalyse(super_search2, isb = true, blksize=blockSize, search=searchAlgo, searchparam=searchRange, pelsearch=searchRangeFinest, 
\ chroma=useChroma, delta=delta, lambda=lambda, lsad=lsad, pnew=pnew, plevel=plevel, global=globalMotion, overlap=overlap, overlapv=overlapv, 
\ divide=divide, badSAD=badSAD, badrange=badRange, meander=meander, temporal=temporal, trymany=trymany)
fv2 = MAnalyse(super_search2, isb = false, blksize=blockSize, search=searchAlgo, searchparam=searchRange, pelsearch=searchRangeFinest, 
\ chroma=useChroma, delta=delta, lambda=lambda, lsad=lsad, pnew=pnew, plevel=plevel, global=globalMotion, overlap=overlap, overlapv=overlapv, 
\ divide=divide, badSAD=badSAD, badrange=badRange, meander=meander, temporal=temporal, trymany=trymany)
inter2 = fps_only.MFlowFPS(super_render2, bv2, fv2, num=FPS_NUM*2, den=FPS_DEN, mask=mask_fps, ml=maskScale, thSCD1=threshold, thSCD2=100)
fps_only2 = inter2.SelectOdd()


delimiter = "; "

inter_yv12 = fps_only2.ConvertToYV12()
orig_yv12 = orig.ConvertToYV12()

# for comparison original must be forwarded one frame
orig_yv12 = trim(orig_yv12,1,0)

inter_yv12 = inter_yv12.Trim(MIDDLE_FRAME - TEST_FRAMES/2 + (TEST_FRAMES%2==0?1:0), MIDDLE_FRAME + TEST_FRAMES/2)
orig_yv12 = orig_yv12.Trim(MIDDLE_FRAME - TEST_FRAMES/2 + (TEST_FRAMES%2==0?1:0), MIDDLE_FRAME + TEST_FRAMES/2)
last = inter_yv12

global total = 0.0
global ssim_total = 0.0
global avstimer = 0.0
frame_count = FrameCount()
FrameEvaluate(last, """
	global ssim = SSIM_FRAME(orig_yv12, inter_yv12)
	global ssim_total = ssim_total + (ssim == 1.0 ? 0.0 : ssim)	
""", args="orig_yv12, inter_yv12, delta, frame_count")	

# NOTE: AvsTimer call should be before the WriteFile call
AvsTimer(frames=1, type=0, total=false, name="Optimizer")

# per frame logging (ssim, time)
resultFile = "D:\optimizer\perFrameResults.txt"	# output out1="ssim: MAX(float)" out2="time: MIN(time) ms" file="D:\optimizer\perFrameResults.txt"
WriteFile(resultFile, "current_frame", "delimiter", "ssim", "delimiter", "avstimer")
WriteFileIf(resultFile, "current_frame == frame_count-1", """ "stop " """, "ssim_total", append=true)

return last

You will need to change the source (obviously) and the folder of the result file. Change MIDDLE_FRAME to find a good spot for the test. TEST_FRAMES sets the number of frames tested, larger than 10 will make the script more robust but will make the optimization process slower. Set the frame rate of your clip using FPS_NUM and FPS_DEN.

If your video has a large resolution it might be a good idea to crop it to a smaller size, leaving just the interesting motion in the center. Make a cropped video and load it directly into this script to make the optimization faster.

My approach at frame rate doubling is this: I double the rate of the original video, remove original frames, then double the rate again and compare against the original frames. That has been working well but I haven't done comparisons to other ways (like dropping every other frame and then doubling the rate). When you write out the optimized scripts take out the comment in front of "return inter" to see the frame-doubled version (ie without the second pass which recreates the original frames).

This script has a lot of parameters and the ranges are mostly not restricted in any way (some parameters like lambda and lsad don't have an upper limit so I just used a very large number there). I did take out dct=1 because with some parameter combinations it takes *extremely* long time to calculate (like 8 minutes when normal time is 1,5 seconds). That's a shame because in my testing dct=1 is usually the best quality-wise.

Instead of using truemotion true/false this script sets the parameters controlled by it (lambda, lsad, pnew, plevel, global) individually.

If you already know for example which blockSize is good then just adjust the search ranges accordingly.

This script doesn't have MRecalculate, I recommed that you first try to find optimal values without it and once you have them add MRecalculate and optimize only its parameters (I can give an example script of that as well). Actually you could stack as many recalculates as you want always using smaller and smaller blocksize... I think it could be a way to make a really robust script, starting with blocksize 64 and halving it with every recalculate. I haven't tried that yet so it's just a theory.

Note that not all the parametes of MVTools2 are there, I haven't gotten around to trying everything yet...

The script includes preprocessing the search clip with RemoveGrain, in my tests I have found it can help. Obviously we are searching an optimal value for that function as well.

Oh and one more thing, I came up with a way to limit the ugly artifacts you often get with MFlowFPS when good motion vectors are not found. Basically you reconstruct the frame created with MFlowFPS using MCompensate and the original frames. You just need to find good parameters for that, which you can do with the optimizer. I will give an example of that later.

[parbruek]

Confirming the same issue Sharc saw previously:
https://forum.doom9.org/showthread.php?p=1808322&highlight=access+violation#post1808322

FrameRateConverter runs into Access Violation whenever I set the speed to anything slower than "normal". The error seems to be occuring in line 143:

bak = MAnalyse(superfilt, isb=true, blksize=BlkSize, blksizev=BlkSizeV, overlap = BlkSize>4?(BlkSize/4+1)/2*2:0, overlapv = BlkSizeV>4?(BlkSizeV/4+1)/2*2:0, search=3, dct=Dct)

Just tested with blu-ray and dvd compliant material with the same result, so it probably isn't triggered by resolution or bit depth...

I've tested and found the same issue with current versions of Avisynth 32 bit and Avisynth+. Should I fiddle with BlkSize, or does someone else know of a fix?

[parbruek]

Just found the issue - My apologies! I had a bad version of FFTW3.dll. And I'm not even sure where it was from. I grabbed the version from here and dropped it in syswow64. Everything runs properly now.

[attackworld]

Sorry for lack of technical terms but...

Tested this with many many different settings, and the best interpolation I can get is just a blend of the two different frames. Only very small movement will the frame look like its a new frame altogether, but with bigger movements you can see a faded version of the two original frames in the "interpolated" frame.

At least with SVP it attempts to create an image that is somewhere between the two original frames.

[Dogway]

I'm unable to run this at all at a reasonable speed. I'm in AVS+ x64, and using fftw338-x64-AVX2. It falls into sub 1fps. Either with or without mp_pipeline using slower preset. What fftw do you guys use? Is there a special MT mode I have to use for FrameRateConverter.dll?