Frame Blending with Max 80/20 Ratio

[MysteryX]

Has anyone ever written a version of frame blending that blends with a maximum ratio of 80/20 so that each resulting frame shows clear details of a source frame? And it could be set with a configurable ratio. I don't see any reason why that couldn't be done, and I expect it should actually look pretty neat.

If someone can find or write such a script, I'd put it into FrameRateConverter to see how it performs. I really don't see any reason why that wouldn't work. I'd want a plugin that takes a parameter value between 0 and 50. If 0, it simply repeats source frames. If 50, it behaves like frame blending (blending some frames 50/50). Every other value would provide a result in-between.

It would thus produce a frame blending result where each frame is at least 80% of a source frame. No blurry mess.

Has this been done before? If not, any easy way to implement it?

Otherwise the easiest is probably to take ConvertFps' source code and edit to change the blending ratio calculation.

[StainlessS]

Your requirements are a little unclear, methinks.

maximum ratio of 80/20
Does that mean, at most 1 in 5 frames blended ?

(blending some frames 50/50)
It would thus produce a frame blending result where each frame is at least 80% of a source frame. No blurry mess.

Seems contradictory, does that mean at least 80% of output frames are unblended ? [EDIT: Same as 'at most 1 in 5 blended']

Every other value would provide a result in-between.
Based on what ?

I'm not volunteering for this, just want clarification.

[johnmeyer]

My interpretation is that he is asking for the new intermediate frame to not be a 50/50 blend of the two adjacent frames, but to have more weight from either the preceding or succeeding frame so that the result would be sharper, but at the risk of losing some temporal smoothness. In other words, provide just a "hint" of what that intermediate frame would look like.

Interesting idea.

[Sharc]

Perhaps some ideas for weighted blending can be taken from deinterlacers?

[StainlessS]

So,
Arg=00) = repeat frames only, no blends.
Arg=50) = unrestricted color blending ratio.

It would thus produce a frame blending result where each frame is at least 80% of a source frame. No blurry mess.

Assuming Arg=25 for above, where 'source frame' should perhaps read something like 'one or other source frame', as there are obviously gonna be 2 source frames to any blend,
ie between (100:0) and (80:20)% OR (0:100) and (20:80%)

Maybe ? [even with above, still unclear]

EDIT: Oh, think I got it, Arg, always applies to say, earlier of the two blend src frames [or always later of the two blend frames, depends on implementation].
(100:0) -> (80:20)% OR (0:100) -> (20:80%), amount to the same thing:- :stupid:

EDIT: Nah, was confused again. :(
maybe where arg range 0.0 -> 50.0


if (Ratio < Arg || (100.0-Ratio) < Arg) Then Blend, else Copy.

Is above as required MX ?

EDIT:
Also, what if decision COPY ie dupe, any benefit where dupe source is Src1 OR Src2, based on luma diff between (Src1-1)<->Src1 and Src1<->Src2),
any benefit, or dont care ?

[MysteryX]

Is above as required MX?

EDIT:
Also, what if decision COPY ie dupe, any benefit where dupe source is Src1 OR Src2, based on luma diff between (Src1-1)<->Src1 and Src1<->Src2),
any benefit, or dont care ?

You're confusing me. Johnmeyer got it. An interpolated frame is based on the previous and next frame. The only question is with what ratio you blend them together.

As to how that would be calculated, that's a different story.

I have no clue what you're saying about dupes.

[videoh]

80:20 is going to look like a jerky mess.

[StainlessS]

Otherwise the easiest is probably to take ConvertFps' source code and edit to change the blending ratio calculation.

Sorry MX. above ChangeFPS got me thinking you were talking bout simple blending FrameRate change, Not interpolation, hence confusion. Just ignore me, not feeling too chippa.

EDIT:
Has anyone ever written a version of frame blending that blends with a maximum ratio of 80/20
If 0, it simply repeats source frames. If 50, it behaves like frame blending
Of course above (+ more) also assisted in the confusion. [EDIT: Repeats == Dupes]

EDIT: And if Interpolating, would be Time related rather than blend ratio, not quite the same thing.

Think maybe VideoH nailed it [right into the coffin :) ]

[MysteryX]

80:20 is going to look like a jerky mess.
I don't think so. Not more than watching a video at... 24fps!! with duplicate frames.

Or yes jerky mess but much less than interpolating with ChangeFps.

It's about finding a compromise between ChangeFps (blend) and ConvertFps (copy) when interpolating 24fps to 60fps.

This will be especially useful with FrameRateConverter as it does a really good job at identifying interpolation artifacts but none of the 2 does a good job at filling those zones. ConvertFps makes the image "jump" around while ChangeFps gives "shadow" effects by blending an arm 50/50 between 2 frames. If we're displaying 80/20, we eliminate the double-shadow effect.

[Andouille]

Average plugin ?
http://avisynth.nl/index.php/Average

[MysteryX]

Maybe something could be done with Average, by preparing 2 clips with the right opacity. The challenge is that the opacity will vary frame by frame depending on the position of the intermediate frame between the 2 source frames around it. That, I really don't see how to calculate that other that writing a plugin -- which really should be quite simple.

[StainlessS]

Are you able to compute the required time (0.0->1.0) distance between the two source frames, and is there ever more than a single interp/blend frame to be synthesized ?

EDIT: Required time being after you have done any messing due to your 80%:20% type stuff.

EDIT: And I point out this again, but using T=0.0->1.0 rather than percent Ratio, and Arg 0.0->0.5 rather than 0->50 .
if (Ratio < Arg || (100.0-Ratio) < Arg) Then Blend, else Copy.
Something like
if (T < Arg || (1.0-T) < Arg) Then Blend, else Copy.

EDIT: Maybe you can extract some code out of this (BlendDrops_MI) :- http://forum.doom9.org/showthread.php?p=1764486#post1764486

relevant bit supports multiple blends between two src frames, with a few comments added

if(Replace==1) {
StartU = RT_DBaseGetField(DB,n,2) # Src1 frame
EndU = RT_DBaseGetField(DB,n,3) # Src2 Frame
Dif = RT_LumaPixelsDifferent(clp,clp,n=StartU,n2=EndU,Thresh=DifByN,Matrix=Matrix) # scene change between StartU and EndU ?
Pos = n-StartU # blend index, 1 = first blended frame, EndU-1 is last blended frame index
frames = EndU-StartU-1 # Number of frames that will be blended
if(Dif<Th2) { # No scene change
Weight = 1.0 * Pos / (Frames+1)
Last = Merge(clp.Trim(StartU,-1),clp.Trim(EndU,-1),Weight)
(Show)?RT_Subtitle("REPLACED: StartU=%d EndU=%d\nPos=%d Frames=%d BlendWeight=%f\nDif Start->End=%f",
\ StartU,EndU,Pos,Frames,Weight,Dif,align=5):NOP
} Else { # Scene change
Dif2 = RT_LumaPixelsDifferent(clp,clp,n=StartU,n2=EndU-1,Thresh=DifByN,Matrix=Matrix)
if(Frames > 1 && Dif2 < Th2) {
Weight = 1.0 * Pos / Frames
Last = Merge(clp.Trim(StartU,-1),clp.Trim(EndU-1,-1),Weight)
(Show)?RT_Subtitle("REPLACED: StartU=%d EndU=%d\nPos=%d Frames=%d BlendWeight=%f\nDif Start->End=%f" +
\ "\n\a!Because StartU->EndU dif > Th2,\nwe used frame before EndU as terminial Frame\n" +
\ "Dif Start->(EndU-1) is %f)"
\ ,StartU,EndU,Pos,Frames,Weight,Dif,Dif2,align=5):NOP
} Else {
(Show)?RT_Subtitle("SKIP REPLACMENT: StartU=%d EndU=%d\nDif Start->End=%f(Th2)",
\ StartU,EndU,Dif,align=5):NOP
}
}
}


EDIT: StartU and EndU are Unique frames, ie non dupes, distinct from their precdessor frame..
EDIT: Above code, input and output frames count same, ie not upping framerate, so [B]Frames would be the number of frames to insert between your two source frames.

[MysteryX]

Doing it with a GScript might be possible but will be complex and slow; but could serve as proof of concept. Taking the source code of ChangeFps and altering a few lines is probably the way to go. And yes if you go from 24fps to 60fps there is more than 1 interpolated frame between each frames.

[MysteryX]

I've got way too much work to do on other stuff to spend time programming Avisynth, but here's what I'm thinking.

ChangeFps has SSIM code to blend 2 frames with a specified ratio, and has the code to calculate that ratio based on the inter-frame position within 2 adjacent frames. All we need to alter that ratio-calculation function. We can use the exact same SSIM code.

[Dreamject]

Does it have any sense?

Frame A, Frame B =FA,FB
1 blended frames between them mean FA, 50%of FA and 50%of FB, FB
2 mean FA, 100%/(1*3) FA 100%-100%/(1*3) FB, 100%/(2*3) FA 100%- 100%/(2*3)FB, FB
...
x mean i=x-1, then FA,..., 100%(i*(1+x)) FA 100%-100%(i*(1+x))FB...,FB I can make mistake, but...

If you force using 80% of FA it's not good.

Its like FA,80%FA,FB

If you wanna interpolate 24fps to 60 you need 2.5 multipler

Like A B C D (4*2.5)=10
A 80%A B 80%B 80%C C 80%C 80%D D 80%D

Imho its just more laggy

[MysteryX]

Hah! Got it working. Figured out it would be faster to just do it than to talk about it.

Mostly only had to insert this in ConvertFps, but did run into other issues making the job longer.


if (mix_ratio < half)
mix_ratio = mix_ratio * ratio / 100;
else
mix_ratio = one - ((one - mix_ratio) * ratio / 100);


It's looking great. A 50% reduction of blending ratios removes the ghost effect on moving hands.