EstimateWeights: New script for frameblending estimation + blending pattern question

[redfordxx]

I made a script which can be helpful to estimate blending pattern of some video, where was change framerate via blending. However, at least sample of source clip is necessary to make the analysis. There are two outputs. Visual and information in the file.
Function:
#function EstimateWeights v0.1
#by redfordxx
#Needed:
#Blendweight plugin by mg262
#RedAverage by redfordxx
function EstimateWeights (clip src, clip dst, float fps_src, float fps_dst, int "shift") {
#src ......clip before the framerate change (for example nonblended 24/1.001 clip)
#dst ......clip after the framerate change (for example blended 25fps clip)
#fps_src...framerate of source (for example 24000.0/1001)
#fps_dst...target framerate (for example 25)
#shift..... positive or negative number of frames needed to shift the source clip to match the destination
shift=default(shift,0)
global sep1=" -- "
global LabelNumbers="Estimated frame: "
file="BlendWeight log.txt"

src=src.AssumeFPS(fps_src)#.ScriptClip("Subtitle(String(current_frame))")
dst=dst.AssumeFPS(fps_dst)

sh_cnt=(shift>=0) ? shift+1 : 0
sh_rng=(shift>=0) ? -1 : -shift
prev=src.FrameEvaluate( "global frp=current_frame").loop(sh_cnt,0,sh_rng).ChangeFPS(fps_dst,1).loop(2,0,-1)
curr=src.FrameEvaluate( "global frc=current_frame").loop(sh_cnt,0,sh_rng).ChangeFPS(fps_dst,1).loop(1,0,-1)
next=src.FrameEvaluate( "global frn=current_frame").loop(sh_cnt,0,sh_rng).ChangeFPS(fps_dst,1).loop(0,0,-1)

iii=Interleave(prev,dst,curr,dst,next, prev,curr,next, curr,dst)
blendprev=iii.ScriptClip("BlendWeight(current_frame)").SelectEvery(10,1)
blendnext=iii.ScriptClip("BlendWeight(current_frame)").SelectEvery(10,3)
blendsrc=iii.ScriptClip("BlendWeight(current_frame)").SelectEvery(10,6)
tmp=Interleave(blendprev,dst,blendnext)

eval1="""
global q_n=YDifferenceToNext
global q_p=YDifferenceFromPrevious
"""
dx=10
s1=StackHorizontal(prev,curr,next)
s2=StackHorizontal(blendprev,tmp.FrameEvaluate(eval1).SelectEvery(3,1),blendnext)
dp=RAverageW(blendprev,dx,dst,-dx,bias=128)
dc=RAverageW(curr,dx,iii.FrameEvaluate("global q_c=YDifferenceFromPrevious").SelectEvery(10,9),-dx,bias=128)
dn=RAverageW(blendnext,dx,dst,-dx,bias=128)
s3=StackHorizontal(dp,dc,dn).Greyscale()
sp=RAverageW(curr,dx,iii.FrameEvaluate("global s_p=YDifferenceToNext").SelectEvery(10,5),-dx,bias=128)
sn=RAverageW(curr,-dx,iii.FrameEvaluate("global s_n=YDifferenceFromPrevious").SelectEvery(10,7),dx,bias=128)
s4=StackHorizontal(sp,blendsrc,sn).Greyscale()
StackVertical(s1,s2,s3,s4)
WriteFile(file, "LabelNumbers", "current_frame", """" <== """", "frp","sep1","frc","sep1","frn","""" Quality("""","q_p","sep1","q_c","sep1","q_n","""") Flow("""","s_p","sep1","s_n","""")"""")
WriteFile(file, "Chr(32)")
}

Usage example:
black=BlankClip(pixel_type="YV12", color=$000000,width=160,height=160)
white=BlankClip(pixel_type="YV12", color=$FAFAFA,width=160,height=160)
s1=StackVertical(StackHorizontal(black,white),StackHorizontal(white,white))
s2=StackVertical(StackHorizontal(white,black),StackHorizontal(white,white))
s3=StackVertical(StackHorizontal(white,white),StackHorizontal(white,black))
s4=StackVertical(StackHorizontal(white,white),StackHorizontal(black,white))
bbb=Interleave(s1,s2,s3,s4).AssumeFPS(4,1)
bbb2=bbb.ConvertFPS(5,1)
EstimateWeights(bbb,bbb2,4,5)

You can use real live clips, but this is test clip to see how does conversion from 4 to 5 fps look like. As expected, the coefficient are increasing by 0.2, which is shown in the logfile.

To understand the meaning of the output clip better use one real life clip instead of bbb first, but just shortly, what you will see Stacked:
previous, current, next source
first pair estimation, current destination, second pair est
difference of current destination to 1st est, current src, 2nd est
src different to previous, current dst estimation, src difference to next

Logfile sample:
Frame 1001 = 0.0000 x prev. Frame + 1.0000 x next Frame
Frame 1003 = -1.#IND x prev. Frame + -1.#IND x next Frame
Frame 1006 = 0.0000 x prev. Frame + 1.0000 x next Frame
Estimated frame: 100 <== 79 -- 80 -- 80 Quality(0.000000 -- 0.000000 -- 177.250000) Flow(107.500000 -- 0.000000)

Frame 1011 = -1.#IND x prev. Frame + -1.#IND x next Frame
Frame 1013 = 0.2000 x prev. Frame + 0.8000 x next Frame
Frame 1016 = 1.0000 x prev. Frame + 0.0000 x next Frame
Estimated frame: 101 <== 80 -- 80 -- 81 Quality(177.250000 -- 86.000000 -- 0.250000) Flow(0.000000 -- 107.500000)

Frame 1021 = 0.3000 x prev. Frame + 0.7000 x next Frame
Frame 1023 = 0.4000 x prev. Frame + 0.6000 x next Frame
Frame 1026 = 0.5000 x prev. Frame + 0.5000 x next Frame
Estimated frame: 102 <== 80 -- 81 -- 82 Quality(64.750000 -- 64.500000 -- 0.250000) Flow(107.500000 -- 107.500000)

Frame 1031 = 0.2000 x prev. Frame + 0.8000 x next Frame
Frame 1033 = 0.6000 x prev. Frame + 0.4000 x next Frame
Frame 1036 = 0.5000 x prev. Frame + 0.5000 x next Frame
Estimated frame: 103 <== 81 -- 82 -- 83 Quality(43.250000 -- 43.000000 -- 0.250000) Flow(107.500000 -- 107.500000)

Frame 1041 = 0.1000 x prev. Frame + 0.9000 x next Frame
Frame 1043 = 0.8000 x prev. Frame + 0.2000 x next Frame
Frame 1046 = 0.5000 x prev. Frame + 0.5000 x next Frame
Estimated frame: 104 <== 82 -- 83 -- 84 Quality(21.750000 -- 21.500000 -- 0.250000) Flow(107.500000 -- 107.500000)

Frame 1051 = 0.0000 x prev. Frame + 1.0000 x next Frame
Frame 1053 = -1.#IND x prev. Frame + -1.#IND x next Frame
Frame 1056 = 0.0000 x prev. Frame + 1.0000 x next Frame
Estimated frame: 105 <== 83 -- 84 -- 84 Quality(0.000000 -- 0.000000 -- 177.250000) Flow(107.500000 -- 0.000000)

Where for example the second block means:
for the output (dst) frame were three input (src) frames tested: 80, 80 and 81
However, it can happen like for example here, that because there was framerate change in order to match the destination clip, some frame can be duplicated.
These three frames (say, f1,f2,f3) were tested in two couples to find the weight (f1,f2) and (f2,f3)
For each couple were found best possible coefficients to calculate destination frame, see above:
For first couple it was not successful for second couple it was 0.2*f2+0.8*f3.
This information you can find in the lines with caption "Frame <framenumber>1" or "Frame <framenumber>3".
The line "Frame <framenumber>6" shows the coefficients c1,c3 for calculation of the f2=c1*f1+c3*f3. Here, c1=1 and c3=0. c1 close to one tells us that f1 and f2 are close to identical.
There are three Quality values q1,q2,q3 which tell us how well the destination frame is estimation by the source frames by means of LumaDifference function. More exactly:
q1=LumaDifference(dst,c1*f1+c2*f2)
q2=LumaDifference(dst,f2) #if this is the lowest value, the frame is probably not blended but exactly the original.
q3=LumaDifference(dst,c2*f2+c3*f3)
Finally, Flow values are luma differences from previous and to next frames in the source file. For good estimation this should be high, i.e.there should be lot of motion and scenechanges.

For the correct blend values you have to choose the low and possibly lowest quality number with high Flow value and coeficients in the interval (0,1)
If the first resp second Flow value is very low (even lower than the quality value), then the first resp second pair of frames are not good predictors.
If the quality value is high, the estimation is not sufficient (it was not possible to find coefficients for that particular couple which would create destination frame.

So here is reading the logfile:
Frame 100: q1 is low flow1 is high=>we can use first interpolation which says dst100=src80 (same info comes from q2=0...which means dst=src)
Frame 101: q3 is low flow2 is high=> we can use second interpolation:dst101=0.2*src80+0.8*src81
Next:
dst102=0.4*src81+0.6*src82
dst103=0.6*src82+0.4*src83
dst104=0.8*src83+0.2*src84
....
We cannot use for example values in line 1021 for frame 102 calculation, coz q1 is too high

[redfordxx]

BTW, does anyone have a clue, what could be the reason wht the BlendWeight plugin is not writing to file from MeGUI, but is writing to file from AvsPMod?

[redfordxx]

And here comes the trouble.
Using this script I processed file, which 25fps blended obviously coming from 24000/1001, and I am not sure, whether there was some dancing in 30000/1001 fps.

I can get rough estimation of weights. The problem is I don't see any pattern there, to be able to reconstruct the original. Maybe someone with more knowledge will know. Here is a sample (but it does not go all the clip same):Dest 1597 = Src1488
Dest 1598 = Src1489
Dest 1599 = Src1490
Dest 1600 = Src1491
Dest 1601 = Src1492
Dest 1602 = Src1493
Dest 1603 = Src1494
Dest 1604 = Src1495
Dest 1605 = 0,156*Src1495 + 0,844*Src1496
Dest 1606 = Src1497
Dest 1607 = 0,362*Src1497 + 0,638*Src1498
Dest 1608 = Src1499
Dest 1609 = 0,583*Src1499 + 0,417*Src1500
Dest 1610 = 0,185*Src1500 + 0,815*Src1501
Dest 1611 = 0,819*Src1501 + 0,181*Src1502
Dest 1612 = 0,381*Src1502 + 0,619*Src1503
Dest 1613 = 0,935*Src1503 + 0,065*Src1504
Dest 1614 = 0,633*Src1504 + 0,367*Src1505
Dest 1615 = Src1505
Dest 1616 = 0,835*Src1506 + 0,165*Src1507
Dest 1617 = Src1507
Dest 1618 = Src1508
Dest 1619 = Src1509
Dest 1620 = Src1510
Dest 1621 = Src1511
Dest 1622 = Src1512
Dest 1623 = Src1513
Dest 1624 = Src1514
Dest 1625 = Src1515
Dest 1626 = Src1516
Dest 1627 = Src1517
Dest 1628 = Src1518
Dest 1629 = Src1519
Dest 1630 = 0,221*Src1519 + 0,779*Src1520
Dest 1631 = Src1521
Dest 1632 = 0,456*Src1521 + 0,544*Src1522
Dest 1633 = 0,096*Src1522 + 0,904*Src1523
Dest 1634 = 0,667*Src1523 + 0,333*Src1524
Dest 1635 = 0,26*Src1524 + 0,74*Src1525
Dest 1636 = 0,86*Src1525 + 0,14*Src1526
Dest 1637 = 0,451*Src1526 + 0,55*Src1527
Dest 1638 = Src1527
Dest 1639 = 0,729*Src1528 + 0,271*Src1529
Dest 1640 = Src1529
Dest 1641 = 0,898*Src1530 + 0,102*Src1531
Dest 1642 = Src1531
Dest 1643 = Src1532
Dest 1644 = Src1533
Dest 1645 = Src1534
Dest 1646 = Src1535
Dest 1647 = Src1536
Dest 1648 = Src1537
Dest 1649 = Src1538
Dest 1650 = Src1539
Dest 1651 = Src1540
Dest 1652 = Src1541
Dest 1653 = Src1542
Dest 1654 = Src1543
Dest 1655 = 0,268*Src1543 + 0,732*Src1544
Dest 1656 = Src1545
Dest 1657 = 0,513*Src1545 + 0,487*Src1546
Dest 1658 = 0,111*Src1546 + 0,889*Src1547
Dest 1659 = 0,727*Src1547 + 0,273*Src1548
Dest 1660 = 0,32*Src1548 + 0,68*Src1549
Dest 1661 = 0,884*Src1549 + 0,117*Src1550
Dest 1662 = 0,539*Src1550 + 0,461*Src1551
Dest 1663 = Src1551
Dest 1664 = 0,751*Src1552 + 0,25*Src1553
Dest 1665 = Src1553
Dest 1666 = Src1554
Dest 1667 = Src1555
Dest 1668 = Src1556
Dest 1669 = Src1557
Dest 1670 = Src1558
Dest 1671 = Src1559
Dest 1672 = Src1560
Dest 1673 = Src1561
Dest 1674 = Src1562
Dest 1675 = Src1563
Dest 1676 = Src1564
Dest 1677 = Src1565
Dest 1678 = 0,134*Src1565 + 0,866*Src1566
Dest 1679 = Src1567
Dest 1680 = 0,359*Src1567 + 0,641*Src1568
Dest 1681 = 0,054*Src1568 + 0,946*Src1569
Dest 1682 = 0,576*Src1569 + 0,424*Src1570
Dest 1683 = 0,186*Src1570 + 0,814*Src1571
Dest 1684 = 0,803*Src1571 + 0,197*Src1572
Dest 1685 = 0,351*Src1572 + 0,649*Src1573
Dest 1686 = Src1573
Dest 1687 = 0,607*Src1574 + 0,393*Src1575
Dest 1688 = Src1575
Dest 1689 = 0,845*Src1576 + 0,155*Src1577
My guess is the numbers have some error due to lossy compression etc., maybe 0.05, I don't know, so the coefficients do not reflect the pattern exactly.

If I don't find pattern, I can always generate file which will have 150000 lines to calculate the original like this: o=o+RAverageW(d.trim(1576,-1), -0,037, d.trim(1577,-1), 0.001, d.trim(1578,-1), 1.036).trim(1470,-1) #calc frame no 1470 @ 24000/1001

But I would prefer something less hardcore, so anyone

EDIT: now it looks like there is a pattern, pretty nasty one...it's like every 25 frames or so there 5 and 5 blended series interleaved with decreasing weights:
like around 0.9 0.7 0.5 0.3 0.1 and mixed with similar series little different. So if someone knows this pattern, I'd appreciate more exact description than this guess.

[jmac698]

Pretty neat; however an early version of srestore had an info mode which showed inherent blendweights; the source file isn't needed. The basic algorithm has to do with finding the angle of a triangle. Try searching cdeblend.

[jmac698]

Here it is:
http://forum.doom9.org/showthread.php?t=94091&page=4

The pattern you get is probably something like leap years; it's making up the difference between 23.976 and 24/25 whatever.