MV interpolation/interframes explained -- Correctly?

[markfilipak]

Frankly, I find the documentation of Super(), Analyse(), and SmoothFps() somewhat vague. It's undoubtedly my shortcoming. How do these illustrations and text look to you? Of course they are consistent and seem to make sense, but that doesn't mean they aren't wrong. Kindly don't say "Good, Mark" unless you absolutely know they are correct. I suppose that means you're familiar with the source code.

Principles: Show, don't tell. Short, simple declarative sentences. Don't be pedantic.

I eagerly await your thoughts. Thanks, folks!
--Mark.

                              ¸__________¸   [diagram 8A] Image fragments and motion.
                              |    frag C|
                              |          |   This texipix diagram illustrates how an image fragment can appear to move frame to frame:
              ¸__________¸    |          |   Imagine 3 source video frames, 'N-1' 'N' & 'N+1'; imagine time flows 'N-1'->'N'->'N+1'.
              |    frag B|    |in frame N+1  Imagine that 'N-1' 'N' & 'N+1' are conveniently stacked and that we can 'see' through them.
              |          |    |__________|   Imagine that 'B' is an image fragment of 'N', and likewise for 'A' (of 'N-1') & 'C' (of 'N+1').
    ¸_________|¸         |                   In the diagram, 'A'->'B'->'C' shows how the image fragment 'moves' during 'N-1'->'N'->'N+1'.
    |    frag A|     in frame N              Current: Imagine that 'N' is the 'current' frame and that, consequentially, 'B' is the 'current' fragment.
    |          |_________|
    |          |
    |      in frame N-1       ¸__________¸   [diagram 8B] Interframes with linear positioning.
    |__________|         .....|......   C|
                    .....:....|.    :    |   This texipix diagram illustrates an example of the processing objective.
              ¸_____:____:    |:    :    |   Objective: Synthesize interframes between source frames to increase frame rate.
           ...|........ B|    |:    :    |   In this example, 2 interframes are synthesized to triple frame rate.
       ....:..|....   :  |    |:____:____|   Interframes are 'filled in' by copying the fragments, 'A' 'B' 'C', from the frames to the interframes.
    ¸__:___:__|¸  :   :  |.....:....:        The copied fragments are positioned linearly, by proportional parts, along straight paths.
    |         A|  :   :  |.....:             The directions, 'A'->'B' & 'B'->'C', point the way; the pointers are the vectors!
    |          |__:___:__|
    |          |..:...:
    |          |..:           ¸__________¸   [diagram 8C] Interframes with spline positioning.
    |__________|        ......|.....    C|
                   .....:.....|    :     |   This texipix diagram illustrates spline positioning based on 'ballistics'.
              .____:_____.    :    :     |   Ballistics implies expanding the scope from 'N-1'->'N'->'N+1' to 'N-2'->'N-1'->'N'->'N+1'->'N+2'.
          ....|.......  B|    :    :     |   Spline positioning would produce 'smoother' vectors along curved paths (not straight paths).
       ...:...|....  :   |    :____:_____|   It would produce slightly better results.
    .__:__:___|.  :  :   |....:....:         However, in the quest for simplicity, spline positioning is not done.
    |         A|  :  :   |....:              Overlap (i.e. Analyse.block.overlap) is meant to compensate for less precision (but does it, really?).
    |          |__:__:___|
    |          |..:..:
    |          |..:
    |__________|
   ¸_____________frame_N________________     [diagram 8D] Grid blocks in 'current' frame.
   |          ¸          ¸          ¸
   |          ¸          ¸    ¸_____¸____¸   This texipix diagram illustrates the block grid in frame 'N' (the 'current' frame).
   |          ¸          ¸    |     ¸   C|   All frames have identical grid layouts and block sizes, but only the 'current' frame's grid is active.
   |          ¸          ¸    |     ¸    |   This diagram reveals how image fragment 'B' is chosen; it's simply the image in its particular grid block.
   |¸ ¸ ¸ ¸ ¸ ¸__________¸¸ ¸ | ¸ ¸ ¸ ¸ ¸|   Each grid block contains its own fragment, its own matches in 'N-1' & 'N+1', and its own motion vector.
   |          |         B|    |     ¸    |   Note that the locations of fragments 'A' or 'C' will rarely align with the grid.
   |          |          |    |_____¸____|   Also note that frames 'N-1' 'N' & 'N+1' (and their images) are never altered.
   |¸_________|¸         |          ¸
   ||         A|         |          ¸        During 'B's processing, 'A' (and its location in 'N-1') and 'C' (in 'N+1') are found by searching for them.
   || ¸ ¸ ¸ ¸ ¸|_________|¸ ¸ ¸ ¸ ¸ ¸ ¸ ¸    The Analyse() function -- configured by configAnalyse -- does the searching & matching.
   ||         ¸|         ¸          ¸        Except for completely static video, exact matches are not possible.
   ||         ¸|         ¸          ¸        All the image fragments in all the grid blocks of 'N' are searched for and matched in 'N-1' & 'N+1'.
   ||_________¸|         ¸          ¸        'B' is just an example.
   |          ¸          ¸          ¸
   ¸_____________frame_N________________     [diagram 8E] Backward interpolation.
   |          ¸          ¸          ¸
   |          ¸          ¸          ¸        Focusing on the 'current' fragment, 'B', this texipix diagram illustrates 'backward' interpolation.
   |          ¸          ¸          ¸        'Backward' means backward in time, from the 'current' frame, 'N', to the previous frame, 'N-1'.
   |          ¸          ¸          ¸        'B-1' & 'B-2', are the 'backward' copies of 'B' along 'B'->'A' (and are identical to 'B').
   |¸ ¸ ¸ ¸ ¸ ¸__________¸¸ ¸ ¸ ¸ ¸ ¸ ¸ ¸    The positions of 'B-1' & 'B-2' are interpolated from the 'B'-to-'A' vector length & direction.
   |       ...|........ B|          ¸
   |   ....:..|....B-1:  |          ¸
   |   :   :  |B-2:   :  |          ¸
   |   :   :  |   :   :  |          ¸
   |¸ ¸:¸ ¸:¸ |___:___:__|¸ ¸ ¸ ¸ ¸ ¸ ¸ ¸
   |   :   :..¸...:...:  ¸          ¸
   |   :......¸...:      ¸          ¸
   |          ¸          ¸          ¸
   |          ¸          ¸          ¸
   ¸_____________frame_N________________     [diagram 8F] Bidirectional interpolation.
   |          ¸          ¸          ¸
   |          ¸          ¸          ¸        Focusing on fragment 'B', this texipix diagram illustrates bidirectional interpolation.
   |          ¸          ¸          ¸        'Bidirectional' means 'backward', 'N' to 'N-1', plus 'forward', 'N' to 'N+1'.
   |          ¸     .....¸......    ¸        'B-1' is the 'backward' copy of 'B' (along 'B'->'A'); 'B+1' is the 'forward' copy (along 'B'->'C').
   |¸ ¸ ¸ ¸ ¸ ¸_____:____¸¸ B+1:¸ ¸ ¸ ¸ ¸    The position of 'B-1' is interpolated from the 'B'-to-'A' vector length & direction.
   |       ...|........ B|     :    ¸        The position of 'B+1' is interpolated from the 'B'-to-'C' vector length & direction.
   |       :  |    B-1:  |     :    ¸
   |       :  |       :  |     :    ¸
   |       :  |       :  |.....:    ¸
   |¸ ¸ ¸ ¸:¸ |_______:__|¸ ¸ ¸ ¸ ¸ ¸ ¸ ¸
   |       :..¸.......:  ¸          ¸
   |          ¸          ¸          ¸
   |          ¸          ¸          ¸
   |          ¸          ¸          ¸
                              ¸__________¸   [diagram 8G] Backward interpolation and image fragment transitions.
                         .....|......   C|
                    .....:....|. C-1:    |   This texipix diagram illustrates the 'A'->'B'->'C' transition for backward interpolation.
              ¸_____:____:  C-2:    :    |   'A' & 'B-2' (copied from 'B') abut (in time) -- the transition is 'A'->'B-2'.
           ...|........ B|    |:    :    |   'B' & 'C-2' (copied from 'C') abut (in time) -- the transition is 'B'->'C-2'.
       ....:..|....B-1:  |    |:____:____|
    ¸__:___:__|B-2:   :  |.....:....:
    |         A|  :   :  |.....:
    |          |__:___:__|
    |          |..:...:
    |          |..:           ¸__________¸   [diagram 8H] Bidirectional interpolation and image fragment transitions.
    |__________|         .....|......   C|
                    .....:....|. C-1:    |   This texipix diagram illustrates the 'A'->'B'->'C' transition for bidirectional interpolation.
              ¸_____:____:  B+1:    :    |   'A+1' (copied from 'A') & 'B-1' (copied from 'B') abut (in time) -- the transition is 'A+1'->'B-1'.
           ...|........ B|    |:    :    |   'B+1' (copied from 'B') & 'C-1' (copied from 'C') abut (in time) -- the transition is 'B+1'->'C-1'.
       ....:..|....B-1:  |    |:____:____|
    ¸__:___:__|A+1:   :  |.....:....:        It's inferred that bidirectional interpolation is superior to backward interpolation.
    |         A|  :   :  |.....:             The difference cannot be the image fragments because they're the same in both cases.
    |          |__:___:__|                   The difference must be the vectors: backward is longer, bidirectional is half the length.
    |          |..:...:                      It's possible that when all fragments are written to a particular interframe, gaps may be left.
    |          |..:                          In aggregate across an entire interframe, shorter vectors produce fewer such gaps, and smaller.
    |__________|                             Fewer, smaller gaps in the abuting interframes means better overall images needing less 'blending'.

The goodness of interframe images depends on how well Super.pel, Analyse.block.w, Analyse.block.h, Analyse.block.overlap, and SmoothFps.algo harmonize.