Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
23rd April 2018, 22:36 | #21 | Link | |
Registered User
Join Date: Mar 2018
Posts: 447
|
Quote:
So, let's take a look at the masks. I don't know how MMask generates the colors from the vectors, but we can see that the forward mask is lighter than average and the backward mask is darker than average. Since we are showing horizontal component of motion vector and the movement is horizontal it's safe to assume that the lighter and darker colors represent horizontal motion vectors with opposite directions and the middle gray represents no motion. Since frame n+delta has the forward vectors (pointing from frame n to frame n+delta), the vector direction should be towards left and therefore lighter color means vectors towards left. Crudely depicted here: Code:
frame N+delta FORWARD vectors shown along with pixel colors (0 = black, X = white) vector direction is towards left (<) 0 0 0 X< X< X< X< X X X X X X X 0 0 0 X< X< X< X< X X X X X X X 0 0 0 X< X< X< X< X X X X X X X One pixel and its motion vector +-----+ | | | <--------+ | | +-----+ Things get more interesting when we look at the backward vectors at frame n. The vector direction is now towards right Code:
frame N BACKWARD vectors 0 0 0 0> 0> 0> 0> X X X X X X X 0 0 0 0> 0> 0> 0> X X X X X X X 0 0 0 0> 0> 0> 0> X X X X X X X +-----+ | | +--------> | | | +-----+ Ok, this is not how it works so let's try again. This time we swap the pixels and the motion vectors. Code:
frame N+delta FORWARD vectors shown along with pixel colors of frame N 0 0 0 0< 0< 0< 0< X X X X X X X 0 0 0 0< 0< 0< 0< X X X X X X X 0 0 0 0< 0< 0< 0< X X X X X X X One pixel and its motion vector +-----+ | | | <--------+ | | +-----+ Does it also work with the backward vectors? Code:
frame N BACKWARD vectors shown along with pixel colors of frame N+delta 0 0 0 X> X> X> X> X X X X X X X 0 0 0 X> X> X> X> X X X X X X X 0 0 0 X> X> X> X> X X X X X X X +-----+ | | +--------> | | | +-----+ So how can we make it work? There is at least one way, we must draw the arrow so that the back end is at the center of the pixel: Code:
+-----+ | | | +--------> | | +-----+ Last edited by zorr; 24th April 2018 at 00:12. Reason: Added screenshot |
|
24th April 2018, 02:12 | #22 | Link | |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
I'm not allowing myself to get too confused (despite the assistance of your post) ,
the arrow only shows the frame where vectors are stored (arrow head), and the frame used as the pixel source (arrow flight/feathers/rear end), the actual horizontal distance/direction is in the color (above or below Mid Grey, which might be 127, 128 or even 126 [rounded 125.5 mid point for TV levels], I'm guessin' that it might be 128). I'm quite happy to accept that it works, now stop trying to confuse me [But if you figure it out 100.0%, and find different then do let us know] EDIT: Although this kinda makes things a little more 'interesting' From MFlowInter Docs, last arg tclip, Quote:
(not really, it just allows specification of the time arg for every pixel individually, the how does not really matter unless you want to use it tclip arg and if so just do as instructed in above doc). The arrow in the title bar above the graphic[at RHS] shows the direction of the white block travelling across the frame, right to left (when HFlip=False). For the forward vector, pixels at n moved left for the n to n+delta transition, so the color at 'FVEC[n+delta=62] aligned' is lighter than mid grey and so must mean move pixels at n left to match the pixel at n+delta. In the backwards vector is is just the other way around (from frame n+delta to n) and so is opposite and darker than mid grey. So to recap, lighter than mid grey means Move_Left, darker, means Move_Right, absolute difference from mid grey is the distance to move. If you change to HFlip=True, white block will move left to right, and n->n+Delta_Aligned will be darker than mid grey instead of lighter. In a perfect world the results would always be exact opposite (relative mid grey), both vector sets are just a check on each other, that both lots of calculations agree.
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 24th April 2018 at 06:54. |
|
24th April 2018, 17:36 | #23 | Link |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
Post #16 script updated.
Shows colors of the vector greys, and little white (single pixel) dot overlaid where taken from (Align MUST be true otherwise not shown). eg Code:
CS="Y8" #CS="YV24" VectorTest(Delta=2,time=50.0,HFlip=False,CS=CS) Fwd/Bak, Is mostly opposite (relative mid grey, 128), but sometimes goes awry (usually at edge of grey, we try take mid pixel). EDIT: With CS="YV24" EDIT: The movement in above graphics should be DELTA(2) * XSTEP(8)=16, but in MSuper, Pel=2, ie 2*8*2=32, so for the vector Greys = +/- 32 , seems about right. [XSTEP is the horizontal pixel movement per frame, Delta is the number of frames apart, and Pel=2 is at 1/2 pixel resolution].
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 24th April 2018 at 18:51. |
24th April 2018, 23:23 | #24 | Link | |||||
Registered User
Join Date: Mar 2018
Posts: 447
|
Quote:
Quote:
Quote:
Some clarification for this explanation: Quote:
Quote:
Code:
FORWARD VECTORS (MMask lighter than average, positive x value) 0 1 2 3 4 5 6 7 8 +-----+-----+-----+-----+-----+-----+-----+-----+-----+ |(8,0)| | | | | | | |(0,0)| motion vector (delta x, delta y) | <-----------------------------------------------+ | | | | | | | | | | | +-----+-----+-----+-----+-----+-----+-----+-----+-----+ +-----+ +-----+ +-----+ | | | | | | | A | | X | | B | | | | | | | +-----+ +-----+ +-----+ Like we agreed the vector points towards left. The vector is stored at pixel A (there is a vector at pixel B too but it has delta zero). The goal here is to move PIXEL B along the vector towards PIXEL A into location of PIXEL X which is at 50% of the vector length. This means we have to draw the vector so that the arrowhead is at PIXEL A and the rear end is at PIXEL B. So we have an algorithm like this: Code:
1) place the arrowhead at the pixel which contains the motion vector (PIXEL A) 2) calculate position delta by adding motion vector to current pixel position (x=0+8, y=0+0) = (8,0) 3) place the rear end of arrow to this location (PIXEL B) 4) move the pixel at the rear end towards the arrowhead by 50% of arrow length (PIXEL X) (in reality: render the pixel color of PIXEL B to the location of PIXEL X) Code:
BACKWARD VECTORS (MMask darker than average, negative x value) 0 1 2 3 4 5 6 7 8 +-----+-----+-----+-----+-----+-----+-----+-----+-----+ |(-8,0) | | | | | | |(0,0)| motion vector (delta x, delta y) | | | | | | | | | | | | | | | | | | | | +-----+-----+-----+-----+-----+-----+-----+-----+-----+ I haven't really figured it out, just making observations which show that the current theory leads to some complications. I believe there is a simpler way to use the vectors. To really figure out what MVTools2 is doing would mean diving into the source code and I don't really have the time nor competence to do that. |
|||||
25th April 2018, 13:00 | #25 | Link |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
You are an evil and repulsive person who delights in the suffering of others.
Dont think I had my programming head on when I wrote the vector color direction stuff. Light grey does indeed mean move pixel left for (forward) frames n to n+delta, but that is stating it the wrong way around programmatically. If you iterate over frame n for every x, putting pixels in frame n+delta at x + Some_Offset, you could end up eg setting only a single pixel many times (and all the rest would hold rubbish). For forwards component pixel, You should actually iterate over the destination for every x, and use the vector grey stuff to take the source pixel from frame n (the back end of the forward vector arrow) at x+((Fgrey-128)/pel), so instead of moving pixel left in forward prediction, you actually get it from right (so light grey, get it from the right, dark grey get it from the left). [It also corrects the weird sign issue we had so (rel 128) +ve grey means take from the right, -ve from left, and so it matches raster layout in memory]. For backwards component pixel, iterate over the destination for every x, and use the vector grey stuff to take the source pixel from frame n+delta (the back end of the backward vector arrow) at x+((Bgrey-128)/pel), so instead of moving pixel right in backward prediction, you actually get it from left. [when considering VectorTest(HFlip=False)] The "x+((Fgrey-128)/pel)" and "x+((Bgrey-128)/pel)" above should of course be limited to valid pixel coords (min,max), and also need to apply the Time arg to both. Code:
# Something like (where Time=33.33%), # EDIT: Added lines below FGrey = FGrey_at_nPlusDelta[x] # @ head of Forward vector arrow, the horiz mask from fvec BGrey = BGrey_at_n[x] # @ head of Backward vector arrow, the horiz mask from bvec pel=2.0 # whatever pel is in use, as float FCoord_X = x + Round(((Fgrey-128)/pel) * 33.33 / 100.0) # 33.33% of forward distance (from n) BCoord_X = x + Round(((Bgrey-128)/pel) * (100.0 - 33.33) / 100.0) # 66.66% of backward distance (from n+delta) FPixel = Frame_n[Min(Max(FCoord_X,0),Width-1)] BPixel = Frame_nPlusDelta[Min(Max(BCoord_X,0),Width-1)] # Self Check blurry stuff ResultClipFrame_n[x] = Round((FPixel + BPixel)/2.0) # This frame needs shift to fix BAD frame @ Time=33.00% EDIT: I have not looked at the source, no idea how the self check blurry stuff works, just a wild guess. (maybe [EDIT: likely] the other masks play a part)
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 25th April 2018 at 22:17. |
25th April 2018, 23:09 | #26 | Link | |||
Registered User
Join Date: Mar 2018
Posts: 447
|
Quote:
Quote:
We will see about that. Quote:
Code:
PROCESSING... -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8 9 +-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+ | | | | | | |(9,0)| | | | | | | | |(0,0)| forward vector | <-----------------B (66.6%)---------+-----F (33.3%)---> | | | | | | | | | | | | | |(-9,0) | | | | | | | |(0,0)| backward vector +-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+ +-----+ +-----+ +-----+ | | | | | | | A | | X | | B | | | | | | | +-----+ +-----+ +-----+ N+delta 33.3% N ...<<<<<<<<<......... backward vectors (-9,0) at frame N ...>>>>>>>>>......... forward vectors (+9,0) at frame N+delta 000000000000XXXXXXXXX frame N 000000000XXXXXXXXXXXX interpolated at 33.3% 000XXXXXXXXXXXXXXXXXX frame N+delta --> forward offset (33.3%), read pixel from frame N <----- backward offset (66.6%), read pixel from frame N+delta 000000000000XXXXXXXXX--> frame N with offset -3 <-----000XXXXXXXXXXXXXXXXXX frame N+delta with offset +6 000000000000XXXXXXXXXXXX average color CONCLUSION: STAINLESSS IS A GENIUS |
|||
25th April 2018, 23:45 | #27 | Link | |||
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
Quote:
Quote:
Quote:
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 26th April 2018 at 15:13. |
|||
26th April 2018, 23:31 | #28 | Link | |
Registered User
Join Date: Mar 2018
Posts: 447
|
Actually when I wrote that I was confident that I could crush your hopes and dreams about that algorithm, but when I went through it I realized it works... but I decided to leave that comment there in order not to spoil the ending.
Quote:
Let's start with a screenshot of the two frames and the motion vector masks used when creating the interpolated frame. The interpolated frame is created at 33.3% between frames N and N+delta (like in the previous example). I'm changing the XSTEP to 4.5 in order to make the math simpler, this results in 9 pixels of movement in two frames (delta is 2). Looks like this in ASCII: Code:
PIXEL COLOR: 0 = BLACK, X = WHITE VECTOR DIRECTION: < = LEFT, > = RIGHT, . = ZERO LENGTH 000000000000000000XXXXXXXXXXXXXXXXXX FRAME N 000000000000000000XXXXXXXXXXXXXXXXXX 000000000000000000XXXXXXXXXXXXXXXXXX 000000000000000000XXXXXXXXXXXXXXXXXX 000000000XXXXXXXXXXXXXXXXXXXXXXXXXXX FRAME N+delta 000000000XXXXXXXXXXXXXXXXXXXXXXXXXXX 000000000XXXXXXXXXXXXXXXXXXXXXXXXXXX 000000000XXXXXXXXXXXXXXXXXXXXXXXXXXX .........>>>>>>>>>.................. FRAME N+delta FORWARD VECTORS .........>>>>>>>>>.................. .........>>>>>>>>>.................. .........>>>>>>>>>.................. .........<<<<<<<<<.................. FRAME N BACKWARD VECTORS .........<<<<<<<<<.................. .........<<<<<<<<<.................. .........<<<<<<<<<.................. 000000000000000XXXXXXXXXXXXXXXXXXXXX INTERPOLATED FRAME AT 33.33% BETWEEN N AND N+DELTA 000000000000000XXXXXXXXXXXXXXXXXXXXX 000000000000000XXXXXXXXXXXXXXXXXXXXX 000000000000000XXXXXXXXXXXXXXXXXXXXX Code:
FORWARD VECTORS CLOSEUP ... (0,0) (0,0) (0,0) (9,0) (9,0) (9,0) (9,0) ... BACKWARD VECTORS CLOSEUP ... (0,0) (0,0) (0,0) (-9,0) (-9,0) (-9,0) (-9,0) ... These vectors can now be used to point the coordinates where colors should be read for the interpolated frame. We loop every pixel on the screen and read a pixel from the location pointed by the forward and backward vectors. Code:
FORWARD VECTOR AT PIXEL (6,0) 0 1 2 3 4 5 6 7 8 9 +-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+ | | | | | | |(3,0)| | | | | | | | | | | | +-----------------> | | | | | | | | | | | | | | +-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+ BACKWARD VECTOR AT PIXEL (6,0) 0 1 2 3 4 5 6 7 8 9 +-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+ | | | | | | |(-6,0) | | | | | <-----------------------------------+ | | | | | | | | | | | | | | | | | +-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+ This is all the information we need to create the interpolated frame, but it's not very straightforward to do manually. We would have to look at every pixel and its vectors, look at where the vectors are pointing at and what the colors are at those locations. Luckily there's a shortcut we can use. Since all the forward vectors are identical we can think of the operation as shifting the frame N in the opposite direction the vector is pointing at. So in this case the vector (3,0) means we should shift the frame left by 3 pixels. Once we have done that operation we can read the pixel color at the same coordinate as the target pixel is at. We do the same thing with backward vectors, this time we shift frame N+delta in the opposite direction, in this case 6 pixels to the right. Code:
000000000000000000XXXXXXXXXXXXXXXXXX FRAME N 000000000000000000XXXXXXXXXXXXXXXXXX 000000000000000000XXXXXXXXXXXXXXXXXX 000000000000000000XXXXXXXXXXXXXXXXXX 000000000XXXXXXXXXXXXXXXXXXXXXXXXXXX FRAME N+delta 000000000XXXXXXXXXXXXXXXXXXXXXXXXXXX 000000000XXXXXXXXXXXXXXXXXXXXXXXXXXX 000000000XXXXXXXXXXXXXXXXXXXXXXXXXXX 000000000000000000XXXXXXXXXXXXXXXXXX... FRAME N shifted left by 3 pixels 000000000000000000XXXXXXXXXXXXXXXXXX... 000000000000000000XXXXXXXXXXXXXXXXXX... 000000000000000000XXXXXXXXXXXXXXXXXX... ......000000000XXXXXXXXXXXXXXXXXXXXXXXXXXX FRAME N+delta shifted right by 6 pixels ......000000000XXXXXXXXXXXXXXXXXXXXXXXXXXX ......000000000XXXXXXXXXXXXXXXXXXXXXXXXXXX ......000000000XXXXXXXXXXXXXXXXXXXXXXXXXXX Note: new pixels not visible before marked with '.' Code:
000000000XXXXXXXXXXXXXXXXXX AVERAGE COLOR OF SHIFTED FRAMES N AND N+DELTA 000000000XXXXXXXXXXXXXXXXXX 000000000XXXXXXXXXXXXXXXXXX 000000000XXXXXXXXXXXXXXXXXX 000000000000000XXXXXXXXXXXXXXXXXXXXX INTERPOLATED FRAME AT 33.33% BETWEEN N AND N+DELTA (THE GOAL) 000000000000000XXXXXXXXXXXXXXXXXXXXX 000000000000000XXXXXXXXXXXXXXXXXXXXX 000000000000000XXXXXXXXXXXXXXXXXXXXX Here's what it looks like if we also process the zero vectors (lowercase used where final pixel color was read with zero vector): Code:
.........>>>>>>>>>.................. FRAME N+delta FORWARD VECTORS .........>>>>>>>>>.................. .........>>>>>>>>>.................. .........>>>>>>>>>.................. ooooooooo000000XXXxxxxxxxxxxxxxxxxxx ooooooooo000000XXXxxxxxxxxxxxxxxxxxx ooooooooo000000XXXxxxxxxxxxxxxxxxxxx ooooooooo000000XXXxxxxxxxxxxxxxxxxxx |
|
Thread Tools | Search this Thread |
Display Modes | |
|
|