Horizontal alignment of fields within each frame

[johnmeyer]

I have one small, but interesting problem left in the video transfer project I described here:

http://forum.doom9.org/showthread.php?t=106837

You can download a sample clip from that project here:

http://download.yousendit.com/4769F7937CBF6A84

The problem is that once I have recovered the original progressive frames from the video transfer, some of them have a horizontal shift between the two fields that make up each frame. These fields were created at different instants in time, and were both taken of the same frame of film while it was stationary in the projector's film gate. Therefore, they should be offset vertically by the amount of the field offset, but there should be no horizontal shift.

I think the horizontal shift is due to some residual misalignment in my projector, and the film is still sliding sideways after it should have come to rest. It doesn't happen on every frame, but when it does happen, it results in a slight combing.

I have used Separatefields() on the raw capture and have verified that the problem definitely exists in the raw capture and is not the result of some later manipulations by my script.

So, what I need to do is to separate each frame into two fields, and then use something like Depan to get those two frames to align horizontally. However, I only want to operate on two fields at a time; I don't want to look at the second field from the current frame and compare it to the first field from the next frame.

So that is my problem: How can I operate on the two fields within a frame, and line them up, and then once I've done that, do the same thing for the two fields on the next frame?

I started with this code:

separatefields()
mdata2 = DePanEstimate(range=1, dxmax=20,dymax=0)
DePanStabilize(data=mdata2,prev=0,next=0,dxmax=20,dymax=0,info=true)
but then realized that it was trying to stabilize from field-to-field-to-field, instead of just between the two fields of each frame. As I think through this, I believe the solution probably involves doubling the number of frames, immediately after the separatefields() line, and then after doing the stabilization, doing a SelectEvery().

But that's as far as I've gotten. Any hints, help, or ideas would be greatly appreciated.

P.S. For extra credit: It is also possible that the film is not stationary in the vertical direction either. The result of that would be a slight overlap of the two fields, resulting in something that looks a little soft. It would be nice to solve this problem as well, to the extent it exists. I think this involves doing a Bob() and then doing the vertical stabilization, but that's as far as I've gotten on that one.

[AVIL]

Hi,

You can try this script:

#obtain a bobbed clip. even and odd frames are almost equal
#but exists blurred frames
x=avisource("Film Sample.avi").separatefields().eedi2()

#changes blurred frames by the next frame
#if is blurred, a sobel filter gives a flat frame
v=x.crop(16,8,-16,-8).mt_edge()

#blurred frame are replaced, discard the odd frames
conditionalfilter(v,x,x.trim(1,0),"AverageLuma(v)",">","5").selecteven()
Some result frames are duplicated. Left to maintain clip length.

I'v used as filters:

mt_masktools v 2.0a28
eedi2 v091

[johnmeyer]

I think you're on the right track. Last night I was thinking about doubling frame rate and then doing a SelectEvery(). Now I realize that won't work, and that ConditionalFilter is the way to go. Thanks for the help.

BTW, my wife's name is Avril (like the singer) so when I saw your handle, I thought she had showed up.

[johnmeyer]

This is driving me nuts. I'm so close, but I can't quite get what I want. Here's the key part of the script:

a=separatefields()
b=a.colorYUV(autogain=true)
mdata2 = DePanEstimate(b,range=1,dxmax=16,dymax=0,pixaspect=0.9091,info=true)
c=a.Depan(data=mdata2,matchfields=false,offset=-1,info=true)
d=conditionalfilter(a,a,c,"current_frame % 2","=","0")
d


Again, each frame of video is a capture from one frame of film. Because the film is jittering in the gate, the two fields do not align. For the moment, I am only trying to compensate for the horizontal shift. Thus, I only want Depan to look at the previous frame, and forget any frames before that. The conditional filter passes through the even fields without alteration, and the odd fields should be moved to match their mate.

So, in a nutshell: How do you get Depan to force a pair of fields to match, and then reset for the next pair of fields so that only the motion data between each pair of fields is used?

[AVIL]

@johnmeyer

Depan reset reference frame on scene change. Based in that you can interleave fake black (or white) frames between the odd frames and the next even one. I.e.:

after separatefields() every field is a different frame (frames 0 and 1 from original frame 0, frames 2 and three from original frame 1 and so. We use frames after:

Frame 0, frame 1, black frame, frame 2, frame 3, black frame, ...

You can use selecteven, selectodd, blankclip and interleave.

Depan will try to match frame 1 to frame 0, desists on black frame and frame 2 and newly will tray to match frame 3 to frame 2 and so on.

After that, with selectevery you can strip black frames and (teorically) voilà.

Good luck

[johnmeyer]

Absolutely brilliant. Thank you. It works, but there appears to be a bug/issue in Depan. I don't seem to be able to stop it from adjusting in the Y direction. I only want to compensate in the X direction. It is moving the two fields so that they align spatially in the Y direction. I either need to figure out how to get it to stop doing that, or I need to figure out how to shift the odd (or even fields) by one scan line.

BTW, here's the code I developed, using the hints given in the last post:# Script to recover film frames from film projected on shutterless 16mm projector.
# Second Pass
# Copyright 2006 John H. Meyer
# Revision April 18, 2006
#-----------------------------

loadPlugin("c:\Program Files\AviSynth 2.5\plugins\TIVTC.dll")
loadPlugin("c:\Program Files\AviSynth 2.5\plugins\Depan.dll")
loadplugin("c:\Program Files\AviSynth 2.5\plugins\MultiDecimate.dll")

AVISource("D:\OPRFHS\Intersquad 1st reel)0002.avi")

converttoYUY2(interlaced=true)
AssumeBFF()
tfm(display=false,micout=2,mode=0,cthresh=45,mi=1600,pp=6,metric=0,field=1,micmatching=2,slow=2,blockx=256,blocky=256,sco=-1,debug=false,input="d:\tfm.txt")

a=MultiDecimate(pass=2)
b=AssumeFrameBased(a)
c=b.separatefields()
d=c.colorYUV(autogain=true)

e=interleave(blankclip(c),selectodd(c),selecteven(c))
f=interleave(blankclip(d),selectodd(d),selecteven(d))


mdata2 = DePanEstimate(f,range=1,dxmax=16,dymax=0,pixaspect=0.9091,info=false)
g=e.Depan(data=mdata2,matchfields=false,offset=-1,info=false)

h=g.selectevery(3,0,1)
i=weave(h)
AssumeFPS(18)
i