Automatically Detect & Correct Field Shifts (again)

[johnmeyer]

I need help determining when all even fields in a video have been shifted upwards.

Back in 2013 I was given an interlaced video where the even fields had been shifted, but only on some scenes. I posted about this problem back then, and the script I provided in post #7 fixes the shift, but the detection logic sometimes fails to see the shift.

Automatically Detect & Correct Field Shifts (https://forum.doom9.org/showthread.php?t=168009)

I have another video with the same problem. Here's a clip:

Field Shift Sample (https://www.mediafire.com/file/g3hjrq84wmlh0w0/Field_Shift.mpg/file)

Here is my current script. It too fails to detect all frames where the field has been shifted. I've enabled the Interleave statement at the end of the script so you can see the original and then when you go to the next frame you see the fixed version. You'll see it begin to fail starting at frame 89.

#Script to shift fields, but only on frames where problem exists

input=AVISource("e:\fs.avi").ConvertToYV12()
source=input.assumetff().bob(0,1)

#Perform the shift on bobbed source by first cropping and then adding border
Hshift = 0
Vshift = 2
even = selecteven(source)
odd = selectodd (source)

even = crop(even,Hshift,Vshift,0,0)
even = addborders(even,0,0,Hshift,Vshift)

output = interleave(even,odd)

/*
#Debugging: make sure YDifference metrics reliably detect field shift
script = """Subtitle("\nsource = " + String(YDifferenceFromPrevious(source)) + \
"\noutput = " + String(YDifferenceFromPrevious(output)), lsp=0)"""
Scriptclip(output, script)
*/

fixed = output.scriptclip("""YDifferenceFromPrevious(source) > \
YDifferenceFromPrevious(output) ? output : source""")

#Convert back to interlaced
evenbob = selecteven(fixed)
oddbob = selectodd(fixed)
EvenOrig = separatefields(evenbob).selecteven()
OddOrig = separatefields(oddbob).selectodd()
interleave(EvenOrig,OddOrig)
fixed_field = weave()

#return fixed_field

#Alternate output to view before/after
Interleave(input,fixed_field)

You'll have to replace the AVISource command with something which reads MPG files (I frameserve from Vegas into all my AVISynth scripts, so I always use AVISource).

Is there a better approach than simply measuring differences between the bobbed version of the original and comparing them to the difference metrics for the bobbed version of the field shifted (corrected) video?

[lollo2]

I have the field shift problem in all of my captures, as the users of VCRs with TBC know very well, because the line correction leads sometimes to a even or a odd field or both shifted up or down (2 or 3 in 1 hour capture in average, sometimes more).

It is not easy automatically detect them in general, is a little bit easier when you have a logo on the source (for instance the logo of a TV) because the "difference" between the shifted field and the previous one is augmented.

I developped my own script based on a filtering process suggested by StainlessS in this thread for another problem: https://forum.doom9.org/showthread.php?t=183582
It is not perfect at all and fail to detect shifts where the difference between the fields is minimal. Being beased on a "difference" concept, it also introduce many false positive.

What I do in general is that I run the script to find most of the defects, but then watch the video in "slow motion" with AssumeFPS(10, true) to identify all the shifted fields, a very boring and time consuming procedure, but is the only reliable way.
I then fix the problems by simply shifting back up or down the problematic fields.

FWIW here my scripts:

# https://forum.doom9.org/showthread.php?t=183582
# https://forum.doom9.org/showthread.php?p=1960487#post1960487

function detect_bad_frames_GMa(clip c, string video_file, int "threshold")
{
video_tff_sep=c.AssumeTFF().separateFields()
global video_tff_sep_even=video_tff_sep.selectEven()
global video_tff_sep_odd=video_tff_sep.selectOdd()

global th = threshold

global frames_file = video_file+"_bad_frames_threshold_"+string(threshold)+".txt"

RT_FileDelete(frames_file)

report = """
n = current_frame
Top2Bot = RT_FrameDifference(video_tff_sep_even, video_tff_sep_odd, n=n, n2=n, ChromaWeight=0.0)
Prv2CurE = RT_FrameDifference(video_tff_sep_even, video_tff_sep_even, n=n, n2=n-1, ChromaWeight=0.0)
Cur2NxtE = RT_FrameDifference(video_tff_sep_even, video_tff_sep_even, n=n, n2=n+1, ChromaWeight=0.0)
Prv2CurO = RT_FrameDifference(video_tff_sep_odd, video_tff_sep_odd, n=n, n2=n-1, ChromaWeight=0.0)
Cur2NxtO = RT_FrameDifference(video_tff_sep_odd, video_tff_sep_odd, n=n, n2=n+1, ChromaWeight=0.0)
defect = (Top2Bot > th && Prv2CurE > th && Cur2NxtE > th) || (Top2Bot > th && Prv2CurO > th && Cur2NxtO > th)
(defect) ? RT_WriteFile(frames_file,"%d",n,Append=True) : NOP
subtitle(RT_String("%d] Top2Bot=%.2f : Prv2CurE=%.2f : Cur2NxtE=%.2f : Prv2CurO=%.2f : Cur2NxtO=%.2f",n,Top2Bot,Prv2CurE,Cur2NxtE,Prv2CurO,Cur2NxtO),\
text_color=(defect)?$FF40FF:$FFFF00,y=20)
"""

ScriptClip(c,report)

RT_ForceProcess

FrameSel(Cmd=frames_file, Show=true) # display defective frames
#return(last) # display all frames
}

# https://forum.doom9.org/showthread.php?t=183582
# https://forum.doom9.org/showthread.php?p=1960487#post1960487

function display_bad_frames_GMa(clip c, string video_file, int "threshold")
{
frames_file = video_file+"_bad_frames_threshold_"+string(threshold)+".txt"

FrameSel(c, Cmd=frames_file, Show=true)
}

function shift_fields_GMa(clip c, int frame_number, line_shift_even, line_shift_odd)
{
# separate fields tff
c_tff_sep=c.AssumeTFF().separateFields()

# separate fields tff even
c_tff_sep_even=c_tff_sep.SelectEven()

# separate fields tff odd
c_tff_sep_odd=c_tff_sep.SelectOdd()

# shift field even
c_tff_sep_even_rep = (line_shift_even > 0) ?\
c_tff_sep_even.trim(0,frame_number-1)\
++c_tff_sep_even.trim(frame_number,frame_number).crop(0,0,0,-line_shift_even).addborders(0,line_shift_even,0,0)\
++c_tff_sep_even.trim(frame_number+1,0)\
:\
c_tff_sep_even.trim(0,frame_number-1)\
++c_tff_sep_even.trim(frame_number,frame_number).crop(0,-line_shift_even,0,0).addborders(0,0,0,-line_shift_even)\
++c_tff_sep_even.trim(frame_number+1,0)

# shift field odd
c_tff_sep_odd_rep = (line_shift_odd > 0) ?\
c_tff_sep_odd.trim(0,frame_number-1)\
++c_tff_sep_odd.trim(frame_number,frame_number).crop(0,0,0,-line_shift_odd).addborders(0,line_shift_odd,0,0)\
++c_tff_sep_odd.trim(frame_number+1,0)\
:\
c_tff_sep_odd.trim(0,frame_number-1)\
++c_tff_sep_odd.trim(frame_number,frame_number).crop(0,-line_shift_odd,0,0).addborders(0,0,0,-line_shift_odd)\
++c_tff_sep_odd.trim(frame_number+1,0)

# repaired video
c_rep=interleave(c_tff_sep_even_rep,c_tff_sep_odd_rep).Weave()

return(c_rep)
}

[Frank62]

What is at the edge of the shifted fields? Blackness? A doubled line? Or rather chaotic things?

[lollo2]

In the contest of Analog Capture, blackness.

The TBC of the VCR while compensating potential dropouts properly places all of the recovered lines, and then some are missing (according to how much the field is shifted).

So, when shifiting back the fields, on the top of the image there is one or more missing line, which is irrelevant and not noticeable at standard playing speed. The bottom of the picture is not important because there is the head switching noise that needs to be masked anyhow.

Somebody interpolates the missing line(s) based on lines below, but I prefer to do not introduce additional artifacts, and just shift the fields back to the original presumed position.

[Frank62]

For detection then, why don't you just test if the edge is completely black or not?

[lollo2]

Because a top edge black is not constant across a whole capture, and you miss detection if the upper part of the active picture is black.

In addition we're talking about fields here, not frames, and the even and odd scanlines are bobbing.

[real.finder]

here the script as avs function with some updates like an option to control the Vshift since I see a source that need -2 of Vshift
#Script to shift fields, but only on frames where problem exists
#Hard-wired for shifting fields by even number
#original script by John Meyer
Function CorrectFieldShift(clip c, int "Vshift", bool "FirstField"){
source=GetParity(c) ? c.assumetff().bob(0,1) : c.assumebff().bob(0,1)

#Perform the shift on bobbed source by first cropping and then adding border
Hshift = 0
Vshift = Default(Vshift, 2)
FirstField = Default(FirstField, true)
even = selecteven(source)
odd = selectodd (source)

even = FirstField ? crop(even,Hshift,Vshift > 0 ? Vshift : 0,0,Vshift < 0 ? Vshift : 0) : even
even = FirstField ? addborders(even,0,Vshift < 0 ? Abs(Vshift) : 0,Hshift,Vshift > 0 ? Vshift : 0) : even

odd = !FirstField ? crop(odd,Hshift,Vshift > 0 ? Vshift : 0,0,Vshift < 0 ? Vshift : 0) : odd
odd = !FirstField ? addborders(odd,0,Vshift < 0 ? Abs(Vshift) : 0,Hshift,Vshift > 0 ? Vshift : 0) : odd

output = interleave(even,odd)

fixed = output.scriptclip("""YDifferenceFromPrevious(source.aBlur(5,blurv=0)) > \
YDifferenceFromPrevious(aBlur(5,blurv=0)) ? last : source""", args="source")
fixed.interlaced60or50(BFF=!(GetParity(c)))
}

[johnmeyer]

Lollo2, thanks for the script. I did look at StanlessS' RT_Stats package to see if any of his functions might provide better detection. I looked at RT_FrameDifference but chose not to use it because it "returns the average pixel difference for the two comparison frames." By comparison, the YDiff built-in functions returns "the absolute difference of pixel value between the current and previous frame." Gavino described it further in this post (https://forum.doom9.org/showpost.php?p=1602687&postcount=2), "it is the average absolute luma difference between pixels at the same location - for each pixel position, the absolute luma difference is calculated, the results are summed and divided by the total number of pixels to find the average."

So, the Ydiff algorithms makes the number "blow up" pretty quickly when there are any differences on a pixel-by-pixel level, even if the average is similar. When comparing a scan line that is two rows down, the overall average is likely to be similar, but because the location of pixels has moved, the pixel-by-pixel comparison can be quite different.

My original idea was to compare the even line that had been shifted up with the even line estimated from the two adjacent odd lines. I thought I could do that by using Bob(0,1) for the shift and Bob(0.3,0.3), which is the default, for the "estimated" line. That didn't work. I then tried using MVTools MFlowInter to create a "real" estimated line, but the artifacts created by that more sophisticated estimation made the detection blow up.

I do like your script's idea of using more than one comparison, although I'd have to look at the actual number generated to see if they are redundant, or if one works when the other doesn't, and vice versa.

To Frank62's question, I did look at the top and bottom of the frame to see if the shift created a black line, but as Lollo2 points out, because this is an analog capture there is too much garbage in those lines. Also, the source of my capture has already been encoded before it got to me and, as you can see from the capture, there is a tremendous amount of encoding noise introduced by the low bitrate MPEG-2 encode. I was able to clean that up quite nicely as the last step in my restoration, but when doing the analysis to decide whether to shift the field, it further masked what was happening at the top and bottom frame boundaries.

I am going to give this one last shot, as time permits. I am still trying to understand what the default Bob(0.3,0.3) actually does. I think I understand Bob(0,1): it creates a full frame out of each field without altering the field itself, so the original information can be recovered (i.e., it is reversible). I thought the standard Bob (0.3,0.3) created some sort of estimated "blended" field. If I can create an estimated field from the two adjacent odd fields, and do that with some sort of simple averaging, it might provide better detection.

[poisondeathray]

I am still trying to understand what the default Bob(0.3,0.3) actually does. I think I understand Bob(0,1): it creates a full frame out of each field without altering the field itself, so the original information can be recovered (i.e., it is reversible). I thought the standard Bob (0.3,0.3) created some sort of estimated "blended" field. If I can create an estimated field from the two adjacent odd fields, and do that with some sort of simple averaging, it might provide better detection.


The bob parameters are bicubicresize b,c values or blurring and ringing.
http://avisynth.nl/index.php/Bob
http://avisynth.nl/index.php/BicubicResize#BicubicResize

Bob not reversible for YV12 chroma such as your mpeg2 clip .


This filter uses BicubicResize to do its dirty work. You can tweak the values of b and c (see BicubicResize)

Bob(0.0, 1.0) preserves the original fields for RGB and YUY2 and preserves the Luma but not the Chroma for YV12.

The parameters b and c can be used to adjust the properties of the cubic; they are sometimes referred to as "blurring" and "ringing" respectively.




In contrast, Yadif or QTGMC(lossless=1) are "reversible" for all supported pixel types . This script will be uniform grey for lossless / reversible

eg.

MPEG2Source("Field Shift.d2v")
src=last

bob(0,1)
#QTGMC(lossless=1)
#yadif(1,1)

SeparateFields() # weave the original fields back together
SelectEvery(4,0,3)
Weave()

Overlay(src,last , mode="Difference", pc_range=true)
Levels(127, 1, 129, 0, 255, false)

[johnmeyer]

I'm trying to remember why I had to convert to YV12. I knew that Bob wasn't reversible for YV12, and I did try to get the script to work with YUV, but one of the functions complained.

At this point I'm just going to get on to the next project. The client was super happy with the results and the detection worked correctly for the vast majority of the frames, and for those frames where it didn't, the impact wasn't too noticeable.

Based on the input I've received, I think my current detection scheme is as good as I'm going to get with this technology.

Thanks to everyone for their input!

[StainlessS]

Lollo2, thanks for the script. I did look at StanlessS' RT_Stats package to see if any of his functions might provide better detection. I looked at RT_FrameDifference but chose not to use it because it "returns the average pixel difference for the two comparison frames." By comparison, the YDiff built-in functions returns "the absolute difference of pixel value between the current and previous frame." Gavino described it further in this post, "it is the average absolute luma difference between pixels at the same location - for each pixel position, the absolute luma difference is calculated, the results are summed and divided by the total number of pixels to find the average."

So, the Ydiff algorithms makes the number "blow up" pretty quickly when there are any differences on a pixel-by-pixel level, even if the average is similar. When comparing a scan line that is two rows down, the overall average is likely to be similar, but because the location of pixels has moved, the pixel-by-pixel comparison can be quite different.

RT_FrameDifference(Chromaweight=0.0), LumaDifference(), and YDiff built-in functions, are all similar-ish and as Gavino described,
YDiffXXXX() just has single source clip and comparison is with eg previous or next frame.
LumaDifference() takes two source clips, and compares same frameno of each clip.
RT_FrameDifference(Chromaweight=0.0) or RT_LumaDifference(), takes two source clips (c and c2) (which may or may not be same clip)
Returns difference between clip c frame n area x,y,w,h and clip c2 frame n2 area x2,y2,w,h. (range 0.0 -> 255.0).
And using AltScan (ie interlaced alternate field scanning) arg, can compare fields, eg even/odd of current frame, current even with next even, current odd, prev odd, prev odd with next even,
etc. [EDIT: by mods to n,n2 for relative frames, and y,y2 for fields of those frames]



RT_FrameDifference(clip c,clip c2,int "n"=current_frame,int "n2"=current_frame,Float "ChromaWeight"=1.0/3.0,
int "x"=0,int "y"=0,int "w"=0,int "h"=0,int "x2"=x,int "y2"=y,bool "Altscan"=false,bool "ChromaI"=false,
Int "Matrix"=(Width>1100||Height>600)?3:2)
Returns difference between clip c frame n area x,y,w,h and clip c2 frame n2 area x2,y2,w,h. (range 0.0 -> 255.0).

Args:-
c, c2, clips to difference.
n, default = current_Frame. Frame from clip c.
n2, default = current_Frame. Frame from clip c2.
ChromaWeight, default 1.0/3.0. Range 0.0 -> 1.0.
Y8, returns same as LumaDifference. ChromaWeight ignored.
YUV,
Weighting applied YUV chroma:- (1.0 - ChromaWeight) * Lumadif + ChromaWeight * ((Udif + Vdif)/2.0).
RGB,
If ChromaWeight > 0.0, then returns same as RT_RGBDifference() [ie average RGB pixel channel difference].
If ChromaWeight == 0.0, then returns same as RT_LumaDifference() using Matrix arg to convert RGB to YUV-Y Luma.
x,y,w,h. All Default 0 (full frame), for clip c.
x2,y2. x2 defaults x, y2 defaults y, for clip c2.
Altscan, default false. If true then scan only every other scanline starting at clip c y coord, and clip c2 y2 coord.
ChromaI, default false. If YV12 and ChromaI, then do YV12 interlaced chroma scanning.
Ignored if not YV12. Both YV12 clips must be same, cannot difference one that is progressive and one that has interlaced chroma.
Matrix, default (c.Width>1100||c.Height>600||c2.Width>1100||c2.Height>600)?3:2
Conversion matrix for conversion of RGB to YUV-Y Luma. 0=REC601 : 1=REC709 : 2 = PC601 : 3 = PC709.
Default = (c.Width > 1100 OR c.Height>600 OR c2.Width > 1100 OR c2.Height>600) then 3(PC709) else 2(PC601). YUV not used
Matrix only Used if RGB and ChromaWeight == 0.0, where returns same as RT_LumaDifference().
Note, 'x2' and 'y2' default to 'x' and 'y' respectively.
v2.0, c and c2 need not be same dimensions but BEWARE, x2 and y2 default to x and y also w and h default to 0
which is converted to c.width-x and c.height-y, may be best to provide x2, y2, w and h where c2 not same dimensions as c.


EDIT:

Common to many RT_ funcs
The compiletime/runtime clip functions share some common characteristics.
The 'n' and (where used) 'n2' args are the optional frame numbers and default to 'current_frame' if not specified.
The x,y,w,h, coords specify the source rectangle under scrutiny and are specified as for Crop(), the default 0,0,0,0 is full frame.
If 'interlaced' is true (sometimes use Altscan arg name instead), then every other line is ommited from the scan, so if eg x=0,y=1, then
scanlines 1,3,5,7 etc are scanned, if eg x=0,y=4 then scanlines 4,6,8,10 etc are scanned. The 'h' coord specifies the full height scan ie
same whether interlaced is true or false, although it will not matter if the 'h' coord is specified as eg odd or even, internally the
height 'h' is reduced by 1 when interlaced=true and 'h' is even.
Some of the functions have 'x2' and 'y2' args for a second frame (which could be the same frame).
Note, 'x2' and 'y2' default to 'x' and 'y' respectively.
v2.0, Where two clip comparison, c and c2 need not be same dimensions but BEWARE, x2 and y2 default to x and y also w and h default to
0 which is converted to c.width-x and c.height-y, may be best to provide x2, y2, w and h where c2 not same dimensions
as c.


EDIT: from lollo2's first link, see stuff in PURPLE.

More configurable. More Bullet proof.


# FieldBadAreaDetect.avs # https://forum.doom9.org/showthread.php?p=1960487#post1960487

# Requires, RT_Stats
# If FrameSel line uncommented [would also require RT_ForceProcess line uncommented] then shows bad frames only. [And will require FrameSel plugin].

AviSource(".\spots_beg_s9_ss5a_amtv_v2_cut.avi")
#AssumeTFF # Set appropriately [Not required in this script so far]

#### CONFIG ####
TOPFIELDBAD = True # True=Top field BAD, False = Bottom field BAD

BADAREA = 4 # Approximate area of frame or field where crap resides. [Make for better more bullet proof detection, can make TH a lot higher without failed detect].
# 0=1st or Top quarter of frame or field.
# 1=2nd quarter of frame or field.
# 2=3rd quarter of frame or field.
# 3=4th or bottom quarter of frame or field.
# 4=Top half of frame or field,
# 5=Bottom half of frame or field,
# 6=full frame or field,
# Usable settings for Lollo2 clip are 0,1,4 and 6. [Lollo2 clip:- bad top field, top half of frame or field]
# 6 is as previous version testing full field, but only the top of lollo2 field is bad, so will get better numbers using 0,1 or 4, than original equivalent of 6.
# If using 0,1 or 4, suggest raising TH.

TH = 20.0 # Picked out of the hat. [Suggest about 40 or above if using BADAREA=0 or 1 or 4 for lollo2 clip]

CW = 1/3.0 # Below from RT_Stats docs. Switching off, ie CW = 0.0 might give better results [only test luma diff].
# ChromaWeight, default 1.0/3.0. Range 0.0 -> 1.0.
# Y8, returns same as LumaDifference. ChromaWeight ignored.
# YUV,
# Weighting applied YUV chroma:- (1.0 - ChromaWeight) * Lumadif + ChromaWeight * ((Udif + Vdif)/2.0).
# RGB,
# If ChromaWeight > 0.0, then returns same as RT_RGBDifference() [ie average RGB pixel channel difference].
# If ChromaWeight == 0.0, then returns same as RT_LumaDifference() using Matrix arg to convert RGB to YUV-Y Luma.
###

SUBS_Y = 20 # Subtitles offset from top, make a bit lower than FrameSel metrics position.
Frames = ".\Frames.txt"
#### END CONFIG ####

Assert(0 <= BADAREA <= 6,"0 <= BADAREA <= 6")

nAREA = (BADAREA<4) ? 4 : (BADAREA<6) ? 2 : 1 # 4=Areas are quarters, 2=Areas are halves, 1=Areas are full frame/field.
QUAD = Height / 16
BadH = QUAD*16/nAREA
QOFF = Select(BADAREA, 0*BadH, 1*BadH, 2*BadH, 3*BadH, 0*BadH, 1*BadH, 0*BadH)

BadY = QOFF + ((TOPFIELDBAD ) ? 0 : 1)
GoodY = QOFF + ((TOPFIELDBAD ) ? 1 : 0)

Frames = Frames.RT_GetFullPathName
RT_FileDelete(Frames)
#RT_DebugF("BADAREA=%d BadY=%d GoodY=%d BadH=%d",BADAREA,BadY,GoodY,BadH,name="LOLLO2: ") # just paranoia.

SSS = """
n = current_frame
Top2Bot = RT_FrameDifference(Last,Last,n=n,n2=n ,y=BadY,y2=GoodY,h=BadH,AltScan=True,ChromaWeight=CW,ChromaI=True) # Diff top and bottom fields of current frame n
Prv2Cur = RT_FrameDifference(Last,Last,n=n,n2=n-1,y=BadY,y2=BadY ,h=BadH,AltScan=True,ChromaWeight=CW,ChromaI=True) # Diff bad field Prev and bad field Curr
Cur2Nxt = RT_FrameDifference(Last,Last,n=n,n2=n+1,y=BadY,y2=BadY ,h=BadH,AltScan=True,ChromaWeight=CW,ChromaI=True) # Diff bad field Curr and bad field Next
Bingo = (Top2Bot > TH && Prv2Cur > TH && Cur2Nxt > TH)
(Bingo) ? RT_WriteFile(Frames,"%d",n,Append=True) : NOP
Subtitle(RT_String("%d] Top2Bot=%.2f : Prv2Cur=%.2f : Cur2Nxt=%.2f",n,Top2Bot,Prv2Cur,Cur2Nxt),Text_Color=(Bingo)?$FF40FF:$FFFF00,Y=SUBS_Y)
"""

ScriptClip(SSS)
# Try uncomment both below [Will need Framesel]
#RT_ForceProcess # UnComment to force Frames file to be fully written, and then available to any further scripting after this line.
#Return FrameSel(Last,Cmd=Frames,Show=True,REJECT=FALSE) # Change to REJECT=TRUE to view only non bad frames

Return Last

We test for bad top field, and only upper half of field instead of entire field [EDIT: BADAREA = 4], better numbers, about 60.0 instead of original ~40.0.
EDIT: BADAREA = 1 or 2, would get similar-ish numbers [top or 2nd quarter of field], better than original equivalent of BADAREA = 6 [full field].

https://i.postimg.cc/brYZvjvf/Test-02.jpg (https://postimages.org/)

EDIT: Added CW ChromaWeight config.

EDIT: Changed

BADAREA = 4 # Approximate area of frame or field where crap resides. [Make for better more bullet proof detection, can make TH a lot higher without false detect

to

BADAREA = 4 # Approximate area of frame or field where crap resides. [Make for better more bullet proof detection, can make TH a lot higher without failed detect

[Frank62]

I did look at the top and bottom of the frame to see if the shift created a black line, but as Lollo2 points out, because this is an analog capture there is too much garbage in those lines.
So it's NOT complete blackness. Clear then.

[lollo2]

So it's NOT complete blackness. Clear then.

It is. For example, if a field is shifted by 1 line down, you have 1 black line more and the field starting from line+1.
The problem is that at the top there are always 2/4/6/ more black lines inside an analog capture, so the blackness of the top of the frame can't be a reference.
The same for field shifted up with the bottom lines.

Not sure if explains something, but here a link: https://forum.videohelp.com/threads/409177-VHS-Horizontal-Stabilisation-part-2-Electric-Boogaloo#post2686220

In addition, here an example of a field shift: https://drive.google.com/file/d/1F5hrYigHnGPbdcSslZOp51RIBUvfq6Yl/view?usp=sharing

If you have any brillant idea about detection and need more explanation/samples just let me know ;)

[johnmeyer]

So I think StainlessS is saying that his RT_FrameDifference function does the same comparison as the built-in YDiff functions, but has (like so may of his wonderful RT_Stats functions) a lot of additional features and controls. I didn't realize that it was summing the absolute value differences between each pixel in the two clips, within the specified area. I thought it was taking averages.

It seems like there must be a better way to sense when all fields in a clip have been displaced upward or downward. If you have a diagonal black line on a white background, and you move all even lines up by two lines (i.e., to the next even line above), the visual effect is obvious and looks very much like what you get when you take a freeze frame of interlaced video with horizontal motion: you get "teeth." (Although this is strictly a spatial issue whereas interlacing teeth are temporal and are the result of taking a snapshot of two different moments in time, but then displaying them at the same time.)

In thinking about this trivial example, it seems like one approach would be to predict where each even line should be by interpolation, using the two adjacent odd lines, and then compare this prediction with the actual even line. If the prediction logic is good, when the fields are where they are supposed to be, the difference metrics should be small compared to what happens if the fields have been shifted.

On another issue, the two long videos I was given each had the field displacement last for minutes; it was not going back and forth every few frames. Therefore, it might be useful to add additional "flywheel" logic which would not engage the fix (to displace the field downwards) unless the problem was detected for several frames, and would then go backwards a few frames to start the fix when the metrics first showed a problem. The same thing would be true for turning off the fixing logic.

I guess I won't shut this down completely just yet and will see if I can come up with a more reliable solution.

[StainlessS]

YDiff is not the difference between the average luma of the two frames. Instead, it is the average absolute luma difference between pixels at the same location - for each pixel position, the absolute luma difference is calculated, the results are summed and divided by the total number of pixels to find the average.

As Gavino said, is the SAD (Sum Of Absolute Difference) / the number of summed samples(pixels). [It probably has a fancy name like 'SAD', but I dont recall it]
Where YUV and RT_FrameDifference(ChromeWeight>0.0) also does weighted differencing on chroma. ChromeWeight=1.0 totally ignores luma and only differences chroma.
When differencing YV12 chroma and ChromaI=True, then chroma samples corresponding to the selected area luma pixels are differenced but taking into account Field Order (AssumeTFF/AssumeBFF)
and the Interlaced/AltScan arg. JMac968 asked for interlaced/altscan on interlaced YV12 chroma, sounded only a bit tricky so I said I would do it, turned out to be as pleasant as consuming a bucket of dog poop.
EDIT: Or is it JMac698?

[poisondeathray]

This is oversimplified for generalized use, but for John's specific clip, where there is only 1 permutation of shifting, adding YDifferenceToPrevious + YDifferenceToNext seems to help . Of course check other sections and for false positives


ConditionalFilter(source, output, source, "YDifferenceFromPrevious(source) + YDifferenceToNext(source)", ">", " YDifferenceFromPrevious(output) + YDifferenceToNext(output)")



Other manipulations to increase "accuracy" for "automatic" detections you can try are to use a central area (crop pixels off the top/bottom, because of the analog crap/noise may be skewing results) , use other metrics PSNR/VMAF/SSIM etc... VMAF might be interesting because it has a temporal component built in. AVS version doesn't have a XML reader so'd you have to convert it to a format suitable for ConditionalReader/ConditionalSelect .

[johnmeyer]

Poisondeathray,

You always have such great ideas. I don't know why I didn't think to look both ways (i.e., Prev/Next). What's more, reading your suggestion made me remember that YDiff detection logic often works better by using ratios. This is because the YDiff metrics often vary widely from one part of the video to the next so absolute values can be misleading, but ratios often make the detection more sensitive and accurate. (I have written many scene detection scripts, and ratios are the key to making them work.)

So, I took your idea, and then did my test based on ratios and voilà, the darn thing started to work about 10x better than before. It still fails to detect some times, but in far, far fewer places and more importantly, only fails where the residual field shift is almost unnoticeable. It also does still sometimes apply the fix where it isn't needed, but also in far, far fewer places, and doesn't call attention to itself when it moves the even field when it shouldn't.

It works perfectly on the sample clip I provided, whereas my previous scripts all failed on certain frames.

So, the following script is massively better than what I did in 2015 and also much better than what I posted at the beginning of this thread.

I should add that I have spent quite a few hours trying all sorts of other techniques and approaches, including using MVTools2 to estimate the even field and compare that with both the shifted and unshifted version. I also tried Yadif instead of Bob, and experimented with all its settings. All these things just shifted the problem around.

However, looking in both direction combined with using a ratio was the key to making this work.

Thanks to everyone for their help. I am not going to take this any further. I know it would be more useful if it automatically detected parity (as was suggested in posts above), but I'm sick of working on this so I'm not going to refine it any further.
#Script to shift fields, but only on frames where problem exists
#Hard-wired for shifting even fields
#John Meyer
#Written 2025

#These plugins are for denoising
Loadplugin("E:\Documents\My Videos\AVISynth\AVISynth Plugins\plugins\MVTools\mvtools2.dll")
LoadPlugin("E:\Documents\My Videos\AVISynth\AVISynth Plugins\plugins\Cnr2.dll")
LoadPlugin("E:\Documents\My Videos\AVISynth\AVISynth Plugins\plugins\Film Restoration\Script_and_Plugins\RemoveGrainSSE2.dll")

#Get input and then Bob.
input=AVISource("e:\fs.avi").killaudio().ConvertToYV12()
source=input.assumetff().bob(0,1)

#Perform the shift on bobbed source by first cropping and then adding border
Hshift = 0
Vshift = 2
even = selecteven(source)
odd = selectodd (source)

even = crop(even,Hshift,Vshift,0,0)
even = addborders(even,0,0,Hshift,Vshift)

#This is the fixed, bobbed output. Reinterlace at the end of the script
output = interleave(even,odd)

/*
#Debugging: make sure YDifference metrics reliably detect field shift
script = """Subtitle("\nMetric = " + String( \
(YDifferenceFromPrevious(source) + YDifferenceToNext(source)) \
/ (YDifferenceFromPrevious(output) + YDifferenceToNext(output))), lsp=0)"""
Scriptclip(output, script)
*/

#Choose the original source if no field shift detected. Otherwise use the fixed version
fixed = output.scriptclip("""(YDifferenceFromPrevious(source) + YDifferenceToNext(source)) \
/ (YDifferenceFromPrevious(output) + YDifferenceToNext(output)) > 1\
? output : source""")


#Convert back to interlaced
evenbob = selecteven(fixed)
oddbob = selectodd(fixed)
EvenOrig = separatefields(evenbob).selecteven()
OddOrig = separatefields(oddbob).selectodd()
interleave(EvenOrig,OddOrig)
fixed_field = weave()

#Enable next two lines for denoising
chroma=fixed_field.Cnr2("oxx",8,16,191,100,255,32,255,false) #VHS
fixed_field = MDegrain2i2(chroma,8,4,400,0)

return fixed_field

#Alternate output to view before/after
#Interleave(input,fixed_field)

# Enable MT!
Prefetch(6)

[lollo2]

I don't know why I didn't think to look both ways (i.e., Prev/Next)

In the tecnique developped by Stainless used in my script you already have comparison (difference) at field level for:

Top versus Bottom for odd versus even field of the same frame
Previous versus Current for even fields
Current versus Next for even fields
Previous versus Current for odd fields
Current versus Next for odd fields

In my sources, this detects almost all fields shifted by multiple lines (>1) and many single line shift (not all). It also generates many false positive, as I said.

[StainlessS]

Top versus Bottom for odd versus even field of the same frame
Previous versus Current for even fields
Current versus Next for even fields
Previous versus Current for odd fields
Current versus Next for odd fields
Could if required also difference even/odd of either prev or next frame. [Prev: n=n-1, n2=n-1 :: Next n=n+1, n2=n+1] and compare with even/odd of current frame.
Also could difference Prev even with Next even, or Prev odd with Next odd, maybe comparing with prev to curr or next to curr even or odd field.
Or previous even to current odd, previous odd to current even: Or next even to current odd, next odd to current even.
Using those, in some kind of extended logic to avoid false detects. [have to keep in mind scene cuts though].
EDIT: Above for any kind of source problem detect, not just for OP problem.

[lollo2]

Using those, in some kind of extended logic to avoid false detects. [have to keep in mind scene cuts though].

Yes. However, I do not see an easy way to avoid false positive at scene cuts.

EDIT: Above for any kind of source problem detect, not just for OP problem.

Yes, I use your (modified) script to detect at the same time field/frame corruption and field shift.
Unfortunately we can only specify 1 threshold (th) values, which is somehow a compromize. Otherwise you have to run the script twice.
But because I run a visual check at slow motion anyhow I do not do that, I just use it once for a preliminary gross check.

[real.finder]

It is. For example, if a field is shifted by 1 line down, you have 1 black line more and the field starting from line+1.
The problem is that at the top there are always 2/4/6/ more black lines inside an analog capture, so the blackness of the top of the frame can't be a reference.
The same for field shifted up with the bottom lines.

Not sure if explains something, but here a link: https://forum.videohelp.com/threads/409177-VHS-Horizontal-Stabilisation-part-2-Electric-Boogaloo#post2686220

In addition, here an example of a field shift: https://drive.google.com/file/d/1F5hrYigHnGPbdcSslZOp51RIBUvfq6Yl/view?usp=sharing

If you have any brillant idea about detection and need more explanation/samples just let me know ;)

AVISource("field_shift_example.avi").AssumeFPS(25).ConvertToYV16
Crop(0, 0, -718, -0)
CorrectFieldShift (https://forum.doom9.org/showpost.php?p=2016270&postcount=7)(-6).CorrectFieldShift(-6,false)

https://i.postimg.cc/7YrzFdTT/sample-1-original-ts000004.png (https://postimages.org/)

[lollo2]

Thanks real.finder!

Fixing the problem is quite easy, just shift in the appropriate direction the field by the appropriate number of lines, as you showed.

The difficulty, and the whole thread, is about the detection of the shifts, especially when it consists of a single line :(

[StainlessS]

Yes. However, I do not see an easy way to avoid false positive at scene cuts.

...

Unfortunately we can only specify 1 threshold (th) values, which is somehow a compromize. Otherwise you have to run the script twice.


1) Well part of the detect logic has to ensure that is not a scene cut, AND is good detect.

2) Have more than a single threshold, easily done.

Simpler if detect (possibly multiple) states separately, and then apply logic to establish final decision.

Detect=(diff1>TH1 && diff2 > TH2 && diff3 < TH3)
sc = (diff4 < diff5 && diff6 > diff7)
Bingo = (Detect && !sc)

[johnmeyer]

Scene changes would be nice icing on the cake, but they are not a major issue, since the visual disruption of the scene change almost always masks the problem.

I would have to look at more than my two videos in order to know if multiple thresholds would help. One of the reasons I use ratios is that it often eliminates the need to test for different thresholds. As one example, as I walked through hundreds of frames over my 90-minute video, the YDiff values went from 2 up to 30, but generally stayed in a narrow range within most scenes. There is no way a threshold could be used because of that wide variation. If you start using four different values (my final script) or six (I think that was what StainlessS did), it gets really complicated if you use thresholds.

I am not a fan of adding more and more tests unless there is a specific situation which only one test can determine and which the others miss. I find that as the number of tests increase, they tend to either cancel each other out, thus decreasing the detection sensitivity, or they move in lockstep, in which case they are not needed and just obscure whether the detection is working.

[StainlessS]

My prev post was intended as general detect, not specifically your OP problem.
A big benefit to having separate detect and other states (eg scene cut) is that can show status on-frame
of each of the states, and so makes it easier to tweak settings/thresholds etc for each state separately.

Whenever you make some state detector, it is also a pretty good idea to see if you can also implement another
to detect false +ve's. [try avoid creating more problems]

EDIT: Above certainly true during development, maybe combine states / reduce variables or logic in final script,
but only if you are certain that you will not need to go back to development mode.

[real.finder]

Thanks real.finder!

Fixing the problem is quite easy, just shift in the appropriate direction the field by the appropriate number of lines, as you showed.

The difficulty, and the whole thread, is about the detection of the shifts, especially when it consists of a single line :(

do have video samples for consists of a single line?

[lollo2]

do have video samples for consists of a single line?

Here a sample: https://drive.google.com/file/d/1h0fvA9nyagxEp6SFLQhvxYMK7LqpW0y-/view?usp=sharing

At frame 10 even field is shifted by 1 line up; not detected by my script.

The TV logo helps finding the shift when performing a "manual" check.

[johnmeyer]

My script also failed to detect the problem. However, part of the problem is that this is a progressive source. This eliminates the temporal aspect of the artifact.

[lollo2]

Yes John, the source is PsF (Progressive segmented Frame), i.e. the 2 fields are from the same moment in time. All captures of sources originated by "films" are like that.

[real.finder]

Here a sample: https://drive.google.com/file/d/1h0fvA9nyagxEp6SFLQhvxYMK7LqpW0y-/view?usp=sharing

At frame 10 even field is shifted by 1 line up; not detected by my script.

The TV logo helps finding the shift when performing a "manual" check.

#Script to shift fields, but only on frames where problem exists
#Hard-wired for shifting fields by even number
#original script by John Meyer
#modded to use "StackVertical" instead of "addborders" to make it more accurate
#mod V1.2
Function CorrectFieldShift(clip c, int "Vshift", bool "FirstField"){
source=GetParity(c) ? c.assumetff().bob(0,1) : c.assumebff().bob(0,1)

#Perform the shift on bobbed source by first cropping and then adding border
Vshift = Default(Vshift, 2)
FirstField = Default(FirstField, true)
even = selecteven(source)
odd = selectodd (source)

even = FirstField ? crop(even,0,Vshift > 0 ? Vshift : 0,0,Vshift < 0 ? Vshift : 0) : even
even = FirstField ? Vshift > 0 ? StackVertical(even,crop(even,0,0,0,-(Height(even)-Vshift))) : StackVertical(crop(even,0,Height(even)+Vshift,0,0),even) : even

odd = !FirstField ? crop(odd,0,Vshift > 0 ? Vshift : 0,0,Vshift < 0 ? Vshift : 0) : odd
odd = !FirstField ? Vshift > 0 ? StackVertical(odd,crop(odd,0,0,0,-(Height(odd)-Vshift))) : StackVertical(crop(odd,0,Height(odd)+Vshift,0,0),odd) : odd

output = interleave(even,odd)

fixed = output.scriptclip("""YDifferenceFromPrevious(source.aBlur(5,blurv=0)) > \
YDifferenceFromPrevious(aBlur(5,blurv=0)) ? last : source""", args="source")
fixed.interlaced60or50(BFF=!(GetParity(c)))
}
with CorrectFieldShift(-2,false).CorrectFieldShift(-2) and this mod it seems work fine

[lollo2]

Did not check the code deeper, but when running:

AviSource("ufo_s1a_amtv_v2_slfs.avi")
CorrectFieldShift(-2,false).CorrectFieldShift(-2)

Function CorrectFieldShift(clip c, int "Vshift", bool "FirstField"){
source=GetParity(c) ? c.assumetff().bob(0,1) : c.assumebff().bob(0,1)

#Perform the shift on bobbed source by first cropping and then adding border
Vshift = Default(Vshift, 2)
FirstField = Default(FirstField, true)
even = selecteven(source)
odd = selectodd (source)

even = FirstField ? crop(even,0,Vshift > 0 ? Vshift : 0,0,Vshift < 0 ? Vshift : 0) : even
even = FirstField ? Vshift > 0 ? StackVertical(even,crop(even,0,0,0,-(Height(even)-Vshift))) : StackVertical(crop(even,0,Height(even)+Vshift,0,0),even) : even

odd = !FirstField ? crop(odd,0,Vshift > 0 ? Vshift : 0,0,Vshift < 0 ? Vshift : 0) : odd
odd = !FirstField ? Vshift > 0 ? StackVertical(odd,crop(odd,0,0,0,-(Height(odd)-Vshift))) : StackVertical(crop(odd,0,Height(odd)+Vshift,0,0),odd) : odd

output = interleave(even,odd)

fixed = output.scriptclip("""YDifferenceFromPrevious(source.aBlur(5,blurv=0)) > \
YDifferenceFromPrevious(aBlur(5,blurv=0)) ? last : source""", args="source")
fixed.interlaced60or50(BFF=!(GetParity(c)))
}

I get https://drive.google.com/file/d/1Y50wEbxJkFkEMja-ZxGgnfWeHQbRJ1hH/view?usp=drive_link

[real.finder]

I get https://drive.google.com/file/d/1Y50wEbxJkFkEMja-ZxGgnfWeHQbRJ1hH/view?usp=drive_link

http://avisynth.nl/index.php/GRunT

[lollo2]

Sure the run time enviroment (GRunT)!

Now it works: https://drive.google.com/file/d/1ERKR3iiCMiCl1MQE52FITx_v4OKcHdLf/view?usp=sharing

Is it possible to modify your function to just have the detection of frame number where the shift of the fields occurs, the type (even / odd) and the amount of the line shifted, and report all this in a file?

I was thinking in automatically increasing the parameter "Vshift" in your function to check from 1 line shift to a high number (8 or 9 or more) and iterate the process, but you may have a better idea ;)

In any case thanks for your help in this subject!

[real.finder]

Is it possible to modify your function to just have the detection of frame number where the shift of the fields occurs, the type (even / odd) and the amount of the line shifted, and report all this in a file?

I think it can

fixed = output.scriptclip("""YDifferenceFromPrevious(source.aBlur(5,blurv=0)) > \
YDifferenceFromPrevious(aBlur(5,blurv=0)) ? last : source""", args="source")
fixed.interlaced60or50(BFF=!(GetParity(c)))

the Modifications should be done in this part, you can use some previous StainlessS code to do it

[lollo2]

Ok thanks for the hints