Is it possible to stabilize this horizontally jerkyvideo?

[rs008f]

It's a PAL/SECAM G tape. I'm using a TBC-1000.

https://mega.nz/#!T19inJjK!nRCI9349rOQHJ1USeewaUD3y7oJ9klz5s87fDxZDuTE

[juhok]

Yes. Fix your VCR.

[johnmeyer]

Is this VHS?

If so, is it recorded in the EP (6-hour) mode?

Have you tried adjusting the tracking manually?

Do other PAL G tapes play this way, or is the problem just with this one tape?

The last question is an attempt to determine if this is indeed a VCR problem, or just a problem with this one tape.

There are multiple things going on, judging by the various horizontal offsets that are happening at regular intervals, looking vertically down the left edge. In addition, you have the considerable wobble.

[jmac698]

I'm glad to have a sample. Perhaps I can use this to work on my software TBC.

I'm dying to work on that but I have to warn you, I'm taking classes and homework due tomorrow so the earliest I should even think about this is Sunday.

[robert.n]

I developed my own tool to fix horizontal shaking in digitized videos: https://github.com/rsnitsch/vhs-deshaker

[lollo2]

Does it work for vertical shift as well? Is it able to go at field level? It happens sometimes in a capture that the even or the odd fields are shifted by 1 or more lines. Easily fixable by hand with crop().addborders(), but if an automatic tool is available it may have a large interest

[Selur]

Any plans for an Avisynth or Vapoursynth plugin?

[robert.n]

Does it work for vertical shift as well? Is it able to go at field level? It happens sometimes in a capture that the even or the odd fields are shifted by 1 or more lines. Easily fixable by hand with crop().addborders(), but if an automatic tool is available it may have a large interest
The tool, as of now, can only handle horizontal deshaking. In fact at first I wanted to name it "vhs-horizontal-deshaker"...

I did not know that digitized VHS videos can also suffer from vertical shaking or shifted lines. Do you know why that happens? Can you upload some example video files (ideally separate files for each problem type). If it's easy to fix then I might develop another tool in my free time... or extend vhs-deshaker.

My first recommendation for vertical shaking would be to try and use a normal deshaking tool. In my case the performance of traditional deshaking tools was bad because - I guess - the shaking was very severe and irregular between subsequent frames. Therefore I developed my own tool, which yielded much better video quality. But the traditional deshaking tools DID help somewhat at least... if I recall correctly about 50% of the shaking was gone. With my own tool I got to about 90%.

Any plans for an Avisynth or Vapoursynth plugin?
Nope, sorry.

[Boulder]

Would it be enough to use TurnLeft, deshake and then TurnRight?

[lollo2]

Do you know why that happens?

Sometimes, when the tapes has a problem, the lineTBC of the VCR is not able to full rebuild the lines architecure, and there is a shift in the even or in the odd field.

Can you upload some example video files (ideally separate files for each problem type).

Here a sample: https://drive.google.com/file/d/1SwPfLYxGZ_rZ6Zbl_Njor7CoigLOEvev/view?usp=share_link

My first recommendation for vertical shaking would be to try and use a normal deshaking tool.

They are not able to fix the problem; the required adjustement is very small (i.e. crop(0,0,0,-1).addborders(0,1,0,0)) and they all fail. The manual adjustement is easy and perfect, but it is tedious if you have many shifted fields.

Would it be enough to use TurnLeft, deshake and then TurnRight?

It should work, if the tool is able to fix down to 1 line shift

[robert.n]

Sometimes, when the tapes has a problem, the lineTBC of the VCR is not able to full rebuild the lines architecure, and there is a shift in the even or in the odd field.

Here a sample: https://drive.google.com/file/d/1SwPfLYxGZ_rZ6Zbl_Njor7CoigLOEvev/view?usp=share_link

What is the problem with that video? It looks fine to me at first sight. (Maybe I need glasses. ;) ) Is it about the faulty lines at the very bottom?

[poisondeathray]

Sometimes, when the tapes has a problem, the lineTBC of the VCR is not able to full rebuild the lines architecure, and there is a shift in the even or in the odd field.

Here a sample: https://drive.google.com/file/d/1SwPfLYxGZ_rZ6Zbl_Njor7CoigLOEvev/view?usp=share_link

They are not able to fix the problem; the required adjustement is very small (i.e. crop(0,0,0,-1).addborders(0,1,0,0)) and they all fail. The manual adjustement is easy and perfect, but it is tedious if you have many shifted fields.

It should work, if the tool is able to fix down to 1 line shift


A "hacky" solution for that would be to detect the top non Y=16 line, and use conditionalfilter for the line shift . But it would be a less generalizable solution to generic problem - Is that pattern consistent for that setup ? ie. In that sample, it's always the same field shifted in the same direction, and the same magnitude - the pattern does not change

This will "fail" if the content is dark, maybe a fade in , that sort of thing - it will give a false positive


o=AVISource("PATH\ufo_sIII3c_amtv_4_v2.avi").ConvertToYV24(interlaced=true)

f1=o.assumebff().separatefields().selecteven()
f2=o.assumebff().separatefields().selectodd().crop(0,0,0,-1,true).addborders(0,1,0,0)
interleave(f1,f2)
weave()
f2up=last

#r=blankclip(o, height=1 , color=color_red)
#overlay(o, r, y=10, opacity=0.8) #find line position

det=o.crop(0,10,0,-565,true)

#ConditionalFilter(det, f2up, o, "AverageLuma()", ">", "30", show=true) #debug threshold
ConditionalFilter(det, f2up, o, "AverageLuma()", ">", "30", show=false)
ConvertToYV16(interlaced=true)

[lollo2]

Thanks poisondeathray for your attempt, higly appreciated. I will experiment it.

Is that pattern consistent for that setup ?

The pattern is not always the same: it can be even field shift up, even field shift down (rare), odd field shifted up (rare) , odd field shifted down. By a number of lines from 1 to 5. Rarely both fields are shifted.

To detect them I use this approach (developed with help from StainLessS); it produces many false positive and does not detect many of the 1 line only shifts:

AviSource(video)

# plugins directory
plugins_dir="C:\Users\giuse\Documents\VideoSoft\MPEG\AviSynth\extFilters\"

# RT_Stats
loadPlugin(plugins_dir + "RT_Stats_25&26_x86_x64_dll_v2.00Beta12_20181125\Avisynth26\RT_Stats_x86.dll")

# FrameSel
loadPlugin(plugins_dir + "FrameSel_x86_x64_dll_v2-20_20180420\Avisynth+_x86\FrameSel_x86.dll")

AssumeTFF()

threshold = 7

frames_file = video+"_bad_frames_threshold_"+string(threshold)+"_v2.txt"
frames_file = frames_file.RT_GetFullPathName
RT_FileDelete(frames_file)

report = """
n = current_frame
Top2Bot = RT_FrameDifference(last.separateFields.selectEven(),last.separateFields.selectOdd(),n=n,n2=n,ChromaWeight=0.0)
Prv2CurE = RT_FrameDifference(last.separateFields.selectEven(),last.separateFields.selectEven(),n=n,n2=n-1,ChromaWeight=0.0)
Cur2NxtE = RT_FrameDifference(last.separateFields.selectEven(),last.separateFields.selectEven(),n=n,n2=n+1,ChromaWeight=0.0)
Prv2CurO = RT_FrameDifference(last.separateFields.selectOdd(),last.separateFields.selectOdd(),n=n,n2=n-1,ChromaWeight=0.0)
Cur2NxtO = RT_FrameDifference(last.separateFields.selectOdd(),last.separateFields.selectOdd(),n=n,n2=n+1,ChromaWeight=0.0)
Bingo = (Top2Bot > threshold && Prv2CurE > threshold && Cur2NxtE > threshold) || (Top2Bot > threshold && Prv2CurO > threshold && Cur2NxtO > threshold)
(Bingo) ? RT_WriteFile(frames_file,"%d",n,Append=True) : NOP
subtitle(RT_String("%d] Top2Bot=%.2f : Prv2CurE=%.2f : Cur2NxtE=%.2f : Prv2CurO=%.2f : Cur2NxtO=%.2f",n,Top2Bot,Prv2CurE,Cur2NxtE,Prv2CurO,Cur2NxtO),\
text_color=(Bingo)?$FF40FF:$FFFF00,y=10)
"""

ScriptClip(report)

RT_ForceProcess
return FrameSel(last,Cmd=frames_file,Show=true,REJECT=false)

return last

and then manually shift the fields:

function shift_fields_GMa(clip c, int frame_number, line_shift_even, line_shift_odd)
{
# separate fields tff
c_tff_sep=c.AssumeTFF().separateFields()

# separate fields tff even
c_tff_sep_even=c_tff_sep.SelectEven()

# separate fields tff odd
c_tff_sep_odd=c_tff_sep.SelectOdd()

# shift field even
c_tff_sep_even_rep = (line_shift_even > 0) ?\
c_tff_sep_even.trim(0,frame_number-1)\
++c_tff_sep_even.trim(frame_number,frame_number).crop(0,0,0,-line_shift_even).addborders(0,line_shift_even,0,0)\
++c_tff_sep_even.trim(frame_number+1,0)\
:\
c_tff_sep_even.trim(0,frame_number-1)\
++c_tff_sep_even.trim(frame_number,frame_number).crop(0,-line_shift_even,0,0).addborders(0,0,0,-line_shift_even)\
++c_tff_sep_even.trim(frame_number+1,0)

# shift field odd
c_tff_sep_odd_rep = (line_shift_odd > 0) ?\
c_tff_sep_odd.trim(0,frame_number-1)\
++c_tff_sep_odd.trim(frame_number,frame_number).crop(0,0,0,-line_shift_odd).addborders(0,line_shift_odd,0,0)\
++c_tff_sep_odd.trim(frame_number+1,0)\
:\
c_tff_sep_odd.trim(0,frame_number-1)\
++c_tff_sep_odd.trim(frame_number,frame_number).crop(0,-line_shift_odd,0,0).addborders(0,0,0,-line_shift_odd)\
++c_tff_sep_odd.trim(frame_number+1,0)

# repaired video
c_rep=interleave(c_tff_sep_even_rep,c_tff_sep_odd_rep).Weave()

return(c_rep)
}

[johnmeyer]

For those who might not find it, if you click on Robert.n's link in post #5, and then click on the Readme.md file and you'll get lots of good information on his software, along with an excellent before/after video.

It looks to me like the software works on the entire frame rather than on individual lines. It therefore will probably not be useful for TBC errors which change a vertical flag pole into a wavy mess.

Since this is correcting time base errors, it is worth mentioning past efforts on this problem. Building a software TBC has been a holy grail in this forum for many years, and while a few authors have come close, no one has ever produced something that really works, at least as far as I know. Here are links to some previous attempts. The length of some of these threads clearly shows the interest in such a project:

New Script: Software TBC 0.6 & Sample (was Fast Line Shifter 0.53) (https://forum.doom9.org/showthread.php?t=162726)

Software TBC (Multi-page thread 1 2 3 4) (https://forum.doom9.org/showthread.php?t=152706)

Could a software equivalent to TBC be done in Avisynth... (Multi-page thread 1 2) (https://forum.doom9.org/showthread.php?t=117511)

However, even if this software isn't developed any further, it looks like it will still be very useful for certain types of VHS capture problems.

[lollo2]

Thanks a lot johnmeyer for the hints.

The problem here is not really the implementation of a (frame)TBC in software, which is a difficult (impossible?) and not accomplished task.

What happens is that the (line)TBC of the VCR is active, and corrects the V synch of the signals (is then effective) but sometimes it makes an error of 1 or more lines. In general it occurs only in a couple of frames over 100.000 frames, in this special case happens more frequently.

This can be fixed easily with just a shift of one of the field, but there is not (I have not found yet) an accurate way of detecting the shifted fields and fixing them, especially if they are 1 line only and consecutive.
So I have to check at 1/3 of the speed the original video, and manually write the corrective actions. The final result will then be ok, but you understand that this manual operation is ok for few videos, it is crazy for many of them.

Poisondeathray approch is working very well for a single pattern (even field shift 1 line up) and if no black areas are inside the top active image of the frame/field.
I will work on it trying to extend its capabilities, but the problem is always the "right" detection.

[robert.n]

Thank you for the valuable information, in particular @johnmeyer.

As I mentioned in a related thread (https://forum.videohelp.com/threads/392186-Way-too-shakey-captured-VHS-video#post2684412) in the VideoHelp forum, I initially named my tool post-mortem-tbc. In the end I went with vhs-deshaker because it is easier found by laymen who don't know about TBC. Also, to be honest, I am not sure I have understood what TBC is all about.

Over time I have seen various clips that demonstrate VHS playback with and without TBC. The symptoms (lines are horizontally shifted) are very similar (maybe identical, dunno) to what I encountered in my own VHS videos and what vhs-deshaker is meant to fix. My conclusion therefore is that indeed my tool is likely able to clean up many of these clips. Although in extreme cases it is probably necessary to tweak some internal smoothing parameters...

Suggestion: You can upload some example clips with such TB problems and I can try and process them with my tool. Then we can see how good (or bad) it handles such cases. Ideally I get examples with varying degrees of TB problems (mild, medium, extreme). Lossless or quasi-lossless compression would be best (but x264 with CRF 18 should be sufficient).

[johnmeyer]

Time base correction is not well understood by most people. I am an electrical engineer but since I never had a job in the TV industry, I only partially understand the detailed complexities of how an analog TV signal is created, stored on tape, and then played back. However, I certainly understand enough to know that a key part of the very complex process is that the analog video camera, which initially captures the image, embeds a variety of different timing signals before the beginning of each video field, and also in between each half frame. The TV set which receives the video signal uses these timing signals to synchronize the scanning beam so that the capacitor which stores the ramp voltage which deflects the beam that puts the spot on the phosphor tube starts charging that capacitor at precisely the right time.

Unfortunately, when the analog video signal gets stored on tape, those timing signals get degraded. A time base corrector uses those degraded signals and, using the knowledge of how those signal should look, attempts to reconstruct them.

Here is a link to a really short, simple explanation of what video can look like if the time base signals are corrupted, along with an "after" picture showing how a time base corrector can help:

Using Time Base Correctors (https://www.tgrantphoto.com/sales/index.php/content/video-dubbing-tips)

There are several problems created by a poor time base that would be very difficult to correct using software, after the capture. However, one of the most common problems where each scan line starts a little too early or a little too late compared to the adjacent lines, is something that almost certainly could be corrected in software. It is this particular problem that was discussed in those early thread I linked to in my previous post.

On paper, the problem seems simple enough to solve especially if you are lucky enough to have a capture which has not been cropped in any way and therefore includes pure black down either the left or right edge of the frame. The idea is that you shift each line so that you get a "clean edge" up and down the extreme left or right of the frame.

However, while I think the problem is solvable, it requires some reasonably advanced programming skills.

[Selur]

@johnmeyer: I get '403 Permission Denied' for https://www.tgrantphoto.com/sales/index.php/content/video-dubbing-tips

[johnmeyer]

@johnmeyer: I get '403 Permission Denied' for https://www.tgrantphoto.com/sales/index.php/content/video-dubbing-tipsI just clicked on the link in my post and also the quoted link in your post and had no problem.

Can you get to their site at all by just going to the main page?

https://www.tgrantphoto.com

If not, there must be something blocking you, either from your ISP, from the German government, or at some other point.

You could try a VPN, if you have one installed.

[poisondeathray]

The pattern is not always the same: it can be even field shift up, even field shift down (rare), odd field shifted up (rare) , odd field shifted down. By a number of lines from 1 to 5. Rarely both fields are shifted.



If you have the combinations defined, it should be possible if you have a way of detecting it . You can do multi conditionals, scriptclip , but it gets complicated quickly. My gut says that sounds like too many combinations for that type of approach .


To detect them I use this approach (developed with help from StainLessS); it produces many false positive and does not detect many of the 1 line only shifts:

If SS already had a look at the combinations and couldn't come up with something more accurate, I doubt it's possible in avs

[Selur]

an you get to their site at all by just going to the main page?

https://www.tgrantphoto.com
nope.

You could try a VPN, if you have one installed.
Works with VPN/WebProxy.

[lollo2]

My gut says that sounds like too many combinations for that type of approach

Yes. I will keep trying...

If SS already had a look at the combinations and couldn't come up with something more accurate, I doubt it's possible in avs

The original script was developed for another kind of defect (corrupted field): https://forum.doom9.org/showthread.php?t=183582. I modified it for the field shift problem.

[robert.n]

Time base correction is not well understood by most people. (...)
Great post, thank you! This helped connect a lot of dots for me regarding TBC.

On paper, the problem seems simple enough to solve especially if you are lucky enough to have a capture which has not been cropped in any way and therefore includes pure black down either the left or right edge of the frame. The idea is that you shift each line so that you get a "clean edge" up and down the extreme left or right of the frame.

However, while I think the problem is solvable, it requires some reasonably advanced programming skills.
This is exactly the method that is implemented in vhs-deshaker. The pure black on the left-hand and right-hand borders of the frames is used to re-align/unshift all rows of the frames. This can fix mild to medium instances of horizontal shaking/distortions caused by lack of TBC.

[robert.n]

Today I greatly improved the "How does it work" description of vhs-deshaker, including visualizations of what happens. You can read it here if you're interested: https://github.com/rsnitsch/vhs-deshaker#how-does-vhs-deshaker-work

[TCmullet]

For those who might not find it, if you click on Robert.n's link in post #5, and then click on the Readme.md file and you'll get lots of good information on his software, along with an excellent before/after video.

It looks to me like the software works on the entire frame rather than on individual lines. It therefore will probably not be useful for TBC errors which change a vertical flag pole into a wavy mess.

Since this is correcting time base errors, it is worth mentioning past efforts on this problem. Building a software TBC has been a holy grail in this forum for many years, and while a few authors have come close, no one has ever produced something that really works, at least as far as I know. Here are links to some previous attempts. The length of some of these threads clearly shows the interest in such a project:

New Script: Software TBC 0.6 & Sample (was Fast Line Shifter 0.53) (https://forum.doom9.org/showthread.php?t=162726)

Software TBC (Multi-page thread 1 2 3 4) (https://forum.doom9.org/showthread.php?t=152706)

Could a software equivalent to TBC be done in Avisynth... (Multi-page thread 1 2) (https://forum.doom9.org/showthread.php?t=117511)

However, even if this software isn't developed any further, it looks like it will still be very useful for certain types of VHS capture problems.
Wow, I can't believe this thread is being updated now! Thanks John Meyer! I spent half my inheritance in the 90s (my last parent died in '91) on expensive VHS/S-VHS video equipment. Most is gone now. But after 20 years of many tapes (over 1000) staying safely in storage, I'm embarking now on selectively digitizing them. I've hunted but not found a cheap hardware TBC. I owned (let me count) at one time, 5 units, but over the course of that decade had experience with 7. Panasonic AG-7750 (broadcast quality) had an excellent one. My 2 AG-1970s had TERRIBLE ones, which I realized no one should use. Hotronic outboard monster; terrible in several ways. Two GVP Amiga based cards, which did a good job but were not neutral-based in their procamp controls. (We don't need procamp controls; we have "Tweak".) I still have the GVPs but don't know if they can be hooked into a PC and whether there is PC software to control them. Heck, I just want ONE! (I'm not doing 2 camera shoots of anything; really no shoots of anything!)

But yes, having been a programmer from college freshman days (1972) till gradually having to drop out roughly 20 years ago, and having started using Avisynth occasionally since about 2004 (best I can estimate), I HAVE wondered why software can't be written. I will eagerly devour the links you all have given EXCEPT for deshaker type info. (EDIT: Oops, I realize I may need to jump on the deshaker tool.)

The most common problem does not need VERTICAL correction, as a decent tape in a machine with a good video head will not need that. It's the individual LINES that need it. I'm 69.5 and in middle of battling stage 4 melanoma, as well as being a very low-income senior. After the melanoma's over (and I am encouraged at the probabilities it won't be over via the grave), I may be able to devote some time and brain to programming IF I had access to knowledge. My skills in Avisynth are relatively weak. I'm about to buy a $100+ price range SD to HDMI upscaler. I doubt even those more expensive units have a TBC in them. (Don't buy $50 and under units.) Again, mainly JohnMeyer and Robert.N, thanks for bringing up this subject just a MONTH or so before I needed to jump into the fray! I want a solution BEFORE I do any serious captures. (I do have at least one VCR head to replace first.)

[TCmullet]

Great post, thank you! This helped connect a lot of dots for me regarding TBC.


Originally Posted by johnmeyer View Post
On paper, the problem seems simple enough to solve especially if you are lucky enough to have a capture which has not been cropped in any way and therefore includes pure black down either the left or right edge of the frame. The idea is that you shift each line so that you get a "clean edge" up and down the extreme left or right of the frame.

However, while I think the problem is solvable, it requires some reasonably advanced programming skills.


This is exactly the method that is implemented in vhs-deshaker. The pure black on the left-hand and right-hand borders of the frames is used to re-align/unshift all rows of the frames. This can fix mild to medium instances of horizontal shaking/distortions caused by lack of TBC.

For YEARS, I've visualized in my head that THIS is exactly how the problem is solved. When you look at a frame of video in any video editor (VirtualDub, AviDemux, etc.) you see the beginnings of lines in the left black border as not being lined up. I did not think of the right border as I figured the timing was being lost at tape playback time between lines, leaving HSync pulses not at the right times. But yes, it makes sense that even after you line up the lines at the left edge, the right edge might be somewhat jagged due to drift between the start and end of a line. I'd LOVE to try this, as it sounds like with most video it should work! (I thought any "deshaker" is dealing with camera shake, not timebased errors.) But I'm scared of going outside the realm of running an Avisynth script. Let me study some more and I may ask you questions, Robert.

[TCmullet]

Today I greatly improved the "How does it work" description of vhs-deshaker, including visualizations of what happens. You can read it here if you're interested: https://github.com/rsnitsch/vhs-deshaker#how-does-vhs-deshaker-work
Robert, though I was considered a very good programmer from my first job in 1978 thru about 1991, I have NEVER been able to acquire a Windows .exe file from that place called Github. I've always been very intimidated by the zillion buttons with no explanations of anything there. Could you please walk me through getting your Windows executable?? (Or anyone?) I do not want to compile; I do not want (or have the ability at present) to set up a development environment of any kind except Notepad for editing Avisynth scripts to feed into the last version of VirtualDub obtainable before "Phaeron" brought the Vdub forum totally down years ago. Just need to know how to get the .EXE. I'll let you all know how the tool performs for me.

EDIT: OOPS!!! Just after I sent the link to a local programmer friend, I happened to notice in the lower right region that "Releases, Latest" region. The joke's been on me all these years! (I'm so used to normal webpages where it's generally easy to download an application by a non-programmer.)

[TCmullet]

Robert, I will be using Avisynth for everything except the direct running of vhs-shaker.exe. What are the rules for what codecs I can use when I create my input.avi? Does it have to be an AVI or can it be something else like an MP4? Am not worried about audio. I can fix all that after the video is done. I do wonder though why you're not using VFW so that I can pick an output codec in VirtualDub to read it later.

[robert.n]

What are the rules for what codecs I can use when I create my input.avi?

Internally vhs-deshaker relies on OpenCV's VideoCapture feature, which in turn is based on ffmpeg (among others). Therefore, vhs-deshaker supports a wide variety of input codecs/formats. As a rule of thumb: If ffmpeg can process it, then vhs-deshaker can likely open it, too.

[TCmullet]

Yep, found that out. It easily processed my mp4 TS that my capture card had created. Now to figure out why it's working terribly. I have an idea; will report back after further testing.

[TCmullet]

My first test was pretty bad. Much jerking around left to right and back. After staring at it a minute, I concluded the noise bar at the bottom was about as far offset as was the sideways jumping. I concluded that I'd need to strip that off.

The sad news was that after stripping it off, the jerking was almost as bad. So there's something incompatible between your program and my videos. I have to give up on this for now. I can't think of anything else to fix it. Note: My video is not bad in need of correction. I had thought that the very slight fast jitter I see in some spots made me think this was the problem in this case.