New Script: Software TBC 0.6 & Sample (was Fast Line Shifter 0.53)

[jmac698]

Hi,
This is a software "line" TBC. There is a new version, but it doesn't have more quality, just fast enough to be practical (in those cases where it can be used.)

Yes, it has worked for some people's real clips, which had severe "wobbly lines", but don't expect much. I still need to do some basic research here.

Update: New plugin version 0.61
http://www.sendspace.com/file/s78pp4
Installation note:
Please extract the included plugins and place into your plugins directory.
jitter02.zip -> jitter.dll -> plugins\jitter.dll
findpos01.zip -> findpos.dll -> plugins\findpos.dll
You may also need the VC2008 runtime.

If you get an error when trying to load the plugin, try these versions. They are called "static builds", a programmer's term, which means they are fully self contained. The only disadvantage is a bigger download. They are no different in functionality.

jdejitter03_static
http://www.sendspace.com/file/qk1ifv

findpos02_static
http://www.sendspace.com/file/e3jblt


The following applies to the old 0.53, and the theory in general can be used with the new version
----------------------------------

This script is in development and is only for the following cases:
-#1 Video was captured with HSYNC area (via driver tweaks)
-#2 Video contains easily distinguishable black borders on both sides (again, probably needing tweaks).
-If you have a hardware TBC there's no point, but case #2 in a digital file with the original analog source unavailable might be useful
-It's not necessarily any better than existing plugins for case #2
-If you have a supported card, it's possible to have the equivalent effect of a hardware TBC (they work very similar), at least for the "wobbly" function.

How to get the capture
The capture section has moved to "How to Capture with HSYNC" in the capture forum. That topic is unrelated to script usage.
http://forum.doom9.org/showthread.php?t=162832

Results (this only worked so well because I have HSYNC)
http://screenshotcomparison.com/comparison/88810

Results for case#2 from a user supplied video:
http://screenshotcomparison.com/comparison/91853

Theory of Operation
Before I get into usage, it would help to know what we're trying to accomplish. An analog video signal starts and ends each line with a pulse called an HSYNC. On a VCR, the timing is not stable. The capture card starts with the first HSYNC and reads in the video for a set time-period, then waits for the next HSYNC. The result is that the video can appear shifted by a different amount on each line. A hardware TBC probably works by reading the full line and re-adjusting it's timing to the standard amount, thus when captured, you get the entire video line.
With Case #1 we are capturing the HSYNC (the right half of it) and a large window of the video line - thus including all of it, regardless of it's timing. Now it's simply an image that needs lining-up. The script looks for the HSYNC line at the left, then a black area at the right, and resizes the line to a standard size (exactly equivalent to how a hardware TBC re-times). The HSYNC is very easy to find and is a one-pixel or so line. The right edge relies on a short black area which is 'blacker' than anything the video could be, so it's also reliable.

With Case #2 we're relying on black borders being in the video and the video being relatively bright. We simply line-up the black borders to the same spot. Dark scenes in the video could confuse the border detection so it's not as reliable, and that is the case with any other plugin (which is why I say mine is not necessarily any better in this case).

Usage
Quite simple, there is an autothresh which looks for black pixels in the left border. A small amount is added to this to form the real thresh. Adjust the added amount to tweak, until the picture lines up. I used AvsPmod to look at the raw video, and moving my cursor over the black area, I saw it was a noisy 15-25 (in a user test clip). I decided to just set 32. Values slightly lower would leave a few lines wrong, seen as little black stripes at the left edge of the video. For that particular video, only the bright scenes worked correctly. Having it work for dark scenes is being experimented with.

Advanced Usage
The script first makes a mask, in this case every pixel at thresh or above is marked to luma=255 in the mask or 0 elsewhere.
Next I pass my mask to findpos_h which searches for the first 255 value on each line, within the searchwindow. It also simultaneously searches from right to left by searchwindow pixels. It saves the results in a special clip, which records the number of pixels before the mask was found. For example if the HSYNC line occurs at x=4, the luma of the shift clip contains luma=4 for that line, at the left hand side. The right hand side contains the offset from the right of where the video ended, for example if it ended at x=710 and the video is 720 pixels wide, the luma of shift is 10 on the right hand side.
alignbyluma now reads the original video and the shift video and resizes to a standard size. Hopefully this is enough information to do anything else you want here.

What about the aspect ratio?
I still need to calibrate to return the correct aspect ratio. Currently the fixed video is probably wider than it should be.

Limitations
Check the history. Currently searchwindow is not working correctly, but it's enough to work. Also I need to add subpixel detection/shifting to get a really stable result. Currently the result looks like a little noise because of subpixel jitter.


#Fast line shifter Ver 0.53 by jmac698
#Lines up either or both edges of a video. Can also be used as displacement for 3d scripts.
#Requires Masktools v2a45+ (mt_lutspa mode), GRunT, GScript
#MinMax http://forum.doom9.org/showthread.php?p=1532124#post1532124
#Limitations: still no subpixel shifting
#0.53: Avoid possible "ScriptClip: Function did not return a video clip of the same colorspace as the source clip!"
#0.52: Less blurry resize
#0.51: Autothresh (uses 2 pixels of left border as a starting point, then adds a small amount to avoid noise)
#0.5: Slow, but using a completely new approach, and can resize whole lines
#0.4: Fast, can detect and line up on left or right edges
#note: sample was frame 167, http://screenshotcomparison.com/comparison/88810

#Modified to work with wide-window sample capture, which includes hsync

src=AVISource("D:\project001a\tbc2\vhs hysnc sample.avi").converttoyuy2
#crop(8,0,0,0)#Uncomment and adjust to remove extra black left border
thresh=int(findthresh(src))+3#This may not always work, try to manually set to 32 for example. Pick the lowest value which lines up picture.
ScriptClip(src, """
#Mark video edges
converttoyv12
m=mt_binarize(thresh)
#Line up video
findpos_h(m, searchwidth=22)
alignbyluma(src,last)
""")
addborders(56,0,0,0)

function findpos_h(clip m, int "searchwidth", int "x1", int "x2"){
#Searches m from left to right in the range x1 to x1+searchwidth-1 and right to left in the range width-1-x2 to x2-searchwidth-1
#for the first luma=255 pixel, then colors the output line with the offset from x
#for example m is 0 0 255 255 255 0 0 0, width=8, x1=0, x2=0, searchwidth=4 becomes 4 4 2 3 3 4 4 4, then 2 2 2 2 3 3 3 3
#Can only search for 255 pixels (as the luma output is only 8 bit)
#c and m should have the same clip properties (same size)
#searchwidth should be <=width/2
searchwidth=default(searchwidth,32)
x1=default(x1,0)
x2=default(x2,0)
rampexpr="x "+string(m.width/2)+" < x "+string(m.width-1)+" x - ?"#x w/2 < x w-1 x - ?
ramp=mt_lutspa(m, mode="absolute",expr=rampexpr)
notfound=searchwidth#Value to return if no mask on this line, should be >=searchwidth or you'll find the wrong minimum later
maskmarker=255#The luma value in the mask which indicates a detected pixel
#(if m=maskmarker return ramp else notfound), 255 means x>=255, x<searchwidth or notfound
mt_lutxy(m,ramp,yexpr="x "+string(maskmarker)+" = y "+string(notfound)+" ?")
#now make solid lines based on min luma found in each line
l=crop(0,0,-width/2,0)
r=crop(width/2,0,0,0)
l=l.minmax(0,0)
r=r.minmax(0,0)
StackHorizontal(l,r)
}

function alignbyluma(clip src, clip shift, int "mode"){
#Shift/scale each line of clip src by the x offset defined by the luma of shift
#for example if shift were all luma=8, the entire src clip would move 8 pixels to the (dir)
#This works on a pixel basis, so solid horizontal lines in shift can shift src by variable amounts per line
#It uses a simple replacement strategy, where each pixel in shift is tested and replaced by the same pixel in a shifted copy
#Currently handles only 0-15 shifts
#Magnify everything to get full color resolution
mode=default(mode, 2)
shiftuv=shift
shift=shift.pointresize(shift.width*2,shift.height*2)
shift=ytouv(shiftuv,shiftuv,shift)
src=src.pointresize(src.width*2,src.height*2)#We double here to preserve chroma rez
GScript("
for (y=0, src.height/2-1, 1) {
l=int(getpixel(shift,0,y).YPlaneMin)
r=int(getpixel(shift,shift.width/2-2,y).YPlaneMin)
getline(src, y*2)
align(l*2, r*2, 4, 4)
out=y==0?last:stackvertical(out,last)
}#for y
")#GScript
out
converttoyuy2
bilinearresize(src.width/2,src.height/2)
}

function align(clip v, int xl, int xr, int lb, int rb, int "mode") {
v#shift an image, x>0 shifts left, xl is amount to shift left, xr is amount to shift right
#mode 0 is shift left only, 1 shift right, 2 scale to shift left and right
mode=default(mode, 2)
offx=mode==0?xl:-xr
mode<2?pointresize(last.width, last.height, offx, 0, last.width, last.height):crop(xl,0,-xr,0).addborders(lb, 0, rb, 0).Spline36Resize(last.width,last.height)
}

function getpixel(clip v, int x, int y) {
#get color of a single pixel and return as a fat 2x2 yv12 pixel
v
#pointresize(last.width*2,last.height*2)
crop(x>0?x*2:0,y>0?y*2:0,-(last.width-x*2-2),-(last.height-y*2-2))
}

function getline(clip v, int y) {
v#return a line of height 2 from y to y+1
crop(0,y,0,-(last.height-y-2))
}

function findthresh(clip v){
#Find a resonable starting point for thresh by searching border
current_frame=0
v.converttoyv12
crop(0,16,-last.width+2,-16)
AverageLuma
}

[sven_x]

The subject has been discussed several times. Please take a look at this (http://forum.doom9.org/showthread.php?t=152706) thread.

There were some other attempts to correct line sync jitter (linked in the thread above). I have tested all of these functions - they are very vulnerable to noise which is an element of VHS recordings. Sometimes the resulting line jitter after correction is bigger and more random than that of the input video.
I do not know if someone has tested using a cross correlation function to find the beginning of a line which might reduce the influence of noise.

[jmac698]

sven,
I'm doing something that hasn't been done before. I have access to the raw signal including hsync so I have a very strong image to align to. See http://screenshotcomparison.com/comparison/86066
I have aligned the image, even though it was bright, and this line which seems to be hsync, but neither work correctly. My theory now is that it needs both sides. I've written a correlation function, but I don't need it as my test cases were quite clear but they didn't work. The problem must be somewhere else, not in our technique.

[johnmeyer]

As has been pointed out in previous posts on this subject, a good time base corrector, used properly, and inserted at the appropriate point in the analog signal path, will probably completely eliminate this problem.

[jmac698]

john,
That doesn't seem to make sense - I have access to the full raw signal, I should be able to make a real hardware TBC with only software. If you could d/l a program that does a TBC directly on capture, wouldn't that save you money compared to buying a TBC?
It only takes a low level access driver and any capture card can turn into a TBC.
I just need to develop the technique.

[johnmeyer]

Software is wonderful, but some things can only be done in hardware, especially when dealing with an analog signal. You really cannot create a digital TBC because the sync signal is only available in the analog domain and is not available once the analog video has been digitized. You cannot do the same thing simply by looking at the resulting badly captured video and then trying to figure out what to do with the bad video.

If you want really good video from your old VHS analog video, then you MUST use a TBC. I admire your desire to do many things in software, but this is one situation where you are not going to get very good results.

[jmac698]

john,
I appreciate your sentiment, however I seem to have not managed to communicate that I *do* have access to the sync signal, through special driver tweaks, you can see a picture of it (including the color burst) above.
So I actually agree with you, you need the sync, and I do have it.

[ronnylov]

The time between two line sync pulses should in theory be a constant but on a recorded tape it may fluctuate a little bit during playback because of tape tension. So instead of just syncing the start maybe you need to stretch back the length of each line to the nominal value. Check the time distance between each pair of hsync pulses and readjust the signal back to nornal. Then you can align the lines.

Just an idea you can try if you have captured the complete video signal. I don't know how a real hardware TBC work.

[AVIL]

@jmac698.

I think your approach is good. But IMHO the result is worst than original (as seen in your pictures). Could be that image pixels must be moved in opposite direction to sync ones. Perhaps the line must be streched/expanded instead moving the pixels. But I like this method.

BTW. Where can be obtained the modified driver that does sync signal capture? I find it very useful

[sven_x]

I have applied a very basic approach to your screenshots:
1. Analysing the provided VHS grabbings
2. Trying to sync the lines manually in photoshop and looking if this is possible and how the result looks.

Analysis

The picture below shows begin, middle part and end of a group if lines taken from your screenshot. One can see, that offset (time delay) is nearly constant in each line, that is the pixel shift is the same at beginning and end of the lines.
It does not correlate to the sync signal position.
http://www.engon.de/temp/avisynth/jmac698_line_sync1.png

Correction
I have imported the synced screenshot into photoshop. Line heigth was increased to 300% to get a better view.
Using a fixed selection window (720px x 3px) I copied each line into a new have layer and than manually shifted that line to a position that looked good.

The result is given below:
http://www.engon.de/temp/avisynth/jmac698_line_sync2.png


Conclusions
When a correction of the image could be done by eyes view it also must be possible to put that in a computer algorithm.


Lines from VHS grabbings cannot be synced by using what was regarded as sync signal
Lines from VHS grabbings can be synced using cross correlation between subsequent lines.
There is also a small jitter in line length, that needs an extra treatment.
The line offset jitter does not correlate very well to the beginning of the lines (which is the beginning of total black pixels) (but not as bad as to the sync signal)


Many thanks to jmac698 for the interesting approach and for the screen shots which helped me a lot.

[johnmeyer]

I think Sven's amazing post shows clearly that there is no correlation between the sync pulses you have captured, and the artifacts you are trying to remove. The easy conclusion is that they are caused by something other than time base errors. However, another conclusion is that you have not really "captured" time base information (or at least have not captured it correctly) with your modified driver.

I do not say this in order to be difficult, but at the risk of sounding like I only have one thing to offer, I would once again like to suggest trying to borrow a real hardware TBC and use it. I say this because the visual disturbance you are trying to correct is something I have seen many times, and a TBC circuit has, for me, always been able to remove the line displacements.

[jmac698]

:goodpost:sven,
That was incredible! I know this has been tackled many times, but I believe it's the first time there's been an analysis including the hidden area of hsync. It's very inspiring to me. However I can do a better job in making examples. I should record a test signal which is easier to line up, then we can measure directly the stretch/offset. My only concern is that the result won't be as bad as this tape. I picked this tape initially because it is a well-worn kids movie and showed massive jitter.
Does anyone know how I can make a tape with *bad* jitter? Perhaps if I copy many generations?

I had already felt that something was inherently wrong with lining up just by the left edge. People thought it failed because of their line-up algoritm (due to noise, etc.), but I had thought of the possibility of tape stretch and/or slight variations in head speed causing the line-stretching effect. In this case there's bad news, because I can only capture hsync once per line, if I capture the next hsync then I've missed the hsync for the next line and essentially never get any more lines.
Why that line is not good for lining-up is confusing to me however.

:goodpost:

[jmac698]

john,
You may be correct that the line is not an hsync, I will verify this. I'm not even trying to TBC that image; what's more interesting to me is to develop a method of software TBC. So I'm not trying to be practical here, just doing research and development. The research itself is interesting, but I still believe there's a good chance of a practical result in the end, and I'm using some techniques that haven't been tried before. That it failed initially doesn't matter, research is about answering a question. I haven't given up until I have fully understood the problem and concluded that it's impossible (at least for someone with my abilities).

[Ghitulescu]

Shifting is not the only algorithm in a HW TBC. It stretches and squeezes the analog line too. You need to do also a scaling in your algorithm.

[IanB]

...
Does anyone know how I can make a tape with *bad* jitter?
...
Using a tape you definitely do not love, unspool an amount of tape and carefully stretch it a little, then respool it. Viola jitter video.

When the tape is stretched it is not uniform. Better quality tapes use stronger base material which is more resistant to deformation. Commercial video releases notoriously use tape from the lower quality end of the range, penny pinching misers.

[jmac698]

I'm still looking to answer my original question: Can you suggest a way to speed up the dejitter script I first posted?
General discussion about the practicality of a software TBC has been moved to
http://forum.doom9.org/showthread.php?t=152706&page=2
Thanks.
(Where's the amazing Gavino? I bet you can optimize the script! And I finally had to use GScript :)

[Gavino]

I'm still looking to answer my original question: Can you suggest a way to speed up the dejitter script I first posted?
A very minor speedup to the existing script would be to initialise 'out' to a fixed blank line instead of calling getline(), since that first line is a dummy that gets cropped at the end.

Nice to see you using GScript :), but I wonder if it could be done more quickly using some Masktools wizardry to process the rows in parallel. I don't have time just now to figure out how.

[jmac698]

That's a good idea, you can do a pixel shift with a convolution, let me try.
Update:
You can shift the luma 10 pixels to the right like this:

mt_convolution(horizontal="1 0 0 0 0 0 0 0 0 0 0",chroma="process")

This also shifts the luma:

mt_luts( last,last, mode = "0 0 0 1 0 0 0 0 0", pixels = mt_square(3), expr = "y" ,chroma="process")

[jmac698]

Fast Line Shifter 0.2
[deleted]

[StainlessS]

Hi,
The script I have is extremely slow but it works - is there any way to speed it up?


How about moving GScript(" ... ") # GScript outside of ScriptClip,
I think that there is some kind of initialiser involved [Executed on every frame???] and it might
(or might not) make a bit of an improvement in speed, or maybe even try the GImport option. I think perhaps Gavino would have mentioned this if it would be useful, & so perhaps may not help.

[jmac698]

Good point.
My new version is faster; I'm still working on it.

[Gavino]

How about moving GScript(" ... ") # GScript outside of ScriptClip,
I think that there is some kind of initialiser involved [Executed on every frame???]...
A call to GScript has a relatively small overhead, equivalent to calling Eval, in which the code string is parsed and evaluated. It's all done at compile-time, so there is a per-frame penalty only when inside ScriptClip.

Doing it the other way round, calling ScriptClip inside a GScript string, has a problem in that GScript constructs cannot be used directly inside the ScriptClip part (without calling GScript again), as ScriptClip creates a new standard parser instance for each frame, bypassing the GScript parser.

What you can do is extract the GScript code into a function defined inside GScript (yes, entire functions can be GScript'ed), and then call the function from within ScriptClip. This reduces the per-frame parsing overhead to a minimum. Using this scheme, the original code could be rewritten as:
GScript("
function f(clip src) {
out=getline(src,0)
for (y=0, src.height-1, 1) {
line=getline(src, y)
shiftx=0
for (x=0, 25, 1) {
if (getpixel(src, x, y)>200) {
shiftx=x
}#if
}#for x
out=stackvertical(out,line.shift(shiftx-20,0))
}#for y
out.crop(0,1,0,0)
} # end f
")#GScript

ScriptClip("""
f()
""")#ScriptClip

I'm not sure how much difference this would make - probably most of the overhead is coming from the pixel manipulation anyway.

[jmac698]

Sorry, it was close, but pixel manipulation in any kind of loop is too slow, so I don't have a first use for gscript after all :(. If you could add a pixel reading/writing feature, that would make it extremely useful!

Update: See first page for a new version of a line shifter script.

[Perepandel]

The subject has been discussed several times. Please take a look at this (http://forum.doom9.org/showthread.php?t=152706) thread.

Wow Sven, you're referencing a thred in which jmac698, the user you're replying to, has already participated xD



As has been pointed out in previous posts on this subject, a good time base corrector, used properly, and inserted at the appropriate point in the analog signal path, will probably completely eliminate this problem.


John, you seem to miss the point: he's trying to make a time base corrector.


Software is wonderful, but some things can only be done in hardware, especially when dealing with an analog signal. You really cannot create a digital TBC because the sync signal is only available in the analog domain and is not available once the analog video has been digitized. You cannot do the same thing simply by looking at the resulting badly captured video and then trying to figure out what to do with the bad video.

If you want really good video from your old VHS analog video, then you MUST use a TBC. I admire your desire to do many things in software, but this is one situation where you are not going to get very good results.


Maybe we are not talking properly then. The truth is that it would be a hardware+software TBC. A bt878-based capture card is used to capture the (raw?) video signal, and then a software algorithm is used to correctly horizontally allign the video lines.

A friend of mine lend me a SVHS video with an integrated TBC. I already captured my tapes with it and the horizontal jitter went of; it really made miracles with some badly jittered tapes. But I still have the intereset in this project, sharing the motivations with jmac698.

The label on the SVHS says, about the TBC, "Digital 3-D circuit" and "Digital 3R picture system". If the advertising is true, then it means that the signal is digitized somewhere, then an algorithm applied to the digital signal to retore the stabilization, etc. That is the same principle that what we are trying to archieve with the so-called "software TBC". So John, I wouldn't be that radical in my opinion.

The time between two line sync pulses should in theory be a constant but on a recorded tape it may fluctuate a little bit during playback because of tape tension. So instead of just syncing the start maybe you need to stretch back the length of each line to the nominal value. Check the time distance between each pair of hsync pulses and readjust the signal back to nornal. Then you can align the lines.

I'm glad ronnylov had the same idea I shared with jmac698 in one of my previous private messages with (before seeing this thread and posting in the forum). That's were I would focus my efforts right now.

One of the problems would be the amount of pixels we could get per raw line or digitizing resolution. If we are limited to 800 and some, and we have to adquire the active video from it and translate the result to 720 pixels, then maybe we could get a lot of aliasing/lack of resolution, etc. But we should try it to see if it is acceptable or not. Also jmac698 said somewhere that he's able to get about twice per line that values, so in that case we'd probably have by far enough information to get good results.

Just an idea you can try if you have captured the complete video signal. I don't know how a real hardware TBC work.

That's also another thing I wanted to say: we need more information on how a stand-alone TBC works, in order to make ours.

[jmac698]

Hi,
I made a new sample, it's quite interesting
http://screenshotcomparison.com/comparison/87833
This seems to be a partial success; overall it's much better, but it seems to need lining up on both sides. That will be tough to script.
I tried to get as much of the video as I could this time, it's brighter, there's no filtering, as much of the left border, as much resolution as possible, and even all of the right hand side of the picture.
The bright green line turns out to not be relevant; I can move it with a register called AGC delay so it's not part of the video.
I've also made a package for you to analyze;
http://www.sendspace.com/file/jdh9j9

I don't think the cause of jitter is what we think at all! It seems to be related to picture content, and it can vary dramatically between lines. It's also common over both fields. It seems to happen more in bright areas but it's not consistent. Anyhow have any observations?

[sven_x]

Great job!
The corrected image points out that there is also a jitter in line length (wich might be a delay in head rotation or tape stretch or a delay in tape speed or even rippled tape). This jitter is very fast( i.e. a 1/30 of the time of a frame, about 900 Hz). A VHS player uses several loops to control frame synchronisation. So the source of this effect could also be any ringing or disturbance in the electrical signal flow.

I do not have the impression that jitter is related to picture content. It has an amplitude envelope that is rather a sinus curve. For me that speeks for electrical or mechanical "ringing".

using nnedi3(field=-2,nsize=2) one would notice, that
a) the jitter is different in the even and odd field
b) in both fields it is more present in the upper part of the frame and also in the same regions (line numbers). That is strange.

The AGC of the video signal reacts on the negative sync impulse. The AGC produces an overshoot because it tries to compensate the negative impulse (AGC tries to amplify video amplitude to 16...235). The AGC delay should be set so fast, that sync jitter does not affect luma of the first video pixels in a line. On the other hand a very fast AGC delay might lead to less stable synchronisation. Have you noticed any influences of the value to the quality of synchronisation, jitter frequency or luma modulations?

Another aspect is, the problem that appears in your grabbing can be described in other words with "the video grabber card does not synchronize to a proper line synchronisation signal". Have you ever tried to grab the same tapes with another card?

[Mounir]

I'm not sure if you have chosen the right video sample because animes are supposed to be 24fps right ? Perhaps a video with live content (concert) which is filmed at 29.97 fps, truly interlaced would be suited for the test ? The challenge would be to find one that has defects (lines shifts)

[jmac698]

@mounir
I don't follow your reasoning, I like this content because it's fairly steady and you can tell if the picture itself is not lined up. I'm really looking for straight edges in the video. With animation, there's a few frames that aren't interlaced so I can view the frame as a whole (though we could also analyze through separatefields; I don't care about picture quality/interlacing at all just the jitter). The video is still 29.97 btw; just that entire pictures repeat themselves.

@sven
The AGC register seems to set the point in time where the amplitude is measured. I've positioned it in the hsync area. The picture brightness changes if I move it into various zones. The line it causes to be drawn on the video seems to be as you say; the ringing as it reacts to the amplitude. The video was far brighter than it had to be too, but it doesn't matter for now.

Notice the first scene change - the jitter area instantly changes. We also have to consider if macrovision could be influencing our analysis, I should try another example. Anyhow I'll try lining up both sides and some other experiments. I am still hopeful this can be done based on the partial improvement.
Btw I'm seen "bowed" video which bent to the right on the lines which had high average brightness, I suspect that's an electrical effect, possibly due to aged components in the circuit which is not operating properly.

[*.mp4 guy]

I have not read the entirety of the thread however, I do not think this has been mentioned. It looks like instead of shifting only, you must shift, and stretch so that both ends of the signal space line up. If you think about how the jitter errors are created (stretching of tape / time signal), it makes sense that the entire "time" (horizontal) dimension would need to be resampled to enforce consistency.

So what must be done is:
1 shift signal so one edge is aligned
2 calculate stretch/squash (resample factor) needed to maintain correct signal length
3 resample signal accordingly
4 hope that most time-scale variance is between lines rather then within lines

[jmac698]

@*.mp4

This seems to be a partial success; overall it's much better, but it seems to need lining up on both sides.


@sven
Oops, also I can capture with two cards at once - but have no doubt, even modern cards show jitter. In fact, some chips have a feature such as Ultralock, which are supposed to be a line TBC (horizontal shift only), however they don't work. It will be interesting to test the performance of various capture cards in this way, but also to finally discover why it's not working as well as a typical TBC.

[sven_x]

@jmac698
I just had a second look at your latest screen shots (http://screenshotcomparison.com/comparison/87833). Now it looks to me that not only the sync impulse could be taken to line up the lines, but also the transition from very black to grey would do. If the very black borders on the left and right are deeper black than the rest of the image, an algorithm should be possible that finds the very first and very last pixel of the line video content. Doing so we'll get length and pixel offset for each line so that we can
- shrink the line to standard length
- shift the beginning of the line to a standard position

Then all of the driver manipulation would not be necessary and the method might work with USB grabbers as well.
If the borders are not in a deeper black this method will fail in some cases.

[jmac698]

Ok, I have an alpha version of the stretching function. It works:
http://screenshotcomparison.com/comparison/88691
So we've solved software TBC - it was so simple, why didn't anyone do this before? All you need to do is line up both sides. I doubt we even need hsync at all. It's still going to help to see the porch area.

[ronnylov]

It seems to introduce aliasing in areas that did not have aliasing in the original sample, like on the strings across the chest of the hunter.
Maybe it needs some more tweaking?

[Mounir]

that is surprizingly good result to me! (with some aliasing, true that) ; if you could give the procedure to tweak the drivers...

[jmac698]

I don't think you need to tweak the drivers at all, it's just a simple script to line up the edges. The aliasing and top line are just bugs. I'll be tweaking for speed and quality.

[Ghitulescu]

:goodpost:
I can hardly wait for the script.

[jmac698]

Thanks, I suppose I should stop playing video games and try to finish it :)
Maybe I'll get a coffee first...
There's speed and quality issues still.

[sven_x]

@jmac698
I am very impressed! Fantastic result.
Perhaps the algorithm to find borders could become a bit more robust. There is still some ripple. (Using a small cross correlation function for border areas only?)
I am just starting to tweak my VHS grabbings with avisynth. Didée has posted some scripts that use NNDEDI(-2) as deinterlacer. It extracts both fields of a frame, wich gives the odd or even lines of two successing frames and interpolates the missing lines. Then the two resulting frames are merged back into one. With digital sources this also acts as good antialiasing algorithm.


oo=last

nnedi3(field=-2,nsize=0,nns=3) #DVD content: nns=3
merge(selecteven(),selectodd())

D1=mt_makediff(oo,last)
D2=mt_makediff(last,last.removegrain(11,-1))
last.mt_adddiff(D2.repair(D1,13,-1).mt_lutxy(D2,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?"),U=2,V=2)
o=last

But in VHS sources (at least in my own sources) we have the problem, that every second frame has a slightly different position as every first, after deinterlacing. Just browse through the frames after a NNEDI3(-2). You will see it.
Now when both frames are merged back into one the offset between the two frames leads to a blurring effect. The resulting composed frame is less sharp. On the other hand it has more details as one of both single frames, because NNEDI3 cannot invent details that are smaller than three lines.
His method also reduces the noise, because tape noise from both frames is not correlated much. (But film grain of course is not reduced, when both frames originate from a 25 fps movie).
Nevertheless -- his method would work much better, if we succeed in lining up both interpolated frames at exactly the same position.

@Mounir
The aliazing might result from the fact, that the grabbed frame is still interlaced, not processed.

[jmac698]

Thanks,
The antialiasing is a bug in my script which is easily fixed. I've seen Didee's method on a german forum, it was very impressive! But for cartoons I have another method already for this, I call relative jitter. It correlates the same background image over several frames, so even if there is jitter, it's the *same* jitter for several frames, so the frames can be temporally processed. The result is less noise, but still has a static jitter pattern. It doesn't help your problem.

There is a slight jitter still in my sample because I need a subpixel algorithm. That will take some more work.

[jmac698]

New version, faster, bugs fixed.
http://screenshotcomparison.com/comparison/88810

[sven_x]

Just wanted to apply your script to my own source, but I get an error: Invalid arguments to function "YPlaneMin" (GScript line 3 and 8).

[jmac698]

It was tested on 2.6a3
I dunno what to say, it works for me.. maybe I should repost.. I actually did edit it some online

[Gavino]

Just wanted to apply your script to my own source, but I get an error: Invalid arguments to function "YPlaneMin" (GScript line 3 and 8).
You need to use the GRunT (http://forum.doom9.org/showthread.php?t=139337) version of ScriptClip, which allows run-time functions like YPlaneMin to be called inside a user function.

[jmac698]

In other words you need GRunT installed...

[Mounir]

Error: "compare plane: this filter can only be used within run-time filters" (Gscript line3 ,line8)

tested with avisynth 2.6.0.2 and good filters versions i believe

[jmac698]

I used the version I posted and it works, do you have GRunT installed as well? I think I'll just remove the dependency on Gscript/Grunt...

New comparison
http://screenshotcomparison.com/comparison/88873
This shows lining up with video only. To me it looks like it's worse. So that means my hsync technique does help and the side should look ragged when TBC'd.

[sven_x]

@Gavino
Thanks! Works with Avisynth 2.5.8 and GRunT installed.

To me it looks like it's worse.
The last 10 lines or so are shifted to the left, but they should'nt. Looking at the source frame you see a black vertical line there in the lower left corner that the algorithm is falsely taking for the left border (indeed it is content of the frame).

If the black inside the line cannot be separated from the black of the border perhaps some plausibility checks must be involved (see this (http://forum.doom9.org/showpost.php?p=1530806&postcount=26) post).

[jmac698]

Aha. There's other differences, but I can't tell which is better. I should run a test signal through. I left detection as a mask just for the purpose of problems like this. The resulting shift mask can just be blurred to reduce sudden changes! The same ideas of limiting it can be turned into a vertical convolution...

How does it look on your sources?

[johnmeyer]

I just tried the script, unaltered, on some 720x480 NTSC interlaced footage. The left side is the original footage and the right side is the footage processed by the script. As you can see, it looks like the script is actually producing the very problems it is supposed to eliminate! I didn't alter anything in the script -- just downloaded it and ran it.

http://i177.photobucket.com/albums/w208/johnmeyer/Before-After.jpg

Do I need to have the additional information that your capture drivers provide? If so, how do I get that?

[sven_x]

@johnmeyer
Did you feed the script with the left input? So there are the borders missing that the script uses to find the transition to the beginning of the "real" line beginning and end. Without borders it cannot estimate anything.

@jmac698
My sources are grabbed with an 15 Euro USB Video Grabber and a fairly good, very old Toshiba VHS recorder. With this setup sync disturbances are rather small. In most cases your script does not find another solution to line up the lines. I can see no differences between source and processed (switching with AvsP through the scripts).
One video is the copy of a copy from a camcorder. This is the only one with some ripple at the border that needs adjustment. The result is better.

In one case the result looks pure: lots of lines out of row that were okay in the source. I have no idea what is causing this. The borders look clear and smooth in the source.
PAL recordings have 576 lines. A part of the algorithm seem to work with 480 lines only.

I have the impression that removing the blur effect (that occurs when merging two frames) requires sub-pixel accuracy (at least 1/2 pixel).