Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 9th October 2011, 03:56   #1  |  Link
jmac698
Registered User
 
Join Date: Jan 2006
Posts: 1,867
New Script: Software TBC 0.6 & Sample (was Fast Line Shifter 0.53)

Hi,
This is a software "line" TBC. There is a new version, but it doesn't have more quality, just fast enough to be practical (in those cases where it can be used.)

Yes, it has worked for some people's real clips, which had severe "wobbly lines", but don't expect much. I still need to do some basic research here.

Update: New plugin version 0.61
http://www.sendspace.com/file/s78pp4
Installation note:
Please extract the included plugins and place into your plugins directory.
jitter02.zip -> jitter.dll -> plugins\jitter.dll
findpos01.zip -> findpos.dll -> plugins\findpos.dll
You may also need the VC2008 runtime.

If you get an error when trying to load the plugin, try these versions. They are called "static builds", a programmer's term, which means they are fully self contained. The only disadvantage is a bigger download. They are no different in functionality.

jdejitter03_static
http://www.sendspace.com/file/qk1ifv

findpos02_static
http://www.sendspace.com/file/e3jblt


The following applies to the old 0.53, and the theory in general can be used with the new version
----------------------------------

This script is in development and is only for the following cases:
-#1 Video was captured with HSYNC area (via driver tweaks)
-#2 Video contains easily distinguishable black borders on both sides (again, probably needing tweaks).
-If you have a hardware TBC there's no point, but case #2 in a digital file with the original analog source unavailable might be useful
-It's not necessarily any better than existing plugins for case #2
-If you have a supported card, it's possible to have the equivalent effect of a hardware TBC (they work very similar), at least for the "wobbly" function.

How to get the capture
The capture section has moved to "How to Capture with HSYNC" in the capture forum. That topic is unrelated to script usage.
http://forum.doom9.org/showthread.php?t=162832

Results (this only worked so well because I have HSYNC)
http://screenshotcomparison.com/comparison/88810

Results for case#2 from a user supplied video:
http://screenshotcomparison.com/comparison/91853

Theory of Operation
Before I get into usage, it would help to know what we're trying to accomplish. An analog video signal starts and ends each line with a pulse called an HSYNC. On a VCR, the timing is not stable. The capture card starts with the first HSYNC and reads in the video for a set time-period, then waits for the next HSYNC. The result is that the video can appear shifted by a different amount on each line. A hardware TBC probably works by reading the full line and re-adjusting it's timing to the standard amount, thus when captured, you get the entire video line.
With Case #1 we are capturing the HSYNC (the right half of it) and a large window of the video line - thus including all of it, regardless of it's timing. Now it's simply an image that needs lining-up. The script looks for the HSYNC line at the left, then a black area at the right, and resizes the line to a standard size (exactly equivalent to how a hardware TBC re-times). The HSYNC is very easy to find and is a one-pixel or so line. The right edge relies on a short black area which is 'blacker' than anything the video could be, so it's also reliable.

With Case #2 we're relying on black borders being in the video and the video being relatively bright. We simply line-up the black borders to the same spot. Dark scenes in the video could confuse the border detection so it's not as reliable, and that is the case with any other plugin (which is why I say mine is not necessarily any better in this case).

Usage
Quite simple, there is an autothresh which looks for black pixels in the left border. A small amount is added to this to form the real thresh. Adjust the added amount to tweak, until the picture lines up. I used AvsPmod to look at the raw video, and moving my cursor over the black area, I saw it was a noisy 15-25 (in a user test clip). I decided to just set 32. Values slightly lower would leave a few lines wrong, seen as little black stripes at the left edge of the video. For that particular video, only the bright scenes worked correctly. Having it work for dark scenes is being experimented with.

Advanced Usage
The script first makes a mask, in this case every pixel at thresh or above is marked to luma=255 in the mask or 0 elsewhere.
Next I pass my mask to findpos_h which searches for the first 255 value on each line, within the searchwindow. It also simultaneously searches from right to left by searchwindow pixels. It saves the results in a special clip, which records the number of pixels before the mask was found. For example if the HSYNC line occurs at x=4, the luma of the shift clip contains luma=4 for that line, at the left hand side. The right hand side contains the offset from the right of where the video ended, for example if it ended at x=710 and the video is 720 pixels wide, the luma of shift is 10 on the right hand side.
alignbyluma now reads the original video and the shift video and resizes to a standard size. Hopefully this is enough information to do anything else you want here.

What about the aspect ratio?
I still need to calibrate to return the correct aspect ratio. Currently the fixed video is probably wider than it should be.

Limitations
Check the history. Currently searchwindow is not working correctly, but it's enough to work. Also I need to add subpixel detection/shifting to get a really stable result. Currently the result looks like a little noise because of subpixel jitter.

Code:
#Fast line shifter Ver 0.53 by jmac698
#Lines up either or both edges of a video.  Can also be used as displacement for 3d scripts.
#Requires Masktools v2a45+ (mt_lutspa mode), GRunT, GScript
#MinMax http://forum.doom9.org/showthread.php?p=1532124#post1532124
#Limitations: still no subpixel shifting
#0.53: Avoid possible "ScriptClip: Function did not return a video clip of the same colorspace as the source clip!"
#0.52: Less blurry resize
#0.51: Autothresh (uses 2 pixels of left border as a starting point, then adds a small amount to avoid noise)
#0.5: Slow, but using a completely new approach, and can resize whole lines
#0.4: Fast, can detect and line up on left or right edges
#note: sample was frame 167, http://screenshotcomparison.com/comparison/88810

#Modified to work with wide-window sample capture, which includes hsync

src=AVISource("D:\project001a\tbc2\vhs hysnc sample.avi").converttoyuy2
#crop(8,0,0,0)#Uncomment and adjust to remove extra black left border
thresh=int(findthresh(src))+3#This may not always work, try to manually set to 32 for example.  Pick the lowest value which lines up picture.
ScriptClip(src, """
#Mark video edges
converttoyv12
m=mt_binarize(thresh)
#Line up video
findpos_h(m, searchwidth=22)
alignbyluma(src,last)
""")
addborders(56,0,0,0)

function findpos_h(clip m, int "searchwidth", int "x1", int "x2"){
    #Searches m from left to right in the range x1 to x1+searchwidth-1 and right to left in the range width-1-x2 to x2-searchwidth-1
    #for the first luma=255 pixel, then colors the output line with the offset from x
    #for example m is 0 0 255 255 255 0 0 0, width=8, x1=0, x2=0, searchwidth=4 becomes 4 4 2 3 3 4 4 4, then 2 2 2 2 3 3 3 3 
    #Can only search for 255 pixels (as the luma output is only 8 bit)
    #c and m should have the same clip properties (same size)
    #searchwidth should be <=width/2
    searchwidth=default(searchwidth,32)
    x1=default(x1,0)
    x2=default(x2,0)
    rampexpr="x "+string(m.width/2)+" < x "+string(m.width-1)+" x - ?"#x w/2 < x w-1 x - ?
    ramp=mt_lutspa(m, mode="absolute",expr=rampexpr)
    notfound=searchwidth#Value to return if no mask on this line, should be >=searchwidth or you'll find the wrong minimum later
    maskmarker=255#The luma value in the mask which indicates a detected pixel
    #(if m=maskmarker return ramp else notfound), 255 means x>=255, x<searchwidth or notfound
    mt_lutxy(m,ramp,yexpr="x "+string(maskmarker)+" = y "+string(notfound)+" ?")
    #now make solid lines based on min luma found in each line
    l=crop(0,0,-width/2,0)
    r=crop(width/2,0,0,0)
    l=l.minmax(0,0)
    r=r.minmax(0,0)
    StackHorizontal(l,r)
}

function alignbyluma(clip src, clip shift, int "mode"){
    #Shift/scale each line of clip src by the x offset defined by the luma of shift
    #for example if shift were all luma=8, the entire src clip would move 8 pixels to the (dir)
    #This works on a pixel basis, so solid horizontal lines in shift can shift src by variable amounts per line
    #It uses a simple replacement strategy, where each pixel in shift is tested and replaced by the same pixel in a shifted copy
    #Currently handles only 0-15 shifts
    #Magnify everything to get full color resolution
    mode=default(mode, 2)
    shiftuv=shift
    shift=shift.pointresize(shift.width*2,shift.height*2)
    shift=ytouv(shiftuv,shiftuv,shift)
    src=src.pointresize(src.width*2,src.height*2)#We double here to preserve chroma rez
    GScript("
        for (y=0, src.height/2-1, 1) {
            l=int(getpixel(shift,0,y).YPlaneMin)
            r=int(getpixel(shift,shift.width/2-2,y).YPlaneMin)
            getline(src, y*2)
            align(l*2, r*2, 4, 4)
            out=y==0?last:stackvertical(out,last)
        }#for y
    ")#GScript
    out
    converttoyuy2
    bilinearresize(src.width/2,src.height/2)
}

function align(clip v, int xl, int xr, int lb, int rb, int "mode") {
    v#shift an image, x>0 shifts left, xl is amount to shift left, xr is amount to shift right
    #mode 0 is shift left only, 1 shift right, 2 scale to shift left and right
    mode=default(mode, 2)
    offx=mode==0?xl:-xr
    mode<2?pointresize(last.width, last.height, offx, 0, last.width, last.height):crop(xl,0,-xr,0).addborders(lb, 0, rb, 0).Spline36Resize(last.width,last.height)
}

function getpixel(clip v, int x, int y) {
    #get color of a single pixel and return as a fat 2x2 yv12 pixel
    v
    #pointresize(last.width*2,last.height*2)
    crop(x>0?x*2:0,y>0?y*2:0,-(last.width-x*2-2),-(last.height-y*2-2))
}

function getline(clip v, int y) {
    v#return a line of height 2 from y to y+1
    crop(0,y,0,-(last.height-y-2))
}

function findthresh(clip v){
    #Find a resonable starting point for thresh by searching border
    current_frame=0
    v.converttoyv12
    crop(0,16,-last.width+2,-16)
    AverageLuma
}

Last edited by jmac698; 14th July 2012 at 06:16. Reason: Updated
jmac698 is offline   Reply With Quote
Old 9th October 2011, 16:20   #2  |  Link
sven_x
Registered User
 
Join Date: Oct 2011
Location: Germany
Posts: 39
The subject has been discussed several times. Please take a look at this thread.

There were some other attempts to correct line sync jitter (linked in the thread above). I have tested all of these functions - they are very vulnerable to noise which is an element of VHS recordings. Sometimes the resulting line jitter after correction is bigger and more random than that of the input video.
I do not know if someone has tested using a cross correlation function to find the beginning of a line which might reduce the influence of noise.
sven_x is offline   Reply With Quote
Old 9th October 2011, 20:23   #3  |  Link
jmac698
Registered User
 
Join Date: Jan 2006
Posts: 1,867
sven,
I'm doing something that hasn't been done before. I have access to the raw signal including hsync so I have a very strong image to align to. See http://screenshotcomparison.com/comparison/86066
I have aligned the image, even though it was bright, and this line which seems to be hsync, but neither work correctly. My theory now is that it needs both sides. I've written a correlation function, but I don't need it as my test cases were quite clear but they didn't work. The problem must be somewhere else, not in our technique.
jmac698 is offline   Reply With Quote
Old 9th October 2011, 21:00   #4  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,691
As has been pointed out in previous posts on this subject, a good time base corrector, used properly, and inserted at the appropriate point in the analog signal path, will probably completely eliminate this problem.
johnmeyer is offline   Reply With Quote
Old 9th October 2011, 21:27   #5  |  Link
jmac698
Registered User
 
Join Date: Jan 2006
Posts: 1,867
john,
That doesn't seem to make sense - I have access to the full raw signal, I should be able to make a real hardware TBC with only software. If you could d/l a program that does a TBC directly on capture, wouldn't that save you money compared to buying a TBC?
It only takes a low level access driver and any capture card can turn into a TBC.
I just need to develop the technique.
jmac698 is offline   Reply With Quote
Old 10th October 2011, 01:51   #6  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,691
Software is wonderful, but some things can only be done in hardware, especially when dealing with an analog signal. You really cannot create a digital TBC because the sync signal is only available in the analog domain and is not available once the analog video has been digitized. You cannot do the same thing simply by looking at the resulting badly captured video and then trying to figure out what to do with the bad video.

If you want really good video from your old VHS analog video, then you MUST use a TBC. I admire your desire to do many things in software, but this is one situation where you are not going to get very good results.
johnmeyer is offline   Reply With Quote
Old 10th October 2011, 02:51   #7  |  Link
jmac698
Registered User
 
Join Date: Jan 2006
Posts: 1,867
john,
I appreciate your sentiment, however I seem to have not managed to communicate that I *do* have access to the sync signal, through special driver tweaks, you can see a picture of it (including the color burst) above.
So I actually agree with you, you need the sync, and I do have it.
jmac698 is offline   Reply With Quote
Old 10th October 2011, 07:45   #8  |  Link
ronnylov
Registered User
 
Join Date: Feb 2002
Location: Borås, Sweden
Posts: 492
The time between two line sync pulses should in theory be a constant but on a recorded tape it may fluctuate a little bit during playback because of tape tension. So instead of just syncing the start maybe you need to stretch back the length of each line to the nominal value. Check the time distance between each pair of hsync pulses and readjust the signal back to nornal. Then you can align the lines.

Just an idea you can try if you have captured the complete video signal. I don't know how a real hardware TBC work.
__________________
Ronny
ronnylov is offline   Reply With Quote
Old 10th October 2011, 14:50   #9  |  Link
AVIL
Registered User
 
Join Date: Nov 2004
Location: Spain
Posts: 408
@jmac698.

I think your approach is good. But IMHO the result is worst than original (as seen in your pictures). Could be that image pixels must be moved in opposite direction to sync ones. Perhaps the line must be streched/expanded instead moving the pixels. But I like this method.

BTW. Where can be obtained the modified driver that does sync signal capture? I find it very useful
AVIL is offline   Reply With Quote
Old 10th October 2011, 15:38   #10  |  Link
sven_x
Registered User
 
Join Date: Oct 2011
Location: Germany
Posts: 39
An analysis of the provided VHS signals

I have applied a very basic approach to your screenshots:
1. Analysing the provided VHS grabbings
2. Trying to sync the lines manually in photoshop and looking if this is possible and how the result looks.

Analysis

The picture below shows begin, middle part and end of a group if lines taken from your screenshot. One can see, that offset (time delay) is nearly constant in each line, that is the pixel shift is the same at beginning and end of the lines.
It does not correlate to the sync signal position.


Correction
I have imported the synced screenshot into photoshop. Line heigth was increased to 300% to get a better view.
Using a fixed selection window (720px x 3px) I copied each line into a new have layer and than manually shifted that line to a position that looked good.

The result is given below:



Conclusions
When a correction of the image could be done by eyes view it also must be possible to put that in a computer algorithm.
  1. Lines from VHS grabbings cannot be synced by using what was regarded as sync signal
  2. Lines from VHS grabbings can be synced using cross correlation between subsequent lines.
  3. There is also a small jitter in line length, that needs an extra treatment.
  4. The line offset jitter does not correlate very well to the beginning of the lines (which is the beginning of total black pixels) (but not as bad as to the sync signal)

Many thanks to jmac698 for the interesting approach and for the screen shots which helped me a lot.

Last edited by sven_x; 12th October 2011 at 18:10.
sven_x is offline   Reply With Quote
Old 10th October 2011, 16:34   #11  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,691
I think Sven's amazing post shows clearly that there is no correlation between the sync pulses you have captured, and the artifacts you are trying to remove. The easy conclusion is that they are caused by something other than time base errors. However, another conclusion is that you have not really "captured" time base information (or at least have not captured it correctly) with your modified driver.

I do not say this in order to be difficult, but at the risk of sounding like I only have one thing to offer, I would once again like to suggest trying to borrow a real hardware TBC and use it. I say this because the visual disturbance you are trying to correct is something I have seen many times, and a TBC circuit has, for me, always been able to remove the line displacements.

Last edited by johnmeyer; 10th October 2011 at 16:35. Reason: typo
johnmeyer is offline   Reply With Quote
Old 10th October 2011, 16:42   #12  |  Link
jmac698
Registered User
 
Join Date: Jan 2006
Posts: 1,867
sven,
That was incredible! I know this has been tackled many times, but I believe it's the first time there's been an analysis including the hidden area of hsync. It's very inspiring to me. However I can do a better job in making examples. I should record a test signal which is easier to line up, then we can measure directly the stretch/offset. My only concern is that the result won't be as bad as this tape. I picked this tape initially because it is a well-worn kids movie and showed massive jitter.
Does anyone know how I can make a tape with *bad* jitter? Perhaps if I copy many generations?

I had already felt that something was inherently wrong with lining up just by the left edge. People thought it failed because of their line-up algoritm (due to noise, etc.), but I had thought of the possibility of tape stretch and/or slight variations in head speed causing the line-stretching effect. In this case there's bad news, because I can only capture hsync once per line, if I capture the next hsync then I've missed the hsync for the next line and essentially never get any more lines.
Why that line is not good for lining-up is confusing to me however.


Last edited by jmac698; 10th October 2011 at 16:46.
jmac698 is offline   Reply With Quote
Old 10th October 2011, 16:50   #13  |  Link
jmac698
Registered User
 
Join Date: Jan 2006
Posts: 1,867
john,
You may be correct that the line is not an hsync, I will verify this. I'm not even trying to TBC that image; what's more interesting to me is to develop a method of software TBC. So I'm not trying to be practical here, just doing research and development. The research itself is interesting, but I still believe there's a good chance of a practical result in the end, and I'm using some techniques that haven't been tried before. That it failed initially doesn't matter, research is about answering a question. I haven't given up until I have fully understood the problem and concluded that it's impossible (at least for someone with my abilities).
jmac698 is offline   Reply With Quote
Old 10th October 2011, 17:15   #14  |  Link
Ghitulescu
Registered User
 
Ghitulescu's Avatar
 
Join Date: Mar 2009
Location: Germany
Posts: 5,769
Shifting is not the only algorithm in a HW TBC. It stretches and squeezes the analog line too. You need to do also a scaling in your algorithm.
__________________
Born in the USB (not USA)
Ghitulescu is offline   Reply With Quote
Old 10th October 2011, 22:35   #15  |  Link
IanB
Avisynth Developer
 
Join Date: Jan 2003
Location: Melbourne, Australia
Posts: 3,167
Quote:
Originally Posted by jmac698 View Post
...
Does anyone know how I can make a tape with *bad* jitter?
...
Using a tape you definitely do not love, unspool an amount of tape and carefully stretch it a little, then respool it. Viola jitter video.

When the tape is stretched it is not uniform. Better quality tapes use stronger base material which is more resistant to deformation. Commercial video releases notoriously use tape from the lower quality end of the range, penny pinching misers.
IanB is offline   Reply With Quote
Old 11th October 2011, 23:32   #16  |  Link
jmac698
Registered User
 
Join Date: Jan 2006
Posts: 1,867
I'm still looking to answer my original question: Can you suggest a way to speed up the dejitter script I first posted?
General discussion about the practicality of a software TBC has been moved to
http://forum.doom9.org/showthread.php?t=152706&page=2
Thanks.
(Where's the amazing Gavino? I bet you can optimize the script! And I finally had to use GScript
jmac698 is offline   Reply With Quote
Old 12th October 2011, 00:21   #17  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,431
Quote:
Originally Posted by jmac698 View Post
I'm still looking to answer my original question: Can you suggest a way to speed up the dejitter script I first posted?
A very minor speedup to the existing script would be to initialise 'out' to a fixed blank line instead of calling getline(), since that first line is a dummy that gets cropped at the end.

Nice to see you using GScript , but I wonder if it could be done more quickly using some Masktools wizardry to process the rows in parallel. I don't have time just now to figure out how.
__________________
GScript and GRunT - complex Avisynth scripting made easier
Gavino is offline   Reply With Quote
Old 12th October 2011, 00:36   #18  |  Link
jmac698
Registered User
 
Join Date: Jan 2006
Posts: 1,867
That's a good idea, you can do a pixel shift with a convolution, let me try.
Update:
You can shift the luma 10 pixels to the right like this:
Code:
mt_convolution(horizontal="1 0 0 0 0 0 0 0 0 0 0",chroma="process")
This also shifts the luma:
Code:
mt_luts( last,last, mode = "0 0 0 1 0 0 0 0 0", pixels = mt_square(3), expr = "y" ,chroma="process")

Last edited by jmac698; 12th October 2011 at 01:27.
jmac698 is offline   Reply With Quote
Old 12th October 2011, 04:14   #19  |  Link
jmac698
Registered User
 
Join Date: Jan 2006
Posts: 1,867
Fast Line Shifter 0.2
[deleted]

Last edited by jmac698; 16th October 2011 at 02:51. Reason: Obsolete
jmac698 is offline   Reply With Quote
Old 12th October 2011, 07:38   #20  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
Quote:
Originally Posted by jmac698 View Post
Hi,
The script I have is extremely slow but it works - is there any way to speed it up?
How about moving GScript(" ... ") # GScript outside of ScriptClip,
I think that there is some kind of initialiser involved [Executed on every frame???] and it might
(or might not) make a bit of an improvement in speed, or maybe even try the GImport option. I think perhaps Gavino would have mentioned this if it would be useful, & so perhaps may not help.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 12th October 2011 at 07:40.
StainlessS is offline   Reply With Quote
Reply

Tags
tbc

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 08:51.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.