New Script: Software TBC 0.6 & Sample (was Fast Line Shifter 0.53)

[StainlessS]

I've had it confirmed that Jmac698 is in fact as dead as a doornail, sad but true.

EDIT: Jmac was on-line when I posted this, he disappeared without responding, must be in mourning.
(Nah, he was probably looking up his below response)

Jmac698, Aug 1969 - Aug 2013, sorely missed. R.I.P. :)

[jmac698]

Rumours of my death are greatly exaggerated :P

[anathema]

I'm trying to use the Fast Line Shifter script to fix up a chunk of video which has some fairly major line-displacements. The problem I've hit is that the method used to detect where the blanking ends and the active picture begins is getting tripped up by the rather variable black levels in the picture. If I set 'thresh' to 34 then the displaced lines are pulled back into position, but unfortunately a dark area of the picture ends up being distorted. If I set 'thresh' low enough to prevent the distortion, the displaced lines don't move far enough.

This is the captured image (luma-channel only - for some reason the chroma didn't suffer from the timebase problem):
http://www.nightshade.org.uk/raw.jpg

With 'thresh=34' the displaced lines are detected in their entireity but you can see a large chunk of black in the lower-left of the frame, which corresponds to a dark area of the original picture:
http://www.nightshade.org.uk/thresh34.jpg

Setting 'thresh=24' gets rid of the error, but at the expense of missing bits of the displacement problem:
http://www.nightshade.org.uk/thresh24.jpg

Looking at the luma values in the blanking area it's pretty clear that, while the black areas are noisy, there is a marked jump where the active picture starts. I was thinking that a better algorithm might be to look for the rate-of-change in the luma, rather than for an absolute difference between two adjacent pixels. Before I set about writing one, does anyone know: a) does such a detector already exist; or b) can one be written using existing filters?

[Ludvig Friberg]

Very interesting thread, keep up the good work!
jmac: I tried sending a PM but your quota was full.

[StainlessS]

Jmac's quota is always full.

Not because popular, cos does not clear old ones. Long standing problem. (EDIT: been full for a year or so).

[Ludvig Friberg]

Ahhh. I see. I want to get hold of him to ask about a different project. I am looking for a way to construct and filter video from raw sampled waveforms. Doing chroma separation, TBC and everything else in software.

[lansing]

hi I wanted to use this filter for my vhs record, I copied the script in the first post, and it's giving me an error about no function name minmax(), where does this function come from?

edit: nvm, I just found it on the top of the first post.

edit2: Now I'm getting "MinMax: this filter can only be used within run-time filters, [GScript], line x..." message. My script is a just copy from the sample script in the first post.

[lansing]

I also tried vcmohan dejitter, it runs more easily. But there's a lot of false positive detections.

I played around with it for an hour, I think it's missing something like a jitter minimum parameter for better control. And the threshold parameter doesn't seem to do what it suppose to do. For example, in one jitter scene anything above threshold 30 will detect it, on another scene I need to raise it to 80. But on the 3rd scene, the 80 threshold will not detect the jitter, I have to lower it back to 30 for it to work, which doesn't make sense to me.

[rs008f]

The download links no longer works. Where can I download the files?

Is this script a good candidate to fix this horizontally moving back and forth video.

https://mega.nz/#!T19inJjK!nRCI9349rOQHJ1USeewaUD3y7oJ9klz5s87fDxZDuTE

[lordsmurf]

The download links no longer works. Where can I download the files?
jmac had the bad habit of using freebie "file sharing" sites, which are infamous for purging/losing files. I kept telling him not to do that. He and I discussed his software TBC work quite a bit at digitalFAQ.com. And at that site, we have a habit of attaching files to forum posts, to prevent exactly this sort of loss. So a lot of his files can still be found there, though sadly not all.

His tbc06 zip is here: http://www.digitalfaq.com/forum/video-restore/2742-power-motion-tracking-2.html#post21874
As are the Robotech samples we were working with.

But, as you can see, the TBC never really worked. It was marginal improvements at best. So don't get your hopes up.

I think he was doing all this in MatLab. It had potential, but he lost interest too quickly. A software TBC is sorely needed, to correct all the botched nth-generation VHS copies that exist, and lousy DVD recordings from amateur equipment.

[johnmeyer]

I agree. It was a cool idea that seemed like it could have worked.

I wonder if some of StainlessS' RT_Stats code could be used? I've been able to use it for all sorts of line-to-line comparisons. The shifting part of the code (shifting an entire line left/right) should be straightforward, as long as you have a clear left edge (which is what he was using). The bigger issue is that time base errors are often not consistent across the entire frame, left to right. However, even a first-order correction would be welcome.

[lordsmurf]

What's missing is temporal intelligence.

When you take a video with horizontal jitter, and hit it with a harsh temporal NR, you'll notice that the wiggling largely dissipates. What you're left with, of course, is nasty temporal artifacts (aka "mouse trails", ghosting). But motion-stable artifacts!

The solution for a software TBC could be to run temporal NR, use the results as a baseline for where the lines should be, and then shift accordingly.

More chaotic timing noise would need some degree of both the line shifting, and temporal NR. Perhaps even something semi-intelligent, maybe based on Donald Graft's dynamic NR, to reduce ghosting artifacts.

I think this filter was going down the wrong path.
(1) You never have the sync data. Proof-of-concept was somewhat stupid, as it required a VHS tape be captured with special extra data. That's just not real-world whatsoever. I think that was a sidetrack tangent to where it could have gone.
(2) Timing errors are almost never static, but continuously move. It's analog. Analog is controlled/restrained chaos. The quicker you realize this, the better you are long-term.

I wish somebody would take up where jmac698 stopped. He was on to something.

[StainlessS]

You need to be logged into your DigitalFAQ.com account to download above linked files.

[jmac698]

@johnmeyer
"The bigger issue is that time base errors are often not consistent across the entire frame, left to right."
Correct. That's why it started as fast line shifter then turned into software tbc. Both the left and right edges have jitter, which means there's some horizontal resizing per line. The filter does work dynamically per line.

@lordsmurf
"You never have the sync data" Semi-true; for a professional, it is assumed they are going to buy the equipment they need to get the sync data. Not only that, there were a huge number of cards based on that chipset which could be made to work, most famously the WinTV series.

Second, I've successfully applied it even without sync data; if the capture window is large to enough to leave black borders on both sides (video supplied by cherbette).

You misunderstand that I never came into this thinking it would be a polished, end-user solution; it was experimental, meaning I was trying to understand the nature of the problem to start. And I did learn something; I learned specifically that left-aligning the sync is not the solution; I have to fit between two edges or two syncs. I just put it out there expecting people to play with the functions, but no one ever did :(

This is how engineering is done, start by a) capturing data on the problem b) creating a model for the problem c) solving the model d) apply the solution to the real-world e) refine and iterate.

I also showed to a lot of people that it can be done in software and helped dispel a lot of the misinformation that you need some kind of hardware box. The internet was adamant that it was impossible without sync, that's not true though.

Anyhow, there is another way. There's a paper which works by using mathematical properties (an L1 norm*) which have a good chance of saying if lines are aligned. Someone else tried to implement it too, but we were both stuck over one sentence which wasn't explained. I was in contact with the author but she was very sick at the time. I can still finish this someday.

Another experiment someone did is detecting alignment with a Fourier Transform, but that didn't work out.
*an L1 norm is based on Least Absolute Deviations

"What's missing is temporal intelligence" what you're getting is a frame blending with random horizontal offset. What you call stable jitter motion is due to this, mathematically speaking:
the jitter is guassian random, and as you average frames, the jitter is reduced by the square root of the number of frames, making it look more stable. However, the ghosting is increasing in the same proportion. Features which don't move will literally be guassian blurred. Features that move will also have motion blur. I don't think using this as a reference would really work.

Using my process of engineering I could attempt to prove this; I would simulate jitter, apply temporal denoising, then try to fix it. That's going through the steps of modelling and simulation.

I still can't explain why there is jitter to begin with; the capture card clearly adjusts sample rate based on the sync timing, it is in fact a built-in TBC.

[lordsmurf]

I have at least one sample clip that could be right-aligned, as the black image border is visible. If you can align that, the video should be steady.

I'm still not sure you understand what I mean by "temporal intelligence". You'd not average anything, aside from analysis. If you sample the same line across (X) [3-7] frames, and at least (Y) [25-75] % is still the same (noting "the same" occurs AFTER the pixel shift), then decide if the line shall be moved in the direction of the majority position. Temporal analysis, not temporal application.

This would process slow as hell, but in the modern computing era (Skylake+), it can probably be done. I'd imagine its probably not worse than Mercalli on an old CPU.

I can think it up, but programmers (even mathematicians) more gift than I must create it. That part is beyond me (although I could probably learn, given many years).

[jmac698]

As to the person who couldn't get it to work 3 years ago; you were using it wrong. I believe I can script around the problems with the varying thresh levels.
I agree with your conclusions that the border detection wasn't good enough. I was expecting that other scripters around here would easily be able to address that problem. If anyone can post a sample I could work on that problem and it will still work.
The basics of this are sound and proven; all you need is some black border, and a better border detection. No need for sync.

[jmac698]

Post your sample.

Yes I understand you meant it as a detect clip. Pans would obviously make it fail, because is it a pan or jitter? What about diagonal lines? I think it could work in some scenes but not others. It's worth an experiment though; I believe to always keep an open mind and each failure only helps you learn more :)

[jmac698]

ohh... I have to post this before I forget. I had an idea for static scenes (stationary scenery with just characters moving, end credits, titles etc.). All you have to do is align the frame relative to the last one; more than that, to the average offset of all the static frames, this then has to average close to the true position.


**** +4
**** +1
**** +2

2nd line, being the left-most aligned, is taken as a reference line. From that, we get +1 and +3 offsets, average that to +2 offset and move that line (in all frames) to that position. It is now much closer to the true offset, and it's guaranteed to converge to the true offset by the properties of statistics. I'd say a 1 second scene is easily enough to get nearly perfect sync (30 frames can account for 5 pixels of offset). Each line will independently line up with those above/below like magic :)

*1 proof:
https://math.stackexchange.com/questions/715629/proving-a-sample-mean-converges-in-probability-to-the-true-mean

*2 assuming jitter is i.i.d.

[lordsmurf]

Pans would obviously make it fail
Not necessarily.

It would make it again hit the CPU and/or RAM, but this one is (probably?) easy. Analyze by frame temporally, then by the line. If the entire frame shows signs of panning, then said motion should be accounted for. Yet another %/slider/option should be present, to provide threshold for "what is a pan". Because, as you say, all clips are different.

To not waste CPU cycles, put the entire frame range into RAM.

I have 16gb RAM, i7-6700k CPU. I routinely get annoyed by Avisynth filters not taking advantage of the resources, usually using less than 25% (even with MT options).

[jmac698]

@lordsmurf
Great idea! Though I thought of something like that after I posted.

[lordsmurf]

I'm going to fully recapture the footage I want you to test a correction with. I pulled the tape tonight, and will get back to you sometime soon. I'll see what my capturing options are.

[jmac698]

My suggestion is, try to get black on both sides, and turn up the brightness (with procamp) so we can distinguish the blacker than black from the dark. In cases I've seen, there was a clear difference.

Ohh.. now I had another idea. By the same process, if you capture a scene many times (too many to be practical, tbh), it should naturally average out as well.

How would you feel about capturing a few seconds 9 times? haha :)

[jmac698]

I think what I need now is an edge filter followed by RANSAC (this is an algorithm to get rid of outliers; what should remain is true corresponding edges) and can use that for alignment.
The other way is correlation/fourier, that tries to find the best "sum" that matches position and scale.

[lordsmurf]

RANSAC (this is an algorithm to get rid of outliers;
Ah, that gets into statistics, which I do have some skill at. :)

You need to be careful about the stats method across all frames. You don't want 100% averaging, as outliers can drag the norm in its direction. What you want is a truncated mean of some sort, and I find a winsorized mean to often be more accurate.

The other way is correlation/fourier, that tries to find the best "sum" that matches position and scale.
I'm not sure a fourier transform is the way to go. For one thing, even with a still image, it just kills a CPU. Does the sum exclude outliers?

Ohh.. now I had another idea. By the same process, if you capture a scene many times (too many to be practical, tbh), it should naturally average out as well.
I think that would be more academic than anything else.

Generally damaged footage will already be digital, with access to the tape unavailable. Aside from really bad nth gen errors embedded/compounded on the tape, a quality S-VHS VCR with line TBC (or Es10 on passthrough) would easily correct the timing problem, thus negating the filter's need.

I mean, yeah, sure, I can do it. But I don't think it will help much. In my experience, multi capturing doesn't result in a big difference, if at all. Again, embedded errors, meaning the positioning would remain mostly or entirely unchanged.

[jmac698]

If your sample has embedded errors, don't bother then.

I've been looking at robust statistics lately, and ran across windsoring, but it has a bias. That would still leave the line with an outlier of jitter sticking out (just not as much).

Robust methods are rated by their breakdown point, which means what % of the data has to be corrupted in order to give you a bad estimate. A method called least median of squares has 50% breakdown, which is really good.

Imagine a graph where the x positions are non-linear, based on the cummulative sum of a guassian. It will be .5 halfway, increase quickly then taper off..

* * **** * *

The Y positions, you plot your data, but sorted and cummulative. If the data comes from a guassian, this should now form a straight diagonal line. It's similar to the concept of log/log graph paper. This straight line has an intercept (where the line touches the x-axis, ideally 0), which is the mean, and a slope (normally 1) which is the standard deviation. At this point you would call it the linear regression method. If the mean of the data is higher, the line should shift to the right. If the sample is 'fatter' (more variation), the area under the curve is always higher, so the line slope should increase more, and slant up.

Modify this concept again, but choose only the points that cause the least deviation from a straight line. Yes that means trying every set of points and see how good of a line they make. However this is now called the least median of squares. But you can see how it's getting rid of outliers, it chooses only the most 'clean' data. It works like a majority vote; outliers are completely eliminated from consideration. Using the chosen points to form a slope, is like taking an average of the best data. You can see it's adaptive; it uses the most clean data available and then reduces to a simple average (the sample mean is the population mean, it's consistent, efficient, unbiased, but not robust).

You might think that a guassian is infinite, however our data isn't, and it's not an actual graph but a virtual calculation, so you can include all the data arbitrarily.

This method of choosing only certain points is what puts 'median' in the name. It's also what gives it it's robustness, as you know a median isn't affected by outliers. In the worst case think of 3 points in a triangle; the longest side would be the best fit line.

There is an even more powerful method called maximum likelihood, but if you solve it for the case of Gaussian/normal, you get the method above.

This method is the best you can do; it's consistent, which means it always gets closer to the population mean (the true mean) as you use more data, it's efficient, which means it reaches the true conclusion the quickest as you use more data, it's unbiased.

[jmac698]

Ok I finally realized your point, this whole process is moot because if you have the tape then you can use a TBC to begin with. So baked in jitter is the only problem that needs solving here? Does TBC also fail sometimes?
I mean, it was still useful to me and some other people who couldn't get a TBC for whatever reason.

[lordsmurf]

I've been looking at robust statistics lately, and ran across windsoring, but it has a bias.
That's the point. You need bias. If the frame exceeds x pixels, then it should be ignored. Otherwise erratic skew can throw off the mean used for the line readjustment.

This should also be a variable switch, letting you adjust it higher or lower. Note the minimum must be 3 pixels, while something like 20 disables it.

this whole process is moot because if you have the tape then you can use a TBC to begin with.
Exactly.

So baked in jitter is the only problem that needs solving here?
Yes.

Does TBC also fail sometimes?
TBC is useless on embedded errors, and fails 99% of the time. It's rare to have an embedded error respond to TBC corrections.

I mean, it was still useful to me and some other people who couldn't get a TBC for whatever reason.
You'd be wasting time reinventing the wheel solely for the cheapness of others. You can buy a good ES10 for under $150, or JVC for under $300. If you're cheap, then buy it, use it, resell it.

Let's focus on currently-uncorrectable problems with video, creating something that does not exist to address it. And software TBC (lack thereof) is still a major issue.

I think we're on to something here. :)

I know MUCH more than I did 6-7 years ago when we met and this project was started, and you're probably the same. Let's finish it.

[FranceBB]

Hi there,
I was actually interested in a software TBC through Avisynth but the link is dead. Did someone archive the plugin?

[johnmeyer]

I have one version of the script, copied below. However, I think this was created before he had plugins. Are you looking for the jitter.dll and findpos.dll?

My guess is that those two plugins use somewhat the same algorithms that he used in this original script.
#Fast line shifter Ver 0.5 by jmac698
#Lines up either or both edges of a video. Can also be used as displacement for 3d scripts.
#Requires Masktools v2a45+ (mt_lutspa mode), GScript & GRunT http://forum.doom9.org/showthread.php?t=147846
#MinMax http://forum.doom9.org/showthread.php?p=1532124#post1532124
#Limitations: still no subpixel shifting
#0.5: Slow, but using a completely new approach, and can resize whole lines
#0.4: Fast, can detect and line up on left or right edges
#note: sample was frame 167, http://screenshotcomparison.com/comparison/88810

#Modified to work with wide-window sample capture, which includes hsync


loadPlugin("C:\Program Files\AviSynth 2.5\plugins\GScript.dll")
loadPlugin("C:\Program Files\AviSynth 2.5\plugins\minmax.dll")
loadplugin("C:\Program Files\AviSynth 2.5\plugins\mt_masktools-25.dll")
loadplugin("C:\Program Files\AviSynth 2.5\plugins\GRunT.dll")

src=AVISource("e:\frameserver.avi")
#crop(58,0,0,0)#Uncomment to line up by picture edge
out=ScriptClip(src, """
#Mark video edges
#thresh=72
thresh=55
converttoyv12
m=mt_binarize(thresh)
#Line up video
#findpos_h(m, searchwidth=22)
findpos_h(m, searchwidth=8)
alignbyluma(src,last)
""")
stackhorizontal(src,out)

function findpos_h(clip m, int "searchwidth", int "x1", int "x2"){
#Searches m from left to right in the range x1 to x1+searchwidth-1 and right to left in the range width-1-x2 to x2-searchwidth-1
#for the first luma=255 pixel, then colors the output line with the offset from x
#for example m is 0 0 255 255 255 0 0 0, width=8, x1=0, x2=0, searchwidth=4 becomes 4 4 2 3 3 4 4 4, then 2 2 2 2 3 3 3 3
#Can only search for 255 pixels (as the luma output is only 8 bit)
#c and m should have the same clip properties (same size)
#searchwidth should be <=width/2
searchwidth=default(searchwidth,32)
x1=default(x1,0)
x2=default(x2,0)
rampexpr="x "+string(m.width/2)+" < x "+string(m.width-1)+" x - ?"#x w/2 < x w-1 x - ?
ramp=mt_lutspa(m, mode="absolute",expr=rampexpr)
notfound=searchwidth#Value to return if no mask on this line, should be >=searchwidth or you'll find the wrong minimum later
maskmarker=255#The luma value in the mask which indicates a detected pixel
#(if m=maskmarker return ramp else notfound), 255 means x>=255, x<searchwidth or notfound
mt_lutxy(m,ramp,yexpr="x "+string(maskmarker)+" = y "+string(notfound)+" ?")
#now make solid lines based on min luma found in each line
l=crop(0,0,-width/2,0)
r=crop(width/2,0,0,0)
l=l.minmax(0,0)
r=r.minmax(0,0)
StackHorizontal(l,r)
}

function alignbyluma(clip src, clip shift, int "mode"){
#Shift/scale each line of clip src by the x offset defined by the luma of shift
#for example if shift were all luma=8, the entire src clip would move 8 pixels to the (dir)
#This works on a pixel basis, so solid horizontal lines in shift can shift src by variable amounts per line
#It uses a simple replacement strategy, where each pixel in shift is tested and replaced by the same pixel in a shifted copy
#Currently handles only 0-15 shifts
#Magnify everything to get full color resolution
mode=default(mode, 2)
shiftuv=shift
shift=shift.pointresize(shift.width*2,shift.height*2)
shift=ytouv(shiftuv,shiftuv,shift)
src=src.pointresize(src.width*2,src.height*2)#We double here to preserve chroma rez
GScript("
for (y=0, src.height/2-1, 1) {
l=int(getpixel(shift,0,y).YPlaneMin)
r=int(getpixel(shift,shift.width/2-2,y).YPlaneMin)
getline(src, y*2)
crop(l*2,0,-r*2,0).addborders(4,0,20,0).BilinearResize(last.width,last.height)
out=y==0?last:stackvertical(out,last)
}#for y
")#GScript
out
converttoyuy2
bilinearresize(src.width/2,src.height/2)
}

function align(clip v, int xl, int xr, int "mode") {
v#shift an image, x>0 shifts left, xl is amount to shift left, xr is amount to shift right
#mode 0 is shift left only, 1 shift right, 2 scale to shift left and right
mode=default(mode, 2)
offx=mode==0?xl:-xr
mode<2?pointresize(last.width, last.height, offx, 0, last.width, last.height):crop(xl,0,-xr,0).BilinearResize(last.width,last.height)
}

function getpixel(clip v, int x, int y) {
#get color of a single pixel and return as a fat 2x2 yv12 pixel
v
#pointresize(last.width*2,last.height*2)
crop(x>0?x*2:0,y>0?y*2:0,-(last.width-x*2-2),-(last.height-y*2-2))
}

function getline(clip v, int y) {
v#return a line of height 2 from y to y+1
crop(0,y,0,-(last.height-y-2))
}

[StainlessS]

Not Jmac TBC, but I have this from 2008:- https://code.google.com/archive/p/avisynthrestoration/downloads

Copyright 2008 halifaxgeorge licenced under GPL v3

tbcv01.avs [link has additional stuff]

#~ This program is free software: you can redistribute it and/or modify
#~ it under the terms of the GNU General Public License as published by
#~ the Free Software Foundation, either version 3 of the License, or
#~ (at your option) any later version.

#~ This program is distributed in the hope that it will be useful,
#~ but WITHOUT ANY WARRANTY; without even the implied warranty of
#~ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
#~ GNU General Public License for more details.

#~ You should have received a copy of the GNU General Public License
#~ along with this program. If not, see <http://www.gnu.org/licenses/>.

#TBC 0.1
#Released: Sep 24, 2008
#License: Copyright 2008 halifaxgeorge licenced under GPL v3
#Author: halifaxgeorge
#Description: Avisynth Script which stabilizes timing jitter in video from analog VCRs, similiar in function to a TimeBaseCorrector.
#Limitations: This script is very slow and uses a lot of memory due to limitations in Avisynth.
#Simple things that are possible to program are not practical to run, I consider this a limitation. There is nothing compute intensive here.
#Functions:
#greybars(clip "vid", int "count", float "starty", int "endy", int "n", int "w", int "h") - create testpattern in length frames. strty,endy - starting
#and ending luminence values. n number of vertical greybars. w width of resulting clip. h height of resulting clip. count, used internally for loop counter.
#vid, used internally for stacking greybars together.
#greybars() creates a 300 frame video.
#hexcolor(float "c1", float "c2", float "c3"), the values c1-c3 are y,u,v. The result is a avisynth colorvalue, which can be used for example in blankclip.
#hexcolor(y) - specify only the Y value.
#jitter(clip in, int jitter) - simulates timing jitter. in is video to be de-stabilized. jitter is pixels/2 to horizontally shift video sections
#limitations: this is a very rough simulation. The video is sliced into 5 horizontal sections, of only 16 pixels, each slice is randomly shifted
#in the horizontal direction. A true jitter would shift every line randomly.
#Now what is this shift operation? It's a scaling in x with revealing a black background behind, equivalent to the timescaling
#of a mechanical video head turning too fast (thus shorter video line).
#tbc(clip) - the supplied clip is stabilized as a whole, thus ideally, one would submit a one-pixel high line.
#however, avisynth doesn't handle 1 pixel lines in YV12, so practically we will send a point-resized 2 pixel high line for each
#true line of video.
#The stabilization algorithm: the luminence values are searched from far right to left by 16 pixels (ie 704-719) until
#a luminence above thresh is found. Thresh is hardcoded into the function at the moment. The result is the x value
#where the thresh was found. Based on this x value, the line is resized to full size. Full size is normally 720 pixels,
#however another hardcoded value, the offset, resizes to less than 720 pixels.
#The purpose of offset is, that searching for thresh consumes some part of the right edge of the video, thus
# the last few pixels (less than thresh) would be cutoff. Therefore, now one can consider that *detection* of the video edge
#is separated from the re-scaling.

#Results: This script does work, and was able to stabilize a real analog video. However it was clear that many improvements
#could be made. One is that thresh is too simple to detect the edge of the video. The presence of a spike of noise would
#cause instant pixel jumps of that line when it shouldn't. Also varying subpixel posiitioning and luma variances by line
#can cause imprecise de-jittering. What is needed is a stable, subpixel edge detection.
#Also the left side of the video can be stabilized as well; this is easy if you flip the video and pass it through twice,
#though the roundoff of the calculations would build up. A true double-sided one pass filter would be better.
#In summary, it's amazing that this actually works and demonstrates advanced per-pixel and per-line manipulation
#solely in a script, where AviSynth is mostly per-frame based. It is useful to prototype stabilization algorithms
#with rapid prototyping before writing a filter (which I currently don't know how to do anyhow :( ).

function greybars(clip "vid", int "count", float "starty", int "endy", int "n", int "w", int "h") {
starty=defined(starty)?starty:16
endy=defined(endy)?endy:235
n=defined(n)?n:6
w=defined(w)?w:720
h=defined(h)?h:480
vid=defined(vid)?vid:blankclip(30*10,w/n,h,"YV12",color_yuv=hexcolor(starty))
count=defined(count)?count:0
step=(endy-starty)/(n-1.0)
count<n-1?stackhorizontal(blankclip(30*10,w/n,h,"YV12",color_yuv=hexcolor(starty+count*step)),greybars(vid=vid,count=count+1,starty=starty,endy=endy,n=n,w=w,h=h)):blankclip(30*10,w/n,h,"YV12",color_yuv=hexcolor(starty+count*step))
}
function hexcolor(float "c1", float "c2", float "c3") {
#This function returns a hex color value from 3 decimal values.
#It defaults to black as defined in the YUV colorspace, which is "$108080".
#This is an example: hexcolor(235) returns "$EB8080".
c1=defined(c1)?c1:16
c2=defined(c2)?c2:128
c3=defined(c3)?c3:128
c1=round(c1)
c2=round(c2)
c3=round(c3)
c1*65536+c2*256+c3
}
#To use on a real video:
#v=MPEG2Source("C:\MyVideos\vcr\rca.d2v", cpu=0)
#remove jitter line
#remove stackvertical line, which is just for visual comparison of before and after
#adjust hardcoded offset and thresh in tbc function
#adjust hardcoded '80' in slice to 480.
#hope it doesn't crash - it's slow and takes a lot of memory. Adjust slice value slowly upward.
#this works! The first 80 lines are TBC'd and highlighted
v=greybars()
#v=MPEG2Source("C:\MyVideos\vcr\rca.d2v", cpu=0)
v
jitter(4+1)
slice(last,0)
pointresize(last.width,last.height/4)
stackvertical(last.coloryuv(off_y=24),v.crop(0,0,0,480-80))
# pointresize(last.width,last.height*2)
# crop(0,4*2,0,2).pointresize(720,4)
# tbc
function tbc(clip last) {
ScriptClip("""
v3=last
function getpixel(clip v2, int x1, int y1) {
v2
pointresize(last.width*2,last.height*2)#double to avoid problems reading single pixel of YV12
crop(x1*2,y1*2,2,2)
}
y=0
v3
#find where >=thresh
p0=getpixel(last,719,0).averageluma
p1=getpixel(last,718,0).averageluma
p2=getpixel(last,717,0).averageluma
p3=getpixel(last,716,0).averageluma
p4=getpixel(last,715,0).averageluma
p5=getpixel(last,714,0).averageluma
p6=getpixel(last,713,0).averageluma
p7=getpixel(last,712,0).averageluma
p8=getpixel(last,711,0).averageluma
p9=getpixel(last,710,0).averageluma
p10=getpixel(last,709,0).averageluma
p11=getpixel(last,708,0).averageluma
p12=getpixel(last,707,0).averageluma
p13=getpixel(last,706,0).averageluma
p14=getpixel(last,705,0).averageluma
p15=getpixel(last,704,0).averageluma
thresh=85
offset=4
x=p0>=thresh?719:p1>=thresh?718:p2>=thresh?717:p3>=thresh?716:p4>=thresh?715:p5>=thresh?714:p6>=thresh?713:p7>=thresh?712: \
p8>=thresh?711:p9>=thresh?710:p10>=thresh?709:p11>=thresh?708:p12>=thresh?707:p13>=thresh?706:p14>=thresh?705:p15>=thresh?704:720
v3.bilinearresize(last.width,last.height,0,last.height,x+offset,last.height)#+4 moves detection point in
#subtitle(string(x),x=100)
""")
}
function jitter(clip in, int jitter) {
n=5#this works and make 5 slices of horizontally resized areas
j1=rand(jitter)#determined once per clip
j2=rand(jitter)
j3=rand(jitter)
j4=rand(jitter)
j5=rand(jitter)
w=720
h=480
h1=h/n
h1=16
slice=blankclip(width=w,height=h1,pixel_type="YV12")
c1=in.bilinearresize(720-j1*2,h1,0,0*h1,w,h1)
c1=overlay(slice,c1,opacity=1).subtitle(string(j1))
#c1=c1.bilinearresize(720,h1,0,h1,720-j1*2,h1)#test resizing back to original size
c2=in.bilinearresize(720-j2*2,h1,0,1*h1,w,h1)
c2=overlay(slice,c2,opacity=1).subtitle(string(j2))
#c2=c2.bilinearresize(720,h1,0,h1,720-j2*2,h1)
c3=in.bilinearresize(720-j3*2,h1,0,2*h1,w,h1)
c3=overlay(slice,c3,opacity=1).subtitle(string(j3))
c4=in.bilinearresize(720-j4*2,h1,0,3*h1,w,h1)
c4=overlay(slice,c4,opacity=1).subtitle(string(j4))
c5=in.bilinearresize(720-j5*2,h1,0,4*h1,w,h1)
c5=overlay(slice,c5,opacity=1).subtitle(string(j5))
stackvertical(c1,c2,c3,c4,c5)
}
function slice(clip v5, int y) {
#warning! this function is very slow. It eats a lot of memory also. Only good for experimentation.
v5
pointresize(last.width,last.height*2)
crop(0,y*2,0,2).pointresize(720,4)
tbc
y<80-1?stackvertical(last,slice(v5,y+1)):last
}


EDIT: Later version of JMac698 Tbc.avs than posted by JohnMeyer, v0.53 (from 1st post) rather than JM v0.5 [ Last edited by jmac698; 14th July 2012 at 06:16. Reason: Updated ]

#Fast line shifter Ver 0.53 by jmac698
#Lines up either or both edges of a video. Can also be used as displacement for 3d scripts.
#Requires Masktools v2a45+ (mt_lutspa mode), GRunT, GScript
#MinMax http://forum.doom9.org/showthread.php?p=1532124#post1532124
#Limitations: still no subpixel shifting
#0.53: Avoid possible "ScriptClip: Function did not return a video clip of the same colorspace as the source clip!"
#0.52: Less blurry resize
#0.51: Autothresh (uses 2 pixels of left border as a starting point, then adds a small amount to avoid noise)
#0.5: Slow, but using a completely new approach, and can resize whole lines
#0.4: Fast, can detect and line up on left or right edges
#note: sample was frame 167, http://screenshotcomparison.com/comparison/88810

#Modified to work with wide-window sample capture, which includes hsync

src=AVISource("D:\project001a\tbc2\vhs hysnc sample.avi").converttoyuy2
#crop(8,0,0,0)#Uncomment and adjust to remove extra black left border
thresh=int(findthresh(src))+3#This may not always work, try to manually set to 32 for example. Pick the lowest value which lines up picture.
ScriptClip(src, """
#Mark video edges
converttoyv12
m=mt_binarize(thresh)
#Line up video
findpos_h(m, searchwidth=22)
alignbyluma(src,last)
""")
addborders(56,0,0,0)

function findpos_h(clip m, int "searchwidth", int "x1", int "x2"){
#Searches m from left to right in the range x1 to x1+searchwidth-1 and right to left in the range width-1-x2 to x2-searchwidth-1
#for the first luma=255 pixel, then colors the output line with the offset from x
#for example m is 0 0 255 255 255 0 0 0, width=8, x1=0, x2=0, searchwidth=4 becomes 4 4 2 3 3 4 4 4, then 2 2 2 2 3 3 3 3
#Can only search for 255 pixels (as the luma output is only 8 bit)
#c and m should have the same clip properties (same size)
#searchwidth should be <=width/2
searchwidth=default(searchwidth,32)
x1=default(x1,0)
x2=default(x2,0)
rampexpr="x "+string(m.width/2)+" < x "+string(m.width-1)+" x - ?"#x w/2 < x w-1 x - ?
ramp=mt_lutspa(m, mode="absolute",expr=rampexpr)
notfound=searchwidth#Value to return if no mask on this line, should be >=searchwidth or you'll find the wrong minimum later
maskmarker=255#The luma value in the mask which indicates a detected pixel
#(if m=maskmarker return ramp else notfound), 255 means x>=255, x<searchwidth or notfound
mt_lutxy(m,ramp,yexpr="x "+string(maskmarker)+" = y "+string(notfound)+" ?")
#now make solid lines based on min luma found in each line
l=crop(0,0,-width/2,0)
r=crop(width/2,0,0,0)
l=l.minmax(0,0)
r=r.minmax(0,0)
StackHorizontal(l,r)
}

function alignbyluma(clip src, clip shift, int "mode"){
#Shift/scale each line of clip src by the x offset defined by the luma of shift
#for example if shift were all luma=8, the entire src clip would move 8 pixels to the (dir)
#This works on a pixel basis, so solid horizontal lines in shift can shift src by variable amounts per line
#It uses a simple replacement strategy, where each pixel in shift is tested and replaced by the same pixel in a shifted copy
#Currently handles only 0-15 shifts
#Magnify everything to get full color resolution
mode=default(mode, 2)
shiftuv=shift
shift=shift.pointresize(shift.width*2,shift.height*2)
shift=ytouv(shiftuv,shiftuv,shift)
src=src.pointresize(src.width*2,src.height*2)#We double here to preserve chroma rez
GScript("
for (y=0, src.height/2-1, 1) {
l=int(getpixel(shift,0,y).YPlaneMin)
r=int(getpixel(shift,shift.width/2-2,y).YPlaneMin)
getline(src, y*2)
align(l*2, r*2, 4, 4)
out=y==0?last:stackvertical(out,last)
}#for y
")#GScript
out
converttoyuy2
bilinearresize(src.width/2,src.height/2)
}

function align(clip v, int xl, int xr, int lb, int rb, int "mode") {
v#shift an image, x>0 shifts left, xl is amount to shift left, xr is amount to shift right
#mode 0 is shift left only, 1 shift right, 2 scale to shift left and right
mode=default(mode, 2)
offx=mode==0?xl:-xr
mode<2?pointresize(last.width, last.height, offx, 0, last.width, last.height):crop(xl,0,-xr,0).addborders(lb, 0, rb, 0).Spline36Resize(last.width,last.height)
}

function getpixel(clip v, int x, int y) {
#get color of a single pixel and return as a fat 2x2 yv12 pixel
v
#pointresize(last.width*2,last.height*2)
crop(x>0?x*2:0,y>0?y*2:0,-(last.width-x*2-2),-(last.height-y*2-2))
}

function getline(clip v, int y) {
v#return a line of height 2 from y to y+1
crop(0,y,0,-(last.height-y-2))
}

function findthresh(clip v){
#Find a resonable starting point for thresh by searching border
current_frame=0
v.converttoyv12
crop(0,16,-last.width+2,-16)
AverageLuma
}


EDIT: Maybe Reel.Deel has any missing plugins.
@jmac698

I believe I have most of those plugins. Do you mind if I upload them?
EDIT:
Ok here they are:

addcode02.zip (https://dl.dropbox.com/s/yaj13afjr5ttpd8/addcode02.zip)
corr03.zip (https://dl.dropbox.com/s/rjnwwz94izl4pyd/corr03.zip)
decomet04.zip (https://dl.dropbox.com/s/67iybsgmah7hnnj/decomet04.zip)
dejitter03.zip (https://dl.dropbox.com/s/1c09rcpbjwmzfd6/dejitter03.zip)
findpos02.zip (https://dl.dropbox.com/s/hpk9n9wkr2i10g6/findpos02.zip)
mandelbrot01a.zip (https://dl.dropbox.com/s/sfyiiir2ptqv6eu/mandelbrot01a.zip)
slicer02.zip (https://dl.dropbox.com/s/n2j8tdhz092m2la/slicer02.zip)
taverage 01.zip (https://dl.dropbox.com/s/9lzco9qggolza64/taverage 01.zip)
taverage 01 src.zip (https://dl.dropbox.com/s/m45ryrmub2zvgxk/taverage 01 src.zip)
tbc061.zip (https://dl.dropbox.com/s/3epcmnd5pyk7xrt/tbc061.zip)


Above updated TBC061.zip has dejitter03 and findpos02 dll's.

[FranceBB]

Thanks to you both! I'm gonna try it out and see whether it does what it's supposed to do with my source which definitely would have benefited from an hardware time base correction (but the tape is on another site 500 km away and it's not exactly the time to... uh... travel... you know...). ;)

[Reel.Deel]

EDIT: Maybe Reel.Deel has any missing plugins.

Those are all of jmac's plugins. He also lost the source code to all of them :(.

[StainlessS]

Yep, Sendspace only keeps them for 30 days after latest download. [ what a silly billy :) ]

RD, did you update this on Wiki: http://avisynth.nl/index.php/Filter_SDK/Env_SaveString

Thats a pretty awful example of env->SaveString() use, eg

Filter SDK/Env SaveString

env->SaveString is given to allow users to pass strings to AVSValue, and ensure that they are being deallocated on unload.

An Example:
fnpluginnew = new char[string_len];

strcpy(fnpluginnew, fnplugin.AsString());
strcat(fnpluginnew, " ");
strcat(fnpluginnew, name);

env->SetGlobalVar("$PluginFunctions$", AVSValue(env->SaveString(fnpluginnew, string_len)));

// Since fnpluginnew has now been saved it can safely be deleted.

delete[] fnpluginnew;


What newbie is gonna know what "$PluginFunctions$" is, and if thats not bad enough, looks like it will squash your current plugin functions list to me.

I think previous example was taken from one of my posts in dev forum, way better .
I'll find it if you want.

EDIT: Here:- https://forum.doom9.org/showthread.php?p=1633936#post1633936

Here how to return strings:-


char * strlit="AnyOldName";
int len=strlen(strlit);
char * s=new char[len+1];
if(s==NULL)
env->ThrowError("Cannot Allocate string mem");
strcpy(s,strlit); // dup
char *e=s+len; // point at nul

// make safe copy of string (freed on Avisynth closure)
AVSValue ret = env->SaveString(s,e-s); // e-s is text len only (excl nul) {SaveString uses memcpy)
// AVSValue ret = env->SaveString(s); // alternative, Avisynth uses strlen to ascertain length
// AVSValue ret = env->SaveString(s,-1); // alternative, Avisynth uses strlen to ascertain length
delete [] s; // delete our temp s buffer
return ret; // return Saved Str as AVSValue

// return strlit; // Alternative to MOST of above code char* converted to AVSValue.
// return "AnyOldName"; // Alternative to ALL of above code char* converted to AVSValue.
// String literals are read only and at constant address and so need not be saved.

above untested
EDIT: Dont use the one in red above (Its a bit naughty).


EDIT:
Avisynth and your plugin both have their own heap memory [new/malloc] and when you return a string allocated on your plugin heap,
you MUST hand over a string copy to avisynth, you must make a copy on avisynth heap (using avisynth's env->SaveString()) and give it the copy,
this gives avisynth full control over the string memory and also the responsibility to delete the memory when avisynth closes down.
After you make a string copy for avisynth, you can then delete/free your temp string memory buffer.
It is not necessary to use SaveString() on a string literal, "String literals are read only and at constant address and so need not be saved".

Set a Global var named MyGlobalString="Some GlobalVar Text" (actually does not need to use SaveString as "Some GlobalVar Text" is a String Literal).

env->SetGlobalVar("MyGlobalString", AVSValue(env->SaveString("Some GlobalVar Text"))); // Set Global [SaveString will use strlen() to establish length of string]
...
env->SetVar("MyLocalString", AVSValue(env->SaveString("Some LocalVar Text"))); // Set Local [SaveString will use strlen() to establish length of string]

[B]Also NOTE above, if MyGlobalString and MyLocalString variable name strings above were NOT string literals and located in some dynamic memory buffer, you would
similarly have to use env->SaveString() to make copies of them, and give the copies as the names to SetGlobalVar() and SetVar() functions.


Set Global variable MyPi=3.1415926 [again, below MyPiName is a pointer to "MyPi" string literal (text wrapped in double quotes) and does not need env->SaveString, is just for illustration purposes]

char *MyPiName = "MyPi";
float pi=3.1415926f;
env->SetGlobalVar(env->SaveString(MyPiName), AVSValue(pi));
...
env->SetGlobalVar(env->SaveString(MyPiName), pi); // Implicit type conversion of float pi to an AVSValue

[jmac698]

Thanks for finding my old plugins. I still don't know where a good place to post stuff is though. What site lasts 10 years these days?

I actually did work on an aspect of this recently. What I called the perfect TBC method was using a correlation to find horizontal line jitter. This used several samples so was able to be more robust to noise, besides changes in brightness and contrast. I was able to use ChatGPT to solve all the math behind it. I've learned much more since then. There's a bunch of approaches to my problem and they can be measured against something called the Rao bound. Math like this is used for many things such as photogrammatry and aligning audio from different sources, measuring echo, and other types of alignment problems.

The baked in type of jitter could probably be trained with AI now.
Good thing I turned into a data scientist :)

Meanwhile we have vhsdecode which is improving all the time.
https://www.videohelp.com/software/VHS-Decode

You need to solder two wires to your direct head feed. You don't need an expensive Doomsday decoder, you can use a cheap TV USB stick as used in SDR or Software Defined Radio.
https://www.rtl-sdr.com/buy-rtl-sdr-dvb-t-dongles/

Since VHS is an FM signal, the capture doesn't device doesn't have to be precise, it only needs to measure a frequency.

[jmac698]

Also my mailbox is not full, I don't know what's wrong. mail away

[poisondeathray]

I still don't know where a good place to post stuff is though. What site lasts 10 years these days?


Authors, contributors of many code projects (not just avisynth related) use Github . It's owned by Microsoft now, but unlikely to "disappear"

[Reel.Deel]

Thanks for finding my old plugins. I still don't know where a good place to post stuff is though. What site lasts 10 years these days?


Too bad you were hesitant on releasing the source code back then :p. But having something instead of nothing is better.

If you don't want to host on GitHub as poisondeathray suggested, VideoHelp offers 512mb of storage. They have proven to be reliable and you can easily archive it also. I've done exactly that with a handful of plugins that are on the wiki.