Your video restoration scripts

[steptoe]

Is it possible that users could post their scripts or commands they use for restoring video to her or mail them to me

Mainly I'm looking for scripts or commands that I can use with DVD-RB to help me out with my restoration of my poor quality VHS transfers to DVD

These cover old movies and series and also anime stuff I've had for quite a while and want to transfer to DVD

I have quite a few black and white movies and series that I want to try to bring some quality back to, using filters to remove the usual scratches and filickering white spots plus try to bring back some sharpness

Speed isn't a requirement just the final output quality


Many thanks for any help for somebody who is struggling to achive this

[Backwoods]

For anime I use this script:

LoadPlugin("C:\Program Files\AviSynth 2.5\yea\Decomb521.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\yea\SSIQ.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\yea\Cnr2.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\yea\SSETools.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\yea\RemoveGrain.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\yea\Repair.dll")
Import("LRemoveDust.avs")
LoadPlugin("C:\Program Files\AviSynth 2.5\yea\aWarpSharp.dll")
Import("C:\Program Files\AviSynth 2.5\yea\aWarpSharpDering_1.0.avsi")
LoadPlugin("C:\Program Files\AviSynth 2.5\yea\Convolution3DYV12.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\yea\MaskTools.dll")
Import("C:\Program Files\AviSynth 2.5\yea\LimitedSharpen.avs")
Import("FastLineDarken.avs")

AVISource("Ultra.avi")
Telecide(1)
Decimate(5)
ConvertToYV12()
Cnr2("xxx",4,5,255)
LRemoveDust(4,2)
RemoveGrain(1,2,2)
aWarpSharp(depth=16,cm=1)
aWarpSharpDering(depth=20,diffThresh=16)
FastLineDarken(strength=120,thinning=0)
LimitedSharpen(ss_x=2,ss_y=2,Dest_x=720,Dest_y=480,Strength=128)

function LRemoveDust(clip input, int _mode, int "limit")
{
limit=default(limit,4)
repmode = 2
clensed = Clense(input)
rep=Repair(clensed, input, mode=repmode)
rg = RemoveGrain(rep, mode=_mode)
rd=TemporalRepair(rg, rep)
return LimitChange(rd, input, limit, limitU=255)
}

######################
# FastLineDarken 1.3 #
######################
#
# Written by Vectrangle, last update 12 Sept 04
#
# * requires masktools 1.5.1 -- http://jourdan.madism.org/~manao/
# * requires yv12 input
#
# Usage is FastLineDarken(strength, luma_cap, threshold, thinning),
# named parameters are supported eg FastLineDarken(thinning=0)
#
# Note that you must import this avs into your script using import("...\FastLineDarken 1.3.avs")
#
# Parameters are:
# strength (integer) - Line darkening amount, 0-256. Default 48. Represents the _maximum_ amount
# that the luma will be reduced by, weaker lines will be reduced by
# proportionately less.
# luma_cap (integer) - value from 0 (black) to 255 (white), used to stop the darkening
# determination from being 'blinded' by bright pixels, and to stop grey
# lines on white backgrounds being darkened. Any pixels brighter than
# luma_cap are treated as only being as bright as luma_cap. Lowering
# luma_cap tends to reduce line darkening. 255 disables capping. Default 191.
# threshold (integer) - any pixels that were going to be darkened by an amount less than
# threshold will not be touched. setting this to 0 will disable it, setting
# it to 4 (default) is recommended, since often a lot of random pixels are
# marked for very slight darkening and a threshold of about 4 should fix
# them. Note if you set threshold too high, some lines will not be darkened
# thinning (integer) - optional line thinning amount, 0-256. Setting this to 0 will disable it,
# which is gives a _big_ speed increase. Note that thinning the lines will
# inherently darken the remaining pixels in each line a little. Default 24.
#
# Changelog:
# 1.3 - added ability to thin lines, now runs much slower unless thinning=0. Changed the defaults (again)
# 1.2 - huge speed increase using yv12lutxy =)
# - weird darkening issues gone (they were caused by yv12layer)
# - show option no longer available due to optimizations. Use subtract() instead
# 1.1 - added luma_cap option
# 1.0 - initial release
#


function FastLineDarken( clip c, int "strength", int "luma_cap", int "threshold", int "thinning") {
str = string(default(strength, 48) /128.)
lum = string(default(luma_cap, 191))
thr = string(default(threshold, 4))
thinning = default(thinning,24)
thn = string(thinning /16.)
exin=c.expand().inpand()
diff = yv12lutxy(c,exin,yexpr="y "+lum+" < y "+lum+" ? x "+thr+" + > x y "\
+lum+" < y "+lum+" ? - 0 ? 127 +",uexpr="x",vexpr="x",u=2, v=2)
linemask = yv12lut(diff.inpand(),"x 127 - "+thn+" * 255 +")\
.yv12convolution("1 1 1","1 1 1",y=3,u=0,v=0)
thick = yv12lutxy(c, exin, yexpr="y "+lum+" < y "+lum+" ? x "+thr+" + > x y "\
+lum+" < y "+lum+" ? - 0 ? "+str+" * x +",uexpr="x",vexpr="x",u=2, v=2)
thin = yv12lutxy(c.expand(),diff,yexpr="x y 127 - "+str+" 1 + * +",u=2, v=2)
return (thinning == 0) ? thick : maskedmerge(thin,thick,linemask,y=3,u=2,v=2)
}

Taken from this thread: http://forum.doom9.org/showthread.php?t=91460

[mg262]

I'm a lot less good at this than some on this forum, but for what it's worth:

I write a different script for every source. I tend to try and strip away any artefacts (e.g. spots, speckles, chroma weirdness on alternate lines, Gibbs phenomenon referred to below) in the capture 'layer by layer' to get back to a noisy version of the original source and then to denoise it. I ran a test of about 25 different denoisers a year ago and I found that while quite a few performed decently on static scenes, Dust was in a class of its own on moving scenes. OTOH, I haven't had the time to test everything again using MVTools interleaving for more than compensated denoising.

The single thing I found most useful when when building scripts is this:function CompareVersions(clip A, clip B, bool "copy")
{
A=A.ConvertToYUY2()
B=B.ConvertToYUY2()

copy = default(copy, false)

halfwidth = A.Width()/2

left = A.crop(0, 0, halfwidth,0)
right = copy ? B.crop(0, 0, halfwidth,0) : B.crop(halfwidth, 0, 0, 0)

return stackHorizontal(left, right)
}
function CV(clip A, clip B, bool "copy"){ CompareVersions(A, B, copy)} So the point of that is to apply a filter to the left-hand half of the screen only, so you can play back video with some effect (e.g. Dust) on one half only... or you can try different filters on different halves, or different parameters on different halves. This was the only way I ever managed to compare filters successfully (or to decide on parameters).

This is the script I used to restore "Tom's Midnight Garden", a VG quality VHS tape but with some 'echoes' (Gibbs phenomenon; can't remember the term used to describe it in video).#Audio c-10dB, declick, denoise, normalise, resample.

AviSource("e:\garden.avi")

AssumeTFF()
SeparateFields()

#Correct Errors
mergechroma(blur(0,1))
chromashift(c=-2, l=-3)
mergeluma( converttorgb32().VD_Exorcist(6, -20).converttoyuy2() )

#Remove Noise
PixieDust(3)

Crop(6,1,-2,-7)

Weave()

TomsMoComp(-1,15,0)

BicubicResize(720,576)

Levels(0,1,205,0,255, coring = false)

Trim(696, 213545)

assumefps(25)
KillAudio()I might post an animation script when I can dig it out.

[Pookie]

mg262 - Would you be kind enough to post an example of how to utilize your compare versions function in an actual script ? Thanks in Advance.

[twonky]

Any scripts (or otherwise insights) on how to handle cleanup/archiving on old color and B&W live-action television programs? The ones I've been working on vary from 35mm originals to 16mm (which has grain/dirt/scratches that are 250% larger than what 35mm has), and some are broadcast versions (i.e. "real telecine") and some are homebrew transfers (with inconsistant 3:2 pulldown irregularities)...mostly for output to DVD, but am experimenting with SVCD as well. Masters are mostly VHS with some from 3/4" tapes.

My caps are all pro full D-1 (interlaced, BFF 720X486 in 4:2:2 RGB colorspace) lossless MJPEGB quicktime files, and yes, these are HUGE files. It helps to have a full Media100 broadcast quality capture/edit system in your home...no dropped frames ever, and always high quality caps and dependable.

The files load just fine in avisynth DirectShowSource BTW.

My problem is that I've only been experimenting on these for about a month and am just beginning to get comfortable with the scripting part. I've used Fulci's Convolution3D script to decent effect, but that's just a start, as there's a lot of dirt and schmutz on most of these shows. Haven't really gotten into IVTC seriously, due to the irregular pulldown patterns.

I really like Avisynth and it's getting a little less daunting every day, but I could sure use some clues here on filterchains to make these ancient taped shows look good. TIA ;)

[Chainmax]

Backwoods: a while ago, I slightly updated the filterchain, the main change being the replacement of aWarpSharpDering with a more effective, less problematic deringer (HQDering). The new filterchain is:

*ConvertToYV12(interlaced=true)
*DeDot()
**TFM(mode=3,PP=7,chroma=true,mi=50)
**TDecimate(mode=1)
***TDeint(type=1) or (type=2)
Crop(your_settings,align=true)
****SSIQ(diameter=11,strength=50,interlaced=false)
ConvertToYV12() if needed
*****Cnr2("xxx",4,5,255)
LRemoveDust_YV12(17,4) or (17,2)
aWarpSharp(depth=16,cm=1)
BicubicResize(H_Res,V_Res,0,0.5) or Lanczos4Resize(H_Res,V_Res)
RemoveGrain(mode=5) or mode=1
HQDering()
LimitedSharpen(ss_x=2,ss_y=2)

* = If the source has dotcrawl or strange moving noise
** = If IVTCing is needed
*** = If Deinterlacing is needed
**** = Rainbow removal
***** = extra rainbow removal, color bands removal

And here are the function versions I use:

HQDering: (needs the latest Deen (www.avisynth.org/warpenterprises))
######
##
## HQDering v0.1 by mf
##
## Applies derining by using a smart smoother near edges (where ringing occurs) only.
##
## Usage: Import("HQDering-v0.1.avs")
## HQDering()
##
####

function HQDering(clip input, int "strength", int "overall", string "smoother", string "params") {

strength = Default(strength, 255) # strength, 0-255
overall = Default(overall, 0) # overall smoothing, 0-255
smoother = Default(smoother, "Deen") # filter that smooths
#params = default defined below # filter parameters - use Chr(34) for quotes

defaultnull = Default(params, "")
defaultdeen = Default(params, Chr(34)+"a3d"+Chr(34)+", 4, 15, 15, 20")

params = (smoother=="Deen") ? defaultdeen : defaultnull

try {
smoothedwparams = Eval(smoother + "(input, " + params + ")")
}
catch(err_msg) {
smoothedwoparams = Eval(smoother + "(input)")
}

smoothed = (params=="") ? smoothedwoparams : smoothedwparams

input.EdgeMask(3, 255, 255, 255, "sobel", Y=3, V=1, U=1)
normalmask = last

normalmask.Levels(0, 3.3, 90, 0, 255).Blur(1.0)
amplifiedmask = last

normalmask.Inflate().Inflate().Inflate().Levels(0, 3.3, 90, 0, 255).Blur(1.0).Inflate().Inflate().Inflate().Levels(0, 3.3, 255, 0, 255).Inflate().Inflate()
thickmask = last

Overlay(amplifiedmask.Invert(), thickmask, mode="multiply").greyscale

Levels(60, 3.0, 140, overall, strength)

ringingmask = last

MaskedMerge(input, smoothed, ringingmask)

}


LimitedSharpen: (needs the latest MaskTools (http://manao4.free.fr/))
# LimitedSharpen()
#
# A multi-purpose sharpener by Didée
#

function LimitedSharpen( clip clp,
\ float "ss_x", float "ss_y",
\ int "dest_x", int "dest_y",
\ int "Smode" , int "strength", int "radius",
\ int "Lmode", bool "wide", int "overshoot",
\ bool "soft", int "edgemode", bool "special",
\ int "exborder" )
{
ox = clp.width
oy = clp.height
ss_x = default( ss_x, 1.5 )
ss_y = default( ss_y, 1.5 )
dest_x = default( dest_x, ox )
dest_y = default( dest_y, oy )
Smode = default( Smode, 3 )
strength = Smode==1
\ ? default( strength, 160 )
\ : default( strength, 100 )
strength = Smode==2&&strength>100 ? 100 : strength
radius = default( radius, 2 )
Lmode = default( Lmode, 1 )
wide = default( wide, false )
overshoot= default( overshoot, 1)
overshoot= overshoot<0 ? 0 : overshoot
soft = default( soft, false )
edgemode = default( edgemode, 0 )
special = default( special, false )
exborder = default( exborder, 0)
#radius = round( radius*(ss_x+ss_y)/2) # If it's you, Mug Funky - feel free to activate it again
xxs=round(ox*ss_x/8)*8
yys=round(oy*ss_y/8)*8
smx=exborder==0?dest_x:round(dest_x/Exborder/4)*4
smy=exborder==0?dest_y:round(dest_y/Exborder/4)*4

clp.isYV12() ? clp : clp.converttoyv12()

ss_x != 1.0 || ss_y != 1.0 ? last.lanczos4resize(xxs,yys) : last
tmp = last

edge = logic( tmp.DEdgeMask(0,255,0,255,"5 10 5 0 0 0 -5 -10 -5", divisor=2)
\ ,tmp.DEdgeMask(0,255,0,255,"5 0 -5 10 0 -10 5 0 -5", divisor=2)
\ ,"max").levels(0,0.86,128,0,255,false)

bright_limit = (soft == true) ? tmp.blur(1.0) : tmp
dark_limit1 = bright_limit.inpand()
bright_limit1 = bright_limit.expand()
dark_limit = (wide==false) ? dark_limit1 : dark_limit1 .inflate.deflate.inpand()
bright_limit = (wide==false) ? bright_limit1 : bright_limit1.deflate.inflate.expand()
minmaxavg = special==false
\ ? yv12lutxy(dark_limit1,bright_limit1,yexpr="x y + 2 /")
\ : maskedmerge(dark_limit,bright_limit,tmp,Y=3,U=-128,V=-128)

Str=string(float(strength)/100.0)
normsharp = Smode==1 ? unsharpmask(strength,radius,0)
\ : Smode==2 ? sharpen(float(strength)/100.0)
\ : yv12lutxy(tmp,minmaxavg,yexpr="x x y - "+Str+" * +")

OS = string(overshoot)
Lmode == 1 ? yv12lutxy( bright_limit, normsharp, yexpr="y x "+OS+" + < y x "+OS+" + ?")
\ : yv12lutxy( bright_limit, normsharp, yexpr="y x "+OS+" + < y x y x - "+OS+" - 1 2 / ^ + "+OS+" + ?")
Lmode == 1 ? yv12lutxy( dark_limit, last, yexpr="y x "+OS+" - > y x "+OS+" - ?")
\ : yv12lutxy( dark_limit, last, yexpr="y x "+OS+" - > y x x y - "+OS+" - 1 2 / ^ - "+OS+" - ?")

edgemode==0 ? NOP
\ : edgemode==1 ? MaskedMerge(tmp,last,edge.inflate.inflate.blur(1.0),Y=3,U=1,V=1)
\ : MaskedMerge(last,tmp,edge.inflate.inflate.blur(1.0),Y=3,U=1,V=1)

(ss_x != 1.0 || ss_y != 1.0)
\ || (dest_x != ox || dest_y != oy) ? lanczos4resize(dest_x,dest_y) : last

ex=blankclip(last,width=smx,height=smy,color=$FFFFFF).addborders(2,2,2,2).coloryuv(levels="TV->PC")
\.blur(1.3).inpand().blur(1.3).bicubicresize(dest_x,dest_y,1.0,.0)
tmp=clp.lanczos4resize(dest_x,dest_y)

clp.isYV12() ? ( exborder==0 ? tmp.mergeluma(last)
\ : maskedmerge(tmp,last,ex,Y=3,U=1,V=1) )
\ : ( exborder==0 ? tmp.mergeluma(last.converttoyuy2())
\ : tmp.mergeluma( maskedmerge(tmp.converttoyv12(),last,ex,Y=3,U=1,V=1)
\ .converttoyuy2()) )

return last
}
#

LRemoveDust: (needs Repair, SSETools and RemoveGrain from the RemoveGrain package (http://www.removegrain.de.tf/))
function LRemoveDust_YUY2(clip input, int clmode, int "limit")
{
clmode=default(clmode,17)
limit=default(limit,2)
repmode = 2
clensed = Clense(input, grey=true)
rep = Repair(clensed, input, mode=repmode, modeU=-1)
rg = RemoveGrain(rep, mode=_mode, modeU=-1)
return LimitChange(rg, input, limit, limitU=255)
}

function LRemoveDust_YV12(clip input, int clmode, int "limit")
{
clmode=default(clmode,17)
limit=default(limit,2)
repmode = 2
clensed = Clense(input)
rep = Repair(clensed, input, mode=repmode)
rg = RemoveGrain(rep, mode=clmode)
return LimitChange(rg, input, limit)
}

LRemoveDust usage: YUY2 sources:

xxxSource()
org=last
ConverttoYV12()
LRemoveDust_YUY2(17,2)
ConverttoYUY2()
MergeChroma(org)



YV12 sources:

xxxSource()
LRemoveDust_YV12(17,2)

This new version of the script and its usage were made by Boulder.


SSIQ can be found here (http://yatta.mellbin.org/misc/), the latest CNR2 can be found here (http://www.missouri.edu/~kes25c/) and DeDot_YV12 can be downloaded from here (http://nullinfo.s21.xrea.com/#DeDot_YV12).

[mg262]

I wrote a reply to @pookie's request last week, but it ended up being *very* long (although not as I planned it complete!), so I sent it to him via PM. Here's a short version:

If you take a clip and apply

CompareVersions(info(), greyscale())

then you will see very quickly how the clip functions. In practice, you want to write things like this:

CompareVersions(FaeryDust(), PixieDust()) #to choose between filters

CompareVersions(PixieDust(3), PixieDust(5)) #to choose parameters

CompareVersions(last, PixieDust()) #to decide whether to use a filter

Obviously you don't need to apply this immediately after AviSource... I use it to build up a script line by line to see the effect of each line.

If you're trying to compare filter sequences, either put them in script functions or save the results of both using "original=last" "a=last", "b=last" and then apply CompareVersions(a,b) .

Last and most important thing: save the result of the function and look at it in a media player. It will be much much more effective than simply looking at a screen shot. when choosing parameters or applying subtle filters, look for the vertical line down the middle of the screen -- once you can see it, it becomes much easier to notice subtle effects.

Edit: also cf http://forum.doom9.org/showthread.php?t=40675
_______________________________________
@Chainmax:

I assume that's the version of awarpsharp used in the mfToon thread? By the way, your HQDering v0.1 by mf seems to differ from the one on http://mf.creations.nl/avs/functions/ ... did you just change it to work with the most recent MaskTools? Incidentally, thank you for rooting out all those links... it's very very helpful.

[Pookie]

mg262 - I hope you don't mind if I post your reply to me as additional information to this thread. And thanks again.
-------------------------------------------------------------------------
First thing to note is that all the scripts below should be saved as AVIs and viewed in a media player -- it makes a huge difference over just scrolling through frames in VirtualDub.

I would typically add something like Trim(2000, 2500) to the end of the script before saving it, to save time. Switching on the repeat option in your media player can also be useful.

1) First a simple example of what it actually does:

If

AviSource("replace_this_with_path_and_name_of_your_file.avi")

produces a frame like this:


Then

AviSource("replace_this_with_path_and_name_of_your_file.avi")
CompareVersions( last, greyscale(last) )

produces this:


Of course, I could have got that output with any of the following scripts:

AviSource("replace_this_with_path_and_name_of_your_file.avi")
CompareVersions( last, greyscale() )

or

AviSource("replace_this_with_path_and_name_of_your_file.avi")
last.CompareVersions( greyscale() )

or

AviSource("replace_this_with_path_and_name_of_your_file.avi")
CompareVersions( greyscale() )

or even

AviSource("replace_this_with_path_and_name_of_your_file.avi")
CompareVersions( greyscale )

2) Sometimes rather than comparing with/without a filter, you want to compare the effect of two different filters:

AviSource("video file.avi")
CompareVersions( info(last), greyscale(last) )

or more simply

AviSource("video file.avi")
CompareVersions( info, greyscale )

I'm going to type cv from now on because that's what I always use.

3) A couple more syntactic alternatives:

It can get messy to match brackets in something like

AviSource("video file.avi")
cv( TemporalSoften(4,4,8,15,2).mergeluma(spatialsoften(4,4,8)) ) #not meant to be a sensible script!

so sometimes I just write

AviSource("video file.avi")
TemporalSoften(4,4,8,15,2).mergeluma(spatialsoften(4,4,8)).cv(last)

This will flip the sides -- in the first case the original appeared on the LHS, but using .cv(last) will make it appear on the RHS. ".cv(last)" is mainly convenient because you can pasted it onto the end of a line.

A better way to write that last script would be like this:

AviSource("video file.avi")
o = last

o
TemporalSoften(4,4,8,15,2)
mergeluma(spatialsoften(4,4,8))

cv(o, last)

And 'saving the state' by writing e.g. a = last, b = last is even more useful when you're comparing the effects of two different complex processes. Here's a real example:

Loadplugin("D:\cp\FindPan\Release\FindPan.dll")

avisource("basic.avs")
Trim(4415, 4416)
k=last
swapfields ()
AssumeTFF()
separatefields()
#info()

o = last
metric = CapAbsoluteMetric(o, 95)
white = blankclip(o, color = $ffffff)


FindFrameShift(white,o, metric)
ScaleMotion(last, -1.0)
ApplyBilinearTranslate(o, last, blankclip(o))

#WriteMotion (last)

#FindIntegerPan (white, o, metric)
#motion = RefinePan(white, o, metric, last)
#WriteMotion (motion)

AssumebFF()
weave

cv(k, true)

[Chainmax]

@Chainmax:

I assume that's the version of awarpsharp used in the mfToon thread? By the way, your HQDering v0.1 by mf seems to differ from the one on http://mf.creations.nl/avs/functions/ ... did you just change it to work with the most recent MaskTools? Incidentally, thank you for rooting out all those links... it's very very helpful.

The aWarpSharp I use is what I assume is the latest version, the one from MarcFD's site (http://ziquash.chez.tiscali.fr/). I wasn't the one that came up with the MaskTools fix, it was suggested by Didée to SoulHunter in mf's forum.
About the links, I know how difficult it can be sometimes to track down old filters/functions or how to figure out what DLLs are needed, so you're very welcome :).


P.S: use aWarpSharp with caution, ChronoCross told me that "if you
look at the windows they look totally messed up as if they are moving
from top to bottom. this is a warpsharpening effect". The effect should be reduce by using lower depth settings.