The power of Avisynth: restoring old 8mm films.

[johnnytaco]

Hi Everyone, Ive been lurking for a while as I play around with my workflow for my new wolverine hawkeye - scanning super8 frames as 2560x1920 TIFF then pumping AVI through Avisynth for stabilization, dust, scratches etc.. My days with Avisynth started with DVD and ivtc, then converting interlaced DV, then inverse pulldown from my Canon HV20 to get 24p from 30i. Now Im converting my dad's super 8 dating back to the 60s. So far so good, Ive been playing with both Fred and John version of script but have a question that might save me hours of reading and looking. Is there a recent multi-threaded version of Fred's script. More to the point, can anyone tell me which plugins support MT and which don't? Im using the 32 bit versions it for everything except interpolation and color correction. I pump TIFFs back through Davinci for that. That said, any pointers on how I can speed up the script that seems to be limited to 1 core by adding MT statements for specific plugins? This will save me time from investigating each plugin separately. Im on latest Avisynth package and the scripts are working fine so far, just slow - and Id like to use the 16 thread potential of my i7 CPU. Thanks!

[johnmeyer]

My version supports MT. It was one of the main reasons I developed my branch of Fred's script: speed.

BTW 2560x1920 is way more resolution than you need for Super 8. Try scanning and/or saving at 1/4 that resolution (1280x960) and see if you spot any difference at all. Scanning and using more resolution than is in the media just slows down your workflow, as well as eating up more disk space than you need to. I find that SD resolution is more than enough and therefore I suspect that 1280x960 will be 100% the same as 2560x1920.

[johnnytaco]

I want to test my rig to the full potential (modified wolverine hawkeye with 12MP camera) to see if I can tell the different on my 65" 4k OLED. I have used your 2017 script and like the results. Curious that I dont see the SetFilterMTMode statements, but my i7 10700 did hit around 80-90% with threads=4. I liked the output from Fred's script a little bit better, but not sure what the differences are in the settings - it looks a little softer and less grainy. I use option4 mainly for stabilizing, cropping and cleanup then do color correction, interpolation and rendering to 1080p 24p MP4 in Resolve.

[StainlessS]

Wiki :- http://avisynth.nl/index.php/AviSynth+#Help_filling_MT_modes

Help filling MT modes

The following script contains MT modes for various plugins, save it as mtmodes.avsi and place in your auto-load folder. The script is a work-in-progess, there's still lots of plugins that need to be tested and validated. When the script is finalized, the only thing the user will have to write in his script is the Prefetch call, all SetFilterMtMode calls will be hidden in a single .avsi script.

You can find the latest revision here: AviSynth+ MT modes, if you like to contribute please do so here. Report any issues here.

Note for filter writers: filters can report their mt modes in their SetCacheHint, in this case the manual setting is not necessary.
BEST GET IT FROM ABOVE WIKI LINK as might change, below as it is at time of this post.

AviSynth+ MT modes:- https://publishwith.me/ep/pad/view/ro.rDkwcdWn4k9/latest

I have in my plugins as "SetMTMode.avsi", {many others do too I think} but others as MtModes.avsi.

Header of {at time of posting} file,

###########################################################################
# DO NOT OPEN THIS DOCUMENT IN GOOGLE TRANSLATE OR BAD THINGS WILL HAPPEN #
# Reverted back to last English version 898 and cleaned up text. #
# Please, let's stick to English text to document the script. #
###########################################################################
# Latest version of this script can always be found here: http://avisynth.nl/index.php/AviSynth+#Help_filling_MT_modes
###########################################################################
# General information:
# 1. Do not include core filters in the list.
# 2. Specify the lowest mode that produces correct output with any settings.
# For example, if something works with MT_MULTI_INSTANCE on some parameters and MT_SERIALIZED with others, MT_SERIALIZED should be used.
# You can, of course, specify this in comments so people who do want to get MAXIMUM PERFORMANCE can modify the script themselves.
# 3. Do not specify MT modes for runtime filters. Behavior of runtime filtering with mt is not very well known yet.
# 4. In case it isn't obvious, MT_NICE_FILTER - mt mode 1, MT_MULTI_INSTANCE - mt mode 2, MT_SERIALIZED - mt mode 3. Variables are defined by avs+ core.
# 5. Starting from AviSynth+ r2069, AviSynth+ will now automatically recognize source filters. If it sees a source filter which has no MT-mode specified at all,
# it will automatically use mode 3 instead of the default MT mode. This is great for source filters as it rarely hurts performance and they often need mode 3 anyway,
# and it will help yet unclassified source filters perform correctly without any user action. One step closer to the "it just works" target.
##################################################################################
# Other Lists:
# https://gist.github.com/tp7/8899021
# https://github.com/realfinder/UniversalPluginsFolders/blob/master/plugins64%2B/MtModes.avsi
##################################################################################

[johnnytaco]

Fred, Ive seen different metrics that line up with your view that 720p is sufficient relative to lines per mm for Super 8, but here is an interesting article that makes me feel better about capturing at 2k. https://forums.kinograph.cc/t/8mm-super-8-16mm-scanning-resolution/2297

[johnmeyer]

As StainlessS says, the SetMTMode is usually done inside of a single file that your script references. That is why you don't see it explicitly in my scripts.

As for resolution, that link you provided has some really great discussion, but even though I have no problems with what was posted there, you ALWAYS have to do your own tests. Do those tests with your own film and your own transfer setup.

Even 16mm film can be captured at less than full HD with very little degradation. I had a chance to test this when some 1934 16mm film of Nazi rallies in Dresden Germany that I had transferred was licensed to the Smithsonian Channel. They wanted to have the transfer re-done by a "Hollywood" transfer house, using a professional Spirit transfer system. I had the transfer house send the transfer to me before I sent it to the Smithsonian Channel. As I expected, their transfer, which was delivered in 1920x1080 ProRes, did a little better job on shadows than my projector/camcorder system (the original was very contrasty German film stock), but the sharpness and increase in detail, while evident, was not strikingly better. It was a matter of 10-20% better (I'm trying to put my evaluation into numbers), not twice as good, or four times as good, which is how much more resolution 1920x1080 is compared to my 720x480 (non-square pixel) transfers.

It is from this test that I come to the conclusion that Super 8 film, which has only about 1/3 the picture area of 16mm, will not benefit from scanning/capturing at HD, much less 4K resolution.

Do your own tests, and then make your decision.

[johnnytaco]

Import("plugins/SetMTMode.avsi") makes it much simpler to deal with SetMTMode.

I understand where youre coming from on the resolution bit, and largely agree I have a reel that was scanned by Cinepost at 480p with their wetgate system years back. Im going to compare that to a few of my own versions.

Has anyone successfully used ImageSource to pump TIFFs directly into Avisynth? Im seeing if I can skip the step using Virtualdub2 making an AVI from RGB32 TIFFs.

I tried this in the JM script but I get alot of dropped (black) frames in the AVI I save from virtualdub2 regardless of settings.

source1=ImageSource("H:\veta eng\Image%04d.tiff", 1, 3589, fps=18, use_DevIL = true).converttoYV12().FlipVertical()

[poisondeathray]

Has anyone successfully used ImageSource to pump TIFFs directly into Avisynth? Im seeing if I can skip the step using Virtualdub2 making an AVI from RGB32 TIFFs.

I tried this in the JM script but I get alot of dropped (black) frames in the AVI I save from virtualdub2 regardless of settings.

source1=ImageSource("H:\veta eng\Image%04d.tiff", 1, 3589, fps=18, use_DevIL = true).converttoYV12().FlipVertical()


ImageSource works ok

Do you have gaps in your numbering ?

Black frame are inserted when there are gaps

eg.
Image0001.tiff, Image0002.tiff, Image0004.tiff... there would be a black frame inserted where Image0003.tiff should have been

Do you see black frames when you preview in vdub with the avs script loaded ?

What format are you saving it as (what video compression), and how are you viewing it (what software) ? If you re-import the export into vdub2 , are there any black frames ?

[StainlessS]

@johnnytaco
for future projects suggest use 6 digit index on image files, just more flexible and avoids probs if you have more than 9999 images.
ie,

"H:\veta eng\Image%04d.tiff"


"H:\veta eng\Image%06d.tiff"

[johnnytaco]

ImageSource works ok

Do you have gaps in your numbering ?

Black frame are inserted when there are gaps

eg.
Image0001.tiff, Image0002.tiff, Image0004.tiff... there would be a black frame inserted where Image0003.tiff should have been

Do you see black frames when you preview in vdub with the avs script loaded ?

What format are you saving it as (what video compression), and how are you viewing it (what software) ? If you re-import the export into vdub2 , are there any black frames ?

I dont see any gaps in the sequence. These are RGB32 2560x1920 TIFFs from IC Capture. The black frames show up both in the preview and in the final AVI. I am not compressing at the moment. I use virtualdub2 to normally make an AVI and then run a script (no missing frames this way - works fine). Nothing changes except trying to skip the AVI creation from TIFFs before I pump through Virtualdub/Avisynth script. Im on Win10x64 with an i7 10700 and 64GB RAM, I like using Media Player Classic for checking avisynth outputs, but same black frames show up in VLC when viewing the AVI.

I did some troubleshooting and its not an issue with ImageSource. The standalone function works well. As soon as I change result="stab" (to isolate the first function in JM's script after source) the black frames show up. I suspect DePanEstimate or DePanStabilize might be looking back/ahead at frames and results in the black frames showing up.

[poisondeathray]

I did some troubleshooting and its not an issue with ImageSource. The standalone function works well. As soon as I change result="stab" (to isolate the first function in JM's script after source) the black frames show up. I suspect DePanEstimate or DePanStabilize might be looking back/ahead at frames and results in the black frames showing up.

What version of JM script, can you post a link or post the script ?

Depan/DepanStablilize works ok for me. What version of Depan are you using ? It's supposed to look at other frames, otherwise it couldn't establish the relationship between adjacent frames and stabilize

[videoFred]

BTW 2560x1920 is way more resolution than you need for Super 8. Try scanning and/or saving at 1/4 that resolution (1280x960) and see if you spot any difference at all. Scanning and using more resolution than is in the media just slows down your workflow, as well as eating up more disk space than you need to. I find that SD resolution is more than enough and therefore I suspect that 1280x960 will be 100% the same as 2560x1920.

I agree! It's not the size that counts, but the quality from the scanning device.

Fred.

[videoFred]

Depan/DepanStablilize works ok for me.

You might want to try this: much better stabilisation and scene detection can be set with "error"
SetFilterMTMode("DEFAULT_MT_MODE", 2)

clip= AVISource("Z:\VDP\VdP_Sp2.avi").converttoYV12()

vectors= clip.MSuper().MAnalyse(isb=false)
mdata= MDepan(clip,vectors,rot=false, zoom=false, error= 65)

stab= clip.DePanStabilize(data=mdata,dxmax=40,dymax=40,zoommax=0, rotmax=0,method=0, mirror=0, info=true)

stab

Prefetch(4)

SetFilterMTMode and Prefetch is for use with Avisynth+ of cource.

Forgot to mention:
Requires depan.dll vs. 2.13.1.3 or higher modified versions by Printerf
mvtools2.dll vs. 2.7.23.0 or higher
https://github.com/pinterf/mvtools/releases

Fred.

[BabaG]

getting the following error on trying to load a proreshq clip:

Script error: can't have a named (quoted) parameter followed by an ordinary parameter.

the line it highlights is this function line:
function RemoveDirt(clip input, int "limit", bool _grey)

i'm guessing we're referring to the "limit" parameter. how do i get around this? is it as simple as removing the quotes? will try that while waiting for response.

thanks,
babag

[VoodooFX]

Try:

function RemoveDirt(clip input, int "limit", bool "_grey")

[BabaG]

thanks, voodoofx. that got me on to the next error.

now getting:

DePanEstimate: Can not load fftw3.dll or libfftw3f-3.dll!

i do have fft3dfilter.dll in my plugins64+ folder. maybe that doesn't cover it?

thanks again,
BabaG

[VoodooFX]

Can not load fftw3.dll or libfftw3f-3.dll!

FFTW 3.3.5 (http://www.fftw.org/install/windows.html) (fftw-3.3.5-dll32.zip or fftw-3.3.5-dll64.zip)

*** 32-bit libfftw3f-3.dll needs to be in the search path (C:\Windows\SysWOW64 64-bit OS or C:\windows\system32 32-bit OS)
*** 64-bit libfftw3f-3.dll needs to be in the search path (C:\windows\system32 64-bit OS)

[BabaG]

hmm. i've got the lib files in the proper places but getting crashes in avspmod on trying to preview the image. tried copy/renaming the lib files to fftw3.dll as per posts here and elsewhere and placed them in every conceivable combination of location. no luck. also tried simply copying the fft3.dll that VideoFred includes with his download to, again, every conceivable location. same thing. crash on preview.

out of ideas.
babag

[Arx1meD]

BabaG, try LoadDLL (http://avisynth.nl/index.php/LoadDLL). Perhaps you first need to delete fftw3.dll from C:\Windows\...

[StainlessS]

posted by BabaG:
tried copy/renaming the lib files to fftw3.dll as per posts here and elsewhere and placed them in every conceivable combination of location.

You need the zip for the bittage of avisynth that you are using [32 bit Avisynth, then 32 bit zip, else 64 bit zip].

Make sure that you use the file with EXACT source name libfftw3f-3.dll
and make a copy named as fftw3.dll, and copy both of these files to ONLY ONE location as below,

32bit_OS_32bit_Avisynth: C:\Windows\system32
64bit_OS_64bit_Avisynth: C:\Windows\system32
64bit_OS_32bit_Avisynth: C:\Windows\SysWOW64

as outlined by VoodooFX, and delete any others that you mistakenly copied to the wrong directory [ie, "every conceivable combination of location"].

Some plugins use dll with filename libfftw3f-3.dll, and some with filename fftw3.dll, and some plugins may use either name.

[BabaG]

thanks all.

avisynth version shows:AviSynth+ 3.7.3 (r4003, 3.7, x86_64)

placed libfftw3f-3.dll and fftw3.dll in:

C:\Windows\system32

neither in SysWOW64. system is 64 bit.

same crash. other scripts load and run fine.

fwiw, my process has been to try the files in various locations and, after trying in that location, to remove them before trying in the next location. i do this to avoid having duplicates or wrong versions cluttered around the system.

thanks again,
babag

[poisondeathray]

thanks all.

avisynth version shows:AviSynth+ 3.7.3 (r4003, 3.7, x86_64)

placed libfftw3f-3.dll and fftw3.dll in:

C:\Windows\system32

neither in SysWOW64. system is 64 bit.

same crash. other scripts load and run fine.


Try vdub2 to preview , any crash ?

How do you know the crash now is related to libfftw3f-3.dll ? That was for the previous error with DepanEstimate ? Do you have a different error message with the new crash ?

What other types of scripts run ok ? Simple ones ?

I'm thinking a bad (other) plugin .


fwiw, my process has been to try the files in various locations and, after trying in that location, to remove them before trying in the next location. i do this to avoid having duplicates or wrong versions cluttered around the system.


You can try avsinfo tool for information and to identify versions of plugins

[BabaG]

thanks, pdr.

i was actually just doing that. vdub2 also crashed but did give this as its best guess as to the cause:An exception occurred in module 'warpsharp'.

thanks again,
babag

[BabaG]

ran avsinfo tool and the 32 bit reported no problems but the 64 bit reported this:[Plugin errors/warnings]
________________________________________________________________________________

Function duplicates:

"m4" : "C:\Program Files (x86)\AviSynth+\plugins64+\GrainFactory3.avsi"
"m4" : "C:\Program Files (x86)\AviSynth+\plugins64+\HistogramRGBParade.avsi"
"undefined" : "[InternalFunction]"
"Undefined" : "C:\Program Files (x86)\AviSynth+\plugins64+\Zs_RF_Shared.avsi"

don't really know what to do about that.

thanks,
babag

[poisondeathray]

Function duplicates shouldn't cause a crash


Where is the modified JM script ?


An exception occurred in module 'warpsharp'.

Check warpsharp version and is it actually being used? Move it into a different directory and/or comment out the LoadPlugin call


Check the dates of the other .dll's in avsinfo . Older ones that haven't been updated are more likely the culprit

If you still have problem - clean out the plugins directory , and one by one add them back according to the error message, checking the script each time . Since other scripts work ok without crashing, it's likely a bad/old plugin

[johnmeyer]

Where is the modified JM script ?I've skimmed these new posts and had the same exact question. He's obviously made a lot of changes. As one example, the modification VoodooFX made by putting the quotes in the correct place corrected an error that was NOT in any version of my script I ever posted. That syntax error, as BabaG has found, causes the script to fail, and my scripts did not have any obvious syntax errors like that.

Therefore, without seeing the script he is actually using, we cannot assume that it is actually the same as anything that I wrote.

[BabaG]

here's the copy of the script, minus the head notes for space.

part 1 of 2.#GENERAL PARAMETERS #moved before file load by BabaG to define play_speed before it is first called (line 90)
#----------------------------------------------------------------------------------------------------------------------------
result="result4" #specify the wanted output here; valid results are "result1" through "result4" and "resultS1" through "resultS4" for before/after
play_speed=18 #play speed (8mm=16; Super8=18; 16mm sound=24)


#VIDEO FILE
#----------------------------------------------------------------------------------------------------------------------------
#Change the following line to point to your video file
#film="e:\fs.avi" #original johnmeyer line
#film="path\to\prores_file.mov"
film="Appointment_Room_512_2048x1536p_GV-HQX_18fps____CLEANUP_MASTERING_SCRIPT_Lvl_4400x65535_TESTCLIP.mov"
#source1= LSmashVideoSource(film).killaudio().assumefps(play_speed).converttoYV12() #supports proreshq import; suggested by poisondeathray
source1= LSmashVideoSource(film).killaudio().assumefps(play_speed).z_ConvertFormat(pixel_type="YUV420P10") #supports proreshq import; suggested by poisondeathray
#z_ConvertFormat(pixel_type="YUV420P10") suggested by Boulder
#z_ConvertFormat(pixel_type="YUV420P16") suggested by Boulder also


#COLOR AND LEVELS PARAMATERS
#----------------------------------------------------------------------------------------------------------------------------
saturation=1.0 #for all outputs
gamma=1.0 #for all outputs
blue= -0 red=-0 #manual color adjustment, when returning result3 & result4. Values can be positive or negative
black_level=0 white_level=255 output_black=0 output_white=255 #manual levels, when returning result2 & result4


#SIZE, CROP AND BORDERS PARAMETERS
#----------------------------------------------------------------------------------------------------------------------------
CLeft=16 CTop=16 CRight=16 CBottom=16 #crop values after Depan and before final resizing
W=720 H=480 #final size after cropping
bord_left=0 bord_top=0 bord_right=0 bord_bot=0 #720p= borders 150
in_bord_left=0 in_bord_top=0 in_bord_right=0 in_bord_bot=0 #Borders around input that must be removed


#STABILISING PARAMETERS
#----------------------------------------------------------------------------------------------------------------------------
maxstabH=05 #maximum values for the stabiliser (in pixels) 20 is a good start value
maxstabV=20
est_left=40 est_top=40 est_right=40 est_bottom=40 #crop and contast values for special Estimate clip
est_cont=1.6

#DENOISING PARAMETERS
#----------------------------------------------------------------------------------------------------------------------------
#Fastest parameters
#denoising_strength=600 #denoising level of first denoiser: MVDegrain()
#denoising_strength=300 #denoising level of first denoiser: MVDegrain()
#block_size= 16 #block size of MVDegrain
#block_size_v= 16
#block_over= 8 #block overlapping of MVDegrainMulti()

#Best compromise between speed and quality
denoising_strength= 600 #denoising level of first denoiser: MDegrain()
block_size= 8 #block size of MVDegrain
block_size_v= 8
block_over= 4 #block overlapping of MVDegrainMulti()

dirt_strength=23 #sets amount of dirt removal (big spots)


#FOUR STEP SHARPENING PARAMETERS
#--------------------------------------------------------------------------------------------------------------------------------
PRE_sharp_ness= 120 PRE_radi_us= 3 #presharpening (UnsharpMask) just after first denoising
Sharp_Strength= 0.1


#AUTO LEVELS PARAMETER
#--------------------------------------------------------------------------------------------------------------------------------
X=4 #X is a special parameter for reducing the autolevels effect on the whites
X2=4 #X2 is a special parameter for reducing the autolevels effect on the blacks


#NUMBER OF THREADS
#--------------------------------------------------------------------------------------------------------------------------------
threads=5


# END VARIABLES, BEGIN SCRIPT
#=================================================================================================================================

# Using AVISynth version 2.60, build August 28, 2012 (Ben-Rudiak-Gould, etc.)
# Using VirtualDub 1.8.6

#Change the following (lower or higher) if you have stability problems with multi-threading
SetMemoryMax(800)

#Load plugins explicitly all commented by BabaG to avoid error that they are not found; make sure they are all in the proper avisynth plugin folders so as to auto load
#LoadPlugin ("mvtools2.dll") #Version 2.5.11.9 2/24/2012
#LoadPlugin("autolevels.dll") #Version 0.6.0.0 1/09/2011
#LoadPlugin("Deflicker.dll") #Version 0.4.0.0 8/16/2004
#Loadplugin("Depan.dll") #Version 1.10.0.0 4/09/2007
#LoadPlugin("DepanEstimate.dll") #Version 1.9.2.0 3/25/2007
#Loadplugin("mt_masktools.dll") #Version 2.0.23.0 3/14/2008
#loadplugin("RemoveDirtSSE2.dll") #Version 0.9 5/05/2005
#Loadplugin("RemoveGrainSSE2.dll") #Version 0.9 5/01/2005
#Loadplugin("warpsharp.dll") # 4/05/2010

#Use the following for alternative frame interpolation
#loadplugin("svpflow1.dll") #Version 1.2.1.0 5/29/2012
#loadplugin("svpflow2.dll") #Version 1.3.1.0 6/02/2012
#Import("InterFrame2.avsi") #Version 2.1.0 6/27/2012

#Open and crop the video
#Remove all setmtmode statements (there are four in this script) if not using multi-threaded (MT) AVISynth
#setmtmode(5,threads) #commented by BabaG
#source1= Avisource(film).killaudio().assumefps(play_speed).converttoYV12() #killaudio() improves stability (in my experience -- others disagree)
source1= LSmashVideoSource(film).killaudio().assumefps(play_speed).z_ConvertFormat(pixel_type="YUV420P10") #supports proreshq import; suggested by poisondeathray
#z_ConvertFormat(pixel_type="YUV420P10") suggested by Boulder
#z_ConvertFormat(pixel_type="YUV420P16") suggested by Boulder also

cropped_source=source1.crop(in_bord_left,in_bord_top,-in_bord_right,-in_bord_bot) #temporarily remove any black borders on input video
#setmtmode(2) #commented by BabaG

#cropped_source=filldrops(cropped_source) #Use this when removing bad frames that have been removed by duplicating previous frame
#(see notes in function at end of script)

#STABILIZING
#....................................................................................................................................................................
stab_reference= cropped_source.crop(est_left,est_top,-est_right,-est_bottom).tweak(cont=est_cont).MT_binarize(threshold=80).greyscale().invert()
mdata=DePanEstimate(stab_reference,trust=1.0,dxmax=maxstabH,dymax=maxstabV)
stab=DePanStabilize(cropped_source,data=mdata,cutoff=0.5,dxmax=maxstabH,dymax=maxstabV,method=1,mirror=15)

#Alternative line that includes deflicker. I find that this usually is not needed.
#stab=DePanStabilize(cropped_source,data=mdata,cutoff=0.5,dxmax=maxstabH,dymax=maxstabV,method=1,mirror=15).deflicker()


#DENOISING
#...................................................................................................................................................................

#Remove dirt
input_to_removedirt=stab.crop(CLeft,CTop,-CRight,-CBottom)
stabcrop=RemoveDirtMC(input_to_removedirt,dirt_strength,false)

#Reduce grain
prefiltered = RemoveGrain(stabcrop,2)
superfilt = MSuper(prefiltered, hpad=32, vpad=32,pel=2)
super= MSuper(stabcrop, hpad=32, vpad=32,pel=2)

halfblksize= (block_size>4) ? block_size/2 : 4
halfoverlap= (block_over>2) ? block_over/2 : 2

bvec1 = MAnalyse(superfilt, isb = true, delta = 1, blksize=block_size, overlap=block_over,dct=0)
bvec1 = MRecalculate(super, bvec1, blksize=halfblksize, overlap=halfoverlap,thSAD=100)

fvec1 = MAnalyse(superfilt, isb = false, delta = 1, blksize=block_size, overlap=block_over,dct=0)
fvec1 = MRecalculate(super, fvec1, blksize=halfblksize, overlap=halfoverlap,thSAD=100)

bvec2 = MAnalyse(superfilt, isb = true, delta = 2, blksize=block_size, overlap=block_over,dct=0)
bvec2 = MRecalculate(super, bvec2, blksize=halfblksize, overlap=halfoverlap,thSAD=100)

fvec2 = MAnalyse(superfilt, isb = false, delta = 2, blksize=block_size, overlap=block_over,dct=0)
fvec2 = MRecalculate(super, fvec2, blksize=halfblksize, overlap=halfoverlap,thSAD=100)

denoised=stabcrop.MDegrain2(super, bvec1,fvec1,bvec2,fvec2,thSAD=denoising_strength).levels(0,gamma,255,0,255).tweak(sat=saturation).unsharpmask(PRE_sharp_ness,PRE_radi_us,0)


#SHARPENING
#...................................................................................................................................................................
sharp1=denoised.sharpen(Sharp_Strength)
PreBorderFrame = sharp1.Lanczos4Resize(W - bord_left - in_bord_left - bord_right - in_bord_right, H - bord_top - in_bord_top - bord_bot - in_bord_bot)

[BabaG]

paret 2 of 2.#FRAME INTERPOLATION (optional)
#...................................................................................................................................................................

#Interpolate frames
#Use this for progressive output -- example shows 29.97 progressive
#PreBorderFrame=InterFrame(NewNum=30000,NewDen=1001,PreBorderFrame,GPU=true,Cores=threads)

#Use this for interlaced output -- example shows 29.97 interlaced
#PreBorderFrame=InterFrame(NewNum=60000,NewDen=1001,PreBorderFrame,GPU=true,Cores=threads).SeparateFields().SelectEvery(4, 0, 3).Weave()


#RESULT1: AUTOLEVELS,AUTOWHITE
#......................................................................................................................................................................
#SetMTMode(5) #Turn off SetMTMode for Autolevels #commented by BabaG
result1= PreBorderFrame.coloryuv(autowhite=true).addborders(X,0,0,0,$FFFFFF).addborders(0,0,X2,0,$000000).autolevels(filterRadius=2).crop(X,0,-X2,-0).addborders(bord_left+in_bord_left, bord_top+in_bord_top, bord_right+in_bord_right, bord_bot+in_bord_bot)

#RESULT3: AUTOLEVELS, MANUAL COLOR CORRECTIONS
#.....................................................................................................................................................................
result3= PreBorderFrame.coloryuv(off_U=blue,off_V=red).addborders(X,0,0,0,$FFFFFF).addborders(0,0,X2,0,$000000).autolevels(filterRadius=2).crop(X,0,-X2,-0).addborders(bord_left+in_bord_left, bord_top+in_bord_top, bord_right+in_bord_right, bord_bot+in_bord_bot)
#SetMTMode(2) #Re-enable SetMTMode after Autolevels #commented by BabaG

#RESULT2: MANUAL LEVELS, AUTOWHITE
#......................................................................................................................................................................
result2= PreBorderFrame.levels(black_level,gamma,white_level,0,255).coloryuv(autowhite=true).addborders(bord_left+in_bord_left, bord_top+in_bord_top, bord_right+in_bord_right, bord_bot+in_bord_bot)

#RESULT4: MANUAL LEVELS, MANUAL COLOR CORRECTIONS
#.....................................................................................................................................................................
result4= PreBorderFrame.coloryuv(off_U=blue,off_V=red).levels(black_level,gamma,white_level,0,255).addborders(bord_left+in_bord_left, bord_top+in_bord_top, bord_right+in_bord_right, bord_bot+in_bord_bot)


#PARAMETERS FOR THE COMPARISONS
#.....................................................................................................................................................................
W2= W+bord_left+bord_right
H2= H+bord_top+bord_bot
source4=Lanczos4Resize(source1,W2,H2)


#COMPARISONS: ORIGINAL VS RESULTS
#......................................................................................................................................................................
resultS1= stackhorizontal(subtitle(source4,"original",size=28,align=2),subtitle(result1,"autolevels, autowhite",size=28,align=2))
resultS2= stackhorizontal(subtitle(source4,"original",size=28,align=2),subtitle(result2,"autowhite, manual levels correction",size=28,align=2))
resultS3= stackhorizontal(subtitle(source4,"original",size=28,align=2),subtitle(result3,"autolevels, manual color correction",size=28,align=2))
resultS4= stackhorizontal(subtitle(source4,"original",size=28,align=2),subtitle(result4,"manual colors and levels correction",size=28,align=2))

#Finish and go home
Eval(result)


# END SCRIPT, BEGIN FUNCTIONS
#=================================================================================================================================


#REMOVE DIRT FUNCTION
#......................................................................................................................................................................
function RemoveDirt(clip input, int "limit", bool "_grey") #quotes added around _grey per forum suggestion for error
{
clensed=input.Clense(grey=_grey, cache=4)
alt=input.RemoveGrain(2)
return RestoreMotionBlocks(clensed,input,alternative=alt,pthreshold=6,cthreshold=8, gmthreshold=40,dist=3,dmode=2,debug=false,noise=limit,noisy=4, grey=_grey)

# Alternative settings
# return RestoreMotionBlocks(clensed,input,alternative=alt,pthreshold=4,cthreshold=6, gmthreshold=40,dist=1,dmode=2,debug=false,noise=limit,noisy=12,grey=_grey,show=true)
# return RestoreMotionBlocks(clensed,input,alternative=alt,pthreshold=6,cthreshold=8, gmthreshold=40,dist=3,tolerance= 12,dmode=2,debug=false,noise=limit,noisy=12,grey=_grey,show=false)
}

#This function provides motion compensation for the remove dirt function, thus improving quality.
function RemoveDirtMC(clip,int "limit", bool "_grey")
{
_grey=default(_grey, false)
limit = default(limit,6)

prefiltered = RemoveGrain(clip,2)
superfilt = MSuper(prefiltered, hpad=32, vpad=32,pel=2)
super=MSuper(clip, hpad=32, vpad=32,pel=2)

bvec = MAnalyse(superfilt,isb=true, blksize=16, overlap=2,delta=1, truemotion=true)
fvec = MAnalyse(superfilt,isb=false, blksize=16, overlap=2,delta=1, truemotion=true)

bvec_re = Mrecalculate(super,bvec,blksize=8, overlap=0,thSAD=100)
fvec_re = Mrecalculate(super,fvec,blksize=8, overlap=0,thSAD=100)

backw = MFlow(clip,super,bvec_re)
forw = MFlow(clip,super,fvec_re)

clp=interleave(forw,clip,backw)
clp=clp.RemoveDirt(limit,_grey)
clp=clp.SelectEvery(3,1)
return clp
}

# The following function will remove near duplicates ("0.1") or exact duplicates (change to "0.0").
# It replaces these duplicates with a motion estimated frame. It works really well (IMHO).
#
# Use: In your video editor, replace any single bad frame (burned frame, jump, missing frame from splice, etc.) with a duplicate of the
# previous frame. Then, include a call to this function (see script above, where it is commented out).
#
# Here's an example of how this function can be use:
#
# http://www.youtube.com/watch?v=uzMFodrGHDs
#

function filldrops (clip c)
{
super=MSuper(c,pel=2)
vfe=manalyse(super,truemotion=true,isb=false,delta=1)
vbe=manalyse(super,truemotion=true,isb=true,delta=1)
filldrops = mflowinter(c,super,vbe,vfe,time=50)
fixed = ConditionalFilter(c, filldrops, c, "YDifferenceFromPrevious()", "lessthan", "0.1")
return fixed
}


i moved the 'play_speed' variable definition ahead of everything else as i was getting an error that avisynth didn't know what it meant. i assume that meant it hadn't been defined before it was called. moving it allowed the script to continue.

i commented out the 'loadplugins' commands and put the plugins in the main avisynth plugin folders.

setmtmode was causing errors so i commented those for the time being.

thanks,
babag

[poisondeathray]

i commented out the 'loadplugins' commands and put the plugins in the main avisynth plugin folders.


I meant don't even move them to autoload directory. Move them somewhere else. They are all older versions and some of them are known to cause problems with modern avs+ versions .

Or stick with old avs classic with old plugins.

If you're going to use avs+ x64:

Don't use the plugin warpsharp.dll. It's probably the old avs 2.5 plugin.
#Loadplugin("warpsharp.dll") # 4/05/2010
There is a newer awarpsharp2 , that has a compatibility function for old scripts and it works for avs+ x64
But it looks like warpsharp isn't even being called. Sharpen is using the internal sharpen function in that script

Another common problem is mt_masktools.dll - this version is known to cause various problems with modern avs+
#Loadplugin("mt_masktools.dll") #Version 2.0.23.0 3/14/2008
#LoadPlugin ("mvtools2.dll") #Version 2.5.11.9 2/24/2012

Use the pinterf masktools and mvtools2
https://github.com/pinterf/mvtools/releases
https://github.com/pinterf/masktools/releases

In avsinfo tool , look at the dates of the plugins - that should be a big clue. Move any old versions to some other folder and update them to the updated versions


setmtmode was causing errors so i commented those for the time being.


That is for avs classic . For avs+ you would use SetFilterMTMode and Prefetch
http://avisynth.nl/index.php/SetFilterMTMode

[BabaG]

here's the avsinfo file for 64:[OS/Hardware info]
Operating system: Windows 10 (x64) (Build 18362)

CPU: Intel(R) Core(TM) i7-4930K CPU @ 3.40GHz / Coffee Lake (Core i7)
MMX, SSE, SSE2, SSE3, SSSE3, SSE4.1, SSE4.2, AVX
6 physical cores / 12 logical cores


[Avisynth info]
VersionString: AviSynth+ 3.7.3 (r4003, 3.7, x86_64)
VersionNumber: 2.60
File / Product version: 3.7.3.0 / 3.7.3.0
Interface Version: 10
Multi-threading support: Yes
Avisynth.dll location: C:\windows\system32\avisynth.DLL
Avisynth.dll time stamp: 2023-07-16, 07:48:08 (UTC)
PluginDir2_5 (HKLM, x64): C:\Program Files (x86)\AviSynth+\plugins64
PluginDir+ (HKLM, x64): C:\Program Files (x86)\AviSynth+\plugins64+


[C++ 2.5 Plugins (64 Bit)]
C:\Program Files (x86)\AviSynth+\plugins64+\Decomb.dll [2013-12-02]
C:\Program Files (x86)\AviSynth+\plugins64+\warpsharp.dll [2011-06-15]

[C++ 2.6 Plugins (64 Bit)]
C:\Program Files (x86)\AviSynth+\plugins64+\AutoLevels_x64.dll [0.12.3.0]
C:\Program Files (x86)\AviSynth+\plugins64+\avsresize.dll [0.0.0.0]
C:\Program Files (x86)\AviSynth+\plugins64+\aWarpSharp.dll [2016-06-24]
C:\Program Files (x86)\AviSynth+\plugins64+\ConvertStacked.dll [2023-07-16]
C:\Program Files (x86)\AviSynth+\plugins64+\Deflicker.dll [0.6.0.0]
C:\Program Files (x86)\AviSynth+\plugins64+\DePan.dll [2.13.1.6]
C:\Program Files (x86)\AviSynth+\plugins64+\DePanEstimate.dll [2.10.0.4]
C:\Program Files (x86)\AviSynth+\plugins64+\DirectShowSource.dll [2023-07-16]
C:\Program Files (x86)\AviSynth+\plugins64+\ffms2.dll [2020-08-22]
C:\Program Files (x86)\AviSynth+\plugins64+\fft3dfilter.dll [2.10.0.0]
C:\Program Files (x86)\AviSynth+\plugins64+\FluxSmooth.dll [1.4.0.0]
C:\Program Files (x86)\AviSynth+\plugins64+\hqdn3d.dll [1.1.1.0]
C:\Program Files (x86)\AviSynth+\plugins64+\ImageSeq.dll [2023-07-16]
C:\Program Files (x86)\AviSynth+\plugins64+\KNLMeansCL.dll [2020-11-12]
C:\Program Files (x86)\AviSynth+\plugins64+\LSMASHSource.dll [2022-05-05]
C:\Program Files (x86)\AviSynth+\plugins64+\masktools2.dll [2.2.30.0]
C:\Program Files (x86)\AviSynth+\plugins64+\mvtools2.dll [2.7.45.0]
C:\Program Files (x86)\AviSynth+\plugins64+\neo-fft3d.dll [1.0.0.0]
C:\Program Files (x86)\AviSynth+\plugins64+\nnedi3.dll [0.9.4.60]
C:\Program Files (x86)\AviSynth+\plugins64+\RemoveDirt.dll [0.9.3.0]
C:\Program Files (x86)\AviSynth+\plugins64+\RgTools.dll [1.2.0.0]
C:\Program Files (x86)\AviSynth+\plugins64+\Shibatch.dll [2023-07-16]
C:\Program Files (x86)\AviSynth+\plugins64+\TimeStretch.dll [2023-07-16]
C:\Program Files (x86)\AviSynth+\plugins64+\TIVTC.dll [1.0.26.0]
C:\Program Files (x86)\AviSynth+\plugins64+\VDubFilter.dll [2023-07-16]
C:\Program Files (x86)\AviSynth+\plugins64\AddGrainC.dll [1.8.4.0]

[Scripts (AVSI)]
C:\Program Files (x86)\AviSynth+\plugins64+\colors_rgb.avsi [2022-10-07]
C:\Program Files (x86)\AviSynth+\plugins64+\GrainFactory3.avsi [2022-10-23]
C:\Program Files (x86)\AviSynth+\plugins64+\HistogramRGBParade.avsi [2024-01-18]
C:\Program Files (x86)\AviSynth+\plugins64+\QTGMC.avsi [2022-10-24]
C:\Program Files (x86)\AviSynth+\plugins64+\SMDegrain.avsi [2023-12-16]
C:\Program Files (x86)\AviSynth+\plugins64+\TemporalDegrain.avsi [2023-03-01]
C:\Program Files (x86)\AviSynth+\plugins64+\TransformsPack - Main.avsi [2023-12-16]
C:\Program Files (x86)\AviSynth+\plugins64+\TransformsPack - Models.avsi [2023-12-16]
C:\Program Files (x86)\AviSynth+\plugins64+\TransformsPack - Transfers.avsi [2023-12-16]
C:\Program Files (x86)\AviSynth+\plugins64+\Zs_RF_Shared.avsi [2022-10-24]

[Uncategorized files]
C:\Program Files (x86)\AviSynth+\plugins64+\colors_rgb.txt [2022-10-07]

omitted the duplicate functions section as i already posted that. i think things seem to be pretty current. don't really know what i'm doing as far as checking though.

thanks,
babag

[StainlessS]

Avisynth Wiki:- http://avisynth.nl/index.php/Main_Page
External AviSynth+ x64 plugins:- http://avisynth.nl/index.php/AviSynth%2B_x64_plugins
External AviSynth+ x86 plugins:- http://avisynth.nl/index.php/External_filters
Internal filters:- http://avisynth.nl/index.php/Internal_filters

NOTE, Warpsharp is deprecated, use Warpsharp2.

EDIT:
[Uncategorized files]
C:\Program Files (x86)\AviSynth+\plugins64+\colors_rgb.txt [2022-10-07]
Delete that.

[BabaG]

ok. so, i'm in the middle of, one-by-one, bringing filters and functions into my plugins64 folder and, sure enough, have hit a snag with this line (230):denoised=stabcrop.MDegrain2(super, bvec1,fvec1,bvec2,fvec2,thSAD=denoising_strength).levels(0,gamma,255,0,255).tweak(sat=saturation).unsharpmask(PRE_sharp_ness,PRE_radi_us,0)
the error is:Script error: There is no function named 'unsharpmask'.
not sure how to get past this as pdr has recommended to use another filter for this. do i have to change something in the line (230) to match the syntax of the substitute filter?

as i look into this, it's looking to me as if the warpsharp2 suggestion is what would address this? how woul the 230 line be changed?

edit:
thanks, stainlesss. i deleted the file you pointed out.

thanks,
babag

[poisondeathray]

You can replace unsharpmask with updated functions that support high bit depth, so you don't have to downgrade your prores. The original unsharpmask in warpsharp is a 2.5 plugin and does not support other avs+ pixel types like YUV422P10 . You can change the function name to UnsharpMask or change the script UnsharpMask calls to UnsharpMask_avsi
https://github.com/realfinder/AVS-Stuff/blob/master/avs%202.5%20and%20up/UnsharpMask_avsi.avsi

Or dogway's sharpener's pack has a variation
https://github.com/Dogway/Avisynth-Scripts/blob/master/MIX%20mods/SharpenersPack.avsi

[BabaG]

hey! that's awesome! it finally opens. very dark, though. will look into that later. thanks so much for all the help! (unfortunately, i'll probably be back soon.)

thanks again,
babag

[poisondeathray]

hey! that's awesome! it finally opens. very dark, though.

Levels filter. 10bit goes 0 to 1023. Edit all the lines with levels . There might be a few more


black_level=0 white_level=255 output_black=0 output_white=255 #manual levels, when returning result2 & result4


denoised=stabcrop.MDegrain2(super, bvec1,fvec1,bvec2,fvec2,thSAD=denoising_strength).levels(0,gamma,255,0,255).tweak(sat=saturation).unsharpmask(PRE_sharp_ness,PRE_radi_us,0)


result2= PreBorderFrame.levels(black_level,gamma,white_level,0,255).coloryuv(autowhite=true).addborders(bord_left+in_bord_left, bord_top+in_bord_top, bord_right+in_bord_right, bord_bot+in_bord_bot)

[BabaG]

that's got it! thanks!

[johnmeyer]

hey! that's awesome! it finally opens. very dark, though. will look into that later. If it is darker than the original, try one of the two settings that doesn't use autolevels. I usually see no change in gamma, but I almost always use the number 4 setting which doesn't touch either levels or color.

[BabaG]

thanks, johnmeyer! great tip. much appreciated. changing 255 to 1023 seems to have worked here.

i think the last thing i need to try in order to really test this on my materials is to get the change from setmtmode to setfiltermtmode working. for example, with this line:setmtmode(5, threads)
what should the replacement setfiltermtmode line be? if i just say:SetFilterMTMode(5, threads) it throws an error. wrong syntax, i suppose. not sure what to put into it.

edit:
this seems to allow the clip to load:SetFilterMTMode("source1", 2)
i only replaced the first instance of setmtmode with this and the clip does load. don't understand if i should replace the remaining instances of setmtmode with it. they are currently commented.

thanks again, all!
babag

[johnmeyer]

Here is a link to my current script:

John Meyer Film Restoration Script 2024 (https://drive.google.com/file/d/1eLUhz-LL3v_tSLlaH1spnwCNUVWMJn1_/view?usp=sharing)

It is designed for the more modern versions of AVISynth which no longer use the SetMTMode version of multithreading.

This version used GamMac for color, and provides the option of using SpotLess instead of RemoveDirt for getting rid of film dirt.

(Note: I added SpotLess only a few years ago but forgot to update my revision log at the beginning of the script.)

[BabaG]

wow! thanks johnmeyer. just downloaded it. would like to get the version i've been currently working with fully functional before i jump into something new. guess i should study it while waiting for info on these last few questions about the version i've had up to now.

thanks again!
babag

[johnmeyer]

The main reason for making the script available was to answer your question about getting multithreading to work in more modern versions of AVISynth. These versions use the more modern version of multithreading so you should be able adapt whatever you almost have working. The "Prefetch(threads)" line is the main thing. It needs to be the very last statement in the main body of the script (the function calls, of course, can be put anywhere, so "last statement" doesn't apply to them).

I assume you have read this:

AVISynth+ Multithreading (http://avisynth.nl/index.php/AviSynth+#Multithreading)

[BabaG]

thanks again, johnmeyer.

sadly, it's all getting to be a bit more than i can handle. the differences between setmtmode and setfiltermtmode and avisynth 32 and 64. i'm terrible at anything like coding but keeping this all in line is just too much for me. i just can't figure it out. i don't know how you do it.

thanks,
babag

[johnmeyer]

You seem to have gotten pretty far, so I think you can do it. A few suggestions.

1. If you still have a computer using Windows 7, use that.

2. Don't use 64-bit. I still have yet to see any compelling reason why that would be better. You will find that the 32-bit version will have fewer compatibility problems.

3. Turn off all multi-threading options until you have it working. This means commenting out the "Prefetch" line (for the current versions of AVISynth).

[manolito]

Please forgive me if it looks like I want to highjack this discussion. I am asking about this film restauration script because a few days ago a good friend asked me if I had any ideas if this video from 1979 could be improved.

It is an amateur live recording from a band called "Jump Street" shot at the Catalyst in Santa Cruz, the quality is awful, but it does have a sentimental value for my friend (and for me).

Here is the link:
https://www.youtube.com/watch?v=NBxBFOI3rSQ&pp=ygUUanVtcCBzdHJlZXQgY2F0YWx5c3Q%3D

I am not too optimistic if this is even worth the effort, but if you think that this clip could be improved using this script, please let me know...


Cheers
manolito

[BabaG]

1. If you still have a computer using Windows 7, use that.

unfortunately, no windows 7 here.

2. Don't use 64-bit. I still have yet to see any compelling reason why that would be better. You will find that the 32-bit version will have fewer compatibility problems.

this is one of the areas with which i have difficulty: distinguishing between things intended for either version. can they be run alongside each other? how would one make clear which is being called? pretty much everything in my system is 64 bit so i went with 64 bit avisynth+.

3. Turn off all multi-threading options until you have it working. This means commenting out the "Prefetch" line (for the current versions of AVISynth).

i thought i'd pretty much gotten to that point. that's why i was trying to get the multithreading up now. setting the max memory made a huge difference, though. one thing i noted between your two scripts was that, i think it was for autolevels, the earlier version that used setmtmode had a comment about turning autolevels off and then back on. the newer version didn't seem to have any use of the replacement, setfiltermtmode, at all so i couldn't figure out how to update setmtmode in the earlier script.

@manolito, i think this script, and variants, are intended for captured film, in the case of the videofred version, film captured at good quality and, in the johnmeyer version, maybe more tolerant of source quality. obviously, i'm not someone to pay too much attention to but, in this case, i think i have it right.

thanks again,
babag

[StainlessS]

this is one of the areas with which i have difficulty: distinguishing between things intended for either version. can they be run alongside each other? how would one make clear which is being called? pretty much everything in my system is 64 bit so i went with 64 bit avisynth+.
If you load avs file with 64 bit MPC-HC/VirtualDub2/AvsPMod/etc, then it uses Avisynth 64 bit, which in turn uses (AND REQUIRES) 64 plugins in 64 bit plugins directory,
similarly, 32 bit players require 32 bit everything.
EDIT: Also, 64 bit MegUI (or other encoder), uses 64 bit Avisynth etc, and 32 bit ecoder uses 32 bit avisynth etc.
Above bittage requirements apply to all Windows executables and dll's, not just avisynth, n-bit executables require n-bit dll's.

Wiki:- http://avisynth.nl/index.php/Main_Page

[BabaG]

thanks stainlesss. that's helpful.

[rod93]

hi
try to restore a 8mm with "John Meyer Film Restoration Script 2024" but it's a mp4 file so i put "source1= DirectShowSource(film, fps=25, convertfps=true)" don't know if it is ok, then ive got a script error " there is no function named "prefetch" thk for your help

[Emulgator]

Full script, please. And, for starters:
3. Turn off all multi-threading options until you have it working. This means commenting out the "Prefetch" line (for the current versions of AVISynth).
And you will want to use LSmashVideoSource(), even better LWLibavVideoSource() instead of DirectShowSource().
and: converting any film source to 25fps within the sourcefilter, even later based on MVTools2 might do more harm than good.

[rod93]

thk i turn off "prefetch" line the video loading into virtualdub but nothing happen...
try to run the script with virtualdul in x86 but got an another error "mu_binarize:masktools:unsupported color space" thk