The power of Avisynth: restoring old 8mm films.

[videoFred]

Originally Posted by 2Bdecided

he could charge a lot for the quality of work he's doing, but he's shared so much information for free.

Thank you David.

I have worked 5 years on this project and it's not finished yet. Frank Vine is developing a very special RGB Led backlight source. It's almost ready and once I have this running, the project is finished.
http://www.cine2digits.co.uk/

But because for me this is a pure free time project, I have decided to throw everything on the internet. Lots of people have build a similar transfer unit like mine by now.

It'll discourage this level of sharing from others in future if people see one act of generosity, and assume this means that the author has infinite time to debug other people's problems for free!

Indeed, I can not do this. I own and run a company (nothing with film) and this takes most of my time.

Fred.

[rfmmars]

Many thanks for the information, it's the purest approch to the subject.

Richard

[boqa]

Hi,

First of all, a huge thanks to Fred for sharing this great work with us. It's truly inspiring!

Secondly, I am complete newbie in this field, so I hope for your understanding if my questions are to ignorant

Anyway; I have the 8mm film that has been transferred to dvd by a company. I think that the transfer could be been done better, but this dvd is all I have, unfortunately a re-transfer is not a possibility at the moment.

The dvd is PAL and interlaced. So, I used Fred's script and a deinterlace filer on my source and the result is quite nice, definitely better than my original material.

But I was wondering if result could be better. It has been stressed many times in this discussion that the input must be de-interlaced with progressive non-duplicate frames. So my question is this; is it possible for me to remove any duplicate frames from my source dvd, that is, besides deinterlacing, how can I get my input as close to the original as possible (for using it with Fred's script)? I mean, my DVD seem to be 25 fps (as PAL should be), but the original 8mm must have been 18 fps, so it seems to me that "something" has been done by the transferring company.

I know that you may need further details for answering my question (problem is, I don't know exactly how the company transferred the 8mm to dvd), but maybe you still have some advice for me on how to best prepare my dvd input for being processed through Fred's wonder-script

Any advice will be appreciated, thanks
Jakob

[videoFred]

Originally Posted by boqa

I know that you may need further details for answering my question (problem is, I don't know exactly how the company transferred the 8mm to dvd), but maybe you still have some advice for me on how to best prepare my dvd input for being processed through Fred's wonder-script

Hello Jacob,

The best advice I can give you is: get yourself a new transfer, frame by frame this time and no duplicate frames

The second best advice: remove both the interlacing and the duplicate frames. There are filters available for this but I am not the expert on this.

To judge the quality of the transfer, I must see an example. I have seen very good transfers on dvd format and I have seen very bad transfers on dvd format.

Fred.

[kenpachi]

Freddy, Your results are awesome! As many have said before, You are truly a 8mm-transfer Master. Comparing to You and many other guys I'm a total newbie on Avisynth, especially 8mm transfer. I decided to post because of a source that reminded me 8mm quality. It is a NTSC TV broadcast from '68 delivered on DVD. I guess it isn't anything like 8mm but I asked myself how it was recorded in TV studio? I don't know what type of video cameras the used those days but it couldn't be much different from 8mm, could it?

This is the video:

Video
ID                               : 224 (0xE0)
Format                           : MPEG Video
Format version                   : Version 2
Format profile                   : Main@Main
Format settings, Matrix          : Default
Duration                         : 18mn 49s
Bit rate mode                    : Variable
Bit rate                         : 6 854 Kbps
Nominal bit rate                 : 7 700 Kbps
Width                            : 720 pixels
Height                           : 480 pixels
Display aspect ratio             : 4:3
Frame rate                       : 29.970 fps
Standard                         : NTSC
Colorimetry                      : 4:2:0
Scan type                        : Interlaced
Scan order                       : Bottom Field First
Bits/(Pixel*Frame)               : 0.662

And here is an example:
http://www.speedyshare.com/files/22586894/example.zip

@Freddy: What I wonder is if Your script would be any use on this source? I have just tried the default params but it seems it won't do the trick at all. Should I study the script and modify params to obtain results as awesome as in Your clips or just leave it and try standard [ColorYUV, sharpen, AddGrain, TemporalSoften]?

PS. I don't know if this is some good place to discuss such things or I should've posted outside the topic. If so, sorry.

[videoFred]

Originally Posted by kenpachi

Freddy: What I wonder is if Your script would be any use on this source?

The original source might have been real film, I do not know. But this looks like old and low quality video tape to me. My script can not do anything with a source like this, I'm sorry.

PS. I don't know if this is some good place to discuss such things or I should've posted outside the topic. If so, sorry.

Here is the place to discuss this

Fred.

[2Bdecided]

If you want to attack this source properly, it needs a separate thread.

I think it's video > film > video (VHS?) > DVD, without a TBC.

There are VHS artefacts, film artefacts, wrong levels, etc - and even IVTCing it isn't completely straight forward. I think I'd try that experimental software TBC for a start, though don't expect miracles - might make things worse rather than better.

Cheers,
David.

[johnmeyer]

I posted about my film to video transfer system:

High Speed Film to Video Transfer Machine

I finally got the software to work perfectly using my IVTC method. Also, I still use a Workprinter for 8mm and Super8.

The reason for posting here is that I want to do a better job cleaning and adjusting the video that results from these film transfers. There are some things I like about VideoFred's script, some things I don't like, and some things I don't understand.

Like:
The Autolevels function seems to work quite well. It is more subtle and reliable than the HDRAGC filter that both VideoFred and I used a long time ago.

I like the sharpening functions, but I have some significant issues with these, and have several questions (see below). The defaults seem way too strong for 8mm film.

Dislike:
I don't care for the dirt and grain removal techniques. I have gotten much better results with two scripts I developed. One is based on MDegrain, but using the version in MVTools2. It is faster and better. I use this in conjunction with a motion-compensated version of Despot. Another approach I use in a script I sometimes use instead often provides even better results. It uses RemoveDirt. It took me awhile to get the right settings for this because the documentation for RemoveDirt is bad, even by AVISynth standards, but I think I now have it working.

Questions:
My main reason for posting is to ask a few questions. It seems to me that for both levels and for sharpening that you have too many places where these are being done, and as a result, the script sometimes delivers unexpected results. For levels, you first adjust them here:

noise_baseclip= stab2.coloryuv(off_U=blue,off_V=red).levels(0,gamma,255,0,255).tweak(sat=saturation)

but you then adjust them when computing whichever result is going to be returned.

Sharpening is also done more than once (three times, in fact):

stab5= Lanczos4Resize(stab2,W,H).sharpen(0.5)

denoised= noise_baseclip.MVDegrainMulti(vectors, thSAD=denoising_strenght, SadMode=1, idx=2).unsharpmask(PRE_sharp_ness,PRE_radi_us,0)

followed by the entire sharpening section. This seems like a lot of sharpening, and indeed it can produce artifacts, as you can see here:



This is fairly grainy film stock and the grain is being emphasized because of the underexposure. Also note that the highlights are being blown out (look at the trailing edges of the seagulls' wings), something I noticed in the clips posted by VideoFred on Vimeo. I think the double sharpening and double levels increase are partly the reason for this. Obviously I can reduce the sharpening and levels settings, but I'm not sure how they interact and whether there is some subtle reason to have them both.

I also mentioned being disappointed in the dirt removal. Here's a before/after for the default script settings:



and here is the same before and after using the RemoveDirt script I normally use:



Note that the dirt above the girl's head is not cleaned by this script, but is nicely cleaned by the RemoveDirt script (the second of the two photo pairs above). Also note that in my script, the blurred parts of the seagulls' wings still looks normal, whereas this script makes them look grainy and unnatural.

But, I'm not posting to bash this script. Quite the opposite. It works really well on some clips. In particular, it often does a remarkable job of extracting details. I am almost certain that this is more than just sharpening, and it is this aspect of the script that I'd really like to understand. Take a look at this clip. The script performs absolute magic:



Look at the vertical members of the railing on the porch. They are almost invisible in the original, but are quite clear in the corrected version. What's more, this isn't just a single-frame anamoly: they are clear and visible on every frame in the scene on the corrected version.

So, what I'd like to do is combine the vastly superior dirt and grain removal of my scripts with a little of the autolevel feature of this script. But most of all, I want to figure out what part of this script is making those railings suddenly appear. That is very good stuff indeed.

[videoFred]

Hello John,

Thank you for testing my script, here we go:

Originally Posted by johnmeyer

The Autolevels function seems to work quite well. It is more subtle and reliable than the HDRAGC filter that both VideoFred and I used a long time ago.

HDRAGC is very good for restoring details on clips made with low dynamic range digital cameras. But it is introducing artefacts at the same time. When using a decent quality digital camera for film transfering, there is no need for HDRAGC.

The Autolevels plugin works fine indeed. But I had to 'soften' it a bit by adding small pure white and black borders to the picture, then applied Autolevels, then remove the borders again.

I like the sharpening functions, but I have some significant issues with these, and have several questions (see below). The defaults seem way too strong for 8mm film.

Well, it depends on the source and the amount of grain removing. You can set all sharpening parameters to zero to start, and then increase with small steps.

It uses RemoveDirt. It took me awhile to get the right settings for this because the documentation for RemoveDirt is bad, even by AVISynth standards, but I think I now have it working.

RemoveDirt is very good and very fast. But sharpening afterwards is introducing artefacts.

My main reason for posting is to ask a few questions. It seems to me that for both levels and for sharpening that you have too many places where these are being done, and as a result, the script sometimes delivers unexpected results. For levels, you first adjust them here:

noise_baseclip= stab2.coloryuv(off_U=blue,off_V=red).levels(0,gamma,255,0,255).tweak(sat=saturation)

You have tracked a mistake, I will examine this further.

Sharpening is also done more than once (three times, in fact):

Four times actualy It will only produce artefacts when strenght is to strong.

I also mentioned being disappointed in the dirt removal.

It works quite well when averaging 4 or more frames. But I agree that RemoveDirt work better and faster. But difficult to sharpen afterwards.

In particular, it often does a remarkable job of extracting details. I am almost certain that this is more than just sharpening, and it is this aspect of the script that I'd really like to understand. Take a look at this clip. The script performs absolute magic:

It's the avaraging of multiple frames (MVDegrainMulti). Grain is random, never the same on different frames. A small detail, visible on frame 1, is hidden by the grain on frame 2. Averaging multiple frames makes this detail visible on all frames.

But most of all, I want to figure out what part of this script is making those railings suddenly appear.

Again, MVDegrainMulti() followed by small sharpening steps, big radius first. This approach needs a top quality digital transfer to begin with and I know you have this quality, John.

That is very good stuff indeed.

Thank you! I would like to see a small not modified original clip from you and see what I can do with it. It's realy a matter of setting the parameters right.

Fred.

[videoFred]

Like John has noticed, this line:

noise_baseclip= stab2.coloryuv(off_U=blue,off_V=red).levels(0,gamma,255,0,255).tweak(sat=saturation)

must be:

noise_baseclip= stab2.levels(0,gamma,255,0,255).tweak(sat=saturation)

The color correction should not be there at this point of the script. The gamma and saturation is OK. It is better to do this before the degraining.

The following must be corrected too:

result3= sharp3.addborders  etc.....

this must be:

result3= sharp3.coloryuv(off_U=blue,off_V=red).addborders etc....

It was a test to see if color correction before denoising would change anything. And I have forgotten to change it back to the original script.

I will modify the script as soon as possible and update the download.

Fred.

[johnmeyer]

Freddy,

Thanks for the very complete, and very useful reply. I have spent the past fifteen minutes trying to get information on MVDegrainMulti. That function appears to have been quietly added to MVTools and then quietly removed. I think it was developed by someone else (Josey?). As a result, I can't find much documentation. This is both the fun and the frustration of working with AVISynth plugins: the technology is sometimes amazing, but the documentation is sometimes really, really bad. I have a suspicion that this function, because of its multi-frame averaging, is one of the things that makes this script so slow. Also, it probably is never going to work using the multi-CPU AVISynth hack. I just tested, and my script operates at 78 fps, and yours at 5 fps. Obviously I'd like to improve the performance of your script somewhat. I just transferred another eight hours of film for a client for whom I previously transferred sixteen hours of film. The difference in performance becomes a major issue when trying to do this amount of material, against a deadline, for a client.

I just tried substituting the MVTools2 "equivalent" to MVDegrainMulti that was suggested earlier in this thread by "tedkunich:"

http://forum.doom9.org/showthread.ph...77#post1240377

I had to make a few modifications to what he suggested in order to add back the levels and sharpening commands (even though I'm going to take them back out because, as noted above, some of them are redundant). After making those changes, I was unable to detect any difference between your original script, and your script with the MVTools2 code substituted for MVDegrainMulti. Most important to me, the "magic" appearance of the railings on the porch still happened, and looked identical.

So, when I get time tomorrow, I will go through the script and eliminate any other use of the code from the original MVTools (substituting instead the code from MVTools2) and then see if I can get the whole thing to work with a SetMTMode(2,0) command. My goal is to get my variations to your script to work at something approaching 30 fps on my 3.3 GHz i7 computer. I think that with a few tweaks to the sharpening, and with this change, the quality and speed of this already amazing script can be significantly improved. Once I achieve that, I will see if I can further improve the dirt and dust removal without diminishing the other excellent aspects of this script.

[videoFred]

Originally Posted by johnmeyer

My goal is to get my variations to your script to work at something approaching 30 fps on my 3.3 GHz i7 computer.

Be my guest to do this, John! It would be very nice if you would post the modified script here. The speed is the bottleneck indeed.

Fred.

[Boulder]

Originally Posted by videoFred

Be my guest to do this, John! It would be very nice if you would post the modified script here. The speed is the bottleneck indeed.

Fred.

Would you also mind wrapping the whole script together as a function? It would be very useful if certain parts (such as stabilizing or sharpening) could be disabled with a single boolean. It would also be easier to create batch operations.

[johnmeyer]

Originally Posted by videoFred
Be my guest to do this, John! It would be very nice if you would post the modified script here. The speed is the bottleneck indeed.

I'll definitely post the result. My son graduated this weekend (from high school) so I didn't get a chance to work on this as planned. I should get to it today or tomorrow.

Would you also mind wrapping the whole script together as a function? It would be very useful if certain parts (such as stabilizing or sharpening) could be disabled with a single boolean. It would also be easier to create batch operations.

As for making it a function, that is a good idea, but I'm not sure it will add much value. Freddy has done an excellent job putting all the variables in the header, so the difference in adjusting those versus changing the settings via a function call is not that great. However, I'll look into it when I am working on things. The script is really clean and understandable, and I don't want to wreck a good thing.

[johnmeyer]

Well, after about twenty hours and hundreds of trials, I have created a heavily modified version of videoFred's excellent script. Here are the reasons I made these modifications:

1. I wanted better dirt removal. Much of the film I deal with is old home movies which have been badly stored. Even after cleaning, they still have lots of mold and dirt.

2. I wanted a much faster script.

3. I felt the original script used too much sharpening, leading to a result that sometimes didn't feel like film.

The complete changelog is incorporated into the script below.

The original script runs at about 5 fps on my computer. This one runs at 15 fps. I was able to get several versions to run at close to 50 fps, but they were not stable.

I did eliminate the ability to interpolate frames. I have done extensive work with frame synthesis in my Kinescope to video scripts (which turn Kinescopes into something that feels like the original video) so I am well aware of what can be done. However, once you start down this path, you end up with something that definitely no longer "feels" like film. When converting Kinescopes back to video, this is exactly what I want, but in this case, I still want the film to feel like film. Of course everyone gets to do what they want, but since one of my main objectives was a script that ran faster, I took this out in the name of performance.

Many, many thanks to VideoFred for this amazing script. Hopefully I have not done too much violence to his original vision, and perhaps others will find a use for this somewhat less capable, but definitely faster script. I think you will find the dirt removal rather amazing. You'll need to download the RemoveDirt plugin:

http://www.removedirt.de.tf/

To get full performance from this script, you must use the MT version AVISynth. I used the 2.5.8 MT version 5 (August 16, 2009) build.

The script is contained in the next post (too long to fit here).

[johnmeyer]

Here is the modified script:

# film restoration script by videoFred.
# denoising, resizing, stabilising, sharpening, auto-levels and auto-white balance.
#
# Modified by John Meyer, June 8, 2010
#
# Changes made by Meyer:
#   1. Replaced the MVDegrainMulti function, which was part of the original MVTools
#      with the equivalent function using the newer and faster MVTools2.
#   2. Added RemoveDirt function prior to Degrain in order to eliminate large dust spots. This
#      substantially improves the dirt removal capabilities of this script. The user will need to 
#      download this plugin at:
#          http://www.removedirt.de.tf/
#   3. Eliminated a great deal of sharpening. The original script did sharpening in at least four places.
#      With grainy film stock, this sometimes created objectionable grain. Also, the limitedSharpenFaster
#      function, while excellent for video, is uncessarily slow, and isn't really needed for this low 
#      resolution source. Too much sharpening can make the film look too much like video.
#   4. Added multithreading. I was able to roughly triple the script performance. It may be possible to
#      substantially increase this, perhaps as much as 12x instead of 3x. However, the autolevels function
#      would have to be replaced.
#   5. Fixed several small errors I found. The result3 option didn't have the manual color correction code, so I 
#      added that back in. All the numbered "stab" variables (stab1, stab2, etc.) aren't needed except 
#      for providing a test function for stabilization. I eliminated all of this to streamline the script.
#   6. Deflicker seemed redundant, given all the averaging that takes place with MDegrain, and also the averaging
#      that is done when the autolevels outputs are selected, so I took it out.
#   7. I reduced the number of frames used for averaging autolevels from the default (which is 5) to 2.
#      I probably should add a variable in the header so the user can change this. Something else to do
#      in the future ...
#   8. I added yet another set of crop parameters. I did this because both my capture and my output are 
#      done using NTSC DV AVI which is 720x480. However, 8mm film is almost exactly square, so the captured
#      720x480 video has black bars on the side. These need to be cropped off prior to doing motion 
#      stabilization, but then added back prior to the final output, which must still be 720x480 with the
#      black bars on the side.
#   9. I reduced the default depan settings to 20, which is what was recommended in the original script. I
#      also reduced the post-depan cropping. I did this so I could keep as much of the original frame
#      as possible.
#  10. I removed the second denoising and sharpening function. It just seemed to be too much, and made the 
#      result too artificial
#  11. I removed the MVFLowFPS interpolation. I did this during one of dozens and dozens of attempts
#      to improve the speed of the script. I should probably add this back, but if I do so, I also 
#      need to make it work correctly for interlaced output. If the goal is to show this on an NTSC or PAL
#      television set, then it is not correct to convert from the film fps to 25 fps progressive (PAL) or
#      29.97 fps progressive (NTSC). Instead, this should be done as follows (example given is NTSC):
#
#         MFlowFPS(source,super,backward_vec, forward_vec, num=60000, den=1001,ml=200)
#         SeparateFields()
#         SelectEvery(4, 0, 3)
#         Weave()
#
#      This yields interlaced 29.97, which has twice the temporal resolution as 29.97 progressive, and will
#      therefore look correct on a TV set. I have done a lot of this as part of my Kinescope to video
#      conversion scripts. For those scripts, the goal is to make the filmed version of a TV show look
#      like it was actually videotaped. However, for something that originated on film, this "does violence"
#      to the original feel of the media. It is true that it can make horizontal pans less "juddery," but
#      it won't feel like film anymore. Also, this technique does break down, espcially with fast motion
#      in the foreground.
#  12. Removed the unecessary "coloryuv(off_U=blue,off_V=red)" statement from the denoising section.
#  13. Added killaudio() statement to prevent lockups when using SetMTMode().
#====================================================================================

#Change the following line to point to your video file

film="e:\frameserver.avi"

#====================================================================================


#GENERAL PARAMETERS
#----------------------------------------------------------------------------------------------------------------------------
result="result3"                                                #specify the wanted output here 
trim_begin=0  play_speed=15                                     #trim frames and play speed (PAL: 16.6666 or 18.75)

#COLOR AND LEVELS PARAMATERS
#----------------------------------------------------------------------------------------------------------------------------
saturation=1.2                                                  #for all outputs
gamma=1.2                                                       #for all outputs 
blue= -0  red=-0                                                #manual color adjustment, when returning result3 & result4. Values can be positive or negative
black_level=0  white_level=255 output_black=0  output_white=255 #manual levels, when returning result2 & result4
 

#SIZE, CROP AND BORDERS PARAMETERS
#----------------------------------------------------------------------------------------------------------------------------
CLeft=16  CTop=16  CRight=16  CBottom=16                         #crop values after Depan and before final resizing 
W=720  H=480                                                     #final size after cropping 
bord_left=0  bord_top=0  bord_right=0  bord_bot=0                #720p= borders 150
in_bord_left=68  in_bord_top=0  in_bord_right=68  in_bord_bot=0  #Borders around input that must be removed


#STABILISING PARAMETERS
#----------------------------------------------------------------------------------------------------------------------------
maxstabH=20                                                         #maximum values for the stabiliser (in pixels) 20 is a good start value 
maxstabV=20
est_left=40  est_top=40  est_right=40  est_bottom=40  est_cont=1.6  #crop and contast values for special Estimate clip


#DENOISING PARAMETERS
#----------------------------------------------------------------------------------------------------------------------------
denoising_strength=600                     #denoising level of first denoiser: MVDegrain() 
block_size= 16                             #block size of MVDegrain
block_size_v= 16
block_over= 8                              #block overlapping of MVDegrainMulti()
dirt_strength=23                           #sets amount of dirt removal (big spots)


#FOUR STEP SHARPENING PARAMETERS
#--------------------------------------------------------------------------------------------------------------------------------
PRE_sharp_ness= 120   PRE_radi_us= 3                      #presharpening (UnsharpMask) just after first denoising


#AUTO LEVELS PARAMETER
#--------------------------------------------------------------------------------------------------------------------------------
X=4   #X is a special parameter for reducing the autolevels effect on the whites
X2=2  #X2 is a special parameter for reducing the autolevels effect on the blacks 


# END VARIABLES, BEGIN SCRIPT
#=================================================================================================================================


SetMemoryMax(800)  #set this to 1/3 of the available memory

Loadplugin("Depan.dll")
LoadPlugin("DepanEstimate.dll")
Loadplugin("removegrain.dll")
LoadPlugin("MVTools2.dll")
Loadplugin("mt_masktools.dll")
Loadplugin("warpsharp.dll")
LoadPlugIn("LimitedSupport_09Jan06B.dll")
LoadPlugin("autolevels.dll")
loadplugin("C:\Program Files\AviSynth 2.5\plugins\RemoveDirtSSE2.dll")

#Remove all setmtmode statements (there are three in this script) if not using multi-threaded (MT) AVISynth
setmtmode(5)
source1= Avisource(film).killaudio().assumefps(play_speed).trim(trim_begin,0).converttoYV12()
cropped_source=source1.crop(in_bord_left,in_bord_top,-in_bord_right,-in_bord_bot)               #remove any black borders on input video
setmtmode(2,0)


#STABILIZING
#....................................................................................................................................................................
stab_reference= cropped_source.crop(est_left,est_top,-est_right,-est_bottom).tweak(cont=est_cont).MT_binarize(threshold=80).greyscale().invert()
mdata=DePanEstimate(stab_reference,trust=1.0,dxmax=maxstabH,dymax=maxstabV)
stab=DePanStabilize(cropped_source,data=mdata,cutoff=0.5,dxmax=maxstabH,dymax=maxstabV,method=1,mirror=15)


#DENOISING
#...................................................................................................................................................................
input_to_removedirt=stab.crop(CLeft,CTop,-CRight,-CBottom)
stabcrop=RemoveDirtMC(input_to_removedirt,dirt_strength,false)
super = stabcrop.MSuper(pel=2)
bvec1 = MAnalyse(super, isb = true,  delta = 1, blksize=block_size, overlap=block_over)
fvec1 = MAnalyse(super, isb = false, delta = 1, blksize=block_size, overlap=block_over)
bvec2 = MAnalyse(super, isb = true,  delta = 2, blksize=block_size, overlap=block_over)
fvec2 = MAnalyse(super, isb = false, delta = 2, blksize=block_size, overlap=block_over)
#bvec3 = MAnalyse(super, isb = true,  delta = 3, blksize=block_size, overlap=block_over)
#fvec3 = MAnalyse(super, isb = false, delta = 3, blksize=block_size, overlap=block_over)


#Use the second line below, and un-comment two lines above if you want to average more frames. Doesn't seem necessary for most work IMHO.
denoised=stabcrop.MDegrain2(super, bvec1,fvec1,bvec2,fvec2,thSAD=denoising_strength).levels(0,gamma,255,0,255).tweak(sat=saturation).unsharpmask(PRE_sharp_ness,PRE_radi_us,0)
#denoised=stabcrop.MDegrain3(super, bvec1,fvec1,bvec2,fvec2,bvec3,fvec3,thSAD=denoising_strength).levels(0,gamma,255,0,255).tweak(sat=saturation).unsharpmask(PRE_sharp_ness,PRE_radi_us,0)


#SHARPENING
#...................................................................................................................................................................
sharp1=denoised.sharpen(0.9)	
PreBorderFrame = sharp1.Lanczos4Resize(W - bord_left - in_bord_left - bord_right - in_bord_right, H - bord_top - in_bord_top - bord_bot - in_bord_bot)


#RESULT1: AUTOLEVELS,AUTOWHITE
#......................................................................................................................................................................
setmtmode(5)

result1= PreBorderFrame.coloryuv(autowhite=true).addborders(X,0,0,0,$FFFFFF).addborders(0,0,X2,0,$000000).autolevels(filterRadius=2).crop(X,0,-X2,-0).addborders(bord_left+in_bord_left, bord_top+in_bord_top, bord_right+in_bord_right, bord_bot+in_bord_bot)

#RESULT2: MANUAL LEVELS, AUTOWHITE
#......................................................................................................................................................................
result2= PreBorderFrame.levels(black_level,gamma,white_level,0,255).coloryuv(autowhite=true).addborders(bord_left+in_bord_left, bord_top+in_bord_top, bord_right+in_bord_right, bord_bot+in_bord_bot)

#RESULT3: AUTOLEVELS, MANUAL COLOR CORRECTIONS
#.....................................................................................................................................................................
result3= PreBorderFrame.coloryuv(off_U=blue,off_V=red).addborders(X,0,0,0,$FFFFFF).addborders(0,0,X2,0,$000000).autolevels(filterRadius=2).crop(X,0,-X2,-0).addborders(bord_left+in_bord_left, bord_top+in_bord_top, bord_right+in_bord_right, bord_bot+in_bord_bot)

#RESULT4: MANUAL LEVELS, MANUAL COLOR CORRECTIONS
#.....................................................................................................................................................................
result4= PreBorderFrame.coloryuv(off_U=blue,off_V=red).levels(black_level,gamma,white_level,0,255).addborders(bord_left+in_bord_left, bord_top+in_bord_top, bord_right+in_bord_right, bord_bot+in_bord_bot)


#PARAMETERS FOR THE COMPARISONS
#.....................................................................................................................................................................
W2= W+bord_left+bord_right
H2= H+bord_top+bord_bot
source4=Lanczos4Resize(source1,W2,H2)


#COMPARISONS: ORIGINAL VS RESULTS
#......................................................................................................................................................................
resultS1= stackhorizontal(subtitle(source4,"original",size=28,align=2),subtitle(result1,"autolevels, autowhite",size=28,align=2))
resultS2= stackhorizontal(subtitle(source4,"original",size=28,align=2),subtitle(result2,"autowhite, manual levels correction",size=28,align=2))
resultS3= stackhorizontal(subtitle(source4,"original",size=28,align=2),subtitle(result3,"autolevels, manual color correction",size=28,align=2))
resultS4= stackhorizontal(subtitle(source4,"original",size=28,align=2),subtitle(result4,"manual colors and levels correction",size=28,align=2))


Eval(result)

# END SCRIPT, BEGIN FUNCTIONS
#=================================================================================================================================


#REMOVE DIRT FUNCTION
#......................................................................................................................................................................
function RemoveDirt(clip input, int limit, bool _grey)
{
  clensed=input.Clense(grey=_grey, cache=4)
  alt=input.RemoveGrain(2)
  return RestoreMotionBlocks(clensed,input,alternative=alt,pthreshold=6,cthreshold=8, gmthreshold=40,dist=3,              dmode=2,debug=false,noise=limit,noisy=4, grey=_grey)

  # Alternative settings
  # return RestoreMotionBlocks(clensed,input,alternative=alt,pthreshold=4,cthreshold=6, gmthreshold=40,dist=1,dmode=2,debug=false,noise=limit,noisy=12,grey=_grey,show=true)
  # return RestoreMotionBlocks(clensed,input,alternative=alt,pthreshold=6,cthreshold=8, gmthreshold=40,dist=3,tolerance= 12,dmode=2,debug=false,noise=limit,noisy=12,grey=_grey,show=false)
}

function RemoveDirtMC(clip,int "limit", bool "_grey")
{
  _grey=default(_grey, false)
  limit = default(limit,6)
  i=MSuper(clip,pel=2)
  bvec = MAnalyse(i,isb=false, blksize=8, delta=1, truemotion=true)
  fvec = MAnalyse(i,isb=true, blksize=8, delta=1, truemotion=true)
  backw = MFlow(clip,i,bvec)
  forw  = MFlow(clip,i,fvec)
  clp=interleave(forw,clip,backw)
  clp=clp.RemoveDirt(limit,_grey)
  clp=clp.SelectEvery(3,1)
  return clp
}

[videoFred]

Originally Posted by johnmeyer

Here is the modified script:

It looks great, John! The dirt removing is quite impressive. I will test it further on different sources.

Fred.

[johnmeyer]

I plan on doing more work today. Your script definitely does a better job of extracting small details in fine-grained film that contains pictures with a lot of "structure," meaning that they don't have huge areas of sky or water, but instead are filled with objects that have lots of varied details.

On the other hand, the performance of this script is key. I still believe that I can get this to operate at least 2x faster, and perhaps more, because I'm still only using about 20% of my CPUs, and many of my other scripts peg all eight cores on my machine at 100%.

I have run through clips from over eight hours of 8mm film through my modification of your script. The film I used for my tests was taken over a thirty year period on dozens of different film stocks, and contains a huge range of terrible exposures, camera shakes, subject matter, etc. Thus, this amateur film helps show off the advantages and disadvantages of every script approach. So far, I've had no bad surprises from my modified script. Since I want to be able to treat large amounts of film, and don't have the time to fine-tune each and every scene, the other goal I had was to create a script where the settings could be kept constant for an entire reel of film. I'm going to start on the eight hours of film today. We'll see how it goes.

P.S. I also have increased the X and X2 parameters since I posted the script. I kept having "flashes" from the autolevels locking on borders. This probably has to do with the fact that my 720x480 videos have some residual borders. I now crop these in the script, but I think some residual may be creeping through. As near as I can tell, these two parameters can be made quite large before the exposure would be adversely affected.

[johnmeyer]

Here's a short before/after clip using the revised script:

http://www.youtube.com/user/johnmeye.../0/4Gy9ZG23TTk

(And yes, I did clean the film prior to transferring with my Workprinter ...)

[Leinad4Mind]

johnmeyer, can u please upload, the "LimitedSupport_09Jan06B.dll" plugin, I can just find the 2005 version of it.

best regards, and great work improving it.