The power of Avisynth: restoring old 8mm films.

[StainlessS]

HaHa great stuff, especially the Puffin' Bilko ad, cant even drag on an electronic ciggy here today in public (crazy PC-ness)

[footageforpro]

Well John, congratulation for the home movies you posted. I have a big collection, but it's difficult for me to buy so great footage because there are no garage sales here in Italy and also transport cost and taxes for Ebay american auctions are usually too much for my small budget.
Anyway, I use to work with uncompressed AVI files because after restoring footage with AVI synth I transform them with MPEG streamclip in Apple Prores to edit in Final Cut Pro. My idea is that is not fine to have to compress videos with 2 different codecs. I can work with Photojpeg or Mjpeg, they will save me a lot of time and space but I feel more confortable with uncompressed. Anyway, my workflow is fine with my 2 laptops (8 Gb and 6 Gb Ram), but it crashes with my Desktop with only 2 Gb Ram. Where do you think I can change settings of the script to make it work?

[johnmeyer]

In my version of the script, change SetMemoryMax(). In Fred's script, I don't know.

A lot depends on whether it crashes right away; whether it crashes after just a few frames, or whether it makes it most of the way and then crashes. Crashes that happen later usually indicate some sort of memory issue. Crashes that happen right away often indicate that a DLL conflicts with the version of AVISynth you are using.

I think you have already made sure that you are using only the plugins that came bundled with Fred's script. If your system has other versions of the plugins, you must make sure your script doesn't "grab" those instead. Loading the DLLs explicitly (drive and path) is the ultimate way to ensure that the proper plugins get loaded.

BTW, none of the film I posted is stock footage that I purchased. Instead, these are film transfers that I have done for friends and clients.

[wdwms]

Sorry to dig up an old thread, but I wanted to thank everyone for their efforts on these scripts. I've been lurking here for at least 1.5 years, queitly absorbing everything.

Now I'd like to show people what I have been doing and the results of everyone's input and efforts.

I've taken johnmeyers script, watered it down a bit, paired it with my own scripts, virtualdub plugins and then added in the full paid version of Neat Video. It is a two step process (three if you count the compression with handbrake), but the results are nothing but amazing.

All of the films I've restored are on YouTube and Vimeo:
https://vimeo.com/retrowdw/videos
https://www.youtube.com/c/retrodisneyworld

Our most popular films, and the best in terms of restoration are:
http://www.retrodisneyworld.com/imag...-movie-footage
http://www.retrodisneyworld.com/imag...al-restoration
http://www.retrodisneyworld.com/imag...ilm-30-minutes

I've struggled a bit when the color is really bad. We haven't found a plugin for virtualdub or avisynth that can auto color correct well with scene changes. So if you have any ideas please let me know. A few of the restoration are less than perfect when it comes to color.

We also have blog setup on my website that discusses each film in detail that we restore:
http://www.retrodisneyworld.com/imageworks/films

I have nearly 15,000 feet of personal film to restore, but I've been buying disney world footage from ebay and restoring it as well. So i've been busy to say the least!

With that said, again thank you to all who have put in their time and efforts in participating in this thread, much appreciated!

Todd
www.retrodisneyworld.com

[johnmeyer]

Originally Posted by wdwms

... I wanted to thank everyone for their efforts on these scripts. I've been lurking here for at least 1.5 years, queitly absorbing everything ... I've struggled a bit when the color is really bad. We haven't found a plugin for virtualdub or avisynth that can auto color correct well with scene changes. So if you have any ideas please let me know. A few of the restoration are less than perfect when it comes to color.

Your color restoration actually looks amazing. In my experience, there is no "auto white balance" plugin that works very well. This includes free plugins for VirtualDub, as well as high-end paid plugins for NLEs. The best path is to do the color correction scene-by-scene using your NLE's color balance tools. Hopefully it lets you balance the shadows, highlights, and midtones using separate settings.

What I'd like to know is exactly what did you do to get such great color correction results? I just tried to do a better job, using your "before" clip from Vimeo, and couldn't get close to the color quality that you achieved. I too do this for a living, so if you had to use some sort of commercial product, I'd be willing to buy it. Please share!

Finally, I want to commend you on your restoration sensibilities. You resisted the temptation to turn up the "knobs" on each restoration step, and as a result ended up with a very natural-looking result, even though it still has a little dirt, and a little motion. The only thing I would have done differently was to do a little more motion stabilization to get rid of the gate weave. It looks as though you decided to do very little motion stabilization, perhaps none at all.

John

P.S. This is my attempt to restore some "red and white" 8mm footage that someone purchased in 1962 at the Seattle Worlds Fair. Like most print movie film, everything except for the red dye layer was very unstable and had faded. This result was the best I could do, but nowhere near as good as the result you achieved:

1962 Seattle Worlds Fair (stock shots)

[wdwms]

john,
Thanks for the comments! Its been a lot of trial and error, let me type up a quick few comments here; if you want a more detailed dive into the process and scripts, please let me know.

For color correction I am using ColorMill for Virtualdub. Believe it or not, the color adjustments you see are 100% done by eye. I sit there and move the sliders inside ColorMill until I am satisfied. I generally apply a color correction that works for 90% of the film. I don't have the time to break each film into scenes and color correct each one. Generally most 3 minute reels were shot within the same day so exposure and such is generally balanced. Its the commercial films from Disney that are a disaster to work on, not only are the dyes crap, but each scene had different exposure and color saturation to begin with.

Thanks for realizing I didn't crank up the knobs. You have to be delicate with this, too much sharpness brings out artifacts and increases grain. There is a fine balance between all as I've learned.

For dirt, i'm using the RemoveDirtSSE2.dll avisynth plugin in conjunction with the RemoveDirtMC function. I usually set that between 60 and 80. Each film is different and some are so far gone.

Neat Video is the other secret weapon. It removes grain, and does a basic sharpen, but I purposely don't crank it up. Leaving some grain and nostalgic feel. Again, the final sharpen then has to be carefully done in order to not make the grain that is left look artifical.

Motion stabilization is on maxstabH=6 and maxstabV=8. I found that higher was actually causing artificial gate weave - my films are scanned on some crazy $100k machine and there is 0 gate weave when I get the uncompressed files from Pixcel.com Higher stab #s were also causing very odd artifacts and motion issues especially on horizontal pans. I probably could crop a bit more, I'm possibly too much of a fan of seeing everything and need to get over that and crop another 10 pixels all around.

The end result that I'm going for is not one that turns film into video, i still want a bit of older fee, some grain, but it to be nostalgic, colorful and clear. if that makes sense. I had a hard time debating leaving them at 18fps or moving to 24 or 29.97fps. In the end, they looked best at 29.97 for the medium we now view these on as much as pained me to nearly double the frame rate and make fake frames.

I can try to see what i can do to your seattle clip, however what is on youtube has lost much of its color information. Do you have the original scan perhaps? I can see what I can do with it. Seems to be in much better shape than this one that I restored:



Red doesn't have to be dead!

Todd

[papcom]

Originally Posted by wdwms

...I had a hard time debating leaving them at 18fps or moving to 24 or 29.97fps. In the end, they looked best at 29.97 for the medium we now view these on as much as pained me to nearly double the frame rate and make fake frames. Todd

@todd - nice work You do. Congratulation.
I do not really understand what You did with the 18fps and what You mean with the "double Frame rate" and the "fake Frames". I am searching for an effective method to bring my 18 films into a good and fluent Motion flow .

[johnmeyer]

Originally Posted by papcom

I do not really understand what You did with the 18fps and what You mean with the "double Frame rate" and the "fake Frames". I am searching for an effective method to bring my 18 films into a good and fluent Motion flow .

You have four choices:

1. Encode at 18 fps and play back on a device that accepts that frame rate. Your computer can play back any frame rate, and most modern TV sets can also play back at any frame rate if you play from a memory stick inserted into the TV's USB port.

2. Add pulldown frames to play at the standard 29.97 (NTSC) or 25 (PAL) frame rate. This is the traditional way of doing it and has the advantage that you introduce zero spatial artifacts. Each frame will be just as clear and sharp as the original. The downside is that the natural judder of the low 18 fps frame rate will be accentuated, and horizontal camera panning will look quite jumpy.

3. Synthesize additional, new frames in order to bring the frame rate to 25, 29.97, 50, or 59.94 fps. This will provide very fluid, smooth motion, but you almost always create significant spatial artifacts on some scenes. Also, the result will no longer "feel" like film, but instead will feel like video. People have strong opinions and preferences both ways, and in the end the "feel" part of the decision to synthesize these frames is an artistic one. Be warned however, that when this technology fails, it often fails spectacularly, with people's legs "breaking" as they walk quickly across the frame, and picket fences warping and breaking as the camera pans horizontally across the fence.

4. Blend adjacent frames in order to create the additional frames needed to play more frames per second. This creates no surprise artifacts, and does create smoother motion. However, because the new frames are simply blends of adjacent frames, you can get very fuzzy results on fast motion.

I always use #1 when I can. It is the easiest and produces no new artifacts.

When I encode for DVD or any other situation where the playback has to be at one of the traditional TV standard frame rates, I use #2. Why? It just plain works. Yes, it results in judder, but judder has always been an issue with film, and if you use a little motion stabilization, that technology also smooths the panning so that the judder isn't quite as noticeable.

Once in awhile, when I have a lot of panning, I will create two versions, one with pulldown fields (#2) and the other with motion estimation (#3) and I switch between them, using the motion estimation for horizontal panning, and the pulldown fields elsewhere.

I have occasionally used #3 (motion estimation) for entire clips when the frame rate of the original was really low, and the amount of horizontal panning was unusually high. Here is an example of a 1930 15 fps original clip. I increased the frame rate to 30 fps, which means that half of the frames are exactly the same as the original. You'd need to see the original to be able to appreciate the huge difference, but when I originally uploaded this clip to YouTube at 15 fps, it was almost unwatchable.

1930s San Francisco

Skip ahead to the 1:05 mark to see the most interesting scenes.

The parts of the clip where the camera pans wildly across the streetcars at the intersection of Embarcadero and Market was particularly hard to follow at the native 15 fps.

The difference between 15 fps and 18 fps is actually quite large, and I seldom feel like I need to do motion estimation when doing 18 or 24 fps film transfers.

I should also point out that because of all the dirt on film, if you can't get each frame perfectly clean, any technique which synthesizes additional frames, rather than simply repeating fields or frames, will produce really nasty surprises.

[papcom]

@johnmeyer - thank you for the listing of all possibilities.
Usually I use method 2, because it preserves the quality of the original Picture. Unfortunately Scenes with movements in the film are displayed rather juddery.(as You write)

I experimented with lots of other methods, e.g. with After Effects, or Dynapel, or Optical Flow on macs. Some results are almost perfect others are not satisfying. The reason You explained very well in Your post.

A good method is the small script You posted some time ago, as:

AssumeFPS(16.0)
ChangeFPS(50,1)
AssumeTFF() # or BFF, as required
SeparateFields()
SelectEvery(4, 0, 3)
Weave()

This method works fine for interlaced material for a DVD.

Now I saw the post of @wdwms, and I took a look to his sample Videoclip, which has a fairly good fluent movement. So I am interested to know how he did this.

[johnmeyer]

He describes his process, in detail, on this page:

Restoration Process

He is a little vague on which of the options I listed he is actually using. Here is what he says on that page:

The computer software uses interpolation, that is it actually creates those 6 frames by comparing the 18 it already knows about. Confused? Don't be, it works and the end result is great!

Since he is using much of the code developed by VideoFred, and later modified by me, and since VideoFred's code uses motion interpolation, my educated guess is that he is using MVFLowFPS or MFlowFPS to interpolate new intermediate frames. As I already mentioned, this works great for large objects and for camera panning, but can break down rather badly not only with the situations I already mentioned, but even more so when something enters or leaves the frame close to the camera.

I've used Twixtor, Dynapel (MotionPerfect), and MVTools2, and have also seen the results from AE. They all suffer from the same problems, to one degree or another. One advantage of MVTools2 is that you can modify a lot of internal options that, under some circumstances, can make quite a dramatic difference. In most circumstances, I have found that using a large block size reduces artifacts. I have published some of my settings in various thread elsewhere in this forum. There is no "one size fits all" script, and you have to tune your settings for each film or video.

[wdwms]

I'm actually using Interframe to do all my 18->24 or 18-> 29.97 conversion. Very little of Fred or Johns' code is left, in fact I think just the dirt removal and stabilization are the only components. The rest is all through my own scripts, majority of it being in virtualdub plugins.

Interframe is here: http://www.spirton.com/interframe/

All my FPS upconversion is done after the initial cleaning. Here is how my restoration works:

Pass #1
Stabilize (avisynth), dirt removal (avisynth), color adjustment (ColorMill in VD), grain removal (NeatVideo in VD), minor sharpening (VD) - Save as full frame AVI
Pass #2
Interframe to 30pfs (avisynth), Crop, resize, final sharpen - Save as Lossless AVI via Lagarith codec - Archived as "restored"
Pass #3
Handbrake with custom settings to compress for distribution. Uses H.264 and outputs as Mp4 - Archived as "distribution file" to share with friends/family


I plan on changing it to this soon:


Pass #1
Stabilize (avisynth), dirt removal (avisynth), color adjustment (ColorMill in VD), grain removal (NeatVideo in VD), minor sharpening (VD), Crop/resize/final sharpen (VD), Save as full frame AVI, uncompressed as "RESTORED"
Pass #2
Interframe to 30pfs (avisynth), Save as Lossless AVI via Lagarith codec - Archived as Restored/Upconverted
Pass #3
Handbrake with custom settings to compress for distribution. Uses H.264 and outputs as Mp4 - Archived as "distribution file" to share with friends/family

[johnmeyer]

Just to be clear, so Papcom is not confused, Interframe is, at its core, the same thing as MVTools2.

MVTools was updated by its author to MVTools2. That code was then re-written by someone else to run on certain GPUs, thus allowing real-time interpolation. Finally, someone wrote a front end to SVP called Interframe which shields the user from the slightly convoluted SVP syntax. Read about it here:

InterFrame

The reason I post this is to point out that the results from Interframe will be nearly identical (perhaps exactly identical ??) to what you get with MVTools2, if you set the MVTools2 settings to be the same.

The reason I don't use either SVP or Interframe (although I do include the commented out code in my version of Fred's scripts) is that I found that the Interframe front end made it difficult for me to adjust several parameters which I found made a huge improvement when interpolating low frame rate film transfers. Remember that the main point for the existence of the SVP and Interframe extensions was to improve the speed of motion interpolation for real time interpolation while watching normal TV. I put that in italics to emphasize the point that the settings have been tuned for fairly high frame rate material, and you therefore may not get good results with material that contains large temporal gaps between adjacent frames.

As I noted in my previous posts, I find wdwms' results to be really excellent, especially his custom color corrections. However, the motion estimation is neither better, nor worse, than what I've seen elsewhere. It contains all of the usual advantages as well as the usual artifacts.

So, my recommendation is to use MVTools2 and modify the MAnalyze and MFlowFPS settings to reduce the inevitable interpolation artifacts.

FWIW, here is the commented out code in my scripts, showing all three versions: MVTools2/MFlowFPS; SVP; and Interframe.

#FRAME INTERPOLATION (optional)
#...................................................................................................................................................................

#superfps= MSuper(PreBorderFrame,pel=2)
#backward_vec2 = MAnalyse(superfps, isb = true,blksize=16)
#forward_vec2  = MAnalyse(superfps, isb = false,blksize=16)
#PreBorderFrame = MFlowFps(PreBorderFrame,superfps, backward_vec2, forward_vec2, num=60000, den=1001, ml=200).SeparateFields().SelectEvery(4, 0, 3).Weave()

#...................................................................................................................................................................
#The following alternative can provide synthesized frames for hi-def material that causes MSuper to choke
#////// Begin alternative to MFlowFPS ////
#super_params="{pel:2,gpu:1}"
#analyse_params="""{block:{w:32,h:32}, 
#	main:{search:{coarse:{distance:-10}}},
#	refine:[{thsad:200}]
#        }"""

#smoothfps_params="{rate:{num:60,den:16,abs:false},scene:{mode:0,limits:{scene:8500}},algo:21,cubic:1}"

#...................................................................................................................................................................
#The following provides smooth output
#super = SVSuper(PreBorderFrame,super_params)
#vectors = SVAnalyse(super, analyse_params)
#PreBorderFrame = SVSmoothFps(PreBorderFrame,super, vectors, smoothfps_params, url="www.svp-team.com", mt=threads).SeparateFields().SelectEvery(4, 0, 3).Weave().assumefps(29.97)
#////// End alternative to MFlowFPS ////

#...................................................................................................................................................................
#The following is another alternative that can provide synthesized frames for hi-def material that causes MSuper to choke. Probably better than the one above.
#////// Begin alternative to MFlowFPS ////
#PreBorderFrame=InterFrame(NewNum=30000,NewDen=1001,PreBorderFrame,GPU=true,Cores=threads)
#////// End alternative to MFlowFPS ////

[wdwms]

Thanks John, i'll give your code a whirl in my next project to see how it goes

[wdwms]

Just wanted to share with you all some more comparisons, this is not final but wanted to show you what i'm doing with some 16mm I just got digitally transferred and I'm almost done with the restoration... This time around I think i've been able to restore the color even better than the last film I did..

I'll make a full length comparison film so you can see the side by side..
-t

[videoFred]

Your Colormill color correction is incredible! I will have a closer look at this filter myself.

Fred.

[johnmeyer]

As I've said in countless posts in both this thread and elsewhere, automatic white balance is a noble goal, but one that remains elusive. I have yet to see anything that doesn't fail (make the video worse) as often as it succeeds.

When he posted the last time, and said he got his amazing results using just the Colormill plugin, I was amazed by his results, so I downloaded and used Colormill. Here's what I found: there is nothing unusual or different about it, and in fact it does the same thing that I do inside of Vegas with the color corrector included in that NLE.

So, why is he able to get such stunning results? I think there are three reason these results look so good:

1. The Colormill plugin provides the necessary controls, although nothing more than standard color channel sliders.

2. The samples chosen still have enough residual color in the two faded channels that the color can be recovered. This is extremely important. I've dealt with a LOT of red&white, blue&white, and yellow&white film to know that sometimes there simply isn't any color left in the other channels.

3. The guy doing this has a great eye, and understands what he's doing.

#3 is the key. People who do "color timing" for movies are skilled artisans, and the quality of their work could not be duplicated by me, or most other people, even if we had access to the same tools.

This guy is good!! Hire him for your next film project.

[wdwms]

John,
Thanks for #3, humbled by your comments. Maybe my monitor and eye are configured to work in harmony, or I may have gotten lucky w/the films I'm restoring that they have just the right amount of color left. I love doing this and bringing films back to life!

I will say that I've tried other color plugins, none gave me the adjustments I needed to get the color just right. Also I've been doing the color restoration in 2 steps. In the final I had to adjust the hue a bit after the initial correction.

I work in the red gamma first, bringing that down, then I adjust the dark-mid-light for each channel. Final tweaks are the contrast and brightness and such, as well as saturation.

Now if i could just do something about those scratches! (aside from frame by frame restoration).

Would you guys like to see a full side by side comparison of the 30 minute film?

-t

[StainlessS]

Now if i could just do something about those scratches! (aside from frame by frame restoration).

Perhaps this could assist if using Descratch, AreaOp() limit affect to an area of frame.
http://forum.doom9.org/showthread.ph...43#post1603343

[wdwms]

Originally Posted by StainlessS

Perhaps this could assist if using Descratch, AreaOp() limit affect to an area of frame.
http://forum.doom9.org/showthread.ph...43#post1603343

I tried descratch and it was a mess, the scratches are so bad that it really isn't salvageable..

[johnmeyer]

I'm always interested in before/after footage, although I wouldn't be able to watch all 30 minutes.

As for scratches, go ahead and try StainlessS' suggestion, but don't expect much. The problem with scratches is that, unlike dirt, they persists from frame to frame, so you can't use the tricks used by RemoveDirtMC which looks at adjacent frames to determine what to remove, and then fills in the removed mark with material from those frames (at least that is how I think it works).

If scratches were in precisely the same place from frame-to-frame, like a TV logo, then they could be removed using masking technology, like Delogo. However, as you well know, they tend to wander around, left to right, and also change length from one frame to the next. The only thing you can count on, because of how they are created, is that they are almost perfectly vertical.

The only solution I know of that works is to do a wet gate transfer. This involves coating the film with a slow drying solution that physically fills in the scratches. You can Google "wet gate transfer" and see how amazingly effective this technique is. You will probably find several posts by VideoFred because he uses wet gate transfer for his amazing film transfers.