The power of Avisynth: restoring old 8mm films.

[johnmeyer]

Who has made RemoveDirtMC() and what is the difference with the original RemoveDirt()?

RemoveDirt was created by Rainer Wittmann:

Remove Dirt

RemoveDirtMC is, as you can see from the modification to your script that I posted earlier in this thread, simply a function that calls RemoveDirt after motion compensating the clip. So, it is not a different DLL, but simply a function that uses the RemoveDirt DLL.

BTW, I have some alternative documentation for RemoveDirt that was published a long time ago. It refers to an earlier version, so not all the settings are the same. However, it explains far better what the heck is going on, and may help you if you want to tweak some settings. Send me a PM and I'll be happy to send it to you.

The use of MT. I must place SetMTMode(5) on top of the script, if I understand it good. But then, where to place SetMTMode(2)?

Don't use the MT calls anywhere in the script. Instead use the SetMTMode statements. I think this is what you are doing, but your question made it sound like you might be using the MT function. This won't work in this script.

The general usage is to put SetMTMode(5) before you open the clip, and then immediately following the call to get the clip, you put the SetMTMode(2,0) statement. I have found several things to look out for when trying to make a script work with multi-threading:

1. Perfect the script without it, and make sure it works as you expect.
2. Process and save a clip without the multi-threading
3. Add multi-threading and then process and save the same clip. Carefully compare the result to what you did without multi-threading. I have seen some cases where the results are not identical because of how things are done in parallel.
4. Some imported functions (such as the LimitedSharpen function in your script) may not work reliably (they crash when multi-threaded). You may have to incorporate the function into the main script (although I am not sure whether the "Import" function is the cause of the problem) or possibly re-write the LimitedSharpen function to eliminate whatever is causing the problem. Alternatively, you can use a SetMTMode(5) call before the bothersome function, and then put SetMTMode(2,0) after it. Doing this, however, will probably significantly compromise the speed.

I had intended to try to use one of the replacement functions for LimitedSharpenFaster, perhaps the "mod" version, but ran out of time.

I would like to ask, how are you guys transfering your 8mm films to a digital format?

Two ways:

1. Workprinter
2. My own invention: High Speed Film to Video Transfer Machine

VideoFred has his own invention, and you should definitely go to his page and have a look. It is an absolutely beautiful piece of design, and the craftsmanship is spectacular.

My own invention only requires that you remove the shutter from the projector. After that, you simply capture the film using 1/500 or faster shutter speed on your video camera. The software does everything else and now works flawlessly. I can even capture sound film, in real time, using this technique.

[Undead Sega]

Wow, your machines are pretty nice at what resolution are you capturing in?

[tormento]

Originally Posted by videoFred

It looks like RemoveDirtMC() followed by MVDegrainMulti() gives realy spectacular results.

Are they scripts or functions? I searched almost anywhere but couldn't find them.

[Nightshiver]

Obviously you haven't searched hard enough. They are functions. MVDegrainMulti is from, what else, MVTools.

[johnmeyer]

Wow, your machines are pretty nice at what resolution are you capturing in?

720x480.

I know that Roger Evans (Workprinter's inventor) says you can get better results by capturing at full HD resolution (1920x1080). While I can capture in HDV (1440x1080 1.33 PAR), with my technique this doesn't produce very good results because the inter-frame compression gets hosed by the pulldown fields. The solution would be to capture the component output from my FX1, but the capture card and software needed would be over $500, even on eBay.

In theory, 720x480 should be enough to capture most detail from 8mm and Super8, but again, since I haven't tried true HD capture, I don't know. I'm quite certain that 16mm would benefit from HD capture.

Of course none of these consumer film formats are 16:9, so some of the resolution in an HD capture isn't used (black bars on the side). With 8mm, which is almost perfectly square, almost half the area of the capture isn't used. I haven't run the math, but my guess is that instead of getting 1920, you'd probably only get about 1200 horizontal pixels when capturing most of these formats. So, the real increase in resolution would be something more like going from 720x480 (345,600 pixels) to 1200x1080 (1,296,000 pixels) an increase of 3.75.

And, just to anticipate, no the whole progressive vs. interlaced doesn't mean anything because we are capturing the film while it is at rest in the projector gate, and the results obtained with interlaced video when filming a stationary object are exactly identical to progressive video. So, a 60p camera won't improve the quality. The only way to get better results would be to slow down the projector and use a still camera with megapixel resoltuion. It's been a long time since I looked into this, but most still cameras can't sustain multiple frame capture for more than a few seconds, and I'm not sure if they would breakdown from either heat or shutter wear (I think most still cameras use a physical shutter).

I guess I could take a dozen photos with my Nikon SLR and compare the results to what I get with my FX1.

Quite frankly, once you do this for awhile, you realize that resolution is the least important issue for getting a good result. The whole point of VideoFred's script is to overcome these serious issues.

The toughest thing to get right when transferring film to video: gamma. I kick myself every day for buying the FX1 instead of the Z1. These two cameras were supposed to be identical except for the XLR audio on the Z1. Wrong. The Z1 has the ability to create custom gamma curves so you can create your own look. This would let me flatten the highlights and gain the shadows.

Oh well ... next lifetime I'll do everything right.

[videoFred]

Originally Posted by johnmeyer

RemoveDirt was created by Rainer Wittmann:

Remove Dirt

Thank you!

BTW, I have some alternative documentation for RemoveDirt that was published a long time ago. It refers to an earlier version, so not all the settings are the same. However, it explains far better what the heck is going on, and may help you if you want to tweak some settings. Send me a PM and I'll be happy to send it to you.

OK.

The general usage is to put SetMTMode(5) before you open the clip, and then immediately following the call to get the clip, you put the SetMTMode(2,0) statement.

I'm experimenting with the SetMTMode() calls in the script right now.


Fred.

[videoFred]

Originally Posted by johnmeyer

The toughest thing to get right when transferring film to video: gamma.

That's one of the reasons why I use a machine vision camera. All settings can be done manual, with the capture software. Another very useful feature is the ROI (region of interest). With the ROI you can set any aspect ratio and any resolution you want, within the limits of the max. resolution of the sensor of cource.

Fred.

[videoFred]

I have made a new script with RemoveGrainMC() followed by MDegrainMulti() and some more subtile 3-step sharpening.

Example clip:


Fred.

[johnmeyer]

Wow, the newest test clip example looks absolutely stunning! The improvements to the sharpening are much better.

I tried downloading the script from the first post in this thread, but the site is down, and it doesn't look like you updated the link. I hope you will be willing to share the results of your efforts.

John

[videoFred]

Originally Posted by johnmeyer

I tried downloading the script from the first post in this thread, but the site is down

?? This site should not be down.. never. Please try again by right click on the links and then 'save target'.

I have updated the first page of this thread and the new script is available for download now. If you can not download it, I will send it to you by mail. I have made a separate *avs file from RemoveDirtMC() and I load it in the script now.

Please lower all sharpening settings if the sharpening should be to heavy. Or increase the degraining...

It looks like all filters are working fine with SetMTMode(1,0) but you can change this of cource. At the end of the script, I had to place SetMTMode(5) otherwise it crashes.... I run it on a dual core Dell, in Vista. It's still slow... 3fps.

Fred.

[Gerald1937]

Johnmeyer

Capturing 8mm film at full HD resolution (1920x1080) is possible for modest expenditure but the problem to me is the cost of storage/distribution on Blueray or practical alternatives.

Capture method used.

Canon HV20 + 43mm to 49mm step-up ring + 49mm coupling ring + Pentax 50mm f1.7 lens reversed; provides the video camera with a high quality macro facility. The video camera HDMI sensor output is used to feed into a Blackmagic Intensity card to capture a series of sequentially numbered BMP images at full HD resolution. Each BMP image is 6MB so sustained capture rate is limited by HDD write speed. Typically my SATA II drives as measured using the Blackmagic disk test utility indicate 30 to 60 MB/s. There are other delays within the video camera and frame capture software due to the asynchronous relationship with the film transport. Typically my system provides an average capture rate of 2.7 fps measured over 500 frames using a 60MB/s partition on my SATA II drives.

The film transport system should ideally provide repeatable positional accuracy of the film frame to frame to within 1 pixel. The picture height of a Standard 8mm film is 3.68mm covered by 1080 pixels so 1 pixel = 0.0034mm. This level of accuracy is difficult to achieve and measure. Initially to check my own design of film transport mechanism I used Xnview's > View > display colour information of X and Y values as a measurement tool to accurately locate the sprocket hole edge i.e. the transition to full white RGB all 255. Fringing at the sprocket hole edge is a problem and detracts from the accuracy of this method of measurement. Subsequently I have written a program to automate the measurement over 50 or more frames. This indicates my overall film transport system is within +5 to -5 pixels. Elimination of vibration and film winder take-up during frame capture is also needed to get top quality images. I have no experience of testing modified film projector systems that VideoFred and your Workprinter use. The image taken should be of the emulsion side of the film to obtain the best quality. The resultant images taken are upside-down and 16:9 format so I crop the image to 1440x1080 to get 4:3 format and then flip it vertically before doing the fps change. Finally I add back the borders left and right to restore the 1920x1080 format. I think this order of processing reduces the number of pixels processed to a minimum.

The series of BMP frames are then assembled and processed for PAL using this Avisynthh script:

# /LOADING THE PLUGINS...
LoadPlugin("F:\AviSynth 2.5\plugins\mvtools2.dll")
#
# STARTING THE SCRIPT
# Load the sequence of 500 bmp images: the end number needs to be changed for more or less !!!!!
ImageSource("O:\frames\frame%06d.bmp", start = 0, end = 499, fps = 16, use_DevIL=false) converttoyv12
# Crop the image and flip before the speed change from 16 fps to 25 fps to reduce CPU load?
crop(240, 0, -240,0)
# FlipVertical to get the image right way up.
FlipVertical()
# Do fps change
super = MSuper(pel=2)
backward_vec = MAnalyse(super, isb = true)
forward_vec = MAnalyse(super, isb = false)
#
MFlowFps(super, backward_vec, forward_vec, num=25, den=1, ml=100)
# Add the borders left and right to restore the 1920 x 1080 format.
addborders(240, 0, 240, 0)
#

Open the AVS into Virtualdub then output a Huffyuv compressed avi file for subsequent editing with Premiere Pro 2. The edited version is then exported back to tape in the HV20 and a copy of the HDV2 file created by Premiere Pro as part of the export to tape is kept by renaming it before PPro deletes it. The export to tape process has converted the avi to a HDV (1440x1080 1.33 PAR) format so there will be some degradation in quality. The playback from HV20 HDMI output to Panasonic Viera TV HDMI input shows the capture to best advantage. The observed displayed quality is good and did not show significant signs of film grain from the Kodachrome II film used even from Standard 8mm film BUT is it that much better than a PAL 720x576 transfer from the same film to justify the cost, complexity and time to effect such a transfer to HD? The technical challenge for me was to investigate and know how I could make a successful HD transfer. I doubt if I will be making many more HD transfers, as my SD transfers suffice to bring back the memories of the people and places that others and I filmed. You can view an SD transfer made on this type of system at the www.vtoldboys.com on the LHS of the page click on Trips > Christchurch 1974 > Stan’s film and The Band buttons at the bottom of the page.

Gerald

[Boulder]

Have you guys (all of you doing these projects) given any thought to using DGPulldown? I got 18fps out of my source so it was easy to first encode to MPEG2 using HCEnc (the latest 024 beta from April 2010) with AssumeFPS(25) in the script and then running the result through DGPulldown doing a 18fps --> 25fps pulldown. Lots of bits saved and all those problem rising from framerate changes avoided I just had to restrict the max GOP length to 10 frames for DVD compliance.

[johnmeyer]

The observed displayed quality is good and did not show significant signs of film grain from the Kodachrome II film used even from Standard 8mm film BUT is it that much better than a PAL 720x576 transfer from the same film to justify the cost, complexity and time to effect such a transfer to HD?

One of the first things I would do is capture one minute of film using your technique, and then capture the same one minute simply using the DV output of the HV20. Do the post production in the same way for both. I expect that the differences with 8mm and Super8 will be very, very slight, especially if you use any of the post-processing described in this long thread.

Capturing 8mm film at full HD resolution (1920x1080) is possible for modest expenditure but the problem to me is the cost of storage/distribution on Blueray or practical alternatives.

I'm not sure what you mean by this. Aren't Blu-Ray blanks pretty cheap now?

The video camera HDMI sensor output is used to feed into a Blackmagic Intensity card to capture a series of sequentially numbered BMP images at full HD resolution. Each BMP image is 6MB so sustained capture rate is limited by HDD write speed.

There are ways you can capture direct to Cineform or the the Blackmagic intermediate. This will require far less disk performance, and will avoid the step of assembling video from the individual BMP files. I asked about this in the Sony Vegas forum awhile back:

Intensity Pro Cineform FX1 Non-RAID

The film transport system should ideally provide repeatable positional accuracy of the film frame to frame to within 1 pixel.

Yes this is a desirable goal, but darn near impossible with anything other than a Rank Cintel. Gate weave (which results from the fact that no projector provides "repeatable positional accuracy") is part of the film experience. A much simpler way to deal with this is to do small motion compensated stabilization on the resulting capture. This is something VideoFred included in his script via the Depan plugin.

BTW, I note that you are setting the film playback speed using MFlowFPS, something that VideoFred does in his script as well. This is an interesting thing to do, and I am not entirely against it. However, everyone who uses it needs to understand that it WILL introduce artifacts, and these can sometimes be very strong and obnoxious. The more usual way to handle the speed conversion when showing film on a television set is to add pulldown. When starting with 24 fps and going to 25 fps PAL interlaced, many companies simply speed up the film (i.e., change the header flag in the video file). Thus, the frames themselves are not touched. When going from 15, 16, or 18 fps film, this is where pulldown is used. For instance, to show 18 fps film on a 29.97 NTSC television, I use this code to add pulldown:

separatefields()
selectEvery(6, 0,1, 0,3, 2,3, 2,5, 4,5)
weave()
AssumeFPS(29.97, true)

[manono]

Originally Posted by johnmeyer

For instance, to show 18 fps film on a 29.97 NTSC television, I use this code to add pulldown:

separatefields()
selectEvery(6, 0,1, 0,3, 2,3, 2,5, 4,5)
weave()
AssumeFPS(29.97, true)

I think much more efficient from an encoding point of view would be:

ChangeFPS(19.98)

followed by DGPulldown set for 19.98->29.97 after the encoding is complete. All encoding will be of progressive frames with many fewer of them, resulting in a much higher quality result for the same file size. I do this quite often for silent films.

[johnmeyer]

Originally Posted by manono

I think much more efficient from an encoding point of view would be:

ChangeFPS(19.98)

followed by DGPulldown set for 19.98->29.97 after the encoding is complete. All encoding will be of progressive frames with many fewer of them, resulting in a much higher quality result for the same file size. I do this quite often for silent films.

I am not sure I follow the logic of this. The AVISynth documentation states that ChangeFPS "changes the frame rate by deleting or duplicating frames." I added the emphasis.

You never want to duplicate entire frames if instead you can duplicate just one field. Duplicating frames instead of fields substantially increases the stutter.

The beauty of pulldown is that it duplicates fields. This isn't some esoteric distinction, but instead is fundamental to making film look good on video.

In traditional 3:2 pulldown (the pattern used to get 24 fps progressive to 29.97 interlaced), the first frame of film is transferred to one frame of video. But, the next frame of film is copied not only to the next frame of video, but to the first field of the video frame after that. The next frame of film (the third in this sequence) starts by filling in the remaining field of video and then the second field is copied to the first field of the next frame of video. If you merely duplicate entire frames of film, you end up with a very jerky result.

There are dozens of excellent explanations on the web. Here is an illustration I found that shows what I'm describing:

Film Pulldown Illustration

For those who think this will give you blended frames -- a common misperception -- this doesn't happen. The reason is that you have to realize that the whole process of displaying fields is continuous, and it really doesn't matter if a field belongs to a certain "frame" or not, as long as both fields of a given frame of film get displayed in the proper order. Thus, when developing pulldown patterns for various "odd" film speeds, you need to make sure that the fields in the selectevery statement always alternate even-odd-even-odd (never have two even fields or two odd fields in a row). And, you never want to skip ahead beyond the next frame.

There is no "right" or "wrong" way to do these conversions from frame rates other than 24 fps. The solution that VideoFred uses, of actually creating new frames using MFlowFPS, produces some very interesting results. I've experimented with this and, as I indicated in my previous post, thought that it produced too many artifacts. However, I have found that if I use large block sizes, and only change from 18 fps progressive to 24 fps progressive, the results are generally quite good. I then encode this to DVD using 23.976 with the pulldown flag set. If it is played on an NTSC TV, the DVD player adds the pulldown during playback. However, if it is played to a non-scanning device (like an LCD TV) and if the DVD player provides progressive output, you will get the 24fps result without further pulldown.

When I developed the scripts to convert kinescopes back to "real" video, I needed to go all the way from 24 fps to 60i. If you do all the right things (degrain, deshake, gamma correction, etc.) prior to the fps conversion, the result really does look like "real" video. However, when starting with something that really is film, I'm not sure this is what most people want. This, of course, is entirely an aesthetic decision, and, like letterboxing or colorizing movies, there will be people who come down on both sides of the argument.

[manono]

You're duplicating frames as well. Play your encoded script any way you want (except by blend deinterlacing which creates something much worse) and the deinterlacing will create duplicate frames, more than I have. Anything below 19.98fps (for NTSC) by necessity creates duplicate frames.

The beauty of pulldown is that it duplicates fields.

That's what DGPulldown does, and it doesn't encode them into the video stream, thus much better encoding quality.

[johnmeyer]

Originally Posted by manono

You're duplicating frames as well. Play your encoded script any way you want (except by blend deinterlacing which creates something much worse) and the deinterlacing will create duplicate frames, more than I have. Anything below 19.98fps (for NTSC) by necessity creates duplicate frames.

That's what DGPulldown does, and it doesn't encode them into the video stream, thus much better encoding quality.

I'm afraid neither thing you say is entirely correct, although perhaps it depends on how you define "duplicating frames." To me, duplicating frames means that I display the entire frame twice in a row. This is NOT what 3:2 pulldown does. I already explained in my last post what is really going on, so I won't repeat that here.

I've tested all my scripts by adding a separatefields() statement at the end, and then looking, field-by-field, at what the script has produced.

Now, it is true that if you get to a slow enough starting frame rate, like 15 fps, then you are 100% correct: to get to 30 fps, each frame is duplicated twice. And it is also true that when going from 16 fps or 18 fps, [I]some[I] frames do get repeated. But, some do not. So you are partly right because some frames get repeated in their entirety, but not all of them.

Now as for DGPulldown, I just went to Donald Graft's site to make sure I understood what DGPulldown does:

DGPulldown

To quote directly from that page:

"DGPulldown works by altering the MPEG2 RFF/TFF flags to increase the number of display frames to exactly the right number to generate the specified output frame rate during playback."

So, this plugin is only useful when encoding to MPEG2, and it does its magic by changing the pulldown flags and not by actually altering the video. For it to work, the playback device must understand what to do with these flags. If encoding for DVD, there is a very narrow limit to what flags can be used, and I don't think this plugin would be useful.

[manono]

Originally Posted by johnmeyer

I'm afraid neither thing you say is entirely correct,

Everything I've said is entirely correct.

This is NOT what 3:2 pulldown does.

Who's talking about 3:2 pulldown? That's for 23.976->29.97. This is 19.98->29.97. It's 3:3 pulldown.

And it is also true that when going from 16 fps or 18 fps, some frames do get repeated.

Which is what I said. Nowhere did I say they all get repeated. What I said was that with your script more get repeated than with 'my' method ('my' because I didn't invent it. neuron2 gets credit for DGPulldown).

So, this plugin is only useful when encoding to MPEG2

Isn't that what you're doing here - providing a script for encoding progressive 18fps sources as interlaced 29.97fps for DVD?

...it does its magic by changing the pulldown flags and not by actually altering the video.

Well, it sets the flags, similar to the way applying 3:2 pullldown sets the TFF/RFF flags.

If encoding for DVD, there is a very narrow limit to what flags can be used, and I don't think this plugin would be useful.

Oh, it's very useful. Its greatest use, perhaps, is for PAL to NTSC conversions without altering the audio. It allows the video to stay the same length and progressive. But it's also very useful for many other kinds of unusual framerates (I used the example of silent films earlier), allowing them to be encoded progressively, rather than as interlaced. It's 100% DVD compliant, plays properly on almost all software and hardware DVD players, and if you'd open your mind just a little bit you may find it useful even in your work.

[videoFred]

Originally Posted by johnmeyer

The solution that VideoFred uses, of actually creating new frames using MFlowFPS, produces some very interesting results. I've experimented with this and, as I indicated in my previous post, thought that it produced too many artifacts. However, I have found that if I use large block sizes, and only change from 18 fps progressive to 24 fps progressive, the results are generally quite good.

The artifacts created by MFlowFPS() are way less when using RemoveDirtMC() before. MFlowFPS() is great for low motion scenes. Nice smooth panning scenes etc. On high motion scenes it fails sometimes.

I might be wrong, but is the future not HD progressive? I have done some tests with 720p30 and even 720p60. Very simple with duplicate frames. AssumeFPS() followed by ChangeFPS(). This plays remarkable good on a fast computer.

But for a nice experiment it is possible to create extreme slow motion with MFlowFPS(). Try AssumeFPS(1) followed by MFlowFPS(num=30) for example.

Fred.

[Gavino]

Originally Posted by johnmeyer

Thus, when developing pulldown patterns for various "odd" film speeds, you need to make sure that the fields in the selectevery statement always alternate even-odd-even-odd (never have two even fields or two odd fields in a row). And, you never want to skip ahead beyond the next frame.

Note that instead of working out the pulldown pattern, you can just use ChangeFPS to do it for you - much simpler, especially for 'odd' film speeds. See here.