Telecide and Decimate for 8mm film

[stephen22]

Just been exploring Donald Graft's amazing Decomb filter. I'm remastering old 8mm film (~18fps), pointing a video camera (25fps) at the screen.

Telecide does a nice job, and I am using Decimate to discard duplicate frames, trying to end up with (slightly less than) 18 fps, like the original.

I'm using

fname="1978-1"

loadplugin("C:\Program Files (x86)\AviSynth 2.6\plugins\Decomb.dll")
main=telecide(main)

main=decimate(main,cycle=25)
main=decimate(main,cycle=24)
main=decimate(main,cycle=23)
main=decimate(main,cycle=22)
main=decimate(main,cycle=21)
main=decimate(main,cycle=20)
main=decimate(main,cycle=19)

return main

which works well but seems pretty inefficient. Can't see how to use floating or rational number values. Is there a better way?

[videoh]

Tritical's re-working of Graft's original Telecide()/Decimate() concept can be used, as it will allow you to remove n frames from every m with one filter invocation. Tritical's decimator is called TDecimate(). You would remove 7 frames from every 25.

Graft's MultiDecimate() can also do the job but TDecimate() will be more straightforward to use. MultiDecimate() is useful when there is not a fixed pattern of duplicates, i.e., no fixed n/m, or when n/m is irrational. For example, it can globally remove all duplicates, while optionally protecting static scenes from decimation. That may produce a non-standard frame-rate, depending on the source video. Sometimes it is followed by non-standard pulldown (e.g., DGPulldown) to get a standard rate.

Tritical's TFM()/TDecimate() generally do the same job as Telecide()/Decimate() but they have more features.

manono is your go-to guy on the kind of work you are talking about.

I'm a bit confused why you need Telecide() here. I assume you record progressively. If so, then the only issue may be a one-field offset, which can be solved much more efficiently than TFM() or Telecide(). If you like to post a sample it could help us to work out the best processing.

[stephen22]

Thanks and what a kind offer.

I use an old analogue camera with my trusty DAC by Videodata, and I think the result actually is interlaced. I've had a lot of success with VD's Telecide plus Multidecimate but the latter is a bit fiddly to use.

(Does a one-field offset mean a mixture of 2 frames? I thought a field implied interlacing)

This is the kind of composite frame I need to identify and remove.

[videoh]

Thanks and what a kind offer.

I use an old analogue camera with my trusty DAC by Videodata, and I think the result actually is interlaced. I've had a lot of success with VD's Telecide plus Multidecimate but the latter is a bit fiddly to use.

(Does a one-field offset mean a mixture of 2 frames? I thought a field implied interlacing)

This is the kind of composite frame I need to identify and remove. I can't see the JPG because it is not approved yet. However, much better would be an actual video sample of your capture before any processing. You could upload it to a hosting site and post the link here. 10MB would probably be sufficient.

Be aware that if it is truly interlaced then Telecide()/TFM() are inappropriate and should not be used. Rather you should deinterlace before decimating if you want progressive output. This is why it is critical to see a video sample.

About the one-field offset. Suppose your camera captured this, where t and b stand for top and bottom fields, the capital letters denote the progressive frame they come from, and the grouping shows how they are paired in frames:

AtAb BtBb CtCb ...

Now suppose you lost the first field. You have:

AbBt BbCt CbDt ...

This will look interlaced but the progressive frames can be restored by removing the first field. Telecide()/TFM() can also rematch them but it is more efficient to simply remove the orphaned first field, as long as things don't change later in the file. If the pairing changes, then Telecide()/TFM() are great solutions because they adapt to it.

Waiting for your video sample...

[johnmeyer]

Copying movies by pointing a video camera at the movie screen is the worst possible way to transfer film. The result will flicker because each video frame captures an unknown portion of each frame. Some video frames have blends of adjacent frames, and some include the time during which the movie projector's shutter is closed. That brief portion of pure black dramatically changes the exposure for that frame, resulting in flicker.

One of the doom9.org gurus helped someone develop some recovery software that does a pretty good job of unjumbling the mess created by this sort of transfer. The person who received this help was so grateful that he created a very useful thread that you should read:

The power of Avisynth: salvaging "botched" transfers of old 8mm films to DVD (http://forum.doom9.org/showthread.php?p=1505175)

Also, here's a link to the middle of one of the long film restoration threads where you see how this script came into being:

http://forum.doom9.org/showthread.php?p=1498809#post1498809

[manono]

And as videoh mentioned, a short sample (10 seconds or so with steady movement) will be most useful. It will certainly confirm johnmeyer's suspicions.

[johnmeyer]

Now that I can see your still frame, it is certainly interlaced, as you would expect it to be. When using an old SD camcorder to capture film, you will get interlaced video. If you run your capture through a simple AVISynth script that is nothing more then separatefields(), you will see that some of your frames have two fields from the same point in time, and some have fields from different points in time. What's more some fields will contain blends and/or under-exposed fields because the projector shutter was closed. The "trick" to the script I linked to is that it figures out how to decimate individual fields that contain blends or blanks, while reassembling upper and lower fields so that you end up with the original 16- or 18-fps progressive film. That is what you need to do.

So, the fact that it is interlaced is actually a very good thing because it gives you 60 fields per second to choose from in order to create a clean 16/18 fps result. I've used the script I linked to and it is actually capable of producing a pretty decent result.

[StainlessS]

Just a little info about an alternative to Multidecimate(), MDec2() :- http://forum.doom9.org/showthread.php?t=168397&highlight=mdec2

In addition to YUY2, supports Planar (standard v2.6 planar variants) and RGB.
It still requires some of the support files from Multidecimate, namely Multidecimate.Exe and ProcessMD.Exe.
There is no source available for any of the MultiDecimate dll's exe's, was written using modified detection from FDecimate [also (c) Graft],
not exactly the same args as original MultiDecimate, but functions pretty much the same.

[stephen22]

2 files here, before = raw footage, afterx = after processing with Telecide and Decimate (script similar to the one in my first post.) I thought it did a pretty good job. I did actually delete 5 frames by hand, but using rational values with TDecimate this could presumably be automated. (I also used a deflicker filter in VD)

Incidentally can anybody explain to me why before has more kBps than afterx? Both had the same xvid settings in VD.

https://drive.google.com/file/d/0BxpFyO_R0t5JS0JxTl9UaXZDSE0/view?usp=sharing

[StainlessS]

Presumably, because there were more frames per second before hand (there will be a certain amount of overhead per frame for the avi container).
Thats my guess anyway, perhaps other could give a more enlightened answer.

[videoh]

First, I don't discount johnmeyer's approach, but if you want to stay simple as you are, then I remark as follows.

The frames do appear progressive but with blended fields. That's what produces the apparent interlacing. Usually with blended fields one uses a bob followed by srestore(). That may be worth trying. If you don't want to go that route then you can do something simple like this:

loadplugin("decomb.dll")
loadplugin("tivtc.dll")
avisource("before.avi")
fielddeinterlace(full=false,blend=false)
tdecimate(mode=0,cycleR=7,cycle=25)

Then deflicker if you think it is needed.

As I said this is far from the optimal way to handle this but if it meets your needs... I suppose my main point is that you do not need field matching for this. The reason it appears to help is that the post-processing does deinterlacing.

Note that you could replace fielddeinterlace() with TDeint() or any other adaptive deinterlacer you like.

[johnmeyer]

That link you gave is for an EXE file. I don't download EXE files from strangers, so I won't be able to help.

The various decimation links people are providing will certainly do something, but the tools created for undoing telecine must be modified heavily in order to do useful things for film capture. I did a lot of work in this area over a decade ago, long before VideoFred's scripts. My work was oriented towards recovering the original film frames from a capture done on a film projector from which I removed the shutter. I then set the shutter speed on the camcorder to 1/1000 second. You can find my posts in this forum. The bottom line is that TFM can be tuned to not only decimate but also to recombine fields that were originally in different frames (which is what you must do).

You will also find that some of the fields are blended (not blended frames, but blended fields). These must either be decimated or else replaced by motion estimated fields, using the two adjacent good fields (which are found after removing the bad fields for which good fields can be found in the adjacent frames).

Finally, the settings you will use depend very much on the speed of the original film and on the speed of the capture projector. If you are using a variable speed projector, you should probably slow that down to a fairly slow speed. The produces more duplicates and, at a slow enough speed, you are 100% guaranteed to always have two good fields from each frame of film. This simplifies the problem to one of simple decimation.

For my work, I realized that the decimation pattern was too long and too complex to ever be done by TFM alone, so I exported all of TFM metrics and then produced an Excel spreadsheet that looks forward and backwards about ten frames in order to figure out the local pattern of duplicates. Because the projector is running at (nominally) 18 fps (for Super 8, or 24 fps for a 16mm sound projector) and the NTSC camera is running at (nominally) 30 fps, but neither is running at precisely these speeds and neither of them is locked to the other, you end up with very interesting beat patterns of duplicates and blends. This is why the decimation logic has to be adaptive, and you just can't decimate using an "n in m" TDecimate logic.

[edit]P.S. I just noticed that you are using a PAL SD camera. That lower frame rate (compared to 30 fps NTSC) will make it much more difficult to do a near-perfect recovery of the original frames (your goal should be a "frame accurate" transfer). So, my advice above about slowing down the projector is actually extremely important, and you should slow it down to the absolutely minimum speed possible.

[videoh]

You can extract from the exe using 7zip. You don't have to execute the EXE.

[johnmeyer]

You can extract from the exe using 7zip. You don't have to execute the EXE.
OK, I'll try that.

[Sharc]

How does this look?
http://www.mediafire.com/file/p2vb4g493zrbzpe/stephen.ts

[Sharc]

...Finally, the settings you will use depend very much on the speed of the original film and on the speed of the capture projector. If you are using a variable speed projector, you should probably slow that down to a fairly slow speed. The produces more duplicates and, at a slow enough speed, you are 100% guaranteed to always have two good fields from each frame of film. This simplifies the problem to one of simple decimation.....
Maybe I got this wrong, but if the projector operates at 18fps and the 25fps PAL videocam takes the fields at 50 fields per second, there will be 2 clean fields recorded of every film picture. These fields are taken from the same film picture without temporal movement, hence the 2 clean fields constitute progressive video. Am I missing something?

Edit:
Ooops! I see my mistake. Considering the aperture time one will get with 18fps sampled with 50 fields per second repeatedly "orphaned" single fields in between 2 blended (to be discarded) dirty fields during which the projector advances 1 picture.
(Still, this single orphaned field can be interpolated to a clean frame).

[stephen22]

Very good! A few duplicate frames, but my projector was running at slightly under 18fps. All the combing artifact is gone. How did you do it?

[stephen22]

you can do something simple like this:

loadplugin("decomb.dll")
loadplugin("tivtc.dll")
avisource("before.avi")
fielddeinterlace(full=false,blend=false)
tdecimate(mode=0,cycleR=7,cycle=25)



That gives exactly the same as my script using telecide, making your point that it's the deinterlace post-processing that does the trick.

Actually using telecide with post=0 produces a very similar effect, with subtle differences in the residual combed frames. I have always assumed that Telecide checks each frame against the adjacent ones, and if major components of both adjacent frames are present it replaces it with one or the other. Is this a bit simplistic?

[Sharc]

Very good! A few duplicate frames, but my projector was running at slightly under 18fps. All the combing artifact is gone. How did you do it?
ffmpegSource("before.avi")
QTGMC()
denom=1.000 #adjust this for better alignment with your projector's speed
srestore(frate=18.000/denom,omode=6)
deflicker()
assumefps(18000,1000)

[videoh]

That gives exactly the same as my script using telecide, making your point that it's the deinterlace post-processing that does the trick.

Actually using telecide with post=0 produces a very similar effect, with subtle differences in the residual combed frames. I have always assumed that Telecide checks each frame against the adjacent ones, and if major components of both adjacent frames are present it replaces it with one or the other. Is this a bit simplistic? Yes, Telecide/TFM both try to pair fields into frames based on the differences between the fields. I'm a bit surprised field matching alone helps much. It could be that it doesn't do much and the worst frames get ditched by Decimate. Try comparing without the Decimate.

[stephen22]

ffmpegSource("before.avi")
QTGMC()
denom=1.000 #adjust this for better alignment with your projector's speed
srestore(frate=18.000/denom,omode=6)
deflicker()
assumefps(18000,1000)

I had already tried srestore using Fielddeinterlace instead of QTGMC. Not nearly such a good result!


I'm a bit surprised field matching alone helps much. It could be that it doesn't do much and the worst frames get ditched by Decimate. Try comparing without the Decimate.

It does actually get rid of nearly all the combing

[Sharc]

Yes, Telecide/TFM both try to pair fields into frames based on the differences between the fields. I'm a bit surprised field matching alone helps much. It could be that it doesn't do much and the worst frames get ditched by Decimate. Try comparing without the Decimate.
Hmmm... what does TFM do with orphaned fields which are in between 2 dirty fields (due to projector advancing 1 picture during the camera field aperture time)?

[johnmeyer]

Maybe I got this wrong, but if the projector operates at 18fps and the 25fps PAL videocam takes the fields at 50 fields per second, there will be 2 clean fields recorded of every film picture. These fields are taken from the same film picture without temporal movement, hence the 2 clean fields constitute progressive video. Am I missing something?

Edit:
Ooops! I see my mistake. Considering the aperture time one will get with 18fps sampled with 50 fields per second repeatedly "orphaned" single fields in between 2 blended (to be discarded) dirty fields during which the projector advances 1 picture.
(Still, this single orphaned field can be interpolated to a clean frame).It just barely works with a 24 fps projector and 30 fps camera (60 fields per second). You have to actually draw it out with a timing diagram, and remember to include the pulldown time for each film frame. During each pulldown (the time during which the film advances to the next frame) you will either get a blur (if you remove the projector shutter, as I did) or you will get pure black (if the shutter is still installed). The field which captures that mess absolutely must be discarded. If you ue a really high shutter speed while capturing, it helps to sort things out.

[johnmeyer]

Before you go ahead and process all your film, make sure to take a very critical look for missing frames. These are much harder to spot than duplicates. Go through a few hundred frames at about 1/8 normal speed and look for obvious jumps or glitches. I've spent weeks of my life doing this stuff (decimating frames from film to video transfers), and I know what to look for, and what the likely deficiencies of the scripts posted in this thread are likely to be. They should do about 90% of the job, but if you want to get a better result, you really need to try out the script in that thread I already linked to.

[videoh]

Hmmm... what does TFM do with orphaned fields which are in between 2 dirty fields (due to projector advancing 1 picture during the camera field aperture time)? For Telecide, it depends on whether the orphaned field is a top or bottom field. Telecide always keeps the top field and then matches it to the following or preceding field. So we say Telecide matches on the top fields. If I remember correctly, TFM can be configured to match on top or bottom fields. In practice that may not be so helpful if the orphaned fields are randomly top and bottom. But it may be helpful if they appear predominantly in the top or bottom.

Generation 4 Decomb (old Decomb) did a 3-way match such that it tried matching on top or bottom for each frame. Generation 5 removed that for performance improvement. I'm not aware of any real-world cases where 3-way matching improves things but I'll keep an open mind about it.

[stephen22]

The qtgmc/srestore script suggested by Mr Sharc seems to do an immaculate job - all combing is gone and in 181 frames there are a couple of duplicates but I am sure no missing frames. Amazing what these clever scripts will do (if you're prepared to wait for the processing - less than 2 frames/sec!).

Thank you all for your interest and help.

[videoh]

You could make it faster by replacing QTGMC() with another adaptive bobber. I would guess that yadif or sbdeint could do a good enough job. What say you, Sharc?

[Sharc]

You could make it faster by replacing QTGMC() with another adaptive bobber. I would guess that yadif or sbdeint could do a good enough job. What say you, Sharc?
You are most probably right. It's definitely worth to try other (and faster) adaptive bobbers.
My next step would be QTGMC(preset="fast") or QTGMC(preset="superfast"). QTGMC also helps to deflick and includes more nice things. It's just my habit to try with QTGMC when I see some "messy" footage. It often (but not always) does wonders ;)

[Groucho2004]

You are most probably right. It's definitely worth to try other (and faster) adaptive bobbers.
My next step would be QTGMC(preset="fast") or QTGMC(preset="superfast").
The default preset is "Slower" which is, well, pretty slow. Before using the fast presets I'd try "Medium" or even "Slow". From a quality point of view, this is still way better than yadif.

[Sharc]

The qtgmc/srestore script suggested by Mr Sharc seems to do an immaculate job - all combing is gone and in 181 frames there are a couple of duplicates but I am sure no missing frames. Amazing what these clever scripts will do (if you're prepared to wait for the processing - less than 2 frames/sec!).

Thank you all for your interest and help.
You may try this:
QTGMC()
denom=1.0
srestore(frate=18.000/denom,omode=6)
deflicker()
selectevery(30,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28)
assumefps(18000,1000)
It should remove the duplicates. Watch out for skipped frames with a longer test sequence.

[johnmeyer]

You may try this:

It should remove the duplicates. Watch out for skipped frames with a longer test sequence.The other thing you absolutely must do with all of these scripts is to play around with the assumed film frame speed. Sharc's uses 18.00 fps for srestore. However, you actually have no way of knowing what the correct projector frame rate was, except by counting the cadence (number of duped fields). With my setup, I have a 24 fps sound projector (a really nice Eiki) with a synchronous motor that is slaved to the 60 Hz. oscillations of the mains. Even with that, there is drift between the 29.97 fps camera and the projector. With a normal variable speed Super 8 projector, you won't even be close to 18 fps. It could be 17.35 or 18.16 ... or anything.

So, to make any of these scripts work, you have to make an effort to determine how many frames per second your projector actually produces.

I am in no way criticizing any of the scripts; I'm only saying that the person actually doing the work cannot simply take the scripts, as given, and get optimal results without quite a bit of fine-tuning.

[Sharc]

The other thing you absolutely must do with all of these scripts is to play around with the assumed film frame speed. Sharc's uses 18.00 fps for srestore. However, you actually have no way of knowing what the correct projector frame rate was, except by counting the cadence (number of duped fields). With my setup, I have a 24 fps sound projector (a really nice Eiki) with a synchronous motor that is slaved to the 60 Hz. oscillations of the mains. Even with that, there is drift between the 29.97 fps camera and the projector. With a normal variable speed Super 8 projector, you won't even be close to 18 fps. It could be 17.35 or 18.16 ... or anything.

So, to make any of these scripts work, you have to make an effort to determine how many frames per second your projector actually produces.

I am in no way criticizing any of the scripts; I'm only saying that the person actually doing the work cannot simply take the scripts, as given, and get optimal results without quite a bit of fine-tuning.
Agree. That's why I put the experimental variable "denom" in the script to account for the actual projector speed (fine tuning). Maybe there is a better way to do this? Based on my selectevery() finding I would guess that his projector speed was probably about 17.4 fps. Sort of reverse engineering .....
As you mention a short shutter speed of the videocam would help to reduce the "dirty" (black or blended) fields. On some (most?) videocams this is a setting option as far as I know.

[stephen22]

Yes it is 17.4 fps - I had already found that from the framecount using VD Telecide + Multidecimate (which has a threshold for deletion rather than an arbitrary number). And as you found, there were 6 duplicates in 181 frames at 18fps.

With this frate the results seem to be perfect!! (it's not hard at 18fps to spot missing frames) and changing the preset to "medium" or "fast" does just as good a job in half the time.

My ancient camera doesn't have many useful facilities for telecine but I have found that minimising processing by turning off aperture control, autofocus, steadyshot etc significantly improves exposure time.

[johnmeyer]

My ancient camera doesn't have many useful facilities for telecine but I have found that minimising processing by turning off aperture control, autofocus, steadyshot etc significantly improves exposure time.Yes, manual settings are mandatory, although you may still want to use automatic exposure. Here are a few quick hints, although it sounds like you figured most of this out yourself:

1. Set manual focus, manual color balance (use the indoor preset).

2. Expose for the highlights. Repeat: expose for the highlights. To do this, use the zebra settings on your camera (hopefully it has these). This is one of the most important things you can do. Most film transfers use the average exposure setting on the camera, and the highlights get totally blown out because film has a much wider gamut than video.

3. Check focus after each reel. It is really easy to accidentally nudge the focus control on either the projector or the camera while changing reels. Doing this extra focus check adds about one minute per reel, but you will get paid back the first time you avoid having to re-capture a reel that was taken out of focus.

Back to #2 for a moment. If your camera has a "spotlight" function, use it. This is a feature on most Sony camcorders designed to expose for the highlights. It usually results in slightly under-exposed footage, which is what you generally want for film transfers. You then gain the shadows in post production, while leaving the highlights alone. You can almost always retrieve detail from the shadows of an under-exposed video, but once highlights are over-exposed the detail is lost forever. Another trick is to use the auto-exposure adjustment (AE adjust is what it is usually called). This subtracts a fixed amount from whatever the autoexposure calculates. You usually want to use autoexposure because amateur film exposure often varies wildly from scene to scene (most 16mm and 8mm cameras did not have any automatic exposure; that feature was first added when Super 8 hit the market in the early 1960s), but to make sure the highlights aren't blown out, you want to subtract from the exposure. If you try to ride the manual exposure up and down yourself, you have to watch every second of the film during transfer, and even then, you'll always be way behind when the exposure changes because you just cannot react fast enough.

[stephen22]

Good advice.

(Actually I do adjust exposure manually, and if there is a sudden change I simply wind the film back a bit and do it again.)

[johnmeyer]

Actually I do adjust exposure manually, and if there is a sudden change I simply wind the film back a bit and do it again.That is definitely the best way to do it, if you have the time.