AviSynth script for removing double/bad frames

[Ericus]

Is there a script for removing double and bad frames from a video?

[johnmeyer]

I wrote this script. It may do what you want.

Finding individual "bad" frames in video; save frame number; or repair (https://forum.doom9.org/showthread.php?t=174104)

[poisondeathray]

Is there a script for removing double and bad frames from a video?

Clarify what that means

"Double frames" as in duplicates ? Or double as in "blends" ?

What are "bad" frames ? In what are they "bad" ?

If you remove them, what do you want to replace them with ? Otherwise there will be a gap . Do you care about sync ?

[johnmeyer]

You definitely do need to answer all of poisondeathray's questions. Otherwise you will not get useful advice.

In the meantime, here are two other scripts I wrote which may be of some value, depending on what your actual problem may be.

Automatically fix dups followed (eventually) by drops (https://forum.doom9.org/showthread.php?t=161758)

This is one which a lot of people use. It replaces a duplicate frame with a frame synthesized using motion estimation (a.k.a., optical flow). The number of frames stays the same so you won't get audio sync issues, something that poisondeathray asked about.

Fix duplicate frames resulting from bad analog capture (https://forum.doom9.org/showthread.php?t=150460)

[Ericus]

Clarify what that means

"Double frames" as in duplicates ? Or double as in "blends" ?

What are "bad" frames ? In what are they "bad" ?

If you remove them, what do you want to replace them with ? Otherwise there will be a gap . Do you care about sync ?

Yes, duplicates is the right word, not blends. Bad frames are frames which are jumpy or partial frames caused by digitizing old 8mm videos to digital.
When doing telecine digitizing 8mm films in 16,66fps with a camera running at 50fps there are duplicate frames as a result. I want to get rid of those duplicates as I want only one copy of each film frame. Doing it manually is quite time-consuming.

[Frank62]

Sounds regular. Then you just need TDecimate.

[poisondeathray]

Yes, duplicates is the right word, not blends. Bad frames are frames which are jumpy or partial frames caused by digitizing old 8mm videos to digital.
When doing telecine digitizing 8mm films in 16,66fps with a camera running at 50fps there are duplicate frames as a result. I want to get rid of those duplicates as I want only one copy of each film frame. Doing it manually is quite time-consuming.

If it was a decent transfer, and the duplicates looked basically the same, then TDecimate(cycleR=2, cycle=3) would drop 2 duplicates in each group of 3 . Sometimes increasing the "window" size helps , such as 4 in 6.

But "Jumpy, partial frames" might interfere with the accuracy of the selection - because "duplicates" might not be real duplicates. You might have to adjust some of the settings.

If the cadence was perfect and the "cleanest" film frame was always the same one in a group of 3, then you could use SelectEvery . e.g. if it was always the 1st in a group of 3 - SelectEvery(3,0)

If you can't get the desired results, post a sample clip and someone should give suggestions

[johnmeyer]

Yes, a sample would help a lot.

I've built my own transfer system, and purchased two others: a Goko (which I immediately sold back on eBay because it is technically a horrible transfer system) and one of Roger Evan's original Moviestuff "Workprinters." I've looked at many other transfer system, including the original "point the video camera at the movie screen and film the results." If your system does not 100% ensure "frame accurate" results (meaning exactly one frame of video for each original film frame), then you are going to have to do some serious work.

But here is my main point: the script required is going to have to be customized to the artifacts created by your particular transfer system.

What system was used to transfer these films? Did you build it? If not, what is it called?

[Ericus]

Yes, a sample would help a lot.

I've built my own transfer system, and purchased two others: a Goko (which I immediately sold back on eBay because it is technically a horrible transfer system) and one of Roger Evan's original Moviestuff "Workprinters." I've looked at many other transfer system, including the original "point the video camera at the movie screen and film the results." If your system does not 100% ensure "frame accurate" results (meaning exactly one frame of video for each original film frame), then you are going to have to do some serious work.

But here is my main point: the script required is going to have to be customized to the artifacts created by your particular transfer system.

What system was used to transfer these films? Did you build it? If not, what is it called?

I have two different types of scanning systems, first, the original WorkPrinter which I bought from Roger in 2003, and secondly, I have a Chinon 4000 projector with adjustable speed. I need the script or software to remove the duplicates with the latter system. Chinon is modified with an adjustable light system and I use it only for the films that need to be done fast and with the sound. I use Lumix 4K camera in both systems.
I am going to modify the Chinon projector to be able to use frame-by-frame shooting. Then after that, I don´t need the script anymore.

[johnmeyer]

I built my 16mm transfer system from a really nice Eiki projector. I removed the shutter and did the transfer by pointing the video camera, configured with a 1/1000 shutter speed, directly into the projector lens. I then created an extremely complicated script to remove duplicates. I use TFM to provide matching metrics. They blow up when the camera records a field during pulldown (they are blurred even at 1/1000 second). The TFM metrics also provide information for matching fields. The setup is pretty complicated because when I built it, professional quality 60 fps progressive cameras were not available, so I used my Sony FX-1 which is interlaced. I therefore have to look at fields, do the decimation of the pulldown fields, re-match the remaining fields (which involves using an upper field from one frame and a lower field from an adjacent frame to make a new frame), and then do a decimation. I can't do a simple "n in m" decimation because the pattern is not perfectly repeatable, even though my projector uses a synchronous motor and the camera is also synced to the mains 60 Hz frequency. Therefore I prototyped a complicated field matching decimation in Excel and use that to drive an AVISynth decimation program.

I had also intended to code the whole thing, but after almost twenty years I'm still using the prototype.

I too was going to modify the projector to add a sensor to tell the camera when the shutter was closed. I told Roger about my various software inventions, but he was never a software guy and wanted to do everything in hardware. I also told him about how I discovered that if you put a telephoto lens in the projector you point the video camera directly into that lens and eliminate the weird aerial lens approach he used on his early Workprinter. He did follow that suggestion, and the aerial lenses disappeared.

His hardware-only approach is much better than my complicated software but, of course, his current units cost 3-5 times what you and I paid for our Workprinters.

He told me about his sales numbers back when he was buying old projectors on eBay, modifying them, and then selling them as Workprinters. The numbers were absolutely spectacular. I was a startup consultant during the 1990s and was very familiar with early-stage financials, and his little operation was doing fantastic business.