The power of Avisynth: restoring old 8mm films. - Page 48

wdwms · 6th November 2015, 03:15

At this point we are going to leave the restoration as it is. Maybe someday I'll find a better print. But as it was, the cost was pretty high to get this xferred, and we did it via a crowdfunding effort by our followers...

johnmeyer · 6th November 2015, 03:45

P.S. Here is a link to a before/after showing the significant improvement from a wet gate transfer:

Wet Gate Transfer

Even if you don't want to re-transfer for this project, you should know about this for the future.

greg · 14th December 2015, 18:28

Hi,

I just got my reels (2 hours) of regular 8 film (recorded in the 70s with a Yashica UL Power Zoom camera) professionally transferred with the MWA Flashscan HD telecine. I'd like to clean them with Fred's script or JohnMeyer's one but I have three main problems:

-The video files I have are ProRes 422 .mov, uncompressed. I tried QTReader plugin and I opened the avs. file with VirtualDubMod, but I see the frame upside down and if I press play I get the error "Couldn't initialize timer!"

-The original filming speed was 16 fps, but now the .mov file is 25 fps progressive. I can see there are duplicate frames and I know I have to delete them with Avisynth but I can't understand which plugin I have to use (Decimate? Decomb?) and I don't know the parameters

-The video files I have are HD 16:9 so there are black pillarboxes on the sides. How can I crop them with Avisynth? Do I have to use ''AutoCrop''?

I uploaded on Dropbox some samples if you want to see them (uncompressed, same format I have on my PC).

Here's the link to the folder: https://www.dropbox.com/sh/vbe5vggru...lKi6BXgha?dl=0

If someone can help me, I'd really appreciate it.

Thanks in advance

manono · 14th December 2015, 20:51

Hello and welcome to the forum,

I used FFVideoSource (part of FFmpegSource) to open the first sample in Virtual Dub and had no problem.

I used TDecimate (part of the TIVTC filter) to remove the duplicate frames and return it to 16fps:

TDecimate(Cycle=25,CycleR=9)

I suppose you could use Autocrop, but I do such things like this:

Crop(240,0,-240,0)

That crops to 1440x1080 (1.33:1) but doesn't remove all the black. You can play around with it yourself. It's all explained in the Crop page on the AviSynth site.

johnmeyer · 14th December 2015, 22:50

[edit]My TDecimate logic (below) is wrong. Manono's (directly above) is correct.[/end edit]

I used ffmpeg to convert to uncompressed, and then imported that. You can also use ffmpeg to convert to lossless or other formats that consume less disc space. This is an example of how to convert to uncompressed:

Code:

ffmpeg -i "8mm sample_01.mov" -an -vcodec rawvideo -pix_fmt yuv420p -y output.avi

I looked at your film, and every other frame is a repeat. Thus, the pulldown that was added was NOT trying to go from 16 fps to 25 fps, but simply to get the file to play back at 25 fps. Therefore, the TFM/TDecimate parameters given in that previous post actually are not what you want to use. Instead, use this:

Code:

TDecimate(Cycle=3,CycleR=1)
AssumeFPS(16)

This will recover the original frames, so you have a frame accurate result, and then the AssumeFPS will cause those frames to be played back at 16 fps, the common speed for 8mm and silent 16mm film.

You can then add a crop, as needed:

Code:

Crop(250,0,-260,0)

This seemed to be about right for your first example (the only one I downloaded).

manono · 15th December 2015, 00:49

Quote:

Originally Posted by johnmeyer

I looked at your film, and every other frame is a repeat.

Maybe you meant to say every third frame is a repeat? Either you're not decoding it correctly, or I'm not (always possible), or you're not counting enough cycles to be sure what it is. greg said it was converted from 16->25fps with the addition of duplicate frames which is what I see. You seem to be saying one of three frames is a duplicate. I'm seeing sometimes one of four frames being a duplicate. Every eighth cycle comprises four frames and not three. You're seeing 16.67fps and I'm seeing 16fps. Here's what I get with the script I suggested:

https://mega.nz/#!ThQCGLZR!yi1UrDGj0...fkPsS0z3o6ErWI

johnmeyer · 15th December 2015, 04:00

I went back and looked at "8mm sample_01.mov" again, and I clearly didn't "walk" far enough through the clip to see the more complex pattern. I looked at fields (just in case in was telecined for interlaced viewing) and got this field pattern:

4-2-4-2-4-2-4-2-2

Translating to frames:

2-1-2-1-2-1-2-1-1

So in each group of thirteen frames, each of the "2's" must be decimated. There are 13 frames total (Cycle) and 4 that must be decimated (CycleR).

Code:

TDecimate(Cycle=13,CycleR=4)

However, after my previous error, I wanted to test this code, so I walked through the entire clip, using the above decimation.

Darn, I still found several repeats.

I am an idiot.

So the OP should definitely follow Manono's advice because I haven't had the time to walk the entire clip and pick up on what is a more complex pattern than I initially expected.

His code will exactly translate a 25 fps clip back to 16 fps, and some of you may wonder why I didn't just go with that to begin with. After all, the math is trivial (25 - 9 = 16). The reason is that I've done enough film transfers to know that the playback speed set for the frame accurate transfer (16 fps in this case) and for the playback (25 fps) don't necessarily have to reflect the speed of the original film, and the telecine sometimes first gets done, and then the playback speed adjusted, or the speed is first adjusted to something other than 16 fps, and then the telecine is done. So, sometimes these "oddball" telecines don't go according to the math.

This one did.

I'll edit my original post so no one gets misled.

manono · 18th December 2015, 08:02

I came back to have a look and something didn't seem right and I began counting again and realized I had been wrong, too. There's a large cycle of 25 frames before everything repeats again, and within the grand cycle are eight 'mini-cycles', like so:

3 3 3 3 3 3 3 4

Each number represents the number of unique frames plus a duplicate. The '3's represent 122 (unique, unique, dupe), and the '4' 1233 (unique, unique, unique, dupe). It's a different counting convention than yours, johnmeyer, so I'm not sure it makes sense to you. All those 3s and 4s total up the the 'grand cycle' of 25 frames and there are 8 'mini-cycles' which also tells you that within the 25 frame cycles there are 8 duplicate frames. The correct TDecimate line should be:

TDecimate(Cycle=25,CycleR=8)

That makes 17fps and greg can use an AssumeFPS line to slow it to 16fps as you showed earlier, if he wishes. I'm an idiot, too.

greg · 19th December 2015, 11:56

Thanks for helping me!

EDIT: Problem below solved
--------------------------------------------------------------------------------------------------------------------------------
I seem to have problems with the basics: I tried using FFmpeg source, but once opened the .avs file with the command

Quote:

LoadPlugin("C:\Program Files (x86)\AviSynth\plugins\ffms2.dll")
FFVideoSource("D:\8mm sample_1.mov")

with VirtualDubMod, I get the error

Quote:

Avisynth open failure. Evaluate: System exception - Acces Violation (C:\Users\Giacomo\Desktop\test.avs, line 2)

To install the FFmpeg source plugin I downloaded the file ''ffms2-2.22-msvc.7z'' from here https://github.com/FFMS/ffms2/releases (I got the link through this page http://avisynth.nl/index.php/FFmpegSource), then I just unzipped it and copied the files in the folder "x86" (there's a .dll) into Avisynth plugins folder.

Is that the right way to install the plugin?
--------------------------------------------------------------------------------------------------------------------------------

The other question is: how do I know that the telecine duplication cycle pattern always starts at the same point in every .mov file?
I mean, maybe in a .mov file the video starts when the cycle pattern is already halfway through, in that particular case the command

Quote:

TDecimate(Cycle=25,CycleR=8)

will not work?

Thanks again

Groucho2004 · 19th December 2015, 12:17

Quote:

Originally Posted by greg

with VirtualDubMod, I get the error

To install the FFmpeg source plugin I downloaded the file ''ffms2-2.22-msvc.7z'' from here https://github.com/FFMS/ffms2/releases (I got the link through this page http://avisynth.nl/index.php/FFmpegSource), then I just unzipped it and copied the files in the folder "x86" (there's a .dll) into Avisynth plugins folder.

Is that the right way to install the plugin?

That version (2.22) crashes for me as well with various files. Try the previous one (2.21).
Also, either put the plugin into the auto-load directory or use an explicit "loadplugin()", not both.

Edit: 2.22 doesn't crash if I use "threads = 1" in the ffvideosource() call.

greg · 19th December 2015, 13:18

Quote:

Originally Posted by Groucho2004

That version (2.22) crashes for me as well with various files. Try the previous one (2.21).
Also, either put the plugin into the auto-load directory or use an explicit "loadplugin()", not both.

Edit: 2.22 doesn't crash if I use "threads = 1" in the ffvideosource() call.

Wow thanks! I just tried the 2.21 version and it works!

Now I can open and play the .mov video in VirtualDubMod

greg · 19th December 2015, 13:56

Ok, I tried this

Quote:

LoadPlugin("C:\Program Files (x86)\AviSynth\plugins\ffms2.dll")
FFVideoSource("D:\8mm sample_1.mov")

LoadPlugin("C:\Program Files (x86)\AviSynth\plugins\TIVTC.dll")
TDecimate(Cycle=25,CycleR=8)
AssumeFPS(16)

Crop(250,0,-260,0)

and it works, I can play it in VirtualDubMod.

Now I'd like to run some Fred's scripts. What should I do to save the video in .AVI?

File->Save As... -> Video Mode: Full processing mode? Compression: Uncompressed RGB?

Sorry but I'm a total newbie

StainlessS · 19th December 2015, 14:42

Quote:

Originally Posted by Groucho2004

Also, either put the plugin into the auto-load directory or use an explicit "loadplugin()", not both.

Mobile:
You are still doing it.
Assuming yv12, select fast recompress, select eg huffyuv (yuy2) or ut_video codec (search) , then saveas.

Saving as rgb is not lossless due to conversion, and file sizes would be huge.

Groucho2004 · 19th December 2015, 15:07

Quote:

Originally Posted by StainlessS

Mobile:
You are still doing it.

Huh? What?

StainlessS · 19th December 2015, 15:53

Not you g2004, the op [EDIT: Not 'op', I meant that greg was ignoring Groucho2004 advise].

manono · 19th December 2015, 20:30

Quote:

Originally Posted by greg

how do I know that the telecine duplication cycle pattern always starts at the same point in every .mov file?

It doesn't have to. No matter where it begins, in every 25-frame cycle 8 duplicates will be removed. This assumes the other files were like the first, which was the only one I looked at. Now that you know how to do the counting, you can check them yourself. Pick any place to start and count off 25 frames and there should be 8 duplicate frames within those 25 frames.

StainlessS · 20th December 2015, 06:42

@Greg, Clarification: decimators will leave out the frames that are most similar to their previous frames, there is however
an exception in FDecimate, which just throws away the earliest frames that are below a difference threshold. FDecimate is perhaps best
avoided, as even the author is not too impressed with it.

Pol Fieldman · 25th February 2016, 23:09

Many thanks to Video Fred for sharing your Avisynth work!
it was a great help to complete my telecine DIY project.

See : https://www.youtube.com/watch?v=eZniFhJR17M

Pol

johnmeyer · 26th February 2016, 01:14

Amazingly well-done, both the hardware capture rig (wow what a lot of work!), and the various software steps. I really appreciate the excellent gamma transfer, preserving highlights, midtones, and shadows. Also, I appreciate that you didn't turn up the sharpening controls too much. Your results look very natural.

EHarlen · 26th February 2016, 16:22

Sharpening introduce halo artifacts and there is a good sharpening result in VideoFred's example clips
But I can't see any halo artifacts.How is it possible even though he don't use any dehalo filter.

14th December 2015, 22:50	#945 \| Link
johnmeyer Registered User Join Date: Feb 2002 Location: California Posts: 2,695	[edit]My TDecimate logic (below) is wrong. Manono's (directly above) is correct.[/end edit] I used ffmpeg to convert to uncompressed, and then imported that. You can also use ffmpeg to convert to lossless or other formats that consume less disc space. This is an example of how to convert to uncompressed: Code: ffmpeg -i "8mm sample_01.mov" -an -vcodec rawvideo -pix_fmt yuv420p -y output.avi I looked at your film, and every other frame is a repeat. Thus, the pulldown that was added was NOT trying to go from 16 fps to 25 fps, but simply to get the file to play back at 25 fps. Therefore, the TFM/TDecimate parameters given in that previous post actually are not what you want to use. Instead, use this: Code: TDecimate(Cycle=3,CycleR=1) AssumeFPS(16) This will recover the original frames, so you have a frame accurate result, and then the AssumeFPS will cause those frames to be played back at 16 fps, the common speed for 8mm and silent 16mm film. You can then add a crop, as needed: Code: Crop(250,0,-260,0) This seemed to be about right for your first example (the only one I downloaded). Last edited by johnmeyer; 15th December 2015 at 04:01. Reason: Pointed out error in my original post

15th December 2015, 04:00	#947 \| Link
johnmeyer Registered User Join Date: Feb 2002 Location: California Posts: 2,695	I went back and looked at "8mm sample_01.mov" again, and I clearly didn't "walk" far enough through the clip to see the more complex pattern. I looked at fields (just in case in was telecined for interlaced viewing) and got this field pattern: 4-2-4-2-4-2-4-2-2 Translating to frames: 2-1-2-1-2-1-2-1-1 So in each group of thirteen frames, each of the "2's" must be decimated. There are 13 frames total (Cycle) and 4 that must be decimated (CycleR). Code: TDecimate(Cycle=13,CycleR=4) However, after my previous error, I wanted to test this code, so I walked through the entire clip, using the above decimation. Darn, I still found several repeats. I am an idiot. So the OP should definitely follow Manono's advice because I haven't had the time to walk the entire clip and pick up on what is a more complex pattern than I initially expected. His code will exactly translate a 25 fps clip back to 16 fps, and some of you may wonder why I didn't just go with that to begin with. After all, the math is trivial (25 - 9 = 16). The reason is that I've done enough film transfers to know that the playback speed set for the frame accurate transfer (16 fps in this case) and for the playback (25 fps) don't necessarily have to reflect the speed of the original film, and the telecine sometimes first gets done, and then the playback speed adjusted, or the speed is first adjusted to something other than 16 fps, and then the telecine is done. So, sometimes these "oddball" telecines don't go according to the math. This one did. I'll edit my original post so no one gets misled. Last edited by johnmeyer; 15th December 2015 at 06:52.

19th December 2015, 15:53	#955 \| Link
StainlessS HeartlessS Usurer Join Date: Dec 2009 Location: Over the rainbow Posts: 10,980	Not you g2004, the op [EDIT: Not 'op', I meant that greg was ignoring Groucho2004 advise]. __________________ I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 20th December 2015 at 06:43.

20th December 2015, 06:42	#957 \| Link
StainlessS HeartlessS Usurer Join Date: Dec 2009 Location: Over the rainbow Posts: 10,980	@Greg, Clarification: decimators will leave out the frames that are most similar to their previous frames, there is however an exception in FDecimate, which just throws away the earliest frames that are below a difference threshold. FDecimate is perhaps best avoided, as even the author is not too impressed with it. __________________ I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

6th November 2015, 03:15	#941 \| Link
wdwms Registered User Join Date: May 2014 Posts: 19	At this point we are going to leave the restoration as it is. Maybe someday I'll find a better print. But as it was, the cost was pretty high to get this xferred, and we did it via a crowdfunding effort by our followers...

6th November 2015, 03:45	#942 \| Link
johnmeyer Registered User Join Date: Feb 2002 Location: California Posts: 2,695	P.S. Here is a link to a before/after showing the significant improvement from a wet gate transfer: Wet Gate Transfer Even if you don't want to re-transfer for this project, you should know about this for the future.

14th December 2015, 18:28	#943 \| Link
greg Registered User Join Date: Dec 2015 Posts: 4	Hi, I just got my reels (2 hours) of regular 8 film (recorded in the 70s with a Yashica UL Power Zoom camera) professionally transferred with the MWA Flashscan HD telecine. I'd like to clean them with Fred's script or JohnMeyer's one but I have three main problems: -The video files I have are ProRes 422 .mov, uncompressed. I tried QTReader plugin and I opened the avs. file with VirtualDubMod, but I see the frame upside down and if I press play I get the error "Couldn't initialize timer!" -The original filming speed was 16 fps, but now the .mov file is 25 fps progressive. I can see there are duplicate frames and I know I have to delete them with Avisynth but I can't understand which plugin I have to use (Decimate? Decomb?) and I don't know the parameters -The video files I have are HD 16:9 so there are black pillarboxes on the sides. How can I crop them with Avisynth? Do I have to use ''AutoCrop''? I uploaded on Dropbox some samples if you want to see them (uncompressed, same format I have on my PC). Here's the link to the folder: https://www.dropbox.com/sh/vbe5vggru...lKi6BXgha?dl=0 If someone can help me, I'd really appreciate it. Thanks in advance

14th December 2015, 20:51	#944 \| Link
manono Moderator Join Date: Oct 2001 Location: Hawaii Posts: 7,406	Hello and welcome to the forum, I used FFVideoSource (part of FFmpegSource) to open the first sample in Virtual Dub and had no problem. I used TDecimate (part of the TIVTC filter) to remove the duplicate frames and return it to 16fps: TDecimate(Cycle=25,CycleR=9) I suppose you could use Autocrop, but I do such things like this: Crop(240,0,-240,0) That crops to 1440x1080 (1.33:1) but doesn't remove all the black. You can play around with it yourself. It's all explained in the Crop page on the AviSynth site.

18th December 2015, 08:02	#948 \| Link
manono Moderator Join Date: Oct 2001 Location: Hawaii Posts: 7,406	I came back to have a look and something didn't seem right and I began counting again and realized I had been wrong, too. There's a large cycle of 25 frames before everything repeats again, and within the grand cycle are eight 'mini-cycles', like so: 3 3 3 3 3 3 3 4 Each number represents the number of unique frames plus a duplicate. The '3's represent 122 (unique, unique, dupe), and the '4' 1233 (unique, unique, unique, dupe). It's a different counting convention than yours, johnmeyer, so I'm not sure it makes sense to you. All those 3s and 4s total up the the 'grand cycle' of 25 frames and there are 8 'mini-cycles' which also tells you that within the 25 frame cycles there are 8 duplicate frames. The correct TDecimate line should be: TDecimate(Cycle=25,CycleR=8) That makes 17fps and greg can use an AssumeFPS line to slow it to 16fps as you showed earlier, if he wishes. I'm an idiot, too.

25th February 2016, 23:09	#958 \| Link
Pol Fieldman Registered User Join Date: Feb 2016 Posts: 2	Many thanks to Video Fred for sharing your Avisynth work! it was a great help to complete my telecine DIY project. See : https://www.youtube.com/watch?v=eZniFhJR17M Pol

26th February 2016, 01:14	#959 \| Link
johnmeyer Registered User Join Date: Feb 2002 Location: California Posts: 2,695	Amazingly well-done, both the hardware capture rig (wow what a lot of work!), and the various software steps. I really appreciate the excellent gamma transfer, preserving highlights, midtones, and shadows. Also, I appreciate that you didn't turn up the sharpening controls too much. Your results look very natural.

26th February 2016, 16:22	#960 \| Link
EHarlen Registered User Join Date: Oct 2014 Posts: 34	Sharpening introduce halo artifacts and there is a good sharpening result in VideoFred's example clips But I can't see any halo artifacts.How is it possible even though he don't use any dehalo filter.