8mm film denoising/deflickering fine-tuning

[elbennodiablo]

Hello everyone, I am filtering my parents wedding video, after a so-so capture that included double frames. I had a short discussion on the filtering already at the Video Help Forum (http://forum.videohelp.com/threads/367890-8mm-film-restoration-fine-tuning?p=2352289).

After removing the doubles with srestore, I end up with a 20 fps video, of which you can download some frames here: frames.rar (https://dl.dropboxusercontent.com/u/48820378/8mm-film-doom9/frames.rar)

I stabilized the footage for comparisement reasons, but still have the nonstabilized footage. John Meyer was so kind to suggest some filtering, which I am implementing and trying to work with these frames. This is my current script:

Loadplugin("C:\Program Files (x86)\AviSynth 2.5\plugins\Depan.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\DepanEstimate.dll")
Loadplugin("C:\Program Files (x86)\AviSynth 2.5\plugins\removegrain.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\removedirt.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\MVTools.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\MVTools2.dll")
Loadplugin("C:\Program Files (x86)\AviSynth 2.5\plugins\warpsharp.dll")
loadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\mt_masktools-26.dll")
loadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\mt_masktools-25.dll")
Import("C:\Program Files (x86)\AviSynth 2.5\plugins\RemoveDirtMC.avs")

### VIDEO

ImageSource("%03d.jpg", start=001, end=100, fps=20, use_DevIL=true)
converttoyv12(matrix="Rec709",interlaced=false)

### DENOISING AND DEFLICKERING
rest=last

calm = rest.temporalsoften(1,255,255,32,2).merge(rest,0.25)#.repair(rest,1).removegrain(11)
calm = calm.temporalsoften(1,255,255,16,2).merge(calm,0.25)

LOP = calm.bicubicresize(480,270).bicubicresize(1392,1048,1,0)
HIP = rest.bicubicresize(480,270).bicubicresize(1392,1048,1,0)
HIP = mt_makediff(rest,HIP)
mix = LOP.mt_adddiff(HIP,U=2,V=2)

sup1=rest.msuper(levels=1)
sup2=mix.removegrain(11).msuper()

bv2=sup2.manalyse(isb=true, delta=2,blksize=16,overlap=8,DCT=5)
bv1=sup2.manalyse(isb=true, delta=1,blksize=16,overlap=8,DCT=5)
fv1=sup2.manalyse(isb=false,delta=1,blksize=16,overlap=8,DCT=5)
fv2=sup2.manalyse(isb=false,delta=2,blksize=16,overlap=8,DCT=5)

rest.mdegrain2(sup1,bv1,fv1,bv2,fv2,thSAD=640)

### REMOVING DIRT
RemoveDirtMC(40,false)

### SHARPEN
ConverttoRGB32(matrix="rec709",interlaced=false)
LoadVirtualDubPlugin("F:\Losse Bestanden\Software\VirtualDub-1.10.4\plugins32\msu_sharpen.vdf", "MSUSmartSharpen", 1)
MSUSmartSharpen(2) # from 0 to 5 (mad)
converttoyv12(matrix="Rec709",interlaced=false)

frame 05 (https://dl.dropboxusercontent.com/u/48820378/8mm-film-doom9/05.jpg)
frame 44 (https://dl.dropboxusercontent.com/u/48820378/8mm-film-doom9/44.jpg)
frame 80 (https://dl.dropboxusercontent.com/u/48820378/8mm-film-doom9/80.jpg)

As you can see up here, I end up with ghosting in the image, because of the denoising step. How can this be solved? Or are the differences between the frames simply to big to be solved? Thank you!

[feisty2]

rest.mdegrain (xxx).repair (rest,mode=13)

[feisty2]

u can choose one or a combo of the following options
1 repair, like what I wrote above
2 more accurate motion estimation (blksize=4,overlap=2)
3 lower thSAD to filter out unwanted compensated motion blocks (replace mdegrain (thsad=640) with mdegrain (thsad=200) x 3 times)

[johnmeyer]

I am not sure where you got the script you posted. It doesn't look like anything that came from the VideoFred scripts for improving film. Most of the degraining portion of the script looks like it came from the script used to recover the original frames from a "botched" transfer, which is something I assume you already did. That is the wrong code to use for the video you posted.

I ran your video through both VideoFred's script and also my heavily modified version of his script and both worked as expected and produced a good result.

[elbennodiablo]

Feisty2 thanks for this addition!

Hi John, I took part of your posted script at the videohelp forum. I learnt a lot again by reading into the botched transfer topic. But I might be still combining script parts in a wrong way. Sorry for this. Good to hear that your script is working so I will give this a try! Thanks!

For my understanding, why does the deflickering in this script not work as intended, because in the original it was bobbed first?

[johnmeyer]

For my understanding, why does the deflickering in this script not work as intended, because in the original it was bobbed first?The video you posted didn't have much flicker.

Since the script you posted is the wrong script for the job, I don't want to spend time troubleshooting it, and neither should you.

[elbennodiablo]

Your modified version of Fred's film restoration script works great John! :) I get this memory error though (https://dl.dropboxusercontent.com/u/48820378/8mm-film-doom9/avisynth%20error.jpg), how can I fix this?

The only things I changed in the script, was removing all SetMTMode(): modified script (https://dl.dropboxusercontent.com/u/48820378/8mm-film-doom9/JHM%20Restoration%20based%20on%20film%20restoration%20%28revised%29_01%20-%20Benno.avs)

Thanks again for the help. Really appreciated!

[johnmeyer]

I do most of my film restoration work using VirtualDub to read the output of the AVISynth scripts, and I then encode from there. I do use MeGUI, but have never used it with any of these particular scripts. I am not familiar with the particular error message you posted.

A few things you can try.

1. Open the script in VirtualDub and see if you can get it to play without any errors. This is intended as a troubleshooting step to determine whether the problem is mostly in the script or mostly in MeGUI.

2. I used MS Word to compare your script to my original. Your changes all look OK, but one thing that might cause problems is using higher resolution. Just for grins, try changing the:

W=1392 H=1048

variables back to either NTSC or PAL SD resolution (e.g., W=720 H=480 for NTSC). This isn't meant as a solution, but as another troubleshooting step. I did a quick Google search on your MeGUI error message, and some posts indicate MeGUI issues when fed "odd" resolutions.

3. Try changing the SetMemoryMax(1000) statement to use less memory, perhaps something like SetMemoryMax(768).

Finally, while I am not a MeGUI expert, within MeGUI you can change the number of "workers" so that it doesn't consume all your cores.

[StainlessS]

Hi, dont know what version MeGUI you are using but if developer build (or recent) with "Settings/Main configuration/Improved AVS Opening" ticked,
then try with it unticked, I have had problems with that and disable it (some kind of exception I think, dont remember exactly).
The given image looks to be that of the 'Video Preview' (ie set Aspect Ratio), where it seeks to middle frame, perhaps seek is involved.

In your initial script, you should not load both v2.5 and v2.6 versions of masktools (not that it matters here).

[foxyshadis]

elbennodiablo, can you edit the screens into links?

[StainlessS]

Foxy, Right click (In Firefox) and Copy Link Location (assuming that you want the link, OR, Save Image As).

[elbennodiablo]

Hey guys, just made links out of the pictures. I adjusted the output resolution, unticked the 'improved avs opening' and changed the memory. I also replaced

source1= Avisource(film).killaudio().assumefps(play_speed).converttoYV12()

with

source1= Avisource(film).killaudio().assumefps(play_speed).converttoYV12().LanczosResize(720,480)

All did not prevent megui to mention the same error:

Megui Encountered a fatal error and may not be able to proceed. Reason: Attempted to read or write protected memory. this is often an indication that other memory is corrupt

Previewing the script works and I can scan through it, but conversion is blocked. Other scripts work fine. So it might be script related(?) I am using MeGUI_2507_x86, fully updated. AvsPmod has no troubles going through the screens. Virtualdub is unable to open the script. Strange!

[foxyshadis]

Previewing the script works and I can scan through it, but conversion is blocked. Other scripts work fine. So it might be script related(?) I am using MeGUI_2507_x86, fully updated. AvsPmod has no troubles going through the screens. Virtualdub is unable to open the script. Strange!

MeGUI has some bugs with Avisynth 2.6 (or maybe it's newer plugins?), the preview crashes all the time for me too. Oddly, VirtualDub is rock solid though. The encoding itself should go fine.

Foxy, Right click (In Firefox) and Copy Link Location (assuming that you want the link, OR, Save Image As).

More to prevent the Hueg Liek Xbox forum-busting. This old version of VBulletin doesn't have any way to resize anything.

[StainlessS]

Add a "Return Source1" immediately after "source2=Avisource(...)", does it work OK up till there (ie are the dll's loaded OK).

I was getting problems with UnsharpMask missing from warpsharp.dll (old version from 2002/2003).
Replaced with 2008 version on Wiki, works OK with original res 1392x1048 in VDMod.

EDIT: Ooops no, if AVSPmod works ok then cannot be as above. Unless you mean OTHER scripts work OK but this
one [with LoadPlugin("warpsharp.dll"),] does not.

Where does one get the 2010 version John ?
EDIT: OK John, found it in Fred's package.

EDIT:

preview crashes all the time for me too.

Try switch off "Settings/Main configuration/Improved AVS Opening" as in post #9. (EDIT: Switches itself back on again after update I think)

More to prevent the Hueg Liek Xbox forum-busting.
Yep, was not sure if that were reason.

EDIT:
elbennodiablo,
RT_Stats has RT_Debug and RT_DebugF functions [You need DebugView (google) to view output].
You could eg put either side of the Loadplugins lines

RT_Debug("Attempting Load Plugins")

# Here Load Plugin() funcs

RT_Debug("Loaded Plugins OK")


And insert additional debug lines to track how far it got before the problem bites.
NOTE, MeGUI re-opens script about 4 or five times before Preview window is shown, but VirtualDub only once.
(So in MeGUI you will see complete set of RT_Debug messages multiple times when MeGUI is working OK).

[StainlessS]

elbennodiablo
I've added these lines to script (Here: http://www.mediafire.com/download/p847l3d2w8hwidb/Frames.zip)


RT_DebugF("\n\nFILM = %s",film)
RT_DebugF("DOING PARAMETERS")
RT_DebugF("DONE PARAMETERS : Set MemoryMax")
RT_DebugF("Set MemoryMax DONE : Load Plugins")
RT_DebugF("Load Plugins DONE : Get AviSource")
RT_DebugF("Get AviSource DONE : Crop")
RT_DebugF("Crop Source DONE : Stabilizing")
RT_DebugF("Stabilize DONE : Denoise")
RT_DebugF("Denoise DONE : Sharpen")
RT_DebugF("Denoise DONE : Interpolate") # Oops
RT_DebugF("Interpolate DONE : Result1")
RT_DebugF("Result1 DONE : Result3")
RT_DebugF("Result3 DONE : Result2")
RT_DebugF("Result2 DONE : Result4")
RT_DebugF("Result4 DONE : Source4")
RT_DebugF("Source4 DONE : Stack")
RT_DebugF("Stack DONE : Eval")
RT_DebugF("Eval DONE")

and a "return Last" after the Eval.

Should produce something like this in Debugview


00000005 0.60150105 RT_DebugF: FILM = D:\FRAMES\frames.avi
00000006 0.60159677 RT_DebugF: DOING PARAMETERS
00000007 0.60190696 RT_DebugF: DONE PARAMETERS : Set MemoryMax
00000008 0.60225672 RT_DebugF: Set MemoryMax DONE : Load Plugins
00000010 0.61894488 RT_DebugF: Load Plugins DONE : Get AviSource
00000011 0.67595273 RT_DebugF: Get AviSource DONE : Crop
00000012 0.67660767 RT_DebugF: Crop Source DONE : Stabilizing
00000013 0.70261818 RT_DebugF: Stabilize DONE : Denoise
00000014 0.80626643 RT_DebugF: Denoise DONE : Sharpen
00000015 0.81613147 RT_DebugF: Denoise DONE : Interpolate # oops
00000016 0.81626463 RT_DebugF: Interpolate DONE : Result1
00000017 0.81971866 RT_DebugF: Result1 DONE : Result3
00000018 0.82288140 RT_DebugF: Result3 DONE : Result2
00000019 0.82504261 RT_DebugF: Result2 DONE : Result4
00000020 0.82685977 RT_DebugF: Result4 DONE : Source4
00000021 0.82758445 RT_DebugF: Source4 DONE : Stack
00000022 0.83403766 RT_DebugF: Stack DONE : Eval
00000023 0.83468759 RT_DebugF: Eval DONE


The Debug lines are output in script order and any Filter has its constructor also called in script order (non MT avisynth, no idea about MT).
If it gets all the way to the Eval line then it really would have to be problem in MeGUI I think.

EDIT: Denoise should read Sharpen, easily changed in linked script.
Also see NOTE, in last paragraph of previous post.

EDIT: OK, I've put it through MeGUI OK.
Here is result I got, 4 times opened in MeGUI (Before Preview, number probably depends upon settings eg source detector etc)


00000001 0.00000000 FFMS2 avs plugin: Initializing...
00000002 0.00004937 FFMS2 - avs 2.6 mode
00000003 0.11882889 RT_DebugF:
00000004 0.11884817 RT_DebugF:
00000005 0.11887187 RT_DebugF: FILM = D:\FRAMES\frames.avi
00000006 0.11896720 RT_DebugF: DOING PARAMETERS
00000007 0.11926990 RT_DebugF: DONE PARAMETERS : Set MemoryMax
00000008 0.11961066 RT_DebugF: Set MemoryMax DONE : Load Plugins
00000009 0.13258466 RemoveDirt 0.9
00000010 0.13494150 RT_DebugF: Load Plugins DONE : Get AviSource
00000011 0.19484177 RT_DebugF: Get AviSource DONE : Crop
00000012 0.19537163 RT_DebugF: Crop Source DONE : Stabilizing
00000013 0.21945123 RT_DebugF: Stabilize DONE : Denoise
00000014 0.32367137 RT_DebugF: Denoise DONE : Sharpen
00000015 0.33339244 RT_DebugF: Sharpen DONE : Interpolate
00000016 0.33352518 RT_DebugF: Interpolate DONE : Result1
00000017 0.33696708 RT_DebugF: Result1 DONE : Result3
00000018 0.34014246 RT_DebugF: Result3 DONE : Result2
00000019 0.34266844 RT_DebugF: Result2 DONE : Result4
00000020 0.34410685 RT_DebugF: Result4 DONE : Source4
00000021 0.34481767 RT_DebugF: Source4 DONE : Stack
00000022 0.35119697 RT_DebugF: Stack DONE : Eval
00000023 0.35183230 RT_DebugF: Eval DONE
00000024 0.78333014 FFMS2 avs plugin: Initializing...
00000025 0.78337759 FFMS2 - avs 2.6 mode
00000026 0.90251386 RT_DebugF:
00000027 0.90253317 RT_DebugF:
00000028 0.90255731 RT_DebugF: FILM = D:\FRAMES\frames.avi
00000029 0.90265137 RT_DebugF: DOING PARAMETERS
00000030 0.90294999 RT_DebugF: DONE PARAMETERS : Set MemoryMax
00000031 0.90328592 RT_DebugF: Set MemoryMax DONE : Load Plugins
00000032 0.91621315 RemoveDirt 0.9
00000033 0.91855925 RT_DebugF: Load Plugins DONE : Get AviSource
00000034 0.95205951 RT_DebugF: Get AviSource DONE : Crop
00000035 0.95271254 RT_DebugF: Crop Source DONE : Stabilizing
00000036 0.97634923 RT_DebugF: Stabilize DONE : Denoise
00000037 1.08068645 RT_DebugF: Denoise DONE : Sharpen
00000038 1.09038162 RT_DebugF: Sharpen DONE : Interpolate
00000039 1.09051287 RT_DebugF: Interpolate DONE : Result1
00000040 1.09484911 RT_DebugF: Result1 DONE : Result3
00000041 1.09801018 RT_DebugF: Result3 DONE : Result2
00000042 1.10007620 RT_DebugF: Result2 DONE : Result4
00000043 1.10188591 RT_DebugF: Result4 DONE : Source4
00000044 1.10271096 RT_DebugF: Source4 DONE : Stack
00000045 1.10907829 RT_DebugF: Stack DONE : Eval
00000046 1.10970640 RT_DebugF: Eval DONE
00000047 1.44251180 FFMS2 avs plugin: Initializing...
00000048 1.44255936 FFMS2 - avs 2.6 mode
00000049 1.56131518 RT_DebugF:
00000050 1.56133366 RT_DebugF:
00000051 1.56136513 RT_DebugF: FILM = D:\FRAMES\frames.avi
00000052 1.56145990 RT_DebugF: DOING PARAMETERS
00000053 1.56176162 RT_DebugF: DONE PARAMETERS : Set MemoryMax
00000054 1.56211233 RT_DebugF: Set MemoryMax DONE : Load Plugins
00000055 1.57448709 RemoveDirt 0.9
00000056 1.57684755 RT_DebugF: Load Plugins DONE : Get AviSource
00000057 1.59341848 RT_DebugF: Get AviSource DONE : Crop
00000058 1.59389615 RT_DebugF: Crop Source DONE : Stabilizing
00000059 1.61701894 RT_DebugF: Stabilize DONE : Denoise
00000060 1.72179067 RT_DebugF: Denoise DONE : Sharpen
00000061 1.73217332 RT_DebugF: Sharpen DONE : Interpolate
00000062 1.73230469 RT_DebugF: Interpolate DONE : Result1
00000063 1.73578584 RT_DebugF: Result1 DONE : Result3
00000064 1.73894203 RT_DebugF: Result3 DONE : Result2
00000065 1.74101222 RT_DebugF: Result2 DONE : Result4
00000066 1.74282789 RT_DebugF: Result4 DONE : Source4
00000067 1.74354339 RT_DebugF: Source4 DONE : Stack
00000068 1.75003195 RT_DebugF: Stack DONE : Eval
00000069 1.75066531 RT_DebugF: Eval DONE
00000070 2.15759468 RT_DebugF:
00000071 2.15761828 RT_DebugF:
00000072 2.15764475 RT_DebugF: FILM = D:\FRAMES\frames.avi
00000073 2.15773964 RT_DebugF: DOING PARAMETERS
00000074 2.15804648 RT_DebugF: DONE PARAMETERS : Set MemoryMax
00000075 2.15837908 RT_DebugF: Set MemoryMax DONE : Load Plugins
00000076 2.16456151 RemoveDirt 0.9
00000077 2.16592050 RT_DebugF: Load Plugins DONE : Get AviSource
00000078 2.17438102 RT_DebugF: Get AviSource DONE : Crop
00000079 2.17504430 RT_DebugF: Crop Source DONE : Stabilizing
00000080 2.17974997 RT_DebugF: Stabilize DONE : Denoise
00000081 2.28370476 RT_DebugF: Denoise DONE : Sharpen
00000082 2.29205322 RT_DebugF: Sharpen DONE : Interpolate
00000083 2.29218745 RT_DebugF: Interpolate DONE : Result1
00000084 2.29684854 RT_DebugF: Result1 DONE : Result3
00000085 2.29884410 RT_DebugF: Result3 DONE : Result2
00000086 2.30090427 RT_DebugF: Result2 DONE : Result4
00000087 2.30284047 RT_DebugF: Result4 DONE : Source4
00000088 2.30355978 RT_DebugF: Source4 DONE : Stack
00000089 2.30998969 RT_DebugF: Stack DONE : Eval
00000090 2.31122875 RT_DebugF: Eval DONE


The FFMS2 line is from FFMpegSource FFMS2.avsi script, in my plugins (I added a few newlines in 'RT_DebugF("\n\nFILM = %s",film)' line
as whoever writes that FFMS2 line to Debug forgot to add a newline to the end of line [messes up formatting, and I also switched on
'Forced Carriage Returns' In Debugview]).
Also, If you use Avisynth v2.58, it outputs lots of debugging stuff to the Debugview window, to avoid the v2.58 deluge,
in Debugview, "Menu/Edit/Filter/Highlight", set Include to ':' instead of '*' as RT_Debug sends ':' with all messages and
so DebugView will exclude those message without ':' in them [Above I did not set filter and so removedirt line is there even without ':'].

I think the last group above must be some kind of seek to frame 0 (for Preview Window when Imporved AVS opening switched OFF,
seeks to middle frame when switched ON) as there is no FFMS2 line shown for that group. {Some Kind of seek to Frame 0 == Re-open script
without AutoLoading plugins again, Groucho2004 could probably explain that.}
The number of groups opened when error occurs might be an indicator of what MeGUI was doing when error occurred.

Script opened via VDMod

00000001 0.00000000 FFMS2 avs plugin: Initializing...
00000002 0.00004664 FFMS2 - avs 2.6 mode
00000003 0.59475178 RT_DebugF:
00000004 0.59477013 RT_DebugF:
00000005 0.59479302 RT_DebugF: FILM = D:\FRAMES\frames.avi
00000006 0.59488684 RT_DebugF: DOING PARAMETERS
00000007 0.59518802 RT_DebugF: DONE PARAMETERS : Set MemoryMax
00000008 0.59553158 RT_DebugF: Set MemoryMax DONE : Load Plugins
00000009 0.60842794 RemoveDirt 0.9
00000010 0.61096829 RT_DebugF: Load Plugins DONE : Get AviSource
00000011 0.66322827 RT_DebugF: Get AviSource DONE : Crop
00000012 0.66387904 RT_DebugF: Crop Source DONE : Stabilizing
00000013 0.68979824 RT_DebugF: Stabilize DONE : Denoise
00000014 0.79376590 RT_DebugF: Denoise DONE : Sharpen
00000015 0.80353099 RT_DebugF: Sharpen DONE : Interpolate
00000016 0.80366617 RT_DebugF: Interpolate DONE : Result1
00000017 0.80714786 RT_DebugF: Result1 DONE : Result3
00000018 0.81030709 RT_DebugF: Result3 DONE : Result2
00000019 0.81235421 RT_DebugF: Result2 DONE : Result4
00000020 0.81419826 RT_DebugF: Result4 DONE : Source4
00000021 0.81491786 RT_DebugF: Source4 DONE : Stack
00000022 0.82208782 RT_DebugF: Stack DONE : Eval
00000023 0.82282597 RT_DebugF: Eval DONE

[elbennodiablo]

Thanks for the tips, StainlessS and foxyshadis. I am gonna try some older versions of Megui and also try an install on my 32bit laptop with XP. If that does not work, also will try the debug mode!

[elbennodiablo]

I just installed avisynth on my 32bit laptop and it works there, so I will be using this system for the conversions. The script does wonders, I am really fascinated by the quality of the filtered output! My parents will love this when they see it. :)

Tonight I will be checking the output.

Everyone here, lots of :thanks:
I really appreciate this active community and all the time spent to help other people!!