Filter debate: Averaging multiple captures and blur filters

[fasttimes]

OK, so there is some debate over the use of Blur, and the usefulness Layering/averaging multiple VHS captures. I suggested that both are valuable tools, when used correctly. This post will mainly deal with the issue of layering; in particular, the usefulness of multiple capture of VHS tape, even from the exact same VHS tape. But, I'll get into the use of blur, as well. I hope to spark some debate and exploration of these issues.

Some have said that you will only be re-capturing the exact same analog tape noise with each pass, thus negating any averaging that could be accomplished. However, after testing, it is clear that multiple captures, of even the same VHS tape, does make an notable improvement in the final MPEGs quality! This makes sense, as any analog format never play back in exactly the same manner each time, and these minor differences create lots of high frequency noise that can averaged out (and make MPEG compression more efficient.) This is in addition to the normal noise/static, that gets picked up from every point in the chain, from Tape->capture card DAC, during a capture. All that can be averaged out as well.

If you would like all the sample JPGs of all steps, for review, you can download them as a self-extracting RAR at the following URL. If you have WindowsXP (maybe other versions) Picture and FAX Viewer (installed I think if you install the FAX features,) you can select as many pictures at you want in Window Explorer (use the CTRL and/or SHIFT key,) then right click and select "Preview" from the menu. You can then toggle between the selected images with the keyboard arrow keys! Makes it very simple to compare different images to eachother.
http://www.fasttimesatridgemonthigh.org/vidcap/files/filtersamples.exe (1.2MB) All JPGs were made in PaintShop Pro, with a very low compression setting of "5".

First, I'll list the filter chain that I used:

I captured the exact same footage, from the exact same VHS tape and VCR, three times, using the lossless HuffYUV codec. More passes should improve these results even more. In theory, 4 captures should double the random noise reduction of two captures, and 8 captures should double 4 captures, ect.

Please view L1SourceCap1.jpg, L1SourceCap1.jpg, L2SourceCap2.jpg, and S1SourceCap.jpg (same cap as L2SourceCap2.jpg) for a sample of the condition of the raw, unaltered VHS captures. Take special note of L1SourceCap1.jpg for the strange blue streak, in the bottom video tear region, that is not there on the other two. Also take note of the thin white lines, just above the video tearing.

After trimming all raw caps to the same starting position and length (using VDub's direct stream copy), I ran each AVI through Decomb's IVTC, using the following AVISynth AVS script, and saved each out to a new AVI:

LoadPlugin("decomb.dll")
AVISource("c:\layer[1/2/3].avi")
Telecide(guide=1, post=false)
Decimate(cycle=5)

Now that I had each capture as progressive 29.976FPS material, I created the layered source cap with the following AVS, and saved it out to a new AVI:

layer1=avisource("c:\layer1.avi")
layer2=avisource("c:\layer2.avi")
layer3=avisource("c:\layer3.avi")
Interleave(layer1,layer2,layer3)
TemporalSoften(1,255,255)
SelectEvery(3,1)

Please see L4CapsLayered.jpg for a sample of the result of the layering. Note that the blue streak in the bottom video tearing is gone, as well as the thin white lines, just above this area. Also, take a look at the jagged scan lines in the left man's hair, the right man's apron, and the upper section of the "black bar" along the left side. And, take a look at the NBC TV logo, that is over sharpened on the individual caps. Even with only three layers, it is a much cleaner image. The is a slight bit of blur imposed, but this is caused by the slight timing differences between each individual capture, and would not be there with a more stable video source than VHS tape. The worst of this blur can be seen in the left man's tie, which loses it's fine detail.

Now that I had my Layered Cap, I picked "layer2.avi" as the Single Cap to compare against the layered one, and renamed it "single.avi". "Single.avi" was the mid-sized AVI of the three captures, implying it had about the average amount of noise of the three captures. You can also compare the sample JPEG file sizes, showing the Single sample being 31.77% larger than the layered (layered being 74.74% smaller):

Stage_________________Size__Compres%__Layered vs Single Compres%
Layer 1 Source_______125119__100.000
Layer2/Single Cap____123227___98.488__131.77
Layer 3 Source_______121800___97.347
Layered Caps_________106124___84.818___74.74

Now, both Layered.avi and Single.avi went through the exact same filter chains. First, ran both Layered.avi and Single.avi through AVISynth's Convolution3D using the following AVS and saved the results to a new AVI.:

LoadPlugin("Convolution3d.dll")
Convolution3d(AVISource("c:\[filename].avi"),0, 3, 64, 6, 64, 10, 0)

Please see L5Convolution3D.jpg and S2Convolution3D.jpg for samples of the results. Take a look at how much cleaner the layered version is! Also take a look at the left man's tie. C3D blurred those fine details out on both versions, so losing it when I did the layered capture cost nothing over not layering! I think these results alone are very clear proof, that multiple captures of the exact same VHS material, has a very positive influence on quality! If you don't agree, please speak up as to why you disagree. Here is the file size data for the sample JPGs, showing the single version being 12.91% larger than the layered results:

Stage_________________Size__Compres%__Layered vs Single Compres%
Layered C3D___________93517___74.742___88.57
Single C3D___________105589___84.391__112.91

I'm really amazed at the results at this stage, and might have just encoded the layered results right here. However, I continued to process a little more, to get some additional noise reduction and a wee more sharpness.

Finally, both Layered.avi and Single.avi were fed to VirtualDub using the following AVS, that applied a slight blur, in preparation for WarpSharp in VirtualDub:

source=AVISource("C:\[filename].avi")
Blur(source,.50)

Please see L6BlurAdded.jpg and S3BlurAdded.jpg for samples of the results. Both are a tad blurry (duh!,) but most of that will get fixed next. As always, the layered caps results are much cleaner than the single cap. The single layer JPG is still 7.57% larger than the layered results:

Stage_________________Size__Compres%__Layered vs Single Compres%
Layered Blur__________83161___66.466___92.96
Single Blur___________89458___71.498__107.57

Now with the blur AVS scripts loaded in VirtualDub, I applied WarpSharp 1.1 at a strength of 32, then Border Control to matte 8 pixels all around the image, and saved new final AVIs. Please see L7WarpSharp&BorderControl.jpg and S4WarpSharp&BorderControl.jpg for a sample of the final results. Comparing the JPG file sizes shows the single cap version still 6.45% larger in size:

Stage_________________Size__Compres%__Layered vs Single Compres%
Layered WS&BC_________81054___64.782___93.94
Single WS&BC__________86286___68.963__106.45

Last, but not least, I compressed both of the final AVIs in Cinema Craft Encoder SP (CCE SP) v2.66.01.07, using the following settings, and bitrates:

- 1-pass VBR Min=2Mb/s Max=15Mbs (not DVD compliant)
- Q settings of 3, 15, 30 and 60
- All built-in filtering disabled
- GOP 3/5
- Quantization matrices set to MPEG Standard
- Quantizer Characteristics set to 16
- Sequence End Code and DVD Video Compliance unchecked
- Zigzag Block Scanning Order

Please see L8FinalMPEG.jpg and S5FinalMPEG.jpg for sample screen captures, taken with PowerDVD. You can see the additional noise, in the single cap version. That little bit of extra nose may not seem like much, but when the MPEGs are played back at full speed, there is a big difference in visible noise! Here is the screen capture file size comparison, showing the 5.11% increased file size. That is all additional noise!:

Stage_________________Size__Compres%__Layered vs Single Compres%
Layered Final MPEG____89875___71.832___95.14
Single Final MPEG_____94470___75.504__105.11

Lastly, I encoded MPEG-2 files at various 1-pass VBR rates-- Q setting 3, 15, 20, and 60. At Q3, there is only a 2.14% difference, but this is due to both of them getting capped at the max bitrate. At Q15, Q30, and Q60, the single cap is larger by 20.11%, 19.43%, and 26.1% respectively. Those are huge bitrate savings, and the only difference was using layering!:

Stage_________________Size__Compres%__Layered vs Single Compres%
Q3 Layered_________41477352_____________97.9
Q3 Single__________42366568___________102.14
Q15 Layered________26543604____________83.26
Q15 Single_________31881288___________120.11
Q30 Layered________15899872____________83.73
Q30 Single_________18989616___________119.43
Q60 Layered_________8199844____________79.30
Q60 Single_________10339812___________126.10

If you would like to see what the final MPEG videos look like, you can download them from the following links:
http://www.fasttimesatridgemonthigh.org/vidcap/files/layered60.m2v (8.9MB)
http://www.fasttimesatridgemonthigh.org/vidcap/files/single60.m2v (10.1MB)

My conclusions:

There are several things that I think I have been proven by my testing. The following only relates to VHS material, as I have not tested yet with higher quality source material. If you disagree with anything, please speak up, so I can do further testing to address your points.

First, that layering (averaging) clearly helps reduce noise, increases the effectiveness of further post-process filtering, and increases compression by 20-25%.

Second, that blur filters, when used correctly, are not evil, and can have a positive effect on both noise reduction, and image quality.

And last, that Convolution3D is one kick-ass filter! I find the defaults way too heavy (like most filter defaults) but, works well when toned down. I've only played with it for about a day, so I still have lots to learn about using it to best effect.

I'd like you to play around, comparing the different samples I provided, for some interesting insight. For example, compare any of the source caps with L8FinalMPEG.jpg, and see if the blur filter hurt the quality (and be prepared for a shock.) Or compare L5Convolution3D.jpg with L7WarpSharp&BorderControl.jpg, for the amount of noise reduction vs the tiny bit of blur. There are lots of ways to compare the samples to each other.

The time to do the this testing, and write this message was about 12 hours (most of it was writing this damn message,) but I think it was time well spent. The issues of filtering is probably one of the most difficult aspect of desktop video (especially analog video) to do learn to do effectively, and at the same time, is the most important factor in image quality! There are very few resources on the net the talk in-depth about filtering, and I hope the doom9.org forums will consider creating a forum, just for this specialized subject.

In closing, I'd like to invite everyone to try these tests for themselves, on your own footage. The only needed software is VirtualDub, AVISynth v2.x, HuffYUV, Decomb AVISynth plugin, and Border Control and WarpSharp 1.1 VirtualDub plugins. Then, just use all the sample AVS scripts, above. Here are software links:

VirtualDub http://www.virtualdub.org
AVISynth http://www.avisynth.org
HuffyYUV http://math.berkeley.edu/~benrg/huffyuv.html
AVISynth and VirtualDub plugins http://shelob.mordor.net/dgraft/

Hope you found this interesting and helpful. Let the debates begin. :)

Update:
I thought I'd also compress the raw original, unfiltered/unprocessed "Single.avi", for comparison with how much the filtering improved compressibility. Below is MPEG-2 file size of the unprocessed Single.avi vs the layered and processed version. All CCE encoder settings were the same as mentioned previously:

Stage Size Processed vs
Q3 Layered 41477352 Unprocessed Compres%
Q15 Layered 26543604 59.22
Q15 Single Unprocessed 44824240 168.87
Q30 Layered 15899872 41.98
Q30 Single Unprocessed 37877524 238.23
Q60 Layered 8199844 42.84
Q60 Single Unprocessed 19142548 233.45

Ouch! Unprocessed is 69%, 138% and 133% larger. Also notice, a Q3 encode on the layered and processed source for less bitrate as a Q15 encode of the raw footage!

[CableCat]

Wow, good work. I will try to do my own 3 time capture of one VHS source.

I was thinking if there is a way to use a median filter instead of a blur filter over the 3 layers.

[fasttimes]

The main problem, is getting multiple good captures of VHS material, without any frame drops. VHS has very unstable timing, and is prone to drops.

In theory, you should be able to "realign" the frames to compensate for the occasional frame drop, but I'm not good enough with AVISynth to know how to write that script!

[fasttimes]

Cool. Nobody disagrees with my findings. :devil:

BTW, here is my personal "universal" script for VHS source material. It can work with a single or multiple source(s), the capture(s) can be trimmed to match, and 2 seconds black trailer/leader is added (recommended for DVD encoding).

The footage can be optionally layered (multi caps), IVTCed, color corrected/tweaked, filtered (add the ones you like to use), resized to 1/2-D1 (with correct aspect ratio), and bottom head switching noise matted. Just remove the leading left "#" to enable an option.

Watch out, because a few of lines wrap around in this post, and should be fixed!

PS: AVISynth scripts are a pain in the butt! I was playing around for hours, trying to make everything work the way I wanted it to!


LoadPlugin("C:\Program Files\AVISynth2\plugins\Convolution3d.dll")
LoadPlugin("C:\Program Files\AVISynth2\plugins\DecombLegacy.dll")
LoadPlugin("C:\Program Files\AVISynth2\plugins\WarpSharp.dll")

### AVI SOURCE(S) ###
cap1=Trim(AVISource("[DRIVE]:\[PATH]\[FILENAME].avi"),0,0) # Single/First AVI (Audio Source)
#cap2=Trim(AVISource("[DRIVE]:\[PATH]\[FILENAME].avi"),0,0)
#cap3=Trim(AVISource("[DRIVE]:\[PATH]\[FILENAME].avi"),0,0)
#cap4=Trim(AVISource("[DRIVE]:\[PATH]\[FILENAME].avi"),0,0)
#cap5=Trim(AVISource("[DRIVE]:\[PATH]\[FILENAME].avi"),0,0)
#cap6=Trim(AVISource("[DRIVE]:\[PATH]\[FILENAME].avi"),0,0)
#cap7=Trim(AVISource("[DRIVE]:\[PATH]\[FILENAME].avi"),0,0)
#cap8=Trim(AVISource("[DRIVE]:\[PATH]\[FILENAME].avi"),0,0)
#cap9=Trim(AVISource("[DRIVE]:\[PATH]\[FILENAME].avi"),0,0)
last=cap1

### LAYER CAPTURES ###
#Interleave(cap1,cap2,cap3).TemporalSoften(1,255,255).SelectEvery(3,1) # 3 Layers
#Interleave(cap1,cap2,cap3,cap4,cap5).TemporalSoften(2,255,255).SelectEvery(5,1) # 5 Layers
#Interleave(cap1,cap2,cap3,cap4,cap5,cap6,cap7).TemporalSoften(3,255,255).SelectEvery(7,1) # 7 Layers
#Interleave(cap1,cap2,cap3,cap4,cap5,cap6,cap7,cap8,cap9).TemporalSoften(4,255,255).SelectEvery(9,1) # 9 Layers

### ADD 2 SECOND BLACK LEADER/TRAILER ###
BlankClip(cap1,60)+last+BlankClip(cap1,60)

### INVERSE TELECINE ###
#Telecide(guide=1,post=false,chroma=true).Decimate(cycle=5,mode=0,quality=3)

### COLOR CORRECTION / PICTURE TWEAKING ###
#Levels(0,1,255,0,255) # (int input_low, float gamma, int input_high, int output_low, int output_high)
#Tweak(0.0,1.0,0.0,1.0) # (float "hue", float "sat", float "bright", float "cont")
#ConvertToRGB32().RGBAdjust(1.0,1.0,1.0,1.0).ConvertBackToYUY2() # (float red, float green, float blue, float alpha)

### FILTERING - VHS - PROGRESSIVE/IVTC/DEINTERLACED SOURCE ###
#Convolution3D(0,3,64,6,64,10,0) # Light
#Convolution3D(0,3,128,16,64,10,0) # Heavy
#Blur(.50) # Set to .25-.50
#WarpSharp(depth=60,blur=1,bump=128) # (int depth, int blur, int bump,float cubic)

### FILTERING - VHS - INTERLACED SOURCE ###
#SeparateFields()
#Convolution3D(0,3,64,6,64,10,0) # Light
#Convolution3D(0,3,128,16,64,10,0) # Heavy
#Blur(.50) # Set to .25-.50
#WarpSharp(depth=60,blur=1,bump=128) # (int depth, int blur, int bump,float cubic)
#Weave()

### ASPECT RATIO CORRECTION & RESIZING / MATTING ###
#Crop(8,0,-8,-0).HorizontalReduceBy2().Letterbox(0,8,0,0) # Resize to Half-D1 & Matte
#Letterbox(0,8,8,8) # Matte only

[^^-+I4004+-^^]

first of all,it's too long of a post....
no one will read it completely.....
(me included,although i had my moments too....)
let me state few facts that your screenshots helped me
realize.....(is that correct syntax in english?LOL..)

let me say this:
i don't regard compressibility checks as something relevant,as they can
be deceiving.....(and i say this even i don't use
GK at all.....ie. never did that tets anyhow...)
here's why:i can make my video completely noiseless (at least in static scenes),so it would of have great compress.% figures,but it's deceiving: although it look fine to compress to GK,it's not the image i aim to have...it'll be too blurred to watch....
so,those checks are out of question.....we aim to reduce noise AND to preserve image sharpness.....

seen your screenshots and now i see that averaging of multiple
captures results in SOME smoothing too.......
how come?
averaging was promoted as the ultimate noise reduction technique that
does magic:reduce the noise while keeping the image sharp.....
now i see there's no magic : simillar results are obtainable with
some of the avs filters that remove the noise...(ie.reduction of noise is ALWAYS payed with loss of sharpness....averaging included...)
so why would i f*** myself with averaging if i can do it in elegant,
1 capture way....(althogh even than it's 2 steps: 1 . capturing
2. encoding )

and how about that "inconvenience" when we have to wait for the rerun
to capture 2nd time,and what if there's NO rerun at all?
averaging for VHS (by the look of your screenshots) does denoise,but also looses much of the details,so where the edge of averaging over the
normal filtering techniques...?

and on the "blur" filter usage....
there are clever ways than simple blur filter:have you heard of
"smartsmootherhiq"?
it smooths in space,but it's picky on what it can and what it can't smooth....
blur filter is not smart at all.....
that's why you have to use it so moderately....

also:c3d is not a good filter for analog capturing,it's too mild...i said it before,i'll said it again....
and if you're using c3d on top of other denoiser,then what's the point of that first denoiser/method and what's the point of c3d?
this shows only that original denoiser/method is not good enough...
(ie. averaging alone is not good enoigh....)
so:c3d alone is too mild,and if averaging needs c3d,then what's the point in averaging?
c3d will still destroy some detail in the end...


in the end,looking at screenshots,i still think you don't have the case here:details are lost,and you're probably better off,and instead using some fine and mild avs filtering on noisy videos.....

if you have enough noise,just leave it on VHS:digital denoising will only destroy the details (or what's left of the details on that
noisy VHS recording...still it'll look much better than blurred digital version....) so that's leading nowhere....
or alternatively,leave the noise and the detail in the video by the means of higher bitrate.....
(still,VHS noise will look more acceptable to human eye than digital noise of mpeg (1,2 or 4 ) codecs.......)

so,in my view,leaving real noisy stuff on VHS tape is doing it justice.....(in the sense of image quality)
if it's precious,and you need backup,and you need all the detail you can get,don't filter at all and go for real high bitrate of codec....

stories of "better than VHS source" digital versions are just that : stories.......
yeap,perfect denoisers don't exist.....

cheers

Ivo

ps. told you i'll reply when i got some more time.....

[dar1us]

Interesting, I will read in full later. Seems quite clever.

Yes, was a bit of a long post, but what do u expect, him/her to leave out useful info.

Kinda hypocrytical really, I cant be bothered to read all the stuff that IVO wrote, I will eventually read it all, though I just had a breakthrough in my XVID encoding, I got some beautiful results with todays captured voyager. The quality is really amazing, the encode that is, the capture was OK but encode rules. I think the B-Frame capability of VDub Mod 1.4.13 just kicked in as I can see the flowing of the frames.

l8rz

Goodwork fasttimes

[^^-+I4004+-^^]

Originally posted by dar1us
Interesting, I will read in full later. Seems quite clever.

Yes, was a bit of a long post, but what do u expect, him/her to leave out useful info.

Kinda hypocrytical really, I cant be bothered to read all the stuff that IVO wrote, I will eventually read it all, though I just had a breakthrough in my XVID encoding, I got some beautiful results with todays captured voyager. The quality is really amazing, the encode that is, the capture was OK but encode rules. I think the B-Frame capability of VDub Mod 1.4.13 just kicked in as I can see the flowing of the frames.

l8rz

Goodwork fasttimes

i said i had my moments too,seems to me my post was a lot smaller than ftimes' lamentations...i still say it's unreadable ie. too
long.......and that's why no one responded to his thread.......
(you didn't,did you?.....b-frames mean smooth motion?,or is it just that you got source that could get decent telecide or so....as star trek is notourious for
mixed content stuff and so....just a thought.....)

how will you read my stuff if ftimes' stuff took all of your soul and steam?

do a few tests (baring in mind all stuff said) tell us who has the point in this one.....
(but how will you do multiple captures for a start...hehe)

but surely at least your xvid breakthrough can be discussed.....
(heh)

anyhow....

cheers

[dar1us]

You have NO idea how right you are about trek being a complete mix of stuff.

I will end up reducing vertical by 2 for best results, too much noise on such a pure source. Convo isn't doing good at all at reducing it.

One episode may be a beautiful noise free, progressive broadcast, the next will be packed to death with noise and interlacing.


Less of that, what about the original article, any useful replies yet? People?


Like the signature IVO:)

[dar1us]

!!LAYERING 3-9 CAPTURES!!

Are you MAD?!?! That will take forever! All the capturing, all the space stolen (not so much of a problem), but say, if you have something like a wedding, captured at 576,576, all in Huff, 1 hour long, jheeze! That would take forever, it is an intreging (sp?) idea, but not really that practical.

Unless you could totally automate the whole process, it would be a little tiresome. Automating a VHS machine to work perfectly with a computer, yeh, right.

Wish that you dont get any frame drops.

Saying all of that, just looking at it in a little more detail, the findings seem quite solid, good theory, very indepth, how long did you spend reasearching (making it all up) then writing it all up.

[^^-+I4004+-^^]

Originally posted by dar1us



Like the signature IVO:)

some mod disagreed,see.......
but nevermind you know where i stand.....


on the issue of averaging,i agree
>!!LAYERING 3-9 CAPTURES!!

this is mad,and will probably produce results which add more
blur with every single new layer.......

unusable technique if you ask me.....

>how long did you spend reasearching (making it all up) then writing it all up.

even i got to that part(heh) : he said 12hrs...

[fasttimes]

Guys, I'll have to re-do the test. I think I may have screwed up the layering in the tests, by setting one of them off by one frame. There was some blur introduced in my first testing above, but I have seen no indication of blur in further testing. So, give me a few days to retest everything, and I'll start a new thread with the new results.

Regarding practicality, that's not the point of the original message. The point of the tests is does it work for VHS sources at all. Some think real-time MPEG capture is the only thing "practical". All I want to discover, is if layering is effective or not. I'll leave the practicality decision up to the individual. One thing I will say, if effective, it is the *only* filtering technique without side effects, and without loss of detail. IMO, it would have a place in cleaning up important/rare material.

But, let me run the tests again, as I think I screwed up the first time around. ;)

PS: Again, *NO* blur was introduced in any use of layering that I have done, except in these tests. That's why I suspect I screwed up the layering.

[wotef]

good post, but i really don't think you should bother re-testing again

the stacking principle is a sound one but you will quickly reach diminishing returns with a vhs source (where the main problem is a fundamental lack of detail on the source tape, and not noise introduced from playback); for a cleaner source (such as satellite), it has a far greater impact

if you are determined to re-test, i think it would be more than sufficient to just cap a short 5 or 10 minute segment a few times over - i think you will see there is little or no benefit after approx 3 repetitions

[^^-+I4004+-^^]

>PS: Again, *NO* blur was introduced in any use of layering that I have done, except in these tests. That's why I suspect I screwed up the layering.

doubt it....
some other folks posted some samples too..
(perhaps even wotef....)it was a scene in front of some
store (coffeshop or so) and it got blurred too.......
(there was a jpeg of original and averaged version...)

wotef,if you have good source,then why cleaning it at all?
VHS is where the clean-up is most needed........

and i still stand by my words: big amounts of noise
freeze when i hit "pause-still" button on my VHS machine,
so no use in capturing it 10 times.....
perhaps only 5-10% of noise is added by playback itself,and the rest is
on the tape.......
but you can do retesting...i'll be glad if you got better results this time.......

cheers_

[killerhis]

Well this has saved my life :D! I'm now using layerprocessing for more then 2 monthes... workes perfect.. thanks fasttimes.

I use it on very old DVD's with lost of noise to remove the noise without killing to much details.. works perfect.... In stead of recording 1 eps 3 times (like you do on vhs) I enocode 1 eps 3 times (in the script) and use other filtersettings on eache encode.. then with layerprocessing I get rid of the noise with minimum detail lost..works perfect.. but veeery slow :D...

It takes about 12H to encode 1pass with xvid on a AMD2000XP for a 20min eps... but looks dahm good :devil:

thankz

[BaronVlad]

:D

oh, think I missed a thread. I will have a deeper look into it the next days. Thanks for your tests...

[IJM]

Hi,

I've been working on capturing and cleaning up a movie from a PAL VHS ex-rental tape with a view to putting it on DVD. It's an obscure straight-to-video movie from the early nineties called "Mystery Date". I've been capturing it using an ADVC-100 and have made several attempts to clean it of noise using various filters and techniques with limited success.

I came across this thread a couple of months ago, and have wanted to try averaging multiple captures for some time. Well, with a new 250GB drive installed, I was able to try using fasttimes' script for the first time last week.

Holy Cowdung! The results are absolutely fan-flipping-tastic! (For maximum effect, please substitute appropriate expletives into the previous two exclamations). :D

I'd used five captures of the same 1 hr 33 minute movie. Averaged them, and then run the resultant file through Convolution3D using fasttimes' "light" parameters. I worked in YUY2 all the way through storing intermediate files using Huffyuv.

Now I have a movie that is almost completely devoid of noise, with almost all the detail retained (which is to say that there must be some missing detail, but I haven't found it yet). There is still a very slight amount of noise, but you have to look hard to see it, and I feel that it serves to give the picture "life".

This has been exactly what I'm looking for. I can thoroughly recommend this method. Thankyou fasttimes.

Ian.

[ronnylov]

I have used this method sometimes. What is a little boring is to find the dropped frames. To do this I layer only 2 captures at the time first (keeping one as reference) and scrolls through the capture until I find ghosting, or double edges. Then I know there has been a dropped or doubled frame. I'm using avisynth and either deletes or add the corresponding frame in the second clip to keep them in sync.

Then I compare the reference clip with the third clip and go through the averaged clip to remove all the doubled, or dropped frames. (if you have a dropped frame you can add it from the other clip, if it is doubled just delete one).

At last when all clips are synced it is time to average them.

The noise reduction is in the theory relative to the root of the number of clips. You need 4 clips to get half the noise, 9 clips to get a third of the noise. And it is only the noise generated by the playback and capturing system that is removed, the actual noise of the original recorded signal is still present. That's why additional noise filters may be necessary.

[ronnylov]

Originally posted by killerhis
Well this has saved my life :D! I'm now using layerprocessing for more then 2 monthes... workes perfect.. thanks fasttimes.

I use it on very old DVD's with lost of noise to remove the noise without killing to much details.. works perfect.... In stead of recording 1 eps 3 times (like you do on vhs) I enocode 1 eps 3 times (in the script) and use other filtersettings on eache encode.. then with layerprocessing I get rid of the noise with minimum detail lost..works perfect.. but veeery slow :D...

It takes about 12H to encode 1pass with xvid on a AMD2000XP for a 20min eps... but looks dahm good :devil:

thankz

Instead of making 3 clips at a low bitrate and average them at playback, can't you just make one clip with higher bitrate? The space needed on disc would be the same. Or do you encode these 3 averaged clips again? Then you are using the encoder as some kind of noise filter before the final encoding. Is this really a good method?

It's not the same thing as averaging 3 losslessly encoded clips before the final encoding.

[IJM]

Originally posted by ronnylov
I have used this method sometimes. What is a little boring is to find the dropped frames.Sounds, like a nightmare. I haven't had to deal with dropped frames yet. But my source was relatively good quality (for VHS) and the ADVC-100 is excellent for not dropping frames.

I did have a weird problem to begin with though. I originally tried running the video through a TBC (an ACE Converter by GTH Electronics) first. Each capture would have an interlaced section, then switch to non-interlaced and then back again, all the way through the film. And it was different on each capture. I couldn't get two captures the same. It wasn't even switching at the same points in the film every time. In the end I took the TBC out and played the VCR directly into the ADVC-100. Only then did I get five totally clean 1hr 33min captures.

Now the video (PAL) was made from the original film by running the 24fps film 4% faster to get 25fps. So both fields in each frame should be made from the same frame of film, and therefore be progressive. Now the TBC is supposed to be a full frame TBC, so it was almost like the TBC was skipping the odd field at random every now and again. As I say, very strange. I might ring up GTH Electronics and see what they say.

Ian.

[Edit Sep 5th 2003 17:17]
I've spoken to GTH Electronics about my TBC. The skipped and/or repeated fields are caused by subtle timing differences between my VCR and the ACE Converter. Say my VCR plays ever so fractionally faster than the TBC; eventually over the course of say 15 minutes (45000 fields), the fields catch up with the TBC and it misses one; or it doesn't get them fast enough and has to duplicate one to keep time. That's why it skips between interlaced to non-interlaced and back again. Well, that's what they've said.

[Malcolm]

@fasttimes,
i worte you an email/message through the forum some days ago, but didn't get any reply.
i'm very interested in your results. unfortunately the links to your samples (jpegs and videos) are dead. could you please reactivate them?

Thanks,
Malcolm

[scharfis_brain]

to find drops, there will be a semi-automatic solution:

use the subtract()-function of avisynth to subtract to of the three captures from each other.
after that apply the levels()-function to make the differenceds more visible.

load this script into virtualdub and press the scenechangedetection button (red/green)
VDub should stop now, when heavy ghosting appears...

[ronnylov]

Originally posted by scharfis_brain
to find drops, there will be a semi-automatic solution:

use the subtract()-function of avisynth to subtract to of the three captures from each other.
after that apply the levels()-function to make the differenceds more visible.

load this script into virtualdub and press the scenechangedetection button (red/green)
VDub should stop now, when heavy ghosting appears...

Thanks!

I'll try this next time.

[Malcolm]

has anyone downloaded the testpics and videos and can provide me a link to download them please? the links from fasttimes aren't working anymore.

Thanks,
Malcolm

[scharfis_brain]

Today, I've made a multiple capture from a VHS.

I've only done a dual-cap.

my script to layer both videos is very easy:
a=avisource("cap1.avi")
b=avisource("cap2.avi")
layer(a,b,"fast")
for more than 2 caps just cascade the layer fuctions:
4 caps: layer(layer(a,b,"fast"),layer(c,d,"fast),"fast")

when getting framedrops, both (or all) videos will get out of sync. To restore the sync, use this script:
a=avisource("cap1.avi")
b=avisource("cap2.avi")
subtract(a,b)
everytime when the videos are going out of sync (subtraction-imgage contains visible imageinforamtion), just append (alternating) a duplicateframe(xxx-1) or deleteframe(xxx) at the first line (a=...)
xxx marks the number of the frame delivered by this sript.

a complete resynced video may look like this:
a=segmentedavisource("1st.avi").trim(72,0).
\duplicateframe(611).deleteframe(947).duplicateframe(4166).deleteframe(6829).
\duplicateframe(11957).deleteframe(13453).duplicateframe(15640).deleteframe(16883).
\duplicateframe(18260).deleteframe(18656).duplicateframe(22108).deleteframe(22149).
\duplicateframe(22428).deleteframe(25720).duplicateframe(31991).deleteframe(32411).
\duplicateframe(34401).deleteframe(36199).duplicateframe(38301).deleteframe(39406).
\duplicateframe(42189)

b=segmentedavisource("2nd.avi").trim(90,0)
layer(a,b,"fast")
the trim-commands are used to achieve the 1st-time-sync.

[fasttimes]

Originally posted by Malcolm
@fasttimes,
i worte you an email/message through the forum some days ago, but didn't get any reply.
i'm very interested in your results. unfortunately the links to your samples (jpegs and videos) are dead. could you please reactivate them?

Thanks,
Malcolm
Wow, this is spooky! I haven't been on this board for year, and I just happen to take a look here, and this thread has risen from the dead, just today! :)

Sorry, I don't have the clips any longer. They were not valid anyway, because a script error on my part was actually blending the wrong frames together (each clip was offset by one frame). I never did get around to making a corrected set.

The theory is still sound. It should help eliminate the random noise introduced into the tape playback chain (capturing the same VHS tape multiple times) or eliminate the random noise introduced during direct live capture (capturing live broadcast material when it's broadcast multiple times.)

However, this is not magic, and will still require additional processing before encoding, unless you have really fine source material.

[IJM]

Originally posted by fasttimes
... a script error on my part was actually blending the wrong frames together (each clip was offset by one frame).
Sorry, to resurrect this again, but is the fault the offsets specified in the "SelectEvery" clauses?

Here's what you had as quoted in your full script as posted near the top of the thread:
### LAYER CAPTURES ###
#Interleave(cap1,cap2,cap3).TemporalSoften(1,255,255).SelectEvery(3,1) # 3 Layers
#Interleave(cap1,cap2,cap3,cap4,cap5).TemporalSoften(2,255,255).SelectEvery(5,1) # 5 Layers
#Interleave(cap1,cap2,cap3,cap4,cap5,cap6,cap7).TemporalSoften(3,255,255).SelectEvery(7,1) # 7 Layers
#Interleave(cap1,cap2,cap3,cap4,cap5,cap6,cap7,cap8,cap9).TemporalSoften(4,255,255).SelectEvery(9,1) # 9 Layers
Am I right in thinking that these should read:
### LAYER CAPTURES ###
#Interleave(cap1,cap2,cap3).TemporalSoften(1,255,255).SelectEvery(3,1) # 3 Layers
#Interleave(cap1,cap2,cap3,cap4,cap5).TemporalSoften(2,255,255).SelectEvery(5,2) # 5 Layers
#Interleave(cap1,cap2,cap3,cap4,cap5,cap6,cap7).TemporalSoften(3,255,255).SelectEvery(7,3) # 7 Layers
#Interleave(cap1,cap2,cap3,cap4,cap5,cap6,cap7,cap8,cap9).TemporalSoften(4,255,255).SelectEvery(9,4) # 9 Layers
I'm not familiar with the finer points of AVISynth scripts, so I'd love an answer to this.

Many Thanks.

Ian.

[mustardman]

Very interesting indeed. I am particulary enthused by the comments made by 'IJM'.

I like the suggestions for finding dropped frames (using subtract) and for averaging multiple frames (using layer) - although it does have the disadvantage of only working in pairs. I did find using 'fasttimes' technique a bit difficult to follow!

I have a clip which is only about 20 seconds long (home video) that is VHS, and I am looking forward to capturing and "averaging" it. For such a short clip, I won't have to worry about frame drops, and 4 captures sounds easy. When I get it done, I will post the results.

It will be interesting to see how much noise is introduced in playback and capture, as opposed to the noise in the original. I would think the original tape would not have too much noise introduced during recording - but I could be wrong!

[Malcolm]

Hi,
i tested the described techniques to average multiple captures with _very_very_ bad vhs material. my results are that - as fasttimes already said - only the very slight noise introduced by the analogue chain (VHS - VCR - TV-card) could be removed. the noise from the bad vhs is really static (that was not _so_ clear for me). Could have been that the bad vhs produces some random noise like flickering or color aberrations on different places during playback. but that isn't the case.
so i would say: this technique brings you a lot if you have very good source-material that you only have to average to get it clean. If you have to use some noisefilters anyway because your source is bad, then you can eliminate the noise from the analogue chain along with the other noise.

bye, Malcolm

[brett]

Originally posted by IJM
Hi,

I've been working on capturing and cleaning up a movie from a PAL VHS ex-rental tape with a view to putting it on DVD. It's an obscure straight-to-video movie from the early nineties called "Mystery Date".

...

I'd used five captures of the same 1 hr 33 minute movie. Averaged them, and then run the resultant file through Convolution3D using fasttimes' "light" parameters. I worked in YUY2 all the way through storing intermediate files using Huffyuv.


I just noticed this. This gentleman did 5 captures of a VHS tape and averaged them to get a decent DVD copy. Now check this out:

http://www.amazon.com/exec/obidos/tg/detail/-/B0000714C0/qid=1073724140//ref=sr_8_xs_ap_i0_xgl74/103-0895474-7047858?v=glance&s=dvd&n=507846
http://www.netflix.com/MovieDisplay?movieid=60025212

Mystery Date has been available on DVD for a year. It's $13.46 at Amazon.com, and Netflix has it for rent.
:p


Originally posted by ronnylov
I have used this method sometimes. What is a little boring is to find the dropped frames. To do this I layer only 2 captures at the time first (keeping one as reference) and scrolls through the capture until I find ghosting, or double edges. Then I know there has been a dropped or doubled frame. I'm using avisynth and either deletes or add the corresponding frame in the second clip to keep them in sync.


You scroll through the whole movie looking for dropped frames? Now that would take awhile. And other people are also suggesting methods for finding dropped frames.

You guys should capture with Virtual VCR. It will simply give you a list of dropped frames.

[IJM]

Originally posted by brett
I just noticed this. This gentleman did 5 captures of a VHS tape and averaged them to get a decent DVD copy. Now check this out:

http://www.amazon.com/exec/obidos/tg/detail/-/B0000714C0/qid=1073724140//ref=sr_8_xs_ap_i0_xgl74/103-0895474-7047858?v=glance&s=dvd&n=507846
http://www.netflix.com/MovieDisplay?movieid=60025212

Mystery Date has been available on DVD for a year. It's $13.46 at Amazon.com, and Netflix has it for rent.
:p

Yes, I am that gentleman. I discovered the availability of a R1 version of Mystery Date just a couple of months ago (I'm in the UK/R2). My principles are such that if I can get an legit version, I will. So I bought it.

It was quite interesting comparing my efforts with a proper production copy. I have to say that despite the fact that mine was sourced from an ex-rental VHS copy, I hadn't done such a bad job. The picture quality I had achieved was not far off the production copy.

In fact, I think I'd managed to keep more detail in mine. It looked as though whoever'd encoded the R1 copy had processed it to produce a cleaner look, but had smoothed out some of the finer detail.

For instance, for those of you that might know the film, the very first scene, the "fantasy/dream" sequence on the beach features the principle character, "Tom". In my version, you could see the hairs on his arms, but the production R1 version had smoothed out this kind of fine detail.

The colour was different too. The R1 copy was much lighter, mine was darker and appeared more saturated. Though that would have been a result of the fact that my only reference was the VHS tape that I'd captured from. So no big surprise there.

Overall, the legit copy was better with the sharper picture and a cleaner more noise free appearance.

But it was really quite interesting to compare the two.

Ian.

[brett]

I wasn't trying to give you too much of a hard time... I just know I'd be pretty mad if I spent the time to do 5 VHS captures then the movie showed up on DVD.

[ronnylov]

When a dropped frame has caused a problem then it affects all the rest of the frames in the averaged clip until another frame is dropped in the second capture which may resync it again, so it is rather easy to quickly scroll trough the averaged clip to find the problem frame. But it sure is a bit boring job to do. I only use it when backing up old home videos when I need to restore as much quality as possible.

I tried the subtract function in avisynth and it makes it a lot easier but still it does not autodetect it in virtualdub. I have some ideas how to make a log with SSIM in avisynth to find problem frames, comparing the averaged video with one of the captures (I use one capture as reference to sync the other captures).

[IJM]

Originally posted by brett
I wasn't trying to give you too much of a hard time... I just know I'd be pretty mad if I spent the time to do 5 VHS captures then the movie showed up on DVD.
Oh no. I have no problem with your post. In fact, thanks for letting me know. If I hadn't have known already, I'd be ordering it now.

I decided to abandon my work in the end because I got hold of that R1 DVD. But I was pleased and surprised with how close I got with my efforts despite it being a VHS source (albeit a good quality one). And it proves that the techniques are sound.

But for me, the effort wasn't really the five captures. My source was decent quality, and I capture using the famous Canopus ADVC-100. So getting five identical captures with no dropped frames was no problem.

What was annoying was throwing away a lot of the artwork I did for the menus (that's the bit that really took the time). But I learnt a helluva lot doing it and I've kept all the menu bits and pieces, so I'll just use 'em for another film. (You should have seen the chapter menu and all the menu transitions. Not wishing to blow my own trumpet, but they were good).

The thing to remember is that averaging by itself won't remove all the noise from a VHS source. As stated by many, it can only really reduces the noise introduced by the capture chain and gets you closer to the recording from which you're capturing.

But consider this. I now regard averaging as an essential part of the process. It serves to reduce the noise threshold, so that the conventional noise filters can be used with much lighter settings than you would otherwise be able to, thus enabling you to retain much more detail. In other words, the noise filters still do the main work, but the averaging makes it much easier for them.

Thanks.

Ian.