Median multiple captures of the same video (VHS)

[BBugsBunny]

Is there a filter like average(clip1,0.25,clip2,0.25,clip3,0.25,clip4,0.25) see here:
http://forum.doom9.org/showthread.php?p=1129919#post1129919
that does perform median instead of average?

I want to use it to remove random spikes from vhs captures. The median filter would discard the extreme values (spikes) that is only present in one of the clips in time.

I've found threads here that talk about median (clip1, clip2, clip3...) but regarding an actual filter I only found the average filter that performs average instead of median).

[AVIL]

Median is generally referred to an odd number of clip:

Read this two thread:

http://forum.doom9.org/showthread.php?p=1169990#post1169990

http://forum.doom9.org/showthread.php?p=1331776#post1331776

Good luck

[BBugsBunny]

Thanks a lot AVIL!
Did some few tests and it seems to work very good.

[jmac698]

Function Median1(clip input_1, clip input_2, clip input_3, string "chroma")
{# median of 3 clips from Helpers.avs by G-force

chroma = Default(chroma,"process") #default is "process". Alternates: "copy first" or "copy second"

Interleave(input_1,input_2,input_3)
chroma == "process" ? Clense(reduceflicker=false) : Clense(reduceflicker=false,grey=true)
SelectEvery(3,1)

chroma == "copy first" ? last.MergeChroma(input_1) : chroma == "copy second" ? last.MergeChroma(input_2) : last

Return(last)
}

[BBugsBunny]

Been using the median of multiple captures with success. I also successfully used average.
http://forum.doom9.org/showthread.php?t=100626

While average reduces random playback noise, median can eliminate random spikes (white lines and the likes) present in one capture but not the other.

So now I was thinking about a combination that can do both. That would be a median of an even amount of captures with a minimum of 4 captures of the same video.

First median calculates the two values closest to the middle and then an average is applied to these two values.

Such a filter/script could eliminate spikes and reduce random playback noise form VHS captures.

Anyone here got a clue how to do this?

[Didée]

Basically, Median filter is not possible / not well-defined for an even number of elements.

However, the median of MaskTools:mt_lutf() iirc works this way - cut the outliers & average the remainder. *.mp4guy once told me about this behaviour.

If so, the following maybe does what you want:

clip1 = ...
...
clip4 = ...

c13 = interleave(clip1,clip3).assumefieldbased.assumetff.weave
c24 = interleave(clip2,clip4).assumefieldbased.assumetff.weave
c1234 = interleave(c13,c24).assumefieldbased.assumetff.weave

mt_lutf(c1234,c1234,mode="median",pixels="0 0 0 1 0 2 0 3",expr="y",U=3,V=3)

pointresize(c1.width,c1.height)
Disclaimer: Script is written "dry", hence "may contain nuts". ;)

Similar can be done for 8 elements quite easily. A bit different for 6 elements - absolutely possible, but more difficult to fiddle together.

In any case: mt_lutf("median") is quite slow ... be prepared for sluggish speeds.


Another possibility: use 4 or 5 captures, create 3 different simple medians of 3 elements via clense (which is very fast), then average the medians.

[BBugsBunny]

Thanks for your response Didée!
Finally I had time to test this.
Downloaded masktools-v2.0a48.zip (seems to be the latest)

VirtualDub shows an error: Script error: mt_lutf does not have a named argument "pixels"

The mt_lutf description says:
clip clip1, clip clip2, string mode("avg"), string expr("y"), string yexpr("y", string uexpr("y"), string vexpr("y")
and mt_luts:
clip clip1, clip clip2, string mode("avg"), string pixels(""), string expr("x"), string yexpr("x"), string uexpr("x"), string vexpr("x")

whereas mt_lutf does have "median" but mt_luts not.
Further my source is YUY2. Not sure if masktools will work with this color space.

I also tried this (median_average4.avs) - seems to work and it is relatively fast (~19 fps)

clip1=AVISource("D:\cap1.avi")
clip2=AVISource("D:\cap2.avi")
clip3=AVISource("D:\cap3.avi")
clip4=AVISource("D:\cap4.avi")
M1=Median1(clip1,clip2,clip3)
M2=Median1(clip2,clip3,clip4)
M3=Median1(clip1,clip3,clip4)
M4=Median1(clip1,clip2,clip4)
Average(M1, 0.25, M2, 0.25, M3, 0.25, M4, 0.25)

Function Median1(clip input_1, clip input_2, clip input_3, string "chroma")
{# median of 3 clips from Helpers.avs by G-force

chroma = default(chroma,"process") #default is "process". Alternates: "copy first" or "copy second"

Interleave(input_1,input_2,input_3)
chroma == "process" ? Clense(reduceflicker=false) : Clense(reduceflicker=false,grey=true)
SelectEvery(3,1)

chroma == "copy first" ? last.MergeChroma(input_1) : chroma == "copy second" ? last.MergeChroma(input_2) : last

Return(last)
}

I am rather a beginner in avs scrips, so not sure if this is correct / makes sense...
Took the function of a post AVIL pointed me to.

Also I'm interested in how the clense median works in this function. Is it by sub pixel / pixel? Does it compare/median the values of each sub pixel eg. pixel 1 of clip1 has sub pixel values of YUV 20/5/60; clip2 21/4/62; clip3 250/253/252. Sub pixel median would be 21/5/62. Is this actual the case or is the result 21/4/62 pixel median or even something else?

Did compare the result of the above script with an individual capture (via subtract) and in general it is an improvement, but there can be some rare cases where some frames of an individual capture can be better eg. at the start of a recording where the player searches the best tracking one of the individual captures can be better than the median.

Did dig out a ~25 year old recording. Actually the remaining part of a recording because I did copy over some newer stuff on most part of the tape.
Here's an example of one frame that's considerable enhancement with median+averaging multiple recordings (completely unfiltered).
Frame of one single capture:
http://img163.imageshack.us/img163/8821/single02.jpg
above script (4 captures):
http://img3.imageshack.us/img3/1144/medianaverage02.jpg

[Didée]

Oops, yes. I meant to write mt_lutS, not mt_lutF. Sorry for the confusion. Also, I'm pretty sure mt_lutS does have the "median" operator, since I've used it numerous times. Is it not in the documentation?

Clense does what you call "sub pixel" filtering. All three planes - Y, U and V - are processed independently.

MaskTools does support YUY2 ... with a trick. You need "SSE2Tools" by kassandro. Then IIRC it goes like

YUY2_clip
ConvertInterleavedToPlanar()
MaskToolsFilter()
ConvertPlanarToInterleaved()


*fallen asleep*

[BBugsBunny]

Thanks again Didée!

Been searching quite a while to find the SSE2Tools. They are no longer inside the RemoveGrain zip/rar files hosted on kassandros site.
Anyway found the dll in some package:

File Size
SSE2Tools.dll 30.720
SSE3Tools.dll 49.664
SSETools.dll 28.672
SSEToolsS.dll 69.632
File date for all: 11.04.2005
Tried all of them one by one but I always get:
mt_luts : unsupported color space. masktools only supports planar YUV colorspaces (YV12, YV15, YV24)

clip1=AVISource("D:\cap1.avi")
clip2=AVISource("D:\cap2.avi")
clip3=AVISource("D:\cap3.avi")
clip4=AVISource("D:\cap4.avi")

c13 = interleave(clip1,clip3).assumefieldbased.assumetff.weave
c24 = interleave(clip2,clip4).assumefieldbased.assumetff.weave
c1234 = interleaved2planar(interleave(c13,c24).assumefieldbased.assumetff.weave)

planar2interleaved(mt_lutS(c1234,c1234,mode="median",pixels="0 0 0 1 0 2 0 3",expr="y",U=3,V=3))

pointresize(clip1.width,clip1.height)
Probably the error could have one of these causes:
1) error in the script
2) wrong version of sse2tools
3) wrong version of masktools

File / size
mt_masktools-26.dll 966.656
31.12.2010

:confused:

[BBugsBunny]

Forgot to mention - I do my captures with the Lagarith codec using YUY2 mode.

Anyone got an idea why I run into this error? Is it at least safe do say that the script I use is OK?

[BBugsBunny]

Solved the problem by replacing mt_masktools-26.dll with mt_masktools-25.dll
No idea what's the difference between the two and why it works with one but not the other, but it works and that is of most importance.

[poisondeathray]

Solved the problem by replacing mt_masktools-26.dll with mt_masktools-25.dll
No idea what's the difference between the two and why it works with one but not the other, but it works and that is of most importance.

"26" is for avisynth 2.6.x

"25" is for avisynth 2.5.x

[BBugsBunny]

poisondeathray, thanks for clearing me up!

Did compare Didée script with the one I posted earlier (with the sample screen) and it seems as they are doing exactly the same (compared them with substract) only that the script I posted is much faster then Didée's.
In fact the script I posted is based on Didée's idea posted earlier:
Another possibility: use 4 or 5 captures, create 3 different simple medians of 3 elements via clense (which is very fast), then average the medians.
So if anyone else wants to experiment with multiple captures...

[Ghitulescu]

Is is worth doing this once the series appeared on DVD? I don't know exactly which series is this, but I think I remember seeing the robot somewhere ....

[BBugsBunny]

Is is worth doing this once the series appeared on DVD? I don't know exactly which series is this, but I think I remember seeing the robot somewhere ....

No clearly not worth it. I searched one of the oldest recordings I had to do some testing. It was on a tape that I recorded newer stuff over, but on the end of the tape there is some of this old stuff left.
But I do have some other recordings that are worth it.

BTW the screencap is from transformers that aired ~in the mid 80'.
The interlacing (field blending?) of this recording is also badly messed up. The recording is 25fps but it seems as if they took a 29.97 fps source (or 24 fps ->29.97?) and converted it to pal. Just out of curiosity I played a bit with RestoreFPS but I could get no good result. Not sure what the original fps was and how they messed it up. Not sure if there's a way to correct this and how to do it.
But this stuff is only some testing anyway.