Noise Reduction Script Problems

[paddycook]

Hi all,

I'm having problems with a script I've put together to reduce noise on captured PAL VHS videos. The script is being opened in Procoder and encoded to half D1 dvd.

Script is as follows. The UnfoldfieldsVertical and Setparity functions are from StickyBoy:
--------------------------------------------
AVISource("F:\Video\Capture\Barbarians\Barbarians.avi")
GuavaComb(Mode="PAL", Recall = 75, MaxVariation = 25, Activation = 90)
Setparity(false)
UnfoldFieldsVertical(flip=true)
PeachSmoother(NoiseReduction=30, Stability=30, Spatial=200, NoiseLevel=4.144, Baseline=2.763)
MSharpen()
FoldFieldsVertical(flip=true)
Levels(16,1.1,192,16,235)
------------------------------------------------
The script does a pretty good job, but every so often, at irregular intervals, I get a frame of complete pixilated garbage mixed with a frame that is from a completely different part of the video. Something seems to be getting out of synch. Is there something obviously wrong with my script?

Regards,

Patrick

[paddycook]

Okay,

Looks like no one's interested in my script problems, so in a desperate attempt to generate interest please find a new script that I've put together to solve the problem outlined in the post above:

------------------------------------------------
AVISource("F:\Video\Capture\Barbarians\Barbarians.avi")
assumefieldbased()
assumebff()
separatefields()
a=selecteven().PeachSmoother(NoiseReduction=30, Stability=30, Spatial=200, NoiseLevel=4.144, Baseline=2.763)
b=selectodd().PeachSmoother(NoiseReduction=30, Stability=30, Spatial=200, NoiseLevel=4.144, Baseline=2.763)
interleave(a,b)
weave()
MSharpen()
Levels(16,1.1,192,16,235)
-----------------------------------------------------
My question is, is it ok to use Peachsmoother in this way? (looks ok in Vdub)

[High Speed Dubb]

I’m not sure whether that will work. Peach Smoother does use a few global variables, so multiple calls in a script might cause problems.

More importantly — Running it separately on the odd and even fields isn’t going to do quite as well as running it simultaneously on both. The temporal correlation in Peach is meant for use on fields, not frames — and if anything is a bit too strong. So taking out half the fields will cause Peach to expect too much correlation between frames.

Also, running it separately on the the odds and evens means that you’ll be ignoring any motion information from half of the fields.

Instead, I’d recommend running it so...

SeparateFields().PeachSmoother(Your parameters here).Weave().etc

[JuanC]

Originally posted by High Speed Dubb
I’m not sure whether that will work. Peach Smoother does use a few global variables, so multiple calls in a script might cause problems. @paddycook: I've tried it. Using it more than once causes problems indeed. The problem is common to many AVS Noise filters. That was the idea behind the Unfold before / Fold after just only one PeachSmoother call.

Originally posted by High Speed Dubb
Instead, I’d recommend running it so...
SeparateFields().PeachSmoother(Your parameters here).Weave().etc @High Speed Dubb: What do you think about separating the fields and also unfolding them vertically, like in paddycook's first post? (Then folding and weaving them together after Peach) FYI: UnfoldFieldsVertical()/FoldFieldsVerical() are a couple of scripts by stickboy (http://forum.doom9.org/showthread.php?s=&postid=354834#post354834)

@paddycook: For my analog TV captures I've been using the same script you posted first, without any problem, only difference is I don't include the MSharpen filter. Hope that helps...

[High Speed Dubb]

JuanC,

So Unfold moves the odd and even scanlines to their own halves of the screen, to prevent blurring between them?

That should work okay, but not as well (or as fast) as the
SeparateFields().PeachSmoother().Weave()
approach.

By putting the odd and even fields apart from one another in the same frame, any information from motion which is shared between fields (which is very common) will be lost.


The global variables in Peach are just a legacy from the DScaler version — Writing a DScaler filter without globals is possible, but you have to jump through some tiny hoops.

[JuanC]

Thanks Lindsey, I'm getting there...

The whole idea of unfolding/folding, as I understood it, had to do with the use of spatial-temporal noise filters, dealing with info on previous frames. So in general, I thought that because of the spatial difference of the top/bottom fields (I can visually distinguish it), when separated and interleaved, it could be more accurate to apply the S/T filters on the unfolded fields. But then, you’re right, I was loosing the motion information which is common to the fields… :(

Anyway, I will do a couple of encodes tonight and will apply your advice.

Thanks a lot, Juan

[High Speed Dubb]

JuanC wrote
The whole idea of unfolding/folding, as I understood it, had to do with the use of spatial-temporal noise filters, dealing with info on previous frames...

Yep, that’s true enough.

Whether the SeparateFields().SomeSpatioTemporalFilter().Weave() trick is a good idea varies from filter to filter. It works well for Peach only because the filter looks explicitly for AVISynth’s “This is a bunch of interlaced fields” flag and uses it to match the even and odd fields properly. Without a check like that, a temporal filter would mess up and apply temporal blending between vertically adjacent scanlines.

Who knows? Even after these convoluted explanations, maybe my recommendation will actually work for you. :)

[JuanC]

Originally posted by High Speed Dubb
Whether the SeparateFields().SomeSpatioTemporalFilter().Weave() trick is a good idea varies from filter to filter. It works well for Peach only because the filter looks explicitly for AVISynth’s “This is a bunch of interlaced fields” flag and uses it to match the even and odd fields properly. Without a check like that, a temporal filter would mess up and apply temporal blending between vertically adjacent scanlines. This is really great :) It's a lot faster this way also! (I'm encoding already) This recommendation should somehow make it to the help file of PeachSmoother.

Thanks, Juan

[paddycook]

Wow! Woke up this morning to a ton of useful advice. thanks guys :)

In the interim I actually used the following script:
---------------------------------------------------
AVISource("F:\Video\Capture\Barbarians\Barbarians.avi")
GuavaComb(Mode="PAL")
Viewfields()
PeachSmoother(NoiseReduction=30, Stability=30, Spatial=200, NoiseLevel=4.144, Baseline=2.763)
MSharpen()
Unviewfields()
Levels(16,1.1,200,16,235)
--------------------------------------------------------

Got the same pixilated garbage frame 43 minutes into the video. Aargh! I'm wondering if it's something to do with Procoder?

I'm going to run an encode following Lindsey's suggestion and see what happens.

Thanks for the help!

[Darksoul71]

@paddycook:

Looks like no one's interested in my script problems....
Learn patience. It sometimes takes several days until people find the time to try out and answer your question (or may be not :))

Got the same pixilated garbage frame 43 minutes into the video. Aargh! I'm wondering if it's something to do with Procoder?
I really can´t comment on your script but I would like to point out that I´ve experienced also problems with Procoder. This programm showed flashing pixels/squares (like corrupted frames in DV) while CCE didn´t show up with any problems in the final encoded material.

Source were VHS-tapes captured via Canopus ADVC100.

-D$

[High Speed Dubb]

JuanC,

Yeah, that should certainly be in the docs — It only occurred to me about a month ago, when Wilbert was asking some similar questions.

[paddycook]

DarkSoul71,

Thanks for your post. Apologies for my earlier impatience.

MY tapes were VHS PAL also captured via a Canopus ADVC100 and the flashing pixels/squares are exactly what I'm getting so it looks as if you've solved my problem! Either the ADVC100 in combination with Procoder or Procoder by itself looks like the likely cause.

I do have CCE so I'll give that a whirl.

Thanks again.

[Yuri]

Encoding VHS captures with CCE, I am using the script and have no problem at all - the quality is superb:

function ApplyInterlacedFilter(clip v1, string filter) {

v2 = separatefields(v1)
selecteven(v2)
even = Eval(filter)
selectodd(v2)
odd = Eval(filter)
interleave(even,odd)
return weave()
}

ApplyInterlacedFilter(last, "Convolution3D(1,32,128,16,64,10,0)") #VHS Capture Bad Quality

ApplyInterlacedFilter(last,"PeachSmoother(NoiseReduction=45,Stability=30,DoSpatial=false,NoiseLevel=4.185,Baseline=2.810)")

... etc.

the idea of the script is similar to Paddycook's. Probably the culprit is the ProCoder he's using.

[JuanC]

Originally posted by Yuri
selecteven(v2)
even = Eval(filter)
selectodd(v2)
odd = Eval(filter)
This part of your script is calling the filter twice. I don't know anything about Procoder, but you guys are risking to have problems when using C3d or PS (see Linsey's posts) more than once in the same script. For me it is clear that you don't need to do so with PS, using the advice from Lindsey.

I have had some troubles with C3D and PS when using them twice in the same script.

@paddycook: You could probably avoid the need for MSharpen by lowering your settings in PS. Hope that helps.

[Yuri]

Haven't had any problems calling PS or c3d or deen so far, but I did when tried PixieDust the wonderful slowpoke.
I decided to try the method

separatefields()
...
weave()

and the UnfoldFieldsVertical script described in this forum on PS, c3d and deen and the they resulted in a far more smoothed picture and shaky horizontal lines. So I'd rather stick to the
function ApplyInterlacedFilter(clip v1, string filter) {

v2 = separatefields(v1)
selecteven(v2)
even = Eval(filter)
selectodd(v2)
odd = Eval(filter)
interleave(even,odd)
return weave()
}
BTW it comes from http://www.avisynth.org/index.php?page=InterlacedSmoothing

[sh0dan]

The filter isn't being invoked twice on the same material, but rather once on each field separately. This is important to temporal filters (as c3d and peach) to ensure that there is no crosstalk between fields, as they are jumping half a line up+down every second frame.

[paddycook]

ok, I've tried the original script with CCE and it works fine - no garbage frames. Also tried it with Mainconcept using VDubMod as an intermediary frameserver and that works fine too. Must be Procoder, which is a damn shame as Procoder is much better at lower bitrate encodes than CCE or MainConcept.

Thanks for all the info folks. I'm learning a hell of a lot and I really appreciate the knowledge being shared.

[stickboy]

Originally posted by Yuri
and the UnfoldFieldsVertical script described in this forum on PS, c3d and deen and the they resulted in a far more smoothed picture and shaky horizontal lines. So I'd rather stick to the
function ApplyInterlacedFilter(clip v1, string filter) {

v2 = separatefields(v1)
selecteven(v2)
even = Eval(filter)
selectodd(v2)
odd = Eval(filter)
interleave(even,odd)
return weave()
}
BTW it comes from http://www.avisynth.org/index.php?page=InterlacedSmoothingHmm... For an RGB32 or YUY2 source, I don't see why UnfoldFieldsVertical/FoldFieldsVertical would give you a significantly different result than ApplyInterlacedFilter. Fundamentally, they both operate the same way (separate the fields and apply the filter independently on the even fields and the odd fields), except Unfold.../Fold... avoids calling the filter twice.
Originally posted by sh0dan:
The filter isn't being invoked twice on the same material, but rather once on each field separately.In some cases (e.g. Dust), it doesn't matter the filter is called once only on different material; what matters is that it's called more than once in the same script at all.

BTW, paddycook and JuanC: it's stickboy, not stickyboy. The latter is just disgusting. :p

[Yuri]

I'm all astonishment myself! You're quite right that the two do the same thing but not with a "significantly" but a "slightly" different result. I tried one and the same interlaced TFF source, a cartoon, on the UnfoldFieldsVertical/FoldFieldsVertical, the SeparaeteFields()...Weave(), and the ApplyInterlacedFilter function. The filters involved were pixiedust/deen/C3D + PS. All the combinations were working just fine. Deen+PS in ApplyInterlacedFilter function produced the best sharpness and quality for the/a 2D cartoon. PixieDust unfortunately failed to work under the ApplyInterlacedFilter function but was quite fine in the SeparaeteFields()...Weave() code giving a very slightly blurred picture. If PixieDust did, it would be irreplacable for natural image.
the UnfoldFieldsVertical/FoldFieldsVertical function takes the mid-position. Anyway, one can blend them all up to achieve best results.
Thanks.

[JuanC]

Originally posted by stickboy

Originally posted by sh0dan: The filter isn't being invoked twice on the same material, but rather once on each field separately.

In some cases (e.g. Dust), it doesn't matter the filter is called once only on different material; what matters is that it's called more than once in the same script at all. @shOdan: I get problems with video getting corrupted or some other filters in the script not being applied when using PeachSmoother more than once in the same script (applying each call to different material), because of the reasons explained by Lindsey himself in this same thread. Also, I have had some problems (encoder crashing randomly during the process) when encoding to MPEG2 a script that calls C3D more than once. This last problem hasn't shown up when using C3D only once in the same script...

Originally posted by stickboy
BTW, paddycook and JuanC: it's stickboy, not stickyboy. The latter is just disgusting. :p @stickboy:
Oooops! How embarrassing! Corrected... Please accept my excuse... :o :J