heavy noise

[yingx2]

Hi, all

I think I need some suggestions on denoising.
How would you denoise a very noisy source like this?
http://myweb.hinet.net/home1/joefox/MJ.avi
(Yes, I will deinterlace it later)

I've tried Convolution3D (0, 32, 128, 32, 128, 10, 0), and vdub's DNR, 2d cleaner and smart smoother HQ, but I could not get an acceptable result. I guess that it's probably not possible to keep too much detail there, but still I want it as sharp as possible. At least I want to remove those vertical wavering lines to some degree.

Any reply is appreciated. Thanks in advance!

[dar1us]

Whats the source of that.

I havn't downloaded it yet as I am updating a freshly installed windows, but is from a terrest' broadcast (through aerial)? Satellite, cable...

A better idea would possibly be to try and stop the noise in the first place. You need to give more info on how you are capturing. Which OS, what source, what cap card... Those kinda things.

[yingx2]

It's cable.

I'm sure I have no problem capturing clean (not too clean, of course)video signal from CATV. What I mentioned above is actually very rare situaton to me. Perhaps it's the service provider's fault or some kind of temporary interference, I don't know. I just need some input on denoising from experienced users.

Thanks for your reply.:)

[Mike456]

Try the filter "motion blur" and encode it with Xvid (h263 quantizer) with a data rate >1000 kbit/s at full resolution. Divx movies doesn't look good, if the source was noisy.

[cjv]

Originally posted by yingx2
I've tried Convolution3D (0, 32, 128, 32, 128, 10, 0), and vdub's DNR, 2d cleaner and smart smoother HQ
You have mainly spatial filters in there, and your temporal settings for c3d _will_ cause ghosting in high action shots. Try GuavaComb() to get rid of those rainbow specs, and PeachSmoother() to reduce the noise somewhat, but mainly to improve picture quality. PeachSmoother() keeps some noise, but it is supposed to. You may also want to try Undot() to attack those vertical lines, and FluxSmooth() with lowish settings as your temporal/spatial filter.
These are all Avisynth filters.

cjv

[^^-+I4004+-^^]

i see that the source was NBA game or so...
(i dloaded just first few mb's...)
sports and fast motion scenes CANNOT be denoised properly...
instead u can just use CNR (and with mild settings) to get rid of chroma noise at least ,and crank codec's bitrate

(i don't agree that divx is worse than xvid: even with much noise divx wil still give sharper image...i have tested it myself...with h263 and mpeg xvid quantizer...)

btw. c3d is not a good denoiser in my opinion,but no denoiser will give you good results with that much noise and that much motion
(btw. i use temporalsmother+smartsmother(vd or avs versions) for hardcore denoise,and peachsmother or grapesmoother (avs) for mild denoising.....for moto races i don't do any denoising...as author of cnr for avs didn't include motion detection cut-off to YUY2 version of cnr filter..)

cheers

Ivo

[fellaw]

You could do a field split via the deinterlace filter of vdub, then run a normal denoise, chroma noise reduction or similar, fold the fields together and then run smart deinterlace or deinterlace map. IMHO this makes sense, because the final deinterlacer won't interpolate the noise but the filtered image. And the denoising won't garble up with the interlace artifacts.
Sometimes, this method gave me far good results on such issues.
Btw, there's a spartial smoother which I'm using very often. Its from Dividee, You can d/l it here (http://users.win.be/dividee/virtualdub.html).

[wotef]

quote:
"sports and fast motion scenes CANNOT be denoised properly... "
"c3d is not a good denoiser in my opinion..."

heh, i totally disagree

try this avisynth 2.0x script - no need to deinterlace:
loadplugin("e:\convolution3d.dll")
loadplugin("e:\simpleresize.dll")
a=avisource("e:\mj.avi").tweak(bright=-8,cont=1.05,sat=1.1).reduceby2().convolution3d(0,10,10,6,6,2,0)
b=a.bilinearresize(480,360)
c=a.simpleresize(480,360)
d=b.layer(c,"fast")
return d

quote:
"i don't agree that divx is worse than xvid: even with much noise divx wil still give sharper image...i have tested it myself...with h263 and mpeg xvid quantizer..."

heh, i utterly disagree, but that's not relevant for this thread

[dar1us]

I use telecide on most of my captures, no fancy options, just .Telecide() and it usually deinterlaces fine. The only problems are the huge issue of encoding artifacts. I get no horizontal lines, no shadows...

Nothing.

And yeh, this isn't the place to bitch about XVID vs DivX:)

[yingx2]

Thanks for all the responses.
I can see that you are all trying to be helpful, thanks a lot.
I'll do some tests and hopefully report the results :)

[^^-+I4004+-^^]

>You could do a field split via the deinterlace filter of vdub, then run a normal denoise, chroma noise reduction or similar, fold the fields together and then run smart deinterlace or deinterlace map.

and what have you done to combat ghosting?
every field of that game is diferent instance of time....
cnr alone is not enough...there's lots of luma noise there too....


>b=a.bilinearresize(480,360)

bilinear?

[wotef]

the tiny blur in the bilinear resize is useful here; it's then layered with simpleresize which is a bilinear resize that doesn't blur, thus averaging out some more noise

lanczos and/or bicubic are no good for this clip, the precision will retain all the vertical rubbish...and there's no point staying at 640x480...

try it!!!!!!!

[^^-+I4004+-^^]

Originally posted by wotef
the tiny blur in the bilinear resize is useful here; it's then layered with simpleresize which is a bilinear resize that doesn't blur, thus averaging out some more noise

lanczos and/or bicubic are no good for this clip, the precision will retain all the vertical rubbish...and there's no point staying at 640x480...

try it!!!!!!!

layered means averaged,right?
and you're averaging one bilinear with another?
enough said.......

not true that bicubic is not good.....EVERY resizer destroys some noise as a side-effect.......(the more you resize,the more noise is gone)
the ones that blur more (obviously) destroy more noise too.....but you can blur
with any denoise filter if you wan't blur.(i don't...and most of the time i would be happy to leave video on 720x576...but my cpu wouldn't like that...and i think in terms of HERE+NOW....)

and where in the script have you done deinterlacing or teleciding (ok,obviously not teleciding for sports)?
or are you counting on the deinterl. effect of the resizer itself?
well,i wouldn't count ton that ........for the sports anyhow.....

in my mind it's impossible:some people will mention averaging (perhaps?)
but i've seen that those people in addition to averaging use temp.smoothers and therefore averaging alone is not enough.....
(and how does one get 10's of vhs tapes with different noise on them to average it.......funny+overkill concept,i must say...)

as i say:for optimal sharpness leave that noise and crank bitrate
(eventually cnr....eventually,as it can produce color artefacts on hi-motion..use mild settings)

for lousy ghosting and complete destruction of image use hard core smoothing
you don't have much choice there i must say.....
for noise,temporal info has to be modified,and that's not good enough
spatial smoothing just blurs too much.......

resizing as a denoise modell is lousy too:resize enough and you have too small image and blurred too........

but sure,he SHOULD try it,and report if he liked the end results.....
i doubt he will.....but who knows.....not all have same standards....

[wotef]

ivo, so easy to critique; not so easy to contribute something!

you didn't even bother to run my script and empirically test the results, did you?

instead of theorising all day, why don't you actually help this guy produce a better clip? in fact, i challenge you to put your money where your mouth is and produce something better than my script!

here is my version, encoded using the "unsharp" (well, you tested it) xvid, quant 2

http://www.zenadsl5618.zen.co.uk/mj_bilinear_is_good.avi

[dar1us]

Sorry, I havn't tried your script either (too much work to be done!)

but reduceby2?! are you mad. that may get rid of the noise, but at a pretty harsh cost! how bad is the peoples noise in the first place, us 'propers' dont get any noise at all anyway...

sorry, must sound like we are ganging up on you here:) - but I am just trying to help people and myself to better it for everyone

[wotef]

don't you even have enough time for right-click "file save as"??? i already denoised and encoded the file!

am i "mad" for reduce by 2? doh, go on then, show me a better way to denoise this clip

...not everybody has a nice clean source all the time...

very odd that you "propers" don't actually seem able to provide any constructive help here...

[dar1us]

What is your source then?

How bad can it be?

btw... the propers bit was a joke... shame huh

I have ISDN, and the original file posted was 11.1 megs, I dont have the time, especially considering that BT is pants and I can only get about 2k a sec.

edit: anyways, me and IVO (his nick is too long to write that in and dont point out the irony that I just said all of this about it) have added quite a bit to this Board (capturing) in the last few months.

[^^-+I4004+-^^]

>...not everybody has a nice clean source all the time...

so it should be cut into half and then blown to 480x360 ?
with that process you have blurred the image COMPLETELY !
look at the logo,man....just LOOK AT THE LOGO !
(i'm pointing the tv-station logo out because often i have subtitles hardwired
into video....how would i look at that videos if i do to them this you just did to MJ.avi?you have killed this video my friend..just killed it...)

>instead of theorising all day, why don't you actually help this guy produce a better clip?

denoising this won't produce better clip:only worse!

i have spent some time writing about it,and then you come saying :
"that's worthless and show me how will you denoise this"!

and before that i said: sports CAN'T be denoised properly!
i'm still with that!
denoising works on principles of temporal redundancy (2 simillar frames,but with different noise:it's easy to spot noise and remove it...),and can't be applied if there's none!
there's NO temp. redundancy in sports!
most of the time frames are DIFFERENT!

recently i've done RallyGB to divx and applied NO noise filtering for the same reason:i only resized,and put "only" 50' per cd (yes,i ALWAYS use prec.bicubic because i feel that codec itself blurs and resizer itself blurs.....if you ask me both do it too much....)

so i proposed 2 solutions:
1:denoise and have some ghosting/blurring
(you've showed pretty graphically how far blurring can go!)
2:don't denoise and leave bitrate real high
(there's no good mpeg4 with extra lo-bitrate.....sports in particular..and i would discard xvid right away too,as i've seen some terrible motion compensation problems with moto-races+xvid....)

i would pick "2",because i do mind noise,but i mind blurring/ghosting even more!
(and i think having the source in divx5 doesn't help either...)

some while ago i did experiments on moto races and temp smoothing...and let me tell you:i didn't liked how does that asphalt looked like!
your method is even worse,as it blurs too much.....
(and then i wouldn't see any asphalt at all.....)
that's extreme case,but proves something.......

there's price to pay for denoising/lower bitrate target--->that price is image quality.......
i'm less and less willing to compromise image quality for that reasons:
as i've said cdr's are really cheap!
with a lot of movement (sports) it's important NOT to compromise that(or image quality itself as with this resize denoising) with extreme denoising.......

till now i have only once did that kind of thing (resize denoise) and it was the noisiest VHS tape i have (some porn tape)...and for that purpose it was good (LOL!).....
for REAL (important) video i wouldn't use that.......

>his nick is too long

copy-paste?
hehe
my "civil" name is fine too.....


wotef,no disrespect ,man, but just regard this clip as unrepairable
(sure,i was geared up to denoise it and provide better
version on my web space,but when i saw the source and i saw
it was sports.....i lost my will.....)

out of pure curiosity i've toyed a bit with aviutil (i don't have any intention on installing divx5 on my machine...ever...) because aviutitl can open it with divx412 codec (and VD can't),and i used it's temp.smoother to denoise and resize (i don't know first thing about aviutil ,this is second time i'm using it..VD is better:i entered 512x384 and it added 16 pixels to every side making it 528x400....i don't know why..)
and i regard that video better than yours..(i've just watched that on tv-out...)
it's not denoised completely,but at least it's not blurred completely....
and complete game (i guess 90' ? ) would fit 2x74' cdr......
if you wan't i'll send you that(1" of video 500kB or 135kB rar archive with source frame,my aviutil experiment frame and wotef's frame.....)
furthermore i'll send this to anyone that expresses a wish (just PM me...) so they can pick a method themselves.....
perhaps there are people that love unsharp images....
(like wotef ?....hehe)

cheers_

[fasttimes]

My current experimental VHS source filter chain:

1. IVTC with Decomb, if the material is film sourced. If doing layering in the next step, IVTC each capture first.:

LoadPlugin("decomb.dll")
avisource("c:\rawcap.avi")
Telecide(guide=1,quality=3)
Decimate(cycle=5)

2. Capture multiple times, and "layer" to average out noise:

cap1=avisource("c:\IVTCcap1.avi")
cap2=avisource("c:\IVTCcap2.avi")
cap3=avisource("c:\IVTCcap3.avi")
Interleave(IVTCcap1,IVTCcap2,IVTCcap3)
TemporalSoften(1,255,255)
SelectEvery(3,1)

Above script can be slightly altered to layer any odd number of captures.

3.Finally, do hard-core filtering:

LoadPlugin("Convolution3D.dll")
Convolution3D(avisource("c:\layeredIVTCcap.avi")0,3,128,16,64,10,0)
Blur(.50)

Then load AVS script into VDub, and apply WarpSharp 1.1 to some value under 32 or so. Also, at some point, apply Border Control to black out any noise around the edges of the image, or crop them out (depending on your final format.)

That's it! Here is a quick sample, but still needs to be tweaked a bit, as it is a bit over-agressive at the settings listed above. One is a raw capture, and the other is filtered (MOD, please approve the Pics):

[fasttimes]

And here is filtered:

[^^-+I4004+-^^]

>2. Capture multiple times, and "layer" to average out noise

you said the magic word "VHS"

have you ever paused some noisy video (freeze frame)?
have you noticed that if you freeze it,noise stops too?
(given that you have proper VCR which has acceptable freeze frame
quality...)
what does that mean?
it means that you're capturing SAME video over and over!
that video has the SAME noise EVERY TIME!
layering is ment to be used with DIFFERENT sources
( but as i say it's such an overkill concept! doesn't pay-out... )
the sources that have DIFFERENT noise!!!
(there's NO POINT in layering same video over and over...or if you think different?)

so,probably all of the cleaning effect you can thank to c3d (less)
or temp.soften (probably more,but i prefer VD's counterpart t.smoother...)

also:if you introduced some temporal smoothing,then why layering at all?
with temp.smoothing all you get is ghosting etc.

but i see this:
>Blur(.50)

so i don't have much more to say.......

(but yes,i encourage original poster to try every and any method on his video--->btw. it seemed like he only has 1 capturing of that video and didn't use VHS to record it,so he can't even try that...)

[fasttimes]

There is lots of random noise in VHS! Give me a day to put a full series of JPGs together to demonstrate the very positive effect of layering. No, it's not magic, thus removing all the noise. But it does remove a chunk of the worst of it, even when capping from VHS.

Also, IMO blur in not evil! It has a place in the chain, when needed. You either blur to kill noise, or have the DCT quantinization do it for you, and cause artifacting during the MPEG encode! Blur can be used with skill, witout destroying real picture information.

Hopfully you will see the two pics I put up, soon. They are a little over-procesed, because I didn't have time to tweak the settings, but demonstrate the point well enough. Also notice, that the source was not layered in this case-- it's just a single noisy analog cap from VHS source.

Added:
OK, I have put something together, in this thread. (http://forum.doom9.org/showthread.php?postid=239817)

[High Speed Dubb]

Just thought I’d add another vote against using a temporal smoother on this video — Basketball has too much motion for temporal smoothing to do any good. Peach Smoother might be able to help (because of its spatial smoothing), but it might also take some hand holding (manual NoiseLevel and Baseline settings) if the video never sits still enough for it to estimate the amount of noise.

I wouldn’t bother with a comb filter, either. Aside from the Star Sports logo, there isn’t any obvious crosstalk.

Your best bet is probably a mild spatial smoother. At a guess, the vertical noise lines might be handled best by a frequency based smoother, such as Wavelet Noise Reduction. Be careful not to use too much smoothing, though, or you’ll smoothe over Michael’s head.

[Steady]

You might want to play with this script for high-motion interlaced source. It will however result in aliasing on narrow lines. Bob deinterlacing is generally a better choice when the emphasis is on motion rather than detail.

LoadPlugin("dustv4.dll")
AVIsource("roseparade2.avi")

#use ComplementParity if needed
ComplementParity
bob
pixiedust
#leave out selecteven for 60/50 fps output
selecteven

Dust is available in this (http://forum.doom9.org/showthread.php?s=&threadid=42749) thread.

[Ookami]

@Steady

:)

Welcome... Since when are you back? Hope to read more from you in the future...

[Steady]

Hi Ookami!

I didn't realize how long I had been gone. I am showing off my new noise filter in the avisynth forum. It is fun to have a new project.

[Ookami]

Originally posted by Steady
Hi Ookami!

I didn't realize how long I had been gone. I am showing off my new noise filter in the avisynth forum. It is fun to have a new project.

Well, we all missed you... I was offline for a few weeks so I have much to read. I've downloaded your new filter and will try it when my new machine is ready (in a week or two) :) . Thanks for your great work.

Cheers,

Mijo.

[onesoul]

From what I read from avisynth, you should always put () after any command, you'll get higher speed video processing, i.e. selecteven -> selecteven() Don't ask me why... :)
Cheers

[ygfdert]

Originally posted by yingx2
Thanks for all the responses.
I can see that you are all trying to be helpful, thanks a lot.
I'll do some tests and hopefully report the results :)

Almost the same capture source
Clip is also about sport of NBA
http://myweb.hinet.net/home3/ygfdert/CVB.avi

To yingx2:
Could you share the AviSynth script often to deal with this kind of cilp ??
(I am from Taiwan too)