Log in

View Full Version : Noisiest + Grainiest Clip I Have Ever Seen

R3Z
11th April 2007, 11:40
Hi Guys,

This here is a real shocker, someone has taken a rare concert footage VHS and sharpened the hell out of it resulting in massive noise and grain artifacts.

Normally i would cut my losses and move on, but because its a nirvana video and the masters are being hoarded due to geffen being aholes - i have to work with what i got.

Heres a sample (38mb);

http://users.on.net/inet_s51/x264/nirvana_chunk_1.mpg

I have tried Didee's magical script from this thread;(http://forum.doom9.org/showthread.php?t=122069)

Heres my results with my modifications (not sure if they are stupid or not) ;

Original

http://www.users.on.net/~inet_s51/x264/orig1_progressive.jpg

Processed

http://www.users.on.net/~inet_s51/x264/pro1_progressive.jpg

My script is as follows;

DGDecode_mpeg2source("F:\nirvana 92 brazil\nirvana_chunk_1.d2v",info=3)
ColorMatrix(hints=true,interlaced=true)

Bob(0,0.5)

bobbed = last
base = last.minblur(2).fft3dfilter(sigma=0.5,bt=4,plane=0,degrid=1.0,sigma2=0.6,sigma3=0.7,sigma4=0.8)

bw4_vec = base.MVAnalyse(isb=true, truemotion=false,delta=4,lambda=512,pel=2,sharp=2,blksize=16,overlap=16/2,pnew=32,idx=1)
bw3_vec = base.MVAnalyse(isb=true, truemotion=false,delta=3,lambda=512,pel=2,sharp=2,blksize=16,overlap=16/2,pnew=32,idx=1)
bw2_vec = base.MVAnalyse(isb=true, truemotion=false,delta=2,lambda=512,pel=2,sharp=2,blksize=16,overlap=16/2,pnew=32,idx=1)
bw1_vec = base.MVAnalyse(isb=true, truemotion=false,delta=1,lambda=512,pel=2,sharp=2,blksize=16,overlap=16/2,pnew=32,idx=1)
fw1_vec = base.MVAnalyse(isb=false, truemotion=false,delta=1,lambda=512,pel=2,sharp=2,blksize=16,overlap=16/2,pnew=32,idx=1)
fw2_vec = base.MVAnalyse(isb=false, truemotion=false,delta=2,lambda=512,pel=2,sharp=2,blksize=16,overlap=16/2,pnew=32,idx=1)
fw3_vec = base.MVAnalyse(isb=false, truemotion=false,delta=3,lambda=512,pel=2,sharp=2,blksize=16,overlap=16/2,pnew=32,idx=1)
fw4_vec = base.MVAnalyse(isb=false, truemotion=false,delta=4,lambda=512,pel=2,sharp=2,blksize=16,overlap=16/2,pnew=32,idx=1)

bw4_comp = bobbed.MVCompensate(bw4_vec,idx=2)
bw3_comp = bobbed.MVCompensate(bw3_vec,idx=2)
bw2_comp = bobbed.MVCompensate(bw2_vec,idx=2)
bw1_comp = bobbed.MVCompensate(bw1_vec,idx=2)
fw1_comp = bobbed.MVCompensate(fw1_vec,idx=2)
fw2_comp = bobbed.MVCompensate(fw2_vec,idx=2)
fw3_comp = bobbed.MVCompensate(fw3_vec,idx=2)
fw4_comp = bobbed.MVCompensate(fw4_vec,idx=2)


Interleave(bw4_comp,bw3_comp,bw2_comp,bw1_comp,bobbed,fw1_comp,fw2_comp,fw3_comp,fw4_comp)
MedianBlurT(0,0,0,4)
SelectEvery(9,4)

base2 = last
bobbed2 = last

bw4_vec2 = base2.MVAnalyse(isb=true, truemotion=false,delta=4,lambda=256,pel=2,sharp=2,blksize=8,overlap=8/2,pnew=32,idx=3)
bw3_vec2 = base2.MVAnalyse(isb=true, truemotion=false,delta=3,lambda=256,pel=2,sharp=2,blksize=8,overlap=8/2,pnew=32,idx=3)
bw2_vec2 = base2.MVAnalyse(isb=true, truemotion=false,delta=2,lambda=256,pel=2,sharp=2,blksize=8,overlap=8/2,pnew=32,idx=3)
bw1_vec2 = base2.MVAnalyse(isb=true, truemotion=false,delta=1,lambda=256,pel=2,sharp=2,blksize=8,overlap=8/2,pnew=32,idx=3)
fw1_vec2 = base2.MVAnalyse(isb=false, truemotion=false,delta=1,lambda=256,pel=2,sharp=2,blksize=8,overlap=8/2,pnew=32,idx=3)
fw2_vec2 = base2.MVAnalyse(isb=false, truemotion=false,delta=2,lambda=256,pel=2,sharp=2,blksize=8,overlap=8/2,pnew=32,idx=3)
fw3_vec2 = base2.MVAnalyse(isb=false, truemotion=false,delta=3,lambda=256,pel=2,sharp=2,blksize=8,overlap=8/2,pnew=32,idx=3)
fw4_vec2 = base2.MVAnalyse(isb=false, truemotion=false,delta=4,lambda=256,pel=2,sharp=2,blksize=8,overlap=8/2,pnew=32,idx=3)

bw4_comp2 = bobbed2.MVCompensate(bw4_vec2,idx=3)
bw3_comp2 = bobbed2.MVCompensate(bw3_vec2,idx=3)
bw2_comp2 = bobbed2.MVCompensate(bw2_vec2,idx=3)
bw1_comp2 = bobbed2.MVCompensate(bw1_vec2,idx=3)
fw1_comp2 = bobbed2.MVCompensate(fw1_vec2,idx=3)
fw2_comp2 = bobbed2.MVCompensate(fw2_vec2,idx=3)
fw3_comp2 = bobbed2.MVCompensate(fw3_vec2,idx=3)
fw4_comp2 = bobbed2.MVCompensate(fw4_vec2,idx=3)

Interleave(bw4_comp2,bw3_comp2,bw2_comp2,bw1_comp2,bobbed2,fw1_comp2,fw2_comp2,fw3_comp2,fw4_comp2)
fft3dfilter(sigma=0.5,bt=4,plane=0,degrid=1.0,sigma2=0.6,sigma3=0.7,sigma4=0.8)
SelectEvery(9,4)

maxx = bobbed.mt_logic(bw1_comp,"max")
\ .mt_logic(fw1_comp,"max")
minn = bobbed.mt_logic(bw1_comp,"min")
\ .mt_logic(fw1_comp,"min")
#.mt_logic(bw2_comp,"max").mt_logic(bw3_comp,"max")
#.mt_logic(fw2_comp,"max").mt_logic(fw3_comp,"max")
#.mt_logic(bw2_comp,"min").mt_logic(bw3_comp,"min")
#.mt_logic(fw2_comp,"min").mt_logic(fw3_comp,"min")

ssharp = mt_lutxy(last,last.RemoveGrain(11), \
"x x y - abs 12 / 1 4 / ^ 12 * x y - 2 ^ x y - 2 ^ 2.3 + / * x y - x y - abs 0.0001 + / * 1 x y - abs 16 / 1 4 / ^ + / +",U=2,V=2)

ssharp.mt_Clamp(maxx,minn,0,0,U=2,V=2)

interleave( selectevery(5,0).FluxSmoothT(7),selectevery(5,1).FluxSmoothT(7),
\ selectevery(5,2).FluxSmoothT(7),selectevery(5,3).FluxSmoothT(7),
\ selectevery(5,4).FluxSmoothT(7) )

#SeparateFields().SelectEvery(4,0,3).Weave() # re-interlace stage

SelectEven().trim(23470,23670)

return(last)


#=================================

# Nifty Gauss/Median combination
function MinBlur(clip clp, int r, int "uv")
{
uv = default(uv,3)
uv2 = (uv==2) ? 1 : uv
rg4 = (uv==3) ? 4 : -1
rg11 = (uv==3) ? 11 : -1
rg20 = (uv==3) ? 20 : -1
medf = (uv==3) ? 1 : -200

RG11D = (r==1) ? mt_MakeDiff(clp,clp.RemoveGrain(11,rg11),U=uv2,V=uv2)
\ : (r==2) ? mt_MakeDiff(clp,clp.RemoveGrain(11,rg11).RemoveGrain(11,rg11),U=uv2,V=uv2)
\ : mt_MakeDiff(clp,clp.RemoveGrain(11,rg11).RemoveGrain(20,rg20).RemoveGrain(20,rg20),U=uv2,V=uv2)
RG4D = (r==1) ? mt_MakeDiff(clp,clp.RemoveGrain(4,rg4),U=uv2,V=uv2)
\ : (r==2) ? mt_MakeDiff(clp,clp.medianblur(2,2*medf,2*medf),U=uv2,V=uv2)
\ : mt_MakeDiff(clp,clp.medianblur(3,3*medf,3*medf),U=uv2,V=uv2)
DD = mt_lutxy(RG11D,RG4D,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?",U=uv2,V=uv2)
(r==0) ? clp : clp.mt_MakeDiff(DD,U=uv,V=uv)
return(last)
}
#end

I basically tried to decrease the strength of the denoising and cut out as much bluring or smoothing as possible. I did notice that without minblur, the scenes would rapidly change from clean to noisy back and forth.

I would really love to know how this script works and maybe how to incorporate mvdegrain2 as the denoiser.

I have also seen the results from this post (http://forum.doom9.org/showthread.php?p=885124#post885124) which appear to have the exact same type of noise my video does.

Didée - It would be great to get that script :) :script:

Can i maybe get some suggestions on how to do things better ? Seriously - any time you spend looking at this is highly appreciated. This video means alot to me and so does other peoples help :)
DarkZell666
11th April 2007, 13:11
The result is stunning already :eek:

Maybe adding Undot() as the very last filter ? I see some orphan pixels here and there, or very fine grain (Salt & pepper ?) left by your script.

My knowledge stops there, I'm not really into MC'ed stuff yet =)
R3Z
11th April 2007, 13:35
The result is stunning already :eek:

Maybe adding Undot() as the very last filter ? I see some orphan pixels here and there, or very fine grain (Salt & pepper ?) left by your script.

My knowledge stops there, I'm not really into MC'ed stuff yet =)

I love the results, thanks of course to Didée. I just really dont understand the script that well. The changes i made are probably screwed.

I get weird effects such as it looks great with noise being killed but then when a scene changes you get a flash of the noisy again then it goes clean again. When the light flickers (especially when the strobes come onto kurt) it happens as well.
Didée
11th April 2007, 23:45
I just really dont understand the script that well. The changes i made are probably screwed.
Not screwed, no. In the middle part, some ressources go down the drain because more MVanalyze's & MVCompensate's are created+interleaved than the applied FFT3DFilter actually does need. In fact, the middle part seems a bit overdone - all that MV-stuff, only for an FFT3D with sigma=0.5? That's very, very much effort to achieve a tiny effect.

I get weird effects such as it looks great with noise being killed but then when a scene changes you get a flash of the noisy again then it goes clean again. When the light flickers (especially when the strobes come onto kurt) it happens as well.
No surprise. The script originally was made for a "static" source with limited motion. That's why the script is so simple. :D

The problem is that temporal denoisers (which is what this script basically is) usually don't denoise well at scenechanges, because the available temporal radius is shortened at the boundary frames. Usually this goes by unnoticed, but if the noise is strong and the radius of temporal denoising is big, then this effect becomes clearly noticeable.
And since the basic filter chain of the posted script in fact is *simple*, it suffers exactly that problem. "Simple" means that the "raw" MV-compensations directly are fed into the used denoisers, without any additional safety or error checks. Being fully at the mercy of MVCompensate's output, at times you're lost, and hence there are those "effects".

Somewhere in my script cellar there's that MV-denoising framework which tries to do it all, inclusive full automagic error correction, handle high noise, and denoise even at scenechanges and over strobo effects ... but even if it would be ready (which it is not):
That script is cOmPlIcAtEd. I'll never dare to post it ... if I would, they'd put me in that white jacket with the veeery long sleeves, and won't let me out for many years. :D
R3Z
12th April 2007, 01:11
Not screwed, no. In the middle part, some ressources go down the drain because more MVanalyze's & MVCompensate's are created+interleaved than the applied FFT3DFilter actually does need. In fact, the middle part seems a bit overdone - all that MV-stuff, only for an FFT3D with sigma=0.5? That's very, very much effort to achieve a tiny effect.

No surprise. The script originally was made for a "static" source with limited motion. That's why the script is so simple. :D

The problem is that temporal denoisers (which is what this script basically is) usually don't denoise well at scenechanges, because the available temporal radius is shortened at the boundary frames. Usually this goes by unnoticed, but if the noise is strong and the radius of temporal denoising is big, then this effect becomes clearly noticeable.
And since the basic filter chain of the posted script in fact is *simple*, it suffers exactly that problem. "Simple" means that the "raw" MV-compensations directly are fed into the used denoisers, without any additional safety or error checks. Being fully at the mercy of MVCompensate's output, at times you're lost, and hence there are those "effects".

Somewhere in my script cellar there's that MV-denoising framework which tries to do it all, inclusive full automagic error correction, handle high noise, and denoise even at scenechanges and over strobo effects ... but even if it would be ready (which it is not):
That script is cOmPlIcAtEd. I'll never dare to post it ... if I would, they'd put me in that white jacket with the veeery long sleeves, and won't let me out for many years. :D

:thanks:

I am really thankfull for your response :) You explained everything i wanted and needed to know.

I guess i just need to shelve this video for a while. I was up all night thinking of ways to overcome this problem, but with my limited technical knowledge its a little hard :p

Cheers mate :)
Fizick
12th April 2007, 05:39
did you try simple mvdegrain2 example from mvtools doc?

for luma flicker stability you may add dct=1 parameter.

function MVDegrain2i(clip "source", int "dct", int "idx")
{ # MVDegrain2i - motion compensated denoiser for interlaced source clip
# uses MVTools plugin
dct=default(dct,0) # use dct=1 for clip with light flicker
idx=default(idx,1) # use various idx for different sources in same script
fields=source.SeparateFields()
backward_vec2 = fields.MVAnalyse(isb = true, delta = 2, pel = 2, overlap=4, sharp=2, idx = idx,dct=dct)
forward_vec2 = fields.MVAnalyse(isb = false, delta = 2, pel = 2, overlap=4, sharp=2, idx = idx,dct=dct)
backward_vec4 = fields.MVAnalyse(isb = true, delta = 4, pel = 2, overlap=4, sharp=2, idx = idx,dct=dct)
forward_vec4 = fields.MVAnalyse(isb = false, delta = 4, pel = 2, overlap=4, sharp=2, idx = idx,dct=dct)
fields.MVDegrain2(backward_vec4,backward_vec2,forward_vec2,forward_vec4,thSAD=400,idx=idx)
Weave()
}

source=MPEG2Source("F:\Internet\070710\nirvana_chunk_1.d2v", cpu=0)
mvdegrain2i(source,1,1)


finally you may add to end of the script (without motion compensation):
fft3dfilter(sigma=1.5)
vcmohan
17th April 2007, 04:07
Try just varianslim()
The results would be almost same
Didée
17th April 2007, 12:06
Try just varianslim()
The results would be almost same
Example, please?

Note - do not *only* compare single frames side-by-side:
Checking temporal stability by frame-stepping is an interesting evaluation, too ... :)
vcmohan
18th April 2007, 02:33
The example is:
http://img89.imageshack.us/img89/9265/vslim0gd6.th.jpg (http://img89.imageshack.us/my.php?image=vslim0gd6.jpg)

The script used is

varianslim()
var2 = varianslim(minw = 17000)
stackvertical(var,var2).reduceby2()


There is still some grain compared to the processed image posted by R3z above. However I did not experiment with more aggressive values for various other params of varianslim.

Yes. Varianslim does not do temporal processing. However from a sample frame derived internal params process can be done over several frames. As I do not have a very fast connection I do not download long files.

I am particularly intrigued by the greenish band of the background in the top left half. May be its a real pattern.
Didée
19th April 2007, 09:48
The script used is

varianslim()
var2 = varianslim(minw = 17000)
stackvertical(var,var2).reduceby2()

Nah, this can NOT be the script that you used:
- you're stacking the clip "var", which is not defined anywhere
- "var2" has varianslim() as input, and does varianslim again. That's double processing whith that filter, and gives me exactly this result:

http://img133.imageshack.us/img133/3866/doubleslimqj5.th.jpg (http://img133.imageshack.us/my.php?image=doubleslimqj5.jpg)

(But then, even a single varianslim (minw=17000) does give me this overfiltered look.)
Do we agree that this is not what you showed in your (cheatingly downsized;)) example? Any chance you were a bit sloppy with posting your script? ;)

(BTW- applying a spatial filter to an interlaced source isn't really the recommended way ...)


Some small files: (for your ease of DL'ing...)

a snip of the source (http://www.mytempdir.com/1300694) (<1MB)

source.bob(0,0.5).varianslim(minw=8640) (http://www.mytempdir.com/1300658) (~3.5MB)

some spatio-temporal filtering (http://www.mytempdir.com/1300660) *) (~2.1MB)

*)source.fft3dfilter(sigma=12,bt=4,interlaced=true,bw=24,bh=24,plane=4)
bob(0,0.5).reduceflicker(strength=2).temporalrepair(source.bob(0,0.5))


This is not exactly "dedicated" filtering, don't mind. Something quickly hacked together, only to show that mere spatial filtering is pretty insufficient for sources like this one, where temporal filtering is so much more mighty.

The problem is not only "grain-like" noise (being of rather high spatial frequency). The problem is the general temporal instability/flickering. Spatial noise filtering acts mostly on high spatial frequencies. The temporal flicker - to me the more annoying part - is almost untouched by spatial filtering. Only temporal filtering is able to make the source calm, by removing the temporal flicker.


The green band probably is a VHS-usual chroma problem ... perhaps even related to some light effect in the background.

What "intrigued" me was the comment "varianslim would look almost the same", which is definetly not the case. In situations like this here, a spatial filter comes nowhere near the result of temporal filtering. There's lightyears between them.
vcmohan
20th April 2007, 03:35
what you showed in your (cheatingly downsized;) example?
I never intended to cheat. As on my 17" LCD screen unless I do a reduceby2() I do not see both frames to gether I reduced it.
Any chance you were a bit sloppy with posting your script? ;)
Yes. Sorry for that. For the first line var = was missed while copying from my notebook.

(BTW- applying a spatial filter to an interlaced source isn't really the recommended way ...)
I know. Since I did not find much difference between the fields I did not separate them.
But in many instances I find temporal processing is used while spatial alone would have done almost a near equally good job ( for my old eyes atleast). Of course there are situations (many) where one has to resort to temporal processing also.
As far as flickering is concerned theoritically homomorphic filtering in spatial domain should eliminate it.

Thanks for posting short clips and script for processing.