Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 13th February 2015, 17:12   #1  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,119
a simple approach for mosquito noise removal

here's my point of view about how to vaporize those goddamn MPEG ringing/mosquito noises from DVDs or other low bitrate clips
ah... mosquito noise, the most hated DVD bullcrap to me personally, hard to separate it from real image, if you ever try to remove it, it costs you twice more on detail losses, and artifacts, also

so, my personal solution...
first, mosquito noise is a high frequency artifacts, we don't wanna waste any low frequency detail for it, so, separate high frequencies from the image
Code:
clip=last
hf=dither_sub16 (clip,clip.dither_removegrain16 (11),dif=true)
"hf" is the high frequencies we want
now, let's remove those pains in the ass from high frequencies
use an insane ultra strong denoise filter on it
we can still save some hifi details since mosquito noise is pretty obvious, in another word, the dif is sort of strong..
we'll do it with dither_limit_dif16
and finally, we will repair the denoised high frequencies to fix side effects caused by insane denoise
Code:
difnr=hf.smdegrain (tr=12,thsad=2000,lsb_in=true,lsb_out=true,prefilter=1)
fdif=hf.dither_limit_dif16 (difnr,thr=3.0,elast=1.5).dither_repair16 (hf,13)
now, we apply the dif back to the low passed clip, and the job is done
Code:
clip.dither_removegrain16 (11).dither_add16 (fdif,dif=true)

source clip

result
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 13th February 2015, 18:07   #2  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,057
Looks promising. It did a great job with the overhead bins on the left side of the pic; less good on the ceiling covers on either side, and behind, the flight attendant's head. Still, a very good result, and I didn't see any obvious new artifacts.

As for your choice of test clip, the mosquito noise was isolated to just a few locations and, overall, not too bad. It would be interesting to see how well the approach would work on a lower bitrate encode with really serious noise.

The biggest problem with mosquito noise is that it is very nicely correlated with the image and therefore cannot be removed with any temporal solution. I've tried, unsuccessfully, to remove it using the multiple frequency sigma values in fft3D, but by itself, that didn't work. However, I wonder if fft3D might be a better tool for your first step, setting it to only remove only high frequency artifacts?
johnmeyer is offline   Reply With Quote
Old 14th February 2015, 07:44   #3  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,119
tried fft/dft stuff... and they suck....
tryna pull median filter in the game to protect edges from the harm of strong denoising
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 14th February 2015, 08:50   #4  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,119
new


Code:
clip=last
median=dither_median16 (1,1,0)
gaussian=dither_removegrain16 (11)
conv=dither_min_dif16 (median,gaussian,clip)
freq=dither_sub16 (clip,conv,dif=true)
pref=freq.smdegrain (tr=6,thsad=400,lsb_in=true,mode=-1,pel=1).smdegrain (tr=6,thsad=400,pel=1).smdegrain (tr=6,thsad=1200,pel=1)
cleanf=freq.smdegrain (tr=6,thsad=400,lsb_in=true,lsb_out=true,prefilter=pref,pel=1).smdegrain (tr=1,thsad=2000,lsb_in=true,lsb_out=true,prefilter=pref,pel=1,mode=-1)
finalf=freq.dither_limit_dif16 (cleanf,thr=1.0,elast=3.0)
conv.dither_add16 (finalf,dif=true).dither_repair16 (clip,9)

Function Dither_min_dif16 (clip input1, clip input2, clip ref)
{
    maxdif     = Dither_max_dif16 (input1, input2, ref, y=3, u=1, v=1)
    bin_stack  = mt_lutxy (input1, maxdif, "x y == 255 0 ?", y=3, u=1, v=1)
    bin_msb    = bin_stack.Dither_get_msb ()
    bin_lsb    = bin_stack.Dither_get_lsb ()
    bin        = mt_logic (bin_msb, bin_lsb, "min", y=3, u=1, v=1)
    
    output     = Dither_merge16_8 (input1, input2, bin, luma=False, y=3, u=1, v=1)

    return output
}
added median filter to protect sharp edges
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 14th February 2015, 11:04   #5  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,119
now.. finally
an almost ideal mosquito noise remover
Code:
clip=last
median=dither_median16 (1,1,0) # blur the image without destroying edges
gaussian=dither_removegrain16 (11) # pure low pass filter
conv=dither_min_dif16 (median,gaussian,clip) # low pass filter without destroying edges
conv=conv.dither_resize16nr (1440,960).dither_removegrain16 (11).dither_resize16 (720,480,a1=-1,a2=0,kernel="cubic") #blur it a tiny little bit, or the result's gonna suffer from raw high frequency artifacts
freq=dither_sub16 (clip,conv,dif=true) #high frequencies without edges
cleanf=freq.smdegrain (tr=6,thsad=400,lsb_in=true,mode=-1,pel=1).smdegrain (tr=6,thsad=400,pel=1).smdegrain (tr=6,thsad=1200,pel=1,lsb_out=true)# ultra strong denoising, remove mosquito noise from high frequencies
finalf=freq.dither_limit_dif16 (cleanf,thr=2.0,elast=3.0)#restore some high frequency details
conv.dither_add16 (finalf,dif=true).dither_repair16 (clip,9)#apply the fixed high frequencies back, and repair the result to fix side effects

Function Dither_min_dif16 (clip input1, clip input2, clip ref)
{
    maxdif     = Dither_max_dif16 (input1, input2, ref, y=3, u=1, v=1)
    bin_stack  = mt_lutxy (input1, maxdif, "x y == 255 0 ?", y=3, u=1, v=1)
    bin_msb    = bin_stack.Dither_get_msb ()
    bin_lsb    = bin_stack.Dither_get_lsb ()
    bin        = mt_logic (bin_msb, bin_lsb, "min", y=3, u=1, v=1)
    
    output     = Dither_merge16_8 (input1, input2, bin, luma=False, y=3, u=1, v=1)

    return output
}
EDIT: compares







EDIT2: sorry, I got no super low bitrate clips to test how it works on extreme cases...
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated

Last edited by feisty2; 14th February 2015 at 11:37.
feisty2 is offline   Reply With Quote
Old 14th February 2015, 12:04   #6  |  Link
TheSkiller
Registered User
 
Join Date: Dec 2007
Location: Germany
Posts: 637
Trying to run your latest script I get this error message.

Avisynth open failure:
Dither_sub16: clip "src2" has a height of 960 instead of 576.

I did change the resize part for my PAL test clip (which, by the way, is an old Britney DVB-S capture of mine with lots of mosquitos )


Quote:
Originally Posted by feisty2 View Post
vaporize those goddamn MPEG ringing/mosquito noises from DVDs or other low bitrate clips
Since when are DVDs low bitrate clips?
Mosquito noise is present in many DVDs, I agree, but that's mostly because whoever set up the encoder did not care and used some stupid standard settings such as very restricted max bitrate, low avg bitrate, standard MPEG matrix, unnecessary short GOPs, closed GOPs, alternate scan for progressive content... the list goes on and on.

Last edited by TheSkiller; 14th February 2015 at 12:08.
TheSkiller is offline   Reply With Quote
Old 14th February 2015, 12:16   #7  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,119
Quote:
Originally Posted by TheSkiller View Post
Since when are DVDs low bitrate clips?
anything fails to present transparent quality is low bitrate to me

below transparent - low bitrate
transparent - balanced bitrate
lossless (mathematically) - high bitrate
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 14th February 2015, 12:21   #8  |  Link
TheSkiller
Registered User
 
Join Date: Dec 2007
Location: Germany
Posts: 637
Nevermind, it works now. You should have noted that this is to be used with 16 bit stacked input.
Screenshots coming up.


Edit: OK, still doesn't work. The script loads but it's painfully slow (like 5 seconds per frame) and the frame borders are green and magenta.

Any idea what's the problem?

Last edited by TheSkiller; 14th February 2015 at 12:39.
TheSkiller is offline   Reply With Quote
Old 14th February 2015, 12:42   #9  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,119
Dither_min_dif16 throws all chroma values away (u=1,v=1)
use it only on plane Y
clip=last
median=dither_median16 (1,1,0) # blur the image without destroying edges
gaussian=dither_removegrain16 (11) # pure low pass filter
conv=dither_min_dif16 (median,gaussian,clip) # low pass filter without destroying edges
conv=conv.dither_resize16nr (1440,960).dither_removegrain16 (11).dither_resize16 (720,480,a1=-1,a2=0,kernel="cubic") #blur it a tiny little bit, or the result's gonna suffer from raw high frequency artifacts
freq=dither_sub16 (clip,conv,dif=true) #high frequencies without edges
cleanf=freq.smdegrain (tr=6,thsad=400,lsb_in=true,mode=-1,pel=1).smdegrain (tr=6,thsad=400,pel=1).smdegrain (tr=6,thsad=1200,pel=1,lsb_out=true)# ultra strong denoising, remove mosquito noise from high frequencies
finalf=freq.dither_limit_dif16 (cleanf,thr=2.0,elast=3.0)#restore some high frequency details
conv.dither_add16 (finalf,dif=true).dither_repair16 (clip,9).mergechroma (clip) #apply the fixed high frequencies back, and repair the result to fix side effects

Function Dither_min_dif16 (clip input1, clip input2, clip ref)
{
maxdif = Dither_max_dif16 (input1, input2, ref, y=3, u=1, v=1)
bin_stack = mt_lutxy (input1, maxdif, "x y == 255 0 ?", y=3, u=1, v=1)
bin_msb = bin_stack.Dither_get_msb ()
bin_lsb = bin_stack.Dither_get_lsb ()
bin = mt_logic (bin_msb, bin_lsb, "min", y=3, u=1, v=1)

output = Dither_merge16_8 (input1, input2, bin, luma=False, y=3, u=1, v=1)

return output
}

EDIT: the main speed sucker is the smdegrain chain, it's just a raw concept, I didn't do any optimization for it, it's not even wrapped in a function...
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated

Last edited by feisty2; 14th February 2015 at 12:49.
feisty2 is offline   Reply With Quote
Old 14th February 2015, 13:38   #10  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,119
wrapped up!
Code:
Function MosquitoBuster_Get (clip input, bool "freqflag")
{
  freqflag   = Default (freqflag, False)
  w          = input.width ()
  h          = input.height ()/2

  median     = input.dither_median16 (1, 1, 0)
  gaussian   = input.dither_removegrain16 (11, 11, 11)
  conv       = dither_min_dif16 (median, gaussian, input)
  conv       = conv.dither_resize16nr (w*2, h*2).dither_removegrain16 (11, 11, 11).dither_resize16 (w, h, a1=-1, a2=0, kernel="cubic")
  freq       = dither_sub16 (input, conv, dif=true)

  output     = freqflag ? freq : conv

  return output
}

Function MosquitoBuster_Go (clip input, clip "conv", clip "freq", clip "filtered", float "thr", float "elast", int "repmode")
{
  thr        = Default (thr, 2.0)
  elast      = Default (elast, 3.0)
  repmode    = Default (repmode, 9)

  finalf     = freq.dither_limit_dif16 (filtered, thr=thr, elast=elast)
  apply      = conv.dither_add16 (finalf, dif=true)
  rep        = apply.dither_repair16 (input, repmode, repmode, repmode)

  return rep
}

Function Dither_min_dif16 (clip input1, clip input2, clip ref)
{
  maxdif    = Dither_max_dif16 (input1, input2, ref, y=3, u=3, v=3)
  bin_stack = mt_lutxy (input1, maxdif, "x y == 255 0 ?", y=3, u=3, v=3)
  bin_msb   = bin_stack.Dither_get_msb ()
  bin_lsb   = bin_stack.Dither_get_lsb ()
  bin       = mt_logic (bin_msb, bin_lsb, "min", y=3, u=3, v=3)

  output    = Dither_merge16_8 (input1, input2, bin, luma=False, y=3, u=3, v=3)

  return output
}

#demo usage
#conv=MosquitoBuster_Get (freqflag=False)
#freq=MosquitoBuster_Get (freqflag=True)
#filtered=freq. #your fav ultra strong denoise filter#
#MosquitoBuster_Go (last, conv, freq, filtered)
EDIT: You'll get exact the same result like those compares above with the script beneath, and it processes chroma too
Code:
conv=MosquitoBuster_Get (freqflag=False)
freq=MosquitoBuster_Get (freqflag=True)
filtered=freq.smdegrain (tr=6,thsad=400,lsb_in=true,mode=-1,pel=1).smdegrain (tr=6,thsad=400,pel=1).smdegrain (tr=6,thsad=1200,pel=1,lsb_out=true)
MosquitoBuster_Go (last, conv, freq, filtered)
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated

Last edited by feisty2; 14th February 2015 at 13:54.
feisty2 is offline   Reply With Quote
Old 14th February 2015, 14:09   #11  |  Link
TheSkiller
Registered User
 
Join Date: Dec 2007
Location: Germany
Posts: 637
Alright. So I used FFT3dFilter(sigma=7, interlaced=true) for my test clip. I have to say, there's detail loss and the ringing is still mostly there. Admittedly, this is a quite heavily compressed video, suffering not only from mosquito but also from block noise.

TheSkiller is offline   Reply With Quote
Old 14th February 2015, 14:36   #12  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,119
ah, told ya fft/dft stuff sucks, it just ain't the right job for them
I don't have fft3d here, I'll show you with dfttest

shit's still there even with sigma=128, then sure it's useless, cuz the filtered clip is pointless
the script aims to remove mosquito noise without too many detail losses
but first, you have to use a filter that actually removes mosquito noise, then the script would try to recover something to make it less crappy
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 14th February 2015, 15:08   #13  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,119
@TheSkiller
found this clip from a reeeaaally old DVD, shitty like hell, but still better than ur test clip



it still suffers from blocks, the script's not designed to remove blocks anyways
I can't see much mosquito noise from either this clip or your clip, both of them are just "way too crappy to even contain any noise other than blocks"
I wrote this script aiming at removing mosquito noise that would become residual noisy spots or whatever after a general denosing, run the script before general denosing, then result would be clean.
for shitty clips like these... a deblock filter is required..
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated

Last edited by feisty2; 14th February 2015 at 15:10.
feisty2 is offline   Reply With Quote
Old 21st February 2015, 21:26   #14  |  Link
martin53
Registered User
 
Join Date: Mar 2007
Posts: 405
Quote:
Originally Posted by feisty2 View Post
found this clip from a reeeaaally old DVD
Besides the blocking, there are striking compression artefacts at the shoulder, the belly and the lower jaw. I also look for a filter to remove those rough artefacts in otherwise smooth areas, but yet didn't find a breakthrough.

What I did and what might be a suggestion for further experiments is a mask around edges, because the artefacts come from the compression of edges and thus are located around them.

Basically, I do it like this

cMask = c.mt_edge(mode="min/max", thY1=28, thY2=38, chroma="128")
cMask = cMask.mt_expand(mode=mt_circle(10))
mt_merge(c, cF, cMask)


where cF is a filtered clip - the ideal filter still to find. However, areas not close to edges are saved from distorting blur.

cF=TfNLMeans(c, Ax=6, Ay=6)

yields the best result of the filters I compared yet (which is not to say it is good, only the others are worse), but is, alas, very slow.
martin53 is offline   Reply With Quote
Old 22nd February 2015, 04:37   #15  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,119
tried those mask stuff and.... didn't work quite well, gave'em up finally
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 23rd February 2015, 11:03   #16  |  Link
colours
Registered User
 
colours's Avatar
 
Join Date: Mar 2014
Posts: 312
Quote:
Originally Posted by martin53 View Post
cMask = cMask.mt_expand(mode=mt_circle(10))
This is slow because you're effectively doing roughly 300 maximum operations per pixel and there's no SIMD code path for custom mt_xxpand modes. Use mt_xxpand_multi instead (which comes with Dither tools).

Code:
cMask = cMask.mt_expand_multi(mode="ellipse",sw=10,sh=10)
The only caveat is that this does an octagonal dilation of your mask rather than a circular dilation, but it should be about six hundred times faster.
__________________
Say no to AviSynth 2.5.8 and DirectShowSource!
colours is offline   Reply With Quote
Old 23rd February 2015, 13:59   #17  |  Link
mawen1250
Registered User
 
Join Date: Aug 2011
Posts: 104
I wonder what's your purpose to use thSAD=1200 for the last pass of SMDegrain.
Also I suppose there should be a 'prefilter=conv.DitherPost(mode=-1)' in SMDegrain which will produce meaningful motion vectors.

Last edited by mawen1250; 23rd February 2015 at 14:06.
mawen1250 is offline   Reply With Quote
Old 23rd February 2015, 14:15   #18  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,119
Quote:
Originally Posted by mawen1250 View Post
I wonder what's your purpose to use thSAD=1200 for the last pass of SMDegrain.
you simply can't resolve MPEG ringings at thSAD=400 (maybe would work on other clips with softer noises, but not on this one)




Quote:
Originally Posted by mawen1250 View Post
Also I suppose there should be a 'prefilter=conv.DitherPost(mode=-1)' in SMDegrain which will produce meaningful motion vectors.
maybe, I'll try
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 14:15.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.