Denoiser producing the least artifacts (again?)

[PiXuS]

I searched and searched and (yes!) made my own tests, but I still
can't figure out which denoiser gives the be.. hmmm.. most not bad
picture quality.

I tried:


Deen (default params)
FluxSmoothT(4)
FluxSmoothT(5,7)
VagueDenoiser(threshold=0.8,method=1,nsteps=6,chroma=true)
TemporalSoften(2,3,3,mode=2,scenechange=6)
MergeChroma(blur(1.3))
Convolution3D("moviehq")
STMedianFilter (default params)
RemoveGrain(1)


My goal is to remoive a small quantity of noise on my DVD-Backups to
increase compressability while maintaining very good picture quality.
I am using DVD+R for backup purposes so file size isn't much of an
issue (but still, I don't like the idea of wasting bits on noise that
I don't like in the first place).

I liked Deen very much. It gave me the best compression rate and a
very good picture quality. FluxSmoothT(4), MergeChroma,
TemporalSoften and Convolution3D aren't bad at all too.

The thing is, I wouldn't bother you with this post if it wasn't for
the fact that I don't have as much experience as some of you do. By
searching on the forums I noticed some people are reporting problems
with artifacts.

So without asking which one is the BEST.. what about which one is the
least susceptible to generate unwanted artifacts?

Thanks!

[pelle412]

My suggestions:

PixieDust(limit=2)

or

PeachSmoother()

Both are in the same class when it comes to compressibility gain vs. picture quality but PeachSmoother is probably 5 times faster. If time is not an issue, PixieDust has the upper edge.

[PiXuS]

Originally posted by pelle412
My suggestions:

PixieDust(limit=2)

or

PeachSmoother()

Both are in the same class when it comes to compressibility gain vs. picture quality but PeachSmoother is probably 5 times faster. If time is not an issue, PixieDust has the upper edge.

Geez, I was going to reply saying PixieDust doesn't work even when using the LoadPluginEx2.dll plugin (whenever PixieDust() was called VirtualDubMod 1.5.10.1/2439 would crash. But I had read Didée is in LOVE with this plugin so I tried and tried and tried to make it work).

I JUST found out that you need to convert to the YUY2 color format for PixieDust to work!

So, for people trying PixieDust but unsuccessful and too shy to ask why, here is the recipe:


LoadPlugin("C:\AviSynthPlugins\LoadPluginEx2.dll")
LoadPlugin("C:\AviSynthPlugins\DustV5.dll")

mpeg2source()
ConvertToYUY2()
PixieDust(5)


p.s.: I am using testing PixieDust right now. Thanks!!

[pelle412]

If your source is clean you can probably go even lower on PixieDust limit. Compared to many denoisers, the PixieDust limit isn't a treshold for detecting noise, it's a limit on how much PixieDust can alter the image to reduce noise. I've found a limit of 2 works well for clean DVD sources, and 5 to be reasonable on TV captures.

[PiXuS]

Originally posted by pelle412
If your source is clean you can probably go even lower on PixieDust limit. Compared to many denoisers, the PixieDust limit isn't a treshold for detecting noise, it's a limit on how much PixieDust can alter the image to reduce noise. I've found a limit of 2 works well for clean DVD sources, and 5 to be reasonable on TV captures.

I just made a test with PixieDust(5) and PixieDust(2). Wow. The
increase in compressability they provide is even greater than Deen()
with it's default settings. (!) PixieDust(5) gave me a file size of
52% the size of the original (unfiltered, only resized) and
PixieDust(2) 56%. PixieDust(2) seems, indeed, very interesting !
But, why is crusty saying PixieDust generates artifacts
(http://forum.doom9.org/showthread.php?s=&postid=297959#post297959)?

I found this post from Didée regarding his use of PixieDust:

http://forum.doom9.org/showthread.php?s=&postid=319313#post319313

His filter chain is the following:

crop(8,80,-8,-80).undot().LanczosResize(640,272).undot()
\ .ConvertToYUY2().pixiedust(limit=2).ConvertToYV12()

Shouldn't the conversion to the YUY2 color format be done first
(before cropping?). And is the return to the YV12 color format
mandatory before feeding to XviD?

Thanks a lot !

[pelle412]

My choice of avisynth sequence:

1. Deinterlacing or IVTC
2. Denoiser
3. Crop
4. Resize
5. Post resize denoiser (e.g. UnDot)

Example:

#TomsMoComp(1,5,1) # if interlaced source
ConvertToYUY2().PixieDust(limit=2)
crop()
LanczosResize()
Undot()

You don't have to convert back to YV12. I think XviD takes care of that automatically.

[Didée]

Originally posted by PiXuS
I found this post from Didée regarding his use of PixieDust:

http://forum.doom9.org/showthread.php?s=&postid=319313#post319313

His filter chain is the following:

crop(8,80,-8,-80).undot().LanczosResize(640,272).undot()
\ .ConvertToYUY2().pixiedust(limit=2).ConvertToYV12()

Shouldn't the conversion to the YUY2 color format be done first
(before cropping?). And is the return to the YV12 color format
mandatory before feeding to XviD?
Ah, that's not my "standard" usage of Dust - that was by the time when I was fighting with putting LOTR-FOTR SEE onto 1 CD, hence I applied PixieDust after the resizing. Under "normal" conditions, I'd advice to use Dust before any resizing.

The placing of the colorspace conversion doesn't play much of role. It's just that my processing chains usually work in YV12 from start to end, except for Dust. So I convert directly before and after it.

BTW, I'm not in LOVE with that plugin (what should I tell my wife? ;) ). I like Dust's denoising capabilities. And I hate its speed - but that's an argument of zero weight.

Regarding Dust's potential of producing artefacts, this thread (http://forum.doom9.org/showthread.php?s=&threadid=78959) might be of interest.

[Didée]

Originally posted by pelle412
Example:

#TomsMoComp(1,5,1) # if interlaced source
ConvertToYUY2().PixieDust(limit=2)
crop()
LanczosResize()
Undot()

You don't have to convert back to YV12. I think XviD takes care of that automatically.
Sure, you don't have to. But as long as you start out with a source being YV12, you should keep it YV12 as long as possible. There is no benefit from going YUY2 (exept for some very specific processing thingies) - but most filters are working *faster* in YV12 than in YUY2.

edit: Also, it's a good idea to put undot() before PixieDust. The SpaceDust-part of PixieDust tends to let stay some "hot" pixels, so Dust becomes a little more effective if they're removed beforehand.

[PiXuS]

@Didée

Thanks a billion for the precise information. I think PixieDust will become my filter of choice.

I will use the following chain filter when using PixieDust:

ConvertToYUY2()
PixieDust(2)
ConvertToYV12()

BTW.. any idea regarding my SharpResize thread?

Edit
p.s.: undot() before pixiedust() after having done the color conversion or after (or doesn't matter) ?

[Didée]

Yep, doesn't matter. Therefore before the conversion.

For your SharpResize'ing - yes, something is really off there. But I've to leave and are out of time, so in a nutshell:

SharpResize is known to alter the colors - it's probably better to disable the chroma processing. But in your pictures there is also something pretty wrong with the luma processing. Dunno what's going on there.

You might want to give a try on LimitedSharpen - the result is pretty similar to SSXSharpen, but it's a little faster ;)

Bye

[PiXuS]

Originally posted by Didée
Yep, doesn't matter. Therefore before the conversion.

For your SharpResize'ing - yes, something is really off there. But I've to leave and are out of time, so in a nutshell:

SharpResize is known to alter the colors - it's probably better to disable the chroma processing. But in your pictures there is also something pretty wrong with the luma processing. Dunno what's going on there.

You might want to give a try on LimitedSharpen - the result is pretty similar to SSXSharpen, but it's a little faster ;)

Bye

Well, actually, I have found this (http://forum.doom9.org/showthread.php?s=&postid=444415#post444415) post where you mention that Unfilter darkens the picture. I verified with VirtualDubMod and, indeed, it does.

So now, I decided to remove both SharpResize and Unfilter from my basic "template" avs script because of the decoloration they induce.

I will look into LimitedSharpen. Though, I don't really care about the lenght of encoding. My puny PentiumIV-M can work all day. We have electricity to burn around here. ;-)

BTW, I wanted to thank you so much for SixOfNine. Not too long ago, I thought the MPEG QM was good enough and that using a CQM was too "dangerous" and would only result in minor picture quality difference. How wrong was I! I mostly use constant quants @ 4. The jump in file size is rather big though... 150% increase from q4 to q3 and 200% increase from q3 to q4 (from my very limited testing). And yes, I know the quants of SixOfNine don't map to the quants of the MPEG matrix. That's why I am happy with either 3 or 4.

Again, thanks !

[PiXuS]

@Didée

Is this (http://forum.doom9.org/showthread.php?s=&postid=532809#post532809) the latest version of LimitedSharpen? I want to give it a shot before doing another encode.

And LimitedSharpen(ss_x=2.0,ss_y=2.0,Smode=2,strength=100,Lmode=1,overshoot=1,wide=false,soft=false,radius=1,edgemode=0) is still recommended to get near 4*-supersampled-XSharpen() results?

Edit: Also, the way to call it in the filter chain looks like the following?

Crop().Undot().Lanczos4Resize().LimitedSharpen().Undot()

I mean.. my clip is originally 720x480 big and I want it to be 640x464 big. So I need to resize it to 640x464 before feeding it to LimitedResize, right?

Again, thanks. ;-)

PiXuS

[PiXuS]

I tried LimitedSharpen with the parameters given in my previous post. I encoded 2% of the "SNL 25th Anniversary" DVD.

Filesize with XSharpen was: 41.2 MB
Time to encode with XSharpen was: 74 minutes

Filsize with LimitedSharpen was: 44.4 MB (108% bigger than XSharpen)
Time to encode with LimitedSharpen was: 51 minutes (69% of the time used by XSharpen)

Here is a comparision between... well... you click on the picture and see. ;-)

http://img2.exs.cx/img2/7509/comp1.th.png (http://img2.exs.cx/my.php?loc=img2&image=comp1.png)

In the following picture, you can see the difference between the LimitedSharpen and the XSharpen pictures.

http://img68.exs.cx/img68/1657/diff.th.png (http://img68.exs.cx/my.php?loc=img68&image=diff.png)

Conclusion: LimitedSharpen rocks. ;-)

PiXuS

p.s.: the script used for the encodes is the following (of course the resizing section is modified accordingly!) :


# PLUGINS
LoadPlugin("C:\PROGRA~1\GORDIA~1\AviSynthPlugins\dgdecode.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\AviSynthPlugins\UnDot.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\AviSynthPlugins\TomsMoComp.dll")

LoadPlugin("C:\AviSynthPlugins\warpsharp.dll")
LoadPlugin("C:\AviSynthPlugins\LoadPluginEx.dll")
LoadPlugin("C:\AviSynthPlugins\DustV5.dll")
LoadPlugin("C:\AviSynthPlugins\MaskTools.dll")

Import("C:\AviSynthPlugins\LimitedSharpen.avs")

# VARIABLES
widthResized = 640
heightResized = 464
superSampling = 4
pixieDustLimit = 2

# SOURCE
mpeg2source("C:\dvd-rips\SNL_25\VIDEO_TS\snl25.d2v")

# CROPPING
crop(0,0,720,480)

# DEINTERLACING (1)
TomsMoComp(1,5,1)

# DENOISING: choose one combination (or none)
Undot()

ConvertToYUY2()
PixieDust(limit=pixieDustLimit)
ConvertToYV12()

# RESIZING
Lanczos4Resize(widthResized,heightResized)
LimitedSharpen(ss_x=2.0,ss_y=2.0,Smode=2,strength=100,Lmode=1,overshoot=1,wide=false,soft=false,radius=1,edgemode=0)

# MORE DENOISING: choose one combination (or none)
Undot()

# COMPRESSIBILITY CHECK
SelectRangeEvery(700,14)


Edit: Oh yeah... SixOfNine-HVS with q=5 was used
Edit#2: Added picture to show the diff. between LimitedSharpen and 4X Sharpen

[Chainmax]

Some time ago I read somewhere that the Dust filters replace the last frame with the penultimate one, is that true? If so, is there a workaround for that?