Denoising 1080p content in a realistic amount of time

[nibus]

I just got a BluRay drive and have started backing up my small collection.

I'm trying to get each movie on a single DVD-R, so I would like to use some denoising filters.

Here's what I've tried, and what I've used on pretty much everything up until this point (with adjustments of course):

(I believe SMDegrain is just a quick way of using MVDegrain for the code-impaired individual like myself.)



AVCSource("F:\Encode\BluRay\X2\BDMV\STREAM\X2.dga")

crop( 0, 140, 0, -140)

LoadPlugin("C:\Program Files (x86)\Multimedia\AviSynth 2.5\plugins\FluxSmooth.dll")
FluxSmoothT(temporal_threshold=2)

SMDegrain(tr=3, thSAD=300)

LSFMod(defaults="slow",ss_x=1.0,ss_y=1.0,Smode=5,strength=30)



This gets me to about the point where I think I can squeeze it all on one disc without losing much quality.

Problem is it is VERY slow... less than ~2fps on a i7 930 w/6gb triple channel ram.

Can anyone recommend some denoising filters that will get me in the same ballpark as the above settings, but at least twice as fast?

If it's not possible then I'll just double the bitrate and split the files into two DVD's, but I just hate having to mess with discs in the middle of a movie. :mad:

[Dark Shikari]

hqdn3d? fft3dgpu?

[nibus]

hqdn3d? fft3dgpu?

I'll check those out. Can you give me a basic usage example for fft3dgpu for light denoising?

[nibus]

In case anyone is looking for something similar, I found one that has worked very well on my BluRay sources - TTempSmooth. And it's not horrendously slow. On an i7 930 it takes about 24hrs for one encode, which to me is just about bearable.

It calms the noise on the stationary parts of the picture (ie walls / ceilings etc) but leaves the moving parts of the picture alone.

So far it's done a great job. My biggest beef with denoising is it can really smooth things out too much.


This is how I've used it:
TTempsmoothF(maxr=2, strength=3)

[linyx]

MVTools2 on an i7 930 should at least be double that, I would surmise. Heck, with the script below, I get ~2 fps @ 1080p (rendering, not encoding) on an Athlon 64 X2 4400.

My recommendation for keeping some grain and being reasonably fast would be something like this:
#Source
MT("""
Super = MSuper(pel=2, sharp=1)
Backward_Vector = MAnalyse(super, isb = true, delta = 1, overlap=4)
Forward_Vector = MAnalyse(super, isb = false, delta = 1, overlap=4)
MDegrain1(super, Backward_Vector,Forward_Vector,thSAD=250)""",4,4)

To use the MT (MultiThreading), you will need MT avisynth (http://forum.doom9.org/showthread.php?t=148117) which is fairly easy to install, provided you follow the directions (http://avisynth.org/mediawiki/MT_support_page). You *may* want to raise the value of the first 4 in the last line to however many threads your computer is able to run--I don't know what a 930 can do, maybe 8?

For removing more or less grain, first change the parameter "thSAD=250" to higher to remove more (upto ~450) and lower to remove less; to more "cleanly" remove more, I would suggest using more frames (i.e. MDegrain2 or MDegrain3 (http://avisynth.org.ru/mvtools/mvtools2.html#examples)--look at, "To denoise by MDegrain2 with overlapped blocks (blksize=8) and subpixel precision:").

[Blue_MiSfit]

If you're okay with speeds that are ~2x faster than your current script, lynx's copy-paste example of vanilla MDegrain1 is a great place to start. I typically go whole-hog and use MDegrain2 with thSAD=400, but I have a lot of 8 core servers at my disposal :)

I would experiment with increasing the thread count to 8, since you have 8 logical cores on your i7 920 CPU.

If you want some "free" denoising, there are some options to explore.

DeGrainMedian on defaults is incredibly fast and does a decent job on boosting compressibility without killing TOO much detail or risking major artifacts.

I often use DeGrainMedian(6,12,1), which is quite a bit stronger. It helps a lot when you're encoding 1080p at less than 3mbps, and can't afford any speed hit.

fft3dgpu is cool, but doesn't seem to run so fast on new GPUs... Dunno. It's at least real-time on 1080p when using my Radeon 4830, but my Quadro FX 2800M isn't any faster (and it's TONS faster in other apps)

FFT3DGPU(plane=3, sigma=2) denoises only the chroma planes
FFT3DGPU(plane=4, sigma=2) denoises all the planes, with linear strength
FFT3DGPU(plane=4, sigma=1, sigma2=2, sigma3=2.25, sigma4=1.5) denoises all the planes with non linear strength (kind of a bell curve). I find this approach usually gives better results.

fft3dgpu is very powerful. Sometimes too much so. It's very important if you're thinking of doing strong denoising to set the values yourself for each encode. If not, you're going to bleach out the image, and end up with plastic banding all over the place :devil:

sigma<=1 is usually okay in all cases, unless you're trying to be totally transparent (and can spend the bits to do so). I almost always clean up chroma grain.

It drives me nuts, and is easy with fft3dgpu.

Also keep in mind that fft3dgpu doesn't play nice with MT AviSynth last I checked. It is also broken by Remote Desktop (as is anything GPU dependent)

Derek

[aegisofrime]

fft3dgpu frequently spews out out of texture memory errors for 720p and above content on my 512MB 4870. I don't think there are that many 1GB 4830s are there Blue Misfit?

[Blue_MiSfit]

Mine is 512, and it works fine. Heck, my old 7800gt was just fine with 720p even with 256mb of RAM!

I only use one instance, and no MT. What about yourself?

[aegisofrime]

I use it together with TempGaussMC like so:


SetMTMode(2,4)
DGdecode_mpeg2source("D:\Raws\Work.d2v")
tempgaussmc_beta1u(1,1,0,0,0,0,edimode="--",SVthin=0.0,pelsearch=1)

Distributor()

SetMTMode(5,4)
FFT3DGPU(mode=2,precision=2)

The SetMTMode(5,4) is there so as to avoid funny artifacts in the video, following the advice in this post:

But not all is lost. SetMTMode(5,0) may be inserted before line with FFT3DGPU and then script should be working.

[TheRyuu]

fft3dgpu gets about 48fps (2x realtime?) on 1080p for me on my ATI 5870 (that's with precision=2 (slowest), mode=1).
With the default settings (precision=0, mode=1) I get about 68fps although I may be decoding speed bound (ffmpeg-mt via dss2), not sure.
I'm not exactly sure what adding SetMTMode(5,0) will do to it. My guess is that it will either have no effect or it will make it slower. I can't seem to test it, avisynth likes to crash when I try it. I'll have to go find a different source to try and test it on or maybe try using dgavcindex with this source.

In comparison fft3dfilter gets about 8.6fps using ncpu=8. Using MT("",8,8) it gets ~16fps (not sure if there are visual artifacts though). SetMTMode didn't work (probably due to dss2).

Settings used (precision and mode omitted when testing fft3dfilter and ncpu left out while testing fft3dgpu, other than that I think they have the same defaults for the rest, please correct me if I'm wrong):
fft3dgpu(sigma=2.0, bw=32, bh=32, ow=16, oh=16, bt=3, precision=0, mode=1)

And just for comparison purposes, dfttest with:
dfttest(threads=8,opt=3,tbsize=3) #tbsize set so it's 'like' bt=3, tbsize default is 5 (slower)
Gets ~2.9fps.

All testing done with avs2avi (-o n -c null). CPU is an i7-920 @ 3.8ghz.

Edit:
Even with dgavcindex the script still doesn't seem to work with setmtmode probably due to have setmtmode works and it just runs out of vram or something I guess (5870 has 1GB of vram).

[dansrfe]

I don't quite understand how to properly use Distributor in conjunction with setmtmode and MT because usually when I put setmtmode(2) at the start of my script and Distributor() at the bottom of my script. It crashes about 20 seconds into encoding, but it's fast as hell for those 20 seconds.

[SilaSurfer]

FFT3DGPU(plane=4, sigma=1, sigma2=2, sigma3=2.25, sigma4=1.5) denoises all the planes with non linear strength (kind of a bell curve). I find this approach usually gives better results.

Blue Misfit

Would you be so kind please to explain to me how do you choose max strength for sigma2, sigma3 and sigma4. Is there any special procedure how you came to pick those values. FFt3Gpu is great, but I'm only using it with standard calling you know

FFT3dgpu(plane=4, sigma=1, bt=4, bw=32, bh=32, ow=16, oh=16)
Changing values for plane and sigma
.
Thanks in advance

SilaSurfer