Which DeNoise/DeGrain Filter Should i use?

[tasusq]

Hi,

I am trying to encode the movie Elektra using MeGUI and DXVA-SD-HQ profile. But the problem is, even after reading countless threads and using different filters i have not been able to determine which Filter should i use to remove artifacts/noise from the Video without losing detail.

I have tried doing some test encodes using different filters but the result wasn't good. In all of them i used Deblock -1:=1 but still in all of my test encodes the video appeared a bit blocky. Take a look at the screenshot:

http://i39.tinypic.com/os8xew.png

http://i39.tinypic.com/ankh83.png

I used this script:

Import("C:\Program Files\AviSynth 2.5\plugins\SeeSaw.avs")
global MeGUI_darx = 160
global MeGUI_dary = 69
DGDecode_mpeg2source("C:\Movies\Elektra\Elektra_Full.d2v", info=3)
ColorMatrix(hints=true, threads=0)
SmoothLevels(gamma=1.0, chroma=50, limiter=0, Lmode=0, DarkSTR=75, Smode=-2, useGF=true, RGmode=11)
crop( 0, 54, -2, -58)
Lanczos4Resize(720,368)
fft3dGPU(mode=1,precision=2, sigma=4, bt=3, bw=32, bh=32, ow=16, oh=16, sharpen=0.1)
a= last
b=a.DeGrainMedian(limitY=5,limitUV=7,mode=0)
SeeSaw(a,b, NRlimit=3, NRlimit2=4, Sstr=1.5, Slimit=5, Spower=5, Sdamplo=6, Szp=16)
Trim(3505,126730)
SelectRangeEvery(20000,1049)

I just want to make it look clean and detailed.

I am still having problems with a lots of scripts here because some of Filters were updated but the scripts weren't. Like the scripts which used the old MVtools and now i cannot make them work with the new MVtools.

Please give me your suggestions on how can i make the video look excellent after ripping and which filters and MeGUI settings should i use? If possible point me to the link of the filter/setting/script/Custom Matrice and how should i use it.

Here is a small part of the DVD VOB file as a sample:

http://hotfile.com/dl/7592591/01d44b4/Elektra_Sample.rar.html

[Adub]

Honestly, that sources is pretty much perfect already. You can pretty much remove all of your degraining.

The only thing that I *might* consider filtering with (just to calm it down a bit) is using something like:
TTempsmoothF(maxr=2, strength=3)

It just calms down the parts of the picture that aren't moving. Other than that, your source is pretty darn clean.

[lansing]

fft3dGPU will cause banding if you set it too high, that's what you have in your pictures. For deband look here (http://forum.doom9.org/showthread.php?t=108681)
.

For the denoising part, if speed is not the issue for you, try TemporalDegrain, MCTemporalDenoise, or MC_Spuds, either one will give good result

[Nightshiver]

As adub said above, you honestly do not need much of any degraining or any filtering for that matter.

[tasusq]

Thanks a lot Adub, lansing & Nightshiver.

I tried what you said and it worked. Can you also tell me which sharpener should i use? Or should i not sharpen at all?

[Chainmax]

For clean sources (and by that I mean clean without denoising), SeeSaw is pretty cool. For a more general-use sharpener, nothing beats LimitedSharpenFaster IMHO.

[tasusq]

Hi i did another test encode with the following setting:
Import("C:\Program Files\AviSynth 2.5\plugins\GradFunkMirror.avs")
Import("C:\Program Files\AviSynth 2.5\plugins\GradFun2DBmod.v1.3.avsi")
global MeGUI_darx = 47
global MeGUI_dary = 20
DGDecode_mpeg2source("C:\Movies\Elektra\Elektra_Full.d2v", info=3)
ColorMatrix(hints=true, threads=0)
crop( 0, 54, -2, -58)
Spline36Resize(704,368) # Lanczos4 (Sharp)
TTempsmoothF(maxr=7, strength=7)
Function gradfunkmirror( clip c, int "strength" )
{
strength = default( strength, 1.2 )
w = c.width()
h = c.height()
vflip = c.FlipVertical()
hflip = c.FlipHorizontal()

stackhorizontal( hflip.crop( w-16, 0, 16, h ).addborders( 0, 16, 0, 16 ),
\ stackvertical( vflip.crop( 0, h-16, w, 16 ), c, vflip.crop( 0, 0, w, 16 ) ),
\ hflip.crop( 0, 0, 16, h ).addborders( 0, 16, 0, 16 ) )
gradfun2db( strength )
crop( 16, 16, -16, -16 )

Return last
}
fft3dGPU(mode=1,precision=2, sigma=1.5, bt=3, bw=32, bh=32, ow=16, oh=16, sharpen=0.4)
AddGrain(1,0,0)
Trim(3505,126730)
SelectRangeEvery(20000,1049)

In the encoded video there is a thin line on the top.
Please take a look at the screenshot:
http://i41.tinypic.com/j12iqo.png

I have done several test encodes and it is in all of them. It is not in the source and i don't have any idea why it appears after encoding.

Am i using the filters in the right order? Can anyone please tell me how can i get rid of that thin line from the video?

[TiGR]

In the encoded video there is a thin line on the top.

When you’re getting artifacts near the edges, it usually helps when you mirror them. If you look closely at the gradfunkmirror function, this is exactly what it does. Usually you don’t have to actually mirror the image, a simple border will suffice. Try something like:

addborders(0,8,0,0)

# do your processing here

crop(0,8,0,0)

[tasusq]

Thanks for the prompt reply TiGR.

I will try it out as soon as possible.

[m3mbran3]

I would take a look at Temporal Degrain (http://avisynth.org/mediawiki/Temporal_Degrain). After playing around with various denoising/degraining scripts I have found this to be well-suited to my needs. For me, using MDegrain3 is too strong, causing plasticy faces and losing fine details like the grain in wood or grass.

My simplest process for touching up DVDs is to increase the dynamic range with HDRAGC, use Temporal Degrain , then followed by LSFmod.

[Didée]

Hint - Adding black borders will reconstitute exactly the conditions that caused the problem in the first place ...

[tasusq]

Hint - Adding black borders will reconstitute exactly the conditions that caused the problem in the first place ...

So can you kindly suggest what should i do to get rid of that thin line/artifact on the edge?

[Didée]

[...] there is a thin line on the top [...] It is not in the source
Look closer, it is already in the source. If you look close, you'll see there is a *dark* line beneath the top matte. It is weak, yes, but it is there. Since it's pretty much static, the filters are likely to treat it as "detail" rather than an unwanted artifact. The denoisers will hardly touch that line, and the sharpening filter(s) will further enhance it: the dark line becomes a bit darker, and additionally gets a bright halo. And that's what you're seeing in the end.

Options:
Simple & brute: overcrop a bit on the top to remove the artefact.
Or, more dedicated: filter the border regions accordingly, so that the artifact is removed before sharpening, or at least won't get enhanced any further. (MaskTools is your friend.)


Edit - In addition, the ringing-behaviour of FFT3D doesn't exactly help on the matter *cough* ...

a) Try reducing FFT3D's blocksize.
b) try reducing the blocksize even more, and aid FFT3D with a quick'n dirty MV-compensation.
c) try dfttest (though slower than FFT3DGPU)

[tasusq]

Didée you are a geniuse. But the problem is i didn't understand how to apply most of the things you said.

Can you please tell me how should i apply it on the script?

[Didée]

In case I should get bored - which is unlikely.

For now, I'll leave it "as an exercise for the reader".

[tasusq]

Didee you can give me the solution now and you are asking me to wait? :(

Please tell me now.

[*.mp4 guy]

There is no simple " A = 5 " solution, thats the point, you can fix it in any number of ways. Didée listed some of those ways, but he isn't going to sit there fiddling with your source to figure out which one works best in his opinion, atleast not unless he has nothing better to do. Still you have all the information you need to solve the problem.

The people on this forum cant simply write an avisynth script for every person that has a problem that they don't know how to fix. The solution is to try to provide generalised solutions, and information on how to apply different techniques to solve problems, this lets everyone find the best solution for them, and avoids duplication of work.

[TiGR]

Having taken a closer look at the source, I think you can just crop it. There isn’t much to be gained from preserving four lines of the source (about 1% of image), after all.

Crop(0,56,-2,-60)
Spline36Resize(704,368)

That aside, you might want to replace FFT3D’s sharpening by a dedicated sharpener (search for LSFmod).

[tasusq]

Thanks TiGR for the Cropping part. But the artifact was on the upper part.
So something like this works fine:
Crop(0,58,-2,-58)
Spline36Resize(704,368)

*.mp4 guy you totally misunderstood my question. It is my fault and i should have mentioned which part i didn't understand. I never asked anyone to do a test encode for me and tell me the best results.

I just wanted to look at the example of a script, not the best script for this movie.

I only didn't understand this part:
b) ....and aid FFT3D with a quick'n dirty MV-compensation.

I don't know how to use FFT3D with MVTools. I just wanted to know how to make these 2 filters work together. If i was a pro like you guys i would have never registered and posted in this forum for help.

This is not the first time i am encoding a movie. I did this a few times before but never faced a problem like this. For example just take a look at the screenshot and tell me does it look Ok or not:
http://i39.tinypic.com/n2h8ac.png

http://i41.tinypic.com/o5dc89.png

I know it has some artifacts and i don't have any idea which filter should be used to get rid of these artifacts. I did at least 20 different test encodes. I tried blurring the image which hides these artifacts but takes away most of the detail. So now i am stuck and i don't know which filter should be used to remove these artifacts.

Again i am asking for you suggestions because i wasn't able to figure it out even after 20 test encodes. Each of these encodes took approximately 1 hour to finish.

Thanks *.mp4 guy for your comment.

[*.mp4 guy]

There is nothing /wrong/ with your encode, imo it doesn't look as good as it possibly could, but what ever does? I don't think I have a single encode that I am 100% happy with.

mvtools documentation (http://avisynth.org.ru/mvtools/mvtools.html) : iirc it has an example script for using mvtools with an external denoiser.

[m3mbran3]

Look at using Temporal Degrain (http://avisynth.org/mediawiki/Temporal_Degrain) for cleaning up your source. I have found it quite good using the default settings and degrain = 1 or 2. You have to understand that it is a compromise between noise / defect removal and detail preservation. If you are too aggressive with noise removal then you will lose details like fine lines, so you just have to choose what you like best.

Then for minor sharpening I use LSFmod (http://forum.doom9.org/showthread.php?t=142706), usually with strength < 80 just so text and details appear a bit sharper without making everyone look like they are wearing clown makeup.

Your encodes look perfectly watchable, but not great. You mention you are using x264 DXVA-SD-HQ profile but not what bitrate. If you want more of a quality based encode and less concern for filesize then you can try using the crf (constant quality profile) option with values between 18-22.

[tasusq]

There is nothing /wrong/ with your encode, imo it doesn't look as good as it possibly could, but what ever does? I don't think I have a single encode that I am 100% happy with.

mvtools documentation (http://avisynth.org.ru/mvtools/mvtools.html) : iirc it has an example script for using mvtools with an external denoiser.

Thanks for the mvtools link but i already saved the Mvtools2 help file and now i just found out that example you were pointing me to.:D
Is this the right way:
super = MSuper()
backward_vectors = MAnalyse(super, isb = true)
forward_vectors = MAnalyse(super, isb = false)
forward_compensation = MFlow(super, forward_vectors, thSCD1=500) # or use MCompensate
backward_compensation = MFlow(super, backward_vectors, thSCD1=500)
# create interleaved 3 frames sequences
interleave(forward_compensation, last, backward_compensation)

fft3dgpu(mode=1, precision=2, sigma=1.9, bt=4, bw=32, bh=32, ow=16, oh=16)

selectevery(3,1) # return filtered central (not-compensated) frames only

Actually the second video of Stormbreaker is full of artifacts. These artifacts are not present in the source and i am having problem removing them.

Can you kindly suggest which filter should i use?

[tasusq]

Look at using Temporal Degrain (http://avisynth.org/mediawiki/Temporal_Degrain) for cleaning up your source. I have found it quite good using the default settings and degrain = 1 or 2. You have to understand that it is a compromise between noise / defect removal and detail preservation. If you are too aggressive with noise removal then you will lose details like fine lines, so you just have to choose what you like best.

Then for minor sharpening I use LSFmod (http://forum.doom9.org/showthread.php?t=142706), usually with strength < 80 just so text and details appear a bit sharper without making everyone look like they are wearing clown makeup.

Your encodes look perfectly watchable, but not great. You mention you are using x264 DXVA-SD-HQ profile but not what bitrate. If you want more of a quality based encode and less concern for filesize then you can try using the crf (constant quality profile) option with values between 18-22.

Sorry i myself was writing the previous post when you posted it and that is why i saw it after i posted.

I am doing a size based encode and i like to keep the bitrate below 1600 kbps if possible. What setting should i use when using the "Unrestricted 1st Pass Const. Quality HQ" so that the bitrate stays below 1600kbps?

I tried using Temporal Degrain but the Grain is still visible even when i use degrain=2.

Can anyone tell me what filters should i use to get rid of the artifacts in the Second set of screenshots i posted?

[sreg0r]

If the artefacts you are getting only appear AFTER encoding then it is due to your encoding settings not your avisynth script.

A quality based encode can not accurately predict what the output size will be, it depends on the complexity of the video and the effects of the avisynth script you are using. For example, my last encode with a crf of 20 was 1800kbps, that same encode when using temporal degrain reduced the bitrate to 1600kbps due to the decreased complexity of the picture from the removal of the film grain.

I would probably try encoding using a crf of 18 (very high quality) and see if you still get artefacts. Then you can decrease bitrate (increase crf value) until you end up with an encode that you are happy with.

[tasusq]

Thanks sreg0r for your helpful comment.

Can anyone please tell me what filters should i use to remove these artifacts:
http://i39.tinypic.com/n2h8ac.png

There are a lot of artifacts around the face. These artifacts mainly appear around and in moving objects.

http://i41.tinypic.com/o5dc89.png

In this picture the artifacts are around the White Letters.

I have tried these filters: Convolution3d(preset=movieLQ), FluxSmoothST(7,7), FluxSmoothT(7), BlindDehalo(PPMode=-1), LSFMod(Strength=70, Smode=5) fft3dfilter(sharpen=0.5, sigma=1.5), Deblock_QED() & many other filters.

I checked the source and the artifacts are present there. In other words these artifacts are not caused by using any filter.

But i just want to know which filters should i try to see if those filters can remove these artifacts?

[7ekno]

MCTemporalDenoise(settings="low") or MCTemporalDenoise(settings="medium")

if Halos persist (those "artifacts around the letters"), follow up with either DeHalo_alpha(), a version of Blind Dehalo or turn edgeclean on with MCTemporalDenoise(settings="<your prefered level>", edgeclean=true, ECrad=8, ECMode="""dfttest().dehalo_alpha()""") - adjust ECrad to affect Halos further from edges ...

Tek

[tasusq]

7ekno, you are a real life saver. I was giving up hope and i was about to stop encoding this movie.

I will definitely try out what you suggested. I really appreciate your help man.

[10L23r]

also try mc_spuds() at strength=1,2, or 3

i use fluxsmoothst(0,9) to remove any spatial noise (such as the non-moving artifacts around text) left by more effective denoisers such as mc_spuds or mctemporaldenoise

[tasusq]

Ok 10L23r i will try that also. Finished doing Elektra and the result was excellent. It looks better than the source.
I will try out your and 7ekno suggestions when encoding Stormbreaker.

Thanks for the comment.