Need Suggestions for VERY GRAINY source

[Didée]

[continued]

Four Zoomings to quickly point out differences:

vs.

vs.

vs.

vs.


Hope this is sufficient to make my point: removing grain is okay if it annoys, but I'd want my result to keep the "natural" look. Sharp soup I like to eat, but not to watch.


Edit - the script: (compact version)

# LoadPlugins: FFT3DFilter.dll / mt_masktools.dll / MVTools.dll / RemoveGrain.dll / Repair.dll

AVCSource("X:\original.dga",deblock=true)

o    = last
fft  = o.fft3dfilter(sigma=16,sigma2=10,sigma3=6,sigma4=4,bt=5,bw=16,bh=16,ow=8,oh=8)
fftD = mt_makediff(o,fft)

b3vec1 = fft.MVAnalyse(isb = true,  delta = 3, pel = 2, overlap=4, sharp=2, idx = 1)
b2vec1 = fft.MVAnalyse(isb = true,  delta = 2, pel = 2, overlap=4, sharp=2, idx = 1)
b1vec1 = fft.MVAnalyse(isb = true,  delta = 1, pel = 2, overlap=4, sharp=2, idx = 1)
f1vec1 = fft.MVAnalyse(isb = false, delta = 1, pel = 2, overlap=4, sharp=2, idx = 1)
f2vec1 = fft.MVAnalyse(isb = false, delta = 2, pel = 2, overlap=4, sharp=2, idx = 1)
f3vec1 = fft.MVAnalyse(isb = false, delta = 3, pel = 2, overlap=4, sharp=2, idx = 1)

NR1   = o .MVDegrain3(b1vec1,f1vec1,b2vec1,f2vec1,b3vec1,f3vec1,thSAD=300,idx=2) 
NR1D  = mt_makediff(o,NR1)

DD     = mt_lutxy(fftD,NR1D,"x 128 - abs y 128 - abs < x y ?")
NR1x   = o.mt_makediff(DD,U=2,V=2)

NR2    = NR1x.MVDegrain3(b1vec1,f1vec1,b2vec1,f2vec1,b3vec1,f3vec1,thSAD=300,idx=3)

s    = NR2.minblur(1,1)                   
allD = mt_makediff(o,NR2)                 
ssD  = mt_makediff(s,s.removegrain(11,-1))
ssDD = ssD.repair(allD,1) .mt_lutxy(ssD,"x 128 - abs y 128 - abs < x y ?")

NR2.mt_adddiff(ssDD,U=2,V=2)

return(last)

#------------------------------------------
# Taken from MCBob.avs:
function MinBlur(clip clp, int r, int "uv")
{
uv   = default(uv,3)
uv2  = (uv==2) ? 1 : uv
rg4  = (uv==3) ? 4 : -1
rg11 = (uv==3) ? 11 : -1
rg20 = (uv==3) ? 20 : -1
medf = (uv==3) ? 1 : -200

RG11D = (r==1) ? mt_makediff(clp,clp.removegrain(11,rg11),U=uv2,V=uv2)
 \    : (r==2) ? mt_makediff(clp,clp.removegrain(11,rg11).removegrain(20,rg20),U=uv2,V=uv2)
 \    :          mt_makediff(clp,clp.removegrain(11,rg11).removegrain(20,rg20).removegrain(20,rg20),U=uv2,V=uv2)
RG4D  = (r==1) ? mt_makediff(clp,clp.removegrain(4,rg4),U=uv2,V=uv2)
 \    : (r==2) ? mt_makediff(clp,clp.medianblur(2,2*medf,2*medf),U=uv2,V=uv2)
 \    :          mt_makediff(clp,clp.medianblur(3,3*medf,3*medf),U=uv2,V=uv2)
DD    = mt_lutxy(RG11D,RG4D,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?",U=uv2,V=uv2)
clp.mt_makediff(DD,U=uv,V=uv)
return(last)
}

Note#1: The green part in the "ssDD=..." line was actually missing when I was creating the sample! Without that lutxy(), in some places a small amount of the original low-freq flicker is reintroduced. Consequentially, the sample is worse than it should have been. (a bit, at least)

Note#2: The script uses MVDegrain3. If you don't have MVTools v1.9.0, you can either use MVDegrain2 instead (with slightly worse results), or make a donation to Fizick (recommended!) to get the goodies more early.

[Zanejin]

Didée, both download links for your clip are... mis-pasted, shall we say? The results look great, though, and the zoomed-in screen-captures definitely show that your encode removes grain at a lower cost to detail—especially facial detail and skin features.

[jeffy]

Didée, please, could you:
1) correct the links to your samples
2) post the final script you used for encoding
EDIT: added: 3) the x264 commandline you used
Thank you.

[Didée]

The links are correct now. Sorry, it was in the middle of the night ...

Originally Posted by jeffy

post the final script you used for encoding

With pleasure. The final script used for encoding was this:

l = AviSource("processed_left_half.avi") .crop(0,0,640,528)
r = AviSource("processed_right_half.avi") .crop(64,0,640,528)

stackhorizontal( l, r )

The commandline was

--bframes 3 --b-pyramid --ref 4 --deblock -3:-3 --crf 18.75 --ipratio 1.25 --partitions all --direct temporal 
--weightb --me hex --subme 5 --mixed-refs --bime --8x8dct --no-fast-pskip --aq-strength 0.2 --aq-sensitivity 12.5 
--deadzone-inter 5 --deadzone-intra 9  -o "X:\easy.mkv" "X:\glue.left.right.avs"

[Terranigma]

Originally Posted by jeffy

EDIT: added: 3) the x264 commandline you used
Thank you.

use avinaptic to get the info.

Originally Posted by Didée

The x264 encoding surely could be done better too - I'm an old-fashioned Xvid user, tweaking x264 is too complicated for me. (truley, I'm not fully satisfied with what I managed to get from x264. Is someone giving lessons?)

People tend to optimize their settings for maximum efficiency, and what I mean by this is optimizing for top quality alongside speed and grain retention. Because of how trellis-2 operates, no one really uses it (unless adaptive quantization is used as well to help with grain retention.)

Settings that aren't, and shouldn't be used for this type of processing, is b-rdo, subme-7, and trellis-2 for one. Also, sometimes, cpu usage plays a part in this as well, so mixed references is usually a no no

If you'd' like to see what the experts tends to leans towards, then you can get an idea from this commandline:

--pass 2 --bitrate xxxx --keyint 250 --min-keyint 25 --ref 5 --no-fast-pskip --bframes 3 --b-pyramid --bime --weightb --filter -2,-2 --subme 6 --trellis 1 --partitions all --8x8dct --direct auto --ratetol 1.5 --me umh --threads auto --thread-input --cqmfile "Prestige CQM.cfg" --progress --no-psnr --no-ssim --output "C:\Output Video.mkv" "C:\Input Video.avs"

Of course, getting to know what bitrate should be used, a crf encode is usually done first of a rate between 18.0 - 22.0. Mostly, 20.0

nice grain/quality/cpu usage/encoding speed. Adaptive quantization most of the times isn't needed with the above.

[jeffy]

Originally Posted by Didée

The links are correct now. Sorry, it was in the middle of the night ...


With pleasure. The final script used for encoding was this:

l = AviSource("processed_left_half.avi") .crop(0,0,640,528)
r = AviSource("processed_right_half.avi") .crop(64,0,640,528)

stackhorizontal( l, r )

Didée: you know what I meant, we're not playing scrabble, are we?
Could you please post *cough* all the scripts you used until you reached these final encodes in their final form?
(I don't think think I will find a better wording.)

[foxyshadis]

The interesting thing about yours, Didée, is that it seems to meld the grain into what looks like fine detail. It's probably not the detail that was on the film prints, but it's essentially the same fake detail that the eye creates out of the grain.

Frankly, I'd love to use whatever you did on some detail-free videos, by first overlaying them with layers of grain. I've been trying to do that for years, but I could never find the right add/remove cycle to keep it from dancing and swimming.

[Sagekilla]

Very nicely done Didee.. Your script looks like it retains much more detail than Zep's does, with that said I have to hand it to both of you.. Both are an amazing improvement over the original, well done

I'd really love to see the filter chain you used to degrain the movie like that Didee! Also, it seems like the reason why yours was a slightly higher bitrate could because of the more detail yours retains. I noticed in the close up of faces yours had more skin detail than Zep's.

[Terranigma]

Originally Posted by Sagekilla

I noticed in the close up of faces yours had more skin detail than Zep's.

Perhaps because zep's was oversharpened? Sure, it may look great while playing, but when zoomed, you can see the detail that was destroyed. It's never good to oversharpen; there's a detail analysis out there that talks about edge-enhancement; maybe you could find it.

[Didée]

Originally Posted by jeffy

Could you please post *cough* all the scripts you used until you reached these final encodes in their final form?

You man ALL the scripts I used until finally arriving where we are now? Are you serious? That's probably thousand scripts and more, over all the years!


@ Terranigma: Thanks for the tips. In fact it's not too surprising ... it's a few details which are hard to judge from the theoretical side, while practice then may be different even more. (And doing much of testing is no fun for me ... if you had my PC, you would understand why...)
BTW, the latest plaything of Dark Shikari (new AQ) seems to come along very nice. I like it much. Hope that it's "generic" enough to become an official feature...

* * *

OK, I added the script to post#61. The only change is that I added a small check that ultimately should have been there, but wasn't ...

In my opinion, the script's processing is rather easy, technically. It's kind of slow because quite some MV-stuff is used - but the mere processing chain is in no way fancy, but pretty straightforward. (It's a stripped-down-to-bare-bone version of really fancy grain removal scripts...)

The script again, blown up with some comments and hook-in points for easier modification:

AVCSource("X:\original.dga",deblock=true)

o   = last
fft = o.fft3dfilter(sigma=16,sigma2=10,sigma3=6,sigma4=4,bt=5,bw=16,bh=16,ow=8,oh=8)


# "srch" is a prefiltered clip on which the motion serach is done.
# Here, we simply use FFT3DFilter. There're lots of other possibilities. Basically, you shouldn't use 
# a clip with "a tiny bit of filtering". The search clip has to be CALM. Ideally, it should be "dead calm".
srch = fft


# "spat" is a prefiltered clip which is used to limit the effect of the 1st MV-denoise stage.
# For simplicity, we just use the same FFT3DFilter. There're lots of other possibilities.
spat  = fft
spatD = mt_makediff(o,spat)


# motion vector search (with very basic parameters. Add your own parameters as needed.)
b3vec1 = srch.MVAnalyse(isb = true,  delta = 3, pel = 2, overlap=4, sharp=2, idx = 1)
b2vec1 = srch.MVAnalyse(isb = true,  delta = 2, pel = 2, overlap=4, sharp=2, idx = 1)
b1vec1 = srch.MVAnalyse(isb = true,  delta = 1, pel = 2, overlap=4, sharp=2, idx = 1)
f1vec1 = srch.MVAnalyse(isb = false, delta = 1, pel = 2, overlap=4, sharp=2, idx = 1)
f2vec1 = srch.MVAnalyse(isb = false, delta = 2, pel = 2, overlap=4, sharp=2, idx = 1)
f3vec1 = srch.MVAnalyse(isb = false, delta = 3, pel = 2, overlap=4, sharp=2, idx = 1)


# 1st MV-denoising stage. Usually here's some temporal-median filtering going on.
# For simplicity, we just use MVDegrain.
NR1   = o .MVDegrain3(b1vec1,f1vec1,b2vec1,f2vec1,b3vec1,f3vec1,thSAD=300,idx=2) 
NR1D  = mt_makediff(o,NR1)

# limit NR1 to not do more than what "spat" would do
DD     = mt_lutxy(spatD,NR1D,"x 128 - abs y 128 - abs < x y ?")
NR1x   = o.mt_makediff(DD,U=2,V=2)


# 2nd MV-denoising stage. We use MVDegrain.
NR2    = NR1x.MVDegrain3(b1vec1,f1vec1,b2vec1,f2vec1,b3vec1,f3vec1,thSAD=300,idx=3)


# contra-sharpening: sharpen the denoised clip, but don't add more to any pixel than what was removed previously.
# (Here: a simple area-based version with relaxed restriction. The full version is more complicated.)
s    = NR2.minblur(1,1)                     # damp down remaining spots of the denoised clip
allD = mt_makediff(o,NR2)                   # the difference achieved by the denoising
ssD  = mt_makediff(s,s.removegrain(11,-1))  # the difference of a simple kernel blur
ssDD = ssD.repair(allD,1)                   # limit the difference to the max of what the denoising removed locally.
ssDD = SSDD.mt_lutxy(ssD,"x 128 - abs y 128 - abs < x y ?") # abs(diff) after limiting may not be bigger than before.

NR2.mt_adddiff(ssDD,U=2,V=2) # apply the limited difference. (sharpening is just inverse blurring)

return(last)

This is what is done:

1) Make a pre-filtering with FFT3DFilter. This one will be used for analysis and probability comparisons.

2) search motion vectors on the (calm) pre-filtered clip.

3) Cut down the grain's amplitude by:

- 3a) Make a motion-compensated denoising (optimally by temporal medianfiltering with temp.radius=2, but I'm having serious issues with the [only] available filter)

- 3b) Limit the effect of 3a) by that of a spatial filter

4) Make a motion-compensated temporal averaging of the clip with flattened grain

5) Sharpen the denoised clip. This is done in such a way that the sharpener is not allow to "add" more than what was previously "removed" by the denoising.

... and that's all, nothing spectacular.


The biggest difference to Zep's denoising is - most probably, since we didn't see his script up to now - the usage of fft3dfilter. It seems that he used to much of it, or at to strong settings. All the ringing and texture echoing is a typical side-effect fft3dfilter, when not used with enough caution. Judging from the overall look, I'd guess that Zep first did an initial filtering with fft3dfilter, and then continued to further process the result with MV-denoising. (Some places look like fft3d-banding that has been temporally averaged, but it's hard to judge after x264-compression.)

In comparison, my script never applies fft3dfilter in the chain that produces the output clip. Instead, it is only used as a "brake" to keep the 1st temporal filter within reasonable bounds. (Which is essential in the bigger scripts where the 1st stage is done with median filtering.)


@ foxyshadis
Yeah, the effect of producing "fixed grainage" ... it's interesting that the effect is almost impossible to avoid when strictly using only temporal averaging. Earlier I tried to get rid of it (it looks poor when the grainage is too strong), but that alwas comes at the cost of serious detail loss. I settled for keeping the effect low, then it looks quite convincing, mostly.

Can't tell if you get the same effect with synthetically generated noise. I suspect the effect comes from the fact that compressed film grain has pretty much invariance in its variance (hope this makes sense), where the gaussian noise of the usual noise generators seems to be much more regular. So, it could be you'd need a way to make the added noise, well, more "dirty".

[Sagekilla]

How do you convert the parameters for MVDegrain3 to MVDegrain 2? I'm looking at the code and the documentation but I'm having a very hard time converting it.

Edit: Figured it out. To anyone else confused, just replace the MVDegrain3 function with the following in both places it appears in: MVDegrain2(b1vec1,f1vec1,b2vec1,f2vec1,thSAD=300,idx=2)

Edit 2: Love how the filter works, I got a 43% reduction in bitrate when I used the filter chain. I also skewed the results slightly because I had Dark Shikari's new AQ running in both the degrained and straight from source encodes.

[Spuds]

Didée

As always very nice script, I really like the contra-sharpening section I think I will put that to some use in my overly complicated script Seeing the results of degrain3 is also very nice.

There were a couple of things I did not quite understand so if you could be so kind as to help me

1) on the nr1= line you use idx=2, should that be idx=1 or does the call to mvdegrain3 need to point to a different cache then the idx=1 vectors?

2) on the nr2= line you reuse the original motion vectors in mvdegrain3 but on a different (denoised) frame. At what point do motion vectors become invalid for reuse? I was under the impression that once you changed the center frame that new wing vectors should be computed. I'm also confused by idx in this line but the answer to 1 should help here.

Thanks!

EDIT: Ah now I see what you did, that center frame is not changed but limited. Should have followed the nr1x creation a bit more closely before I typed in my question number 2, now I see why those vectors can be used. Still confused by idx.

[Sagekilla]

Also, I'd love to see this more complicated version if you know how to/have it! As slow as the script runs with my insane x264 settings (0.8 fps, 20 hours for 60k frames!) I have no qualms with running a several day long encode to degrain 300 at a higher quality. Even a simple little suggestion on how to improve it more would be great I wish I could test out MVDegrain3 to see how much better it works, but there's no way for me to make a donation to get it (Seeing as I'm only 17 = no way for me to make that donation)

[Didée]

Spuds: don't think too complicated!
Motion vectors never get "invalid" in that sense. With idx, it's always the same game:

vec = clip1.MVAnalyse( ..., idx = _idx1 )
foo = clip2.MVToolsFunction( vec,.., idx = _idx2 )

As long as clip1 == clip2 (identical), you can (and should) use identical values for _idx1 and _idx2.

As soon as clip1 != clip2 (not identical), you have to use different values for _idx1 and _idx2.

The vectors stored in 'vec' stay valid as long as clip1 and clip2 are of same size, and are in sync. (E.g. If one of both clips got trim()'ed, then you need new vectors, obviously.)

[Fizick]

Note in addition: If you use single MVTools function with some input clip, idx may be simply omitted (it will be set to unique numbers internally for every such call).

Didée,
you must provide some name for your new script!
LowFrequencyFlickerRemover (LFFR) or somewhat like MiddleFreqDegrain ?

Sagekilla, be patient.

[Didée]

Some things get posted in form of a Function(). Some things get posted in form of a plain script.

Functions do have names. Plain scripts are just nameless plain scripts.

Usually, there is a reason for the decision of whether to post something as function or as plain script.


(If it has to be by all means, how about YAMCGC_EV - "Yet another motion compensated grain cruncher (easy version)" ...) :P

[Zep]

Originally Posted by dbzgundam

Be grateful you guys can at least enjoy the grain as an unmuddied mess in the Blu-Ray version.

The DVD version is not as forgiving

The heavy grain look as of late is just a mystery to me. I understand wanting SOME grain in an image, but film stocks have improved drastically over the last few years in constant efforts to reduce this if anything, and the advent of digital video has only led to nearly-grain-free (Well it is grain free, but has noise now) picture quality... Oh well.

I actually used degrain1 for this beast, and it handled it pretty well. The movie is actually fairly compressible due to the softness of the picture, large amount of slow motion scenes, and while it does have high contrast, a lot of blown out areas make it that much less difficult to compress.

What really annoys me is that they ADDED the grain in post production on purpose. The I-MAX version has NO GRAIN.

[Sagekilla]

Originally Posted by Fizick

Note in addition: If you use single MVTools function with some input clip, idx may be simply omitted (it will be set to unique numbers internally for every such call).

Didée,
you must provide some name for your new script!
LowFrequencyFlickerRemover (LFFR) or somewhat like MiddleFreqDegrain ?

Sagekilla, be patient.

Will do I'm really curious as to how much of an improve MVDegrain 3 will be making, and it seems like (from what I can infer) it uses an extra forward and backward motion vector.

[Zep]

Originally Posted by Sagekilla

Yes I did look at the encode, and personally I think if I were to use your script for encoding 300 it wouldn't be too bad since I'm using a much smaller resolution. (864x368 vs 1920x816) Plus I do use 1-pass crf so wouldn't have to worry about it being slow on a first pass since I am doing only one

haha yeah that should speed things up a good bit

Also I use pretty high x.264 settings. When I do get the best script from this thread and all the great ideas from everyone.
I plan to run a HQ-INSANE (Every setting maxed out) MEGA final run which should look awesome for only take like a week to encode

Going by what Didee said I have done more tests today and I need to better be able to selectively mask areas so the cons he was talking about are not so bad.

[Sagekilla]

Originally Posted by Zep

haha yeah that should speed things up a good bit

Also I use pretty high x.264 settings. When I do get the best script from this thread and all the great ideas from everyone.
I plan to run a HQ-INSANE (Every setting maxed out) MEGA final run which should look awesome for only take like a week to encode

Going by what Didee said I have done more tests today and I need to better be able to selectively mask areas so the cons he was talking about are not so bad.

Mmm, yes it is quite a bit faster than an HD source, a blazing 0.87 fps! Only settings not maxed are refs (6) and ME (umh)

But, back on topic though, was your script (generally speaking) about the same methodology that Didee was using for degraining?