Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 8th July 2011, 12:29   #1  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 2,542
Lock stock and two smoking barrels - Dancing noise, unable to remove

I am trying to clean the BD of Lock stock and two smoking barrels but the usual scripts are useless.

Here is a crop of the m2ts.

I have tried to apply the following scripts:

PHP Code:
SetMTMode(5)
SetMemoryMax(1000)
LoadPlugin("D:\eseguibili\media\dgdecnv\x64 binaries\DGDecodeNV.dll")
DGSource("E:\in\lock_stock\lock_stock2.dgi")
ChangeFPS(last,last,true)
SetMTMode(2)
SMDegrain (tr=3,thSAD=400,blksize=16,overlap=8,pel=1
Result

PHP Code:
SetMemoryMax(1000)
LoadPlugin("D:\eseguibili\media\dgdecnv\x64 binaries\DGDecodeNV.dll")
DGSource("E:\in\lock_stock\lock_stock2.dgi")
ChangeFPS(last,last,true)
MT("""
MCTemporalDenoise(settings="
high",useEEDI2=true,bwbh=16,owoh=4,chroma=true,sharp=true,pel=1)
"""
,threads=4overlap=8
Result

PHP Code:
SetMemoryMax(1000)
LoadPlugin("D:\eseguibili\media\dgdecnv\x64 binaries\DGDecodeNV.dll")
DGSource("E:\in\lock_stock\lock_stock2.dgi")
ChangeFPS(last,last,true)
MT("""
MCTemporalDenoise(settings="
very high",useEEDI2=true,bwbh=16,owoh=4,chroma=true,sharp=true,pel=1)
"""
,threads=4overlap=8
Result

Could you please give me advice on how to remove the dancing noise? Look at the blue screen...

Thanks
__________________
@turment on Telegram
tormento is offline   Reply With Quote
Old 9th July 2011, 03:11   #2  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,389
Multilevel-MDegrain?

http://www.mediafire.com/?tbqgtfi7r7mb47v
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline   Reply With Quote
Old 9th July 2011, 05:19   #3  |  Link
Vitaliy Gorbatenko
viterra
 
Join Date: Feb 2003
Location: St. Peterburg, Russia
Posts: 142
Luxurious result! How do you pick up the levels for Multilevel-MDegrain?
Vitaliy Gorbatenko is offline   Reply With Quote
Old 9th July 2011, 09:10   #4  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 2,542
Didée, my hat off.

You know you have to post me the avs you used, don't you =)
__________________
@turment on Telegram

Last edited by tormento; 9th July 2011 at 10:14.
tormento is offline   Reply With Quote
Old 9th July 2011, 10:47   #5  |  Link
dfan
Registered User
 
Join Date: May 2011
Posts: 6
Quote:
Originally Posted by Didée View Post
amazing!!
dfan is offline   Reply With Quote
Old 9th July 2011, 12:04   #6  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,389
Multi"Level" in the sense of using multiple "scalings". The observation was that when downscaling the source to a smallish resolution, then a vanilla MDegrain can produce a very stable result. Hence, it's an obvious approach to first make a small-but-stable-denoised clip, and then work the way upwards to the original resolution.

This is the main filterchain that was used:

Code:
DGsource("lock_stock2.dgi")
crop(0,20,-0,-20,true)

o   = last
ox  = o.width
oy  = o.height
ox1 = 1280
oy1 =  704
ox2 =  640
oy2 =  360

sm1 = o.bicubicresize(ox1,oy1)   # medium scale
sm2 = sm1.bicubicresize(ox2,oy2) # small scale

D12 = mt_makediff(sm2.bicubicresize(ox1,oy1),sm1,U=3,V=3) # residual of (small)<>(medium)
D1o = mt_makediff(sm1.bicubicresize(ox,oy),o,U=3,V=3)     # residual of (medium)<>(original)

lev2 = sm2.MCDegrain(3,soft=true) # filter on smalle scale
up1 = lev2.bicubicresize(ox1,oy1)
up2 = up1.bicubicresize(ox,oy)

D12.MCDegrain(3,up1,soft=false,thSAD=480) # filter on medium scale
lev1 = up1.mt_makediff(last,U=3,V=3)
up3 = lev1.bicubicresize(ox,oy)

D1o.MCDegrain(3,up2,soft=true,thSAD=480)  # filter on original scale
up3.mt_makediff(last,U=3,V=3)

return(last)
Also required is this modification of the "MCDegrain" script: (originally made to help an MVTools novice, lately)
Code:
function MCDegrain(clip c, int "frames", clip "srch", bool "soft", int "thSAD")
{
frames = default(frames, 2)
srch   = default(srch,c)
soft   = default(soft,false)
thSAD  = default(thSAD,400)
bs = (c.width>960) ? 16 : 8
rg = (c.width<640) ? 11 : (c.width<1200) ? 11 : 20
rg = soft ? rg : 0
super = srch.MSuper(pel=2, sharp=1)
super2 = soft ? c.mt_adddiff(mt_makediff(c,c.removegrain(rg)),U=2,V=2).MSuper(pel=2,sharp=1,levels=1)
 \            : c.MSuper(pel=2,sharp=1,levels=1)
bvec3 = MAnalyse(super, isb = true, delta = 3, blksize=bs, overlap=bs/2)
bvec2 = MAnalyse(super, isb = true, delta = 2, blksize=bs, overlap=bs/2)
bvec1 = MAnalyse(super, isb = true, delta = 1, blksize=bs, overlap=bs/2)
fvec1 = MAnalyse(super, isb = false, delta = 1, blksize=bs, overlap=bs/2)
fvec2 = MAnalyse(super, isb = false, delta = 2, blksize=bs, overlap=bs/2)
fvec3 = MAnalyse(super, isb = false, delta = 3, blksize=bs, overlap=bs/2)
(frames<=0) ? c :\
(frames==1) ? c.removegrain(rg).MDegrain1(super2, bvec1,fvec1,thSAD=thsad) :\
(frames==2) ? c.removegrain(rg).MDegrain2(super2, bvec1,fvec1,bvec2,fvec2,thSAD=thsad) :\
              c.removegrain(rg).MDegrain3(super2, bvec1,fvec1,bvec2,fvec2,bvec3,fvec3,thSAD=thsad)
return(last) }
There's a bunch of hardcoded assumptions (e.g. scale-resolutions) that should be generalized, and probably some refinements missing (e.g. evolving the edges of the searchclip.)

But in general, it seems to do a reasonable job.

Oh, one point: In particular on 1080-ish FullHD, it is ...

... Slooooow!
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)

Last edited by Didée; 9th July 2011 at 12:22. Reason: Syntax error ;-)
Didée is offline   Reply With Quote
Old 9th July 2011, 15:19   #7  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 2,542
Thanks, I'll try it as soon as I'll be at home!
__________________
@turment on Telegram
tormento is offline   Reply With Quote
Old 10th July 2011, 00:13   #8  |  Link
Nephilis
--preset WTF!
 
Join Date: Feb 2009
Posts: 86
Didée; Are you sure about this line ?
Code:
rg = (c.width<640) ? 11 : (c.width<1200) ? 11 : 20
Perhaps you meant
Code:
rg = (c.width<640) ? 11 : (c.width<1200) ? 12 : 20
or
Code:
rg = (c.width<640) ? 11 : (c.width<1200) ? 19 : 20

Last edited by Nephilis; 10th July 2011 at 00:19. Reason: Syntax error
Nephilis is offline   Reply With Quote
Old 10th July 2011, 02:42   #9  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,389
Flutterby. First I was thinking about 17/11/20, but then left it at 11/11/20. Maybe it should rather be sbr()/11/20, that would be the logical adaption to the frame scales.

Howsoever. A strength option instead of a lazy 100% on/off would be more important than the kernel itself.
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline   Reply With Quote
Old 10th July 2011, 10:59   #10  |  Link
Nephilis
--preset WTF!
 
Join Date: Feb 2009
Posts: 86
You have to pay mr. jiminy 2 pennys for saying flutterby
Nephilis is offline   Reply With Quote
Old 11th July 2011, 11:35   #11  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 2,542
Quote:
Originally Posted by Didée View Post
... Slooooow!
Indeed. I'll try to MT it. Where do you thing is the step that takes more time to be performed? Resize or MC?
__________________
@turment on Telegram
tormento is offline   Reply With Quote
Old 11th July 2011, 12:48   #12  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,389
Quote:
Originally Posted by tormento View Post
Where do you thing is the step that takes more time to be performed? Resize or MC?
Erh, surely not the resize. There are three calls of MDegrain3, with the 2nd and 3rd call being reliant on the earlier ones.(!!) - Now what do you think where all the time is spent .....


Aahh, you also need a SetMemoryMax command. I found that a serious fps breakdown happens with SMM(500) (which is Avisynth's default upper boundary). With SMM(600) the fps were ~almost~ on par. To have some safety margin, I'd go for a SMM(800) or so.
BTW: Single-threaded, I got 1.33 fps on an i7-860.

Also, after a short re-visit I found the smalle scale actually is set a little too big. (Originally had it at 480x260, then set it to 640x360 and had let it there.) I think that sm2=[576x312] and sm1=[1088x584] should be okay.

The script is heavyweight with ressources. Most probably you will not get it successfully multithreaded in 32bit Avisynth. Maybe in 64bit Avisynth, but there you'll very likely get crashes from the buggyness in itself.

If you have enough free HD space, then in this case I'd really go for a (near-) lossless intermediate, with [CPU cores] instances running in parallel.

And finally, it's not like this would be the one & only way that can work out. I just tried the idea (because it hasn't been done yet in this way), and although it seems to work out generally, it's quite possible that other approaches can work out quite as well. With the "traditional" way of FFT3dFilter-prefiltering, maybe the culprit is that the usually-used sigma cascade (16/10/6/4) has too low sigma2/3 for this kind of noise.
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline   Reply With Quote
Old 12th July 2011, 06:47   #13  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 2,542
What about reducing pel? Do you think it could impact too much about quality?
__________________
@turment on Telegram
tormento is offline   Reply With Quote
Old 12th July 2011, 11:29   #14  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,389
Loosely guessing:
- The smallest scale should ultimately have pel=2.
- The original(largest) scale probably can get away with pel=1.
- Medium scale: maybe yes, maybe no.

"Try, compare, judge."
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline   Reply With Quote
Old 17th July 2011, 10:11   #15  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 2,542
Mmm.. the scripts works well. Any idea of why x264 crashes after a random time if I multithread it? It's a pity as I can get almost 4 fps.

No other script I currently use is so greedy, I suppose.
__________________
@turment on Telegram
tormento is offline   Reply With Quote
Old 17th July 2011, 12:01   #16  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,389
Look again at post#12 - I already predicted that multithreading won't work with this script.
You have to ask the Avisynth fairies to cast a spell making FullHD+MVTools+MultiThreading less ressource hungry. With what we have now, it is like it is. Nothing that could be done.

Be assured my disappointment about this situation is not smaller than yours. However:

Spending two days to run a script single-threaded still is clearly more efficient than spending a week trying to run it multithreaded, to no avail.
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline   Reply With Quote
Old 17th July 2011, 18:55   #17  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 2,542
You are the wise man, I'll listen to your words of wisdom
__________________
@turment on Telegram
tormento is offline   Reply With Quote
Old 19th July 2011, 07:23   #18  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 2,542
Ok, I did it with a MT thread. I compared the result with MCTD(very high) and I found that the Didee script is very good to remove bigger dancing noise, while MCTD is better in smaller dancing noise. As a matter of fact Didee's one produces a bigger file, probably meaning there is more noise/detail left.

Didee, what should I do? Increase thsad in your/mcdegrain script? Process the output again with a "simple" mcdenoise?

Thanks for the time
__________________
@turment on Telegram
tormento is offline   Reply With Quote
Old 19th July 2011, 09:02   #19  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,389
Quote:
Originally Posted by tormento View Post
what should I do?
x264 --preset fast, lay on couch, watch movie.
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline   Reply With Quote
Old 22nd July 2011, 11:10   #20  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 2,542
I watched the movie and I was disapponted to see the same behaviour of dancing noise on some "pure colors", such as red, again.

Are you interested in investigating more or should I go to hell? =)
__________________
@turment on Telegram
tormento is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 09:04.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.