Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 17th February 2008, 19:03   #261  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,389
ATM I've not the energy (a really bad flu', can't concentrate) to cut through all the stuff posted on the last few pages.

Not sure if it's to rate as "tragedy" or "comedy" ... In any case, lots of nonsense has been posted.

There've been really screwed-up findings & correlations about blocksizes, bogus inventions like "detection thresholds", funny alternate filters (dfttest with default tbsize=1 versus fft3dfilter with bt=5 ... hell, you cannot compare that!), instances of non-functional functions, even posted to the wiki (Sagekilla, do you actually try your own stuff? MVAnalyse has no argument "ov", only "overlap", and why is "ov" responsible for "blocksize"), backwards-evolutions of proven concepts (in the longer-ago past, the biggest complain about MVDenoise was blockyness due to not-yet implemented block-overlap modes ... nowadays block overlapping is hardcoded'ly taken away because of performance...welcome in stoneage), and what not else.

Back to sick bed, crying in silence.
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline   Reply With Quote
Old 17th February 2008, 19:42   #262  |  Link
Terranigma
*Space Reserved*
 
Terranigma's Avatar
 
Join Date: May 2006
Posts: 953
Welcome back Didée. Hope you get well soon, and then you can explain to us about some of the nonsense that was posted (I know that i'm responsible for most of it. ). Honestly though, Fizick would have to update his document if blocksize 4 is weaker than blocksize 8 for denoising. I was thinking that they also functioned like the blockx, blocky arguments in tfm. Again, from comparisons i've done, blocksize 4 seemed to be stronger than 8 and 16 just like how Fizick has it documented, but whatever: you're the expert around these parts.

Last edited by Terranigma; 17th February 2008 at 19:44.
Terranigma is offline   Reply With Quote
Old 17th February 2008, 20:54   #263  |  Link
jeffy
Registered User
 
Join Date: Jan 2007
Posts: 943
Didée,
please get well soon and forget all the nonsense for a while!
for everything.

Back to sick bed, crying in silence.
jeffy is offline   Reply With Quote
Old 17th February 2008, 22:22   #264  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,558
It's not so much that blocksize 4 is stronger, it's that it has a higher chance of making bad correlations and subsequently denoising places that shouldn't by rights have been affected. Meaning smudgier details and edges in random places. blocksize 8 or 16 with a soft threshold, as Didée's been asking for forever, would be a better improvement.
foxyshadis is offline   Reply With Quote
Old 17th February 2008, 23:32   #265  |  Link
Zanejin
Registered User
 
Zanejin's Avatar
 
Join Date: Jan 2007
Posts: 24
Quote:
Originally Posted by foxyshadis View Post
It's not so much that blocksize 4 is stronger, it's that it has a higher chance of making bad correlations and subsequently denoising places that shouldn't by rights have been affected. Meaning smudgier details and edges in random places. blocksize 8 or 16 with a soft threshold, as Didée's been asking for forever, would be a better improvement.
So for MVAnalyse, based purely on results for MVDegrain, the blksize value that gives the greatest speed also happens to give the strongest and most accurate filtering?
Zanejin is offline   Reply With Quote
Old 18th February 2008, 03:45   #266  |  Link
Nikos
Registered User
 
Join Date: Jun 2002
Location: Greece
Posts: 242
Welcome back Didée.
Please explain to me the differents if any between 1 and 2:

Quote:
1.
source=last
b1_vec = source.MVAnalyse(isb = true, delta = 1, blksize = 8, overlap =4, pel = 2, sharp= 2, idx=1)
f1_vec = source.MVAnalyse(isb = false, delta = 1, blksize = 8, overlap =4, pel = 2, sharp= 2, idx=1)

NR1=source.MVDegrain1(b1_vec, f1_vec, thSAD = 400, idx = 1)
NRa=NR1.MVDegrain1(b1_vec, f1_vec, thSAD = 400, idx = 1)
Quote:
2.
source=last
b1_vec = source.MVAnalyse(isb = true, delta = 1, blksize =8, overlap = 4, pel = 2, sharp = 2, idx = 1)
f1_vec = source.MVAnalyse(isb = false, delta = 1, blksize =8, overlap = 4, pel = 2, sharp = 2, idx = 1)

NR1=source.MVDegrain1(b1_vec, f1_vec, thSAD = 400, idx = 1)
NRb=NR1.MVDegrain1(b1_vec, f1_vec, thSAD= 400, idx = 2)
And the correlation, if any between the two above with this:
Quote:
3.
source=last
b2_vec = source.MVAnalyse(isb = true, delta = 2, blksize = 8, overlap = 4, pel = 2, sharp = 2, idx = 1)
b1_vec = source.MVAnalyse(isb = true, delta = 1, blksize = 8, overlap = 4, pel = 2, sharp = 2, idx = 1)
f1_vec = source.MVAnalyse(isb = false, delta = 1, blksize = 8, overlap = 4, pel = 2, sharp = 2, idx = 1)
f2_vec = source.MVAnalyse(isb = false, delta = 2, blksize = 8, overlap = 4, pel = 2, sharp = 2, idx = 1)

NRc=source.MVDegrain2(b1_vec, f1_vec, b2_vec, f2_vec, thSAD = 400, idx = 1)
I know that here (3) we have more frames.

Thank you.
__________________
Greece PAL User...

Last edited by Nikos; 18th February 2008 at 04:03.
Nikos is offline   Reply With Quote
Old 18th February 2008, 03:58   #267  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,558
Strongest and fastest, but least accurate. On the other hand, it can be more accurate on some edges. Using large blocksizes in most areas and progressively smaller ones in occulded areas is the best way, as x264 does, but what can you do at the moment, y'know? This was discussed in the mvtools thread a while back.
foxyshadis is offline   Reply With Quote
Old 18th February 2008, 15:17   #268  |  Link
Nikos
Registered User
 
Join Date: Jun 2002
Location: Greece
Posts: 242
Thank you foxyshadis for the reply. Yoy mean that 1 and 2 are strongest and fastest but least accurate than 3 in my understanding.

What's the differents between 1 and 2?
Both of them are correct or only one?
__________________
Greece PAL User...
Nikos is offline   Reply With Quote
Old 18th February 2008, 19:34   #269  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,558
Sorry, that was a reply to Zanejin.

You always have to use a new idx when you use a new source (unless you use pel=1, then idx doesn't offer a speedup). So you have one idx for MV with source, and another idx for MV on NR1.
foxyshadis is offline   Reply With Quote
Old 18th February 2008, 20:25   #270  |  Link
Nikos
Registered User
 
Join Date: Jun 2002
Location: Greece
Posts: 242
Thank you foxyshadis for the explain.
With simple words the right script is the 2 with idx=2.
Quote:
2.
source=last
b1_vec = source.MVAnalyse(isb = true, delta = 1, blksize =8, overlap = 4, pel = 2, sharp = 2, idx = 1)
f1_vec = source.MVAnalyse(isb = false, delta = 1, blksize =8, overlap = 4, pel = 2, sharp = 2, idx = 1)

NR1=source.MVDegrain1(b1_vec, f1_vec, thSAD = 400, idx = 1)
NRb=NR1.MVDegrain1(b1_vec, f1_vec, thSAD= 400, idx = 2)
__________________
Greece PAL User...
Nikos is offline   Reply With Quote
Old 19th February 2008, 07:37   #271  |  Link
thetoof
Sleepy overworked fellow
 
Join Date: Feb 2008
Location: Maple syrup's homeland
Posts: 933
@ Didée

From what you said... I guess/hope you'll rewrite the script to correct everything and I was wondering if you could add (or simply explain how to do it) a function to use dfttest and fft3dfilter at the same time. Maybe it's just a bad idea to use 2 denoisers... but the source I have requires additional chroma denoise with specific s2/3/4 values with fft3d to be clean.

I tweaked the parameters of dfttest in the TemporalDegrain.avsi and I got pretty good results, but couldn't find a way to add fft3d... hope you'll be willing to help me.
thetoof is offline   Reply With Quote
Old 25th February 2008, 00:16   #272  |  Link
archaeo
Banned
 
archaeo's Avatar
 
Join Date: Apr 2004
Location: Usa
Posts: 522
Just tried sagekilla's TemporalDegrain() on a very grainy old film transfer: "I am a Fugitive from a Chain Gang" (1932), and I am very impressed with the results (See attachments). The grain on the original transfer was extremely heavy, and other degrain combinations never got close to the results I got with TD... Slow...YES, (even with an E6700, I only got about 2 fps) but well worth the wait.
I have also tried this on a couple of other older films, with similar results - grain virtually gone, but excellent detail retention. Very nice script!

I only wish there was a way to speed it up a bit with MT

Original clip: http://www.mediafire.com/?mi11znznz3r
TD clip: http://www.mediafire.com/?its3m3z9yud

stills:

Name:  original.jpg
Views: 1029
Size:  30.0 KB

Name:  temporal degrain.jpg
Views: 973
Size:  18.9 KB

Last edited by archaeo; 26th February 2008 at 17:46.
archaeo is offline   Reply With Quote
Old 28th February 2008, 02:01   #273  |  Link
Nikos
Registered User
 
Join Date: Jun 2002
Location: Greece
Posts: 242
I read an old Didee's post in another forum and i replace the prefilter with this:
Code:
soft = hqdn3d(source).bicubicresize(source.width/4/4*4,source.height/4/4*4).bicubicresize(source.width,source.height,1,0)
filter=mt_lutxy( source, soft, "x y = x x x y - abs 1 + 1 2 / ^ x y - x y - abs / * - ?", U=2, V=2 )
The different in rendering speed was respectable, +40% with overlap=4, and +70% with overlap=2.
I use MVDegrain2 and MT(2,2).
The quality in my eyes was the same, but i want an opinion from experienced user.

Any idea for another correct "spat" filter?

Didee help us
__________________
Greece PAL User...

Last edited by Nikos; 28th February 2008 at 02:05.
Nikos is offline   Reply With Quote
Old 28th February 2008, 08:42   #274  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,389
Several issues.

1) The string in lutxy is wrong. That syntax was for YV12lutxy() of old MaskTools v1.
For mt_Lutxy of MaskTools v2, the string has to be "x y == ..." (TWO equal signs. Try to retun "filter" with your current syntax, you'll immediately see that something's wrong.)

2) With that order of filters, you're actually nullifying the effect of HQDN3D. In this case, the more reasonable ordering is
Code:
soft = source.bicubicresize(source.width/4/4*4,source.height/4/4*4).bicubicresize(source.width,source.height,1,0)
filter=mt_lutxy( source, soft, "x y == x x x y - abs 1 + 1 2 / ^ x y - x y - abs / * - ?", U=2, V=2 ).hqdn3d()
3) Whether pre-denoising is needed at all depends on the stronginess of grain. I see slight danger that this sort of processing now gets thrown at all kind of sources, even those that don't need such a processing method.

4) That combo is a very speedy trick-filter. It might work out for the most part, but there'll be cases where it bites you back. Dumb filters can destroy the motion of rather smooth regions (e.g. close-up of smooth faces + head-movement) strong enough so that MVTools won't reckognize motion anymore.
BTW, HQDN3D is generally not fully safe in this respect.

4) We have about one thousand different denoising filters or -scripts. If source has strong grain, the pre-filtering should be barely strong enough to make static areas calm.
Which filter that is or could be, depends on the source. If you're going to ask about all thousand possibilities, this thread will become long.
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline   Reply With Quote
Old 28th February 2008, 14:52   #275  |  Link
Nikos
Registered User
 
Join Date: Jun 2002
Location: Greece
Posts: 242
Thanks Didée for the correction. That syntax was for YV12lutxy(). The reverse polish notation is a little difficult. What's the different beetween the "==" and "="? These days i read your old post to find out solutions for my problems
If you want translate in simple maths the function.

I need a speedy prefilter whith acceptable (not high) quality with for High Definition sources with moderate or low noise. The prefilter must be spatial, temporal, or spatiotemporal; I must put the spatial first or the temporal;

The FFT3DGPU is suitable for prefilter?
It's a good idea to write a "magic" script for prefilter in MVDegrain with options for heavy, medium or light denoise

Any help.....
__________________
Greece PAL User...

Last edited by Nikos; 28th February 2008 at 15:20.
Nikos is offline   Reply With Quote
Old 28th February 2008, 16:35   #276  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,389
"=" is an assignment, whereas "==" means "is equal to" in logical expressions.

a = 3 # assign the value "3" to the variable "a"

a == 4 # this is "TRUE" if "a" has the value "4", and is "FALSE" if "a" has ar value other than "4".

The syntax used by MaskTools v1 ever was wrong in that respect. MaskTools v2 now has it correct.


Quote:
The reverse polish notation is a little difficult.
No, not really. It's just a matter of practice.

"x y == x x x y - abs 1 + 1 2 / ^ x y - x y - abs / * - ?"

reads in english

IF (x is equal to y)
THEN (use x)
ELSE (apply the square root of (difference between x and y, plus 1) to x)


Quote:
I need a speedy prefilter whith acceptable (not high) quality with for High Definition sources with moderate or low noise.
For low noise you need exactly no prefiltering at all. Use MVDegrainX directly as stated in MVTools' documentation.


Quote:
The FFT3DGPU is suitable for prefilter?
When asking questions where the only objective answer is "there is no objective answer", it won't get better if you guys keep asking over and over again.
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)

Last edited by Didée; 28th February 2008 at 17:17.
Didée is offline   Reply With Quote
Old 28th February 2008, 17:09   #277  |  Link
Nikos
Registered User
 
Join Date: Jun 2002
Location: Greece
Posts: 242
Now with your help i begin to understand the mask tool and the Reverse Polish notation.
Now i am going for practice. I will be back soon with more serious questions

Thanks for your time to answer my questions.
__________________
Greece PAL User...
Nikos is offline   Reply With Quote
Old 28th February 2008, 19:23   #278  |  Link
jeffy
Registered User
 
Join Date: Jan 2007
Posts: 943
Quote:
Originally Posted by Didée View Post

"x y == x x x y - abs 1 + 1 2 / ^ x y - x y - abs / * - ?"

reads in english

IF (x is equal to y)
THEN (use x)
ELSE (apply the square root of (difference between x and y, plus 1) to x)


Didée, could you please rewrite the lime part in a normal (infix) notation? Thank you.
jeffy is offline   Reply With Quote
Old 28th February 2008, 21:01   #279  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,389
In infix notation it is

(x == y) ? x : x - ( sqrt( abs( x - y ) + 1 ) * ( x - y ) / abs( x - y ) )

That's very basic math. Everyone once has used something like

clip1 = last
clip2 = clip1.blur(1)
result = clip1 .merge( clip2, 0.4 )

What is this? It is: apply a filter to your source (here: blur filter), but use only 40% of the result. (Or the resulting effekt). That's trivial, isn't it?

From a slightly different angle, it is this: calculate something, but instead of just using the resulting difference, multiply the difference by 0.4 before using it. That's a linear function: F(diff) = diff * p

Now, what the initial code does is just this: instead of using a linear function to reduce the effect, it uses a square root function: F(diff) = sqrt(diff)

That's all, and hardly worth all the babbling.
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline   Reply With Quote
Old 28th February 2008, 22:48   #280  |  Link
Lorax2161
Registered User
 
Lorax2161's Avatar
 
Join Date: Oct 2007
Location: Forest of the Trufulas
Posts: 37
Actually, thank you for "babbling." Not everyone is at the same level, and I learned some things from it. A good explanation goes a long way to helping people open doors they thought were locked.

BTW, to further on what archaeo said, I ran TemporalDegrain() on a somewhat noisy cable tv capture and the results were stunning. (I didn't know how noisy it was until after it was cleaned.) I only get 1.8fps on my dual core machine, but I haven't seen anything at any speed that comes close to that quality, especially true if I follow it with LSF. So to you and sagekilla I say thanks.
Lorax2161 is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 15:38.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.