Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 14th November 2013, 04:40   #1  |  Link
Chainmax
Huh?
 
Chainmax's Avatar
 
Join Date: Sep 2003
Location: Uruguay
Posts: 3,103
Trying to make MDegrain3 not apply on a portion of a movie, need help.

I am backing up a series and this latest episode I'm tackling has a section that is particularly nasty noise-wise. After some experimentation with nuke-it-from-orbit filtering applied to that section, I decided to leave it be, as the loss of detail and huge amount of extra processing time was not worth it.

Since the scene is short and mostly static, leaving the brunt of the denoising out of it is a reasonable compromise. Thing is, the main denoising is done with MDegrain3, like so:

Code:
source=last
preNR=source.dfttest(sigma=1.00,tbsize=3).FFT3DFilter(sigma=9,plane=3,bw=32,bh=32,bt=3,ow=16,oh=16)
preNR_super=preNR.MSuper(pel=2, sharp=1)
source_super=source.MSuper(pel=2, sharp=1)
backward_vec3 = MAnalyse(preNR_super,isb = true, delta = 3, overlap=4)
backward_vec2 = MAnalyse(preNR_super,isb = true, delta = 2, overlap=4)
backward_vec1 = MAnalyse(preNR_super,isb = true, delta = 1, overlap=4)
forward_vec1 = MAnalyse(preNR_super,isb = false, delta = 1, overlap=4)
forward_vec2 = MAnalyse(preNR_super,isb = false, delta = 2, overlap=4)
forward_vec3 = MAnalyse(preNR_super,isb = false, delta = 3, overlap=4)
source.MDegrain3(source_super,backward_vec1,forward_vec1,backward_vec2,forward_vec2,backward_vec3,forward_vec3,thSAD=350)
The problem section is frames 5535 to 5658. Having never had to make multi-line filters like this one not work on a range, I am at a loss on how to proceed without overcomplicating things. Is there an elegant-ish way to do this, or will I have to cough a mess of Trims and splices?
__________________
Read Decomb's readmes and tutorials, the IVTC tutorial and the capture guide in order to learn about combing and how to deal with it.

Last edited by Chainmax; 14th November 2013 at 04:45.
Chainmax is offline   Reply With Quote
Old 14th November 2013, 08:01   #2  |  Link
cretindesalpes
͡҉҉ ̵̡̢̛̗̘̙̜̝̞̟̠͇̊̋̌̍̎̏̿̿
 
cretindesalpes's Avatar
 
Join Date: Feb 2009
Location: No support in PM
Posts: 610
Code:
f1 = filter1 ()
f2 = filter2 ()
f1.rfs (f2, "[5535 5658]")
rfs is a modification of ReplaceFramesSimple and can be found here.
You can add more ranges in the mappings string if required.
__________________
dither 1.27.2 for AviSynth | avstp 1.0.3 for AviSynth development | fmtconv r19 for Vapoursynth | trimx264opt segmented encoding
cretindesalpes is offline   Reply With Quote
Old 14th November 2013, 08:04   #3  |  Link
martin53
Registered User
 
Join Date: Mar 2007
Posts: 405
The usual solution is wrap the multi-line process in a function
Code:
function filter(clip source) {
    preNR=source.dfttest(sigma=1.00,tbsize=3).FFT3DFilter(sigma=9,plane=3,bw=32,bh=32,bt=3,ow=16,oh=16)
    preNR_super=preNR.MSuper(pel=2, sharp=1)
    source_super=source.MSuper(pel=2, sharp=1)
    backward_vec3 = MAnalyse(preNR_super,isb = true, delta = 3, overlap=4)
    backward_vec2 = MAnalyse(preNR_super,isb = true, delta = 2, overlap=4)
    backward_vec1 = MAnalyse(preNR_super,isb = true, delta = 1, overlap=4)
    forward_vec1 = MAnalyse(preNR_super,isb = false, delta = 1, overlap=4)
    forward_vec2 = MAnalyse(preNR_super,isb = false, delta = 2, overlap=4)
    forward_vec3 = MAnalyse(preNR_super,isb = false, delta = 3, overlap=4)
    source.MDegrain3(source_super,backward_vec1,forward_vec1,backward_vec2,forward_vec2,backward_vec3,forward_vec3,thSAD=350)
}
and call the function
Code:
source=last
last.trim(...,...).filter
\++ last.trim(...,...)
\++ last.trim(...,...).filter
martin53 is offline   Reply With Quote
Old 14th November 2013, 08:20   #4  |  Link
TurboPascal7
Registered User
 
TurboPascal7's Avatar
 
Join Date: Jan 2010
Posts: 270
That usual solution is a really bad idea for caching, performance and stability. Plus with temporal filters you will get a bit different processing on first/last frames.

A better solution would be doing it like
Code:
source=last
filtered = filter() # you can just put all the code here and use source instead of last later
filtered.trim(...,...)
\++ last.trim(...,...)
\++ filtered.trim(...,...)
But this gets unreadable pretty fast. cretindesalpes's advice works better for this kind of filtering.
__________________
Me on GitHub | AviSynth+ - the (dead) future of AviSynth
TurboPascal7 is offline   Reply With Quote
Old 14th November 2013, 08:31   #5  |  Link
martin53
Registered User
 
Join Date: Mar 2007
Posts: 405
Quote:
Originally Posted by TurboPascal7 View Post
That usual solution is a really bad idea for caching, performance and stability. Plus with temporal filters you will get a bit different processing on first/last frames.
Please indicate how the filter call will affect stability.
There is nothing to be cached beyond the trim points, so what should be the matter here. And same with performance: there are 2 instances of the filter functions, but these are only called for their smaller part of the clip. In sum, exactly the same number of frames is sent through the filter.
We are in consent that there is a difference in processing adjacent to the trim points.
But in my opinion, it is not a good, but in contrary a bad idea to include some frames of the unfiltered part in the filtering. Probably there's a scene change, so these frames have nothing to do with the frames at the other side of the fence. That's why I deliberately put the filter to the end of the graph. And Cretindesalpes' proposal suffers from the same issue.
martin53 is offline   Reply With Quote
Old 14th November 2013, 08:39   #6  |  Link
TurboPascal7
Registered User
 
TurboPascal7's Avatar
 
Join Date: Jan 2010
Posts: 270
Quote:
Originally Posted by martin53 View Post
Please indicate how the filter call will affect stability.
There is nothing to be cached beyond the trim points, so what should be the matter here. And same with performance: there are 2 instances of the filter functions, but these are only called for their smaller part of the clip. In sum, exactly the same number of frames is sent through the filter.
In perfect world - maybe. Avisynth, however, likes to crash when you create too many filter instances. The most reasonable explanation for this I've heard is "imagine filters allocating 1.5GB of memory (total) in constructors and avisynth's cache being limited to 1GB. After allocating 500MB of its cache, avs will happily continue allocating more memory eventually crashing thanks to the 32-bit memory limit". Exact numbers are not important - I had a short (1700 frames) script crashing every 300 frames or so with avs2yuv patched for 4GB memory.

If only there was a way for avisynth to detect that filter XXX was called on clip YYY with the same parameters before and reuse the same filter instance...
__________________
Me on GitHub | AviSynth+ - the (dead) future of AviSynth

Last edited by TurboPascal7; 14th November 2013 at 08:41.
TurboPascal7 is offline   Reply With Quote
Old 14th November 2013, 10:42   #7  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,377
The original question had only one unfiltered section, so martin53's approach is reasonable for that (particularly considering his point about scene change). However, if there are many sections, cretindesalpes's approach is prefererable to avoid excessive memory requirements.
__________________
GScript and GRunT - complex Avisynth scripting made easier
Gavino is offline   Reply With Quote
Old 14th November 2013, 17:18   #8  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 6,054
ClipClop is another similar alternative to rfs.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 17th November 2013, 15:10   #9  |  Link
Chainmax
Huh?
 
Chainmax's Avatar
 
Join Date: Sep 2003
Location: Uruguay
Posts: 3,103
I tried martin53's suggestion. I will update on how the encoding process goes. Thanks.
__________________
Read Decomb's readmes and tutorials, the IVTC tutorial and the capture guide in order to learn about combing and how to deal with it.
Chainmax is offline   Reply With Quote
Old 22nd November 2013, 18:00   #10  |  Link
raffriff42
Retried Guesser
 
raffriff42's Avatar
 
Join Date: Jun 2012
Posts: 1,377
Quote:
Originally Posted by martin53 View Post
The usual solution is wrap the multi-line process in a function...
Quote:
Originally Posted by TurboPascal7 View Post
That usual solution is a really bad idea for caching, performance and stability. Plus with temporal filters you will get a bit different processing on first/last frames.
Don't want to sidetrack the thread, but do you have more information on this? I thought, reading the the Avisynth wiki, that any filter not in the graph for the frame currently being fetched will not be called. I suppose caching might make this more complicated. Anyway, I like using lot of small functions for code re-use and would like to know if there is really a performance hit.
raffriff42 is offline   Reply With Quote
Old 22nd November 2013, 21:04   #11  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,377
Quote:
Originally Posted by raffriff42 View Post
I thought, reading the the Avisynth wiki, that any filter not in the graph for the frame currently being fetched will not be called.
That's correct. However, all filters are loaded at compile-time and remain loaded (and occupying memory) until the script is unloaded, even if they are not being used for the current frame.
Quote:
Anyway, I like using lot of small functions for code re-use and would like to know if there is really a performance hit.
There is no performance hit with functions (compared with using the same code outside a function).
TurboPascal7's objection was not to the function itself, but the way it was used. In martin53's script, the function was called twice (and in a larger example potentially many more times), whereas with cretindesalpes's approach (and with TurboPascal7's own script), the function would be called just once and the different frame ranges obtained from that single instance.
__________________
GScript and GRunT - complex Avisynth scripting made easier
Gavino is offline   Reply With Quote
Old 24th November 2013, 00:06   #12  |  Link
raffriff42
Retried Guesser
 
raffriff42's Avatar
 
Join Date: Jun 2012
Posts: 1,377
Quote:
Originally Posted by Gavino View Post
In martin53's script, the function was called twice (and in a larger example potentially many more times), whereas with cretindesalpes's approach (and with TurboPascal7's own script), the function would be called just once and the different frame ranges obtained from that single instance.
Thank you for the answer. So, if I understand you: if a function call appears twice in a script, two instances of the function are generated, and the filter calls within each instance will have redundant caches, and this is where the inefficiency (and bugs) come in? This being true even if (and this is where I was confused) the function would only be called once for any given frame? (compile time != run time)
raffriff42 is offline   Reply With Quote
Old 24th November 2013, 01:20   #13  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,377
Quote:
Originally Posted by raffriff42 View Post
if a function call appears twice in a script, two instances of the function are generated, and the filter calls within each instance will have redundant caches, and this is where the inefficiency (and bugs) come in?
Yes, more or less.
More accurately, each time the function is called in the script, its body is evaluated, generating a fresh section of filter graph, with new instances of the filters used. (The function itself does not exist at run-time, but the filter instances do.)

It's not just the caches, the filters themselves (depending on their requirements) may allocate memory and each instance will have its own copy. So it makes sense to create as few instances as possible when you have the choice.
__________________
GScript and GRunT - complex Avisynth scripting made easier
Gavino is offline   Reply With Quote
Old 25th November 2013, 03:05   #14  |  Link
WorBry
Registered User
 
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 978
Great stuff; I was contemplating more or less the same thing myself, although in the context of a predefined ('wrapped', as it were) function (SMDegrain specifically).

So, just to be clear - the consensus is that when seeking applying a given function to multiple ranges in a clip, the most efficient (least resource-demanding) way is cretindesalpes's rfs approach?

I'm just trying to figure how to apply that to a defined function with variable parameters when you want to apply the function with different settings to different segments of the clip.

Taking SMDegrain, just as an example

http://doom10.org/index.php?topic=2178.0

Lets say I have a clip of 500 frames and wish to apply:

SMDegrain(tr=1) to frame ranges 50 - 100 and 250 - 300
and
SMDegrain(tr=3) to frame ranges 150 - 200 and 350 - 400

I can see how that might be done using martin53's sequential 'trim/filter' call method, but I just can't figure how you would write that using rfs. Does each of those ranges require a separate rfs/function call in the mapping string, or is possible to apply more than one range to a given rfs/function call?
__________________
Nostalgia's not what it used to be

Last edited by WorBry; 25th November 2013 at 04:34.
WorBry is offline   Reply With Quote
Old 25th November 2013, 10:46   #15  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,377
Quote:
Originally Posted by WorBry View Post
So, just to be clear - the consensus is that when seeking applying a given function to multiple ranges in a clip, the most efficient (least resource-demanding) way is cretindesalpes's rfs approach?
Yes, as long as the same function parameters are to be used for each range and, for temporal filters, that martin53's point (post #5) about possible processing differences at range boundaries is not a problem.

Quote:
Lets say I have a clip of 500 frames and wish to apply:

SMDegrain(tr=1) to frame ranges 50 - 100 and 250 - 300
and
SMDegrain(tr=3) to frame ranges 150 - 200 and 350 - 400

I can see how that might be done using martin53's sequential 'trim/filter' call method, but I just can't figure how you would write that using rfs. Does each of those ranges require a separate rfs/function call in the mapping string, or is possible to apply more than one range to a given rfs/function call?
You can have more than one range in a call, but since it replaces frames from one clip by those from another clip, you need two replacement clips, one for each distinct SMDegrain call. In your example, you would have two rfs calls, each with two ranges.
Code:
d1 = SMDegrain(tr=1)
d3 = SMDegrain(tr=3)
rfs (d1, "[50 100] [250 300]")
rfs (d3, "[150 200] [350 400]")
__________________
GScript and GRunT - complex Avisynth scripting made easier
Gavino is offline   Reply With Quote
Old 25th November 2013, 14:44   #16  |  Link
WorBry
Registered User
 
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 978
Quote:
Originally Posted by Gavino View Post
You can have more than one range in a call, but since it replaces frames from one clip by those from another clip, you need two replacement clips, one for each distinct SMDegrain call. In your example, you would have two rfs calls, each with two ranges.
Code:
d1 = SMDegrain(tr=1)
d3 = SMDegrain(tr=3)
rfs (d1, "[50 100] [250 300]")
rfs (d3, "[150 200] [350 400]")
Yes, exactly that. So it's that simple. Process efficiency aside, that definitely makes for greater ease and economy in the scripting - not having to account for the un-filtered segments. Brilliant.
__________________
Nostalgia's not what it used to be
WorBry is offline   Reply With Quote
Old 25th November 2013, 20:44   #17  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 6,054
ClipClop allows for up to 255 replacement clips without multiple plugin instances (and unlimited number of ranges).

see MediaFire in sig
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 2nd December 2013 at 15:29.
StainlessS is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 13:52.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.