Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 24th June 2012, 13:34   #1  |  Link
zero9999
Registered User
 
Join Date: Oct 2011
Posts: 52
Running out of memory when processing parts of a script differently

i'm running a script that debands parts of a video (1080p, 16-bit stacked) differently, which includes using different masks for protecting details.

Code:
Trim(0,23) +\
Trim(24,100).DebandStrong("aamask_fine",false,true) +\
Trim(101,174).DebandLight("aamask_fine",false) +\
Trim(175,196).DebandStrong("aamask_fine",false,true) +\
Trim(197,948).DebandLight("aamask_fine",false) +\
Trim(949,984).DebandStrong("aamask_dull",false,true) +\
Trim(985,1269).DebandLight("aamask_fine",false) +\
Trim(1270,1313).DebandStrong("aamask_fine",true,true) +\
Trim(1314,1629).DebandLight("aamask_fine",false) +\
Trim(1630,1671).DebandStrong("aamask_veryfine",true,true) +\
Trim(1672,1933).DebandLight("aamask_fine",false) +\
Trim(1934,2044).DebandStrong("aamask_fine",true,true) +\
the functions look like this:

Code:
function DebandStrong(clip c, string "mask_name", bool "less", bool "addgrain", string "mask_file")
{
mask_name=Default(mask_name,"aamask_fine")
addgrain=Default(addgrain, false)
mask_file=Default(mask_file, "none")
less=Default(less, false)

mask=BlankClip(c,width=c.Width,height=c.Height/2).KillAudio().Subtitle(mask_name)   
mask = (mask_name=="aamask_dull") ? c.DitherPost(mode=6).Dither_Luma_Rebuild(lsb=false,S0=1.1,c=0.35).tcanny(sigma=1.2, mode=1, plane=7).RemoveGrain(27).mt_lut("x 13 < 0 255 ?" , U=1, V=1).mt_binarize() : mask
mask = (mask_name=="aamask_fine") ? c.DitherPost(mode=6).Dither_Luma_Rebuild(lsb=false,S0=1.1,c=0.35).tcanny(sigma=1.2, mode=1, plane=7).RemoveGrain(27).mt_lut("x 10 < 0 255 ?" , U=1, V=1)\
.mt_logic(c.DitherPost(mode=6).MinBlur().Dither_Luma_Rebuild(lsb=false,S0=1.4,c=0.45).FastlinedarkenMOD(strength=40,thinning=0, luma_cap=255, threshold=0).MinBlur(2).mt_edge(mode="hprewitt"), mode="or").mt_binarize() : mask
mask = (mask_name=="aamask_veryfine") ? c.DitherPost(mode=6).Dither_Luma_Rebuild(lsb=false,S0=1.1,c=0.35).tcanny(sigma=1.2, mode=1, plane=7).RemoveGrain(27).mt_lut("x 7 < 0 255 ?" , U=1, V=1)\
.mt_logic(c.DitherPost(mode=6).MinBlur().Dither_Luma_Rebuild(lsb=false,S0=1.4,c=0.45).FastlinedarkenMOD(strength=40,thinning=0, luma_cap=255, threshold=0).MinBlur(2).mt_edge(mode="hprewitt"), mode="or").mt_binarize() : mask
mask = (mask_name=="aamask_ultrafine") ?  c.DitherPost(mode=6).Dither_Luma_Rebuild(lsb=false,S0=1.4,c=0.75).tcanny(sigma=1.1, mode=1, plane=7).mt_lut("x 7 < 0 255 ?" , U=1, V=1)\.mt_logic(c.DitherPost(mode=6).MinBlur().Dither_Luma_Rebuild(lsb=false,S0=1.4,c=0.45).FastlinedarkenMOD(strength=40,thinning=0, luma_cap=255, threshold=0).MinBlur(2).mt_edge(mode="hprewitt"), mode="or").mt_binarize() : mask
mask = (mask_name=="special") ? ImageSource(mask_file,0,c.FrameCount-1,c.FrameRate).ConvertToYV12(matrix="PC.709").mt_binarize() : mask


debanded= !less ? c.GradFun3(lsb=true, lsb_in=true, thr=0.75, radius=16, smode=2, elast=3, thr_det=2, mask=0) : \
                  c.GradFun3(lsb=true, lsb_in=true, thr=0.55, radius=16, smode=2, elast=2.5, thr_det=2, mask=0)
grainy_debanded = addgrain ? debanded.GrainFL(lsb=true, lsb_in=true, g1str=3, g2str=10, g3str=5, g1shrp=50, g2shrp=60, g3shrp=80, g1size=0.9, g2size=0.7, g3size=0.6, th1=16, th2=24, th3=128, th4=180, g1soft=15, g2soft=15, g3soft=15) : \
                  debanded
debanded=addgrain ? mt_lutxy(debanded, grainy_debanded, expr="x 2 * y 4 * + 6 /") : debanded
merged=dither_merge16_8(debanded,c,mask,luma=false)
return merged
}
(the light variant uses f3kdb instead.

the debanding is run in a separate process (via mp_pipeline) to make use of the full 4GB memory available to a 32-bit Avisynth process.

When called only once (or twice), those functions run just fine, but calling them many times on different video sections causes out of memory errors.
Lower values for SetMemoryMax cause this error:
Code:
avs [error]: SharedMemoryClient: Server got error: GetFrameBuffer: Returned a VFB with a 0 data pointer!
size=8601664, max=268435456, used=1693698752
I think we have run out of memory folks! occurred while reading frame 13
Higher values cause different errors related to frame buffer errors or the process just hangs.
I also tried using ApplyRange instead of Trims while building the masks outside of the debanding functions (in a different process) and passing them as clips to the DebandLight()/DebandStrong()... to no avail.
The error happens regardless whether i use mp_pipeline or not and occurs both in AviSynth 2.5.8 (ST/MT) and AviSynth 2.6 (ST/MT).
I tried both piping the script output directly to x264 (via avs4x264mod) and encoding to UtVideo with VirtualDub. Tested this on a machine with 32GB RAM and a machine with 8GB RAM.

Is there any sane way to rearrange the script to avoid clogging up avisynths frame cache without encoding 4+ debanded versions and doing masking/stitching afterwards?

Last edited by zero9999; 24th June 2012 at 13:37.
zero9999 is offline   Reply With Quote
Old 24th June 2012, 16:10   #2  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,390
Each call to a filter or function invokes a full incarnation of that filter or funktion. The way you do now, If you stitch one thousands trims, then you (would) have to have one thousand copies of the filter in memory.


Instead: Prepare pre-processed clips as needed, and stitch those together.

Code:
vid = last
light = vid.DebandLight("aamask_fine",false)
strong = vid.DebandStrong("aamask_fine",false,true)

vid   .Trim(   0,  23) +\
strong.Trim(  24, 100) +\
light .Trim( 101, 174) +\
strong.Trim( 175, 196) +\
light .Trim( 197, 948) +\
strong.Trim( 949, 984) +\
light .Trim( 985,1269) +\
strong.Trim(1270,1313) +\
light .Trim(1314,1629) +\
strong.Trim(1630,1671) +\
light .Trim(1672,1933) +\
strong.Trim(1934,2044) +\
..etc..
This way you can make one thousand trims, but have only two copies of the filter actually in memory.
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline   Reply With Quote
Old 24th June 2012, 16:16   #3  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,380
Quote:
Originally Posted by Didée View Post
Each call to a filter or function invokes a full incarnation of that filter or funktion. The way you do now, If you stitch one thousands trims, then you (would) have to have one thousand copies of the filter in memory.

Instead: Prepare pre-processed clips as needed, and stitch those together.

This way you can make one thousand trims, but have only two copies of the filter actually in memory.
Right strategy, but he needs more than two because the filters are not called with the same arguments each time. You need a separate instance for each combination of (function + arguments).
__________________
GScript and GRunT - complex Avisynth scripting made easier
Gavino is offline   Reply With Quote
Old 24th June 2012, 17:46   #4  |  Link
mastrboy
Registered User
 
Join Date: Sep 2008
Posts: 287
Replaceframessimple might help: http://avisynth.org/stickboy/
mastrboy is offline   Reply With Quote
Old 24th June 2012, 18:04   #5  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 5,909
As might ClipClop().
http://forum.doom9.org/showthread.php?t=162266

EDIT: ClipClop deals with multiple replacement clips (up to 255),
which would seem to be a requirement in this case.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 25th June 2012 at 03:17.
StainlessS is offline   Reply With Quote
Old 25th June 2012, 08:23   #6  |  Link
cretindesalpes
͡҉҉ ̵̡̢̛̗̘̙̜̝̞̟̠͇̊̋̌̍̎̏̿̿
 
cretindesalpes's Avatar
 
Join Date: Feb 2009
Location: No support in PM
Posts: 610
With the same kind of strategy you could go even further and factorize almost all the code from the Deband* functions. You'll probably end up with everything at the top level or in a single bigger function. Filters in the script will be instantiated only once, not more. Just switch the signal path in your flowgraph with a ClipClop or ReplaceFramesSimple at each switching point. The latter is much easier to use than Trim because you can merge multiple sets of intervals just by concatenating the mapping strings.
__________________
dither 1.27.2 for AviSynth | avstp 1.0.3 for AviSynth development | fmtconv r19 for Vapoursynth | trimx264opt segmented encoding
cretindesalpes is offline   Reply With Quote
Old 26th June 2012, 03:34   #7  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 5,909
Quote:
Originally Posted by cretindesalpes View Post
The latter is much easier to use than Trim because you can merge multiple sets of intervals just by concatenating the mapping strings.
Hi Cretindesalpes, would it be possible to clarify what you meant there, I've spent more than a few few minutes trying to figure it out. Are you saying that both ClipClop and ReplaceFramesSimple are easier than trim or that ReplaceFramesSimple is
easier than ClipClop, You have a unique (almost mystical) way of describing things.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 26th June 2012 at 03:45.
StainlessS is offline   Reply With Quote
Old 26th June 2012, 06:16   #8  |  Link
cretindesalpes
͡҉҉ ̵̡̢̛̗̘̙̜̝̞̟̠͇̊̋̌̍̎̏̿̿
 
cretindesalpes's Avatar
 
Join Date: Feb 2009
Location: No support in PM
Posts: 610
Quote:
Originally Posted by StainlessS View Post
Hi Cretindesalpes, would it be possible to clarify what you meant there, I've spent more than a few few minutes trying to figure it out. Are you saying that both ClipClop and ReplaceFramesSimple are easier than trim or that ReplaceFramesSimple is
easier than ClipClop, You have a unique (almost mystical) way of describing things.
Sorry, English is not my native language. I meant that ReplaceFramesSimple is easier than Trim for this kind of job, because you can easily do things like :

Code:
r1 = "[123 456] [999 6666]"
r2 = "[789 888] [5000 10000]"

SourceFilter ()
tmp = Filter1 ()
ReplaceFramesSimple (tmp, mappings=r1)
tmp = Filter2 ()
ReplaceFramesSimple (tmp, mappings=r2)
Filter3 ()
tmp = Filter4 ()
ReplaceFramesSimple (tmp, mappings=r1+" "+r2)
And you don't have to worry whether ranges from r1 and r2 are overlapping or not, or their order within the string. With ClipClop it's a bit more complicated because you have to care about clip names/numbers too.
__________________
dither 1.27.2 for AviSynth | avstp 1.0.3 for AviSynth development | fmtconv r19 for Vapoursynth | trimx264opt segmented encoding
cretindesalpes is offline   Reply With Quote
Old 26th June 2012, 14:34   #9  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 5,909
@cretindesalpes,
Thanks for your clarification, that is how I should have interpreted it.
Yes, I suppose that you are correct in that only supporting 1 source clip,
you do not need to care what the source clip name or number is, possibly
a case of less is more. Too late to change ClipClop(), to assume clip index
1 if only 1 source clip supplied, as the frame/range specification is so very
loose and flexible in what it will accept, would have required a much tighter
spec (perhaps the 'R' in 'Rx' should not have been made optional).

EDIT: @All,
In Clipclop, it also does not matter about range order in string command nor
overlapping ranges , (as in ReplaceFramesSimple), the later one takes effect.
Also, you can concatenate command/mapping strings using either the end of line
character 'Chr(10)' or the Clipclop soft end of line character ';', whereas in
previous post code sample the space character was used when concatenating
two mapping strings ie 'mappings=r1+" "+r2', in Clipclop you could use eg 'SCmd=r1+";"+r2'.
In both cases, the separation character is just to ensure that the parsers can recognise where
one range/frame ends and the next begins.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 27th June 2012 at 12:00.
StainlessS is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 09:44.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.