Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
|
|
Thread Tools | Search this Thread | Display Modes |
14th February 2013, 02:09 | #61 | Link |
Registered User
Join Date: Aug 2007
Posts: 218
|
I can't really handle it in my code (except for forcibly terminating the slave process itself on exit, but I don't want to do that since some plugin can't properly clean up in that case). Maybe you can try another version of QuickTime, or just use ffms2?
__________________
f3kdb 1.5.1 / MP_Pipeline 0.18 ffms2 builds with 10bit output hack: libav-9a60b1f / ffmpeg-1e4d049 / FFmbc-0.7.1 Built from ffms2 6e0d654 (hack a9fe004) Mirrors: http://bit.ly/19TwDD3 |
20th May 2013, 23:23 | #63 | Link |
Registered User
Join Date: Sep 2012
Posts: 156
|
Hello,
latest adopter of MP_Pipeline here. As I was suggested, I copied all the x86 files into avs plugins directory (using SET's 2.6 MT), then made the following script: MP_Pipeline(""" f3kdb(sample_mode=2,dynamic_grain=false,keep_tv_range=false,dither_algo=3,y=48,cb=24,cr=24,grainY=48,grainC=24) ### ### BlindDehalo3(rx=2.5, ry=2.5, strength=125) ### lock threads to cores """) But for the life of me, I can't get it to work. All it gives me is this error msg: MP_Pipeline: Unable to create slave process. Message: Script Error: Script Error: Invalid arguments to function 'f3kdb' But everything works when I remove MP stuff. What am I doing wrong? Thanks! |
21st May 2013, 01:08 | #64 | Link | |
Registered User
Join Date: Aug 2007
Posts: 218
|
Quote:
Code:
MP_Pipeline(""" FFVideoSource("abcde.mkv") f3kdb(sample_mode=2,dynamic_grain=false,keep_tv_range=false,dither_algo=3,y=48,cb=24,cr=24,grainY=48,grainC=24) ### ### BlindDehalo3(rx=2.5, ry=2.5, strength=125) ### lock threads to cores # By the way, you need to put another block splitter here, # because all special statement won't be effective for code # after the last splitter. ### ### """)
__________________
f3kdb 1.5.1 / MP_Pipeline 0.18 ffms2 builds with 10bit output hack: libav-9a60b1f / ffmpeg-1e4d049 / FFmbc-0.7.1 Built from ffms2 6e0d654 (hack a9fe004) Mirrors: http://bit.ly/19TwDD3 |
|
21st May 2013, 01:58 | #65 | Link |
Registered User
Join Date: Sep 2012
Posts: 156
|
Yup, figured out the source filter bit meanwhile.
But now it's saying that there's no function named BlindDehalo3 even if I'd call it explicitly via import after source filter. Interestingly, f3kdb now works, though. Thanks a lot for the reply btw, not only directly from the author him/herself, but so promptly indeed |
21st May 2013, 02:09 | #66 | Link | |
Registered User
Join Date: Aug 2007
Posts: 218
|
Quote:
Actually, it will be much easier to put all plugins/imported scripts into your avisynth plugins folder, so that they will be loaded automatically.
__________________
f3kdb 1.5.1 / MP_Pipeline 0.18 ffms2 builds with 10bit output hack: libav-9a60b1f / ffmpeg-1e4d049 / FFmbc-0.7.1 Built from ffms2 6e0d654 (hack a9fe004) Mirrors: http://bit.ly/19TwDD3 |
|
21st May 2013, 23:00 | #67 | Link | |
Registered User
Join Date: Sep 2012
Posts: 156
|
Well, they all are in the plugins folder, but I still have to load them manually...
I don't know what to say. Even if I wrap the MP_Pipeline around all the script, with breaks (### ###) between parts, I still get the same msg about invalid arguments. For instance, let's take this one: Quote:
it comes back with "invalid arguments" on degrainmedian. I don't know what else to try. I'm sure it's something minor I'm missing as usual. Last edited by aldix; 21st May 2013 at 23:12. |
|
22nd May 2013, 16:22 | #71 | Link | |
Registered User
Join Date: Sep 2012
Posts: 156
|
Apologies for late reply, too busy.
It's basically just the script above, with MP stuff inserted. Anyway, however I place the MP calls, it comes back with some filter having 'invalid arguments'. Quote:
edit: Sorry, forgot to add source filter at first, fixed. Last edited by aldix; 22nd May 2013 at 20:50. |
|
23rd May 2013, 07:21 | #73 | Link |
Registered User
Join Date: Aug 2007
Posts: 218
|
Variables won't be automatically transferred to next script block. For clip variables you need "### export clip" to use it in next block, for others you need "### inherit". Please see the example script in the package for details, because I am on vacation now, I can't provide another example for you.
__________________
f3kdb 1.5.1 / MP_Pipeline 0.18 ffms2 builds with 10bit output hack: libav-9a60b1f / ffmpeg-1e4d049 / FFmbc-0.7.1 Built from ffms2 6e0d654 (hack a9fe004) Mirrors: http://bit.ly/19TwDD3 |
23rd May 2013, 16:48 | #74 | Link | |
Registered User
Join Date: Sep 2012
Posts: 156
|
Quote:
I really, really appreciate all the help You've given so far SAPIkachu, I just can't seem to wrap my head around it, scarce script example isn't much use to me, I'm afraid. Thank You again. Last edited by aldix; 8th January 2014 at 00:56. |
|
24th May 2013, 01:18 | #75 | Link |
Registered User
Join Date: Aug 2007
Posts: 218
|
Actually, you should use "export clip" like this:
Code:
source = ... ### export clip: source
__________________
f3kdb 1.5.1 / MP_Pipeline 0.18 ffms2 builds with 10bit output hack: libav-9a60b1f / ffmpeg-1e4d049 / FFmbc-0.7.1 Built from ffms2 6e0d654 (hack a9fe004) Mirrors: http://bit.ly/19TwDD3 Last edited by SAPikachu; 25th May 2013 at 02:48. |
24th May 2013, 15:11 | #76 | Link | |
Registered User
Join Date: Sep 2012
Posts: 156
|
Quote:
Well, for this I followed the example script/tutorial. There it was said to list 'em separated by commas. And regardless, now it whines for other variables, like hpad. I don't know. Perhaps I just shouldn't try to use it or something. |
|
24th May 2013, 18:18 | #77 | Link |
Registered User
Join Date: Jan 2012
Location: Mesopotamia
Posts: 2,587
|
you can use the mp_pipleline without split by now, until you learn all its settings
like: Code:
MP_Pipeline(""" ### platform: win32 SetMemoryMax(1500) MPEG2Source("C:\Users\redacted\smileys people ep3\VTS_02_1.d2v", cpu=0) Crop(2, 0, -2, -0) source = last blksize = 16 overlap = blksize/2 hpad = blksize vpad = blksize thSAD = 300 halfblksize = blksize/2 halfoverlap = overlap/2 halfthSAD = thSAD/2 chroma = true search = 5 preNR = source.degrainmedian(mode=3,limity=8,limituv=10).fft3dfilter(wintype=1,degrid=1,bw=32,bh=32,ow=16,oh=16,bt=3,sigma=2.5,sigma2=2.2,sigma3=1.8,sigma4=0.5,plane=4,ncpu=1).GradFun2DB(1.01) preNR = source.degrainmedian(mode=3,limity=8,limituv=10).dfttest(sigma=6, ftype=1, tbsize=1, threads=1).GradFun2DB(1.01) preNR_super = preNR.MSuper(hpad=hpad, vpad=vpad, pel=2, sharp=2, rfilter=2, chroma=chroma) source_super = source.MSuper(hpad=hpad, vpad=vpad, pel=2, sharp=2, chroma=chroma, levels=1) Recalculate = preNR.MSuper(hpad=hpad, vpad=vpad, pel=2, sharp=2, rfilter=2, chroma=chroma, levels=1) vb2 = MAnalyse(preNR_super, isb=true, truemotion=false, delta=2, blksize=blksize, overlap=overlap, search=search, chroma=chroma) vbr2 = MRecalculate(Recalculate, vb2, overlap=halfoverlap, blksize=halfblksize, thSAD=halfthSAD, search=search, chroma=chroma) vb1 = MAnalyse(preNR_super, isb=true, truemotion=false, delta=1, blksize=blksize, overlap=overlap, search=search, chroma=chroma) vbr1 = MRecalculate(Recalculate, vb1, overlap=halfoverlap, blksize=halfblksize, thSAD=halfthSAD, search=search, chroma=chroma) vf1 = MAnalyse(preNR_super,isb=false, truemotion=false, delta=1, blksize=blksize, overlap=overlap, search=search, chroma=chroma) vfr1 = MRecalculate(Recalculate, vf1, overlap=halfoverlap, blksize=halfblksize, thSAD=halfthSAD, search=search, chroma=chroma) vf2 = MAnalyse(preNR_super,isb=false, truemotion=false, delta=1, blksize=blksize, overlap=overlap, search=search, chroma=chroma) vfr2 = MRecalculate(Recalculate, vf2, overlap=halfoverlap, blksize=halfblksize, thSAD=halfthSAD, search=search, chroma=chroma) maskp1 = MMask(vfr1, kind=1, ysc=255).UtoY() maskp2 = MMask(vfr2, kind=1).UtoY() maskp3 = MMask(vbr1, kind=1, ysc=255).UtoY() maskp4 = MMask(vbr2, kind=1).UtoY() tmask = average(maskp1, 0.25, maskp2, 0.25, maskp3, 0.25, maskp4, 0.25).spline36resize(source.width, source.height) source2 = mt_merge(source,preNR,tmask,Y=3,U=3,V=3) KEEP = "0.23" den = source2.MDegrain2(source_super,vbr1,vfr1,vbr2,vfr2,thSAD=thSAD,thSCD1=256,thSCD2=92) \.mt_adddiff(mt_makediff(source,preNR,U=3,V=3).mt_lut("x 128 - abs 1 < x x 128 - abs 1 - "+KEEP+" * x 128 - x 128 - abs 0.001 + / * 128 + ?",U=2,V=2),U=3,V=3) # PROTECTING threshold = 16 cutoff = 64 maxdiff = 4 rg17 = den.removegrain(17,-1) iOB = source.mt_lut("x "+string(cutoff)+" >= x 0 ?",U=1,V=1) mB = mt_makediff(iOB,rg17,U=1,V=1).mt_binarize(128+threshold,upper=false,U=1,V=1).removegrain(5,-1) lB = mt_lutxy(den,source,"x y - abs "+string(maxdiff)+" <= x x y - 0 < y "+string(maxdiff)+" - x ? ?",U=1,V=1) smB = mt_merge(den,lB,mB,U=2,V=2) # EDGECLEANING mP = mt_edge(smB,"prewitt",0,255,0,0,V=1,U=1) mS = mP.mt_expand(mode=mt_square(radius=2),U=1,V=1).mt_inflate(U=1,V=1) mD = mt_lutxy(mS,mP.mt_inflate(U=1,V=1),"x y - "+string(32)+" <= 0 x y - ?",U=1,V=1).mt_inflate(U=1,V=1).removegrain(20,-1) smE = mt_merge(smB,Eval("smB." + "Removegrain(2,0)"),mD,luma=true,U=3,V=3) # MASKING mE = mt_edge(smE,"prewitt",0,255,0,0,V=1,U=1).mt_lut(expr="x 1.8 ^",U=1,V=1).removegrain(4,-1).mt_inflate(U=1,V=1) mL = mt_logic(tmask.invert(),mE,"min",U=1,V=1).removegrain(20,-1) mF = mt_logic(tmask,mE,"max",U=1,V=1).removegrain(20,-1) # SHARPENING b1c = source.MCompensate(source_super,vb1) f1c = source.MCompensate(source_super,vf1) #Sclp = smE.LSFmod(defaults="slow", preblur="ON", strength=100) Sclp = ContraSharpen(smE,source) Tmax = source.mt_logic(f1c,"max",U=1,V=1).mt_logic(b1c,"max",U=1,V=1) Tmin = source.mt_logic(f1c,"min",U=1,V=1).mt_logic(b1c,"min",U=1,V=1) shrp = Sclp.mt_clamp(Tmax, Tmin, 2, 2, U=1, V=1) sL = mt_merge(smE,shrp,mL,U=2,V=2) # ENHANCING #GFc = sL.f3kdb(sample_mode=2,precision_mode=3) GFc = sL.GradFun2DBmod(thr=1.4,thrC=1.8,mode=2,str=0.8,strC=0.0,temp=50,adapt=64) Frs = mt_merge(GFc,sL,mF,luma=true,U=3,V=3).BlindDehalo3(rx=1.25, ry=1.25, strength=90, sharpness=1, ppmode=2) Frs#.mergechroma(den) YlevelsS(0,1.0,255,0,255,false) ### lock threads to cores ### prefetch: 16, 12 ### ### """) AssumeFrameBased Last edited by real.finder; 24th May 2013 at 18:23. |
25th May 2013, 01:03 | #78 | Link |
Registered User
Join Date: Sep 2012
Posts: 156
|
Well, thanks for this. Now I got it to work and going through the frames in AvspMod is very fast. However, encoding in x264 (with Simple Launcher GUI) first
hangs with 'potential deadlock', then nonetheless starts but very slowly and without using 100% cpu. Just sits at 40-50%. Memory meanwhile is 70%+. What's up with that? I did some investigating on my own and discovered that by removing 'lock threads to cores' statement I can get some speed back - 0,5fps and about 10% cpu load, but that's it. At least I'm glad that there was some little thing I missed and everything works for me. It's just not what I expected... |
25th May 2013, 02:45 | #79 | Link | |
Registered User
Join Date: Aug 2007
Posts: 218
|
Quote:
"lock threads to cores" is an advanced function, don't use it unless you really know how it works. Actually nearly no one told me that it is useful.
__________________
f3kdb 1.5.1 / MP_Pipeline 0.18 ffms2 builds with 10bit output hack: libav-9a60b1f / ffmpeg-1e4d049 / FFmbc-0.7.1 Built from ffms2 6e0d654 (hack a9fe004) Mirrors: http://bit.ly/19TwDD3 |
|
25th May 2013, 04:49 | #80 | Link |
Registered User
Join Date: Sep 2012
Posts: 156
|
Well, I don't know what to tell you guys. I've tried both with and without ### inherit start/end wrapping the variables, and with lowering/rising the setMemoryMax.
I'm only getting ~1 fps on a crf encode (once it finally starts to encode after the deadlock msg) on a script that used to give 1.6 fps with just a single SetMTMode(2)/SetMemoryMax(640) pair inserted in the middle. Now that doesn't work (x264 deadlocks eventually and cpu load doesn't achieve 100%) and this pipeline doesn't appear to increase any speed at all. If such set-up is just to make big scripts run with less load on cpu (haven't yet seen cpu at 100% with this) while increasing the memory load, it's fine. Currently I just need to script to run, period, don't really care how long it takes. But I was getting giddy under the (false?) impression that this set-up actually speeds things along? Was I wrong or still doing something incorrectly? I really, really appreciate all the replies and help. It's just frustrating. |
Tags |
avisynth, multi-process, pipeline |
|
|