Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
|
|
Thread Tools | Search this Thread | Display Modes |
1st December 2011, 04:20 | #1 | Link |
Registered User
Join Date: Aug 2007
Posts: 218
|
MP_Pipeline 0.18 - run parts of avisynth script in external processes [2014-04-06]
This plugin is originally written for my friend to work-around the [del]2GB[/del] 4GB problem of 32-bit process. (Well, also for fun. ) Don't know whether it is useful for others, but I decided to post it here anyways.
As of 0.11, overhead of the plugin is much smaller, it may be possible to use it to speed up more scripts. Change log: Code:
0.18 * Fix deadlock when exported clip is consumed by multiple script block 0.17 * Properly terminate slave processes when initialization fails * Fix "Not a clip" error when using ### inherit and the last block is empty 0.16 * Try to silent all error dialogs on exit of slave process * Slave process shouldn't be stuck on exit anymore, it will terminate itself if it doesn't exit cleanly after 15 seconds * Fix ### branch statement, previously it incorrectly rejects some input 0.15 * Properly clean script environment up on exit * Allow using different avisynth dll to run script block (### dll) 0.14 * Fixed another crashing bug 0.13 * Fixed a bug that causes occasional crashing 0.12 * Fixed a problem that makes scripts unable to be loaded in some programs 0.11 * Greatly improved performance, maximum 80% overhead reduction * New feature: Ability to lock threads to cores, may improve performance in some cases * (0.10 is skipped to avoid confusion) 0.9 * New feature: Frame prefetching * New feature: Exporting multiple clip variables in a single process * New feature: Code block can be shared between processes 0.3 * Binaries in the x86 folder are in correct version now (In 0.2 the win64 slave is actually win32...) * Integrated a patched TCPDeliver, no longer depend on the external one * Fixed random crash when filter chain is destroyed * Thunked branching 0.2 * x64 support (please copy TCPDeliver.dll in the package to respective plugin folder) * x86/x64 mixed slave process (requires both x86/x64 version of AviSynth to be installed) * Add a script variable in branch slave process, make it distinguishable in script * Since each process have its own script environment, all script variables and loaded plugins won't be inherited, they must be re-initialized if needed * Due to the limitation above, manually-loaded plugins and imported scripts need to be reloaded/re-imported before they can be used in new process (Or use inherited script snippet, please see MP_Pipeline_readme.avs for details) * Clips before MP_Pipeline will be ignored * Audio is not supported * Every script block must return a clip (i.e. "last" must be a clip), otherwise MPP will raise this error: Invalid arguments to function "MPP_PrepareDownstreamClip" Binary: http://nmm.me/z6 Source code: https://github.com/SAPikachu/MP_Pipeline/tree/0.18 Some example: 1. Basic usage: Code:
MP_Pipeline(""" FFVideoSource("SomeVideo") QTGMC() ### prefetch: 16, 0 ### ### """) MCTD() # MCTD and QTGMC will be run parallelly in 2 separate processes Code:
# Must be 64bit system with at least 8GB memory to run this script MP_Pipeline(""" # This may be smaller, but I only tested this number SetMemoryMax(3072) FFVideoSource("SomeVideo") MCTD(settings="high") ### prefetch: 16, 0 ### ### """) # Some time ago I used a script similar to this one for encoding, it is about 20% ~ 30% faster than plain MCTD. Code:
MP_Pipeline(""" FFVideoSource("SomeVideo") TNLMeans() ### prefetch: 16, 0 ### branch: 4 ### ### """) # TNLMeans will be run in 4 processes with branching (please see example script in the package for details) Code:
MP_Pipeline(""" FFVideoSource("SomeUnseekableVideo", seekmode=-1) TNLMeans() ### prefetch: 32, 24 # It is important to use a big backward cache since we can't seek ### ### MCTD() """)
__________________
f3kdb 1.5.1 / MP_Pipeline 0.18 ffms2 builds with 10bit output hack: libav-9a60b1f / ffmpeg-1e4d049 / FFmbc-0.7.1 Built from ffms2 6e0d654 (hack a9fe004) Mirrors: http://bit.ly/19TwDD3 Last edited by SAPikachu; 6th April 2014 at 10:57. |
Tags |
avisynth, multi-process, pipeline |
Thread Tools | Search this Thread |
Display Modes | |
|
|