Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
8th August 2008, 11:46 | #941 | Link |
Registered User
Join Date: Feb 2003
Location: Russia, Moscow
Posts: 854
|
josey_wells!
Good work. I find even MVTools.dll from folder MSVC08 work little faster, but I can not run IC10 version. At home I have old CPU Number of processors in system: 1 Current processor: #1 Cores per processor: 1 Disabled cores per processor: 0 Processor Name: Intel(R) Pentium(R) 4 CPU 2.53GHz Type: 0 Family: F Model: 2 Stepping: 7 Revision: 37 Maximum CPUID Level: 2 L1 Trace Cache: 12 Kµops L1 Data Cache: 8 KБ L2 Cache: 512 KБ Packaging: FC-PGA2 EIST: No MMX(TM): Yes SSE: Yes SSE2: Yes SSE3: No SSE4: No Enhanced Halt State: No Execute Disable Bit: No Hyper-Threading Technology: No Intel(R) 64 Architecture: No Intel(R) Virtualization Technology: No Expected Processor Frequency: 2.53 ГГц Reported Processor Frequency: 2.53 ГГц Expected System Bus Frequency: 533 МГц Reported System Bus Frequency: 533 МГц My CPU support SSE2 but when I open avs file in VirtualDub, VirtualDub close without error report. Please advice. yup. |
8th August 2008, 11:50 | #942 | Link |
Registered User
Join Date: Dec 2007
Location: VA, USA
Posts: 116
|
Tschniede is basically correct.
In short, MVDegrain1, MVDegrain2, & MVDegrain3 were all changed to be multithreaded for the LumaY, ChromaU, and ChromaV processing where the threads to do the processing are created in the constructor and destroyed in the destructor. However, all frame getting operations are done on the mainline thread so in general little speed improvement will be achieved as the MVAnalyse calls will all be run on the main line thread only. A new function MVAnalyseMulti was added which creates a seperate thread to do the analysing for each forward or backward frame. The threads are only created in the constructor and destroyed in the desctructor. The output is packed into an arrayed output only compatible at this time with MVDegrainMulti but other functions can easily be adopted to handle this arrayed format. MVAnalyseMulti is the same as MVAnalyse with the following changes: 1. isb and delta were removed 2. new parameters refframes and deltamult were added refframes - is the number of forward and backward reference frames to analyse and can be set between 1-32. Default is 1. 1 would be 2 frames, 2 would be 4 frames, etc. deltamult - is the multiplier used for the frame stepping. If deltamult is 1 then the offsets are 1, 2, 3. If deltamult is 2 then the offsets are 2, 4, 6, etc. Default is 1 and can be set between 1-32. For example: Code:
backward_vec2 = source.MVAnalyse(isb = true, lambda = 1000, delta = 2) backward_vec1 = source.MVAnalyse(isb = true, lambda = 1000, delta = 1) forward_vec1 = source.MVAnalyse(isb = false, lambda = 1000, delta = 1) forward_vec2 = source.MVAnalyse(isb = false, lambda = 1000, delta = 2) Code:
ForBackVec = source.MVAnalyseMulti(refframes=2, lambda = 1000) A new MVDegrainMulti was added which still does the multithreaded Luma, ChromaU, and ChromaV processing and has the following syntax: Code:
MVDegrainMulti(clip, clip "MVForBack", int "refframes", int "thSAD", int "thSADC", int "plane", int "limit", clip "pelclip", int "idx") refframes - the number of reference frames used in MVAnalyseMulti to produce MVForBack. Default is 1. These two calls allow all MVDegrain calls equivalent to MVDegrain1 up to MVDegrain32. For example a MVDegrain16 would be Code:
MVForBack = MVAnalyseMulti(refframes=16, pel = 2, overlap=4, sharp=1, idx = 1) last.MVDegrainMulti(MVForBack, refframes=16, thSAD=400,idx=1) I fixed a memory leak in the desctructor of MVMask and MVShow. Many other numerous small changes. These calls can be used with the latest AviSynth V2.5.8 RC3 to due multithreaded processing or they can be combined with some of the MT or SetMTMode variants if desired. |
8th August 2008, 12:18 | #943 | Link | |
Registered User
Join Date: Dec 2007
Location: VA, USA
Posts: 116
|
Quote:
Here are two versions one built for P4 with SSE2 and one built for Core2 Duo with SSE2 & SSE3 http://rapidshare.com/files/13578584...uilds.rar.html FYI: Do not use the Core2 Duo IC10 build. The intel compiler has an issue when I tested this build. Use the P4-SSE3 build which was the original IC10 build posted with the final, or the P4-SSE2 build posted here. Or use the MSVC08 build which should work for anyone. Which is fastest is trial and error Last edited by josey_wells; 8th August 2008 at 14:14. Reason: Warning |
|
8th August 2008, 13:23 | #945 | Link |
x264aholic
Join Date: Jul 2007
Location: New York
Posts: 1,752
|
I've just finished running a test encode (Planet Earth) and the whole thing went through without a problem. I used the IC10 version, and got roughly ~1.36 fps using two computers, hah. This is far better than what I got on one computer and more modest x264 settings for that matter.
__________________
You can't call your encoding speed slow until you start measuring in seconds per frame. |
8th August 2008, 17:02 | #946 | Link | |
Avisynth language lover
Join Date: Dec 2007
Location: Spain
Posts: 3,431
|
Quote:
|
|
8th August 2008, 17:12 | #947 | Link |
Pig on the wing
Join Date: Mar 2002
Location: Finland
Posts: 5,718
|
That doesn't work in situations where you use the same vectorset for two MVDegrainMultis with different temporal radii.. One solution would be to use the value used in MVAnalyseMulti unless specified otherwise.
__________________
And if the band you're in starts playing different tunes I'll see you on the dark side of the Moon... |
8th August 2008, 17:38 | #949 | Link |
Registered User
Join Date: Dec 2007
Location: VA, USA
Posts: 116
|
Actually both of you bring up a good point. I do not check now for a differing refframes than the original set created from MVAnalyseMulti.
I could see where one would want to create an MVAnalyseMulti set with say refframes=3 but only use temporal radius 2, i.e. refframes=2 in MVDegrainMulti. I will have to make these fixes. On second though I could default refframes to 32 and if it is larger than the set passed then use the entire set from MVAnalyseMulti. If a smaller refframes was specified then the subset would be utilized. Last edited by josey_wells; 8th August 2008 at 17:53. Reason: added comment |
8th August 2008, 17:56 | #950 | Link | |
Pig on the wing
Join Date: Mar 2002
Location: Finland
Posts: 5,718
|
Quote:
__________________
And if the band you're in starts playing different tunes I'll see you on the dark side of the Moon... |
|
8th August 2008, 18:00 | #951 | Link | |
Registered User
Join Date: Dec 2007
Location: VA, USA
Posts: 116
|
Quote:
I propose to change so that refframes is specified in MVAnalyseMulti and if unspecified in MVDegrainMulti then will be the same as used MVAnalyseMulti, i.e. entire set. If specified and smaller thatn refframes used in MVAnalyseMulti then the subset of smaller temporal radius would be used. |
|
8th August 2008, 19:55 | #953 | Link |
AviSynth plugger
Join Date: Nov 2003
Location: Russia
Posts: 2,183
|
josey_wells,
You make a real progress with new functions MVDegrainMulti and MVAnalyseMulti! It is really faster! With new Multi syntax, the mvtools lose some ...universality (do not know right word). But speed is more important! However, the clear doc is very important too. If your version is a final, what is state of other functions? is it possible to mix multi and non-multi mvanalyse with other functions, is it possible to have several MVDegrainMulti in script, may idx be same. is some functions compatible with SetMTMode or MT()? As I understand, ...Multi functions is now free from thead locking. Other (non-multi) functions (MVFlowFPS, etc) is still not, right ? In this case, we should use for these (other) functions the v.1.9.5.7 threading implementation (it works fine for most people besides Mutant_Fred). Is it possible to revert all your threading changes for old functions and combine it with new MVDegrainMulti and MVAnalyseMulti? IMO, it would be the optimal way of MVTools evolution. If it is possible, then your version is not final. If it is not possible, then your version is not final too as for version number, v1.10 is seems most appropriate.
__________________
My Avisynth plugins are now at http://avisynth.org.ru and mirror at http://avisynth.nl/users/fizick I usually do not provide a technical support in private messages. |
9th August 2008, 00:27 | #954 | Link |
dumber every day
Join Date: Dec 2006
Location: Planet Earth
Posts: 154
|
@josey_wells,
First awesome work, thanks for sticking with it! I have not been able to get it to crash or freeze Now of course I have a question. Are the idx values between MVAnalyseMulti and MVAnaylse completely separate or do they refer to the same pool. For example if I did a forback=input.MVAnalyseMulti(refframes=2, idx=1) would a for=input.mvanalyse(isb=false,delta=1,idx=1) be able to retrieve the vector frame that MVAnalyseMulti already computed or does it recompute the frame since its in a different function? If different is there a way to get specific vector frames out of the MVAnalyseMultii array? |
9th August 2008, 10:40 | #955 | Link | |
Registered User
Join Date: Sep 2004
Location: Near LA, California, USA
Posts: 1,545
|
Quote:
I'm not exactly skilled with MVtools and I'm trying to modify MCBob to work with this. So far the obstacle is trying get MVFlowInter to use this.
__________________
Pirate: Now how would you like to die? Would you like to have your head chopped off or be burned at the stake? Curly: Burned at the stake! Moe: Why? Curly: A hot steak is always better than a cold chop. |
|
9th August 2008, 13:32 | #957 | Link | |
likes to tinker
Join Date: Jan 2004
Location: girt by sea
Posts: 635
|
Quote:
|
|
9th August 2008, 14:05 | #958 | Link | ||||
Registered User
Join Date: Dec 2007
Location: VA, USA
Posts: 116
|
Wow, lots of questions. Let me address the questions in order and then wrap up with talk about a new version that I believe will make this applicable to everyone until more functions can be modified.
Quote:
Quote:
2. It is possible to have multiple MVDegrainMulti and multiple MVAnalyseMulti in the same script. The idx values may be different or the same and work as before. 3. All new functions should be just as compativle with SetMTMode or MT() and stability using these functions would be the same as before. For higher stability using the newer AviSynth without SetMTMode or MT is recommended. Quote:
Quote:
I spent most of the day yesterday tracing an excessive memory usage condition and making some more enhancements. The memory usage problem turned out to be that if you pass pointers which increment a refcount to a thread which doesn't die between frame calls then the thread will hold references to frames no longer needed between calls and substantially increase memory usage. I have corrected by freeing pointers in the threads. Now for the good stuff. First it is now no longer necessary to add refframes to MVDegrainMulti. It will automatically know how many temporal frames were passed to it and perform the appropriate denoising. For example if MVAnalyseMulti had refframes=3 then MVDegrainMulti would be equivalent to MVDegrain3. If a smaller refframes is passed then a reduced temporal denoising can be done. If refframes=2 was specified in MVDegrainMulti then a MVDegrain2 will be performed even though MVAnalyseMulti was refframes=3. Code:
MVMulti=c.MVAnalyseMulti(refframes=iLevel, pel=iPel, blksize=iBlksize, overlap=iOverlap, chroma=bChroma, truemotion=bTrueMotion, search=iSearch, sharp=iSharp, dct=iDct, idx=iIdx1) last.MVDegrainMulti(MVMulti, thSAD=iTHSAD, idx=iIdx1) MVAnalyseMulti with refframes 3 will produce the following frames corresponding to the array elements. 0 - Backward 3, 1 - Backward 2, 2 - Backward 1, 3 - Forward 1, 4 - Forward 2, 5 - Forward 3 And so on for higher refframes I have added a MVMultiExtract Function which allows one to extract the regular MVAnalyse frames from a multi function call to be used in other MVTools functions until they can be modified to accept MVAnalyse directly. Thus everyone can take advantage of MVAnalyseMulti which seems to take the most processing power. For example. Code:
MVMulti=c.MVAnalyseMulti(refframes=1, pel=iPel, blksize=iBlksize, overlap=iOverlap, chroma=bChroma, truemotion=bTrueMotion, search=iSearch, sharp=iSharp, dct=iDct, idx=iIdx1) back1=MVMulti.MVMultiExtract(0) for1=MVMulti.MVMultiExtract(1) last.MVDegrain1(back1, for1, thSAD=iTHSAD, idx=iIdx1) So finally here it is V1.9.6.1: http://rapidshare.com/files/13604247...9.6.1.rar.html There are several versions in here use the one most appropriate for your system. MSVC08 - Everyone IC10-P4-SSE2 - Pentium 4 or above SSE2 IC10-P4-SSE3 - Pentium 4 or above with SSE3 IC10-Core2-SSE3 - Core 2 duo or above with SSE3 Now that the oylmpics are on I need a break, plus my DVD collection is backing up. Have fun! Last edited by josey_wells; 9th August 2008 at 15:22. |
||||
9th August 2008, 14:37 | #959 | Link |
Registered User
Join Date: Oct 2003
Location: Germany
Posts: 377
|
You made big improvements to the MVTools. For future development maybe it is intresting to note the development in avisynth:
http://forum.doom9.org/showthread.php?t=139629 So if i understand the result of the idea of IanB right it should make just the same as you have realized with the MVAnalyseMulti-Function. So if this will be reality in near future such functions should't be usefull then? |
9th August 2008, 15:00 | #960 | Link | |
Registered User
Join Date: Dec 2007
Location: VA, USA
Posts: 116
|
Quote:
The new AviSynth is looking to multithread individual frames, i.e. running the script in parallel for different frame numbers. There exists lots of issues with this with scripts that depend on previous/forward frames. AviSynth is the place to perform this since it has control of the frame getting mechanism which is why I had to go the way I did. When AviSynth makes these modifications as well as pipeline improvements both modifications can exist together and benifit. |
|
Thread Tools | Search this Thread |
Display Modes | |
|
|