Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 29th November 2005, 12:23   #141  |  Link
Badness
Registered User
 
Join Date: Mar 2003
Posts: 18
Any further luck using MT on Dual Xeon's encoding XviD?
__________________
Badness
Badness is offline   Reply With Quote
Old 1st December 2005, 17:37   #142  |  Link
tsp
Registered User
 
tsp's Avatar
 
Join Date: Aug 2004
Location: Denmark
Posts: 807
Badness: It seems to depend on the script you plane to use. Some (Koroshiya) did experience faster encoder while others (lcksg) had slower encoding.
__________________
Get my avisynth filters @ http://www.avisynth.org/tsp/
tsp is offline   Reply With Quote
Old 2nd December 2005, 20:01   #143  |  Link
mg262
Clouded
 
mg262's Avatar
 
Join Date: Jul 2003
Location: Cambridge, UK
Posts: 1,148
tsp,

Can I trouble you with another abstract question? I'm planning a filter that needs intermediary working space (i.e. a big array or VideoFrame) in each GetFrame call. If two GetFrames are called simultaneously, they each need their own working space, *not* shared. So far so good, except that I don't want to allocate/deallocate every GetFrame call, since there are no guarantees on the allocation speed.

Apart from the intermediate space, everything is fully parallel-friendly, so I would like to find a similarly parallel-friendly way of providing intermediate space. The best way I can think of to deal with this is to have a pool of appropriately sized arrays -- i.e., effectively to build a custom allocator that is called in every GetFrame... but then the allocator might be called twice simultaneously. Any advice on a clean way to go? Should I try and put some kind of lock on the allocator,* or should I just leave it for MT(mode = 2) to deal with or is there a simpler solution?

*which will presumably affect portability.

It's going to be a while before I try building something like this, so there really is no hurry to answer the question. (It's not terribly important anyway... but I would much rather build things to be parallel-friendly where possible.)

Thanks,
M.
__________________
a.k.a. Clouded. Come and help by making sure your favourite AVISynth filters and scripts are listed.

Last edited by mg262; 2nd December 2005 at 20:09.
mg262 is offline   Reply With Quote
Old 2nd December 2005, 22:18   #144  |  Link
AssassiNBG
Registered User
 
Join Date: Mar 2005
Location: Pleven, Bulgaria, Europe.
Posts: 45
OMG I'm so tired, I can't think straight. Someone tell me why this syntax doesn't work please ?

Quote:
MT("ffdshow("default")")
It says it expected a "," or a ")". Hmm... don't I have them ?
AssassiNBG is offline   Reply With Quote
Old 2nd December 2005, 22:29   #145  |  Link
tsp
Registered User
 
tsp's Avatar
 
Join Date: Aug 2004
Location: Denmark
Posts: 807
mg262:

with mode=2 there will never be two GetFrame calls to the same filter instance as there are as many filter instances as there are threads. So it should be safe to allocate the working space in the constructor and release it in the destructor.
If you wanted to share some variables between the different instances of your filter you would need to use the PClipLocalStorage in avisynth 2.6.

Quote:
Originally Posted by AssassiNBG
OMG I'm so tired, I can't think straight. Someone tell me why this syntax doesn't work please ?

MT("ffdshow("default")")

It says it expected a "," or a ")". Hmm... don't I have them ?
try this
Code:
 MT(""" ffdshow("default") """)
__________________
Get my avisynth filters @ http://www.avisynth.org/tsp/

Last edited by tsp; 2nd December 2005 at 23:15.
tsp is offline   Reply With Quote
Old 2nd December 2005, 22:59   #146  |  Link
AssassiNBG
Registered User
 
Join Date: Mar 2005
Location: Pleven, Bulgaria, Europe.
Posts: 45
Yay it worked! Thankz!

PS Although it didn't do any changes. Speed is exactly the same. Second core doesn't seem to work according to EVEREST (benchmark app). I've got a P4 HT @ 3000 GHz.

Last edited by AssassiNBG; 2nd December 2005 at 23:05.
AssassiNBG is offline   Reply With Quote
Old 3rd December 2005, 03:55   #147  |  Link
Revgen
Registered User
 
Join Date: Sep 2004
Location: Near LA, California, USA
Posts: 1,545
hmm...interesting.

I used to use this in order to get it to work.


d="default"

MT("ffdshow(d)")
__________________
Pirate: Now how would you like to die? Would you like to have your head chopped off or be burned at the stake?

Curly: Burned at the stake!

Moe: Why?

Curly: A hot steak is always better than a cold chop.
Revgen is offline   Reply With Quote
Old 3rd December 2005, 08:22   #148  |  Link
AssassiNBG
Registered User
 
Join Date: Mar 2005
Location: Pleven, Bulgaria, Europe.
Posts: 45
Still not doing anything.
AssassiNBG is offline   Reply With Quote
Old 3rd December 2005, 11:16   #149  |  Link
tsp
Registered User
 
tsp's Avatar
 
Join Date: Aug 2004
Location: Denmark
Posts: 807
try seing how many cores windows task manager shows if the total cpu utilization is already 100% before running MT() when you will not gain much by running it.
__________________
Get my avisynth filters @ http://www.avisynth.org/tsp/
tsp is offline   Reply With Quote
Old 3rd December 2005, 11:35   #150  |  Link
AssassiNBG
Registered User
 
Join Date: Mar 2005
Location: Pleven, Bulgaria, Europe.
Posts: 45
It is not. Both are going not more than 50%. And a benchmark app shows only the second core is actualing processing, the first one is idling.
AssassiNBG is offline   Reply With Quote
Old 3rd December 2005, 14:47   #151  |  Link
tsp
Registered User
 
tsp's Avatar
 
Join Date: Aug 2004
Location: Denmark
Posts: 807
it might be because ffdshow avisynth filter doesn't support multithreading. I will try to figure out.
__________________
Get my avisynth filters @ http://www.avisynth.org/tsp/
tsp is offline   Reply With Quote
Old 3rd December 2005, 14:58   #152  |  Link
mg262
Clouded
 
mg262's Avatar
 
Join Date: Jul 2003
Location: Cambridge, UK
Posts: 1,148
Quote:
Originally Posted by tsp
If you wanted to share some variables between the different instances of your filter you would need to use the PClipLocalStorage in avisynth 2.6.
Thank you!
__________________
a.k.a. Clouded. Come and help by making sure your favourite AVISynth filters and scripts are listed.
mg262 is offline   Reply With Quote
Old 5th December 2005, 20:55   #153  |  Link
tsp
Registered User
 
tsp's Avatar
 
Join Date: Aug 2004
Location: Denmark
Posts: 807
Here is an exsample on how the PClipLocalStorage can be used to share a cache between multiple instances(that are created with mode=2,4):
Code:
class Cache
{
public:
  //These function should be threadsafe. The most simple way is to use a 
  //critical section like this
  PVideoFrame GetCachedFrame(int framenumber)
  {
  EnterCriticalSection(&cs);
  //Code
  //...
  
  
  LeaveCriticalSection(&cs);
  return retval;
  }
  SetCachedFrame(PVideoFrame frame);
private:
  CRITICAL_SECTION cs;
}



class Sample : public GenericVideoFilter{
  public:
  Sample(PClip _child, IScriptEnvironment* env);
  ~Sample();
  PVideoFrame __stdcall GetFrame(int n, IScriptEnvironment* env);
  protected:
  PClipLocalStorage cls;
  Cache* FrameCache;
}

Sample::Sample(PClip _child, IScriptEnvironment* env) :
GenericVideoFilter(_child),cls(env)
{
//if the cache has not been created yet GetValue will return 0
if(cls->GetValue()==0) {
  //create the cache and save the address in the PClipLocalStorage
  FrameCache = new Cache();
  cls->SetValue(static_cast<void*>(FrameCache));
  }
// The cache has been created so assign the address to FrameCache
else  {
  FrameCache=static_cast<cache*>(cls->GetValue());
  }  
}

Sample::~Sample()
{
//only delete FrameCache if it is not delete yet.
if(cls->GetValue()!=0)  {
  delete FrameCache;
  cls->SetValue(0);//Signal that the cache is deleted
  }
}
__________________
Get my avisynth filters @ http://www.avisynth.org/tsp/
tsp is offline   Reply With Quote
Old 6th December 2005, 02:45   #154  |  Link
shpitz
Registered User
 
Join Date: Sep 2004
Posts: 113
Quote:
Originally Posted by lcksg
Script same as above but something is not right with the encoding times & CPU usage

MT mode | time mm:ss | CPU usage in Taskmgr

HT-on
MT-off | 1:35 | 45% CPU
MT-20 | 2:16 | 92% CPU
MT-22 | 1:46 | 67-70% CPU
MT-40 | 3:58 | 36%-41% CPU
MT-42 | 2:12 | 48%-54% CPU

HT-off
MT-off | 1:29 | 87-90% CPU
MT-20 | 2:00 | 88%-100% CPU
MT-40 | 2:26 | 82%-89% CPU

the weirdest thing...

i have the same problem:

if i don't set mtmode in the script, it encodes at a certain speed.

if i use any of the mtmodes, cpu will go to 100% but encoding speed is reduced to half in some of the mt modes !

so from the testing i've done it appears that using mt mode actually worsens instead of speed up which really doesn't make any sense.

i've placed the avisynth with mt-included dll from TSP's website in system32.

my spec is dual xeon 3.2ghz with 1gb ram.

my script is as follows:

Code:
#  PLUGINS
LoadPlugin("D:\TBS\Filters\Decomb521.dll")
LoadPlugin("D:\TBS\Filters\DGDecode.dll")
LoadPlugin("D:\TBS\Filters\UnDot.dll")
LoadPlugin("D:\TBS\Filters\MSharpen.dll")
LoadPlugin("D:\TBS\Filters\RemoveGrainSSE3.dll")

SetMTMode(2,2)

mpeg2source("F:\hd_capture.d2v",idct=3)
SelectEven()

trim(5017, 6815)

Crop(162,4,-164,-4)

undot()

BicubicResize(512,384,0,0.75)
am i missing something?
shpitz is offline   Reply With Quote
Old 6th December 2005, 11:29   #155  |  Link
tsp
Registered User
 
tsp's Avatar
 
Join Date: Aug 2004
Location: Denmark
Posts: 807
shpitz: No not really. Might try SetMTMode(2) instead but I don't think it will improve it. You could try to but some of the more cpu demanding filters in like limitedSharpen or vaguedenoise because it seems to be very dependent on what script is used. Rather strange with these dual Xeon. Could anyone with an Athlon X2 test if they got the same problem with the above script?
__________________
Get my avisynth filters @ http://www.avisynth.org/tsp/
tsp is offline   Reply With Quote
Old 6th December 2005, 14:54   #156  |  Link
Socio
Registered User
 
Join Date: May 2004
Posts: 291
Quote:
Originally Posted by shpitz
the weirdest thing...

i

my script is as follows:

Code:
#  PLUGINS
LoadPlugin("D:\TBS\Filters\Decomb521.dll")
LoadPlugin("D:\TBS\Filters\DGDecode.dll")
LoadPlugin("D:\TBS\Filters\UnDot.dll")
LoadPlugin("D:\TBS\Filters\MSharpen.dll")
LoadPlugin("D:\TBS\Filters\RemoveGrainSSE3.dll")

SetMTMode(2,2)

mpeg2source("F:\hd_capture.d2v",idct=3)
SelectEven()

trim(5017, 6815)

Crop(162,4,-164,-4)

undot()

BicubicResize(512,384,0,0.75)
am i missing something?
Don't you have to load the MT.dll as well?
Socio is offline   Reply With Quote
Old 6th December 2005, 14:57   #157  |  Link
shpitz
Registered User
 
Join Date: Sep 2004
Posts: 113
thanks for the reply TSP.

the really weird thing about all this thing is the fact that the cpu usage goes up while the encoding speed goes down. it's like it is going up-hill, more power is needed yet the climbing rate is getting slower and slower, it is really bizarre.

what really gets me is the fact that during an encode, cpu usage is around 40%, no more, no less. and this gives me the feeling the pc is not used to its whole potential...

i could run 2 instances of vdub and encode 2 clips at the same time, but still the encoding speed of 1 will affect the other, so 2 instances doesn't really mean double the speed.

so all i want is to maximize the speed of 1 single encoding session.
shpitz is offline   Reply With Quote
Old 6th December 2005, 14:57   #158  |  Link
tsp
Registered User
 
tsp's Avatar
 
Join Date: Aug 2004
Location: Denmark
Posts: 807
not if he only uses SetMTMode because that is included in the modified avisynth.dll
__________________
Get my avisynth filters @ http://www.avisynth.org/tsp/
tsp is offline   Reply With Quote
Old 6th December 2005, 14:59   #159  |  Link
shpitz
Registered User
 
Join Date: Sep 2004
Posts: 113
Quote:
Originally Posted by tsp
not if he only uses SetMTMode because that is included in the modified avisynth.dll
that's kinda confusing TSP... what does setmtmode do if i don't use mt.dll on filters in the script? what does each one of them do? how does it work?

thanks
shpitz is offline   Reply With Quote
Old 6th December 2005, 15:13   #160  |  Link
tsp
Registered User
 
tsp's Avatar
 
Join Date: Aug 2004
Location: Denmark
Posts: 807
well all the information about the modified avisynth.dll is scattered around in this thread. Some day I really should sit down and write some proper documentation for this.
Basicly SetmtMode change the script parser and the internal cache so that more than one frame can be processed at a time. To do this I had to modify avisynth so that is the reason why it's not included in mt.dll.
Mt on the other hand split the frame up in smaller pieces and run each piece on a different thread/processor. This can be done with a plugin allthough it works best with the modfied dll.
For more information try reading the whole thread (or just my post ).
The strange thing as you said is that it runs slower while using 100% cpu time. There are of cource an overhead by running avisynth multithreaded but not that high. Maybe running all these instances pollutes the cache? Does it makes any difference to disable HT?
__________________
Get my avisynth filters @ http://www.avisynth.org/tsp/
tsp is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 07:12.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.