Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 8th November 2007, 13:13   #701  |  Link
badshah
Registered User
 
badshah's Avatar
 
Join Date: Jan 2007
Posts: 42
contents of plugins folder .............

badshah is offline   Reply With Quote
Old 8th November 2007, 18:44   #702  |  Link
badshah
Registered User
 
badshah's Avatar
 
Join Date: Jan 2007
Posts: 42
I don't know what happened.... its working fine now ... but speed is very slow with setmtmode(2)

without that line i am getting 20fps ... while with that line speed is 2 to 4 fps
badshah is offline   Reply With Quote
Old 8th November 2007, 21:13   #703  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 4,680
I've tried using HC with SetMTMode. It seems that there are no extra threads being used at all! The number of threads is the same even if SetMTMode(2) is called before the source is loaded. When I do the same in VDubMod, the number of threads is higher with SetMTMode.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 8th November 2007, 22:19   #704  |  Link
mroz
Registered User
 
mroz's Avatar
 
Join Date: Sep 2006
Posts: 201
@badshah: given SetMTMode is meant to be the first command in a script, is it safe to have LimitedSharpenFaster.avsi autoloading from plugins? I don't know if this makes any difference; just mentioning in case.

@Boulder: Try adding as the last line to your script
Distributor()
before giving the script to HC. This might be the same issue as with avs input to MEncoder & to Megui's Analysis pass.
mroz is offline   Reply With Quote
Old 9th November 2007, 12:33   #705  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 4,680
Quote:
Originally Posted by mroz View Post
@Boulder: Try adding as the last line to your script
Distributor()
before giving the script to HC. This might be the same issue as with avs input to MEncoder & to Megui's Analysis pass.
Yep, this works. Too bad that using SetMTMode seems to kill performance, going from 6fps down to 3fps
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 9th November 2007, 13:40   #706  |  Link
mroz
Registered User
 
mroz's Avatar
 
Join Date: Sep 2006
Posts: 201
Quote:
Originally Posted by Boulder View Post
Yep, this works. Too bad that using SetMTMode seems to kill performance, going from 6fps down to 3fps
Just with HC or with VDub as well? What's the script?
mroz is offline   Reply With Quote
Old 9th November 2007, 16:14   #707  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 4,680
I only tried HC.

The script is:
Code:
SetMTMode(2)
MPEG2Source("path\clip.d2v",cpu=4)
Crop(12,78,-12,-78,true)
den=DegrainMedian(mode=2)
DegrainFFTC(last,den,sad=200,ol=4)
AddBorders(16,78,8,78)
Distributor()
And the function DegrainFFTC is in degrain.avsi which contains a few functions similar to each other. The function itself is:
Code:
global idx_counter = 10
global idx_counter_2 = 50

function DegrainFFTC( clip c, clip cleaned, int "blk", int "ol", int "sh", int "sad", int "pl", int "div", int "limy", int "limuv" )
{
global idx_counter = idx_counter + 1
global idx_counter_2 = idx_counter_2 + 1
blk = default( blk, 16 )
ol = default( ol, 0 )
sh = default( sh, 2 )
sad = default( sad, 200 )
pl = default( pl, 4 )
div = default( div, 0 )
limuv = default( limuv, 255 )
vbw1=MVAnalyse(cleaned,isb=true,truemotion=true,delta=1,pel=2,chroma=true,blksize=blk,idx=idx_counter,sharp=sh,overlap=ol,divide=div)
vfw1=MVAnalyse(cleaned,isb=false,truemotion=true,delta=1,pel=2,chroma=true,blksize=blk,idx=idx_counter,sharp=sh,overlap=ol,divide=div)
vbw2=MVAnalyse(cleaned,isb=true,truemotion=true,delta=2,pel=2,chroma=true,blksize=blk,idx=idx_counter,sharp=sh,overlap=ol,divide=div)
vfw2=MVAnalyse(cleaned,isb=false,truemotion=true,delta=2,pel=2,chroma=true,blksize=blk,idx=idx_counter,sharp=sh,overlap=ol,divide=div)
nolimit = MVDegrain2(c,vbw1,vfw1,vbw2,vfw2,thSAD=sad,idx=idx_counter_2,plane=pl)
defined(limy) ? LimitChange(MVDegrain2(c,vbw1,vfw1,vbw2,vfw2,thSAD=sad,idx=idx_counter_2,plane=pl),c,limy,limuv) : nolimit
}
The CPU usage goes to 90-99% but speed drops down seriously.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 9th November 2007, 22:29   #708  |  Link
IanB
Avisynth Developer
 
Join Date: Jan 2003
Location: Melbourne, Australia
Posts: 3,173
HC requests frames slightly out of order, if this makes it thru to MPEG2Source worst case you can end up re-decoding each source GOP every time instead of just decoding the next delta frame.

Add a SetMemoryMax(big??) to increase the available cache buffers.

Also try abusing ChangeFPS(Last, Last, True). Add it after your Distributor call, it will act as a 10 frame EnsureVBRMP3Sync for video.
IanB is offline   Reply With Quote
Old 10th November 2007, 05:21   #709  |  Link
badshah
Registered User
 
badshah's Avatar
 
Join Date: Jan 2007
Posts: 42
Quote:
Originally Posted by mroz View Post
@badshah: given SetMTMode is meant to be the first command in a script, is it safe to have LimitedSharpenFaster.avsi autoloading from plugins? I don't know if this makes any difference; just mentioning in case.
I did import of LSF in begining lines & its running better now. but still encoding speed is 60% of total CPU capacity .... how to increase the speed further
badshah is offline   Reply With Quote
Old 10th November 2007, 09:55   #710  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 4,680
Quote:
Originally Posted by IanB View Post
HC requests frames slightly out of order, if this makes it thru to MPEG2Source worst case you can end up re-decoding each source GOP every time instead of just decoding the next delta frame.

Add a SetMemoryMax(big??) to increase the available cache buffers.

Also try abusing ChangeFPS(Last, Last, True). Add it after your Distributor call, it will act as a 10 frame EnsureVBRMP3Sync for video.
I have SetMemoryMax(512) as an avsi file so that should suffice I think. I'll try the ChangeFPS trick later and report back.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 11th November 2007, 00:38   #711  |  Link
mroz
Registered User
 
mroz's Avatar
 
Join Date: Sep 2006
Posts: 201
Quote:
Originally Posted by badshah View Post
I did import of LSF in begining lines & its running better now. but still encoding speed is 60% of total CPU capacity .... how to increase the speed further
The script you quoted isn't very complex so maybe the bottleneck is the encoder. What is it? x264 will get to about 80% on a first pass & around 100% on a second, however ime xvid only manages about 60% - increasing the number of threads it uses will up cpu usage, but actually drops frame rate; it doesn't seem optimised for multithreading (*) (this is on a Q6600). Of course I might simple have it misconfigured (though I was using default profiles in Megui).

(*) By which I mean while it will run multithreaded with improved performance, there does seem to be this limitation preventing full cpu exploitation.

Last edited by mroz; 11th November 2007 at 20:14.
mroz is offline   Reply With Quote
Old 11th November 2007, 10:09   #712  |  Link
badshah
Registered User
 
badshah's Avatar
 
Join Date: Jan 2007
Posts: 42
^^ its Xvid ... and exactly, it doesnt go above 60 to 70 %. anyway, its fine.
badshah is offline   Reply With Quote
Old 11th November 2007, 15:12   #713  |  Link
mroz
Registered User
 
mroz's Avatar
 
Join Date: Sep 2006
Posts: 201
You can always up efficiency manually on a per job basis, by splitting the encode in two & running both at the same time (in separate workers, if you're using Megui), then join the outputs when done.

It's not something I've bothered with due to the hassle & the facts I do little xvid encoding & can usually find another use for those cpu cycles.
mroz is offline   Reply With Quote
Old 11th November 2007, 16:49   #714  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 4,680
Quote:
Originally Posted by Boulder View Post
I have SetMemoryMax(512) as an avsi file so that should suffice I think. I'll try the ChangeFPS trick later and report back.
The ChangeFPS trick seemed to fix things. The performance is better with SetMTMode than without it when encoding with HC.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 11th November 2007, 21:49   #715  |  Link
IanB
Avisynth Developer
 
Join Date: Jan 2003
Location: Melbourne, Australia
Posts: 3,173
I have plans for a "LinearAccess" filter for 2.6, I'll bump it up the todo list a bit.
IanB is offline   Reply With Quote
Old 13th November 2007, 04:00   #716  |  Link
vcmohan
Registered User
 
Join Date: Jul 2003
Location: India
Posts: 735
Can MT convey plugin computed params to both threads?

In some plugins(of mine ) process parameters are derived from analysis of a particular field (or frame) of the clip and applied all through. In case of MT in the mode where alternate fields are processed in different threads whether the params will be computed in both threads or is there a control on the place from which forking takes place?

If this querry does not belong to this thread pl bump it to a new thread.
__________________
mohan
my plugins are now hosted here
vcmohan is offline   Reply With Quote
Old 13th November 2007, 12:29   #717  |  Link
mroz
Registered User
 
mroz's Avatar
 
Join Date: Sep 2006
Posts: 201
@IanB, tsp

So far we've seen MEncoder, MeGUI & HC being unable to exploit MT for the same reason that Distributor is never invoked.

tsp, I think, said it would be very hard/impossible to modify MT/Avisynth so that this call is made automaticaly at the end of a script, other than in the AVIFile case as at present.

Since the problem is affecting many tools even in my limited experience, I'm brought back to considering this.

Is there really no way to automate a solution internal to AviSynth? Is it not possible to insert the relevant code immediately prior to processing of the first frame request?

If this can't be done, how about simplifying the modifications that need to be made to the third party programs affected? Rather than require they check to see if MT is in use & then invoke Distributor conditional on the current MT mode, why not hide all of that in a single function which is safe to call regardless of MT usage & mode? Then the only third party mod is to ensure they invoke this function at the end of script building.

Obviously this is only a full solution once MT becomes an official part of AviSynth, since prior to that the above function invokation will error, but that's probably safe to handle by documenting the function with the instruction to ignore any returned error/catch & ignore any exception.

I think simplifying the mods needed will make them more likely to be accepted. For instance, when I described the changes needed in respect of MeGUI's analysis pass option, the response I got failed to comment on the involved mode check but concentrated on confirming that switching to AVIfile access would eliminate the problem at the loss of certain abilities/controls; this strongly suggested to me that a quick simply change was being sought that wouldn't require any MT specific research prior to implementation, which makes sense for a busy dev.

Comments?
mroz is offline   Reply With Quote
Old 13th November 2007, 14:31   #718  |  Link
squid_80
Registered User
 
Join Date: Dec 2004
Location: Melbourne, AU
Posts: 1,963
IMO any apps accessing avisynth directly (bypassing avifile) should know exactly what they're doing and make the call themselves. Otherwise use AVIFile.
squid_80 is offline   Reply With Quote
Old 13th November 2007, 15:01   #719  |  Link
mroz
Registered User
 
mroz's Avatar
 
Join Date: Sep 2006
Posts: 201
The problem with that is that atm they can know exactly what they're doing in respect of AviSynth & won't know about this, as MT isn't integrated into AviSynth.

And even well after it is, surely it still makes sense, at the very least, to require no more than the script ends with a simple native function invokation which internally deals with checking MT mode & calling distributor if needed.

If in the future it becomes necessary for AviSynth to know when a script is complete for other reasons, this would also facilitate that.

Furthermore, it would allow the particular call to be included in scripts opened via AVIfile without any penalties - at the moment a call to distributor will screw up efficiency if included directly in a script which is also opened via AVIfile. I can't see the harm in at least avoiding that problem.
mroz is offline   Reply With Quote
Old 13th November 2007, 23:03   #720  |  Link
IanB
Avisynth Developer
 
Join Date: Jan 2003
Location: Melbourne, Australia
Posts: 3,173
Well the thing is you were never meant to directly call "Distributor" in scripts, I mearly suggested it as a test for a certain issue as I became aware of it. And yes it certainly makes sense to make MT as transparent as possible.
IanB is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 11:25.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.