Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 21st February 2012, 06:03   #21  |  Link
leiming2006
Author of lmx264gui
 
leiming2006's Avatar
 
Join Date: Mar 2006
Location: Fujian, China
Posts: 198
Quote:
Originally Posted by ChaosKing View Post
New Version doesn't crash anymore, Speed seems to be the same as in previous versions with my script: ~4.5 fps (without sorathread) -> ~6.9 fps (with sorathread)

With DataCopy= true it's slightly slower ~6.8 fps

I also tested sorathread on "light" filters like deen or fastlinedarken. Here it is just not worth to run soratread, worst case: your encode becomes slower

I love your filter, very easy to use with nice speed improvements!
I'm glad to see the problem disappeared.
The CPU of my computer is T2390 (2 cores),
some problems may happened more rarely in CPU with 2 cores than CPU with 4 or more cores

I'm happy that you like it

Quote:
Originally Posted by zerowalker View Post
Okay, well i use the Avisynth 2.6 version that is posted in this forum with this;O

But am i supposed to write SoraThread(X,X) before every filter?

Thanks
You may insert SoraThread() between filters to separate them into different threads.
For example,
AVISource(...)
SoraThread(...)
Crop(...)
Lanczosresize(...)

Then, the [decoding] and [cropping and resizing] will work in different threads.

AVISource(...)
Crop(...)
SoraThread(...)
Lanczosresize(...)

Then, the [decoding and cropping] and [resizing] will work in different threads.

Last edited by leiming2006; 21st February 2012 at 06:05.
leiming2006 is offline   Reply With Quote
Old 21st February 2012, 08:05   #22  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 766
It's a nice plugin but there is a great problem: it doesn't multithread avsi scripts as SetMTMode does. Am I wrong?
__________________
tormento@ircnet
tormento is offline   Reply With Quote
Old 21st February 2012, 20:05   #23  |  Link
zerowalker
Registered User
 
Join Date: Jul 2011
Posts: 1,100
how can i make QTGMC() alone work in 4 threads then;O?
zerowalker is offline   Reply With Quote
Old 27th February 2012, 19:55   #24  |  Link
mastrboy
Registered User
 
Join Date: Sep 2008
Posts: 286
@zerowalker, you have to modify QTGMC.avsi by adding sorathread calls various places inside the function definition.

I finally got around to testing sorathread today, adding a SoraThread() after a call to nnedi3 the fps was increased by 30%, adding more of SoraThread() other places did not increase speed any further.

@leiming2006, how do one decide what the appropriate buffer value is for a script?
mastrboy is offline   Reply With Quote
Old 28th February 2012, 07:53   #25  |  Link
turbojet
Registered User
 
Join Date: May 2008
Posts: 1,840
I gained about 25% by adding 3 sorathread(1) between source, tfm, tdecimate and crop/resize, thanks leiming2006. Threadrequest was about a 15% gain, MT and SetMTMode slowed down the script considerably.

1 frame buffer was much faster than the default of 3 in my case.
turbojet is offline   Reply With Quote
Old 28th February 2012, 16:17   #26  |  Link
leiming2006
Author of lmx264gui
 
leiming2006's Avatar
 
Join Date: Mar 2006
Location: Fujian, China
Posts: 198
In my friends' tests, concurrently reading frames from the same filter instance will cause crash.
for a meaningless (but should have not crashed) example,
(this script is offered by "304". I have made it more simple)
Code:
ffvideosource(...)
assumeframebased()
separatefields()
selectevery(8, 0, 1, 0, 3, 2, 5, 4, 5, 6, 7)
weave()
w=width
h=height

TFM(mode=5, pp=0, slow=2).TDecimate(1)
sorathread()
src=last
c=last.spline64resize(w,h,2,2,-2,-1)

clip1=c.sorathread()
msk=c.sorathread()
clip2=src.sorathread()
mt_merge(clip1, clip2, msk, luma=false, u=3, v=3).sorathread()
I'm thinking about whether it's because the object which comes (type: IScriptEnvironment) from Avisynth engine is not thread-safed.
I'll try locking the environment object during processing next time, to see if the crash could be avoided.

Quote:
Originally Posted by mastrboy View Post
@leiming2006, how do one decide what the appropriate buffer value is for a script?
I'm sorry I don't know.
In fact the default value 5 is what I "guess".
Just thought that "maybe... 5 is enough"

As my designing, higher value will cause more buffer-maintaining overhead, more locking conflict (thread waiting for each other) and a value which is not enough will also cause threads waiting for data happened more frequently.

Last edited by leiming2006; 28th February 2012 at 16:26.
leiming2006 is offline   Reply With Quote
Old 18th March 2012, 18:13   #27  |  Link
Pat357
Registered User
 
Join Date: Jun 2006
Posts: 400
Has the development stopped ?

It's a pitty that we don't see any new versions of this nice tool...
Why is this ?
Has the developer (Sora) lost his interest ?
Pat357 is offline   Reply With Quote
Old 18th March 2012, 19:31   #28  |  Link
GEfS
Registered User
 
Join Date: Aug 2011
Posts: 19
Code:
ffms2_FFVideoSource("bluray remux source.mkv")
SetMTMode(2,0)
Spline36Resize(1280,534,0,140,0,-140)
LSFmod(ss_x=1.0,ss_y=1.0,Smode=3,Smethod=3,Lmode=3,strength=60,overshoot=2,undershoot=1)
I'm using a avs script like that.
Can you give me a "Sora" script for this?
My x264 setting:
Code:
--level 4.1 --deblock -3:-3 --bframes 8 --b-adapt 2 --ref 12 --aq-mode 2 --aq-strength 0.8 
--cqmfile "C:\Users\Administrator\Desktop\eqm_avc_hr.cfg" --merange 32 --me umh --direct auto --subme 11 
--partitions all --trellis 2 --psy-rd 1.15:0.45 --no-dct-decimate --no-fast-pskip
Pass 1: ~17.5 fps
Pass 2: ~5 fps
GEfS is offline   Reply With Quote
Old 18th March 2012, 22:31   #29  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 154
Quote:
Originally Posted by GEfS View Post
[code]
Can you give me a "Sora" script for this?
Just add sorathread() before LSFmod()
Code:
Spline36Resize(1280,534,0,140,0,-140)
sorathread()
LSFmod(ss_x=1.0,ss_y=1.0,Smode=3,Smethod=3,Lmode=3,strength=60,overshoot=2,undershoot=1)
I don't know how SetMTMode would affect sorathread, so maybe you need to remove it from your script.
__________________
Search and denoise
ChaosKing is offline   Reply With Quote
Old 19th March 2012, 02:29   #30  |  Link
Pat357
Registered User
 
Join Date: Jun 2006
Posts: 400
Quote:
Originally Posted by GEfS View Post
Code:
ffms2_FFVideoSource("bluray remux source.mkv")
SetMTMode(2,0)
Spline36Resize(1280,534,0,140,0,-140)
LSFmod(ss_x=1.0,ss_y=1.0,Smode=3,Smethod=3,Lmode=3,strength=60,overshoot=2,undershoot=1)
I'm using a avs script like that.
Can you give me a "Sora" script for this?
My x264 setting:
Code:
--level 4.1 --deblock -3:-3 --bframes 8 --b-adapt 2 --ref 12 --aq-mode 2 --aq-strength 0.8 
--cqmfile "C:\Users\Administrator\Desktop\eqm_avc_hr.cfg" --merange 32 --me umh --direct auto --subme 11 
--partitions all --trellis 2 --psy-rd 1.15:0.45 --no-dct-decimate --no-fast-pskip
Pass 1: ~17.5 fps
Pass 2: ~5 fps
What CPU do you have ?

Take in mind that the script and x264 run at the same time.
Speeding up your script makes only sense if the script runs slower than x264, what is probably not true in this case.
Measure the speed from your script. (AVSmeter ,...) and speed up the slowest part in the chain.
Further speeding up is only possible if your CPU is not already fully loaded.
Looking at the x264 parameters, you gone need a heavy multi-CPU system if you want the whole chain to run faster by speeding up the script using Sora's multi-threading

An alternative is to separate (decoding + filters) and encoding by using a "temporary file".
(decoding + filters) -> lossless encoding --> temp file
temp file --> lossless decoding --> x264 encoding.

BTW : have you tried to run your script ? You have a good chance it will stuck on line 1...

Last edited by Pat357; 19th March 2012 at 02:34.
Pat357 is offline   Reply With Quote
Old 19th March 2012, 03:51   #31  |  Link
GEfS
Registered User
 
Join Date: Aug 2011
Posts: 19
I'm using Xeon X3430
GEfS is offline   Reply With Quote
Old 15th April 2012, 09:07   #32  |  Link
cybersharky
Registered User
 
Join Date: Jul 2008
Posts: 131
How exactly do you use sorathread/sorasmsource? Where must I have the .dll's? As far as I can see the examples have steps left out?
cybersharky is offline   Reply With Quote
Old 26th September 2012, 20:23   #33  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 703
very nice speed up for filter chains.

tried with a simple
Code:
sorathread()
tfm()
sorathread()
tdecimate()
speed went from 70fps to 120fps, very sweet for my AMD athlon x4.
lansing is offline   Reply With Quote
Old 27th September 2012, 11:34   #34  |  Link
active1
Registered User
 
Join Date: Nov 2011
Location: spain
Posts: 45
will sorathread work with vapoursynth?
active1 is offline   Reply With Quote
Old 27th September 2012, 11:46   #35  |  Link
Keiyakusha
契約者
 
Keiyakusha's Avatar
 
Join Date: Jun 2008
Posts: 1,567
vapoursynth have its own threading built into the core isn't it? does it really needs hacks like this?


by the way I tried SoraThread() recently. documentation is very nice but I can't seem to get much cpu load with 8 threads. There is some noticeable speedups but ideally I want to see some solution that can easily fill all cpu resources without wasting cycles for some redundant work. This is too much to ask I guess?

Last edited by Keiyakusha; 27th September 2012 at 11:50.
Keiyakusha is offline   Reply With Quote
Old 27th September 2012, 13:47   #36  |  Link
Hiritsuki
Novice of AVS
 
Join Date: Oct 2009
Posts: 156
I had test lot of sora or mt, the conclusion is pixel too big to make faster.
__________________
My PC
Hiritsuki is offline   Reply With Quote
Old 27th September 2012, 14:18   #37  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,390
I tried to use SoraThread on a MDegrain3 script, by using one sorathread for each motionvector.

Code:
mpeg2source("PAL-DVD-720x576.d2v")

src = last
sup = src.msuper()

bv3 = sup.manalyse(isb=true, delta=3).sorathread()
bv2 = sup.manalyse(isb=true, delta=2).sorathread()
bv1 = sup.manalyse(isb=true, delta=1).sorathread()
fv1 = sup.manalyse(isb=false,delta=1).sorathread()
fv2 = sup.manalyse(isb=false,delta=2).sorathread()
fv3 = sup.manalyse(isb=false,delta=3).sorathread()

src.mdegrain3(sup,bv1,fv1,bv2,fv2,bv3,fv3)

return(last)
I don't know how to explain, please try for yourself. It crashes as fast as i can say "oops". Playing with buffer/cache didn't help. Setting CopyData=true didn't help.
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline   Reply With Quote
Old 27th September 2012, 16:39   #38  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 703
I did a few tests with fft3dfilter, lsfmod, and tnlmeans, comparing the mt mode and sorathread, as well as combining them.
Source resolution 720x480, test run using AVSmeter, and over 5000 frames for each test, i didn't bother running that many frames for tnlmeans though.

Code:
	         no mt     SetMtMode     Sorathread     Sorathreadx2     SetMtMode+Sorathread
fft3dfilter	 28.2	     37	           32.56           33.28            55
lsfmod	         39.83       85.8          52              49.5             108 (no respond)
tnlmeans          1.21       4.44          1.21            1.21             4.47
combining mt and sorathread together tends to get the fastest speed, and cpu usages were close to 100%. Lsfmod was running close to 3 times faster (for 2000+ frames) before no respond, and i didn't know what's causing it.
lansing is offline   Reply With Quote
Old 28th September 2012, 00:22   #39  |  Link
active1
Registered User
 
Join Date: Nov 2011
Location: spain
Posts: 45
Quote:
Originally Posted by Keiyakusha View Post
vapoursynth have its own threading built into the core isn't it? does it really needs hacks like this?
yeah but not for avisynth plugins. i want to run avisynth plugins in vapoursynth like avisynth MT does but with an external plugin without affecting vapoursynth
but it looks like that there isn't such a plugin right now
active1 is offline   Reply With Quote
Old 28th September 2012, 00:43   #40  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Ikea Chair
Posts: 1,520
Quote:
Originally Posted by active1 View Post
yeah but not for avisynth plugins. i want to run avisynth plugins in vapoursynth like avisynth MT does but with an external plugin without affecting vapoursynth
but it looks like that there isn't such a plugin right now
Vapoursynth is quite effective at multithreading avisynth filters too so you should never use anything like this. All you'll get is a crash at best.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Reply

Tags
multi thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 00:31.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2017, vBulletin Solutions Inc.