Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 26th September 2011, 19:12   #301  |  Link
DarkT
Registered User
 
Join Date: Dec 2006
Posts: 280
Quote:
Originally Posted by um3k View Post
Simple, you can't use 64 bit plugins with 32 bit avisynth. Stick to the 32 bit plugins if you're using 2.6.
Wait... Wait... so this 2.6 MT version of set... it's 32bits?! Damn... I though it's 64bits >.< *sighs* that would indeed explain everything... *sighs* Ah well, this is even better *Grins* no need to torment myself now with "nice that you got MT working... BUT... if it were MT AND 64bit, it would prolly be even FASTER!!!" *chuckles*

Edit:

I am encoding as we speak... wow... Jsut... freaking... WOW... I did NOT expect that...

The script:

LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\EXTRA\nnedi.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\EXTRA\addgrainc.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\EXTRA\TDEINT.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\EXTRA\TMM.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\EXTRA\TIVTC.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\EXTRA\mvtools2.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\EXTRA\mt_masktools.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\EXTRA\removegrainsse2.dll")
Import("C:\Program Files (x86)\AviSynth 2.5\EXTRA\LimitedSharpenFaster.avs")
Import("C:\Program Files (x86)\AviSynth 2.5\EXTRA\MCTemporalDenoise.v1.4.20.avsi")
Import("C:\Program Files (x86)\AviSynth 2.5\EXTRA\LSFmod.v1.9.avsi")
Import("C:\Program Files (x86)\AviSynth 2.5\EXTRA\GradFun2DBmod.v1.5.avsi")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\EXTRA\fft3dfilter.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\EXTRA\gradfun2db.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\EXTRA\RepairSSE2.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\EXTRA\Yadifmod.dll")

SetMemoryMax(512)
SetMTMode(3)
AVISource("Lulz.avi").ConvertToYv12(interlaced=true)
SetMTMode(2)
FFT3DFilter(sigma=1.0, sharpen=0.3,interlaced=true)
Yadifmod(order=0, field=-1, mode=0, edeint=nnedi(field=-1))
LimitedSharpenFaster(strength=37)
MCTemporalDenoise()
Lanczos4Resize(872,480)
Crop(12,0,-12,-0)
Trim(1,66562)

Thus far - when I have tried it without MT it was encoding at 4fps... Now? 12 fps... It is THRICE as fast!!! 300% gain, WTF?! AMAZING... SET, MAKE ME A KID!!! *grins*

It did crash when I set memory to 768. Crashed each time I tried it.

It hadnt finished encoding yet, but it will in an hour or so, I'll report how it went.

Last edited by DarkT; 28th September 2011 at 23:20.
DarkT is offline   Reply With Quote
Old 30th September 2011, 11:40   #302  |  Link
ryrynz
Registered User
 
ryrynz's Avatar
 
Join Date: Mar 2009
Posts: 3,646
Can't say I've had any major stability problems with 2.58MT, I run about six filters in realtime, I tested 2.6MT but found it to be considerably slower with my script even with using SetMTMode(1). I guess everybody's mileage will vary.
Will have to try limiting threads and see what happens, how does 2.6MT run compared to 2.58MT for you guys performance wise generally speaking?

Last edited by ryrynz; 30th September 2011 at 11:43.
ryrynz is offline   Reply With Quote
Old 8th October 2011, 13:36   #303  |  Link
Mosfetov
Registered User
 
Join Date: Apr 2008
Posts: 2
Quote:
Originally Posted by BigDid View Post
Some syntax works some do not.
Is it still possible to use mt() with current 2.6.0MT?
Every syntax resuilt in error "wrong avisynth version...".

Last edited by Mosfetov; 8th October 2011 at 13:40.
Mosfetov is offline   Reply With Quote
Old 8th October 2011, 15:40   #304  |  Link
Zep
Registered User
 
Join Date: Jul 2002
Posts: 587
Quote:
Originally Posted by Mosfetov View Post
Is it still possible to use mt() with current 2.6.0MT?
Every syntax resuilt in error "wrong avisynth version...".
if i recall correctly Set said no more support for mt(). just use the SetMTmode() calls

Last edited by Zep; 8th October 2011 at 21:24.
Zep is offline   Reply With Quote
Old 8th October 2011, 17:15   #305  |  Link
SEt
Registered User
 
Join Date: Aug 2007
Posts: 374
I'm not maintaining MT(), so it quite possible broken. MTMode is way to go.
SEt is offline   Reply With Quote
Old 21st October 2011, 21:22   #306  |  Link
DarkT
Registered User
 
Join Date: Dec 2006
Posts: 280
Got a problem with fft3dfilter and fftw.dll on MT on HD material.

Ok, if you look a few posts up, you can see I used fft3dfilter with MT mode 2, just fine, but it was on SD material

Now, I am trying to use it on HD material, and it's a no go, When I set it to MT mode 1 it does work, and it does help - without it it's at 2fps, with it it's at ~5fps.

I just wonder what's up.

I tried the same fft3dfilter line I used on teh SD material on the HD material... It's a no go... how strange... Any ideas? I'll try to re-run it on SD material, perhaps something in the environment changed, and it'd no longer be possible on SD as well, will edit later on.

edit:

"An exception occurred in module 'KERNELBASE'."

Crashed Gonna try and go with fft3d only.

Edit2:

IT finished the encoding, but the video is messed up, 1 second you see it, tthe next you see half the pic, etc etc... how weird..

Last edited by DarkT; 22nd October 2011 at 10:53.
DarkT is offline   Reply With Quote
Old 23rd October 2011, 14:01   #307  |  Link
SEt
Registered User
 
Join Date: Aug 2007
Posts: 374
Looks like crash due to memory usage. Read the first post.

Do not use mode 1 unless you know filter is designed for that mode. Otherwise it won't necessary crash but likely produce very wrong result like described.
SEt is offline   Reply With Quote
Old 23rd October 2011, 21:10   #308  |  Link
DarkT
Registered User
 
Join Date: Dec 2006
Posts: 280
What's weird though is that fft3d on SD material works fine with mode 2, but on HD it insta-crashes... So weird... Meh, I'm running lagarith lossless 1st pass only for fft3d purposes... 25hours, and 100gb for it... *sighs* ah well...
DarkT is offline   Reply With Quote
Old 24th October 2011, 07:11   #309  |  Link
TheRyuu
warpsharpened
 
Join Date: Feb 2007
Posts: 787
Quote:
Originally Posted by DarkT View Post
What's weird though is that fft3d on SD material works fine with mode 2, but on HD it insta-crashes...
A larger size frame taking up more memory? Who would've thunk it.

SetMemoryMax is your friend.
The thing with SetMTMode is that it seems mostly dependent on the amount of RAM that you can give it, which generally means (at least) SetMemoryMax(2048) for HD material for even remotely complicated scripts, which means you better have a 64-bit operating system which can give processes 4GB of address space, which means your program better have the LARGEADDRESSAWARE flag set.

Then after a bit of praying and a script which actually threads well, it just might work.

That being said I don't really like SetMTMode. What it all boils down to is that SetMTMode is a hack and shouldn't really be relied upon or expected to work in any capacity. If it works congrats, you got lucky, if it doesn't oh well and just move on. 64-bit avisynth is the same way, except that's way more hacky and a giant piece of shit, MTMode is at least a respectable attempt.

Last edited by TheRyuu; 24th October 2011 at 07:40.
TheRyuu is offline   Reply With Quote
Old 24th October 2011, 16:24   #310  |  Link
SEt
Registered User
 
Join Date: Aug 2007
Posts: 374
64-bit Avisynth is not 'hacky'. It has only one problem - no one maintains it.

SetMemoryMax is not as trivial as 'set to 2GB and you are done' - sometimes smaller values give better result than larger ones.
SEt is offline   Reply With Quote
Old 24th October 2011, 23:20   #311  |  Link
TheRyuu
warpsharpened
 
Join Date: Feb 2007
Posts: 787
Quote:
Originally Posted by SEt View Post
64-bit Avisynth is not 'hacky'. It has only one problem - no one maintains it.
Several internal functions don't work and for a time it didn't even properly abide by the Windows x64 ABI (and I'm pretty sure the version people recommend to use doesn't properly abide by it). Sure does seem a little hacky to me.

Have you even looked at the msvc project files for x64? They have a huge amount of useless (optimization) settings turned on which give the impression whoever did it had little to no idea what they were doing.

Quote:
Originally Posted by SEt View Post
SetMemoryMax is not as trivial as 'set to 2GB and you are done' - sometimes smaller values give better result than larger ones.
Well it was more of a guesstimate than anything else really, just based on playing around with it a little. 2GB does seem like a good starting point though at least for HD material. As soon as you start throwing temporal filters into the mix the need for a larger cache size seems to go up real quick.

I don't know how exactly avisynth (and MT related company) are doing their frame caching but if my SetMemoryMax value is too small it seems as though avisynth just chews up cpu cycles for nothing and bogs down to something extremely slow (further hurting avisynth-mt's credibility in the process). I don't really see how having too large of a frame cache could hurt performance. On a side note I guess the function name "SetMemoryMax" isn't exactly the best name for it since all it really equates to is "UseThisAmountOfMemory()", and doesn't actually define a maximum amount of memory which avisynth can chose to use up to as the name implies.
TheRyuu is offline   Reply With Quote
Old 25th October 2011, 03:41   #312  |  Link
-Vit-
Registered User
 
Join Date: Jul 2010
Posts: 448
If you increase the memory for the frame cache then you reduce the memory available for the filters themselves. Some filters use little memory, but others allocate quite a lot. Setting a huge frame cache increases the chance of the filters running out of memory and crashing. SetMemoryMax is a misleading name - it really sets a memory balance between cache and free memory for filters. There is a sweet spot for any given script for SetMemoryMax, where you have reserved just enough for its caching needs and no more. I.e. you should SetMemoryMax to the smallest value that doesn't slow down processing.

Since MT processes several frames simultaneously its memory and caching demands are multiplied. MT is much more likely to run out of memory than non-MT given the 32-bit constraint of avisynth. Setting thread counts / SetMemoryMax values that just won't fit in 32-bit memory is a frequent cause of MT problems.

Last edited by -Vit-; 25th October 2011 at 03:44.
-Vit- is offline   Reply With Quote
Old 25th October 2011, 07:03   #313  |  Link
TheRyuu
warpsharpened
 
Join Date: Feb 2007
Posts: 787
Quote:
Originally Posted by -Vit- View Post
Setting thread counts / SetMemoryMax values that just won't fit in 32-bit memory is a frequent cause of MT problems.
Both of which can be reduced by uses processes which have the LARGEADDRESSAWARE flag set and be running them on a 64-bit OS (yay 4GB per process!).
TheRyuu is offline   Reply With Quote
Old 25th October 2011, 14:41   #314  |  Link
SEt
Registered User
 
Join Date: Aug 2007
Posts: 374
I wasn't talking about exact x64 Avisynth build but rather about x64 version of Avisynth in general - it's stalled project no one works on. Haven't looked at that x64 build so can't comment about it. Windows x64 ABI is plain retarded in part about saving xmm registers. As for msvc project settings - I see that all the time, got used to almost always recreating them myself, sigh.

As for memory consumption - I think it's too early to blame 32-bit address space. Current MT Avisynth cache wastes way more memory than required.
SEt is offline   Reply With Quote
Old 25th October 2011, 15:19   #315  |  Link
-Vit-
Registered User
 
Join Date: Jul 2010
Posts: 448
Quote:
Originally Posted by SEt View Post
I think it's too early to blame 32-bit address space. Current MT Avisynth cache wastes way more memory than required.
Yes, I know what you mean. Running out of memory is the problem, but it shouldn't be. A 32-bit address space should be plenty for extreme HD processing. But avisynth caches the output of every single filter in your script (and the final output frame - unnecessary for encoding). But really only filter outputs that are accessed temporally (+ some subtle dependencies) need caching beyond the current frame, very few in the typical script. I analyzed QTGMC and it has maybe 10 intermediate frames that really need to be cached (per output frame), but 100s are actually considered for caching. I tried to use SetCacheHints to help reduce this, but at that time (2.58, haven't looked at the code recently) SetCacheHints seemed to have been rather hacked about and I had limited success.

Last edited by -Vit-; 25th October 2011 at 21:57. Reason: clarity
-Vit- is offline   Reply With Quote
Old 25th October 2011, 21:59   #316  |  Link
DarkT
Registered User
 
Join Date: Dec 2006
Posts: 280
Hmm, I see I see... So what you guys are basically saying is that I should go FFT3DGPU? *grins*
DarkT is offline   Reply With Quote
Old 16th November 2011, 02:26   #317  |  Link
cattolicboy
Registered User
 
Join Date: Nov 2011
Posts: 4
Hi SEt, do you think to implement the possibility to load LoadVirtualDubPlugin in the future?
At the moment seems not working, maybe someone knows some trick to load it...
cattolicboy is offline   Reply With Quote
Old 16th November 2011, 21:20   #318  |  Link
SEt
Registered User
 
Join Date: Aug 2007
Posts: 374
I've never used such feature and can't comment about it. It looks like a request for official Avisynth: http://forum.doom9.org/showthread.php?t=149113
SEt is offline   Reply With Quote
Old 30th November 2011, 21:24   #319  |  Link
debpk77
Registered User
 
Join Date: Feb 2008
Location: Philippines
Posts: 1
Quote:
Originally Posted by cattolicboy View Post
Hi SEt, do you think to implement the possibility to load LoadVirtualDubPlugin in the future?
At the moment seems not working, maybe someone knows some trick to load it...
Quote:
Originally Posted by SEt View Post
I've never used such feature and can't comment about it. It looks like a request for official Avisynth: http://forum.doom9.org/showthread.php?t=149113
SEt,
Is not a request for official avisynth...
I've tried it too and avisynth exits with the following code:
Code:
Traceback (most recent call last):
  File "AvsP.pyo", line 6291, in OnSliderReleased
  File "AvsP.pyo", line 8822, in ShowVideoFrame
  File "AvsP.pyo", line 9256, in UpdateScriptAVI
  File "pyavs.pyo", line 167, in __init__
  File "avisynth.pyo", line 117, in Invoke
WindowsError: exception: access violation reading 0xA6446895
I'm using x64 version of windows on an intel 64 bit processor.
Its not a memory issue. I've tried various values for setmemorymax() - all the way up to 4096 and it gives the same error.

That means we cant access the vdub filters (*.vdf) from within avisynth. which is a pity..

grateful if someone can help...

Last edited by debpk77; 30th November 2011 at 21:29.
debpk77 is offline   Reply With Quote
Old 1st December 2011, 12:22   #320  |  Link
Youka
Registered User
 
Youka's Avatar
 
Join Date: Mar 2011
Location: Germany
Posts: 64
AvsP has it's own problems, so it crashes if a plugin throws an error during filtering process and not just at initialization. Try it simply with MPC and look for an error message.
Youka is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 19:03.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.