Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
26th September 2011, 19:12 | #301 | Link | |
Registered User
Join Date: Dec 2006
Posts: 280
|
Quote:
Edit: I am encoding as we speak... wow... Jsut... freaking... WOW... I did NOT expect that... The script: LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\EXTRA\nnedi.dll") LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\EXTRA\addgrainc.dll") LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\EXTRA\TDEINT.dll") LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\EXTRA\TMM.dll") LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\EXTRA\TIVTC.dll") LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\EXTRA\mvtools2.dll") LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\EXTRA\mt_masktools.dll") LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\EXTRA\removegrainsse2.dll") Import("C:\Program Files (x86)\AviSynth 2.5\EXTRA\LimitedSharpenFaster.avs") Import("C:\Program Files (x86)\AviSynth 2.5\EXTRA\MCTemporalDenoise.v1.4.20.avsi") Import("C:\Program Files (x86)\AviSynth 2.5\EXTRA\LSFmod.v1.9.avsi") Import("C:\Program Files (x86)\AviSynth 2.5\EXTRA\GradFun2DBmod.v1.5.avsi") LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\EXTRA\fft3dfilter.dll") LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\EXTRA\gradfun2db.dll") LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\EXTRA\RepairSSE2.dll") LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\EXTRA\Yadifmod.dll") SetMemoryMax(512) SetMTMode(3) AVISource("Lulz.avi").ConvertToYv12(interlaced=true) SetMTMode(2) FFT3DFilter(sigma=1.0, sharpen=0.3,interlaced=true) Yadifmod(order=0, field=-1, mode=0, edeint=nnedi(field=-1)) LimitedSharpenFaster(strength=37) MCTemporalDenoise() Lanczos4Resize(872,480) Crop(12,0,-12,-0) Trim(1,66562) Thus far - when I have tried it without MT it was encoding at 4fps... Now? 12 fps... It is THRICE as fast!!! 300% gain, WTF?! AMAZING... SET, MAKE ME A KID!!! *grins* It did crash when I set memory to 768. Crashed each time I tried it. It hadnt finished encoding yet, but it will in an hour or so, I'll report how it went. Last edited by DarkT; 28th September 2011 at 23:20. |
|
30th September 2011, 11:40 | #302 | Link |
Registered User
Join Date: Mar 2009
Posts: 3,646
|
Can't say I've had any major stability problems with 2.58MT, I run about six filters in realtime, I tested 2.6MT but found it to be considerably slower with my script even with using SetMTMode(1). I guess everybody's mileage will vary.
Will have to try limiting threads and see what happens, how does 2.6MT run compared to 2.58MT for you guys performance wise generally speaking? Last edited by ryrynz; 30th September 2011 at 11:43. |
21st October 2011, 21:22 | #306 | Link |
Registered User
Join Date: Dec 2006
Posts: 280
|
Got a problem with fft3dfilter and fftw.dll on MT on HD material.
Ok, if you look a few posts up, you can see I used fft3dfilter with MT mode 2, just fine, but it was on SD material Now, I am trying to use it on HD material, and it's a no go, When I set it to MT mode 1 it does work, and it does help - without it it's at 2fps, with it it's at ~5fps. I just wonder what's up. I tried the same fft3dfilter line I used on teh SD material on the HD material... It's a no go... how strange... Any ideas? I'll try to re-run it on SD material, perhaps something in the environment changed, and it'd no longer be possible on SD as well, will edit later on. edit: "An exception occurred in module 'KERNELBASE'." Crashed Gonna try and go with fft3d only. Edit2: IT finished the encoding, but the video is messed up, 1 second you see it, tthe next you see half the pic, etc etc... how weird.. Last edited by DarkT; 22nd October 2011 at 10:53. |
23rd October 2011, 21:10 | #308 | Link |
Registered User
Join Date: Dec 2006
Posts: 280
|
What's weird though is that fft3d on SD material works fine with mode 2, but on HD it insta-crashes... So weird... Meh, I'm running lagarith lossless 1st pass only for fft3d purposes... 25hours, and 100gb for it... *sighs* ah well...
|
24th October 2011, 07:11 | #309 | Link | |
warpsharpened
Join Date: Feb 2007
Posts: 787
|
Quote:
SetMemoryMax is your friend. The thing with SetMTMode is that it seems mostly dependent on the amount of RAM that you can give it, which generally means (at least) SetMemoryMax(2048) for HD material for even remotely complicated scripts, which means you better have a 64-bit operating system which can give processes 4GB of address space, which means your program better have the LARGEADDRESSAWARE flag set. Then after a bit of praying and a script which actually threads well, it just might work. That being said I don't really like SetMTMode. What it all boils down to is that SetMTMode is a hack and shouldn't really be relied upon or expected to work in any capacity. If it works congrats, you got lucky, if it doesn't oh well and just move on. 64-bit avisynth is the same way, except that's way more hacky and a giant piece of shit, MTMode is at least a respectable attempt. Last edited by TheRyuu; 24th October 2011 at 07:40. |
|
24th October 2011, 23:20 | #311 | Link | ||
warpsharpened
Join Date: Feb 2007
Posts: 787
|
Quote:
Have you even looked at the msvc project files for x64? They have a huge amount of useless (optimization) settings turned on which give the impression whoever did it had little to no idea what they were doing. Quote:
I don't know how exactly avisynth (and MT related company) are doing their frame caching but if my SetMemoryMax value is too small it seems as though avisynth just chews up cpu cycles for nothing and bogs down to something extremely slow (further hurting avisynth-mt's credibility in the process). I don't really see how having too large of a frame cache could hurt performance. On a side note I guess the function name "SetMemoryMax" isn't exactly the best name for it since all it really equates to is "UseThisAmountOfMemory()", and doesn't actually define a maximum amount of memory which avisynth can chose to use up to as the name implies. |
||
25th October 2011, 03:41 | #312 | Link |
Registered User
Join Date: Jul 2010
Posts: 448
|
If you increase the memory for the frame cache then you reduce the memory available for the filters themselves. Some filters use little memory, but others allocate quite a lot. Setting a huge frame cache increases the chance of the filters running out of memory and crashing. SetMemoryMax is a misleading name - it really sets a memory balance between cache and free memory for filters. There is a sweet spot for any given script for SetMemoryMax, where you have reserved just enough for its caching needs and no more. I.e. you should SetMemoryMax to the smallest value that doesn't slow down processing.
Since MT processes several frames simultaneously its memory and caching demands are multiplied. MT is much more likely to run out of memory than non-MT given the 32-bit constraint of avisynth. Setting thread counts / SetMemoryMax values that just won't fit in 32-bit memory is a frequent cause of MT problems. Last edited by -Vit-; 25th October 2011 at 03:44. |
25th October 2011, 14:41 | #314 | Link |
Registered User
Join Date: Aug 2007
Posts: 374
|
I wasn't talking about exact x64 Avisynth build but rather about x64 version of Avisynth in general - it's stalled project no one works on. Haven't looked at that x64 build so can't comment about it. Windows x64 ABI is plain retarded in part about saving xmm registers. As for msvc project settings - I see that all the time, got used to almost always recreating them myself, sigh.
As for memory consumption - I think it's too early to blame 32-bit address space. Current MT Avisynth cache wastes way more memory than required. |
25th October 2011, 15:19 | #315 | Link |
Registered User
Join Date: Jul 2010
Posts: 448
|
Yes, I know what you mean. Running out of memory is the problem, but it shouldn't be. A 32-bit address space should be plenty for extreme HD processing. But avisynth caches the output of every single filter in your script (and the final output frame - unnecessary for encoding). But really only filter outputs that are accessed temporally (+ some subtle dependencies) need caching beyond the current frame, very few in the typical script. I analyzed QTGMC and it has maybe 10 intermediate frames that really need to be cached (per output frame), but 100s are actually considered for caching. I tried to use SetCacheHints to help reduce this, but at that time (2.58, haven't looked at the code recently) SetCacheHints seemed to have been rather hacked about and I had limited success.
Last edited by -Vit-; 25th October 2011 at 21:57. Reason: clarity |
16th November 2011, 21:20 | #318 | Link |
Registered User
Join Date: Aug 2007
Posts: 374
|
I've never used such feature and can't comment about it. It looks like a request for official Avisynth: http://forum.doom9.org/showthread.php?t=149113
|
30th November 2011, 21:24 | #319 | Link | ||
Registered User
Join Date: Feb 2008
Location: Philippines
Posts: 1
|
Quote:
Quote:
Is not a request for official avisynth... I've tried it too and avisynth exits with the following code: Code:
Traceback (most recent call last): File "AvsP.pyo", line 6291, in OnSliderReleased File "AvsP.pyo", line 8822, in ShowVideoFrame File "AvsP.pyo", line 9256, in UpdateScriptAVI File "pyavs.pyo", line 167, in __init__ File "avisynth.pyo", line 117, in Invoke WindowsError: exception: access violation reading 0xA6446895 Its not a memory issue. I've tried various values for setmemorymax() - all the way up to 4096 and it gives the same error. That means we cant access the vdub filters (*.vdf) from within avisynth. which is a pity.. grateful if someone can help... Last edited by debpk77; 30th November 2011 at 21:29. |
||
Thread Tools | Search this Thread |
Display Modes | |
|
|